iASK: Assessment of Skills & Know-how

Dynamically evolving technology and employer needs necessitate a workforce capable of rapid skill development. Current skills / learning assessment systems lack precise measurement of skills in context of performance, categorizing certain learners as unemployable.  

iASK provides educators and learners with specific skills repertoire-building to achieve competence in a career of their choice.

It starts and ends with the learner: this evidence-based system provides educators immediately implementable learning plans.

1. Determine values, career goals using pre-assessment survey
2. Determine skill domains to measure (+500 skills)
3. Conduct assessment based on actual observation and rate
4. Produce Skillgram comparing learner skill to goal
   • option of using Standard Celeration Chart

iASK impacts lives because

• learners and educators see performance as skills to be acquired;
• learners are supported to own their progress;
• educators can precisely measure performance and time to gain a skill and reinforce actions

In 2014 we started "Career Development Academy" program with people with disabilities in jobs and pre-employment training with years of only “sheltered” employment. We wanted to create a system to help learners:

• identify their skill levels compared to skills required for their target career / job; 
• get actual work experiences with local employers;
• identify, obtain and sustain meaningful employment;
• increase their knowledge and access to ongoing skill-building opportunities.

 The learning and root cause analysis from our work:

• individuals lacked understanding what a job entailed if they had not done the job.
• Metaphorical explanation of a job, shadowing, tours were ineffective to build either motivation or skills.
• If a person had complex verbal skills, deficits in component level skills were often not visible; e.g. several learners with complex verbal skills were unable to read at first grade level.
• Finding dignity, pride, and success applies to every learner.

Given the absence of accurate assessments and quick actionable plans, learners can despair.

Many statistics show meaningful employment as number one indicator of success, health and fulfillment. iASK provides learners with specific, measurable skills in the context of achieving their goals, supporting people to achieve lives of meaning and purpose in rapidly-changing world.

iASK is an evidence-based proprietary learning system where a trained assessor:

• Begins with earnest questions, “what does the person want to do”, what skills they have to achieve them (Pre-assessment survey).
• Determines domains to assess relevant to goal;
• Conduct assessment (measure observable and quantifiable behavior)
  • 500+ skills organized in a 3-level hierarchy;
  • structured in a component-composite relationships
  • skills can be drilled down to specific pinpoints.

 Assessment

• developed with proven scientific methods—applied behavior analysis and precision teaching;
• uses fluency as a central measure, when possible and applicable instead of Percent correct which is often not a reflection of mastery.
• shows progress or not for learner over time with repeated assessments.
• measures barriers to learning; 
• is applicable for any learner at any age.

Learning plan

• Generates analysis, actionable, measurable steps for learner;
• Lays out component level skills necessary to achieve towards a goal;
• Can be adjusted as per analysis.

Current state:

• 2 facilitation guides (500+ pages): complete
  • step-by-step instructions
  • over 110 reference citations
• Remote CEU trainings for trainers
• Cloud-based software development under way
  • Currently piloting largest skill domain using software 

While much of the history of this innovative learning system has been developed to support people with disabilities, it can be used by any learner anywhere.

• Pilot data on complex verbal behavior indicated that typical learners can improve skill;
• Sensitivity in measurement allows for continuous development;
• Software will facilitate access to any learner and any assessor anywhere;
• Virtual assessment capability allows reach to underserved communities, or continuous service during pandemic.
• Software provides controls and guardrails for fidelity of assessments across different assessors;
• Collection of initial skills and ongoing progress data will build understanding of skill correlations and “time to fluency” based on a learner’s start point.

Data suggests unemployment is a major cause of homelessness and mental health challenges. Inability to obtain and sustain meaningful employment maybe the result of having never acquired component level skills such as fluent use of language and numbers.

iASK utilizes the broad principles of behavior science that are likewise applicable to:

• children and transition-aged young adults,
• typically functioning adults transitioning
  • from military to civilian life,
  • from prison to re-entry,
• aging populations with memory care issues.

Our solution and mission touch multiple dimensions. iASK process defines employment in context of a person’s goals and as an outcome of performing complex skills. If learner has performance problems, it may be due to past experience with learning. iASK specifically identifies gaps in skill repertoires, provides plans in achievable increments to build learner’s success and confidence. As learners succeed, educators also succeed. 

iASK is a method for identification and skill-building, creating quicker and more efficient ways to enter the workforce, transition between jobs or phases of life.

iASK is age, ability, cultural or socio-economic background agnostic.

Real example https://hsireno.org/news/

Solution is part of our social enterprise - High Sierra Industries.

Project lead: Stu Law is a Ph.D candidate at the University of Nevada, Reno, under the guidance of Dr. Steven Hayes, foundational scholar for Acceptance and Commitment Therapy, and Relational Frame Theory. Stu’s dissertation is focused on the skills which underlie self-advocacy, independent decision-making, and dignified risk-taking.

Solution team:

FT: 5

Partner contractor: 3

External subject matter experts: 2

Inclusivity begins with senior leadership committing to achieve it. Inclusivity is one of our strategic goals, measured equally with financial and operational goals.

We ask questions below annually and where the answer is no, we do the work to make it a yes. Work assigned to all leaders and their staffs. Identified outcome definitions, time-frame, measured and reported.

• Does our organization show our mission in action?
• Does our mission incorporate inclusivity directly or indirectly?
• Do our policies and procedures support hiring and sustaining an inclusive organization? 

Given the mission of our organization, we found we fell short of including people with disabilities at all levels in the organization. We set that goal, measured, and reported inclusivity measures monthly. While the work is continuous, we currently are a majority female-led organization and have people with documented disabilities from executive leadership to direct service staff

• We completed a comprehensive market analysis by identifying all assessments used that were appropriate to compare to iASK (=102)
• 102 were narrowed to 12 – those that met 7 characteristics of a behavior analytic approach (Applied, Behavioral, Analytic, Technological, Conceptually Systematic, Effective and Generalizable to environments that matter to the learner)
• 12 other assessment processes were compared to
  • skill domains assessed: iASK is the only tool that measures all domains.
    • Pivotal Skills
    • Movement
    • Barriers to Growth
    • Complex Verbal Behavior
    • Committed Action
    • Close Relationships
    • Community Contact
    • Self-care
  
  
  • applicable age: iASK is one of only 2 tools without age cap.  The other tool does not have a software available, and measures one skill domain.

There is a growing community of scientists who believe that
“ideographic” data and statistics (focused on individual people, through
time), hold the keys to the future of psychology, education, and
business

iASK is

• developed with evidence-based learning science and incorporates leading-edge modern science, Relational Frame Theory and Acceptance and Commitment Therapy, proven to positively impact lives.
• focused on a more complete and precise picture of individual psychological processes. It challenges assessments based on a false assumption of averages and norms.

The ubiquity of technology, and the advancements in the science are moving us towards approaches which can adequately address the level of the individual learner.

We define technology in two ways: learning science technology and information systems. 

The software we are developing is a cloud-based Angular web application with multiple relational databases which capture learner:

• skill data measures,
• analysis of data
• and learning plans.
• With more data and repeated assessments, it reflects skill data change over time.

It is scaleable, resilient and robust. It will comply with HIPAA and FERPA requirements. Our ultimate goal is to eventually utilize AI technology to explore trends and correlations between skills. As the product adoption grows, the data collection will support that goal.

Current and future workers face clear challenges. iASK audaciously
proposes a naturalistic science of psychological application where:
precise and individualized skill development plans in small achievable
increments may help us improve dignity and self-worth in our labor
force.

The learning science technology is the product of several evidence-based practices mentioned previously. Additionally, we incorporate a philosophy called ‘functional contextualism.’(https://contextualscience.org/functional_contextualism_0).

Precision Teaching: PT is a growing community of psychologists and educators with core beliefs: The learner is always right. Fluency matters. Make contact with reliable, and minimally biased data often. Be inductive. Always improve.

Process-Based Analysis: Best example, Todd Rose’s book, The End of Average suggests that comparative metrics like academic grading systems, personality tests etc., generally, have built systems based on a false assumption of averages and norms. In reality, no one is ‘average.

Our organization, High Sierra Industries, has been continuously adopting and using this philosophy and learning science methods in our own services and staff training for multiple years. We have first-hand experience and feedback from the learners.  A summary of a real case (Luk's story) can be found at https://hsireno.org/news/

Academic evidence:

• The processes underlying Acceptance and Commitment Therapy are vital to employment. Committed action towards clear goals and values is at the heart of effective and dignified work. As over 400 randomized controlled trials have demonstrated, it is arguably at the heart of a lot of human concerns: https://contextualscience.org/ACT_Randomized_Controlled_Trials
• The academic journal and other resources regarding Precision teaching: https://celeration.org  
• Dr. Stephan Hoffman: recently awarded the highly prestigious Humboldt Professorship (the most highly endowed research prize in Germany): https://www.humboldt-foundation.de/en/explore/newsroom/new-alexander-von-humboldt-professors-selected
• Dr. Steven Hayes: https://journals-sagepub-com.ezproxyberklee.flo.org/doi/full/10.1177/2167702618772296
• Precision Teaching: Precision Teachers deliver some of the most effective instructional designers on the planet. For example, students (who have historically fallen further behind each academic year) can be guaranteed to demonstrate at least two years growth in every subject, for each year of enrollment.
• Process-Based Analysis: Idiographic analysis is somewhat in its infancy. Statistical models which are sophisticated enough model individualized data streams are only beginning to emerge over the last decade: https://www-tandfonline-com.ezproxyberklee.flo.org/doi/abs/10.1207/s15366359mea0204_1
• Extended Evolutionary Model: Applied use of Relational Frame Theory (RFT) has produced some promising research and may represent a major bridge between basic behavior science, and human complexity. As a very small representative sample: RFT can move ‘perspective-taking’ abilities: https://link.springer.com/article/10.1007/BF03395719
• IQ: https://www.ijpsy.com/volumen16/num1/428/the-effect-on-intelligence-quotient-of-training-EN.pdf

We believe that a system can only be changed from inside out, and when people who are impacted by a solution are part of the development of the solution.  We believe in diversity, inclusion and equity and reflect those beliefs as we drive change. 

Both for the solution that we develop, and for changes we adopt in our organization, we believe that decisions should be evidence and data-based; debated and agreed upon; analyzed from different perspectives to ensure all stakeholders are heard.

• Raise awareness. Continuously engage with multiple stakeholders. Establish credibility. Share success stories, evidence and lessons learned.
  • Seminars, webinars
  • Articles (academic & whitepaper)
  • Pilot programs
  • Conferences
• Create curiosity: Understand what the enforcers will encourage educators to be curious about a new approach
  • Invite stakeholders to a demonstration or a pilot
  • Get stakeholders to pilot it themselves
  • Share what it means for them
• Collect feedback from stakeholders and users. Analyze.   
• Identify and if possible mitigate the upstream and downstream negative impacts and obstacles
• Ask stakeholders and users to identify whether they would adopt the tool and what it would take for them to make the change
  • If interested: we would collaborate with them on an implementation plan. Once collaboration is created, it is part of the ecosystem. Continue to monitor data for fidelity.
  • If not interested: gather feedback and continue to keep communications line open 
  • If interested but part of a larger organization (like school district): Work with the interested party to understand their organization’s politics, priorities, procedures. Get senior leader & middle management champions’ buy-in. Demonstrate the tool to different parties as necessary. Request them to identify a cross-organization implementation team to pilot the tool within a specific group supportive of its success (e.g. high school juniors at a specific school). Once pilot is complete, cross-organizational implementation team generates a pilot report where results and feedback are discussed. Ideally organization decides to expand the pilot. We would repeat the same steps until the organization is comfortable with a wider expansion / adoption.

For 2021, our goal is to complete the development of the software, all the training material and relevant materials by in first half part of the year.  We will continue to pilot the method offline and online (in staging instance) until all privacy and security controls are in place and the tool meets our UI/UX expectations. 

We are already able to assess learners remotely.  We will focus on improving our methods and communication systems.  Given our current funding, by 2022, we planned to impact ~1,000 novel learners every year.

If we become a solution partner through this challenge, it would allow us to complete these items in less time. It would also give us the ability to expand geographical reach and obtain legal, IP and marketing support.

The biggest impact is COVID-19.  We are a social enterprise generating our own funding. COVID-19 impact on our fee-for-service contract revenue is significant. 

Our second main concern is resistance:

• iASK is a powerful technology and it works. It is also new and represents a different approach to learning.
• Putting the learner in the center of education / skill learning progress will threaten certain professionals.
• Individualized approach to learning is typically attributed to people who need ‘special plans’. Learners or family members may not want to participate for fear of being labeled.
• iASK builds in accountability providing specific, measurable and achievable learning plans.

We have been and will continue to be successful financially by identifying iASK programs goals, measurable outcomes to those goals and a budget to support that.   

Adoption and support of the solution through this challenge and partnerships will provide us with more resources to create change (see our process of change detailed above). 

Our vision is to constantly improve the time it takes for a learner to gain fluency in a skill.  To achieve that vision, we need to understand long-term outcomes and correlational data between different skills and pinpoints.  These require time and thousands of learner data in the system for stronger predictive analysis.

We have all the relevant data collection points identified in the software. We will need to expand storage capacity, develop and implement AI with predictive capabilities.

This solution was created to support our organization’s vision and mission.

Vision: People with disabilities are visible, valued members in the community.

Mission:We develop, deliver and use learning systems that benefit people with disabilities and those who support them.

Our solution development team and our leadership team come from diverse cultural, professional and academic backgrounds.  Their uniting and unbreakable link are

• absolute passion to support learners to find their voice and succeed to their goals;
• vision to improve the process of teaching and learning;
• and refusal to accept assumptions, stereotypes and traditional ways for the sake of ease.

Collectively the full-time solution development team have over 30 years of hands-on teaching experience ranging from extremely challenging learners to typical learners.

• State of Nevada: range of fee-for-service contracts
• Haugland Learning Center (Ohio): pilot partner site, focused on students with Autism
• Washoe County School District (Nevada): special-ed teacher training
• Alpha Productions (Nevada):  training for employees and new hire onboarding training
• RAVE Family Foundation (Nevada): training for employees and volunteers

Peer-review and scientific enhancement is important for our organization and solution. We continue to be in communication with the scientific peers at University of Reno, Standard Celeration Society, Association of Behavior Analysis to maintain professional relationships and knowledge of latest technology, as well as provide information on our work and create awareness.

A social enterprise model providing several services:

• Skills assessment to facilitate goal attainment on major life activities, including career entry or transitions
• Day Habilitation programs focused on skills acquisition or maintenance plans
• Job training & readiness programs
• Production and assembly fee-for-service contracts (no 14c wages)
• Diversity and evidence-based decision-making training to organizations and schools;
• Providing residential support in a unique, shared living arrangement creating self-employment for providers and natural support for people with disabilities

Services that we provide in addition to the learning system solutions are detailed above and have sustained the organization financially for 43 years.  Where a funding opportunity is in alignment with our mission and vision, we also apply for grants and/or create mutually beneficial collaborations.

Our organization utilizes sophisticated Materials Resource Planning (MRP) System and has scalable infrastructure and space to adjust to different demands. We are also audited independently.

We have not raised funds (e.g. seed funding, debt, etc) specifically for the development of solution. Until 2018, the solution development was an internal project to support our services.  Since 2018, the organization invested its own funds generated through social enterprise model to develop the software prototype, actual software and run pilot collaborations. Approximately $60k in revenue per year was generated by team by delivering training as detailed above

To date we have applied for two grands to raise funds for the solution. One grant for $20k was awarded this year to provide 20 learners with the opportunity to do assessment.

We are waiting for the outcome of the second grant application which was for ~$50k towards the security testing of the software. 

~$200k for software development, cloud-based storage, security and maintenance

~$240k for wages and benefits of development team

As a non-profit, we have positively impacted the community in Nevada for the last 43 years. We would like to exponentially increase our impact across the nation. We developed a technological solution that can be scaled. We believe we can change the world. However, we need partnerships like this challenge can provide to help us understand the latest with core technology advancements, build a successful business model and increase our exposure geographically. 

This partnership would also help us verify and validate our dreams and ambitions. Collaborations we would build through this challenge would help us to make greater impact in any of these areas. 

We have some of the best minds in the field of precision teaching, relational frame theory and science of learning that contribute to our solution. However, we currently outsource our software development and marketing efforts.  Additional partnerships will help to more specifically define our outyear goals, as well as provide us more specialized expertise.   

Business model & Service Distribution: iASK can be used to assess skills and provide learning plans for any transition environment.  Explore different business models that can help scale the solution nationally and internationally in efficient ways.  

Solution technology: software is being developed using Angular, hosted on AWS. We need more technical support to drive our conversations with the developers, as well as actual development know-how for incorporating advanced analytics. 

Marketing & Legal: specialized expertise to help with marketing reach; Intellectual Property protections and software regulations

Bill & Melinda Gates Foundation:

• Technology development and security solution support for our organization,
• Sponsorship of implementation of iASK in disadvantaged and underserved communities.  

MIT Lab: Technology development support; predictive analysis / algorithms  

Society of HR Management (SHRM): work with organizations to change mindsets about employment skillsets and the ability to measure on an individual level. The focus with SHRM would be to works towards employment of people with disabilities, as well as those transitioning from prison and re-entering into life.   

Autism Speaks: to scale our solution and our experience with a national partner and advocate  

Veterans Administration: focusing on scalable partnerships to help veterans transitioning out and looking for careers in the labor market.

Craig Newmark Philanthropies: focusing on employment opportunities for veterans and military families 

Other organizations and partners fighting homelessness interested to use iASK to provide employment solutions