BAA-RQKH-2015-0001-IndustryDay-Charts.pdf

Integrity Service Excellence

Methods and Technologies for Personalized Learning, Modeling and Assessment Industry Day Presentation

Date: 6 Nov 2014

Dr. Winston Bennett, 711 HPW/RHA, PM Helen Williams, AFRL/RQKHA, CO

PURPOSE

To Provide the Communities of Interest with Details on Our New Open BAA Opportunities

PROGRAM/PROJECT

INFORMATION

• The Methods and Technologies for Personalized Learning, Modeling and Assessment

• Scope: Science and technology development, experimentation, and demonstration in the areas of improving and personalizing individual, team, and larger group instructional training methods.

• Competency definition and requirements analysis

• Measuring, diagnosing, and modeling human expertise and performance

• Rapid development of models of human cognition

• Specifying and validating, both empirically and practically, new classes of synthetic, computer-generated agents and teammates.

• Training and rehearsal strategies and models

• Environments that support learning and proficiency achievement and sustainment during non-practice or under novel contexts

• Replicate human cognitive processing in synthetic instructors, teammates

• Tools for personalized, persistent training

• Create and validate learning and assessment methods to accelerate proficiency and readiness

• Demonstrate affordable, adaptive, and open solutions and options

Create and validate methods, models and technologies to promote learning and persistent readiness in and across mission contexts

INFORMATION

Learning theory and pedagogical underpinnings not leveraged/validated

Content and objectives are the same for everyone – one-size-fits-all

No persistent assessment or monitoring of performance

Naïve representations of humans, threats, and capabilities

3-6 month prep for live and virtual exercises

12/1 support to trainee ratio for design, delivery, mgt, and AAR

Persistent, secure LVC integration/access for all partners does not exist

Challenge 1: Principled Training Design and Assessment

- Validated tools to tailor and optimize content and metrics across individuals and teams

Challenge 2: Realistic, Adaptive and Interactive Scenario Based Training

- Persistent integration of real world events and content into scenarios and syllabi

Challenge 3: Persistent, Affordable, Integrated LVC Training

- Seamless, secure integration of training systems across services and coalition partners

5th gen Models, Scenarios, Data

• Threat models and authoring

• Assessment methods Common Data Exchanges for Gaming and Training

• Distributed perf tracking Common After Action Review Tools:

• Integrated AAR/Reporting

• Adaptive Feedback and Mgt LVC Demos, Data and Lessons Learned

Tech Availability

Methods and tools for realistic and secure training and assessment in and across LVC contexts

• Validated models: human behavior and threats

• Performance models for automated instructional delivery and management

• Agent-based scenario and syllabus delivery/mgt

• Adaptive strategies for feedback and AAR

• Common methods to capture/analyze/report performance data across L and V contexts

Notional Technology Investment Schedule (FY) As of 5 Nov 2014 Prior 15 16 17 18 19

711 HPW/RHA

Technology

Benefits to the Warfighter

Description

• Tactically relevant and Instructionally-rich training •Methods for adaptive Individualized training •Models for enhancing learning and performance in DMO and LVC rehearsal and exercise events

• Performance-based design, delivery and evaluation

• Persistent readiness assessment and tracking •Operational impact assessments of technology integration

Integrated live and virtual performance- based debrief Constructive

Virtual

Live Performance Measurement and Tracking of Live and Virtual Entities

Scenario Development, Delivery, and Management

5 5 6

DELIVERING: Training and assessment development, delivery, and management models and metrics

Proficiency

Time

Future Today

Mission Ready Airmen

More Career Time as a Mission Ready Airmen

– Purpose: Solicit unique, creative, revolutionary ideas and concepts that support the overall mission of the Warfighter Readiness Research Division (711 HPW/RHA) and generate research not already defined and planned by RHA as part of its core S&T portfolio

• Capture non-core funded projects that are complimentary to RHA core programs

– Examples: AFOSR, Sec 219, STT, external funds

– Core program funds may be utilized for smaller, seedling projects

– Awards may accomplish sustainment research efforts

– Benefit: Enhances RHA core research by leveraging non-core S&T efforts

– Improves capability, expedites schedule, mitigates core funding shortfalls

– Conserves ceiling on core program contracts

INFORMATION

• This is a Performance-Based Service Acquisition

• Technical Areas:

• Area 1. Competency-Based Approaches for Training and

Performance Requirements Analysis

– Develop, refine, and/or validate methods and technologies to define and individualize workforce knowledge, skills, experiences, and other attributes to increase training and job success

– Create/refine competency-based approaches that can inform and drive models to forecast and analyze alternative personnel and system requirements to meet evolving weapons system capabilities and mission contexts

– Develop requirements traceability and learning trajectory development such that we can define competency needs from the time an individual enters the pipeline to the time they leave the Service and to develop career long learning models and criteria based on known proficiencies and experiences that track with the changes to work domains and mission sets

– Develop and validate linkages between competency specifications and models to forecast personnel requirements for new systems, mission, and occupations

INFORMATION

• Technical Areas:

• Area 2. Criterion Development and Readiness Standards

Definition for Non-Traditional Work Applications

– Develop, evaluate and validate novel methods to measure and model individual and team learning, performance, and proficiency

– Methods for measurement and tracking in formal job and work structures as well as ad-hoc and just-in-time work and performance instances are desired

– Automated tools to persistently monitor and diagnose performance and identify knowledge and skill requirements and gaps are also needed

– Criterion development and assessment may center on one or another area or construct - the following are examples (but there are potentially numerous others of relevance to this area of need): communication (speech, text/chat) monitoring, body-worn physiological state tracking and mobile performance tracking and analysis; team interaction either collocated or at a distance;

collaboration and trust – especially in human and machine teams; situated knowledge or judgment; decision making; understanding; and learning

INFORMATION

• Technical Areas:

• Area 3. Ontology Development for Representing Subject Matter

Knowledge and Skills

– Develop representational formalisms and encoding processes to allow for strategic and tactical knowledge from subject matter experts (SME) to be translated into formal structures that can be used by cognitive and machine learning modelers in the development of models that instantiate psychologically valid representations of expert task performance

– Formalisms must capture SME knowledge, skills, abilities and interactions while minimizing or eliminating the knowledge engineering requirements that are typical for the implementation of such knowledge bases, and that today limit improvements in computer generated force capabilities, adaptive and intelligent tutoring applications, and performance aiding and supporting agents, coaches and teammates

INFORMATION

• Technical Areas:

• Area 4. Approaches for Using Real World Performance Data to

Improve the Efficiency and Validity of Constructive Models and Agents in Live-Virtual-Constructive (LVC) Environments

– Develop methodologies and algorithms to inform computational and machine learning models/architectures to acquire context from human- and system-level performance and interaction data in and across LVC environments

– One opportunity is to develop more efficient and unobtrusive ways to characterize these data such that models/architectures can obtain and use information more directly through experience and interaction with LVC environments and to use the data as part of the model development, refinement and validation process

– Rapid generation of content contextualization and tokens that become part of the data pedigree is critical to this area of need. Developing and demonstrating techniques for contextualizing data to “train” and “teach” models how to perform in complex environments is essential to overcoming knowledge engineering bottlenecks that limit the scope and scale of contemporary modeling activities today

INFORMATION

• Technical Areas:

• Area 5. Modeling for Learning and Agent Development

– Develop theoretically- and empirically-motivated models of understanding and performance that ….

• (a) account for major phenomena in the relevant published literature

• (b) leverage existing computational process models developed in the basic cognitive modeling research effort and/or ….

• (c) enhance the state of the art in the representation of highly interactive synthetic humans as “friendly assets,” synthetic teammates and instructors, adversary players, agents and coaches, and assessors for live and virtual environments and contexts

INFORMATION

• Technical Areas:

• Area 6. Complimentary Family of Trainers Development and

Validation

– Develop, integrate, and evaluate infrastructure and enterprise architectures and tools for realistic and secure, multi-technology environments that support personalized live, virtual, and/or constructive learning and training

– Create and demonstrate live, virtual, and/or constructive environment and content development, delivery, and assessment tools for workforce training, tracking, and management

– Approaches and demonstrations of environments, content, and methods that can be generalized beyond military applications to civilian/military, and civilian work environments are desired

• Examples include but are not limited to first responder/receiver, emergency operations, humanitarian assistance, border reconnaissance and protection, cyber operations, remotely operated vehicles, command and control, intelligence, surveillance, and reconnaissance resource allocation, and science technology engineering and mathematics (STEM)

Thank You!

Questions?

Helen. Williams@us.af.mil Winston.Bennett@us.af.mil mailto:Williams@us.af.mil

INFORMATION

PROGRAM/PROJECT

INFORMATION

PROGRAM/PROJECT

INFORMATION