BAA-RQKH-2015-0001-SOO.doc

Attachment 1 to BAA-RQKH-2015-0001

BAA-RQKH-2015-0001

Statement of Objectives

Methods and Technologies for Personalized Learning, Modeling and Assessment

7 Nov 2014

Objective: This program deals with science and technology development, experimentation, and demonstration in the areas of improving and personalizing individual, team, and larger group instructional training methods for the airman. The approaches relate to competency definition and requirements analysis, training and rehearsal strategies, and models and environments that support learning and proficiency achievement and sustainment during non-practice or under novel contexts. This science and technology effort focuses on measuring, diagnosing, and modeling airman expertise and performance, rapid development of models of airman cognition and specifying and validating, both empirically and practically, new classes of synthetic, computer-generated agents and teammates. The program’s focus is expanding based on very successful collaborations with civilian industry, academic and other agencies to use the extensive research foundation and ongoing military work to support similar needs in the private sector and in fundamental research activity. This expansion includes, but is not limited to, the following: (A) having very specific and valid knowledge, skill, experience and performance (K, S, E and P) information on what airmen in various roles are expected to be able to know and do and how they are expected to operate and perform; (B) identifying individuals and teammates with this K, S, E and P information already, matching them with mission requirements and putting them to work as quickly as possible; (C) developing methods to define and quantify what is meant by terms such as “proficient,” “mission ready” and “effectively and efficiently operating”; (D) developing models that can help airmen to operate at a more effective level of capability sooner – i.e. synthetic coaches, assistants, and teammates that can help airmen fill in gaps in their understanding, capture critical knowledge from experts to enhance learning and performance in future trainees, work side-by-side with airmen to ensure mission success, or assume responsibilities for tasks to reduce workload on AF operators or to more effectively operate in constrained or contested environments ; (E) improving the fidelity and the precision and immersion of the environments airmen are placed in so that training, rehearsal, exercise ,and actual operations become seamless and synonymous with one another and in contexts that are fully and securely integrated live, virtual, and constructive contexts; (F) developing accurate representations of airmen and their behavior to predict individual knowledge, skill, understanding, and proficiency to inform training interventions, instruction, and feedback; (G) linking individual models to reactions to tactics and doctrinal changes, as well as map responses to trust, cultural and psychosocial characteristics for future training and assessment approaches to conflict resolution and decision making; (H) knowing what environments (simulation and live operations) are available and that can be appropriately tailored and mixed to support getting the necessary K, S, and Es needed to develop the knowledge and skills to operate effectively and matching learning requirements and objectives to environments and vice versa; and (I) creating the technologies, tools, instrumentation, and enterprises and standards, to ensure high fidelity immersive LV, and C training and experimentation.

Background: Research in this area has historically focused on the Service, Joint, and Coalition levels of play and on developing environments, methods, and data on best practices in research and application that serve the end user – in this case, the warfighter today and decision makers in the future – at the tactical, operational, and eventually the strategic levels of operations.

Previous and on-going collaborations address not only US needs, but also involve several nations around the world to develop and validate the data needed to make national investment decisions on distributed high fidelity simulations. The collaborations supported by the RHA research team represent a seamless partnership of military, civilians, contractors, academic, and operational collaborators. They have resulted in investment decisions that enable distributed modeling and simulation research and operations internationally. In several key areas, this research has helped to define the requirements for, and to quantify the payoffs from, multinational distributed training and engineering research activities. The work has helped to define the Mission Essential Competencies (MECs)SM needed for mission success and to help define a common set of metrics that can be used to demonstrate quantitatively training effectiveness and transfer to actual operations. Finally, this research has been instrumental in establishing the minimum acceptable airman-centered data standards and toolsets for collaborative mission planning, briefing and after action review; tools that can be used irrespective of whether warfighters are using high fidelity simulation, actual operational systems, or exercises on instrumented ranges and in immersive environments.

Scope: The scope of this effort includes the development, test, and validation of training approaches, methods, tools, instrumentation, and enterprise infrastructure for learning and performance. It includes modeling and the design and execution of laboratory and field studies to evaluate alternative live, virtual, and constructive approaches, solutions, and practices in military and civilian mission areas such as autonomous operations, manned and unmanned air and ground operations, command and control, cyber operations, emergency response, intelligence, surveillance, and reconnaissance, tactical air combat, and integrated air, land, sea and space operations.

Technical Requirements: This effort shall include work in any or all of the following technical areas.

Area 1. Competency-Based Approaches for Training and Performance Requirements Analysis: Develop, refine, and/or validate methods and technologies to define and personalize workforce knowledge, skills, experiences, and other attributes to increase training and job success. Also needed are competency-based approaches that can inform and drive models to forecast and analyze alternative personnel and system requirements to meet evolving weapons system capabilities and mission contexts. Further, there is a need for requirements traceability and learning trajectory development such that we can define competency needs from the time an individual enters the pipeline to the time they retire and to develop career long learning models and criteria based on known proficiencies and experiences. Finally, developing and validating linkages between competency specifications and models to forecast personnel requirements for new systems, mission, and occupations.

Area 2. Criterion Development and Readiness Standards Definition for Non-Traditional Work Applications: Develop, evaluate and validate novel methods to measure and model individual and team learning, performance, and proficiency. Methods for measurement and tracking in formal job and work structures as well as ad-hoc and just-in-time work and performance instances are desired. Automated tools to persistently monitor and diagnose performance and identify knowledge and skill requirements and gaps are also needed. Criterion development and assessment may center on one or another area or construct - the following are examples (but there are potentially numerous others of relevance to this area of need): communication (speech, text/chat) monitoring, body-worn physiological state tracking and mobile performance tracking and analysis; team interaction either collocated or at a distance; collaboration and trust – especially in airmen and machine teams; situated knowledge or judgment; decision making; understanding; and learning.

Area 3. Ontology Development for Representing Subject Matter Knowledge and Skills: Develop representational formalisms and encoding processes to allow for strategic and tactical knowledge from subject matter experts (SME) to be translated into formal structures that can be used by cognitive and machine learning modelers in the development of models that instantiate psychologically valid representations of expert task performance. These formalisms must capture SME knowledge, skills, abilities and interactions while minimizing or eliminating the knowledge engineering requirements that are typical for the implementation of such knowledge bases, and that today limit improvements in computer generated force capabilities, adaptive and intelligent tutoring applications, and performance aiding and supporting agents, coaches and teammates.

Area 4. Approaches for Using Real World Performance Data to Improve the Efficiency and Validity of Constructive Models and Agents in Live-Virtual-Constructive (LVC) Environments: Develop methodologies and algorithms to inform computational and machine learning models/architectures to acquire context from airman and system-level performance and interaction data in and across LVC environments. One opportunity is to develop more efficient and unobtrusive ways to characterize these data such that models/architectures can obtain and use information more directly through experience and interaction with LVC environments and to use the data as part of the model development, refinement and validation process. Further, rapid generation of content contextualization and tokens that become part of the data pedigree is critical to this area of need. Developing and demonstrating techniques for contextualizing data to “train” and “teach” models how to perform in complex environments is essential to overcoming knowledge engineering bottlenecks that limit the scope and scale of contemporary modeling activities today.

Area 5. Modeling for Learning and Agent Development: Develop theoretically- and empirically-motivated models of understanding and performance that (a) account for major phenomena in the relevant published literature, b) leverage existing computational process models developed in the basic cognitive modeling research effort and/or (c) enhance the state of the art in the representation of highly interactive synthetic humans as “friendly assets,” synthetic teammates and instructors, adversary players, agents and coaches, and assessors for live and virtual environments and contexts.

Area 6. Complimentary Family of Trainer Development and Validation: Develop, integrate, and evaluate infrastructure and enterprise architectures and tools for realistic and secure, multi-technology environments that support personalized live, virtual, and/or constructive learning and training. Create and demonstrate live, virtual, and/or constructive environment and content development, delivery, and assessment tools for workforce training, tracking, and management. Approaches and demonstrations of environments, content, and methods that can be generalized beyond military applications to civilian/military, and civilian work environments are desired. Examples include but are not limited to first responder/receiver, emergency operations, humanitarian assistance, border reconnaissance and protection, cyber operations, remotely operated vehicles, command and control, intelligence, surveillance, and reconnaissance resource allocation, and science technology engineering and mathematics.