Attachment 3_MATRIX TO 0002 SOO - Machine Learning_FINAL.pdf

FA8650-21-S-6002

Attachment 3

Task Order 0002 - Statement of Objectives (SOO) Machine Learning Analytics

Date: 12 Apr 2021

1.0 Background

Machine Learning (ML) and Artificial Intelligence (AI) have seen a resurgence of interest and utility as computational power dramatically increases while high-end computing costs plummet. At the same time, technological advances in ubiquitous platforms such as cell phones, tablets, personal computers, and gaming systems frequently feature applications integrating leading-edge AI and ML capabilities. The US Air Force strives to maintain international scientific leadership, and widen the technology gap, providing our Airman with state-of-the art capabilities and decision-making tools far beyond those of our adversaries. Research and utilization of leading-edge ML and AI is a significant component of this strategy.

Specifically, this task order seeks to apply machine learning, data science, signal processing, and computer science techniques to process and exploit multi-dimensional data across a variety of modalities and media types, focusing on multimodal human language technologies, explainable artificial intelligence (Topological Data Analysis, Knowledge Graphs, etc.), and machine-learning and statistics-based data science analytics. Applications of these capabilities could be leveraged by the US Air Force when integrated into their existing systems and designed into new systems being introduced into the research and acquisition pipelines. Air Force Intelligence organizations and a variety of operational units are interested in supporting research and deploying products using ML and AI in ways that improve their efficiency and effectiveness, and rely on AFRL to provide such capabilities.

2.0 Scope

This effort includes research and development of mathematical and computational techniques and models for understanding machine learning (ML) and artificial intelligence (AI), and applying those methods in ways that benefit US Air Force and US Space Force Warfighters and other DoD personnel. This includes the development of applications built around these methods and models, as well as collaboration with other organizations for integration into existing applications.

The broad scope of research in Machine Learning Analytics also involves the collection and aggregation of data, and includes, but is not limited to, work to identify, collect, store, condition, aggregate, analyze, measure, visualize, and manipulate data. Many of these steps and processes are directly integrated into the current research products outlined below, such as explainable artificial intelligence, human language processing, and aggregation and visualization of publically available information.

Modeling and simulation (M&S) is also valuable as both a mechanism for testing ML and AI techniques, as well as a useful tool integrating specific applications for ML and AI. M&S applications of ML and AI may take place in simulated, virtual, augmented, or operational environments, and may also support research evaluating the suitability and impact of human interface technologies using these methods. M&S may include providing mission scenarios to support human interface technology assessments, integrating hardware and associated software into virtual environments or augmented environments to evaluate human interface technologies, and the development and execution of research study protocols to include the collection and analysis of data obtained from human subjects performing in augmented and virtual environments.

3.0 Technical Objectives

3.1 Conduct research to develop and extend breakthrough advances in the sciences and engineering that have enabled the collection and exploitation of massive amounts of high resolution data across a variety of modalities and in various domains of interest. Technical focus areas include: developing machine learning and explainable machine intelligent data exploitation approaches, providing dynamic visualization strategies to convey machine learning derived solutions, and developing advanced environments for human-machine teaming, data exploration, and exploitation. Research should include techniques such as Deep Learning, Knowledge Graphs, Topological Data Analysis (TDA), and other methods which can be used to create explainable architectures to inform descriptive visualizations, create comparative and counterfactual examples, and automate hypothesis generation and testing.

3.2 Conduct research to understand, develop, and assess technologies for processing human language found in diverse types of digital media files and formats. Investigate and implement techniques for converting speech to text (including automatic speech recognition), text to speech synthesis, and machine translation between languages. Augment machine translations and perform advanced queries by including multimodal data sources. Incorporate extraction of specific topics, formulae, tables, and plots from scientific, technical, and medical literature. Integrate and expand research in human dialog and conversational systems. Implement novel AI and ML techniques to enhance machine translation capabilities. Design and adapt solutions for militarily-relevant languages and domains.

3.3 Develop, mature, and apply novel, contemporary, and traditional data analytic, statistical, and machine learning technologies for a range of research and operational missions. Utilize network simulation, characterization, analysis, and monitoring for datasets and data streams. Define and construct an ensemble of machine learning utilities, analytical tools, and user interfaces within an automated structure to produce appropriate analysis and data visualization that does not require expert mathematical or domain-expert knowledge to create, comprehend, and apply results. Research examples include, but are not limited to:

o Analytics and visualizations enabling rapid knowledge discovery from textual and multimodal sources to identify and explore relationships between topics of interest;

o Exploitation and analysis of publicly available information (PAI) to automatically generate domain data representations and standardized workflows to increase domain understanding and situational awareness;

o Generate representations to monitor and improve the health, wellness, and operational effectiveness of Airmen based on accurately and unobtrusively tracking biological details and communications data related to their workflows and operational activities.

3.4 Explore the continual development and advancement of hardware and software to meet the previously identified technical objectives. Identify gold-standard practices for data analytics and real-time analysis, or establish such standards if none exist. Develop or modify existing warfighter interface software for operational and/or relevant representative environments as well as develop and use software packages that provide optimal research capability. This includes design and development of user interface designs, Input/Output (I/O) devices and specialized hardware, algorithms, integration with existing/new technologies, modeling, simulation, wargaming, and implementation of software for the development of technology, data collection, and data analysis. Software and system development must conform to industry best practices, including the use of structured and documented development methodologies and source code control repositories.

3.5 Evaluate software models, simulations, and systems involving machine learning research in laboratory settings and operationally-relevant environments to evaluate data collection for information analysis tasks and provide systems and solutions enhancing Airman performance and situational awareness, and improving communication effectiveness. Perform test setup, test operation, data collection, data plotting, and data analysis and reporting to meet test program research objectives. Utilize best-practices and knowledge of experimental methodologies to maintain a high level of research integrity, conduct appropriate descriptive and inferential statistical analyses using the collected data, verify the integrity and accuracy of the data collection and analysis capabilities, and develop methods to ensure sanitization of data or software.

3.6 Analyze objective and subjective data from the evaluations and summarize qualitatively and quantitatively as appropriate, using accepted industry practices (e.g., graphical procedures, parametric repeated measures Analysis of Variance, and non-parametric statistical methods). Document the methodology and results from data analyses for each evaluation. Prepare materials in support of presentations and publications. Conduct evaluations as formal research using accepted experimental design processes:

o Some tools and architectures can be validated through case studies.

o Usability assessments are more appropriate for some research, and should be conducted using accepted industry practices. Example usability assessments include user surveys, design (cognitive) walks-through, think-aloud protocols, paper and pencil evaluations, expert evaluations, usability testing, and field studies.

3.7 Develop test plans and human-use protocols (including Institutional Review Board (IRB) human use research protocols), perform the acquisition and setup of specialized laboratory and test facility equipment to meet test program objectives, develop and use software packages that provide optimal research capability, and perform data collection, analysis, and reporting of research results.

3.8 Collaborate with, and leverage technologies from, national and international government, academic, and industry agents to enhance advanced research capabilities. Form and secure partnerships and research relationships both within and outside of AFRL. Participate in local, national, and international workshops, conferences, and events.

4.0 Operations Security (OPSEC)

The contractor shall provide OPSEC protection for all sensitive/critical information and indicators involved in execution of this Task Order, as defined by AFI 10-701 (Operations Security). 711 HPW Critical Information and Indicators are protected under the 711 HPW Operations Security Program and the 711 HPW Critical Information and Indicators List (CIIL). Contractor employees granted access to critical information and indicators will be provided initial OPSEC training by the 711 HPW OPSEC Coordinator upon in-processing and prior to being granted access to CIIL items related to the Task Order. The contractor will also participate in 711 HPW’s annual OPSEC training and education programs, which includes periodic updates and refresher training on CIIL items applicable to the Task Order.

The 711 HPW OPSEC coordinator will evaluate the OPSEC posture of AF contract activities and operations.

All contractor personnel require a minimum of a Tier 1 background check

(T1)/SF85 for any position that requires access to the internet, use of automated information systems to cover standalone computers or unescorted entry into restricted or controlled areas prior to reporting for duty in support of any requirement. The investigation is not for a security clearance; it is for a position of trust. This is a mandatory requirement set forth in DoDM 5200.02_AFMAN 16- 1405, Air Force Personnel Security Program. All documentation required for security certification shall be the responsibility of the contractor.

5.0 Additional Requirements

• Highest Security Classification: TS/SCI

• Deliverables

• Reports: CDRLs: A001, A002, A003, A004, A005, A006, A007, A008, A009, A010, A011, A012, A013, A014, A015, A016, A017, A018, A019

• Software: All Laboratory software developed

• Hardware: All Special test equipment developed or acquired.

• ITAR: Applicable

• Available Base Support/Facilities: The contractor will be provided access to the following Building 248 and Building 33 laboratory facilities to conduct research:

• SCREAM Laboratory (Bldg 248, Rm 109)

• Battlespace Visualization Laboratory (Bldg 248, Rm 306B)

• Observatory, with 17” Telescope (Bldg 33 bridge, Rm 3001A)

• Classified processing areas (Bldg 248, Rms B07, B03)

• GFP/GFI: None

• Flight Test: No

• Human Use/Protection of Human Subjects: Yes

• Animal Use: No

• Hazardous Materials: No

• Radioactive Materials: No

• Period of Performance: 57 months technical plus 3 months for Final Technical

Report (60 month POP)

• Estimated Funding Profile: The estimated funding for this task order is:

Fiscal Year FY 22 FY 23 FY 24 FY 25 FY 26 Total Total Amount

$1.0M $1.0M $2.0M $2.5M $2.5M $9.0M