BAA-AFRL-RQKH-2015-0008-Atch7.pdf

Attachment 7

BAA-AFRL-RQKH-2015-0008

15 July 2015

STATEMENT OF OBJECTIVES

Task Order 0002 “Trust-based Design Through Transparency”

1.0 Background

This effort is soliciting innovative research concepts that advance the overall mission of the Human Trust and Interaction Branch (711 HPW/RHXS), Human Insight and Trust (HIT) Team which is to conduct research to develop, test, evaluate and transition methods to foster human-robot teaming, methods to calibrate trust, and methods to foster transparency in human-machine contexts. This effort seeks to understand how the construct of transparency (methods to foster shared awareness and shared intent between a human and a machine) influence trust and performance within a human-machine context. Transparency methods can involve proving the human operator with insight into the systems’: capabilities and limitations, environmental awareness/sensing capabilities, analytical underpinnings, purpose, social intent and social interactions, as well as human state monitoring and adaptation. Assessing human reactions to these various transparency methods is a key tenant of this research. Scientific advances in the construct of transparency can support a need for additional research from both a robot perspective and human-machine interaction perspective by identifying (and validating through experiments – or field studies) design principles to foster shared awareness and shared intent between humans and machines. Research is also needed to empirically identify the gamut of factors that shape the human-machine trust process.

2.0 Scope

The contractor will provide basic, applied, and advanced technology development research, development and demonstration for understanding the trust calibration process. The domains of interest include human-machine interaction with actual automated systems that have been fielded or are close to being fielded or automated systems that are under development for the Air Force. The present research may include both laboratory and field research. The contractor will collaborate and perform in-house research conducted at Wright-Patterson AFB;

conduct field research at government facilities; and/or industrial settings; and/or conduct research at university settings. “In-house” refers to the collaboration, teaming, and execution of research that will be done by the contractor at the government facility. The research proposed herein aligns to two core areas:

(1) Human-Machine Transparency Methods – This effort seeks to empirically demonstrate the costs and or benefits of advanced transparency methods (i.e., design principles or training tactics used to foster shared awareness and shared intent between human operators and their tools.

(2) Trust-based evaluation – This effort will examine operator trust and reliance on actual systems in the field through interactions with operators (e.g., pilots, analysts, software coders, engineers).

3.0 Tasks and Technical Requirements/Objectives

The work under this effort will be implemented through an indefinite delivery, indefinite quantity (IDIQ) contract task order.

3.1 Technical Areas/Objectives

3.1.1 Human-Machine Transparency Methods

This technical area seeks to provide empirically-driven design principles for transparency methods validated through experimental methods. Transparency is the process for establishing shared awareness and shared intent between humans and machines.

3.1.4 Trust-Based Evaluation

This technical area will involve capturing, studying, and observing operator evaluations of fielded systems to garner: 1) an understanding of the level of trust and reliance evidenced by the operator, 2) an understanding of what factors shape operator trust on the system, and 3) an understanding of what factors can be shaped to alter these beliefs among operators. The key attribute of this research will be the distilling of design principles related to transparency in human-machine contexts.

3.2 Specific Technical Objectives

3.2.1 RHXS In-house laboratories

3.2.1.1 Research

The contractor shall perform in-house research conducted at Wright-Patterson AFB in the Human Insight and Trust Laboratory, Human-Robot Teaming Laboratory, and government-affiliated laboratories associated with RHXS. This research shall require the contractor to perform at various levels within the research domain, to include research assistants, and subject matter experts (SMEs). The contractor shall collaborate with government researchers to create and execute experimental designs, IRB preparations, data collections, data analyses, and technical writing for RHXS research. The contractor may provide hardware or software creation or acquisition as the research deems necessary.

3.2.1.1.1 Human-Machine Transparency Methods

The contractor shall research transparency and transparency methodology as it as it pertains to shared awareness and intent between humans and machines. The contractor shall research and perform experiments on transparency methods, which may include: 1) highlighting the system’s social intent as it relates to the human operator; 2) providing dynamic or static contextual bounds signaling the system’s capabilities and or limitations; 3) providing the human operator insight into the analytical underpinnings of the system’s algorithms, decision rationale, performance assumptions; 4) provided the human and understanding of the system’s method for sensing/interacting with the environment; 5) providing the human operator insight into the system’s purpose, goals, and assumptions about teamwork; and 6) using human state factors (stress, emotion, performance limitations) to shape the performance, interaction strategy, and behavior of the system. The contractor shall conduct experiments that manipulate aspects of transparency and evaluate the impact on human operator trust and overall human-machine performance.

3.2.1.1.2 Trust Based Evaluation

The contractor shall create methods to and participate in capturing, studying and observing operator evaluations of fielded systems. The contractor shall garner: 1) an understanding of the level of trust and reliance evidenced by the operator, 2) an understanding of what factors shape operator trust on the system, and 3) an understanding of what factors can be shaped to alter these beliefs among operators. The contractor shall create appropriate technical reports to document these findings and create experiments to further develop the knowledge base in this field. The contractor shall lead evaluations of systems such as the Automated Ground Collision Avoidance System, Air Collision Avoidance System, and other forms of advanced automation and autonomy within Air Force systems. During these evaluations the contractor shall identify what design features influenced operator trust of the system. The contractor shall conduct operator interviews, cognitive task analyses, observations, and surveys to gauge and assess operator perceptions of the technology. The contractor shall develop technical reports detailing design principle recommendations, analyze data, and write conference papers and journal articles.

3.2.1.2 Participant Recruitment

The nature of the current and planned research in RHXS requires the use of human subjects. Some research may require specialized groups of subjects (e.g., those with software coding experience), which necessitates exploration by the contractor to recruit for and maintain a database of said subjects for in-house research conducted in the Human Insight and Trust Laboratory, Human-Robot Teaming Laboratory, and other government-affiliated laboratories for HIT. Other subject pools may be required, and would include: military participants, non-student samples, and student samples. The contractor shall maintain these samples in the same database as for specialized subjects.

3.2.2 RHXS External Laboratories

The contractor shall collaborate and perform research related to the RHXS mission with academia to expand RHXS research objectives. The contractor shall collaborate with researchers at these facilities and follow all government and facility-specific regulations.

3.2.3 Field Research

The contractor shall perform field research including cognitive tasks analysis, interviews, and survey research in the domain of human-machine trust. Field research may occur at Air Force bases, or other government or industry locations.

3.2.4 Demonstration

The contractor shall perform systems integration of software and hardware components as necessary to perform various technology demonstrations and evaluations.

3.2.5 Management of Effort

3.2.5.2 The contractor shall manage technical schedule and cost satisfaction of the task order;

plan, prepare, conduct and document reviews; track CDRL satisfaction; and interface with the customer to assess performance satisfaction and re-plan the project as needed. Contractor shall comply with all applicable human use and animal use regulations.

3.2.5.3 Contract Holidays Clause Will Apply To This Effort As Follows

The prices/costs in Section B of the contract include holiday observances; accordingly, the Government will not be billed for such holidays, except when services are required by the Government and are actually performed on a holiday. The following days are contract holidays:

New Year’s Day, Martin Luther King Jr. Day, Presidents Day, Memorial Day,4th of July, Labor Day, Columbus Day, Veterans Day, Thanksgiving Day, and Christmas Day.

4.0 Security Requirements

This effort is unclassified.

4.1 Position of Trust

All contractor personnel require a minimum of a National Agency Check with written Inquires (NACI)/SF85 for any position that requires access to the internet, use of automated systems or unescorted entry into restricted or controlled areas prior to reporting for duty in performance of any task order. The investigation is not for a security clearance; it is for a position of trust. This is a mandatory requirement set forth in DoD 5200.2-R, Personnel Security Program and AFI 31-501, Personnel Security Program Management. All documentation required for security certification shall be the responsibility of the contractor. No foreign nationals shall be employed for any task order issued under this contract without prior approval of the Government.

4.2 Operations Security (OPSEC)

The contractor shall provide OPSEC protection for all sensitive/critical information as defined by AFI 10-701 (Operations Security), the 711 HPW OPSEC Plan, and critical information list. The contractor shall participate in the 711 HPW sustained OPSEC awareness training or include OPSEC training as part of their on-going security program. The 711 HPW OPSEC coordinator will evaluate the OPSEC posture of AF contract activities and operations.

4.3 Information Security and Force Protection

The contractor shall provide Information Security and Force Protection training as defined by AFI 16-1404, Air Force Information Security Program and AFI 10-245 (Air Force Antiterrorism Standards). The contractor shall participate in the 711 HPW sustained information Security and Force Protection/Antiterrorism training or include this training as part of their on-going security program. The 711HPW Security Managers will evaluate the training posture of AF contract activities and operations. This requirement is set forth in AFI 16-1404, AFI 10-245 and applicable AFMC and local supplements.

5.0 CDRLs:

CDRL # CDRL NAME FREQUENCY

A001 Scientific and Technical Reports – Final Report ONE/R A002 Funds and Man-Hour Expenditure Report MTHLY A003 Contract Funds Status Report QRTLY A004 Status Report MTHLY A005 Presentation Material ASREQ