BAA-AFRL-RQKH-2015-0008-Amd03.pdf

BAA-AFRL-RQKH-2015-0008 - Amendment 03

Broad Agency Announcement Title: Trust in Autonomy for Human-Machine Teaming

Broad Agency Announcement Number: BAA-AFRL-RQKH-2015-0008

The following corrections/amendments are being made to the BAA:

1) Paragraph IV(2)(a)(iv), page 13 of 24 of the BAA, is hereby amended as follows:

FROM:

iv. Offerors must submit one (1) original and five (5) hard copies of their proposals via mail to the Contracting POC, identified in Section VII.

TO:

iv. Offerors must submit one (1) original and five (5) hard copies of their proposals to the Contracting POC, identified in Section VII.

BASIC SOO

2) BAA Basic IDIQ Statement of Objectives paragraph 2.0 Scope is amended as follows

(changes in bold):

2.0 Scope

The contractor will provide basic, applied, and advanced technology development research, development and demonstration for understanding the trust calibration process. The domains of interest include semi-autonomous, symbiotic, human-machine teaming (HMT) research; research to identify the psychological and physiological antecedents and consequences of suspicion within a human-machine context; research to understand the factors that shape trust calibration within a human-machine context; and research to understand the most significant components driving trust and performance within human-robotic interaction. The domains of application for this research may include but is not limited to automated tools within cockpits; analyst tools/aids for ISR analysis and exploitation; software code; interactive platforms such as automated translation capabilities; and advanced robotic systems within the Air Force. The present research may include both laboratory and field research. The contractor will collaborate and may perform in-house research conducted at Wright-Patterson AFB. The contractor may conduct field research at government facilities; and/or industrial settings; and/or conduct research at university settings; and/or other contractor proposed alternative sites. “In-house” refers to the collaboration, teaming, and execution of research that may be done by the contractor at the government facility. The research proposed herein may align to one of the following core areas:

3) BAA Basic IDIQ Statement of Objectives paragraph 3.2.1 Laboratory Research is amended as follows (changes in bold):

3.2.1 Laboratory Research

The contractor may perform laboratory research conducted at Wright-Patterson AFB in the Human

Insight and Trust Laboratory, Human-Robot Teaming Laboratory, and government-affiliated laboratories associated with RHXS. The contractor may conduct field research at other government facilities, industrial settings, university settings, and/or other contractor proposed alternative sites. This research shall require the contractor to perform at various levels within the research domain, to include research assistants and subject matter experts, to name a few. The contractor will collaborate with government researchers to create and execute experimental designs, IRB preparations, data collections, data analyses, and technical writing for RHXS research. The contractor may provide hardware or software creation or acquisition as the research deems necessary.

Task Order 0001 SOO

4) BAA Task Order 0001 Statement of Objectives paragraph 2.0 Scope is amended as follows

(changes in bold):

2.0 Scope

The contractor will provide basic, applied, and advanced technology development research, development and demonstration for understanding the trust calibration process. The domains of interest include semi-autonomous, symbiotic, human-machine teaming (HMT) research; research to identify the psychological and physiological antecedents and consequences of suspicion within a human-machine context; research to understand the factors that shape trust calibration within a human-machine context; and research to understand the most significant components driving trust and performance within human-robotic interaction. The domains of application for this research may include but is not limited to automated tools within cockpits, analyst tools/aids for ISR analysis and exploitation, software code, and advanced robotic systems within the Air Force. The present research may include both laboratory and field research. The contractor will collaborate and may perform in-house research conducted at Wright-Patterson AFB. The contractor may conduct field research at government facilities; and/or industrial settings; and/or conduct research at university settings;

and/or other contractor proposed alternative sites. “In-house” refers to the collaboration, teaming, and execution of research that may be done by the contractor at the government facility. The research proposed herein aligns to two core areas:

5) BAA Task Order 0001 Statement of Objectives paragraph 3.2.1.1 Research is amended as follows (changes in bold):

3.2.1.1 Research

The contractor may perform in-house research conducted at Wright-Patterson AFB in the Human Insight and Trust Laboratory, Human-Robot Teaming Laboratory, and government-affiliated laboratories associated with RHXS. The contractor may conduct field research at other government facilities, industrial settings, university settings, and/or other contractor proposed alternative sites. This research shall require the contractor to perform at various levels within the research domain, to include research assistants, and subject matter experts (SMEs). The contractor will collaborate with government researchers to create and execute experimental designs, IRB preparations, data collections, data analyses, and technical writing for RHXS research. The contractor may provide hardware or software creation or acquisition as the research deems necessary.

6) BAA Task Order 0001 Statement of Objectives paragraph 3.2.1.1.1 Human-Robot Teaming (2nd Paragraph) is amended as follows (changes in bold):

3.2.1.1.1 Human-Robot Teaming (2nd Paragraph)

The research in this technical area may utilize government in-house resources (e.g., laboratory spaces and equipment). The contractor may utilize industrial settings, university settings, and/or other contractor proposed alternative resources. The contractor will collaborate with government researchers. The contractor shall develop experimental designs, platforms to support experimental design, collect data, analyze data, and write research reports, conference papers, and journal articles.

7) BAA Task Order 0001 Statement of Objectives paragraph 3.2.1.1.2 Trust Calibration (2nd

Paragraph) is amended as follows (changes in bold):

3.2.1.1.2 Trust Calibration (2nd Paragraph)

The research in this technical area may utilize government in-house resources. The contractor may utilize industrial settings, university settings, and/or other contractor proposed alternative resources. The contractor will collaborate with government researchers. The contractor shall develop experimental designs, platforms to support experimental design, collect data, analyze data, and write research reports, conference papers, and journal articles.

Task Order 0002 SOO

8) BAA Task Order 0002 Statement of Objectives paragraph 2.0 Scope is amended as follows

(changes in bold):

2.0 Scope

The contractor will provide basic, applied, and advanced technology development research, development and demonstration for understanding the trust calibration process. The domains of interest include human-machine interaction with actual automated systems that have been fielded or are close to being fielded or automated systems that are under development for the Air Force. The present research may include both laboratory and field research. The contractor will collaborate and may perform in-house research conducted at Wright-Patterson AFB. The contractor may conduct field research at government facilities; and/or industrial settings; and/or conduct research at university settings; and/or other contractor proposed alternative sites. “In-house” refers to the collaboration, teaming, and execution of research that may be done by the contractor at the government facility. The research proposed herein aligns to two core areas:

9) BAA Task Order 0002 Statement of Objectives paragraph 3.2.1.1 Research is amended as follows (changes in bold):

3.2.1.1 Research

The contractor may perform in-house research conducted at Wright-Patterson AFB in the Human Insight and Trust Laboratory, Human-Robot Teaming Laboratory, and government-affiliated laboratories associated with RHXS. The contractor may conduct field research at other government facilities, industrial settings, university settings, and/or other contractor proposed alternative sites. This research shall require the contractor to perform at various levels within the research domain, to include research assistants, and subject matter experts (SMEs). The contractor will collaborate with government researchers to create and execute experimental designs, IRB preparations, data collections, data analyses, and technical writing for RHXS research. The contractor may provide hardware or software creation or acquisition as the research deems necessary.

Industry Day Questions and Answers – 26 August 2015:

1) Question: Task orders may allow for great variability of performance due to the potential of specific topics and baselines. How can we narrow our focus to better respond with a proposal?

Answer: We do not want to direct you on how to formulate your proposal as this would limit the creativity potential of your proposal submission.

2) Question: Are you looking more toward fundamental understanding of trust first or specific applications?

Answer: Our research is centered on human-machine trust, so a proposal that is focused on a specific application without looking at trust will not be very relevant to this BAA. If you have access to a specific application that has trust issues or trust-related problems that could be of interest. However we are very interested in the fundamentals of the trust process that can span very different applications.

3) Question: Given the primary emphasis on experimental research, what would you consider to be an acceptable level of software/hardware development?

Answer: To study trust and autonomy you need some form of technology interaction, an affordance for interaction. Therefore if software is readily available we do not need you to develop something new, however if you have something novel we are open to new software design.

4) Question: How interested are you in developing technological solutions that could facilitate future research opportunities in this area?

Answer: The Government seeks proposals that are responsive to the BAA and meet the evaluation criteria.

5) Question: Are there any other recommended research references that we should review?

Answer: Review of literature in the following areas would be useful; human-machine interaction, human factors, trust in automation, and trust models in management domain.

6) Question: Will AFRL consider delaying the due date for the proposals in response to the

BAA to allow industry to form teams and write their proposals? As it take significant time to get agreement is place.

Answer: See Amendment 02.

7) Question: Who are the current support contractors for RHXS?

Answer: The current contracts are under FA8650-09-D-6939 and FA8650-14-C-6585.

The awards have been synopsized in FedBizOps.

8) Question: How does this new effort relate to the current contracts?

Answer: The response would be dependent upon the work proposed by the offeror. The Government is seeking proposals responsive to the BAA, and offerors should rely on the BAA and not previous contracts.

9) Question: Is there an anticipated skill mix for the staff who will be on site? Are there specific positons needed?

Answer: There are numerous skills sets that would be appropriate but none have been specifically identified.

10) Question: Will the contractor have responsibility for supporting any of the labs we saw today?

Answer: You are not required to use or support any specific lab, and should propose what you feel is best. The BAA and Statement of Objectives (SOOs) have identified certain labs as being available for use. Your proposal must identify which, if any, of those labs that you would like to use. Use of Government labs is limited to research; no operations and maintenance support will be required.

11) Question: Will AFRL/RH lead the research, with the contractor supporting? Or will the contractor be expected to lead the research? Who will develop the research concepts and approaches, RH or the contractor or jointly? How will the awardee interact with the companies who developed the current test-bed in use if changes are needed?

Answer: The SOOs provided for the Basic, Task Order 0001 and Task Order 0002 encompass the Government’s Objectives (i.e., problem to be solved). It is up to the offeror to address those objectives in their SOW. The SOW shall detail the technical tasks proposed to be accomplished under the effort.

12) Question: Are there ongoing or future research/experiments slated for this BAA that require Yale, UD, MIT, Syracuse, interaction?

Answer: None of them are required for this BAA. These are our current collaborators and were provided to communicate our current research and they can be considered for your proposal.

13) Question: Is it possible to propose to just Task Order 1 or 2?

Answer: As stated in the BAA on page 13 of 24, paragraph IV (2)(b), Offerors must propose/submit against the Basic IDIQ and both Task Orders in order to be considered for an IDIQ award. It is anticipated that awards under this BAA will generally be Indefinite Delivery/Indefinite Quantity (IDIQ) contract(s) with Cost Plus Fixed Fee Completion type task orders. See page 2 of the BAA.

14) Question: What avenues are there for collaboration/funding outside this BAA?

Answer: There will be other BAAs in the future with RHXS. Our Division has an existing open BAA. Watch FedBizOps and Grants.gov for future opportunities.

15) Question: What type of funding will the BAA use? What percentage of RHXS’s funding is dedicated to internal versus external funding?

Answer: This BAA traverses 6.1-6.3 funding. It is anticipated that all of the funding will be Air Force dollars.

16) Question: How open are you to using labs and facilities in new industry locations?

Answer: We are open to new locations and ideas.

17) Question: Do you expect all contractor support to be on base at WPAFB?

Answer: No, an offeror may propose as much off-base support as they feel appropriate.

18) Question: How much off-base support will you allow, especially to get to SMEs that will not move to Dayton?

Answer: It is not a requirement for SMEs to move to Dayton. An offeror may propose as much off-base support as they feel appropriate.

19) Question: Are you willing to give a 30 day extension to give industry more time to create teams to respond?

Answer: See Amendment 02.

20) Question: Is the open dialogue to be formal/written only?

Answer: Per the BAA, Communications of a technical nature may be addressed to Dr.

Joseph Lyons via phone or via email. Communications of a contractual nature may be addressed to Mr. Devon Banks via phone or via email. See page 3 of the BAA. It is preferred that any questions be submitted via email; however, questions via phone will be accepted as well. Per AFRL BAA Guide for Industry, March 2015, Chapter 3, Section C, “If information concerning the objectives/goals/requirements of the BAA [are provided] to one proposer, this information must be provided to all proposers with the CO’s coordination. All questions (written or oral) deemed necessary to be provided to all proposers will be made publicly available to all potential proposers in a Questions and Answers document.

21) Question: What is a robot?

Answer: When we talk about human machine teaming, we are talking about something that can involve a physical entity or a cyber-physical entity. Also, automated technologies are relevant for this research as well. The key is that it must involve some level of interaction and the technology should offer affordances for human interaction with a technology that has some level of decision initiative.

22) Question: The BAA mentions a maintenance component, please explain?

Answer: That was mentioned as a possible area of interest if desired due to the expanding use of technology in the maintenance domain. This area was included to keep the call broad enough to include potential ideas in this area but it is not a domain we currently work in. Although we work in various areas of technology our focus is on science.

Questions received via Email:

1) Question: Sample Tasks 1 and 2 account for all of the anticipated funding in the first 3 fiscal years (FY15-17) of the contract; however, additional funding under the IDIQ is anticipated for FY18-20. Does the Government expect an Offeror’s cost volume to contain pricing for those years?

Answer: Offerors should propose in accordance with the specific funding profile that is listed for each Task Order. See page 11 of 24 of the BAA, paragraphs II(3) and II(4).

Offerors are only required to submit cost proposals for each Task Order and not the Basic IDIQ. See page 14 of 24 of the BAA, paragraphs IV(2)(b)(i), IV(2)(b)(ii), and IV(2)(b)(iii).

2) Question: The SOWs for the IDIQ and sample tasks are limited to 5 double-spaced pages while the SOOs are 5-6 pages single-spaced. The 5 page limit appears insufficient to adequately address all requirements from the SOO in the SOW. Will the Government please consider increasing the page count to 15 pages for each SOW?

Answer: Amendment 01 increased the page count for the SOW from 5 pages to 10 pages.

3) Question: During the tour, it was stated that Industry Labs could be used on TO-1 and TO-

2. However, the language in section 3.2.1.1 of each TO suggests that the TOs should be executed in the Human Insight and Trust Laboratory, Human-Robot Teaming Laboratory, and government-affiliated laboratories associated with RHXS. Are industry labs then precluded for execution of TO-1 and TO-2. If so, could additional information on the capabilities of these labs be provided? Available hardware and software are of particular interest.

Answer: No. The offeror may propose using labs other than those identified by the

Government. See corrections/amendment above.

4) Question: Can the proposal be hand-delivered, or must it be delivered by mail as stated on page 13, 2.a.iv?

Answer: Proposals may be hand-delivered. See corrections/amendment above.

5) Question: The BAA requests costing at the “task level” can you verify whether this means costing by Task Order e.g.:

TO 1 $XXX

TO 2 $XXX

or does it mean by the Technical Tasks listed in the SOO e.g.:

TO 1 $XXX

In-house research laboratories $XXX External laboratories $XXX Field research $XXX Demonstration $XXX

Management of Effort $XXX

TO 2 $XXX

In-house research laboratories $XXX External laboratories $XXX Field research $XXX Demonstration $XXX

Management of Effort $XXX

Answer: An offeror is only required to propose at the Task Order level in accordance with the funding profile on page 11 of 24 of the BAA.

6) Question: Per the BAA instructions, the BASIC SOW and Task Order SOWs are limited to five (5) pages each, double-spaced. However, the SOOs alone, when put into the format identified in the BAA, cover 7 pages each. If we keep the SOWs to five pages, key content from the SOOs may be omitted or condensed to such a degree that it loses the RHX’s intent. Would you consider expanding the SOW page limits to 10 pages?

Answer: Amendment 01 increased the page count for the SOW from 5 pages to 10 pages.

7) Question: I wanted to get clarification re anticipated awards structure for task orders. At last week's meeting, you said we could propose against all or some elements of the BAA. I took this to mean that a team could propose a single project against a specific TO - is that right?

Answer: As stated in the BAA on page 13 of 24, paragraph IV (2)(b), Offerors must propose/submit against the Basic IDIQ and both Task Orders in order to be considered for an IDIQ award. It is anticipated that awards under this BAA will generally be Indefinite Delivery/Indefinite Quantity (IDIQ) contract(s) with Cost Plus Fixed Fee Completion type task orders. See page 2 of the BAA.

8) Question: I recognize that there will be an IDIQ in place, but it was not clear to me if a contract could be awarded to a team that was not proposing to be the IDIQ vendor. For instance, if Company X wins the IDIQ contract and they have Companies 1, 2, and 3 on their team. Does that mean that only those companies will be awarded contracts in 2016?

Answer: With IDIQ contracts, they are awarded to one contractor, the “Prime.” It is the responsibility of that contractor to identify the best method and team to meet the requirement. Future task orders are not guaranteed, but if future research needs are identified, they could be awarded as additional task orders under the IDIQ.