RFI - 832016142.docx

REQUEST FOR INFORMATION

The Joint Artificial Intelligence Center (JAIC) / Defense Information Systems Agency (DISA)/Defense Information Technology Contracting Organization (DITCO) is seeking information from industry to assist with the development and planning of a potential new requirement.

THIS IS A REQUEST FOR INFORMATION (RFI) NOTICE ONLY. THIS IS NOT A REQUEST FOR PROPOSALS (RFP). NO SOLICITATION IS AVAILABLE AT THIS TIME.

1. Overview/Purpose/ Description of Procurement

JAIC, under the Department of Defense (DoD) Chief Information Officer (CIO), was established to accelerate the adoption of Artificial Intelligence (AI) across the DoD. The Services have a growth need for AI technologies to augment warfighters conducting repetitious and/or dangerous tasks. AI has the potential to provide timely and critical support to the warfighter, increasing operational tempo, thereby providing an advantage in the battlespace. The JAIC has established several key Mission Initiatives (MI) (see Figure 1), where AI can enhance operations across the Services.

Figure 1. The JAIC Mission Initiatives To facilitate the adoption of AI technologies, the JAIC is creating a Joint Common Foundation (JCF). The JCF, shown in Figure 2, is a central location for Components cross the DoD to discover, develop, and test solutions for their AI problems and serves as a host to the AI technologies developed by the Mission Initiatives.

Figure 2. The JAIC Joint Common Foundation To ensure that safe and effective products are produced by each of the Mission Initiatives, JAIC has a Test and Evaluation (T&E) office dedicated to providing testing services to be hosted on JCF. The JAIC T&E office is responsible for evaluating all JAIC products and providing independent analysis of AI performance. The JAIC T&E office conducts algorithm testing, system testing, and operational testing.

In addition, the JAIC is the AI Center of Excellence for the DoD. The JAIC T&E team is responsible for developing DoD wide AI T&E processes, tools, and standards. To accomplish this, the team will develop and host AI test harnesses on the JCF for use by the wider DoD community. In addition, the JAIC T&E team is required to be a centralized point of contact for AI T&E subject matter expertise.

2. Scope of Effort and Technical Characteristics

The JAIC T&E office is requesting information on the development of T&E software for AI, creation of AI T&E processes and best practices, and the ability of vendors to provide AI test and evaluation services. The JAIC T&E team has needs in three of the following areas:

Test Technology and Tools

The JAIC T&E office seeks to acquire test technology and tools for a variety of AI applications. The anticipated AI T&E focus areas may include development of T&E capabilities to support the DoD’s full spectrum of emerging AI technologies to include Machine Learning (ML), Deep Learning (DL), and Neural Networks / Neural Nets (NN) with focus on the following:

1. Conversational interface applications using voice to text.

2. Speech enabled products and services for DoD applications and systems.

3. Image analysis, testing deep learning based visual search and image classifier.

4. Natural Language Processing (NLP) enabled products and services.

5. Humans augmented by machines, to include human-machine interfaces and improved methods to measure warfighter cognitive and physical workloads, to include augmented reality (AR) and virtual reality (VR) test services.

6. Autonomous systems.

All proposed solutions must align with JCF and Mission Initiatives infrastructure needs, i.e., cloud-based test tools that are containerized.

Test and Evaluation Services

The JAIC T&E team is seeking the services of organizations in government, academia, and industry to assist in the test and evaluation of technologies provided by Mission Initiative or other DoD performers. Support is needed in the following five areas:

1. Data Set Development/Curation: Each Mission Initiative or DoD AI project has unique data that must be acquired and curated such that it can be used by modern AI software systems. Efforts include, but are not limited to, data quality assessment, image annotation, and data coverage analysis.

2. Test Harness Development: Creation of automated, containerized, cloud-enables test harnesses that facilitates automatic ingestion of data into a containerized AI model and aggregates the results.

3. Model Output Analysis: Analysis expertise/software is needed to assess the quality of results from the AI models developed by the Mission Initiatives. This effort involves both development of containerized, cloud-based, analysis software and creation of context specific and operationally relevant performance metrics.

4. Test Planning, Documentation, and Reporting: Testing expertise to create the required documentation for an AI algorithm undergoing the testing lifecycle (Algorithm test, System test, and Operational test). This effort includes, but is not limited to, creation of Test and Evaluation Master Plans (TEMPS), Test Plans, creation of Test Report documenting findings, and management of a test schedule.

5. Testing Services: This effort involves the services and expertise to conduct test and evaluation of AI products. This includes, but is not limited to, performing testing on AI algorithms and/or AI enabled systems across all T&E areas (Algorithm test, System test, Developmental test, and Operational test).

All proposed solutions must align with JCF and NMI infrastructure needs, i.e., cloud-based test tools that are containerized.

Other Technologies

This category allows potential performers to identify other technology not covered within the scope of the topic areas above that may be beneficial to AI T&E efforts at the JAIC. Potential performers must explicitly indicate the benefit of the proposed technology.

3. Requested Information

Interested parties may identify their interest, recommendations, and capabilities by responding to this RFI and submitting a capability statement that includes the following:

1. A discussion of your organizations familiarity/experience, to include a vendor provided attachment titled “Test Technology and Tools”, with a supporting narrative.

2. A submission of recommended solutions for applying innovative development approaches to the “Test and Evaluation Services.”

3. A list of any omissions should be attached, along with an explanation of why the proposed additional considerations should be added (i.e. “What did we forget, and why is it important?”).

RESPONSE GUIDELINES:

Interested parties are requested to respond to this RFI with a white paper. Submissions cannot exceed 10 pages, single-spaced, 12-point type with at least one-inch margins on 8 1/2” X 11” page size. The response should not exceed a 5 MB e-mail limit for all items associated with the RFI response. Responses must specifically describe the contractor’s capability to meet the requirements outlined in this RFI. Oral communications are not permissible. Beta.sam will be the sole repository for all information related to this RFI.

Companies who wish to respond to this RFI should send responses via email no later than 10:00 AM CST on 12 May 2020 to tyler.j.kehrer.civ@mail.mil.

INDUSTRY DISCUSSIONS:

DISA representatives may choose to meet with potential offerors and hold one-on-one discussions. Such discussions would only be intended to obtain further clarification of potential capability to meet the requirements, including any development and certification risks.

QUESTIONS:

Questions regarding this announcement shall be submitted in writing by e-mail to Mr. Tyler Kehrer at tyler.j.kehrer.civ@mail.mil. Verbal questions will NOT be accepted. Answers to questions will be posted to beta.sam. The Government does not guarantee that questions received after 10:00 AM CST on 21 April 2020 will be answered. The Government will not reimburse companies for any costs associated with the submissions of their responses.

DISCLAIMER:

This RFI is not a RFP and is not to be construed as a commitment by the Government to issue a solicitation or ultimately award a contract. Responses will not be considered as proposals nor will any award be made as a result of this synopsis.

All information contained in the RFI is preliminary as well as subject to modification and is in no way binding on the Government. FAR clause 52.215-3, “Request for Information or Solicitation for Planning Purposes”, is incorporated by reference in this RFI. The Government does not intend to pay for information received in response to this RFI. Responders to this invitation are solely responsible for all expenses associated with responding to this RFI. This RFI will be the basis for collecting information on capabilities available. This RFI is issued solely for information and planning purposes. Proprietary information and trade secrets, if any, must be clearly marked on all materials. All information received in this RFI that is marked “Proprietary” will be handled accordingly. Please be advised that all submissions become Government property and will not be returned nor will receipt be confirmed. In accordance with FAR 15.201(e), responses to this RFI are not offers and cannot be accepted by the Government to form a binding contract.

image1.png image2.png