A23 - HR001125R0003 - Attachment S03 - Sample Task Order 1 - 2025-03-21.pdf

Defense Advanced Research Projects Agency

(DARPA)

Information Technology Directorate (ITD)

ManAged eNterprise Network sUpport serviceS (MANNUS)

Indefinite Delivery/Indefinite Quantity (IDIQ)

Solicitation HR001125R0003

Attachment S03 - Sample Task Order 1

Statement of Objectives (SOO)

Dated: 21 March 2025

1.0 GENERAL

1.1 INTRODUCTION

The Defense Advanced Research Projects Agency (DARPA or Agency) is the research and development agency for the U.S. Department of Defense (DoD). The DARPA mission is to maintain U.S. technological superiority over potential adversaries by identifying, developing, and supporting breakthrough technologies of interest to the military. DARPA is located at 675 North Randolph Street, Arlington, Virginia.

The key to DARPA’s success has been, and continues to be, its ability to attract the brightest minds from Government, industry, and academia and provide them an environment that facilitates their ability to turn ideas into practical, leading-edge technology. DARPA requires Information Technology (IT) services that facilitate this creativity by providing world-class Government service, state-of-the-art tools and services, and rapid, flexible and innovative response to mission essential and evolving Government requirements.

1.2 TASK ORDER BACKGROUND AND SCOPE

The rapid advancement of artificial intelligence (AI) and machine learning (ML) has led to the development of large language models (LLMs) that can generate human-like text. However, these models often suffer from hallucinations, which are instances where the model generates text that is not based on actual facts or evidence. This can lead to the spread of misinformation and undermine the credibility of AI-generated content.

To address this challenge, the Agency is seeking to develop a website that can host a competition to create an AI model that can expedite the ingestion of large language models while eliminating hallucinations through fact-checking and referencing. The goal of this competition is to encourage the development of innovative solutions that can improve the accuracy and reliability of AI-generated content.

The objective of this task order is to design, develop, test, and deploy a fully functional competition website that meets the requirements outlined in the Website Requirements Document provided below.

The website shall provide a user-friendly interface for users to register, submit entries, and view competition information, as well as a secure and reliable system for managing submissions, judging, and displaying results.

1.3 TASK ORDER OBJECTIVES

• Development of a user registration and authentication system for the tracking of participants and verification that teams are only allowed to submit one entry.

• Implementation of administration features for managing user accounts, team registrations, and submissions.

• Development of a website that can host the competition, including user registration, dataset access, and results viewing.

• Implementation of a secure and reliable system for uploading and storing submissions, as well as judging and evaluating entries.

• Development of a leaderboard that can rank participants based on their performance, including metrics such as accuracy, precision, and recall.

• Development of a fact-checking and referencing system that can verify the accuracy of AI-generated content, including the use of external datasets and knowledge graphs.

• Conducting unit testing, integration testing, and user acceptance testing (UAT).

• Deployment of the website to a production environment.

• Configuration of monitoring and logging mechanisms.

• Development of a maintenance and monitoring plan to track and limit participant expenses within the hosted cloud environment.

1.4 TASK ORDER ASSUMPTIONS

• The competition will be open to a wide range of participants, including academia, industry, and individual researchers.

• The competition will be hosted on a website that will provide a platform for participants to submit their models, access datasets, and view results.

• The website will be provisioned and hosted from DARPA’s publicly available Cloud Service Provider (CSP) in Amazon Web Services (AWS).

• The competition will involve a series of challenges and evaluations, including a leaderboard that will rank participants based on their performance.

• The dataset used for the competition will be a large, publicly available dataset that is representative of the types of text that the AI model will be expected to generate.

3.0 KEY PERSONNEL

Offerors shall list any proposed key personnel position(s) and the requirements of the proposed key personnel position(s), for their proposed performance work statement (PWS). Offerors must indicate if they do not intend to propose key personnel positions.

4.0 TRAVEL

Offerors shall include all travel requirements needed to perform the scope of work in a separate travel section of the proposed PWS.

If travel is proposed, Offerors shall include the following language in the travel section of the proposed PWS: “All travel is to be submitted to the COR no less than three (3) business days prior to the requested travel dates. Travel requests shall include all costs within GSA prescribed per diem rates for the location of travel and will require submission of final expenses not later than the month following completion of travel or the second month following completion of travel if travel concluded within three (3) business days of month’s end.”

5.0 PLACE AND PERIOD OF PERFORMANCE

Work performed under this effort shall be performed at the contractor facilities that meet enterprise security requirements as required in Defense Federal Acquisition Regulation Supplement Clause 252.204-7020, NIST SP [National Institute of Standards and Technology Special Publication] 800-171, DoD Assessment Requirements.

Offerors shall assume a period of performance not to exceed 24 weeks from the date of task order award to complete the proposed PWS.

6.0 GOVERNMENT FURNISHED INFORMATION/EQUIPMENT

Offerors shall assume the following Government Furnished Equipment (GFE) will be provided as required to support the proposed PWS: high performance laptops, authentication tokens for enterprise access, facilities badging as required, and mobile phones with access to DARPA enterprise email and approved applications software as requested/required.

Government Furnished Information (GFI) provided under this effort will include all pertinent data for the challenge site creation such as dates of challenge, constraints for participants, and format of submissions, as they are identified. Offerors’ proposed PWS shall include any additional GFI which will be required to perform their proposed PWS.

7.0 DELIVERABLES

The offeror shall identify all deliverables as part of their proposed solution in the deliverables section/table of the proposed PWS in addition to the mandatory deliverables listed in the table below. All deliverables in the proposed PWS must have a corresponding task and acceptable quality measurement. The offeror shall include the following language in the deliverables section of the proposed PWS: “The Contractor shall electronically post all deliverables for online and/or secure WEB access by designated DARPA personnel. Deliverables shall be in Government-approved format. The Contractor shall recommend the format of each deliverable to the task order COR for approval prior to submission.”

Deliverable Title Recipients Frequency Location

1 Competition Website

Contracting Officer (KO)/Contracting Officer’s Representative

(COR)/

Government Program Manager (GPM)

Once As determined by the COR and GPM.

2 Project schedule and timeline KO/COR/GPM Once

Vault and Confluence, or similar location, as decided by the Government.

Deliverable Title Recipients Frequency Location

3 Communication Plan KO/COR/GPM

Once and updated as needed.

Vault and Confluence, or similar location, as decided by the Government.

Feasibility Study and Risk Assessment Report

KO/COR/GPM

Once and updated as needed.

Vault and Confluence, or similar location, as decided by the Government.

Wireframe and User Interface (UI) Design Document

KO/COR/GPM Once Vault and Confluence, or similar location, as decided by the Government.

6 Functional Prototype KO/COR/GPM Once As determined by the

COR and GPM.

7 Test Plan and Test Cases KO/COR/GPM

Once and updated as needed.

Vault and Confluence, or similar location, as decided by the Government.

8 Maintenance Plan KO/COR/GPM Once Vault and Confluence, or similar location, as decided by the Government.

Access keys and admin passwords for developed site(s)

GPM Once As determined by the

GPM.

End of Task Order Transition Plan

KO/COR/GPM Once Vault and Confluence, or similar location, as decided by the Government.

11 Non-Disclosure Agreement KO/COR

Once for each contractor personnel and updated as needed.

Vault and Confluence, or similar location, as decided by the Government.

8.0 QUALITY MANAGEMENT

8.1 QUALITY ASSURANCE PLAN

Offerors shall include a quality assurance plan in their proposed PWS. Offerors’ proposed quality assurance plan shall align to the tasks and deliverables contained in their proposed PWS. Offerors shall include, at a minimum, timeliness of the project and user satisfaction as quality assurance measurements.

Website Requirements Document

Key Performance Indicators (KPIs):

• Number of registered participants (assume approximately 200 – 500 registrations)

• Number of model submissions (assume approximately 100 submissions)

• Quality of model submissions (as measured by evaluation criteria): 80% of submissions meet or exceed the minimum evaluation criteria

• User engagement and satisfaction (as measured by feedback and survey responses following successful participant submissions): 90% of users report being satisfied with the challenge website and submission process

Requirements:

• User registration and authentication

• Model submission and upload functionality

• Model evaluation and scoring

• Leaderboard display

• Results display

• Forum or discussion board for participants to ask questions and share information

• Administrative interface for challenge administrators to manage the competition

Non-Functional Requirements:

• Security: The website must be secure and compliant with DoD security standards

• Scalability: The website must be able to handle a large number of users and model submissions

• Performance: The website must be able to evaluate and score models in a timely manner

• Usability: The website must be easy to use and navigate for participants and administrators

Assumptions and Dependencies:

• The website will be built using a modern web framework and programming language

• The website will use a relational database management system to store user and model data

• The website will use a secure protocol for data transmission and storage