Attachment 0002-JAIC Data BPA Performance Work Statement V4.docx

Joint Artificial Intelligence Center Data Readiness Services Blanket Purchase Agreement Performance Work Statement DRAFT as of 18 December 2020

Contents

1.0 Introduction

1.1 Background

The Department of Defense (DoD) Joint Artificial Intelligence Center (JAIC) was established in 2018 to be the focal point of the DoD Artificial Intelligence (AI) Strategy to accelerate scaling AI and its impact across the Department. Working closely with the Services, Combatant Commands, and other Components, JAIC identifies appropriate use cases for AI across DoD, rapidly pilots solutions, and scales impact across our enterprise.

The mission of the JAIC, as specified in the DoD AI Strategy, is “to accelerate the delivery of AI-enabled capabilities, scale the Department-wide impact of AI, and synchronize DoD AI activities to expand Joint Force advantages.”

1.2 Scope

The contractor shall accomplish all tasks needed to prepare or create data sets that are ready for use in Artificial Intelligence applications. Examples of such tasks include but are not exclusive to Extract Transform Load (ETL) and Data Engineering development, Database Design and Development, Data Science Development, Data Analysis, Project and Outreach Program Management.

Services Include:

· Project and Program Management

· Data Science

· Data Engineering

· Data Architecture

· Data Quality and Analysis

· Synthetic Data Generation and Data Anonymization

· Software Development, Modification and Configuration

· Enterprise Information Management and Governance

1.3 Type of Contract

This is a Blanket Purchase Agreement (BPA). Type of contract(s) allowed under this agreement are Firm Fixed Price (FFP), Time and Materials (T&M), and FFP – T&M hybrid. The type of contract(s) will be established by the Ordering Contracting Officer (OCO) at the individual call order level.

1.4 Authorized BPA Users

The BPA is open for ordering to the U.S. Government in support of Artificial Intelligence Data Readiness. This vehicle is established for decentralized ordering. The OCO is responsible for reviewing the Ordering Guide in its entirety and following all pertinent rules and regulations for awarding a BPA call order against this BPA.

1.5 Lengths of Agreements

The Data Readiness BPA will be active for five years of performance. Each BPA will be reviewed annually, and, if necessary, updated as needed. The Government will consider off boarding contractors during this time, if it is determined that it is in the best interest of the Government. As needed, the Government will also explore onboarding new contractors. During the final year, the BPA will be reviewed to determine if it will be renewed.

1.6 Non – Personal Services

The Government shall neither supervise Contractor employees nor control the methods by which the Contractor performs the required tasks. Under no circumstances shall the Government assign tasks to, or prepare work scheduled for, individual Contractor employees. It shall be the responsibility of the Contractor to manage its employees and to guard against any actions that are of the nature of personal services, or give the perception of personal services. If the Contractor believes that any actions constitutes, or are perceived to constitute personal services, it shall be the Contractor’s responsibility to notify the Contracting Officer (KO).

1.7 Thresholds

Individual call orders shall not exceed the dollar threshold listed in FAR 13.5(c).

2.0 Description of Services/Work Product Overview

2.1 Key Personnel

Key Personnel will be determined at the call order level, with the exception of the Program Manager.

Qualifications for the PM are listed below:

Program Manager (PM): The Contractor shall identify a PM by name who shall provide management, direction, administration, quality management, and leadership of the execution of the BPA. The PM shall act as the overall lead, manager, and administrator for the effort. The PM shall direct efforts of cross-competency teams including Contractors at multiple locations and shall serve as the primary interface and POC with Government program authorities and representatives on technical and project issues. The PM shall be responsible for regularly briefing leadership on program status and milestones. The PM shall oversee Contractor personnel project operations by developing procedures, planning and directing execution of the contractual, technical, multi-disciplinary engineering, programming, maintenance, and administrative support effort, and monitoring and reporting progress. The PM shall manage acquisition and employment of project resources and control financial and administrative aspects of the project.

It is required that the PM has the following qualifications:

· Must be an employee of the prime offeror or have an offer of employment from the prime offeror that the PM intends to accept in the event of an award being made to the offeror.

· Possess a minimum of ten years of Project Management experience managing complex projects in an IT engineering or big data environment.

Possess an active certification in one of the following:

· Project Management Institute (PMI) Project Management Professional (PMP), PMI Program Management Professional (PgMP) certification, or equivalent program or project management certification.

· Federal Acquisition Certification for Program and Project Managers (FAC P/PM) Level 3.

· Defense Acquisition Workforce Improvement Act (DAWIA) Level III Program Management certification.

· Possess a minimum of five years of experience managing projects utilizing non-traditional program management methodologies (e.g., agile) in a big data environment.

· Possess a minimum of ten years of experience in an IC, military, or other Government environment performing in a related subject area (e.g., project management, engineering, computer science) to that of the CO.

It is desired that the PM has the following qualifications:

· Possess a Master of Business and Management (MBM), a Master of Business Administration (MBA), or a master’s degree in a related technical discipline (e.g., Engineering, Computer Science, or Electronics).

2.2 Labor Categories

See Appendix A labeled, “JAIC Data BPA Labor Categories Descriptions” for more information.

3.0 General Information:

3.1 Security Requirements

The Contractor shall safeguard all Government property, documents and controlled forms provided for Contractor use and adhere to the Government property requirements contained in this contract. At the end of each work day, all Government facilities, equipment, and materials shall be secured by a Government POC. Contractors are not allowed to secure Government facilities, equipment, and materials.

The Contractor shall establish and implement methods of ensuring that no building access instruments issued by the Government are lost, misplaced or used by unauthorized persons. Access codes shall not be shared with any person(s) outside the organization. The Contractor shall control access to all Government provided lock combinations to preclude unauthorized entry. The Contractor is not authorized to record lock combinations without written approval by the Government COR. Records with written combinations to authorized secure storage containers, secure storage rooms, or certified vaults, shall be marked and safeguarded at the highest classification level as the classified material maintained inside the approved containers.

Call orders outside of the DoD will need to determine specific security requirements at the call order level.

3.2 Other Security Requirements

A DD Form 254, Contract Security Classification Specification, applies to this BPA. Call orders on this BPA can support up to Top Secret (SCI/SAP). Classification requirements will be established at the call order level. When responding to COPR, proposed personnel shall possess the minimum security clearance, or higher required, prior to proposal submission or being issued a Common Access Card (CAC). There is one exception that pertains to Contractor personnel with an interim security clearance. Fingerprints must be complete, favorable and on file for validation by 11th Wing Information Protection office (this is only applicable for contractors providing supporting in the NCR, all other locations shall follow local procedures). Contractor personnel must have a SECRET security clearance prior to performance on a Call order if they require a Pentagon-sponsored government e-mail. All Contractor personnel must maintain the level of security required for their contracted duties for the life of the contract.

The security requirements are in accordance with the attached DD Form 254. Any extracts or use of such data shall require the Contractor to apply derivative classifications and markings consistent with the source from which the extracts were made in accordance with DoDM 5200.1 Volumes 1-4, Information Security Program, a manual for safeguarding classified information; and Executive order 12356, Classified National Security Information. Pursuant to DoD 5220.22M; Chapter 6, Visits and Meetings; the contractor shall submit visit requests to applicable security managers (see call order CORs for security manager POCs). Contractor shall complete necessary documentation for sub-contractor(s) associated with each BPA holder.

The Contractor’s facility identified in Block 6 in the DD Form 254 shall be cleared as a Top Secret facility at the time of proposal. Contractor facility verification is through the Industrial Security Facilities Database (ISFD) using the company commercial and Government entity (CAGE) code submitted with its proposal. In the event that a company has more than one CAGE Code assigned, the CAGE Code submitted with its proposal shall be for the facility where the actual work that is performed in support of this BPA. “Personnel” clearance is different from “Facility” clearance.

3.3 Government Furnished Property

Government furnished items or services will be specified by the OCO in the individual call orders.

3.4 Service Contract Reporting (SCR)

The Office of the Secretary of Defense (OSD) mandates reporting of manpower data with regard to the performance of DoD services contracts consistent with the requirements of title 10, U.S.C., section 235 and 2330a, as amended. Contractors are to report manpower data relating to the performance of services contracts into SAM, consistent with existing Service Contract Reporting (SCR) requirements under the FAR Subpart 4.17 - Service Contracts Inventory (SCI). When reporting on call orders issued under an indefinite-delivery contract or agreement, reports are completed at the order level. No reports are collected at the contract or agreement level. Contractors are required to report data in SAM on an annual basis when awarded a DoD contract or call order valued in excess of $3 million in obligations. The Contractor shall completely fill in all the information in the format using the following web address: http://sam.gov. SCI reporters are required to report the following information: Total Amount Invoiced, Prime Contractor Hours Expended, and, if applicable, Tier 1 subcontractor information. The reporting period for Contractors in SAM is open from mid-October through mid-December for reporting against the prior government fiscal year. Contractors shall complete the SCR before the reporting period closes against the prior government fiscal year.

3.4.1 COR

No COR will be established at the base BPA level. Each call order will have a separately identified COR and designation letter that specifies their specific authorities and responsibilities. The COR will be identified by the designation letter issued by the KO. A copy of the letter will be sent to the Contractor and the Contractor will acknowledge each COR’s designation letter. Under no circumstances does a COR have the authority to authorize substantive changes to any terms and conditions of the resulting order with respect to cost/price, and delivery dates.

3.4.2 Quality Control

Contractors performing under this BPA shall ensure that they have an approved Quality Management System that is certified to standards determined at the call order level. The Contractor shall develop a Quality Control Plan (QCP) within 30 day of Notice To Proceed (NTP) of a call order under this BPA.

3.4.3 Quality Assurance [Surveillance Plan]

All call orders under this BPA shall include a Quality Assurance Surveillance Plan (QASP) outlining the Government’s surveillance methodology and assessment of the Contractor’s performance in terms of an acceptable level of quality.

3.4.4 Size Standards

Contractors must certify size standard for each call order. All size standards will be confirmed by the OCO at the call order level for eligibility to compete.

3.4.5 Subcontracting Plans

The Small Business Subcontracting Plan requirement is required for Contractors with a business size designation of “other than small” in accordance with FAR Clause 52.219-9, Small Business Subcontracting Plan, Alt II and DFARS clause 252.219-7003, Small, Disadvantaged, and Women-Owned Small Business Subcontracting Plans (Government Contracts). Unrestricted call orders shall include Small Business Subcontracting Plan requirements. The Small Business Subcontracting Plan shall be defined by the OCO at the call order level.

3.4.6 Small Business Participation

The Small Business Participation Plan shall be defined by the OCO at the call order level.

3.5 Recognized Holidays

Work at a Government site shall not take place on Federal holidays or weekends (but may require off-hour work due to network loading or other disruptions that could occur) unless direct by the OCO on individual call orders.

4.0 Tasks

This is not an all-encompassing list, as technology changes, additional tasks could be required.

4.1. Required General Services (4.1.1 - 4.1.3)

The Contractor shall provide General Services for all additional selected Task options below.

1.

2.

3.

4.

4.1.

4.1.1. Project Management and Documentation:

The contractor shall provide project management to an assortment of initiatives necessary to leverage enterprise data sets as strategic assets for the development of Data Science and Artificial Intelligence applications. The contractor shall provide project management and oversight of initiatives including, but not limited to, data acquisition, ingest, data governance and metadata management, integration and preparation of data, including labeling and quality assessment. The contractor shall also provide project management for efforts involving the generation of synthetic data, anonymized data, and development of data availability interfaces such as web services and data streams and will plan and coordinate hackathons and data science outreach events that leverage these technologies. The contractor shall provide regular updates including project plans, status updates, and progress towards major milestones.

The contractor shall provide technical documentation for data sets handled and processed. The contractor will author research style data quality evaluations and technical methodology papers to facilitate knowledge transfer.

4.1.2. Cloud Services Integration:

For DoD components, Contractors shall be prepared to transition services into the Joint Common Foundation (JCF): (1) Contractors shall provide artifacts, datasets, machine learning models, and source code into JCF repositories, as directed by the Government. (2) Contractors shall register source and delivered datasets within DoD and JCF data catalogues. For planning purposes, the JCF will have the following IOC capabilities with environments capable of Information Impact level 2 & 4: Cloud-based workspaces with Amazon and Azure platform services for data science, data management, machine learning, and software deployment; and artifact, data, and container repositories. Contractors will provide use-cases or requirement descriptions of the required tools and services for any data services to be provided.

If the Contractor is unable to transition services to the JCF, a DoD provisional cloud service, or a FedRAMP approved option, the Contractor must provide a plan to authorize the required tooling and infrastructure, if not provided for in the respective Government PWS.

In the future, the JCF will be deployed to the Joint Enterprise Defense Infrastructure (JEDI). The network design will remain the same and it is anticipated that the Contractor will be able to seamlessly transition its services to JEDI when it becomes operational. The JCF team will provide direction to the Contractor to build capabilities in line with the maturation of the JCF

4.1.3. Software Configuration

The Contractor shall configure and secure software as needed.

Optional Task Selections

4.2. Data Ingestion and Pipelines

4.2.

4.2.1. Data Ingestion Services

The contractor shall provide data ingestion services to include the establishment and development of new data pipelines utilizing technologies including but not exclusive to web services and streaming data services. These data transfer pipelines will be designed as interchangeable and reusable components that may facilitate data exchange for multiple Data Science and Artificial Intelligence projects. The contractor shall develop methods to secure the data transfer to the Impact Level (IL) of the data and provide transfer capabilities to move data between systems of different Impact Levels. These capabilities will include fine grained user access controls for data at rest and secure encrypted technologies for data in transfer.

4.2.2. Streaming Transformation

The constructed pipelines should have capabilities for streaming data transformation including but not exclusive to joining, merging, filtering, and performing Change Data Capture techniques on ingested data sets.

4.2.3. Data Storage

The contractor shall design very large capacity, scalable data storage systems including secure user and role based access controls.

4.2.4. Metrics and Documentation

Key data ingest deliverables shall be provided by the contractor in relation to the size of data in both storage space as well as number of data entities, and the timeliness of data refresh. Data lineage and provenance shall be recorded at all steps of processing.

4.3. Enterprise Information Management

4.3.1. Cataloging Data

The contractor shall catalog metadata for all data ingested in the readiness effort. Complete collections of metadata and metrics for data assets including table data and elements such as columns will be gathered. The contractor shall also provide administrative support for the collection of data element definitions and entity relationships.

4.3.2. Making Data Visible

Metadata and metrics collected shall be formatted to facilitate searching, display in, and portability between Enterprise Information Systems. These collections shall be organized in searchable interfaces such that users may be able to easily locate organizational information and its metadata. The contractor shall also provide visibility into cross domain data relationships through integrated entity relationship models that show relationships between data generated or stored in disparate systems.

4.3.3. Metadata Integration

The contractor shall ensure that the collection, formatting and storage of metadata and metrics meets all existing format standards for the awarding Government Organization and integrates with existing metadata catalog tools.

4.4. Data Integration

The contractor shall develop an enterprise data integration model that defines the data flow requirements for confidential and restricted information. The model must address all potential data flows internal and external to the enterprise. The contractor will develop or provide technologies which allow for the integration of various data sources into common visualization and data science platforms and libraries.

The contractor shall also provide integration techniques to improve the quality and usefulness of data collections. Techniques such as Master Data Management and Data Enrichment shall be utilized to enhance data and assist increase its predictive power in Data Science and AI applications.

4.5. Data Architecture and Preparation

The contractor shall provide a scalable technical architecture necessary to enable the application of prepared data sets by Data Science and Artificial AI workflows. This architecture shall not result in reduced performance of data source systems because of increased data consumption workloads. Additionally, the architecture will be documented in entity relation diagrams and supporting documentation to maximize usefulness of processed data sets.

The contractor shall perform all Data Preparation tasks necessary to transform data from its raw form into a format ready for consumption by Data Science and AI workflows. These tasks include, but are not limited to, data export standards and Extract-Transform-Load activities to maximize data usage for AI and analytical usage.

The contractor shall act as an integrator of technologies and solutions. They shall provide technological solutions that enhance and complement existing products currently in use, as well as future technologies provided by outside vendors.

4.6. Data Quality

The contractor shall provide technologies for assessing the quality of data.metrics gathered through this assessment will be displayed in reporting interfaces and data readiness dashboards. Data Quality may be assessed through processes including but not exclusive to, Outlier Analysis, Sparsity Analysis, and additional statistical methodologies.

4.7. Data Security

The contractor shall provide technologies for assessing the security of the data with respect to its impact on the security of the AI system. This will include methods to detect and remove data designed to interfere with the training and/or operation of an AI model or application, as well as securing the data throughout all points of the data lifecycle.

4.8. Automated Data Collection

The contractor shall work to improve data quality by implementing applications and pipelines for the automated capture and collection of data including technologies such as sensors, IOT, or other intelligent data capture devices.

4.9. Data Labeling

The contractor shall provide methods and technologies capable of providing proper labels to datasets that require labeling. The contractor will provide methods and technologies that are capable of evaluating and reporting on the accuracy and confidence of automated labeling efforts. Finally, the contractor will provided methods and technologies of evaluating the percentage of missing labels on any previously labeled data.

The contractor shall support labeling efforts by developing applications in support of this effort, including object detection applications and pipelines.

The contractor will seek out and deploy technologies which assist in or reduce the labeling effort. Technologies such as weak supervision automated or assisted labeling and data reduction algorithms should be utilized to their maximum potential to build highly efficient reusable labeling pipelines.

4.10. Synthetic Data

The contractor will provide technologies for generating synthetic data within user-defined statistical constraints. The contractor will develop methodologies for validating statistical similarity and lack of any secure or confidential original data. Generated synthetic data will maintain originating entity relationships and statistical or distribution lineage to its source such that developed and trained algorithms may be used with interchangeability between data sets.

4.11. Data Anonymization

The contractor will provide technologies for anonymizing data within user-defined statistical constraints. The contractor will provide methodologies for validating the degree to which the data has been anonymized.

4.12. Data Wrangling

The contractor shall perform data wrangling including all activities necessary to profile, connect or join, filter, and transform data into Machine Learning and AI Feature Sets. The contractor will write custom data wrangling code where applicable and will also leverage existing data processing tools and pipelines in this effort.

4.13. Data Availability

The contractor will provide technologies and resources which allow for data and storage redundancy while providing data security as indicated by the appropriate impact level. The contractor shall make data available through the development of multiple interface forms including but not exclusive to web services, sFTP transfers, cloud based data buckets, and database ODBC/JDBC connections. These interfaces will be designed to encourage high levels of data sharing and collaboration while maximizing the security of sensitive data sources.

4.14. Feature Engineering

The contractor shall perform development of Data Science and AI features by transforming data sets using programming languages, libraries, and ETL tools. The contractor shall join and integrate data sources as needed and use techniques to provide data structures that meet the input needs of Data Science and AI algorithms. The contractor shall seek to increase the power of such algorithms by using combinations of data elements to create calculated features.

4.15. NLP and Text Processing

The contractor shall perform processing of structured and unstructured text data to create feature sets for Machine Learning and AI applications. The contractor shall perform text reduction, extraction, tokenization, and mining efforts to prepare data for AI pipelines. The contractor shall also build applications to tag and label text data.

4.16. Transfer Learning and Algorithm Management

The contractor shall manage and maintain an organized library of trained algorithms to facilitate reuse and rapid development of artificial Intelligence solutions. The contractor shall include storage structures, maintenance, and documentation including metadata and statistics.

4.17. Standardized Data Science and AI Pipelines

The contractor shall leverage existing infrastructure to develop and customize standardized Data Science and AI tools and pipelines which can be leveraged for multiple projects. Pipelines shall be deployed with flexible tools allowing for future scale-ability and growth. The tools should be available as micro-services in a containerized environment and should be designed to be inter-operable with new and existing technologies.

4.18. Data Science and AI Outreach and Innovative Development Mechanisms The contractor shall leverage the processed and generated data sets for outreach projects and innovative AI application development mechanisms, such as competitions that leverage development inputs from academic and industry communities. A competitive development model shall be designed that allows for outside entities and individuals to engage in hackathons and competitions using synthetic or unclassified openly available data in a manner that encourages innovative solutions. The contractor will plan and execute these events.

5.0 Post Award

5.1 Conferences

As needed, the Government will hold post-award conferences. The Contractor shall attend any post award conference convened by the contracting activity or contract administration office in accordance with Federal Acquisition Regulation Subpart 42.5. The contracting officer, PM, Contracting Officer's Representative (COR), and other Government personnel, as appropriate, may meet periodically with the Contractor to review the Contractor's performance. At these meetings, the contracting officer will apprise the Contractor of how the government views the Contractor's performance and the Contractor will apprise the Government of problems, if any, being experienced. Appropriate action shall be taken to resolve outstanding issues. These meetings shall be at no additional cost to the Government.

5.2 Transition In/Transition Out

Transition in and out for contractors on this BPA will be specified for each call order.

5.2.1 Transition-Out Plan. The Contractor shall provide a Transition-Out Plan NLT than 30 calendar days prior to expiration of the contract. The plan shall facilitate the accomplishment of a seamless transition from the incumbent to an incoming Contractor (if applicable). In addition, the Contractor will continue to accomplish all tasks as outlined in the contract during this period. The current Contractor shall identify how it will coordinate with the incoming Contractor and Government personnel to transfer knowledge regarding the following:

• Project management processes

• Points of contact

• Location of technical and project management documentation

• Status of ongoing technical initiatives

• Transition of key personnel

• Schedules and milestones

• Actions required of the Government

• Coordination of IT related programs, issues

5.2.2 Communication. The Contractor shall also establish and maintain effective communication with the Government personnel for the period of the transition via weekly status meetings per the COR’s direction.

6.0 Definitions/Acronyms

7.0 Attachments

Appendix A. JAIC Data BPA Labor Categories Descriptions