DRAFT Pluto PWS dated 7 March 2024.docx

E-4-1523

AIR FORCE RESEARCH LABORATORY

ROME RESEARCH SITE

ROME, NEW YORK

PERFORMANCE WORK STATEMENT (PWS)

FOR

PLUTO

E-4-1523

7 MARCH 2024

(Contract Number FA8750-24-C-XXXX)

TABLE OF CONTENTS

1.0 BACKGROUND & OBJECTIVES

1.1 The AFRL/RIEB mission is to deliver superior technological advantage to the warfighter through agile Information Technology (IT) innovation, research, development, and transition, and sustain advanced IT solutions to provide timely and secure intelligence information access, analysis, and dissemination for National Defense and IC decision makers at all echelons.

1.2 The objectives of this Technical Order (TO) are to:

a. Provide AFRL/RIEB and its stakeholders with the ability to rapidly move from discovery to full operations, with limited operator/use evaluations, for IT Systems Architecture and Development, Testing, Integration, transition, and sustainment services.

b. Reduce the time-to-field of critical ISR and Command and Control (C2) mission systems through development of containerized Artificial Intelligence (AI) and Machine Learning (ML) capabilities supporting U.S. Code (U.S.C.) Title 10-Armed Forces and U.S.C. Title 50-War and National Defense including PED missions.

c. Capitalize on Science and Technology (S&T) advancements through rapid development and integration of completed basic, applied, and advanced research supporting information dominance when engaged with peer and near-peer adversaries.

d. Support increased demand from AFRL/RIEB mission partners for data science that postures resilient cybersecurity and zero trust architecture, ML, and rapid implementation of analytics deployment within the modern cloud Infrastructure as a Service (IaaS) infrastructures and associated Software/Platform as a Service (SaaS/PaaS) environments based on evolving mission areas and threats.

e. Support the AFRL/RI Strategic Plan for FY 2024-2028, which is aligned with the 2022 National Defense Strategy (NDS):

· To protect and defend both the homeland and the values at the heart of the American way of life, the NDS defines the following Defense priorities:

· I) Defending the homeland, paced to the growing multi-domain threat posed by China

· II) Deterring strategic attacks against the United States, Allies, and partners

· III) Deterring aggression, while being prepared to prevail in conflict when necessary, prioritizing the PRC challenge in the Indo-Pacific, then the Russia challenge in Europe

· IV) Building a resilient Joint Force and defense ecosystem

f. Support the strategic lines of effort of the 2023 National Defense Science and Technology Strategy in order to reduce the acute threat to our national and technological interests by increasing demand for cutting-edge S&T.

· Focus on the Joint Mission

· II) Create and Field Capabilities at Speed and Scale

· III) Ensure the Foundations for Research and Development

g. Support the Secretary of the Air Force (SecAF) Operational Imperatives, which relate to the urgent need to modernize and improve our operational posture in response to challenges imposed by our adversaries, including our most pacing challenge: China.

· Space Order of Battle

· Operationally Focused ABMS

· Moving Target Engagement

· Tactical Air Dominance

· Resilient Basing

· Global Strike

· Readiness to Deploy and Fight

h. Support the priorities and initiatives of the Office of the Chief Information Officer (CIO) within the DAF, (SAF/CN):

· Objective 1 | Accelerate Cloud Adoption

· Objective 2 | Future of Cybersecurity

· Objective 3 | Workforce Development

· Objective 4 | IT Portfolio Management

· Objective 5 | Excellence in Core IT & Mission-Enabling Services

· Objective 6 | Data & AI

i. Support the following priorities of the DIA CIO Strategy 2023-2027:

· IT Workforce Experience

· JWICS Modernization

· DoDIIS Modernization

· FVEY/2-Party Integrees (2PI) on JWICS

· Multi-Cloud/DevSecOps Governance

1.3 The AFRL/RIEB IDHS portfolio is currently undergoing enhancements and modifications to maintain and improve the readiness of the DoDIIS and IC. As such, support associated with these efforts will include migration of applications from dedicated hardware to cloud environments.

1.4 AFRL/RIEB researches, develops, transitions, and sustains advanced IT solutions to support timely and secure intelligence information access, analysis, and dissemination for National Defense and IC decision makers at all echelons. The AFRL/RIEB currently supports numerous agencies and MAJCOMs within DoD and numerous members of the IC, including Navy, AF Life Cycle Management Center (AFLCMC), ACC, AFSPC, United States (U.S.) Battlefield Information Collection and Exploitation System (BICES), and National Security Agency (NSA); AF IC is one of the most frequent and largest partners. Others are USSF, USAF enterprise, Secretary of the Air Force Administrative Assistant, Information Management (SAF/AAI), Rapid Capabilities and C2 AO for OVL, SAF/CDM Fast Track, HAF/A1, and USAFE.

1.5 AFRL/RIEB is licensed by the National Cross Domain Services Management Office (NCDSMO) to conduct cybersecurity assessments on Cross Domain Systems seeking approval for operation in SABI environments. Several prominent missions are Joint Strike Fighter, USBICES, DIA, and Radiant Mercury. AFRL/RIEB is tasked with development, enhancement testing, deployment, and sustainment of cross-domain transfer and access solutions for DoD AF and IC.

2.0 DESCRIPTION OF SERVICES: The contractor shall perform technical support for IT services to facilitate ISR A&F missions as follows:

2.1 The purpose of this TO is to provide AFRL/RIEB and its clients with specialized Information Technology (IT) Engineering Services (ES) and analytical support within the ISR domain for the AFRL/RIEB and its mission partners. Services will be provided at multiple Government installations throughout the Contiguous United States (CONUS) and field support at CONUS and Outside CONUS (OCONUS) DoD locations. The services include:

a. Intelligence IT Systems Analysis.

b. Requirements Analysis.

c. Solutions Analysis.

d. Operational and Technical Assessment.

e. Operational Architecture Analysis.

f. Systems Engineering.

g. Configuration and Database Management.

h. Life cycle engineering of systems through certification.

2.2 The scope of this TO includes all aspects of IT systems engineering including, configuration, database management, life cycle engineering of systems through certification and fielding, requirements and solution analysis, and engineering operational and technical assessment. Specifically, emphasis is on integration of requirements across the enterprise including incorporating emerging sensors into the Intelligence Community (IC) supporting global Processing, Exploitation, and Dissemination (PED) architecture, or managing integration between IC members and DoD Command and Control (C2) mission partners, and Intelligence producers.

2.3 In order to implement the NDS, AFRL, and its stakeholders, AFRL/RIEB will be tasked with providing cybersecurity support to the Joint Forces. These objectives, priorities, and operational imperatives demand that AFRL will need an increased ability to rapidly move from discovery to operations through abbreviated limited operator/user evaluations and then directly into full operations transition. Further, in response to the NDS, AFRL/RIEB is tasked with establishing an AFRL-wide repository to maintain and improve the readiness of the DoDIIS and IC. As such, support associated with these efforts will include migration of applications from dedicated on premises hardware to cloud environments.

2.4 The Air Force Research Laboratory (AFRL) provides programmatic and technical support for research and development, design, prototyping, demonstration, modeling, evaluation, test, migration, installation, and transition for Air Force (AF) Intelligence, Surveillance, and Reconnaissance (ISR) intelligence systems and programs in support of numerous Major Commands (MAJCOMs), Intelligence Community (IC), and AFRL/Information Handling Branch (RIEB) programs and initiatives. AFRL/RIEB provides the following services to its clients:

a. Information Assurance and Assessment (IA&A): As part of this capability, The Air Force Research Laboratory (AFRL) provides fact-based risk assessments of AF Enterprise, AF Space, SAF/AAI, HAF/A1 and IC systems to the United States Space Force (USSF) Headquarters (HQ) Space Operations Command (SpOC) S3/6CS and United States Air Force (USAF) Enterprise. AFRL/RIEB assists programs through the Assessment and Authorization (A&A) process, which includes the transition from the Department of Defense (DoD) Information Assurance (IA) Certification and Accreditation (C&A) Process (DIACAP) to Risk Management Framework (RMF).

b. Test and Evaluation (T&E): As part of this capability, AFRL/RIEB provides cybersecurity assessments of Cross Domain Systems (CDS) for various DoD PMOs.

c. Technical Assessment (TA): As part of this capability, AFRL/RIEB provides Cybersecurity Engineering, IA T&E services, and A&A support to DoD Programs. As an IA-focused extension of programs, AFRL/RIEB performs Command Cybersecurity Readiness Inspection (CCRI) preparation services. Additionally, AFRL/RIEB reviews, prepares and submits RMF packages and software assessments to obtain Interconnection Security Agreements (ISA), Interim Authorities to Test (IATT), Certificate to Field (CtF), Authority to Operate (ATO), and Security Impact Assessments (SIA).

d. Software Development and Maintenance: AFRL/RIEB predominantly engages in development and deployment of software and systems; however, due to AFRL’s mission, AFRL/RIEB also engages in piloting and prototyping as part of its Research, Development, Test, and Evaluation (RDT&E) efforts. As part of this capability, AFRL/RIEB provides the full spectrum of software development as well as maintenance to rapidly advance ISR capabilities for the DoD and IC.

2.1 The task areas of this TO are:

a. Task 1 - DoDIIS RDT&E Services

b. Task 2 - Cybersecurity Testing Services

c. Task 3 - Data Analytics Research Development and Integration

d. Task 4 - IT Systems Architecture, Development, Integration, and Maintenance

2.1.1 As part of this TO, the contractor may be tasked with acquiring Government-approved software and/or hardware, incidental to the enhancement, design, and development, to support system requirements; developing, enhancing, or demonstrating prototypes; or testing and prototyping system development and/or enhancement activities. In these instances, the contractor shall provide applicable license, maintenance, and diagnostics support software and all applicable arrangements for delivery of such hardware, software, and licenses to Government-designated sites.

2.1.2 The contractor shall conduct testing (e.g., function, system, interoperability, regression, security, and performance) of software throughout the development life cycle using industry best practices of continuous integration methods and automated regression test utilities. Tests (e.g., scripts, configurations, utilities, tools, plans, and results) shall be maintained under configuration control using methods that apply to source code.

2.2 TASK 1 – DODIIS RDT&E SERVICES

The contractor shall provide innovative architecture, solution designs, and DCM solutions that increase the AFRL/RIEB, USC Title 10 and USC Title 50 (Title 10/50) mission partners’ ability to reduce the time-to-field of critical business systems, ISR and C2 mission capabilities.

2.2.1 SUBTASK 1 – ANALYSIS OF ALTERNATIVES (AoA)/DESIGN SERVICES

2.2.1.1 The contractor shall review and evaluate system designs to ensure user and functional requirements are satisfied in a sound technical and cost-effective manner.

2.2.1.2 The contractor shall ensure that reviewed system designs address software requirements, hardware requirements, network requirements, security requirements, personnel requirements, and facility requirements.

2.2.1.3 The contractor shall ensure the designs conform to Government and Industry Standards and that best practices are consistent with unique user, service, command, or community-oriented policies and procedures. These designs can be for on premises, cloud, or hybrid solutions including provisions for disconnected operations.

2.2.1.4 The contractor shall include minimizing the cost of cloud compute, network, storage and data transmission in AOA activities,

2.2.1.4.1 SUBTASK 1.1 – ENTERPRISE SYSTEMS DESIGN

The contractor shall review, analyze, and provide system designs based on Government-provided constraints identified in individual Technical Solution & Cost Estimate (TSCE) (CDRL 008). The contractor shall review, analyze, and document that:

a. Designs are applicable to a single site, to multiple sites, on premises, cloud, hybrid, or other Government-specified environment including disconnected operations.

b. Designs (when applicable) address hardware, software, databases, visualization, documentation, environmental constraints, policy and directives, and security and documentation.

c. Designs conform to Government and Industry Standards and best practices and Modular Open Systems Approach (MOSA), Model Based Systems Engineering (MBSE), principles in support of Service-Oriented Architecture (SOA) and Enterprise Architectures (EA) as specified in individual TSCEs.

d. Designs are reviewed for compatibility with interfaces to existing systems, easily integrated into existing systems, scalable, and shall be easily tailored to meet individual site requirements.

2.2.1.4.2 SUBTASK 1.2 – DATABASE/INFORMATION STANDARDS

The contractor shall monitor and analyze the ongoing design and development progress of the systems for development of community of interest standards and potential impacts to program initiatives. (See CDRLs, A003, A010 and A011) Examples of databases are SQL, Oracle, PostgreSQL, MongoDB, AWS, and Azure.

2.2.1.4.2.1 As part of this task, the contractor shall:

a. Attend and participate in various technical reviews and working groups, review available documentation, and document potential issues and proposed strategies for solution.

b. Provide analysis that includes all information defined by data plans/dictionaries and is consistent and compliant with data standards, naming conventions, and logical rules.

c. Support compartmentalization of data, based on classification/caveats.

d. Be consistent with approved DoD and IC standard data modeling efforts.

e. Automate data exchange applications between data repositories and analysis tools.

2.2.1.4.2.2 The system/software documentation that could be monitored, analyzed, and revised as part of this task may include: the following:

a. Applications Programming Interface (API).

b. Interface Control Documents (ICD).

c. XML Schema.

d. Data Flow Diagrams.

e. Department of Defense Architectural Framework (DoD AF) Views

2.2.2 SUBTASK 2 – ENTERPRISE SYSTEMS MANAGEMENT AND PLANNING SERVICES

2.2.2.1 The contractor shall provide policy, technical, and programmatic analyses/planning activities and IT technical ES to support the Government’s strategic, tactical, business, and operational plan development and implementation. As part of this task, the contractor shall provide technical expertise to support Government-sponsored Technical Exchange Meetings (TEMs). The contractor shall participate in TEMs and reviews (e.g., system reviews, program/project reviews, site visits, working groups, functional and physical configuration audits, CCBs, and user groups) and provide associated documentation (i.e., meeting minutes and technical discussions for future reference). (See CDRL, A004)

2.2.2.2 The contractor shall conduct requirements and policy/guidance analyses (e.g. analysis/recommendations for System Design Document (SDD), Requirements Management Plan (RMP), Systems Engineering Plan (SEP), Configuration Management Plan (CMP), IT Support Plan (ITSP) Test and Evaluation Master Plan (TEMP), and Life Cycle Support Plan (LCSP)). (See CDRL, A013) The contractor shall:

a. Provide impact analysis of enterprise new and existing requirements.

b. Provide analysis for policy and guidance changes as identified by the Government.

c. Provide an analysis of high-level integration issues between projects and technical input to planning including consideration for planning, requirements definition, engineering, solution fielding, training, operations, and customer support.

d. Provide technical analysis and dependency identification to refine Government-identified requirements and decompose analysis into program level requirements.

e. Research and investigate technologies, approaches, and concepts with potential use as solutions to identified issues including improving system capabilities.

2.2.2.3 The contractor shall provide technical ES for ongoing support to new and existing programs.

(See CDRL, A013) As part of this subtask, the contractor shall:

a. Evaluate the technical impacts of new and existing requirements including policy, guidance, and infrastructure changes.

b. Provide technical recommendations to address changes and their impacts across strategic, tactical, and operational planning including consideration for policy, directives, requirements definition, engineering, solution fielding, training, operations, and customer support.

c. Develop modernization strategies and integrated mission area plans for ISR, C2 and Business Systems support plans and associated documents.

2.2.2.4 The contractor shall provide analysis of new (proposed) and existing technologies/capabilities including consideration of national systems impacts, interface control requirements, and impacts on functional analysis capabilities. Additionally, the contractor shall develop recommendations related to Concept of Operations (CONOPS); roadmaps; Tactics, Techniques, and Procedures (TTP); operational architectures; strategic plans; implementation plans; and action plans (CDRL 011). The contractor shall:

a. Document ranked recommendations for integration strategy(s) based on technical compliance and impacts along with other technical parameters provided by the Government in TSCEs (CDRL 008).

b. Develop designated integration/migration strategy(s).

c. Identify, verify, and provide requirements traceability for new Hardware/Software (HW/SW) capabilities and/or baselines.

2.2.2.5 The contractor shall identify, verify, and provide detailed descriptions of programmatic shortfalls, needs, or required changes to the program or system. This may include database design activities (logical and physical), applications software design and development, or technology insertion activities (document/demonstrate). (See CDRL, A019)

2.3 TASK 2 – CYBERSECURITY TESTING SERVICES

2.3.1 The contractor shall provide AFRL/RIEB and its mission partners with innovative cybersecurity and testing services, including Zero Trust, which comply with the RMF requirements and assist AFRL/RIEB and its mission partners with ATO, Fast Track ATO (RMF via 3 evidential components: (1) review of cybersecurity baseline, (2) adversarial risk assessment, and (3) Information System Continuous Monitoring (ISCM) strategy, as defined in Fast Track Authorization to Operate Handbook V2.0 dated 17 June 2020), and Continuous ATO processes. This task includes support of all six phases of testing identified in National Institute for Science and Technology (NIST) Special Publication (SP) 800-37 (Guide for Applying the RMF to Federal Information Systems) and NIST SP 800-53 (Security and Privacy Controls for Federal Information Systems and Organizations). As part of this task, the contractor shall provide AFRL/RIEB with innovative solutions to streamline RMF processes, which would allow AOs to make informed risk decisions at the speed of relevance. Per 19 October 2020 MFR directing DAF to use Fast-Track ATO as the primary process for assess risk for new IT, new platform systems, and for renewing ATOs, the Fast-Track RMF process will be used unless otherwise noted in the Fast Track ATO Handbook as a system not prepared for the Fast-Track ATO process or if AO requires the use of traditional RMF processes.

2.3.2 In general, unless otherwise specified in this task or individual TSCEs (CDRL 008), AFRL/RIEB considers a standalone system as a medium complexity system and a system of systems as a high complexity. Historically, AFRL/RIEB is tasked with providing medium to high complexity system/software testing services.

2.3.3 SUBTASK 1 – SYSTEM TESTING SERVICES

2.3.3.1 The contractor shall provide IT ES to support testing of software developed and/or enhanced under this TO.

2.3.3.2 As part of this subtask, the contractor shall provide support services for the development of a Government T&E Management Plan including roles and responsibilities of all phases of the system.

2.3.3.3 The contractor shall provide engineering support for system T&E based on the requirements and the technical nature of the design. Requirements of this task may include development of test plans, test documents, test cases, test procedures, test steps, and their allocation to the test events described above.

2.3.3.4 The contractor shall complete A&A testing and certification testing to verify compliance with current and emerging standards including, as applicable, the following:

a. DoDIIS, Joint DoDIIS/Cryptologic Information Systems Security Standards (JDCSSS).

b. IT Systems Security Risk Management.

c. C&A.

d. Security and Privacy Controls for Federal Information Systems and Organizations (NIST SP 800-53).

e. SABI/Top SABI (TSABI).

f. DISA.

g. Joint Chiefs of Staff (JCS) policies and requirements of Intelligence Mission Applications and Vulnerability.

h. AFI 99-103, Capabilities Based T&E.

2.3.3.5 Developed test documentation shall support approval to connect to JWICS, NSA Network (NSANET), SIPRNET, and Mission Networks including the AF DCGS Campus Area Network and WAN. The contractor shall provide a logical test process (for all test and test activities) that minimizes interruptions, avoids sustained downtime, and presents a contingency procedure to be implemented in the event of system(s) failure during testing (See CDRLs, A015 and A017). The contractor shall address and account for the following in its deliverable submission:

a. Security Assessment Report (See CDRL, A017)

b. Security Assessment Out Brief (See CDRL, A010)

c. SELinux Policy Analysis (See CDRL, A013)

d. Policy Bypass Test Results (See CDRL, A017)

e. Security Test Procedures (See CDRL, A016)

f. Security Test Report (See CDRL, A017)

g. CDS test plan/results (See CDRL, A015 and A017)

h. Malware Analysis (See CDRL, A028)

i. Penetration Test Plan/Results (See CDRLs, A015 and A017)

j. Source Code Review (See CDRL, A027)

k. Site Based Assessment Plan (See CDRL, A015)

l. Site Based Assessment Report (See CDRL, A017)

2.3.4 SUBTASK 1.1 – REGRESSION TESTING

2.3.4.1 The contractor shall perform testing in a Government integration environment for the entire system including each upgrade or patch installed to ensure continuing functionality. The Government integration environment will include:

a. Tools.

b. Test suites.

c. Support databases.

d. A software test lab.

e. Configuration management.

f. Hardware spares.

g. Process and procedure documentation and delivered source code.

2.3.4.2 The contractor shall develop scripts and conduct testing for the application, database, and OS IAW test plans. (See CDRL, A015, A016 and A017)

2.3.4.3 If a test fails, the contractor shall analyze and document test data for each component and rework the system to establish functional equilibrium.

2.3.4.4 Testing shall be performed to gain Government acceptance and integration; development testing shall ensure that operational and technical requirements have been satisfied; and operational testing shall ensure the system meets the needs of the user community. Throughout these test events, the system’s Information and Assurance and interoperability shall be assessed.

2.3.5 SUBTASK 1.2 – PRODUCT/SYSTEM INTEGRATION TESTING

2.3.5.1 The contractor shall perform testing and inspections of all system services to ensure the technical adequacy and accuracy of all work, including reports and other documents, required in support of that work. The contractor shall support on-site testing when requested. When specified by the Government in a TSCE, (See CDRL, A008) the contractor shall participate with the Government in testing the complete system or application at the Government integration environment. After appropriate corrective action has been taken, all tests including those previously completed, related to the failed test, and corrective action shall be repeated and successfully completed prior to Government acceptance. (See CDRL, A016)

2.3.5.2 Pre-cutover audits shall consist of verification of all testing completed by the contractor such that the system is deemed ready for functional cutover. As part of this audit, any engineered changes or approved waivers applicable to the installation shall be reviewed and agreed upon between the contractor and the Government.

2.3.5.3 Post-cutover audits shall verify that all post-cutover acceptance testing has been performed satisfactorily IAW the standard practices and identify those tests, if any, that have not been successfully completed and must be re-tested prior to acceptance.

2.3.6 SUBTASK 1.3 – DEVELOPMENT TESTING (DT)/OPERATIONAL TESTING (OT)

2.3.6.1 A Lead Development Test Organization (LDTO) shall conduct DT events, and a Lead Operational Test Organization (LOTO) shall conduct OT events. The contractor shall support DT and OT IAW the Government-approved test plan(s) as specified in the individual TSCE(s). (See CDRL, A008) This testing may include Cooperative Vulnerability and Penetration Assessment (CVPA) and Adversarial Assessments (AA). A test plan shall consist of a program of tests, inspections, and demonstrations to verify compliance with the requirements and validate operational suitability and effectiveness of the IT system tested. (See CDRLs, A015, A016 and A017) The contractor shall implement changes and/or fixes from deficiencies identified during DT and OT, and track them until they are resolved (e.g., fix, patch, redeliver, etc.). (See CDRL, A019) In the event the system or application does not meet the availability rate, the acceptance testing shall continue on a day-by-day basis until the availability rate is met.

2.3.7 SUBTASK 1.4 – GOVERNMENT ACCEPTANCE TESTING (GAT)

2.3.7.1 The contractor shall support GAT as required in individual TSCEs. (See CDRL, A008) The contractor shall implement changes and/or fixes from deficiencies identified during GAT, and track them until they are resolved (e.g., fix, patch, redeliver, etc.) (Section F, Deliverables 39 and 40).

2.3.8 SUBTASK 1.5 – INTEROPERABILITY TESTING

2.3.8.1 The contractor shall support integration and interoperability testing and provide technical expertise to support system installation; T&E at the site, consisting of the execution of the test and/or operational (system) procedures; and any site specific tests deemed necessary by the site or the Government. (See CDRLs, A015, A016 and A017)

2.3.9 SUBTASK 1.6 – SECURITY CERTIFICATION/ACCREDITATION AND IA TESTING AND SUPPORT

2.3.9.1 Depending on client requirements and systems readiness, the contractor shall:

a. Conduct security accreditation and IA testing. (See CDRLs, A015 and A017)

b. Provide technical expertise to support system security and IA testing. (See CDRLs, A025 and A013(if applicable))

c. Draft supporting documentation for CtF and ATO. (See CDRLs, A015 and A017)

2.3.9.2 The contractor shall provide system testing and documentation as described in the software accreditation test plan, similar documentation, or as directed by the Government in a TSCE (See CDRL, A008) to meet unique accreditation requirements. (See CDRLs, A015, A016 and A017)

2.3.9.3 As part of certification testing, the contractor shall provide compliance self-testing and documentation as described in the software accreditation plan, similar documentation, or as directed by the Government in a TSCE to meet or exceed system certification requirements. (See CDRL, A017)

2.3.9.4 As part of this task, the contractor shall provide:

a. Copies of all security related documentation.

b. Explanation and demonstration of test procedures.

c. Resolution of anomalies.

2.3.9.5 In addition, the contractor shall prepare the security accreditation test report or similar documentation to be loaded into the Government’s tracking system (e.g., Xacta).

2.3.10 SUBTASK 1.7 – CROSS-DOMAIN SOLUTIONS (CDS) TESTING

2.3.10.1 The contractor shall review and assess CDS against NIST 800-53 RMF controls and Raise the Bar (RtB) initiatives as specified by the NCDSMO. Generally, AFRL/RIEB is tasked by NCDSMO to provide CDS testing for up to eight medium to high complexity systems at any given interval. Each system could be in a different stage (i.e., phase I or II) of testing. AFRL/RIEB considers a medium complexity system as a single box solution transferring data from a NIPRNet to a SIPRNet or a SIPRNet to a NIPRNet. AFRL/RIEB considers a high complexity system as a multiple box solution transferring data from a NIPRNet to SIPRNet and vice versa. Historically, the Government provided the system (hardware and software) for testing; however, the contractor may be required to procure hardware and software as part of a TSCE (See CDRL, A008) requirement.

2.3.10.2 As part of this task, the contractor shall:

a. Prepare CDS Security Test Plan(s) for systems being tested. (See CDRL, A015)

b. Support the CDTAB Security Test Plan Review (STPR).

c. Support system installation and familiarization, in preparation for testing.

d. Conduct the CDS Test Readiness Review (TRR) and document the readiness of the system (See CDRL, A030)

e. Record all procedures, findings, and data.

f. Conduct CDS testing including penetration testing.

g. Provide briefing materials for presentation to the customer, CDTAB, and Defense Information Assurance Security Accreditation Working Group (DSAWG).

h. Complete Site-Based Security Assessments (SBSA), as needed.

2.3.10.3 The contractor shall deliver the following security test documentation for Cross Domain Systems:

a. Presentation Material - Briefings to CDTAB. (See CDRL, A010)

b. Security Test Plan and Procedures. (See CDRL, A015)

c. Test Readiness Review Reports. (See CDRL, A030)

d. Security Assessment Report. (See CDRL, A017)

e. Technology Risk Reports. (See CDRL, A017)

f. Policy Bypass Reports. (See CDRL, A021)

g. Penetration Test Reports. (See CDRL, A017)

h. Site-Based Security Assessment Final Reports. (See CDRL, A017)

2.3.11 SUBTASK 1.8 – PENETRATION TESTING

2.3.11.1 The contractor shall support and perform penetration testing. Penetration testing (See CDRL, A017) could be conducted as part of the ATO process including Fast Track ATO, cross-domain services, IA support, continuous monitoring, as well as part of other software/system assurance efforts. AFRL/RIEB considers penetration testing of a large enterprise environment (e.g., cloud infrastructure) that hosts multiple applications as high complexity. Testing of a web application, for example, is considered medium complexity. As part of penetration testing, the contractor may be tasked with providing qualified red/blue penetration test teams. This testing may include Cooperative Vulnerability Identification (CVI), Cooperative Vulnerability and Penetration Assessment (CVPA) and Adversarial Assessments (AA).

2.3.11.2 The techniques of penetration testing (through CVI or CVPA) include:

a. Web application / Application Programming Interface (API) security testing, with techniques focusing on invalid input testing (such as SQL Injection, Cross Site Scripting, and Command Injection), Session Management (such as session cookie/web token replay attacks), and application misconfigurations (authentication bypass, stack tracing, cross site request forgery, click jacking).

b. Network/Domain enumeration, with techniques focused on network device security, Active Directory enumeration (with a focus on least privilege, kerberoasting, SAM/SECURITY/SYSTEM hive dumping, and NTLM hash cracking.

c. Endpoint security testing, including binary exploitation, privilege escalation, lateral movement, and Linux/Windows vulnerability identification and exploitation.

2.3.11.3 Techniques of an adversarial assessment/red team assessment may include:

a. Social engineering/phishing

b. Trojan deployment

c. Physical security auditing (Close Action Team)

d. Mission-based threat replication

e. Similar actions as compared to penetration testing, with a focus more on mission objective and impact vs overall attack surface validation

2.3.11.4 Specific tools required for a penetration test instances will be specified in TSCEs (See CDRL, A008) and may either be provided by the Government or the contractor, depending on the mission partner requirements.

2.3.11.5 Historically, the following tools have been used as part of penetration testing:

a. Burp Suite Pro (licensed)

b. OWASP ZAP (open source)

c. Core Impact (licensed)

d. Shodan (publicly hosted)

e. NMap (open source)

f. Mimikatz (open source)

g. Bloodhound (open source)

h. Wire shark (open source)

2.3.11.6 The contractor may be tasked with providing, testing, and utilizing exploration tools including:

a. Android Debug Bridge (protocol connection tool)

b. HP Fortify (source code analysis)

c. QARK (source code analysis)

d. IBM Application Security on Cloud

e. Drozer Android Debugger

2.3.11.7 When required in a TSCE, the contractor may be tasked with utilizing wireless network exploitation tools including:

a. HackRF

b. Aircrack-no

c. Kismet

d. inSSIDer

2.3.11.8 The contractor shall comply with all applicable regulations related to data contained within systems including, but not limited to, Health Insurance Portability and Accountability Act of 1996 (HIPAA), Personally Identifiable Information (PII), and/or Payment Card Industry/Data Security Standard (PCI/DSS).

2.3.12 SUBTASK 2 – IA&A SUPPORT SERVICES

2.3.12.1 The contractor shall provide IA&A support and test services for customers seeking assistance in obtaining Information System C&A. AFRL/RIEB generally supports a higher volume of high complexity vs. medium complexity system assessment support. For example, 50 or higher mission applications within a system of systems is considered a high complexity requirement. In support of this subtask, the contractor shall ensure that all system or application deliverables meet the requirements of DoD and AF policy. The deliverables associated with this task could be individual or a combination of any of the following depending on the individual TSCE (See CDRL, A008) requirements:

2.3.12.2 Deliver all created documentation and any software or hardware developed, enhanced, assembled or acquired in accordance with the following:

· Presentation Material - CDSSA Briefings to CDTAB (classified reports) for Cross Domain systems (See CDRL, A010)

· Meeting Minutes for Design Reviews, Technical Exchanges, and other meetings attended. (See CDRL, A003)

· Trip Reports (See CDRL, A007)

· Risk Assessment Reports for the CA on the risk to the operational environment by fielding the system under evaluation (See CDRL, A017)

· E/APL Certification Memos (See CDRL, A031)

· Security/Design/Test Readiness Report (See CDRL, A030)

· Security Design Review Reports for Cross Domain systems

· Test Readiness Review Reports for Cross Domain systems

· CDSSA Test Procedures for Cross Domain systems (See CDRL, A023)

· CDSSA Test Reports for Cross Domain systems (See CDRL, A024)

· Technical Risk Rating Reports (classified reports) for Cross Domain systems (See CDRL, A032)

· Policy Bypass Reports (classified reports) for Cross Domain systems (See CDRL, A022)

· Site Preparation Requirements and Implementation Plan (SPRIP) (See, CDRL A021)

· Source and object (executable) Code (See CDRL, A027)

· Complete software documentation (See CDRL, A008)

· Acquired commercial off-the-shelf hardware and software (See CDRL, A029)

2.3.12.3 The contractor shall provide:

a. Independent system risk assessments in support of the RMF. As part of the deliverables listed above, the contractor shall address RMF IA control validation.

b. Testing and validation services of implemented IA controls.

c. Penetration testing of systems.

d. Assessments of security posture of AF systems.

e. Assist programs with control interpretation, validation, and scoring.

f. Review and validation of system artifacts.

g. On-site system assessments; review and validate mitigation approaches for A&A packages identified by the Information System Security Manager (ISSM).

h. Security T&E (ST&E) for systems.

i. Analyses of Security Change Notice (SCN) submissions and system changes.

j. CtF packages and SIA memos

2.4 TASK 4 – DATA ANALYTICS RESEARCH DEVELOPMENT AND INTEGRATION

2.4.1 The contractor shall provide data science support to AFRL/RIEB across various domains (i.e., open source and operational sensor data) and ML, including generating large scale labeled data sets, developing algorithms for and rapid implementation of analytics, and deploying within the modern Cloud IaaS infrastructures and associated SaaS/PaaS environments based on evolving mission areas and threats. The objective of this task is to provide an innovative solution and increase the Government’s capacity to process raw ISR data to assist and augment ISR analysts’ tasks through the application of AI and ML technology. As part of this task, the contractor shall create algorithms, from either Government or contractor provided datasets, analyze, and process datasets containing categorical labels that span in excess of one million distinct values. Data inputs will be provided from various sources. To support AFRL/RIEB’s efforts with information sharing across DoD, AF, and IC, the contractor may be tasked with developing and/or providing a shared data repository and a model library in the cloud, IAW the domain that the data resides.

2.4.2 Integration requirements for this task for AI/ML may include, but are not limited to, major systems such as DCGS and AOC.

2.4.3 As part of this task, the contractor may be required to integrate trained algorithms into one or more mission partner chosen operational environments. Algorithm outputs shall comply with Government specification to guarantee long-term interoperability. Algorithm performance scores and all trained algorithm (e.g., models) shall be classified at the highest security classification of the data they are trained upon; or the Tactic, Technique, or Procedure (TTP) the algorithm is attempting to assist with in operations; or whichever is highest, unless otherwise is specified in the TSCE.

2.4.4 SUBTASK 1 – AI/ML ALGORITHM DEVELOPMENT

2.4.4.1 The contractor shall provide competitive algorithm development and algorithm evaluation. The developed algorithms shall comply with Government specification as defined in the challenge competition documentation and TSCE (See CDRL, A008) to ease both algorithm performance evaluation and integration into existing ML frameworks already in operations. Requirement of this subtask include:

a. Developing and executing systematic methods and processes to gather intelligence analyst input, from multiple sites (physical locations) on desired AI/ML capabilities and gathering continual feedback on delivered AI/ML capabilities. (See CDRL, A010)

b. Engaging with mission partners to prioritize desired analysts capabilities. (See CDRL, A010)

c. Continual development, deployment, testing, and integration of efficient data labelling techniques and workflows. (See CDRL, A027)

d. Integrating data labeling capabilities into existing and evolving Government-owned, crowd-sourced data labeling apps and/or as part of larger Government-owned dataset repositories. (See CDRL, A027)

e. Integrating data labeling capabilities as tasks within current or improving intelligence analyst workflows, referred to as “Labeling on the Line” capabilities. (See CDRL, A027)

f. Developing prototype algorithm using existing AI/ML techniques for data including, but not limited to, computer vision (e.g., Imagery Intelligence (IMINT)), natural language processing (e.g., Human Intelligence (HUMINT), and audio (e.g., speech to text) including environmental signatures (e.g., Measurement and Signature Intelligence (MASINT) processing (e.g., Signals Intelligence (SIGINT)). (See CDRL, A027)

g. Developing and prototyping algorithms utilizing innovative techniques. (See CDRL, A027)

h. Developing automated or semi-automated techniques to divide large-scale datasets into train and validation/test splits. In the event an automated or semi-automated test is not feasible, a human (e.g., visual) evaluation may also be conducted by mission partner in a blind fashion (without knowledge of which developer created the model) at the discretion of the mission partner. (See CDRL, A027)

2.4.5 SUBTASK 2 – DATASET CHALLENGES SUPPORT

2.4.5.1 The contractor shall develop, organize, and administer dataset challenges, which could include five to 20 participants, on average, with challenges occurring once every six months. Algorithm outputs shall comply with Government specification (as identified in TSCE) to allow for long- term interoperability. Objectives of this subtask include, but are not limited to:

a. Discovering and framing.

b. Dataset acquisition.

c. Challenge participating solicitation and completion administration.

d. Algorithm integration for the purposes of dataset challenges.

e. Mission partner end-user feedback gathering and analysis.

f. Analysis of solutions and action forward plans.

2.4.5.2 Both non-proprietary and proprietary models may be developed and delivered as part of this TO. For dataset competitions, any non-proprietary algorithms will be delivered with Restricted Rights as defined in DFARS 252.227-7014. (See CDRL, A027) For proprietary trained algorithms, short-term (defined as three to 12 months) licenses shall be provided for the purposes of deploying the algorithm into operations on a limited time basis. Long-term (greater than one year) license sustainment costs for algorithms are outside of scope for this TO. Requirements of this subtask include, but are not limited to:

a. Developing and submitting a Labeled Data Campaign Plan (See CDRL, A017) and providing subject matter expertise with correcting and/or refining mislabeled data.

b. Developing and refining final dataset splits (training versus validation/testing sets). (See CDRL, A013)

c. Identifying, defining, and preparing computational resources and tools for use by algorithm developers across varying security domains. (See CDRLs, A023 and A024)

d. Defining evaluation metrics of performance. (See CDRL, A019)

e. Submitting Challenge Problem Documentation (CPD). (See CDRL, A017)

f. Identifying dataset challenge participants, the selections, depending on mission partner requirements, may be based on data science and ML experience, as well as licensing model and pricing for algorithms. The documentation shall include any proprietary algorithm license options and may include usage-based, user-based, site-wide, specific systems, and AF, DoD, and/or DoD/IC-wide pricing (See CDRL, A026)

g. Supporting/training algorithm developers on chosen algorithm development tools. (See CDRL, A010)

h. Developing algorithm with software delivery conforming to Government-identified framework and model specifications. (See CDRL, A027)

i. Completing automated T&E of algorithms throughout the challenge. (See CDRL, A019) Data gathered throughout the discovery (e.g., problem/gap analysis) and framing (i.e., solution proposal) process shall be provided to the Government.

j. Submitting trained algorithms for evaluation on a bi-weekly basis through the competition and a maximum of twice in one week.

k. Ensuring that provided algorithms include unlimited Government rights for the duration of evaluation through completion, for the purposes of the dataset challenges.

l. Out-briefing documentation and dataset challenge final technical memorandums. (See CDRL, A033) The final report shall at a minimum address analysis of the solution (algorithm) and action forward plan.

2.4.6 SUBTASK 3 – LESSONS LEARNED, ACTION PLANS, AND DEPLOYMENT

2.4.6.1 As part of evaluation, algorithm performance scores shall be delivered with unlimited Government rights and may be provided to other dataset challenge participates throughout competitions.

2.4.6.2 The contractor shall provide the following services:

a. Developing, integrating, and testing system security accreditation and deploying of ML pipelines that support rapid development and fielding of AI/ML algorithms. (See CDRL, A027)

b. Developing and/or outlining AI/ML algorithm output/inference/prediction standards that shall conform to Government specifications (i.e., Motion Imagery Standards Board), wherever feasible (See CDRL, A027)

c. Providing rapid (within 30 days) deployment of minimum viable products and continuous deployment of best of breed algorithms into operational environments. (See CDRL, A027)

d. Conducting automated T&E of algorithm performance (i.e., speed and accuracy across varying operational conditions) within development and operational environments. Algorithm performance evaluation shall be supported by pushing a trained algorithm to a server where automated evaluation software may reside. (See CDRL, A019)

e. Providing scaling offerings within and across various cloud IaaS, SaaS/PaaS environments to enable hybrid cloud deployment (including at the edge) of AI/ML capabilities. (See CDRL, A027)

f. Developing common data structures, indexing, and large-scale metadata storage produced during data labeling tasks, AI/ML predictions, and intelligence products generated by intelligence analysts during operations. (See CDRL, A027)

2.5 TASK 5 – IT SYSTEMS ARCHITECTURE, DEVELOPMENT, INTEGRATION, AND MAINTENANCE

2.5.1 The contractor shall provide innovative solutions for IT systems architecture, engineering, development, enhancement, integration, sustainment, and rapid deployment of mission applications to AFRL/RIEB and mission partners. The purpose of this task is to provide AFRL/RIEB and its mission partners with the ability to move rapidly from discovery to full operations, with limited operator/user evaluations. In support of this task, the contractor shall provide certified SCRUMMasters to support agile software development including DEVSECOPS environments. For architecture development, integration, and maintenance, the contractor shall adapt to the existing mission partner processes, policies, and existing architecture.

2.5.2 SUBTASK 1 – SYSTEM DEVELOPMENT AND ENHANCEMENT SERVICES

2.5.2.1 The contractor shall develop new and enhance existing software in support of Government system requirements. As part of this task, the contractor shall provide technical and engineering development services that may consist of coding, engineering development to validate and demonstrate a design, as well as final system development for tools and infrastructure. (See CDRL, A027) The deliverable associated with this task may incorporate (based on applicability) the following:

a. Software Version Description Document (SVDD)

b. Software Design Document

c. User Stories and Epics

d. SW Unit Test Plan

e. Data Flows

f. SW Sustainment and Maintenance Plan

g. Information Exchange Requirement

h. Application Program Interface

i. Interface Control Document

2.5.2.2 The contractor shall, as identified in individual TSCEs, (See CDRL, A008) respond to and resolve items identified in a PR/CR and assess/verify that solution(s) meet the requirements and solve the problem(s). The contractor shall provide an assessment of the impacts to system performance, security impacts, and scheduled release dates. (See CDRL, A018)

2.5.2.3 Based on the information provided in a PR/CR the contractor shall be tasked (if applicable) with developing a resolution that may include cost estimate, proposed solution technical plan, Commercial Off-the-Shelf (COTS), etc.. (See CDRL, A019)

2.5.3 SUBTASK 2 – IT SYSTEM(S)/SOFTWARE SYSTEMS ENGINEERING

2.5.3.1 The support provided by the contractor shall cover the entire IT systems engineering life cycle including requirements gathering, system design and development, installation, integration and testing, and sustainment (CDRL 002). Software/system development plans are project and mission partner specific and will be identified, as applicable, in each TSCE. (See CDRL, A008)

2.5.4 SUBTASK 2.1 – SYSTEMS/SOFTWARE ENGINEERING

2.5.4.1 As part of this task, the contractor shall provide both Life Cycle Systems Engineering and SEP services. The contractor deliverable documentation shall address and incorporate SDD, RMP, SEP, CMP, TEMP, and LCSP. (See CDRL, A011)

2.5.4.2 The contractor shall provide:

a. Life Cycle Systems Engineering: The contractor shall employ disciplined SEP through the period of performance of this TO IAW AFI 63-1201 (Life Cycle Systems Engineering) and AFI 63-101 (Integrated Life-Cycle Management), which at a minimum include:

1. Requirements development.

2. Technical management and control.

3. System/software design and architecture.

4. Integrated risk management.

5. Configuration management.

6. Data management and test, evaluation, verification, and validation practices.

b. SEP: The contractor shall, as set forth in TSCEs, (See CDRL, A008) follow the AF Program Executive Office (AFPEO) Business Enterprise Systems (BES) SEP. The contractor shall develop solutions that employ principles of open technology development and a modular open systems architecture for hardware and software as described in the DoD Open Technology Development Guidebook and Net-Centric Enterprise Solutions for Interoperability (NESI) body of knowledge. The contractor’s SEP and design activities shall also adhere to the DoD Information Sharing and Net-Centric Strategies published by the DoD Chief Information Officer (CIO) and the engineering body of knowledge and lessons learned accumulated in NESI.