HC102821R00060001.pdf

AMENDMENT OF SOLICITATION/MODIFICATION OF CONTRACT

Except as provided herein, all terms and conditions of the document referenced in Item 9A or 10A, as heretofore changed, remains unchanged and in full force and effect.

15A. NAME AND TITLE OF SIGNER (Type or print)

30-105-04EXCEPTION TO SF 30

APPROVED BY OIRM 11-84

STANDARD FORM 30 (Rev. 10-83) Prescribed by GSA

FAR (48 CFR) 53.243

The purpose of this amendment is to:

1. Provide the questions and answ ers document as attachment L1A for questions recevied by the question due date.

2. Update the solicitation as identif ied in the responses to the questions in the questions and answ ers document.

3. Extend the proposal due date by 30 days to September 30, 2021.

1. CONTRACT ID CODE PAGE OF PAGES

Z 1 111

16A. NAME AND TITLE OF CONTRACTING OFFICER (Type or print)

16C. DATE SIGNED

BY 26-Aug-2021

16B. UNITED STATES OF AMERICA15C. DATE SIGNED15B. CONTRACTOR/OFFEROR

(Signature of Contracting Officer)(Signature of person authorized to sign)

8. NAME AND ADDRESS OF CONTRACTOR (No., Street, County, State and Zip Code) X HC102821R0006

X 9B. DATED (SEE ITEM 11)

29-Jul-2021

10B. DATED (SEE ITEM 13)

9A. AMENDMENT OF SOLICITATION NO.

11. THIS ITEM ONLY APPLIES TO AMENDMENTS OF SOLICITATIONS

X The above numbered solicitation is amended as set forth in Item 14. The hour and date specified for receipt of Offer X is extended, is not extended.

Offer must acknowledge receipt of this amendment prior to the hour and date specified in the solicitation or as amended by one of the following methods:

(a) By completing Items 8 and 15, and returning 1 copies of the amendment; (b) By acknowledging receipt of this amendment on each copy of the offer submitted;

or (c) By separate letter or telegram which includes a reference to the solicitation and amendment numbers. FAILURE OF YOUR ACKNOWLEDGMENT TO BE RECEIVED AT THE PLACE DESIGNATED FOR THE RECEIPT OF OFFERS PRIOR TO THE HOUR AND DATE SPECIFIED MAY RESULT IN REJECTION OF YOUR OFFER. If by virtue of this amendment you desire to change an offer already submitted, such change may be made by telegram or letter, provided each telegram or letter makes reference to the solicitation and this amendment, and is received prior to the opening hour and date specified.

12. ACCOUNTING AND APPROPRIATION DATA (If required)

13. THIS ITEM APPLIES ONLY TO MODIFICATIONS OF CONTRACTS/ORDERS.

IT MODIFIES THE CONTRACT/ORDER NO. AS DESCRIBED IN ITEM 14.

A. THIS CHANGE ORDER IS ISSUED PURSUANT TO: (Specify authority) THE CHANGES SET FORTH IN ITEM 14 ARE MADE IN THE

CONTRACT ORDER NO. IN ITEM 10A.

B. THE ABOVE NUMBERED CONTRACT/ORDER IS MODIFIED TO REFLECT THE ADMINISTRATIVE CHANGES (such as changes in paying office, appropriation date, etc.) SET FORTH IN ITEM 14, PURSUANT TO THE AUTHORITY OF FAR 43.103(B).

C. THIS SUPPLEMENTAL AGREEMENT IS ENTERED INTO PURSUANT TO AUTHORITY OF:

D. OTHER (Specify type of modification and authority)

E. IMPORTANT: Contractor is not, is required to sign this document and return copies to the issuing office.

14. DESCRIPTION OF AMENDMENT/MODIFICATION (Organized by UCF section headings, including solicitation/contract subject matter where feasible.)

10A. MOD. OF CONTRACT/ORDER NO.

2. AMENDMENT/MODIFICATION NO. 5. PROJECT NO.(If applicable)

6. ISSUED BY

3. EFFECTIVE DATE

26-Aug-2021

CODE

DISA/DITCO-SCOTT-PL83

2300 EAST DRIVE, BUILDING 3600

SCOTT AFB IL 62225-5406

HC1028 7. ADMINISTERED BY (If other than item 6)

4. REQUISITION/PURCHASE REQ. NO.

CODE

See Item 6

FACILITY CODECODE

EMAIL:TEL:

HC102821R0006

SECTION SF 30 BLOCK 14 CONTINUATION PAGE

SUMMARY OF CHANGES

SECTION A - SOLICITATION/CONTRACT FORM

The required response date/time has changed from 31-Aug-2021 05:00 PM to 30-Sep-2021 05:00 PM.

SECTION B - SUPPLIES OR SERVICES AND PRICES

CLIN 0005

The CLIN extended description has changed from:

Cost Reimbursable ODCsPeriod of Performance: Four (4) year base period - Contract Years 1 through 4.

To:

Cost Reimbursable ODCsPeriod of Performance: Four (4) year base period - Contract Years 1 through 4.

NOTE: The requirements in DFARS 252.211-7003, Item Identification and Valuation, are applicable for this line item. The contractor shall provide DoD unique identification or a DoD recognized unique identification equivalent..

CLIN 1005

Cost Reimbursable ODCsPeriod of Performance: Five (5) year option period - Contract Years 5 through 9.

Cost Reimbursable ODCsPeriod of Performance: Five (5) year option period - Contract Years 5 through 9.

NOTE: The requirements in DFARS 252.211-7003, Item Identification and Valuation, are applicable for this line item. The contractor shall provide DoD unique identification or a DoD recognized unique identification equivalent..

CLIN 2001

Firm Fixed Price (FFP) All Variations - This CLIN applies to all variations of contract types under fixed price.Period of Performance: one (1) year option period - Contract Period 10.

Firm Fixed Price (FFP) All Variations - This CLIN applies to all variations of contract types under fixed price.Period of Performance: one (1) year option period - Contract Year 10.

CLIN 2002

Cost Reimbursable (CR) All Variations - This CLIN applies to all variations of contract types under cost reimbursable.Period of Performance: one (1) year option period - Contract Period 10.

Cost Reimbursable (CR) All Variations - This CLIN applies to all variations of contract types under cost reimbursable.Period of Performance: one (1) year option period - Contract Year 10.

CLIN 2003

Time and Material (T&M) - This CLIN applies to all variations of contract types under T&M.Period of Performance: one (1) year option period - Contract Period 10.

Time and Material (T&M) - This CLIN applies to all variations of contract types under T&M.Period of Performance: one (1) year option period - Contract Year 10.

CLIN 2004

Labor Hours (LH) - This CLIN applies to all variations of contract types under LH.Period of Performance: one (1) year option period - Contract Period 10.

Labor Hours (LH) - This CLIN applies to all variations of contract types under LH.Period of Performance: one (1) year option period - Contract Year 10.

CLIN 2005

Cost Reimbursable ODCsPeriod of Performance: one (1) year option period - Contract Period 10.

Cost Reimbursable ODCsPeriod of Performance: one (1) year option period - Contract Year 10.

NOTE: The requirements in DFARS 252.211-7003, Item Identification and Valuation, are applicable for this line item. The contractor shall provide DoD unique identification or a DoD recognized unique identification equivalent..

CLIN 2006

Requirements shall be completed in accordance with the Performance Work Statement, contract clauses, and contract requirements.Requirements will be awarded through the issuance of individual task orders that will indicate the specific CLIN, quantity, duration, price, invoicing instructions.Period of Performance: one (1) year option period

- Contract Period 10.

Requirements shall be completed in accordance with the Performance Work Statement, contract clauses, and contract requirements.Requirements will be awarded through the issuance of individual task orders that will indicate the specific CLIN, quantity, duration, price, invoicing instructions.Period of Performance: one (1) year option period

- Contract Year 10.

CLIN 2007

Requirements shall be completed in accordance with the Performance Work Statement, contract clauses, and contract requirements.Requirements will be awarded through the issuance of individual task orders that will indicate the specific CLIN, quantity, duration, price, invoicing instructions.Period of Performance: one (1) year option period

- Contract Period 10.

Requirements shall be completed in accordance with the Performance Work Statement, contract clauses, and contract requirements.Requirements will be awarded through the issuance of individual task orders that will indicate the specific CLIN, quantity, duration, price, invoicing instructions.Period of Performance: one (1) year option period

- Contract Year 10.

SECTION C - DESCRIPTIONS AND SPECIFICATIONS

The following have been modified:

C.1. - PWS

PERFORMANCE WORK STATEMENT (PWS)

Version Version Date Solicitation/Amendment Date of Solicitation/Amendment Issuance

Contract Number: (completed by the KO at time of contract or TO award)

Task Order (TO) Number: (completed by the KO at time of TO award or N/A) Contractor Name (completed by the KO at time of contract or TO award) Tracking Number: 832016230 Follow-on to Previous Contract Number:

HC102817D0006

1. Contracting Officer’s Representative (COR).

a. Primary COR.

Name: (completed by the KO at time of contract or TO award) Organization:

DODAAC:

Address:

Phone Number:

Fax Number:

E-Mail Address:

b. Alternate COR.

Department of Defense Activity Address Code (DODAAC):

Address:

Phone Number:

c. Alternate COR.

DODAAC:

Address:

Phone Number:

d. Property Administrator

DODAAC:

Address:

Phone Number:

2. Contract Title. Test, Evaluation, and Certification (TEC) II Services (TEC II)

3. Background.

The Defense Information Systems Agency (DISA)/ Joint Interoperability Test Command (JITC) TEC II Services is a follow-on to the DISA/JITC TEC Services contract, valued at $480,000,000.00. The DISA/JITC TEC Services contract was awarded using full and open competition. It is a single award Indefinite-Delivery/Indefinite-Quantity (ID/IQ) contract with Firm-Fixed-Price and Cost-Plus-Fixed-Fee TOs.

This PWS addresses the Test and Evaluation (T&E) and T&E Enabling Service Requirements of the DISA, JITC.

JITC is the Department of Defenses (DoD's) sole Joint Interoperability Certifier and only non-Service Operational Test Agency (OTA) for Information Technology/National Security Systems (IT/NSS). JITC provides its T&E services to the DoD Combatant Commands/Services/Agencies (CC/S/A); federal, state, and local Governmental agencies; United States (US) Coalition Partners; and commercial vendors and service providers. JITC provides these services to aid in mitigating the risks associated with designing, developing, acquiring, integrating, deploying, using, and sustaining new and enhanced IT/NSS capabilities. JITC’s T&E services help to ensure that joint warfighting Information Technology (IT) capabilities are interoperable and enable mission needs by:

• Determining to what extent applicable requirements and standards have been satisfied

• Identifying and helping to reduce technical and operational risks

• Baselining and characterizing levels of performance

• Documenting operational capabilities and limitations

• Inform acquisition, deployment, and operational use milestone decisions

General information concerning DISA’s and JITC’s missions, functions, organizational structures, operational capabilities, and service offerings are available through their respective public websites, as are regulations, directives, instructions, manuals, policies, and guidelines referenced in this PWS. The public website addresses are:

• DISA: http://www.disa.mil

• JITC: http://jitc.fhu.disa.mil

4. Objectives. The objective of the TEC II Services contract is for JITC to carry out unique T&E roles and responsibilities for which it is assigned within the DoD:

• Joint Interoperability Certifier – JITC has sole responsibility for certifying joint and combined interoperability of all DoD IT and NSS. JITC performs critical interoperability and standards conformance T&E to reduce the risks faced by warfighters in the field. JITC often provides evaluation and certification services based in part or in whole on testing conducted by other organizations.

• OTA – JITC is the OTA for DISA-managed programs. JITC also serves as the OTA for other DoD agencies such as the Defense Logistics Agency, the Defense Finance and Accounting Service (DFAS), the Defense Commissary Agency, the Defense Health Agency, the United States Special Operations Command, and the National Security Agency. As an OTA, JITC conducts Operational Test and Evaluation (OT&E) to assess system operational effectiveness, suitability, interoperability, and cybersecurity. JITC is the sole non-Service OTA recognized by the DoD Director, Operational Test and Evaluation (DOT&E).

• Warfighter Support – JITC aids the CC/S/As during exercises and contingency operations, helping the warfighter with on-the-spot evaluations of problem areas and viable mission-oriented solutions to include providing 24x7-hotline response.

• Major Range and Test Facility Base (MRTFB) – JITC is the only non-Service member of the 23-element MRTFB. As such, JITC is considered a national asset focused on testing IT/NSS. Under MRTFB guidelines, JITC must account for all direct costs at the job order level. MRTFB services provided include T&E capabilities, infrastructure, and resources to respond to DoD acquisition programs and other Governmental and commercial customers on a reimbursable basis.

http://www.disa.mil/ http://jitc.fhu.disa.mil/

• Developmental Test and Evaluation (DT&E) – JITC provides a full range of DT&E services. Through these services, JITC responds to DISA and other customers in their delivery of verified, reliable, fully functional, and secure technical solutions.

The TEC II services contract will provide JITC with T&E products and services that reflect a higher level of quality, innovation, and technical sophistication than JITC has required in the past. This greater emphasis on quality, innovation, and technical sophistication will raise the trust that customers have in JITC’s capabilities, with the end goal of increasing the demand for JITC’s value-added TEC services. To achieve this goal, JITC requires a contract that is staffed with a highly qualified, technically competent, professionalized workforce possessing knowledge and experience in many diverse technical areas.

5. Scope. The scope of the TEC II Services requirements includes T&E planning, test conducting, data collection and analysis, and results reporting for a wide range of functional areas and technologies to include: critical Command, Control, Communications, Computers, Combat Systems, Intelligence, Surveillance and Reconnaissance (C5ISR); defense enterprise business, logistics, and medical systems; emerging network, mobility, cloud, and Department of Defense Information Network (DoDIN) capabilities; cybersecurity, tactical, strategic, and national intelligence systems; and other IT/NSS capabilities. IT/NSS capabilities include infrastructure and environments such as the DoD Joint Information Environment which use both traditional waterfall development and deployment processes, as well as more modern Agile/Development Security Operations (DevSecOps) processes. JITC’s TEC II Services needs also include contractor-provided maintenance of the underlying policies, processes, and capabilities that enable JITC to satisfy its customers’ T&E needs in a responsive and cost effective manner. In addressing these needs, JITC develops and applies proven methods, tools, instrumentation, automation, modeling, simulation, statistical and analytical methods, and other advanced technologies and techniques to ensure the T&E rigor needed is in place to mitigate the risks and uncertainty its customers face in developing, fielding, operating, and sustaining IT/NSS capabilities. Surge support may be required and will be determined at the TO level.

In addition to this TEC II Services contract, DISA has two other large contracts, which provide T&E services: the System Engineering, Technology, and Innovation (SETI) contract and the Encore III contract. By design, neither the SETI nor Encore III contracts provide DISA OT&E or DoD Joint Interoperability Certification services. Both types of these T&E services are the exclusive responsibility of JITC to perform, and in turn, the TEC II Services contract to perform. The TEC II Services contract will complement the T&E services available through the SETI and Encore III contracts. All three contracts encourage the use of integrated testing, the reuse and sharing of T&E data, and the coordination of test events that enable the collection of data to efficiently address multiple T&E objectives.

Evolving DoD priorities and missions, coupled with continuous technological advancements, result in ever-evolving T&E requirements and challenges. Given this uncertainty and the rapid pace of change, it is important that the TEC II Services contract provide the depth, breadth, and flexibility of services required by JITC to remain relevant and responsive in satisfying its customers’ individual and unique T&E needs throughout the life of the contract.

In 2009, DoD OTA Commanders signed a memorandum of agreement committing to the use of Design of Experiments (DOE) in OT&E. In 2010, the DOT&E issued a memorandum requiring the OTAs to use scientific and statistical methods in developing rigorous and defensible test plans and in thoroughly evaluating resulting test data. The TEC II Services Contractor will be required to implement the 'Operational Test Agencies Six Core Test Principles' memo from DOT&E dated 31 May 2019.

The Deputy Assistant Secretary of Defense, Developmental Test and Evaluation (DASD(DT&E)) in turn established the term “Scientific Test and Analysis Techniques (STAT)” to refer to the use of scientific and statistical methods for the purposes of developing efficient, rigorous test strategies that yield highly accurate and repeatable results. As an OTA and provider of both DT&E and OT&E services, JITC has embraced the use of these DOT&E and DASD (DT&E) endorsed approaches. Online sources for information pertaining to Science Based Test Design (SBTD) concepts and approaches include the DOT&E website, http://www.dote.osd.mil/; the DASD(DT&E) sponsored STAT T&E Center of Excellence, www.afit.edu/STAT/; and the DOT&E sponsored Test Science Knowledge Center, www.testscience.org.

http://www.afit.edu/STAT/ http://www.testscience.org/

6. Performance Requirements.

To operate in an effective, efficient, and responsive manner, JITC requires a coordinated unity of effort across the full spectrum of T&E services it provides its customers. In satisfying JITC’s customers’ T&E needs, the scope of the TEC II Services contract must address requirements in the following distinct but interrelated performance areas:

• T&E Performance Requirements – Those service requirements enabling JITC to directly fulfill its customers’ T&E needs.

• T&E Capability Management Requirements – Those service requirements enabling JITC to maintain and enhance its core organizational T&E policies, processes, and capabilities.

• T&E Capability Requirements – Those service requirements enabling JITC to identify, research, respond to, and adopt emerging technologies and innovative approaches.

• Overarching Contract and TO Requirements – Those service requirements pertaining to the overall administration of the JITC contract and TOs.

To ensure JITC and its customers realize maximum benefit from the finite resources available to perform T&E requirements, the Contractor must possess the knowledge, skills, and abilities to seamlessly and efficiently perform all performance areas. The specific form, format, and content for the deliverables associated with each of the performance areas will be specified in individual TOs. The general requirements associated with these performance areas are as follows.

6.1 T&E Performance Requirements.

The following performance requirement areas identify the services JITC requires to ensure “unity of effort” in meeting its customers’ T&E needs:

• Performance Area 1.1: DT&E

• Performance Area 1.2: OT&E

• Performance Area 1.3: Interoperability T&E

• Performance Area 1.4: Standards Compliance and Conformance T&E

• Performance Area 1.5: Cybersecurity T&E

• Performance Area 1.6: Unified Capabilities (UC) T&E

• Performance Area 1.7: Other T&E and Subject Matter Expert Support

• Performance Area 1.8: DevSecOps

The DoD Chief Information Officer (CIO), Joint Staff (JS), DOT&E, Defense, Deputy Assistant Secretary of Defense for Developmental Test & Evaluation, CC/S/A level Milestone Decision Authorities, Component Acquisition Executives, and Program Managers (PMs) for DoD Acquisition Category (ACAT) and non-ACAT programs are all reliant on JITC and its T&E services to perform various program oversight functions. JITC’s T&E results and findings serve as critical input to program acquisition and deployment decisions and to particular program review board processes such as Network Readiness Review Boards and Acquisition Review Boards.

Because of the importance of the information JITC provides to leaders providing program oversight and decision processes, it is vital that the Contractor is thorough, objective, and fully responsive in providing the T&E services required in this PWS. It is equally vital that the products and deliverables the Contractor produces in providing these T&E services be of high quality, free of defects and inaccuracies, and fully compliant with all DoD, DISA, JITC, and other applicable policies, instructions, guidelines, and reporting and formatting requirements.

The Contractor shall possess the technical competencies and mission expertise to provide full T&E lifecycle services to include T&E planning, execution, data collection, analysis, and reporting:

• T&E Planning – As part of T&E planning, the Contractor shall review and comment on system/program requirements; recommend specific T&E scope and objectives; perform in-depth analyses and risk assessments to identify specific T&E requirements and data sources; review testing conducted by other elements outside of JITC to help determine what data can be re-used and identify gaps to help focus JITC efforts; establish effective strategies based on SBTD approaches and instrumented data collection methods to reliably achieve T&E objectives; develop T&E documentation; and plan for and coordinate the activities needed to carry out these strategies and plans in a rigorous, disciplined, and cost efficient fashion.

• T&E Execution and Data Collection – In T&E execution, the contractor shall conduct the test and ensure the required data is collected according to approved T&E strategies and plans, including the effective implementation of manual and automated data collection processes, and the performance of those activities required to manage, validate, and reduce collected data. Throughout T&E execution, the Contractor shall also identify and mitigate any limitations with the potential to affect the accomplishment of T&E objectives.

• T&E Analysis – The Contractor shall apply systematic and rigorous statistical methods and analytical tools to thoroughly evaluate collected data to efficiently establish T&E objectives and ensure the resulting conclusions are at the confidence levels targeted during the T&E planning phase. In analyzing the collected data, the Contractor shall focus on documenting the operational significance and mission impact of any capabilities and limitations associated with the System Under Test (SUT).

• T&E Reporting – In performing T&E reporting functions, the Contractor shall apply appropriate methods to clearly, concisely, and objectively document T&E results and findings in a timely and responsive manner, delivering high quality test reports that comply with JITC-approved processes and satisfy end-customer information needs. Reporting of T&E results should emphasize the operational significance of test findings, and when possible, provide actionable value-added recommendations.

The Contractor shall have extensive knowledge of the functional and operational requirements, intended users, mission needs, and environmental conditions that the systems, technologies, and devices undergoing T&E are intended to fulfill. The Contractor shall possess a thorough understanding of the Tactics, Techniques, and Procedures, Concepts of Operation, Standard Operating Procedures (SOPs), Military and Commercial Standards, and operational issues pertaining to each SUT. The Contractor shall apply this understanding to help specify and establish the operationally representative test environments and conditions needed to effectively determine to what extent each SUT satisfies operational mission needs. The Contractor shall apply techniques such as mission thread analysis and decomposition, message traffic generation, models and simulation, stress and load testing, and other techniques to provide operational realism and mission context in the conduct of testing.

In providing T&E services, the Contractor shall identify and apply innovative methods and technologies to improve the overall efficiency and cost effectiveness by which JITC satisfies its customers’ T&E needs. In executing specific T&E objectives, the Contractor shall consider all valid data sources, which includes data collected by JITC or other Governmental and non-Governmental organizations through dedicated test events, technology demonstrations, exercises, actual system operations, or other relevant sources of information. To the extent possible, the Contractor is encouraged to propose and apply automated and instrumented capabilities that will both lead to affordable and accurate results and establish processes and capabilities that can be reused to satisfy similar T&E requirements in the future. To improve overall T&E efficiency, the Contractor should exploit all opportunities for integrated test events led by JITC or other testing organizations that allow multiple T&E objectives to be concurrently and cost effectively achieved.

To ensure accurate and defensible T&E outcomes, and to comply with DOT&E Assistant Secretary of Defense for Research & Engineering mandates for increased T&E rigor and efficiency, the Contractor shall adopt SBTD methods as part of their execution of JITC’s TEC II Services requirements. Methods such as regression analysis, DOE, Analysis of Variance and other statistical approaches, combinatorial test techniques, and test automation and instrumentation should be considered as means to accurately and efficiently evaluate the performance characteristics of systems or products undergoing test, over the full range of required operating conditions. The Contractor shall help define formal analysis structures and evaluation frameworks as the basis for a program’s T&E, to ensure that T&E is performed in a thorough, systematic, and structured manner. In T&E planning, the Contractor shall make use of SBTD approaches to determine the relevant combination of factors to be systematically controlled, varied, or simply recorded to achieve efficient test designs. For each test design, the Contractor must justify test data requirements by citing statistical measures of merit (applicable statistical models, measures of orthogonality, prediction variance, power, confidence, etc.). Each test design must also account for the statistical methods of analysis to be used to determine if correlation exists between changes in test conditions to measured variations in performance. In cases where non-statistical methods are used to achieve particular T&E objectives, the Contractor shall justify such test design choices and explain how sufficient statistical confidence will be achieved, based on the available data. To mitigate physical or resource constraints, the Contractor shall make use of computer models and simulations, virtualized test environments, and other innovative valid approaches for cost effectively satisfying particular T&E objectives, where appropriate. To provide accurate and well-informed post-test analysis and reporting, the Contractor shall determine the statistical uncertainty and levels of confidence associated with test findings. The Contractor shall document SBTD activities, results, and findings in a manner that is clear, concise, and easily understood by JITC customers who may not possess an in-depth knowledge of technical, scientific, or statistical concepts.

The TEC II Service contract T&E requirements are divided into eight performance areas. Many JITC customers require T&E services in more than one of these performance areas. In these cases, the Contractor shall satisfy the requirements in the most efficient and cost effective manner possible, using methods such as combined or integrated test events, where appropriate. The eight T&E requirement performance areas follow.

6.1.1 Performance Area 1.1: DT&E.

The Contractor shall perform planning, conducting, and reporting on DT&E. The Contractor shall verify the design, development, integration, and technical characteristics of new or enhanced IT/NSS capabilities prior to their acceptance and consideration for operational use. These DT&E services will be provided for capabilities being delivered by developers using both conventional development methods, as well as advanced methods such as Agile/SCRUM and Test Driven Design approaches. The Contractor shall apply SBTD methods such as combinatorial test methods, instrumentation and automation, and other advanced approaches, where appropriate, to improve the overall efficiency, effectiveness, rigor, and timeliness of JITC’s DT&E activities. The Contractor may provide DT&E services to DISA programs pertaining to enterprise services, cloud computing, mobility, UC, and other emerging technologies being adopted by DoD. As part of this service, the Contractor shall attend and provide subject matter expertise to advise Working-level Integrated Product Teams (WIPTs) to facilitate program T&E planning and provide DT&E input to programs’ Test and Evaluation Master Plan (TEMP). The DT&E services the Contractor shall provide includes:

6.1.1.1 Unit, Module, Device, Subsystem, and System Testing.

The Contractor shall perform DT&E planning, execution, data collection and analysis, and reporting to verify that software and hardware modules, devices, subsystems, and systems undergoing development satisfy their stated requirements for functionality, interoperability, standards conformance, performance, capacity, scalability, reliability, availability, maintainability, cybersecurity, and other required technical characteristics as specified in applicable requirements sources and design specifications. The Contractor shall perform risk assessments to help focus DT&E resources on areas of highest technical risk. The Contractor shall conduct regression testing to verify that previously evaluated system characteristics are not adversely affected by subsequent configuration changes.

The Contractor shall provide services that include:

• Develop DT&E strategies and methodologies

• Develop Risk Assessment plans and reports

• Develop Requirements Traceability Matrices (RTMs)

• Develop use cases, test cases, detailed test procedures, and test scripts

• Develop DT&E Test Plans

• Develop Product Deficiency Reports

• Develop DT&E Test Reports

• Provide input to the TEMP and/or Test and Evaluation Strategy (TES)

6.1.1.2 Integration and Qualification Testing.

The Contractor shall perform T&E planning, execution, data collection and analysis, and reporting to verify that software and hardware modules, devices, subsystems, and systems perform as intended when integrated together in an approved system design or architecture. The Contractor shall perform risk assessments to help focus T&E resources on areas of highest technical and operational risk. The Contractor shall perform T&E to determine the degree to which the integrated system satisfies applicable design requirements for functionality, interoperability, standards conformance, performance, capacity, scalability, reliability, availability, maintainability, cybersecurity, and other required technical characteristics as specified in applicable requirements sources and design specifications.

The Contractor shall conduct regression testing to verify that previously evaluated system characteristics are not adversely affected by subsequent configuration changes. The Contractor shall provide services that include:

• Develop Integration/Qualification strategies and methodologies

• Develop risk assessment plans and reports

• Develop RTMs

• Develop use cases, test cases, detailed test procedures, and test scripts

• Develop and present Test Concept Briefs (TCBs)

• Develop Integration/Qualification Test Plans

• Develop Product Deficiency Reports

• Develop Integration/Qualification Test Reports

• Provide input to the TEMP and/or TES

6.1.1.3 Acceptance Testing.

The Contractor shall perform T&E planning, execution, data collection and analysis, and reporting to verify that the developer has satisfied applicable functional requirements. Such testing may take the form of a software/system acceptance test, government acceptance test, or user acceptance test; normally involves test participation by representatives of the system’s intended user community; and precedes any Government commitment to limited deployment and formal OT&E. The Contractor shall conduct regression testing to verify that previously evaluated system characteristics are not adversely affected by subsequent configuration changes. The Contractor shall provide services that include:

• Develop Component Qualification Test Plans, including associated strategies and methodologies evaluating the integration of system components during development

• Develop Risk Assessment plans and reports

• Develop RTMs

• Develop TCBs

• Develop use cases, test cases, detailed test procedures, and test scripts

• Develop Acceptance Test Plans

• Develop Product Deficiency Reports

• Develop Acceptance Test Reports

• Provide input to the TEMP and/or TES

6.1.2 Performance Area 1.2: OT&E.

The Contractor shall perform OT&E planning, conduct, data collection and analysis, and reporting for purposes of assessing or determining the operational effectiveness, suitability, and survivability of the IT/NSS SUT (OT&E survivability requirements are addressed in PWS section 6.1.5.4, OT&E Cyber Survivability). JITC performs such OT&E activities to inform program acquisition decisions, both for programs relying on traditional waterfall development approaches as well as those utilizing Agile/DevSecOps development methods. The Contractor shall provide these services IAW: USC Title 10, Section 139 requirements, latest DOT&E guidance and directives (https://www.dote.osd.mil/), and per the processes set forth in the JITC OT&E Guidebook (http://jitc.fhu.disa.mil/).

In providing these services, the contractor shall apply the 'Operational Test Agencies Six Core Test Principles' presented in the OTA Commanders’ memo to DOT&E, 31 May 2019. Where appropriate, the Contractor shall apply SBTD approaches, T&E automation, instrumentation, simulation, and stimulation methods to ensure efficient collection of quantifiable, operationally representative, and highly accurate test data that feeds into OT&E objectives with increased confidence and mission relevance. OT&E activities may take the form of either Operational Assessments (OAs), to include Field Assessments (FAs) and Limited User Tests (LUTs); or Operational Tests (OTs), to include Initial Operational Test and Evaluation or Follow-On Operational Test and Evaluation. Whether an OA or an OT, OT&E necessitates evaluation based on typical trained users and maintainers exercising an operationally representative system configuration, operated under realistic conditions, to determine to what extent the SUT satisfies operational mission needs. The Contractor shall provide services that include:

• Identify operational requirements through review of program documentation such as Joint Capabilities Integration and Development System (JCIDS) documents and other requirements sources.

• Perform mission analysis through review of applicable Joint Capability Areas (JCAs), Universal Joint Task Lists, Mission Essential Task Lists, Mission Essential Tasks, mission threads, use cases, etc.

• Conduct risk assessments to determine appropriate levels of OT&E.

• Develop OT&E concepts and strategies.

• Document OT&E strategies within TCBs and TEMPs.

• Attend and provide input to T&E, User Working Groups, and WIPTs with stakeholders (PMs, DOT&E representatives, user representatives, T&E participants, etc.) to plan OT&E activities and resource requirements.

• Develop OT&E Analysis Structures composed of an evaluation framework and a data source matrix.

• Capitalize on existing relevant test or operational data, and opportunities for integrated and/or combined testing where DT&E, Cyber, or other relevant T&E events can be leveraged to feed OT&E objectives, thus potentially reducing the scope, duration, expense, and delay associated with dedicated phases of OT&E.

• Construct user surveys, questionnaires, interview forms, and other qualitative and quantitative data collection instruments.

• Employ SBTD methods, automation, and instrumentation as means to carry out OT&E objectives in an efficient and rigorous manner.

• Develop detailed OT&E test plans.

• Attend and provide input to Operational Test Readiness Reviews (OTRR).

• Conduct pilot tests, not for the OT&E of the SUT, but to verify and ‘dry run’ test and data collection procedures, etc., prior to the start of formal phases of testing.

• Carry out OT&E execution and data collection in accordance with approved test plans.

• Provide input to ‘hot washes,’ test coordination meetings, and test status reporting throughout the conduct of OT&E events.

• Attend and provide input to Data Authentication Group (DAG) functions performed to authenticate and validate test data.

• Analyze collected data to arrive at an operational effectiveness, suitability, and cyber survivability determination.

• Prepare final reports and briefings to convey OT&E results and findings to Senior Executive Service (SES) and/or General Level officials.

• Develop OT&E TCBs

• Provide OT&E input to program TEMPs

• Develop Analysis Structures

• Develop Risk Assessments Plans and Reports

• Develop OT&E Test Plans

• Develop OTRR Reporting

• Develop Operational Quick Look Reports

• Develop OT&E Reports

• Develop OT&E Results Briefings

• Develop DAG Reports

6.1.2.1 Traditional OT&E.

The Contractor shall perform T&E planning, execution, data collection and analysis, and reporting to address the objectives of assessing or determining operational effectiveness, suitability, and cyber survivability. JITC conducts such testing for major program acquisition decisions. OT&E activities may take the form of either OAs, to include FAs and LUTs; or OTs, to include Initial Operational Test and Evaluation or Follow-On Operational Test and Evaluation. Whether an OA or an OT, OT&E necessitates evaluation based on typical trained users and maintainers exercising an operationally representative system configuration, operated under realistic conditions, to determine to what extent the SUT satisfies operational mission needs. In providing services to OT&E activities, the Contractor shall follow the processes set forth in the JITC OT&E Guidebook, accessible via the JITC public website at http://jitc.fhu.disa.mil/. The Contractor shall also be responsive to the latest guidance from DOT&E, accessible at http://www.dote.osd.mil/guidance.html. The Contractor shall provide services that include:

• Identify operational requirements through review of program documentation such as JCIDS documents and other requirements sources.

• Perform mission analysis through review of applicable JCAs, Universal Joint Task Lists, Mission Essential Task Lists, Mission Essential Tasks, mission threads, use cases, etc.

• Conduct risk assessments to determine appropriate levels of OT&E.

• Develop OT&E concepts and strategies.

• Document OT&E strategies within TCBs and TEMPs.

• Attend and provide input to T&E, User Working Groups, and WIPTs with stakeholders (PMs, DOT&E representatives, user representatives, T&E participants, etc.) to plan OT&E activities and resource requirements.

• Develop OT&E Analysis Structures composed of an evaluation framework and a data source matrix.

• Capitalize on existing relevant test or operational data, and opportunities for integrated and/or combined testing where DT&E, Cyber survivability, or other relevant T&E events can be leveraged to address OT&E objectives, thus potentially reducing the scope, duration, expense, and delay associated with dedicated phases of OT&E.

• Construct user surveys, questionnaires, interview forms, and other qualitative and quantitative data collection instruments.

• Employ SBTD methods, automation, and instrumentation as means to carry out OT&E objectives in an efficient and rigorous manner.

• Develop detailed OT&E test plans.

• Attend and provide input to OTRR.

• Conduct pilot tests, not for the OT&E of the SUT, but to verify and ‘dry run’ test and data collection procedures, etc., prior to the start of formal phases of testing.

• Carry out OT&E execution and data collection in accordance with approved test plans.

• Provide input to ‘hot washes,’ test coordination meetings, and test status reporting throughout the conduct of OT&E events.

• Attend and provide input to DAG functions performed to authenticate and validate test data.

• Analyze collected data to arrive at an operational effectiveness, suitability, and cyber survivability determination.

• Prepare final reports and briefings to convey OT&E results and findings to SES and/or General Officer

Level officials.

6.1.2.2 Capability Assessments.

The Contractor shall perform OT&E planning, execution, data collection and analysis, and reporting for capability assessments. As part of these services, the Contractor shall apply streamlined OT&E techniques to provide an operational perspective to system evaluations performed without the associated structure, rigor, or formality normally associated with traditional OT&E. The basis for a capability assessment may include the application of approaches outlined in the JITC OT&E Guidebook Version 2.0 (2017). Examples of capability assessment activities include observational studies performed as part of normal system operations; abbreviated field assessments to identify operational capabilities and limitations; limited assessments conducted as part of military exercises, proofs of concept, and technology demonstrations; and assessments which reach or infer operational conclusions from DT&E or actual system usage data (e.g., component reliability effects on overall system reliability). The Contractor shall provide services that include:

• Develop Capability Assessment TCBs

• Develop Capability Assessment Plans

• Develop Capability Assessment Quick Look Reports http://www.dote.osd.mil/guidance.html

• Develop Capability Assessment Reports

• Develop and Present Capability Assessment Results Briefings

6.1.3 Performance Area 1.3: Interoperability T&E.

6.1.3.1 Interoperability Certification.

The Contractor shall provide Interoperability T&E planning, execution, data collection, analysis, and reporting to address the objectives of Joint Interoperability Certification and assessment as outlined in DoD Instruction (DoDI) 8330.01, “Interoperability of Information Technology (IT), Including National Security Systems (NSS),” the JITC Interoperability Process Guide (IPG), and JITC Instruction 380-50-02, “JITC Interoperability and Standards Conformance Test and Evaluation (T&E) Certification Instruction.” Additional guidance, such as implementing the 'Operational Test Agencies Six Core Test Principles' memo from DOT&E, 31 May 2019, the JITC Interoperability (IOP) Service Area Standard Operating Procedure (SOP), and other instructions and policies pertaining to DoD interoperability roles, responsibilities, and requirements are documented in DoDIs, Chairman, Joint Chief of Staff Instructions (CJCSIs), and DoD guides and manuals to include: U.S. Code (USC) Title 10-Armed Forces, Sections 2223; DoDI 5000.02; CJCSI 5123.01H, Content Guide for the Net-Ready Key Performance Parameter (NR KPP).

These reference materials are available through the TEC II Services RFP Bidders Library on SAM.gov. Additional information concerning interoperability certification can be found at JITC’s public website: http://jitc.fhu.disa.mil

In cases where JITC will rely solely on tests conducted by other organizations to make a certification determination, the Contractor shall collaborate with other organizations to ensure appropriate data is collected and analyzed. In cases where JITC’s customers may not require or be ready for Joint Interoperability Certification, JITC may perform an interoperability assessment. For JITC customers’ interoperability certification or assessment needs, the Contractor shall provide services that include:

• Provide interoperability T&E input to program TEMPs.

• Attend and provide subject matter expertise to program T&E working groups and WIPTs to coordinate test data requirements and interoperability T&E activities with Program Management Offices, users, OTAs and other test agencies.

• Provide guidance to programs with NR KPP, Performance Attributes, JCIDS, and Business Capability Lifecycle Defense Business Systems product reviews and development to ensure valid requirements are Joint Staff approved, testable, and measurable.

• Review and provide feedback on DoD Architecture Framework (DoDAF) viewpoints, or other architectural products, and interface control documentation.

• Assess the levels of risk associated with a program’s specific interoperability-related requirements (e.g., interfaces, information exchanges, and standards).

• Prepare Joint Interoperability Evaluation Plans based on Requirements Analysis for Test (RAFT), T&E strategies, evaluation methodologies, and Data Management and Analysis Plans (DMAPs) in accordance with the Service Area SOP.

• Develop a RAFT that identifies gaps and/or omissions within DoDAF viewpoints, which includes DMAPs, coordinated in conjunction with the program offices to manage data, products, and programs artifacts addressing Joint Interoperability Certification.

• Design T&E strategies to enable determination of program interoperability leading to appropriate Joint Interoperability Certification product.

• Apply SBTD methods, automation, and instrumentation ensuring rigor, accuracy, repeatability, and cost effectiveness for Joint Interoperability Certification.

• Provide Interoperability Statements as input to OTRRs.

• Collect and analyze interoperability certification data per approved Test Plan/Test Support Package.

• Report interoperability T&E findings via the appropriate Joint Interoperability product (e.g. Quick Look

Report, Test Report, and Joint Interoperability Certification).

• Prepare certification products.

• Provide program specific interoperability and T&E status, results, findings, and reports.

6.1.3.2 Other Interoperability T&E.

The Contractor shall perform T&E planning, execution, data collection and analysis, and reporting of interoperability T&E objectives other than those specified by DoDI 8330.01. An example of such services includes Mission Partner Environment testing performed in accordance with CJCSI 5128.01/02. The Contractor shall provide services that include:

• Analyze system requirements documentation, such as DoDAF views, and develop RAFT

• Develop Interoperability T&E strategies within TCBs

• Develop Interoperability Evaluation Plans

• Develop T&E Readiness Reviews and Briefings

• Develop Interoperability Quick Look Reports

• Develop Interoperability Assessment Reports

• Develop Request for Information (RFI) Questionnaires

• Develop Interoperability Desktop Analysis (DTA) Reports

• Develop Assurance, Verification & Validation (AV&V) Plans/Reports

• Develop Component Interoperability Process, Training, Guidance, Certification and Testing.

6.1.4 Performance Area 1.4: Standards Compliance and Conformance T&E.

6.1.4.1 Standards Conformance and Compliance Verification.

The Contractor shall perform planning, execution, data collection and analysis, and reporting to determine system, product, or device compliance and conformance with particular standards and formal specifications. JITC provides standards compliance/conformance verification addressing a broad range of different types of standards and regulations to include: North Atlantic Treaty Organization Standardization Agreements, tactical data links, electronic messaging formats, imagery, communication waveforms, finance and accounting, health and medical affairs, records management, and other standards conformance and compliance areas as listed in section 6.4.4, subsection ”Standards Conformance” of this PWS, “Technical Skills, Training, Credentials, and Experience.” The

• Develop standards compliance and conformance concepts, methods, and detailed test plans and procedures as outlined in JITC Instruction 380-50-02, Chapter 8.

• Apply SBTD methods, automation, and instrumentation to improve standards compliance/conformance rigor, accuracy, repeatability, and cost effectiveness.

• Conduct standards compliance/conformance assessment and verification activities per approved plans.

• Prepare standards compliance/conformance assessment and certification reports.

6.1.4.2 Standards Compliance Program Administration.

The Contractor shall perform research and administrative functions to maintain and improve existing conformance/compliance certification policies, procedures, and capabilities. The Contractor shall also perform research and administrative functions in establishing policies, procedures, and capabilities required to address the evaluation and conformance/compliance certification of new standards. The Contractor shall maintain standards conformance reference profiles, Approved Product Lists (APL), or other conformance registry information for particular standards compliance programs under JITC’s oversight. Within the scope of this performance area, the Contractor shall also provide technical solutions to JITC as Subject Matter Experts (SMEs) for particular standard programs and provide input to the JITC Risk Assessment Database and other standards compliance capabilities. The

• Develop standards compliance instructions and procedures.

• Develop standards compliance reference documentation.

• Develop APL, and other registry updates to facilitate dissemination of compliance status information.

6.1.4.3 Integrated Broadcast Service (IBS) Lab T&E.

The…