TEC.pdf

SOLICITATION, OFFER AND AWARD

1. THIS CONTRACT IS A RATED ORDER

UNDER DPAS (15 CFR 700)

RATING PAGE OF PAGES

1 167

2. CONTRACT NO. 3. SOLICITATION NO. 4. TYPE OF SOLICITATION [ ] SEALED BID (IFB)

5. DATE ISSUED 6. REQUISITION/PURCHASE NO.

HC1028-16-R-0007

[ X ]

NEGOTIATED (RFP) 29 Sep 2016

7. ISSUED BY

DISA/DITCO-SCOTT-PL83

2300 EAST DRIVE

SCOTT AFB IL 62225-5406

TEL:

FAX:

CODE HC1028 8. ADDRESS OFFER TO

See Item 7

(Ifother than Item 7)

TEL:

FAX:

CODE

NOTE: In sealed bid solicitations "offer" and "offeror" mean "bid" and "bidder".

SOLICITATION

9. Electronic offers for furnishing the services in the Schedule will be received at the web address specified in Section L.2.3 until 1:00 p.m. (CST) on Nov 14, 2016.

CAUTION - LATE Submissions, Modifications, and Withdrawals: See Section L, Provision No. 52.214-7 or 52.215-1. All offers are subject to all terms and conditions contained in this solicitation.

10. FOR INFORMATION A. NAME

CALL:

B. TELEPHONE (Include area code)

11. TABLE OF

CONTENTS

(NO COLLECT CALLS) C. E-MAIL ADDRESS

(X) SEC. DESCRIPT ION

PART I - THE SCHEDULE

PAGE(S) (X) SEC. DESCRIPT ION PAGE(S)

PART II - CO NTRACT CLAUSES

X A SOLICITATION/ CONTRACT FORM 1 X I CONTRACT CLAUSES 100 - 119

X B SUPPLIES OR SERVICES AND PRICES/ COST S 2 - 8 PART III - LIST OF DO CUMENTS, EXHIBITS AND OTHER ATTACHMENTS

X C DESCRIPTION/ SPECS./ WORK STATEMENT

X D PACKAGING AND MARKING

9 - 89

X J LIST OF ATTACHMENTS

PART IV - REPRESENTATIONS AND INSTRUCTIO NS

120 - 121

X E INSPECTION AND ACCEPTANCE

X F DELIVERIES OR PERFORMANCE

91 - 92 93 - 94 X K

REPRESENTATIONS, CERTIFICATIONS AND

OTHER STATEMENTS OF OFFERORS

122 - 133

X G CONTRACT ADMINISTRATION DATA 95 - 98 X L INSTRS., CONDS., AND NOTICES TO OFFERORS 134 - 159

X H SPECIAL CONTRACT REQUIREMENTS 99 X M EVALUATION FACTORS FOR AWARD 160 - 167

OFFER (Must be fully completed by offeror) NOTE: Item 12 does not apply if the solicitation includes the provisions at 52.214-16, Minimum Bid Acceptance Period.

12. In compliance with the above, the undersigned agrees, if this offer is accepted within calendar days (60 calendar days unless a different period is inserted by the offeror) from the date for receipt of offers specified above, to furnish any or all items upon which prices are offered at the price set opposite each item, delivered at the designated point(s), within the time specified in the schedule.

13. DISCOUNT FOR PROMPT PAYMENT

(See Section I, Clause No. 52.232-8)

14. ACKNOWLEDGMENT OF AMENDMENTS

(The offeror acknowledges receipt of amendments to the SOLICITATION for offerors and related documents numbered and dated):

AMENDMENT NO. DATE AMENDMENT NO. DATE

15A. NAME

AND

CODE FACILITY 16. NAME AND TITLE OF PERSON AUTHORIZED TO

ADDRESS SIGN OFFER (Type or print)

OF

OFFEROR

15B. TELEPHONE NO (Include area code) 15C. CHECK IF REMITTANCE ADDRESS

IS DIFFERENT FROM ABOVE - ENTER

SUCH ADDRESS IN SCHEDULE.

17. SIGNATURE 18. OFFER DATE

AWARD (To be completed by Government)

19. ACCEPTED AS TO ITEMS NUMBERED 20. AMOUNT 21. ACCOUNTING AND APPROPRIATION

22. AUTHORITY FOR USING OTHER THAN FULL AND OPEN COMPETITION: 23. SUBMIT INVOICES TO ADDRESS SHOWN IN ITEM

10 U.S.C. 2304(c)( ) 41 U.S.C. 253(c)( ) (4 copies unless otherwise specified)

24. ADMINISTERED BY (Ifother than Item 7) CODE 25. PAYMENT WILL BE MADE BY CODE

26. NAME OF CONTRACTING OFFICER (Type or print) 27. UNITED STATES OF AMERICA 28. AWARD DATE

TEL: EMAIL: (Signature of Contracting Officer) IMPORTANT - Award will be made on this Form, or on Standard Form 26, or by other authorized official written notice.

Previous Edition is Unusable 33-134 STANDARD FORM 33 (REV. 9-97)

Prescribed by GSA FAR (48 CFR) 53.214(c)

Section B - Supplies or Services and Prices

ITEM NO SUPPLIES/SERVICES QUANTITY UNIT UNIT PRICE AMOUNT

0001 12 Months

TEC Services - Firm Fixed Price

FFP

Firm Fixed Price (FFP) All Variations - This CLIN applies to all variations of contract types under fixed price. See Schedule B labor rate tables.

Period of Performance: Two (2) year base period - Contract Years 1 through 2.

FOB: Destination

NET AMT

0002 Lot

TEC Services - Cost

COST

Cost Reimbursable (CR) All Variations - This CLIN applies to all variations of contract types under cost reimbursable. See Schedule B labor rate tables.

ESTIMATED COST

0003 Lot

Other Direct Costs (ODCs)

COST

Cost Reimbursable ODCs - IAW PWS 6.3.13 (Contractor Purchasing Procedures) and 7.1.2.3.6 (Travel)

1001 12 Months OPTION TEC Services - Firm Fixed Price

FFP

Period of Performance: Option Year 1 - Contract Year 3.

1002 Lot OPTION TEC Services - Cost

COST

contract types under cost reimbursable. See Schedule B labor rate tables

AMOUNT

OPTION Other Direct Costs (ODCs)

COST

2001 12 Months OPTION TEC Services - Firm Fixed Price

FFP

Period of Performance: Option Year 2 - Contract Year 4.

ITEM NO SUPPLIES/SERVICES QUANTITY UNIT UNIT PRICE

1003 Lot

2002 Lot OPTION TEC Services - Cost

COST

2003 Lot OPTION Other Direct Costs (ODCs)

COST

3001 12 Months OPTION TEC Services - Firm Fixed Price

FFP

Period of Performance: Option Year 3 - Contract Year 5.

OPTION TEC Services - Cost

3002 Lot

3003 Lot

4001 12 Months OPTION TEC Services - Firm Fixed Price

FFP

Period of Performance: Option Year 4 - Contract Year 6.

OPTION TEC Services - Cost

SECTION B

B.1 MAXIMUM AMOUNTS, INDEFINITE-DELIVERY/INDEFINITE QUANTITY (ID/IQ)

a. This TEC Services contract provides technical solutions for the Department of Defense (DoD) in support of: (1) Test and Evaluation (T&E) services including Interoperability (IOP), Operational, Developmental, Standards

4002 Lot

4003 Lot

Compliance and Conformance, Cybersecurity, Enterprise Service, and other T&E and Subject Matter Expert Support services, and (2) T&E enabling services such as DoD IOP status development and management, management of TEC policies and processes, advanced technologies research, knowledge management, requirements review and documentation management, customer relationship management, test automation and instrumentation, testbed management, and laboratory support. TEC Services are provided to DoD Combatant Commanders/Services/Agencies, federal and state agencies, coalition partners, and commercial entities to aid in the design, development, acquisition, integration, deployment, operational use, and sustainment of new and enhanced Information Technology/National Security System (IT/NSS) capabilities.

b. This is an ID/IQ contract with Fixed-Price (FP) and Cost-Reimbursement (CR) type task orders (TOs) in accordance with FAR Subpart 16.5 - Indefinite- Delivery Contracts. The guaranteed minimum amount is $10,000.00 for the overall IDIQ contract, including all option periods. The minimum guaranteed amount will be satisfied via a TO issued at contract award. The total amount of all orders placed against all contracts resulting from this solicitation shall not exceed $480,000,000.00 over a five and one half year period: two (2) year base period of performance, three (3) one-year option periods and one six (6) month option period.

c. The Government anticipates a single contract award resulting from this unrestricted competitive solicitation.

B.2 LABOR RATE TABLES

The labor rate tables represents the offeror’s proposed fully-burdened hourly labor rates for each skill classification for work performed at Government sites and work performed at Contractor sites. Labor rates proposed for the TOs cannot exceed these labor rates.

B.3 INCORPORATION OF PROPOSED PRICING

The offeror’s proposed pricing and labor rate tables submitted in response to Section J, Attachment 6 (TO 0001 (Firm-Fixed-Price (FFP)), Attachment 7 (Cost-Plus-Fixed-Fee (CPFF) Pricing Scenario Template), and Attachment 8 (FFP Pricing Scenario Template) shall be incorporated into the contract as attachments to Section B.

Pricing for all cost-reimbursement contract actions (including, but not limited, to proposal and billing of future Tos and modifications) shall be restricted to the labor categories and fully burdened labor rates established in Attachment 7, CPFF Pricing Scenario Template, found in Section J.

Pricing for all FFP contract actions (including, but not limited to, proposal and billing of future Tos and modifications) shall be restricted to the labor categories and fully burdened labor rates established in Attachment 8, FFP Pricing Scenario Template, found in Section J.

B.4 CONTRACT DATA REQIREMENTS LIST (CDRL’s)

Contractors are advised that data items listed on the DD Form 1423, CDRL's, shall not be separately priced.

Section C - Descriptions and Specifications

Performance Work Statement (PWS)

Joint Interoperability Test Command (JITC) – Test, Evaluation, and Certification (TEC) Services

Contract Number: Solicitation #: HC1028-16-R-0007 Task Order Number: (completed by the KO at time of TO award or N/A) Tracking Number: (completed by the KO when complete package is received) Follow-on to Previous Contract and Task Order Number:

DISA Test and Evaluation (T&E) Mission Support Services (MSS) contract:

1. Alion Science & Technology (Alion), HC1028-12-D-0023

2. TASC, Inc. (acquired by Engility), HC1028-12-D-0024

3. ManTech Telecommunications & Information Systems Corporation (ManTech), HC1028-12-D-0026

4. Lockheed Martin Global Training & Logistics, HC1028-12-D-0028

5. Oberon, HC1028-12-D-0029

6. AMERICAN SYSTEMS, HC1028-12-D-0027

7. Science Applications International Corporation (SAIC)*, HC1028-12-D-0025

* Effective 28 April 2013, AMERICAN SYSTEMS purchased the testing arm of SAIC, resulting in six prime contractors on the DISA T&E MSS contract.

1 Contracting Officer’s Representative (COR).

a. Primary COR.

Name: Information to be provided at time of contract award Organization:

Department of Defense Activity Address Code (DODAAC):

Address:

Phone Number:

Fax Number:

E-Mail Address:

b. Alternate COR.

Name: Information to be provided at time of contract award Organization:

DODAAC:

Address:

Phone Number:

Fax Number:

E-Mail Address:

2 Contract Title. Test, Evaluation, and Certification (TEC) Services.

3 Background.

This PWS addresses the T&E and T&E Enabling Service Requirements of the Defense Information Systems Agency (DISA), Joint Interoperability Test Command (JITC). JITC is the DoD's sole Joint Interoperability Certifier and only non-Service Operational Test Agency (OTA) for IT/NSS. JITC provides its T&E services to the DoD CC/S/A;

federal, state, and local Governmental agencies; United States (US) Coalition Partners; and commercial vendors and service providers. JITC provides these services to aid in mitigating the risks associated with designing, developing, acquiring, integrating, deploying, using, and sustaining new and enhanced IT/NSS capabilities. JITC’s T&E services help to ensure that joint warfighting IT capabilities are interoperable and support mission needs by:

a. Determining to what extent applicable requirements and standards have been satisfied

b. Identifying and helping to reduce technical and operational risks

c. Baselining and characterizing levels of performance

d. Documenting operational capabilities and limitations

e. Supporting acquisition, deployment, and operational use milestone decisions

General information concerning DISA’s and JITC’s missions, functions, organizational structures, operational capabilities, and service offerings are available through their respective public websites, as are regulations, directives, instructions, manuals, policies, and guidelines referenced in this PWS. The public website addresses are:

a. DISA: http://www.disa.mil/

b. JITC: http://jitc.fhu.disa.mil/

4 Objectives.

The objectives of the TEC Services contract are to support JITC in carrying out the unique T&E roles and responsibilities for which it is assigned within the DoD:

a. Joint Interoperability Certifier – JITC has sole responsibility for certifying joint and combined interoperability of all DoD IT and NSS. JITC performs critical interoperability and standards conformance T&E to reduce the risks faced by warfighters in the field. JITC often provides evaluation and certification services based in part or in whole on testing conducted by other organizations.

b. OTA – JITC is the OTA for DISA-managed programs. JITC also serves as the OTA for other DoD agencies such as the Defense Logistics Agency, the Defense Finance and Accounting Service (DFAS), the Defense Commissary Agency, the Defense Health Agency, the United States Special Operations Command, and the National Security Agency (NSA). As an OTA, JITC conducts Operational Test and Evaluation (OT&E) to assess system operational effectiveness, suitability, interoperability, and cybersecurity. JITC is the sole non-Service OTA recognized by the DoD Director, Operational Test and Evaluation (DOT&E).

c. Warfighter Support – JITC supports the CC/S/As during exercises and contingency operations, helping the warfighter with on-the-spot evaluations of problem areas and viable mission-oriented solutions to include providing 24x7-hotline support.

d. Major Range and Test Facility Base (MRTFB) – JITC is the only non-Service member of the 23-element MRTFB. As such, JITC is considered a national asset focused on testing IT/NSS. Under MRTFB guidelines, JITC must account for all direct costs at the job order level. MRTFB services provided include T&E capabilities, infrastructure, and resources to support DoD acquisition programs and other Governmental and commercial customers on a reimbursable basis.

e. Developmental Test and Evaluation (DT&E) – JITC is increasing its capabilities to provide a full range of DT&E services. Through these services, JITC will support DISA and other customers in their delivery of verified, reliable, fully functional, and secure technical solutions.

The TEC Services contract is intended to provide JITC with T&E support products and services that reflect a higher level of quality, innovation, and technical sophistication than JITC has required in the past. This greater emphasis on quality, innovation, and technical sophistication will raise the trust that customers have in JITC’s capabilities, with the end goal of increasing the demand for JITC’s value-added TEC services. To achieve this goal, JITC requires a contract that is supported by a highly qualified, technically competent, professionalized workforce possessing knowledge and experience in many diverse technical areas.

5 Scope.

The scope of the TEC Services requirements includes support for T&E planning, test conduct, data collection and analysis, and results reporting. JITC performs these T&E functions in the support of a wide range of critical http://www.disa.mil/ http://jitc.fhu.disa.mil/

Command, Control, Communications, Computers, Combat Systems, Intelligence, Surveillance and Reconnaissance (C5ISR), and defense business and health affairs systems. In addition, JITC evaluates emerging network, mobility, cloud, enterprise, cybersecurity, and other IT/NSS capabilities that include infrastructure, and environments such as the DoD Joint Information Environment. JITC’s TEC Services needs also include contractor-provided support for maintaining the underlying policies, processes, and capabilities that enable JITC to satisfy its customers’ T&E needs in a responsive and cost effective manner. In addressing these needs, JITC develops and applies proven methods, tools, instrumentation, automation, modeling, simulation, statistical and analytical methods, and other advanced technologies and techniques to ensure the T&E rigor needed to mitigate the risks and uncertainty its customers face in developing, fielding, operating, and sustaining IT/NSS capabilities.

JITC provides T&E support to numerous functional areas and technologies to include enterprise and cloud services;

tactical, strategic, and national intelligence systems; logistics, business, and medical applications; command and control; battlespace awareness; force application; intelligence; communications and collaboration; weapon system platforms; Unified Capabilities (UC); mobility; and cybersecurity.

In addition to this TEC Services contract, DISA is also awarding two large contracts which provide T&E services:

the System Engineering, Technology, and Innovation (SETI) contract and the Encore III contract. By design, neither the SETI nor Encore III contracts will provide DISA OT&E or DoD Joint Interoperability Certification services. Both types of these T&E services are the exclusive responsibility of JITC to perform, and in turn, the TEC Services contract to support. The TEC Services contract will complement and support the T&E services available through the SETI and Encore III contracts. All three contracts encourage the use of integrated testing, the reuse and sharing of T&E data, and the coordination of test events that enable the collection of data to efficiently address multiple T&E objectives.

Evolving DoD priorities and missions, coupled with continuous technological advancements, result in ever-evolving T&E requirements and challenges. Given this uncertainty and the rapid pace of change, it is important that the TEC Services contract provide the depth, breadth, and flexibility of support required by JITC to remain relevant and responsive in satisfying its customers’ individual and unique T&E needs throughout the life of the contract.

In 2009, DoD OTA Commanders signed a memorandum of agreement committing to the use of Design of Experiments (DOE) in OT&E. In 2010, the Director, Operational Test and Evaluation (DOT&E) issued a memorandum requiring the OTAs to use scientific and statistical methods in developing rigorous and defensible test plans and in thoroughly evaluating resulting test data. The Deputy Assistant Secretary of Defense, Developmental Test and Evaluation (DASD(DT&E)) in turn established the term “Scientific Test and Analysis Techniques (STAT)” to refer to the use of scientific and statistical methods for the purposes of developing efficient, rigorous test strategies that yield highly accurate and repeatable results. As an OTA and provider of both DT&E and OT&E services, JITC has embraced the use of these DOT&E and DASD(DT&E) endorsed approaches. Online sources for information pertaining to SBTD concepts and approaches include the DOT&E website, http://www.dote.osd.mil/;

the DASD(DT&E) sponsored STAT T&E Center of Excellence, www.afit.edu/STAT/; and the DOT&E sponsored Test Science Knowledge Center, www.testscience.org.

6 Performance Requirements.

To operate in an effective, efficient, and responsive manner, JITC requires a coordinated unity of effort across the full spectrum of T&E services it provides its customers. In satisfying JITC’s customers’ T&E needs, the scope of the TEC Services contract must address requirements in three distinct but interrelated performance areas:

a. T&E Requirements – Those service requirements enabling JITC to directly support its customers’ T&E needs.

b. T&E Enabling Capability Requirements – Those service requirements enabling JITC to maintain and enhance its core organizational T&E policies, processes, and capabilities.

c. Overarching Contract and TO Requirements – Those service requirements pertaining to the overall administration of the TEC Services contract and TOs.

http://www.dote.osd.mil/ http://www.afit.edu/STAT/ http://www.testscience.org/

To ensure JITC and its customers realize maximum benefit from the finite resources available to support T&E requirements, the Contractor must possess the knowledge, skills, and abilities to seamlessly and efficiently support all three performance areas. The specific form, format, and content for the deliverables associated with each of the three performance areas will be specified in individual TOs. The general requirements associated with the three performance areas are as follows.

6.1 T&E Performance Requirements.

The following seven performance requirement areas identify the services JITC requires to ensure “unity of effort” in meeting its customers’ TEC needs:

• Performance Area 1.1: Interoperability T&E

• Performance Area 1.2: Standards Compliance and Conformance T&E

• Performance Area 1.3: DT&E

• Performance Area 1.4: OT&E

• Performance Area 1.5: Cybersecurity T&E

• Performance Area 1.6: Enterprise Service T&E

• Performance Area 1.7: Other T&E and Subject Matter Expert Support

In support of specific TOs issued for these seven performance areas, the Contractor shall possess the technical competencies and mission expertise to provide full T&E lifecycle support to include T&E planning, execution, data collection, analysis, and reporting:

a. T&E Planning – As part of T&E planning, the Contractor shall review and comment on system/program requirements; recommend specific T&E scope and objectives; perform in-depth analyses and risk assessments to identify specific T&E requirements and data sources; review testing conducted by other elements outside of JITC to help determine what data can be re-used and identify gaps to help focus JITC efforts; establish effective strategies based on SBTD approaches and instrumented data collection methods to reliably achieve T&E objectives; develop supporting T&E documentation; and plan for and coordinate the activities needed to carry out these strategies and plans in a rigorous, disciplined, and cost efficient fashion.

b. T&E Execution and Data Collection – In performing T&E execution, the Contractor shall ensure that required data is collected according to approved T&E strategies and plans, including the effective implementation of manual and automated data collection processes, and the performance of those activities required to manage, validate, and reduce collected data. Throughout T&E execution, the Contractor shall also help to identify and mitigate any limitations with the potential to affect the accomplishment of T&E objectives.

c. T&E Analysis – The Contractor shall apply systematic and rigorous statistical methods and analytical tools to thoroughly evaluate collected data to support T&E objectives and to ensure that resulting conclusions are at the confidence levels targeted during the T&E planning phase. In analyzing the collected data, the Contractor shall focus on documenting the operational significance and mission impact of any capabilities and limitations associated with the System Under Test (SUT).

d. T&E Reporting – In performing T&E reporting functions, the Contractor shall apply appropriate methods to clearly, concisely, and objectively document T&E results and findings in a timely and responsive manner, delivering high quality test reports that comply with JITC-approved processes and satisfy end-customer information needs. Reporting of T&E results should emphasize the operational significance of test findings, and when possible, provide actionable value-added recommendations.

The Contractor shall have extensive knowledge of the functional and operational requirements, intended users, mission needs, and environmental conditions that the systems, technologies, and devices undergoing T&E are intended to support. The Contractor shall possess a thorough understanding of the Tactics, Techniques, and Procedures, Concepts of Operation, Standard Operating Procedures (SOPs), Military and Commercial Standards, and operational issues pertaining to each SUT. The Contractor shall apply this understanding to help specify and establish the operationally representative test environments and conditions needed to effectively determine to what extent each SUT satisfies operational mission needs. The Contractor shall apply techniques such as mission thread analysis and decomposition, message traffic generation, models and simulation, stress and load testing, and other techniques to provide operational realism and mission context in the conduct of testing.

In providing T&E support, the Contractor shall identify and apply innovative methods and technologies to improve the overall efficiency and cost effectiveness by which JITC satisfies its customers’ T&E needs. In support of specific T&E objectives, the Contractor shall consider all valid data sources, which include data collected by JITC or other Governmental and non-Governmental organizations through dedicated test events, technology demonstrations, exercises, actual system operations, or other relevant sources of information. To the extent possible, the Contractor is encouraged to propose and apply automated and instrumented capabilities that will both lead to affordable and accurate results and establish processes and capabilities that can be reused to satisfy similar T&E requirements in the future. To improve overall T&E efficiency, the Contractor should exploit all opportunities for integrated test events led by JITC or other testing organizations that allow multiple T&E objectives to be concurrently and cost effectively achieved.

To ensure accurate and defensible T&E outcomes, and to comply with DOT&E and DASD(DT&E) mandates for increased T&E rigor and efficiency, the Contractor shall adopt SBTD methods as part of their support of JITC’s TEC Services requirements. Such methods as regression analysis, DOE, Analysis of Variance, and other statistical approaches, combinatorial test techniques, and test automation and instrumentation should be considered as means to accurately and efficiently evaluate the performance characteristics of systems or products undergoing test, over the full range of required operating conditions. The Contractor shall help define formal analysis structures and evaluation frameworks as the basis for a program’s T&E, to ensure that T&E is performed in a thorough, systematic, and structured manner. In T&E planning, the Contractor shall make use of SBTD approaches to determine the relevant combination of factors to be systematically controlled, varied, or simply recorded to achieve efficient test designs. For each test design, the Contractor must justify test data requirements by citing statistical measures of merit (statistical models supported, measures of orthogonality, prediction variance, power, confidence, etc.). Each test design must also account for the statistical methods of analysis to be used to determine if correlation exists between changes in test conditions to measured variations in performance. In cases where non-statistical methods are used to support particular T&E objectives, the Contractor shall justify such test design choices and explain how sufficient statistical confidence will be achieved. To mitigate physical or resource constraints, the Contractor shall make use of computer models and simulations, virtualized test environments, and other innovative valid approaches for cost effectively satisfying particular T&E objectives, where appropriate. To provide accurate and well informed post-test analysis and reporting, the Contractor shall determine the statistical uncertainty and levels of confidence associated with test findings. The Contractor shall document SBTD activities, results, and findings in a manner that is clear, concise, and easily understood by JITC customers who may not possess an in-depth knowledge of technical, scientific, or statistical concepts.

The DoD Chief Information Officer (CIO), Joint Staff (JS), DOT&E, Defense, Deputy Assistant Secretary of Defense for Developmental Test & Evaluation (DDT&E), CC/S/A level Milestone Decision Authorities, Component Acquisition Executives, and PMs for DoD Major Automated Information Systems Acquisition Category (ACAT) and non-ACAT programs are all reliant on JITC and its T&E services to support various program oversight functions. JITC’s T&E results and findings serve as critical input to program acquisition and deployment decisions and to particular program review board processes such as Network Readiness Review Boards (NRRBs) and Acquisition Review Boards (ARBs). Because of the importance of the information JITC provides to support program oversight and decision processes, it is vital that the Contractor is thorough, objective, and fully responsive in providing the T&E services required in this PWS. It is equally vital that the products and deliverables the Contractor produces in providing these T&E services be of high quality, free of defects and inaccuracies, and fully compliant with all DoD, DISA, JITC, and other applicable policies, instructions, guidelines, and reporting and formatting requirements.

The TEC Service contract T&E requirements are divided into seven performance areas. Many JITC customers require T&E support in more than one of these performance areas. In these cases, the Contractor shall satisfy the requirements in the most efficient and cost effective manner possible, using methods such as combined or integrated test events, where appropriate. The seven T&E requirement performance areas follow.

6.1.1 Performance Area 1.1: Interoperability T&E.

6.1.1.1 Interoperability Certification.

The Contractor shall support JITC in T&E planning, execution, data collection and analysis, and reporting to support the objectives of Joint Interoperability Certification and assessment as outlined in DoD Instruction (DoDI) 8330.01, “Interoperability of Information Technology (IT), Including National Security Systems (NSS),” the JITC Interoperability Process Guide (IPG), the JITC Net Ready Key Performance Parameter (NR KPP) Evaluation Guidebook, and JITC Instruction 380-50-02, “JITC Interoperability and Standards Conformance Test and Evaluation (T&E) Certification Instruction.” Additional guidance, instructions, and policies pertaining to DoD interoperability roles, responsibilities, and requirements are documented in DoDIs, Chairman, Joint Chief of Staff Instructions (CJCSIs), and DoD guides and manuals to include: U.S. Code (USC) Title 10-Armed Forces, Sections 2222 and 2223; DoDI 5000.02; CJCSI 5123.01G; CJCSI 3170.01I; and the Joint Capabilities Integration and Development System (JCIDS) Manual, Content Guide for the Net-Ready KPP. These reference materials are available through the TEC Services RFP Bidders Library at: https://www.fbo.gov/spg/DISA/D4AD/DITCO/HC1028- 16-R-0007/listing.html. Additional information concerning interoperability certification can be found at JITC’s public website: http://jitc.fhu.disa.mil

In cases where JITC will rely on testing conducted by other organizations to make a certification determination, the Contractor shall collaborate with other organizations to ensure appropriate data is collected and analyzed. In cases where JITC’s customers may not require or be ready for Joint Interoperability Certification, JITC may perform an interoperability assessment or component-level interoperability certification instead.

In support of JITC customers’ interoperability certification or assessment needs, the Contractor shall perform the following activities per the processes outlined in the JITC IPG:

a. Assist programs with NR KPP, JCIDS, and Business Capability Lifecycle Defense Business Systems product reviews and development to ensure valid and testable requirements.

b. Participate in program T&E working groups and Working Integrated Process Teams (WIPTs) to coordinate test data requirements and interoperability T&E activities with PMs, users, OTAs and other test agencies.

c. Provide interoperability T&E input to program Test and Evaluation Master Plans (TEMPs).

d. Assess the levels of risk associated with a program’s specific interoperability-related requirements (e.g., interfaces, information exchanges, and standards).

e. Review and provide feedback on DoD Architecture Framework (DoDAF) viewpoints.

f. Develop accurate Integrated Architecture Traceability Matrices (IATMs) that account for any gaps or omissions within DoDAF viewpoints.

g. Design T&E strategies to enable program Joint Interoperability Certification.

h. Apply SBTD methods, automation, and instrumentation to improve Joint Interoperability Certification rigor, accuracy, repeatability, and cost effectiveness.

i. Create Data Management and Analysis Plans (DMAPs) to manage data, products, and programs artifacts supporting Joint Interoperability Certification.

j. Prepare Joint Interoperability Evaluation Plans (JIEPs) based on IATMs, T&E strategies, evaluation methodologies, and DMAPs.

k. Provide Interoperability Statements as input to Operational Test Readiness Reviews (OTRRs).

l. Collect and analyze interoperability certification data per approved JIEPs.

m. Report interoperability T&E findings.

n. Prepare certification products.

o. Provide program specific interoperability and T&E status, results, findings, and recommendations in support of NRRBs, ARBs, and other program review boards and oversight functions.

Deliverables (as specified in individual TOs may include, but are not limited to):

a. Comment Resolution Matrices (CRMs)

b. Test Concept Briefs (TCBs) https://www.fbo.gov/spg/DISA/D4AD/DITCO/HC1028-16-R-0007/listing.html https://www.fbo.gov/spg/DISA/D4AD/DITCO/HC1028-16-R-0007/listing.html

c. IATMs

d. DMAPs

e. JIEPs

f. OTRR Interoperability Statement

g. Quick Look Reports

h. Test Results Briefings

i. Joint Interoperability Assessment/Certification Memorandums

j. Joint Interoperability Assessment/Certification Reports

6.1.1.2 Other Interoperability T&E.

The Contractor shall support JITC in T&E planning, execution, data collection and analysis, and reporting of interoperability T&E objectives other than those specified by DoDI 8330.01. An example of such support includes Mission Partner Environment testing performed in accordance with CJCSI 5128.01.

a. IATMs

b. TCBs

c. Interoperability Evaluation Plans

d. T&E Readiness Reviews and Briefings

e. DMAPs

f. Quick Look Reports

g. Test Results Briefings

h. Interoperability Assessment Reports

6.1.2 Performance Area 1.2: Standards Compliance and Conformance T&E.

6.1.2.1 Standards Conformance and Compliance Verification.

The Contractor shall support JITC in planning, execution, data collection and analysis, and reporting to determine system, product, or device compliance and conformance with particular standards and formal specifications. JITC provides standards compliance/conformance verification in support of a broad range of different types of standards and regulations to include, but not limited to: UC, North Atlantic Treaty Organization Standardization Agreements, tactical data links, electronic messaging formats, imagery, communication waveforms, finance and accounting, health and medical affairs, records management, and other standards conformance and compliance areas as listed in section 6.3.4.s of this PWS, “Technical Skills, Training, Credentials, and Experience.” In providing standards conformance/compliance support, the Contractor shall:

a. Develop standards compliance and conformance concepts, methods, and detailed test plans and procedures as outlined in JITC Instruction 380-50-02, Chapter 8.

b. Apply SBTD methods, automation, and instrumentation to improve standards compliance/conformance rigor, accuracy, repeatability, and cost effectiveness.

c. Conduct standards compliance/conformance assessment and verification activities per approved plans.

d. Prepare standards compliance/conformance assessment and certification reports.

Deliverables (as specified in individual TOs may include, but are not limited to):

a. Compliance/Conformance Plans and Procedures

b. Standards Conformance/Compliance Assessment/Certification Reports

6.1.2.2 Standards Compliance Program Administration.

The Contractor shall support JITC in performing research and administrative functions to maintain and improve existing conformance/compliance certification policies, procedures, and capabilities. The Contractor shall also perform research and administrative functions to support JITC in establishing policies, procedures, and capabilities required to address the evaluation and conformance/compliance certification of new standards. The Contractor shall maintain standards conformance reference profiles and Approved Product List (APL) information for particular standards compliance programs under JITC’s oversight. Within the scope of this performance area, the Contractor shall also provide technical support to JITC as SMEs for particular standard programs and provide input to the JITC Risk Assessment Database and other standards compliance support capabilities.

a. TEC instructions and procedures

b. Standards conformance reference documentation

c. APL updates

6.1.3 Performance Area 1.3: DT&E.

The Contractor shall support JITC in planning, conducting, and reporting on DT&E performed in accordance with (IAW) USC Title 10, Section 139b requirements. The Contractor shall verify the design, development, integration, and technical characteristics of new or enhanced IT/NSS capabilities prior to their acceptance and consideration for operational use. These DT&E services will be provided in support of capabilities being delivered by developers using both conventional development methods, as well as advanced methods such as Agile/SCRUM and Test Driven Design approaches. The Contractor shall apply SBTD methods such as combinatorial test methods, instrumentation and automation, and other advanced approaches, where appropriate, to improve the overall efficiency, effectiveness, rigor, and timeliness of JITC’s DT&E support. The types of systems for which the Contractor may be requested to provide DT&E support include DISA programs pertaining to enterprise services, cloud computing, mobility, UC, and other emerging technologies being adopted by DoD. As part of this support, the Contractor shall participate in WIPTs to facilitate program T&E planning and provide DT&E input to program TEMPs. The DT&E support the Contractor shall provide includes, but is not limited to:

6.1.3.1 Unit, Module, Device, Subsystem, and System Testing.

The Contractor shall perform T&E planning, execution, data collection and analysis, and reporting to verify that software and hardware modules, devices, subsystems, and systems undergoing development satisfy their stated requirements for functionality, interoperability, standards conformance, performance, capacity, scalability, reliability, availability, maintainability, cybersecurity, and other required technical characteristics as specified in applicable requirements sources and design specifications. The Contractor shall perform risk assessments to help focus T&E resources on areas of highest technical risk. The Contractor shall conduct regression testing to verify that previously evaluated system characteristics are not adversely affected by subsequent configuration changes.

a. DT&E strategies and methodologies

b. Risk Assessment plans and reports

c. Requirements Traceability Matrices (RTMs)

d. Use cases, test cases, detailed test procedures, and test scripts

e. Test Plans

f. Product Deficiency Reports

g. Test Reports

h. TEMP input

6.1.3.2 Integration and Qualification Testing.

The Contractor shall perform T&E planning, execution, data collection and analysis, and reporting to verify that software and hardware modules, devices, subsystems, and systems perform as intended when integrated together in an approved system design or architecture. The Contractor shall perform risk assessments to help focus T&E resources on areas of highest technical and operational risk. The Contractor shall perform T&E to determine the degree to which the integrated system satisfies applicable design requirements for functionality, interoperability, standards conformance, performance, capacity, scalability, reliability, availability, maintainability, cybersecurity, and other required technical characteristics as specified in applicable requirements sources and design specifications.

The Contractor shall conduct regression testing to verify that previously evaluated system characteristics are not adversely affected by subsequent configuration changes.

a. DT&E strategies and methodologies

b. Risk Assessment plans and reports

c. RTMs

d. Use cases, test cases, detailed test procedures, and test scripts

e. TCBs

f. Test Plans

g. Product Deficiency Reports

h. Test Reports

i. TEMP input

6.1.3.3 Acceptance Testing.

The Contractor shall perform T&E planning, execution, data collection and analysis, and reporting to verify that the developer has satisfied applicable functional requirements. Such testing may take the form of a software/system acceptance test, government acceptance test, or user acceptance test; normally involves test participation by representatives of the system’s intended user community; and precedes any Government commitment to limited deployment and formal OT&E. The Contractor shall conduct regression testing to verify that previously evaluated system characteristics are not adversely affected by subsequent configuration changes.

a. RTMs

b. TCBs

c. Use cases, test cases, detailed test procedures, and test scripts

d. Test Plans

e. Product Deficiency Reports

f. Test Reports

g. TEMP input

6.1.4 Performance Area 1.4: OT&E.

The Contractor shall support JITC in planning, conducting, and reporting on OT&E activities to assess the operational capabilities and limitations of IT/NSS and to determine the operational effectiveness, suitability, interoperability, and cybersecurity of such systems under realistic conditions and representative mission support scenarios. The Contractor shall provide such support IAW USC Title 10, Section 139 requirements. In providing such support, the Contractor shall apply SBTD approaches, and T&E automation, instrumentation, simulation, and stimulation methods, where appropriate, as means to efficiently collect quantifiable, operationally representative, and highly accurate test data to provide increased confidence and mission relevance to OT&E results and findings.

The Contractor shall provide support to the following forms of OT&E:

6.1.4.1 Traditional OT&E.

The Contractor shall perform T&E planning, execution, data collection and analysis, and reporting to support the objectives of assessing or determining operational effectiveness, suitability, interoperability, and cybersecurity.

JITC conducts such testing in support of major program acquisition decisions as outlined in DoDI 5000.02, “Operation of the Defense Acquisition System.” OT&E activities may take the form of either Operational Assessments (OAs), to include Field Assessments (FAs) and Limited User Tests (LUTs), or Operational Tests

(OTs), to include Initial Operational Test and Evaluation or Follow-On Operational Test and Evaluation. Whether an OA or an OT, OT&E necessitates evaluation based on typical trained users and maintainers exercising an operationally representative system configuration, operated under realistic conditions, to determine to what extent the SUT satisfies operational mission needs. In providing support to OT&E activities, the Contractor shall follow the processes set forth in the JITC OT&E Guidebook, accessible via the JITC public website at http://jitc.fhu.disa.mil/. In providing OT&E support, the Contractor shall also be responsive to the latest guidance from DOT&E, accessible at http://www.dote.osd.mil/guidance.html. The Contractor shall provide support to OT&E activities that include, but are not limited to:

a. Identify operational requirements through review of program documentation such as JCIDS documents and other requirements sources.

b. Perform mission analysis through review of applicable Joint Capability Areas (JCAs), Universal Joint

Task Lists, Mission Essential Task Lists, Mission Essential Tasks, mission threads, use cases, etc.

c. Conduct risk assessments to determine appropriate levels of OT&E.

d. Develop OT&E concepts and strategies.

e. Document OT&E strategies within TCBs and TEMPs.

f. Participate in T&E and User Working Groups and WIPTs with stakeholders (PMs, DOT&E representatives, user representatives, T&E participants, etc.) to plan OT&E activities and resource requirements.

g. Develop OT&E Analysis Structures.

h. Capitalize on existing relevant test or operational data and opportunities for integrated and combined testing where DT&E, Interoperability, Cybersecurity, or other relevant T&E events that can be leveraged to support OT&E objectives, thus potentially reducing the scope, duration, expense, and delay associated with dedicated phases of OT&E.

i. Construct user surveys, questionnaires, interview forms, and other qualitative and quantitative data collection instruments.

j. Employ SBTD methods, automation, and instrumentation as means to carry out OT&E objectives in an efficient and rigorous manner.

k. Develop detailed OT&E plans.

l. Participate in OTRRs.

m. Conduct pilot tests, not for the OT&E of the SUT, but to verify and ‘dry run’ test and data collection procedures, etc., prior to the start of formal phases of testing.

n. Carry out OT&E execution and data collection in accordance with approved test plans.

o. Supply input to ‘hot washes,’ test coordination meetings, and test status reporting throughout the conduct of OT&E events.

p. Assist in Data Authentication Group (DAG) functions performed to authenticate and validate test data.

q. Analyze collected data to arrive at an operational effectiveness, suitability, interoperability, and cybersecurity determination.

r. Prepare final reports and briefings to convey OT&E results and findings.

Deliverables (as specified in individual TOs may include, but are not limited to):

a. TCBs

b. TEMPs

c. Analysis Structures

d. Risk Assessments Plans and Reports

e. OT&E Plans

f. OTRR Reporting

g. Quick Look Reports

h. OT&E Reports

i. OT&E Results Briefings

j. DAG Reports

6.1.4.2 Capability Assessments.

http://www.dote.osd.mil/guidance.html

The Contractor shall perform T&E planning, execution, data collection and analysis, and reporting in support of the conduct of capability assessments. As part of this support, the Contractor shall apply streamlined OT&E techniques to provide an operational perspective to system evaluations performed without the associated structure, rigor, or formality normally associated with traditional OT&E. The basis for a capability assessment may include the application of the JITC Mission Thread Analytic Framework Assessment Handbook approach. Examples of capability assessment activities include observational studies performed as part of normal system operations;

abbreviated field assessments to identify operational capabilities and limitations; limited assessments conducted as part of military exercises, proofs of concept, and technology demonstrations; and assessments which reach or infer operational conclusions from DT&E or actual system usage data (e.g., component reliability effects on overall system reliability).

a. TCBs

b. Assessment Plans

c. Quick Look Reports

d. Assessment Reports

e. Results Briefings

6.1.5 Performance Area 1.5: Cybersecurity T&E.

The Contractor shall support JITC in planning, conducting, and reporting on cybersecurity T&E. Cybersecurity T&E shall be performed to verify that requirements for information confidentiality, integrity, and availability are adequately addressed, and to assess system capabilities enabling data protection, detection of unauthorized activities, reaction to compromises, system restoration, and continuous monitoring of threats. JITC conducts cybersecurity T&E across the full system acquisition lifecycle, supporting both Defensive Cyber Operations (DCO) and DoD mission systems. Cybersecurity testing may be conducted as part of DT&E, OT&E, Enterprise Service T&E, interoperability T&E, Combatant Command exercises, and Commercial Product evaluations. In enabling JITC to meet its customers’ T&E cybersecurity needs, the Contractor shall be knowledgeable and experienced with the cybersecurity laws, policies, regulations, and instructions that direct DoD system acquisition and DCO as outlined in the DoD Chief Information Officer (DoD CIO) “Build and Operate a Trusted DoD Information Network (DoDIN)” chart, available at http://iac.dtic.mil/csiac/ia_policychart.html. The Contractor shall consider the use of SBTD approaches, in addition to T&E automation, instrumentation, simulation, and stimulation methods, as means to efficiently collect quantifiable, operationally representative, and highly accurate test data to support cybersecurity T&E objectives. In support of JITC customers’ cybersecurity T&E needs, the Contractor shall provide the following services:

6.1.5.1 Cybersecurity T&E Methodologies.

The Contractor shall assist JITC in the development, maintenance, and enhancement of standardized cybersecurity T&E methods, test plans, and report templates for use by JITC and other DoD test activities. These methods support cybersecurity assessment across the entire system lifecycle from system and product concept development, design and integration, commercial product evaluation, acceptance, fielding, to operational use and sustainment. Such methods may address a wide range of technologies and system types to include cloud computing infrastructure, mobility, UC, C5ISR architectures, and DoD health and financial systems. The Contractor shall research, identify, acquire, and maintain instrumentation, automation, and SBTD approaches to improve the overall efficiency, effectiveness, rigor, and responsiveness of capabilities supporting cybersecurity T&E. The Contractor shall identify, and as appropriate provide the means to simulate particular high risk threats and vulnerability exploitation methods that must be accounted for to ensure thorough and complete cybersecurity assessment.

a. Cybersecurity T&E concepts and methodologies

b. Cybersecurity T&E Plans

c. Research white papers http://iac.dtic.mil/csiac/ia_policychart.html

d. Instrumentation/Automation Implementation Plans

6.1.5.2 Security T&E (ST&E), Cybersecurity T&E (CT&E), and Cybersecurity Certification and Accreditation (C&A).

The Contractor shall perform ST&E planning, execution, data collection and analysis, and reporting in support of Information Assurance (IA) and cybersecurity C&A objectives that include, but are not limited to, the conduct of the following activities:

a. Assess program Risk Management Framework (RMF) compliance.

b. Conduct vulnerability assessments.

c. Ensure SUT configurations are per approved cybersecurity configurations.

d. Provide cybersecurity assessment support to military exercises.

Deliverables (as specified in individual TOs may include, but are not limited to):

a. ST&E/CT&E Plans

b. ST&E/CT&E Reports

c.…