Draft_PWS_FA704615R0001-0002.docx

PERFORMANCE WORK STATEMENT

FOR

F-35 Operational Test and Evaluation – OT&E Verification Simulation (VSim) Verification and Validation (V&V) 1 June 2015

1. SCOPE. This Performance Work Statement addresses technical support required by the Joint Strike Fighter (JSF) Operational Test Team (JOTT) for use of the Lockheed Martin (LM) F-35 Verification Simulation (VSim) for Operational Test and Evaluation (OT&E). The contractor shall support the JOTT in establishing and providing verification and validation (V&V) expectations to the JSF Program Office (JPO). The JOTT requires highly experienced technical expertise to assess the V&V plans developed by LM and the JPO and to make recommendations to those plans. The recommendations will be based on a technically, credible, physics-based analysis to identify limitations of effects-based models in VSim. The analysis will identify high-risk areas and methodologies to support validation. The contractor will perform engagement-level modeling or detailed physics based analysis (if engagement-level modeling does not exist) and review V&V and accreditation products to identify the operational impacts of limitations in VSim capabilities.

1.1. Background.

1.1.1. The VSim is a man-in-the-loop federation that includes federates such as the F35, Blue aircraft, Red aircraft, Red SAMs, Red integrated air defense system (IADS), air-to-surface weapons, GPS, GPS jamming, data links, radar frequency (RF) and infrared (IR) sensors, air-to-air weapons, terrain, surface targets, countermeasures, obscurants, clutter, multipath, synthetic aperture radar (SAR) mapping, and infrared and radio frequency emissions and countermeasures. In order to run in real-time, VSim incorporates many effects-based models.

1.1.2. The VSim is being developed to support F-35 contract compliance, principally for mission effectiveness and integrated system performance. The JOTT, a multi-service and multi-national team, will use the VSim to support OT&E mission-level evaluations in the context of a Defense Planning Scenario at F-35 initial operational capability (IOC) to IOC+10 years. Pre-Initial Operational Test and Evaluation (IOT&E) activities are planned for July 2015 - June 2016.

1.1.3. LM has responsibility for planning and completing V&V of LM (and subcontractor supplied) models integrated into VSim. The JPO provides V&V for all Government furnished models. The JSF Executive Committee (EXCOM) is the accreditation authority for OT&E use of VSim. The JOTT is the accreditation agent. V&V accomplished by LM and the JPO will provide the evidence to support the accreditation decision by EXCOM.

1.2. Issues. The JOTT completed a review of a draft V&V plan prepared by LM and found that the V&V plan was not technically adequate to support accreditation of VSim for its intended use during OT&E. To resolve VSim V&V issues, the JOTT will provide specific recommendations to restructure the VSim V&V plan, and in a separate effort will develop VSim acceptability criteria for OT&E, and write the OT&E accreditation plan, which will identify limitations on the credible use of VSim prior to and during OT&E. The contractor’s role will include scientists, engineers, and other qualified personnel experienced in mission systems requirements reviewing V&V testing, analysis, and products to determine impacts to the credible use of VSim for OT&E. The JOTT will provide the JPO with findings and recommendations.

1.3. Period of performance. The period of performance is for 60 months. The time period includes pre-IOT&E activities, Operational Test and Evaluation (OT&E), and associated VSim verification, validation and accreditation.

1.4. Locations. Lockheed Martin is developing and integrating VSim at two locations: 1) Lockheed Martin Marietta and 2) Lockheed Martin Fort Worth. OT&E will be conducted at the Marietta location. The primary work location will be the JOTT, located at Edwards AFB, CA. Other work locations will be approved by the Combined Test Director on a case-by-case basis.

2. Applicable Documents. The documents listed below include the JOTT review of the LM V&V plan and recommended practices for VV&A.

2.1. JOTT Memorandum. “JOTT Initial Feedback Regarding Adequacy of the F-35 VSim Verification and Validation Plan,” 28 May 2013

2.2. Department of Defense standards. MIL-STD-3022 w/CHANGE 1, “DoD Standard Practice Documentation of Verification, Validation, and Accreditation (VV&A) for Models and Simulations,” 5 April 2012

2.3. Other Government documents.

2.3.1. DoD Modeling and Simulation Coordination Office (M&SCO), "Verification, Validation, & Accreditation (VV&A) Recommended Practices Guide (RPG)," 31 January 2011 (Current Release), http://msco.mil/VVA_RPG.html

2.3.2. Office of the Director, Defense Research & Engineering (ODDR&E), NSAD-R-2010-037,“Best Practices for the Development of Models and Simulations,” June 2010

2.4. Industry documents (reference DoD M&SCO VV&A RPG).

2.4.1. IEEE Std 1278.4-1997(2010),”Recommended Practice for Distributed Interactive Simulation – Verification, Validation, and Accreditation,” 1 July 2010

2.4.2. IEEE Std 1516.4-2007, “Recommended Practice for Verification, Validation, and Accreditation of a Federation – An overlay to the High Level Architecture Federation Development and Execution Process,” 20 December 2007

2.4.3. IEEE Std 1516.3-2003, “Recommended Practice for High Level Architecture (HLA) Federation Development and Execution Process (FEDEP),” 23 April 2003 (Superseded by IEEE Std 1730-2010)

2.4.4. IEEE Std 1730-2010,”Recommended Practice for Distributed Simulation Engineering and Execution Process (DSEEP),” 24 January 2011

2.5. Documents not readily available except through Government channels will be provided upon request.

3. REQUIREMENTS.

3.1. Provide a management structure to ensure efficient management of this effort in compliance with Section 3.1 through 3.8 of the F-35 Operational Test and Evaluation, OT&E VSim Verification and Validation Performance Work Statement (PWS). (A001, A002, A003)

3.2. Location. The contractor shall perform work at Government provided facilities and allow regular visits (as-needed-basis) to Lockheed Martin Marietta, Lockheed Martin Fort Worth, and Naval Air Systems Command (NAVAIR) at Naval Air Station Patuxent River.

3.3. Preparation.

3.3.1. Security Clearances. (ref: para 6.1)

3.3.1.1. The contractor personnel shall have a current Top Secret clearance within 5-years and be able to obtain special program accesses and sensitive compartmented information (SCI). Note: SCI is required to work with the Intelligence Centers.

3.3.1.2. Requests for clearances shall be initiated within 30 days of the start of work. Individuals should be able, to the maximum extent possible, to meet background requirements in order to minimize time to be granted clearances or accesses.

3.3.2. Technical Assistance Agreement (TAA). (A001)

3.3.2.1. The contractor shall obtain a TAA for foreign national mission partners under the International Traffic in Arms Regulations (ITAR) in order to release technical and approved classified information. Note: Minimum processing time by the US Department of State is 60 days. (A001)

3.3.2.2. Application for a TAA shall be submitted within 45 days of the start of work. (A001)

3.3.3. Planning. Within 30 days of the start of work, the contractor shall outline a broad plan (including a schedule of activities and deliverables), provide recommendations on work priority to the JOTT, and identify additional requirements to proceed. Work will be reviewed and prioritized with the JOTT at regularly scheduled meetings (telecon, VTC, or travel), and delivery dates adjusted, as required, by the Government. (A001) (A004)

3.4. VSim Acceptability Criteria. The contractor shall provide recommendations and sensitivity analysis to support federation-level, acceptability criteria for OT&E. (A004) (A021) (A020).

3.5. VSim Conceptual Model Validation.

3.5.1. The contractor shall conduct a conceptual model validation of VSim to identify inherent capabilities and limitations of the VSim federation architecture and modeling approach. (A004) (A021)

3.5.2. The contractor shall assess the impact of VSim capabilities and limitations on use of VSim for OT&E. The goal is to identify conditions under which VSim produces credible results and to identify conditions and/or operations that should be avoided in VSim. Use cases shall be considered to address test conditions from the OT&E test design for VSim. (A004) (A021)

3.6. Review Lockheed-Martin and JSF Program Office VSim V&V plans.

3.6.1. The contractor shall provide detailed assessments on the adequacy of LM V&V plans and reports to support accreditation of the VSim for its intended use during OT&E. (A004) (A021) (A020)

3.6.2. The contractor shall provide specific recommendations to restructure V&V plans and V&V methodologies to support OT&E. (A004) (A021) (A020)

3.6.3. The contractor shall review the V&V packages provided by the Department of Defense Intelligence Centers and the wrappers developed by LM in order to assess the limitations of threat models in VSim. (A004) (A021) (A020)

3.6.4. The contractor shall recommend V&V methods and techniques for individual federates, grouping of federates, sub-federations, and the entire VSim federation. (A004) (A021) (A020)

3.6.5. The contractor shall analyze VSim models as implemented within the VSim architecture to ensure it replicates aircraft installed system level performance, and shall address the credibility of federates, grouping of federates, sub-federations, and the overall VSim federation. The review shall include: F35, Blue aircraft, Red aircraft, Red SAMs, Red integrated air defense system (IADS), air-to-surface weapons, GPS, GPS jamming, data links, radar frequency (RF) and infrared (IR) sensors, air-to-air weapons, terrain, surface targets, countermeasures, obscurants, clutter, multipath, synthetic aperture radar (SAR) mapping, and infrared and radio frequency emissions and countermeasures. (A004) (A021) (A020)

3.6.6. The contractor shall evaluate the capabilities and limitations to include recorded data of VSim federates, and V&V methods, to include the following (A004) (A021) (A020):

3.6.6.1. Electro-optic/infrared (EO/IR) and radio frequency (RF) propagation effects

3.6.6.2. Physics of EO/IR/RF signatures

3.6.6.3. EO/IR/RF sensors

3.6.6.4. System integration, avionics, networking

3.6.6.5. Electronic support, electronic attack, electronic protection

3.6.6.6. Mission planning modeling and interfaces

3.6.6.7. Sensor fusion

3.6.6.8. Multispectral data base, terrain modeling, atmospherics, obscurants

3.6.6.9. Camouflage concealment and deception

3.6.6.10. Multipath

3.6.6.11. Integration into a distributed interactive simulation

3.6.6.12. Design of experiments to structure V&V

3.6.6.13. V&V procedures and techniques

3.6.6.14. Sensitivity analysis to evaluate impacts affecting credible use for OT&E

3.7. VSim Trials. The contractor shall analyze and evaluate the impact of VSim limitations on results of the OT&E VSim trial conducted prior to and during OT&E. (A021) (A020)

4. SERVICE SUMMARY ITEMS

5. GENERAL INFORMATION

5.1 Government Furnished Property-Base Support. The Government will furnish or make available applicable working space, equipment, databases and network access to F-35 contractor personnel at their work location, whether it is the JOTT or another work location approved by the Combined Test Director on a case-by-case basis. All materials provided by the Government will remain the property of the Government and will be returned to the AFOTEC Det 1 Contracting Officer Representative (COR) upon request or at the end of the task order period of performance.

5.2 Travel Requirements. Contractor employees will be required to travel to various locations within the continental United States (CONUS) for performance of this task order. The Travel Ceiling for this task order is not-to-exceed (NTE) amount as specified in the contract.

5.3 Transition. Not used.

5.4 Associate Contractor Agreements. The contractor shall enter into Associate Contractor Agreements (ACA) for any portion of the contract requiring joint participation in the accomplishment of the Government’s requirement. Agreements shall include the basis for sharing information, data, technical knowledge, expertise, and/or resources essential to the integration of the F-35 OT&E which shall ensure the greatest degree of cooperation for the development of the program to meet the terms of the contract. Associate Contractors are listed below.

5.4.1 ACAs shall include the following general information:

5.4.1.1 Identify the associate contractors and their relationships.

5.4.1.2 Identify the program involved and the relevant Government contracts of the associate contractors.

5.4.1.3 Describe the associate contractor interfaces by general subject matter.

5.4.1.4 Specify the categories of information to be exchanged or support to be provided.

5.4.1.5 Include the expiration date (or event) of the ACA.

5.4.1.6 Identify potential conflicts between relevant Government contracts and the ACA; include agreements on protection of proprietary data and restrictions on employees.

5.4.2 A copy of such agreement shall be provided to the Contracting Officer for review before execution of the document by the cooperating contractors.

5.4.3 Nothing in the foregoing shall affect compliance with the requirements of the clause at 5352.209-9000, Organizational Conflict of Interest (OCI).

5.4.4 The contractor is not relieved of any contract requirements or entitled to any adjustments to the contract terms because of a failure to resolve a disagreement with an associate contractor.

5.4.5 Liability for the improper disclosure of any proprietary data contained in or referenced by any agreement shall rest with the parties to the agreement, and not the Government.

5.4.6 The following contractors are associate contractors with whom agreements are required:

Lockheed Martin Contract Number: N00019-02-C-3002

5.5 CONTRACT DATA REQUIREMENTS LIST (DD 1423s).

5.5.1 Data Items and Deliverables. With the exception of contractor management documents (e.g. Monthly Status Reports), and the identified CDRLs, all documents and reports generated by the F-35 joint and combined test team are to be U.S. Government reports due to the nature of ITAR requirements. Contractor format is acceptable except where AFOTEC OT&E guide or policy provides specific templates. The contractor’s participation must comply with required program protection criteria, joint and combined requirements, and AFOTEC standards, unless given other direction (e.g., document not required to meet current standards due to grandfathering). The history of this program has shown that many of the program’s documents will require multiple versions due to program protection criteria, and require extensive additions and updates after initial development. The contractor shall provide inputs to these revisions or updates, as required. All documents shall be marked with the following statement: Export Control Warning. “WARNING- This document contains technical data whose export is restricted by the Arms Export Control Act (Title 22, U.S.C., Sec 2751, et. seq.) or the Export Administration Act of 1979, as amended, Title 50, U.S.C., App. 2401 et. seq. Violations of these export laws are subject to severe criminal penalties. Disseminate in accordance with provisions of DoD Directive 5230.25. All reports shall be delivered to:

AFOTEC Det 1 Attn: Contracting Officer Representative 187 N. Flightline Road Edwards AFB, CA 93524 The contractor shall provide or participate in:

5.5.2 Management Plan (A001) Reference paragraph 3.3.3. Management plan shall be presented in contractor format at the time of proposal submittal and updated within 30 days of the start of work and will be updated monthly to reflect any changes made during regularly scheduled meetings with the JOTT.

5.5.3 Funds & Manpower Expenditure (A002) and Status Reports (A003) – Reference paragraph 3.1. Immediately highlight in writing to the JOTT COR areas of concern.

5.5.4 Management Plan (A001) - Reference paragraph 3.3.2, 3.3.2.1, 3.3.2.2. The contractor shall deliver a copy of the TAA(s) to the AFOTEC Det 1 Foreign Disclosure Officer (FDO) within 60 days of the task order award date.

5.5.5 Conference Minutes (A004) - Reference paragraph 3.3.3, 3.4, 3.5.1, 3.5.2, 3.6.1, 3.6.2, 3.6.3, 3.6.4, 3.6.5, 3.6.6. Document all meetings, conferences, etc. in trip report format.

5.5.6 Scientific and Technical Report – Generic (A021), Presentation Material (A020) - Reference paragraph 3.4. Contractor shall provide recommendations and results of sensitivity analysis on acceptability criteria 7 days prior to report due date. Contractor format is acceptable. Report and presentation are due on the dates established by the Government. Updates will be provided quarterly or as required.

5.5.7 Scientific and Technical Report (A021), Presentation Material (A020) - Reference paragraph 3.5.1, and 3.5.2. Contractor shall document the VSim conceptual model validation. Report and presentation are due on the dates established by the Government. Updates will be provided quarterly or as required.

5.5.8 Scientific and Technical Report – Generic (A021), Presentation Material (A020) - Reference paragraph 3.6.1, 3.6.2. Contractor shall provide detailed assessments of plans and provide recommendations to restructure plans and methodologies. Contractor format is acceptable. Report and presentation are due on the dates established by the Government. Updates to report and presentation will be provided quarterly or as required.

5.5.9 Scientific and Technical Report – Generic (A021), Presentation Material (A020) - Reference paragraph 3.6.3. Contractor shall provide detailed assessments of intelligence center V&V reports and associated wrapper developed by LM. Contractor format is acceptable. Report and presentation are due on the dates established by the Government. Updates to report and presentation will be provided quarterly or as required.

5.5.10 Scientific and Technical Report – Generic (A021), Presentation Material (A020) - Reference paragraph 3.6.4. Contractor shall provide detailed assessments and recommendations for V&V methods and techniques. Report and presentation are due on dates established by the Government. Updates shall be provided on a quarterly basis or as required. Contractor format is acceptable.

5.5.11 Scientific and Technical Report – Generic (A021), Presentation Material (A020) - Reference paragraph 3.6.5., 3.6.6. Contractor shall provide detailed analysis, evaluation, and recommendations. Report and presentation are due on the dates established by the Government. Updates shall be provided on a quarterly basis or as required. Contractor format is acceptable.

5.5.12 Scientific and Technical Report – Generic (A021), Presentation Material (A020) - Reference paragraph 3.7. Contractor shall provide input into the development and rationale for the JOTT accreditation concept, plan, report and briefings due on the dates established by the Government. Reports and presentations are due on the dates established by the Government. Material shall be in the required AFOTEC format, available from the COR.

6 SECURITY REQUIREMENTS

6.1 Security Clearances

6.1.1 All contractor personnel are required to access Top Secret, Special Program Access, and Sensitive Compartmented Information (SCI). Required information on a regular basis.

6.1.2 The contractor shall adhere to all security policies and procedures in the execution of this Task Order. Security policies pertaining to all facets of the F-35 test program to include, applicable references in the DD254, overarching JPO security policy, all AFOTEC security policies including facilities security, computer security, personnel security and program security. Since performance on this task will be conducted in an international environment, the contractor must adhere to all policies and procedures pertaining to interactions with foreign national mission partners and foreign disclosure guidance as defined in applicable Technical Assistance Agreements.

6.1.3 The contractor shall provide the Government with required information for program access requests to include justifications, letters of compelling need, and review all program access packages to ensure they are accurate, complete, and ready for Det 1/JOTT submittal IAW the AFOTEC/JOTT SAP Security Officer guidance. Contractor shall submit visit requests for personnel forecasting travel and/or in travel status.

6.2 Safeguarding Information. The contractor shall:

6.2.1 Comply with DoD 5400.7-R, Chapter 4, DoD Freedom of Information Act (FOIA) Program, and AFI 33-332, Privacy Act Program requirements. Executive Order 13526, DODM 5200.1, Volumes 1-4, JAFAN Manual 6/0, 6/4 and 6/9, ICD 705, DoD 5105.21-M Volume 1-3, DoDM 5205.07 Volume 3, AFI 31-401 and AFI 14-304. ISOO Marking Guide, United States Air Force Security Marking Guide for Special Access Programs and CAPCO Marking Guide. DODM 5205.07 SAP Security Manual Volume 1 (V1) General Procedures; Reference Enclosure 14, Information Assurance in this volume; Volume 2 (V2) SAP Access Determination Process; Volume 3 (V3) Physical Security Guidance for Special Access Program Facilities (SAPFs); Volume 4 (V4) Classification and Marking Guidance; NIST Special Publications (SP) Joint Task Force (JTF) Initiative documents; NIST SP 800-53, Rev 3, Recommended Security Controls for Federal Information Systems and Organizations; NIST SP 800-53A, Guide for Assessing the Security Controls in Federal Information Systems and Organizations, Building Effective Security Assessment Plans; NIST SP 800-37, Guide for Applying the Risk Management Framework to Federal Information Systems; NIST SP 800-39, Managing Information Security Risk: Organization, Mission, and Information System View; NIST SP 800-30, Guide for Conducting Risk Assessments; CNSSI 1253, Security Categorization and Control Selection for National Security Systems and the Joint SAP Implementation Guide (JSIG). These regulations set policy and procedures for the disclosure of records to the public and for marking, handling, transmitting and safeguarding FOUO material.

6.2.2 Report to an appropriate authority any information or circumstances of which they are aware may pose a threat to the security of DoD personnel, contractor personnel, resources and classified or unclassified defense information. Contractor employees shall be briefed by their immediate supervisor upon initial on-base assignment.

6.3 CACs/Badges. The Contractor shall:

6.3.1 Submit Application for Department of Defense Common Access Card/DEERS Enrollment, DD Form 1172-2, for initial request of contractor employee; contractor to furnish updated CAC List as needed throughout the PoP; and update an annual CAC listing and provide to the JOTT Security Office, 187 North Flightline Road, Edwards AFB CA 93524.

7 SAFETY REQUIREMENTS

7.1 Safety Considerations. The contractor shall plan for and implement safety procedures and protective equipment to reduce/negate injury to personnel engaged in activities supporting this contract. Safety areas of concern include but not limited to ground, flight, occupational, and environmental safety. Contractor shall comply with all applicable OSHA, Federal, State and Local regulations.

7.2 Program Safety Considerations. The contractor shall support field events. Contractor employees shall be trained and certified IAW applicable OSHA standards as required. The contractor shall maintain current training records and/or certificates of completion and provide them to the Government upon request.

8. ORGANIZATIONAL CONFLICT OF INTEREST

8.1 In accepting and performing the work required by this task order, and all amendments thereto, the contractor certifies that the requirements of clause 5352.209-9000, Organizational Conflict of Interest (OCI) have been or will be adhered to. The contractor must immediately bring any OCI issues to the attention of the Procuring Contract Officer and the Contracting Officer’s Representative for a determination of whether a mitigation plan is required.

9. CONTRACTOR FULL-TIME EQUIVALENT REPORTING ADDENDUM

9.1 Contractor Full-Time Equivalent Reporting: The contractor shall report ALL contractor labor hours (including subcontractor labor hours) required for performance of services provided under this contract for AFOTEC via a secure data collection site. Data will be collected and reported at both the task order and specific program level. The contractor is required to completely fill in all required data fields at http://www.ecmra.mil. Reporting inputs will be for the labor executed during the period of performance for each Government fiscal year (FY), which runs 1 October through 30 September. While inputs may be reported any time during the FY, all data shall be reported no later than 31 October* of each calendar year. Contractors may direct questions to the CMRA help desk. * Reporting Period: Contractors are required to input data by 31 October of each year. Uses and Safeguarding of Information: Information from the secure website is considered to be proprietary in nature when the contract number and the contractor identity are associated with the direct labor hours and direct labor dollars. At no time will any data be released to the public with the contractor name and contract number associated with the data. User Manuals: Data for Air Force service requirements must be input at the Air Force CMRA link. However, user manuals for Government personnel and contractors are available at the Army CMRA link at http://www.ecmra.mil.