N0018920R0031.pdf

SEE ADDENDUM

(No Collect Calls)

N0018920R0031 18-May-2020

b. TELEPHONE NUMBER

757-443-1395

8. OFFER DUE DATE/LOCAL TIME

12:00 AM 17 Jun 2020

5. SOLICITATION NUMBER 6. SOLICITATION ISSUE DATE

AUTHORIZED FOR LOCAL REPRODUCTION

PREVIOUS EDITION IS NOT USABLE

STANDARD FORM 1449 (REV. 2/2012)

Prescribed by GSA – FAR (48 CFR) 53.212

(TYPE OR PRINT)

(SIGNATURE OF CONTRACTING OFFICER)

ADDENDA X ARE

26. TOTAL AWARD AMOUNT (For Gov t. Use Only )

23.

CODE 10. THIS ACQUISITION IS

SUCH ADDRESS IN OFFER

17b. CHECK IF REMITTANCE IS DIFFERENT AND PUT

BELOW IS CHECKED

TELEPHONE NO.

N001899. ISSUED BY

18b. SUBMIT INVOICES TO ADDRESS SHOWN IN BLOCK 18a. UNLESS BLOCK

7. FOR SOLICITATION

INFORMATION CALL:

a. NAME

TIERRA K. WATSON

2. CONTRACT NO. 3. AWARD/EFFECTIVE DATE 4. ORDER NUMBER

(TYPE OR PRINT)

30b. NAME AND TITLE OF SIGNER 30c. DATE SIGNED 31b. NAME OF CONTRACTING OFFICER

30a. SIGNATURE OF OFFEROR/CONTRACTOR 31a.UNITED STATES OF AMERICA

1 27a. SOLICITATION INCORPORATES BY REFERENCE FAR 52.212-1. 52.212-4. FAR 52.212-3. 52.212-5 ARE ATTACHED.X

25. ACCOUNTING AND APPROPRIATION DATA

1. REQUISITION NUMBER

20.

ADDITIONAL SHEETS SUBJECT TO THE TERMS AND CONDITIONS SPECIFIED.

OFFEROR TO COMPLETE BLOCKS 12, 17, 23, 24, AND 30

SOLICITATION/CONTRACT/ORDER FOR COMMERCIAL ITEMS

ARE NOT ATTACHED

27b. CONTRACT/PURCHASE ORDER INCORPORATES BY REFERENCE FAR 52.212-4. FAR 52.212-5 IS ATTACHED. ADDENDA ARE ARE NOT ATTACHED

(BLOCK 5), INCLUDING ANY ADDITIONS OR CHANGES WHICH ARE

SET FORTH HEREIN, IS ACCEPTED AS TO ITEMS:

. YOUR OFFER ON SOLICITATION

28. CONTRACTOR IS REQUIRED TO SIGN THIS DOCUMENT AND RETURN

% FOR:SET ASIDE:UNRESTRICTED OR X

SMALL BUSINESS

17a.CONTRACTOR/ CODE FACILITY

OFFEROR CODE

NAVSUP FLC NORFOLK CONTRACTING

NORFOLK OFFICE

ATTN: T. WATSON

1968 GLIBERT STREET, SUITE 600

NORFOLK VA 23511-3392

18a. PAYMENT WILL BE MADE BY CODE

RATED ORDER UNDER

DPAS (15 CFR 700)

13a. THIS CONTRACT IS A

13b. RATING

CODE15. DELIVER TO CODE N57023 16. ADMINISTERED BY

12. DISCOUNT TERMS11. DELIVERY FOR FOB DESTINA-

TION UNLESS BLOCK IS

MARKED

SEE SCHEDULE

14. METHOD OF SOLICITATION

RFQ IFB RFPX

OPERATIONAL TEST & EVALUATION FORCE

TIMOTHY BURROWS

7970 DIVEN STREET

NORFOLK VA 23505-1498

TEL: 757-282-5546 X3136 FAX:

FAX:

TEL: 757-443-1395 SERVICE-DISABLED

VETERAN-OWNED

SMALL BUSINESS

X 8(A)

HUBZONE SMALL

BUSINESS

SIZE STANDARD:

41,500,000

NAICS:

541330

X

OFFER DATED

29. AWARD OF CONTRACT: REF.

DELIVER ALL ITEMS SET FORTH OR OTHERWISE IDENTIFIED ABOVE AND ON ANY

COPIES TO ISSUING OFFICE. CONTRACTOR AGREES TO FURNISH AND

EMAIL:

TEL:

31c. DATE SIGNED

SEE SCHEDULE

SCHEDULE OF SUPPLIES/ SERVICESITEM NO. QUANTITY UNIT UNIT PRICE AMOUNT

24.22.21.19.

WOMEN-OWNED SMALL BUSINESS (WOSB)

ELIGIBLE UNDER THE WOMEN-OWNED

SMALL BUSINESS PROGRAM

EDWOSB

32g. E-MAIL OF AUTHORIZED GOVERNMENT REPRESENTATIVE

SOLICITATION/CONTRACT/ORDER FOR COMMERCIAL ITEMS

(CONTINUED)

PAGE 2 OF108

ACCEPTED, AND CONFORMS TO THE CONTRACT, EXCEPT AS NOTED: ______________________________________________________

32a. QUANTITY IN COLUMN 21 HAS BEEN

RECEIVED INSPECTED

32b. SIGNATURE OF AUTHORIZED GOVERNMENT

REPRESENTATIVE

32c. DATE 32d. PRINTED NAME AND TITLE OF AUTHORIZED GOVERNMENT

REPRESENTATIVE

32e. MAILING ADDRESS OF AUTHORIZED GOVERNMENT REPRESENTATIVE 32f . TELEPHONE NUMBER OF AUTHORIZED GOVERNMENT REPRESENTATIVE

37. CHECK NUMBER

FINALPARTIALCOMPLETE

36. PAYMENT35. AMOUNT VERIFIED

CORRECT FOR

34. VOUCHER NUMBER

FINAL

33. SHIP NUMBER

PARTIAL

38. S/R ACCOUNT NUMBER 39. S/R VOUCHER NUMBER 40. PAID BY

41a. I CERTIFY THIS ACCOUNT IS CORRECT AND PROPER FOR PAYMENT

41b. SIGNATURE AND TITLE OF CERTIFYING OFFICER 41c. DATE

42a. RECEIVED BY (Print)

42b. RECEIVED AT (Location)

42c. DATE REC'D (YY/MM/DD) 42d. TOTAL CONTAINERS

STANDARD FORM 1449 (REV. 2/2012) BACK

Prescribed by GSA – FAR (48 CFR) 53.212

AUTHORIZED FOR LOCAL REPRODUCTION

PREVIOUS EDITION IS NOT USABLE

SEE SCHEDULE

20.

SCHEDULE OF SUPPLIES/ SERVICES

21.

QUANTITY UNIT

22. 23.

UNIT PRICE

24.

AMOUNT

19.

ITEM NO.

N0018920R0031

Section SF 1449 - CONTINUATION SHEET

PERFORMANCE WORK STATEMENT

PERFORMANCE WORK STATEMENT FOR ANALYTICAL OPERATIONAL TEST AND

EVALUATION SUPPORT SERVICES FOR COMMAND, CONTROL, COMMUNICATIONS,

COMPUTERS, INTELLIGENCE, AND SPACE WARFARE DIVISION (CODE 60)

1. Scope of Tasking. Contractor will provide technical and analytical support to Commander, Operational Test and Evaluation Force (COMOPTEVFOR) in the definition, conduct and analysis of structured Operational Test and Evaluation (OT&E) of Command, Control, Communications, Computers, Intelligence, and Space Warfare (C4IS) systems. This support will be required in one or more of the following forms: Documentation Review, Operational Test Design, Test Planning Support, Test Execution, Data Management, Test Operations Support, Test Analysis and Reporting, Modeling and Simulation (M&S) Support, and Program Management Support.

COMOPTEVFOR is charged with leading the Operational Test community with highly skilled testers and staff that adapt to change, and provide credible, prompt, warfighting-focused test results to Navy, Marine Corps and Coast Guard forces and acquisition leadership.

1.1. Background There are five warfare divisions at COMOPTEVFOR that are supported by competency divisions. The warfare divisions include Undersea Warfare (40), Air Warfare (50), C4IS (60), Surface Warfare (70), Expeditionary Warfare/Littoral Combat Ship (80) Division. Each warfare division has a Navy Captain as the division Director with a senior civil servant as the Deputy or a senior civil servant as the division Director and a Navy Commander as the Deputy.

The warfare divisions represent the traditional core of the COMOPTEVFOR organization. This is where the Fleet operators reside. The work there allows COMOPTEVFOR to bridge the gap from technical to the tactical, successfully. The warfare divisions are staffed by uniformed military personnel and Navy government civilians, augmented by a contractor workforce.

There are four competency divisions: Policy and Operations (01A), Test Design (01B), Test Planning and Analysis (01C) and Cybersecurity Testing (01D). In addition, the Technical Director (00TD) oversees the technical aspects of the Command’s mission. Other support divisions include the Staff Commanding Officer and Administration (10), Chief Information Officer (CIO) (20), Contracts (01), and the Comptroller (30). The members of the competency divisions generally work within the test team to ensure that the Commander’s policies are adhered to and that best practices are applied.

1.2. For scoping purposes, a description of the C4IS Warfare division and competency divisions are listed below, as well as a current list of C4IS evaluation programs being performed. This list is current as of time of this solicitation, and additional programs may be assigned to COMOPTEVFOR for contractor support under this contract.

1.2.1. The C4IS Warfare Division (Code 60) Tests and evaluates all ashore and afloat Command, Control, Communications, Computers, Intelligence, Space, Business, and Enterprise Information systems such as radars, radio frequency transmitters/receivers, digital switching networks, identification friend or foe, tactical decision-making systems, intelligence systems, combat direction systems, tactical links, information warfare systems, cryptologic systems, space and satellite systems, and non-tactical automated information systems. The division's main effort is to assist, through early involvement, in the acquisition process conduct tests, and provide timely and accurate evaluations to characterize system performance and resulting capability. Current C4IS test and evaluation programs include, but are not limited to :

1.2.1.1. Command and Control Processor (C2P)

1.2.1.2. Electronic Procurement System (ePS)

1.2.1.3. LINK16

1.2.1.4. LINK Monitoring and Management Tool (LMMT)

1.2.1.5. Maritime Tactical Command and Control (MTC2)

1.2.1.6. NOBLE

1.2.1.6.1. Naval Aviation Maintenance System (NAMS)

1.2.3.6.2 Naval Operational Maintenance Environment (NOME)

1.2.3.6.3 Naval Operational Supply System (NOSS)

1.2.1.7. Surface Electronic Warfare Improvement Program (SEWIP)

1.2.3.7.1 SEWIP Block 2

1.2.3.7.2 SEWIP Block 3

1.2.1.8. Mobile User Objective System (MUOS)

1.2.1.9. Battle Force Tactical Network (BFTN)

1.2.1.10. Enhance Polar Satellite (EPS)

1.2.1.11. Global Broadcasting System (GBS)

1.2.1.12. Digital Modular Radio (DMR)

1.2.1.13. Navy Multiband Terminal (NMT)

1.2.1.14. Protected Tactical Enterprise Service (PTES)

1.2.1.15. Consolidated Afloat Network and Enterprise Services (CANES)

1.2.1.16. Distributed Common Ground System – Navy Increment 1 (DCGS-N Inc 1)

1.2.1.17. DCGS-N Increment 2 (DCGS-N Inc 2)

1.2.1.18. Department of Defense Health Management System Modernization (DHMSM)

1.2.1.19. Naval Maritime Maintenance Enterprise Solution—Technical Refresh (NMMES—TR)

1.2.1.20. Manpower, Personnel, Training, and Education (MPT&E)

1.2.1.21. Naval Integrated Tactical Environment System—Next Generation (NITES Next)

1.2.1.22. Counter Insider Threat Capability (CITC)

1.2.1.23. Computer Network Defense (CND)

1.2.1.24. Shark Cage

1.2.1.25. Navy Cyber Situational Awareness (NCSA)

1.2.1.26. Weather Satellite Follow-On Microwave (WSF-M)

1.2.1.27. United States Naval Observatory—Precise Time and Astronomy (USNO-PTA)

1.2.1.28. Global—Theatre Security Cooperation Management System (G-TSCMIS)

1.2.1.29. Enterprise Resource Planning (ERP)

1.2.1.30. Command and Control Air Operations Suite (C2AOS)

1.2.1.31. Identification Friend or Foe (IFF) Mode 5

1.2.1.32. Shipboard Air Traffic Control Radar (SATR) AN/SPN-50

1.2.1.33. Network Tactical Common Data Links (NTCDL)

1.2.1.34. Nuclear Command, Control, and Communications (NC3)

1.2.1.35. Advanced Off-board Electronic Warfare (AOEW)

1.2.1.36. Spectral

1.2.1.37. Joint Military Global Positioning System (GPS) User Equipment (MGUE)

1.2.1.38. Ship’s Signal Exploitation Equipment Modifications (SSEE Mods)

1.2.1.39. Tactical Cryptologic System (TCS)

1.2.1.40. Cryptologic Carry-On Program (CCOP)

1.2.1.41. Integrated Communications and Data System (ICADS)

1.2.1.42. Electronic Chart Display and Information System—Navy (ECDIS-N)

1.2.1.43. Navy—Electronic Chart Display and Information System (N-ECDIS)

1.2.1.44. GPS-Base Positioning, Navigation, and Timing Service (GPNTS)

1.2.1.45. Inertial Navigation System-Replacement (WSN-12)

1.2.1.46. Protected Tactical SATCOM (PTS)

1.2.1.47. Navy Personnel and Pay (NP2)

1.2.1.48. Joint Operational Medicine Information Systems (JOMIS)

1.2.2. Test Policy, Operations and Training (Code 01A) ensures COMOPTEVFOR policy conforms to Navy Test and Evaluation (T&E) Policy; monitors production of T&E products; leads continuous process improvement efforts; coordinates external OT&E inquiries Inspector General and Government Accountability Office; manages all government T&E training (including budget requests); provides T&E editorial support to the command; and plans, analyzes and evaluates the effectiveness of command processes.

1.2.3. Test Design and M&S (Code 01B) is responsible for the Mission- Based Test Design (MBTD) process, M&S technical review, test planning and post-test iterative process analysis support, test design production, Capabilities Based Test and Evaluation coordination, and Targets, Ranges, and Test Resources coordination.

1.2.4. Test Planning and Analysis Division (Code 01C) is responsible for policy, processes and standardization for test planning, test execution, post-test analysis and evaluation (reporting). This includes Lead Test Engineer support to the COMOPTEVFOR Warfare Divisions and Fleet Resources coordination to support execution of OT&E.

1.2.5. Cybersecurity Division (Code O1D) OT&E integrates critical cybersecurity testing into the acquisition lifecycle and tests Systems under Test (SUT) per Department of Defense Instruction

5000.02 and Director, Operational Test and Evaluation (DOT&E) Memorandum of 01 August 2014.

The Warfare division will be responsible for executing acquisition test design, planning support to ultimately provide the Cybersecurity OT&E group (O1D) support in cybersecurity execution and reporting. Through a rigorous and iterative test process, Cybersecurity OT&E assists in the development and fielding of more secure and resilient Information Technology (IT) systems supporting the warfighters. Code 01D Cybersecurity OT&E group includes the following support positions: Technical Director, Future Capabilities and Threat Emulation Developer, Analysis Manager, Training and Lab Manager, Test Team Leads and Certified Ethical Hackers.

1.3. Period of Performance. 16 November 2020 through 15 May 2026, inclusive of the 52.217-8.

1.4. Place of Performance. Unless required to support mission accomplishment, work shall primarily be performed at the customer’s site. If the number of personnel exceeds onsite capabilities, unclassified work will be expected to be performed offsite. The customer location is:

Commander, Operational Test and Evaluation Force 7970 Diven Street Norfolk, VA 23505-1498

While onsite at COMOPTEVFOR, the contractor shall comply with Office of the Chief of Naval Operations (OPNAV) Instruction (INST) 5239 (series), with specific regard to non-government hardware and software.

1.4.1. Travel. Travel can be expected to occur Continental United States (CONUS) and Outside the

Continental United States (OCONUS) and will be required throughout the course of this contract.

All travel costs shall be in accordance with the Federal Travel Regulations (FTR). Invoices shall include actual costs experienced. Airfare shall be based on the lowest available cost for coach or economy in writing, in advance. Travel dates and locations are to be confirmed through the Technical Assistant and approved by the Contracting Officer’s Representative (COR) in writing, prior to commencement of any travel.

1.4.1.1. Travel Instructions.

1.4.1.1.1. Ensure all Contractors supporting this requirement meet the minimum requirements of the Commander, U.S. Fleet Forces Command/Commander, U.S. Pacific Fleet Instruction 6320.3B, dated 07Apr14, Medical Screening for U.S. Government Civilian Employees, Contractor Employee’s, and Guests and Visitors Prior to Embarking Fleet Units.

1.4.1.1.2. Any travel OCONUS by the Contractor will need to review the Foreign Clearance Guide at http://www.fcg.pentagon.mil and select the country of intended travel. During OCONUS travel, the contractor shall operate in accordance with International Traffic in Arms Regulation (ITAR) Part 124, approved by US Department of State, Office of Defense Trade Controls. Specific guidance can be found at:

https://www.pmddtc.state.gov/regulations_laws/itar.html.

1.4.1.1.3. The Contractor is not authorized to perform any travel that is not in conjunction with this contract. It is the Department of the Navy policy not to allow a charge of profit or fee on reimbursable items.

1.4.1.1.3.1. The Government will make no reimbursements for a contractor’s local travel

(defined as less than 50 miles from the place of performance).

1.4.1.1.3.2. In no instance shall the contractor exceed the amount specified in the contract for travel Contract Line Item Number (CLIN) and Other Direct Cost (ODC) CLIN expenses.

1.4.1.1.3.3. Locations include CONUS, OCONUS and at sea locations and any other location, which may be added on a Task Order (TO) basis based on future test requirements.

1.4.1.1.3.4. For planning purposes any travel location OCONUS must be considered as a possible travel location unless travel restrictions at time of test do not permit entry. Possible OCONUS locations include, but are not limited to the following:

Sasebo, Japan, Naples Italy, Australia, Hawaii, Halifax, Nova Scotia, Nanoose, British Colombia, and any other location, which may be added on a TO basis based on future test requirements.

1.5. Personal Services. This is not a personal services TO. The contractors/sub-contractors should be well educated in respect to “personal services”. No one (Government employee or military) may task the contractor to perform work that is not listed in the delivery order, nor do the contractors take direction in the performance of their task from anyone other than their corporate manager and/or prime contractor, as appropriate.

1.6. Privity Defined. The legal term that defines the relationship between two parties of a contract, does not exist between the Government and the sub-contractor. Therefore, the Prime contractor is responsible for ensuring, through its own internal processes, that quality work is performed and delivered on time, by any subcontractors to meet the objectives of the prime contract. Formal direction to sub-contractors flows only thought the prime contractor. There can be simultaneous communication between the prime contractor and the subcontractor, but the Government and the subcontractor cannot bypass the prime contractor to communicate directly with each other.

1.7. Automated Data Processing (ADP) Requirements. The Government will provide, if required, the necessary computer, work station/system access for on-site work.

1.7.1. Those offerors assigned as off-site will not be assigned government furnished ADP assets.

1.7.2. Additionally, the offerors assigned ADP Government Furnished Property (GFP) will be held accountable to respond to the governments ADP inventory requirements conducted semi-annually re-occurring every Mar and Sep to ensure 100% accountability of all ADP assets.

1.7.3. Any offeror not adhering to this call for inventory of ADP assets may be directed to forfeit their ADP

GFP at no additional cost to the government. In Accordance With (IAW) FAR 52.245-1

1.8. Common Access Card (CAC)/Public Key Infrastructure, System Authorization Access Request (SAAR-N). All contractor personnel requiring access to Government Information Technology (IT) systems shall have an approved System Authorization Access Request (SAAR-N) Form OPNAV 5239/14 (Rev Sep 2011) on file, and complete required Annual Information Awareness Training. New employees must submit their SAAR forms upon check in of their first day of work. Instructions for processing the SAAR-N forms are available at:

http://www.cnrc.navy.mil/publications/Forms/OPNAV_5239_14_SAAR_N.pdf.

1.8.1. SAAR-N forms shall be submitted to the CIO Department, specifically the Information Systems

Security Manager.

1.9. CAC/Local Badges. Contractor CACs and facility specific identification badges will be approved using the Trusted Agent Sponsorship system and issued by the Government to on-site contractor personnel and shall be visible at all times while personnel are at the Government site. The contractor shall furnish all requested information required to facilitate issuance of identification badges and shall conform to all instructions/guidance required by the facility. All CACs and identification badges issued to Contractor employees shall be returned to the Government Security Department at the Government site in accordance with the instructions/ guidance required by your facility following completion of the contract, relocation or termination of an employee, or upon request from the Contracting Officer’s Representative. The Government will provide the contractor access to Government facilities, as required, for performance of tasks under this contract. Contractor personnel shall comply with the instructions/guidance required by your facility.

1.10. ODCs, Increased Operational Requirements and Contingency Support.

1.10.1. If Applicable, ODCs consist of test equipment and supplies as required for performance of TO. In no instance shall the contractor exceed the amount specified in the contract for ODC CLIN expenses.

1.10.2. When applicable, contingency services will be estimated for and ONLY used when:

1.10.2.1. An unanticipated requirement or contingency arises due to changes in testing events or operations;

1.10.2.2. Is within scope (within para 2.1 thru 2.7.7) of the contract/task order and necessary to complete the work required in the task order;

1.10.2.3. Will take the contractor over their normal 40 hour work (5 days) week; and

1.10.2.4. After concurrence via email from the Operational Test Director (OTD) and COR is received.

1.10.2.5. These services are not considered overtime. These services will be included as a total price of the task order, if ordered.

1.11. Receipt and Acceptance of Deliverables. All Deliverables Contractor Data Requirement List listed will be delivered by the Contractor in accordance with the Quality Assurance Surveillance Plan (QASP).

These delivered items shall be sent in an email by the contractor to the Operational Test Director (OTD), Section Head, Warfare Division Deputy Director, (and other applicable recipients to include the COR.

Upon receipt, the OTD will send an email to the COR facilitating the formal government’s position for the respective product for timeliness and/or accuracy. The COR will forward a copy of this email to the prime of the TO and the email will become part of the COR folder for that respective TO.

2. Specific Tasking. In accordance with the PWS and the basic contract the contractor shall provide support in evaluating the operational effectiveness and suitability of the SUT that has been assigned to COMOPTEVFOR for OT&E, in accordance with the format prescribed in reference (a), COMOPTEVFORINST 3980.2 series, OTD Guide (or other governing references listed in paragraph 5 as applicable). The manual maybe found at:

http://www.public.navy.mil/cotf/OTD/OTD%20Manual.pdf

2.1. Overall Task Guidance. To gain efficiencies and synergy in contract execution, the Government encourages the Contractor to consider:

2.1.1. Flexibility in production. COMOPTEVFOR produces information which may take many forms.

Support teams should be willing to experiment, bring up new ideas or better ways of doing things, but remain willing to execute stated processes when directed.

2.1.2. Present concerns about the validity of the information produced, and value to the end customer (Fleet or Program). Remain willing to defend a contrary position until government makes a final decision.

2.1.3. Remain Efficient and Economical in production. Strive to produce information as quickly as possible without sacrificing quality, and with the appropriate level of expertise.

2.1.4. Maintain a broad perspective. Awareness of testing outside of a specific program to seek and recommend testing efficiencies and overall kill chain evaluation opportunities.

2.1.5. Technical competence. Improve T&E community’s ability to understand, analyze, communicate, and ultimately visualize T&E issues from a warfighter and technical perspective.

2.2. Test Design

2.2.1. Mission Analysis

2.2.1.1. The Contractor shall perform MBTD mission analysis IAW Integrated Evaluation Framework

(IEF) Checklist reference (b) and IEF Template reference (c).

2.2.1.1.1. The contractor shall analyze source data to perform MBTD Mission Based Test Design mission analysis.

2.2.1.1.2. The contractor shall identify and document the mission areas.

2.2.1.1.3. The contractor shall identify and document system capabilities.

2.2.1.1.4. The contractor shall derive and document operator tasks.

2.2.1.1.5. The contractor shall define and document the operational conditions applicable to the system to be tested.

2.2.1.1.6. The contractor shall document the mission analysis in the Mission Based Test and

Evaluation System (MBTES).

2.2.1.1.7. The contractor shall document the mission analysis in an IEF document.

2.2.1.1.8. The contractor shall review the mission analysis during MBTD reviews.

2.2.1.1.9. The contractor shall provide meeting minutes, to include any required comment resolution matrix for issues, to document the results of mission analysis MBTD reviews.

2.2.2. Requirements Analysis

2.2.2.1. The contractor shall perform MBTD Design requirements analysis in accordance with references (b) and (c).

2.2.2.1.1. The contractor shall analyze source data to perform MBTD requirements analysis.

2.2.2.1.2. The contractor shall identify and document measures of system performance, inclusive of critical measures.

2.2.2.1.3. The contractor shall link measures of system performance to subtasks and document this linkage.

2.2.2.1.4. The contractor shall identify and document the data required to quantify the measures of system performance.

2.2.2.1.5. The contractor shall identify and document the data required to quantify the operational conditions applicable to the system to be tested.

2.2.2.1.6. The contractor shall document requirements analysis in the MBTES.

2.2.2.1.7. The contractor shall review the requirements analysis during MBTD reviews.

2.2.2.1.8. The contractor shall provide meeting minutes, to include any required comment resolution matrix for issues, to document the results of requirements analysis MBTD reviews.

2.2.3. Test Design

2.2.3.1. The Contractor shall perform MBTD test design in accordance with references (b) and (c).

2.2.3.1.1. The contractor shall analyze source data to perform MBTD.

2.2.3.1.2. The contractor shall identify and document the candidate response variables for statistical test design.

2.2.3.1.3. The contractor shall identify and document factors/conditions (with corresponding levels) associated with each response variable or critical measure used in test design.

2.2.3.1.4. The contractor shall participate in and document the statistical test design for candidate response variables and/or critical measures.

2.2.3.1.5. The contractor shall identify and document the test vignettes.

2.2.3.1.6. The contractor shall determine and document the appropriate test method.

2.2.3.1.7. The contractor shall determine and document a test schedule supporting execution of the test design.

2.2.3.1.8. The contractor shall identify and document the expected use of M&S in the execution of the test design.

2.2.3.1.9. The contractor shall determine and document the resources required to conduct the test.

2.2.3.1.10. The contractor shall determine and document the limitations to test based on known availability of resources and assets to conduct the test.

2.2.3.1.11. The contractor shall develop and document data sheets and test instrumentation (CDRL

A001) to support test execution.

2.2.3.1.12. The contractor shall document the test design in the MBTES.

2.2.3.1.13. The contractor shall provide meeting minutes, to include any required comment resolution matrix for issues, to document the results of test design MBTD reviews.

2.2.4. Stakeholder Collaboration Working Integrated Product Team (WIPTs), outreach, etc

2.2.4.1. The contractor shall make recommendations to the warfare division in accordance with

COMOPTEVFOR processes found in references (b) and (c).

2.2.4.2. The contractor shall identify and recommend outside organization(s) for participation and inclusion in Test Design activities.

2.2.4.3. When directed, the contractor shall coordinate briefings/meetings in support of

COMOPTEVFOR Test Design activities in accordance with references (b) and (c).

2.2.4.4. When directed, the contractor shall participate in briefings/meetings in support of

COMOPTEVFOR Test Design activities in accordance with references (b) and (c).

2.2.4.5. When directed, the contractor shall lead briefings/meetings in support of COMOPTEVFOR

Test Design activities in accordance with references (b) and (c).

2.2.4.6. The contractor shall provide a trip report (CDRL A002) when the activities occur away from the COMOPTEVFOR site in Norfolk, VA.

2.2.5. Document and Brief Preparation

2.2.5.1. The contractor shall prepare and document briefings (CDRL A001) to include the IEF, Executive In-Process Review and stakeholder collaboration briefings, as directed.

2.2.6. Test and Evaluation Master Plan (TEMP) Input Preparation

2.2.6.1. The contractor shall analyze source data to provide input to the TEMP, to include program schedules, the IEF, applicable policy documents and any other program-provided materials

2.2.6.2. The contractor shall develop and document input to the TEMP IAW the DOT&E TEMP Guidebook, reference (d).

2.2.6.3. The contractor shall develop and document comments to the TEMP related to or affecting

OT&E.

2.2.7. Stakeholder Collaboration

2.2.7.1. The contractor shall make recommendations to the warfare division in accordance with COMOPTEVFOR processes found in references (a) and (b).

2.2.7.2. When directed, the contractor shall coordinate briefings/meetings in support of COMOPTEVFOR TEMP Input activities in accordance with references (a) and (b).

2.2.7.3. When directed, the contractor shall participate in briefings/meetings in support of COMOPTEVFOR TEMP Input activities in accordance with references (a) and (b).

2.2.7.4. When directed, the contractor shall lead briefings/meetings in support of COMOPTEVFOR TEMP Input activities in accordance with references (a) and (b).

2.2.7.5. The contractor shall provide a trip report (CDRL A002) when the activities occur away from

2.3. Test Planning

2.3.1. Plan of Action and Milestones (POA&M)

2.3.1.1. The contractor shall develop and document a POA&M for the Test Planning process in accordance with reference (e).

2.3.1.2. The contractor shall analyze source data to perform Test Planning POA&M development.

2.3.1.3. The contractor shall obtain and maintain access to the program MBTES database to support the test planning process.

2.3.2. Purpose of Test

2.3.2.1. The contractor shall perform the activities associated with development of Purpose of Test in accordance with reference (e) test planning checklist.

2.3.2.1.1. The contractor shall analyze source data to perform Test Planning Purpose of Test development.

2.3.2.1.2. The contractor shall update and document the SUT and SoS descriptions from the IEF for use in the Test Plan.

2.3.2.1.3. The contractor shall identify and document the SUT test configuration.

2.3.2.1.4. The contractor shall identify and document the Developmental Test /Integrated Test contributions to OT&E.

2.3.2.1.5. The contractor shall identify and document regression test requirements for any Follow-on OT&E phases, if applicable.

2.3.2.1.6. The contractor shall identify and document changes to the limitations to test.

2.3.2.1.7. The contractor shall update and document the Mission Critical Subsystem Matrix

(MCSM) from the IEF.

2.3.2.1.8. The contractor shall document the Purpose of Test activities in a Test Plan document in accordance with reference (e) and the appropriate test plan templates.

2.3.2.1.9. The contractor shall review the Purpose of Test activities during test plan reviews.

2.3.2.1.10. The contractor shall identify test equipment/instrumentation needed for test execution.

2.3.2.1.11. The contractor shall provide meeting minutes, to include any required comment resolution matrix for issues, to document the results of Purpose of Test reviews.

2.3.3. Test Schedule

2.3.3.1. The contractor shall perform the activities associated with development of the Test Schedule in accordance with reference (e).

2.3.3.1.1. The contractor shall analyze source data to perform Test Planning Test Schedule development.

2.3.3.1.2. The contractor shall identify, recommend and document data to be qualified for use in

OT&E.

2.3.3.1.3. The contractor shall develop and document the Test Schedule, to include test events, for each test phase and period.

2.3.3.1.4. The contractor shall document the Test Schedule data in the MBTES.

2.3.3.1.5. The contractor shall document the Test Schedule activities in a TP document in accordance with reference (e) and the appropriate TP templates.

2.3.3.1.6. The contractor shall review the Test Schedule activities during TP reviews.

2.3.3.1.7. The contractor shall provide meeting minutes, to include any required comment resolution matrix for issues, to document the results of Test Schedule TP reviews.

2.3.4. Detailed Method of Test (DMOT)

2.3.4.1. The contractor shall perform the activities associated with development of the DMOT in accordance with reference (e).

2.3.4.1.1. The contractor shall analyze source data to perform DMOT development.

2.3.4.1.2. The contractor shall identify and document the data to be collected on each test event.

2.3.4.1.3. The contractor shall develop and document the start and stop criteria for each test event.

2.3.4.1.4. The contractor shall develop and document the GO/NO GO criteria for each test event.

2.3.4.1.5. The contractor shall develop and document the DMOT for each test event.

2.3.4.1.6. The contractor shall document the DMOT in a TP document in accordance with reference (e) and the appropriate TP templates.

2.3.4.1.7. The contractor shall review the DMOT during TP reviews.

2.3.4.1.8. The contractor shall provide meeting minutes, to include any required comment resolution matrix for issues, to document the results of DMOT plan reviews.

2.3.5. Data Collection Plan (DCP)

2.3.5.1. The contractor shall perform the activities associated with development of the DCP in accordance with reference (e).

2.3.5.1.1. The contractor shall analyze source data to perform Test planning DCP development.

2.3.5.1.2. The contractor shall review, update and document, as required, any changes to data requirements.

2.3.5.1.3. The contractor shall develop and document the DCP.

2.3.5.1.4. The contractor shall identify test equipment/instrumentation needed for test data collection.

2.3.5.1.5. The contractor shall identify data repository.

2.3.5.1.6. The contractor shall create and document data sheets, interview sheets, focus group sheets, and survey sheets in support of the DCP.

2.3.5.1.7. The contractor shall document the DCP data in the MBTES.

2.3.5.1.8. The contractor shall document the DCP in a TP document in accordance with reference

(e) and the appropriate TP templates.

2.3.5.1.9. The contractor shall review the DCP during TP reviews.

2.3.5.1.10. The contractor shall provide meeting minutes, to include any required comment resolution matrix for issues, to document the results of DCP TP reviews.

2.3.6. Data Analysis Plan (DAP)

2.3.6.1. The contractor shall perform the activities associated with development of the DAP in accordance with reference (e) and appropriate TP templates.

2.3.6.1.1. The contractor shall analyze source data to perform Test Planning DAP development.

2.3.6.1.2. The contractor will develop and document the Critical Operational Issue (COI) resolution method.

2.3.6.1.3. The contractor will develop and document data scoring criteria.

2.3.6.1.4. The contractor shall identify and document Scoring Board Guidance.

2.3.6.1.5. The contractor shall provide a data analysis repository with Government concurrence.

2.3.6.1.6. When directed, the contractor shall automate data analysis with Government concurrence.

2.3.6.1.7. The contractor shall document the DAP in a TP document in accordance with reference

(e) and the appropriate TP templates.

2.3.6.1.8. The contractor shall review the DAP during TP reviews.

2.3.6.1.9. The contractor shall provide meeting minutes, to include any required comment resolution matrix for issues, to document the results of DAP TP reviews.

2.3.7. Stakeholder Collaboration (WIPTs, outreach, etc)

2.3.7.1. The contractor shall make recommendations to the warfare division in accordance with

COMOPTEVFOR processes found in references (e) and the appropriate TP templates.

2.3.7.2. The contractor shall identify and recommend outside organization(s) for participation and inclusion in Test Planning activities.

2.3.7.3. When directed, the contractor shall coordinate briefings/meetings in support of

COMOPTEVFOR Test Planning activities in accordance with references (e) and the appropriate TP templates.

2.3.7.4. When directed, the contractor shall participate in briefings/meetings in support of COMOPTEVFOR Test Planning activities in accordance with references (e) and the appropriate TP templates.

2.3.7.5. When directed, the contractor shall lead briefings/meetings in support of COMOPTEVFOR Test Planning activities in accordance with references (e) and the appropriate TP templates.

The contractor shall provide a trip report (CDRL A002) when the activities occur away from

2.3.8. Document and Brief Preparation

2.3.8.1. The contractor shall prepare and document briefings (CDRL A001) to include the TP, Data

Collection Plan, Concept of Test, TP Review Board and stakeholder collaboration briefings, as directed.

2.4. M&S Support

2.4.1. M&S Requirements Development

2.4.1.1. The Contractor shall perform M&S Requirements Development in accordance with references

COMOPTEVFORINST 5000.1C, reference (f), and Best Practice series

2.4.1.2. The contractor shall identify and document candidate critical measures/response variables, along with their associated conditions, during the Test Design portion of the MBTD process that will be supplemented by M&S data.

2.4.1.3. The contractor shall develop and document a M&S Requirements letter in accordance with references (f), and Best Practice series.

2.4.1.4. The contractor shall review the M&S requirements during MBTD reviews.

2.4.1.5. The contractor shall provide meeting minutes, to include any required comment resolution matrix for issues, to document the results of M&S Requirements Development.

2.4.2. Develop accreditation plans

2.4.2.1. The contractor shall develop an M&S Accreditation Plan in accordance with reference (f), Accreditation Plan Development Guide, reference (g) and the Best Practice Series.

2.4.2.2. The contractor shall develop and document the accreditation approach method consistent with COTF policy.

2.4.2.3. The contractor shall develop and document the Accreditation Plan.

2.4.2.4. The contractor shall provide meeting minutes, to include any required comment resolution matrix for issues, to document the results of M&S Accreditation Plan Development.

2.4.3. Review Verification &Validation (V&V) Plans and Reports

2.4.3.1. The contractor shall review program M&S V&V plans and reports in accordance with reference (f) and Best Practice Series.

2.4.3.2. The contractor shall review and document disagreements or issues with V&V plan methodologies.

2.4.3.3. The contractor shall provide recommendations for alternative V&V methodologies supporting

COMOPTEVFOR Accreditation Requirements.

2.4.3.4. The contractor shall review program M&S V&V reports and document disagreements or issues with methodologies/results based on comparison with live test results.

2.4.3.5. The contractor shall develop and document the COMOPTEVFOR V&V Report addendum.

2.4.3.6. The contractor shall provide meeting minutes, to include any required comment resolution matrix for issues, to document the results of M&S V&V Plans or Reports reviews.

2.4.4. Accreditation Support

2.4.4.1. The contractor shall perform M&S Accreditation in accordance with references (f) and the

Best Practice series.

2.4.4.2. The contractor shall perform and document, in coordination with COMOPTEVFOR Test

Design analysts, any additional required analysis not provided in the V&V report in support of accreditation.

2.4.4.3. The contractor shall identify and document conclusions and recommendations based on comparison of live test results with M&S test results.

2.4.4.4. The contractor shall identify and document significant M&S limitations that impact the accreditation recommendation.

2.4.4.5. The contractor shall identify and document where the M&S results support the intended use for

OT.

2.4.4.6. The contractor shall develop and document an accreditation recommendation in an Accreditation Letter.

2.4.4.7. The contractor shall provide meeting minutes, to include any required comment resolution matrix for issues, to document the results of M&S Accreditation activities and reviews.

2.4.5. Stakeholder Collaboration

2.4.5.1. The contractor shall make recommendations to the warfare division in accordance with COMOPTEVFOR processes found in reference (f) and the Best Practice series.

2.4.5.2. The contractor shall identify and recommend outside organization(s) for participation and inclusion in M&S Verification, Validation, and Accreditation (VV&A) activities.

2.4.5.3. When directed, the contractor shall coordinate briefings/meetings in support of COMOPTEVFOR M&S VV&A activities in accordance with reference (f) and the Best Practice series.

2.4.5.4. When directed, the contractor shall participate in briefings/meetings in support of COMOPTEVFOR M&S VV&A activities in accordance with references (f) and the Best Practice series.

2.4.5.5. When directed, the contractor shall lead briefings/meetings in support of COMOPTEVFOR M&S VV&A activities in accordance with references (f) and the Best Practice series.

2.4.5.6. The contractor shall provide a trip report (CDRL A002) when the activities occur away from

2.4.6. Document and Brief Preparation

2.4.6.1. The contractor shall prepare and document briefings (CDRL A001) to include the M&S

Requirements Letter, Accreditation Plan, Verification and Validation Report Addendum, Accreditation Letter and stakeholder collaboration briefings, as directed.

2.5. Test Execution

2.5.1. Pre-test Coordination

2.5.1.1. The contractor shall identify and schedule required test services (ranges, targets, personnel, etc.) for the test phase.

2.5.1.2. The contractor shall develop and submit asset scheduling requests to COMOPTEVFOR Fleet Services schedulers in accordance with reference (a).

2.5.1.3. The contractor shall develop and provide the COMOPTEVFOR pre-test letters for test units and ISICs in accordance with reference (a).

2.5.1.4. The contractor shall develop and provide briefing materials as required to support Operational Test Requirements Reviews (OTRRs) IAW SECNAVINST 5000.2 series.

2.5.2. Pre-test Brief

2.5.2.1. The contractor shall prepare and document the Pre-Test briefing (CDRL A001).

2.5.3. Test Operations

2.5.3.1. The contractor shall collect and document test data using data cards/sheets, interview sheets, focus group sheets, and survey sheets in accordance with COMOPTEVFOR procedures and the signed TP.

2.5.3.2. The contractor shall distribute text execution materials to all test participants, when directed.

2.5.3.3. The contractor shall develop and document any test deviations resulting from test variances.

2.5.3.4. The contractor shall develop and document a daily summary of test operations email, when directed.

2.5.3.5. The contractor shall coordinate/operate test equipment/instrumentation with Government concurrence.

2.5.4. Post-Test Activities

2.5.4.1. The contractor shall conduct and document post-test interviews for all test events requiring interviews of participants.

2.5.4.2. The contractor shall conduct and document test team debriefs, when directed.

2.5.4.3. The contractor shall conduct and document test-site scoring boards, when directed.

2.6. Post-Test Iterative Process (PTIP) and Reporting

2.6.1. Data Scoring

2.6.1.1. The Contractor shall perform PTIP Data Scoring in accordance with references (a) and the

Best Practice series.

2.6.1.2. The contractor shall inventory, compile and document all collected data, comparing data collected to that required by the Test Plan scope, DMOT and DCP.

2.6.1.3. The contractor shall develop and document a test execution summary.

2.6.1.4. The contractor shall review the data during Data Scoring reviews.

2.6.1.5. The contractor shall develop and document the End of Test message.

2.6.1.6. The contractor shall provide meeting minutes, to include any required comment resolution matrix for issues, to document the results of Data Scoring reviews.

2.6.2. COI Evaluation

2.6.2.1. The Contractor shall perform PTIP COI Evaluation in accordance with (a) and (Best Practice Series).

2.6.2.1.1. The contractor shall develop and document risks and/or deficiencies on COMOPTEVFOR Blue and Gold Sheets.

2.6.2.1.2. The contractor shall develop and document test results in COMOPTEVFOR test report results paragraph format.

2.6.2.1.3. The contractor shall document any limitations to test identified during test execution.

2.6.2.1.4. The contractor shall develop and document COI evaluation via the COI Evaluation

Chart.

2.6.2.1.5. The contractor shall provide meeting minutes, to include any required comment resolution matrix for issues, to document the results of COI Evaluation reviews.

2.6.3. Data Analysis

2.6.3.1. The Contractor shall perform PTIP Data Analysis in accordance with reference (a) and the

Best Practice series.

2.6.3.1.1. The contractor shall conduct and document an analysis of all measures.

2.6.3.1.2. The contractor shall develop and conduct a Responds Variable Analysis Outbrief in coordination with Test Design Analyst support.

2.6.3.1.3. The contractor shall develop a draft Data Analysis Summary (DAS).

2.6.3.1.4. The contractor shall review the DAS during analysis working groups, when directed.

2.6.3.2. The contractor shall provide meeting minutes, to include any required comment resolution matrix for issues, to document the results of Data Analysis reviews.

2.6.4. Deficiency/Risk Documentation

2.6.4.1. The Contractor shall perform PTIP Deficiency/Risk Documentation IAW reference (a) and the Best Practice series.

2.6.4.1.1. The contractor shall distribute and collect comments on draft Blue/Gold Sheets.

2.6.4.1.2. The contractor shall review the Deficiencies/Risks during PTIP reviews.

2.6.4.1.3. The contractor shall provide meeting minutes, to include any required comment resolution matrix for issues, to document the results of Deficiency/Risk reviews.

2.6.5. System Evaluation

2.6.5.1. The Contractor shall perform PTIP System Evaluation in accordance with reference (a) and the

Best Practice series.

2.6.5.1.1. The contractor shall develop and document evaluation reports to support Command

Systems Evaluation Review Board (SERB) / Executive System Evaluation Review Board (E-SERB) PTIP reviews.

2.6.5.1.2. The contractor shall provide Divisional Summary Briefs.

2.6.5.1.3. The contractor shall provide post-test Fact Sheet.

2.6.5.1.4. The contractor shall provide post-test Fleet Brief.

2.6.5.1.5. The contractor shall provide Lessons Learned.

2.6.5.1.6. The contractor shall review the system evaluation during PTIP reviews.

2.6.5.1.7. The contractor shall provide meeting minutes, to include any required comment resolution matrix for issues, to document the results of System Evaluation reviews.

2.6.6. Stakeholder Collaboration

2.6.6.1. The contractor shall make recommendations to the warfare division in accordance with

COMOPTEVFOR processes found in references (a) and the Best Practice series.

2.6.6.2. The contractor shall identify and recommend outside organization(s) for participation and inclusion in PTIP activities.

2.6.6.3. When directed, the contractor shall coordinate briefings/meetings in support of COMOPTEVFOR PTIP activities in accordance with references (a) and the Best Practice series.

2.6.6.4. When directed, the contractor shall participate in briefings/meetings in support of COMOPTEVFOR PTIP activities in accordance with references (a) and the Best Practice series.

2.6.6.5. When directed, the contractor shall lead briefings/meetings in support of COMOPTEVFOR PTIP activities in accordance with references (a) and the Best Practice series.

2.6.6.6. The contractor shall provide a trip report (CDRL A002) when the activities occur away from

2.6.7. Document and Brief Preparation

2.6.7.1. The contractor shall prepare and document briefings (CDRL A001) to include the Evaluation

Report, Post-Test Brief, Scoring Board, COI evaluation working group products, SERB/E- SERB Brief, DAS and Blue/Gold Sheets, as directed.

2.6.7.2. The contractor shall prepare the Risk/Deficiency cover letter, the DAS, and the test report per references (a,) and the Best Practice series.

2.7. Cyber Survivability Test Planning

2.7.1. Cyber Survivability Test Planning Research

2.7.1.1. The Contractor shall perform Cyber Survivability Test Planning Research in accordance with

Cyber Test Planning Process, reference (h).

2.7.1.2. The contractor shall review and identify technical documents required to support Cyber Survivability Test Planning.

2.7.1.3. The contractor shall identify need dates for all required technical documents and provide dates to the OTD/Operational Test Coordinator (OTC).

2.7.1.4. The contractor shall identify previous and planned Cyber Survivability tests that may contribute to current Cyber Survivability efforts.

2.7.1.5. The contractor shall analyze source data to perform Cyber Survivability Test Planning Purpose of Test development.

2.7.1.6. The contractor shall conduct a Cyber Survivability level of test review with Code 01D.

2.7.1.7. The contractor shall coordinate with Fleet Resources and the OTD/OTC to determine asset availability in support of Cyber Security objectives.

2.7.1.8. The contractor shall develop and provide a Cyber Survivability POA&M to support the test planning effort.

2.7.1.9. When directed, the contractor shall identify and provide risk management and recertification baseline concerns for communication to the Program Management Office (PMO).

2.7.1.10. The contractor shall document the test planning research in a Cyber Survivability TP document.

2.7.1.11. The contractor shall review the test planning research during Cyber Survivability reviews.

2.7.1.12. The contractor shall provide meeting minutes, to include any required comment resolution matrix for issues, to document the results of test planning research Cyber Survivability

2.7.2. Cyber SUT/SoS and Schedule Development

2.7.2.1. The Contractor shall perform Cyber Survivability SUT/SoS and Schedule Development in accordance with reference (h).

2.7.2.2. The contractor shall identify and document the system attack surface.

2.7.2.3. The contractor shall identify and document the system attack vectors.

2.7.2.4. The contractor shall develop and document Cyber Survivability TP site visit objectives.

2.7.2.5. The contractor shall develop and document the test execution schedule.

2.7.2.6. The contractor shall document the SUT/SoS and Schedule in a Cyber Survivability TP document.

2.7.2.7. The contractor shall review the SUT/SoS and Schedule during Cyber Survivability reviews.

2.7.2.8. The contractor shall provide meeting minutes, to include any required comment resolution matrix for issues, to document the results of SUT/SoS and Schedule Cyber Survivability

2.7.3. Cyber Survivability TP Site Visit

2.7.3.1. The Contractor shall perform Cyber Survivability TP Site Visit in accordance with reference (h).

2.7.3.2. The contractor shall coordinate dates, attendance lists and documentation requirements with the PMO…