Solicitation - N0018926RW008.pdf

WOMEN-OWNED SMALL

BUSINESS (WOSB)

SOLICITATION/CONTRACT/ORDER FOR COMMERCIAL PRODUCTS AND COMMERCIAL SERVICES

1. REQUISITION NUMBER PAGE 1 OF

2. CONTRACT NUMBER 3. AWARD/EFFECTIVE

DATE

4. ORDER NUMBER 5. SOLICITATION NUMBER 6. SOLICITATION ISSUE

DATE

7. FOR SOLICITATION

INFORMATION CALL:

a. NAME b. TELEPHONE NUMBER (No collect calls)

8. OFFER DUE DATE/

LOCAL TIME

9. ISSUED BY

13b. RATING

14. METHOD OF SOLICITATION

CODE

15. DELIVER TO 16. ADMINISTERED BY CODE

18a. PAYMENT WILL BE MADE BY CODE17a. CONTRACTOR/

OFFEROR

CODE

FACILITY

CODE

CODE

TELEPHONE NUMBER

17b. CHECK IF REMITTANCE IS DIFFERENT AND PUT SUCH ADDRESS IN

OFFER

18b. SUBMIT INVOICES TO ADDRESS SHOWN IN BLOCK 18a UNLESS BLOCK

BELOW IS CHECKED

REQUEST

FOR QUOTE

(RFQ)

INVITATION

FOR BID

(IFB)

REQUEST

FOR

PROPOSAL

(RFP)

SEE ADDENDUM

19.

ITEM NUMBER

20.

SCHEDULE OF SUPPLIES/SERVICES

21.

QUANTITY

22.

UNIT

23.

UNIT PRICE

24.

AMOUNT

(Use Reverse and/or Attach Additional Sheets as Necessary)

25. ACCOUNTING AND APPROPRIATION DATA 26. TOTAL AWARD AMOUNT (For Government Use Only)

28. CONTRACTOR IS REQUIRED TO SIGN THIS DOCUMENT AND RETURN

COPIES TO ISSUING OFFICE. CONTRACTOR AGREES TO FURNISH

AND DELIVER ALL ITEMS SET FORTH OR OTHERWISE IDENTIFIED ABOVE AND

ON ANY ADDITIONAL SHEETS SUBJECT TO THE TERMS AND CONDITIONS

SPECIFIED

29. AWARD OF CONTRACT: REFERENCE OFFER

DATED . . YOUR OFFER ON SOLICITATION

(BLOCK 5), INCLUDING ANY ADDITIONS OR CHANGES WHICH ARE

SET FORTH HEREIN, IS ACCEPTED AS TO ITEMS:

30a. SIGNATURE OF OFFEROR/CONTRACTOR

30b. NAME AND TITLE OF SIGNER (Type or print) 30c. DATE SIGNED

31a. UNITED STATES OF AMERICA (SIGNATURE OF CONTRACTING OFFICER)

31b. NAME OF CONTRACTING OFFICER (Type or print) 31c. DATE SIGNED

AUTHORIZED FOR LOCAL REPRODUCTION

PREVIOUS EDITION IS NOT USABLE

STANDARD FORM 1449 (REV. 11/2021)

Prescribed by GSA - FAR (48 CFR) 53.212

10. THIS ACQUISITION IS UNRESTRICTED OR

NORTH AMERICAN

INDUSTRY CLASSIFICATION

STANDARD (NAICS):

SIZE STANDARD:

13a. THIS CONTRACT IS A

RATED ORDER UNDER

THE DEFENSE PRIORITIES

AND ALLOCATIONS

SYSTEM - DPAS (15 CFR 700)

SET ASIDE: % FOR:

11. DELIVERY FOR FREE ON

BOARD (FOB) DESTINATION

UNLESS BLOCK IS MARKED

SEE SCHEDULE

12. DISCOUNT TERMS

ARE ARE NOT ATTACHED

ARE ARE NOT ATTACHED

27a. SOLICITATION INCORPORATES BY REFERENCE (FEDERAL ACQUISITION REGULATION) FAR 52.212-1, 52.212-4. FAR 52.212-3

AND 52.212-5 ARE ATTACHED. ADDENDA

27b. CONTRACT/PURCHASE ORDER INCORPORATES BY REFERENCE FAR 52.212-4. FAR 52.212-5 IS ATTACHED. ADDENDA

8(A)

ECONOMICALLY

DISADVANTAGED

WOMEN-OWNED SMALL

BUSINESS (EDWOSB)

SERVICE-DISABLED

VETERAN-OWNED

SMALL BUSINESS

(SDVOSB)

HUBZONE SMALL

BUSINESS

SMALL BUSINESS

NOTE: OFFEROR TO COMPLETE BLOCKS 12, 17, 23, 24, AND 30.

Solicitation/Contract Form Continuation

Undersea Warfare Division 40 Analytical Operational Test and Evaluation Support Services

N0018926RW008

Continuation of Supplies or Services and Prices/Costs

Additional Information/Notes

MINIMUM AND MAXIMUM QUANTITIES (FFP, SINGLE AWARD IDIQ) (SEP 2015)

As referred to in paragraph (b) of the "Indefinite Quantity" clause of this contract, the total contract minimum quantity is a total of $2,500.00 worth of orders at the contract unit price(s). The contract maximum quantity is a total of $___________.

LOCAL TEXT "UNLIQUIDATED OBLIGATIONS"

The Government intends to unilaterally deobilgate any unliquidated obligations 90 days after each period of performance. Prior to doing so, the Contracting Officer (or Administrative Contracting Officer) will notify the contractor of the intended amount of deobligation and ask the contractor to confirm within one week that all invoices for the completed period of performance have been submitted and processed for payment (with the exception of any invoice for rate adjustment upon receipt of DCAA approved final rates for any cost-type CLINs or SLINs). Upon confirmation that all invoices have been submitted and processed for payment, or upon the contractor's failure to respond to the PCO/ACOs notification of intent to deobligate, the Contracting Officer or Administrative Contracting Officer will unilaterally deobligate the potential ULOs by reducing the funded amount to an amount equal to the contractors invoiced/paid amount for that contract period of performance.

Note: Reduction of cost-reimbursement CLINs or SLINs in accordance with the above DOES NOT change either party's obligations under a Final Invoice submitted upon DCMA establishing final rates for each CLIN and or SLIN.

Allowed Per Order Minimum: 1.00 Dollars, U.S.

Government Minimum/Maximum to Order Minimum: 2,500.00 Dollars, U.S.

Item Supplies/Service Estimated Quantity

Unit Unit Price Amount

0001 Test and Evaluation Services

Pricing Arrangement: Firm Fixed Price

1 Unit

0002 Travel

Pricing Arrangement: Cost No Fee

1 Unit

Option Line Item

Option Period - Test and Evaluation Services

Pricing Arrangement: Firm Fixed Price

1 Unit

Option Line Item

Option Period - Travel

Pricing Arrangement: Cost No Fee

1 Unit

Continuation of Description

1.0 Scope. The Contractor shall provide technical and analytical support to Operational Test and Evaluation Force (OPTEVFOR) in the definition, conduct, and analysis of structured Operational Test and Evaluation (OT&E) of US Navy Undersea Warfare systems. This support will be required in one or more of the following forms: Documentation Review, Operational Test Design, Test Execution, Data Management, Test Planning Support, Test Operations Support, Test Analysis and Reporting, Modeling and Simulation Support, and Program Management Support. OPTEVFOR is charged with leading the Operational Test community with highly skilled testers and staff that adapt to change, and provide credible, prompt, warfighting-focused test results to Navy, Marine Corps, Coast Guard forces and acquisition leadership.

2.0 Background. There are five warfare divisions at OPTEVFOR that are supported by competency divisions. The warfare divisions include Undersea Warfare (40), Air Warfare (50), Command, Control, Communications, Computers, Intelligence, Surveillance, and Reconnaissance (C4ISR) (60), Surface Warfare (70), Expeditionary Warfare/Littoral Combat Ship (80) Division. Each warfare division typically has a Navy Captain as the division Director with a senior civil servant as the Deputy or a senior civil servant as the division Director and a Navy Commander as the Deputy.

The warfare divisions represent the traditional core of the OPTEVFOR organization. This is where the Fleet operators reside. The work there allows OPTEVFOR to bridge the gap from technical to the tactical, successfully. The warfare divisions are staffed by uniformed military personnel and Navy government civilians, augmented by a contractor workforce.

There are four competency divisions: Policy and Operations (01A), Test Design (01B), Test Planning and Analysis (01C), and Cybersecurity Testing (01D). In addition, the Technical Director (00TD) oversees the technical aspects of the Command's mission. Other support divisions include the Staff Commanding Officer and Administration (10), Chief Information Officer (CIO) (20), Contracts (01K), and the Comptroller (30). The members of the competency divisions generally work within the test team to ensure that the Director's policies are adhered to, and that Best Practice Series are applied.

Test Policy, Operations and Training (Code 01A) ensures OPTEVFOR policy conforms to Navy T&E Policy; monitors production of T&E products;

leads continuous process improvement efforts; coordinates external OT&E inquiries (IG and GAO); manages all government T&E training (including budget requests); provides T&E editorial support to the command; and plans, analyzes and evaluates the effectiveness of command processes.

Test Design and M&S (Code 01B) is responsible for the Mission Based Test Design (MBTD) process, M&S technical review, test planning and post-test iterative process analysis support, test design production, Capabilities Based Test and Evaluation coordination, and Targets, Ranges, and Test Resources coordination.

Test Planning and Analysis Division (Code 01C) is responsible for policy, processes and standardization for test planning, test execution, post-test analysis and evaluation (reporting). This includes providing Lead Test Engineer (LTE) support to the OTF Warfare Divisions and Fleet Resources coordination to support execution of OT&E.

Cybersecurity Division (Code O1D) integrates critical cybersecurity testing into the acquisition lifecycle and tests Systems under Test (SUT) per DoDI 5000.02 and DOT&E memo of 01 August 2014. The Under Sea Warfare division will be responsible for executing acquisition test design, planning support to ultimately provide the Cybersecurity OT&E group (O1D) support in cybersecurity execution and reporting. Through a rigorous and iterative test process, Cybersecurity OT&E assists in the development and fielding of more secure and resilient Information Technology (IT) systems supporting the warfighters. Code 01D Cybersecurity OT&E group includes the following support positions: Technical Director, Future Capabilities and Threat Emulation Developer, Cybersecurity Test Engineers, Test Design and Analysis Manager, Training and Lab Manager, Test Team Leads, and Certified Ethical Hackers.

3.0 Undersea Warfare Division and Supported Programs. The Undersea Warfare Division (USW) (Code 40) tests and evaluates all undersea warfare systems, including submarine, surface, aviation, and UUV anti-submarine warfare and mine warfare systems under the most operationally realistic conditions possible. The division's main effort is to assist, through early involvement, in the acquisition process, conduct tests, and provide timely and accurate evaluations to characterize system performance and resulting capability. Below is a list of current USW test and evaluation programs that require OT&E support; however, additional programs may be assigned to OPTEVFOR for contractor support under this contract. Current USW Test and Evaluation programs include, but are not limited to:

VIRGINIA Class SSN Program (SSN 774) COLUMBIA Class SSBN Program (SSBN 826) Mk 48 Common Broadband Advanced Sonar System (CBASS) Heavyweight Torpedo (HWT) Mk 54 Lightweight Hybrid Torpedo (LWT) Program Acoustic Device Countermeasure (ADC) MK 5 Compact Rapid Attack Weapon (CRAW) Submarine Launched Unmanned Aerial System (SLUAS) Hammerhead Encapsulated Effector (HHEE) Conventional Prompt Strike (CPS) Maritime Strike Tomahawk (MST) Integration Submarine Launched HARPOON Integration AN/SQQ-89 ASW Combat System Integration Surveillance Towed Array Sensor System (SURTASS), including Low Frequency Active Array, TL-29A Towed Array, and Integrated Common Processor SURTASS-Expeditionary (SURTASS-E) Auxiliary General Ocean Surveillance Ship (T-AGOS(X)) AN/BQQ-10(V) Sonar System, Acoustic Rapid COTS Insertion (A-RCI) AN/BYG-1 Combat Control System (CCS) (ECP4) TB-29 Reduced Length (RL) Array

TB-33 Thin Line Array TB-34 Fat Line Array Low Cost Conformal Array (LCCA) Large Vertical Array (LVA) Wide Aperture Array (WAA) Large Aperture Bow Array (LAB)/Spherical Array Common Submarine Radio Room (CSRR) Electronic Warfare Support (BLQ-10) OE-538/BRC Antenna Common Submarine Imaging System (CSIS) Low Band Universal Communications System (LBUCS)

XLUUV (ORCA)

LDUUV (Snakehead, Dragonfish) Razorback - Torpedo tube launched autonomous unmanned underwater vehicle Tetra - Torpedo tube launched remotely operated underwater vehicle Trident D5LE2 Strategic Weapon System Submarine Launched Cruise Missile - Nuclear (SLCM-N) Link 16 Tactical Data Link

4.0 Performance Period and Location.

4.1 Period of Performance. See schedule.

4.2 Place of Performance. Unless required to support mission accomplishment, work shall primarily be performed at the customer's site. If the number of personnel exceeds onsite capabilities, unclassified work will be expected to be performed at a contractor facility offsite. The government will not determine a situation where the contractor may utilize "Telework" as this is a professional services contract, so any work performed by the contractor away from the customer location will be upon the contractor to ensure all requirements are being met within the respective task orders issued under this contract. The customer location is:

Operational Test and Evaluation Force 7970 Diven Street Norfolk, VA 23505-1498

While onsite at OPTEVFOR, the contractor shall comply with OPNAVINST 5239 (series), with specific regard to non-government hardware and software. Normal workdays at OPTEVFOR are Monday through Friday and normal working hours are from 0730 through 1600 ET. The manned quarter deck watch will unlock the front and back doors to CA-10 at 0500 (M-F) and lock the front and back doors to CA-10 at 1830 (M-F). Any contractor who desires to remain past 1830 in CA-10 to perform services under their support contract MUST notify the quarterdeck CDO/watch stander NLT 1700 on the day they will be working past 1830. Once the contractor has secured all work and access in their respective workspace, they will notify the CDO/watch stander that they are securing for the day.

4.3 Travel. Travel can be expected to occur CONUS and OCONUS and will be required throughout the course of this contract. The need for travel associated with this PWS shall be determined in advance by the COR at the Task Order level and authorized by the COR. Travel shall be paid by the Contractor and reimbursed by the Government. Travel costs/expenses will be considered reasonable and allowable only to the extent that they do not exceed on a daily basis, the maximum per diem rates in effect at the time of the travel. Travel expenses include airfare, per diem, rental car, local travel and miscellaneous expenses as authorized by the Joint Travel Regulations (JTR). Costs for transportation may be based upon mileage rates, actual costs incurred or a combination thereof, provided the method used results in a reasonable charge. No mileage costs will be incurred when travel is within a 50-mile radius of the primary work location. The JTR, while not wholly applicable to Contractors, shall provide the basis for the determination as to reasonable and allowable. Maximum use is to be made of the lowest available customary standard coach or equivalent airfare accommodations available during normal business hours. Exceptions to these guidelines shall be approved in advance by the Contracting Officer or Designee. Travel dates and locations are to be confirmed through the TA and approved by the COR in writing via email, prior to commencement of any travel.

4.4 Observed Federal Holidays

Columbus Day Veterans Day Thanksgiving Christmas Day New Year's Day MLK Jr. Day / Inauguration Day Mon Washington's Birthday Mon Spring Holiday (MIL Only) Sun Memorial Day Juneteenth National Independence Day Independence Day Labor Day

4.5 Travel Instructions.

4.5.1 Any travel OCONUS by the Contractor will need to review the Foreign Clearance Guide at http://www.fcg.pentagon.mil/fcg.cfm and select the country of intended travel. During OCONUS travel, the contractor shall operate in accordance with International Traffic in Arms Regulation (ITAR) Part 124, approved by US Department of State, Office of Defense Trade Controls. Contractor is not authorized to perform any travel that is not in conjunction with this contract. It is the Department of the Navy (DON) policy not to allow a charge of profit or fee on reimbursable items.

4.5.2 The Government will make no reimbursements for a contractor's local travel (defined as less than 50 miles from the place of performance).

4.5.3 In no instance shall the contractor exceed the amount specified in the contract for travel CLIN expenses.

4.5.4 Locations include CONUS, OCONUS and at sea locations and any other location, which may be added on a task order basis based on future test requirements.

4.5.5 For planning purposes, any travel location OCONUS must be considered as a possible travel location unless travel restrictions at time of test do not permit entry. Possible OCONUS locations include, but are not limited to the following: Andros Island, Bahamas, Darwin, Brisbane, Melbourne, Newcastle, Perth, and Sydney, Australia; London, England; Honduras, Singapore; La Spezia, Italy; Guam; Hawaii; Halifax, Nova Scotia; Nanoose, British Colombia; Columbia; Belize; Guatemala; Souda Bay, Crete; Mexico; and any other location, which may be added on a task order basis based on future test requirements.

5.0 Specific Tasking. The contractor shall provide support in evaluating the operational effectiveness and suitability of the system under test that has been assigned to OPTEVFOR for OT&E in accordance with the processes and procedures prescribed in OPTEVFOR M-3980.2 (series), the Operational Test and Evaluation Manual. This manual and other governing OT&E references are listed in paragraph 12 and are available from the OT Resource Library on the OPTEVFOR Sharepoint site:

https://flankspeed.sharepoint-mil.us/sites/OPTEVFOR/SitePages/01A-Division-OT-Resources.aspx.

5.1 All government furnished Non-Secure Internet Protocol Router (NIPR) templates to be used by the contractor shall only be obtained from the authoritative source, the OT Resource Library on the OPTEVFOR Sharepoint site https://flankspeed.sharepoint-mil.us/sites/OPTEVFOR/SitePages/01A-Division-OT-Resources.aspx.

5.2 All government furnished Secure Internet Protocol Router (SIPR) templates to be used by the contractor shall only be obtained from the authoritative source, the OPTEVFOR SIPR Share Drive:

Y: / OT&E Production Library.

5.3 The contractor shall sign up for the OPTEVFOR "Alerts for OT Resource Changes" to receive updates (which can be tailored on what you receive) on all instructions, Best Practices, and templates. Instructions to sign up are found on the OPTEVFOR SharePoint page under 01A - Operations, OT Instructions and Guidance:

https://flankspeed.sharepoint-mil.us/sites/OPTEVFOR/SitePages/01A-Division-OT-Resources.aspx

5.4 The relevant process that requires the use of government furnished templates are (but not limited to):

a. Test Design o LTD Brief o LTD Letter o IEF and TIEF o IEF Change Letter o SWP TIEF Approval Letter o E-IPR Brief o RCRM

b. Test Strategy o TEMP or MTS Letter (O6 Comment, Signature, Change) o TECP Brief

c. Test Planning o Test Plan (IOT&E, FOT&E, OA/EOA, QRA, DCP, VCD) o Concept of Test Brief o Cyber Survivability Test Plan o CP0 Brief o CP2 Brief

d. Post-Test Iterative Process (PTIP) o Blue & Gold Sheets o Report Letter (IOT&E, FOT&E, OA/EOA, QRA) o SERB/E-SERB Brief o Letter of Observation (LOO) o Assessment of Operational Capability (AOC)

5.5 Overall Task Guidance. To gain efficiencies and synergy in contract execution, the Government encourages the Contractor to consider:

5.5.1 Flexibility in production. OPTEVFOR produces information which may take many forms. Support teams should be willing to experiment, bring up new ideas or better ways of doing things, but remain willing to execute stated processes as required by the contract and/or task order.

5.5.2 Present concerns about the validity of the information produced and value to the end customer (Fleet or Program). Remain willing to defend a contrary position until government makes a final decision.

5.5.3 Remain efficient and economical in production. Strive to produce information as quickly as possible without sacrificing quality, and with the appropriate level of expertise.

5.5.4 Maintain a broad perspective. Awareness of testing outside of a specific program to recommend testing efficiencies and overall kill chain evaluation opportunities.

5.5.5 Technical competence. Improve T&E community's ability to understand, analyze, communicate, and ultimately visualize T&E issues from a warfighter and technical perspective.

5.6 Mission Based Test Design.

5.6.1 Mission Analysis.

5.6.1.1 The Contractor shall perform Mission Based Test Design (MBTD) mission analysis in accordance with the Integrated Evaluation Framework

(IEF) Checklist and IEF Template, respectively.

5.6.1.2 The contractor shall analyze source data to perform Mission Based Test Design mission analysis.

5.6.1.3 The contractor shall identify and document the mission areas.

5.6.1.4 The contractor shall identify and document system capabilities.

5.6.1.5 The contractor shall derive and document operator tasks.

5.6.1.6 The contractor shall define and document the operational conditions applicable to the system to be tested.

5.6.1.7 The contractor shall document the mission analysis in the Mission Based Test and Evaluation System (MBTES).

5.6.1.8 The contractor shall document the mission analysis in an IEF document.

5.6.1.9 The contractor shall review the mission analysis during MBTD reviews.

5.6.1.10 The contractor shall provide meeting minutes, to include any required comment resolution matrix for issues, to document the results of mission analysis MBTD reviews.

5.6.2 Requirements Analysis.

5.6.2.1 The contractor shall perform Mission Based Test Design requirements analysis in accordance with the IEF Checklist and IEF Template, respectively.

5.6.2.2 The contractor shall analyze source data to perform Mission Based Test Design requirements analysis.

5.6.2.3 The contractor shall identify and document measures of system performance, inclusive of critical measures.

5.6.2.4 The contractor shall link measures of system performance to subtasks and document this linkage.

5.6.2.5 The contractor shall identify and document the data required to quantify the measures of system performance.

5.6.2.6 The contractor shall identify and document the data required to quantify the operational conditions applicable to the system to be tested.

5.6.2.7 The contractor shall document requirements analysis in the MBTES.

5.6.2.8 The contractor shall review the requirements analysis during MBTD reviews.

5.6.2.9 The contractor shall provide meeting minutes, to include any required comment resolution matrix for issues, to document the results of requirements analysis MBTD reviews.

5.6.3 Test Design.

5.6.3.1 The Contractor shall perform Mission Based Test Design test design in accordance with the IEF Checklist and IEF Template, respectively.

5.6.3.2 The contractor shall analyze source data to perform Mission Based Test Design.

5.6.3.3 The contractor shall identify and document the candidate response variables for statistical test design.

5.6.3.4 The contractor shall identify and document factors/conditions (with corresponding levels) associated with each response variable or critical measure used in test design.

5.6.3.5 The contractor shall participate in and document the statistical test design for candidate response variables and/or critical measures.

5.6.3.6 The contractor shall identify and document the test vignettes.

5.6.3.7 The contractor shall determine and document the appropriate test method.

5.6.3.8 The contractor shall determine and document a test schedule supporting execution of the test design.

5.6.3.9 The contractor shall identify and document the expected use of Modeling and Simulation in the execution of the test design.

5.6.3.10 The contractor shall determine and document the resources required to conduct the test.

5.6.3.11 The contractor shall determine and document the limitations to test based on known availability of resources and assets to conduct the test.

5.6.3.12 The contractor shall develop and document data sheets to support test execution.

5.6.3.13 The contractor shall document the test design in the MBTES.

5.6.3.14 The contractor shall provide meeting minutes, to include any required comment resolution matrix for issues, to document the results of test design MBTD reviews.

5.7 Stakeholder Collaboration (Working Integrated Product Teams (WIPTs), outreach, etc.).

5.7.1 The contractor shall make recommendations to the warfare division in accordance with OPTEVFOR processes found in the OT&E Manual and IEF Checklist.

5.7.2 The contractor shall identify and recommend outside organization(s) for participation and inclusion in Test Design activities.

5.7.3 The contractor shall coordinate briefings/meetings in support of OPTEVFOR TEMP Input activities in accordance with IEF Checklist and IEF Template.

5.7.4 The contractor shall participate in briefings/meetings in support of OPTEVFOR TEMP Input activities in accordance with IEF Checklist and IEF Template.

5.7.5 The contractor shall lead briefings/meetings in support of OPTEVFOR TEMP Input activities in accordance with IEF Checklist and IEF Template.

5.7.6 The contractor shall provide a trip report (DELIVERABLE A002) when contract activities occur away from the OPTEVFOR site in Norfolk, VA.

5.8 Document and Brief Preparation. The contractor shall prepare and document briefings (DELIVERABLE A001) to include the Integrated Evaluation Framework, Executive In-Process Review and stakeholder collaboration briefings.

5.9 Test and Evaluation Master Plan (TEMP) Input Preparation.

5.9.1 The contractor shall analyze source data to provide input to the TEMP, to include program schedules, the Integrated Evaluation Framework, applicable policy documents and any other program-provided materials.

5.9.2 The contractor shall develop and document input to the TEMP in accordance with the DOT&E TEMP Guidebook.

5.9.3 The contractor shall develop and document comments to the TEMP related to or affecting OT.

5.9.4 The contractor shall provide a trip report (DELIVERABLE A002) when contract activities occur away from the OPTEVFOR site in Norfolk, VA.

5.10 Test Planning.

5.10.1 Plan of Action and Milestones (POA&M).

5.10.1.1 The contractor shall develop and document a POA&M for the Test Planning process in accordance with the Test Planning Checklist.

5.10.1.2 The contractor shall analyze source data to perform Test Planning POA&M development.

5.10.1.3 The contractor shall obtain and maintain access to the program MBTES database to support the test planning process.

5.10.2 Purpose of Test.

5.10.2.1 The contractor shall perform the activities associated with development of Purpose of Test in accordance with the Test Planning Checklist.

5.10.2.2 The contractor shall analyze source data to perform Test Planning Purpose of Test development.

5.10.2.3 The contractor shall update and document the System Under Test (SUT) and System of Systems (SoS) descriptions from the Integrated Evaluation Framework for use in the Test Plan.

5.10.2.4 The contractor shall identify and document the SUT test configuration.

5.10.2.5 The contractor shall identify and document the Developmental Test (DT)/Integrated Test (IT) contributions to OT.

5.10.2.6 The contractor shall identify and document regression test requirements for any Follow-on OT&E phases, if applicable.

5.10.2.7 The contractor shall identify and document changes to the limitations to test.

5.10.2.8 The contractor shall update and document the Mission Critical Subsystem Matrix (MCSM) from the Integrated Evaluation Framework.

5.10.2.9 The contractor shall document the Purpose of Test activities in a Test Plan document in accordance with the Test Planning Checklist, and the appropriate test plan templates.

5.10.2.10 The contractor shall review the Purpose of Test activities during test plan reviews.

5.10.2.11 The contractor shall provide meeting minutes, to include any required comment resolution matrix for issues, to document the results of Purpose of Test reviews.

5.10.3 Test Schedule.

5.10.3.1 The contractor shall perform the activities associated with development of the Test Schedule in accordance with the Test Planning Checklist.

5.10.3.2 The contractor shall analyze source data to perform Test Schedule development.

5.10.3.3 The contractor shall identify, recommend, and document data to be qualified for use in OT.

5.10.3.4 The contractor shall develop and document the Test Schedule, to include test events, for each test phase and period.

5.10.3.5 The contractor shall document the Test Schedule data in the MBTES.

5.10.3.6 The contractor shall document the Test Schedule activities in a Test Plan document in accordance with Test Planning Checklist and the appropriate test plan templates.

5.10.3.7 The contractor shall review the Test Schedule activities during test plan reviews.

5.10.3.8 The contractor shall provide meeting minutes, to include any required comment resolution matrix for issues, to document the results of Test Schedule reviews.

5.10.4 Detailed Method of Test.

5.10.4.1 The contractor shall perform the activities associated with development of the Detailed Method of Test in accordance with the Test Planning Checklist.

5.10.4.2 The contractor shall analyze source data to perform Detailed Method of Test development.

5.10.4.3 The contractor shall identify and document the data to be collected on each test event.

5.10.4.4 The contractor shall develop and document the start and stop criteria for each test event.

5.10.4.5 The contractor shall develop and document the GO/NO-GO criteria for each test event.

5.10.4.6 The contractor shall develop and document the Detailed Method of Test for each test event.

5.10.4.7 The contractor shall document the Detailed Method of Test in a Test Plan document in accordance with Test Planning Checklist and the appropriate test plan templates.

5.10.4.8 The contractor shall review the Detailed Method of Test during test plan reviews.

5.10.4.9 The contractor shall provide meeting minutes, to include any required comment resolution matrix for issues, to document the results of Detailed Method of Test reviews.

5.10.5 Data Collection Plan (DCP).

5.10.5.1 The contractor shall perform the activities associated with development of the Data Collection Plan in accordance with the Test Planning Checklist.

5.10.5.2 The contractor shall analyze source data to perform Data Collection Plan development.

5.10.5.3 The contractor shall review, update and document, as required, any changes to data requirements.

5.10.5.4 The contractor shall develop and document the Data Collection Plan.

5.10.5.5 The contractor shall identify test equipment/instrumentation needed for test data collection.

5.10.5.6 The contractor shall identify data repository.

5.10.5.7 The contractor shall create and document data sheets, interview sheets, focus group sheets, and survey sheets in support of the Data Collection Plan.

5.10.5.8 The contractor shall document the Data Collection Plan data in the MBTES.

5.10.5.9 The contractor shall document the Data Collection Plan in a Test Plan document in accordance with Test Planning Checklist and the appropriate test plan templates.

5.10.5.10 The contractor shall review the Data Collection Plan during test plan reviews.

5.10.5.11 The contractor shall provide meeting minutes, to include any required comment resolution matrix for issues, to document the results of Data Collection Plan reviews.

5.10.6 Data Analysis Plan (DAP).

5.10.6.1 The contractor shall perform the activities associated with the development of the Data Analysis Plan in accordance with the Test Planning Checklist and the appropriate test plan templates.

5.10.6.2 The contractor shall analyze source data to perform Data Analysis Plan development.

5.10.6.3 The contractor will develop and document the COI resolution method.

5.10.6.4 The contractor will develop and document data scoring criteria.

5.10.6.5 The contractor shall identify and document Scoring Board Guidance.

5.10.6.6 The contractor shall provide a data analysis repository with Government concurrence.

5.10.6.7 The contractor shall document the Data Analysis Plan in a Test Plan document in accordance with Test Planning Checklist and the appropriate test plan templates.

5.10.6.8 The contractor shall review the Data Analysis Plan during test plan reviews.

5.10.6.9 The contractor shall provide meeting minutes, to include any required comment resolution matrix for issues, to document the results of Data Analysis Plan reviews.

5.10.7 Stakeholder Collaboration (WIPTs, outreach, etc).

5.10.7.1 The contractor shall make recommendations to the warfare division in accordance with OPTEVFOR processes found in the OT&E Manual, Test Planning Checklist, and test plan templates.

5.10.7.2 The contractor shall identify and recommend outside organization(s) for participation and inclusion in Test Planning activities.

5.10.7.3 The contractor shall coordinate briefings/meetings in support of OPTEVFOR Test Planning activities in accordance with Test Planning Checklist and the appropriate test plan templates.

5.10.7.4 The contractor shall participate in briefings/meetings in support of OPTEVFOR Test Planning activities in accordance with Test Planning Checklist and the appropriate test plan templates.

5.10.7.5 The contractor shall lead briefings/meetings in support of OPTEVFOR Test Planning activities in accordance with Test Planning Checklist and the appropriate test plan templates.

5.10.7.6 The contractor shall provide a trip report (DELIVERABLE A002) when contract activities occur away from the OPTEVFOR site in Norfolk, VA.

5.10.8 Document and Brief Preparation. The contractor shall prepare and document briefings (DELIVERABLE A001) to include the Test Plan, Data Collection Plan, Concept of Test, Test Plan Review Board, and stakeholder collaboration briefings.

5.11 Modeling and Simulation (M&S) Support.

5.11.1 M&S Requirements Development.

5.11.1.1 The Contractor shall perform M&S Requirements Development in accordance with OPTEVFORINST 5000.1 (series) and applicable Best Practices.

5.11.1.2 The contractor shall identify and document candidate critical measures/response variables, along with their associated conditions, during the Test Design portion of the MBTD process that will be supplemented by M&S data.

5.11.1.3 The contractor shall develop and document a M&S Requirements Letter in accordance with the M&S Requirements Letter template and applicable Best Practices.

5.11.1.4 The contractor shall review the M&S requirements during MBTD reviews.

5.11.1.5 The contractor shall provide meeting minutes, to include any required comment resolution matrix for issues, to document the results of M&S Requirements Development.

5.11.2 Develop Accreditation Plans.

5.11.2.1 The contractor shall develop an M&S Accreditation Plan in accordance with OPTEVFORINST 5000.1 (series), the Accreditation Plan template, and applicable Best Practices.

5.11.2.2 The contractor shall develop and document the accreditation approach method consistent with OPTEVFOR policy.

5.11.2.3 The contractor shall develop and document the Accreditation Plan.

5.11.2.4 The contractor shall provide meeting minutes, to include any required comment resolution matrix for issues, to document the results of M&S Accreditation Plan Development.

5.11.3 Review Verification & Validation (V&V) Plans and Reports.

5.11.3.1 The contractor shall review program M&S V&V plans and reports in accordance with OPTEVFORINST 5000.1 (series) and applicable Best Practice.

5.11.3.2 The contractor shall review and document disagreements or issues with V&V plan methodologies.

5.11.3.3 The contractor shall provide recommendations for alternative V&V methodologies supporting OPTEVFOR Accreditation Requirements.

5.11.3.4 The contractor shall review program M&S V&V reports and document disagreements or issues with methodologies/results based on comparison with live test results.

5.11.3.5 The contractor shall develop and document the OPTEVFOR V&V Report addendum.

5.11.3.6 The contractor shall provide meeting minutes, to include any required comment resolution matrix for issues, to document the results of M&S V&V Plans or Reports reviews.

5.11.4 Accreditation Support.

5.11.4.1 The contractor shall perform M&S Accreditation in accordance with OPTEVFORINST 5000.1 (series) and applicable Best Practices.

5.11.4.2 The contractor shall perform and document, in coordination with OPTEVFOR Test Design analysts, any additional required analysis not provided in the V&V report in support of accreditation.

5.11.4.3 The contractor shall identify, and document conclusions and recommendations based on comparison of live test results with M&S test results.

5.11.4.4 The contractor shall identify and document significant M&S limitations that impact the accreditation recommendation.

5.11.4.5 The contractor shall identify and document where the M&S results support the intended use for OT.

5.11.4.6 The contractor shall develop and document an accreditation recommendation in an Accreditation Letter in accordance with OPTEVFORINST 5000.1 (series), the Accreditation Letter template, and applicable Best Practices.

5.11.4.7 The contractor shall provide meeting minutes, to include any required comment resolution matrix for issues, to document the results of M&S Accreditation activities and reviews.

5.11.5 Stakeholder Collaboration.

5.11.5.1 The contractor shall make recommendations to the warfare division in accordance with OPTEVFOR processes found in OPTEVFORINST

5000.1 (series) and applicable Best Practices.

5.11.5.2 The contractor shall identify and recommend outside organization(s) for participation and inclusion in M&S VV&A activities.

5.11.5.3 The contractor shall coordinate briefings/meetings in support of OPTEVFOR M&S VV&A activities in accordance with OPTEVFORINST

5000.1 (series) and applicable Best Practices.

5.11.5.4 The contractor shall participate in briefings/meetings in support of OPTEVFOR M&S VV&A activities in accordance with OPTEVFORINST 5000.1 (series) and applicable Best Practices.

5.11.5.5 The contractor shall lead briefings/meetings in support of OPTEVFOR M&S VV&A activities in accordance with OPTEVFORINST 5000.1

(series) and applicable Best Practices.

5.11.5.6 The contractor shall provide a trip report (DELIVERABLE A002) when contract activities occur away from the OPTEVFOR site in Norfolk, VA.

5.11.6 Document and Brief Preparation. The contractor shall prepare and document briefings (DELIVERABLE A001) to include the M&S Requirements Letter, Accreditation Plan, Verification and Validation Report Addendum, Accreditation Letter, and stakeholder collaboration briefings.

5.12 Test Execution.

5.12.1 Pre-test Coordination.

5.12.1.1 The contractor shall identify, and schedule required test services (ranges, targets, personnel, etc.) for the test phase.

5.12.1.2 The contractor shall develop and submit asset scheduling requests to OPTEVFOR Fleet Services schedulers in accordance with the OT&E Manual.

5.12.1.3 The contractor shall develop and provide the OPTEVFOR pre-test letters for Commanding Officers of test units and Immediate Superior in Command (ISIC) in accordance with the OT&E Manual.

5.12.1.4 The contractor shall develop and provide briefing materials as required to support Operational Test Readiness Reviews (OTRR) in accordance with SECNAVINST 5000.2 (series).

5.12.2 Pre-test Brief.

5.12.2.1 The contractor shall prepare and document the Pre-Test briefing to support stakeholder alignment in execution preparations.

5.12.3 Test Operations.

5.12.3.1 The contractor shall collect and document test data using data cards/sheets, interview sheets, focus group sheets, and survey sheets in accordance with OPTEVFOR procedures and the signed test plan.

5.12.3.2 The contractor shall distribute text execution materials to all test participants.

5.12.3.3 The contractor shall develop and document any test deviations resulting from test variances.

5.12.3.4 The contractor shall develop and document a daily summary of test operations email.

5.12.4 Post-Test Activities.

5.12.4.1 The contractor shall conduct and document post-test event interviews.

5.12.4.2 The contractor shall conduct and document test team debriefs.

5.12.4.3 The contractor shall conduct and document test-site scoring boards.

5.13 Post-Test Iterative Process (PTIP) and Reporting.

5.13.1 Data Scoring.

5.13.1.1 The contractor shall perform PTIP Data Scoring in accordance with the Test Reporting Handbook, and applicable Best Practices.

5.13.1.2 The contractor shall inventory, compile, and document all collected data, comparing data collected to that required by the Test Plan, specifically the DMOT and DCP.

5.13.1.3 The contractor shall develop and document a test execution summary.

5.13.1.4 The contractor shall review the data during Data Scoring reviews.

5.13.1.5 The contractor shall develop the End of Test message.

5.13.1.6 The contractor shall provide meeting minutes, to include any required comment resolution matrix for issues, to document the results of Data Scoring reviews.

5.13.2 COI Evaluation.

5.13.2.1 The contractor shall perform PTIP COI Evaluation in accordance with the Test Reporting Handbook and applicable Best Practices.

5.13.2.2 The contractor shall develop and document risks and/or deficiencies using OPTEVFOR Blue and Gold Sheet templates.

5.13.2.3 The contractor shall develop and document test results in OPTEVFOR test report results paragraph format.

5.13.2.4 The contractor shall document any limitations to test identified during test execution or update any identified in test planning and realized during test execution.

5.13.2.5 The contractor shall develop and document COI evaluation via the Platform Mission Task (PMT) View.

5.13.2.6 The contractor shall provide meeting minutes, to include any required comment resolution matrix for issues, to document the results of COI Evaluation reviews.

5.13.3 Data Analysis.

5.13.3.1 The contractor shall perform PTIP Data Analysis in accordance with the Test Reporting Handbook .

5.13.3.2 The contractor shall conduct and document an analysis of all measures.

5.13.3.3 The contractor shall develop and conduct an RV Analysis Out brief in coordination with Test Design Analyst support.

5.13.3.4 The contractor shall develop a draft Data Analysis Summary (DAS).

5.13.3.5 The contractor shall review the DAS during Analysis Working Groups (AWG).

5.13.3.6 The contractor shall provide meeting minutes, to include any required comment resolution matrix for issues, to document the results of Data Analysis reviews.

5.13.4 Deficiency/Risk Documentation.

5.13.4.1 The contractor shall perform PTIP Deficiency/Risk Documentation in accordance with the Test Reporting Handbook and applicable Best Practices.

5.13.4.2 The contractor shall develop, distribute, and collect comments on draft Blue/Gold Sheets.

5.13.4.3 The contractor shall review the Deficiencies/Risks during PTIP reviews.

5.13.4.4 The contractor shall provide meeting minutes, to include any required comment resolution matrix for issues, to document the results of Deficiency/Risk reviews.

5.13.5 System Evaluation.

5.13.5.1 The contractor shall perform PTIP System Evaluation in accordance with the Test Reporting Handbook and applicable Best Practices .

5.13.5.2 The contractor shall develop and document evaluation reports to support Command Systems Evaluation Review Board (SERB)/Executive SERB (E-SERB) PTIP reviews.

5.13.5.3 The contractor shall provide Divisional Summary Briefs.

5.13.5.4 The contractor shall provide post-test Fact Sheet draft.

5.13.5.5 The contractor shall provide post-test Fleet Brief draft.

5.13.5.6 The contractor shall provide Lessons Learned.

5.13.5.7 The contractor shall review the system evaluation during PTIP reviews.

5.13.5.8 The contractor shall provide meeting minutes, to include any required comment resolution matrix for issues, to document the results of System Evaluation reviews.

5.13.6 Stakeholder Collaboration.

5.13.6.1 The contractor shall make recommendations to the warfare division in accordance with OPTEVFOR processes found in the Test Reporting Handbook, PTIP Checklist, and applicable Best Practices.

5.13.6.2 The contractor shall identify and recommend outside organization(s) for participation and inclusion in PTIP activities.

5.13.6.3 The contractor shall coordinate briefings/meetings in support of OPTEVFOR PTIP activities in accordance with the PTIP Checklist and the appropriate test report templates.

5.13.6.4 The contractor shall participate in briefings/meetings in support of OPTEVFOR PTIP activities in accordance with the PTIP Checklist and the appropriate test report templates.

5.13.6.5 The contractor shall lead briefings/meetings in support of OPTEVFOR PTIP activities in accordance with the PTIP Checklist and the appropriate test report templates.

5.13.6.6 The contractor shall provide a trip report (DELIVERABLE A002) when contract activities occur away from the OPTEVFOR site in Norfolk, VA.

5.13.7 Document and Brief Preparation

5.13.7.1 The contractor shall prepare and document briefings (DELIVERABLE A001) to include the Evaluation Report, Post-Test Brief, Scoring Board, Critical Operational Issue (COI) Evaluation Working Group (CEWG) products, SERB/E-SERB Brief, Data Analysis Summary, and Blue /Gold Sheets.

5.13.7.2 The contractor shall prepare the Risk/Deficiency cover letter, the DAS, and the Test Report per the OT&E Manual, Test Reporting Handbook, and applicable Best Practices.

5.14 Cyber Security Subject Matter Expert/Operational (SME).

5.14.1 Cyber Security Support.

5.14.1.1 IAW the DoD Cybersecurity Test and Evaluation Guidebook 2.0 Series, the contractor shall utilize paragraph F.2.1.2, Cybersecurity Subject Matter Experts and Figure F-2 RASCI Table to assist the OTD and 01D Exploitation Analyst (EA) in cyber test preparations and coordination.

5.14.2 Cyber Survivability Test Planning.

5.14.2.1 Cyber Survivability Test Planning Research

5.14.2.2 The Contractor shall perform Cyber Survivability Test Planning Research in accordance with Cyber Test Planning Process guidance.

5.14.2.3 The contractor shall review and identify technical documents required to support Cyber Survivability Test Planning.

5.14.2.4 The contractor shall identify need dates for all required technical documents and provide dates to the OTD/OTC.

5.14.2.5 The contractor shall identify previous and planned Cyber Survivability tests that may contribute to current Cyber Survivability efforts.

5.14.2.6 The contractor shall conduct a Cyber Survivability level of test review with Warfare Division and Code 01D.

5.14.2.7 The contractor shall coordinate with Fleet Resources and the OTD/OTC to determine asset availability in support of Cyber Security objectives.

5.14.2.8 The contractor shall develop and provide a Cyber Survivability POA&M to support the test planning effort.

5.14.2.9 The contractor shall identify and provide risk management and recertification baseline concerns for communication to the Program Management Office (PMO).

5.14.2.10 The contractor shall document the test planning research in a Cyber Survivability Test Plan document.

5.14.2.11 The contractor shall review the test planning research during Cyber Survivability reviews.

5.14.2.12 The contractor shall provide meeting minutes, to include any required comment resolution matrix for issues, to document the results of test planning research during Cyber Survivability Test Planning reviews.

5.14.3 Cyber SUT/SoS and Schedule Development.

5.14.3.1 The Contractor shall perform Cyber Survivability SUT/SoS and Schedule Development in accordance with Cyber Test Planning Process guidance.

5.14.3.2 The contractor shall identify and document the system attack surface.

5.14.3.3 The contractor shall identify and document the system attack vectors.

5.14.3.4 The contractor shall develop and document Cyber Survivability Test Plan site visit objectives.

5.14.3.5 The contractor shall develop and document the test execution schedule.

5.14.3.6 The contractor shall document the SUT/SoS and Schedule in a Cyber Survivability Test Plan document.

5.14.3.7 The contractor shall review the SUT/SoS and Schedule during Cyber Survivability reviews.

5.14.3.8 The contractor shall provide meeting minutes, to include any required comment resolution matrix for issues, to document the results of SUT /SoS and Schedule during Cyber Survivability Test Planning reviews.

5.14.4 Cyber Survivability Test Plan Site Visit.

5.14.4.1 The contractor shall perform Cyber Survivability Test Plan Site Visit in accordance with Cyber Test Planning Process guidance.

5.14.4.2 The contractor shall coordinate dates, attendance lists, and documentation requirements with the PMO prior to execution of the site visit.

5.14.4.3 The contractor shall develop and conduct an informal site visit in-brief covering the purpose of the site visit and what the site visit personnel should expect.

5.14.4.4 The contractor shall collect and document information to validate initial elements of the critical components list, system configuration differences from provided documentation, and technical SME support requirements for test execution.

5.14.4.5 The contractor shall provide a post-visit trip report (DELIVERABLE A002) documenting site visit activities and actions when contract activities occur away from the OPTEVFOR site in Norfolk, VA.

5.14.5 Cyber Survivability Final Technical Planning.

5.14.5.1 The Contractor shall perform Cyber Survivability Final Technical Planning in accordance with Cyber Test Planning Process guidance.

5.14.5.2 The contractor shall develop and document the preliminary set of anticipated attack surfaces and vectors to be used during test execution.

5.14.5.3 The contractor shall develop and document the final Cyber Survivability Critical Components list to be used for test execution.

5.14.5.4 The contractor shall develop and document the Cyber Survivability test objectives.

5.14.5.5 The contractor shall develop and document any Cyber Survivability test limitations, with corresponding mitigations.

5.14.5.6 The contractor will collect and incorporate go/no-go criteria from Code 01D test personnel.

5.14.5.7 The contractor shall document the final technical planning in a Cyber Survivability Test Plan document.

5.14.5.8 The contractor shall review the final technical planning during Cyber Survivability reviews.

5.14.5.9 The contractor shall provide meeting minutes, to include any required comment resolution matrix for issues, to document the results of final technical planning during Cyber Survivability reviews.

5.14.6 Cyber Survivability Test Plan Finalization.

5.14.6.1 The Contractor shall perform Cyber Survivability Test Plan Finalization in accordance with Cyber Test Planning Process guidance.

5.14.6.2 The contractor shall develop and document data requirements for test execution.

5.14.6.3 The contractor shall coordinate with Code 01D to determine test equipment and required resources for…