HC104718R4016_0002.pdf

AMENDMENT OF SOLICITATION/MODIFICATION OF CONTRACT

Except as provided herein, all terms and conditions of the document referenced in Item 9A or 10A, as heretofore changed, remains unchanged and in full force and effect.

15A. NAME AND TITLE OF SIGNER (Type or print)

30-105-04EXCEPTION TO SF 30

APPROVED BY OIRM 11-84

STANDARD FORM 30 (Rev. 10-83)

Prescribed by GSA

FAR (48 CFR) 53.243

The purpose of this amendment is to revise one of the responses to the Q&A's received and corresponding PWS reference (PWS 6.1.1

Subtask 1).

1. CONTRACT ID CODE PAGE OF PAGES

S 1 31

16A. NAME AND TITLE OF CONTRACTING OFFICER (Type or print)

16C. DATE SIGNED

BY 20-Aug-2018

16B. UNITED STATES OF AMERICA15C. DATE SIGNED15B. CONTRACTOR/OFFEROR

(Signature of Contracting Officer)(Signature of person authorized to sign)

8. NAME AND ADDRESS OF CONTRACTOR (No., Street, County, State and Zip Code) X HC104718R4016

X 9B. DATED (SEE ITEM 11)

26-Jul-2018

10B. DATED (SEE ITEM 13)

9A. AMENDMENT OF SOLICITATION NO.

11. THIS ITEM ONLY APPLIES TO AMENDMENTS OF SOLICITATIONS

X The above numbered solicitation is amended as set forth in Item 14. The hour and date specified for receipt of Offer is extended, is not extended.

Offer must acknowledge receipt of this amendment prior to the hour and date specified in the solicitation or as amended by one of the following methods:

(a) By completing Items 8 and 15, and returning copies of the amendment; (b) By acknowledging receipt of this amendment on each copy of the offer submitted;

or (c) By separate letter or telegram which includes a reference to the solicitation and amendment numbers. FAILURE OF YOUR ACKNOWLEDGMENT TO BE

RECEIVED AT THE PLACE DESIGNATED FOR THE RECEIPT OF OFFERS PRIOR TO THE HOUR AND DATE SPECIFIED MAY RESULT IN

REJECTION OF YOUR OFFER. If by virtue of this amendment you desire to change an offer already submitted, such change may be made by telegram or letter, provided each telegram or letter makes reference to the solicitation and this amendment, and is received prior to the opening hour and date specified.

12. ACCOUNTING AND APPROPRIATION DATA (If required)

13. THIS ITEM APPLIES ONLY TO MODIFICATIONS OF CONTRACTS/ORDERS.

IT MODIFIES THE CONTRACT/ORDER NO. AS DESCRIBED IN ITEM 14.

A. THIS CHANGE ORDER IS ISSUED PURSUANT TO: (Specify authority) THE CHANGES SET FORTH IN ITEM 14 ARE MADE IN THE

CONTRACT ORDER NO. IN ITEM 10A.

B. THE ABOVE NUMBERED CONTRACT/ORDER IS MODIFIED TO REFLECT THE ADMINISTRATIVE CHANGES (such as changes in paying office, appropriation date, etc.) SET FORTH IN ITEM 14, PURSUANT TO THE AUTHORITY OF FAR 43.103(B).

C. THIS SUPPLEMENTAL AGREEMENT IS ENTERED INTO PURSUANT TO AUTHORITY OF:

D. OTHER (Specify type of modification and authority)

E. IMPORTANT: Contractor is not, is required to sign this document and return copies to the issuing office.

14. DESCRIPTION OF AMENDMENT/MODIFICATION (Organized by UCF section headings, including solicitation/contract subject matter where feasible.)

10A. MOD. OF CONTRACT/ORDER NO.

2. AMENDMENT/MODIFICATION NO. 5. PROJECT NO.(If applicable)

6. ISSUED BY

3. EFFECTIVE DATE

20-Aug-2018

CODE

DEFENSE INFORMATION SYSTEMS AGENCY

P.O. BOX 549

FORT MEADE MD 20755

HC1047 7. ADMINISTERED BY (If other than item 6)

4. REQUISITION/PURCHASE REQ. NO.

CODE

See Item 6

FACILITY CODECODE

EMAIL:TEL:

HC104718R4016

SECTION SF 30 BLOCK 14 CONTINUATION PAGE

SUMMARY OF CHANGES

SECTION C - DESCRIPTIONS AND SPECIFICATIONS

The following have been modified:

PERFORMANCE WORK STATEMENT

PERFORMANCE WORK STATEMENT (PWS)

As of 20 August 2018

Contract Number: TBD at time of award

Contractor Name TBD at time of award

Tracking Number: 621600368

Follow-on to Previous Contract and

Task Order Number:

HC1028-08-D-2022-HC104718F0004

HC1028-08-D-2025-VC15

HC1028-08-D-2025-VC20

1. Contracting Officer’s Representative (COR).

a. Primary COR.

Identified in Section G, Points of Contact, as included in the contract.

b. Alternate COR.

Identified in Section G, Points of Contact, as included in the contract.

2. Contract Title. Command and Control (C2) Portfolio Test and Evaluation (T&E)

3. Background.

The C2 Portfolio T&E contract will be an overarching approach for fulfilling requirements for independent verification and validation (IV&V) of software across the C2 Portfolio which will consist of four (4) major programs, known today as the Global Command and Control System-Joint (GCCS-J), the Global Combat Support

System-Joint (GCSS-J), the Joint Planning and Execution Services (JPES), and the Global Command and Control

System-Joint Enterprise (GCCS-JE). Each of the programs develops and sustains systems for use by the joint communities and as such each has requirements to independently verify and validate software associated with these systems. The operational systems were developed independently of each other and, as a result, some of the foundational services that support each system are duplicative. The C2 Portfolio will evolve over time to have common modernized foundational services eliminating existing duplicative services. Additionally, the core mission applications that will interface with the common foundational services will also be modernized resulting in the sunset of specific legacy components. As a result, the C2 capabilities that are supported by this PWS will transition over time.

Each of the major C2 systems is described below:

• GCCS-J is the Department of Defense (DoD) joint C2 System of Record for achieving full spectrum dominance. GCCS-J is a suite of mission applications that provides critical joint war-fighting C2 capabilities and is the principal foundation for dominant battlespace awareness, providing an integrated, near real-time picture of the battlespace necessary to conduct joint and multinational operations. GCCS-J provides a robust and seamless C2 capability to the Commander-in-Chief (CINC), Secretary of Defense

(SECDEF), National Military Command Center (NMCC), Combatant Commanders (CCDRs), Joint Force

Commanders, and Service Component Commanders. GCCS-J offers vital interoperability to the systems the joint warfighter uses to plan, execute, and manage military operations. Key capabilities of GCCS-J include two specific Global baselines, v4.3 and v6.x. GCCS-J Global 4.3 has a planned sunset in mid

Fiscal Year (FY) 2019 and GCCS-J Global 6.x has a planned sunset by the end of FY20. The modernized system to replace GCCS-J is referred to as GCCS-J Enterprise (GCCS-JE). Rather than a local installation of the release at each site, the Initial Operational Capability (IOC) of GCCS-JE will be provided regionally.

• The GCSS-J system is the Joint Staff (JS/J4) Logistics System of Record providing an enterprise logistics

Common Operational Picture (LogCOP) for the Combatant Command (CCMD) and Joint Task Force (JTF) commanders. The system integrates data from CCMD and JS/J4 identified authoritative data sources to provide a fused, integrated, near real-time, multi-dimensional view of combat support and combat service support across joint capability areas, providing situational awareness of the battlespace and logistics pipeline (e.g., Supply, Deployment and Distribution, Engineering, etc.) and optimizing joint logistics processes. Utilizing a strategic server concept, GCSS-J capabilities are accessible to users from any

Government approved equipment, anywhere, as long as they have access to a web browser, a GCSS-J account and valid credentials (Secret Internet Protocol (IP) Router Network (SIPRNet): DoD SIPRNet hard token, Non-Secure IP Router Network (NIPRNet): Common Access Card (CAC)). Current GCSS-J initiatives provide solutions which improve collaboration across the CCMDs and facilitate the integration of GCSS-J services with each CCMD developed and maintained LogCOP.

• JPES is a portfolio of capabilities that support the policies, processes, procedures, and reporting structures needed to plan, execute, mobilize, deploy, employ, sustain, redeploy, and demobilize activities associated with Joint Operations. JPES capabilities support the deliberate planning, crisis action planning and global force management processes. Key legacy capabilities within JPES include the Joint Operation Planning and Execution System (JOPES), Joint Capabilities Requirements Manager (JCRM) and Preferred Force

Generation (PFG). JOPES is being modernized in two phases. Phase One consists of the JPES Framework

(JFW) and the JPES Solution (JPESS) which will replace the existing functionality of JOPES, concluding with the sunset of JOPES anticipated by the end of FY19. Phase Two consists of enhancing JFW and

JPESS to replace the functionality provided by JCRM and PFG. Phase Two is planned to conclude by the end of FY21 and will result in the sunset of JCRM and PFG.

• The Global Command and Control System – Joint Enterprise (GCCS-JE) will be the DoD Joint Command and Control (C2) system of record for achieving full battlespace dominance. It will provide critical joint warfighting C2 capabilities, providing an integrated, near real-time picture of the battlespace necessary to conduct joint and multinational operations. GCCS-JE will provide a robust and seamless C2 capability to the Commander-in-Chief (CINC), Secretary of Defense (SECDEF), National Military Command Center

(NMCC), Combatant Commanders (CCDRs), Joint Force Commanders, and Service Component

Commanders. GCCS-JE will offer vital access to the fused information from systems and sources that the

Joint warfighter uses to plan, execute, and manage military operations.

The C2 Portfolio deployable capability shall be interoperable with C2 Portfolio capabilities and shall operate under conditions which are disconnected, intermittent, and limited (DIL) communications. The deployable configuration shall support interfacing with a subset of the C2 Portfolio data sources and systems. The deployable configuration shall support the C2 Portfolio web client and shall expose C2 Portfolio data services to data consumers operating in the target environment.

The C2 Portfolio will include custom baselines configured to support specific Foreign Military Sales (FMS) cases.

The specific software baseline for a given FMS case will be approved by the DISA Foreign Disclosure Officer

(FDO). System-level documentation will be modified as required to ensure it only discusses software components that are part of the approved baseline for a particular FMS case.

4. Objectives: The objective of this PWS is to provide independent validation and verification (IV&V) of C2 developed capabilities/releases.

5. Scope. Ensure that the C2 Portfolio capabilities are fully security compliant, installed and configured per delivered documentation in a Government or contractor managed test environment, and tested per the established test plan in Government or contractor managed test environment as required. The contractor shall participate in full life cycle testing of systems from development through deployment and sustainment.

The methodologies and processes shall be consistent throughout the individual programs; however, due to the evolving nature of the C2 portfolio GCCS-J and JPES support will be required at time of award while GCCS-JE support is anticipated to be needed towards the middle of the base year. For this reason, the Government plans to use part of the SURGE CLIN to exercise GCCS-JE support once the Government becomes aware of the exact requirements. Tables 5.1-5.3 reflect the projected testing events for the GCCS-J and JPES individual programs within the C2 portfolio. Detailed explanations of these test events can be found in Section 6, Performance

Requirements.

In addition, funding sources for the various programs will be different as outlined below

• GCCS-J will use O&M

• JPES (JOPES, JCRM, and PFG) will use O&M

• JPES (JPES and JFW) will use RDT&E

• GCCS-JE will use RDT&E

Table 5.1 – Anticipated GCCS-J (O&M) Test Events per Yearly Period of Performance

Test Events Anticipated Total Number of Test Events Per Year

Major Release Events 4

Minor Release Events 7

IAVA and Emergency Release

Events

FMS-Related Events 3

Total 442

Table 5.2 – Anticipated JPES (O&M) Portfolio Test Events per Yearly Period of Performance of Test Events Per Year

Major Release Events 0

Minor Release Events 7

IAVA and Emergency Release

Events

FMS-Related Events 0

Total 17

Table 5.3 – Anticipated JPES (RDT&E) Portfolio Test Events per Yearly Period of Performance of Test Events Per Year

Major Release Events 1

Minor Release Events 3

IAVA and Emergency Release

Events

FMS-Related Events 0

Total 9

The Government may require surge support during the base or any option period, and surge modifications will be within the scope of the contract and provide increased support for the defined task areas of this PWS. Surge support over the life of the contract will not exceed 40% of the contractor’s total proposed cost/price for the base and all option periods, excluding any six-month extension of services pursuant to Federal Acquisition Regulation (FAR)

52.217-8.

6. Performance Requirements.

6.1 Task 1 – Contract Management (Firm-Fixed-Price (FFP)).

Contract Management is required to provide a structure that integrates the contractor effort with the goals and objectives of the C2 Portfolio. The key elements of task management include the management plan, a schedule of activities (with regular updates), control and reporting of performance, cost reporting, regular in progress reviews, and delivery of materials. Task efforts are described in the subtasks below.

6.1.1 Subtask 1 – Project Management.

• The contractor shall prepare and provide a Contract Management Plan (CMP) describing the technical approach, organizational resources and management controls to be employed to meet the cost, performance and schedule requirements throughout the PWS execution.

• The contractor shall provide technical and functional activities required for integration of all tasks specified within the PWS.

• The contractor shall prepare and provide a Work Breakdown Structure (WBS) and a corresponding

Integrated Master Schedule (IMS) for the overall contract to include schedules for each subtask.

• The contractor shall analyze, define, prepare, and provide Work Packages with planned monthly costs for the entire Period of Performance (PoP) broken out to level four (4) in the WBS and traceable to the IMS.

• The contractor shall integrate and coordinate all activities needed to execute the requirements within this

PWS. The contractor shall manage the timeliness, completeness, and quality of all task areas within this

PWS. The contractor shall ensure customer satisfaction and professional and ethical behavior of all contract personnel.

• The contractor shall establish clear organizational lines of authority and responsibility to ensure effective management of the resources assigned to the requirements. The Prime contractor shall have at least 1 of the following certifications at the time of award: Capability Maturity Model Integration (CMMI) Level 3 or

ISO 9001.

Deliverables for 6.1.1:

� CMP

� WBS to the 4 th level

� T&E IMS

6.1.2 Subtask 2 – Control Progress and Status Reporting.

• The contractor shall prepare and provide a Monthly Status Report (MSR) for all task areas within the PWS.

• The contractor shall prepare and provide a Contractor Performance Review (CPR). For each task of the

PWS, the contractor shall break out costs and hours by systems (e.g., GCCS-J, GCCS-JE, GCSS-J, JPES, PFG, JCRM, JOPES, and JFW), then by specific release of each system and furthermore, shall identify the

Accounting Classification Reference Number (ACRN) and task level.

• The contractor shall conduct a monthly status review meeting. The meeting can be facilitated via Defense

Collaboration Services (DCS), teleconference, or onsite as appropriate.

• The contractor shall generate meeting minutes based on the monthly status review meeting. The minutes shall summarize the discussion of the monthly review meeting and capture actions to be resolved.

Deliverables for 6.1.2:

� MSR

� CPR Report

� Meeting Minutes

6.1.3 Subtask 3 – Kick-Off Meeting (KOM).

The contractor shall prepare for and participate in a KOM with the Government to establish a baseline of understanding after the contract award or as directed by the Contracting Officer’s (KO) at a location to be determined by the COR no later than (NLT) ten (10) business days after contract award. Participants shall include

The Product Leads and Deputy Product Leads, C2 Portfolio Program Managers (PMs), KO, COR, key contractor personnel and other Government staff. At the KOM, the contractor shall present the details of its intended approach and high-level milestone schedule for review. The KOM shall serve to resolve strategic questions, refine goals, define success, and explore the biggest challenges and breakthrough opportunities for the Government. This shall be the beginning of a dialogue between the Government and the contractor to ensure successful execution of this contract. The contractor shall coordinate the agenda for the KOM and include, as a minimum, the following:

• Introduction of management and technical teams

• Presentation of management plans

• Establishment of common understanding of the contract

• Other relevant items may be introduced at the discretion of the contractor and/or the Government

Deliverables for 6.1.3:

� KOM Briefing Materials

� KOM Minutes and Action Items

6.1.4 Subtask 4 – Contract Transition Plan (Optional Contract Line Item Number (CLIN)).

In the event of a contractual change or turnover, the Government will provide written notification to the contractor.

Upon receipt of the notification, the contractor shall provide at a minimum all materials and support necessary to accomplish a seamless and expeditious transition of the tasks identified in this PWS to the incoming contractor. The transition shall include the following: all in-progress working files for the deliverables of this PWS; final files for all tasks, test procedures; all test scripts/procedures (manual, automated, documented); and final phase out meetings with Government, incumbent, and the new contractor.

Deliverables for 6.1.4:

� Transition Plan, transfer of knowledge, experience, and lessons learned to the incoming contractor

6.2 Task 2 – Test Support for GCCS-J (O&M Funding) (Cost-Plus-Fixed-Fee (CPFF)).

All test support shall be coordinated/directed by the Government to support/meet Government established/managed test objectives, expectations and end states for its Family of Systems (FoS). Due to the time critical nature of program level test events, it shall sometimes be necessary for contractor support to be available during extended or non-standard duty hours (including weekends) to support activities under this TO. The contractor PM shall inform the Government COR regarding the availability of staff assigned to this effort, particularly for planned formal test events. The availability of contractor staff during test events and planning activities shall be identified by the contractor 10 business days in advance.

The Contractor shall provide Test and Evaluation (T&E) support for all the systems in the C2 Portfolio. The T&E support includes technical expertise and execution support necessary for conducting all C2 Portfolio test events

(sometimes simultaneously), including support for C2 Portfolio associated FoS, T&E activities, and T&E of individual mission critical systems. Specific T&E tasks include:

• Support to Government T&E Integrated Product Teams (IPT) in coordinating T&E and release activities across the Program Management Offices (PMOs) and external stakeholders

• Support selection, planning and development of test events, to include identification of software requirements mapping and validation processes

• Development of testing schedules, integration with the overall program IMS, test bed architecture, network diagrams, flowcharts, test plans, functional and regression tests, test cases, and development support of

Master Scenario Event Lists (MSEL)

• T&E execution duties to include local and remote tests of development and operationally representative sites, development and analysis documentation of test data/test cases/MSELs, developer delivered documentation, etc.

• Verification, execution of performance testing and metrics gathering, and evaluation/implementation of automated testing tools and processes

• Tracking of testing efforts, associated schedule impact, and risk identification, analysis, and mitigation specific to T&E efforts

• IV&V efforts include the evaluation and reporting of the security posture delivered software

The ranges of test event spans from very small scoped Information Assurance Vulnerability Alert (IAVA) tests to

Major Release test events. Examples/explanations of test events:

Major Release - Consists of a significant change in the architecture or operation of the capability; a major version would indicate that the specification may have changed in a non-backward compatible way. Releases are usually indicated by a first digit versioning change (1.x.x.x). Examples of test events typically consist of the following:

a. Contractor Integration Test (CIT) – A developer led test event which integrates all development efforts prior to formal Government delivery. The contractor shall collaborate with the developer in an effort to derive test cases (manual and automated), while helping the developer identify the quality of the product.

b. Risk Reduction Test Event (RRTE) – Test that allows subject matter experts (SMEs) to perform hands-on testing of the delivered software to identify critical defects. Contractor leads and executes RRTE and may coordinate with externals (if required).

c. System Integration Test (SIT) – Tests that the delivered software interfaces/integrates with other system components. Contractor shall execute SIT and coordinate with external stakeholders to complete SIT.

d. System Acceptance Test (SAT) – Tests that the delivered software meets stated requirements. Contractor leads and executes SAT. These tests may also be executed at external sites with operationally representative test beds and users.

e. Software Quality Test (SQT) – Tests that allows SMEs to perform hands-on testing of the delivered software to identify critical defects early and minimize risks going into formal Operational Test (OT), Interoperability Test (IOP) or Limited User Assessment (LUA). Contractor leads and coordinates with external stakeholders to complete SQT.

f. Interoperability Test (IOP) – Tests the ability to exchange data with external systems as documented in

Interoperability Support Plan (ISP). Joint Interoperability Test Command (JITC) leads the IOP. Contractor supports and coordinates with JITC. May coincide with other test events.

g. Limited User Assessment (LUA) – Tests the end-to-end system functions. Similar to an OT but on a smaller scale. Contractor leads LUA and coordinated with external participants.

h. Performance Testing (PT) – Tests the systems’ ability to meet Key Performance Parameters as well as tests the upper limits of the system. Contractor leads and executes the test. May coincide with other test events.

i. Operational Test & Evaluation (OT&E) – Test events performed by operationally representative test sites and users to verify the release is operationally ready to field. The JITC leads the test with C2 Portfolio contractor support.

Minor Release - Consists of new features being added to the capability but the fundamental architecture remains unchanged; a minor version would indicate that the specification has changed in a backward compatible way. Minor

Releases are generally indicated by a second, third or fourth digit versioning change (1.2.x.x). Examples of Minor

Release Events may include:

a. Contractor Integration Test (CIT) – A developer led test event which integrates all development efforts prior to formal Government delivery. The contractor shall collaborate with the developer in an effort to derive test cases (manual and automated), while helping the developer identify the quality of the product.

b. Application Security Process (ASP) – This is the Agile Process used for software application that require rapid deployment into an accredited DoD Information System.

c. Maintenance Release Process (MRP) – Tests which performed focused evaluation of fixes or enhancements while ensuring the security posture and functionality of the associated release is undisturbed. Contractor leads and executes Maintenance Release tests.

d. System Integration Test (SIT) – Tests that the delivered software interfaces/integrates with other system components. Contractor leads, executes SIT and coordinates with external stakeholders to complete SIT.

e. System Acceptance Test (SAT) – Tests that the delivered software meets stated requirements. Contractor leads and executes SAT. These tests may also be executed at external sites with operationally representative test beds and users.

f. Software Quality Test (SQT) – Tests that allows SMEs to perform hands-on testing of the delivered software to identify critical defects early and minimize risks going into formal OT, IOP or LUA.

Contractor leads and coordinates with external stakeholders to complete SQT.

g. Risk Reduction Test Event (RRTE) – Test that allows SMEs to perform hands-on testing of the delivered software to identify critical defects. Contractor leads and executes RRTE and may coordinate with externals (if required).

h. Interoperability Test (IOP) – Tests the ability to exchange data with external systems as documented in

Interoperability Support Plan (ISP). Joint Interoperability Test Command (JITC) leads the IOP. Contractor supports and coordinates with JITC. May coincide with other test events.

i. Performance Testing (PT) – Tests the systems’ ability to meet Key Performance Parameters as well as tests the upper limits of the system. Contractor leads and executes the test. May coincide with other test events.

Emergency Release Events and IAVA Events:

a. An Emergency Release Test event is conducted in response to a highly critical operational defect.

Development, security, test execution and fielding of the defect mitigation is collapsed. Contractor leads and executes the test.

b. IAVA Events: Integration functional testing and validation of security patches applied to existing systems.

FMS Test Events: Custom events geared to support C2 Portfolio distribution to non-standard Outside Continental

United States (OCONUS) sites.

Please refer to Table 5.1 in Section 5 for the initial projected testing forecast for GCCS-J.

6.2.1 Subtask 1 – Test Management.

The contractor shall provide Test Managers with expertise across the full spectrum of test events. For each test event, the contractor shall identify a Test Manager. Multiple contractor Test Managers are required as multiple test events occur concurrently. Contractor Test Managers shall maintain a final Secret security clearance. The Test manager’s responsibility shall be to ensure each test event is planned, executed, and closed out. The Test Manager shall be well versed with C2 Portfolio software lifecycle processes, acquisition test requirements, with particular emphasis on System Integration Tet (SIT), System Acceptance Test (SAT) and Operational Test processes as well as ISO/IEC/IEEE 29119 Software Testing Standards. All test activities shall be coordinated through the COR. The

Test Manager shall ensure all test artifacts and contractor deliverables are compiled in a Government approved location.

6.2.1.1 Subtask 1.1 – Test Planning.

The Contractor shall be responsible for all test event planning activities. The Contractor shall develop a Test Plan that documents all resources needed to execute the test event, unless the requirement for a Test Plan document is waived by the COR on a case-by-case basis for each test event.. The test cases to be executed during the test event shall be documented in the Test Plan. For each release, the contractor shall test all new functionality as well as all fixes to software defects. Contractor shall plan for integration testing with external systems as well. The Contractor shall assess and plan for regression testing of previously delivered functionality to ensure no breakage. Contractor shall conduct a Risk Assessment of each software release to aid in building the Test Plan.

Contractor shall participate in PMO Release design reviews and PMO sprint reviews in order to develop test cases.

The contractor shall refine and decompose test planning requirements through participation in the following:

• Preliminary Design Review (PDR) as required

• Critical Design Review (CDR) as required

• Daily Developer status briefings during Sprints

• Bi-Weekly Status Update (SUM) Meetings

• Weekly Engineering Review Group (WERG) Meetings

• Weekly Test Management Meetings

• Scrum Meetings as required

• Weekly IPT Meetings

• Daily CIT Briefings

• Contractor Weekly Program Change Review Board (PCRB) Meetings

The contractor Test Manager activities shall include the following:

• Identify and verify availability of test team resources

• Participate and collaborate with software development to include sprint briefings

• Identify and verify availability of systems administrators

• Identify and verify all Government-off-the-shelf (GOTS), and Commercial-off-the-shelf (COTS) software requirements to support test events in all lab test beds

• Identify all interfaces to be evaluated against all test beds

• Perform independent development technical evaluations

• Coordinate with the Government toward resolution of release production issues

• Participate in the generation of any required Risk Assessments

• Identify and report to the Government all projected test conflicts with recommended solutions

• Ensure all test build, software and test execution documentation is collected and stored in a Government-specified repository

• Coordinate projected release test activities between all test sites and stakeholders

• Identify test entrance and exit criteria

• Assist with release schedule generation and maintenance

• Track and report overall planning management of system test events

• Identify, document and track all pre-test software non-compliance, security findings, and functional defect reports

• Assist and supplement the product development team’s development integration event testing prior to formal Government delivery

• Provide weekly status updates on release issues and schedule

Deliverables for 6.2.1.1:

� Risk Assessment

� Test Plans

6.2.1.2 Subtask 1.2 – Test Execution and Reporting.

The contractor shall manage and execute all assigned test events. The contractor shall plan for and conduct a Test

Readiness Review (TRR) prior to the start of the Test. The Government will provide a sample TRR as has been used in the past as a baseline template. The contractor shall modify this TRR template based on the size and scope of the test event. The contractor shall deliver the completed and compiled TRR materials to the Government for review and approval. Prior to the scheduled TRR, the contractor shall coordinate with all participating members of the test event to ensure that all aspects of the TRR have been pre-coordinated, validated and verified. The contractor shall ensure the full execution of the Government approved Test Plan. A build is a collection of application components installed and compiled for testing purposes to ensure a reliable final product. The contractor responsibilities shall include:

• Verify software and required installation/build documents are available to all test build sites

• Confirm all local and remote tester/user account validation

• Confirm any applicable test automation validation

• Review any site validation checklist(s) and/or Site Readiness Reviews (SRR)

• Coordinate TRR

• Execute the Test Plan

• Oversee all defect report generation, execute defect report processes

• Coordinate test status updates with the Government personnel, Release Manager, present issues and recommend mediations to the Release Manager and/or Test Director

• Verify all installation/build, software/system installation teams are assembled

• Confirm network and interface validation

• Facilitate daily defect review meetings

• Prepare and deliver daily situation reports

• Coordinate build validation

• Support required all Cybersecurity validation and network Authority to Test (ATT), Interim Authority to

Test (IATT)

• Capture all installation/build issues, documentation redlines and defect reports

• As required, assemble daily situation reports to include test progress relative to the test schedule, issues discussion, percentage completion pass/fail of test cases, validation of interface and defect reports, identification of any new defect reports

• As required, coordinate cross-team installation/configuration validations

Deliverables for 6.2.1.2:

� TRRs

� Daily Situation Reports

6.2.2 Subtask 2 – System Testing.

The contractor shall provide a team of experienced systems test professionals (referred to as “team” in this section).

The contractor team shall maintain a minimum Secret clearance. The contractor team shall be required to execute multiple tests against multiple baselines concurrently.

The contractor systems test professionals shall participate in the generation of Risk Assessments, Test Plans, Situation Reports and Test Reports. The contractor shall be proficient in executing test processes against client, server, virtual and enterprise C2 Portfolio of systems. Contractor team shall be proficient in the generation of scalable test cases and trained to execute test processes to include formal acquisition required processes and agile test processes. Contractor generated test cases support a range from Emergency Release processes through mission oriented Operational Test events. Test cases shall validate developer delivered capabilities to determine requirements satisfaction. Contractor generated test cases shall be executed during developer sprint testing and

Contractor Integration Testing (CIT) to support the formal Government delivery of quality products. The Contractor team shall store and maintain test cases in a Government-owned shared repository.

The contractor team shall execute and document Key Performance Parameters (KPPs) and systems interface validation. KPP validation shall identify and validate performance requirement criteria. The contractor team shall be proficient in the execution of automated test toolset/scripts. The contractor shall participate in the execution of security integration products and documentation.

The contractor team shall participate in test systems status briefings.

Contractor team shall be proficient in generating, validating and maintaining systems defect reports following the established documented processes.

All data generated during all phases of systems validation shall be stored in a Government-owned shared repository.

The contractor team responsibilities shall include:

• Participation in the generation of Risk Assessments, Test Plans, Situation Reports and Test Reports

• Participation in Scrums, IPTs, PCRBs, WERGs, and Hotwashes

• Participation in test related teleconferences

• Participation in the generation of Site Validation Checklists

• Participation in the generation of test Entrance Criteria and Exit Criteria checklists

• Participation in developmental testing

• Validation of Cybersecurity requirements

• Generation and storage of systems test cases

• Execution of incremental capabilities focused validation

• Execution of full systems test cases from development through Operational Testing

• Execution of automated tests

• Generation, validation, and maintenance of defect reports

• Validation and documentation of all required performance criteria and interface requirements

• Full execution of the Test Plans, any exceptions shall first be approved by the Government

Deliverables for 6.2.2:

� Systems Test Cases

� Systems Defect Reports

6.2.3 Subtask 3 – Build and Maintain Test Environment.

The contractor shall build and maintain classified test environments for clients, servers, virtual systems, and enterprise systems. The contractor shall also validate system builds with strict adherence to installation documentation. Deviations from the system installation documentation shall be documented in defect reports. The contractor shall generate, validate, and update all system defect reports. In addition, the contractor shall be responsible for:

• Generating, validating, and updating system defect reports

• Participation with developer led systems builds, security validation and systems validation

• Participation in the generation of Risk Assessments

• Participation in the generation of systems, network and security architecture diagrams

• Execution of formal test environment systems builds, security validation and reporting, post-installation configurations, data interface network configuration

• Performing systems validation of all client, server, virtual and enterprise systems

• Red-lining all formal environment build, security, and configuration documentation

• Participation in build status meetings

• Performing software and security maintenance of all test sustainment, modernized, and enterprise systems virtual machines

• Completing Site Validation Checklists

• Generating Test Entrance Criteria and Exit Criteria checklists

Deliverables for 6.2.3:

� Site Validation Checklist

� Red-lined formal documentation

� Baseline build validation checklist

� Systems installation/build, security and configuration defect reports

6.2.3.1 Subtask 3.1 – External Project Support (Optional CPFF CLIN).

The contractor shall perform installation, configuration and functional evaluation of custom software as requested by external stakeholders. This software may include full system builds, data population and appliances supporting existing systems.

The contractor shall build, install, configure, and administer systems in support of external projects/initiatives and/or

FMS systems. Estimate up to two server suites in a performance period may be designated for use in supporting various external projects/initiatives.

Deliverables for 6.2.3.1:

� Daily Situation Reports (SITREPs) or as requested by the Government

� Weekly SITREPs or as requested by the Government

6.2.4 Subtask 4 – Cybersecurity and Information Assurance (IA) Compliance Support.

The contractor shall ensure that the product baselines are in compliance with the DoD 8500 (Cybersecurity), 8510

(Risk Management Framework (RMF)), 8551 (Ports, Protocols, and Services Management (PPSM)), 8520 (Public

Key Infrastructure (PKI)), and any other applicable Directives and Instructions requirements. The contractor shall ensure, maintain, and/or improve the product baselines cybersecurity posture by proactively addressing, resolving, and delivering solutions in order to comply with the applicable DoD Directives and Instructions. The contractor shall provide support (i.e., security artifacts, documentation and software solutions) for the resolution of new and previously identified security deficiencies and the development of Plan of Action and Milestone (POA&M) with mitigation strategies to ensure that the residual vulnerability findings for the applicable baseline releases are compliant with the DoD Cybersecurity requirements and Accreditation and Authorization (A&A) process. All contractor developed software items as well as the integrated end product baseline shall be compliant with cybersecurity requirements and free of any known CAT I vulnerabilities that cannot be successfully mitigated, downgraded to a CAT II finding by the Security Control Assessor (SCA), and approved by the Authorizing Official

(AO). The contractor shall deliver software updates to address any applicable IAVAs, COTS and Open Source software updates (e.g., Oracle quarterly Critical Patch Updates), End of Life (EOL) / unsupported software updates, and fixes to address selected POA&M items. Contractors shall proactively monitor, track, and provide the PMO with a strategic way forward regarding any applicable DOD/USCYBERCOM taskings for any of the PMO supported baselines/systems/applications (Rich Site Summary (RSS) feed Uniform Resource Locator (URL)) will be supplied post award).

Deliverables for 6.2.4:

� Developer’s Secure Coding, analysis, remediation, and mitigation Process

� DOD RMF Control assessment, artifact updates/development, and validation

� Developer cybersecurity self-assessments of GOTs Software (S/W) products and baselines

� POA&Ms with Mitigation strategy development

� PMO Cybersecurity IV&V and/or security pre-look/scan events

� Compliance Testing and Analysis Report (CLASSIFIED)

� Developer’s Cybersecurity Self-Assessment Summary report of GOTs S/W (per package) (CLASSIFIED)

� IA Artifacts (Annotated Systems Developer Security Technical Implementation Guide (ASD STIG) checklist and Fortify scan results (i.e. Workbooks &. fpr files)) (CLASSIFIED)

� Security / RMF Documentation updates and/or development

� Integrated Command and Control Support Framework Software Development Kit (ICSF SDK) (as applicable)

� Computer Software Product End Items

6.2.5 Subtask 5 – Reaccreditation Support via the Risk Management Framework (RMF).

The contractor shall provide mitigation and POA&M analysis support for the reaccreditation of software baseline via the RMF certification and accreditation process. The contractor shall support the security team throughout the

RMF process to include: reassessing RMF controls, remediate findings, deliver artifacts, update finding matrix, etc.

Deliverables for 6.2.5:

� Compliance Testing and Analysis Report (CLASSIFIED)

� Developer’s Cybersecurity Self-Assessment Summary Report (per package) (CLASSIFED)

� IA Artifacts (ASD STIG checklist and annotated Fortify scan results) (CLASSIFIED)

� Security/RMF Documentation updates and/or development

6.2.6 Subtask 6 – Test Process Improvement and Test Automation Support.

The contractor shall assist in improving the C2 Portfolio test processes by generating, executing, and reporting Test

Automation to conduct performance and functional testing.

6.2.6.1 Subtask 6.1 – Test Process Improvement/Automation.

The contractor shall observe and coordinate with systems development toward integrating automation tools/programs/scripts to enhance testing performance and redundancy. The contractor shall review and continually evaluate the overall C2 Portfolio testing processes, methods, and tools currently being used and propose and develop improvements to the testing processes that better fit the needs of the C2 Portfolio. The contractor shall maintain expertise in all aspects of Acquisition T&E requirements and Director of OT&E (DOT&E) /Other Transaction

Agreement (OTA) oversight and continually recommend areas of optimization.

The contractor shall investigate, evaluate, and incorporate automated software test tools and associated processes to facilitate test activities in the lab and Global Technical Assistance Center (GTAC) where appropriate. These tools must be able to run in a Windows/Unix/Linux Client/Server PMO test environment. The contractor shall replace repetitive manual testing with automated tests that can be run repeatedly. The selection of automated tools shall enable test engineers to quickly set up and build complex test cases that can be easily customized. The contractor shall develop a plan indicating how and when the tool or process is to be integrated into the test environment.

The approach and cost to automate shall be coordinated with and approved by the COR, prior to implementation.

The Government will purchase all third party automated tools for implementation and provide them to the contractor as Government Furnished Equipment (GFE).

Deliverables for 6.2.6.1:

� Technical Report – Study/Services – Test Process Improvement Report

� Implementation Plan: Automated Software Test Tool Implementation Plan and White Paper on

Recommended Automated Test Tools

6.2.6.2 Subtask 6.2 – C2 Portfolio Automated Performance Testing.

The contractor shall assist the Government in conducting performance tests and analyses in accordance with the

Government’s test strategy. In order to perform this activity, performance tools obtained or provided by the customer may be used. The contractor shall be knowledgeable and experienced in the use and management of the tools. See Appendix A for the list of automated test tools. The contractor shall:

• Assist the Government in planning and execution of performance testing per the KPPs

• Provide installation and setup of the tools on GFE platforms

• Develop and install software test scripts

• Gather and analyze all metrics generated by tools

• Assist the Government with tuning the tools and applications under test to localize defects

• Include in appropriate test reports metrics detailing server delay, network delay, and end user performance with (where applicable) recommendations for performance improvement

• Assist in the development of test plans and scenarios which test the performance of PMO releases, in order to determine how the tested release will perform in a loaded operational environment. The tests and scenarios shall be documented in the test plan and shall be “operationally realistic” in that the load conditions applied to the tested release shall exceed conditions expected in a real-world environment

If the Government approves the test plan, the contractor shall identify the raw metric data to be collected, the method of collection, procedures to be applied to the collected raw metric data, and the final metrics to be calculated from the raw metric data. The contractor shall explain in the test plan the manner in which the final metrics describe the tested release performance under load. Subsequent to release, the contractor shall assist the Government in collecting data and performing analysis comparing actual release performance under load with predicted release performance under load.

Deliverables for 6.2.6.2:

� Test Plan: Performance Test Plan

� Test/Inspection Reports: Performance Test Report

� Computer Software Product End Items: Performance Test Scripts

6.2.6.3 Subtask 6.3 – C2 Portfolio Automated Functional Testing.

The contractor shall assist the Government in performing automated functional testing of any C2 applications. In order to perform this activity, automated tools provided as GFE may be used. The contractor shall be knowledgeable and experienced in the use and management of the tools. The support team shall:

• In coordination with the Government, conduct the planning phase of the functional test

• In coordination with the Government, develop test plans and operationally relevant scenarios, which are to be documented in the functional test plan

• Provide installation and setup guidance of the tools on GFE platforms Develop and install software test scripts

• Gather and analyze all metrics generated by vendor provided tools as use of metrics analysis is dependent on Government need

• In coordination with the Government, assist with tuning the tools and applications under test in order to localize defects and for the calibration of tools

The contractor shall produce a functional test report describing the results of the test using the calculated metrics as descriptors. The test report shall cover all performance related aspects of the measured and observed system behavior under load.

Deliverables for 6.2.6.3:

� Test Plan: Functional Test Plan

� Test/Inspection Reports: Functional Test Report

� Computer Software Product End Items: Functional Test Scripts

6.2.6.4 Subtask 6.4 – Contractor Training of Government Personnel (Optional CPFF CLIN).

The contractor shall train its Government counterparts in the processes and procedures used to perform the T&E activities required under this contract. Training shall be performed on request of the Government, in the form of individual (i.e., one-on-one) and group sessions. Training shall be presented in the form of presentation slides, video presentations, desk-top process guides, or similar materials. T&E training topic areas will be selected by the

Government, as needed for ongoing knowledge transfer.

A preliminary and final Training Plan shall be submitted to the Government for review and approval. The Training

Plan shall describe the training requirements, processes, procedures and materials that the Contractor shall develop to train Government personnel on selected T&E topics. The Training Plan shall include the training schedule, including the expected duration of each type of training, expected number of trainees, and location of the training.

The Training Plan shall have an executive summary providing an overview of the plan’s management and technical approach. Summary schedules of project phases, activities, and tasks, and the corresponding training shall be provided. The extent of the training plan shall be summarized in terms of overall effort, time, impact, expected results, and products. Training materials shall be submitted ten (10) calendar days prior to a scheduled training session.

Deliverables for 6.2.6.4:

� Contractor Training of Government Personnel Training Materials

� Training Plan

6.3 Task 3 – Test Support for JPES (O&M Funding) (Cost-Plus-Fixed-Fee (CPFF)).

As stated above, the testing methodologies and processes shall be consistent throughout the individual programs. All of PWS 6.2 tasks and subtasks should be mirrored as part of 6.3 Test Support for JPES (O&M), including deliverables. Please refer Table 5.2 for JPES (O&M) Portfolio in Section 5 for the initial projected testing forecast.

6.4 Task 4 – Test Support for JPES (RDT&E Funding) (Cost-Plus-Fixed-Fee (CPFF)).

As stated above, the testing methodologies and processes shall be consistent throughout the individual programs. All of PWS 6.2 tasks and subtasks should be mirrored as part of 6.4 Test Support for JPES (RDT&E), including deliverables. Please refer to Table 5.3 for JPES (RDT&E) Portfolio in Section 5.

6.5 Task 5 – Test Support for GCCS-JE (RDT&E Funding)

As stated above, due to the current uncertainty of the required GCCS-JE testing support the Government plans to use part of the SURGE CLIN to exercise GCCS-JE support once the Government becomes aware of exact requirements.

7. Performance…