Attachment_D_-_STS_SOW_A004_(Clean).docx

Bureau of Consular Affairs (CA) Consular Systems and Technology (CST)

Service Transition Support (STS)

Statement of Work (SOW) Amendment 004 1/26/2016

1 INTRODUCTION

This Statement of Work (SOW) provides the requirements for performing Service Transition Support (STS) for the United States (U.S.) Department of State (DoS), Bureau of Consular Affairs (CA), Office of Consular Systems and Technology (CA/CST), hereafter designated as CA/CST.

1.1 Background

The primary mission of CA is to protect and assist U.S. citizens abroad, to issue passports to U.S. Citizens, and to issue immigrant and non-immigrant visas to foreign nationals according to the laws of the U.S. Government. To achieve this mission, CA actively pursues business, technology, and management goals and objectives.

CA/CST is tasked with providing CA's mission critical information technology (IT) resources necessary to support various CA locations including approximately 230 overseas posts, 29 domestic Passport agencies, and an array of information, book / card print, passport centers, DoS annexes, and contractor sites.

1.2 Objectives / Purpose

This SOW provides STS-related services for the software (SW) development lifecycle in support of DoS CA/CST systems, as well as IT Service Management (ITSM) and Business Process Improvement (BPI) for CA/CST. The services described in this SOW capture a standards-based, best practices approach to providing service and support.

The traditional approach for project management and execution is undergoing a transformation that incorporates principles of Lean and Agile practices. Industry experience has shown these practices, when implemented correctly, produce higher quality products. CA/CST is adopting an Agile / Lean approach for its SW development projects. The Contractor shall adhere to the CA/CST practices and guidelines that facilitate this approach.

1.3 Scope of Work

The scope of this contract is defined by the Performance Requirements, which are further defined in Section 3.

2 CONCEPT OF OPERATIONS

With the exception of Task 1: Program Management and Reporting, the services specified in the tasks of this SOW are broadly defined in four (4) groups of services:

· Facility and Transition Management: o Task 2: Transition Activities o Task 3: Facility Management

· Independent Validation and Verification (IV&V):

Service Transition Support (STS) – Statement of Work

Amendment 004 Page 58 of 58

· Task 4: Formal Independent Validation and Verification (IV&V) Services

· Task 5: Commercial Off-the Shelf (COTS) and Government Off-the-Shelf (GOTS) Product Review

· Task 6: Integration Testing and Development Support Services

· Task 7: User Acceptance Test (UAT) Services

· Quality Management (QM):

· Task 9: Process Monitoring and Improvement Services

· Configuration Management (CM):

· Task 8: Information Technology Service Management (ITSM) Support Services

· Task 10: Portal Services (PS)

· Task 11: Tools and Repository Services

· Task 12: Configuration Control Process Services Many of these tasks support the SDLC phases of CA/CST projects. Each task within a group supports each other and may perform as independent services to other groups or external organizations.

2.1 Tasks 2 & 3 Facility and Transition Management

As depicted in Figure 2-1, the tasks within Facility and Transition Management group are Transition Activities and Facility Management Services.

Portal Services

CST Division Management Legend:

External Processes

CLIN

Optional CLIN Input/Output CST Division GTMs

· Infrastructure

· GOTs

· COTs

Facility Management Partners

· GSA

· CA/CST

Facility and Transition Management CLIN Grouping

Contractor Transition Transition Summary Report Facility Management Plan

GFE

Status & Performance Reporting Leasing Approved Facility Management Plan Transition (In/Out) Plans

GFE

Figure 2-1. Facility and Transition Management Services Table 2-1 shows the major inputs and outputs of the two (2) tasks of the Facility and Transition Management Services.

Table 2-1. Facility and Transition Management Tasks Inputs and Outputs

Facility and Transition Management Services

Input

Source

Output

Destination

Contractor Transition

Division GTMs

· Transition (In/Out) Plans

· Facility Facility Management Management Plan

Transition (In/Out) Plans

Contractor Transition

· Facility Management Plan

· GFE

Contractor Transition

Division GTMs

· Status & Performance Reporting

PS

2.2 Tasks 4 through 7 Independent Verification & Validation (IV&V) As depicted in Figure 2-2, the main tasks within IV&V are IV&V, COTS & GOTS Product Review, Integration Testing Development Support Services, and UAT Services.

Development Environment (NPE) Management Services

CST Division GTMs

Formal Independent Verification & Validation (IV&V) Status & Performance Reporting

COTS and GOTS Product Review

COTS & GOTS

Test Incident Report an

Integration Testing Services

Integration Test Incident Report

UAT Services

UAT Test Incident Report

UAT Findings & Recommendations Report Recommendation Report Independent Verification and Validation (IV&V) CLIN Grouping

Validation Results Weekly Test & Monthly Release Scheduling Reports Approved IV&V Test Plan Validation Results & IV&V Test Incident Report Approved Software & Product Requirements List Status & Performance Reporting Approved Product Test Plan Status & Performance Reporting Approved Integration Test Pl Status & Performance Reporting Approved UAT Plan Integration Findings Report CST Division Management

· SSPM

· GTMs IV&V Findings & COTS & GOTS Findings & Recommendations Report

Legend:

External Processes

CLIN

Optional CLIN Input/Output

Figure 2-2. Independent Verification and Validation Services Table 2-2 shows the major inputs and outputs of the five (5) tasks of the IV&V Services.

Table 2-2. IV&V Task Inputs and Outputs

IV&V

Services

Incident Reports (TIRs)

Approved SW &

· IV&V Findings and Recommendat ion Report

COTS &

Incident Report

ion Report

· Integration PS TIR

· Integration CST Division Findings Management Report

Report

IV&V

Services

Scheduling Reports

2.3 Task 9 Quality Management (QM)

As depicted in Figure 2-3, the tasks within the QM group are Process Monitoring and Improvement Services and ITSM Support Services.

Process Monitoring and Improvement Services Audit Findings

CST Division GTMs

IT Service Management (ITSM) Support Services

ITSM Ser Process Appraisal Results Process Change Document Quality Management (QM) CLIN Grouping Status &Performance Reporting Approved Process Improvement Plan Approved Auditing Plan Survey of IT services & service assets Status & Performance Reporting vice Plan Audit Findings CST Division Management

· SSPM

· GTMs

Legend:

External Processes

CLIN

Input/Output

Figure 2-3. Quality Management Services

Table 2-3 shows the major inputs and outputs of the two (2) tasks of the QM Services.

Table 2-3. QM Components Inputs and Outputs

QM Services Input Source Output Destination

CST Division GTMs

· Status & Performance Reporting

· Audit Findings

· Process Change Document

· Process Appraisal Results

· Audit Findings

· ITSM Service CST Division Plan Management

2.4 Tasks 8, 10, 11 and 12 Configuration Management (CM)

As depicted in Figure 2-4, the tasks within the CM group are Configuration Control Process (CCP), Tools / Repository Services, and PS.

MIS

· CA/CST Management

· CCB

· ECB

· ITCCB

· ICAB

· SDLC

· Others as needed External Decision Management

· ICAB

· ITCCB

Legend:

External Processes

CLIN

Input/Output Configuration Control Process Management

· CA-CCB

· ECB

· SDLC Support External Partners

· IRM

· Auditors

· OIG

· Diplomatic Security

· CA Business Partners Configuration Management (CM) CLIN Grouping CST Division GTMs

· Infrastructure

· GOTs

· COTs Portal Services

· Dashboards

· Reports

· Asset Management

Requirements Service Clients

· Developers

· IV&V

· Security

· Others as needed Tools / Repository Services

· Remedy

· Rational/CLM

· SharePoint Status & Performance Data Re Status & Performance Data Requirements Status & Performance Data Status & Performance Reporting

CR & CCR

Artifacts, Applications, Environments Artifacts & Deployed Changes Status, Performance & Asset Data Artifacts Submitted Artifacts Artifacts Status & Performance porting

Figure 2-4. Configuration Management Services External Partners and Service Clients provide input to the CST Division GTMs regarding requested changes and requirements. There are three (3) different sources of changes submitted to the GTMs:

1) Infrastructure: changes in the CA/CST IT environment

2) COTS: commercial products, purchased by CA/CST

3) GOTS: SW products, developed by CA/CST or its Contractors

Those changes are converted into change requests (CR) and configuration change requests (CCR) that are then sent to the CCP task for approval processing. Service Clients may retrieve artifacts, applications, or environments to support their activities and submit CM artifacts when their activities complete. CCP supports the SDLC Stage Gates Reviews (SGRs) and control boards by facilitating meetings and recording decisions and status for the Information Technology Change Control Board (IT-CCB) and the Information Resources Management (IRM) Change Advisory Board (ICAB). Tools and Repository Services provide controlled items from the repository to the members of the Stage Gate Reviewers (SGRs) and control boards. PS host status and performance information for all other tasks and other organizations within CA/CST. Finally, PS reports asset management status and provides other reporting capabilities.

Table 2-4 shows the major inputs and outputs of the three (3) tasks of CM Services.

Table 2-4. CM Services Components Inputs and Outputs

CM Service Input Source Output Destination

CCP

· Status and Performance PS Data

Tools and Repository Services

3 REQUIREMENTS

This section defines the task requirements for the STS Contractor. Each task follows a similar structure, identifying the goals and objectives, activities, deliverables, and status and performance measures. The Contractor shall execute all work activities in a timely and cost effective manner while maintaining the highest quality of performance.

3.1 Task 1: Program Management and Reporting (Firm Fixed Price (FFP))

3.1.1 Goals and Objectives

The Contractor shall effectively manage and administer all efforts performed under the contract.

The Contractor shall conduct a quarterly survey of satisfaction from CA/CST organizations, as directed by CA/CST.

Goals & Objectives

3.1.2 Objectives

The Program Manager shall oversee the following:

· Management Plans

· Progress Reports

· Deliverable Submission

· Invoice Submission

· Staffing

· Security

· Government Furnished Equipment (GFE)

· Travel

· Other Direct Costs (ODC)

· Quality Control

· Quality Assurance

· Special Format Requirements

· Portal and Trend Reporting

· Track risks and assign estimated dates for resolution

· Other topics as directed by the Government Additional activities not explicitly listed will be required to meet the performance requirements for this task.

3.1.2.1 Progress Reports

The Contractor shall provide Weekly Activity Reports (WAR), Bi-Weekly Status Reports, Contract Status Reports (CSR), and Monthly Progress Reports that comply with SSPM standards. Reporting requirements may be adjusted in the future as some reports are consolidated and the Contractor shall comply with any changes, recommended and mutually agreed upon by the Government. Meetings may be scheduled at the discretion of CA/CST and GTM leadership. The Contractor shall submit all reports to the GTM. The Contractor shall provide the following information as required:

· The Contractor shall prepare and present a CSR to CA/CST on a monthly basis.

· The Contractor shall prepare the CSR using the SSPM-provided CSR template.

· At least three (3) days before the CSR presentation, the Contractor shall ensure that the CSR is available through Portal Services and email the CSR presentation to

CA/CST-PMO@State.gov. In the event that the presentation is cancelled, the Contractor shall still complete and submit the CSR as outlined above.

· The Contractor shall track and update CA/CST of all action items assigned during the CSR presentation.

· The Contractor shall present the status of the action items at the next CSR.

3.1.2.2 Deliverable Submission

The Contractor shall submit all deliverables specified within the tasks, as well as any defined by CA/CST management, for control by CM services and dissemination by Portal Services.

The Government will provide comments to the Contractor within ten (10) business days, upon receipt of the draft documents. The Contractor shall provide a final document version within five

(5) business days after receipt of Government comments.

The Contractor shall write documentation in clear, concise language that is verifiable. The Contractor shall produce final documents without typographic and grammatical errors, and shall be formatted according to standards provided by the Government. The Contractor shall send all electronic deliverables to the designated GTM’s DoS OpenNet e-mail inbox (address to be supplied later).

3.1.3 Deliverables

The Contractor shall complete and provide Table 3-1 with details on proposed deliverables. Throughout the life of the project there may be additional deliverables added, deleted, or edited.

Table 3-1. Deliverables for Task 1 (Program Management and Reporting)

Table 3-2. Status and Performance Metrics for Task 1 (Contract Management and Reporting)

· Track number of revisions after scheduled submission

· Track rejected deliverables

· Track authorizations

· Track status of staff OCI/PCI

· Track status of staff badging

· Percentage of Funded (SB) Vice Requirement should also include prime contractor (if applicable) plus any SB subcontractors.

· Compare actual spending versus plan/limit

· Compare risk resolution dates versus estimated date

· Track security incidents

3.2 Task 2: Transition Activities (T&M)

The Contractor shall plan and manage the transition from the incumbent Contractor(s), which shall be no more than 120 calendar days from the award date. The Contractor shall provide a plan to complete the transition of responsibilities.

The Contractor shall successfully plan and manage the transition from the existing STS test facility to a STS test facility (if transitioning to a facility different than current Arlington, VA location). The Contractor shall work with CA/CST COR as appropriate, to transition to the facility, if necessary.

3.2.1 Goals and Objectives Transition-In

The goal of Transition-In is to successfully transition the responsibilities of the current three (3) incumbent Contractors to the new Contractor to include personnel, facility, and all other associated administrative actions. The Government currently has three (3) contracts for IV&V, Configuration Management, and Remedy services, in which the new Contractor shall coordinate and cooperate with to ensure seamless transition-in activities.

Transition-In shall provide a continuity of service in the areas supported by the incumbent Contractors while the new Contractor learns how to support these services. In addition, the Contractor shall work with the previous Contractors to obtain at a minimum, a working knowledge of each of the tasks of this Statement of Work. The Contractor shall include plans in the Transition Summary Report to complete the transition of any service that has not been transitioned or where the service transition is not completed during the transition period.

Another objective of Transition Activities is to provide a fully functional STS test facility at the end of the transition. The Contractor shall ensure that there are no service disruptions to IV&V operations.

3.2.2 Objectives

This section describes the requirements for activities to be performed under this task and their interaction, if appropriate. The activities for this task are as follows:

· Determine Transition Tasks and Requirements

· Develop Transition-In Plan (TiP)

· Implement Transition

· Transition Closeout

· Portal and Trend Reporting

3.2.2.1 Determine Transition Tasks and Requirements

The Contractor, in consultation with the Government, shall determine the transition tasks and requirements, which include:

· Coordinate Contract Kick-Off Meeting (within 14 business days of contract award). At a minimum the kick-off meeting includes:

· Review of schedule activities and milestones

· Roles and responsibilities

· Expectations

· Risks

· Resources

· Communication

· Determine transition tasks, requirements, and transition items needed for the transition plan.

3.2.2.2 Develop Transition-In Plan

The Contractor shall develop a TiP to include, but not be limited to:

· Coordination with the Government and incumbent Contractor to develop a draft TiP.

· Review and comment of the draft TiP by GTM, COR, and incumbent contractor.

· Submission of a final TiP within 15 business days following contract award.

3.2.2.3 Implement Transition

The Contractor shall plan and execute a Transition Implementation to include, but not be limited to:

· Managing the transition from the current Contractor(s).

· Managing how the incoming STS Contractor will work with the incumbent Contractors to transition STS support services.

· Mitigating transition risks and conflict resolution.

· Managing the transition from the current facility to a different facility if necessary.

· Transferring Government Furnished Equipment (GFE)

3.2.2.4 Transition Closeout

The Contractor shall follow FAR 52.237-3, Continuity of Services (JAN 1991) to ensure transition is accomplished without difficulty until the new Contractor can fully provide services under the contract, which must be continued without interruption. Any failure to fully implement this clause, including attendance at all scheduled meetings and timely submission of all required documents, may result in a negative past performance evaluation and unfavorable responsibility determination.

The Contractor shall plan and execute a Transition Closeout to include, but not be limited to:

· A Transition-Out Plan (ToP), in accordance with Federal Acquisition Regulation (FAR) 52.237-3.

· A demonstration of the new contractor’s ability to successfully support the Operation and Maintenance (O&M) of each item transitioned from the incumbent contractor.

· A Transition Summary Report, which is used for Government acceptance of the follow-on Contractor within ten (10) business days after the end of the Initial Transition Period.

· Demonstrate that the new facility is operational.

· Demonstrate that the GFE is returned to the Government.

3.2.3 Deliverables

The Contractor shall complete the items within Table 3-3 with details on proposed deliverables. Throughout the life of the project there may be additional deliverables added, deleted, or edited.

Table 3-3. Deliverables for Task 2 (Transition Activities)

GTM, COR

90 days before the end of the period of performance

N/A

Deliverable 12

Transition Summary Report (Final)

GTM, COR

10 days after resolution of GTM/COR comments

Update as Required

Deliverable Facility Transition Summary Report (Final)

GTM, COR

10 days after resolution of GTM/COR comments Update as Required

Table 3-4. Status and Performance Metrics for Task 2 (Transition Activities)

· Track completion of Transition tasks and requirements

· Identify procedures to be documented

Implement Transition

· Track and monitor Transition Risks and Issues

· Compare planned vs. actual completion of Transition activities

· Track and monitor completion of procedures to be documented

Transition Closeout

· Compare planned vs. actual Transition Closeout activities

· Compare planned vs. actual completion of Transition requirements

· Compare planned vs. actual completion of documented procedures

· Track and monitor operational status of facility (as needed)

· Track and monitor closure of existing facility (as needed)

3.3 Task 3: Facility Management (T&M)

The Contractor shall provide Facility Management services to support O&M of the Contractor provided STS test facility. The facility shall total at least 35,000 square feet in one building to include space for warehouse, labs, conference rooms (minimum of 4 rooms to hold a range of 10-35 people per room), server room, and secure storage space. The Contractor shall provide proof of warehouse by providing an active and valid lease or contingent lease of the facility, as well as a configuration of the facility to show available space and the location for labs, conference rooms, server room, and secure storage space.

The server room shall meet the guidelines found at FAH-10 H-270.

3.3.1 Goals and Objectives

The goal of Facility Management is to manage and maintain facility computing resources, communication and network infrastructure, and facility environment (heating and cooling) remain operational and secure. In the event of scheduled or unscheduled facility maintenance, the Contractor shall manage the maintenance effort. The objectives of Facility Management are:

· The Contractor shall manage and maintain the STS test facility during standard CA/CST operating hours from 8:15 am – 5:00 pm, Monday–Friday Eastern Standard Time.

· The Contractor shall manage scheduled maintenance or facility service needs outside of standard CST operating hours as scheduled, as otherwise specified in this SOW, or as requested by the COR or GTM.

· The Contractor shall develop and maintain the Facility Management Service Level Agreement (SLA) to reflect the needs of the CA/CST STS test facility.

3.3.2 Objectives

This section describes the requirements for activities to be performed under this task and their interaction, if appropriate. The activities for this task are as follows:

· Provide Facility Management Plan (FMP)

· Facility Management

· Facility Management Reporting

· Portal and Trend Reporting Additional activities not explicitly listed will be required to meet the performance requirements for this task.

3.3.2.1 Provide Facility Management Plan

The Contractor shall develop a FMP that includes, but is not be limited to, procedures for:

· HW installation

· SW installation

· Operating and staffing

· Maintenance

· Access Control

· Property Control

3.3.2.2 Facility Management

The Contractor shall:

· Maintain a secure and independent test facility

· Manage scheduled and unscheduled facility maintenance

· Document the operational status of the test facility

· Coordinate and document IT services

· Collect data for performance metrics

3.3.2.3 Facility Management Reporting

The Contractor shall maintain and provide Facility Management Reporting including:

· Current Non-expendable Property Application (NEPA) Inventory

· Federal Purchase Log

· Facility Maintenance Log

· The Contractor shall provide input to the Weekly Activity Report on changes to inventory, purchases, and maintenance

3.3.3 Deliverables

The Contractor shall complete the items within Table 3-5 with details on proposed deliverables. Throughout the life of the project there may be additional deliverables added, deleted, or edited.

Table 3-5. Deliverables for Task 3 (Facility Management)

Deliverable 18

FMP (Final)

GTM, COR

10 days after resolution of GTM/COR comments

Update as Required

GTM

30 days after GTM/COR request Update as Required

GTM

30 days after GTM/COR request Update as Required

GTM

30 days after GTM/COR request Update as Required

Table 3-6. Status and Performance Metrics for Task 3 (Facility Management)

· Track and Monitor Facility operational status during normal operating hours

· Track and Monitor Facility operational status outside of normal operating hours

· Track and monitory mean time between failure

· Track and monitory average length of failure

Facility Management Reporting

· Track and monitor the current NEPA Inventory within 1 hour after receipt of goods

· Track and monitor the Federal Purchase Log within 1 day after receipt of goods or placement of orders

· Track and monitor the Facility Maintenance Log within 1 day after receipt of goods or placement of orders

3.4 Task 4: Formal Independent Verification and Validation Services (T&M) The Contractor shall provide formal IV&V is to provide an (IV&V) of all products developed for CA/CST.

3.4.1 Goals and Objectives

The goal of Formal IV&V Services is to provide an IV&V of all products developed for CA/CST. The Contractor shall conduct all COTS and GOTS patch testing as directed by IRM or CA/CST corporate level IT.

The objectives of IV&V Services include:

· Configuring, administering, and operating testing environment(s), in collaboration with CM Services.

· Conducting IV&V activities independent of any other product development, testing, validation, and verification activities.

· Ensuring that product requirements and acceptance criteria are the basis for an IV&V test.

· Actively participating in the review of product requirements and development of acceptance criteria.

· Ensuring that verification and validation of product operation, maintenance, documentation, and training are included in IV&V testing.

· Producing IV&V Findings and a Go / Go with conditions / No Go recommendation to be distributed to relevant stakeholders.

· Developing verification methods to include, but not limited to, inspections, peer reviews, audits, walkthroughs, analyses, architecture evaluations, simulations, testing, and demonstrations of products.

· Including the definition of support tools, test equipment and SW, simulators and facilities.

· Ensuring that verification provisions are embedded in product and product component requirements, designs, developmental plans, and schedules.

· Executing IV&V testing, the Contractor shall follow delineated SW evaluation guidelines, take advantage of automated tools authorized by the Government, utilize mobile and smart device testing solutions and analysis, and provide analysis reports based on all data received and support remediation of issues.

3.4.2 Objectives

This section describes the requirements for activities to be performed under this task and their interaction, if appropriate. The activities for this task are as follows:

· Validate Release Elements

· Prepare IV&V Test Plan

· Prepare IV&V Test

· Perform IV&V Testing

· IV&V Analysis & Reporting

· Portal and Trend Reporting Additional activities not explicitly listed will be required to meet the performance requirements for this task.

3.4.2.1 Validate Release Elements

The development GTM will provide notification of all product release elements, which shall be retrieved for IV&V testing by CM Services. The Contractor shall develop the Software and Product Requirements List (SWPRL), a set of generic SW requirements that reflect common SW issues that occur in the operational environment, but may not be explicit in the product requirements. The Contractor shall coordinate with CST personnel and management to define and approve the SWPRL. The Contractor shall submit the SWPRL to CM Services for control within the CA/CST repositories. The Contractor shall include appropriate SW requirements from the SWPRL into the release element product requirements to be validated.

The Contractor shall validate release elements to be used for IV&V activities, to include:

· SW components

· Documentation

· Release package items

· Product requirements, i.e. the cumulative set of requirements developed for that product over the entire lifespan (all versions) of the product. The release elements will be validated using criteria that include:

· Is the release element verifiable?

· Is the release element clear and concise?

· Is the release element complete? o Is the release element consistent? o Is the release element traceable? o Is the release element viable?

· Is the release element necessary? o Is the release element testable? o Is the release element correct?

The Contractor shall validate the acceptance criteria for each product requirement. The Contractor shall submit the Validation Results to CM Services for control within the CA/CST repositories. Based on the validation results, the Contractor shall determine whether IV&V can:

· Proceed without constraints

· Proceed with constraints

· Not proceed

3.4.2.2 Prepare IV&V Test Plan

The Contractor shall develop an IV&V Test Plan to include: required test resources (including how many people are using, date, timeframe, and dependencies), test activities (including CCR requirements) and a test schedule.

· IV&V plan shall address increased activity by having procedures to supplement staff and access subject matter experts to meet an increased workload. For instance, the increased activity may be derived from congressional mandates, peak summer season, or special global events.

· IV&V plan may need to address IV&V priority with other ongoing IV&V activities. Critical and high priority projects – in order of priority -- will receive preference when multiple IV&V activities requiring the same limited resources.

The Contractor shall submit the draft IV&V Test Plan to the COR and the GTM for review and to CM Services for control within the CA/CST repositories.

The Contractor shall submit a final IV&V Test Plan to the COR and the GTM for approval and to CM Services for control within the CA/CST repositories a minimum of five (5) days prior to the start of IV&V Testing.

The Contractor shall acquire prior approval by the GTM for any deviations from the IV&V Test Plan.

3.4.2.3 Prepare IV&V Test

The Contractor shall execute a comprehensive set of test cases to verify product requirements and SW requirements for each release. The Contractor shall submit these test cases to CM Services for control within the CA/CST repositories.

The Contractor shall coordinate with the GTM to ensure testing is properly handled.

The Contractor shall prepare test environment(s) that emulates a production like environment. The Contractor shall prepare environments to include domestic and overseas support where necessary.

The Contractor shall install the product to be tested in cooperation with the product development team as needed.

3.4.2.4 Perform IV&V Testing

The Contractor shall evaluate SW functionality using the test cases, retrieved from CM Services control. Where possible, the Contractor shall use innovative and automated tools and resources that can be or are approved for use to evaluate SW functionality against defined requirements.

The Contractor shall ensure that the application processes as expected and test the application efficiencies. Government experience shows that 70% of the Test Incident Reports (TIRs) that are generated identify true defects (i.e., fixed by SW development groups or converted to CCR) as opposed to comments or recommendations.

3.4.2.5 IV&V Analysis and Reporting

The Contractor shall prepare TIRs and Advisory Notices (ANs) to support IV & V findings. The Contractor shall deliver these documents one (1) business day after completion of the test or earlier as required by the COR or GTM.

The Contractor shall present these documents to the appropriate stakeholders (requirements development, system designer, system developers, etc.) with a recommendation of Go / Go with conditions / No Go.

3.4.3 Deliverables

The Contractor shall complete the items within Table 3-7 with details on proposed deliverables. Throughout the life of the project there may be additional deliverables added, deleted, or edited.

Table 3-7. Deliverables for Task 4 (Formal IV&V Services)

GTM, COR

5 days after package is received for test

As required

Deliverable 24

IV&V Test Plan (Draft)

GTM,

COR, and CM Services 10 days prior to commencement of formal testing

N/A

Deliverable 25

IV&V Test Plan (Final)

GTM,

COR, and CM Services 1 day after resolution of GTM/COR comments

Update as Required

Deliverable 26

IV&V TIR

GTM, COR, &

others as needed 1 day after completion of a test

Deliverable 27

IV&V Advisory Notices

GTM, COR, &

others as needed 1 day after completion of a test

As required

GTM, COR

5 days after completion of a test

Table 3-8. Performance Measures for Task 4 (Formal IV&V Services)

· Track and monitor Design anomaly density

· Track and monitor Project Duration

Prepare IV&V Test

· Track and monitor Provisioning Time Objective (PTO)

· Track and monitor Recovery Time Objective (RTO)

· Track and monitor Recovery Point Object (RPO)

Perform IV&V Testing

· Track and monitor Productivity

· Track and monitor Schedule Predictability

· Track and monitor Implementation Anomaly Density

· Track and monitor Test Anomaly Density

Reporting

· Track and monitor Post-release Defects

· Track and monitor Post-release Defect Density

3.5 Task 5: COTS and GOTS Product Review (T&M)

The Contractor shall provide IV&V services for the evaluation of COTS and GOTS products and patches, at the direction of IRM and CA/CST.

3.5.1 Goals and Objectives

The goal of the COTS and GOTS Product Review is to provide an independent review of products that may be used to meet technological need(s) of CA.

3.5.2 Objectives

This section describes the requirements for activities to be performed under this task and their interaction, if appropriate. The activities for this task are as follows:

· Validate Requirements

· Prepare Product Test Plan

· Prepare Independent Product Test

· Perform Product Test

· Product Test Analysis and Reporting

· Define the Technology Need (Optional)

· Define the Technology Need Requirements (Optional)

· Conduct Market Survey (Optional)

· Portal and Trend Reporting

3.5.2.1 Validate Requirements

The Contractor shall validate product requirements to be used for Independent Product Test activities, which include the cumulative set of requirements developed for that product over the entire lifespan (all versions) of the product. The Contractor shall validate the requirements using the criteria specified in Section 3.4.2.1. The Contractor shall determine, based on the validation results, whether the Independent Product Test can proceed without constraints, proceed with constraints, or not proceed.

3.5.2.2 Prepare Independent Product Test Plan (IPTP)

The Contractor shall develop an IPTP to describe the required test resources, test activities (including CCR requirements) and a test schedule.

The Contractor shall address increased activity in the IPTP by having procedures to supplement staff and access subject matter experts (SMEs) to meet an increased workload if required.

The Contractor shall submit the draft “IPTP” to the stakeholders, the COR, and the GTM for review.

The Contractor shall submit the final “IPTP” to the COR and the GTM that addresses the comments made by the stakeholders, the COR, and the GTM on the draft IPTP.

The Contractor shall obtain prior approval of the GTM for deviations from the IPTP.

3.5.2.3 Prepare Independent Product Test

The Contractor shall execute a comprehensive set of test cases to verify product requirements and the SWPRL, as described in Section 3.4.2.1, for each release. The Contractor shall submit these test cases to CM Services for control within the CA/CST repositories.

The Contractor shall deliver an SLA Status Report to stakeholders, the COR, and the GTM prior to the start of Product Testing.

The Contractor shall retrieve a test environment(s) for the Independent Product Test from CM Services that emulates a production environment, including domestic and overseas environments where required.

The Contractor shall operate and configure test environment(s), obtained from CM Services, for the Independent Product Test.

The Contractor shall install the product(s) to be tested utilizing product technical support as needed.

3.5.2.4 Perform Independent Product Test

The Contractor shall evaluate product functionality by taking advantage of innovative and automated tools and resources that can be or are approved for use to evaluate SW functionality against defined requirements. The Contractor shall validate the application efficiencies and processes. The Contractor shall test print exemplars of CA products, and review quality and validate authenticity. Government experience shows that 70% of the TIRs that are generated identify true defects (i.e., fixed by software development groups or converted to CCR) as opposed to comments or recommendations.

3.5.2.5 IV&V Analysis and Reporting

The Contractor shall prepare test incident reports and advisory notices that support Product Test findings are delivered five (5) business days after completion of the test or earlier as required by the COR or GTM. The TIR and ANs are submitted to CM Services for control within the CA/CST repositories.

The Contractor shall provide the validation results and findings to the relevant stakeholders (requirements development, system designer, system developers, etc.) with a recommendation of Go / Go with conditions / No Go.

3.5.2.6 Define the Technology Need (Optional)

The Contractor shall interface with stakeholders to define the goals and objectives of products that may support a technology need, as directed by the GTM.

The Contractor shall provide a Statement of Need (SoN) that defines the goals and objectives of the technology, to include:

· Confirmation that the Need Is Solution Independent

· Validation that the Scope of the Need Is Appropriate

· Definition of the Need Criteria The Contractor shall submit the draft SoN to the stakeholders, the COR and the GTM for review.

The Contractor shall submit the final SoN to the COR and the GTM that addresses the comments made by the stakeholders, the COR, and the GTM on the draft SoN.

3.5.2.7 Define the Technology Need Requirements (TNRs) (Ad hoc) The Contractor shall provide requirements that correspond to the Need Criteria defined in the SoN to meet criteria, as directed by the GTM.

The Contractor shall submit the draft TNRs to the stakeholders, the COR and the GTM for review.

The Contractor shall submit the final TNRs to the COR and the GTM that addresses the comments made on the draft TNRs by the stakeholders, the COR, and the GTM.

3.5.2.8 Conduct Market Survey (Ad hoc)

The Contractor shall perform a Market Survey ,with requirements provided by the Government, using the technology SoN and Need Requirements to assemble a list of viable products for evaluation, as directed by the GTM. The Contractor shall use authoritative and independent product and market surveys and research to determine viable products.

The Contractor shall submit the draft Market Survey Results to the stakeholders, the COR and the GTM for review and to CM Services for control within the CA/CST repositories.

The Contractor shall submit the final Market Survey Results to the COR and the GTM that addresses the comments made by the stakeholders, the COR, and the GTM on the draft Market Survey Results. The Contractor shall submit the draft and final Market Survey Results to CM Services for control within the CA/CST repositories.

3.5.3 Deliverables

The Contractor shall complete Table 3-9 with details on proposed deliverables. Throughout the life of the project there may be additional deliverables added, deleted, or edited.

Table 3-9. Deliverables for Task 5 (COTS and GOTS Product Review)

Item #

Product/Deliverable

Stakeholder

Initial Submission Additional Submission (s)

Validation Results

GTM, COR

5 days after package is received for test

As required

Deliverable 30

IPTP (Draft)

GTM, COR

10 days prior to commencement of formal testing

Deliverable 31

IPTP (Final)

GTM, COR

1 day after resolution of GTM/COR comments

Update as Required

GTM, COR

1 day after completion of a test

As required

Advisory Notices

GTM, COR,

& others as needed 1 day after completion of a test

Deliverable 34 COTS and GOTS Findings and Recommendations Report

GTM, COR

5 days after completion of a test

Deliverable 35

SLA Status Report

GTM, COR,

and other Stakeholders as needed

Table 3-10. Status and Performance Measures for Task 5 (COTS and GOTS Product Review)

· Track and monitor Design anomaly density

· Track and monitor Project Duration

· Track and monitor Recovery Point Object

Perform Product Test

· Track and monitor Productivity

· Track and monitor Schedule Predictability

· Track and monitor Implementation Anomaly Density

· Track and monitor Test Anomaly Density

3.6 Task 6: Integration Testing and Development Support Services (T&M) The Contractor shall provide services to manage and support the Integration Testing and development support environments and resources that allow the product developers to integrate and test their product(s) in a production-like environment. The Contractor shall provide installation, integration, and environment level troubleshooting support for developers as needed. The contractor is to develop a cloud migration plan. The Government anticipates integration testing and development support services to be hosted in a cloud environment during the life of this contract. The Contractor shall be prepared to manage the integration testing and development support services in a cloud environment.

3.6.1 Goals and Objectives

The goal of integration testing and development support services is to provide the environment and resources that allow the product developers to integrate and test their product(s) in a production-like environment and to manage and support integration testing and development support services used.

3.6.2 Objectives

This section describes the requirements for activities to be performed under this task and their interaction, if appropriate. The activities for this task are as follows:

· O&M Support Services

· Development Environment Support Services

· Backup and Retrieval Services

· Migration Support Services

· Verify Test Plans and Schedules

· Prepare Integration Support Test Plan (ISTP)

· Prepare for Integration Testing

· Integration Testing Installation and Support

· Support Test Analysis and Reporting

· Portal and Trend Reporting

3.6.2.1 Operation and Maintenance Support Services

The Contractor shall operate and administer environments, which are obtained from and controlled by CM Services.

The Contractor shall update environments to reflect the CA/CST production environment, in collaboration with CM Services.

The Contractor shall operate, maintain, and keep current HW, SW, and systems that provide failover capabilities for CA/CST.

The Contractor shall operate and maintain environments that are logically isolated from other environments, but representative of the Production environment.

The Contractor shall operate environments that are fully scalable.

The Contractor shall provide management support for all SW servers (database, web servers, and application enablers), operating systems, hypervisors, HW and network infrastructure that are included in the environment boundary.

The Contractor shall provide web hosting infrastructure and application integrations support.

The Contractor shall provide Operations Management and operations of the environment to be inclusive of all the above components and their respective integration.

The Contractor shall provide network storage, server, and virtualization layer management to include performance of internal technology and refresh cycles applicable to the environment.

The Contractor shall provide and document patch management appropriate to all components within the provider’s boundary and to adhere to DoS standards.

3.6.2.2 Customer Environment Support Services

The Contractor shall manage / administer, in collaboration with CM, the HW and SW necessary for service activities, which will be provided by the Government and / or CA/CST.

The Contractor shall provide remote access to testing environments for CA/CST application contractors.

The Contractor shall provide customer support services, in collaboration with CM, including an automated means of promoting or demoting products to alternative environments.

The Contractor shall provide customer support for internal client bases, including, but not limited to, system owners and managers, as required.

The Contractor shall provide tools for analyzing usage specific trends as they relate to the public users of the system.

The Contractor shall use CM Services by following CA/CM processes to control the functionality of the environment without hindering the ability of CA/CST Content Developers / Managers to efficiently add new functionality, integration, and / or content delivery mechanisms.

The Contractor shall provide a trusted secure communication channel to support the Government’s PIV Card authentication (dual factor method) of remote access in accordance with OMB M-11-11.

The Contractor shall provide maintenance of user profiles presented to the user at time of login.

3.6.2.3 Backup and Retrieval Services

The Contractor shall provide, in collaboration with CM, backup and retrieval services for environments and environment artifacts. The Contractor shall manage the logs and data consistent with best practice approaches.