IETSS DRAFT PWS v2 for RFI FINAL to POST.pdf

PERFORMANCE WORK STATEMENT (PWS)

DEPARTMENT OF VETERANS AFFAIRS (VA)

Office of Information and Technology (OIT) Compliance, Risk and Remediation (CRR)

Independent Verification and Validation (IV&V)

Electronic Health Record Modernization Integration Office (EHRM-IO) Integrated Testing (IntT)

Independent Enterprise Testing and Support Services (IETSS)

Date: April 11, 2024

VA-24-00050415

PWS Version Number: 2.0

Independent Enterprise Testing and Support Services (IETSS)

Contents

1.0 BACKGROUND

1.1 OIT IV&V TMO and TC & EHRM-IO Integrated Testing FUNCTIONAL AREAS

1.1.1 Test Management and Operations

1.1.1.1 Operations and Management

1.1.1.2 Test and Evaluation

1.1.1.3 Testing Technology Support

1.1.1.4 Test Process and Quality Management

1.1.2 Test Center

1.1.2.1 Testing Systems Engineering and Implementation & Testing Technology Support 8

1.2 OIT SERVICES AND METHODOLOGY

1.2.1 Services Executed During SDLC and SACI Planning Phase

1.2.2 Services Executed During SDLC and SACI Test Services Execution Phase

1.2.3 Processes Executed in Support of Monitoring and Control

1.2.3.1 Workload Forecasting

1.2.3.2 Support of Test Center Systems

1.3 EHRM-IO INTEGRATED TESTING SERVICES & METHODOLOGY

2.0 APPLICABLE DOCUMENTS

3.0 SCOPE OF WORK

3.1 ORDER TYPE

4.0 PERFORMANCE DETAILS

4.1 PERFORMANCE PERIOD

4.2 PLACE OF PERFORMANCE

4.3 TRAVEL (T&M)

5.0 SPECIFIC TASKS AND DELIVERABLES

5.1 PROJECT MANAGEMENT (FFP)

5.1.1 Contractor Project Management Plan

5.1.2 Contract Key Personnel

5.1.3 Reporting Requirements & Management Meetings

5.2 TEST AND EVALUATION SUPPORT (T&M)

5.2.1 Work Product Reviews

5.2.2 Information Gathering/Requirements Analysis

5.2.3 Testing Intake Assessment/Criticality Analysis and Risk Assessment

5.2.4 Test Plan Summary

5.2.5 Execute Test Plan, Analyze Results, & Report Findings

5.2.5.1 Requirements Validation Testing

5.2.5.2 Performance Testing

5.2.5.3 Integration/Interoperability Testing

5.2.5.4 Test Observation and Validation

5.2.5.5 Software Code Quality Check Scanning

5.2.5.6 Automated Testing

5.2.6 Patient Safety Issue (PSI) Testing

5.2.7 Section 508 Compliance Support Services

5.2.8 Exploratory Testing

5.2.9 Mobile and Portable Device Application Testing

5.2.10 Test and Evaluation Management Support

5.3 Testing and technology support

5.4 TEST SYSTEMS ENGINEERING AND IMPLEMENTATION SUPPORT (T&M) . 31

5.4.1 Administration, Architecture, and Engineering

5.4.2 Operations and Maintenance and/or Continuity of Operations Services

5.4.3 Database Administration Services

5.4.4 Asset Management

5.4.5 Tools Assessment

5.4.6 Environment/Infrastructure Management Support Services

5.5 TEST PROCESS AND QUALITY MANAGEMENT SUPPORT (T&M)

5.5.1 Process Improvement Support

5.5.2 Lessons Learned and Quality Reviews

5.5.3 Artifact Review

5.5.4 Content Reviews

5.5.5 Operations and Management support

5.5.6 Metric and Measures Support

5.5.7 CQIP Briefings

5.5.8 Root Cause Analysis with Corrective Action Plan

5.5.9 Technical Writing Services

5.5.10 Knowledge Management Services

5.5.11 Process Management Development and Implementation

5.5.12 Assessment & analysis for strategic and tactical planning and techinical support of OIT

& EHRM-IO (T&M)

5.6 OPTIONAL TASK - TRANSITION SUPPORT (T&M)

1.0 BACKGROUND

The mission of Department of Veterans Affairs (VA), Office of Information & Technology (OIT) is to collaborate with business partners to create the best experience for Veterans. In meeting these goals, OIT strives to provide high quality, effective, and efficient Information Technology (IT) services to those who provide care to Veterans at point-of-care as well as throughout all points of each Veteran’s health care, in an effective, timely and compassionate manner. VA depends on Information Management/Information Technology (IM/IT) systems to meet mission goals.

OIT is the steward of VA’s IT resources and is responsible for ensuring the efficient and effective operation of VA’s IT Management System to meet mission requirements defined by the VA Secretary and other key VA officials. OIT partners with other VA business units to deliver available, adaptable, secure, and cost-effective technology services to VA. Technology services are introduced into VA via two primary approaches: in-house development or acquisition of Commercial Off The Shelf (COTS) and Government Off The Shelf (GOTS). These approaches follow industry, Government, and VA standards for technology, including but not limited to the phases of Software Development LifeCycle (SDLC) for development and Software Acquisition, Configuration, and Implementation (SACI) for COTS and GOTS.

OIT provides a consolidated enterprise-wide approach to identify, select, prioritize, and successfully execute a portfolio of IT Services. OIT further provides management and oversight for major IT initiatives, to improve project execution and deliver better outcomes to internal and external business partners and to Veterans.

Within OIT and reporting to the VA’s Principal Deputy Assistant Secretary for Information and Technology, is the IT Service area Compliance, Risk, and Remediation (CRR) wherein Independent Verification and Validation (IV&V) Test Management and Operations (TMO) and the Test Center (TC) reside. At present, the services provided by IV&V support many internal and external customers. The two primary customers internal to VA are: 1) OIT’s Software Product Management (SPM), an OIT IT Service area which maintains VA’s Legacy and some modernization Products, and 2) the Electronic Health Record Modernization Integration Office (EHRM-IO), which is a VA major Program office responsible for the implementation of VA’s modernization of their Electronic Health Record (EHR). The VA Supply Chain Modernization (SCM) program is a VA Program office that may require testing activities under this contract.

SCM will acquire and implement department-wide supply chain solutions, and after various research and assessments have concluded may become a customer engaging these services.

As the VA champion for IT quality and a risk-based approach to software testing, IV&V TMO & TC provides testing services and test environments enabling EHRM-IO and OIT program and project managers to deliver and deploy stable, high quality information technology solutions on-time and within budget.

While engaging in partnerships with the EHRM-IO, OIT organizations, VA administrations, and other external agencies, TMO and TC measures and examines delivery and deployment processes, quality, and capabilities within VA to enable a culture of continuous improvement.

This culture leads to enhanced program and project quality through early engagement in development or acquisition lifecycles, agile delivery of test services and test environments, and elimination of process redundancy.

Strategic priorities include:

• Align resources and capabilities with VA mission-critical priorities.

• Communicate, collaborate, and operate transparently with internal and external stakeholders, examples include: EHRM-IO Integrated Testing, OIT CRR, OIT SPM, OIT Infrastructure Operations Service (IOS), OIT leadership, and the U.S. Department of Defense (DoD).

• Promote a flexible and adaptive culture that leverages quality metrics and analytics to optimize services delivery.

• Continuously improve existing core and supporting business processes and services and develop new processes and services as VA and OIT priorities evolve.

• Commit to service excellence through open communication and early engagement with customers.

Independent Verification & Validation TMO & TC are the OIT organization that provides the services through this contract primarily for the independent test and evaluation of software applications within the OIT portfolio. EHRM-IO Integrated Testing is the organization that utilizes TMO and TC services through this contract for the independent test and evaluation of the capabilities and solutions that make up the electronic medical record.

On May 17, 2018, VA entered into a ten-year indefinite-delivery, indefinite-quantity contract with Cerner Government Services, Inc. (Cerner now Oracle Health) to acquire the Electronic Health Record (EHR) system being deployed by the Department of Defense (DoD) and related services for deployment and transition across the VA enterprise in a manner that meets the VA’s needs and enables seamless healthcare to Veterans and qualified beneficiaries. Procurement of a single, common system across VA and DoD will achieve VA’s goal of seamless care for Veterans by facilitating the transition of active-duty military members to VA and improving their timely access to the highest quality of care.

The Oracle Health EHR solution shall provide VA with a single system that can store and retrieve administrative, clinical, laboratory, radiology, pharmacy, and scheduling data, and interact with other internal and external systems. Modernization of VA’s EHR will support a Veteran-centric, team-based care model, with modern decision support tools; clinical information content services; identification, communication and standardization of care paths; and resource provisioning. This will improve interoperability, performance, and user experience for a majority of the health care delivery and ancillary teams responsible for directly or indirectly providing health care services; and in turn will improve quality, safety, and timeliness of health care services delivered to Veterans.

Oracle Health provides a comprehensive EHR solution and services. The VA Electronic Health Record Modernization Integration Office (EHRM-IO) and the DoD Healthcare Management System Modernization (DHMSM) are moving to a joint release management of two release cycles per year and releases of site-specific capabilities with the EHRM-IO site deployments.

EHRM-IO Integrated Testing (IntT) is the champion for EHRM-IO end-to-end IT quality and testing through the provision of software quality assurance (SQA) software testing environments, testing and evaluation services, and software development projects. IntT collaborates with the program and project managers to support them in the delivery and deployment of high-quality, stable, interoperable IT solutions to reduce risk and minimize the impact of production defects to the provided Veteran services.

The EHRM-IO Program is the program executive office charged with the preparation, deployment, and transition to a new electronic health record (EHR) within VA. The program will ensure that the process of deploying the new EHR is done in a manner that meets VA needs and supports seamless healthcare to Veterans and qualified beneficiaries. The Integrated Testing program within the EHRM-IO manages and oversees the execution of the test and evaluation activities across the EHRM program and its products. The testing community that makes up the EHRM-IO Integrated Testing Program is comprised of three main organizations: Integrated Testing Enterprise Test (IntT-ET), Office of Functional Champion, Functional Test & Evaluation (OFC Test/OFCt), and the Oracle Health Testing & Quality Assurance (T&QA). The resources delivering these services are a mix of EHRM-IO IntT VA resources and IntT-ET contractor services through this IETSS contract; OFC Test VA & EHRM-IO contractor resources; and OIT TMO and TC VA resources.

The IETSS Contract will provide services in support of EHRM-IO IntT and OFC Test and OIT TMO and TC VA resources. IETSS resources will provide services that work closely and collaboratively with the Oracle Health T&QA team to ensure an integrated approach and schedule for the EHRM-IO Test and Evaluation lifecycle.

Historically, VA developed software applications in house, so the main operational focus for TMO and TC was to provide test environments and test services for VA-developed applications.

In mid-2017, VA announced a major shift from internally developed software applications to modernization efforts, which includes a significant use of commercial off-the-shelf (COTS) products. A key element of VA modernization is the effort to replace VA’s current EHR system, the Veterans Information Systems and Technology Architecture (VistA), with a COTS product.

For purposes of this PWS, the VA EHR solution will be referred to as the VA Electronic Health Record Modernization system (EHRM).

EHRM is based on the electronic health record solution provided to the Department of Defense (DoD) and is known as the Military Health System GENESIS, which is based on Oracle Health Millennium. The adoption of a single joint system between VA and DoD will enable patient data to reside in a common system with a seamless link between DoD, VA, and community providers. When implemented, the EHR System will provide access to authoritative clinical data sources; as the transition matures, it will become the authoritative source of clinical data to support improved population health, patient safety, and quality of care provided by VA.

1.1 OIT IV&V TMO AND TC & EHRM-IO INTEGRATED TESTING

FUNCTIONAL AREAS

OIT IV&V and EHRM-IO IntT perform services in the following major functional areas:

1.1.1 Test Management and Operations

Test Management and Operations (TMO) Associate Directorate serves as the software testing operations, monitoring, risk assessment, test planning & deliver, and test process management for IETSS. TMO is comprised of 4 key branches.

1.1.1.1 Operations and Management

Operations and Management (O&M) is a part of TMO, and supports all budget planning, funding execution, testing software and hardware acquisitions and maintenance activities for both TMO and TC, as well as EHRM-IntT and other major customers that may arise. O&M is predominantly resourced by VA government staff, and utilizes process management, knowledge management and reporting support.

1.1.1.2 Test and Evaluation

Test & Evaluation (T&E) are part of TMO. T&E are comprised of a multitude of teams, delivering analysis and testing services to the EHRM-IO Program and OIT. T&E teams analyze VA application requirements and COTS stakeholder project requirements to identify opportunities for increase in quality and reduction of risks. Informed from such analysis, T&E testing services teams recommend performing diverse software testing services.

The T&E analysis team are comprised of the initial forward-facing representatives in engagements with customers and analysts who conduct a methodical risk-based review and scoring of identified project requirements and other associated characteristics of a project or system. The analysis team collaborates with all internal and external parties during an engagement to identify appropriate testing services. The T&E analysis team then creates and provides a recommended course of action to engage the customer and prepare for delivery of the tailored suite of testing services unique to the software product.

The T&E testing services delivery team coordinates within the TMO/EHRM-IO IntT teams (including IV&V Test Center), and OIT SPM software project teams to establish testing environments, emulate interfacing systems, manage resources, and create and improve efficient repeatable processes. T&E provides skilled and experienced leadership, technical subject matter experts, testing engineers, and technicians to execute a vast array of software testing, including but not limited to Test Observation and Validation (TOV), interoperability testing, system integration (interface) testing (SIT), data migration, Risk-based Requirements Validation (RV) testing, performance testing (benchmarking, load, ramp-up, stress, endurance, spike), mobile/portable application testing, exploratory testing, static and dynamic Software Code Quality Check (SCQC) scanning and reporting. T&E also supports user acceptance testing and user functional testing of software development, COTS acquisition, and implementation programs. The primary example of which is providing testing resources through this IETSS contract for EHRM-IO Integrated Testing requirements.

1.1.1.3 Testing Technology Support

Testing Technology Support provides testing and technology support for both OIT testing & EHRM-IO IntT and major programs. Services include architecture, interoperability, VistA Data Service (VDS), creation & maintenance of tools to support high volume data migration testing, and service virtualization functionality. TTS provides test environment logical requirements, architecture diagrams, test data, service virtual components, and configuration of test environments for TMO testing projects.

1.1.1.4 Test Process and Quality Management

Test Process and Quality Management (TPQM) is focused on continuous review and improvement of testing capabilities and services provided by TMO, TC and EHRM-IO Integrated Testing. TPQM also directs the establishment of a system-wide definition and approach for executing services for all VA projects requiring testing services from TMO and TC, to include establishing and tracking metrics and measures, continuous review and improvement of processes, and artifact review. The focus of TPQM is ensuring that standardization and quality assurance processes are used within the OIT and EHRM IntT testing program. TPQM resources serve as process consultants and subject matter experts on the VA Testing Process and assist project teams with understanding the various testing processes and how they are performed throughout the SDLC and SACI phases. TPQM resources work closely with the portfolio/program management offices, project teams, and other VA stakeholders to define and improve the Software Quality Assurance program. The Testing Logistics Support (TLS) team within TPQM supports the day-to-day operation of IV&V & ERHM-IO IntT and provides structure to assist in the strategic and tactical planning necessary to accomplish IV&V management goals. TLS coordinates work plans and schedules with IV&V& EHRM-IO IntT managers and customers to provide testing environments and services in a manner that promotes a smooth flow of projects while ensuring an effective distribution of resources. Work performed by TPQM also results in constant monitoring of priorities, identifying the need for changes when appropriate and acting when necessary to implement changes.

1.1.2 Test Center

Test Center (TC) Associate Directorate serves as the test environment and infrastructure operations oversight for IETSS. TC is comprised of a key branch.

1.1.2.1 Testing Systems Engineering and Implementation, the Testing Systems Engineering and Implementation (TSEI) team provides the software, hardware, and physical plant infrastructure to support environments needed for testing activities, including but not limited to enterprise-level software testing, EHRM-IO Integrated Testing program testing, and OIT SPM project team testing (including project-level integration testing, development testing, and operations support for environments and systems). The Test Center (TC) Infrastructure is housed at the Bay Pines Office of Information and Technology Field Office, 10000 Bay Pines Blvd, Building 37, Bay Pines, FL 33744. Additionally, TSEI supports the T&E analysis team in identifying the criticality and risks for technical requirements (as needed) and provides risk assessment on the product architecture. The goal is to expand TC infrastructure as needed to support an increasing number of testing environments, which in turn increases testing capability that TMO can provide to project teams.

1.2 OIT SERVICES AND METHODOLOGY

IV&V TMO and TC is involved early and throughout all SDLC and SACI phases, currently via the VA OIT Veteran-focused Integration Process (VIP) methodology and/or the current Acquisition Lifecycle Framework (ALF), although TMO and TC evolves to accommodate methodology improvements and changes. TMO assists project teams with understanding all aspects of the OIT and EHRM-IO Test Processes, especially those that apply to individual projects. TMO also reviews all identified project requirements to analyze for level of risk, advise on testability, determine special hardware/environment needs for test planning, completes testing intake assessments (TIA) entrance criteria with project team collaboration, performs testing intake assessments, and performs T&E services.

OIT follows VIP Agile and software acquisition, configuration, and implementation (SACI) methodology for development and management of IT projects. VIP is a Lean-Agile framework that provides efficient streamlining of activities within the VA enterprise and is designed to increase the delivery speed of high-quality and secure IT capabilities. VIP methodologies are reviewed on a recurring basis and refinements are made as needed. TMO stays abreast of all the latest changes to the VIP methodologies that is maintained in PAL (Process Asset Library).

Major program SACI processes tend to align with standard VA OIT processes to include VIP, and may streamline, expand, augment, or customize those VIP processes to meet program needs.

IV&V Test Management Operations & Test Center engagement starts with gathering project data for the Testing Intake Assessment (TIA). In the case of OIT, the primary authoritative source for approved and budgeted projects is VA PARS. When a new entry or new build on an existing entry are established, TMO engagement begins. TMO evaluates the VIP Request (VIPR), data elements, and project artifacts to recommend TMO testing services. During product/build start, TMO performs the Criticality Analysis and Risk Assessment (CARA) and recommends appropriate test services. During the remainder of the project lifecycle, TMO monitors product and sprint backlogs in project repositories, with follow-on CARA’s performed and recommendations of further applicable testing services. TMO conducts testing services in parallel with the project build.

TMO processes include reviewing the project artifacts, conducting an initial outreach meeting, project analysis, planning, and testing service determination. When the services start, TMO determines whether test environments exist which could be used for the services, or new ones are built. From there, TMO executes testing, and generates reports of the test findings. TMO also may conduct final regression testing following a project’s Initial Operating Capability (IOC).

TMO continues to monitor product and sprint backlogs until project closeout.

There are two primary entrances into TMO testing service engagement with a Program and/or Project. One is through a Program and/or Project entering a request into TMO’s centralized request management tool for testing services. The other is a proactive approach to monitoring the authoritative source for approved VA projects, both for new entries and software builds within existing entries. TMO engages with the VA project manager (PM) or designee to gain access to the product teams repositories. TMO conducts a high-level evaluation on all the artifacts. After the evaluation, if further engagement is not recommended, TMO enters a passive engagement. If TMO's engagement is recommended, a build-level risk assessment following TMO’s CARA process is conducted. TMO recommended services accepted and scheduled with the project are performed against the current build and sprint based on the build-level CARA at the discretion of the project or program team. If there are additional builds, TMO continues to perform build-level CARAs and any recommended test services accepted by the Program and/or Project that result for those builds until there are no additional builds, at which point TMO enters a passive engagement.

For all VIP methodologies, TMO uses a risk-based approach to determine what T&E services are recommended. This approach involves using a standardized methodology to determine the level of testing for software applications based on risks and their probable adverse impacts to the enterprise. As part of the TIA process, the CARA methodology identifies the functions, system design elements, or software tasks which pose the greatest risk to the success of software development initiatives. The key feature of this approach is the assessment of an appropriate level of analysis for any given function based on risk and criticality. By applying CARA, suitable levels of analysis, as well as the resources required to perform them can be clearly defined and implemented. The services offered by TMO fit into the SDLC and SACI phases and processes of planning, execution, and monitoring and control. The following diagram portrays how the TMO service offerings fit into the planning and execution lifecycle phases.

Figure 1.2-3: Service Offerings and the SDLC and SACI phases

1.2.1 Services Executed During SDLC and SACI Planning Phase

Three TMO services—Testing Intake Assessment (TIA), Criticality Analysis and Risk Assessment (CARA), and Determine Scope and Services—are performed during the Planning phase of the SDLC and SACI phases for products and/or projects. Project teams create respective project documentation (epics, user stories, requirements traceability matrices, etc.) and load them into the appropriate document repositories.

TMO’s Testing Intake Assessment starts by creating a Project Analysis Summary (PAS) record based off VA approved and budgeted projects within the VA’s authoritative source. TMO triages all projects in the authoritative source, using a predefined criteria to identify appropriateness for further engagement. Once criteria are met, TMO requests access to the document repositories. As access is granted and any available project documentation is able to undergo analysis via the TIA service, TMO creates a Build Analysis Summary (BAS) for the project build. At this build level, the CARA process is executed.

From the CARA results, the Determine Scope and Services service is performed. This service involves multi-disciplinary analysis performed by all TMO teams. Once scope and services are identified, TMO create a Risk Analysis Summary (RAS) which is reviewed and approved by TMO VA Management. The RAS is the primary written communication to the project for TMO recommended test services. As applicable, TMO will conduct a debrief with the project team.

Details of these services are identified in Section 5.0.

1.2.2 Services Executed During SDLC and SACI Test Services Execution Phase After Risk Analysis Summary (RAS) delivery to the project team, applicable debriefs, and project acceptance of all or any of the recommended test services, TMO enters the Execution phase. This Execution phase includes coordination with the project team to collect any further information, identify the approach for test environment, and to overcome any other technical challenges in preparation of TMO testing services to be performed. To formally document these interactions and planning efforts TMO creates Test Plan Summary (TPS), obtains TMO VA management concurrence and approval, and then delivers the TPS to the project team.

Overlapping Planning and Execution is the TC Test Environment Construction. These requirements are defined by TC engineers and delivered to TC or the major program test environment managers (for COTS test systems hosted elsewhere) in the form of a Test Environment Logical Requirements (TELR) document. TC and the major program representatives coordinate the creation of the test environments as necessary, and ensures the environment performs as expected via a smoke test. TC and the major program representatives also may coordinate with the project team who validates the build is ready for TMO testing service execution such as Requirements Validation Testing, Performance Testing, System Integration/Interoperability Testing, and Software Code Quality Check (SCQC). When the appropriate components of the system under test (SUT) are available for testing, the TMO team executes planned services and records the activities, findings and results in test service repositories. TMO then creates a Test Analysis Summary (TAS), which is reviewed and approved by TMO VA management. Once approved, the TAS is delivered to the project team and a debriefing with the project team is conducted as applicable. Details of these services are identified in section 5.

1.2.3 Processes Executed in Support of Monitoring and Control Workload Forecasting and Support of Test Center Systems may be executed during the Monitoring and Control Phase of the SDLC and SACI phases. TMO management continuously checks VIPR and Test Center Service Requests so that appropriate plans can be made for providing test services and/or other TMO services.

1.2.3.1 Workload Forecasting

Workload Forecasting is the process in which information from the project team and TMO is used to forecast work for TMO. TMO updates the testing project schedule to align with changes gathered from the Product/Project team’s Schedule. TMO uses the information to forecast project arrival and to project services which enables adequate time and resource preparation.

1.2.3.2 Support of Test Center Systems

TSEI maintains the TC located in Bay Pines, FL, which is used by various project teams for software product-level Quality Assurance testing, as well as hosting software tools and test environments built for delivery of TMO test services to OIT and major programs. TSEI also manages test systems hosted in VA Enterprise Cloud (VAEC) as needed. Services provided by the TC include:

• Establishment of new databases

• Assistance with miscellaneous issues when there is trouble with a test environment

• Database restore/refresh/backup

• User access to an environment

• Install, configure, manage the SUT

• Install, configure and manage the necessary test tools and monitoring tools

In accordance with OIT priorities and timelines, IV&V TC will begin planning for transition to the cloud.

1.3 EHRM-IO INTEGRATED TESTING SERVICES & METHODOLOGY

The Electronic Health Record Modernization (EHRM) Joint Master Test Plan (JMTP) documents the services and methodology used in the EHRM-IO Integrated Testing Program. The JMTP provides the overarching guidance to coordinate and execute integrated testing of the EHR that the Department of Veterans Affairs (VA) acquired from Oracle. Once deployed, the Electronic Health Record (EHR) will co-exist in the same production environment as the Department of Defense’s (DOD) EHR system with VA using an overlapping set of the DOD’s features as well as an additional set of features.

The JMTP also describes the integrated testing lifecycle for the EHRM-IO Program executed across the test community. The test community is comprised of Oracle Testing & Quality Assurance (T&QA), deployment resources, EHRM-IO Integrated Testing (IntT) Enterprise Test (ET) resources, Office of the Functional Champion (OFC) Functional Testing and Evaluation

(OFCt) resources for medical enterprise and local system interfacing, the Office of Healthcare Technology Management (HTM) for medical devices and peripherals testing, field site subject matter experts (SMEs)/solutions experts (SEs), and EHRM-IO Clinical Councils. This plan describes the involvement and relationship of various stakeholders to the test community. The plan will evolve and serve as a living document, providing a framework within which detailed test plans can be generated for the EHRM-IO Integrated Testing Program.

2.0 APPLICABLE DOCUMENTS

In the performance of the tasks associated with this Performance Work Statement, the Contractor shall comply with the most current version of the following:

1. 44 U.S.C. § 3541-3549, “Federal Information Security Management Act (FISMA) of 2002”

2. “Federal Information Security Modernization Act of 2014”

3. Federal Information Processing Standards (FIPS) Publication 140-2, “Security

Requirements for Cryptographic Modules”

4. FIPS Pub 199. “Standards for Security Categorization of Federal Information and

Information Systems,” February 2004

5. FIPS Pub 200, “Minimum Security Requirements for Federal Information and

Information Systems,” March 2006

6. FIPS Pub 201-2, “Personal Identity Verification of Federal Employees and

Contractors,” August 2013

7. 10 U.S.C. § 2224, "Defense Information Assurance Program"

8. 5 U.S.C. § 552a, as amended, “The Privacy Act of 1974”

9. Public Law 109-461, Veterans Benefits, Health Care, and Information Technology

Act of 2006, Title IX, Information Security Matters

10. 42 U.S.C. § 2000d “Title VI of the Civil Rights Act of 1964”

11. VA Directive 0710, “Personnel Security and Suitability Program,” June 4, 2010, https://www.va.gov/vapubs/index.cfm

12. VA Handbook 0710, “Personnel Security and Suitability Program,” May 2, 2016, https://www.va.gov/vapubs/index.cfm

13. VA Directive and Handbook 6102, “Internet/Intranet Services,” August 5, 2019

14. 36 C.F.R. Part 1194 “Information and Communication Technology Standards and

Guidelines,” January 18, 2017

15. Office of Management and Budget (OMB) Circular A-130, “Managing Federal

Information as a Strategic Resource,” July 28, 2016

16. 32 C.F.R. Part 199, “Civilian Health and Medical Program of the Uniformed

Services (CHAMPUS)”

17. NIST SP 800-66 Rev. 1, “An Introductory Resource Guide for Implementing the

Health Insurance Portability and Accountability Act (HIPAA) Security Rule,” October 2008

18. Sections 504 and 508 of the Rehabilitation Act (29 U.S.C. § 794d), as amended, January 18, 2017

19. Homeland Security Presidential Directive (12) (HSPD-12), August 27, 2004

20. VA Directive 6500, “VA Cybersecurity Program,” February 24, 2021

21. VA Handbook 6500, “Risk Management Framework for VA Information Systems

VA Information Security Program,” February 24, 2021 https://www.va.gov/vapubs/index.cfm https://www.va.gov/vapubs/index.cfm http://www.va.gov/vapubs http://www.va.gov/vapubs

22. VA Handbook 6500.2, “Management of Breaches Involving Sensitive Personal

Information (SPI),” March 12, 2019

23. VA Handbook 6500.5, “Incorporating Security and Privacy into the System

Development Lifecycle,” March 22, 2010

24. VA Handbook 6500.6, “Contract Security,” March 12, 2010

25. VA Handbook 6500.8, “Information System Contingency Planning,” April 6, 2011

26. VA Handbook 6500.10, “Mobile Device Security Policy,” February 15, 2018

27. VA Handbook 6500.11, “VA Firewall Configuration,” August 22, 2017

28. OIT Process Asset Library (PAL), https://www.va.gov/process/ . Reference Process

Maps at https://www.va.gov/process/maps.asp and Artifact templates at https://www.va.gov/process/artifacts.asp

29. One-VA Technical Reference Model (TRM) (reference at https://www.va.gov/trm/TRMHomePage.aspx)

30. VA Directive 6508, “Implementation of Privacy Threshold Analysis and Privacy Impact Assessment,” October 15, 2014

31. VA Handbook 6508.1, “Procedures for Privacy Threshold Analysis and Privacy Impact Assessment,” July 30, 2015

32. VA Handbook 6510, “VA Identity and Access Management,” January 15, 2016

33. VA Directive and Handbook 6513, “Secure External Connections,” October 12,

34. VA Directive 6300, “Records and Information Management,” September 21, 2018

35. VA Handbook, 6300.1, “Records Management Procedures,“ March 24, 2010

36. NIST SP 800-37 Rev 2, “Risk Management Framework for Information Systems and

Organizations: A System Life Cycle Approach for Security and Privacy,” December

37. NIST SP 800-53 Rev. 5, “Security and Privacy Controls for Federal Information Systems and Organizations,” September 23, 2020 (includes updates as of 12/10/2020)

38. VA Directive 0735, “Homeland Security Presidential Directive 12 (HSPD-12) Program,” October 26, 2015

39. VA Handbook 0735, “Homeland Security Presidential Directive 12 (HSPD-12) Program,” March 24, 2014

40. OMB Memorandum 05-24, “Implementation of Homeland Security Presidential Directive (HSPD) 12 – Policy for a Common Identification Standard for Federal Employees and Contractors,” August 5, 2005

41. OMB Memorandum M-19-17, “Enabling Mission Delivery Through Improved Identity, Credential, and Access Management,” May 21, 2019

42. OMB Memorandum, “Guidance for Homeland Security Presidential Directive (HSPD) 12 Implementation,” May 23, 2008

43. Federal Identity, Credential, and Access Management (FICAM) Roadmap and Implementation Guidance, December 2, 2011, (NOTE: Part A of the FICAM Roadmap and Implementation Guidance, v2.0, was replaced in 2015 with an updated Architecture (https://arch.idmanagement.gov/#what-is-the-ficam-architecture)

44. NIST SP 800-116 Rev 1, “Guidelines for the Use of Personal Identity Verification (PIV) Credentials in Facility Access,“ June 2018 https://www.va.gov/process/ https://www.va.gov/process/maps.asp https://www.va.gov/process/artifacts.asp https://www.va.gov/trm/TRMHomePage.aspx https://arch.idmanagement.gov/#what-is-the-ficam-architecture

45. NIST SP 800-63-3, 800-63A, 800-63B, 800-63C, “Digital Identity Guidelines,” updated March 02, 2020

46. NIST SP 800-157, “Guidelines for Derived PIV Credentials,” December 2014

47. NIST SP 800-164, “Guidelines on Hardware-Rooted Security in Mobile Devices

(Draft),” October 2012

48. Draft National Institute of Standards and Technology Interagency Report (NISTIR)

7981, “Mobile, PIV, and Authentication,” March 2014

49. VA Memorandum, VAIQ #7100147, “Continued Implementation of Homeland

Security Presidential Directive 12 (HSPD-12),” April 29, 2011 (reference https://www.voa.va.gov/documentlistpublic.aspx?NodeID=514)

50. IAM Identity Management Business Requirements Guidance document, May 2013, (reference Enterprise Architecture Section, PIV/IAM (reference https://www.voa.va.gov/documentlistpublic.aspx?NodeID=514)

51. VA Memorandum “Personal Identity Verification (PIV) Logical Access Policy Clarification,” July 17, 2019, https://www.voa.va.gov/DocumentView.aspx?DocumentID=4896

52. Trusted Internet Connections (TIC) 3.0 Core Guidance Documents, https://www.cisa.gov/publication/tic-30-core-guidance-documents

53. OMB Memorandum M-19-26, “Update to the Trusted Internet Connections (TIC) Initiative,” September 12, 2019

54. OMB Memorandum M-08-23, “Securing the Federal Government’s Domain Name System Infrastructure,” August 22, 2008

55. Sections 524 and 525 of the Energy Independence and Security Act of 2007, (Public Law 110–140), December 19, 2007

56. Section 104 of the Energy Policy Act of 2005, (Public Law 109–58), August 8, 2005

57. Executive Order 13834, “Efficient Federal Operations,” dated May 17, 2018

58. Executive Order 13221, “Energy-Efficient Standby Power Devices,” August 2, 2001

59. VA Directive 0058, “VA Green Purchasing Program,” July 19, 2013

60. VA Handbook 0058, “VA Green Purchasing Program,” July 19, 2013

61. Office of Information Security (OIS) VAIQ #7424808 Memorandum, “Remote

Access,” January 15, 2014, https://www.voa.va.gov/DocumentListPublic.aspx?NodeId=28

62. Clinger-Cohen Act of 1996, 40 U.S.C. §11101 and §11103

63. “Veteran Focused Integration Process (VIP) Guide 4.0,” January 2021, https://www.voa.va.gov/DocumentView.aspx?DocumentID=4371

64. VA Memorandum “Proper Use of Email and Other Messaging Services,” January 2, 2018, https://www.voa.va.gov/DocumentListPublic.aspx?NodeId=28

65. “DevSecOps Product Line Management Playbook” version 2.0, May 2021, https://www.voa.va.gov/DocumentView.aspx?DocumentID=4946

66. NIST SP 500-267B Revision 1, “USGv6 Profile,” November 2020

67. OMB Memorandum M-21-07, “Completing the Transition to Internet Protocol

Version 6 (IPv6),” November 19, 2020

68. Social Security Number (SSN) Fraud Prevention Act of 2017

69. Section 240 of the Consolidated Appropriations Act (CAA) 2018, March 23, 2018

70. “Veteran Focused Integration Process (VIP) Guide 3.2”, December 2018, https://www.voa.va.gov/DocumentView.aspx?DocumentID=4371 https://www.voa.va.gov/documentlistpublic.aspx?NodeID=514 https://www.voa.va.gov/documentlistpublic.aspx?NodeID=514 https://www.voa.va.gov/DocumentView.aspx?DocumentID=4896 https://www.cisa.gov/publication/tic-30-core-guidance-documents https://www.voa.va.gov/DocumentListPublic.aspx?NodeId=28 https://www.voa.va.gov/DocumentView.aspx?DocumentID=4371 https://www.voa.va.gov/DocumentListPublic.aspx?NodeId=28 https://www.voa.va.gov/DocumentView.aspx?DocumentID=4946 https://www.voa.va.gov/DocumentView.aspx?DocumentID=4371

71. “VIP Release Process Guide”, Version 1.4, May 2016, https://www.voa.va.gov/DocumentView.aspx?DocumentID=4411

72. “POLARIS User Guide”, Version 1.9, March 2017, https://www.voa.va.gov/DocumentView.aspx?DocumentID=4412

73. VA Memorandum “Use of Personal Email (VAIQ #7581492)”, April 24, 2015, https://www.voa.va.gov/DocumentListPublic.aspx?NodeId=28

74. VA Memorandum “Updated VA Information Security Rules of Behavior (VAIQ #7823189)”, September, 15, 2017, https://www.voa.va.gov/DocumentListPublic.aspx?NodeId=28

75. EHRM-IO Joint Master Test Plan (JMTP)

3.0 SCOPE OF WORK

The Contractor shall provide support in the areas of test and evaluation of software, infrastructure, environments, and operations of the IV&V TC, operations for TC environments hosted in VA Enterprise Cloud; and continuous quality process and program management support.

EHRM-IO IntT & OIT each require specific services during the period of performance to support all areas of daily operations. This work includes test and evaluation of the Electronic Health Record solutions, and VA applications in support of VA modernization initiatives. These applications include but are not limited to EHRM solutions and all systems integration to the EHR, Financial Management Business Transformation (FMBT), and other Government-off-the-shelf (GOTS) and COTS applications. The required services also involve testing-related activities associated with Legacy VistA (Mumps, Delphi), modernized applications (Java/Oracle, Microsoft.Net/SQL), and other middleware and custom interface and data exchange systems.

While OIT TMO and TC may cover any application in this arena, primary focus is on modernization initiatives, data migration, major system interoperability, and data exchange applications while concurrently supporting testing activities during the transition from VA legacy systems.

3.1 ORDER TYPE

The effort shall be a hybrid Firm-Fixed-Price (FFP) and Time-and-Materials (T&M) order.

4.0 PERFORMANCE DETAILS

4.1 PERFORMANCE PERIOD

The Period of Performance (PoP) shall be 12 months from date of award with four 12-month option periods and one optional task to provide Transition Support starting 30-90 days prior to the end of the period of performance, not to exceed 150 days.

Any work at the Government site shall not take place on Federal holidays or weekends unless directed by the Contracting Officer (CO). (If required, the CO may designate the Contractor to work during holidays and weekends)

There are 11 Federal holidays set by law (USC Title 5 Section 6103) that VA follows:

https://www.voa.va.gov/DocumentView.aspx?DocumentID=4411 https://www.voa.va.gov/DocumentView.aspx?DocumentID=4412

Under current definitions, five are set by date:

New Year's Day January 1 Juneteenth June 19 Independence Day July 4 Veterans Day November 11 Christmas Day December 25

If any of the above falls on a Saturday, then Friday shall be observed as a holiday. Similarly, if one falls on a Sunday, then Monday shall be observed as a holiday.

The other six are set by a day of the week and month:

Martin Luther King's Birthday Third Monday in January Washington's Birthday Third Monday in February Memorial Day Last Monday in May Labor Day First Monday in September Columbus Day Second Monday in October Thanksgiving Fourth Thursday in November

4.2 PLACE OF PERFORMANCE

Tasks performed under this PWS shall be performed at both Contractor and Government facilities, dependent on the project to be tested or required testing support service. The Contractor shall identify the place of performance in its proposal submission.

All support will be within the Continental United States (inclusive of Alaska, Hawaii, and US Territories). Some tasks under this PWS require the on-site presence of contractor resources and shall be performed at the IV&V Test Center located in Bay Pines, Florida. There are two to six seats for contractor services at IV&V Test Center. The Contractor shall prioritize the assignment of resources to the TC based on infrastructure and engineering services requiring physical access to the TC.

Bay Pines Test Center 10000 Bay Pines Blvd, Bldg. 37 Bay Pines, FL 33744

4.3 TRAVEL (T&M)

The Government anticipates the need for travel events to perform tasks associated with the effort, and to attend program-related meetings or conferences throughout the PoP. Below are travel estimates for base and option periods, but the number, location and duration of travel events will be based on operational requirements.

Travel will be processed as Other Direct Costs and will not be included under CLIN 5. Travel will be performed in accordance with Federal Travel Regulations and will be billed after performance.

Base Period

EHRM

OIT

Type of Meeting Location Number of Trips

Number of Persons

Number of Days

EHRM Testing at Deployment Site (6 events)

TBD pending operational requirements 6 3 5

EHRM Integrated Testing Team Meetings Washington DC 2 12 5

OIT IV&V Strategic Planning Austin, TX 1 9 5

Option Period 1

OIT

Type of Meeting

Location Number of Trips

Number of Persons

Number of Days

EHRM Testing at Deployment Site (6 events)

TBD pending operational requirements 6 3 5

EHRM Integrated Testing Team Meetings Washington DC 2 12 5

Option Period 2

OIT

Type of Meeting

Location Number of Trips

Number of Persons

Number of Days

EHRM Testing at Deployment Site (6 events)

TBD pending operational requirements 6 3 5

EHRM Integrated Testing Team Meetings Washington DC 2 12 5

Option Period 3

OIT

Type of Meeting

Location Number of Trips

Number of Persons

Number of Days

EHRM Testing at Deployment Site (6 events)

TBD pending operational requirements 6 3 5

EHRM Integrated Testing Team Meetings Washington DC 2 12 5

Option Period 4

OIT

Type of Meeting

Location Number of Trips

Number of Persons

Number of Days

EHRM Testing at Deployment Site (6 events)

TBD pending operational requirements 6 3 5

EHRM Integrated Testing Team Meetings Washington DC 2 12 5

5.0 SPECIFIC TASKS AND DELIVERABLES

The Contractor shall perform the following in accordance with VA processes to be provided to the contractor:

5.1 PROJECT MANAGEMENT (FFP)

5.1.1 Contractor Project Management Plan

The Contractor shall deliver a Contractor Project Management Plan (CPMP) that lays out the Contractor’s approach, timeline, and tools to be used in execution of the contract. The CPMP shall take the form of both a narrative and graphic format that displays the schedule, milestones, risks, and resource support. The CPMP shall also include how the Contractor shall coordinate and execute planned, routine, and ad hoc data collection reporting requests as identified within the PWS. The initial baseline CPMP shall be concurred upon and updated monthly thereafter.

The Contractor shall update and maintain the VA-approved CPMP throughout the period of performance.

Deliverable:

A. Contractor Project Management Plan

5.1.2 Contract Key Personnel

The Contractor shall assign a single individual to serve as the Contract Program Manager to support the contract, as well as any other key personnel deemed necessary for support of the contract. Any change of key personnel, defined as Program Manager, Project Manager, and Division Leads shall require the Contracting Officer’s Representative (COR) and IETSS VA Senior Management notification and a transition plan to ensure minimum disruption to services delivered via the IETSS contract.

Any personnel the Contractor offers as substitutes shall have capabilities and qualifications at least equal to the key personnel being replaced. If any change to a key personnel position becomes necessary, the Contractor shall immediately notify the VA PM in writing. Whenever possible the Contractor shall notify the VA PM of substitutions of personnel in writing 30 calendar days prior to making any change in key personnel. The notification should provide a detailed explanation of the circumstances necessitating the proposed substitution and shall demonstrate that the proposed replacement is of at least substantially equal ability and qualifications as the individual originally proposed for that position.

The Contractor agrees that it has a contractual obligation to mitigate the consequences of the loss of key personnel and shall promptly secure any necessary replacements in accordance with this PWS section. Failure to replace key personnel without a break in performance of the labor category at issue shall be considered a condition endangering contract performance and may provide grounds for default termination.

5.1.3 Reporting Requirements & Management Meetings

The Contractor shall provide the COR with Monthly Progress Reports in electronic form in Microsoft Word or Excel formats. The report shall include detailed explanations for each required area and ensure that data is accurate and consistent. These reports shall reflect data as of the last day of the preceding month.

The Monthly Progress Reports shall cover all work completed during the reporting period and work planned for the subsequent reporting period. The report shall also identify any problems that arose and a description of how the problems were resolved. If problems have not been completely resolved, the Contractor shall provide an explanation including its plan and timeframe for resolving the issue. The Contractor shall monitor performance against the CPMP and report any deviations. It is expected that the Contractor will keep in communication with VA accordingly so that when issues arise, they are visible to both parties, to enable solutions to be developed without escalation. The Monthly Progress Report shall organize the report into the EHRM-IO Integration Testing Section & OIT Section.

The Contractor shall provide a weekly status update via email and as applicable during weekly management meetings that summarizes services delivered for the past week, active or open services, issues resolved and issues still under consideration that are relevant for VA management visibility and related to the requirements review. Weekly management meetings are attended by VA and Contractor management and team leads.