S02 - PWS - ATCH - L to T - Remote Patient Monitoring Home Telehealth FY 22 09082021.pdf

Home Telehealth VistA Integration Test Plan Validation Testing

Author: George C. Blankenship, Jr.

Creation Date: 2/25/2005 Last Updated: 6/1/2020 Version: 2.3a Vendor: HT Vendor

Attachment L

Attachment L - 37C79121R0008

Change Record Date Author Version Change Reference

3/14/2005 George Blankenship 1.0 Initial release 6/23/2005 George Blankenship 1.1 Add CCOW (patient context) 6/23/2005 George Blankenship 1.1 Add Progress Note testing 7/28/2005 George Blankenship 1.2 Add VA Alpha/Beta testing 7/28/2005 George Blankenship 1.2 Elevate “Acceptance Report” to outline level 1 1/6/2006 George Blankenship 1.3 Add CCOW (patient context) validation table 1/23/2006 George Blankenship 1.4 Changed the document name to accurately reflect its scope 7/9/2009 George Blankenship 1.5 Changed the formatting of the creation/revision dates 7/13/2009 George Blankenship 1.6 Tailor test plan for Viterion VistA Integration 1.0 testing 9/5/2009 George Blankenship 1.7 Add A03 to test plan 9/5/2009 George Blankenship 1.7 Add A04 sequences with care coordinator update 9/5/2009 George Blankenship 1.7 Add A31 sequences 9/10/2009 George Blankenship 1.7a Update acceptance test objectives (Table 7) 9/16/2009 George Blankenship 1.7b Update acceptance test objectives (Table 7) – work location and transactions that should not be processed 1/27/2011 George Blankenship 1.7c Update acceptance test sequences (Table 18) to cover updates while dis-enrolled and legacy patient correlation errors.

1/27/2011 George Blankenship 1.7c Update acceptance test objectives (Table 7) for legacy patient correlation errors and activation 3/9/2011 George Blankenship 1.7d Update for CR recovery 3/9/2011 George Blankenship 1.7d Update for missing Cx 3/9/2011 George Blankenship 1.7d Update for missing Ax 3/9/2011 George Blankenship 1.7d Update for connection failure 3/9/2011 George Blankenship 1.7d Clarify “active” to mean enrolled and Integrated 3/9/2011 George Blankenship 1.7d Clarify “inactive” to include dis-enrolled and Integrated 3/9/2011 George Blankenship 1.7d Update to reference alert generation 3/9/2011 George Blankenship 1.7d Add Phase I configuration 3/9/2011 George Blankenship 1.7d Add Phase II configuration 3/9/2011 George Blankenship 1.7d Add Phase III configuration 8/25/2014 George Blankenship 1.7d1 Reset all completion date in Table 11 due to restart of Phase I after a 9-month hiatus.

8/27/2014 George Blankenship 1.8 Add Phase I configuration table 8/27/2014 George Blankenship 1.8 Add Phase II configuration table (subject to revision at the beginning of testing) 8/27/2014 George Blankenship 1.8 Add Survey (Satisfaction and ADL) validation 8/27/2014 George Blankenship 1.8 Add DMP validation (as a future) 8/27/2014 George Blankenship 1.8 Add Census submission validation 5/27/2015 George Blankenship 1.8a Correct typo in update date 5/27/2015 George Blankenship 1.8a Correct typo in footnote 10 (Table 18) 7/11/2016 George Blankenship 1.9 Preparation for 2016 award validations

Add public test lab port usage table 8/24/2016 George Blankenship 1.9 Correct test 114 3/22/2017 George Blankenship 2.0 Update for 2017 award 4/12/2017 George Blankenship 2.0a Update Table 4 7/18/2017 George Blankenship 2.1 Update Table 18 to reflect automated testing 8/16/2017 George Blankenship 2.1a Correct Table 18 to reflect detailed subtests for Test88-95 1/29/2019 George Blankenship 2.2 Update Table 7 to reflect the revised vital sign update rules 1/29/2019 George Blankenship 2.2 Update Table 18 to include the valid vital sign submissions and updates to submissions with the rejection of unsupported data changes

11/4/2019 George Blankenship 2.3 Updated for 2020 procurement.

Add Section 4

5/29/2020 George Blankenship 2.3a Add preconditions to Table 14, Table 15, Table 16, and Table

Reviewers Name Position

Ellen Hans Home Telehealth Project Manager Ronald Sandreth COTR Catherine Buck OTS Clinical Representative Michael Benner Home Telehealth Architect

Contents

1 Overview

2 Test Environment

2.1 Data Flows

2.2 CCOW Patient Context

2.3 System Functions

2.3.1 Patient Record

2.3.2 Patient Generated Data

3 Test Phases

3.1 Phase I (EITL)

3.1.1 Objective

3.1.2 Tests

3.2 Phase II (Integration Testing)

3.2.1 Objective

3.2.2 Tests

3.3 Phase III (Pre-deployment Testing)

3.3.1 Objective

3.3.2 Tests

4 Testing packages

4.1 PreEITL Test Sets

4.2 EITL Test Sets

5 Acceptance Report

6 References

7 Glossary

Tables

Table 1: Acronyms and Abbreviations

Table 2: Tests (Data Flow Perspective)

Table 3: Function Priorities and Groups

Table 4: Phase I Configuration (Internet EITL)

Table 5: Phase I Configuration (Intranet EITL)

Table 6: Phase I Tests (Acceptance of Valid Messages)

Table 7: Phase I Tests (Valid Message Sequences)

Table 8: Phase I Tests (Invalid Message Content and Message Sequences)

Table 9: Phase II Configuration (ITL)

Table 10: Phase II Tests (Integration with Individual VA Test Systems)

Table 11: Phase II Tests (Integration with All VA Test Systems)

Table 12: Phase II Tests (Integration with VA Applications using CCOW Patient Context)

Table 13: Phase III Tests (Vendor Pre-deployment Validation)

Table 14: Test preconditions

Table 15: PreEITL Test Sets

Table 16: EITL Application Test Sets

Table 17: EITL Message Test Sets

Table 18: Vendor Testing Acceptance Report

Table 19: Testing Log

Table 20: Glossary

Figures

Figure 1: Home Telehealth HL7 Functional Relationships

1 Overview The VHA Office of Connected Care/Telehealth Services (OCC/THS), through the Home Telehealth Program, is currently putting medical devices in patient homes to improve the quality of care and standard of living for veterans throughout the U.S. From the technical point of view, each patient’s medical device communicates with a central collection system (triage system or server) located behind a VA firewall in a VA data center. A critical function of the Home Telehealth systems is their ability to communicate with existing Veteran Health Administration (VHA) computer systems (VistA, MPI, HDR, and Home Telehealth repositories). The communication between the Home Telehealth systems and the VHA systems is based upon the use of the standard medical exchange protocol Health Level Seven (HL7).

HL7 is the VA-adopted standard for the exchange of data between medical systems that support the management and delivery of healthcare services (patient care). As a standard, HL7 is widely accepted and used both nationally and internationally. The VHA national contract requires the Home Telehealth suppliers to incorporate into their systems the ability to accept and send HL7 messages as defined by the VA. The exchange of medical information between multiple diverse systems requires precise definition of transaction contents.

Another component of the HL7 standards group is Clinical Context Object Workgroup (CCOW) standard. This standard defines a context manager shared between applications executing on common workstation. The VA requires the support of two contexts by Home Telehealth vendors: patient context and user context1. Applications that share a patient context will display data for the same patient; and when one application changes the context (switches patient), the other application follows (switches to the same patient). Applications that share a user context will track the user in the peer application. When a CCOW compliant application opens on a workstation, it uses the current workstation context to define the displayed data; for example patient context defines the patient to be displayed. CCOW compliance requires that an application is always in context; that is, patient context compliance requires that the displayed patient is coherent with the context.

This document defines the test plan that used to perform the pre-deployment validation testing of a Home Telehealth vendor system. The name System Under Test (SUT) is used to reference the target system of the testing. The validation uses three distinct phases.

The first phase validates the HL7 implementation using an emulation engine for the major VA systems. This phase is a functional test verifying the correct exchange of messages, the proper processing of transactions, the rejection of messages and transactions that contain improper data, and the recovery from error conditions. The SUT connects to an External Integration Test

1 The support of user context is currently not required.

Laboratory (EITL) for the first phase testing. The laboratory is accessible via the Internet or the VA Intranet. Section 3.1 provides the EITL configuration and testing steps.

The second phase validates the functional data flows involving the vendor server and actual VA systems; this phase is called integration testing. The SUT connects to an Integration Test Laboratory (ITL) for the second phase testing. The laboratory is accessible via the VA Intranet.

Section 3.2 provides the configuration and testing steps.

The final phase is the support of a “slow start” approach to deployment of a new system into the production environment. A limited set of patients are introduced into the new production environment over a set period of time; this phase is a controlled release of a product into general use. Section 3.3 provides the testing steps.

During the second two phases, context switches involving the vendor application and CPRS will verify CCOW patient context compliance is maintained. In the ITL, specific tests will verify that the patient context will be maintained in the presence of a variety of context changes. In the deployment rollout, the clinical staff will monitor patient context compliance.

The rest of this document covers the test environment and test plans. The test environment section discusses the testing tools, testing configurations, and testing objectives from a functional perspective. The test plan section discusses the test sequences, participants in the tests, and the testing objectives of each test.

Table 1 contains the acronyms and abbreviations used in this document.

Table 1: Acronyms and Abbreviations Acronym Description AAC Austin Automation Center ADT Admission, Discharge and Transfer ACK Acknowledgement ADL Activity of Daily Living

Based upon the Veterans RAND 12 Item Health Survey (VR-12) CIO Chief Information Officer CPRS Computerized Patient Records System DFN Data File Number, a two-component index consisting of an integer and a facility number ESM Enterprise Systems Manager Hines Hines Data Center HL7 Health Level 7 protocol HT Home Telehealth ICN Integration Control Number IE Interface Engine IEN Internal Entry Number Legacy Patient A legacy patient is one that has been placed in the Home Telehealth program via a direct entry of the patient demographics into the vendor server. The patient data is not available using any VA system such as CPRS (VistA).

M&IS Messaging & Interfaces Services MPI Master Patient Index (system) MRN Medical Record Number (index into a medical database) MVI Master Veteran Index (system)

Acronym Description OCC/THS VHA Office of Connected Care/Telehealth Services (10A7D) OI Office of Information PD Patient Demographics RDV Remote Data View SSN Social Security Number TCP/IP Transmission Control Protocol/Internet Protocol URL Universal Record Location VA Department of Veterans Affairs VACO Veterans Affairs Central Office VAMC Department of Veterans Affairs Medical Center VIE VistA Interface Engine VISN Veterans integrated service unit VistA Veterans Health Information Systems and Technology Architecture VistA Integration VistA Integration is the term that indicates the patient’s identity and demographics are defined VA systems and the automatic updating of the information by VA systems.

VHA Veterans Health Administration Acronym Description

2 Test Environment

2.1 Data Flows

Figure 1 depicts the functional relationships between the systems that comprise the Home Telehealth environment. The figure identifies whether the functions are implemented using HL7. All functions between the Home Telehealth vendor severs and a VA system must use HL7. This test plan defines the testing that performed to validate the interfaces and data flows.

Home system Telehealth device

Telephone

Vendor Server

(C) Medical observation data (not HL7)

HL7 Functional Relationships A. VistA sends patient sign up and medical order submission to vendor server B. Vendor server notifies MPI of patient activity, MPI will send update if needed C. Home system sends medical observation data to vendor server D. Vendor server sends medical observation data to HDR E. Vendor server sends progress note to VistA (when required)

Health Data Repository

HDR

Master Patient Index

MPI

VistA system

(A) Patient Sign Up/Activation (HL7) Medical order submission (HL7) Patient Inactivation (HL7)

(E) Progress note (HL7)

(B) Patient tracking subscription (HL-7) Patient information update (HL-7)

(D) Medical observation data (HL7)

Care Coordinator Workstation

Vendor unique application interface (not HL7)

Figure 1: Home Telehealth HL7 Functional Relationships

The overall process flow consists of the following steps:

1. Clinician prepares a consult recommending the use of Home Telehealth for a VA patient and forwards recommendation to care coordination.

2. Care coordinator selects Home Telehealth vendor and initiates the sign up process. The sign up process will generate an HL7 message sent to the vendor server with patient identity, demographics, and consult information (relationship A in Figure 1). The patient identity is an index that is used to locate the patient in a database. There are three identities used by the VA. The integration control number (ICN) is the VA national patient identity. The ICN locates the patient in the Master Person Index (MPI). The Data File Number (DFN) is the local patient identity locates the patient in a VistA facility database. The social security number (SSN) is the US national individual identity. The patient demographics include the patient first name, last name, and date of birth (DOB).

3. The vendor server generates subscription request to the MPI for updates to the patient identity linkages or demographics (relationship B in Figure 1).

4. The care coordinator prepares a medical order defining the vitals to be collected and the dialogues to be used (relationship A in Figure 1).

5. The vendor issues a home device to the patient and activates the device.

6. When the home device has collected data, the home device sends the vitals measurements to the vendor server. Upon receipt, the vendor server forwards the vital measurements to the Health Data Repository (HDR) (relationships C and D in Figure 1).

7. The MPI sends updates to the national identity (ICN), VistA identity (DFN), US identity (SSN), or demographics to the vendor server when it is notified of changes (relationship B in Figure 1).

8. Analysis of the collected data performed by the vendor server is the basis of draft progress notes. The vendor server sends the draft progress notes data to VistA (relationship E in Figure 1).

9. Patients receive a Patient Satisfaction Survey every 90 days and a self-assessment of their ability to cope with the activities of daily living (ADL) every 90 days2 with a 45 day offset. The vendor server sends the results of each survey to the survey database for analysis reports.

10. A disease management protocol (DMP) defines the process that evaluates the patient’s current medical condition. The vendor server sends the results of the patient’s responses to the DMP database for analysis reports.3

11. Each vendor system is required to send a weekly census that records the patient receiving care, their care program (DMP) and compliance with the care program to the census database for analysis reports.

The overall process includes both initial system and future system HL7 requirements. Steps 2, 3, 6, 7, 8, 9 and 11 (above) are immediate HL7 integration requirements for the initial deployment.

Steps 4 and 5 are performed manually. The rest are future requirements to be implemented at a later stage of the program. The testing covers only the items required for the initial deployment.

Table 2 discusses the testing done to verify the data flows.

Table 2: Tests (Data Flow Perspective) Data flow item VA system Objectives

(1) Consult construction

VistA (not tested)

(2) Sign Up message

VistA Ability of the vendor server to process the PID constructed by VistA Ability of the vendor server to store data from a valid PID Ability of the vendor server to recognize an invalid PID Ability of the vendor server to construct a correct ACK message

(3) MPI

subscription

MPI Ability of the vendor server to build the MPI subscription request Ability of the vendor server to process the MPI subscription response Ability of the vendor server to recognize an MPI response that creates a local data safety issue

(4) Medical Order VistA (not tested – future capability)

(5) Home device VistA (not tested)

(6) Data observation

HDR Ability of the vendor server to build observation messages for each of the required home measurements

2 Currently the ADL (VR-12) survey is suspended.

3 The DMP response report is a future requirement.

Data flow item VA system

Objectives

Ability of the vendor server to update for a previously submitted observation message Ability of the vendor server to send an observation entered in error.

(7) MPI update MPI Ability of the vendor server to update ICN, DFN, and SSN identity values (MRNs) Ability of the vendor server to recognize a local data safety issue caused by an ICN, DFN, or SSN update Ability of the vendor server to accept patient demographics update

(8) Progress Notes VistA Ability of the vendor server to build the 28-day draft progress note4 Ability of the vendor server to build the Inactivation draft progress note5 Ability of the vendor server to build the “out-of-bounds” draft progress note6.

(9) Survey submissions

HDR Ability of the vendor server to build the satisfaction survey submission documenting the patient responses to the survey Ability of the vendor server to build the ADL survey submission documenting the patient responses to the survey7 Ability of the a VA database to recognize improper survey submission.8

(10) DMP

Submission

HDR Ability of the vendor server to build a DMP response submission documenting the patient responses to the various VA standardized DMPs9

(11) Census submission

HDR Ability of the vendor server to construct the weekly census submission Ability of the a VA database to recognize improper database submission.10

HL7 message handling

(all) Ability to recover from lost messages Ability to properly dispose of misdirected messages Ability to properly dispose of messages with incorrect MSH fields Ability to properly dispose of unsupported message types Ability to properly dispose of transactions with incorrect patient identification Ability to properly dispose of transactions that would cause a data safety issue Ability to comply with the ACID principles

2.2 CCOW Patient Context

The test environment validates the ability of the vendor workstation applications11 to track the patient context with CPRS and other medical application. If CPRS or other CCOW compliant

4 The 28-day progress note is currently suspended and not validated.

5 The progress note sent when a patient is inactivated (dis-enrolled from the program) will be validated in the

EITL, but suspended for the ITL and production.

6 The “out-of-bounds” progress note will be activated in the future.

7 The ADL survey is currently suspended.

8 This test does not involve a “vendor server” 9 The DMP submission is a future requirement.

10 This test does not involve a “vendor server” application is open on a workstation and the vendor application is opened, the vendor application is to display data for the current patient context or a display window with no patient data. If the vendor application is open and CPRS opens, CPRS is to display data for the current patient context or a display window with no patient data. The testing verifies that switching patients in either application forces a switch by the other application. If the patient context changes to a patient that is not available12 to a peer application, the application should not display any patient data13. CPRS is currently CCOW patient compliant.

2.3 System Functions

The following sections discuss the functions that are performed using HL7 messaging.

Activation (Sign up), MPI subscription, Updates from the MPI, Inactivation, Observations, Survey, Census, Progress Notes and Acknowledgement are the functions addressed in this document. Medical Orders and DMP Submission are future requirements. Table 3 defines each of the functions.

The functions fall into one of two groups: maintenance of the patient record and patient generated data. The following sections discuss each function and the testing performed to validate compliance with the requirements. The groups are Patient Record (section 2.3.1) and Patient Generated Data (section 2.3.2). Each section contains an overview of the function and an overview of the testing performed to verify compliance with the requirements.

Table 3: Function Priorities and Groups Function Priority Purpose Group Activation One This function establishes a patient in a vendor server, if the patient record does not exist. If the patient is not currently active14, the patient is activated and a new census record is created. The function is also used to update selected components of the patient record of an active patient. The patient record components that the function updates are patient’s care manager, the patient’s address and the patient’s phone number. The activation function is used to start the collection of 28-day progress note information when the progress note submission is enabled.

2.3.1

Inactivation One This function closes the current census record indicating that the patient is no longer receiving care by the vendor server. Final 28-day progress

2.3.1

11 A vendor application is a program or web browser executed on a workstation.

12 A patient’s data may not be “available” if either the patient is not present in the application database or the user does not have sufficient permission to view the patient’s data.

13 The general method to implement this capability is to display a window with no patient data (such as a patient list), a blank viewing screen, or close the viewer. No patient data should be displayed.

14 A patient is active if the patient is registered for care management.

Function Priority Purpose Group note information is to be submitted.

Observation Submission

One This function sends the patient vitals to the HDR server. New vital sign data originates with the patient’s method of collection. The vital sign status15 can be updated via the clinical desktop.

2.3.2

MPI Subscription One This function informs the MPI that the vendor server is monitoring the patient. The MPI will send updates for medical record index (ICN, DFN, and SSN) and demographics (name and DOB) when the information changes via the MPI update function.

2.3.1

MPI Update One This function informs the vendor server of an update to a patient’s database index or demographics.

2.3.1

Acknowledgements One This function supplies either a message acknowledgment (acceptance for processing) or an application acknowledgement (results of the processing). Every other function includes both forms of the acknowledgement function.

2.3.1 2.3.2

Medical order Three16 This function gives the vendor server the parameters needed to define the use of a home device. This function should imply an activation function.

2.3.2

Progress alarm One This function will build a draft progress note containing an alarm for a validated “out of range” medical submission; that is, the alarm is generated when the care provider decides that the days report from the patient is “out of range”.

2.3.2

Progress note One This function will build a draft progress note containing the patient’s 28-day progress.

2.3.2

Survey (Satisfaction)

One This function will build a message that documents the patient’s responses to a Satisfaction survey. The Satisfaction survey summarizes the patient’s view of the program’s results of the medical care.

2.3.2

Survey (ADL)17 One This function will build a message that documents the patient’s responses to an ADL survey. The Activity of Daily Living summarizes the patient’s ability to cope with normal actions.

2.3.2

DMP Submission Three18 This function will build a message that documents the patient’s responses to the daily DMP.

2.3.2

Census One This function will build a census submission that summarized the testing activity for a patient. The census submission is built weekly (on Sunday) and contains a summary of the patient’s activity for the week.

2.3.2

2.3.1 Patient Record

Through the Patient Record group of functions, a Home Telehealth vendor receives patient identity indices and demographics along with updates to the indices and demographics. A

15 Each vital sign stored in the HDR has a status of valid or questionable. Valid vital signs are displayed on a clinical desktop and reports; questionable vital signs are not used for patient evaluation.

16 The medical order is a future requirement.

17 The ADL survey has been suspended.

18 The submission of the daily DMP responses is a future requirement.

patient record is in one of two states: enrolled for care (active) and not enrolled for care (inactive). When the patient record is in an active state, the vendor system will expect the patient to interact with the Disease Management Protocol program (DMP) according to a proscribed schedule. The DMP presents a survey to the patient. The objective of the survey is to elicit the patient’s current medical condition. The survey includes questions on the patient’s health and activity and may include a request that the patient collect a set of vital signs. The Patient Record group includes the activation, the MPI subscription, the MPI update, and the inactivation functions. This group includes three distinct functions (or events).

The activation function creates a patient record with a Home Telehealth vendor, if a record does not currently exist; otherwise, it updates the existing patient record. The function is starts with an HL7 message sent by a VistA system that contains the patient’s identity indices (ICN, DFN, and SSN) and demographics (name and DOB) and includes the patient’s address, phone number, and official care manager (care coordinator). Activation includes a subscription to the MPI update service; the response to the subscription contains the official identity indices, name and DOB for the patient. If the information supplied by VistA and the information supplied by the MPI do not agree, the vendor must raise an error condition alert. The activation function places the patient in the active state. An active patient is a patient enrolled in the Home Telehealth program with the vendor system providing monitoring for a treating facility (VistA facility).

Error conditions include patient data that would place the vendor’s database in an unsafe state and the receipt of a sign up for a patient that is already active with a different treating facility.

The inactivation function is starts with an HL7 message sent by the VistA system identified by the patient record. The patient record is located using the ICN. The inactivation function places the patient in an inactive state; that is, the patient is dis-enrolled from the Home Telehealth program. The vendor server is not to process inactivation message for a dis-enrolled patient; the proper response would be processing successfully completed. Error conditions include inactivation requests for patients not in the vendor’s database or a request for an active patient from a facility other than the patient’s treating facility.

The MPI update function updates the patient indices or demographics in the vendor database.

The update function is initiated by an HL7 message from the MPI. The updates are identified by the patient’s ICN. If the ICN does not identify a current patient record, the update is to be is to receive a positive response; if a response is requested, a positive response is appropriate. The MPI update The MPI will also send updates to patient demographics. Error conditions include ICN, DFN, or SSN correlation updates that place the vendor’s database in an unsafe state.

Another error condition occurs when the response to the subscription request does not originate with the MPI.

A medical order is a component of this group. The testing of the medical order component will be defined in an update to the test plan.

2.3.2 Patient Generated Data

The Patient Generated Data group is a set of functions that define the processing of HL7 messages generated by a vendor server containing information collected by a patient or program reports generated by a vendor server. The set of functions are DMP Submission, Observation, Progress alarm, Progress note, Survey (Satisfaction), Survey (ADL), and Census. Each of the messages is generated by the vendor server and contains data relative to the care of patients.

The DMP Submission function defines the creation of a message that contains all of the responses by the patient to the question set in the assigned DMP. Generally, the DMP is presented to the patient daily; however, each DMP may have a unique scheduling algorithm.

The DMP response message is sent to the HDR. A vendor will be required to generate each type of DMP for use during the EITL testing. The validation process examines the DMP submissions for proper XML formatting and content. The application ACK will contain indications of an error condition.

The Observation Submission function defines the creation of a message that contains a patient’s vital sign collection results. Generally, the DMP defining the patient’s care program would include collection of vital signs appropriate for the treatment. The vendor is required to flag each collected vital sign as either valid (accurate) or questionable (entered in error); each submission contains a status of valid or questionable. Questionable data is not to be used for clinical purposes. The vendor is also required to supply a method to modify the status of a vital sign already stored. The Observation Submission response message is sent to the HDR. A vendor will be required to generate each type of vital sign for use during the EITL testing; that is, the vendor must be able to create a valid submission, a questionable submission, the change of valid to questionable status of a stored submission, and the change of questionable to valid status of a stored submission. The validation process examines the submissions for proper content.

The application ACK will contain indications of an error condition.

Progress notes are a component of this group. A vendor server must be able to generate a 28-day progress note early and must generate a progress note when a patient is die-enrolled for patient care. The consult number that is associated with the signup allows the progress note to be “found” by the care coordinator using VistA Web or CPRS. The care coordinator associated with the signup allows the progress note to be “signed” so that all clinicians are able to view the note. The application acknowledgement indicates that the progress note was, or was not, accepted by VistA.

Surveys are a component of this group. A vendor generates each type of survey (Satisfaction and ADL) for the patients established during the testing. The validation process examines the surveys for proper XML formatting and content. The application ACK contains indications of an error condition.

The weekly Census submission is a component of this group. A vendor generates a census submission for the patients established during the testing. The validation process examines the census submission for proper XML formatting and content. The application ACK will contain indications of an error condition.

3 Test Phases There are three phases in the validation testing. The purpose of each phase is the identification that the vendor server is ready to move to the next phase. The objective of the overall test plan is the validation that the vendor server is ready to support a full patient complement using HL7 protocol integrated with the VA systems.

3.1 Phase I (EITL)

The phase I testing is performed using a Validation Engine that emulates all the VA systems shown in Figure 1. The validation engine is available upon request and can be installed in a local laboratory; the documentation for the validation engine installation identifies all dependencies.

A laboratory that is accessible from the Internet is also available. The internet based laboratory assumes that the SUT (system under test) in a vendor’s facility. Table 4 lists the access points for the laboratory; the common name for the laboratory is the External Integration Test Laboratory (EITL). The DNS listed in Table 4 are for the VA system; the DNS for the HL7 engine (HL7 Messaging) is HL7.<VA DNS>; for example, HL7.552.VA.GBlankenship.us is the DNS for the HL7 engine for VistA system 552. The assignments in Table 4 follow the standard VA rules.

Table 4: Phase I Configuration (Internet EITL) Facility DNS Port 200HD 200HD.VA.GBlankenship.us 3001 200M 200M.VA.GBlankenship.us 3002 552 552.VA.GBlankenship.us 3552 613 613.VA.GBlankenship.us 3613 657 657.VA.GBlankenship.us 3657 601 601.VA.GBlankenship.us 3601 493 493.VA.GBlankenship.us 3493 995 995.VA.GBlankenship.us 3995 999 999.VA.GBlankenship.us 3999 688HT 688HT.VA.GBlankenship.us 3003 HT vendor (specified by vendor) 5121

A laboratory that is accessible inside the VA Intranet is also available. The intranet based laboratory assumes that the SUT (system under test) VA data center. Table 5 lists the access points for the laboratory. The DNS listed in Table 5 are for the VA systems; the DNS for the HL7 engine (HL7 Messaging) is HL7.<VA DNS>; for example, EITL.HL7.552.CC.MED.VA.GOV is the DNS for the HL7 engine for VistA system 552. The assignments in Table 5 follow the standard VA rules.

Table 5: Phase I Configuration (Intranet EITL) Facility DNS Port

200HD EITL.200HD.CC.MED.VA.GOV 3001

200M EITL.200M. CC.MED.VA.GOV 3002

552 EITL.552. CC.MED.VA.GOV 3552

613 EITL.613. CC.MED.VA.GOV 3613

657 EITL.657. CC.MED.VA.GOV 3657

601 EITL.601. CC.MED.VA.GOV 3601

493 EITL.493. CC.MED.VA.GOV 3493

995 EITL.995. CC.MED.VA.GOV 3995

999 EITL.999. CC.MED.VA.GOV 3999

688HT EITL.688HT. CC.MED.VA.GOV 3003

Facility DNS Port HT vendor (specified by vendor) 5121

The phase I testing is accomplished in three steps. The First Step is a protocol validation; the tests are referenced as PreEITL tests. The Second Step is a protocol sequence validation; the tests are referenced as EITL tests. The Third Step is the processing of invalid protocol messages and sequences; the tests are also referenced as EITL tests.

3.1.1 Objective

The objective of the Phase I testing is the validation of protocol sequences between a vendor server and a VA system. The HL7 Validation Engine verifies the protocol sequences. The engine simulates all peer VA systems. Public access to the engine is available at using the information supplied in Table 4; VA Intranet access to the engine is available at using the information supplied in Table 5.

3.1.2 Tests

In the first step, the engine tests each of the functions described in section 2.3. The engine supplies correctly constructed messages and proper responses. The message sequences tested in this step are the issuance of a message and the acknowledgement of the message processing.

Table 6 defines the messages sent during this step.

Table 6: Phase I Tests (Acceptance of Valid Messages) No Function Source Validation Objective A1 Sign Up

(Activation) VistA Receipt of Application

Acknowledgement Preliminary verification of acceptance of valid message

A2 MPI Subscription Vendor Construction of QBP message and use of DFN from sign up message

Proper construction of subscription message Ability to extract DFN from sign up message

A3 MPI Subscription Response

MPI Receipt of Application Acknowledgement

Preliminary verification of acceptance of valid message

A4 Observation Result Vendor Construction of ORU message for each required measurement

Proper construction of observation message

A5 Identity Update MPI Receipt of Application Acknowledgement

Preliminary verification of acceptance of valid message

A6 Demographics Update

MPI Receipt of Application Acknowledgement

Preliminary verification of acceptance of valid message

A7 Progress Note Vendor Receipt of Application Acknowledgement

Proper construction of progress note message.

A8 Inactivation VistA Receipt of Application Acknowledgement

Preliminary verification of acceptance of valid message

A9 Survey (Satisfaction)

Vendor Receipt of Application Acknowledgement

Preliminary verification of acceptance of valid message

A10 Survey (ADL) Vendor Receipt of Application Acknowledgement

Preliminary verification of acceptance of valid message

A11 DMP Vendor Receipt of Application Acknowledgement

Preliminary verification of acceptance of valid message

A12 Census Vendor Receipt of Application Acknowledgement

Preliminary verification of acceptance of valid message

In the second step, the engine sends valid message constructions and sequences to the vendor server. The engine verifies the vendor server follows the data flow described in section 2.1.

Table 7 describes the message sequence that is tested. Test starts with the engine sending a sign up message (emulation of a VistA facility system). The vendor server sends an MPI subscription using the DFN from the sign up message. The engine will respond with a query response containing the same PID from the sign up message (emulation of the MPI). The vendor server acknowledges the MPI and the VistA facility. The vendor server sends observation messages to the engine (emulation of the HDR). Each observation message contains the PID from the sign up message. The engine sends patient update messages (emulation of the MPI). The vendor server sends observation messages to the emulator (emulation of the HDR). Each observation message should contain an updated PID reflecting the MPI updates.

Table 7: Phase I Tests (Valid Message Sequences) No Function Source Validation Objective B1 Sign Up

(Activation) VistA Receipt of QBP message

Receipt of Application Acknowledgement for RSP message Receipt of Application Acknowledgement for ADT message

Verification message sequence that indicates that vendor server extracted patient data from sign up message and informed MPI of the patient sign up.

Vendor patient panel reflects patient integration.

B2 Observation Result Vendor Construction of ORU message Correct patient identification Correct submission identification Correct vital sign meta-data Correct vital sign identification Detection of illegal submission updates

Verification that message sequence indicates that HDR stored the observation.

B3 Identity Update (active patient)

MPI Receipt of Application Acknowledgement of identity update while patient is active

Verification of acceptance of valid message and vendor patient panel would reflect the update.

B4 Identity Update (inactive patient)

MPI Receipt of Application Acknowledgement of identity update while patient is inactive

Verification of acceptance of valid message and vendor patient panel would reflect the update.

B5 Demographics Update (active patient)

MPI Receipt of Application Acknowledgement of demographics update while patient is active

Verification of acceptance of valid message and vendor patient panel would reflect the update.

B6 Demographics Update (inactive patient)

MPI Receipt of Application Acknowledgement of demographics update while patient is inactive

Verification of acceptance of valid message and vendor patient panel would reflect the update.

B7 Progress Note (generated by date algorithm)

Vendor Construction of MDM message using patient data from sign up message and observation counts

Verification that message sequence indicates that vendor server stored patient data from sign up message and counted observations. Verification that note generated on proper date.

B8 Inactivation VistA Receipt of Application Verification that message sequence

No Function Source Validation Objective Acknowledgement for Inactivation message Receipt of Periodic Progress note indicates that vendor server inactivated patient and generated final progress note.

B9 Change of Care Coordinator

VistA Receipt of Application Acknowledgement of first activation Receipt of Application Acknowledgement of second activation with new care coordinator Receipt of Application Acknowledgement of inactivation Receipt of final progress note addressed to new care coordinator

Verification that message sequence indicates that vendor server inactivated patient and generated final progress note addressed to new care coordinator

B10 Rejection of activation requests for patients active in another VistA facility

VistA Receipt of Application Acknowledgement (rejection) of activation for a patient active in another VistA facility

Verification of message rejection without change to patient data

B11 Rejection of activation requests from non- VistA systems

MPI Receipt of Application Acknowledgement (rejection) of activation from a non-VistA facility

Verification of message rejection without change to patient data

B12 Rejection of activation requests with MPI correlation error

VistA Receipt of Application Acknowledgement (rejection) of activation for a patient with VistA/MPI correlation errors.

Verification of message rejection without change to patient data

B13 Rejection of demographics update from systems other than the MPI

VistA Receipt of Application Acknowledgement (rejection) of patient demographics update from a system other than the MPI

Verification of message rejection without change to patient data

B14 Rejection of messages that should not be processed

VistA

MPI

HDR

(others)

1. Receipt of CA to clear retransmission requirement (if requested)

2. Receipt of Ax to clear retransmission requirement (if requested)

Verification that the message generated the appropriate acknowledgements and was not processed but may have generated an alert. Verification that patient data was not affected.

B15 Rejection of an inactivation message for an active patient in another VistA facility

VistA Receipt of Application Acknowledgement (rejection) of inactivation from a VistA facility that did not activate patient

Verification of message rejection without change to patient data

B16 Rejection of an inactivation message from a non-Vista facility

MPI Receipt of Application Acknowledgement (rejection) of inactivation from a non- VistA facility

Verification of message rejection without change to patient data

B17 Rejection of activation message (workstation

VistA Receipt of Application Acknowledgement (rejection)

Verification of message rejection without change to patient data does not correlate with main facility) of activation from an improper work location

1. Not part of main facility

2. Does not have an account on vendor system

B18 Rejection of activation message when account cannot be found

VistA Receipt of Application Acknowledgement (rejection) of activation

1. Main facility does not have an account

2. Sub-facility (work location) and main facility do not have an account

Verification of message rejection without change to patient data

B19 Proper disposal of messages that cannot be processed

VistA

MPI

HDR

(others)

Receipt of CR to clear retransmission requirement (if requested)

Verification that the message generated a CR (if possible and requested) and was not processed, but did generate an alert

B20 Proper processing of a message when a TCP connection to receiving system cannot be created

Vendor Continued attempt to connect to the target system

Verification that the message is delivered when the connection impediment is removed

B21 Proper processing of a message when a requested Cx is not received

Vendor Continued attempt to deliver (retransmit) the message to the target system

Verification that the message is retransmitted until a Cx is received

B22 Proper processing of a message when a requested Ax is not received

Vendor Continued attempt to deliver (retransmit) the message to the target system

Verification that the message is retransmitted until an Ax is received

B23 Proper processing of a retransmitted message

VistA

MPI

HDR

(others)

Retransmission of the original Cx and Ax

Verification that the message is not reprocess, but is acknowledged

B24 Proper TCP socket for a Cx

VistA

MPI

HDR

(others)

The Cx should be received on the socket that carried the message being acknowledged

Verification that the Cx is not sent of another open TCP socket

B25 Burst messages (same patient)

VistA

MPI

CA and Ax for all messages Verification that messages will be queued for processing

B26 Burst messages (different patients)

VistA

MPI

CA and Ax for all messages Verification that messages will be queued for processing

B27 Sign Up of patient integrated patient with new ICN. SSN is for existing patient.

VistA A CA and AR should be received. The AR diagnostic should be indicate a new ICN for existing patient.

Verification that duplicate patient record is not created

In the third step, invalid message constructions and sequences sent to the vendor server by the engine. Invalid messages are required to generate application rejection messages. Invalid message sequences generate application rejection messages. Table 8 lists the tests that validate the message rejection messages.

Table 8: Phase I Tests (Invalid Message Content and Message Sequences) No Function Source Validation Objective C1 Sign Up

(missing ICN) VistA Receipt of Application

Rejection Preliminary verification of rejection of invalid message

C2 Sign Up (missing DFN)

VistA Receipt of Application Rejection

Preliminary verification of rejection of invalid message

C3 Sign Up (missing SSN)

VistA Receipt of Application Rejection

Preliminary verification of rejection of invalid message

C4 Sign Up (missing name)

VistA Receipt of Application Rejection

Preliminary verification of rejection of invalid message

C5 Sign Up (malformed ICN)

VistA Receipt of Application Rejection

Preliminary verification of rejection of invalid message

C6 Sign Up (malformed DFN)

VistA Receipt of Application Rejection

Preliminary verification of rejection of invalid message

C7 Sign Up (malformed SSN)

VistA Receipt of Application Rejection

Preliminary verification of rejection of invalid message

C8 Sign Up (malformed name)

VistA Receipt of Application Rejection

Preliminary verification of rejection of invalid message

C9 Update (missing ICN)

MPI Receipt of Application Rejection

Preliminary verification of rejection of invalid message

C10 Update (missing DFN)

MPI Receipt of Application Rejection

Preliminary verification of rejection of invalid message

C11 Update (missing SSN)

MPI Receipt of Application Rejection

Preliminary verification of rejection of invalid message

C12 Update (missing name)

MPI Receipt of Application Rejection

Preliminary verification of rejection of invalid message

C13 Update (malformed ICN)

MPI Receipt of Application Rejection

Preliminary verification of rejection of invalid message

C14 Update (malformed DFN)

MPI Receipt of Application Rejection

Preliminary verification of rejection of invalid message

C15 Update (malformed SSN)

MPI Receipt of Application Rejection

Preliminary verification of rejection of invalid message

C16 Update (malformed name)

MPI Receipt of Application Rejection

Preliminary verification of rejection of invalid message

3.2 Phase II (Integration Testing)

The Phase I testing is performed using a laboratory that is accessible from the Internet with the SUT (system under test) in a vendor’s facility. Table 4 gives the access points for the laboratory;

the common name for the laboratory is the Integration Test Lab (ITL).

Table 9: Phase II Configuration (ITL) Facility DNS19 Port 200HD HDRCLUSA-QI.aac.va.gov 6031

200M MPI.SQA.IAM.VA.GOV 5014

552 ISPA05. FO-BAYPINES.MED.VA.GOV 54503

613 ISPA09. FO-BAYPINES.MED.VA.GOV 54441

657 ISPA09. FO-BAYPINES.MED.VA.GOV 54476

688HT vaww.hl7.nhtdb.cc.med.va.gov 5000

Phase II testing is accomp1Blished in two steps. The first step is a test with each VA system in a test environment validating the communication with the individual VA system. The second step tests the full data flow using a test database and test patients. The second step of integration testing includes the verification of CCOW compliance.

3.2.1 Objective

The objective of the Phase II testing is the validation that the vendor is capable of integration into the VA environment. The ITL is the test bed for all Phase II which is available on the VA Intranet.

3.2.2 Tests

In the first step, the test conductor validates each of the functions described in section 2.1 . The VA test systems supply correctly constructed messages and proper responses. All message sequences tested in this step are the issuance of a message and the acknowledgement of the message processing. Table 10 defines the messages sent during this step.

Table 10: Phase II Tests (Integration with Individual VA Test Systems) No Function Source Validation Objective A1 Sign Up VistA Receipt of Application

Acknowledgement Preliminary verification of acceptance of valid message

A2 MPI Subscription Vendor Construction of QBP message and use of DFN from sign up message

Proper construction of subscription message Ability to extract DFN from sign up message

A3 MPI Subscription Response

MPI Receipt of Application Acknowledgement

Preliminary verification of acceptance of valid message

A4 Observation Result Vendor Construction of ORU message for each required measurement

Proper construction of observation message

A5 Identity Update MPI Receipt of Application Acknowledgement

Preliminary verification of acceptance of valid message

A6 Demographics Update

MPI Receipt of Application Acknowledgement

Preliminary verification of acceptance of valid message

A7 Progress Note Vendor Receipt of Application Proper construction of progress

19 ITL DNS and port are subject to change due to the testing requirements in the VA’s Test Laboratory

Acknowledgement note message

A8 Survey (Satisfaction)

Vendor Receipt of Application Acknowledgement

Proper construction of…