36C10B20Q0218-003.pdf

Department of Veterans Affairs

Veteran Benefits Administration (VBA) Performance

Testing

Master Test Plan – March

Coordinated Install

CLIN3002AD

Version 4.4

February 25, 2020

VBA Performance Testing

Test Plan ii February 2020

Revision History

Date Version Description Author

2/25/2020 4.4 March CI Test Plan FTC

1/27/2020 4.3 February CI Test Plan FTC

1/6/2020 4.2 January CI Test Plan FTC

12/30/2020 4.1 December CI Test Plan FTC

10/24/2020 4.0 October CI Test Plan FTC

Test Plan iii February 2020

Table of Contents

1. Report Details (CLIN3002)

2. Introduction

2.1. Purpose

2.2. Test Objectives

2.3. Roles and Responsibilities

2.4. Processes

3. Items to be Tested

3.1. Overview of Test Inclusions

3.2. Overview of Test Exclusions

4. Test Approach

5. Testing Techniques

5.1. Productivity and Support Tools

6. Test Criteria

6.1. Pass/Fail Criteria

6.2. Suspension and Resumption Criteria

7. Test Deliverables

8. Test Schedule

9. Test Environments

9.1. Test Environment Configurations

9.2. Base System Hardware

10. Staffing and Training Needs

11. Risks and Constraints

12. Test Metrics

13. Appendix A – 508 Compliance

Test Plan 1 February 2020

1. Report Details (CLIN3002) Date of Report: February 25, 2020

Contract: VA11816F10060005

Deliverable: CLIN 3002AD – Test Plan Report

Name Contact Information

Kesha Burch Project Manager

Email: Kesha.Burch@va.gov

Telephone: 708-483-5135

Tony Canales Contracting Officer’s Representative (COR)

Email: Tony.Canales@va.gov

Telephone: 512-484-2949

Dana Newcomb Contracting Officer (CO)

Email: Dana.newcomb@va.gov

Telephone: 732-440-9680

Name Contact Information

David Peterson Program Manager, HITC Hines, II

Telephone: 217-390-7820 Email:

david.peterson1@va.gov mailto:Kesha.Burch@va.gov mailto:Tony.Canales@va.gov mailto:Dana.newcomb@va.gov mailto:david.peterson1@va.gov

Test Plan 2 February 2020

2. Introduction

2.1. Purpose

The purpose of performance testing the Coordinated Install is to ensure that the new software does not impact performance adversely.

2.2. Test Objectives

A one-hour test emulating the one-hour peak load on VBAPROD was executed on VBAPERF.

Approximately 15% of the transactions account for over 90% of utilization in Tuxedo and

Oracle. These transactions are the core of the test suite.

There are 13 corporate applications in the test suite. The performance team examined

ClearQuest, Application Turnover Transmittals (ATT) and release notes to determine the requirements to target performance testing of this release. Corporate applications require testing whenever WebLogic applications change because they use the same database.

2.3. Roles and Responsibilities

Table 1 lists the key roles and their responsibilities for this Master Test Plan.

Table 1: Roles and Descriptions

Role Description

Performance Team Performance team builds and executes tests.

Project Manager Person that has overall responsibility for the successful planning and execution of a project; person responsible for creating the Master Test Plan in collaboration with the Development Manager.

Stakeholders Persons that hold a stake in a situation in which they may affect or be affected by the outcome.

Test Environment Team Persons that establish, maintain, and control test environments.

2.4. Processes

The processes that guide the implementation of this Master Test Plan are:

• Test Preparation

• Test Execution

• Evaluation

The references that support the implementation of this Master Test Plan are:

• Performance Testing Process - December 2019

Test Plan 3 February 2020

3. Items to be Tested Table 2 - Coordinated Install

Tuxedo Applications in the Test Suite

Awards Share PIES WINRS

VNARCH FAS RBA CSUM

COVERS HINQ MAPD CovSequenceCheck

CSEM

Web Applications in the Test Suite

WEAMS WSMS FOCAS QAWeb

Web Services

Batch Applications

FAS Batch Monthly FAS Batch Daily FAS Reports EAC

3.1. Overview of Test Inclusions

The target transaction rate for the VBAPERF test suite is 3.9M transactions per hour. The test suite was recalibrated and based on a peak hour load for Production from January 31, 2019.

3.2. Overview of Test Exclusions

Not all WebLogic applications are in scope for this contract.

4. Test Approach The standard VBAPROD test suite was executed using HP/Mercury LoadRunner. Test scripts and test cases are available at the Hines ITC under the following folder:

\\vbahintstrat9\Mercury\Master_Repository\Master_Scripts. The test data bed was established from the December 20, 2019 refresh of VBAPERF from the production system with the

December release applied.

A one-hour test emulating the one-hour peak load on VBAPROD was executed on VBAPERF.

This suite has evolved over the past several years. Approximately 15% of the transactions account for over 90% of utilization in Tuxedo and Oracle. These transactions are the core of the test suite.

5. Testing Techniques There are 13 corporate applications, 4 WebLogic applications and web services in the test suite.

The performance team examined ClearQuest, Application Turnover Transmittals (ATT) and release notes to determine the requirements to target performance testing of this release. All file://///vbahintstrat9/Mercury/Master_Repository/Master_Scripts

Test Plan 4 February 2020 applications require testing whenever database changes are made because they use the same database.

Based on this information, a test suite was created using LoadRunner. The test suite is executed a number of times to ensure consistent results.

5.1. Productivity and Support Tools

Table 3 describes the tools that will be employed to support this Master Test Plan.

Table 3: Tool Category or Types

Tool Category or Type Tool Brand Name Vendor or In-house Version

Performance Testing LoadRunner MicroFocus 12.55

6. Test Criteria

6.1. Pass/Fail Criteria

Pass/Fail criteria are decisions based on the response time of the applications. System resource utilization is also taken into consideration.

6.2. Suspension and Resumption Criteria

Tests are suspended when it is clear that response time goals will not be met or excessive errors are observed.

Resumption starts when root cause has been identified and corrected. The entire test is repeated.

7. Test Deliverables Table 4 lists the test deliverables for the Performance Testing project.

Table 4: Test Deliverables

Test Deliverables Responsible Party

Performance Test Plan David Peterson – Project Manager

Test Schedule David Peterson – Project Manager

Test Scripts Josh Wallgren – Test Architect

Use Cases Josh Wallgren – Test Architect

Production Database and File System

Storage Report (before)

David Peterson – Project Manager

Production Database and File System

Storage Report (after)

David Peterson – Project Manager

Test Issues Report David Peterson – Project Manager

Test Plan 5 February 2020

Test Deliverables Responsible Party

Test System Infrastructure Change Report David Peterson – Project Manager

Test System Data Staging Process Report David Peterson – Project Manager

Load Generation Scripting Process Report Josh Wallgren – Test Architect

Client-Server Application Specific Report David Peterson – Project Manager

Client-Server Application Specific Lessons

Learned Report David Peterson – Project Manager

Client-Server Summary Findings Report David Peterson – Project Manager

Client-Server Threshold Projection Report David Peterson – Project Manager

Client-Server Infrastructure Enhancements

Reports David Peterson – Project Manager

Web-based Application Specific Reports David Peterson – Project Manager

Web-based Application Specific Lessons

Learned Report David Peterson – Project Manager

Web-based Summary Findings Report David Peterson – Project Manager

Web-based Threshold Projection Report David Peterson – Project Manager

Web-based Infrastructure Enhancements

Reports David Peterson – Project Manager

Batch Application Specific Reports David Peterson – Project Manager

Batch Application Specific Lessons Learned

Report David Peterson – Project Manager

Batch Threshold Projection Report David Peterson – Project Manager

Batch Infrastructure Enhancements Reports David Peterson – Project Manager

Test Plan 6 February 2020

8. Test Schedule Table 5: Testing Milestones

Testing Milestones Responsible Party

Application installs Enterprise Operations

Sync changes Performance Team

Run performance test Performance Team

Analyze results Performance Team

Rerun as required Performance Team

Generate reports Performance Team

Restore environments Performance Team

9. Test Environments Hines performance test environment consists of a database server, Tuxedo servers, WebLogic servers, Active Directory servers, Apache servers and SiteMinder servers.

9.1. Test Environment Configurations

The party or parties responsible for configuring and maintaining the test environments are:

ITOPS and the Hines Performance team.

9.2. Base System Hardware

Table 6 sets forth the system resources for the test effort presented in this Test Plan.

Table 6: System Hardware Resources

Resource Quantity

VBAPERF - Oracle 1

WebLogic Servers 9

Tuxedo Servers 4

LoadRunner 6

Apache Server 3

SiteMinder Servers 2

Active Directory Uses Production servers

Test Plan 7 February 2020

10. Staffing and Training Needs Table 7 describes the personnel resources needed to plan, prepare, and execute this Master Test

Plan.

Table 7: Staffing Resources

Testing Task Quantity of

Personnel Needed

Test Process Duration/

Days

Create the Master Test Plan 1 Test

Preparation

1 day

Establish the Test Environment 7 Test

Preparation

2 days

Execute Performance Tests 4 Test Execution 10 days

11. Risks and Constraints Primary issue is software updates. Fixes are applied throughout the test cycle. This marginalizes previous results and requires additional testing.

12. Test Metrics Metrics are a system of parameters or methods for quantitative and periodic assessment of a process that is to be measured. Test metrics may include, but are not limited to:

• Response time

• CPU utilization

• Memory utilization

Test Plan 8 February 2020

13. Appendix A – 508 Compliance All deliverables are compliant with Section 508 standards’ requirements for accessibility and include final test results demonstrating Section 508 compliance.