36C10B20Q0218-001.docx

VBA Performance Testing

VA-20-00031743

PERFORMANCE WORK STATEMENT (PWS)

DEPARTMENT OF VETERANS AFFAIRS

Office of Information & Technology (OIT) IT Operations (ITOPS) Infrastructure Operations (IO) Health, Benefits, Memorial and Corporate (HBMC) Veterans Benefits Administration Performance Testing

Date: March 27, 2020 VA-20-00031743 PWS Version Number: 3.0

Contents

BACKGROUND

The mission of the Department of Veterans Affairs (VA), Office of Information & Technology (OI&T), IT Operations and Services (ITOPS) Infrastructure Operations (IO) Health Benefits Memorial and Corporate (HBMC) Benefits Service Line is to provide benefits and services to Veterans of the United States. In meeting these goals, OI&T strives to provide high quality, effective, and efficient Information Technology (IT) services to those responsible for providing care to the Veterans at the point-of-care as well as throughout all the points of the Veterans’ health care in an effective, timely and compassionate manner. VA depends on Information Management/Information Technology (IM/IT) systems to meet mission goals.

The Veterans Benefits Administration (VBA) Production System (VBA Corporate) is the main payment system for VBA Compensation, Pension, and Vocational Rehabilitation benefits processing. It has interfaces to the Beneficiary Identification and Records Locator System (BIRLS) and the Benefits Delivery Network (BDN). The BIRLS is on an IBM mainframe and the BDN is on a Bull/Atos mainframe. Online traffic into the VBA Corporate environment is via client server technology (currently Tuxedo) and web technology (currently WebLogic). In addition, the VBA Corporate environment has a batch processing component. This infrastructure is subject to change as refresh of equipment and technologies are implemented.

The Veterans Relationship Management (VRM) / Customer Relationship Management (CRM) program goal is to provide improved capabilities to achieve on-demand access to comprehensive VA services and benefits in a consistent, user-centric manner to enhance VA clients’ self-service experience through a multi-channel CRM approach. This initiative is designed to improve the speed, accuracy, and efficiency in which information is exchanged between VA and VA clients, regardless of the communications method (phone, web, email, etc.)

The Veterans Benefits Management System (VBMS) is a web-based, electronic claims processing solution complemented by improved business processes. VBMS assists in eliminating the claims backlog and serves as the enabling technology for quicker, more accurate, and integrated claims processing. Implementation of VBMS has helped VA meet increasing demand while providing timelier and more responsive customer service to Veterans and their families. VBA is replacing the functionality of the existing suite of Veterans Service Network (VETSNET) client server applications with VBMS web-based applications.

VETSNET Compensation and Pension (C&P) is a suite of applications that facilitates the entire C&P claims process. Within the suite, the end user can establish and develop Veterans’ claims; the rating decision, award and notification letter are documented, and payment information is transmitted to Treasury, accomplishing the necessary accounting. Throughout these activities, data are shared and passed between the applications to support end-to-end claims processing, customer service and notification.

Chapter 33 is an education benefit processing system. It is an independent benefits processing system that receives data from VBA Corporate database residing on VBA Production.

In order for the VBA Production environment to be resilient, a performance environment needs to be available to performance test application and system software changes as well as hardware infrastructure changes before production deployments. This task order (TO) is not for hardware and software maintenance. Rather, it is to simulate production system performance in a dedicated test environment. This involves testing the effect of software and hardware changes.

APPLICABLE DOCUMENTS

The Contractor shall comply with the following documents, in addition to the documents in Paragraph 2.0 in the T4NG Basic Performance Work Statement (PWS), in the performance of this effort:

1. “Veteran Focused Integration Process (VIP) Guide 3.2”, December 2018, https://www.voa.va.gov/DocumentView.aspx?DocumentID=4371

2. POLARIS User Guide”, Version 1.2, February 2016, https://www.voa.va.gov/DocumentView.aspx?DocumentID=4412

3. Application List.xls

4. Sample Load/Performance Test Plan

5. Sample Use Cases.txt

6. Sample Test Scripts.zip

7. Planned Hardware/Software Upgrades

8. SIO Fiscal Release Schedule 2021

9. Third Quarter FY 20 Quarterly Capacity Projection Report

SCOPE OF WORK

The Contractor shall analyze and conduct performance testing of system software and hardware infrastructure changes and upgrades. The Contractor shall conduct performance testing on VBA Client-Server, VBA Web-based, Batch, Chapter 33, and VRM/CRM applications. The Contractor shall also provide capacity projections based on the testing conducted on the VBA Performance (VBAPERF) Test Systems.

APPLICABILITY

This TO effort PWS is within the scope of paragraph(s) 4.1 Program Management, Strategy, Enterprise Architecture and Planning Support, 4.2 Systems/Software Engineering, 4.3 Software Technology Demonstration and Transition, 4.4 Test & Evaluation (T&E), 4.5 Independent Verification and Validation (IV&V), 4.8.6 Capacity/Availability Planning and Management, 4.8.10 Database and Data Warehouse of the T4NG Basic PWS.

ORDER TYPE

The effort shall be proposed on a Firm Fixed Price (FFP) basis.

Performance Details

PERFORMANCE PERIOD

The Period of Performance (PoP) shall be one (1) twelve (12) month base period with four (4) twelve (12) month option periods, and three (3) optional tasks.

4.2 PLACE OF PERFORMANCE

Efforts under this TO shall be performed in VA facilities located at one of the following Enterprise Operations (EO) Data Centers: the Austin Information Technology Center (AITC), the Hines Information Technology Center (HITC), the Philadelphia Technology Center (PITC), or the Quantico Information Technology Center (QITC). Work may be performed at remote locations with prior concurrence of the Contracting Officer’s Representative (COR).

4.3 TRAVEL or special requirements

The Government anticipates travel to perform the tasks associated with the effort, as well as to attend an annual program-related meeting or conference throughout the PoP. Travel shall be limited to the Contractor’s Project Manager only, unless otherwise approved in advance by the COR. Include all estimated travel costs in your FFP line items. These costs will not be directly reimbursed by the Government.

The total estimated number of trips in support of the program related meetings for this effort is one (1) per period of performance. The anticipated location will be one of the following, estimated at three (3) days in duration:

1. HITC, Hines IL

1. AITC, Austin, TX

1. VA Central Office, Washington, DC

0. CONTRACT MANAGEMENT

All requirements of Sections 7.0 and 8.0 of the T4NG Basic PWS apply to this effort. This TO shall be addressed in the Contractor’s Progress, Status and Management Report (CPSMR) as set forth in the T4NG Basic contract.

0. GOVERNMENT FURNISHED PROPERTY

VA will provide workspace and appropriate system accesses in accordance with the system access policy. VA may provide parking, dependent upon the Data Center, in accordance with local Data Center policy. VA will issue Personal Identity Verification (PIV) badges for physical and logical access in accordance with VA PIV badge issuance policy.

The Government has multiple remote access solutions available to include Citrix Access Gateway (CAG), Site-to-Site Virtual Private Network (VPN), and RESCUE VPN.

The Government’s issuance of Government Furnished Equipment (GFE) is limited to Contractor personnel requiring direct access to the network to: development environments; install, configure and run Technical Reference Model (TRM) approved software and tools (e.g., Oracle, Fortify, Eclipse, SoapUI, WebLogic, LoadRunner); upload/download/ manipulate code, run scripts, and apply patches; configure and change system settings; check logs, troubleshoot/debug, and test/QA.

When necessary, the Government will furnish desktops or laptops, for use by the Contractor to access VA networks, systems, or applications to meet the requirements of this PWS. The overarching goal is to determine the most cost-effective approach to providing needed access to the VA environment coupled with the need to ensure proper Change Management principles are followed. Contractor personnel shall adhere to all VA system access requirements for on-site and remote users in accordance with VA standards, local security regulations, policies and rules of behavior. GFE shall be approved by the COR and Program Manager on a case-by-case basis prior to issuance.

Based upon the Government assessment of remote access solutions and requirements of this TO, the Government estimates that the following GFE will be required by this TO:

1. 10 Developer laptops

The Government will not provide IT accessories including but not limited to Mobile Wi-Fi hotspots/wireless access points, additional or specialized keyboards or mice, laptop bags, extra charging cables, extra Personal Identity Verification card readers, peripheral devices, or additional Random Access Memory (RAM). The Contractor is responsible for providing these types of IT accessories in support of the TO as necessary and any VA installation required for these IT accessories shall be coordinated with the COR.

The Status of Government Furnished Equipment Report under the T4NG Basic Contract requirements is applicable to this TO and shall be delivered to the COR/VA PM as required.

The Government will ensure the VBAPERF Test System mirrors the Corporate Production System (to include third party software i.e. Oracle, Solaris, Tuxedo; Commercial off the Shelf (COTS) products; and system software) prior to applications performance testing. The Government will also ensure that the applications needed for testing are installed into VBAPERF prior to the Contractor commencing testing.

0. SECURITY AND PRIVACY

All requirements in Section 6.0 of the T4NG Basic PWS apply to this effort. Specific TO requirements relating to Addendum B, Section B4.0 paragraphs j and k supersede the corresponding T4NG Basic PWS paragraphs, and are as follows,

j. The vendor shall notify VA within 24 hours of the discovery or disclosure of successful exploits of the vulnerability which can compromise the security of the Systems (including the confidentiality or integrity of its data and operations, or the availability of the system). Such issues shall be remediated as quickly as is practical, based upon the severity of the incident.

k. When the Security Fixes involve installing third party patches (such as Microsoft OS patches or Adobe Acrobat), the vendor will provide written notice to VA that the patch has been validated as not affecting the Systems within 10 working days. When the vendor is responsible for operations or maintenance of the Systems, they shall apply the Security Fixes based upon the requirements identified within the TO.

It has been determined that protected health information may be disclosed or accessed and a signed Business Associate Agreement (BAA) shall be required. The Contractor shall adhere to the requirements set forth within the BAA, referenced in Section D of the Request for Task Execution Plan (RTEP) and shall comply with VA Directive 6066.

POSITION/TASK RISK DESIGNATION LEVEL(S)

The position sensitivity and the level of background investigation commensurate with the required level of access for the following tasks within the PWS are:

Position Sensitivity and Background Investigation Requirements by Task

The Tasks identified above and the resulting Position Sensitivity and Background Investigation requirements identify, in effect, the Background Investigation requirements for Contractor individuals, based upon the tasks the particular Contractor individual will be working. The submitted Contractor Staff Roster must indicate the required Background Investigation Level for each Contractor individual based upon the tasks the Contractor individual will be working, in accordance with their submitted proposal

SPECIFIC TASKS AND DELIVERABLES

The Contractor shall perform the following:

PROJECT MANAGEMENT

CONTRACTOR PROJECT MANAGEMENT PLAN

The Contractor shall deliver a Contractor Project Management Plan (CPMP) that lays out the Contractor’s approach, timeline and tools to be used in execution of this TO effort. The CPMP should take the form of both a narrative and graphic format that displays the schedule, milestones, risks and resource support. The CPMP shall also include how the Contractor shall coordinate and execute planned, routine, and ad hoc data collection reporting requests as identified within the PWS. The initial baseline CPMP shall be concurred upon and updated in accordance with Section B of the TO. The Contractor shall update and maintain the VA Program Manager (PM) approved CPMP throughout the PoP.

Deliverable:

1. Contractor Project Management Plan

REPORTING REQUIREMENTS

The Contractor shall submit Weekly Status Reports via Primavera Progress Reporter.

The Contractor shall provide daily Risk and Impact Reports to Release Management and Enterprise Operations.

The Contractor shall keep in communication with VA accordingly so that issues that arise are transparent to both parties to prevent escalation of outstanding issues.

Deliverables:

1. Weekly Status Reports

1. Risk and Impact Reports

TECHNICAL KICKOFF MEETING

A technical kickoff meeting shall be held within 10 days after TO award. The Contractor shall coordinate the date, time, and location (can be virtual) with the Contracting Officer (CO), as the Post-Award Conference Chairperson, the VA PM, as the Co-Chairperson, the Contract Specialist (CS), and the COR. The Contractor shall provide a draft agenda to the CO and VA PM at least five (5) calendar days prior to the meeting. Upon Government approval of a final agenda, the Contractor shall distribute to all meeting attendees. During the kickoff-meeting, the Contractor shall present, for review and approval by the Government, the details of the intended approach, work plan, and project schedule for each effort via a Microsoft Office PowerPoint presentation. At the conclusion of the meeting, the Contractor shall update the presentation with a final slide entitled “Summary Report” which shall include notes on any major issues, agreements, or disagreements discussed during the kickoff meeting and the following statement “As the Post-Award Conference Chairperson, I have reviewed the entirety of this presentation and assert that it is an accurate representation and summary of the discussions held during the Technical Kickoff Meeting for the VBS Performance Testing effort. The Contractor shall submit the final updated presentation to the CO for review and signature within three (3) calendar days after the meeting. The Contractor shall also work with the CS, the Government’s designated note taker, to prepare and distribute the meeting minutes of the kickoff meeting to the CO, COR and all attendees within three (3) calendar days after the meeting. The Contractor shall obtain concurrence from the CS on the content of the meeting minutes prior to distribution of the document.

PROJECT COORDINATION AND INTEGRATION

MONTHLY PROGRESS MEETING

The Contractor shall hold a monthly progress review meeting (virtual) with the COR and other Government personnel. The meeting shall be held during the first week of each month. Meeting invites shall be sent, by the Contractor, at least one week before the actual meeting and include a high-level agenda. The Contractor shall provide detailed meeting minutes within 1 (one) week of meeting. The Contractor shall provide the following status in the monthly progress review meetings:

1. Review of results from previous month’s work

1. Review of current work

1. Discussion of technical issues and concerns

VBA DEVELOPMENT AND OPERATIONS MEETINGS

The Contractor shall attend the daily virtual VBA Development and Operations synch up calls to provide daily status reports.

PHASE-IN TRANSITION SUPPORT

The Contractor shall provide phase-in transition and orientation support. The Contractor shall develop and document a transition plan in the CPMP to transition tasks and materials to the Contractor. The Contractor shall present the details of the transition plan at the technical kickoff meeting. After COR concurrence, the Contractor shall monitor the execution of the transition plan. The Contractor shall provide phase-in transition planning to include the following:

0. Roster of key Points of Contact (POCs) with name, role, email address, and telephone numbers.

0. Transition timeline with key milestones.

0. Procedural manuals/guidelines.

0. Templates used in day-to-day operations.

0. Itemized list of additional equipment to be provided.

0. Estimated downtime by site and units (hub or terminal).

0. Process for transfer of on-hand inventory, if applicable.

0. Transition checklist.

0. Signed turnover agreements, if applicable.

0. System design plan.

0. Network architecture plan.

PRE-TEST ACTIVITIES

The Contractor shall coordinate all performance testing on the VBAPERF Test System with the VA COR, HITC/AITC/PITC VBAPERF staff and with other external entities who interface with VBA Corporate and need to test against the environment.

The Contractor shall provide a Test Plan and Test Schedule. The Test Plan and Test Schedule shall outline the test approach and planned activities for the performance testing required in the effort. Specifically, the Contractor shall develop, in conjunction with business and technical stakeholders (Product Development, Enterprise Systems Engineering (ESE), EO, Region 5, PITC), the Test Plans and Test Schedule documenting the relative mix of use cases and their frequency of execution. Each test in the Test Plan shall represent a single operating scenario, (i.e. likely third quarter peak hour peak day, 40% above likely third quarter peak hour peak day, etc). The Test Plan shall be reviewed with technical and business stakeholders and concurrence secured. The Contractor shall provide a Test Plan with content similar to Applicable Document 5, Load/Performance Test Plan.

The Contractor shall update the Test Plan and Test Schedule annually.

The Contractor shall update Use Cases and develop new Use Cases which model the most frequently executed and highest cost user-behaviors associated with processing a claim, document capture and Veteran claim folder review. Historically, 10% of Use Cases need to be modified to accommodate new features. The Use Cases can involve Front End Tool (FET), The Image Management System (TIMS), WEAMS, VA-ONCE, Electronic Certification Automated Processing (ECAP), Veterans On-Line Applications (VONAPP), Web Automated Verification Enrollment (WAVE) and VA Defense Information Repository (VADIR). The Use Cases can also involve BIRLS and other external VA systems. Use Cases shall capture all pertinent interactions both with the primary application system and the referenced external systems. Interactions shall be captured regardless of whether they are programmatically or manually triggered. Use Cases shall be based on documented user requirements and operational directions. Use Cases shall be updated for each planned release of the application software. The Contractor shall provide updated and/or New Use Cases for each of the application performance tests.

The Contractor shall update Test Scripts and develop new Test Scripts using Hewlett Packard LoadRunner V11.5 (software that the VA will provide) and perform any necessary staging of data required to perform the performance tests. Historically, 10% of the approximately 200 Test Scripts need to be modified annually while New Test Scripts are rarely needed to accommodate new features. Use Cases and Test Scripts shall be captured in a VA hosted auditable repository.

Deliverables:

1. Test Plan

1. Test Schedule

1. Use Cases

1. Test Scripts

INFRASTRUCTURE UPGRADE IMPACTS

The Contractor shall analyze performance impacts of system software and hardware infrastructure changes and upgrades identified by the Government (Reference Applicable Document 8, Planned Hardware/Software Upgrades) for the VBAPERF Test System environment. The VBAPERF Test System environment is composed of 5 Weblogic Servers, 5 Apache Servers, 2 SiteMinder Servers, 3 Tuxedo Servers, 1 Oracle database and 6 LoadRunner Servers, Hitachi File Storage System running on Solaris Unix and Windows Operating Systems. The Contractor shall conduct performance testing of software and hardware changes and upgrades after they are implemented by the Government into the performance testing environment, prior to production deployment, to ensure the changes and upgrades do not cause unacceptable performance degradation. The Contractor shall use Oracle Tuxedo and WebLogic applications, or other government approved applications/toolsets, to gather performance metrics. The Contractor shall emulate user loading transactions, observe and record application response time on a peak hour from production basis. The Contractor shall, at a minimum, perform two (2) 150% stress tests annually. The Corporate environment software and hardware changes and upgrades are planned for approximately six (6) system upgrades performed annually. The Contractor shall provide a Test Issues Report detailing all issues associated with each hardware and software infrastructure change and upgrade performance test.

Deliverables:

1. Test Issues Report

INFRASTRUCTURE CHANGES PRIOR TO EACH APPLICATION TEST CYCLE

The Contractor shall provide a VBAPERF Test System Infrastructure Change Report annually describing the infrastructure changes to the VBAPERF Test System made in preparation for each of the application performance test cycles to include Client-Server, Web Based, Batch, and Chapter 33, as well as VRM/CRM Application Performance Testing and Additional Application Performance Testing, if exercised. The Contractor shall provide a VBAPERF Test System Data Staging Process Report documenting the data staging process performed in preparation for each of the application performance test cycles. The Contractor shall provide a VBAPERF Test System Load Generation Scripting Process Report describing the scripting process used to generate a production load and load test execution for each of the application performance test cycles.

The Contractor shall host a virtual meeting within 5 days after each report is delivered. The Contractor shall perform a content walkthrough of the following documents during the meeting:

1. VBAPERF Test System Infrastructure Change Report

1. VBAPERF Test System Data Staging Process Report

1. VBAPERF Test System Load Generation Scripting Process Report The Contractor shall execute the contents of the VBAPERF Test System Load Generation Scripting Process Report with government designated technical resources.

Deliverables:

1. VBAPERF Test System Infrastructure Change Report

1. VBAPERF Test System Data Staging Process Report

1. VBAPERF Test System Load Generation Scripting Process Report

CLIENT-SERVER APPLICATION PERFORMANCE TESTING

The Contractor shall conduct performance testing in support of the approximately twelve (12) coordinated VBA Corporate application installs that occur annually. Applicable Document 8, SIO Fiscal Release Schedule 2021 is a sample preproduction coordinated install schedule which lists the time period to accomplish performance testing in the VBAPERF environment for each coordinated installation. The Contractor shall test all applications resident on the existing platform and the Web infrastructure and up to 4 new VBA Corporate applications that are being placed into production annually. The Contractor shall provide a Coordinated Client-Server Application Specific Report which details each Corporate coordinated application install test cycle results and conclusions, including supporting documentation substantiating test results and conclusions. This report shall also include lessons learned, a summary report of findings, client-threshold projections, and recommendations for enhancing the VBA Corporate infrastructure and improving VBA Corporate application response time and performance.

The Contractor shall conduct performance testing in support of the approximately twelve (12) non-coordinated VBA Corporate application installs per year. The non-coordinated application installs only involve changes to a specific subset of the Corporate applications; however, the Contractor shall include all Corporate applications in the supplemental performance test. The supplemental installs are defined as emergency application corrections for problems that were not discovered during pre-production testing or user acceptance testing. The Contractor shall provide a Non-Coordinated Client-Server Application Specific Report detailing each VBA Corporate non-coordinated application install test cycle results and conclusions, including supporting documentation substantiating testing results and conclusions This report shall also include lessons learned, a summary report of findings, client-threshold projections, and recommendations for enhancing the VBA Corporate infrastructure and improving VBA Corporate application response time and performance.

The Contractor shall execute the aforementioned tests and capture relevant performance data and recommended next steps in the Coordinated/Non-Coordinated Client-Server Application Specific Reports. This shall consist at a minimum of:

1. Anticipated user response times for the elements of the Use Cases

1. Observed resource consumptions on all systems touched by the Use Cases.

1. Observed intersystem data movement

1. Observed bottlenecks and areas of improvement. These shall be prioritized both in terms of their potential impact and the degree of difficulty of their remediation.

1. Documenting details of the tests run, observed results, meaning of the results and recommendations.

The Contractor shall retest as necessary, as fixes are provided to noted performance concerns, and the above test process shall be repeated for each application release.

The Contractor shall include projections, within the Coordinated/Non-Coordinated Client-Server Application Specific Reports, of the production deployment impact of the coordinated and non-coordinated application installs on processor and memory resources on the approved VBA hardware platform and the Web infrastructure.

The Contractor shall include client-server threshold projections, within the Coordinated/Non-Coordinated Client-Server Application Specific Reports, detailing a projection of whether or not the performance testing, for the VBA coordinated installs and VBA non-coordinated installs, indicate a production deployment of the application changes will result in the following thresholds being exceeded on the approved VBA Corporate hardware platform and the Web infrastructure:

· Central Processing Unit (CPU) Usage – 70%

· Memory Usage – 50%

Deliverables:

1. Coordinated Client-Server Application Specific Report

1. Non-Coordinated Client-Server Application Specific Report

CAPACITY PROJECTIONS

The Contractor shall update the Quarterly Capacity Projections Reports based on the testing conducted on the VBAPERF Test Systems. These reports shall include capacity projections and include specific infrastructure upgrade recommendations to ensure VBA Corporate performance issues related to infrastructure capacity will not be encountered. The infrastructure upgrade recommendations shall include supporting data and a trends analysis report for each application tested during the quarterly reporting period detailing transaction volume, CPU utilization, Input/Output (I/O) and memory consumed by each application type; and general system service utility system consumptions and projection. These Quarterly Capacity Projection Reports shall cover a 2-year projection period. The Contractor shall host quarterly virtual meetings, performing an oral walk through of the updated Quarterly Capacity Projection Reports. The Contractor shall provide the updated Quarterly Capacity Projection Reports five (5) days prior to the oral walk through. The meeting shall take place no later than two (2) weeks after the end of a Government fiscal quarter. In the 4th quarter, the meeting shall take place no later than 1 week prior to end of Government fiscal year.

Deliverables:

A. Quarterly Capacity Projections Reports

WEB-BASED APPLICATION PERFORMANCE TESTING

The Contractor shall conduct performance testing in support of the WebLogic environment and Corporate Web-based applications in the approximately twelve (12) coordinated VBA Corporate application installs that occur annually. Applicable Document 8, SIO Fiscal Release Schedule 2021 is a sample preproduction coordinated install schedule which lists the time period to accomplish performance testing in the VBAPERF environment for each coordinated installation. The Contractor shall test all applications resident on the existing platform and the Web infrastructure and up to 4 new VBA Corporate applications that are being placed into production annually. The Contractor shall provide a Web-based Application Specific Report which details each Corporate coordinated application install test cycle results and conclusions, including supporting documentation substantiating test results and conclusions. This report shall also include lessons learned, a summary report of findings, client-threshold projections, and recommendations for enhancing the VBA Corporate infrastructure and improving VBA Corporate application response time and performance.

The Contractor shall conduct performance testing of WebLogic and Web-Based Corporate applications in support of the approximately twelve (12) non-coordinated VBA Corporate application installs per year. The non-coordinated application installs only involve changes to a specific subset of the Corporate applications; however, the Contractor shall include all Corporate applications in the supplemental performance test. The supplemental installs are defined as emergency application corrections for problems that were not discovered during pre-production testing or user acceptance testing. The Contractor shall provide a Non-Coordinated Web-based Application Specific Report detailing each VBA Corporate non-coordinated application install test cycle results and conclusions, including supporting documentation substantiating testing results and conclusions This report shall also include lessons learned, a summary report of findings, client-threshold projections, and recommendations for enhancing the VBA Corporate infrastructure and improving VBA Corporate application response time and performance.

The Contractor shall execute the aforementioned tests and capture relevant performance data and recommended next steps in the Coordinated/Non-Coordinated Web-based Application Specific Reports. This shall consist at a minimum of:

a. Anticipated user response times for the elements of the Use Cases

b. Observed resource consumptions on all systems touched by the Use Cases.

c. Observed intersystem data movement

d. Observed bottlenecks and areas of improvement. These shall be prioritized both in terms of their potential impact and the degree of difficulty of their remediation.

e. Documenting details of the tests run, observed results, meaning of the results and recommendations.

The Contractor shall retest as necessary, as fixes are provided to noted performance concerns, and the above test process shall be repeated for each application release.

The Contractor shall include projections, within the Coordinated/Non-Coordinated Web-based Application Specific Report, of the production deployment impact of the coordinated and non-coordinated application installs on processor and memory resources on the approved VBA hardware platform and the Web infrastructure.

The Contractor shall include a Corporate WebLogic and Web-Based threshold projections, within the Coordinated/Non-Coordinated Web-based Application Specific Report, detailing a projection of whether or not the performance testing, for the VBA coordinated installs and VBA non-coordinated installs, indicate a production deployment of the application changes will result in the following thresholds being exceeded on the approved VBA Corporate hardware platform and the Web infrastructure:

· Central Processing Unit (CPU) Usage – 70%

· Memory Usage – 50%

Deliverables:

A. Coordinated Web-based Application Specific Report B. Non-Coordinated Web-based Application Specific Report

BATCH APPLICATION PERFORMANCE TESTING

The Contractor shall conduct performance testing in support of Corporate Batch Applications in the approximately twelve (12) coordinated VBA Corporate application installs that occur annually. Applicable Document 8, SIO Fiscal Release Schedule 2021 is a sample preproduction coordinated install schedule which lists the time period to accomplish performance testing in the VBAPERF environment for each coordinated installation. The Contractor shall perform Financial Accounting System and Extended Awards Composite Batch application testing. The Contractor shall provide a Corporate Batch Application Specific Report which details each Corporate coordinated application install test cycle results and conclusions, including supporting documentation substantiating test results and conclusions. This report shall also include lessons learned, a summary report of findings, client-threshold projections, and recommendations for enhancing the VBA Corporate infrastructure and improving VBA Corporate application response time and performance.

The Contractor shall conduct performance testing of Corporate Batch Applications in support of the approximately twelve (12) non-coordinated VBA Corporate application installs per year. The non-coordinated application installs only involve changes to a specific subset of the Corporate applications; however, the Contractor shall include all Corporate applications in the supplemental performance test. The supplemental installs are defined as emergency application corrections for problems that were not discovered during pre-production testing or user acceptance testing. The Contractor shall provide a Non-Coordinated Corporate Batch Application Specific Report detailing each VBA Corporate non-coordinated application install test cycle results and conclusions, including supporting documentation substantiating testing results and conclusions This report shall also include lessons learned, a summary report of findings, client-threshold projections, and recommendations for enhancing the VBA Corporate infrastructure and improving VBA Corporate application response time and performance.

The Contractor shall execute the aforementioned tests and capture relevant performance data and recommended next steps in the Coordinated/Non-Coordinated Batch Application Specific Reports. This shall consist at a minimum of:

a. Anticipated user response times for the elements of the Use Cases

b. Observed resource consumptions on all systems touched by the Use Cases.

c. Observed intersystem data movement

d. Observed bottlenecks and areas of improvement. These shall be prioritized both in terms of their potential impact and the degree of difficulty of their remediation.

e. Documenting details of the tests run, observed results, meaning of the results and recommendations.

The Contractor shall retest as necessary, as fixes are provided to noted performance concerns, and the above test process shall be repeated for each application release.

The Contractor shall include projections, within the Coordinated/Non-Coordinated Corporate Batch Application Specific Report, of the production deployment impact of the coordinated and non-coordinated application installs on processor and memory resources on the approved VBA hardware platform and the Web infrastructure.

The Contractor shall include a Corporate Batch Applications threshold projections, within the Coordinated/Non-Coordinated Corporate Batch Application Specific Report, detailing a projection of whether or not the performance testing, for the VBA coordinated installs and VBA non-coordinated installs, indicate a production deployment of the application changes will result in the following thresholds being exceeded on the approved VBA Corporate hardware platform and the Web infrastructure:

· Central Processing Unit (CPU) Usage – 70%

· Memory Usage – 50%

Deliverables:

A. Coordinated Batch Application Specific Report B. Non-Coordinated Batch Application Specific Report

CHAPTER 33 PERFORMANCE TESTING

The Contractor shall conduct performance testing in support of the Chapter 33 suite of web applications (to include TIMS, WEAMS, VA-ONCE, ECAP, VONAPP, WAVE and VADIR) in in the approximately twelve (12) coordinated VBA Corporate application installs that occur annually. Applicable Document 8, SIO Fiscal Release Schedule 2021 is a sample preproduction coordinated install schedule which lists the time period to accomplish performance testing in the VBAPERF environment for each coordinated installation. The Contractor shall provide a Chapter 33 suite of web applications Specific Report which details each Corporate coordinated application install test cycle results and conclusions, including supporting documentation substantiating test results and conclusions. This report shall also include lessons learned, a summary report of findings, client-threshold projections, and recommendations for enhancing the VBA Corporate infrastructure and improving VBA Corporate application response time and performance.

The Contractor shall conduct performance testing of the Chapter 33 suite of web applications in support of the approximately twelve (12) non-coordinated VBA Corporate application installs per year. The non-coordinated application installs only involve changes to a specific subset of the Corporate applications; however, the Contractor shall include all Corporate applications in the supplemental performance test. The supplemental installs are defined as emergency application corrections for problems that were not discovered during pre-production testing or user acceptance testing. The Contractor shall provide a Non-Coordinated Chapter 33 suite of web applications Specific Report detailing each VBA Corporate non-coordinated application install test cycle results and conclusions, including supporting documentation substantiating testing results and conclusions This report shall also include lessons learned, a summary report of findings, client-threshold projections, and recommendations for enhancing the VBA Corporate infrastructure and improving VBA Corporate application response time and performance.

The Contractor shall execute the aforementioned tests and capture relevant performance data and recommended next steps in the Coordinated/Non-Coordinated Chapter 33 Application Specific Reports. This shall consist at a minimum of:

a. Anticipated user response times for the elements of the Use Cases

b. Observed resource consumptions on all systems touched by the Use Cases.

c. Observed intersystem data movement

d. Observed bottlenecks and areas of improvement. These shall be prioritized both in terms of their potential impact and the degree of difficulty of their remediation.

e. Documenting details of the tests run, observed results, meaning of the results and recommendations.

The Contractor shall retest as necessary, as fixes are provided to noted performance concerns, and the above test process shall be repeated for each application release.

The Contractor shall include projections, within the Coordinated/Non-Coordinated Chapter 33 suite of web applications Specific Report, of the production deployment impact of the coordinated and non-coordinated application installs on processor and memory resources on the approved VBA hardware platform and the Web infrastructure.

The Contractor shall include a Chapter 33 suite of web applications threshold projections, within the Coordinated/Non-Coordinated Chapter 33 suite of web Application Specific Report, detailing a projection of whether or not the performance testing, for the VBA coordinated installs and VBA non-coordinated installs, indicate a production deployment of the application changes will result in the following thresholds being exceeded on the approved VBA Corporate hardware platform and the Web infrastructure:

· Central Processing Unit (CPU) Usage – 70%

· Memory Usage – 50%

Deliverables:

A. Coordinated Chapter 33 suite of web applications Specific Report B. Non-Coordinated Chapter 33 suite of web applications Specific Report

OPTION PERIOD ONE (12 months) If Option Period One is exercised by the Government, all the tasks and deliverables in the following sections, including all subparagraphs, shall apply

5.11 OPTION PERIOD TWO (12 months)

If Option Period Two is exercised by the Government, all the tasks and deliverables in the following sections, including all subparagraphs, shall apply

5.12 OPTION PERIOD THREE (12 months)

If Option Period Three is exercised by the Government, all the tasks and deliverables in the following sections, including all subparagraphs, shall apply

5.13 OPTION PERIOD FOUR (12 months)

If Option Period Three is exercised by the Government, all the tasks and deliverables in the following sections, including all subparagraphs, shall apply

5.14 VRM/CRM PERFORMANCE TESTING (OPTIONAL TASK ONE)

The Contractor shall conduct performance testing in support of VRM/CRM’s usage of their suite of applications in in the approximately twelve (12) coordinated VBA Corporate application installs that occur annually. Applicable Document 8, SIO Fiscal Release Schedule 2021 is a sample preproduction coordinated install schedule which lists the time period to accomplish performance testing in the VBAPERF environment for each coordinated installation. The Contractor shall provide a VRM/CRM suite of applications Specific Report which details each Corporate coordinated application install test cycle results and conclusions, including supporting documentation substantiating test results and conclusions. This report shall also include lessons learned, a summary report of findings, client-threshold projections, and recommendations for enhancing the VBA Corporate infrastructure and improving VBA Corporate application response time and performance.

The Contractor shall conduct performance testing of the VRM/CRM suite of applications in support of the approximately twelve (12) non-coordinated VBA Corporate application installs per year. The non-coordinated application installs only involve changes to a specific subset of the Corporate applications; however, the Contractor shall include all Corporate applications in the supplemental performance test. The supplemental installs are defined as emergency application corrections for problems that were not discovered during pre-production testing or user acceptance testing. The Contractor shall provide a Non-Coordinated VRM/CRM suite of applications Specific Report detailing each VBA Corporate non-coordinated application install test cycle results and conclusions, including supporting documentation substantiating testing results and conclusions This report shall also include lessons learned, a summary report of findings, client-threshold projections, and recommendations for enhancing the VBA Corporate infrastructure and improving VBA Corporate application response time and performance.

The Contractor shall execute the aforementioned tests and capture relevant performance data and recommended next steps. This shall consist at a minimum of:

a. Anticipated user response times for the elements of the Use Cases

b. Observed resource consumptions on all systems touched by the Use Cases.

c. Observed intersystem data movement

d. Observed bottlenecks and areas of improvement. These shall be prioritized both in terms of their potential impact and the degree of difficulty of their remediation.

e. Documenting details of the tests run, observed results, meaning of the results and recommendations.

The Contractor shall retest as necessary, as fixes are provided to noted performance concerns, and the above test process shall be repeated for each application release.

The Contractor shall include projections, within the Coordinated/Non-Coordinated VRM/CRM suite of applications Specific Report, of the production deployment impact of the coordinated and non-coordinated application installs on processor and memory resources on the approved VBA hardware platform and the Web infrastructure.

The Contractor shall include a VRM/CRM suite of applications threshold projections, within the Coordinated/Non-Coordinated VRM/CRM suite of Application Specific Report, detailing a projection of whether or not the performance testing, for the VBA coordinated installs and VBA non-coordinated installs, indicate a production deployment of the application changes will result in the following thresholds being exceeded on the approved VBA Corporate hardware platform and the Web infrastructure:

· Central Processing Unit (CPU) Usage – 70%

· Memory Usage – 50%

Deliverables:

A. Coordinated VRM/CRM suite of applications Specific Report B. Non-Coordinated VRM/CRM suite of applications Specific Report

5.15 Additional performance testing (OPTIONAL TASK TWO)

This optional task may be exercised once per performance period (base and each option period) for performance testing of an additional application, or suite of applications, which will be identified at time of option exercise. The performance period shall be up to 12-months but shall not exceed the base or option period it is exercised within. If exercised, the Contractor shall perform the same requirements specified in PWS Section 5.5 for the identified application, or suite of applications.

The price for each option shall be negotiated on a FFP basis for the specific application, or suite of applications identified, prior to each exercise of this optional task. The Government will utilize the proposed firm fixed price labor rates which shall be incorporated into the order at time of award as set forth in the Price Schedule. VA may exercise Optional Task Two upon written notification from the Contracting Officer.