Attachment J4 - Quality Assurance Surveillance Plan (QASP).pdf

Attachment J4 – Quality Assurance Surveillance Plan (QASP) 8/8/2025

Performance based Quality Assurance Surveillance Plan

INTRODUCTION

This Quality Assurance Surveillance Plan (QASP) has been developed to evaluate contractor actions while implementing the SOW. It is designed to provide an effective surveillance method of monitoring contractor performance for each listed objective on the Performance Requirements Matrix in the task order. It also provides a systematic method to evaluate the services the contractor is required to furnish.

STANDARD

The contractor is responsible for management and quality control actions to meet the terms of the task order. The role of the COR is quality assurance to ensure task order standards are achieved. The contractor shall perform all work required in a satisfactory manner in accordance with the requirements of the SOW. The COR shall notify the CO for appropriate action if it is likely that the contractor will not achieve successful final delivery of the software code in accordance with the performance objectives and acceptable quality levels (AQLs) identified below.

PERFORMANCE REQUIREMENTS MATRIX

The COR/ACOR will evaluate the performance objectives through surveillance as reflected below by reviews and acceptance of work products and services. As indicated, the COR/ACOR will assess progress towards the final delivered software code. Note that the performance requirements listed below are required for the final deliverable.

However, the sprints and incremental delivery of code will be assessed by the Government to ensure that the contractor is on a path to successful final delivery. The tools mentioned in this matrix are recommended tool. If the contractor wishes to recommend a different tool, that request can be made to the COR and will be evaluated by the Library.

Deliverable or Required Services Performance Standard(s):

Deliverable or Required

Services

Performance Standard(s)

Acceptable Quality Level

Method of Surveillance

Tested Code (including regression testing)

Code delivered under the order must have substantial test code coverage and a clean code base. Branch coverage must be enabled in this analysis.

Minimum of 90% test coverage of all relevant code. If coverage falls below 90%, contractor can provide rational for this, if acceptable, it won’t count against standard.

Automated testing using a tool like SonarQube during code check-in and prior to each release. The tool will be approved/agreed to by Library during sprint review.

Properly Styled Code

According to LOC Guides and SDLC available at:

https://github.com/LibraryOfC ongress/coding-standards

The dev environments are configured with auto-formatting and the appropriate config files are used.

100% by final delivery

Manual spot checking.

Any improperly formatted code will require an explanation of why it does not meet the standards.

Code Quality Code must be analyzed for code quality which is comprised of “Bugs”, “Smells” and “Vulnerabilities”.

Primary goal of this assessment is to ensure the following:

1- Numbers are trending in the right direction

2- anything which is known to be incorrect / deferred is marked as such in the report

Based on SonarQube reports

Bugs/Smells – B

Vulnerabilities – A(0)

Primary inspection method will be SonarQube. A meeting to discuss results to identify edge-cases/false positives.

https://frontend.18f.gov/#js-style https://github.com/LibraryOfCongress/coding-standards https://github.com/LibraryOfCongress/coding-standards

Performance Planned vs. Actual Work Completed

Minimum of 90% on the calculation of Actual Story Points Completed in a Sprint <devided by> Planned Story Points in a Sprint.

Provides early warning signal of possible issues with performance.

“Velocity Chart” Report in JIRA

Accessible Web Content Accessibility Guidelines 2.1 AA (WCAG

2.1 AA) standards

Valid mark-up

Screen reader user can access all content and functions of web and mobile products

0 errors reported for

WCAG 2.1 AA

standards using an automated scanner and 0 errors reported in manual testing

0 validation errors

Screenreader users have full access to content and functions or https://github.com /pa11y/pa11y or aXe (for web and android):

https:www.deque.com/ axe AND appropriate mark-up validators like https://validator.w3.org/

AND

Manual testing with screenreader, typically JAWs on Desktop and VoiceOver on iOS

Deployed Code must successfully build and deploy into test/pre-production environment via an automated means. Manual steps in the deployment will not be accepted.

Successful build of application that passes smoke testing.

Combination of manual review and automated testing

QA and Software Testing

Manual Test Cases/ Scenarios and Automated test cases delivered under this order will provide full coverage of front-end code once deployed into the test/pre-production environments. Along with providing a Daily, Incident, and Final Test Report.

Successful test execution of all manual and automated test cases.

Test Case Repository/ Management Tool to store Manual and Automated Test Cases, in addition to using tools like Selenium, Cucumber, or Python to automate regression test cases.

https://github.com/pa11y/pa11y https://github.com/pa11y/pa11y https://validator/

Documented All dependencies are listed and the licenses are documented. Major functionality in the software/source code is documented. Individual methods are documented inline using comments that permit the use of tools such as Javadoc. System diagram must be provided.

Documentation must meet LOC PMLC and SDLC standards

Manual review

PROCEDURES

The COR, along with the OCIO Technical Lead and the product owner, will inspect all tasks required by the task order to ensure contractor compliance with the task order requirements at the conclusion of each sprint, which shall have a length of three weeks or less. If inspection results are satisfactory, no action will be taken; otherwise, deficiencies will be noted and discussions will be held with the contractor. The COR may find the delivery satisfactory even though further work is required, provided that the specific requirements of the sprint are met.

At the conclusion of each sprint, the COR, along with the OCIO Technical lead and the product owner, will review the completed user stories and related functionality.

Incomplete or inadequate code and user stories will be noted in a mutually agreed-upon issue tracker, such as Jira, and links to each issue shared with the CO. The contractor may respond in that tracker as appropriate, addressing the accuracy and validity of the defect as well as any planned corrective action (if not already noted). The contractor team will discuss and document actions to prevent recurrence in their sprint retrospectives.

At the conclusion of the period of performance, a similar procedure will be followed to document discrepancies and to assess overall performance.

If any of the services do not conform to the call order requirements, the COR may require the contractor to perform the services again in conformity with call order requirements. Any user stories that are not accepted must be completed in the next sprint, unless the product owner and OCIO Technical manager agree to move it to a later sprint. The COR shall not certify satisfactory performance for the task order until all defects have been corrected. When the defects in services cannot be corrected by re-performance, the Government may:

1. Require the contractor to take necessary action to ensure that future performance conforms to call order requirements; and,

2. Reduce the task order price to reflect the reduced value of the services performed. The COR shall, in addition to providing documentation to the CO, maintain a complete quality assurance file. The file will contain copies of all reports, evaluations, recommendations, and any actions relating to the Government’s performance of the quality assurance function, including originals of all surveillance activity checklists. All such records will be retained for the life of the call order. The COR shall forward these records to the CO at completion or termination of the call order.