J2 - Quality Assurance Surveillance Plan.docx

Performance based Quality Assurance Surveillance Plan (QASP)

INTRODUCTION

This Quality Assurance Surveillance Plan (QASP) has been developed to evaluate contractor actions while implementing the SOW. It is designed to provide an effective surveillance method of monitoring contractor performance for each listed objective on the Performance Requirements Matrix in the task order. It also provides a systematic method to evaluate the services the contractor is required to furnish.

STANDARD

The contractor is responsible for management and quality control actions to meet the terms of the task order. The role of the COR is quality assurance to ensure task order standards are achieved. The contractor shall perform all work required in a satisfactory manner in accordance with the requirements of the SOW. The COR shall notify the CO for appropriate action if it is likely that the contractor will not achieve successful final delivery of the software code in accordance with the performance objectives and acceptable quality levels (AQLs) identified below.

PERFORMANCE REQUIREMENTS MATRIX

The COR/ACOR will evaluate the performance objectives through surveillance as reflected below by reviews and acceptance of work products and services. As indicated, the COR/ACOR will assess progress towards the final delivered software code. Note that the performance requirements listed below are required for the final deliverable. However, the sprints and incremental delivery of code will be assessed by the Government to ensure that the contractor is on a path to successful final delivery. The tools mentioned in this matrix are recommended tool. If the contractor wishes to recommend a different tool, that request can be made to the COR and will be evaluated by the Library.

Deliverable or Required Services Performance Standard(s):

https://github.com/LibraryOfCongress/coding-standards

The dev environments are configured with auto-formatting and the appropriate config files are used.

Primary goal of this assessment is to ensure the following:

1- Numbers are trending in the right direction 2- anything which is known to be incorrect / deferred is marked as such in the report Based on SonarQube reports Bugs/Smells – B Vulnerabilities – A(0) Primary inspection method will be SonarQube. A meeting to discuss results to identify edge-cases/false positives.

Valid mark-up Screen reader user can access all content and functions of web and mobile products

0 errors reported for WCAG 2.1 AA standards using an automated scanner and 0 errors reported in manual testing 0 validation errors Screenreader users have full access to content and functions https://github.com/pa11y/pa11y or aXe (for web and android): https:www.deque.com/axe

AND appropriate mark-up validators like: https://validator.w3.org/ AND Manual testing with screenreader, typically JAWs on Desktop and VoiceOver on iOS

PROCEDURES

The COR, along with the OCIO Technical Lead and the product owner, will inspect all tasks required by the task order to ensure contractor compliance with the task order requirements at the conclusion of each sprint, which shall have a length of three weeks or less. If inspection results are satisfactory, no action will be taken; otherwise, deficiencies will be noted and discussions will be held with the contractor. The COR may find the delivery satisfactory even though further work is required, provided that the specific requirements of the sprint are met.

At the conclusion of each sprint, the COR, along with the OCIO Technical lead and the product owner, will review the completed user stories and related functionality. Incomplete or inadequate code and user stories will be noted in a mutually agreed-upon issue tracker, such as Jira, and links to each issue shared with the CO. The contractor may respond in that tracker as appropriate, addressing the accuracy and validity of the defect as well as any planned corrective action (if not already noted). The contractor team will discuss and document actions to prevent recurrence in their sprint retrospectives.

At the conclusion of the period of performance, a similar procedure will be followed to document discrepancies and to assess overall performance.

If any of the services do not conform to the call order requirements, the COR may require the contractor to perform the services again in conformity with call order requirements. Any user stories that are not accepted must be completed in the next sprint, unless the product owner and OCIO Technical manager agree to move it to a later sprint. The COR shall not certify satisfactory performance for the task order until all defects have been corrected. When the defects in services cannot be corrected by re-performance, the Government may:

1. Require the contractor to take necessary action to ensure that future performance conforms to call order requirements; and,

2. Reduce the task order price to reflect the reduced value of the services performed. The COR shall, in addition to providing documentation to the CO and meve@loc.gov, maintain a complete quality assurance file. The file will contain copies of all reports, evaluations, recommendations, and any actions relating to the Government’s performance of the quality assurance function, including originals of all surveillance activity checklists. All such records will be retained for the life of the call order. The COR shall forward these records to the CO at completion or termination of the call order.

Attachment J: Quality Assurance Surveillance Plan (QASP) – Supplemental Questions and Answers

1. What is the process if a story does NOT meet the success criteria set at the beginning of a sprint? How are these and other defects handled?

If a story fails Quality Assurance (unable to meet the documented Acceptance Criteria), the story moves back to “TO DO” in JIRA and the team will evaluate and immediately rectify, if possible. If the story is not completed by the end of the sprint, no credit is given for work completed and the story either moves to the next sprint or back to the Product Backlog (depending on priority assigned by Product Owner).

If it is deemed that the Story was not completed due to a failure on the Federal side, it will not be counted against the contractor.

1. What is the process when new requirements are discovered during grooming the backlog, were not originally included, or otherwise discovered during execution of the contract?

When additional requirements are defined, either by changing business needs or through “progressive elaboration,” the requirements will be documented in User Story format and placed in the Product Backlog where the Product Owner will assign a relative priority in relation to all outstanding work defined on the project.

1. What is the process for handling technical debt?

Technical Debt will be identified and prioritized separately from features by the Technical Lead (federal FTE) in collaboration with the Development Team. As needed, time will be allocated to Technical Debt remediation on both the Product Roadmap and within the Sprint Schedule. On projects where Technical Debt is substantial, it (Technical Debt) will be allocated a percentage of the Development Team’s capacity within each sprint.

Technical Debt is not a byproduct or poor decision or actions of the contractor. It is a natural by-product of software engineering. It should not be counted against the vendor.

1. What is the process for handling any remaining backlog that exists at or near the end of the contract?

Often, there will be requirements left un-fulfilled at the end of a contract period of performance. Frequently, this is due to changing business needs and/or the process of “progressive elaboration” on stated requirements. Unfrequently, it is due to poor performance of the contractor.

When it is determined that contractor poor performance is the root cause, additional sprints should be added in order to complete a specific set of requirements at no cost to the government. If root cause is determined to be due to additional scope added to the project by the Government, and the additional requirements need to be delivered, then Government should have the ability via the contract to continue work with as little disruption as possible at government cost.