QASP SURPASS Task Order One.pdf

Support for UniFi Replacement Providing an Advanced Streamlined System

(SURPASS)

Quality Assurance Surveillance Plan (QASP)

Office of the Chief Information Officer

Rural Development Washington, D.C.

Date: 02/22/2022 Version 1.0

Contract Number [TBD] Task Order No.: [TBD]

Table of Contents

SECTION 1: INTRODUCTION

1.1 Purpose

1.2 QASP Relation to the Contract

1.2.1 QASP Revisions

1.2.2 Surveillance of Performance Progression

1.3 QASP Relation to the Quality Control Plan (QCP)

SECTION 2: PERFORMANCE DESCRIPTION

2.1 Performance Standards and Acceptable Quality Levels (AQLs)

2.1.1 Allowable Deviation

2.1.2 Substantially Complete

2.2 Non-performance

2.2.1 Documentation

2.2.2 Remedial Actions

SECTION 3: CONTRACTOR AND GOVERNMENT RESPONSIBILITIES

3.1 Contractor Responsibility

3.2 Government Responsibility

3.2.1 Contracting Officer (CO)

3.2.2 Contracting Officer Representative (COR)

3.2.3 Customers: RDTO and RHS SFHGLP

SECTION 4: SURVEILLANCE METHODS TO PERFORM QUALITY ASSURANCE

4.1 Surveillance Methods

4.1.1 How the PRS method of surveillance will be measured

4.1.2 Surveillance Methods

4.2 Analysis and Results

APPENDIX 1. PERFORMANCE REQUIREMENT SUMMARY

APPENDIX 2. CUSTOMER FEEDBACK

SECTION 1: INTRODUCTION

This Quality Assurance Surveillance Plan (QASP) is pursuant to the requirements listed in the Indefinite Delivery /Indefinite Quantity (IDIQ) performance work statement (PWS) Task Order One entitled Support for UniFi Replacement Providing an Advanced Streamlined System. This plan sets forth the procedures and guidelines that the USDA, Rural Development Technology Office (RDTO) and Single Family Housing (SFH) will use to ensure the required performance standards or services levels are achieved by the contractor.

1.1 Purpose

The purpose of the QASP is to describe the systematic methods used to measure performance and to identify the reports required and the resources to be employed. The QASP provides a means for evaluating whether the contractor is meeting the performance standards identified in the PWS Task Order One. This QASP is designed to define roles and responsibilities, identify the performance objectives, define the methodologies used to monitor and evaluate the contractor’s performance, describe quality assurance reporting, and describe the analysis of quality assurance monitoring results.

1.2 QASP Relation to the Contract

1.2.1 QASP Revisions

The Contracting Officer Representative (COR) and Contracting Officer (CO) with agreement from the PMO team, may make revisions or changes to the QASP procedures and surveillance methods or increase or decrease the degree of surveillance methods at any time during the contract performance period. Changes to the Metric must be incorporated into the PWS Task Order One, Performance Requirement Summary (PRS) and QASP by a bilateral modification to contract that is issued by the Contracting Officer (CO). A copy of the QASP is provided to the contractor to enable the contractor to enhance its Quality Control (QC) program to perform in accordance with its Quality Control Plan (QCP).

1.2.2 Surveillance of Performance Progression

As the performance period progresses, the levels of surveillance may be altered for service areas where performance is either consistently excellent or unsatisfactory. If consistently good performance, then the amount of surveillance may be reduced. If observations reveal consistent deficiencies, increased surveillance may be implemented.

1.3 QASP Relation to the Quality Control Plan (QCP)

The QCP is a required element of the contract and the Contractor shall adhere to its established quality control processes and procedures in managing and performing work as described in the contract. While the QCP represents the way in which the Contractor will ensure its quality and timeliness of services, as defined in the PWS Task Order One, the QASP represents the way in which the Government personnel specified in Section 3: Contractor and Government Responsibilities, Paragraph 3.2, Government Responsibilities will evaluate the contractor’s performance. The contractor’s QC program and the residual organization’s QASP should be complementary programs that ensure successful contractor performance.

SECTION 2: PERFORMANCE DESCRIPTION

Performance of the contractor will be monitored through the surveillance methods described in Section 4: Surveillance Methods to Perform Quality Assurance to assess the Contractor’s performance against PWS Task Order One requirements.

2.1 Performance Standards and Acceptable Quality Levels (AQLs)

For selected activities in the PWS Task Order One, the PRS provides a performance standard and an AQL. A performance standard is the expected level of contractor performance. An AQL defines the level of performance that is satisfactory. Depending on the service evaluated and the evaluation method selected, performance standards and AQLs may be stated as several occurrences or as a percentage. Performance standards and AQLs for random sampling and 100 percent inspection are generally stated as percentages. For periodic inspections, performance standards may be stated as either percentages or as absolute numbers.

The contract requires the Contractor to perform all work as specified. Any inaccuracies or omissions in services or products are referred to as “defects” on the part of the Contractor. The Contractor shall be responsible for all identified defects and may be required to perform the work at no cost to the government, within the terms of the contract. e. The AQLs consider that in some instances an allowable level of deficiencies (deviations) is possible while overall performance continues to meet the government’s desired level of service.

2.1.1 Allowable Deviation

The AQLs define the level or number of performance deficiencies the Contractor is permitted to reach under this contract. AQLs consider the difference between an occasional defect and a gross number of defects.

AQLs can be expressed as a percentage of or as an absolute number (e.g., three per month). There may be instances where 100 percent compliance is required, and no deviation is acceptable (e.g., where safety is involved).

2.1.2 Substantially Complete

In some cases, service outputs are evaluated using subjective values (e.g., excellent, satisfactory, unsatisfactory). The criteria for acceptable performance and for defects must be defined for these service outputs. The concept of “substantially complete” should be the basis for inspections based on subjective scales. Work is considered “substantially complete” where there has been no significant departure from the terms of the contract and no omission of essential work. In addition, the Contractor has performed the work required to the best of its ability and the only variance consists of minor omissions or deficiencies.

2.2 Non-performance

Non-performance occurs when the contractor’s performance does not meet the AQL for a given requirement. Requirements may contain multiple performance elements, and therefore, deficiencies may occur in one or more aspects of performance (e.g., timeliness, accuracy, completeness, etc.) or subject areas of effort.

When surveillance indicates that the contractor's service output is not in compliance with the contract requirements, the Contracting Officer’s Representative (COR) must determine whether the Contractor or the Government caused the deficiency. If the cause of the defect rests with the Government, corrective action must be taken through Government channels. If the cause of the defect is due to action or inaction by the contractor, the contractor is responsible for correction of the problem at no additional expense to the Government.

2.2.1 Documentation

Documentation of work non-performed or unacceptable work is essential for tracking Contractor performance.

The COR will document deficient work by compiling facts describing the inspection methods and results and to substantiate nonconformance with the contract. A sample documentation reporting form is provided in Appendix 2, Customer Feedback. The documentation, with any recommendations, will be forwarded to the CO.

In the case of the Contractor, the COR will decide whether to elevate the problem to the CO for corrective action.

2.2.2 Remedial Actions

The Federal Acquisition Regulation allows for penalties if the Contractor fails to perform the required services. Penalties are defined as those actions taken under the direction of the CO against the contractor within the general provisions of the contract for nonconformance to the PWS Task Order One and PRS.

SECTION 3: CONTRACTOR AND GOVERNMENT

RESPONSIBILITIES

3.1 Contractor Responsibility

The Contractor is responsible for delivering products or services in accordance with the contract.

Implementing its QCP, which describes the Contractor’s methods for ensuring all products and services under the contract meet established performance standards and AQLs. Maintaining, and providing for audit, quality control records and reports and all records associated with the investigation and complaint resolutions.

Appointing a single quality control point-of-contact to act as a central recipient of communication from the COR or CO.

3.2 Government Responsibility

3.2.1 Contracting Officer (CO)

The CO is responsible for administering and monitoring contract compliance, contract administration, and cost or price control and for resolving any differences between the observations documented by COR and the contractor’s performance. The CO may delegate various day-to-day contract administration duties to an Administering CO (ACO) and/or the COR for performance management and administrative actions such as invoice approval and issuance of Contract Discrepancy Reports may be, and normally are, delegated by the CO to the COR. The CO shall approve any revisions to the QASP processes or standards.

3.2.2 Contracting Officer Representative (COR)

The COR, is designated in writing, by the CO. The COR will ensure that the QA function is properly executed, plays a key role in contract administration, and performs the contract surveillance and monitoring. Some key contract administration duties include, but are not limited to, performs surveillance as required by this QASP; make recommendations to the CO for issuance of Contract Discrepancy Reports or letters of commendation and acceptance or rejection of completed work and for administrative actions based on unsatisfactory or non-performed work, and revisions or changes to the QASP; and assists the CO in identifying necessary contract modifications and preparing reports of Contractor performance and cost or price.

The COR may use the form(s) included in the Appendices to perform the inspection or other forms as approved by the CO. The Contractor overall guidance is also provided by FAR clause 52.212-4 Contract Terms and Conditions-Commercial in accordance with the PWS Task Order One.

3.2.3 Customers: RDTO and RHS SFHD

Customers or organizations/offices that the Contractor provided support may be requested to assist the COR in conducting QA to provide information relating to the Contractor’s performance through Customer Feedback. Information gained from the Customer Feedback may be used in conjunction with other methods of observation to rate the performance of the Contractor.

SECTION 4: SURVEILLANCE METHODS TO PERFORM

QUALITY ASSURANCE

4.1 Surveillance Methods

The surveillance methods used in the QA process are the Government’s tools to monitor the Contractor’s products and services. The best means of determining whether the Contractor has met all contract requirements is to inspect the Contractor’s service products and analyze the results. Further, documented inspection results are an effective tool in contract administration that can confirm the successful achievement of all performance requirements or highlight areas where defects exist, and improvements are necessary.

4.1.1 How the PRS method of surveillance will be measured

Rigidity or prescriptiveness regarding Agile metrics is counter to the Agile methodology. In Agile, tracking progress does not align to traditional metrics to validate price, schedule, performance, and quality of programs and projects. While price, schedule and performance continue to have meaning in an Agile program, they must be viewed and managed differently for Agile.

Agile Alignment to Traditional Metrics:

Traditional Waterfall Measures Characteristics Agile Measures

Price Dollars Resources

Functionality Complete having met all acceptance criteria per requirements and schedule

Schedule Time Milestones Planning

Sprint completed on time Code deployed on time No less than 95% of planned user stories/features completed per sprint All deliverables are completed per schedule

Performance Customer satisfaction Customer capabilities Quality (of product) Performance (of contractor)

User Story Completion and Acceptance Test Coverage Functionality Complete having met all acceptance criteria per requirements and schedule

Quality Design and Architecture Development (of code) Test and Integration

Meets acceptance criteria Defects Corrected and Verified Test Coverage No more than 4 critical defects per sprint No more than 4 high defects per sprint

The list below provides general guidance on Agile metrics but is not intended to be all-encompassing or definitive. A baseline of metrics should focus on value delivery, quality, and performance (Agile process adoption). Sample metrics include:

• Delivery Metrics: related to pace of delivery and progress Applicable at the Order level. (move details to order level QASP) o Progress against the roadmap (i.e., number of features completed; number of sprints completed;

release burndown). The Release Burndown (also known as Release Burnup) is a good visual indicator of whether the team will be able to complete all the items in the Product Backlog assuming the Release Velocity.

o Progress against the Product Backlog (i.e., number of story points completed; number of stories delivered; percentage of total Backlog completed; release burndown) o Progress against the sprint (i.e., number of story points completed; number of stories delivered;

percentage of total Sprint/iteration work completed; Sprint/iteration burndown). Specifically, the Sprint/iteration burndown metric allows the team to track work progress and make predictions regarding whether the team will complete the work on time.

o Progress against the release (i.e., number of story points completed; umber of stories delivered; percentage of total release work completed; percentage of Product Backlog completed).

• Quality Metrics: related to product quality - Applicable at the Order level. (move details to order level QASP) o Adheres to the user story acceptance criteria agreed to by the Government and the

Contractor o Testing results (e.g., number or percentage of tests that passed/failed) o Number of issues found per sprint (defect count) o Issue type histogram (bar chart that organizes issues by type and count to give a sense of the issues that are the biggest challenge to progress) o Scatter plots of stories and time to completion (a more advanced analysis tool that provides a method to quickly isolate outlier stories to understand why some stories took longer to complete).

• Process/Performance Metrics: related to adherence to the Agile framework and process Applicable at the

Order level. (move details to order level QASP) o Ensure that Agile ceremonies are defined. The ceremonies are usually events organized around planning, demonstrations, retrospectives, or daily standups. Documentation of the Agile process will ensure that all team members and leadership understand the framework, roles, responsibilities, and expectations.

o Roles defined and filled: The Government is not defining a specific Agile framework or methodology, but asking the Contractor to do so, therefore the Government will avoid being overly prescriptive in roles, but the Contractor should ensure that the roles are identified, documented, and assigned as part of the original RFP response, or early after award. Once the work begins, the Government should request information on the team composition and whether the positions are filled.

Specifically, the Government roles related to the business representation (e.g., Product Owner) are critical to the success of the project and to ensuring that the right value is delivered according to the right priorities set by the Business Representative / Product Owner.

o Integration of Contracting, Testing, Security, and Operations personnel as part of the Agile team o Number of stories removed from sprint o Story completion ratio and story point completion ratio (= percent complete/percent planned) o Number of blockers (understanding blocker sources and reasons could help reduce the potential of future blockers) o Training participation. This is another important metric. Training all team members and senior leadership in the Agile framework before the start of the project will help improve overall success.

All contractor staff must also be trained.

o Testing metrics such as lead time for changes (time from requirement definition to delivery), build frequency, deployment frequency, mean time to restore service, and change fail rate are key software delivery performance indicators.

• Price Metrics: related to price measurements Applicable at the Order level. (move details to order level QASP) o Price can be determined based on the number of hours planned and delivered against stories within a sprint. The prices per story metric are subjective and vary according to the complexity of the stories in a sprint and the number of team members contributing to that sprint. Work elements are typically not equivalent to time, and the government takes this into consideration when evaluating these metrics.

Teams may also need several sprints to “normalize” their estimates/velocity, and the price per work element may fluctuate in the beginning. This metric depends heavily on the accuracy of the estimates and stability of the team’s velocity. Recommended metrics to manage price include those listed in this table below.

Metric Used to Answer

Total work elements committed to sprint Projected price of work scheduled

Total hours planned per story in sprint Projected price per story

Actual hours logged against each story in sprint

Actual price of work performed

Actual work elements completed in sprint Price of work completed for potential delivery

4.1.2 Surveillance Methods

The surveillance methods described below include 100 percent inspection, periodic inspection, random sampling, and customer feedback. The number of inspections conducted may be reduced in those instances where the Contractor has established a good performance record. In cases of poor performance, the USDA organization may increase the level of surveillance and focus on known problem areas. In either case, the reasons for the change in surveillance will be documented.

4.1.2.1 100 Percent Inspection The 100 percent inspection method requires complete inspection of a contract requirement and will be used for requirements that are especially critical or where there is some reason for suspecting that the performance standard or AQL is not being met (and therefore, should be more closely monitored). Evaluation schedules for 100 percent inspections will be prepared each month.

4.1.2.1.1 Performance Standards and AQLs

The performance standards and AQLs may be stated as either percentages or absolute numbers.

4.1.2.1.2 Evaluation Procedures

Observed defects for a service monitored by 100 percent inspection is compared to the performance standard and AQL.

4.1.2.2 Periodic Inspection

Periodic inspection provides a systematic way of looking at service outputs and forming conclusions about the Contractor's level of performance in accordance with a planned schedule of surveillance.

Evaluation by periodic inspection is designed to inspect some part but not all the products and services being monitored.

4.1.2.2.1 Application

Specific contract requirements that are to be monitored are selected for evaluation prior to their scheduled accomplishment. Periodic inspection differs from random sampling in the way in which samples are selected – periodic inspection sample selection is based on some subjective rationale and sample sizes are usually arbitrarily determined. With this type of evaluation, the evaluators can direct efforts to those areas where inspections are most needed, and the Contractor knows that those areas are more likely to be monitored than others. Periodic inspection, as compared with random sampling, provides a less sound statistical means of making comparisons between observed and overall performance, and the Contractor’s overall level of performance. Periodic inspection is generally used in two ways. First, it can provide a one-time subjective evaluation of Contractor performance. Second, it can be used to detect a change in the Contractor’s level of performance (i.e., trend analysis). This method requires that the sample selection criteria be well documented and consistently applied from period to period, and that there are no other intervening factors. The price of periodic inspections varies with the level of inspections. Such latitude is important to manage limited resources and focus inspections on known or suspected problems areas.

4.1.2.2.2 Performance Standards and AQLs

Performance standards and AQLs are usually stated in terms of the number of defects detected per time period (e.g., three times per month). There is no specific relationship between sample size and performance standard/AQL. However, when the AQL is expressed as a percentage, it is recommended that the maximum sample size be chosen such that one defect does not exceed the AQL.

4.1.2.2.3 Evaluation Procedures

The levels of evaluation appropriate for periodic inspection are subjective. In order to perform trend analysis from periodic inspection, criteria for sample selection should be applied consistently from period to period. To ensure valid results, the evaluators will use periodic inspection evaluation sheets and follow a detailed inspection schedule. Schedules may be developed monthly to coincide with the Contractor's monthly schedule of work, and regularly updated after receiving the Contractor's definitive weekly schedule. Observed defects for services monitored by periodic inspection will be totaled at the end of each month. For each service, the total number of defects will be compared to the performance standard and AQL.

4.1.2.3 Random Sampling

Random sampling evaluation is a quality assurance method designed to evaluate some, but not all, of a specific contract requirement. This method, based on statistical principles, estimates the Contractor's overall level of performance for a given contract requirement based on a representative sample drawn from a population. Random Sampling is most often used when the number of occurrences of a service is very high.

4.1.2.3.1 Application

The random sampling procedures are based on those set by the American National Standards Institute (ANSI). The random sampling procedures consider the AQL (maximum allowable deviation from the performance standard), the level (intensity) of the evaluation effort, and the population size. There are two ways of applying random sampling for QA surveillance. The first is used only for performance evaluation and allows deductions to be taken only for observed defects; the second is random sampling for performance evaluation and deduction projection (also called extrapolated deductions), which allows deductions against the whole population based on the inspection of the sample. To obtain valid results, random sampling procedures must be followed precisely.

4.1.2.3.2 Performance Standards and AQLs

Performance standards and AQLs may be specified as percentages or absolute numbers.

4.1.2.3.3 Evaluation Procedures

Random Sampling is based solely on a statistical analysis whereby a conclusion is drawn about a population based on a randomly selected sample of that population. For the conclusion to be valid, the sample selected must be representative of the population. A truly representative sample can be achieved by ensuring that the sample is selected randomly, and the size of the sample is enough. A conclusion about Contractor performance can then be made based on the representative sample drawn.

4.1.2.4 Customer Feedback

Validated customer feedback is a quality assurance method based on customer and Contractor interaction. Customers continually receive the outputs of Contractor performance and can evaluate the Contractor on a recurring basis. Because customers have a clear stake in the quality of Contractor services, they are valuable resource for the evaluators.

4.1.2.4.1 Application

Customers are made aware of contract requirements and monitor the services provided by the Contractor, both positive and negative. Where there is a case of poor performance or non-performance, customers notify the evaluators. The evaluators then investigate the report and, if found to be valid, document their findings. The numbers of complaints and resulting inspections depend upon customer awareness and response. If the complaint is valid and caused by poor performance or non-performance by the Contractor, the Contractor must take appropriate corrective action. A valid complaint is one in which the QAE confirms that poor performance or non-performance violates contract requirements.

4.1.2.4.2 Customer Feedback Process

Upon contract award, RHS should send letters to all or selected customer points of-contact.

These letters will inform them of the need for their active participation in the overall Quality Assurance Program. The CO or COR will also provide a Customer Feedback Record (sample at Appendix 2) for the customer to use to either document performance problems or identify when superior services are received. The COR will validate the Customer Feedback Records submitted. It is primarily the responsibility of the Contractor to investigate each complaint to determine the problem. While evaluators can also investigate customer complaints, the responsibility for initial review shall remain with the Contractor. At the Government’s discretion, the evaluators will investigate problems from customer groups and complaints involving major problems with services being provided. The Contractor shall act when a Customer Feedback Record is received. If a valid complaint exists, the Contractor shall re-perform the product or service. The Contractor may use the complaint as an indicator that the QCP needs improvement.

Corrective actions shall be implemented to prevent the recurrence of similar problems in the future or detect and fix such problems before a product or service is delivered to a customer. If the customer complaint is found to be invalid, the CO or COR shall educate the customer regarding contract requirements as they pertain to the customer’s expectations.

4.1.2.4.3 Evaluation Procedure

The Contractor shall report validated complaints each month, so the evaluators may review the valid complaints and formulate action items if necessary. Trend analysis may be used to test for variations in the number of complaints received each month and identify changes in Contractor performance.

4.2 Analysis and Results

When the inspections and customer feedback validations have been completed, the COR will perform an analysis of the Contractor’s performance. The purpose of the analysis is to ensure Government is receiving high-quality products and services from the Contractor. The COR will review the results, rate the Contractor’s compliance with the performance standards and AQLs, and characterize the Contractor’s overall performance. The performance ratings below reflect definitions at FAR 42.15031 Table 42-1-Evaluation Rating Definition. The COR will use these rating to evaluate the quality of contractor’s performance. Analysis of all types of contract monitoring will result in one of the following outcomes:

Rating

(a) Exceptional

(b) Very Good

Definition Note

To justify an Exceptional rating, identify multiple significant events and state how they were of benefit to the Government. A singular benefit, however, could be of such magnitude that it alone constitutes an Exceptional rating. Also, there should have been NO significant weaknesses identified.

To justify a Very Good rating, identify a significant event and state how it was a benefit to the Government. There should have been no significant weaknesses identified.

To justify a Satisfactory rating, there should have

Performance meets contractual requirements and exceeds many to the Government’s benefit. The contractual performance of the element or sub element being evaluated was accomplished with few minor problems for which corrective actions taken by the contractor were highly effective.

Performance meets contractual requirements and exceeds some to the Government’s benefit. The contractual performance of the element or sub element being evaluated was accomplished with some minor problems for which corrective actions taken by the contractor were effective.

(c) Satisfactory

(d) Marginal

Performance meets contractual requirements. The contractual performance of the element or sub-element contains some minor problems for which corrective actions taken by the contractor appear or were satisfactory.

been only minor problems, or major problems the contractor recovered from without impact to the contract/order. There should have been NO significant weaknesses identified. A fundamental principle of assigning ratings is that contractors will not be evaluated with a rating lower than Satisfactory solely for not performing beyond the requirements of the contract/order.

To justify Marginal performance, identify a significant event in each category that the contractor had trouble overcoming and state how it impacted the Government. A Marginal rating should

Performance does not meet some contractual requirements. The contractual performance of the element or sub-element being evaluated reflects a

1 https://www.acquisition.gov/content/part-42-contract-administration-and-audit-services#i42_1503

11 | Page

Rating Definition Note be supported by referencing the management tool that notified the contractor of the contractual deficiency (e.g., management, quality, safety, or environmental deficiency report or letter).

To justify an Unsatisfactory rating, identify multiple significant events in each category that the contractor had trouble overcoming and state how it impacted the Government. A singular problem, however, could be of such serious magnitude that it alone constitutes an unsatisfactory rating. An Unsatisfactory rating should be supported by referencing the management tools used to notify the contractor of the contractual deficiencies (e.g., management, quality, safety, or environmental deficiency reports, or letters).

serious problem for which the contractor has not yet identified corrective actions. The contractor’s proposed actions appear only marginally effective or were not fully implemented.

(e) Unsatisfactor y

Performance does not meet most contractual requirements and recovery is not likely in a timely manner. The contractual performance of the element or sub-element contains a serious problem(s) for which the contractor’s corrective actions appear or were ineffective.

Note 1: Plus, or minus signs may be used to indicate an improving (+) or worsening (-) trend insufficient to change the evaluation status.

Note 2: N/A (not applicable) should be used if the ratings are not going to be applied to an area for evaluation.

APPENDIX 1. PERFORMANCE REQUIREMENT SUMMARY

Performance Requirement

Performance Standard

Method of Measurement

Acceptable Quality Level

(AQL)

Method of Surveillance

Schedule management

All work is completed on agreed upon time per contract requirements and program incremental/sprint planning.

Program Incremental/Sprint completed on time.

No less than 90% of planned user stories/features completed per agreed upon acceptance criteria. All deliverables are completed per schedule

Final deliverable is delivered 90% on time and 90% of stories/features completed per agreed upon acceptance criteria.

100% inspection of due dates and agreed upon scope delivery or customer feedback

Deliverables in Section 3 of the PWS Task Order One

All work is completed on agreed upon time per contract requirements.

Deliverables meets the acceptance criteria.

All deliverables are completed per schedule and meet the acceptance criteria.

Final deliverable is delivered 100% of the time on schedule and meets the acceptance criteria.

100% inspection of due dates and agreed upon scope delivery or customer feedback.

FedRAMP, Moderate Authorization

All work is completed.

All deliverables are completed per schedule and meet the acceptance criteria.

Final deliverable is delivered 100% of the time on schedule and meets the acceptance criteria.

100% inspection of due dates and agreed upon scope delivery or customer feedback.

Sprint Review Summary and Demonstration

All work is completed on agreed upon time per contract requirement.

Deliverable is completed per Vendor and Government agreed upon schedule and meets agreed upon

Deliverable is delivered 90% of the time on schedule and meets the acceptance criteria.

100% inspection of due dates and agreed upon scope delivery or customer feedback.

acceptance criteria.

DSR & Release Notes

All work is completed on agreed upon time per contract requirement.

Meets PSO guidelines and facilitates a successful deployment.

Minimum of 95% successful Deployment Service Request.

100% inspection using a combination of manual review and successful deployment in the specified environment.

Deployed Code Code must successfully build and deploy into USDA designated environments to include UAT.

Sprint completed on time Code deployed on time No less than 95% of planned user stories/features completed per sprint All deliverables are completed per schedule

Successful build in the designated

USDA

environment and code meets acceptance criteria per user stories/features/ep ics.

100% inspection using a combination of manual review and automated testing in the specified environment.

Performance Requirement

Performance Standard

Method of Measurement

Acceptable Quality Level

(AQL)

Method of Surveillance

Code Quality per Sprint

Adheres to the user story acceptance criteria agreed to by the Government and the Contractor

Functionality meets all acceptance criteria per requirements and schedule No more than 4 critical defects per sprint No more than 4 high defects per sprint.

No more than 4 medium defects per sprint.

No more than 10% of test cases that fail.

No more than 4 critical defects per sprint.

No more than 4 high defects per sprint.

No more than 4 medium defects per sprint.

100% inspection using a combination of manual review and automated testing in the specified environment.

Test Scripts and Automated Test Scripts (Selenium)

All work is completed on agreed upon time per contract requirement.

Deliverable is completed per Vendor and Government agreed upon schedule and meets agreed upon acceptance criteria.

Minimum of 95% of test scripts completed per sprint.

100% inspection using a combination of manual review and successful deployment in the specified environment.

Contract weekly status reporting

100% of reports accurately depict current status

Review of contract monthly status report. All acceptance criteria per the executed contract are met.

Contract weekly status report follows the acceptance criteria and is delivered on time per the project schedule.

100% Inspection of weekly reports

Contract monthly status reporting

100% of reports accurately depict current status

Review of contract or task order monthly status report. All acceptance criteria per the executed contract are met.

Contract monthly status report follows the acceptance criteria and is delivered on time per the project schedule.

100% Inspection of monthly reports

APPENDIX 2. CUSTOMER FEEDBACK

DISCREPENCY REPORT 1. DISCREPENCY REPORT NO.

2. TO: (Contractor / Project Manager Name) 3. FROM: (Name of COR)

CONTRACT NUMBER: DATE:

PREPARED ORAL NOTIFICATION RETURNED BY

CONTRACTOR

ACTION COMPLETE

4. DISCREPENCY OR PROBLEM (Describe in Detail. Include PWS Task Order One references. Attach Continuation Sheet if Necessary)

5. SIGNATURE OF COR

6. TO: (Name of COR) 7. FROM: (Contractor / Project Manager)

8. CONTRACTOR RESPONSE AS TO CAUSE, EFFECT, CORRECTIVE ACTION AND ACTIONS TO PREVENT RECURRENCE (Attach Continuation Sheet if necessary. Cite applicable Contractor QC program procedures or new QC procedures)

9. SIGNATURE OF CONTRACTOR REPRESENTIVE 10. DATE

11. GOVERNMENT EVALUATION (Acceptance, partial acceptance, or rejection. Attach Coordination Sheet if necessary)

12. GOVERNMENT ACTIONS (Cure notice, show cause, other)

CLOSE OUT

KTR NOTIFIED

NAME AND TITLE SIGNATURE DATE

COR

1.3 QASP Relation to the Quality Control Plan (QCP)

4.2 Analysis and Results