DRAFT QASP 12_9_21.docx

UNITED STATES DEPARTMENT OF AGRICULTURE (USDA)

RURAL DEVELOPMENT (RD)

DRAFT

Quality Assurance Surveillance Plan (QASP) for Single-Family Housing Nationwide Appraisals Services (SHNAS)

9 December 2021

Contents

Project Name Single-Family Housing Nationwide Appraisal Services (SHNAS)

Solicitation / Contract Number

TBD

Section 1: Introduction This Quality Assurance Surveillance Plan (QASP) is pursuant to the requirements listed in the performance work statement (PWS) entitled Single-Family Housing Nationwide Appraisal Services (SHNOAS). This plan sets forth the procedures and guidelines that the USDA, Rural Development (RD) Procurement Management Office (PMO) will use in ensuring the required performance standards or services levels are achieved by the Contractor.

1.1 Purpose

The purpose of the QASP is to describe the systematic methods used to measure performance and to identify the reports required and the resources to be employed. The QASP provides a means for evaluating whether the Contractor is meeting the performance standards identified in the PWS. This QASP is designed to define roles and responsibilities, identify the performance objectives, define the methodologies used to monitor and evaluate the Contractor’s performance, describe quality assurance reporting, and describe the analysis of quality assurance monitoring results.

1.2 QASP Relation to the Contract

1.2.1 QASP Revisions

The Contracting Officer Representative (COR) may make revisions or changes to the QASP procedures and surveillance methods or increase or decrease the degree of surveillance methods at any time during the contract performance period. Changes to the Metric must be incorporated into the PWS, Performance Requirement Summary (PRS) and QASP by a bilateral modification to the contract that is issued by the Contracting Officer (CO). A copy of the QASP is provided to the Contractor to enable the Contractor to enhance its Quality Control (QC) program to perform in accordance with its Quality Control Plan (QCP).

1.2.2 Surveillance of Performance Progression

As the performance period progresses, the levels of surveillance may be altered for service areas where performance is either consistently excellent or unsatisfactory. If consistently good performance, then the amount of surveillance may be reduced. If observations reveal consistent deficiencies, increased surveillance may be implemented.

1.3 QASP Relation to the Quality Control Plan (QCP)

The QCP is a required element of the contract and the Contractor shall adhere to its established quality control processes and procedures in managing and performing work as described in the contract. While the QCP represents the way in which the Contractor will ensure its quality and timeliness of services, as defined in the PWS, the QASP represents the way in which the Government personnel specified in Section 3: Contractor and Government Responsibilities, Paragraph 3.2, Government Responsibilities will evaluate the Contractor’s performance. The Contractor’s QC program and the residual organization’s QASP should be complementary programs that ensure successful Contractor performance.

Section 2: Performance Description Performance of the Contractor will be monitored through the surveillance methods described in Section 4: Surveillance Methods to Perform Quality Assurance to assess the Contractor’s performance against PWS requirements.

2.1 Performance Standards and Acceptable Quality Levels (AQLs) For selected activities in the PWS, the PRS provides a performance standard and an AQL. A performance standard is the expected level of Contractor performance. An AQL defines the level of performance that is satisfactory. Depending on the service evaluated and the evaluation method selected, performance standards and AQLs may be stated as a number of occurrences or as a percentage. Performance standards and AQLs are generally stated as percentages. For periodic inspections, performance standards may be stated as either percentages or as absolute numbers.

The contract requires the Contractor to perform all work as specified. Any inaccuracies or omissions in services or products are referred to as “defects” on the part of the Contractor. The Contractor shall be responsible for all identified defects and may be required to perform the work at no cost to the government. The AQLs take into account that in some instances of an allowable level of deficiencies (deviations) is possible while overall performance continues to meet the government’s desired level of service.

2.1.1 Allowable Deviation

The AQLs define the level or number of performance deficiencies the Contractor is permitted to reach under this contract. AQLs take into account the difference between an occasional defect and a gross number of defects. AQLs can be expressed as a percentage of or as an absolute number (e.g., three per month). There may be instances where 100 percent compliance is required, and no deviation is acceptable (e.g., where safety is involved).

2.1.2 Substantially Complete

In some cases, service outputs are evaluated using subjective values (e.g., excellent, satisfactory, unsatisfactory). The criteria for acceptable performance and for defects must be defined for these service outputs. The concept of “substantially complete” should be the basis for inspections based on subjective scales. Work is considered “substantially complete” where there has been no significant departure from the terms of the contract and no omission of essential work. In addition, the Contractor has performed the work required to the best of its ability and the only variance consists of minor omissions or deficiencies.

2.2 Non-Performance

Non-performance occurs when the Contractor’s performance does not meet the AQL for a given requirement. Requirements may contain multiple performance elements, and therefore, deficiencies may occur in one or more aspects of performance (e.g., timeliness, accuracy, completeness, etc.) or subject areas of effort.

When surveillance indicates that the Contractor's service output is not in compliance with the contract requirements, the Contracting Officer’s Representative (COR) must determine whether the Contractor or the Government caused the deficiency. If the cause of the defect rests with the Government, corrective action must be taken through Government channels. If the cause of the defect is due to action or inaction by the Contractor, the Government may consider one of the following actions: (1) the Contractor is responsible for correction of the problem at no additional expense to the Government; (2) the Government may charge the Contractor for any costs incurred by the Government that are directly related to having another Contractor perform the work, including the difference in price by the new Contractor that is hired to perform the services requested; (3) terminate the contract and/or individual task order.

All instances of non-conformance must be documented verbally or in writing, in the form of a corrective action request (CAR):

2.2.1 Documentation

Documentation of work not performed or unacceptable work is essential for tracking Contractor performance. The COR will document deficient work by compiling facts describing the inspection methods and results to substantiate nonconformance with the contract. The documentation, with any recommendations, will be forwarded to the CO. In the case of the Contractor, the COR will decide whether to elevate the problem to the CO for corrective action.

2.2.2 Remedial Actions

The Federal Acquisition Regulation allows for penalties in the event that the Contractor fails to perform the required services. Penalties are defined as those actions taken under the direction of the CO against the Contractor within the general provisions of the contract for nonconformance to the PWS and PRS. FAR 52.211-11 Liquidated Damages-Supplies, Services or Research and Development is in Section F Deliveries or Performance of the contract.

Section 3: Contractor and Government Responsibilities

3.1 Contractor Responsibility

The Contractor is responsible for delivering products or services in accordance with the contract. Implementing its QCP, which describes the Contractor’s methods for ensuring all products and services under the contract meet established performance standards and AQLs. Maintaining, and providing for audit, quality control records and reports and all records associated with the investigation and complaint resolutions. Appointing a single quality control point-of-contact to act as a central recipient of communication from the COR or CO.

3.2 Government Responsibility

3.2.1 Contracting Officer (CO)

The CO is responsible for administering and monitoring contract compliance, contract administration, and cost control and for resolving any differences between the observations documented by COR and the Contractor’s performance. The CO may delegate various day-to-day contract administration duties to a Contract Specialist and/or the COR for performance management and administrative actions such as invoice approval. Issuance of Contract Discrepancy Reports (Appendix B) is delegated to the COR. The CO shall approve any revisions to the QASP processes or standards.

3.2.2 Contracting Officer Representative (COR)

The COR, is designated in writing, by the CO. The COR will ensure that the QA function is properly executed, plays a key role in contract administration and performs contract surveillance and monitoring. Some key contract administration duties include, but are not limited to, performs surveillance as required by this QASP; make recommendations to the CO for issuance of Contract Discrepancy Reports (Appendix B) or letters of commendation and acceptance or rejection of completed work and for administrative actions based on unsatisfactory or non-performed work, and revisions or changes to the QASP; and assists the CO in identifying necessary contract modifications and preparing reports of Contractor performance and cost.

The COR may use the form(s) included in the Appendices to perform the inspection or other forms as approved by the CO. The Contractor overall guidance is also provided by FAR clause 52.246-4 Inspection of Services -Fixed Price in Section E Inspection and Acceptance of the contract.

3.2.3 Customers

USDA customers or regional offices that the Contractor provided support may be requested to assist the COR in conducting QA to provide information relating to the Contractor’s performance through Customer Feedback (Appendix C). Information gained from the Customer Feedback may be used in conjunction with other methods of observation (i.e., Inspection (Appendix D), Customer Complaint (Appendix F)) to rate the performance of the Contractor.

Section 4: Surveillance Methods to Perform Quality Assurance

4.1 Surveillance Methods

The surveillance methods used in the QA process are the Government’s tools to monitor the Contractor’s products and services. The best means of determining whether the Contractor has met all contract requirements is to inspect the Contractor’s service products and analyze the results. Further, documented inspection results are an effective tool in contract administration that can confirm the successful achievement of all performance requirements or highlight areas where defects exist, and improvements are necessary.

The required performance standards and/or quality levels are included in the PWS and in Appendix A, “Performance Requirements Summary.” Failure to meet the required service or performance level may result in Liquidated Damages or Nonperformance action.

To minimize the performance management burden, simplified surveillance methods shall be used by the Government to evaluate Contractor performance when appropriate. The primary methods of surveillance are:

1. 100% Inspection – with this method, performance is inspected/evaluated at each month. The COR or designated technical monitor (DTM) shall review the generated documentation and enter summary results into the Surveillance Activity Checklist available in Appendix G.

2. Periodic Inspection- this method evaluates tasks selected during a period of time. In this case, the COR will plan the dates which periodic inspections will take place. The COR or DTM ensure performance standards are implemented properly for each performance objective.

3. Customer Feedback from regional/state offices. The COR or technical representative will require feedback from state and/or regional offices during the ordering period. Customer feedback will be documented using the form available in Appendix C.

4.1.1. CUSTOMER FEEDBACK

The Contractor is expected to establish and maintain professional communication between its employees and customers. The primary objective of this communication is customer satisfaction. Customer satisfaction is the most significant external indicator of the success and effectiveness of all services provided and can be measured through customer complaints. Performance management drives the Contractor to be customer focused through initially and internally addressing customer complaints and investigating the issues and/or problems but the customer always has the option to communicate complaints to the COR or CO as opposed to the Contractor.

Customer complaints, to be considered valid, must set forth clearly and in writing the detailed nature of the complaint and must be signed and forwarded to the COR or Contracting Officer using the form in Appendix F. Customer feedback may also be obtained either from the results of formal customer satisfaction surveys or from random customer complaints.

4.1.2 ACCEPTABLE QUALITY LEVELS (AQL)

The Acceptable Quality Levels (AQLs) included in Appendix A titled Performance Requirements Summary Table, for Contractor performance are structured to allow the Contractor to manage how the work is performed while providing negative incentives for performance shortfalls. For certain critical activities, the desired performance level is established at 90 percent or higher. Other levels of performance are keyed to the relative importance of the task to the overall mission performance.

4.2 Analysis and Results

When the inspections (Appendix D) and “customer feedback,” (Appendix C) validations have been completed, the COR will perform an analysis of the Contractor’s performance. The purpose of the analysis is to ensure Government is receiving high-quality products and services from the Contractor. The COR will review the results, rate the Contractor’s compliance with the performance standards and AQLs, and characterize the Contractor’s overall performance.

Government shall use the monitoring methods cited herein to determine whether the performance standards/service levels/AQLs have been met. If the Contractor has not met the minimum requirements, it may be asked to develop a corrective action plan to show how and by what date it intends to bring performance up to the required levels. At the end of each month, the COR will prepare a written report for summarizing the overall results of the quality assurance surveillance of the Contractor’s performance. This written report, which includes the Contractor’s submitted monthly report and the completed Quality Assurance Monitoring Form (Attachment B), will become part of the QA documentation. It will enable the Government to demonstrate whether the Contractor is meeting the stated objectives and/or performance standards.

Section 5: Quality Assurance Performance, Reporting and Documentation The performance management feedback begins with the communication of expected outcomes. Performance standards are expressed in the PWS and are assessed using the performance monitoring techniques shown in Appendix A of the QASP.

5.1 Monitoring Performance

Government shall use the monitoring methods cited to determine whether the performance standard level have been met. If the Contractor has not met the minimum requirements, it may be asked to develop a corrective action plan to show how and by what date it intends to bring performance up to the required levels.

5.2 Reporting Performance

At the end of each month, the COR will prepare a written report for summarizing the overall results of the quality assurance surveillance of the Contractor’s performance. This written report, which includes the Contractor’s submitted monthly report and the completed Quality Assurance Monitoring form (Appendix E), will become part of the QA documentation. It will enable the Government to demonstrate whether the Contractor is meeting the stated objectives and/or performance standards.

5.3 Monitoring Forms

The Government QA surveillance, accomplished by the COR will be reported using the monitoring forms in Appendices B-G. The forms when completed, will document the Government’s assessment of the Contractor’s performance under the contract to ensure results are being achieved. The COR will retain a copy of all completed QA surveillance forms for review.

5.4 Monthly Reviews and Resolution

The COR may require the Contractor’s Program Manager, to meet with the CO and other Government personnel monthly to discuss performance evaluation. The agenda of the reviews may include:

· Monthly performance assessment.

· Issues and concerns from both parties.

· Projected outlook for upcoming months and progress expected against trends, including corrective action plan analysis.

· Recommendations for improved efficiency and/or effectiveness.

The COR must coordinate and communicate with the Contractor and the CO to resolve issues and concerns regarding the marginal or unacceptable performance. The COR and Contractor should jointly formulate courses of action. Decisions regarding changes to metrics, or service levels should be clearly documented.

Appendix A: Performance Requirement Summaries

meeting with the Agency staff within 2 business days of Contract Award.

The vendor’s disruption of services on Web-based platform exceeding the 4 hours during a normal business day will not exceed no more than 3 business days in a calendar year.

100% Inspection and customers feedback from regional offices.

The vendor’s disruption of services on Web-based platform exceeding the 4 hours during a normal business day will not exceed no more than 3 business days in a calendar year.

100% Inspection and customer feedback from regional offices.

Appendix B: Discrepancy Report

CONTRACT NUMBER: DATE:

4. DISCREPENCY OR PROBLEM (Describe in Detail. Include PWS references. Attach Continuation Sheet if Necessary)

5. SIGNATURE OF COR

8. CONTRACTOR RESPONSE AS TO CAUSE, EFFECT, CORRECTIVE ACTION AND ACTIONS TO PREVENT RECURRENCE (Attach Continuation Sheet if necessary. Cite applicable Contractor QC program procedures or new QC procedures)

11. GOVERNMENT EVALUATION (Acceptance, partial acceptance, or rejection. Attach Coordination Sheet if necessary)

12. GOVERNMENT ACTIONS (Cure notice, show cause, other)

CLOSE OUT

KTR NOTIFIED

COR

Appendix C: Customer Feedback Record

CUSTOMER FEEDBACK RECORD

DATE AND TIME OF COMPLAINT CONTRACT NUMBER/ OR PROJECT:

SOURCE OF COMPLAINT

ORGANIZATION

INDIVIDUAL

NATURE OF COMPLAINT

PWS REFERENCE

VALIDATION

ACTION TAKEN BY CONTRACTOR (Responsible officer):

RECEIVED AND VALIDATED BY

Determination: Complaint Valid Complaint Invalid

Appendix D: Inspection List

CONTRACT NUMBER: ___________________________________

SERVICE FUNCTION:

PWS SECTION:

Performance: Excellent (E), Satisfactory (S), Unsatisfactory (U), Not Applicable (N/A)

Overall Rating Of Inspection (E, S, U or N/A)

COR/Inspector Comments:

Contractor Signature: ___ Date: __

COR Signature: ____________________Date: _________________

Appendix E: Quality Assurance Monitoring

CONTRACT NUMBER: ______________________________________

SERVICE or STANDARD:

SURVEY PERIOD:

SURVEILLANCE METHOD (Check the applicable item):

r Random Sampling r 100% Inspection r Periodic Inspection r Customer Complaint LEVEL OF SURVEILLANCE (Check the applicable item):

ANALYSIS OF RESULTS:

Observed Service Provider Performance Measurement Rate: ______% Service Provider’s Performance (Check the applicable item):

r Meets Standards r Does Not Meet Standards Narrative of Performance During Survey Period:

Prepared By: _______________________________ Date: ______________

Appendix F: Customer Complaint Investigation

CONTRACT NUMBER: _________________________

SERVICE or STANDARD:

SURVEY PERIOD: ___________________

DATE COMPLAINT RECEIVED: ____________

SOURCE OF COMPLAINT: _________________________________________ (Name) _______________________________________________________________ (Organization) ___________________ (Phone No.) _________________________________ (Email Address)

NATURE OF COMPLAINT: ____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

RESULTS OF COMPLAINT INVESTIGATION: ____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

DATE SERVICE PROVIDER INFORMED OF COMPLAINT: _____________

CORRECTIVE ACTION TAKEN BY SERVICE PROVIDER: _________________________________________________________________________________________________________________________________________________________________________________________________________________________________

Received and Validated By: __________________________________________ Prepared By: _____________________________ Date: _________________

Appendix G: Surveillance Activity Checklist

Performance Requirement Performance Standard Method of Measurement

Performance Metrics Method of Surveillance Date Accomplished Compliance (Exceed, Met, Partial Met)