Attachment_B_-_NPSAS-16_Quality_Assurance_Surveillance_Plan_(2).docx

UNITED STATES DEPARTMENT OF EDUCATION

QUALITY ASSURANCE SURVEILLANCE PLAN

2015-16 National Postsecondary Student Aid Study (NPSAS 2016) & Postsecondary Longitudinal Studies

(ED-IES-13-R-0011)

Introduction

This Quality Assurance Surveillance Plan (QASP) sets forth procedures and guidelines that the Department of Education will use to monitor performance of the Contractor under this contract. A copy of this plan will be furnished to the contractor so that the contractor shall be aware of the methods that the Government will employ in monitoring performance on this project and address any concerns that the contractor may have prior to initiating work.

Purpose of the QASP

The QASP is intended to accomplish the following:

· Define the roles and responsibilities of participating Government officials and outside experts; and

· Describe the process of quality assurance surveillance.

Each of these purposes is discussed in detail below.

Roles and Responsibilities of Participating Government Officials

The following Government Officials will participate in assessing the quality of the contractor’s performance. Their roles and responsibilities are described as follows:

Contracting Officer’s Representative (COR). The COR will be responsible for monitoring, assessing, recording, and reporting on the technical performance of the contractor on a day-to-day basis. The COR will also be responsible for inspecting all deliverables submitted under this contract and providing a recommendation regarding the acceptability of the deliverable to the Contracting Officer.

It is essential that the COR establish and maintain a team-oriented line of communication with the Contractor’s Project Director (PD) in order to perform monitoring functions. The COR, CO, and PD must work together as a team to ensure that required work is accomplished in an efficient and proper manner. Meetings should be held on a regular basis in order to resolve serious problems. Less serious problems should be discussed and resolved on an impromptu basis.

The Contracting Officer (CO) will have overall responsibility for overseeing the contractor’s performance. The Contract Specialist will be responsible for the day-to-day monitoring of the contractor’s performance in the areas of compliance, administration, cost control and property control; reviewing the COR’s assessment of the contractor’s performance; and resolving all differences between the COR’s version and the contractor’s version. The CO may call upon the expertise of other Government individuals as required.

Quality Assurance Surveillance Process

Even though the Government, through the COR, will be monitoring the contractor’s performance on a continuing basis, the volume of tasks performed by the contractor makes technical inspections of every task and step impractical. Accordingly, the Department of Education will use a quality-assurance review process to monitor the contractor’s performance under this contract through the contract deliverables, specifically through a review of the timeliness of submission and the acceptability of the content of each deliverable.

The COR will inspect each deliverable and recommend acceptance of the deliverable to the Contracting Officer. In the event the COR has a concern regarding the timeliness or quality of the content of a deliverable, the contractor and the Contracting Officer shall be notified.

The Contractor’s performance shall be documented and measured through the annual past performance evaluation through the Contractor Performance Assessment Reporting System (CPARS). Performance will be evaluated in the areas of:

· Quality of product or service

· Study design

· Data collection

· Data products and publication

· Schedule

· Cost control

· Business relations

· Management of key personnel

· Utilization of small business

The following rating scale will be used to evaluate each element using a 9-point scale:

· Unacceptable (1-3): A level of performance that is not acceptable and which fails to meet the minimum standards of performance;

· Acceptable (4-6): An acceptable level of performance that meets the minimum standards of performance; or

· Superior (7-9): A level of performance that exceeds the minimum standards of performance.

Table 1 provides describes the standards of performance for each rating level.

ATTACHMENT B

Table 1. Rating elements and standards of performance

Element Rating scale

Unacceptable (1-3) Acceptable (4-6) Superior (7-9)

Meets all the requirements for the "Acceptable" rating AND uses new and innovative techniques identified in the literature to improve or enhance the study and/or reduce costs while maintaining quality.

· does not use tracing procedures to locate respondents or uses unethical practices;

· uses call center operations (if applicable) with conversion practices that does not respect the non-respondents decision to withdraw consent (e.g. complaint rates >1%);

· yields low response rates within the contract's specifications or does not yield high enough response rates to provide reliable estimates with low bias; and

· does not provide adequate documentation of practices (and/or unclear and poorly written OMB packages).

The process for generating the deliverable uses generally accepted industry practices when during data collection and:

· uses tracing procedures to locate sample members while respecting privacy and the confidentiality of the data;

· uses call center operations (if applicable) that balances the conversion of non-respondents with the respect for withdrawing consent;

· yields a high enough response rate to provide reliable estimates within the contract's specifications; and

· provides adequate documentation of practices and writes clear and concise OMB packages.

Meets all the requirements for the "Acceptable" rating AND uses techniques to reduce error and/or bias during data collection (e.g. responsive or adaptive design, daily updates to inform call center and contacting procedures, measures that inform decisions to reduce bias and/or reduce costs in the field, etc).

Table 1. Rating elements and standards of performance, continued

Rating scale

Unacceptable

Acceptable (4-6) Superior (7-9)

The Contractor had unsubstantiated disregard for reviewers’ suggestions throughout the process and refused to acknowledge the Department’s edits/comments.

For publications and written work products, the deliverable(s) was not well organized, was incomplete, or was inaccurate; or the writing style and content were not clear.

The deliverable(s) required moderate revisions to meet IES/NCES standards for research design, analytic procedures, and presentation of results.

The Contractor employed thoughtful consideration and acknowledgement of reviewers’ comments and suggestions for revisions throughout the drafting process.

For publications and written work products, the deliverable(s) was well organized, complete, and accurate; and the writing style and content were clear.

The deliverable(s) demonstrated adherence to IES/NCES standards for research throughout, with minor revisions or no revisions needed.

The Contractor employed an efficient method for soliciting edits and comments from the Department that resulted in the product being of high quality and prepared on time.

For publications and written work products, the deliverable(s) was of the highest quality; it was well organized, accurate, and complete; and the content and all aspects of the writing were clear.

Table 1. Rating elements and standards of performance, continued

Rating scale

Unacceptable

Acceptable (4-6) Superior (7-9)

· Ignoring and hiding problems,

· Maintaining irregular and infrequent communications with Government officials, and

· Does not adapt to changing Government requirements.

The contractor maintains a history of collaboration including:

· Evidence of strong corporate commitment and oversight as evidenced by a clear and effective approach to problem identification and resolution,

· Is responsive to the needs of the Government keeping lines of communication open and timely, and

· Adapts to changing Government requirements.

Meets all the requirements for the "Acceptable" rating AND the contractor:

· Provides recommendations to improve data collections and products,

· Is active in providing feedback when Government requirements change,

· Is proactive in identifying was to improve quality and/or use resources ore efficiently.

No controls are present in resource allocations or time spent in creating deliverables.

Mismatches are present with the expertise of staff/contractors and responsibilities yielding substandard deliverables.

Is not proactive in controlling quality of the project and related deliverables.

Efficiently uses staff with clearly defined lines of authority and responsibilities, and staff assignments match their expertise.

Demonstrates effective control of tasks and resource allocations that will ensure the timely completion of tasks/deliverables within the proposed budget

The development of deliverables uses a sound and feasible approach to controlling the quality of project work and products.

Meets all the requirements for the "Acceptable" rating AND effectively manages subcontractors, including:

· monitoring subcontract costs,

· undertake efforts taken to ensure early identification of subcontract problems and the timely application of corporate resources to preclude subcontract problems from impacting overall prime contractor performance, and

· integrate subcontractors as an integral part of the contractor’s team.

2015-2016 National Postsecondary Student Aid Study (NPSAS:16) & Postsecondary Longitudinal Studies

Annual Quality Assurance Monitoring Form

Reviewer:

Contract Year:

Comments:

Comments:

Comments:

Comments:

Comments:

Comments:

Comments:

Comments:

Average Overall Rating:

image1.png