Attachment D - QASP.pdf

UNITED STATES DEPARTMENT OF EDUCATION

Office of the Chief Financial Officer

Contracts and Acquisitions Management

Attachment D

ANALYTIC AND TECHNICAL SUPPORT FOR ADVANCING

EDUCATION RESEARCH

Quality Assurance Surveillance Plan (QASP)

1. Purpose

This performance-based Quality Assurance Surveillance Plan (QASP) sets forth procedures and guidelines that the Department of Education (ED) will use in evaluating the technical performance of the

Contractor. A copy of this plan will be furnished to the Contractor so that the Contractor will be aware of the methods that the Government will employ in evaluating performance on this contract and address any concerns that the Contractor may have prior to initiating work.

2. Scope

The QASP is intended to accomplish the following:

1. Define the roles and responsibilities of participating Government officials

2. Define the key deliverables that will be assessed

3. Describe the rating elements and standards of performance against which the Contractor’s performance will be assessed for each key deliverable

4. Describe the process of quality assurance assessment

5. Provide copies of the quality assurance monitoring forms that will be used by the Government in documenting and evaluating the Contractor’s performance

Each of these purposes is discussed in detail below.

Roles and Responsibilities of Participating Government Officials

The following Government Officials will participate in assessing the quality of the Contractor’s performance. Their roles and responsibilities are described as follows:

Contracting Officer’s Representative (COR): The COR will be responsible for monitoring, assessing, recording, and reporting on the technical performance of the Contractor on a day-to-day basis. The COR will also be responsible for completing the Quality Assurance Monitoring

Forms (described in greater detail below), which will be used to document the inspection and evaluation of the Contractor’s work performance on the key deliverables. It is extremely important for the COR to establish and maintain a team-oriented line of communication with the

Contractor’s Project Director (PD) in order to perform monitoring functions. The COR, Contracting Officer (CO), Contract Specialist (CS), and PD must work together as a team to ensure that the required work is accomplished in an efficient and proper manner. Meetings and conference calls should be held on a regular basis in order to resolve serious problems. Less serious problems should be discussed and resolved on an as-needed basis.

Contract Specialist (CS): The CS will be responsible for the day-to-day monitoring of the

Contractor’s performance in the areas of contract compliance, contract administration, cost control and property control; reviewing the COR’s assessment of the Contractor’s performance;

and resolving all differences between the COR’s version of events and the Contractor’s version of events. The CS may call upon the expertise of other Government individuals as required.

Contracting Officer (CO): The CO ensures performance of all necessary actions for effective contracting, ensures compliance with the terms of the contract, and safeguards the interests of the

United States in the contractual relationship. It is the CO that assures the Contractor receives impartial, fair, and equitable treatment under the contract. The CO is ultimately responsible for the final determination of the adequacy of the Contractor’s performance. The CO’s procurement authorities include the following:

SOLE authority for any decisions which produce an increase or decrease in the scope of the contract;

SOLE authority for any actions subject to the Changes clause;

SOLE authority for any decision to be rendered under the Disputes clause;

SOLE authority to approve the substitution or replacement of the Project Director and other key personnel;

SOLE authority to monitor and enforce Department of Labor promulgated labor requirements;

Authority to arrange for and supervise Quality Assurance activities under this contract; and

Signatory authority for the issuance of all modifications to the contract.

Key Deliverables to be Assessed

Even though the Government will be monitoring the Contractor’s performance on a continuing basis through its COR, the volume of tasks to be performed by the Contractor makes technical inspections of every task impractical. Accordingly, the Department of Education will use a quality-assurance review process to monitor the Contractor’s performance across a set of rating elements for each of the following deliverables outlined in the Performance Work Statement (PWS):

Revised draft of research paper for IES review process (outlined in Activity 1)

Revised draft of issue brief for IES review process (outlined in Activity 1)

Revised draft of research synthesis for IES review process (outlined in Activity 1)

Final draft of compendium (outlined in Activity 1)

Final version of catalogue (outlined in Activity 2)

Catalogue support: Revisions and Updates (outlined in Activity 2)

Rating Elements and Standards of Performance

The Contractor’s performance will be evaluated by the COR in terms of a specific set of products and activities, according to three categories: “Superior,” “Acceptable,” and “Unacceptable.” The criteria for each of these performance levels are outlined below. All products produced by or activities performed by the Contractor shall, at a minimum, meet the level of “acceptable.”

Unacceptable: A level of performance that is not acceptable and which fails to meet the minimum standards of performance;

Acceptable: An acceptable level of performance that meets the minimum standards of performance; or

Superior: A level of performance that exceeds the minimum standards of performance.

In general, the work will be evaluated in terms of how well the requirements of the contract are satisfied, the extent to which the work performed follows the approach found in the Contractor’s technical proposal, clarity of documentation, and timeliness of scheduled task accomplishment. At the discretion of the CO/CS and COR, other government officials approved by the CO/CS may be asked to evaluate a particular deliverable or set of deliverables.

The following specific evaluation criteria will be used when evaluating the revised drafts of research papers, issue briefs, and research syntheses, and the final drafts of compendia:

1) Timeliness: When was the deliverable received? Was it received on the date specified?

Unacceptable: Standard not met—Deliverable was received after the due date.

Acceptable: Standard met—Deliverable was received between the period of two business days earlier than the due date and the due date.

Superior: Standard exceeded—Deliverable was received three or more business days before the due date.

Note: In the event of an excusable delay (as defined in FAR 52.249-14, Excusable Delays and

EDAR 3452.242-71, Notice to the Government of Delays), the Department and the Contractor shall work together to modify the contract with regard to the due dates of the deliverables. If such an event were to occur that would require a modification to the due dates of the deliverables, the

Contractor’s performance shall be measured by the date agreed upon in the modification.

2) Presentation: To what degree was the deliverable free of grammatical, typographical, and formatting errors?

Unacceptable: Standard not met—Substantial number of corrections was needed.

Acceptable: Standard met—Minor corrections were needed.

Superior: Standard exceeded—No corrections were needed.

3) Responsiveness to Technical Work Group/ED’s Comments and Suggestions: Did the contractor thoughtfully consider and acknowledge comments and suggestions from the TWG/ED throughout the drafting process?

Unacceptable: Standard not met—The Contractor had unsubstantiated disregard for reviewers’ suggestions throughout the drafting process and refused to acknowledge

ED’s/TWG’s edits/comments.

Acceptable: Standard met —The Contractor employed thoughtful consideration and acknowledgement of reviewers’ comments and suggestions for revisions throughout the drafting process.

Superior: Standard exceeded—The Contractor employed an efficient method for soliciting edits and comments from the TWG and ED that resulted in the draft being of high quality and prepared on time.

4) Coherence, accuracy, and clarity: Was the deliverable well organized, clear, accurate, and complete?

Unacceptable: Standard not met— The deliverable was not well organized, was incomplete, or was inaccurate; or the writing style and content were not clear.

Acceptable: Standard met— The deliverable was well organized, complete, and accurate;

and the writing style and content were clear.

Superior: Standard exceeded— The deliverable was of the highest quality; it was well organized, accurate, and complete; and the content and all aspects of the writing were clear.

The following specific evaluation criteria will be used when evaluating final versions of catalogues:

1) Timeliness: When was the deliverable received? Was it received on the date specified?

Unacceptable: Standard not met— The deliverable was received after the due date.

Acceptable: Standard met— The deliverable was received between the period of two business days earlier than the due date and the due date.

Superior: Standard exceeded— The deliverable was received three or more business days before the due date.

shall work together to modify the contract with regard to the due dates of the deliverables. If such

2) Responsiveness to ED’s Comments and Suggestions: Did the contractor thoughtfully consider and acknowledge comments and suggestions from ED throughout the drafting process?

Unacceptable: Standard not met—The Contractor had unsubstantiated disregard for ED’s suggestions throughout the drafting process and refused to acknowledge ED’s edits/comments.

Acceptable: Standard met —The Contractor employed thoughtful consideration and acknowledgement of ED’s comments and suggestions for revisions throughout the drafting process.

Superior: Standard exceeded—The Contractor employed an efficient method for soliciting edits and comments from ED that resulted in the catalogue being of high quality and being prepared on time.

3) Coherence, accuracy, and clarity: Was the deliverable well organized, accurate, and complete? To what degree was the deliverable free of typographical and formatting errors?

Unacceptable: Standard not met—The deliverable, including any requested list reports compiled from the catalogue, was not well organized, was incomplete, or was inaccurate.

Substantial number of typographical and formatting corrections were needed.

Acceptable: Standard met— The deliverable and any requested list reports compiled from the catalogue were well organized, complete, and accurate. Minor typographical and formatting corrections were needed.

Superior: Standard exceeded— The deliverable and any requested list reports compiled from the catalogue were of the highest quality. It was well organized, accurate, and complete. No typographical and formatting corrections were needed.

The following specific evaluation criteria will be used when evaluating final versions of catalogues

(continued):

4) User-friendly: Was the deliverable user-friendly? Can users easily update the catalogue, perform searches, and generate list reports of items?

Unacceptable: Standard not met— The deliverable was not user-friendly. There was either no interface or an ineffective interface. Users had difficulty adding or editing items, performing searches, or generating lists of items.

Acceptable: Standard met— The deliverable was user-friendly and fully compatible with standard IES software. Users could amend the catalogue as well as perform searches of it and generate reports from it. The user guide provided enough detail to perform basic tasks.

Superior: Standard exceeded— The catalogue was of the highest quality, allowing for complex searches and professional-quality reports. The structure of the catalogue allowed it to be easily linked to other catalogues or documents (e.g. indices). The user guide provided enough detail to perform basic tasks and more advanced tasks.

The following specific evaluation criteria will be used when evaluating catalogue support (revisions and updates):

1) Timeliness: When was the deliverable received? Were updates made on schedule and within the specified timeframe? Were revisions made within the specified timeframe?

Unacceptable: Standard not met— The deliverable was received after the due date.

Acceptable: Standard met— The deliverable was received between the period of two business days earlier than the due date and the due date.

Superior: Standard exceeded— The deliverable was received three or more business days before the due date.

shall work together to modify the contract with regard to the due dates of the deliverables. If such acknowledge comments and suggestions from ED throughout the updating or revision process?

Unacceptable: Standard not met—The Contractor had unsubstantiated disregard for ED’s suggestions throughout the updating or revision process and refused to acknowledge ED’s edits/comments.

Acceptable: Standard met —The Contractor employed thoughtful consideration and acknowledgement of ED’s comments and suggestions for updates or revisions throughout the process and integrated the changes.

Superior: Standard exceeded—The Contractor employed an efficient method for soliciting edits and comments from ED that resulted in the catalogue being of high quality and updated and revised on time.

Unacceptable: Standard not met—The deliverable, including any requested list reports compiled from the catalogue, was not well organized, was incomplete, or was inaccurate.

Substantial number of typographical and formatting corrections were needed.

Acceptable: Standard met— The deliverable and any requested list reports compiled from the catalogue were well organized, complete, and accurate. Minor typographical and formatting corrections were needed.

Superior: Standard exceeded— The deliverable and any requested list reports compiled from the catalogue were of the highest quality. It was well organized, accurate, and complete. No typographical and formatting corrections were needed.

The following specific evaluation criteria will be used when evaluating catalogue support (revisions and updates) (continued):

searches, and generate lists of items?

Unacceptable: Standard not met— The deliverable was not user-friendly. There was either no interface or an ineffective interface. Users had difficulty adding or editing items, performing searches, or generating lists of items.

Acceptable: Standard met— The deliverable was user-friendly and fully compatible with standard IES software. Users could amend the catalogue as well as perform searches of it and generate reports from it. The user guide provided enough detail to perform basic tasks.

Superior: Standard exceeded— The catalogue was of the highest quality, allowing for complex searches and professional-quality reports. The structure of the catalogue allowed it to be easily linked to other catalogues or documents (e.g. indices). The user guide was updated and provided enough detail to perform basic tasks and more advanced tasks.

Process of Quality Assurance Assessment

The Contractor’s performance will be evaluated by the COR using the appropriate key deliverable evaluation forms (Attachment A). These forms will serve to document and evaluate the Contractor’s performance for each of the key deliverables under this contract. Each form will be completed by the

COR, and the rating element scores will be averaged to generate the final evaluation score for that key deliverable. The form, when completed, will document the COR’s understanding of the Contractor requirements, what was actually completed, and the impact or consequences of what was not completed.

The COR must substantiate all tasks that are judged to be indicative of “superior” or “unacceptable” performance.

The COR will forward copies of all completed QASP monitoring forms to the CO/CS for review. The

COR will then forward the monitoring forms to the Contractor. The Contractor is required to respond in writing to any QASP monitoring form(s) indicating unacceptable performance within five working days after receipt of the form(s).

The CO/CS will review each QASP monitoring form prepared by the COR. When appropriate, the

CO/CS may investigate the event further to determine if all the facts and circumstances surrounding the event were considered in the COR opinions outlined in the forms. The CO/CS will discuss every event receiving a substandard rating with the Contractor to assure that corrective action is promptly initiated.

Attachment A

ANALYTIC AND TECHNICAL SUPPORT FOR ADVANCING EDUCATION RESEARCH

Quality Assurance Monitoring Form

Revised Drafts of Research Papers, Issue Briefs, and Research Syntheses for IES Review Process, and the

Final Drafts of Compendia

Deliverable: Task Number:

Reviewer: Date(s):

Relevant Rating Elements: Average Overall Rating:

1) Timeliness: When was the deliverable received?

___ After the due date (Unacceptable, 2 points)

___ Between the period of two business days earlier than the due date and the due date

(Acceptable, 5 points)

___ Three or more business days before the due date (Superior, 8 points)

2) Presentation: To what degree was the deliverable free of grammatical, typographical, and formatting errors?

1 2 3 4 5 6 7 8 9 or

Unacceptable Acceptable Superior Not applicable

3) Responsiveness to Technical Work Group/ED’s Comments and Suggestions: Did the contractor thoughtfully consider and acknowledge comments and suggestions from the TWG/ED throughout the drafting process?

1 2 3 4 5 6 7 8 9 or

4) Coherence, accuracy, and clarity: Was the deliverable well organized, clear, accurate, and complete?

Discussion of Contractor’s performance during survey period (required for unacceptable or superior performance ratings):

Final Version of Catalogue acknowledge comments and suggestions from ED throughout the drafting process?

Catalogue Support: Revisions and Updates acknowledge comments and suggestions from ED throughout the updating or revision process?