Attachment F - Task 7 NSSC QASP.pdf

Attachment F- Task 7 NCCS QASP

Quality Assurance Surveillance Plan

National Assessment of Educational Progress

NAEP Support and Service Center

Introduction

This performance-based Quality Assurance Surveillance Plan (QASP) Evaluation Plan has been developed in accordance with Federal Acquisition Regulation (FAR)

12.208 and pursuant to the requirements identified in the Statement of Objective

(SOO). The QASP sets forth procedures and guidelines that the Department of Education

(ED) will use in evaluating the technical performance of the contractor. A copy of this plan will be forwarded to the contractor so that the contractor shall be aware of the methods that the Government will employ in evaluating performance on this contract, and to address any concerns that the contractor may have prior to initiating work.

Purpose of the QASP

The QASP is intended to accomplish the following:

• Define the roles and responsibilities of participating Government officials and outside experts.

• Define the key deliverables that will be assessed.

• Describe the rating elements and standards of performance against which the contractor’s performance will be assessed for each key deliverable.

• Describe the process of quality assurance assessment; and

• Provide copies of the quality assurance monitoring forms that will be used by the

Government in documenting and evaluating the contractor’s performance.

Roles and Responsibilities of Participating Government Officials and Experts

The following Government officials and/or experts will participate in assessing the quality of the contractor’s performance. Their roles and responsibilities are described as follows:

The Contracting Officer’s Representative (COR) is responsible for monitoring, assessing, recording, and reporting on the technical performance of the contractor on a day-to-day basis. The COR also is responsible for completing the Quality Assurance Monitoring

Forms (QAMF) (described below and provided in Exhibits A - D), which will be used to document the inspection and evaluation of the contractor’s work performance.

Additional ED staff with technical knowledge and experience related to support and service to states and districts about the NAEP Program may provide input and review deliverables. However, ED reserves the right to have only the COR assess each deliverable.

The Contract Specialist (CS) is responsible for the day-to-day monitoring of the contractor’s performance in the areas of contract compliance, contract administration, price control, property control; reviewing the COR’s assessment of the contractor’s performance; and resolving all differences between the COR’s version and the contractor’s version. The CS may call upon the expertise of other Government individuals as required.

The Contracting Officer (CO) has the following procurement authorities:

• Sole authority for any decisions that produce an increase or decrease in the contract’s scope.

• Sole authority for any decision to be rendered under the “Disputes” clause.

• Sole authority to approve the substitution or replacement of the project manager or other key personnel.

• Sole authority to monitor and enforce Department of Labor promulgated labor requirements.

• Authority to arrange for and supervise quality assurance activities under this contract.

• Sole authority to approve the contractor’s quality control program.

• Signatory authority for the issuance of all modifications to the contract.

Annual Past Performance Evaluation

The Contractor’s overall performance shall be documented and measured through the annual past performance evaluation through the Contractor Performance

Assessment Reporting System (CPARS). Performance will be evaluated in the areas of:

• Quality of Product or Service

• Schedule

• Price Control (if applicable)

• Business Relations

• Management of Key Personnel

• Utilization of Small Business

Quality Assurance Review Process

Even though the Government, through its COR, will monitor the contractor’s performance on a continuing basis, the volume of tasks performed by the contractor makes technical inspection of every task and step impractical. Therefore, ED will use an annual quality assurance review process to monitor the contractor’s performance under the award. Specifically, the COR will assess the contractor’s performance on the following key deliverables:

1. Annual NAEP Report Card Submission

2. Orientation for New Coordinators – Subtask 7.1.1

3. NAEP Report Card Pre-Release Workshops – Subtask 7.1.2.1

4. Assessment Operations Training Workshops – Subtask 7.1.2.2

5. Professional Development Workshops – Subtask 7.1.2.3

6. Continuous Training to Coordinators – Subtask 7.2

7. Provide Technical Support to States and Districts – Subtask 7.3

8. Provide Administration Support Packages and NAEP Products – Subtask

7.3.6

The contractor’s performance will be evaluated annually by the COR using the following adjectival ratings: Excellent, Very Good, Satisfactory, Marginal, and Unsatisfactory.

Descriptions of these adjectival ratings are provided below:

CPARS Adjectival

Rating

Description

Exceptional Performance meets contractual requirements and exceeds many to the

Government's benefit. The element being assessed was accomplished with few minor problems for which corrective actions taken by the contractor were highly effective.

Very Good Very Good: Performance meets contractual requirements and exceeds some to the Government's benefit. The element being assessed was accomplished with some minor problems for which corrective actions taken by the contractor were effective.

Satisfactory Satisfactory: Performance meets contractual requirements. The element being assessed contains some minor problems for which corrective actions taken by the contractor appear or were satisfactory.

Marginal Marginal: Performance does not meet some contractual requirements.

The element being assessed reflects a serious problem for which the contractor has not yet identified corrective actions.

Unsatisfactory Unsatisfactory: Performance does not meet most contractual requirements and recovery is not likely in a timely manner. The element being assessed contains a serious problem(s) for which the contractor's corrective actions appear or were ineffective.

Each key task/subtask shall be separately evaluated using the above scale.

The contractor shall revise any key deliverable determined to be “unacceptable” at no additional price to the Government. That is, the contractor shall not charge the

Government for the additional work to revise unacceptable products, such as reports that do not meet NCES Statistical Standards.

Each year ED will evaluate the contractor's performance by assessing the performance on key deliverables within the task. The work will be evaluated in terms of how well the contract requirements are satisfied; the extent to which the work performed follows the approach found in the contractor’s technical proposal; quality and clarity of documentation; and timeliness of key deliverables.

While quality is closely tied to the performance standards for activities and /or deliverable content, timeliness and price are also important considerations in the assessment of contractor performance.

In the event of an excusable delay (as defined in FAR 52.249-14, Excusable Delays and

EDAR 3452.242-71, Notice to the Government of Delays), the Department and the contractor shall work together to modify the contract with regard to the due dates of the deliverables. If such an event were to occur that would require a modification to the due dates of the deliverables, the contractor's performance shall be measured by the date agreed upon in the modification.

Below are the evaluation periods (Dates may vary based on actual date of award, but will align with the period of performance):

1. Base Year – November 1, 2024 – October 31, 2025

2. Option Period I – November 1, 2025 – October 31, 2026

3. Option Period II – November 1, 2026 – October 31, 2027

4. Option Period III – November 1, 2027 – October 31, 2028

5. Option Period IV – November 1, 2028 – October 31, 2029

Key Task/Subtask #1: – Annual NAEP Report Card

Performance Indicators: The Annual NAEP Report Card is the culmination of NAEP activities from item development to the operational administration of the assessment to students. Findings from the NAEP assessment are made available to the public through the release of the Nations Report Card. NCES is required to release findings from the mathematics and reading report cards six months after the end of the data collection; 12-

15 months are the average release times for all other NAEP reports. The contractors shall, as an Alliance, ensure the established deadlines are made for release of all NAEP reports in the defined contract period. Details of NAEP report card requirements for each contract year time period are included in Chart 1 below. The contractors shall provide a weekly report with issues/problems..

Primary Surveillance Method: CORs review of Nations Report Card.

Evaluation Criteria:

Acceptable Performance

1. Materials prepared by the contractor (i.e., draft and final publications) are of high quality, require no more than minimal revisions, and are consistent with ED’s guidance on content, formatting and publishing requirements.

2. Materials prepared by the contractor are complete and delivered on time to the

COR.

Key Task/Subtask #2: (Subtask 7.1.1) – Orientation for New Coordinators

Performance Indicators: The contractor shall hold in-person orientations for new

Coordinators to provide them with a comprehensive overview of their position expectations and resources available. The purpose of these orientation workshops is to give new Coordinators the information, skills and resources they need to perform their jobs as well as giving them the opportunity to meet and begin to network among their colleagues. The contractor shall develop a plan for orientation to outline the approach including agenda, materials and resources used for the orientation.

Primary Surveillance Method: The COR reviews the deliverables and the summary evaluations of the workshops.

Evaluation Criteria:

Acceptable Performance

1. Materials prepared by the contractor (i.e., draft and final publications) are of high quality, require no more than minimal revisions, and are consistent with ED’s guidance on content, formatting and publishing requirements.

2. Materials prepared by the contractor are complete and delivered on time to the

COR.

Key Task/Subtask #3: (Subtask 7.1.2.1) – NAEP Report Card Pre-Release

Workshops

Performance Indicators: The contractor shall hold a workshop prior to the release of the state and district level NAEP data. The purpose of this type of workshop is to give state and district Coordinators the opportunity to learn more about the assessment(s), to review their state/district NAEP results prior to the official U.S. Department of Education’s release, to prepare for their states/districts own release activities and to help states/districts plan to conduct their own analysis of their data using NAEP data and tools. The contractor shall develop a plan for the pre-release workshop to outline the approach including agenda, materials and resources used for the workshop.

Primary Surveillance Method: The COR reviews the deliverables and the summary evaluations of the workshop.

Evaluation Criteria:

Acceptable Performance

1. Materials prepared by the contractor (i.e., draft and final publications) are of high quality, require no more than minimal revisions, and are consistent with ED’s guidance on content, formatting and publishing requirements.

2. Materials prepared by the contractor are complete and delivered on time to the

COR.

Key Task/Subtask #4: (Subtask 7.1.2.2) – Assessment Operations Training

Workshop

Performance Indicators: The contractor shall conduct the operations training to prepare state and district Coordinators on the variety of tasks associated with the administrative implementation of a large state/district year of NAEP. The contractor shall prepare all presentations, resources and activities that will be used during the workshops.

Coordinators need to understand and be able to complete required communications to districts and schools, understand the systems that will be used for communication with participating districts and schools and understand their overall role in the preparation for a NAEP state/district year. The contractor shall develop a plan for the assessment operations training workshop to outline the approach including agenda, materials and resources used for the workshop.

Primary Surveillance Method: The COR reviews the deliverables and the summary evaluations of the workshop.

Evaluation Criteria

Acceptable Performance

1. Materials prepared by the contractor (i.e., draft and final publications) are of high quality, require no more than minimal revisions, and are consistent with ED’s guidance on content, formatting and publishing requirements.

2. Materials prepared by the contractor are complete and delivered on time to the

COR.

Key Task/Subtask #5: (Subtask 7.1.2.3) – Professional Development Workshops

Performance Indicators: The contractor shall provide the Coordinators with training to further enhance their understanding of large-scale assessment and provide them the skills and resources to successful analyze, interpret and report NAEP data. The focus of these workshops is much broader than the operational assessment trainings. The contractor shall develop a plan for the assessment operations training workshop to outline the approach including agenda, materials and resources used for the workshop.

Primary Surveillance Method: The COR reviews the deliverables and the summary evaluations of the workshop.

Evaluation Criteria

Acceptable Performance

1. Materials prepared by the contractor (i.e., draft and final publications) are of high quality, require no more than minimal revisions, and are consistent with ED’s guidance on content, formatting and publishing requirements.

2. Materials prepared by the contractor are complete and delivered on time to the

COR.

Key Task/Subtask #6: (Subtask 7.3.1) – Provide Technical Support

Performance Indicators: The contractor shall provide data analysis support to the NAEP

Coordinator using the currently available NAEP on-line tools in order to build the

Coordinators’ capacity to analyze data. The contractor shall provide the training and support of the Coordinators in the use of tools such as the NAEP Data Explorer (NDE), the NAEP Questions Tool (NQT), and the State/District Profile tool. In addition to the

NAEP tools, the contractor shall support the Coordinators’ use of non-NAEP on-line tools.

Primary Surveillance Method: The COR reviews the deliverables consisting of the support desk reports created monthly.

Evaluation Criteria

Acceptable Performance

1. Materials prepared by the contractor are complete and delivered on time to the

COR.

Key Task/Subtask #7: (Subtask 7.3.6) – Provide Administrative Support Packages and NAEP Products

Performance Indicators: The contractor shall develop ready-made materials (i.e. support package) to assist the NAEP Coordinator and ensure quality and consistency across states and districts. These materials shall include presentation templates, table shells, state and district report templates, and informational CDs and DVDs, and form letters regarding

NAEP. The contractor shall provide training and support to States and TUDA Districts in using the reporting software as well as the analysis tools, listed previously. The contractor shall make these templates and materials available to the Coordinators via their internal website.

Primary Surveillance Method: The COR reviews the deliverables.

Acceptable Performance

Evaluation Criteria

1. Materials prepared by the contractor (i.e., draft and final publications) are of high quality, require no more than minimal revisions, and are consistent with ED’s guidance on content, formatting and publishing requirements.

2. Materials prepared by the contractor are complete and delivered on time to the

COR.

Process of Quality Assurance Assessment Evaluation

In the event of an excusable delay (as defined in FAR 52.249-14), the Department and the contractor shall work together to modify the contract regarding the due dates of deliverables. If an event occurs that would require modifying deliverable due dates, the contractor’s performance, where applicable in this QASP, shall be measured by the date agreed upon in the modification.

In narrative form, the COR will substantiate all adjectival ratings other than Satisfactory.

Performance at the “Satisfactory” level is expected from the contractor. The COR will forward a report of average scores to the CS.