Attachment H - Task 1 POC QASP.pdf

Attachment H

Quality Assurance Surveillance Plan Evaluation Plan (QASP) National Assessment of Educational Progress NAEP Program and Operations Coordination

Introduction This performance-based Quality Assurance Surveillance Plan (QASP) Evaluation Plan has been developed in accordance with Federal Acquisition Regulation (FAR) 16.401(e)(3) and pursuant to the requirements identified in the Statement of Objectives (SOO). The QASP sets forth procedures and guidelines that the Department of Education (ED) will use in evaluating the technical performance of the contractor. A copy of this plan will be forwarded to the contractor so that the contractor shall be aware of the methods that the Government will employ in evaluating performance on this contract, and to address any concerns that the contractor may have prior to initiating work.

Purpose of the QASP

The QASP is intended to accomplish the following:

• Define the roles and responsibilities of participating Government officials and outside experts.

• Define the key deliverables that will be assessed.

• Describe the rating elements and standards of performance against which the contractor’s performance will be assessed for each key deliverable.

• Describe the process of quality assurance assessment; and

• Provide copies of the quality assurance monitoring forms that will be used by the

Government in documenting and evaluating the contractor’s performance.

Roles and Responsibilities of Participating Government Officials and Experts The following Government officials and/or experts will participate in assessing the quality of the contractor’s performance. Their roles and responsibilities are described as follows:

The Contracting Officer’s Representative (COR) is responsible for monitoring, assessing, recording, and reporting on the technical performance of the contractor on a day-to-day basis. The COR also is responsible for documentation, inspection and evaluation of the contractor’s work performance.

Additional ED staff with technical knowledge and experience related to the program and operations coordination of the NAEP Program may provide input and review deliverables. However, ED reserves the right to have only the COR assess each deliverable.

The Contract Specialist (CS) is responsible for the day-to-day monitoring of the contractor’s performance in the areas of contract compliance, contract administration, cost control, property control; reviewing the COR’s assessment of the contractor’s performance; and resolving all differences between the COR’s version and the contractor’s version. The CS may call upon the expertise of other Government individuals as required.

The Contracting Officer (CO) has the following procurement authorities:

• Sole authority for any decisions that produce an increase or decrease in the contract’s scope.

• Sole authority for any decision to be rendered under the “Disputes” clause.

• Sole authority to approve the substitution or replacement of the project manager or other key personnel.

• Sole authority to monitor and enforce Department of Labor promulgated labor requirements.

• Authority to arrange for and supervise quality assurance activities under this contract.

• Sole authority to approve the contractor’s quality control program.

• Signatory authority for the issuance of all modifications to the contract.

Annual Past Performance Evaluation The Contractor’s overall performance shall be documented and measured through the annual past performance evaluation through the Contractor Performance Assessment Reporting System (CPARS). Performance will be evaluated in the areas of:

• Quality of Product or Service

• Schedule

• Cost Control (if applicable)

• Business Relations

• Management of Key Personnel

• Utilization of Small Business

Quality Assurance Review Process Even though the Government, through its COR, will monitor the contractor’s performance on a continuing basis, the volume of tasks performed by the contractor makes technical inspection of every task and step impractical. Therefore, ED will use an annual quality assurance review process to monitor the contractor’s performance under the award. Specifically, the COR will assess the contractor’s performance on the following key deliverables:

1. Annual NAEP Report Card Submission

2. Update NAEP Strategy Plan – Subtask 1.1.1

3. Convene Executive Steering Committee meetings – Subtask 1.1.2

4. Convene Mid-year NAEP Operations meeting – Subtask 1.2.3

5. Convene Fall NAEP Design Summit – Subtask 1.2.3

6. Convene monthly project planning meetings between Coalition and NCES CORS

– Subtask 1.2.3

7. Prepare response memorandum on emerging matters of action – Subtask 1.2.4

8. Convene R&D planning meeting – Subtask 1.1.3

9. Coordinate NAEP special studies – Subtask 1.1.3.3

10. Develop annual technical report on R&D activities – Subtask 1.1.3

11. Coordinate ad hoc studies – Subtask 1.1.3.5

12. Coordinate Development of the NAEP Information Collection Requests (ICR) for

OMB Approval – Subtask 1.3

The contractor’s performance will be evaluated annually by the COR using the following adjectival ratings: Outstanding, Superior, Acceptable and Unacceptable. Descriptions of these adjectival ratings are provided below:

CPARS Adjectival

Rating Description

Exceptional Performance meets contractual requirements and exceeds many to the Government's benefit. The element being assessed was accomplished with few minor problems for which corrective actions taken by the contractor were highly effective.

Very Good Very Good: Performance meets contractual requirements and exceeds some to the Government's benefit. The element being assessed was accomplished with some minor problems for which corrective actions taken by the contractor were effective.

Satisfactory Satisfactory: Performance meets contractual requirements. The element being assessed contains some minor problems for which corrective actions taken by the contractor appear or were satisfactory.

Marginal Marginal: Performance does not meet some contractual requirements.

The element being assessed reflects a serious problem for which the contractor has not yet identified corrective actions.

Unsatisfactory Unsatisfactory: Performance does not meet most contractual requirements and recovery is not likely in a timely manner. The element being assessed contains a serious problem(s) for which the contractor's corrective actions appear or were ineffective.

Each key task/subtask shall be separately evaluated using the above scale.

ED will rate performance separately for each Key Outcome.

The contractor shall revise any key deliverable determined to be “unacceptable” at no additional cost to the Government. That is, the contractor shall not charge the Government for the additional work to revise unacceptable products, such as reports that do not meet NCES Statistical Standards.

Each year ED will evaluate the contractor's performance by assessing the performance on key deliverables within the task. The work will be evaluated in terms of how well the contract requirements are satisfied; the extent to which the work performed follows the approach found in the contractor’s technical proposal; quality and clarity of documentation; and timeliness of key deliverables.

While quality is closely tied to the performance standards for activities and /or deliverable content, timeliness and cost are also important considerations in the assessment of contractor performance.

In the event of an excusable delay (as defined in FAR 52.249-14, Excusable Delays and EDAR 3452.242-71, Notice to the Government of Delays), the Department and the contractor shall work together to modify the contract with regard to the due dates of the deliverables. If such an event were to occur that would require a modification to the due dates of the deliverables, the contractor's performance shall be measured by the date agreed upon in the modification.

Below are the evaluation periods (Dates may vary based on actual date of award, but will align with the period of performance):

1. Base Year – Base Year – October 15, 2024 – September 30, 2025

2. Option Period I – October 1, 2025 – September 30, 2026

3. Option Period II – October 1, 2026 – September 30, 2027

4. Option Period III – October 1, 2027 – September 30, 2028

5. Option Period IV – October 1, 2028 – September 30, 2029

The specific task/subtask, performance indicators, and primary surveillance method are described below:

Key Task/Subtask #1: – Annual NAEP Report Card

Performance Indicators: The Annual NAEP Report Card is the culmination of NAEP activities from item development to the operational administration of the assessment to students. Findings from the NAEP assessment are made available to the public through the release of the Nations Report Card. NCES is required to release findings from the mathematics and reading report cards six months after the end of the data collection; 12- 15 months are the average release times for all other NAEP reports. Each activity completed by the Coalition is critical to ensure NCES’s ability to successfully meet its obligations to report on student performance, culminating with the release of the NAEP reports. The contractors shall, as a Coalition, ensure the established deadlines are made for release of all NAEP reports in the defined contract period. Details of NAEP report card requirements for each contract year time period are included in Chart 1 below. The contractors shall provide weekly reports with issues/problems and requirements stated in the reporting task.

Primary Surveillance Method: CORs review of Nations Report Card.

Evaluation Criteria:

Acceptable Performance

1. Materials prepared by the contractor (i.e., draft and final publications) are of high quality, require no more than minimal revisions, and are consistent with ED’s guidance on content, formatting and publishing requirements.

2. Materials prepared by the contractor are complete and delivered on time to the

COR.

Key Task/Subtask #2: (Subtask 1.1.1.) – Update NAEP Strategic Plan

Performance Indicators: The contractor shall support NCES in developing updates to the NAEP Strategic Plan by providing recommendations to effectively manage the impact of changes in the large-scale assessment industry. The Plan shall cover a five-year period from the start of the contract year. The contractor shall also revise the plan, as necessary.

Primary Surveillance Method: The COR reviews the deliverables and the recommendations memo on revising the NAEP Strategic Plan.

Acceptable Performance

1. Materials prepared by the contractor (i.e., draft and final publications) are of high quality, require no more than minimal revisions, and are consistent with ED’s guidance on content, formatting and publishing requirements.

Key Task/Subtask #3: (Subtask 1.1.2) – Convene Executive Steering Committee meetings

Performance Indicators: The contractor shall establish or maintain the NAEP Program Executive Steering Committee, comprising Corporate Officers from all Coalition contractors, the NCES NAEP Associate Commissioner, and Assessment division managers, to report out on the status of strategic planning and implementation, and present the rationale for any proposed new directions and actions that are aligned with NAEP’s mission and vision. The contractor shall document all expert panel deliberations and resultant recommendations, monitor progress on action items identified during NAEP

Coalition meetings, and make recommendations to NCES on approaches to operationalize the recommendations of the panel.

Primary Surveillance Method: The COR reviews the deliverables including agenda and meeting minutes from the Committee meetings.

1. Materials prepared by the contractor (i.e., draft and final publications) are of high quality, require no more than minimal revisions, and are consistent with ED’s guidance on content, formatting and publishing requirements.

Key Task/Subtask #4: (Subtask 1.2.3) – Convene Mid-year NAEP Operations meeting

Performance Indicators: The contractor shall coordinate the mid-year NAEP operations meeting among Coalition contractors and NCES to make decisions about the final assessment activities for the upcoming assessment year, including preparing meeting agenda, coordinating presentations, and preparing meeting minutes.

Primary Surveillance Method: The COR evaluates alignment between agenda and meeting discussions and reviews the deliverables.

Evaluation Criteria

Acceptable Performance

1. Materials prepared by the contractor (i.e., draft and final publications) are of high quality, require no more than minimal revisions, and are consistent with ED’s guidance on content, formatting and publishing requirements.

Key Task/Subtask #5: (Subtask 1.2.3) – Convene Fall NAEP Design Summit

Performance Indicators: The contractor shall coordinate the fall NAEP Design Summit meeting among Coalition contractors and NCES to make decisions about the final assessment activities for the upcoming assessment year, including preparing meeting agenda, coordinating presentations, and preparing meeting minutes.

Primary Surveillance Method: The COR evaluates alignment between agenda and meeting discussion and reviews the deliverables.

Evaluation Criteria

Acceptable Performance

1. Materials prepared by the contractor (i.e., draft and final publications) are of high quality, require no more than minimal revisions, and are consistent with ED’s guidance on content, formatting and publishing requirements.

Key Task/Subtask #6: (Subtask 1.2.3) – Convene monthly project planning meetings between Coalition and NCES CORS

Performance Indicators: The contractor shall arrange meetings between Coalition task leads, relevant team members, and NCES CORS to coordinate activities, discuss status and milestones of upcoming projects, and recommend actions. The contractor shall provide meeting agendas and minutes, and coordinate and deliver any resultant recommendation memoranda from Coalition to the COR.

Primary Surveillance Method: The COR reviews the deliverables.

Evaluation Criteria

Acceptable Performance

1. Materials prepared by the contractor (i.e., draft and final publications) are of high quality, require no more than minimal revisions, and are consistent with ED’s guidance on content, formatting and publishing requirements.

Key Task/Subtask #7: (Subtask 1.2.4) – Prepare response memorandum on emerging matters of action

Performance Indicators: The contractor shall coordinate activities requiring input from multiple contractors in response to a request from NCES. For each identified request, the contractor shall submit a recommendations memorandum, and other material such as an analysis file or table, as required. For requests identified as urgent, the contractor shall deliver a response within 24 hours of receiving the request.

Primary Surveillance Method: The COR reviews the deliverables.

Evaluation Criteria

1. Materials prepared by the contractor are complete and delivered on time to the

Key Task/Subtask #8: (Subtask 1.1.3) – Convene R&D planning meeting

Performance Indicators: The contractor shall convene an R&D panel meeting for the purpose of obtaining feedback and recommendations from experts on research directions and objectives that are consistent with NAEP’s overall strategic goals. The panel’s purpose is also to help NCES set the annual R&D agenda. The R& D agenda shall identify areas where there are research gaps and identify new areas of research based on emerging issues or program goals.

Primary Surveillance Method: The COR evaluates the alignment between R&D planning meeting agenda and discussion and reviews the deliverables.

Acceptable Performance

Evaluation Criteria

1. Materials prepared by the contractor (i.e., draft and final publications) are of high quality, require no more than minimal revisions, and are consistent with ED’s guidance on content, formatting and publishing requirements.

Key Task/Subtask #9: (Subtask 1.1.3.3) – Coordinate NAEP special studies

Performance Indicators: The contractor shall coordinate effective collaboration among the multiple contractors involved in Coalition special studies. Such coordination shall include developing recommendations memos and proposals for conducting special studies, suggesting timelines for COR approval, then coordinating the integration of timelines in NAEP’s Master Schedule, periodically informing NCES on the progress of special studies, and delivering the final study reports to NCES in accordance with the agreed upon timeline.

Primary Surveillance Method: The COR reviews the deliverables.

Acceptable Performance

Evaluation Criteria

1. Materials prepared by the contractor (i.e., draft and final publications) are of high quality, require no more than minimal revisions, and are consistent with ED’s guidance on content, formatting and publishing requirements.

Key Task/Subtask #10: (Subtask 1.1.3) – Develop annual technical report on R&D activities.

Performance Indicators: The contractor shall prepare a technical report on the current activities of all active R&D projects.

Primary Surveillance Method: The COR reviews the deliverable.

Acceptable Performance

Evaluation Criteria

1. Materials prepared by the contractor (i.e., draft and final publications) are of high quality, require no more than minimal revisions, and are consistent with ED’s guidance on content, formatting and publishing requirements.

Key Task/Subtask #11: (Subtask 1.1.3.5) – Coordinate ad hoc studies

Performance Indicators: The contractor shall coordinate small to medium sized R&D projects carried out by coalition and non-coalition contractors. These projects shall address topics that must be researched quickly, or that arise over the course of the assessment. Such coordination shall include developing recommendations memos and proposals for conducting projects, suggesting timelines for COR approval, then coordinating the integration of timelines in NAEP’s Master Schedule, periodically informing NCES on the progress of special studies, and delivering the final study report to NCES in accordance with the agreed upon timeline.

Primary Surveillance Method: The COR reviews the deliverables.

Acceptable Performance

Evaluation Criteria

3. Materials prepared by the contractor (i.e., draft and final publications) are of high quality, require no more than minimal revisions, and are consistent with ED’s guidance on content, formatting and publishing requirements.

4. Materials prepared by the contractor are complete and delivered on time to the

Key Task/Subtask #12: (Subtask 1.3) – Coordinate Development of information collection requests for OMB approval

Performance Indicators: The contractor shall develop information collection requests for OMB approval of NAEP information collections. The contractor shall develop one request for a NAEP generic clearance, and a request for information collection in each of the five assessment years. The contractor shall also develop up to 10 requests per year for submittal under the NCES generic clearance. The contractor shall work collaboratively with multiple contractors, and consultants, where necessary, to develop required documentation for inclusion in the ICR, and follow the most recent OMB and NCES OMB liaison instructions for completing the ICRs.

Primary Surveillance Method: The COR reviews the deliverables and also evaluates feedback from NCES OMB liaison on quality of requests.

Acceptable Performance

Evaluation Criteria

1. Materials prepared by the contractor (i.e., draft and final publications) are of high quality, require no more than minimal revisions, and are consistent with ED’s guidance on content, formatting and publishing requirements.

Process of Quality Assurance Assessment

In the event of an excusable delay (as defined in FAR 52.249-14), the Department and the contractor shall work together to modify the contract regarding the due dates of deliverables. If an event occurs that would require modifying deliverable due dates, the contractor’s performance, where applicable in this QASP, shall be measured by the date agreed upon in the modification.

In narrative form, the COR will substantiate all adjectival ratings other than Satisfactory.

Performance at the “Satisfactory” level is expected from the contractor. The COR will forward a report of average scores to the CS.