Attachments_A-G_-_ED-NAG-17-R-0003.pdf

The following additional attachments are provided with the RFP:

Attachment B: Quality Assurance Surveillance Plan (QASP)

Attachment C: Instructions to Offerors

Attachment D: Technical Evaluation Criteria

Attachment E: Past Performance Form

Attachment F: Sample Budget Submission

Attachment G: Small Business Subcontracting Plan

ATTACHMENT A: STATEMENT OF WORK

TECHNICAL SUPPORT IN PSYCHOMETRICS, ASSESSMENT

DEVELOPMENT, AND PREPAREDNESS FOR

POSTSECONDARY ENDEAVORS

April 28, 2017

Solicitation # ED-NAG-17-R-0003 Attachment A: Statement of Work

Contents

I. Background

II. Scope of Work

Task 1: Project Management

a. Kick-off meeting

b. Weekly meetings with COR

c. Other remote meetings (up to 10 per year)

d. Monthly progress reports

Task 2: Research to Inform Governing Board Activities

a. Research studies (1 per year)

Task 3: Attendance at Governing Board and Other Meetings and Preparation of Meeting Minutes

a. Governing Board quarterly meetings

b. Other meetings and conferences

Task 4: Expert Consultant Services

Task 5: Ad hoc Technical and Administrative Services

III. Management and Staffing Plan

IV. Contract Period of Performance

V. Schedule of Project Deliverables

VI. List of Attachments

I. Background

The National Assessment Governing Board (hereafter referred to as the Governing Board or the Board) has a requirement for technical support to assist the Governing Board staff in implementing specific activities to achieve the goals of the Strategic Vision and for other ongoing work requirements based on the Governing Board’s legislative mandates specified in Public Law 107-279.

For over four decades, the National Assessment of Educational Progress (NAEP) has provided information on the achievement and performance of students in the United States. NAEP, also known as “The Nation’s Report Card,” is the most comprehensive and only continuing, valid source of information on what U.S. students know and can do in various subject areas. Tracking student performance has permitted the nation to assess U.S. students’ academic progress over time. The National Center for Education Statistics (NCES), within the Department of Education, has responsibility for administering NAEP. The Governing Board is responsible for setting policy for NAEP. Public Law 107-279— signed on November 5, 2002, contains the authorizing legislation for NAEP as well as for the Governing Board.

The following are among the specific responsibilities assigned to the Governing Board in the legislation:

• Select the subject areas to be assessed;

• Develop appropriate student achievement levels;

• Develop assessment objectives and test specifications that produce an assessment that is valid and reliable, and are based on relevant widely accepted professional standards;

• Develop a process for review of the assessment which includes the active participation of teachers, curriculum specialists, local school administrators, parents, and concerned members of the public;

• Design the methodology of the assessment to ensure that assessment items are valid and reliable, in consultation with appropriate technical experts in measurement and assessment, content and subject matter, sampling, and other technical experts who engage in large scale surveys;

• Measure student academic achievement in grades 4, 8, and 12 in the authorized academic subjects;

• Develop guidelines for reporting and disseminating results;

• Develop standards and procedures for regional and national comparisons;

• Take appropriate actions needed to improve the form, content, use, and reporting of results; and

• Plan and execute the initial public release of NAEP reports.

The Governing Board carries out many of its legislative responsibilities through the work of its standing committees to include the Assessment Development Committee (ADC), the Committee on Standards, Design and Methodology (COSDAM), and the Reporting and Dissemination (R&D) Committee. There is a need for technical support to facilitate some of the work of ADC and COSDAM.1

During its November 2016 quarterly meeting, the Governing Board adopted a Strategic Vision to guide its work through 2020.2 The Strategic Vision contains several technical activities to include planning and design of statistical linking studies; researching how NAEP is used by various audiences; validating the intended uses and appropriate interpretations of NAEP;

developing approaches to updating assessment frameworks while maintaining trends;

reconfiguring the Long-Term Trend assessments; learning best practices from other assessment programs; and exploring uses of NAEP as a measure of preparedness for postsecondary education and careers.

On November 17, 2016, the National Academies of Sciences, Engineering, and Medicine released the final report of their evaluation of the NAEP achievement levels for mathematics and reading.3 The NAEP legislation specifies that the Commissioner of NCES is to use findings from the evaluation to decide whether the achievement levels should continue to be used on a “trial basis” or whether that designation can be removed. In addition, the final report included conclusions and recommendations that have implications for future Governing Board achievement levels-setting activities. Public Law 107-279 specifies that the Governing Board must prepare a formal response to the evaluation.

The Governing Board’s response to the evaluation (included in the March 2017 COSDAM materials4 refers to several follow up activities to be pursued over the next few years. Technical support will be needed to assist the Governing Board with implementing some of these activities that include the following:

• Providing input to inform Board policy revision on setting achievement levels for NAEP;

• Conducting research on the relationship between NAEP achievement levels and concurrent or future performance on measures external to NAEP;

• Conducting research on how the achievement levels are used by various audiences;

• Preparing validity arguments to support the intended uses and interpretations of the achievement levels;

1 Committee materials and minutes from previous Board meetings can be found at:

https://www.nagb.org/what-we-do/quarterly-board-meeting-materials.html 2 See https://www.nagb.org/content/nagb/assets/documents/newsroom/press-releases/2016/nagb-strategic-vision.pdf.

3 https://www.nap.edu/catalog/23409/evaluation-of-the-achievement-levels-for-mathematics-and-reading-on-the-national-assessment-of-educational-progress.

4 See https://www.nagb.org/what-we-do/quarterly-board-meeting-materials/2017-march.html https://www.nagb.org/what-we-do/quarterly-board-meeting-materials.html https://www.nagb.org/content/nagb/assets/documents/newsroom/press-releases/2016/nagb-strategic-vision.pdf https://www.nagb.org/content/nagb/assets/documents/newsroom/press-releases/2016/nagb-strategic-vision.pdf https://www.nap.edu/catalog/23409/evaluation-of-the-achievement-levels-for-mathematics-and-reading-on-the-national-assessment-of-educational-progress https://www.nap.edu/catalog/23409/evaluation-of-the-achievement-levels-for-mathematics-and-reading-on-the-national-assessment-of-educational-progress https://www.nagb.org/what-we-do/quarterly-board-meeting-materials/2017-march.html

• Conducting research and producing guidance on inferences best made with achievement levels and those best made with scale scores.

For more than a decade, the Governing Board has been conducting research in pursuit of NAEP serving as an indicator of academic preparedness for postsecondary endeavors. Starting with the 2013 grade 12 results, the NAEP reading and mathematics assessments have been used to estimate the percentage of students academically prepared for college. The Governing Board undertook considerable research to explore the feasibility of using NAEP as an indicator of academic preparedness for job training, but thus far this use has not been supported (a summary of that work can be found at: https://nagb.gov/content/nagb/assets/documents/what-we-do/preparedness-research/nagb-job-training-report-102215.pdf.

Additional validation research is still underway related to academic preparedness for college.

Further, the Strategic Vision calls for re-conceptualizing preparedness for postsecondary endeavors beyond college (i.e., for the workforce). Technical support will be needed to assist the Governing Board staff with work related to NAEP as an indicator of preparedness for postsecondary endeavors.

Services are required on an ongoing basis to meet the technical aspects of the Board’s policymaking responsibilities as detailed below.

II. Scope of Work

Activities in the base year and each option year will include the following requirements:

Task 1: Project Management

a. Kick-off meeting

Within 10 working days following the award of the contract, the contractor’s key staff will meet with the Contracting Officer (CO), Contracting Officer’s Representative (COR), and other key Governing Board staff in Washington, D.C. to review key elements of the contract requirements, discuss work plans, and review the proposed project schedule and Quality Assurance Surveillance Plan (QASP). Contract communications and key staff roles and responsibilities will also be discussed. A summary of the kick-off meeting minutes and a final QASP shall be submitted electronically to the CO with a copy to the COR, within one week of the meeting. Feedback shall be provided by the Governing Board within five working days, with the final deliverables submitted no later than five working days after Board feedback.

b. Weekly meetings with COR The contractor shall arrange weekly meetings (typically 1 hour) with the COR and other Governing Board staff (as appropriate) at a mutually convenient time, to occur by telephone and/or webinar. The contractor shall submit an agenda to the COR no later than one business day prior to each call and shall submit brief written minutes within two https://nagb.gov/content/nagb/assets/documents/what-we-do/preparedness-research/nagb-job-training-report-102215.pdf https://nagb.gov/content/nagb/assets/documents/what-we-do/preparedness-research/nagb-job-training-report-102215.pdf business days after each call. The minutes are intended to capture the major discussion and decision points from each call.

c. Other remote meetings (up to 10 per year) The contractor shall participate in phone calls and/or webinars (typically 1 hour) on an ad hoc basis with Board members, Board staff, NCES staff, NAEP contractors, or other relevant parties (as determined by the COR) to discuss technical activities. The contractor shall submit brief written minutes within two business days after each call. The minutes are intended to capture the major discussion and decision points from each call.

d. Monthly progress reports The contractor shall submit monthly progress reports, summarizing contractual activities and financial expenditures for the duration of the project. Monthly progress reports shall describe major tasks and accomplishments, problems and suggested solutions; significant activities and events; decisions that may be needed from the Board; and plans for the next reporting period. These progress reports shall be submitted by the fifteenth day of each month (or on a specified date agreed upon by the CO and the contractor). The CO, through the COR, will respond within five calendar days to each monthly progress report with any technical direction or re-direction that may be warranted in order to carry out Board policy and direction.

In addition to monthly progress reports, the contractor shall provide monthly reports on financial expenditures through invoice submissions. Each invoice shall include a summary of the overall project costs broken down by the task areas described in this section. This will include task/contract funding; tasks costs by current reporting period;

cumulative costs for each task; and a balance of funds for the overall contract.

Following this summary, the contractor shall provide, by task, a breakdown of labor and other direct costs. Detail for other direct costs shall be summarized by line item description and cost. Budgeted versus actual costs for each line item will be provided.

Sufficient detail for each cost, such as lodging and per diem, supplies, etc. shall be provided to substantiate billed costs.

e. Plans for secure materials The Governing Board is especially sensitive to the fact that some of this work may involve access to secure NAEP items, data, or other materials. The contractor shall ensure that all secure materials are securely controlled as well as confidentiality maintained by employees at all times. Security is a serious concern and it is the contractor’s responsibility to guarantee security for NAEP materials once in the contractor’s control.

It is a felony to disclose confidential NAEP materials.

If a task requires the use of secure NAEP materials, the contractor must specify a plan in writing to describe how security procedures will be employed and monitored at all times for the duration of the task. General procedures for protecting secure materials shall be discussed in the offeror’s proposal.

NCES requires that any person(s) who will be reviewing or using secure data or materials sign data security compliance forms during the course of this contract.5 These forms will be supplied by the Governing Board COR for the contractor to complete. The contractor shall forward all completed forms to the COR, who will forward a copy to NCES.

Task 2: Research to Inform Governing Board Activities

a. Research studies (1 per year)

The contractor shall perform one research study per year with a duration of approximately 6-12 months. For each research study, the contractor shall develop a research proposal in line with the Governing Board’s goals; prepare and submit a study plan; conduct approved research activities including the preparation of data collection materials and Office of Management and Budget (OMB) and Institutional Review Board (IRB) packages where applicable; prepare an interim presentation and briefing; prepare a study report of approximately 50-75 pages (with an executive summary intended for general audience); and prepare and deliver a presentation on the report at a Governing Board meeting. Initial drafts of all documents shall be reviewed by the Governing Board COR, and the final report will include two rounds of review. Study plans and final reports may also be reviewed by Board members and distributed for public comment.

The topic for the research study in the first contract year shall be an analysis and synthesis of how NAEP data in general and achievement levels in particular are used (both appropriately and inappropriately) by various audiences, including but not limited to: policymakers (federal, state, and local); media; researchers; businesses; parents; and the general public. There are four primary goals to gaining a better understanding of how NAEP results and achievement levels are used: 1) inform a future Board policy statement on intended uses and appropriate interpretations of NAEP; 2) gain understanding of which uses have been most impactful; 3) inform the development of validity arguments for the intended uses and interpretations of NAEP achievement levels, in particular; and

4) inform future reporting and dissemination efforts for increasing appropriate and impactful uses and decreasing inappropriate uses of NAEP results and achievement levels.

The research study on how NAEP data are used is not attempt to produce an exhaustive list or database of every mention of NAEP over the past 50 years; rather the focus should be on identifying and synthesizing the types or categories of uses made by different

5 See http://nces.ed.gov/statprog/instruct_access_faq.asp http://nces.ed.gov/statprog/instruct_access_faq.asp audiences and providing evidence to consider when making a judgment about whether each use is appropriate or inappropriate. The final research proposal shall incorporate feedback from the COR and Governing Board members to include COSDAM members.

Initial plans shall be proposed to demonstrate relevant qualifications and experience with similar work. Budget assumptions shall be clearly specified.

Potential areas of research studies to be conducted in contract option years 2 and 3 include but are not limited to reviews of NAEP achievement level descriptions and/or cut scores; and content alignment studies of NAEP and other assessments. Expected methodologies to be employed for research studies include but are not limited to:

quantitative methods such as statistical procedures and IRT scaling analyses; qualitative methods such as focus groups, interviews, and case studies; and mixed methods such as anchoring studies and content alignment studies. Research studies are not anticipated to be secondary analyses of existing NAEP data, but projects involving original data analyses which may require access to restricted use datasets. Priorities for research studies will be discussed at the contract kickoff meeting and will be planned in collaboration with the COR. As requirements for the research studies become more defined, a revised scope of work will be issued if assumptions have changed from what was proposed and budgeted for a research study.

Should any of the contractor’s proposed plans for research studies require additional data collections (e.g., focus groups or interviews), Office of Management and Budget (OMB) clearances will be necessary. If such is the case, the contractor is responsible for preparing such clearance packages and tracking the process through to completion.

Current OMB regulations require two Federal Register announcements (costs to be assumed by the Governing Board), and approximately 180 days to execute and secure the necessary clearances. The COR will assist in this process by providing copies of the regulations and facilitating a relationship with the OMB liaison in the Department of Education.

b. Technical memos (4 per year) The contractor shall produce four technical memos per year, each of which is anticipated to take approximately 2-3 months to complete. The purpose of each technical memo is to provide technical advice or recommendations on topics related to the Governing Board’s work. For each technical memo, the contractor shall produce a one page concept document describing the goals and approach of the technical memo; prepare a technical memo of approximately 15-20 pages (with a short executive summary for general audiences); and prepare and deliver a final presentation at a Governing Board meeting.

Initial drafts of all documents shall be reviewed by the Governing Board COR and Board members.

Potential topics for technical memos include but are not limited to: ideas for using findings from previous NAEP statistical linking studies to add meaning to NAEP scale scores and achievement levels; suggestions for additional studies to address goals in the Strategic Vision; evaluation of alternative options for exploring the feasibility of a new approach to using NAEP as an indicator of preparedness for postsecondary endeavors;

and summaries of considerations for producing a validity framework to support the intended uses and interpretations of the NAEP achievement levels. Priorities for technical memos will be discussed at the contract kickoff meeting and will be planned in collaboration with the COR. The offeror shall describe how they would approach a technical memo on at least one of the potential topics mentioned above. Budget assumptions for a technical memo shall be clearly specified.

c. Literature reviews and/or syntheses of best practices (4 per year) The contractor shall produce four literature reviews and/or syntheses of best practices per year, each of which is anticipated to take 2-3 months to complete. The purpose of a literature review is to locate, critique, and synthesize existing research and/or practices on topics related to the Governing Board’s work. For each literature review and/or synthesis of best practices, the contractor shall produce a study plan of approximately 2-3 pages describing the proposed approach to locating, critiquing, and synthesizing the research;

prepare a report of approximately 25-30 pages (with a short executive summary for general audiences); and prepare and deliver a presentation of the report at a Governing Board meeting. Initial drafts of all documents shall be reviewed by the Governing Board COR and may also be reviewed by Board members.

Potential topics for literature reviews and syntheses of best practices include but are not limited to procedures used for updating content frameworks; the role of achievement level descriptions in standard setting; and how NAEP Long-Term Trend data have been used. Priorities for technical memos will be discussed at the contract kickoff meeting and will be planned in collaboration with the COR. The offeror shall describe how they would approach a literature review and/or synthesis of best practices on at least one of the potential topics mentioned above. Budget assumptions shall be clearly specified.

For each quarterly Board meeting, the contractor shall prepare a written report of up to 3 pages describing the current status of specific research activities for each subtask, as requested by the COR.

Task 3: Attendance at Governing Board and Other Meetings and Preparation of Meeting Minutes

a. Governing Board quarterly meetings Two contractor staff shall attend each of the Governing Board’s quarterly meetings, and prepare meeting minutes for the plenary sessions (beginning with the November 2017

Board meeting). The schedule of Board meetings can be found at: https://nagb.gov/what-we-do/calendar.html#boardmeeting. For budgeting purposes, assume that meetings will take place in Washington, DC, if the meeting location has not yet been specified.

Within 7 working days after each quarterly Board meeting, the COR will provide the contractor with the attendee list, verbatim transcript, and audio recording of the meeting in order to assist in drafting minutes. The contractor will have two weeks from receipt of these materials to finish drafting the meeting minutes, which are expected to be approximately 25-30 pages in length. Examples of prior meeting minute summaries are posted on the website under meeting materials, available here:

https://www.nagb.gov/what-we-do/quarterly-board-meeting-materials.html.

b. Other meetings and conferences One staff member shall attend up to 4 other meetings and/or conferences on behalf of the Governing Board (as requested), and prepare meeting minutes. For budgeting purposes, assume that all meetings will be two full days and take place in Washington, DC.

Task 4: Expert Consultant Services The contractor shall identify consultants needed for specialized areas of expertise to assist in preparing commissioned papers and reports and/or serving on expert panels on areas within the Governing Board’s purview. Such work includes coordinating the process of acquiring their services in accordance with Federal and Department of Education guidelines for the acquisition of experts and consultants, overseeing the work performed under the contracts, and processing payments as directed by the Governing Board Contracting Officer.

Potential topics for expert papers and reports and/or panels include but are not limited to:

updating the Governing Board policy on setting achievement levels for NAEP, developing principles for reviewing NAEP achievement level descriptors and/or cut scores; exploring the feasibility of a new approach to using NAEP as a measure of preparedness for postsecondary education and careers; clarifying the intended uses and interpretations of NAEP; and determining how the content of NAEP frameworks can remain responsive and innovative.

For budgeting purposes, assume that up to 4 expert panel meetings of two days each shall occur in Washington, DC, with 8 expert consultants per meeting. Preparation for each meeting shall include: specifying the goals for the meeting; identifying expert consultants and contracting for their services; making travel arrangements and processing payments for the consultants;

developing a meeting agenda and advanced materials; facilitating the meeting; taking notes and producing meeting minutes. Meeting goals and outcomes will be established in close collaboration with the COR. In addition, up to 8 white papers per year may be requested.

Priorities for expert consulting services will be discussed at the contract kickoff meeting and will be planned in close collaboration with the COR. As requirements for the expert panel meetings https://nagb.gov/what-we-do/calendar.html#boardmeeting https://nagb.gov/what-we-do/calendar.html#boardmeeting https://www.nagb.gov/what-we-do/quarterly-board-meeting-materials.html and white papers become more defined, a revised scope of work will be issued if assumptions have changed from what was proposed and budgeted for expert consulting services. The offeror shall describe how they would approach expert consulting services on at least one of the potential topics mentioned above. Budget assumptions for a technical memo shall be clearly specified.

Task 5: Ad hoc Technical and Administrative Services The Governing Board has an ongoing requirement for ad hoc technical and administrative services to include the following:

a. Prepare PowerPoint presentations on technical topics;

b. Provide quick turnaround services for technical requests;

c. Provide ad hoc services in preparation for Board meetings;

d. Coordinate payments for expert services and substitute teachers for meetings attended by teachers in accordance with guidance issued by the Department of Education.

e. Provide ancillary services within the Governing Board’s legislative mandates as needed.

Requests issued under this task shall be coordinated by the CO with a specific scope issued as needed. For budgeting purposes, assume up to 500 hours for an administrative professional, up to 100 hours for the project director, and up to 200 hours for another technical staff member working on task 2, for each contract year, and a set amount of Other Direct Costs not to exceed $10,000 per year. The Other Direct Costs will not need to be detailed as the requests issued by the CO will specify the scope per activity with a projected level of effort.

III. Management and Staffing Plan

The offeror is invited to propose a management and staffing plan that maximizes the efficiency of the process and assures adherence to the schedule of activities and project budget. The Governing Board stipulates, however, that a single person be identified as Project Director to provide leadership and direction to the contractor’s project staff and serve as the contractor’s primary contact with the Governing Board management and technical staff and with Governing Board members. The Project Director must participate in all aspects of the project, attend all meetings—both internal and external, and serve as the primary contact with and source of information for the contract.

A chart depicting staff roles and responsibilities and time allocation by task area shall be provided, and percentage of time commitment available for the project shall also be clearly depicted.

All project staff (including subcontractors and consultants) shall have qualifications appropriate for the tasks they are to perform. If a subcontracting arrangement is proposed, the project director and leaders for task 2 and task 3 must be staff of the prime contractor, and the prime organization should possess expertise in all areas noted above. Additional information on qualifications and experience is provided in Attachment C (Instructions to Offerors).

IV. Contract Period of Performance

The contract period of performance is 12 months for the base year with 2 option years of 12 months each, if exercised. The period of performance shall not exceed 36 months.

V. Schedule of Project Deliverables

For each deliverable, an electronic version shall be submitted to the CO, copying the COR, for Board review before final submission. The Governing Board recommends that sufficient time be included in the schedule to consider feedback from the review of the draft in preparation of the final document. It is of paramount importance that the schedule for activities and deliverables is strictly adhered to and that the contractor allows sufficient time for Board review. All final reports have to be submitted in both Microsoft Word and PDF formats, compliant with publishing requirements of Section 508 for posting on the Governing Board website.

The following schedule provides a summary of key project deliverables and due dates.

Tasks and Deliverables Due Date

Task 1: Project Management Kick-off meeting summary and Final QASP Minutes from weekly calls with COR Minutes from other remote meetings Monthly progress reports

5 business days after the kick-off meeting 2 business days after call 2 business days after call By the 15th of each month*

Task 2: Research to Inform Governing Board Activities

Due dates for subtask 2a deliverables shall be proposed by the offeror Schedule to be negotiated with the COR when requests are made for the other subtasks

Task 3: Attendance at Governing Board and Other Meetings Written update for Board materials Minutes from Board plenary sessions

Minutes from other meetings/conferences

1 month prior to each Board meeting 10 business days after receipt of meeting transcripts 5 business days after meeting

Task 4: Expert Consultant Services Schedule to be negotiated with the CO and COR when requests are made

Task 5: Other Ad hoc and Quick Turnaround Requests

To be specified in work requests

*The date can be based on the contractor’s budget cycle, but it must be set for the same date each month.

The contractor shall consult with the COR before making any major technical decision concerning a deliverable product. To facilitate this communication, the contractor shall have communicate electronically to the COR who will communicate with the CO to make sure that the requirements are fully documented, and changed only by contract modification as needed.

VI. List of Attachments

The following additional attachments are provided with the RFP:

Attachment B: Quality Assurance Surveillance Plan (QASP) Attachment C: Instructions to Offerors Attachment D: Technical Evaluation Criteria Attachment E: Past Performance Form Attachment F: Sample Budget Submission Attachment G: Small Business Subcontracting Plan

Attachment B: Quality Assurance Surveillance Plan

ED-NAG-17-R-0003

TECHNICAL SUPPORT IN PSYCHOMETRICS, ASSESSMENT

DEVELOPMENT, AND PREPAREDNESS FOR

POSTSECONDARY ENDEAVORS

OVERVIEW

The Quality Assurance Surveillance Plan (QASP) documents a set of systematic quality assurance methods to be used in the administration of the National Assessment Governing Board’s contract resulting from this solicitation. The QASP provides quality metrics, describes performance standards, defines acceptable levels of quality, and details the steps to inspect performance to determine if the deliverables meet required quality standards. The purpose of the QASP is to ensure that the work is performed as required, and that products and deliverables are of high quality standards, statistically defensible, and adhere to project timelines. Invoiced costs must comply with approved contract budgets. Results from the monitoring will be documented at www.cpars.gov as past performance information utilized in reviewing contractor performance for federal contracts.

The QASP will be an integral component of the contract and will be updated as needed throughout the contract lifecycle. Offerors may propose changes to the QASP that strengthens project quality assurance in proposal submissions.

METHODS OF SURVEILLANCE

Several methods of surveillance shall be used by the government to monitor contract performance, as described below.

Regular Contract Monitoring and Reporting:

An internal contract monitoring plan shall be utilized by the Governing Board’s Contracting Officer’s Representative (COR) to monitor work performance for the duration of the contract.

The contract deliverables include weekly, monthly, and quarterly reports. The monthly report requires the contractor to provide technical and cost information both of which will be utilized by the CO and COR to monitor ongoing work plans and ensure that the contractor complies with the contract terms and conditions.

Periodic Meetings:

The contractor shall meet with the CO, COR, and other Governing Board staff in person or via conference call on a weekly basis. These meetings will provide opportunity to address contract work needs and discuss ongoing issues so that contract requirements are continuously discussed.

Contractor Performance Assessment Report System (CPARS):

The COR will document and evaluate the contractor’s performance for each key requirement in accordance with the adjectival rating system in the federal contractor performance reporting system at www.cpars.gov. In narrative form, the COR will substantiate all adjectival ratings other than Satisfactory. Performance at the “Satisfactory” level is the minimum standard expected from the contractor. The COR will forward copies of all completed ratings to the Contracting Officer, by close of business after completion of ratings. When appropriate, the CO http://www.cpars.gov/ may investigate the ratings to determine if all the facts and circumstances were considered in the opinions outlined on the rating forms. The CO will discuss every area receiving an “Unsatisfactory” rating with the contractor to ensure that corrective action is promptly initiated.

The contractor is required to respond in writing to any negative rating within five business days after receipt of the form. Each key task/subtask shall be separately evaluated.

Favorable ratings on QASP elements will result in correspondingly favorable past performance reporting at www.cpars.gov. Marginal and Unsatistfactory ratings on any elements of the QASP will result in a correspondingly unfavorable past performance reporting.

RATING DEFINITION

Exceptional Performance meets all contractual requirements and exceeds all contract expectations. The performance of the element or sub-element being assessed was accomplished with a few minor problems for which corrective action taken was highly effective. Favorable Past Performance Rating at www.cpars.gov.

Very Good

Performance meets all contract requirements and exceeds most expectations. The performance of the element or sub-element being assessed was accomplished with a few minor problems for which corrective action taken was effective.

Favorable Past Performance Rating at www.cpars.gov.

Satisfactory

Performance meets contract requirements. The performance of the element or sub-element being assessed contains some minor problems and the corrective action taken appears satisfactory. Favorable Past Performance Rating at www.cpars.gov. The Government reserves option to cancel or terminate work as specified in the contract

Marginal

Performance does not meet basic contract requirements. The performance of the element or sub-element being assessed reflects a serious problem for which a corrective action has not been identifed or proposed actions appear to be only marginally effective or not fully implemented. Unfavorable Past Performance Rating at www.cpars.gov. The Government reserves option to cancel or terminate work, as specified in the contract.

Unsatisfactory

Performance does not meet most contract requirement and recovery is not likely in a timely manner. The performance of the element of sub-element being assessed contains a serious problem for which a corrective action is ineffective as determined by the Contracting Officer. Unfavorable Past Performance Rating at www.cpars.gov. The Government reserves the option to cancel or terminate work as specified in the contract.

http://www.cpars.gov/

QASP ELEMENTS AND RATING DEFINITIONS

• QUALITY DETERMINANT – Describes what is to be accomplished during the contract period of performance. The headings reflect the major deliverables from the contract and were developed by the Governing Board as part of the contracting process to assist in the development of the QASP. Offerors may modify the elements to improve the QASP.

• PERFORMANCE STANDARD – Describes the expected overall quality of deliverables.

• ACCEPTABLE QUALITY LEVEL – Provides key aspects of the performance standards to be met.

• MONITORING METHOD – Identifies the methods used for monitoring performance

QUALITY

DETERMINANTS

QUALITY METRICS CONTRACT

RATING

Desired Outcome & Objective What do we want to accomplish for this contract and what is the desired end result for this project?

Performance Standard What are the standards for completeness, reliability, accuracy, timeliness, quality, and/or cost?

Acceptable Quality Level (AQ) How is acceptance defined and how much error will be accepted?

Monitoring Method How will we determine attainment of quality metrics? Monitoring will take place via the Contracting Officer’s Representative (COR) and Contracting Officer (CO).

Exceptional, Very Good, Satisfactory, Marginal, Unsatisfactory

GENERAL PROJECT MANAGEMENT

SCHEDULE AND

REPORTING

The Project Milestone Schedule includes all of the contractual deliverables and agreed-to dates.

Project milestones, including draft and final deliverables are submitted on time and in agreed-upon formats, to include final report compliance with Section 508 for web posting.

Proactive planning for submitting deliverables will be undertaken to account for expected contingencies in the performance work plan.

Weekly updates are submitted by email or on phone call on time.

Monthly operations and financial reports are submitted no later than the 15th of the following month.

Quarterly reports for Governing Board meeting s are submitted per the Project Milestone Schedule.

The COR will compare the expected dates of delivery per the Project Milestone Schedule with actual delivery dates. The COR will also compare agreed-upon deliverable formats for completeness.

Monthly and quaterly reports will be reviewed by the COR and validated with acceptance by the

CO.

Teleconferences to discuss potential issues, challenges, and cost overruns take place on time, with summaries provided as specified in the project deleiverable scehdule.

COSTS Invoices are submitted according to contracted budgets and timelines, in the agreed-upon formats with supporting details.

Monthly invoices shall be submitted by 15th of the month following the work performance period.

The COR will compare the expected dates of delivery per the Project Milestone Schedule with actual dates of delivery.

QUALITY

DETERMINANTS

QUALITY METRICS CONTRACT

RATING

Desired Outcome & Objective What do we want to accomplish for this contract and what is the desired end result for this project?

Performance Standard What are the standards for completeness, reliability, accuracy, timeliness, quality, and/or cost?

Acceptable Quality Level (AQ) How is acceptance defined and how much error will be accepted?

Monitoring Method How will we determine attainment of quality metrics? Monitoring will take place via the Contracting Officer’s Representative (COR) and Contracting Officer (CO).

Exceptional, Very Good, Satisfactory, Marginal, Unsatisfactory

The contractor is expected to be in compliance with all federal cost accounting standards and regulations.

Cost savings are to be obtained for negotiation of other direct costs to ensure cost reasonableness and to remain in the not to exceed costs, following the cost assumptions.

100% compliance with federal guidelines and U.S. Department of Education guidelines.

100% compliance with Governing Board expectations regarding invoiced costs with work performed.

The COR will review the invoice for accuracy and compliance with the contract budget and project milestones.

BUSINESS

RELATIONSHIP

Contractor staff are responsive to work requests, and work is performed as contracted, with details as specified in each contract task.

The contractor is responsive to work requests, is flexible in undertaking changed work requirements within scope, anticipates potential areas of concern, and develops contingency plans.

The CO, COR, and technical staff will monitor the contractor’s business relations on an ongoing basis and via annual assessments conducted at www.cpars.gov.

MANAGEMENT OF

PERSONNEL

Key staff performing work requests are responsive, and perform assigned duties effectively, as specified in each task request.

Contractor staff respond to work requests in a timely manner.

Changes in project priorities are made quickly and effectively. Only assigned key staff provide services as contracted.

COR and CO will review contractor staff work performance on an ongoing basis to assure that staff assignments are executed as detailed in contract tasks.

QUALITY

DETERMINANTS

QUALITY METRICS CONTRACT

RATING

Desired Outcome & Objective What do we want to accomplish for this contract and what is the desired end result for this project?

Performance Standard What are the standards for completeness, reliability, accuracy, timeliness, quality, and/or cost?

Acceptable Quality Level (AQ) How is acceptance defined and how much error will be accepted?

Monitoring Method How will we determine attainment of quality metrics? Monitoring will take place via the Contracting Officer’s Representative (COR) and Contracting Officer (CO).

Exceptional, Very Good, Satisfactory, Marginal, Unsatisfactory

TECHNICAL SCOPE OF WORK

RESEARCH STUDIES Study plans for data collection, analysis and reporting must adhere to technical rigor and quality standards in accordance with the guidelines provided by the COR for each study.

Research activities must comply with approved study plans. If OMB data collection is required, the clearance package must be comprehensive and submitted in a timely manner, with clarifications if needed provided promptly.

Research reports are thorough, error-free, of high quality, and are fully responsive to the statement of work and to staff and Board feedback.

Reports accurately describe all relevant analyses conducted.

Deliverables are well written, easily understood by a third party, and responsive to reporting expectations and requirements.

Up to two rounds of draft review on the study plan with additions, revisions, and/or modifications suggested by Board staff are acceptable.

Technical research adheres to statistical standards, comply with federal data collection methodologies, and accurately portray research findings.

OMB clearance packages must adhere to requirements to minimize respondent burden, and clearance packages must be well written and minimize the need for clarification questions.

Final report demonstrates 100% alignment with project milestone schedule, including documentation of changes or explanation of reasons for non-alignment.

Deliverables represent appropriate, technically sound and valid data analysis and interpretation.

Achievement of desired outcomes will be determined by:

(1) Quality of product(s) as determined by COR on the basis of technical rigor, analysis appropriateness and accuracy, methodological transparency, and clarity in communicating content;

(2) Relevance of elements of plans to producing interim and final deliverables in accordance with contract’s purpose and standards of quality.

(3) COR will review all collected data and summary reports of collected data to check for completeness of final data set.

(4) COR may review feedback from external reviews requested by the Governing Board, as additional verification of technical quality and understandability.

QUALITY

DETERMINANTS

QUALITY METRICS CONTRACT

RATING

Desired Outcome & Objective What do we want to accomplish for this contract and what is the desired end result for this project?

Performance Standard What are the standards for completeness, reliability, accuracy, timeliness, quality, and/or cost?

Acceptable Quality Level (AQ) How is acceptance defined and how much error will be accepted?

Monitoring Method How will we determine attainment of quality metrics? Monitoring will take place via the Contracting Officer’s Representative (COR) and Contracting Officer (CO).

Exceptional, Very Good, Satisfactory, Marginal, Unsatisfactory

All final products or reports presenting data analyses and statistics are free of mathematical errors, and when reviewed, presented analyses have less than 1% errors.

TECHNICAL MEMOS Technical memos must comply with the approved study concepts. The memos must be clear, submitted on time and within budget.

Final memos must reflect findings that are clear and achieve intended research goals. They must demonstrate 100% alignment with project milestone schedules, and document changes or explain reasons for non-alignment.

Deliverables represent appropriate, technically sound and valid data analysis and interpretation.

All final products presenting data analyses and statistics are free of errors, and when reviewed, presented analyses have less than 1% errors.

(1) The quality of products as determined by COR must meet technical rigor, analysis appropriateness and accuracy, methodological transparency, and clarity in communicating content.

(2) Interim and final deliverables meet the contract’s purpose and standards of quality.

(3) COR may review feedback from external reviews requested by the Governing Board, as additional verification of technical quality and understandability.

LITERATURE REVIEWS

AND SYNTHESES OF

BEST PRACTICES

Study plan, which outlines plans for literature review and synthesis of best practices, must adhere to technical rigor and quality standards as approved by COR and CO for each study.

Final reports demonstrate 100% alignment with project milestone schedule, including documentation of changes or explanation of reasons for non-alignment.

(1) Quality of product(s) as determined by COR on the basis of technical rigor, analysis appropriateness and accuracy, methodological transparency, and clarity in communicating content;

QUALITY

DETERMINANTS

QUALITY METRICS CONTRACT

RATING

Desired Outcome & Objective What do we want to accomplish for this contract and what is the desired end result for this project?

Performance Standard What are the standards for completeness, reliability, accuracy, timeliness, quality, and/or cost?

Acceptable Quality Level (AQ) How is acceptance defined and how much error will be accepted?

Monitoring Method How will we determine attainment of quality metrics? Monitoring will take place via the Contracting Officer’s Representative (COR) and Contracting Officer (CO).

Exceptional, Very Good, Satisfactory, Marginal, Unsatisfactory

Deliverables represent appropriate, technically sound and valid synthesis and interpretation.

(2) Relevance of elements of plans to producing interim and final deliverables in accordance with contract’s purpose and standards of quality.

(3) COR may review feedback from external reviews requested by the Governing Board, as additional verification of technical quality and understandability.

MEETING MINUTES

FOR GOVERNING

BOARD PLENARY

SESSIONS

Meeting minutes are accurate, comprehensive, and synthesize key points covered in presentations and discussions.

The minutes are error free and are delivered within agreed timelines.

COR reviews of minutes result in minor edits for clarity or stylistic preference (up to 10 per page) and do not require minutes to be extensively re-written.

Governing Board staff will review meeting minutes and compare to the meeting transcripts. Staff feedback will be provided to the COR to relay to the contractor.

The COR will also review the deliverable submission timeline for compliance with due dates.

EXPERT CONSULTING

SERVICES

Expert consultants have relevant expertise to perform duties for the contracted activities.

Expert panels are diverse and represent various demographic metrics to include race and ethnicity. A spreadsheet of data on all proposed

100% of expert consultants contracted for panels or white papers have relevant expertise and qualifications that match the selection criteria.

Where travel arrangements are required, there must be 100%

COR will review a summary of expert consultant qualifications and provide comments and suggestions.

QUALITY

DETERMINANTS

QUALITY METRICS CONTRACT

RATING

Desired Outcome & Objective What do we want to accomplish for this contract and what is the desired end result for this project?

Performance Standard What are the standards for completeness, reliability, accuracy, timeliness, quality, and/or cost?

Acceptable Quality Level (AQ) How is acceptance defined and how much error will be accepted?

Monitoring Method How will we determine attainment of quality metrics? Monitoring will take place via the Contracting Officer’s Representative (COR) and Contracting Officer (CO).

Exceptional, Very Good, Satisfactory, Marginal, Unsatisfactory expert consultants developed for COR review, details their experiences and expertise relevant to this study.

Expert panel meetings are well-organized, evidencing sufficient advance preparation by project staff for all presentations and meeting transitions.

Arrangements for expert panel meetings comply with all meeting objectives, federal travel regulations, and regulations governing federally sponsored meetings.

Agenda and materials for all meetings will be sent to the COR for review prior to distribution to participants.

Advance materials for expert panelists are sent with adequate lead time.

Advance materials are clear, concise and sufficient to inform discussion of meeting objectives.

Proceedings and documentation of meeting processes and outcomes are complete and transparently conveyed.

compliance with federal travel regulations and ED regulations for federally sponsored meetings.

Reports of meeting costs are submitted for Governing Board review prior to securing meeting arrangements. Five (5) working days are allotted for the review.

Draft materials for expert panel meetings are received by the COR for review at least 7 calendar days before distribution to panelists.