RFP_Attachments_ED-NAG-15-5-0004.pdf

Attachment A: Performance Work Statement Solicitation: ED-NAG-15-R-0004

ED-NAG-15-R-0004

STUDENT ENGAGEMENT IN THE NATIONAL ASSESSMENT

OF EDUCATIONAL PROGRESS (NAEP):

CRITICAL REVIEW AND SYNTHESIS OF RESEARCH

June 18, 2015

TABLE OF CONTENTS

I. OVERVIEW

II. STATEMENT OF NEED

III. SCOPE OF WORK

IV. CONTRACT TASKS

Task 1: Project Management

Task 2: List of Relevant Research for Inclusion in the Bibliography: Design and Final

Task 3: Bibliography and Technical Review……………………………………………..………10

Task 4: Synthesis Report………………………………………………………………………….10

V. DELIVERABLES

VI. LIST OF APPENDICES

I. Overview

The National Assessment Governing Board, hereafter referred to as the Governing Board or the Board, was created by Congress in 1988 and comprises 26 members, including governors, state legislators, local and state school officials, educators, researchers, business representatives, and members of the general public. Members of the Governing Board are appointed by the U.S.

Secretary of Education. The Governing Board sets policy for the National Assessment of Educational Progress (NAEP), commonly known as “The Nation’s Report Card.” More information on the Governing Board and its work can be found at www.nagb.gov.

NAEP is a nationally representative, continuing assessment of U.S. student achievement in key academic subjects. Since its authorization by Congress in 1969, NAEP has provided an independent, objective measure of achievement in reading, mathematics, science, writing, U.S.

history, civics, economics, geography, the arts, and other subjects. NAEP results are reported nationally, and depending on subject, by state and in 21 large urban districts at grades 4, 8, and/or

12. Through results of NAEP assessments, the public, including policymakers, educators, media, as well as business leaders and parents, are informed about the academic achievement of elementary and secondary students in the United States.

NAEP has served an important role in evaluating the condition and progress of American education for more than four decades. The independent and objective information provided by NAEP has led to its reputation as the “gold standard” for monitoring student achievement and progress over time. NAEP is administered by the National Center for Education Statistics (NCES) at the U.S. Department of Education, and NAEP program is overseen by the Governing Board, which identifies subjects to be tested, determines the content and achievement levels for each assessment, approves all test questions, takes steps to improve the reporting of results, and plans and executes initial releases of NAEP Report Cards.

Results from NAEP are not reported for schools or for individual students, thus there are no traditionally-conceived stakes attached to performance on these assessments. The most recent reauthorization of the Elementary and Secondary Education Act, the No Child Left Behind Act signed and enacted in 2002, tied school participation in NAEP grade 4 and grade 8 assessments to Title I funding, but, despite this incentive to participate for schools, students face neither incentive to participate nor repercussion for declining to participate in NAEP or not taking the test seriously.

http://www.nagb.gov/

1.1 NAEP Motivation Work

In 2005, NCES convened a Secondary School Principals Working Group to address twelfth-graders’ motivation to participate in NAEP. 1 Their work resulted in recommended approaches for both school leadership and NCES to increase student test-taking motivation, for example, notifying schools early about their selection for NAEP, tailoring recruitment materials to individual schools, and sharing best practices with administrators and teachers for building motivation. Additionally, NCES distributed to schools an informative video to introduce NAEP, and schools used incentives to encourage student participation, e.g., reduced-price prom tickets and credit for community service. By the next administration of NAEP in 2007, applying these strategies had worked: the more of these strategies schools used, the higher their student participation rates2.

Current work on NAEP motivation centers on the essential issues of ensuring selected students

(1) actually participate and (2) participate meaningfully in the test administration. The increased emphasis on assessments nationwide and the increased media focus on testing may affect how students in all grades perceive the utility and value of participating in a low-stakes test such as NAEP. Because NAEP holds no direct implications for students’ future, exerting effort to perform well on NAEP may not be a priority for students. On the other hand, students may see the test as simply another exam in a long succession of assessments and engage with NAEP similarly to their high stakes exams. Previous research often concludes with a recommendation to make NAEP test administration more appealing to schools and to students34. This conclusion typically derives from research that assumes NAEP is administered through a paper-and-pencil approach.

However, NAEP is transitioning to a digital-based assessment, which should be innately more appealing to students. Considering the changes underway in how NAEP is administered and the myriad perspectives on twelfth-graders’ test-taking motivation, now seems an apt time to revisit, review, and synthesize the extant research literature on motivation to take NAEP.

The Governing Board’s authorizing legislation, Section 302 (5) of Public Law 107-279 describes the Governing Board’s duties, which include taking appropriate actions needed to improve the form, content, use, and reporting of assessment results.5 Understanding how motivation on NAEP relates to student performance will inform the Governing Board’s policy work on student performance on NAEP. The work within this report will not be directly statistical in nature;

instead the work described in this Performance Work Statement refers to a critical, empirical

1http://demo3.westat.com/graphics/naep_bpg/PDFs_Word_Docs/Generic/Secondary_Principals_Working_Group.pdf 2 http://www.ode.state.or.us/initiatives/naep/naep_best_practices_2014_508.pdf 3 Brophy, Jere, and Carole Ames. NAEP Testing For Twelfth Graders: Motivational Issues. Washington, DC:

National Assessment Governing Board, 2005. Web. 20 Apr. 2015.

4 Braun, H., Kirsch, I, and Yamamoto, K. (2011). An Experimental Study of the Effects of Monetary Incentives on Performance on the 12th-Grade NAEP Reading Assessment. Teachers College Record, 113(11), 2309-2344.

5 http://www.nagb.gov/naep/naep-law.html http://www.ode.state.or.us/initiatives/naep/naep_best_practices_2014_508.pdf review of the extant literature, as documented through an annotated bibliography, and a thorough synthesis of research.

II. Statement of Need

The Governing Board has a requirement to conduct a systematic examination of empirical research about the motivation of elementary and secondary school students for taking NAEP in Grades 4, 8, and 12. The review of research sought in this Request for Proposals (RFP) shall include research about motivation and engagement in NAEP, especially research that examines digital-based, online, or computer-based assessment.

The work requires a comprehensive technical review and critical synthesis of research on student engagement on NAEP to learn the extent to which motivation may play a role in student performance on NAEP. This review shall include an updated comprehensive bibliography and synthesis of the extant research on motivation and engagement in NAEP and an expanded perspective, shifting from the current focus only on twelfth-graders to include fourth-graders and eighth-graders in the scope of review and broadening the scope to NAEP assessment topics beyond reading and mathematics. The goal of this project is to centralize what the field knows about the extent to which sub-optimal engagement may affect student performance on NAEP.

Among these studies, student motivation and performance on NAEP may be compared to motivation and performance on other assessments, such as state achievement exams. Differences across assessments, such as stakes attached to outcomes, item type (multiple-choice, constructed response, etc.), and mode of administration (paper-and-pencil or computer-based) must be considered when evaluating studies and synthesizing findings. The review also must consider the extent to which findings from previous research may be current, relevant, and applicable to

NAEP.

The proposed study shall include six essential elements:

1) Preparation of a design document that lays out the proposed plan of research, as well as the processes to be used for identifying studies to include;

2) Identification of all extant studies, reports, papers, and research relevant to the topic;

3) An annotated summary of included studies (may be subset of [2]);

4) A critical evaluation of methods, claims, findings, conclusions among included studies;

5) A comprehensive synthesis of findings from all included studies, especially the extent to which results may be share common findings across studies; and

6) Recommendations for future research.

III. Scope of Work

Offerors responding to this Request for Proposals (RFP) are to provide evidence of the ability to find, critically and technically review, evaluate, and synthesize relevant research. The goal of this technical review is to determine the extent to which sub-optimal engagement on NAEP may affect administration of—and performance on—NAEP. This procurement seeks a contractor to compile, review, and synthesize extant research on this topic. Much of the relevant research covered in previous reviews was published years ago, assumes paper-and-pencil administration, focuses only on twelfth-graders, and/or ignores subjects that NAEP covers beyond reading and mathematics. In addition, the Board currently is not aware of work that compares engagement in NAEP to engagement in state assessments; this relationship is of interest to the Board based on its congressionally mandated role to improve the form and use of NAEP. The deliverables described in this performance work statement should include research that addresses these gaps in knowledge.

A preliminary scan of the literature suggests more than thirty studies that fit the general theme of student motivation to engage fully with NAEP. These studies vary in their choice of method, their rigor, and importantly, in their findings. There is no clear consensus on student test-taking motivation that emerges from this scan. The list of relevant sources and the annotated bibliography, the second and third essential tasks in this project, and the critical technical review should pore through all studies identified as relevant and parse through each study’s methodological rigor, drawn inferences, and the impact of the findings for NAEP.

The tasks included in the scope of work are as follows. A detailed description of each task follows this list.

1. Task 1: Project Management

2. Task 2: Research and Reports

3. Task 3: Bibliography and Technical Review

4. Task 4: Synthesis Report

IV. Contract Tasks

Task 1: Project Management

Task 1 deliverables include minutes summarizing the kick-off meeting discussion and monthly reports with schedules.

The project management plan must provide for effective and efficient project control. The offeror shall demonstrate the ability to manage and support the project overall and on a daily basis. An effective management plan shall include roles and responsibilities of proposed staff, along with time allocations enumerated by both hours and percentage of each respective staff person.

Proposed staff, together, must have both managerial and technical expertise, knowledge, and experience needed to execute the project successfully. The project manager should wield experience in direction of a short-term project and demonstrate capacity to adhere to timelines and budgets.

The project management for this contract should include at least one expert statistical analyst with technical skills to identify, review, and critically evaluate complex research and to synthesize disparate research findings into one coherent final paper. The synthesis report should reflect astute, informed, and insightful perspectives on each study’s merits and the overall picture that emerges from all reviewed studies, delineating points of agreement and disagreement in the literature and the reasons underlying differences. This requires a strong background in research and statistical methodology to understand findings and evaluate studies critically.

Subtask 1.1: Kick-off Meeting Within ten working days of contract award, the contractor’s Project Director and one other key project staff as identified in the contractor’s proposal, shall meet in person in Washington, DC at the Governing Board offices with the Contracting Officer (CO), Contracting Officer’s Representative (COR), and other Governing Board staff.

The purpose of the meeting shall be to identify the roles and responsibilities of Board staff and contractor staff; review and discuss proposed contract work; discuss aspects of contract management such as submission of reports, deliverables, and invoices; and establish communication procedures. At this meeting, the project plan, with a GANTT chart (or similar chart) depicting project timelines, as submitted in the offeror’s response to the Request for Proposals, shall be discussed.

Within five working days following this meeting, the contractor shall submit electronically a written summary of the kick-off meeting discussions to the CO with a copy to the COR, documenting agenda items discussed at the meeting and highlighting outcomes agreed upon by both parties. Feedback on the summary shall be provided by the Governing Board within five working days, with the final deliverable submitted no later than five (5) working days after Board feedback.

Subtask 1.2: Progress Reports The contractor is required to communicate regularly for the duration of the contract with the COR to make certain that the COR is informed of all project activities. These progress reports shall be submitted by the fifteenth day of the month (or on a specified date agreed upon by the CO and the contractor). The CO, through the COR, shall respond within five (5) calendar days to each monthly progress report with any technical direction or re-direction that may be warranted in order to carry out Board policy and direction.

The contractor shall report monthly on the progress made in accomplishing the project tasks, the expenditure of funds, immediate notice of problems encountered as well as possible solutions to such problems, and plans for the next month. The schedule shall be updated monthly and submitted with the monthly report and invoice. For all project tasks, the schedule shall detail the key staff assigned, the projected start and end dates, and completion dates. For any monthly schedule, all completion dates shall be entered and the future three (3) months’ activities shall be described.

The contractor shall provide monthly reports on financial expenditures through invoice submissions. Each invoice shall include a summary of the overall project costs broken down by the task activity. The summary will include task/contract funding; task costs by current reporting period; cumulative costs for each task; and a balance of funds for the overall contract.

Task 2: List of Relevant Research for Inclusion in the Bibliography: Design and Final

Task 2 deliverables include a design document describing the process of compiling the list of relevant research and reports (including any justifications for inclusion) and the list of relevant research and reports.

The review of research should extend beyond reading and mathematics to include any NAEP subject on which there is extant research. Reading and mathematics assessments attract much research attention, but NAEP importantly includes as part of its mission an emphasis on other core academic subjects. Thus the Board requires that this review cover motivation to participate and engage with NAEP in subjects additional to reading and mathematics and in all grades covered by NAEP.

The list of relevant research and reports shall include:

• Preliminary scan of research literature by Board staff;

• Any work under review or ongoing at NCES, as provided by the COR; and

• Any relevant work found by contractor in a comprehensive, extensive search of the field.

Subtask 2.1: Design Document Outlining Process to Select and Review Research The contractor shall develop a comprehensive design document to provide the foundation for the bibliography, critical technical review, and synthesis report. The design document shall describe how studies will be identified and selected for inclusion in the organized list of relevant research to evaluate.

The design document must detail the search terms and selection criteria for studies to warrant inclusion in the critical technical review and synthesis report and the process to identify and select studies. This planning process document must clearly reflect consideration of the following for each study:

• Methodological rigor—sample, data, methodology, technical execution of analysis;

• Drawn inferences and support for such inferences;

• Application and/or impact of the findings to NAEP; and

• Any theoretical and/or statistical comparisons to assessments other than NAEP

The contractor shall document the sources used to locate the relevant studies and reports as well as the searches conducted that led to the studies on the list. Upon approval of this design document by Board staff, the contractor shall provide evidence of adherence to the plan when building the list of relevant sources (Subtask 2.2) through monthly reports, progress updates, and weekly teleconference meetings.

Subtask 2.2: List of Relevant Sources Once the process outlined in the design document is implemented, the resulting list of research and reports then will become the outline for the annotated bibliography and synthesis. The contractor shall identify the universe of relevant research for inclusion in the annotated bibliography and critical technical review and shall organize the work by:

• Date;

• Type of research study (see Common Guidelines for Education Research [NSF-ED]);

• Administration mode (paper-and-pencil or digital-based);

• Participant age group; and

• Assessment subject area.

The contractor shall document the sources used to locate the relevant studies and reports as well as the searches conducted that led to the studies on the list. The contractor shall present this list to the Board staff for review and approval prior to drafting the annotated bibliography and conducting the critical technical review. The Board has the discretion to suggest additional sources or searches to improve the comprehensiveness of the list, if necessary. The final list of sources that will form the focus of the annotated bibliography may include all or some of the sources presented in this deliverable.

Task 3: Bibliography and Technical Review

Task 3 deliverables comprise an initial draft and a final draft of the annotated bibliography and critical technical review, including visual representations of findings.

The contractor shall develop an annotated bibliography of research and reports about student motivation and engagement in NAEP. This annotated bibliography shall cover the extant relevant research on motivation and NAEP, including all grades tested and a broadened scope to all NAEP assessment subjects. The annotated bibliography shall include a technical synopsis with each included study or report that details assessment mode, date, target age, research type, methodology, findings, concerns, and relevance for NAEP along with critiques of the methodology, findings, and conclusions.

The notes associated with each entry in the bibliography should reflect an insightful, critical, technical analysis of each included study’s strengths and weaknesses. This should be conducted according to a clear and systematic coding system, developed by the contractor and approved by the COR, to establish how the search proceeded, under what conditions or filters, and how the research studies were deemed relevant to merit inclusion. The annotated bibliography should highlight findings from all relevant studies, critically evaluate the stated claims, describe each study’s strengths and weaknesses, and draw special attention to particularly rigorous and relevant studies as well as to studies that use digital-based, computer-based, or online administration.

The COR and Board staff may suggest revisions and additions to the interim product(s), in which case the contractor shall document how these suggestions were incorporated or justify why they were not. The contractor shall prepare visual graphics, charts, and/or other pictorial representations of the commonalities and differences in the extant literature, which emerge from the critical technical review.

Task 4: Synthesis Report

Task 4 deliverables include initial and final drafts of the Synthesis Report.

The contractor shall prepare a report that (1) critically evaluates research findings and (2) synthesizes research findings from research studies and reports included in the annotated bibliography. This report shall highlight the extent to which results are consistent across studies, draw conclusions about the extent of the motivation issue, and recommend directions for future research. The report should be geared to policymakers who may be concerned about student engagement with NAEP but sufficiently technical to satisfy policy analysts. The requirement to discern consistency among results is significant. Distinguishing what reports, with what methodologies and conditions, suggest which conclusions based on which results is a vital goal for this task and deliverable. The final report should reflect astute, informed, and insightful perspectives on each study’s merits and the overall picture (from all the studies) that emerges about student motivation, delineating points of agreement and disagreement in the literature and the substantive and technical reasons underlying differences.

The COR as well as Board staff may suggest revisions and additions to the interim report(s), in which case the contractor shall document how these suggestions were incorporated or justify why they were not. The final report should reflect a thoughtful synthesis of extant published and unpublished research on students’ motivation and engagement in NAEP. The report should highlight consistencies across studies and offer directions for future research as well as draw conclusions within the context of NAEP. The contractor shall present findings from this final report to members of the National Assessment Governing Board at the May or August 2016 quarterly Board meeting. For each Board meeting between time of contract award and final presentation, the contractor shall prepare a two-page summary of work progress.

If the findings from this final report potentially merit special interest from the public and/or NAEP stakeholders, the COR may request a brief highlight document for release on the Governing Board website (see NCES Data Point for similar example: http://nces.ed.gov/pubs2015/2015066.pdf). This would represent a repackaging of already conducted work in the service of the final report.

http://nces.ed.gov/pubs2015/2015066.pdf

V. Deliverables

The schedule for project deliverables shall align with the dates listed below.

Tasks and Deliverables Due Date (Actual dates TBD at time of award

Task 1: Project Management

• Summary of kick-off meeting

• Bi-weekly teleconferences for progress updates (ongoing through contract life)

• Monthly progress reports (ongoing through contract life)

Initial draft five working days after award; final draft no more than ten working days after kick-off meeting

Task 2: Research and Reports

• Initial draft

• Final list

Four weeks after kick-off meeting Four weeks after initial draft approved by COR

Task 3: Bibliography and Technical Review

• Draft bibliography and critical review

• Final bibliography and critical review

Eight weeks after final list submitted Six weeks after initial draft approved by COR

• Draft synthesis report

• Final synthesis report

Ten weeks after final list submitted Six weeks after initial draft approved by COR

VI. List of Appendices

Appendix A: Quality Assurance Surveillance Plan

Attachment B: Instructions to Offerors:

Student Engagement in NAEP:

Critical Review and Synthesis of Research

Attachment B: Instructions to Offerors

The offeror is expected to show a thorough understanding of the proposed tasks and the existing resources, as specified in the Performance Work Statement (PWS). The proposal should address the requirements in the PWS, not simply repeat the requirements. The proposal should be written in enough detail so that a review panel can adequately judge its full merits. Although the intent of each task is spelled out in the PWS, it is up to the offeror to propose the most effective method for carrying out the tasks and the most feasible timeline and dates for all key activities. Offerors are encouraged to exercise independent judgment with respect to the best methods of attaining the objectives of this contract.

All information necessary to judge the technical soundness and management capabilities of the offeror will be contained in the technical proposal. The business proposal will contain all information related to the determination of the price associated with each of the project’s tasks.

The technical proposal will not contain any reference to specific costs or prices, but resource information may be included so that the offeror’s understanding of the scope of the work may be evaluated.

This Request for Proposals (RFP) does not commit the Government to pay any costs for the preparation and submission of a proposal. In addition, the Contracting Officer is the only individual who can legally commit the Government to expenditure of public funds in connection with this proposed acquisition. It is understood that your proposal will become part of the official contract file.

Proposal Submission Requirements Offerors are required to follow the submission format as detailed below. Responses shall include three separately packaged hard copy proposals: a technical proposal, a business proposal, and past performance information as described in detail below.

• One original and five copies of the technical proposal

• One original and four copies of the business proposal (Microsoft Excel format with formulas for cost and pricing data)

• Three copies of the past performance

• An electronic copy of all files on a portable drive (with read write capabilities)

The proposal must be submitted by an official authorized to bind your organization. Responses to the RFP shall be evaluated in accordance with the evaluation criteria provided in Attachment C of the RFP.

A. Technical Proposal All information necessary to judge the technical soundness and the capabilities of the offeror shall be contained in the technical response to the RFP). The technical response shall not exceed 20 pages, on 8.5” by 11” paper size and a 12-point font, assuming standard 1” margins. Resumes and other supporting material may be provided in appendices. Each resume should not exceed 4 pages. The technical proposal must not refer to specific costs, but resource information may be included so that the offeror’s understanding of the scope of the work solicited by the Government may be evaluated.

1. Table of Contents The proposal should include a Table of Contents as a means of quickly accessing important points of discussion in the proposal. The Table of Contents shall outline the offeror’s response to the RFP and include an executive summary of the technical approach.

2. Executive Summary The Executive Summary should briefly describe the offeror’s overall plan for achieving the project objectives and highlight key areas of the proposal that meet the requirements. A short summary of the offeror’s qualifications and unique strengths shall be provided in the summary.

3. Technical Approach The technical approach shall address all specified requirements in executing the Statement of Work. While a general statement of strategy may be appropriate, the proposed plan to implement work must be specific with regard to each task area. Potential and/or anticipated problems must be identified. The time frame for completing work within the projected timelines and due dates must be addressed to facilitate successful completion of work.

Offerors must demonstrate an understanding of key areas of work and must propose a viable project implementation plan, with effective quality control and cost-saving strategies. It is particularly important to demonstrate technical understanding of federal requirements for providing communications services, in accordance with specific technical specifications detailed in the Statement of Work.

Creativity in responding to the requirements is encouraged. Approaches that are thoroughly described and supported by effective management and implementation plans are most likely to contribute to the overall efficiency and success of accomplishing project goals.

4. Management Plan The management plan section shall detail the offeror’s overall management plan, including lines of authority, coordination, and communication within the offeror’s organization. This project requires working under tight timelines; therefore a well-formulated project management plan is especially important. The offeror shall describe in this section of the proposal the management control system that will be used internally for planning, scheduling, budgeting, managing, and reporting on all phases of the contract.

For a successful outcome, this project will require an effective management system that enables the contractor to complete work on schedule and within budget while ensuring high technical quality. The system shall include procedures for coordinating and controlling project personnel and tasks, including appropriate time commitments of proposed personnel; ensuring adherence to schedules and deadlines; ensuring high quality products and outcomes; identifying potential problems (technical or cost) early; methods used to mitigate these problems; maintaining close, effective communication with the Contracting Officer’s Representative (COR); and accounting for and controlling project expenditures.

The offeror is invited to propose a management and staffing plan that maximizes the efficiency of all work processes and assures adherence to the schedule of activities and project budget. The

Governing Board stipulates, however, that a single person be identified as a project director to provide leadership and direction to the contractor’s project staff and to serve as the contractor’s primary contact with the Governing Board.

All key personnel, including staff and consultants, shall be identified by name and title. Their position in the project’s management structure must be detailed in an organizational chart. This chart shall depict clearly the lines of authority and responsibility for all persons and organizations involved in the conduct of this project. The proposed management plan should ensure appropriate coordination of activities across relevant groups and individuals in the process and appropriate timing of activities.

A Gantt or similar chart displaying the proposed timeline and key project milestones shall be provided. This section shall include a chronological timeline for all major activities and key deliverables. Included in this timeline shall be the start and completion dates for each task, as well as intermediate dates for precursor steps and draft deliverables, as appropriate. Key staff responsible for each task shall also be identified. The work of subcontractors (if any) is the responsibility of the prime contractor, and the management plan must specify how subcontractors (as applicable) will be managed.

A Quality Assurance Surveillance Plan (QASP) shall be submitted in this section of the proposal to address all phases of the evaluation process and development of recommendations. The plan must provide for assurance of quality control throughout the project, adherence to project timelines, and strategies of cost control. Offerors shall propose a plan to ensure quality control and expand or propose additional metrics to the Quality Assurance Surveillance Plan (QASP) in the RFP.

5. Key Personnel This section of the technical proposal shall identify proposed staff and highlight their educational background and expertise relevant to the proposed work. Proposed staff shall have qualifications appropriate for the labor category in which role they will perform work. Vitae of proposed staff shall document knowledge, skills, abilities, and relevant training associated with the position proposed. Key personnel competencies and experience in performing work of a similar size and scope to that outlined in the Performance Work Statement, and proposed roles must be specified.

The proposed Project Director must have demonstrated skills in managing similar projects, have exceptionally good management and organizational expertise, and must have strong written and oral communication skills. Successful oversight for project quality and cost controls is essential to the contract. The Project Director must participate in all aspects of the project and serve as the primary contact with and source of information for the contract. The proposed Project Director is a key position in the contract. Any changes or substitutions of key personnel will require written advance approval by the Contracting Officer.

To plan, conduct, and complete the work successfully, the offeror must provide staff that collectively possesses technical expertise, knowledge, and experience in the following illustrative areas:

• Expertise in managing effective strategies to achieve project objectives and impact;

• Experience in deploying creative strategies;

• Experience in conducting research to undertake project goals;

• Ability and experience in organizing and categorizing significant amounts of content, including publications, data, text, graphics and images in a streamlined manner;

• Ensuring quality and accurate products;

• Familiarity with the fields of assessment such as NAEP as well as quantitative research and methodology to understand findings and evaluate studies critically;

• Expertise in identifying, reviewing, evaluating, annotating, and synthesizing fields of research work;

• Knowledge and experience in managing government contracts and adhering to federal regulations; and

• Managing projects on time and within budgets.

Resumes shall be included for all key staff for this contract and should document only directly related experience, educational background, publications, and professional activities. All resumes should share a common format and organization.

6. Organizational Experience The offeror’s organizational capability and experience in undertaking all phases of work specified in the PWS must be demonstrated. This includes resources that are necessary for the supporting of all task areas. The offeror’s experience in conducting projects of a similar nature and magnitude must be provided.

The Organizational Experience section shall describe the offeror’s experience and effectiveness in designing and conducting research similar in content, complexity, rigor, and scope to the proposed research described in the Statement of Work. This section should describe the offeror’s ability to meet key milestones for project operations and deliverables according to the approved project schedule. In addition, this section shall outline the offeror’s expertise in critical evaluation of research methods and findings and in the synthesis of complex and myriad research findings. Finally, the section will provide information that demonstrates the capability of the offeror with respect to human and material resource management, effective financial management, accurate record keeping, and timely reporting.

7. Alternative Approaches Offerors are encouraged to suggest efficient and creative ways of conducting task areas outlined in the Performance Work Statement. Additional or alternative strategies that complement, update, or more appropriately address requirements in the PWS may be proposed. Offerors shall fully describe and justify significant differences or alternative strategies from the PWS.

B. Business Proposal

Business Proposal Submission The business proposal shall include completion or attestation of the Representations and Certifications issued in the RFP or attest to online certification, and provide proposed overall contract costs detailed by task. Costs shall be provided in a format similar to the Pricing Schedule (Attachment E). A cumulative total shall be provided. The business proposal shall include a Conflict of Interest Plan and Certificate. The business proposal will contain all information related to the determination of the costs associated with the execution of the project. Offerors shall submit the following:

1. A project budget for each major task.

2. A detailed breakdown of labor costs and other direct costs for each section.

Other Direct Costs need to be identified by cost type, and cost assumptions will need to accompany the proposed costs. All other cost reductions or rebates shall be provided upfront as it is likely that negotiations shall be very limited; an award may be made without extensive cost negotiations. The budget needs to be provided in both print, as well as electronic Microsoft Excel format (with formulas).

Conflict of Interest Plan The offeror shall submit a Conflict of Interest Plan, providing details on its policies and procedures to identify and avoid potential organizational or personnel conflicts of interest (or apparent conflicts of interest). The Plan should also address procedures taken to neutralize or mitigate such conflicts, if they have not been or cannot be avoided. The Plan should indicate that such policies and procedures are operative throughout the period of outside activities financial interests, or other significant connections or identifications that would establish, or give the appearance of establishing, a conflict of interest. A method for periodically reviewing financial interests of employees, subcontractors and consultants, and their immediate families, in order to assess actual or apparent conflicts of interest, should be included in the plan. The term, “potential conflict” means a reasonably foreseeable conflict of interest. The thoroughness, completeness and effectiveness of the Plan shall be evaluated as part of the offeror’s overall proposal. The Plan will be incorporated into the contract awarded to the successful offeror.

C. Past Performance The Request for Proposals shall include submission of past performance information to substantiate offeror’s capabilities in performing work of similar scope. Short abstracts of related work, for four previous projects (one of which must be for a federal agency) completed during the past three years that identify clearly both the names of staff members who were participants and the name, current affiliation, and current telephone number of the sponsor’s project officer should be provided. These project officers may be asked to report their experience with the offeror on relevant projects with regard to the size, problems (if any), cost overruns (if any), responsiveness, flexibility, and project quality. A form for filling in the Past Performance Report is provided as Attachment D to the RFP. The past performance information will be provided a separately bound proposal and will not be counted within the 20-page technical proposal submission.

Contract Award Contract award shall be made to the responsible offeror whose proposal meets all RFP requirements and provides an overall best value to the Government as detailed in Attachment C Technical Evaluation Criteria.

Additional Information The proposal shall list the names and telephones numbers of persons authorized to conduct negotiations. The proposal shall include a statement that the offer is firm for a period of at least 60 calendar days from receipt of offers specified by the Government.

Attachment C: Technical Evaluation Criteria:

Student Engagement in NAEP:

Critical Review and Synthesis of Research

Attachment C: Technical Evaluation Criteria

Evaluation Factors for Award

The award decision will be based on best value to the government which shall documented by the Contracting Officer. While price will be an important factor, price/cost will be significantly less important than technical merit. The importance of price will increase as technical merit becomes more equal.

Total Possible Quality Points (100 points) The technical rating (worth 80 points), combined the offeror’s past performance rating (worth 20 points) will total a maximum 100 points as summarized and detailed below.

Technical Evaluation Criteria and Points

Summary Evaluation Criteria Maximum Points

1. Technical Approach 30

2. Management Plan 20

3. Project Staffing 15

4. Organizational Experience 15

Total Technical Points (for Panel Ratings) 80

5. Past Performance 20

Grand Total 100

Technical Approach (30 points)

Reviewers will look for information that demonstrates the following:

• The extent to which the offeror’s technical approach demonstrates a clear understanding of overall project objectives and specific requirements;

• A comprehensive and viable technical work plan that addresses each required area of support, as detailed in tasks;

• The extent to which the offeror and staff have familiarity with the fields of assessment and large scale assessments such as NAEP as well as quantitative research and methodology to understand findings and evaluate studies critically;

• The extent to which the technical approach reflects an understanding of the challenges facing the project and proposes innovative solutions to these challenges.

Management Plan (20 points)

Each proposal will be reviewed for information that demonstrates the capability and experience of the offeror to successfully manage the work. Reviewers will look for information that includes the following key elements:

• A comprehensive plan for managing the project, with allocation of necessary personnel and resources for the project team;

• Credible evidence that the management of personnel, resources, activities, and work will result in successful and timely completion of work requirements within the project performance period and within the project budget;

• A plan that addresses each individual task with respect to overall project management – specific project management actions needed to accomplish each task, a reasonable timeline for accomplishing each task, and the key personnel assigned to each task;

• Cost-saving measures for managing the work; and

• A comprehensive quality control plan for the project compliant with the requirements detailed in the Quality Assurance Surveillance Plan

Project Staffing (15 points)

Proposals will be reviewed for information that shows proposed staff have the education, training, knowledge, and relevant experience to successfully carry out the project. Reviewers will look for information that demonstrates the following:

• Demonstrated ability of proposed staff to perform work assignments of similar nature;

• Requisite knowledge, skills and abilities of key staff assigned to the project;

• Proposed staff possess the technical expertise, knowledge, and experience in all areas related to the study, including research review and evaluation, motivation and engagement, assessment generally, and large scale assessment projects such as NAEP specifically; and

• Experience checking and improving the quality of project work and products.

Organizational Experience (15 points)

Offerors shall demonstrate the following:

• Organizational capability, resource allocations, and experience in performing work for the federal government;

• Organizational experience in managing contracts of similar size and scope;

• Evidence of strong corporate commitment and oversight as evidence by a clear and effective approach to problem identification and resolution; and

• Demonstrated expertise in the following areas: designing and conducting research reviews; scouring diverse and numerous databases for relevant research; selecting research for inclusion in reports; evaluating research insightfully; synthesizing findings from extant research; and understanding of large scale assessments such as NAEP.

Past Performance (20 points: Rated by Contracting Officer)

Past performance evaluation will be based on information obtained from the references provided in the offeror’s responses to the RFP, information from www.cpars.gov, information from other customers known to the Government, and other sources who have relevant information.

Evaluation of past performance may be quite subjective. Offerors shall be given an opportunity to discuss adverse past performance information, if the offeror has not had a previous opportunity to comment on the information. If no relevant information on past performance is available for an offeror, the offeror will not be evaluated favorably or unfavorably on past performance, meaning half-credit will be given.

The past performance rating (worth 20 points) will be combined with the technical rating (worth 80 points) to produce a combined rating of a maximum 100 points. Past performance sub-factors are detailed below:

(a) Quality of Product or Service – compliance with contract requirements as stated above – technical quality and excellence – responsiveness and accuracy of communication strategies – assignment of qualified and appropriate personnel – understanding of the contract requirements.

(b) Timeliness of Performance – meets milestones and deadlines – quick turnaround time on requests – promptly communicates issues– reliable – stays on schedule – responsive to technical direction – completes tasks on time.

(c) Problem Resolution – anticipates, avoids, or mitigates problems – proactive in accomplishing task activities – satisfactorily overcomes or resolves problems – prompt notification of problems – recommends viable solutions.

(d) Cost Control – adhere to contract budgets – provides current, accurate, and complete billings – costs are properly allocated – unallowable costs are not billed – relationship of negotiated costs to actual costs – cost efficiencies.

(e) Business Relations – effective management – accountability for employee performance – business-like concern for customer’s interests – effective use of website communication strategies – effective supervision of staff – reasonable/cooperative behavior –flexible – maintains high employee morale.

(f) Customer Service – prompt responses and communication – understands and embraces service and program goals – team approach with the customer – satisfaction with the contractor’s services – positive customer feedback –– courteous interactions – initiative.

http://www.cpars.gov/

Appendix A

Quality Assurance Surveillance Plan

Contents

Introduction

Purpose of the Quality Assurance Surveillance Plan

Process of Quality Assurance Assessment/Incentive Evaluation

Roles and Responsibilities

Quality Assurance Review Process

Contract Specific Ratings

Table 1: Quality Assurance Surveillance Plan

Table 2: Specific Contract Areas of Performance

Table 3: Quality Assurance Monitoring and Rating Form

Introduction

In order to ensure compliance with the contract terms and conditions, and pursuant to the requirements of Federal Acquisition Regulations (FAR) Subparts 37-6, Performance-Based Acquisition, FAR Subpart 46-4, Government Contract Quality Assurance, and U.S. Department of Education (ED) regulations and guidelines for contract monitoring plans a Quality Assurance Surveillance Plan (QASP) is established as a contract monitoring tool. The QASP will be used throughout the contract lifecycle to ensure that contract requirements and expectations are fully documented and monitored throughout the contract period of performance.

A performance-based incentive plan has been developed in accordance with FAR 16.401(e)(3) that establishes reasonable and attainable performance targets in specific areas of work, pursuant to the requirements identified in the Performance Work Statement (PWS). The QASP sets forth procedures and guidelines that the Department of Education will use in evaluating the technical performance of the contractor. Following contract award, the QASP may be further refined to document mutual expectations of both the Government and Contractor.

The Contractor agrees to perform work in accordance with the contract terms and conditions, and as detailed in this QASP in order to accomplish project objectives. The QASP will be an integral component of the contract and will be updated as needed throughout the contract lifecycle.

Purpose of the Quality Assurance Surveillance Plan

The purpose of the QASP is to define systematic quality assurance methods used by the government to monitor the contractor’s performance, as documented in the contract terms and conditions, and in contract orders and tasks issued under the contract. The standards and quality levels are identified in the PWS and the contractor’s quality control plan (QCP) submitted by the contractor, and accepted by the government. The QASP summarizes key areas that will be monitored and measured by the government to meet project goals.

The objectives of the QASP are as follows:

• Define the roles and responsibilities of the Government and the contractor with respect to performing work under the contract;

• Identify performance expectations vis a vis key areas of work;

• Describe the rating elements and standards of performance for each area;

• Describe the process of conducting the quality assurance assessment; and

• Provide quality assurance monitoring forms…