Attachment_A_updated.pdf

Attachment A

PERFORMANCE WORK STATEMENT

U.S. Participation in the Progress In International Reading Literacy Study 2016

Contents

I. INTRODUCTION

II. BACKGROUND

III. LEGISLATIVE AUTHORITY

IV. SCOPE

V. PERIOD OF PERFORMANCE

VI. REQUIREMENTS

Task 1. Contract Administration (Firm Fixed Price)

1.1 Project initiation and planning

1.1.1 Post-award conference

1.1.2 Produce master schedule

1.1.3 Project Management Plan

1.1.4 Staffing Plan

1.1.5 Data Security Plan

1.1.6 Staff Security Requirements:

1.1.7 Transmittals of study products and quality of deliverables

1.1.8 Study brochures

1.2 Contract management

1.2.1 Project update meetings with contractor

1.2.2 Other meetings

1.2.3 Monthly study progress and financial reports

1.2.5 Maintenance of data files

1.2.6 Relationship of the contractor to NCES and other agencies

1.2.7 Readiness for work under conditions of unusual speed and flexibility

1.2.8 Quality Control Measures

1.3 Obtain OMB clearance for field test data collection

1.4 Obtain OMB clearance for the main study’s data collection

1.5 Prepare for briefings on project

1.5.1 Prepare project briefing materials

1.5.2 Provide briefings or presentations

Task 2. Preparations for Assessments (Firm Fixed Price) – Field Test

2.1 Field test sampling plan and sample

2.2 Prepare field test instruments

Task 3. Recruiting, schools, teachers, and students, and Collecting Data for Field Test

(Cost Reimbursable)

3.1 Recruit Schools, Teachers, and Students for Field Test

3.2 Collect data for field test

Task 4. Training Data Collectors for Field Test and Delivering Data (Firm Fixed Price)

Preparing, producing and delivering data

4.1 Recruit and train data collectors for field test

4.2 Produce field test dataset for IEA

4.3 Field test data review memo

Task 5. Scoring and Coding Field Test (Firm Fixed Price)

5.1 Score field test assessment and code questionnaire data

Task 6. (Optional) Preparations for Assessments (Firm Fixed Price) – Main Study

6.1 Main study sampling plan and sample

6.2 Prepare main study instruments

Task 7. (Optional) Recruiting, schools, teachers, and students, and Collecting Data for Main Assessment

(Firm Fixed Price)

7.1 Recruit Schools, Teachers, and Students for Main Assessment

7.2 Collect data for main assessment

Task 8. (Optional) Training Data Collectors for Main Study and Delivering Data (Firm Fixed Price)

8.1 Recruit and train data collectors for main assessment

8.2 Produce main assessment dataset for IEA

8.3 Prepare responses to IEA information requests about data collection

8.4 Prepare National curriculum questionnaires

8.5 Produce U.S. chapter for the PIRLS Encyclopedia

Task 9. (Optional) Scoring and Coding Main Study (Firm Fixed Price)

9.1 Score main assessment and code questionnaire data (IDIQ, dependent on number of respondents)

Task 10. (Optional) Dissemination of Results (Firm Fixed Price)

10.1 National report analysis plan

10.2 Produce reports

Non-response bias analysis report Survey operations reports National reports PIRLS school reports PIRLS released item sets

10.3 Produce main study electronic public-use datafiles and user’s guides

10.4 Disseminate Results

Task 11: (Optional) Utilize expert help (Firm Fixed Price)

Expert panel

Task 12: (Optional) Comparison of NAEP/PIRLS (Firm Fixed Price)

Task 13: (Optional) State benchmarking (Firm Fixed Price)

Task 14 (Optional) Parent Questionnaire – Learning to Read Survey (Cost Reimbursable)

Staff Skills and Commitment

Appendix A. Tentative Timelines for PIRLS 2016 Data Collections

Proposed PIRLS 2016 Project Schedule (tentative)

Appendix B. Data Confidentiality, Security, Quality Control, and Contractor Staff Security Clearances

B-1. Overview of IES/NCES Confidentiality Statutes Confidentiality of Individuals and Institutions Protection and Security of Data

B-2. Data Security Plan Definition of Personally Identifiable Information and Direct Identifiers Development of Data Security Plan Implementation of Data Security Plan

B-3. Adherence to NCES Statistical Standards NCES Statistical Standards

B-4. Quality Control Measures Requirement Quality Control Measures

B-5. Contractor Employee Security Screening Contractor Employee Security Screening Requirements

B-6. Requirement of Affidavit of Nondisclosure Affidavit of Nondisclosure

B-7. Security and Confidentiality Training for Contractor Staff Security and Confidentiality Training

B-8 Information Collection Request (ICR) Requirement

Appendix C. Sampling for PIRLS 2016

C-1. PIRLS Target Populations

C-2. PIRLS Sample Design

C-3. Exclusions

C-4. Response Rates School-Level Student-Level

C-5. School Sampling The Frame (School List) School Sample Selection Replacement Schools Treatment of Small Schools

C-6. Assessment Booklet Design

Appendix D. Disclosure Analysis Plan

List of Exhibits

Exhibit 1

PERFORMANCE WORK STATEMENT

U.S. Participation in the Progress in International Reading Study (PIRLS) 2016

I. INTRODUCTION

The National Center for Education Statistics (NCES) in the Institute of Education Sciences (IES) of the U.S. Department of Education (hereafter, the Department) supports a variety of international activities to collect and report data on education in the United States and other countries. It is the NCES’ goal to provide valuable comparative information about the U.S.

students’ educational performance in an international context. NCES participates in international student assessments in order to:

provide comparable indicators on student performance and schooling practices across countries in order to benchmark U.S. student performance; and suggest hypotheses about the relationship between student performance and factors that may influence performance and about areas where students have particular strengths or weaknesses.

The study covered by this procurement is part of the larger international program that NCES has participated in through collaboration with a variety of international organizations and partners, such as the Organization for Economic Cooperation and Development (OECD), the International

Association for the Evaluation of Educational Achievement (IEA), and ministries of education and other government agencies throughout the world.

Through its participation, NCES seeks to strengthen the quality, validity, consistency, and timeliness of international data. To continue this effort, the United States must follow through with well-organized and well-executed data gathering activities within our national boundaries.

These efforts will allow NCES to provide policy-relevant data for international comparisons of the U.S. adult population’s competencies and skills, and to help inform decision making on the part of national, state, and local policy makers, especially those concerned with economic development and workforce training.

This Performance Work Statement (PWS) is written in five sections. The first section provides an overview for the PIRLS 2016 data collection: its background, its enabling legislation, its purpose, and general parameters of the work to be done under this contract. The second section specifies the core tasks, subtasks, and deliverables associated with contract administration for this work (Task 1). The third section specifies the core tasks, subtasks, and deliverables associated with the work for the field test (Task 2 through Task 5). The third section specifies the core tasks, subtasks, and deliverables associated with the work for the main assessment (Task

6 through Task 9). The fourth section describes a set of deliverables related to the preparation of data sets and dissemination of results for the main assessment (Task 10). The fifth section describes optional tasks and subtasks for PIRLS 2016 (Task 11 through Task 14).

II. BACKGROUND

The Progress in International Reading Study (PIRLS) is an international assessment that focuses on the reading knowledge and skills of fourth- graders. PIRLS is organized by the International

Association for the Evaluation of Educational Achievement (IEA), an international organization of universities and governments based in Amsterdam, the Netherlands. PIRLS has been designed to track changes in student achievement over time, and has been administered on a five-year cycle since starting in 2001. There have been three cycles of PIRLS thus far (2001, 2006, and 2011).

In the most recent round of PIRLS, in 2011, 49 countries and other educational systems, including the United States, implemented PIRLS at either fourth- grade, or participated in one or more of the other available options to permit wider participation at the end of the primary school cycle or in prePIRLS. In total, 12,726 students in 370 schools across the United States took the assessment in spring 2011, joining approximately 312,000 other students around the world who took part in PIRLS. To address the interest in states’ benchmarking to international standards, a state public school sample was also drawn at grade 4 for Florida.

For more information about PIRLS, http://nces.ed.gov/surveys/pirls/ and the IEA’s International

Study Center’s PIRLS home page at http://PIRLS.bc.edu/.

III. LEGISLATIVE AUTHORITY

The authority for this study can be found in NCES’ longstanding mission to “collect, and analyze and disseminate statistics and other information related to education in the United States and in other nations” (stated in section 406 of the General Education Provisions Act, as amended (20

U.S.C. 1221e-1) and in the Education Sciences Reform Act of 2002 (P.L. 279-107, Part C, Sec.

153):

(a) GENERAL DUTIES.-The Statistics Center shall collect, report, analyze, and disseminate statistical data related to education in the United States and in other nations, including…(6) acquiring and disseminating data on educational activities and student achievement (such as the Third International Math Study) in the United States compared with foreign nations.

(See http://www.ed.gov/policy/rschstat/leg/PL107-279.pdf for the full Education

Sciences Reform Act.)

IV. SCOPE

The purpose of this contract is to implement U.S. participation in the Progress in International

Reading Study (PIRLS) 2016.

The overall goal for the procurement is data collection in the United States for PIRLS 2016 that meets all international and NCES guidelines and that makes available on a timely basis high-quality internationally comparative assessment data. Furthermore, the study shall result in one or http://nces.ed.gov/surveys/pirls/ http://timss.bc.edu/ http://www.ed.gov/policy/rschstat/leg/PL107-279.pdf more publications that report on U.S. data and the international comparisons from the perspective of an American constituency.

PIRLS 2016 will follow the established design and standards as set forth by the IEA. These design and standards will cover, but are not limited to, administration procedures, sampling guidelines and requirements, survey instruments, data collection, data coding and processing.

The IEA has contracted with the International Study Center at Boston College to oversee the international development and coordination of the study. In the Performance Work Statement

(PWS) to follow, mention of the Study Center or the international organization refers to these institutions. The contractor shall follow the international guidelines for PIRLS 2016 and shall meet established NCES standards and procedures (NCES standards can be found at http://nces.ed.gov/statprog/).

The contractor shall be clear that NCES is the sole authority to manage PIRLS in the United

States and to negotiate with the international consortium on any issues related to the study. The contractor shall not use the items for any other purposes without NCES’ written consent. Under no circumstances shall the contractor conduct any activities related to PIRLS, including conducting the assessment in sub-national jurisdictions (e.g., states or school districts), without permission from NCES.

The contractor shall bid key staff who have experience working in an international project and who have extensive knowledge of and experience with the operations of an international assessment based on a complex sample design. Given the large volume of highly technical work required by this contract and the fast-paced international schedule that must be met, there will not be time after the start of the contract for key contractor personnel to familiarize themselves with the technical content or operations of PIRLS.

V. PERIOD OF PERFORMANCE

The anticipated period of performance for this contract is sixty (60) months.

At the discretion of NCES, this contract may be extended for up to an additional 60 months past its original expiration date to accommodate additional work in conjunction with future PIRLS.

VI. REQUIREMENTS

Independently and not as an agent of the United States Government, the Contractor shall provide all personnel, materials, services, and facilities necessary for the project and perform the tasks as described below.

This contract shall be incrementally funded and includes (1) contract management; (2) preparing and conducting a field test of the PIRLS 2016 instruments, (3) preparing and conducting the main study or main assessment for PIRLS 2016, and (4) producing public- and restricted-use release files, data documentation, and analytic and methodology reports. It may also include http://nces.ed.gov/statprog/ various optional tasks, such as preparing and conducting a study of ePIRLS, a new initiative for

2016.

A preliminary timeline for PIRLS 2016, with major events and activities, is in Appendix A. The contractor shall note that the timelines are subject to change. References herein to “the study” or

“the project” means the PIRLS 2016 study (and may include the ePIRLS 2016 assessment, if the

United States chooses to participate in that additional component of PIRLS and exercises the ePIRLS option, herein). Due to the timing of PIRLS 2016, some activities at the international level may have begun by the time the contractor begins work.

Task 1. Contract Administration (Firm Fixed Price)

The contractor shall work closely with the Contracting Officer’s Representative (COR) to implement the design of the study included under this procurement and the approach presented in the proposal.

1.1 Project initiation and planning

It is assumed that upon the start of the contract, the contractor’s project director(s) and key staff will have reviewed and have a working knowledge of the technical methods and procedures as well as the international operations and production of past administrations of PIRLS (as explained in detail in the PIRLS international technical reports available at http://PIRLS.bc.edu/).

1.1.1 Post-award conference

Within two weeks of the contract award, as approved by the COR, the contractor’s project director and four other key staff members who would be able to answer questions about sampling issues, data collection processes, costs, etc., from the contractor project team shall hold a meeting in Washington D.C. at facilities made available by the Department. The meeting will include the COR, the Contract Officer (CO), key NCES project staff, and the Department’s

Contract Specialist (CS). The contractor shall focus the meeting to review the overall contract tasks, identify information needed from past rounds of PIRLS, identify potential problems and possible solutions in the project and contract, and discuss areas of concern related to the proposed project staffing plan and other management requirements. The primary purpose of this meeting is to refine the management, staffing, and scheduling plans. These refinements shall provide management information for use by both the contractor and the government in monitoring the work to be performed. This meeting shall also provide the opportunity for

Department staff to share project experience and materials with the contractor. The contractor shall prepare and bring all necessary materials for a productive and successful initial meeting, including an agenda, draft master schedule, and list of key staff with contact information

(deliverable 1.1.1).

Deliverables: 1.1.1 Post-award meeting materials: agenda, draft master schedule, and list of key staff with contact information http://pirls.bc.edu/

1.1.2 Produce master schedule

The contractor shall produce a project schedule that details key milestones; critical dates specified in Appendix A and the “Schedule of Deliverables;” and the contractor’s production deadlines for the Office of Management and Budget (OMB) clearance, instrument revision and adaptation, assessment administrator training, data collection and analysis, as well as reporting.

This schedule is required with the submission of the proposal and will be reviewed by

NCES before the contract is awarded.

The project schedule shall include the following information in an easy-to-understand format:

Start and end dates of all tasks and subtasks;

Key milestone dates;

Completion dates of subtasks, drafts, or other events;

Key deliverable dates.

The master schedule shall make clear the primary leads and contact person for each task, subtask, or deliverable (as applicable).

The contractor shall submit a revised master schedule for the field test within two weeks of contract award (1.1.2), and thereafter submit a revised master schedule to the COR at each update meeting (see subtask 1.2.1). The contractor shall submit a revised master schedule for the main study no later than January 2014. The contractor shall be prepared to update the schedule within 24 hours of a request for it by the COR. The IEA International Study Center will provide updated timelines at each meeting of the National Research Coordinators (NRCs).

Deliverables: 1.1.2 Revised master schedule for field test

1.1.3 Project Management Plan

The contractor shall prepare and submit a draft project management plan to the COR within three weeks of contract award (1.1.3.1). The COR will provide comments on the plan within two weeks of receipt. The contractor shall incorporate the changes necessary to respond to the concerns raised by the COR and submit a revised plan within three weeks of receipt of COR comments. Once the project management plan is approved by the COR, the contractor shall manage the project in accord with the approved final plan (1.1.3.2) and, if a change to the plan is needed, shall notify the COR of such change(s) before implementing such.

The plan shall detail the management techniques that will be employed to ensure timely, efficient, and cost-effective data collection, processing, analysis, and reporting. The plan shall reflect the requirements specified in the contract or describe and justify any revisions to the original plan specified in the contract and approved by NCES. These might include changes in data collection approach, data processing, plans for analysis and reporting, as well as changes in the proposed project organization, staffing plan, and schedule. The plan will fully describe the rationale for any proposed changes.

Deliverables: 1.1.3.1 Draft project management plan

1.1.3.2 Final project management plan

1.1.4 Staffing Plan

The contractor shall prepare and submit a draft staffing plan to the COR within three weeks of contract award (1.1.4.1). The COR will provide comments on the plan within two weeks of receipt. The contractor shall incorporate the changes necessary to respond to the concerns raised by the COR and submit a revised plan within three weeks of receipt of COR comments. Once the staffing plan is approved by the COR, the contractor shall implement the approved final plan

(1.1.4.2) and, if a change to the plan is needed, shall notify the COR of such change(s) before implementing such.

The staffing plan shall detail the project’s overall staffing, including contractors and subcontractor staff dedicated to the study, as well as the training or preparatory activities to ensure that all staff are familiar with and abide by the requirements of the study (including, but not limited to, the study’s need for high quality and timely completion of all activities and for strictly enforced confidentiality procedures).

The ability of the COR to obtain timely and accurate information and decisions from contractor staff is considered of the utmost importance, and will remain a high priority of the contractor.

The contractor shall be responsible for the timely and satisfactory completion of work carried out by subcontractors, and shall convey to subcontractors the need to comply with the rules, regulations, guidelines, standards, timelines, and deliverable schedules of all tasks and sub-tasks included under this procurement.

Deliverables: 1.1.4.1 Draft staffing plan

1.1.4.2 Final staffing plan

1.1.5 Data Security Plan

NCES is required by law to develop and enforce standards designed to protect the confidentiality of individually identifiable respondents, including students and schools. These requirements cover the collection, reporting, and publication of data; they are described in detail in Appendix

B. The contractor shall ensure that all contractor-led operations, including those of subcontractors, adhere to federal laws and standards in this regard.

To describe how the contractor will do this, the contractor shall prepare and submit a draft data security plan to the COR within three weeks of contract award (1.1.5.1). The COR will provide comments on the plan within two weeks of receipt. The contractor shall incorporate the changes necessary to respond to the concerns raised by the COR and submit a revised plan within three weeks of receipt of COR comments. Once the data security plan is approved by the COR, the contractor shall implement the approved final plan (1.1.5.2) and, if a change to the plan is needed, shall notify the COR of such change(s) before implementing such.

The security of this study’s data is of utmost importance to NCES and the security plan will be reviewed by NCES’ Chief Statistician. For a detailed discussion of the data security requirements of the contract, see Appendix B.

Deliverables: 1.1.5.1 Draft data security plan

1.1.5.2 Final data security plan

1.1.6 Staff Security Requirements:

Affidavits of Nondisclosure, Security and Confidentiality Training, and Background

Checks

All contractor staff to work on the project for 30 days or more must complete a series of security forms and undergo a federal background check per OMB requirements, with higher levels of security clearance required for those who visit schools or who may come in contact with confidential data. The contractor shall ensure that all staff who may come in contact with confidential data shall (a) be trained in the proper handling of confidential data and (b) first sign and then notarize an affidavit of nondisclosure before having access to any data with confidential information. The contractor shall execute these affidavits of nondisclosure and retain them in a secure location. The contractor shall provide a list to the COR of all the individuals who have signed affidavits, the position in the organization of the person, and the person’s relationship to the study, within four weeks of the award of the contract (1.1.6).

As new persons are assigned to the project, the security forms shall be filled out and an affidavit shall be executed for them on the first working day of assignment to the study and an updated list provided to the COR, no less than on a quarterly basis. The contractor shall notify the COR if any contractor staff leave the project. The contractor shall be aware of and comply with all contractor personnel security regulations, as described in Appendix B, and all Department IT security policy requirements for personnel, as described in Appendix B.

Deliverables: 1.1.6 List of all personnel with signed Affidavits of Nondisclosure

1.1.7 Transmittals of study products and quality of deliverables

The contractor shall transfer and retrieve study-related documents and files through the Internet, to or from NCES, or other designated contractors. The ability to transfer and retrieve documents through the Internet is important to maintaining the integrity of electronic versions of documents that may eventually be posted on the NCES web site or distributed internally. It is thus imperative that the contractor works closely with NCES web site personnel to set up and ensure that all transfer protocols are followed and that transfer and retrieval of documents (especially from the contractor’s management website, see task 1.2.5, and the NCES secure data server on the NCES Member’s website) is completed to the satisfaction of NCES. The contractor shall conduct a test of its transfer/retrieval system(s), within four weeks of the award of the contract and prior to making use of the system for key contract deliverables, and shall eliminate any problems found to ensure a trouble-free system (1.1.7).

The contractor shall transmit all correspondence directly to the NCES COR via e-mail that is compatible with Microsoft Outlook. Compatibility will extend to the ability to access attachments sent as part of e-mails. NCES uses Microsoft Office 2010: Word, PowerPoint, and

Excel as its primary software packages for communicating documents, charts, and tables. NCES uses the Department of Education’s Microsoft Outlook software as its communications package.

All deliverables are to be delivered in native file format, in the version of Word, PowerPoint, Excel, or other software packages used by NCES.

All final deliverables that are to be assigned a NCES publication number or that will be made public through the NCES website will be delivered in PDF format (or another appropriate format approved by the COR) in accordance with the NCES requirements published in “Policies, Guidelines, Standards and Procedures for Web Publishing for IES and NCES” (available at https://members.nces.ed.gov/ webstandards/), and must adhere to Section 508 requirements for accessibility. More information on accessibility is available through the Federal IT Accessibility

Initiative at: http://www.section508.gov.

Although the term “draft” is used throughout this PWS, it is important that the contractor understand what the Government means, and does not mean, by “draft.” A draft is a complete product of high quality that the contractor (and the government) can publish with confidence as to its accuracy and usefulness. A “draft” is not simply what is available on the due date— regardless of completeness or quality. NCES anticipates that only minor changes to drafts will be needed for them to become final deliverables.

Every final deliverable will be sent with a cover memorandum signed or initialed by the project director or someone of higher corporate stature than the project director. The cover memorandum will certify that the contractor’s organization stands behind the quality of the product. This holds for all final versions of products—even those prepared primarily by consultants or subcontractors. At no time will the government take direct delivery of a product from a consultant or subcontractor.

All draft deliverables will be due at the office of the COR and the CS on or before their due date.

Deliverables must be sent in electronic copy and, if requested by the COR, hard copy format.

Hard copies of deliverables must be sent to the COR such that they arrive on or before their due date. No hard copies are to be sent to the CS.

Deliverables: 1.1.7 Test of transfer/retrieval system(s)

1.1.8 Study brochures

The contractor shall assist NCES in developing an informational brochure for the study that will describe the study’s general goals. At a minimum, it will identify the purpose of the project, the sponsoring organizations, the types of data to be collected, and the data collection schedule. The brochures will be distributed to individuals and organizations requesting information about the study and will be included as part of the information packet used to secure cooperation of study participants.

https://members.nces.ed.gov/%20webstandards/ http://www.section508.gov/

The contractor shall submit a draft of the brochure six weeks after contract award (1.1.8.1). The

COR will review the drafts and provide comments and suggestions within two weeks. NCES, or the IEA, will provide the contractor with an electronic version of the logo of the study, which the contractor shall use in the production of the brochure and other published materials. Based on initial COR comments, the contractor shall revise the draft of the brochure and will resubmit it for full NCES review (see Exhibit 1, under Task 6, for a description of the NCES review process). Once approved, an electronic version of the final brochure will be delivered with the finished copies for inclusion on the NCES web site (1.1.8.2). The electronic version of the brochure will meet NCES web site design and formatting guidelines, which currently requires

PDF and HTML formatting, with graphics as .jpg or .gif files. The contractor shall produce enough copies of the final, printed brochure for all information packets needed to secure cooperation of study participants and deliver an additional 50 copies to the COR for NCES. For the purposes of responding to this proposal, the contractor should assume, for administering

PIRLS without options, of needing to contact about 700 schools, 350 school districts, and 40 states

Deliverables: 1.1.8.1 Draft informational brochure

1.1.8.2 Final informational brochure

1.2 Contract management

Because this contract includes a series of time-sensitive and complex activities, it is critical that it be monitored and managed effectively. To that end, NCES requests the following general study management activities.

1.2.1 Project update meetings with contractor

For the life of the contract, the contractor will hold regular weekly or biweekly (or less frequently, if the COR so determines) meetings for key contractor staff to update the COR or other NCES staff on the project’s progress. The COR may propose to hold these meetings via telephone conference calls, over the Internet, or face-to-face if the meeting is critical. For budget purposes, the contractor shall plan up to 40 meetings a year throughout the contract, of which up to 20 meetings a year may be face-to-face. The contractor shall participate in the development of the goals and agenda for each meeting. The contractor shall be responsible for distributing the agenda and other meeting materials, including the current version of the project’s master schedule, to no more than 12 meeting attendees. Materials to be distributed will be approved, in advance, by the COR.

Deliverables: 1.2.1 Regular project meetings materials: agenda and current version of master schedule

1.2.2 Other meetings

The contractor shall send one representative to attend all National Research Coordinator (NRC) meetings for the study, as well as other technical meetings held by the IEA/International Study

Center. The contractor is strongly urged to send the Project Director/Principal Investigator to the meetings, or the staff member who has primary responsibility for conduct of the study.

Additional personnel may be sent, if needed, upon approval of the COR. For the study, the contractor should assume nine international NRC meetings or technical meetings in a foreign location over the course of the contract, and for the purposes of the preparing the Business

Proposal should cost out each meeting separately.

Deliverables: 1.2.2.1 Attendance at NRC meeting or technical meeting

1.2.2.2 Attendance at NRC meeting or technical meeting

1.2.2.3 Attendance at NRC meeting or technical meeting

1.2.2.4 Attendance at NRC meeting or technical meeting

1.2.2.5 Attendance at NRC meeting or technical meeting

1.2.2.6 Attendance at NRC meeting or technical meeting

1.2.2.7 Attendance at NRC meeting or technical meeting

1.2.2.8 Attendance at NRC meeting or technical meeting

1.2.2.9 Attendance at NRC meeting or technical meeting

1.2.3 Monthly study progress and financial reports

The contractor shall submit to the COR and CS a regular monthly study progress and financial report, by the 10th day of the month following the month the report covers. This report shall be delivered before or concurrent with the contractor’s monthly billed invoice so that it is available to check and approve each month’s invoice. At a minimum, in each monthly report, the contractor shall include information on the following aspects of the study and its operation:

The past month’s work, including a description of continuing work, new work, and completed work, with key dates on which tasks or activities were initiated or completed during the month, by task/subtask;

The past month’s cost of work completed, by task, subtask, and cumulatively;

The upcoming month’s work, including a description of the work and key dates by which time tasks or activities will be initiated or completed during the month;

List of products and milestones submitted or achieved during month, with a brief description of each;

Problems/Issues and how each has been or will be addressed by the contractor; and

Financial indicators of the status of the contract, including at a minimum:

o Percent of contracted amount spent to date, by task/subtask;

o Estimated cost to complete, total estimated cost, and total contracted cost;

o Difference between the total estimated cost and the contracted cost; and o Staff hours spent on each task during the prior month;

Appropriate records and information to permit the COR and CS to certify that the services that are listed on monthly bills are actually used and are for official purposes (if this information is not included with the invoice); and

A manpower report prepared and signed by the project director that summarizes actual personnel assignments for the month just completed, showing for each named individual the hours charged, by task. For individuals not named in the contract, the manpower report will show hours spent by position (e.g., data collectors). All hours billed to the contract will be summarized in the report.

The contractor shall include detailed information in the monthly reports on any anticipated or actual cost overruns or cost savings, whether incurred within a single task or cumulatively. From this information, the contractor shall ensure that the COR is clear about the progress being made on tasks, the work that remains to be accomplished, the financial resources used and available, and any problems and proposed solutions. At the request of the CO, if additional issues need to be addressed in either monthly report, the contractor shall submit an addendum within 1 week of the request.

The format for the monthly report will be consistent from month to month. The same format will be used to report on any subcontractor activity costs.

The monthly report shall serve as a justification for all work billed for the month. Therefore, the contractor shall take care to ensure not only that all work billed for the month is described in sufficient detail in the monthly report to support an audit, but that all detailed information (on completed work) contained in the monthly report is transparently and easily associated with its respective cost(s) or billing(s).

Deliverable: 1.2.3 Monthly study progress and financial reports

1.2.4 Maintenance of project files

The contractor shall establish and maintain a complete and up-to-date set of all project files to ensure that all documents are accessible to project staff at any time and to facilitate the archiving of all draft and final project materials. In doing so, the contractor shall establish an application which will allow the files to be stored on the NCES website.

This will aid in ensuring that all project staff know what are the most up-to-date versions of materials and can access them.

The contractor shall produce a web based application that is capable of running on

Microsoft Windows Server 2012 running IIS 8.0 with a Windows SQL 2012 database. The web pages can be written in Microsoft ASP.Net 4.5 or earlier and MCV.

Deliverables: 1.2.4 Project portal for housing electronic copies of project materials

1.2.5 Maintenance of data files

The contractor shall maintain complete data files for all components of PIRLS 2016, and all supplemented files created and obtained in support of the study, for the life of the contract.

Maintenance of data files means keeping data file documentation available for reproduction on request of the COR or other authorized individuals. Certain data files that will be maintained will not be made available to the public because of privacy considerations. In these cases, file maintenance means that the data files will be kept intact, in a form that can be updated or copied.

Data files will not be allowed to expire, to be released, overwritten, or otherwise destroyed.

These files will be maintained with a software system capable of interlinking them for analysis

(or other) purposes while, at the same time, preserving the necessary confidentiality requirements. If prior PIRLS datasets are required by the contractor, the COR will arrange for delivery for those datasets.

1.2.6 Relationship of the contractor to NCES and other agencies

The contractor shall report all activities for this contract and activities related to PIRLS 2016 to the COR. All data, publications, and products of the study are the sole property of the U.S.

government, and shall be provided to it as specified herein and upon request. Under no circumstances will the contractor, or any subcontractors, distribute or release any data, publications, or products of the study in any form to any individual or party without prior permission of the COR. Any contractor or subcontractor conference presentations or papers, based on this study, shall be reviewed and cleared by NCES prior to any such presentation. This restriction will remain in effect throughout the life of the contract, and until NCES officially releases the data, publications, or products to the public.

The contractor shall work through the COR in all relationships and discussions with the IEA, the

International Study Center, foreign governments, foreign education professionals, and all other agencies and individuals related to the study. The contractor shall not convey to any of these entities information that may be construed as NCES, IES, or Department of Education policy, except where related to technical aspects of the study or where given explicit permission by the

COR.

1.2.7 Readiness for work under conditions of unusual speed and flexibility

In the event that the study’s international plans and time schedules change rapidly due to various necessities, the contractor shall be prepared to work with changing requirements. For example, time schedules may become very tight because the IEA are unable to meet their projected study timelines. Whenever necessary, the contractor shall be prepared to perform activities under conditions of emergency speed. Emergency speed will be defined according to four priority levels:

Priority 1: within 24 hours of request

Priority 2: within 72 hours of request

Priority 3: within 5 working days of request

Priority 4: within 10 working days of request

In the event that changes to plans or time schedules require the completion of a task or activity under emergency speed, the COR will notify the contractor of the request, after consultation with the Contracting Officer (CO), and identify the timeframe (i.e., priority level) under which the task or activity will be completed by the contractor.

1.2.8 Quality Control Measures

NCES is firmly committed to determining how well each project meets its goals, the quality of data collected (including response rates at the school and the student levels), and the implications of the experiences and outcomes of the project for future projects. Project quality control should be integrated into every phase of the project. For instance, potential problems should be identified early so that possible solutions are tested as part of the field test. Item reliability and validity tests for questionnaires and assessments shall be built into the data collection efforts, and results from these tests shall inform data collection and processing. Nonresponse analyses shall be conducted as part of the weighting process, and information about the survey methodology and sample design effects shall be made available to data users.

Quality control also includes (1) managing staff in the field to ensure the high quality of their performance; (2) instituting procedures relevant to confidentiality and protection of data

(including the secure transmittal of confidential data); and (3) ensuring deliverables are submitted on time and are of sufficient quality (in terms of accuracy, coherence, clarity, and comprehensiveness) to meet NCES statistical standards, IEA standards, professional standards, and NCES and IES style guidelines. As noted earlier, NCES expects that only minor changes to drafts will be needed for products to become final deliverables.

Again, all work conducted under this contract shall, at a minimum, meet the standards and guidelines set forth in the publication NCES Statistical Standards, and amendments implemented since that time. The standards can be accessed at http://nces.ed.gov/statprog/. The IES guidelines can be accessed at http://nces.ed.gov/statprog/style.asp.

1.3 Obtain OMB clearance for field test data collection

The contractor shall prepare a clearance package for obtaining OMB approval of the PIRLS 2016 field test and non-cognitive survey instruments that will be used in the study, and any other materials that will be distributed to districts, schools, teachers, students, and parents (e.g., parent permission letters), in accordance with current OMB, Department, and NCES requirements for such packages.

Preparing an OMB clearance package entails assembling an extensive amount of information to

(a) justify the data collection as a whole as well as individual background questions; (b) estimate the cost of the data collection to the federal government and the amount of burden (in both time and cost) on respondents; and (c) explain the data’s uses, efforts taken to minimize the burden on respondents, any provisions for payments or gifts to respondents, steps to assure confidentiality, and the statistical methods that are to be employed in collecting and preparing the data (e.g., dealing with nonresponse). OMB requires that all this information be written up in a Supporting

Statement. An OMB clearance package for submission must include the Supporting Statement, copies of all materials that will go to respondents (such as recruitment materials, survey questionnaire, letters of consent, etc.), and information on any national adaptations of the instruments (see subtask 2.3). The contractor shall prepare a draft OMB clearance package for the field test no later than August 1, 2014, and preferably earlier.

http://nces.ed.gov/statprog/style.asp

After COR review and internal NCES review, the contractor shall revise the draft OMB clearance package within 1 week of receipt of comments. NCES will submit the revised OMB clearance packages to the Department and OMB for review and approval. Failure to submit the

OMB clearance package by the deadline may result in the contractor being unable to collect data in a timely manner. The contractor shall be aware that the time between submission of the OMB clearance package and receipt of a clearance number from OMB for inclusion on all survey documents takes a minimum of four months, and upwards of six months, sometimes more. The contractor shall take these time constraints into consideration when preparing and submitting the OMB clearance package. The contractor shall work diligently and expeditiously with the COR to seek OMB clearance prior to recruiting schools.

In preparing the OMB clearance package, the contractor shall follow instructions for completing the Information Collection Submission Worksheet and writing the Supporting Statement required for Paperwork Reduction Act submissions (for information on the requirements for a clearance package, officially called at present an Information Collection Review, see http://www.whitehouse.gov/omb/inforeg_infocoll/). The contractor shall prepare all written materials needed in the formal clearance and approval process according to OMB requirements as outlined in the Information Collection Submission Worksheet. The package will include the

U.S.-adapted background questionnaires but not the cognitive (assessment) instruments for the study. This approval process is expected to involve the following series of steps (called out as deliverables):

Contractor shall provide a draft package to the COR and shall revise it within a week of receiving the COR’s comments (1.3.1).

The COR provides the package to the NCES OMB liaison and chief statistician, and the contractor shall revise package according to comments received within five business days

(1.3.2).

The NCES OMB liaison provides the package to the Department’s OMB liaison office

(RIMS) and the contractor shall revise the package according to comments received within five business days (1.3.3).

The Department provides package to OMB; the contractor shall answer OMB questions

(1.3.4) within five business days (there may be more than one round of OMB questions) and revise the package according to final comments received within three business days (1.3.5).

The contractor staff shall be prepared to join the COR in person or via conference call for a review of the materials with Department and OMB staff. The contractor shall ensure that authorization by OMB has been obtained no less than three weeks prior to start of data collection. All data collection forms will bear the approval number assigned by OMB.

The contractor shall be aware that states, districts, and schools may not be contacted until OMB has approved the study plan. The contractor shall take this into consideration when planning the field tests and main data collections.

http://www.whitehouse.gov/omb/inforeg_infocoll/

Following approval, the contractor shall provide NCES with one hard copy and one electronic copy of OMB’s official approval or clearance for the study (1.3.6).

Deliverables: 1.3.1 Draft OMB field test clearance package

1.3.2 Revised OMB field test clearance package

1.3.3 Revised package based on RIMS comments

1.3.4 Responses to OMB questions on package

1.3.5 Revised final field test clearance package for OMB approval

1.3.6 OMB clearance for field test

1.4 Obtain OMB clearance for the main study’s data collection

OMB may grant clearance for both the field test and the main study activities or may require that after the field test a new clearance be obtained for the study. In the case that new clearance is required, the COR will provide guidance on the type of materials needed to obtain this clearance, which may include either a memo describing any changes made in instruments or procedures from the field test to the main study, or a full new clearance package similar to that prepared for the field test. In either case, the requirements and steps for approval, as described above in task

1.3, shall apply to obtaining the OMB clearance for the main study. The contractor shall provide either the memo or new clearance package no less than six months prior to the scheduled start of the main study data collection.

Again, the contractor may not begin contacting states, districts, and schools until OMB has approved the study plan. This point is of particular importance in planning for the main assessment as schools typically need to be recruited early in the school year for them to be able to participate in a late spring assessment like PIRLS. The contractor shall take this into consideration when planning the main data collection schedule.

Deliverables: 1.4.1 Draft OMB main assessment memo or clearance package

1.4.2 Revised OMB main assessment memo or clearance package

1.4.3 Revised memo or package based on RIMS comments

1.4.4 Responses to OMB questions on memo or package

1.4.5 Revised final main assessment memo or clearance package for OMB approval

1.4.6 OMB clearance for main assessment

1.5 Prepare for briefings on project

1.5.1 Prepare project briefing materials

From time to time throughout the course of the project, NCES will provide interested individuals and agencies with information about the nature, progress of, and findings of the study. In consultation with the COR, the contractor shall be prepared to provide up to 10 sets of materials for five briefings/presentations about the study over the life of the contract. The COR will notify the contractor at least two weeks in advance of each briefing or presentation. The materials for such a briefing/presentation will be no more than 10 typewritten pages of text (double-spaced) with accompanying graphs and tables suitable for display. Requested materials may include overhead or PowerPoint slides for making presentations regarding the study. Within 1 week of a request, a set of materials will be submitted to the COR for review and comment. The contractor shall make necessary revisions to the materials based on this review and resubmit the materials within three days of receipt of comments.

Deliverables: 1.5.1 Prepare briefing materials on PIRLS project

1.5.2 Provide briefings or presentations

The contractor shall also be prepared to provide up to six presentations, briefings, or demonstrations about the study over the life of the contract. Examples of the kinds of meetings at which the contractor shall be asked to discuss the study are the annual meetings of the

American Educational Research Association (AERA) and NCES’ Summer Data Conference.

They might also include briefings with other federal agencies, staff of the U.S. Congress, or the media. The COR will notify the contractor at least two weeks in advance of each briefing or demonstration.

Deliverables: 1.5.2 Make presentations on PIRLS project

Task 2. Preparations for Assessments (Firm Fixed Price) – Field Test

Develop field test school sampling plans and select samples

The contractor shall develop school and student sampling plans for the field test study that meets established…