Attachment G- Table of Deliverables.xls

XLS spreadsheet 251 KB Posted May 23, 2012

Attached to: NATIONAL ASSESSMENT OF EDUCATIONAL PROGRESS (NAEP) ALLIANCE 2013 -2017
Solicitation number: ED-IES-12-R-0021
Issued by: Department of Education Contracts and Acquisition Management

About this file

From the source

Table of Deliverables

View the file

Other files

File	Type	Posted
Amendment 0004.pdf	PDF	Jun 26, 2012
Attachment J-2013-17 Alliance Task List Fixed_price_CR.xlsx	XLSX spreadsheet	Jun 20, 2012
Attachment I- Supplemental Instructions to Offerors.pdf	PDF	Jun 20, 2012
Amendment 0003.pdf	PDF	Jun 20, 2012
Amendment 0002.pdf	PDF	Jun 13, 2012
Amendment 0001.pdf	PDF	May 31, 2012
Attachment H- Cost Proposal Template.xlsx	XLSX spreadsheet	May 23, 2012
Attachment F- Acronyms.pdf	PDF	May 23, 2012
Attachment A- Performance Work Statement	—	May 23, 2012
Attachment E- Billing Instructions.pdf	PDF	May 23, 2012
Attachment I- Supplemental Instructions to Offerors.pdf	PDF	May 23, 2012
Attachment D- Security Risk Levels.zip	ZIP file	May 23, 2012
Attachment C- Past Performance Form.doc.dot	DOT file	May 23, 2012
RFP.pdf	PDF	May 23, 2012
Attachment B- Award Fee Plans.zip	ZIP file	May 23, 2012

Show all 15

On GovTribe

Work with this file

Download the original file
Contacts named in this file
Similar government files
Ask GovTribe AI about this file

Try for free Login

Text version

The file's text, extracted by GovTribe without its formatting.

Alliance Table of Deliverables

Table of Deliverables NAEP Planning and Coordination
Task Number	Deliverable	Due Date
1.1.1 Conduct strategic planning	2 meetings annually with experts and practitioners to identify opportunities and threats that support or detract from efforts to advance NAEP as the leader in large-scale assessment.	One meeting between to occur between mid-October and November 30 One meeting to occur between April and May 31
	Draft meeting agenda to the COR at least 14 days prior to panel meetings.	14 days prior to panel meetings
	Final meeting agenda due to the COR 10 days prior to the meeting, and once approved by COR, disseminated to panel members.	Final agenda delivered 10 days prior to the meeting and distributed on the date of COR approval
	Documentation and dissemination of expert panel deliberations and resultant.	Within 2 business days following the meeting
	Memoranda on approaches to operationalizing recommendations.	To be determined in consultation with COR
	Weekly report to COR on status of action items and implementation of recommendations.	To be determined by COR
	Up to 2 recommendations memoranda annually on approaches for managing the impact of change on NAEP stakeholders and contractors.	March 31 and October 31 annually
	Program Executive Steering Committee	Establish within 30 days of contract start date
	Meeting space and equipment for up to six Executive Steering Committee meetings per contract year.	To be determined in consultation with COR
	Draft meeting agenda to the COR at least 14 days prior to Committee meeting.	At least 14 days prior to Committee meeting.
	Final meeting agenda due to the COR 10 days prior to the meeting, and once approved by COR, disseminated to Committee members.	Final agenda delivered 10 days prior to the meeting and distributed on the date of COR approval
	Meeting minutes and action items from Committee meetings due to the COR and distributed to committee members, via posting to the IMS within two business days following the meeting	Within 2 business days following the meeting
	Weekly report to COR on status of action items.	Weekly
1.1.2 Conduct operational planning towards meeting NAEP’s primary goal	Plan for achieving and evaluating progress towards NAEP’s primary. Annual updates to plan.	30 days after contract start date, and updated October 31 annually
	Recommendations memoranda to the COR, on approaches to achieving NAEP’s high priority goals.	60 days after contract start date
	Quarterly reports on the status of NAEP performance to include progress towards NCES goals.	March 31, June 30, September 30 and December 31 annually
1.1.3 Conduct operational planning towards achievement of NAEP-related organizational performance targets	Plan for achieving and evaluating progress towards achieving GRPA goals pertinent to NAEP operations.	30 days after contract start date, and updated October 31 annually
	Recommendations memoranda to the COR, on approaches to achieving GRPA goals pertinent to NAEP operations.	60 days after contract start date
	Quarterly, cumulative reports on the status of NAEP performance to include progress towards GPRA targets.	March 31, June 30, September 30 and December 31 annually
1.2.1 Manage the implementation of tasks across multiple-contractors	Assessment coordination plan	November 15 of contract award year
	Annual updates to assessment coordination plan	November 15 annually
1.2.2 Coordinate planning and review meetings among NAEP contractors and NCES	Project initiation meeting to be held within two weeks of the start of the contract.	Within two weeks of the start of the contract.
	Mid-year NAEP operations meeting.	Between May and June annually
	Fall NAEP Design Summit.	Between November and December annually
	Draft meeting agenda to the COR at least 14 days prior to major review and planning meeting.	At least 14 days prior to major review and planning meeting
	Final meeting agenda for major planning and review meetings due to the COR 10 days prior to the meeting, and once approved by COR, disseminated to meeting attendees.	Final agenda delivered 10 days prior to the meeting and distributed on the date of COR approval
	Meeting minutes from major review and planning meetings due to the COR and distributed to attendees, via posting to the IMS within two business days following the meeting.	Within 2 business days following the meeting
	Memoranda of recommendations on follow up actions due to the COR and distributed to attendees, via posting to the IMS within two business days following the meeting.	Within 2 business days following the meeting
	Memoranda of approaches to operationalizing recommendations from major review and planning meetings, upon request from the COR.	To be determined in consultation with COR
	Monthly project planning meeting between the Alliance and NCES CORs.	Monthly, to be determined in consultation with COR
	Meeting agenda for monthly project planning meeting between the Alliance and NCES CORs at least 3 days prior to meetings.	At least 3 days prior to meetings.
	Meeting minutes and action items for monthly project planning meeting between the Alliance and NCES CORs, due to the COR and distributed to attendees, via posting to the IMS within two business days following the meeting.	Within 2 business days following the meeting
	Regularly scheduled meetings for up to10 Alliance-NCES working groups, on a schedule to be determined.	To be determined in consultation with COR
	Meeting agenda for working group meetings at least 3 days prior to meetings.	At least 3 days prior to meetings.
	Meeting minutes and action items for working group meetings due to the COR and distributed to attendees, via posting to the IMS within two business days following the meeting.	Within 2 business days following the meeting
1.2.3 Coordinate timely responses to quick turnaround requests	Response memorandum on emerging matters for action, to the COR, within one to five business days of NCES request, as determined by the COR	To be determined in consultation with COR
	Response memorandum on emerging matters for action, to the COR, within one to four hours of NCES request, as determined by the COR	To be determined in consultation with COR
1.3.1 Assist NCES in developing an R&D agenda	Annual R&D planning meeting to include panel of experts	Between October and mid-December annually
	Initial NAEP R&D agenda to COR by December 31. Annual updates to the agenda by January 30.	By December 31. Annual updates to the agenda by January 30.
	Annual R&D review meeting to occur in September.	September annually
	1 meeting annually with panel of experts and practitioners, and NCES-selected NAEP contractors, to set NAEP R&D priority areas.	November annually
	Draft meeting agenda to the COR at least 21 days prior to expert panel meetings.	At least 21 days prior to expert panel meetings.
	Final meeting agenda due to the COR 10 days prior to the meeting, and once approved by COR, disseminated to attendees.	Final agenda delivered 10 days prior to the meeting and distributed on the date of COR approval
	Documentation and dissemination of expert panel deliberations and resultant Alliance recommendations.	Within 2 business days following the meeting
	Memoranda on approaches to operationalizing recommendations.	To be determined in consultation with COR
	Weekly report to COR on status of action items and implementation of recommendations.	Weekly
1.3.2 Coordinate NAEP special studies	As requested by the COR, develop methodology recommendations for the conduct of special studies	To be determined in consultation with COR
	Develop timelines for special studies and ensure integration into NAEP’s Master Schedule	To be determined in consultation with COR
	Provide weekly updates to COR on the progress of special studies	Weekly
	Make recommendations for procedural changes to special studies, where necessary	To be determined in consultation with COR
	Deliver the final study report to NCES on a timeline negotiated with the COR	To be determined in consultation with COR
1.3.3 Maintain an R&D task order fund	Submit to COR upon request, proposals for the conduct of TOC studies	To be determined in consultation with COR
	Develop timelines for TOC studies and ensure integration into NAEP’s Master Schedule	Upon submittal of study proposal
	Provide weekly updates to COR on the progress of TOC studies	Weekly
	Make recommendations for procedural changes to TOC studies, where necessary	To be determined in consultation with COR
	Deliver the final study report to NCES on a timeline negotiated with the COR	To be determined in consultation with COR
1.3.4 Coordinate development of information collection requests (ICR) for OMB approval	Request package to OMB for three-year NAEP system clearance, to the COR, on a timeline negotiated with the COR.	To be determined in consultation with COR
	Request package to OMB for NAEP annual clearance, to the COR, on a timeline negotiated with the COR, which should be a minimum of 4 months prior to printing of booklets for pilot, field, or operational assessment, as needed.	To be determined in consultation with COR
	Up to 8 ICR packages, per year, for submittal under the NCES generic clearance, delivered to the COR.	To be determined in consultation with COR
	ICR guidance to consultants and contractors working on NAEP studies.	Within two weeks of NCES approval of project
	Verification of the accuracy and completeness of statements contained in the ICR package.	Upon delivery to COR
	Draft response memorandum on OMB passback, for each ICR, within 2 to 24 hours of receiving passback from OMB via the COR.	within 2 to 24 hours of receiving passback from OMB via the COR.
	Full documentation of ICR development files and for catalogue of all notices of action (NOA) received from OMB, on the IMS.	Ongoing from beginning of ICR preparation to receipt of NOA
	Logs of review comments and NCES responses to OMB passback, maintained on IMS	Within 24 hours of receipt of review comments and NCES responses
1.3.5 Manage production of technical documentation on the Web	Develop timelines for TD report preparation, and ensure integration into NAEP’s Master Schedule	Within 60 days of contract start date
	Provide weekly updates to COR on the progress of TD report preparation	Weekly
	Make recommendations for procedural changes to TD report preparation where necessary	To be determined in consultation with COR
	Draft TD report for Division review, to the COR on a timeline negotiated with the COR, for each assessment cycle.	To be determined in consultation with COR
	Draft for NCES Center wide review of TD, submitted on a schedule agreed upon with the COR, for each assessment cycle.	To be determined in consultation with COR
	Report of results of TD, as directed.	As directed by the COR

Page &P of &N

DAR Table of Deliverables

Table of Deliverables NAEP Alliance Contract 2013-2017
Task Number	Deliverable	Due Date
2 DESIGN, ANALYSIS, AND REPORTING
2.1 Designing the Assessment
2.1.1 Develop overall design plan	An overall design plan covering all subjects to be assessed during the contract period or that require developmental work for future assessments.	Annually
2.1.2 Create Operational Design Plans	An operational design plan that guides all NAEP assessment activities during a single year. The operational plan shall describe requirements for reporting, scoring, analysis, special studies, and survey questions for those assessments. Plans shall include matrix sampling booklet design and a booklet spiral map to ensure equal coverage for all blocks.	Annually
2.2 Analyses Leading to Reporting Results of the Assessments
2.2.1 Develop Analysis Plans	An Analysis Plan that supports reporting all required metrics, subscale reporting, and, where appropriate, reporting specific competencies. In the case of special studies that focus on psychometric analysis of item-level data the contractor shall describe in the Analysis Plan the types of psychometric modeling to be conducted and how hypotheses will be tested. In the case of operational assessments, the contractor shall develop the Analysis Plan to be consistent with the Design Plan, including descriptions of the statistical methodology that will be used to analyze the data.	On an schedule agreed upon with the COR
2.2.2 Pilot Test Item Analysis	Memoranda identifying problematic items, blocks, and performance scales for the operational assessment. The DAR contractor shall identify blocks containing items that show poor psychometric properties, and provide this information to the ID contractor.	On an schedule agreed upon with the COR
2.2.3 Scaling and Estimation	Final population ability estimates that incorporate preliminary and final sampling weights for item scaling. In addition, the DAR contractor shall estimate standard errors correctly from a sample that has been clustered and stratified.	On an schedule agreed upon with the COR
	A machine-readable version of item parameters	On an schedule agreed upon with the COR
2.2.4 Linking for Trend Maintenance and Cross-Grade Scaling	Linking of scale scores from each new assessment to the scales from earlier assessments. Test of the validity of (a) the Reading cross-grade scale in Main NAEP and (b) the Reading and Mathematics cross-age scales in Long-term Trend NAEP.	On an schedule agreed upon with the COR
	Notification to the COR of problems encountered	Upon encountering such problems
2.2.5 Estimating Population Characteristics and Standard Errors	Proficiency estimates and standard errors for the population and key subgroups	On an schedule agreed upon with the COR
	Standard errors for all parameter estimates	On an schedule agreed upon with the COR
	Electronic version of summary data tables, parameter estimates, standard errors, significance test results	On an schedule agreed upon with the COR
2.2.6 Linking to External Assessments	Design and conduct a linking study in 2015 between Grade 8 science NAEP and grade 8 TIMSS using methodology appropriate for linking by statistical moderation. Analyses to support linking NAEP with international assessments such as PISA and PIRLS	On an schedule agreed upon with the COR
2.2.7 Quick Turnaround Analysis	Analyses to support NCES in the timely analysis and reporting of NAEP data in response to inquiries from the Department of Education headquarters, Congress, the General Accounting Office, or from any other stakeholder in the government or private sector.	On an schedule agreed upon with the COR
2.2.8 Assessment Research and Development	A program of research designed to enhance and advance the psychometric and statistical methodology of NAEP	On an schedule agreed upon with the COR
2.3 Reporting Assessment Results
2.3.1 Reporting Assessment Results	Schedule of reporting activities	Submitted to the COR, in November of the year preceding data collection for the operational assessments
2.3.2 Develop Reports	Final copy of reports in format suitable for GPO printing	Submitted to the COR 2 weeks following NCES sign-off or public release, whichever occurs last.
	Printed copies of the Report Cards to be made available at the public release event, as requested by the COR. Appropriate chapters of technical documentation to be submitted to Alliance Coordination contractor.	Public-release date
2.3.3 Write State Report Generators for State-Level Assessments, and Trial Urban District Report Generators for TUDA Assessments	Electronic versions of all state or urban district reports, posted on NCES Web Site	Public-release date
2.3.4 Release Assessment Items from the Assessments	One set of released items and accompanying information for posting to the NAEP Web Site	Submitted to the COR, 6 weeks prior to initial public release date
2.3.5 Produce Restricted Used National/State/Urban District Data File	25 Copies of final restricted-use national/state data on CD-ROM	On an schedule agreed upon with the COR
2.1.1 Place NAEP Data on the Web Sites with Tools Suitable for Public Use	Maintenance of the NAEP Data Explorer (NDE) consistent with the data tool used for the 2010 assessments including graphic capabilities or provision of one of equal or better functionality and ease of use.	On an schedule agreed upon with the COR
2.4 Additional General Activities
2.4.1 Facilitating the Design and Analysis Committee	Final agendas	1 week prior to each meeting
2.4.2 Develop and Maintain Statistical Analysis Programs	Maintenance and updating of NAEP took kit, DESI program, and NDE or similar interactive data analysis tool.	On an schedule agreed upon with the COR

&CPage &P of &N

SDC Table of Deliverables

Task Number	Milestone	Deliverable	Due Date
1.1 Design and Select Samples	Draft sampling plans for each assessment, pilot test, and special study including selection of replacement schools, contingencies for nonparticipating jurisdictions and students, for COR approval. The draft 2015 sample design shall include information on the HSTS 2015 sample.		Eleven (11) months prior to the beginning of data collection.
		Final revised sampling plans for each operational assessment, pilot test, and special study including selection of replacement schools, contingencies for nonparticipating jurisdictions and students, for COR approval. The final 2015 sample design shall include information on the HSTS 2015 sample.	Six (6) months prior to the beginning of data collection.
1.2 Collect Data
1.2.1 Secure Participation of Schools and Students		Schedule of data collection activities from development of training and field materials to administering the assessment for each operational assessment, pilot test, and special study maintained on the IMS and submitted electronically, for COR approval.	60 days prior to the beginning of data collection.
		Three copies (3) of all materials sent to schools to encourage school participation and the inclusion of all students capable of taking the assessment and maintained on the IMS, for COR approval.	30 days prior to dissemination.
		Log of schools selected and updates on agreements to participate.	Available on demand throughout the assessment cycle and then destroyed within 60 days of data collection completion.
		State and school participation information, updated continuously throughout the assessment cycle and posted on the IMS.	Updated continuously throughout the assessment cycle and posted on the IMS.
		Log of data collection issues maintained in State Participation Tracking System.	Maintained on a schedule agreed upon with the COR.
		Copies of all materials sent to refusing private schools, including letters and other communications, for COR approval.	30 days prior to data collection.
		School refusals and contacts related to refusals log, available to COR on demand.	Available to COR on demand, throughout the assessment cycle and then destroyed within 60 days of data collection completion.
1.2.2 Coordinate Parental Notification Process		Documentation of parental notification and parental refusals, maintained through the MySchool website.	On a schedule agreed upon with the COR.
1.2.3 Select and Prepare Staff		Application materials for potential job applicants delivered to the NAEP Web contractor.	30 days after contract start date and updated annually.
		Five (5) copies of training manuals and ancillary materials for field staff training for COR approval.	30 days prior to training.
		Schedule for training field staff submitted to COR and posted to IMS.	90 days prior to beginning of training cycle.
1.2.5 Collect Assessment Data		Revised schedule of activities for COR approval.	60 days prior to the beginning of the assessment cycle.
		Data collection log with schools assessed, problems, irregularities and complaints maintained in the IMS throughout the assessment cycle.	Maintained in the IMS throughout the assessment cycle.
		Schedule for receipt and distribution of assessment materials for COR approval.	60 days prior to distribution.
		Materials transmittal log of activities including problems that arose and solutions developed to be maintained on the IMS throughout the assessment cycle.	Maintained on the IMS throughout the assessment cycle.
		Data collection log of activities including when schedules were met, where delays occurred, what problems arose and what solutions were developed, to be maintained on the IMS throughout the assessment cycle.	Maintained on the IMS throughout the assessment cycle.
		Accommodations, data security, and complaint logs including problems and resolutions, to be maintained on the IMS throughout the assessment cycle.	Maintained on the IMS throughout the assessment cycle.
		Log of makeup sessions including problems that arose and the solutions developed to be maintained on the IMS throughout the assessment cycle.	Maintained on the IMS throughout the assessment cycle.
1.2.6 Post-Assessment Activities		Schedule for transmitting collected data to the MDPS and DAR contractors for COR approval.	60 days prior to data collection.
		Materials transmittal log of activities including problems that arose and solutions developed to be maintained on the IMS throughout the assessment cycle.	Maintained on the IMS throughout the assessment cycle.
1.3 Generate Sampling Weights
1.3.1 Prepare Sampling Weights		Schedule for preparation of preliminary and final sampling weights for COR approval.	60 days prior to delivery of weights to DAR contractor.
		Data files of preliminary and final sampling weights needed for analyses to the DAR contractor.	On a schedule agreed upon with the COR.
		Non-response bias analysis report for COR approval.	60 days after the delivery of the final sampling weights to DAR contractor.
		Meeting with NCES to present results of quality control checks of sampling and weighting.	Within 60 days after the delivery of the final sampling weights to the DAR contractor.
1.3.2 Prepare Variance Estimation Weights and Stratum Codes		Procedures and schedule for the preparation of variance estimation weights and coding for transmittal to the DAR contractor for COR approval.	Maintained on the IMS on a schedule agreed upon with the COR.
		Data files of variance estimation weights and coding to the DAR contractor.	On a schedule agreed upon with the COR.
1.4 Conduct and Report the High School Transcript Study
1.4.1 Designing the Study		Design plan for the 2015 HSTS that includes required sample sizes for the proposed analyses of transcript data.	Due November, 2013.
1.4.2 Conducting the Study		A HSTS data collection plan.	Nine (9) months prior to data collection.
		A HSTS analysis plan.	Six (6) months prior to data collection.
		Schedule of reporting activities submitted to the COR.	Within six (6) months of data collection completion.
		Memo on the final transcript coding procedures including information on updates to the CSSC.	Within six (6) months of data collection completion.
		Report outline with key data findings.	Within eight (8) months of data collection completion.
		Report for NCES Division review.	Within nine (9) months of data collection completion.
		Report for NCES review (post-Division review).	On a schedule agreed upon with the COR.
		Report for NCES Commissioner’s review.	On a schedule agreed upon with the COR.
		Final Report Card and Highlights report.	Within twelve (12) months of data collection completion.
		Release materials (e.g., PowerPoint slides, Questions & Answers documents).	Within twelve (12) months of data collection completion.
1.4.3 Prepare and Maintain the Data Files		Restricted-use data user’s guide and codebook and technical report.	Within fifteen (15) months of data collection completion.
		Report on disclosure analysis and data confidentiality procedures for the Institute of Education Sciences (IES) Disclosure Review Board.	Within six (6) months of data collection completion.
		Master copy of public use data for use with online data analysis tool.	Within twelve (12) months of data collection completion.
		Master copy of the restricted-use data file.	Within twelve (12) months of data collection completion.

Sheet4

Task	Deliverable & Specifications	Milestones	Completion/Due Dates
3.1 Cognitive Item and Scoring Rubric Development
Subtask 3.1.1 Developing a cognitive development plan	· Development Plan shall include:	· Draft plan with actual calendar dates due no later than six weeks prior to developing item replacements or as negotiated with the COR.	· Final plan due no later than three weeks prior to developing item replacements or as negotiated with the COR.
	- psychometric data, summary tables that show the distribution of items by objective, by item type, by item difficulty, and an annotated set of comments regarding the evaluation of each group of items;
	- the number and types of items that will be developed with rationales for development;
	- schedule for seeking approval of such things as passages and topics to be submitted to include projected numbers;
	- training plan for item writers to include a description of the qualifications of item writers, and training/monitoring and evaluation procedures,
	- plans for cognitive interviews/tryouts, etc. of new item types,
	- plans for obtaining copyrights for new passages/images, and
	- schedule of activities for subtasks 3.1.2 through 3.1.7.
		· A schedule of development created in collaboration with DAR/MDPS/ WTC due to the COR at the start of item development or as negotiated with the COR.
Optional Subtask 3.1.1.1 Implementation plan for assessments with new or updated frameworks	· Implementation Plan shall include:	· Draft plan with actual calendar dates due within one month of receipt of draft frameworks and specifications from Governing Board or as negotiated the COR.	· Final plan with actual calendar dates due within three weeks prior to developing item replacements or as negotiated with the COR.
	- high-level overview of the framework and specifications;
	- key questions and issues left unanswered by the framework and specification documents;
	- measurement model used to guide assessment design, item development, administration, scoring, reporting, and release plans;	· A schedule of development created in collaboration with DAR/MDPS/ WTC due to the COR three weeks from the receipt of a new framework/updated framework or as negotiated with the COR.
	- preliminary and long-term research needed;
	- item development procedures (item writer training, cognitive interviews; item quality reviews, etc.);
	- administration considerations;
	- scheduling and coordination;
	- analysis and scoring; and
	- technical documentation and reporting.
Subtask 3.1.2 Design cognitive items and scoring rubrics	· Approved Passages and Task Concepts:	· Proposed initial passages or task concepts submitted to NCES within in four weeks of receipt of NCES acceptance of task 3.1.1 development plan or task 3.1.1.1 implementation plan.	· Each year by June 15 or according to schedule as negotiated with the COR in task 3.1.1
	- Sufficient number of passages or task concepts. (Submitted Reading passages must include information on the number of words and estimated readability levels.)
		· Passages and task concepts submitted to NCES for Governing Board review due within two weeks of NCES receipt of initial proposed passages or task concepts or as negotiated with the COR.
		· Due within two weeks of the 2nd Standing Committee meeting for pilot items or as negotiated with the COR.	· Final passages and task concepts due within two weeks of receipt of NCES acceptance or as negotiated with the COR.
		· Due within two weeks of the 2nd Standing Committee meeting to select post-pilot items or as negotiated with the COR.
		· Due within two weeks of the 2nd Standing Committee meeting for pilot items or as negotiated with the COR.
			· Due within two weeks of the Governing Board approval of the proposed pilot items or as negotiated with the COR.
		· Due within two weeks of the 3rd Standing Committee meeting to select post-pilot items or as negotiated with the COR.
	· Sufficient numbers of items and scoring rubrics to fulfill the design specified in the development plan		· Final due within four weeks of the Governing Board review of the proposed operational blocks or as negotiated with the COR.
			· Final due within three weeks of the Governing Board review of the proposed pilot items or as negotiated with the COR.
	· Documentation of the rationales for the distracters of multiple choice/selected-response items
			· Due within three weeks of the Governing Board review of the proposed operational blocks or as negotiated with the COR.
	· Item profile record documenting the history of the items
· Develop skill-based cognitive item variable names	· Skill-based variable names for all subject-specific cognitive items	· Draft Item variable names of cognitive items as part of item classification information found in the 1st standing committee review books.	· Final item variable names of cognitive items as part of item classification information found in the 2nd standing committee review books.
· Train cognitive item writers	· Training schedule and training agenda	· Draft schedule and agenda due six weeks prior to the start of training or as negotiated with the COR.	· Final schedule and agenda due four weeks before start of training or as negotiated with the COR.
		· Draft summary report of training due within two weeks of end of training or as negotiated with the COR.	· Final summary report due two weeks after receipt of NCES comments or as negotiated with the COR.
	· Summary report of training activity
· Collaborate with MDPS/SDC and provide scoring support to MDPS	· Final training materials for pilot items		· Final training materials due four weeks after training or as negotiated with the COR.
	· Final scoring materials/documentation for pilot items, probe, and new operational constructed-response items		· Final item scoring materials/ documentation for to NCES four weeks after training or as negotiated with the COR.
		· Draft scoring materials/ documentation for pilot items, probe, and new operational constructed-response items due 4 weeks prior to scoring.
	· Master copy of the final pre-scoring guides and training sets		· Final master pre-scoring guides and training sets four weeks after scoring or as negotiated with the COR.
			· Yearly report due four weeks after the end of scoring of each subject or as negotiated with the COR.
	· A yearly report evaluating the quality of the rubrics and items with suggestions for improvements for each subject area in the field.		· Action Item report of items due two weeks after scoring debriefing or as negotiated with the COR.
	· Action Item report of items, e.g., issues identified during the scoring debriefing that are relevant to item development.		· Report of results of the actions to ameliorate/remedy the issues identified in the Action Item report due as negotiated with the COR.
	· Report of results of the actions to ameliorate/remedy the issues identified in the Action Item report.	· Annotated outline of yearly report due three weeks prior to scoring.	· Final Frequently Asked Questions due within two weeks of receipt of SDC request for training materials or as negotiated with the COR
	· Frequently Asked Questions for cognitive items
· Acquire copyright licenses	· Copyright Releases for Text and Artwork: Report documenting the status of all copyright permissions active in NAEP, the associated costs, and the schedule for renewals. (All copyright permissions shall be obtained before configuring items into blocks.)	· Due within four weeks of the MDPS printing planning meeting and Web uploading planning meeting for computer-based assessment.	· Due on July 15th of each year or as negotiated with the COR.
· Conduct cognitive interviews and/or small- scale pilot testing	· Detailed plans for conducting cognitive interviews/item tryouts/small-scale pilot testing	· Draft plans due six weeks before Forms Clearance submission or as negotiated with the COR.	· Final plans due two weeks after receipt of NCES comments.
	· Report of findings to include a description of the participants, detected problems, and sources of difficulties with plans for revising and improving problematic items for the pilot test.	· Annotated outline of report due to the COR two weeks prior to conducting cognitive interviews or as negotiated with the COR.
			· Final report of findings due two weeks after receipt of NCES comments or as negotiated with the COR.
		· Draft report of findings two weeks after completion of cognitive interviews/tryouts/small-scale pilot testing or as negotiated with the COR.
· Conduct bias and sensitivity review	· Report of findings and plan of action on an item-by-item basis after each analysis and review.	· Draft report of findings and plan of action due within two weeks of completion of analysis and review or as negotiated with the COR.	· Final report of findings and plan of action due within two weeks after receipt of NCES comments or as negotiated with the COR.
· Determine construct validity of cognitive items	· Evaluation report documenting that all items in the operational assessment meet the standards of construct validity described in the item development plans submitted in conjunction with the post-operational item quality review	· Draft outline of evaluation report delineating on a schedule negotiated with the COR	· Final evaluation report due 2 weeks after receipt of NCES comments or on a schedule negotiated with the COR.
Subtask 3.1.3 Configure items into assessment blocks	· Memo based on DAR block design documenting the choice, placement, and rationales for the selection of items to fulfill the within block assembly plan to include psychometric properties of individual items, block-level summary data of new blocks, and item pool summary data of the new and trend blocks combined.	· Memo due four weeks before block assembly or as negotiated with the COR	· Assessment blocks due two weeks after receipt of NCES comments on the documentation memo or as negotiated with the COR.
	· Assembled assessment blocks
Subtask 3.1.4 Prepare and certify assessment booklets	· Certified assessment booklets to include Spanish bilingual and Puerto Rico assessments.	· Schedule for the review and acceptance of certified booklets due to NCES 16 weeks before the negotiated print date or upload date for computer-based assessments or as negotiated with the COR.	· Final ID contractor certified assessment booklets due 4 weeks before the negotiated print date or upload date for technology-based assessments or as negotiated with the COR.
			· Final record of all changes/modifications posted on the IMS signed off by DAR, MDPS, Web, and ID contractors within one week of uploading for computer-based assessments and printing for paper-pencil assessments.
Subtask 3.1.5 Translate cognitive items into Spanish	· Record of item development documenting the quality of set of items translated into Spanish for the bilingual and Puerto Rico versions of the relevant assessments (bilingual mathematics, civics, geography, U.S. history and science, and mathematics for Puerto Rico). Record of item development shall document the characteristics of items and rubrics that may cause difficulty and recommended modifications and resolutions.	• Record of item translations due to the COR one week after the resolution of comments/revisions.
	· Translated assessment blocks
			· Translated assessment blocks due to the COR three weeks after translation committee meeting or as negotiated with the COR.
Subtask 3.1.6 Post-data collection cognitive item quality reviews	· Technical report documenting the design, development, psychometric properties, and target development areas for each operational assessment.	· Schedule and agenda for the in-person meeting due to the COR six weeks before the start of the item development for replacement blocks, or as negotiated with the COR.	· Final report due to the COR two weeks after the meeting, or as negotiated with the COR.
Subtask 3.1.7 Determine cognitive items to release/retire	· Item release plans recommending blocks to release with supporting (e.g., rationales, psychometric, and qualitative documentation).	· Draft item release plan due to the COR at the start of item development or as negotiated with the COR.	· Final release plan due to the COR two weeks after receipt of review comments from the Governing Board or as negotiated with the COR.
			· Released items for the NAEP Questions Tool as negotiated with the COR and Web contractor.
	· Released items for the NAEP Questions Tool
Task 3.2 Developing survey questionnaires
Subtask 3.2.1 Evaluate existing question pool	· Report evaluating the quality of survey question clusters	· A review plan that includes criteria for evaluating the quality of existing questionnaire variables and a review schedule of item clusters for the entire 5-year contract period with actual calendar dates due to the COR five weeks after contract award.	· Due on a schedule agreed upon with the COR that coordinates with NAGB meetings or as negotiated with the COR.
			· Due on a schedule negotiated with the COR.
	· Yearly update of the initial plan that incorporates lessons learned regarding the system in place and recommended modifications.
Subtask 3.2.2Prepare long-range core and subject-specific survey question development plans	· Long-range plans describing how the current survey questions, alternative sources of data and administrative record data fit together	· Schedule for the overall cycle of the survey questions development tasks due to the COR two weeks after contract award or as negotiated with the COR.	· Due to the COR two weeks after receipt of approval of the development schedule for the overall cycle or as negotiated with the COR.
		· Annotated outline due six weeks after contract award or as negotiated with the COR
	· Issues papers to describe subject-specific issues.	· Draft issues paper due four weeks after receipt of NCES comments on outline or as negotiated with the COR.
			· Final issues papers due two weeks after receipt of NCES comments or as negotiated with the COR.
Subtask 3.2.3 Developing new survey questions
· Conducting cognitive interviews	· Detailed plans for conducting cognitive interviews/question tryouts	· Draft plans due six weeks prior to Forms Clearance submission or as negotiated with the COR.	· Final plans due two weeks after receipt of NCES comments.
		· Annotated outline of report due to the COR two weeks prior to conducting cognitive interviews or as negotiated with the COR.
	· Report of findings to include a description of the participants, detected problems, sources of difficulties with plans for revising and improving problematic items for the pilot test.		· Final report of findings due two weeks after receipt of NCES comments or as negotiated with the COR.
		· Draft report of findings two weeks after completion of cognitive interviews/tryouts or as negotiated with the COR.
· Prepare and certify the questionnaires	• Certified survey question booklets to include Spanish bilingual and Puerto Rico assessments.	· Schedule for the review and acceptance of certified booklets due to NCES sixteen weeks before the negotiated print date or upload date for computer-based assessments or as negotiated with the COR.	· Final certified survey question booklets due four weeks before the negotiated print date or upload date for computer-based assessments or as negotiated with the COR.
			· Final record of development for all changes/modifications posted on the IMS signed off by DAR, MDPS, Web, and ID contractors within one week of uploading for computer-based assessments and printing for paper-pencil assessments.
Subtask 3.2.5 Translate survey questions.	· Record of item development documenting the translation of questions translated into Spanish for the bilingual and Puerto Rico versions of the relevant assessments (bilingual mathematics, civics, geography, U.S. history and science, and mathematics for Puerto Rico). The record shall document the characteristics of questions and response options that may cause difficulty and recommended modifications and resolutions.	· Record of question development translation due to the COR one week after the resolution of comments/revisions.
	· Translated assessment blocks
			· Translated question blocks due to the COR three weeks after translation committee meeting or as negotiated with the COR.
Subtask 3.2.6 Post-data collection survey questionnaire quality reviews	· Technical report documenting the design, development, statistical properties, and target development areas for each operational assessment.	· Schedule and agenda for the in-person meeting due to the COR six weeks before the start of the question development, or as negotiated with the COR.	· Final report due to the COR two weeks after the meeting, or as negotiated with the COR.
Task 3.3 Item/survey question reviews
· Subject- specific Review Committees and Translation Review Committee	· Minutes of each meeting that includes an annotated list of all recommendations made during the meeting and actions that shall be taken.	· Annotated outline of meeting minutes due within three weeks of contract award or as negotiated with the COR.	· Minutes of each meeting due to the COR no later than two weeks after each meeting.
		· List of proposed members for each subject standing and translation committees to include information related to affiliation, location, and area of expertise due to the COR within eight weeks of contract award or as negotiated with the COR.
		· An agenda for each meeting of either the entire group or subgroups at least three weeks before each meeting.
		· A yearly schedule of all meetings of each standing committee that coordinates with the overall item development schedule and shall be updated monthly.
			· Annually submitted in June of each year to include in the yearly item development calendar or as negotiated with the COR.
· Prepare cognitive item review materials and address comments	· Review materials to include an agenda, list of participants and affiliations, introduction, content information, items/rubrics, and for post-pilot reviews psychometric data.	• Draft mock-up of the review materials due to NCES five weeks before the scheduled item reviews or as negotiated with the COR.	· Final revised review materials due one week after receipt of NCES comments or as negotiated with the COR.
	-Review materials for the Governing Board formatted to allow the members to review items electronically.		· Final Governing Board review materials due two weeks before the scheduled review or as negotiated with the COR.
	-Review materials for standing committees and NCES (20 copies); translation committee (10 copies) State/Tuda Item Reviews (110 copies); and mathematics expert panel (7 copies).
	· Schedule of all item reviews (Standing Committees, State Item Reviews to include meeting sites and Governing Board)
	· Meeting minutes to include documentation of issues raised and discussed, and actions to be taken.
			· Final schedule of item reviews due on a schedule agreed upon with the COR.
		· Draft schedule of item reviews due on a schedule agreed upon with the COR.
· Conduct cognitive item reviews
		· Annotated outline of meeting minutes due within three weeks of contract award or as negotiated with the COR.
	· Interim item profile record of development (see task 3.1.2 for specification details). Format of item profile record shall be compatible with the IDTS.	-Due within two weeks after the 2nd Standing Committee meeting for pilot items or as negotiated with the COR.	· Final due within two weeks after receipt of NCES comments or as negotiated with the COR.
		· Due within two weeks after adjudication of comments from 2nd Standing Committee, State Item Review, Governing Board, and NCES.
			· Final item profile record of development due within two weeks after adjudication of comments from the 2rd Standing Committee, Governing Board, and NCES to select post pilot items or as negotiated with the COR.
· Prepare survey question review materials and address comments	· Review materials to include an agenda, list of participants and affiliations, introduction, content information, items/rubrics, and for post-pilot reviews statistical data (Sufficient number of review materials for standing committees, Translation committee, and NCES (30 copies)	· Draft mock-up of the review materials due to NCES four weeks before the scheduled item reviews or as negotiated with the COR.	· Final revised review materials due two weeks after receipt of NCES comments or as negotiated with the COR.
	-Review materials for the Governing Board shall be formatted to allow the members to review items electronically.
	-Content for the Office of Management and Budget (OMB) clearance documents
			-Final GB review materials due fifteen days before the scheduled review or as negotiated with the COR.
			-Final OMB clearance documents due six weeks before the scheduled submission to the Alliance coordinator or as negotiated with the COR.
Task 3.4 Develop and Maintain an Item Development Tracking System (ITDS)
	· Functional IDTS on the NAEP web system containing entire pool of NAEP assessment items (cognitive and questions) for both paper-pencil and technology-based assessments, scoring rubrics, survey questions, item level data (meta data), with access for all necessary Alliance contractors and NCES.	· Specifications document delineating the features including functionality and technical and platform requirements/ specifications of the IDTS due to COR within six weeks of award or as negotiated with the COR.	· Specification of the basic features due to COR within four weeks of award or as negotiated with the COR.
		· Within three months of receipt of COR approval of initial set of specifications, a prototype shall be presented to NCES for review.
		· Functional prototype due to NCES for testing as negotiated with the COR.
			· Functional prototype due to COR within eight months of contract award or as negotiated.
	· Annual report summarizing development, recommendations, and coordination activities with the Web and MDPS contractors to enhance/maintain the quality and functionality of the ITDS.
			· Annual report due to the COR within two weeks of the end of the fiscal year or as negotiated with the COR.

NSSC Table of Deliverables

	Table of Deliverables NAEP Alliance
	Task Number		Deliverable		Due Date
	5 State Service Center
	5.1.1 Orientation for New Coordinators		Draft orientation plan including curriculum, materials and training approaches. Included in annual report.		Within 30 days of the issuance of the contract and by November 1 every year there after.
	5.1.1 Orientation for New Coordinators		Final orientation plan including curriculum, materials and training approaches. Included in annual report.		Within 60 days of the issuance of the contract and by December 1 every year there after.
	5.1.1 Orientation for New Coordinators		Orientation workshop and evaluations including recommendations for improvement.		Within 30 days of the end of the workshop.
	5.1.2 Develop and Conduct Training
Activities		Draft annual plan for conducting coordinator trainings, to include plans, materials, procedures for all trainings. Included in annual report.		Within 30 days of the start date of the contract and then annually by November 1.
	5.1.2. Develop and Conduct Training
Activities		Final annual plan for conducting coordinator trainings, to include plans, materials, procedures for all trainings. Included in annual report.		Within 60 days of the start date of the contract and then annually by December 1.
	5.1.2.1 Conduct NAEP Report Card Pre-Release Workshops		Pre-release workshop evaluations including recommendations for improvement.		Within 30 days of the end of the workshop.
	5.1.2.2 Conduct Assessment Operations Training		Assessment operations training including recommendations for improvement.		Within 30 days of the end of the workshop.
	5.1.2.3 Conduct Professional Development Workshops		Professional development workshop evaluations including recommendations for improvement.		Within 30 days of the end of the workshop.
	5.1.3 Continuous Training to Coordinators		Draft annual plan for conducting continuous trainings of coordinators including materials, process, topic outline and presentors. Include in annual report.		Within 30 days of the start date of the contract and then annually by November 1.
	5.1.3 Continuous Training to Coordinators		Final annual plan for conducting continous trainings of coordiantors including materials, process, topic outlines and presentors. Include in annual report.		Within 60 days of the start date of the contract and then annually by December 1.
	5.2.1 Provide Technical Support		Monthly technical support report, outlining support requests, response time, and resolutions.		Monthly
	5.2.2 Respond to Information, Data and Service Requests		Monthly information, data and service requests report, outlining request, response time and resolutions		Monthly
	5.2.3 Support the NAEP Information Network		Draft annual plan for support of the NAEP information network, including resources and content of site.		Within 30 days of the start date of the contract and then annually by November 1.
	5.2.3 Support the NAEP Information Network		Final annual plan for support of the NAEP infomration network, including resources and content of site.		Within 60 day of the start date of the contract and then annually by December 1.
	5.2.4 Provide Individualized Support for each NAEP State and TUDA Coordinator		Monthly individualized support report to Coordinators, including support activities, timelines and resolutions.		Monthly
	5.2.5 Develop materials and processes to support state participation in NAEP.		Draft annual plan of materials and processes for support of states in participation in NAEP. Include in annual report.		Within 30 days of the start date of the contract and then annually by November 1.

This is the start of the file's text. The full file is on GovTribe.

File details come from the government source that posted it. Updated Nov 14, 2017.