Attachment 1 NCEE Report Planning Template.docx

NCEE EVALUATION DIVISION REPORT PLANNING TEMPLATE

OVERVIEW

Potential Title:

Publication Type:

· Report (Up to 15 pages and include an appendix with more detail. Reports also include a rationale to provide context supporting each main research question section. Report examples: implementation evaluation report; impact evaluation report)

Overarching Research Question/Hypothesis:

Identify the high-level research question that best articulates what the proposed publication is about. The question should be evaluative (is the program or policy accomplishing what was intended?) and framed as a hypothesis that you will test using the available data.

Examples:

· (Implementation report) To what extent do state and district practices align with IDEA’s goal of appropriately identifying children with disabilities, including finding children with suspected disabilities, evaluating them to determine their eligibility for IDEA services, and detecting and addressing disparities?

· (Impact report) Does adding a set of personalized text messages to regular Educational Opportunity Centers' (EOCs’) informational services boost clients' Free Application for Federal Student Aid (FAFSA) completion and college enrollment rates?

Rationale for Publication: Why this topic/overarching research question is important

Describe in one or two paragraphs (1) the educational problem the specific policy or practice being evaluated is trying to address, (2) why decision makers should care about it (what is at stake?), (3) how the specific policy or program being evaluated is expected to address the educational problem, and (4) how the study will support decision makers in their efforts to address the problem. This motivation should ‘set-up’ the findings presented. This means that the text needs to motivate each of the main research questions listed below.

Main Research Question 1:

Include a cohesive set of high-level main research questions (between 2-5) that when taken together answer the overarching research question. These questions should be evaluative and articulated in such a way that they can be answered with one sentence, that will serve as report headlines/key policy relevant findings.

Examples:

· (Implementation report) To what extent are states and districts undertaking broad efforts to find children with suspected disabilities? (A report headline answering this question might read: States and districts reported broad efforts to find children with suspected disabilities as encouraged by IDEA, with less emphasis on intensive approaches for younger children.)

· (Impact report) Does providing personalized text messages to EOC participants increase the likelihood of participants enrolling in postsecondary education? (A report headline might read: The Text Ed messages did not increase college enrollment among EOC clients.)

Research Question 1 Rationale:

In one or two paragraphs, lay out the policy relevance of answering the first main research question. Include motivation for why the specific topics, outcomes, indicators, and/or analyses that will be presented to answer the question are important. Avoid simply repeating the motivation provided in the rationale for publication. (A report will always include text to help motivate the main questions.

Examples:

· (Implementation report) Explain (1) what is meant by finding children with suspected disabilities and why it is important, (2) the key practices IDEA encourages states and districts to undertake to find children with suspected disabilities, and (3) why understanding whether states and districts are using these practices is relevant for informing federal policy.

· (Impact report) Explain why the personalized text messaging strategy was intended to increase the likelihood of EOC participants enrolling in college

Sub-questions that will be answered to address the main research question 1:

· Sub-question 1a. The sub-questions taken together will answer main research question 1. The rationale for the main research question will make clear why the answer to these specific questions matter. In a report, the answers to these questions would appear as policy relevant findings in bullet form (subfindings to the main research question).

Examples: To what extent did states intensively coordinate with the various agencies that may be important to identifying children with suspected disabilities? (The finding might read: Most states coordinated with a broad set of entities to find infants and toddlers who may need services, more often through conversations than through time-intensive efforts such as data sharing.)

Did the personalized text messaging strategy increase college enrollment in the year when clients hoped to matriculate?(The finding might read: Text Ed did not increase college enrollment within a year of when clients hoped to matriculate, nor did it have delayed effects.)

Data Sources, Element(s), and Analysis:

For each sub-question: (1) identify the specific data sources that will be used to answer it (e.g., student survey, 2023 Civil Rights Data Collection (CRDC), etc.); (2) describe the specific elements in the data sources that will be used to address it (e.g., specific student survey items, specific variables from CRDC, etc.); and (3) describe how the specific data elements above will be used analytically to arrive at an answer to the sub-question (e.g., tabulate and report the percentage of students with X or Y survey response, create an index measure that combines relevant student survey items, compute the difference between treatment and control students on X assessment, etc.)

· Sub-question 1b.

Main Research Question 2:

Research Question 2 Rationale:

Sub-questions that will be answered to address the 2nd main research question:

· Sub-question 2a.

Data Sources, Element(s), and Analysis:

(OPTIONAL) Main Research Question 3:

Research Question 3 Rationale:

Sub-questions that will be answered to address the 3rd main research question:

· Sub-question 3a.

Data Sources, Element(s), and Analysis:

(OPTIONAL) Main Research Question 4:

Research Question 4 Rationale:

Sub-questions that will be answered to address the 4th main research question:

· Sub-question 4a.

Data Sources, Element(s), and Analysis:

(OPTIONAL) Main Research Question 5:

Research Question 5 Rationale:

Sub-questions that will be answered to address the 5th main research question:

· Sub-question 5a.

Data Sources, Element(s), and Analysis: