Q.23 sample validation work.doc

Long-term Measures for Parent Information Centers Program Measure Methods and Procedures (2009) Measure 2.1 of 1: The percentage of parents receiving Special Education Parent Information Centers services who report enhanced knowledge of IDEA rights and responsibilities.

This measure requires a two-step approach: 1) using existing data (National Parent Center Impact Survey-NCPIS) and creating a proxy measure; and 2) collecting new data that would validate the NCPIS results.

The NPCIS is conducted annually by each Parent Center. Using a standardized data collection instrument, each center randomly contacts 25 parents who received services and administers, via telephone, the survey. Because the centers are funded, in part, on the size of the population in their catchment area (from a high of over $600,000 to a low of $100,000), the percentage of parents from each Center included in the survey will likely vary.. To control some of this variation, we plan to weight the survey responses by center size.

Methods Definition

Step 1: National Proxy Estimate (using extant interview data) Add “yes” responses to items #1, #4, and #6 (all binary—1 or 0) from FY08 Impact Data Collection: Phone Intakes From Parents (NCPIS) survey database and divide by the total number of parents responding. Item #6 will be given a weight of 4 (as most critical to “demonstrating enhanced knowledge”), item #4 will be given a weight of 2 (next most critical), and item #1 will be given a weight of 1. Each response will be weighted by the size of the Parent Center.

Step 2: National Direct Estimate (using follow-up interview data) Select four Centers in each of the six regions (n=24 Parent Centers) that fairly represent the Centers in that region. The RPTAC will randomly select 15 parents of the 25 parents from that Center that responded to the original questions (n=360 parents). The RPTAC will contact each of the 15 parents selected and complete the Data Collection From Parents (new) survey. Questions 4 and 5 directly ask if parents have enhanced knowledge of IDEA rights and regulations.

Scoring Calculation Illustration Step1 National Proxy Estimate (using FY07 NCPIS telephone survey data)

Numerator: Number of parents responding “yes” to item #1+ (Number of parents responding “yes” to item #4) x 2 + (Number of parents responding “yes” to item #6) x 4

Denominator: Total number of parents responding to item #1 + (Total number of parents responding to item #4) x 2 + (Total number of parents responding to item #6) x 4

Using FY07 NCPIS telephone survey data:

2642 + 2223(2) + 2035(4) / 2712 + 2803(2) + 2743(4) = 78.9%

Step 2 National Estimate (new) This involves selecting PTI Centers and asking them to call 15 parents. Four Centers from each of the six regions (for a total of 24 Centers) will be selected to participate. The sample will be selected so that the size of the Centers is fairly represented in the sample. ICF Macro will provide each of the selected Centers with a way to select a random sample of parents that will allow for extra parents and therefore make it easier for each PTI Center to meet its quota of 15 parent interviews. The sample will consist of 360 parents and provide approximately a 95% confidence interval of +/- 5%. Parents would be asked some of the same questions as on the NCPIS survey but asked directly:

· Did the services you received enhance your knowledge of IDEA rights and regulations? and

· How would you rate how much your knowledge of IDEA rights and regulations was enhanced?

Enhanced Greatly Enhanced Somewhat Enhanced Very Little The estimates from the two methodologies will be compared.

Measure 3.1 of 1: The percentage of parents receiving Special Education Parent Information Centers services who promote scientifically or evidence-based practices for their infants, toddlers, children, and youth.

ICF Macro proposes a one-step approach to calculating this measure: collecting new data that would also validate the NCPIS results. (A two-step approach could be followed in the future if the NCPIS is supplemented with items inquiring about promoting evidence-based practices)

Method 1: National Estimate (new data) This involves randomly selecting PTI Centers and asking them to call 15 parents selected at random. Four Centers from each of the six regions (for a total of 24 Centers) will be selected to participate. The sample will be selected so that the size of the Centers is fairly represented in the sample. ICF Macro will provide each of the selected Centers with a way to select a systematic random sample of parents that will allow for extra parents and therefore make it easier for each PTI Center to meet its quota of 15 parent interviews. The sample will consist of 360 parents and provide approximately a 95% confidence interval of +/- 5%. Parents would be asked some of the same questions (Item #5?) as on the Pacer survey but asked directly:

· As a result of your contact with the Center, can you now promote evidence-based practices for your child and others?

By “evidence-based practices”, I mean things like [each center to insert examples that would be familiar to the parents they serve]

Yes

No

· How would you rate how well you believe you can promote evidence-based practices for your child and others?

Promote Very Well Promote Well Promote Poorly Promote Very Poorly