Data_Analysis_Note.docx

DATA ANALYSIS NOTE

Please assume the Demonstration Data was derived from a survey using a complex sampling approach and weighted to be representative of the population of interest; thus, the design effects (i.e., stratification and weighting) must be taken into account when estimating the variance of various statistics (e.g., means, proportions, correlations, regression weights). Alternative approaches to finding effective n sizes based on design effects are required in certain situations for certain types of statistical testing. There are a number of different approaches to estimate the variability of (complex) parameters in complex surveys; two of the more common approaches are referred to as Linearization by Taylor series expansion and Replication, both of which take into account design effects and rely on readily available computer software to remove tedious hand calculations and adjustments.

Users are cautioned not to ignore the design features (i.e., stratification and weighting) of the data collection that was used for this survey in their analyses. Stratification, as done in the data collection for this survey, effectively allows the calculation of variance for a statistic that is based solely on within-stratum variance. This variance estimate is almost always smaller than the one that would be obtained if the data were treated as being collected using only simple random sampling. Ignoring the stratification will typically result in an over-estimation of the variance and bias any statistics calculated.