Attachment 1 Demonstration Test.pdf

Source Selection Information -- See FAR 2.101 and 3.104 – FOR OFFICIAL USE ONLY

1 Ver: October 2016

Attachment 1 Demonstration Test

1. Discuss participant contact strategies (e.g., survey mode, number of contacts, use of incentives) you have used in large scale surveys meant to provide population level estimates and would recommend to JAMRS for the type of efforts described in the RFP PWS, Section 4.1.1

Download the mock youth poll data file (Excel file), annotated variable list (Word document), and analysis note (Word document) provided and read carefully before conducting the following analyses.

2. Using the data provided, please create a table of weighted proportions for each response option, missingness or error codes for each of the four variables listed below. In each of the four tables, please provide splits by a) race/ethnicity, b) gender, c) race/ethnicity by gender.

1. Propensity

2. Consideration

3. Knowledge

4. Recruiter

3. For the same four variables listed above, make a one page infographic focused on gender differences for these variables. The infographic should visually depict the male and female weighted percentages for each variable using the following response options as input to the proportion:

a) Propensity – use the proportion that respond ‘definitely’ or ‘probably’ (i.e., definitely or probably combined)

b) Knowledge – use the proportion that responded? 8-10.

c) Consideration – display each of the three response options proportions (do not display missingness percentages).

d) Recruiter – Use the proportion who answered ‘yes’.

Note. When preforming the above calculation, if present, remove the cases that are coded as ‘multiple responses’ for a given variable from the calculation. Cases with missing data for a given variable should remain in the denominator of the equation.

4. Create a briefing slide deck (maximum ten slides) describing how the variables: ‘knowledge’, ‘consideration’, ‘Good_pay’, ‘Like_me’ and ‘recruiter’ each affect/predict propensity. If one variable predicts propensity notably more than the other variables, also describe how the remaining variables (not including propensity) affect the variable that best predicted propensity in your first analysis. Be sure to use weighted data in your analyses. Craft a non-analyst audience-friendly brief that clearly describes the impact of these variables on propensity and makes clear recommendations related to increasing propensity. Remember, this data is fake created for this exercise. Include a supplement document that describes the methodology you used for any calculations.

2 Ver: October 2016

DATA ANALYSIS NOTE

Please assume the Demonstration Data was derived from a survey using a complex sampling approach and weighted to be representative of the population of interest; thus, the design effects (i.e., stratification and weighting) must be taken into account when estimating the variance of various statistics (e.g., means, proportions, correlations, regression weights).

Alternative approaches to finding effective n sizes based on design effects are required in certain situations for certain types of statistical testing. There are a number of different approaches to estimate the variability of (complex) parameters in complex surveys; two of the more common approaches are referred to as Linearization by Taylor series expansion and Replication, both of which take into account design effects and rely on readily available computer software to remove tedious hand calculations and adjustments.

Users are cautioned not to ignore the design features (i.e., stratification and weighting) of the data collection that was used for this survey in their analyses. Stratification, as done in the data collection for this survey, effectively allows the calculation of variance for a statistic that is based solely on within-stratum variance. This variance estimate is almost always smaller than the one that would be obtained if the data were treated as being collected using only simple random sampling. Ignoring the stratification will typically result in an over-estimation of the variance and bias any statistics calculated.

3 Ver: October 2016

DEMONSTRATION DATA

ANNOTATED QUESTIONNAIRE

NOTE TO USERS

This document is an annotated questionnaire designed to show variable names and data values for the Demonstration Test dataset.

SURVEY VARIABLES CONTAINED IN THE DEMONSTRATION DATASET

Variable Name: ID NOTE: This variable is a unique ID associated with each respondent.

Variable Name: WEIGHT NOTE: This variable is the final weighting variable. It should be used for all weighted analyses.

Variable Name: STRATA NOTE: This variable indicates the stratum each case belongs to in the complex sampling design that is associated with this dataset.

Variable Name: Propensity Question Text: In the next few years, how likely is it that you will be serving in the Military?

1=Definitely 2=Probably 3=Probably not 4=Definitely not -97=Multiple Response -99=Refused

Variable Name: Consideration Question Text: Before taking this survey, how much consideration had you given to the possibility of joining the Military?

1=Never thought about it 2=Gave it some consideration 3=Gave it serious consideration

4 Ver: October 2016

Variable Name: Knowledge Question Text: How knowledgeable are you about the US Military?

(Scale item; 1-10) 1=Not at all knowledgeable 10=Extremely knowledgeable

Variable Name: Recruiter Question Text: Have you ever spoken with a U.S. Military recruiter?

0=No 1=Yes

Variable Names: Like_Me Question Text: Please indicate the extent to which you agree or disagree that the following statements is true: The Military has people like me.

1=Strongly disagree 2=Disagree 3=Neutral 4= Agree 5= Strongly agree

Variable Names: Good_Pay Question Text: Please indicate the extent to which you agree or disagree that the following statements is true: The Military offers good pay.

1=Strongly disagree 2=Disagree 3=Neutral 4= Agree 5= Strongly agree

Variable Name: GENDER Question Text: What is your gender?

5 Ver: October 2016

1=Male 2=Female 3=Unknown

Variable Name: RACE_ETH NOTE: Assume this race/ethnicity variable is a computed variable that was created based on answers provided to multiple other survey questions regarding ethnicity and race.

1=Non-Hispanic White Only 2=Non-Hispanic Black Only 3=Hispanic 4=Non-Hispanic Asian Only 5=Non-Hispanic Other

End of Attachment 1