Data Collection Guidance.docx

Draft: July 3, 2018

DATA COLLECTION, MANAGEMENT, AND REPORTING GUIDANCE

Goals

1. Set requirements that help implementers become more data-driven

2. Facilitate the flow of data from programs back to CSO to inform future analysis Data Sources Programs generally fall into two categories: research/analysis programs and non-research. Data collection is more extensive in research / analysis programs.

Data may be generated from program activities themselves and/or from program monitoring.

Common methods of generating data include surveys, focus groups, key individual interviews, direct observation, and document reviews.

Note: Direct observation: Watching, taking notes, and recording specific actions within a target community, such as communications, spatial interaction, or exclusion (SFCG Designing for Results, Chapter 12) Selection of Methodological Tools and Definitions Implementers will need to work with the relevant regional CSO team and APPL to select the appropriate methods and/or tools for data collection, and CSO will approve any questionnaires and methodologies, to include the Monitoring and Evaluation plan that is generally required after award.

CSO will collaborate with the implementer to determine:

· Data definitions (e.g. violent election = 10 + deaths falling within 6 months before or 3 months after elections)

· Data metrics (e.g. ???)

· Data disaggregation: What demographics will participant information include?

· Survey questions (i.e. How will the individual questions be phrased? What content will they cover?)

· Participant / interviewee selection criteria (i.e. How are you determining who is representative of their community? Who are key figures in the community?)

Data Collection Parameters Data collected should be responsive to the research question / program goal.

Note: This includes collecting data from the correct audience (e.g. if you’re looking at increases in religious tolerance, don’t just collect data from people who believed in religious tolerance before the program).

For monitoring purposes, baseline data should be collected prior to program implementation.

Demographic information collected should be tailored to the specific program:

· Usually relevant: age, gender, and town of current residence

· For town of current residence, provide a specific town (or if possible, latitude/longitude)

· Possibly relevant: ethnicity, religion, birthplace, education level, education type, income level Data needed by source type:

· Surveys: description of survey goal; what implementer conducted the survey; survey dates; method of contact; survey locations (if in person); target population; number of participants; participant selection criteria; relevant demographic information about participants; questions asked and question order; response options (if applicable); responses to questions

· Number of demographic questions partly depends on survey size – it might not make sense to have 10 demographic questions for a 10-question survey

· Focus groups: description of focus group goal; what implementer conducted the focus groups; focus group date; focus group location; target population; number of participants; participant selection criteria; relevant demographic information about participants; questions asked and question order; notes on responses

· If possible collect the full responses to the questions, but transcribing (and translating) those may be too much of a burden

· Key informant interviews: description of interview goal; what implementer conducted the interview; interview date; interview location; selection criteria for the interviewee; relevant demographic information; questions asked and question order; notes on responses

· If possible collect the full responses to the questions, but transcribing (and translating) those may be too much of a burden

· Observation: description of observation goal; what implementer conducted the observation; observation dates; observation locations; notes from observations, including any checklists or guidelines that guided the note-taking

· Document review: explicitly-stated research question; sources used, both in the text and as a bibliography at the end; date of review completion; what role the review played in program decision-making

· If the document review’s findings will be shared outside CSO, such as with Post, require an executive summary or one-pager summarizing the review’s findings Data Reporting Expectations As the NOFO language states, implementers should share raw data with CSO in a clear, structured manner to the extent that is possible, while preserving safety for participants.

· In the past some implementers have said they own the raw data, so this expectation must be clear.

· Some organizations will need more support than others with structuring their raw data in a format CSO can use, so it’s best to identify any support needs early.

For any program that involves collecting information about multiple units of observation (like people or CSOs) the ideal reporting tool is a spreadsheet with a row for every unit of observation and a column for every trait of the observation. That enables easier disaggregation based on traits. APPL can provide templates as needed.

· Example: a row for each participant in the focus group and columns for traits of those participants, like town of residence and gender

· Participants can be identified by a random ID number rather than by name All data should include a data dictionary (codebook) that explains what any codes or shorthand / imprecise terms refer to.

· Example code dictionary: for gender, 0=male, 1=female; for religion, 0=Buddhist, 1=Muslim, 2=Christian

· Example term dictionary: “Location” = city the participant lives in, “Household” = number of individuals living in the participant’s home