RISEStudy - Example from Pre-Propsal Meeting.pdf

The Refugee Integration Survey and Evaluation (RISE) Year Five: Final Report

A study of refugee integration in Colorado

Colorado Department of Human Services Office of Economic Security Division of Employment & Benefits Colorado Refugee Services Program

For more information please contact:

Kit Taintor State Refugee Coordinator kit.taintor@state.co.us

Gary Lichtenstein, Ed.D.

gary@qualityevaluationdesigns.com

Last revised: February 22, 2016

About This Report

This is the fifth and final annual report (October 2014‐September 2015) concerning refugee integration in Colorado, conducted by Quality Evaluation Designs (QED) for the Colorado Refugee Services Program (CRSP) through a grant (PO IHA CRSP1123064) utilizing a portion of Wilson‐Fish program of the federal Office of Refugee Resettlement (ORR).

The views expressed in the report do not necessarily reflect those of the U.S. Department of Health and Human Services, Colorado Department of Human Services or the federal or state government.

The purpose of this work was to create a framework for understanding refugee integration in Colorado, and then evaluating refugee integration through a survey administered once each year to a cohort of refugees who were re‐surveyed for four consecutive years (2011‐12 through 2014‐15). The survey is called the Refugee Integration Survey and Evaluation (RISE). The purpose of the project was not to evaluate of any of the programs, services, or funding sources that assist the resettlement and integration of refugees. A summary of RISE can be found on the QED website: www.QualityEvaluationDesigns.com. Data were collected under the Solutions IRB protocol #1212316.

We thank CRSP staff, including Ms. Kit Taintor and Mr. Joseph Wismann‐Horther, for their commitment to the refugee community in Colorado and for their support of and assistance with this project. Mr. Paul Stein, the former CRSP director, provided critical vision and support for this work. Denver refugee Resettlement Agencies (also called voluntary agencies which are either affiliates or sub‐offices of the nine national Resettlement Agencies that have cooperative agreements with the U.S. Department of State), who administered the RISE survey the first year, were critical to our success. QED is grateful to research consultants Dr. Martin L. Tombari and Dr. Laurie Bennett, who participated in survey design and analysis the first four years of the project. QED is also deeply grateful to the refugees themselves who have participated in the survey, as well as in focus groups and interviews. Thanks to Nicole DuFour for her excellent administrative support, and Jami Loree for editing support. We are especially indebted to our magnificent network of Community Connectors, who remained connected to the survey population, served as interpreters for our qualitative research efforts, and provided us with countless meaningful insights along the way. Without them, this project would not have been possible.

Quality Evaluation Designs

Gary Lichtenstein, Ed.D.

Jini Puma, Ph.D.

Amy Engelman, Ph.D.

Maggie Miller, M.A.

i

Web: www.QualityEvaluationDesigns.com Email: gary@QualityEvaluationDesigns.com

Executive Summary

This report summarizes activities and findings from the Refugee Integration Survey & Evaluation (RISE) project. The purpose of the project was to assess refugee integration into U.S. society in the early years of resettlement by focusing on a single cohort of refugees during the first 3‐4 years of resettlement. Quality Evaluation Designs’ (QED) RISE research team developed and piloted a survey instrument that effectively assesses refugee integration, as well as conducted interviews and focus groups to understand the range of refugees’ experiences during integration. The survey and study were expressly not an evaluation of any of the programs, services, or funding sources that assist the resettlement and integration of refugees.

The RISE survey was first administered to nearly all adult refugees who arrived in Denver from Bhutan, Burma, Somalia, and Iraq during 2011‐12. Fiscal Year 2014‐15 was the fourth consecutive year of data collection and fifth and final year of the grant. In that year the RISE survey was completed by 327 refugees of the original 467 who took it at Baseline. This reflects 70% retention from Baseline.

Section II summarizes the framework of the RISE survey, which is based on Ager & Strang’s integration model. Unique features of the study are highlighted, including the fact that the study was longitudinal, mixed‐methods, and Community‐Based. Section II also provides an overview of RISE survey administration, qualitative interviews, and professional and community presentation activities during 2014‐15.

Data collection depended on a network of Community Connectors, most of whom arrived in the U.S. as refugees years earlier, who were embedded in the communities within which refugees in the study lived. Project success is due to Community Connectors’ commitment to the evaluation study and their extraordinary efforts tracking down respondents. Administrative oversight depended on a hybrid of a professional and a community‐based model, which was effective, although not necessarily efficient. Section III reviews lessons learned during implementation of the Community Connector model, success of which is attributed to: 1) respecting the expertise of the Community Connectors and being flexible to their ever‐changing schedules due to their holding multiple jobs, attending school, and raising children; 2) keeping warm, personal relationships with each Community Connector while maintaining a professional relationship that did not compromise the Project Manager’s supervisory role; and 3), supporting Community Connectors’ sense of ownership and investment in the project.

Qualitative data were collected each year of survey administration. During the first 2‐3 years, interview data sought to strengthen the survey by getting feedback on how respondents understood certain questions, as well as elaborating on survey‐related integration issues.

Beginning year 3, the focus shifted to gaining deeper understanding of refugees’ experiences.

During 2014‐15 (Year 4) interviews focused on challenges faced by refugees with low integration, how those challenges impacted Employment & Economic Sufficiency, Social Bridging & Bonding, Language & Cultural Knowledge, as well as other integration pathways (see ii iii

Section IV). Thirteen refugees who scored in the low integration range during the Year 4 survey administration were selected based on age (55+), gender, and whether they were not employed in order to care for children. Common themes across the sample included a high degree of social isolation, inability to speak English, and lack of agency to alter their integration level. Although respondents themselves evidenced low integration, their support of the family by providing child care, housework, and monthly Supplemental Security Income (SSI) checks facilitated family members along their integration pathways.

Survey data are reviewed in Section V. Although retention of the same refugees taking the survey across the four years of data collection was high overall (70%), poor retention from Somali and Iraqi respondents forced RISE researchers to discontinue sampling from those communities beginning in 2013‐14. Section V includes a brief analysis of findings related to each of the 10 pathways assessed.

Analyses of Overall Integration are particularly revealing. The cohort as a whole steadily progressed from low integration at Baseline, to medium integration at Baseline +1, to high integration at Baseline +2 and Baseline +3. At Baseline +3 (Year 4), 5.4% of respondents experienced low integration, 18.4% experience medium integration, and 76.2% experience high integration. Ninety‐eight percent (98%) of respondents increased from low to high or medium to high integration, but 2% slipped back a category. Men and women progressed at the same rate, but at Baseline +3 years, the means for both were both in the high integration range. In past years, those from Bhutan progressed towards integration at higher rates than those from Burma, but at B+3, both groups average in the high integration range. Age is a distinguishing factor, with those who are 55 and older progressing at significantly slower rates than younger refugees. Mean scores of 17‐34 year‐olds and 34‐54 year‐olds fall into the high integration range, while scores of those 55 and older fall into the medium range. Although as a group those 55 and older have progressed along integration pathways, they do so at a significantly lower rate, and those with low integration tend to be older.

Cluster analysis reveals common characteristics among those who display Above Average Integration and those who display Below Average Integration. Profiles of each group are nearly mirror images of one another, suggesting similar underlying factors affecting whether a refugee travels on an Average or Below Average pathway. Those with Above Average Integration score much higher on Employment and Economic Sufficiency, Social Bridging, and Language & Cultural Knowledge than those with Below Average Integration. This analysis reveals patterns of results that may be of particular interest to refugee service professionals.

Section VI reviews the due diligence of the survey instrument, which has included analysis of inter‐correlations among variables, cognitive interviews to assess item validity, interview and focus group validation of questionable items, and analyses of survey trends over time. QED has confidence that the survey reliably and validly assesses what it purports to.

Section VII summarizes quantitative and qualitative findings. The cohort as a whole has progressed steadily towards High Integration. At Baseline +3 years (Year 4), 92% of the cohort iv was employed over 30 hrs/week. Median family income increased from $700‐$999 at Baseline to $1,300‐$1,599 at B+3. Seventy‐six percent (76%) were covered by medical insurance, the highest proportion since the first year of arrival, when all refugees had Medicaid coverage. English language proficiency steadily improved, with 58% reporting they are fluent or able to speak English in social and work situations. Refugees report feeling safe within their homes (98%) and outside the home (97%). Nearly all state that they wish to become U.S. citizens (99%) and all (100%) have applied for a green card.

In spite of overwhelming successes, challenges remain. Those with disabilities and who are over 55 years‐old struggled with language and adjustment issues. Those who were older and who did not work often helped others in their family with childcare, cooking, and cleaning, but many were socially isolated. Sixty‐one percent (61%) of refugees reported that family income is too low to cover necessary expenses. Few (2.5%) were enrolled in degree programs. Fewer than half (48%) regularly spoke with people whose first language is English, a proportion that did not change much since Baseline.

Employment & Economic Sufficiency, Social Bridging, and Language and Cultural Knowledge are critical indicators of integration. Low scores on Housing and Social Bonding, and/or high scores on Health and Physical Well‐Being may be indicators of integration distress. Civic Engagement might be a long‐term proxy for integration. Safety and Stability had a consistently moderate correlation with critical integration variables. More study would be required to tease out the factors that influence that pathway.

Age is strongly tied to level of integration. Those who were 55 years and older had significantly lower Overall Integration scores than those in other age groups, and progressed at significantly slower rates. Many part‐time workers and unemployed elders we interviewed were often primary care givers, which enabled other family members to work full‐ or part‐time.

Section VIII outlines implications and suggestions for the future of the RISE instrument. Some questions raised that RISE could be used to answer include:

At what point do refugees’ answers to the questions in the survey instrument begin to resemble those of other immigrants and individuals who were not foreign born?

What does longitudinal integration look like, both quantitatively and qualitatively, for individuals in the host society (also known as the receiving community)?

What does longitudinal integration look like, both quantitatively and qualitatively, for different refugee populations in a community different from Denver, Colorado?

What program designs or modifications are informed by the findings and analysis of

RISE?

v

Possible follow up studies include:

Literacy Validation Study. Using a valid, standardized English language proficiency exam, compare self‐report items on RISE with actual proficiency of RISE respondents as well as newly arrived refugees.

Refugee Address Geo‐Mapping Study. By plugging (non‐identifying) refugee addresses into a geo‐mapping program, examine the patterns of mobility of refugees from B to B1 to B2 to B3. Look for correlations between neighborhood and the Safety & Stability pathway scores, as well as Overall Integration scores.

Safety and Stability Pathway Study. Explore factors that influence refugees’ feelings of safety and stability and how those feelings affect other integration pathways.

Self‐Efficacy Study. Explore how refugees’ perceptions of their ability to influence their environment affects their integration success. To what extent is sense of self‐efficacy influenced by location, conditions, and length of stay before resettlement?

Family Integration Study. To what extent is integration a family‐level variable? How does interdependence among family members affect integration outcomes?

Critical Pathways Study. Based on quantitative data collected over four years, are there key variables that best reflect overall integration? If so, can a shortened version of the RISE instrument be created, which could increase the capacity of refugee services agencies to administer it?

QED was charged with creating and administering a survey that assesses refugee integration from arrival through the first few years of resettlement. QED successfully created this instrument, which has the ability to track integration success of groups and individuals during the first few years of resettlement. We believe that the instrument can be a tool for policy, support services, and resource management. The survey confirms the multi‐faceted nature of integration. Study findings confirm that the policy and academic agenda to broaden the focus on refugee resettlement using an integration framework was forward thinking and right‐ headed. It is our hope that the RISE survey will be used by researchers and refugee service professionals to improve understanding and effectiveness of refugee integration through research, policy, and practice.

vi

Table of Contents

Revised & Resubmitted .................................................................................................................... i

About This Report ............................................................................................................................ i

Executive Summary ......................................................................................................................... ii

I. Can a Dream Have a Footprint?

II. Summary of Year 5 (2014‐15) Activities

III. Lessons from Data Collection

Four Years of Data Collection

An Evolving Model of Data Collection

Baseline Year: Working With the Resettlement Agencies The Community Connector Model Respect, Professionalism, and Ownership

IV. Interviews & Focus Groups: Exploring Experiences of Low Integrators

Experiences of “Low” Integration

Methodology

Findings

Elderly Evidencing Low Integration Experienced Significant Barriers Along Their Individual Integration Pathways, Yet Contribute Significantly to Their Family’s Integration Two Young Mothers & Two Different Integration Profiles

Conclusions

Implications

V. Summary of Survey Analyses

Assessing Pathway Variables Over Time

Four Years of Pathway Data

1. Employment & Economic Sufficiency

2. Education & Training

3. Children’s Education

4. Health & Physical Well‐Being

5. Housing

6. Social Bonding

7. Social Bridging

8. Language & Cultural Knowledge

9. Safety & Stability

10. Civic Engagement vii

Assessing Overall Integration Over Time

Integration by Demographic Group Summary

VI. How Effective is the RISE Survey?

The Survey Validation Process

What About Pathway/Domain Reliability and Validity?

Overall Integration

Survey Administration

VII. What Are We Learning About Refugees?

VIII. The Future of RISE

Appendix A—Interview Protocol Appendix B—RISE Survey (Recommended Version) Appendix C—RISE Codebook with Scoring

Figures

2.1. RISE Survey Administration Years

2.2. Ager & Strang’s Conceptual Framework Defining Core Domains of Integration

3.1. Low Integration Interview Sample

5.1. RISE retention from Baseline through B+3

5.2. Demographic Characteristics of the RISE Cohort Over Time

5.3. Frequencies for Employment Variables

5.4. Employment Relative to Education Level

5.5. Barriers to Employment

5.6. Frequencies for Family Income Categories

5.7. Frequencies for Education & Training Variables

5.8. Frequencies for Children’s Education Variables

5.9. Frequencies for Health & Physical Well‐Being Variables

5.10. Frequencies for Housing Variables

5.11. Frequencies for Social Bonding Variables

5.12. Frequencies for Social Bridging Variables

5.13. Frequencies for Language & Cultural Knowledge Variables

5.14. Frequencies for Safety & Stability Variables

5.15. Frequencies for Civic Engagement Variables

5.16. Changes in Pathway Scores Over Time

5.17. Correlations Between Pathways at B+3

5.18. Mean Integration Scores by RISE Administration

5.19. Frequency and Distribution of Integration Scores at each RISE Administration

5.20. Shifts in Integration Groups from Baseline +2 to Baseline +3

5.21. Overall Integration Across Four RISE Administrations by Gender viii

5.22. Overall Integration Across Four RISE Administrations by Country of Origin

5.23. Overall Integration Across Four RISE Administrations by Age

5.24. Cluster Analyses: Two Patterns Through Integration Pathways

I. Can a Dream Have a Footprint?

When I tell people about the Refugee Integration Survey & Evaluation (RISE) project, I explain that refugees are persecuted individuals and families from countries around the world that the U.S. government supports to come and rebuild their lives in a society based upon economic, political, and religious freedom. I am proud that our country upholds ideals of freedom and democracy in this way, giving hope for a better life to millions who suffer throughout the world—regardless whether they ever will have the opportunity to come to America. For those who do come, the plaque affixed to the Statue of Liberty speaks directly to their circumstances:

“Give me your tired, your poor, Your huddled masses, yearning to breathe free, The wretched refuse of your teeming shore, Send these, the homeless, tempest tossed to me, I lift my lamp beside the golden door.”

—Emma Lazarus

This is only part of Lazarus’ now famous poem, and I did not know until Quality Evaluation Designs (QED) began the RISE evaluation in 2010 that in the poem the Statue of Liberty is referred to as Mother of Exiles. Before RISE, I did not understand how apt and inspiring this phrase is for newly arrived refugees, whose greatest loss, sometimes, is their identity.

Compounding the loss of family and friends through war, torture, and disease, as well as the loss of home and property is leaving one’s home country without ID, passport, or official documents. I did not understand the facelessness and hopelessness of those exiled from their homelands without papers. Refugees have no home and no political identity.

I also did not know that refugees are required to repay their airfare and a portion of the costs of their resettlement. These expenses can range from $1,000‐$5,000 or more depending on the size of one’s family. All refugees are required to sign documents agreeing to repay their airfare (interest free). Loan repayment (or not) may be considered when they apply for citizenship.

In major U.S. cities, I typically encounter refugees as taxi drivers and hotel staff. Until I began talking with them, I didn’t understand the tremendous suffering most have endured before coming to the U.S., and the tremendous challenges they face during resettlement in learning English, negotiating transportation, finding jobs, understanding American culture and values, and keeping their families intact in the face of extraordinary pressures. And yet, the refugees with whom I’ve spoken—related to RISE or elsewhere—carry their burdens with patience, grace, and gratitude. RISE data show that no matter how much they struggle to transition into their new lives, 96% feel safe in the U.S. Free from the dangers and restrictions of refugee camps, refugees who come to the United States feel free and filled with hope. They do not worry about being beaten or tortured by police and they are not targeted because of their religion or ethnicity. This is as true for those well‐educated in their home countries who practiced law and medicine who now may be cleaning bedpans in nursing homes, as it is for illiterate farmers trying to adapt to life in an urban center.

A cab driver from Somalia told me, For us, U.S.A. stand for ‘You Start Again.’ This is good life, I have opportunity here. My children can be safe and they can have a better life.

A Denver refugee said, I really appreciate the U.S. government and the people who work with us, and our kids being able to go to school. In our country we do not have school or have the chance to go to school.

I asked a RISE Community Connector who arrived from Nepal whether he would ever return to his home country. He said, There is nothing for me there. I will never go back. My country didn’t want me. They sent me away without papers. In the camps, I still had no country. We were people of nowhere. The United States brought [my family] here. Soon I will have citizenship and then I will have a passport. Now I’ll have an identity. The United States will always be my home.

The American Dream was articulated in 1931 by James Truslow Adams, in his book, The Epic of America. Adams writes:

It is not a dream of motor cars and high wages merely, but a dream of social order in which each man and each woman shall be able to attain to the fullest stature of which they are innately capable, and be recognized by others for what they are, regardless of the fortuitous circumstances of birth or position" (p. 214‐ 215).

Written during the depths of the Great Depression, these words helped renew a spirit of determination and hope for a discouraged people, and refreshed the vision of America as “the land of opportunity” that has from the beginning been a defining quality of the American ethos.

People differ about what, exactly, the American Dream is, but many agree that a major element is the hope (or expectation) that younger generations will have better lives than their parents.

Recent articles and reports question the viability of the American Dream. In a 2014 Forbes online post on, Neil Howe documents trends since the early 1880’s showing that, since 2012— for the first time since the U.S. Census—those 50 years and younger are earning lower median incomes than same‐age cohorts born in earlier years. In another (2015) post, Howe cites data showing that 40‐49 year olds have more debt and $25,000 fewer assets than did their same‐ aged peers 30 years ago.1 A 2012 Gallup poll showed that 58 percent of adults nationally were unsatisfied with the opportunity of the next generation to live better than their parents.2

Yet for refugees, the American Dream remains not only an ideal, but a goal. Those supported to rebuild their lives in the United States in order to escape persecution and the ravages of war have experienced misery, death and living conditions impossible for most of us born in the United States to imagine. Here in America, they work hard to earn money, learn English, keep their families together, and help their children thrive. They move from crowded apartments to modest homes. They seek navigators from within and outside their communities to guide them through confusing legal, social, and cultural systems. Slowly, with extraordinary determination, their circumstances improve.

Can a dream leave a footprint? Only if it’s more than just a dream. RISE data, which show a steady increase over four years in the integration of 95% of our Denver cohort, is the footprint of refugees’ path towards the American Dream.

And yet, some refugees and groups of refugees struggle to overcome obstacles, including finding sufficient and meaningful work, learning English, navigating health care, and accessing education and training. RISE data show that those aged 50 and over face especially stressful challenges.

Working with Community Connectors from within refugee communities to administer the survey over four consecutive years resulted in enduring relationships between refugees and the RISE evaluation team. Perspectives from refugee participants and Community Connectors informed and enhanced our findings. Refugees in our study carved time from extensive work and child‐rearing schedules and braved horrible weather to take the RISE survey and participate in our interviews and focus groups. Community Connectors helped refugees move, provided rides, and assisted them with helpful information to encourage their continued participation.

Community Connectors and respondents—all of whom were former refugees—recognized the unique contribution their participation could make to the lives of refugees in Denver and throughout the United States. They were eager to share their stories, which remain invisible and untold to most Americans.

We on the evaluation team have been inspired by the perseverance and resilience of the refugees in our study and the commitment of our Community Connectors to making RISE a

1 Neil Howe, (2014). Are you born to be better off than your parents? (Part 1 of Generations in Pursuit of the American Dream). Forbes.com. July 16, 2014. Available at:

http://www.forbes.com/sites/neilhowe/2014/07/16/part‐1‐generations‐in‐pursuit‐of‐the‐ american‐dream/. Also see: Neil Howe (2015). Another day younger and deeper in debt.

Available at: http://www.forbes.com/sites/neilhowe/2015/09/16/another‐day‐younger‐and‐ deeper‐in‐debt/. Both articles last accessed January 4, 2016.

2 Lysia Saad (June 4, 2012). Majority in U.S. dissatisfied with next generations’ prospects.

Available at: http://www.gallup.com/poll/155021/Majority‐Dissatisfied‐Next‐Generation‐ Prospects.aspx. Last accessed January 4, 2016.

success. We strived to justify the hope and trust they placed in us by representing their stories with integrity as we map the pathways they travel towards integration into U.S. economy, society, and culture. We think our work can illuminate gaps in the trail, where strategic deployment of resources could provide bridges to facilitate the journey. Ultimately, the evaluation team hopes that RISE will improve the pathways refugees travel toward better lives.

We have met them along these paths, and our own lives are enriched as a result.

Dr. Gary Lichtenstein, Principal Investigator Quality Evaluation Designs

II. Summary of Year 5 (2014‐15) Activities

The Refugee Integration Survey & Evaluation (RISE) project sought to operationalize refugee integration by creating a survey that could be administered longitudinally or cross‐sectionally that would assess refugees’ progress towards integration in U.S. society. The RISE survey was developed in 2010‐11 and administered longitudinally four consecutive years (2011‐2015) to a cohort of 467 newly arrived refugees (70% retention rate), who constituted nearly all adult refugees arriving in Denver in 2011 from the target study groups.

Since the study was originally funded, the field of integration has grown, with increasing focus on the interaction between refugees and their receiving communities. This study focuses only on integration from the perspective of refugee experiences. Other studies that have also looked at this phenomenon commonly use U.S. Census data, which limits exploration of integration to pre‐existing variables that can be mapped onto integration domains or pathways, and tend to be annual, cross‐sectional data. The Migration Policy Institute recently released an excellent study that identifies this limitation:

ACS [American Community Survey] data are not longitudinal, and therefore do not enable us to track individual refugees to see if they have made education gains since their arrival. However, it is possible to compare attainment over time among similar groups of refugees by comparing data from the 2000 U.S. Census and the 2009‐11 ACS (p. 21).3

Another excellent study, published by the National Academies Press, emphasized the need for longitudinal data related to immigrant integration:

The panel was handicapped in its work by the dearth of available longitudinal data to measure immigrant integration. This is a long‐standing problem that has become increasingly critical as immigration to the United States has increased and as immigrants have become dispersed throughout the country (ExecSum

p. 10)4

Because the RISE survey is based on a well‐researched survey framework, data can be analyzed both within and across integration variables, discerning highlights of and patterns in refugees’ integration experiences. One important feature of the RISE study is that data are reported

3 Capps, Randy & Kathleen Newland with Susan Fratzke, Susanna Groves, Gregory Auclair, Michael Fix, and Margie McHugh, (2015). The Integration Outcomes of U.S. Refugees: Successes and Challenges. Washington, D.C., Migration Policy Institute.

4 National Academies of Sciences, Engineering, and Medicine. (2015). The Integration of Immigrants into American Society. Panel on the Integration of Immigrants into American Society, M. C. Waters and M. G. Pineau, eds. Committee on Population. Division of Behavioral and Social Sciences and Education. Washington, DC: The National Academies Press.

directly by refugees themselves, in their home language, to trained, reliable, community‐based data collectors, rather than by unidentifiable data that might be transformed to fit pre‐ established categories. RISE qualitative data, including one‐on‐one interviews and focus groups, supplement survey data, enabling refugees to tell their own stories.

2014‐15 was the fifth year of Quality Evaluation Designs’ (QED) implementation of the RISE project, and the fourth year of administering the RISE survey instrument. The data consisted of nearly all adult Burmese, Bhutanese, Somali, and Iraqi adult refugees who arrived in Denver from January 2011 through March 2012. Data collection involved administering the RISE survey annually to each refugee, and collecting interviews and conducting focus groups with a subset of respondents. During Year 1, the QED Research Team researched integration and created the survey; there was no data collection. Year 2 of the project was the first year of data collection.

Figure 2.1 clarifies the timing and nomenclature of the project.

2.1. RISE Survey Administration Years

Project

RISE Survey

Year Year Activity Administration 2010‐11 Year 1 Study & Survey Design N/A

2011‐12 Year 2 1st Year Data Collection (June‐May) Baseline (B) 2012‐13 Year 3 2nd Year Data Collection (June‐May) B+1 2013‐14 Year 4 3rd Year Data Collection (June‐May) B+2 2014‐15 Year 5 4th Year Data Collection (June‐May) B+3

The RISE survey is based on Ager & Strang’s conceptual framework of refugee integration, which has been accepted as a model of integration worldwide. Ager and Strang developed “The Indicators of Integration Framework” (2004, 2008)5 after years of exploring the construct of “integration” through an exhaustive literature review, fieldwork in refugee settlement communities, and rigorous qualitative data analysis and verification. Rather than attempt to define integration, they delineated its principal components. The framework identifies ten dimensions of integration, which, individually and collectively, are pathways to successful integration. Ager and Strang grouped these ten “core domains” of integration into four themes, as seen in Figure 2.2 below; and any number or combination of pathways might lead to success.

Although this depiction may insinuate a hierarchy of progress, Ager and Strang’s research suggests no such evidence. In fact, the use of both Markers and Means demonstrates how domains can be indicators of integration as well as a pathway to integration.

5 Ager, A., & Strang, A. (2008). Understanding integration: A conceptual framework. Journal of Refugee Studies, v1, n2, pp. 166‐191. UK: Oxford University Press. And Ager, A. and Strang, A.

(2004) Indicators of Integration: Final Report. Home Office Development and Practice Repor t #28, London: Home Office.

Integration: Lessons from Integration Perspectives, Journal of Refugee Studies, 23 (4), 411‐414.

2.2. Ager & Strang’s Conceptual Framework Defining Core Domains of Integration*

*From: Ager, A. and Strang, A. (2004) Indicators of Integration: Final Report. Home Office Development and Practice Report #28, London: Home Office.

In the 11 years since this framework was proposed, it has served as a heuristic for facilitating discussion and integration policy in several countries, including the United States (Ager & Strang, 2008). The framework has influenced national and regional policy formulation (Home Office, 2008)6, and its critique (ICAR, 2004)7 served as a structure for commissioning and/or developing services aimed at supporting refugee integration (Smyth et al, 2010)8, and formed the basis of multiple qualitative studies on aspects of refugee resettlement.

Agar and Strang’s integration framework is currently the only model of integration consistently referred to in published policy articles and documents. Yet, in spite of the influence the model has had on integration policy worldwide, QED found no evidence that the model has been

6 Home Office (2008). Path to Citizenship, U.K. Green Paper, London: HMSO.

7 ICAR (2004) Response to Integration Matters: A National Strategy for Refugee Integration.

London: Information Center on Asylum and Refugees in the U.K.

8 Smyth, G., Stewart, E., and Da Lomba, S. (2010). Introduction: Critical Reflections on Refugee operationalized with any valid and reliable metrics, nor have we found any longitudinal survey study of integration.

The RISE survey assesses integration across 10 pathways, 9 of which align with Ager & Strang’s framework (see Figure 2.3).9

2.3. The 10 Pathways that RISE Assesses

1. Employment & Economic Sufficiency 6. Social Bonding

2. Education & Training 7. Social Bridging

3. Children’s Education 8. Language & Cultural Knowledge

4. Health & Physical Well‐Being 9. Safety & Stability

5. Housing 10. Civic Engagement

Overall Integration Score

The RISE team included:

Dr. Gary Lichtenstein, QED Principal, who was primarily responsible for study design and execution, as well as contracting, project communication, and dissemination.

Ms. Maggie Miller was the RISE Project Manager, responsible for day‐to‐day operations, Community Connector selection and oversight, and data collection.

Dr. Jini Puma, faculty at the University of Colorado, Denver, was the lead on database management and statistical analyses.

Dr. Martin L. Tombari, University of Texas, Austin, advised on survey development, statistical analyses, and research.

Dr. Laurie Bennett, an independent consultant, was the lead on qualitative study design, implementation, and analysis.

Dr. Amy Engelman was an independent consultant who conducted the qualitative study during 2014‐15.

Ms. Nicole DuFour, also an independent consultant, was responsible for survey data entry, transcription, and assisting with qualitative data collection.

The design and implementation of the RISE project has been grounded in principles of Community‐based Research (CBR). CBR is collaborative, change‐oriented research that engages community members, organizational representatives, and researchers in a project that

9 QED uses the term “pathways,” instead of “domains” to better reflect a dynamic means towards progress along the integration continuum. Also, QED modified and collapsed two of Ager & Strang’s domains (Social Links and Social Bridges) into one and added the Children’s Education pathway.

addresses community‐identified needs.10 CBR aims to increase knowledge and understanding of a given phenomenon and integrate the knowledge gained with interventions, policy, and social change to improve the health and quality of life of community members. 11 The RISE project aligns with the following CBR principles (outlined by Israel): (1) recognizes community as a unit of identity; (2) builds on strengths and resources within the community; (3) facilitates collaborative partnerships in all phases of the research; (4) integrates knowledge and action for mutual benefits of all partners; (5) promotes a co‐learning and empowering process that attends to social inequalities; and (6) involves a cyclical and iterative process. The application of these principles in the RISE project is illustrated in Section III.

Over the past year, the QED team continued the elaborate process of administering the RISE survey. We refer to this administration as Baseline +3, indicating that we have now collected three years’ of respondent data since the initial, Baseline administration in 2011‐12. During the current year, the survey was administered to 327 respondents, reflecting 96% of those who took it the prior year, and 70% of respondents who took the survey at Baseline (see Section V for survey analyses).

The RISE team conducted several interviews and focus groups during each year of data collection. In 2014‐15 (Year 4), we interviewed a subset of 13 respondents who had low integration scores. The purpose of the qualitative inquiry was to assess how those with low integration scores are supported and/or impeded along critical integration pathways (see Section IV).

Through the second year of data collection (B through B+1), QED focused on the effectiveness of the RISE survey itself. We did not want to misguide refugee service professionals with results based on findings from a faulty survey. We conducted several validation studies (both quantitative and qualitative) to ensure the validity and reliability of survey items, the integrity of pathway data, and the effectiveness of summed pathway scores12 that yield an Overall Integration variable. Having performed extensive due diligence on the survey instrument, we are confident that the survey does assess what it purports to, and that the results are reliable and valid reflections of refugee experiences (see Section VI). We are still tinkering with some items in some pathway areas. The survey may continue to evolve. But, fundamentally, we believe that the structure of the survey, its scope, and its depth of coverage, provide a sound

10 Strand, K., Marullo, S., Cutforth, N., Stoecker, R., & Donohue, P. (2003). Community‐based research and higher education: Principles and practices. San Francisco: Jossey‐Bass.

11 Israel, B. A., Schulz, A. J., Parker, E. A., & Becker, A. B. (1998). Review of community‐based research: Assessing partnership approaches to improve public health. Annual Review of Public Health, 19, 173‐202. And Israel, B. A., Schulz, A. J., Parker, E. A., Becker, A. B., Allen, A., & Guzman, J. R. (2008). Critical issues in developing and following CBPR principles. In M. Minkler & N. Wallerstein (Eds.), Community‐based participatory research for health: From process to outcomes (2nd ed., pp. 47–66). San Francisco: Jossey‐Bass.

12 Throughout this report, “pathway analyses” refers to analysis of the 10 integration pathways that the RISE survey assesses, not to be confused with the statistical method, pathway analysis.

No pathway analysis methods have been employed.

basis for evaluating refugees’ integration within the first 4 years of arrival (probably the survey will be effective for the first 4 or 5 years; after that, we expect ceiling effects; that is, the cohort would have progressed to the point where nearly everyone receives a high score and the survey no longer distinguishes different integration levels.

We began reporting survey results and their implications in 2014. We elaborate on those results in this final report (see Section V). Cluster analyses are a major deliverable of the project. They show patterns of challenges and successes for those with low vs. high integration.

It has been important to the QED Research Team to disseminate RISE findings to refugee communities and refugee service professionals. For the past two years (once we had confidence in the survey results), we have met with RISE Community Connectors to thank them for their service and review survey results. We have helped prepare PowerPoint slides that have been presented by the Colorado Refugee Services Program staff at national conferences.

We have spoken to officials visiting from other countries who have expressed an interest in the project.

In November 2014 (FY5), QED reviewed RISE findings first with CRSP leadership and then with the Colorado Alliance for Refugee Empowerment and Success (CARES), a council of refugee service professionals, including Resettlement Agencies. In April 2015, we reviewed findings with the African Community Center. In March and April (2015), QED presented to the CARES Employment Consortium, a committee focused on refugee employment.

In the following sections, we report on lessons learned from the unique and adventurous data collection methods on which RISE depended (Section III), methods and results from qualitative and quantitative data collection (Sections IV and V, respectively), lessons learned about the RISE survey and about the refugee population we surveyed (Sections VI and VII). Finally, we summarize the implications of this work and suggested next steps as the five‐year RISE study comes to a close (Section VIII).

III. Lessons from Data Collection

Four Years of Data Collection

There have been few longitudinal studies of refugees in the United States and none that we have found using a quantitative methodology involving hundreds of respondents. One reason for the absence of longitudinal studies has to do with the huge challenges involved in tracking a very mobile population over time. Refugees don’t speak English, their cell numbers change, they move. The RISE study achieved 70% retention of the original 467 respondents over four years of data collection, 93% retention from B+1 to B+2, and 96% retention from B+2 to B+3.

We are unaware of this sort of success in a longitudinal study of refugees anywhere in the world.

The high retention rate resulted from an unusual approach to data collection that is a hybrid of community‐based and professional models of data collection, which we refer to as the Community Connector model. The Community Connector model is characterized by a collaborative relationship between professionals (specifically, the RISE Project Manager) and embedded members of target communities (the Community Connectors). In this section, we review RISE data collection procedures and reflect upon lessons learned from the effort.

An Evolving Model of Data Collection

Baseline Year: Working With the Resettlement Agencies

One of the significant challenges of administering the survey in the first year was determining the most effective way to work with refugees as they arrived in their new community. Since Resettlement Agencies provide initial resettlement services, they were natural partners.

Resettlement Agencies are either affiliates or sub‐offices of the nine national Resettlement Agencies that have cooperative agreements with the U.S. Department of State. In Colorado, the Resettlement Agencies are contracted by the Colorado Refugee Services Program (CRSP) office to provide a variety of resettlement services. Because Resettlement Agency staff establishes the initial relationships with refugees, it made sense that they administer the survey during the last of three, regularly scheduled meetings, which takes place approximately 90 days after an adult refugee first arrives in this country. For Resettlement Agencies, orally administering the 40‐minute RISE survey became just one more task among many for which the extraordinarily busy case workers were responsible.

In order to ensure that the surveys were administered to everyone in the cohort at the designated times, and to maintain quality assurance in administering the survey, the RISE team conducted a number of formal training sessions with the case workers and their supervisors. In addition, the RISE Project Manager made frequent visits to refugee service agencies to catch the busy case workers and their supervisors between tasks, in order to collect completed surveys, answer questions, give reassurances, and address problems of timeliness and completeness as they arose. In so doing, the Project Manager began building trust with the Resettlement Agencies and/or interpreters, who had direct connections with the newly arrived refugees.

Originally QED planned on collecting a sample of 1200 surveys from the total number of adult refugees who arrived in Denver during the 2011‐12 year. However, due to restricted flow of refugee arrivals, a total of only about 500‐600 refugees arrived from the target populations. In order for the study to be credible, we needed nearly every adult refugee from these groups to participate. Creating and maintaining good relationships with the Resettlement Agencies became critical to successful data collection during the first year.

During this first, Baseline year, Resettlement Agencies provided essential information that the RISE team needed in order to collect surveys in subsequent years, such as:

How do Bhutanese names work?

How do the Karenni differ from the Karen?

Which names in Karenni are men’s names, and which are women’s name?

How is Ahmed pronounced differently between the Iraqis and the Somalis?

How is “z” pronounced in Karen?

What do these acronyms stand for: ERIS, LFS, ACC, CAO, GBCC?

These learnings occurred through conversations with Resettlement Agency case workers and with the interpreters who administered many of the surveys—some of whom later became RISE Community Connectors.

The Community Connector Model Beginning 2012‐13 (Baseline +1—the second year of data collection), QED became responsible for administering the survey to those who took the survey in the first year, as well as conducting interviews and focus groups with a subset of respondents. To facilitate these critical facets of the RISE study, QED implemented the Community Connector model.

The premise of the Community Connector model is that refugee communities are opaque to outsiders, but tightly knit from an insider’s perspective. Many refugees come to Denver because they have family and friends here already, and these social networks create familiarity and safety in a new and ambiguous environment. Community Connectors were embedded members of refugee communities. They had lived in the U.S. for at least two years after having arrived in the United States as refugees themselves. They had broad and deep knowledge of individuals, family, and social networks, and were bilingual in the languages of the target populations and in English. Often, the Community Connectors were social connectors within their own communities. Community Connectors were hired to contact survey respondents every three months, track new addresses and phone numbers, and administer the RISE survey once each year.

To do this work, Community Connectors often relied on respondents’ networks of extended family and friends, so if a refugee moved or got a new mobile phone number the Community Connector could locate him or her through these networks. Community Connectors’ exceptional ability to find refugees through a deep understanding of respondents’ social networks enabled them to make contact in unexpected ways. We have amusing and revealing stories that illustrate the vagaries of community‐based data collection: finding the whereabouts of one refugee at an uncle’s wedding; administering a survey while helping a client pack boxes to move.

Collecting surveys using a professional model—scheduling appointments, for example, simply wouldn’t have worked. The fluidity and mild to extreme chaos of refugees’ lives would quickly frustrate the schedules of 9‐5 professionals. Furthermore, it would have been impossible for researchers outside of these communities to identify these networks and follow the movements of individuals and families on a week‐to‐week and month‐to‐month basis.

And yet, the study did take place in a professional context with QED under contract to the Colorado Refugee Services Program (CRSP). The work had to get done and time and expenses needed to be budgeted and accounted for. Administering the survey and qualitative data collection involved many logistics.

To strategize these tasks, QED contracted with Ms. Maggie Miller, an independent consultant based in Denver, to be Project Manager. While research staff created the study design, survey, and interview protocols, Ms. Miller was responsible for survey implementation and accountability. Managing data collection meant working closely with Community Connectors, whose lives were often fragmented by working several jobs, child rearing, and transportation obstacles.

Communicating with Community Connectors presented challenges. They do not all use email and phone access is variable, but they text. Meeting times and places vary. Planning meetings with all four, five or six Community Connectors a month or week in advance was impossible.

Most meetings took place one‐on‐one, often…