eca_performance_monitoring_plan_-_pmp.pdf

| 1

PERFORMANCE MONITORING PLAN (PMP)

Applicant Name: Example Organization Fiscal Year and Program Title: FY## Example Program Funding Opportunity #: SFOP000XXXX

Instructions:

The Monitoring Data for ECA (MODE) Framework provides standard indicators and corresponding survey questions to ensure consistent measures across all ECA programs. This Performance Monitoring Plan (PMP) includes ALL MODE objectives and indicators that were created to collect data from program participants. While ECA recommends the applicant use the format provided, this is not a requirement. However, any performance monitoring reference document the applicant submits should include the information in the column headers (Indicator Name, Definition, Target, Survey Question, etc.) in the table below at a minimum.

To provide a PMP that will meet the solicitation review criteria, the applicant should take the following steps:

1. Fill out the Applicant Name, FY and Program Title and Funding Opportunity # fields highlighted above

2. Carefully review the solicitation document to identify the MODE indicators that were listed

3. Review the additional (those not listed in the solicitation document) MODE Framework indicators listed in the table below to determine if they are relevant (meaning the applicant proposes that data should be collected for those indicators as part of this program)

4. Those indicators that were NOT selected in steps 2 and 3 above can be deleted from the table below (to do this, click your mouse somewhere within the row you would like to delete, right-click, select “Delete Cells”, then “Delete Entire Row”. Repeat this process until the only MODE Framework indicators that remain are those identified during steps 1 and 2)

5. Applicants may also design custom objectives and/or indicators that are specific to the proposed program and this proposal if the MODE Framework indicators are insufficient to cover all aspects of the program and/or if there were other sections in the solicitation that identified additional measures the applicant should plan to collect data against

Note: The placement of two rows for applicant-designed custom indicators in the PMP is arbitrary – the applicant can add or delete rows as necessary

For all indicators in the PMP that remain after steps 2-5 above, insert the target into the appropriate column. A target specifies a feasible and appropriate level of an output or outcome that is hoped for, expected, or intended. For more information on how to set a target, you can watch a MELI Evaluation Seminar on the topic here: https://www.youtube.com/watch?v=KUCQbc0I7Nw)

More resources and guidance documents on the MODE Framework and PMP are available online (https://eca.state.gov/impact/eca-monitoring-evaluation-learning-and-innovation-meli-unit/mode-framework-eca-applicants-and). PMP-specific resources include:

MODE Framework Indicator Book (https://app.box.com/s/7ms6e5f66kji0h8sgao8uno13drvsg1z):

MODE Performance Indicator Reference Sheets (https://app.box.com/s/mf5m9ul4cjyljlesgk958vqagsp7xqm1)

| 2

MODE Framework Indicator Selection Webinar (https://www.youtube.com/watch?v=u2THUI00RO0) PMP Creation Webinar (https://app.box.com/s/gwv7t8eol9n8e883q8xp2pfjssdxhdbq) Creating Baselines and Targets Webinar (https://www.youtube.com/watch?v=KUCQbc0I7Nw)

Definitions of PMP Columns

Performance Indicator: An observable or measurable characteristic that indicates the extent to which a program objective is being achieved.

Definition: Clarifies all terms necessary and provides details on how to calculate the data collected to measure the change expected.

Disaggregation (“Disagg”): Lists how the data will be broken down into subgroups (such as: gender, age, state, country, etc.)

Target: Specific, planned level of result to be achieved within an explicit timeframe Data Source and Required Survey Question: Specifies how the data will be obtained (such as a survey, observation, interview, etc.)

and the mandatory question linked to the data source. Note that all MODE Framework survey questions are required (i.e. should be forced response); please see the Consent Language in the MODE Framework Indicator Book for an example of how to convey this to participants/survey respondents.

When to Collect: Specifies when the indicator will be measured, for example: post-program, end of award, etc.

o A post-program collection should occur no later than 6 weeks after the participant has completed the program

Award recipients are responsible for collecting indicator data on participant outputs and outcomes during the period of performance of the exchange itself. Note: The Bureau will measure outcomes of ECA participants at one, three, five, and 10 years after the exchange has ended to capture the long-term impact of ECA programming unless otherwise specified in the solicitation for which you are applying. In this instance, the recipient will be responsible for coordinating with ECA on any alumni surveys to de-duplicate questions and minimize potential survey fatigue.

| 3

Performance Indicator

Definition Disagg. Target Data Source and Required* Survey Question

When to Collect

Demographic Questions

Home Country (Demographic)

Used to identify subgroups for Country of Origin disaggregation. (Foreign participants only)

N/A N/A Survey (self-reported data)

What is your home country?

[Drop Down: List of program countries] – See Appendix C in the Indicator Book

Post- Program

Home State (Demographic)

Used to identify subgroups for Home State disaggregation.

(American participants only)

N/A N/A Survey (self-reported data)

What is your home state?

[Drop Down: List of U.S.

States] – See Appendix D in the Indicator Book

Post- Program

Age (Demographic)

Used to identify subgroups for Age disaggregation. N/A N/A Survey (self-reported data)

What is your age? Fill in the blank ________

[Validate answer numeric, with accepted answers ranging from 15 to 100]

-- OR--

Are you between the ages of 15 and 35?

Yes No

Post- Program

Gender (Demographic)

Used to identify subgroups for Gender disaggregation. N/A N/A Survey (self-reported data)

| 4

Indicator

Definition Disagg. Target Data Source and Required* Survey Question

When to Collect

Please check the option that best describes your gender:

Male Female Non-binary I do not wish to respond

Race

(Demographic) Used to identify subgroups for Race disaggregation.

(American participants only)

N/A N/A Survey (self-reported data)

What is your race or ethnicity?

Select all that apply.

American Indian or Alaskan

Native Asian Black or African American Hispanic or Latino Native Hawaiian or Other

Pacific Islander White Some Other Race I do not wish to respond

Post- Program

Participant Type (Demographic)

Used to identify the subgroup “Reciprocal” for Participant Type disaggregation.

(American participants only)

Award recipients will need to identify whether a participant is designated as 'Foreign', 'American', or 'Reciprocal' as part of the disaggregations. The assumption is that those who answer the "Home Country" question can be designated as a 'Foreign participant', those who answer "Home State" can be designated as an 'American

N/A N/A Survey (self-reported data)

Did you participate in a reciprocal exchange?

A reciprocal participant is an American who travels to an exchange participant's home country to assist the participant with their individual project

| 5

Indicator

Definition Disagg. Target Data Source and Required* Survey Question

When to Collect participant', and those who answer the associated question with a "yes" can be designated as a 'Reciprocal participant'

Yes No

Placement State (Demographic)

Used to identify subgroups for Placement State disaggregation. (Foreign participants only)

This only applies if an exchange participant was based in one state for more than one month; for example, for IVLP participants, the placement state would not need to be reported.

N/A N/A Survey (self-reported data)

In which U.S. State were you placed during your exchange?

[Drop Down: List of U.S.

States] – See Appendix D in the Indicator Book

Post- Program

ECA Objective 1: Advance participant and beneficiary cross-cultural competence and global perspective ECA Sub-Objective 1.1: Promote cultural exchanges and enhance understanding between participants and their host communities

E1.1.01: Percent of participants reporting that their exchange experience offered opportunities to engage with other cultures

Participant - A person who has directly participated in an ECA exchange program. Note that participants also include ECA’s Office of Private Sector Exchange’s (ECA/EC) “Exchange Visitors.”

Counting Frequency Each respondent should only be counted once in the reporting year following their program completion.

Calculations To calculate percentage, numerator will be number of respondents who answered “yes” while the denominator will be the total number of survey respondents. Divide the numerator by the denominator and then multiply by 100 for the percent value.

Gender Age Country of origin Home state Race Participant type:

American, foreign, reciprocal

To be completed by the applicant

Survey (self-reported data)

Do you agree with the following statement?

My exchange program offered opportunities for making cultural connections and sharing. For example:

Conversations about each other’s’ countries, Attending cultural events, etc.

E1.1.04:

Percent of participants with more favorable

Participant - A person who has directly participated in an ECA exchange program. Note that participants also include ECA’s Office of Private Sector Exchange’s (ECA/EC) “Exchange Visitors.”

Gender Age Country of origin Home state

To be completed by the applicant

Survey (self-reported data)

Did your views of the United States government change as a

| 6

Indicator

Definition Disagg. Target Data Source and Required* Survey Question

When to Collect opinions of the United States Government (core indicator)

Counting Frequency Each respondent should only be counted once in the reporting year following their program completion.

Calculations To calculate percentage, numerator will be number of respondents who select “much more favorable” OR “somewhat more favorable” on the survey question. The denominator will be the total number of survey respondents. Divide the numerator by the denominator and then multiply by 100 for the percent value.

Race Participant type:

American, foreign, reciprocal result of your exchange program participation?

If YES:

How did your views of the United States government change? Are your views:

Much less favorable Somewhat less favorable Somewhat more favorable Much more favorable

E1.1.09:

Percent of participants who traveled abroad for the first time because of their program (core indicator)

Participant - A person who has directly participated in an ECA exchange program. Note that participants also include ECA’s Office of Private Sector Exchange’s (ECA/EC) “Exchange Visitors.”

Abroad – A country outside of one’s home country

Counting Frequency Each respondent should only be counted once in the reporting year following their program completion.

Calculations To calculate percentage, numerator will be number of respondents who answered “no” to the first question while the denominator will be the total number of survey respondents. Divide the numerator by the denominator and then multiply by 100 for the percent value.

Gender Age Country of origin Home state Race Participant type:

American, foreign, reciprocal

To be completed by the applicant

Survey (self-reported data)

Before this exchange program, had you traveled outside of your home country?

Why did you travel outside of your home country? Select all that apply.

Tourism Visit friends/family Participation in another U.S.

Department of State

| 7

Indicator

Definition Disagg. Target Data Source and Required* Survey Question

When to Collect exchange program (including U.S. Embassy-sponsored exchanges)

Participation in a non-U.S.

Department of State exchange

Study Work Other: ______ [write-in]

E1.1.10:

Percent of foreign participants who traveled to the United States for the first time during their program (core indicator)

Participant - A person who has directly participated in an ECA exchange program. Note that participants also include ECA’s Office of Private Sector Exchange’s (ECA/EC) “Exchange Visitors.”

Counting Frequency Each respondent should only be counted once per reporting year following their program completion.

Calculations To determine the percent value, numerator is number of respondents who reply “no” to the first question while denominator will be the total number of survey respondents. Divide the numerator by the denominator and then multiply by 100 for the percent value.

Gender Age Country of origin

To be completed by the applicant

Survey (self-reported data)

Before this exchange program, had you traveled to the United States?

Why did you travel to the United States? Select all that apply.

Tourism Visit friends/family Participation in another U.S.

Department of State exchange program (including U.S. embassy-sponsored exchange programs)

| 8

Indicator

Definition Disagg. Target Data Source and Required* Survey Question

When to Collect

Participation in a non-U.S.

Department of State exchange

Study Work Other: ______ [write-in]

E1.1.11:

Percent of American participants indicating a change in their understanding of their host country's culture and values

Participant - A person who has directly participated in an ECA exchange program. Note that participants also include ECA’s Office of Private Sector Exchange’s (ECA/EC) “Exchange Visitors.”

Host country - The country in which the participant traveled to for their exchange program.

Counting Frequency Each respondent should only be counted once in the reporting year following their program completion.

Calculations Responses to each item should be scored in the following manner:

No change – 1 Minimal change – 2 Moderate change – 3 Substantial change – 4

The response scores for each item should be summed and divided by the number of items they responded to in order to give an average question score for each respondent.

Gender Age Home state Race Alumni type:

American, reciprocal

To be completed by the applicant

Survey (self-reported data)

How much did participation in the exchange program change your understanding or knowledge of the following topics?

Host country political system Host country economy Foreign affairs of my host country Domestic affairs in my host country Host country values and culture Daily life in my host country Religious and ethnic diversity in my host country Freedom of speech and press in my host country Voluntary community service in my host country

| 9

Indicator

Definition Disagg. Target Data Source and Required* Survey Question

When to Collect

To determine the percent value, the numerator is the number of respondents whose average question score is 2 or greater while the denominator will be the total number of survey respondents. Divide the numerator by the denominator and then multiply by 100 for the percent value.

[Scale: No change, Minimal change, Moderate change, Substantial change]

E1.1.12:

Percent of foreign participants indicating a change in understanding of third-party countries’ cultures and values

Participant - A person who has directly participated in an ECA exchange program. Note that participants also include ECA’s Office of Private Sector Exchange’s (ECA/EC) “Exchange Visitors.”

Third-party country - A country that is not the participant/alumni’s home or host country. Generally, this refers to the home countries of fellow participants.

Counting Frequency Each respondent should only be counted once in the reporting year following their program completion.

Calculations Responses to each item should be scored in the following manner:

No change – 1 Minimal change – 2 Moderate change – 3 Substantial change – 4

The response scores for each item should be summed and divided by the number of items they responded to in order to give an average question score for each respondent.

Gender Age Country of origin

To be completed by the applicant

Survey (self-reported data)

Thinking about the exchange participants you met from other countries (not your home country or the U.S.), how much did participation in the program change your understanding or knowledge of the following topics related to those countries?

Political system Economy Foreign affairs Domestic affairs Values and culture Daily life Religious and ethnic diversity Freedom of speech and press service

| 10

Indicator

Definition Disagg. Target Data Source and Required* Survey Question

When to Collect

To determine the percent value, numerator is number of respondents whose average question score is 2 or greater while the denominator will be the total number of survey respondents. Divide the numerator by the denominator and then multiply by 100 for the percent value.

[Scale: No change, Minimal change, Moderate change, Substantial change]

E1.1.13: Percent of foreign participants who are more likely to recommend the United States as a good place to study

Participant - A person who has directly participated in an ECA exchange program. Note that participants also include ECA’s Office of Private Sector Exchange’s (ECA/EC) “Exchange Visitors.”

Counting Frequency Each respondent should only be counted once in the reporting year following their program completion.

Calculations To determine the percent value, numerator is number of respondents who reply “much more likely” or “somewhat more likely” while denominator will be the total number of survey respondents. Divide the numerator by the denominator and then multiply by 100 for the percent value.

Gender Age Country of origin

To be completed by the applicant

Survey (self-reported data)

Compared to before your exchange program participation, how likely are you now to recommend the United States as a good place to study?

Much more unlikely Somewhat more unlikely Somewhat more likely

Much more likely

Post- Program

E1.1.16:

Percent of virtual exchange

(VE)

participants who report VE as their only likely way to interact with citizens from other

Participant - A person who has directly participated in an ECA exchange program. Note that participants also include ECA’s Office of Private Sector Exchange’s “Exchange Visitors.”

Virtual exchange - A sustained, multi-way interaction, that are online, used to build networks and shared knowledge around policy priorities. For a program to be considered a virtual exchange, participants must engage 51% or more virtually. A series of virtual components can be considered an exchange if they are (1) originated as a unit with a

Gender Age Country of origin Home state Race Participant type:

American, foreign, reciprocal

To be completed by the applicant

Survey (self-reported data)

In the future, are virtual exchanges likely to remain your only way to interact with citizens from other countries?

Yes No

I don’t know

| 11

Indicator

Definition Disagg. Target Data Source and Required* Survey Question

When to Collect countries in the future collective start and end date and (2) are organized around a unifying theme. For example: a series of MOOCs would be considered an exchange if they are offered as a set (of say six classes) and are centered on a specific topic (for example, media literacy). Offering MOOCs as an online resource is not considered a virtual exchange program.

Virtual element - Expand the reach of programs and increase engagement for participants and alumni. Unlike virtual programs, virtual elements are single, stand-alone components, offered either as add-on enrichment to an in-person exchange or as general outreach by ECA. For example, offering single MOOCs to the general public or virtual pre-departure orientation webinars are both considered virtual elements. Additionally, online spaces for participants to communicate (such as a program Facebook page) is considered a virtual element.

Count Frequency Each respondent should only be counted once per reporting year following their program completion.

Calculations To calculate percentage, numerator will be number of respondents who answered “yes” while the denominator will be the total number of survey respondents. Divide the numerator by the denominator and then multiply by 100 for the percent value.

This indicator counts participants involved in virtual exchange programs (see definition above) and WILL NOT

| 12

Indicator

Definition Disagg. Target Data Source and Required* Survey Question

When to Collect count participants engaging with virtual program elements as part of a non-virtual program.

E1.1.17:

Percent of foreign participants with more favorable opinions of the American people (core indicator)

Participant - A person who has directly participated in an ECA exchange program. Note that participants also include ECA’s Office of Private Sector Exchange’s (ECA/EC) “Exchange Visitors.”

Counting Frequency Each respondent should only be counted once in the reporting year following their program completion.

Calculations To determine the percent value, numerator is number of respondents who reply “much more favorable” OR “somewhat more favorable” while the denominator will be the total number of survey respondents. Divide the numerator by the denominator and then multiply by 100 for the percent value.

Gender Age Country of origin

To be completed by the applicant

Survey (self-reported data)

Did your views of the American people change as a result of your exchange program participation?

How did your views of the American people change? Are your views:

Much less favorable Somewhat less favorable Somewhat more favorable Much more favorable

Post- Program

E1.1.18:

Percent of foreign participants indicating an increase in understanding of United States culture and values (core indicator)

Participant - A person who has directly participated in an ECA exchange program. Note that participants also include ECA’s Office of Private Sector Exchange’s (ECA/EC) “Exchange Visitors.”

Counting Frequency Each respondent should only be counted once in the reporting year following their program completion.

Calculations

Gender Age Country of origin

To be completed by the applicant

Survey (self-reported data)

How much did participation in the exchange program change your understanding or knowledge of each of the following topics?

United States democracy United States economy Foreign affairs of the United

States

| 13

Indicator

Definition Disagg. Target Data Source and Required* Survey Question

When to Collect

Responses to each item should be scored in the following manner:

No change – 1 Minimal change – 2 Moderate change – 3 Substantial change – 4

The response scores for each item should be summed and divided by the number of items they responded to in order to give a total question score for each respondent.

To determine the percent value, numerator is number of respondents whose total question score is 2 or greater while the denominator will be the total number of survey respondents. Divide the numerator by the denominator and then multiply by 100 for the percent value.

Domestic affairs in the United States

United States values and culture

Daily life in the United States

Religious and ethnic diversity in the United States

Freedom of speech and press in the United States service in the United States

[Scale: No change, Minimal change, Moderate change, Substantial change]

E1.1.19:

Percent of participants agreeing with statements in support of democratic values (core indicator)

Participant - A person who has directly participated in an ECA exchange program. Note that participants also include ECA’s Office of Private Sector Exchange’s (ECA/EC) “Exchange Visitors.”

Democratic values - The ideas or beliefs that make a society fair, including: democratic decision-making, freedom of speech, equality before the law, social justice, equality, social justice.

Count Frequency Each respondent should only be counted once in the reporting year following their program completion.

Calculations Responses to each item should be scored in the following manner:

Gender Age Country of origin Home state Race Participant type:

American, foreign, reciprocal

To be completed by the applicant

Survey (self-reported data)

To what extent do you agree or disagree with the statements below?

Voting is important because real decisions are made in elections

Free and fair elections are the cornerstone of democracy

An independent media is important to the free flow of information

| 14

Indicator

Definition Disagg. Target Data Source and Required* Survey Question

When to Collect

Strongly disagree – 1 Disagree – 2 Neither disagree nor agree – 3 Agree – 4 Strongly agree – 5 I don’t know/I can’t ascertain – 0

To determine the percent value, numerator is the number of respondents whose average question score is greater than or equal to 4 on a 1-5 scale while the denominator will be the total number of survey respondents. Divide the numerator by the denominator and then multiply by 100 for the percent value.

All citizens in a country should have equal rights and protections under the law, regardless of circumstances

The rule of law is fundamental to a functioning democracy

Individuals have the right to free speech and to voice opposition

Organizations have the right to free speech and to voice opposition

Democratic principles enhance the workplace.

Supervisors should incorporate democratic principles into their management practices

[Scale: Strongly disagree, Disagree, Neither disagree nor agree, Agree, Strongly agree, I don’t know/I can’t ascertain]

ECA Objective 2: Increase the impact that participants and alumni have on their communities / countries

E2.0.01: Percent of foreign participants that volunteer in their host communities

Participant - A person who has directly participated in an ECA exchange program. Note that participants also include ECA’s Office of Private Sector Exchange’s (ECA/EC) “Exchange Visitors.”

Volunteer - To offer one’s time or services to an organization or community effort for free.

Gender Age Country of origin

To be completed by the applicant

Survey (self-reported data)

Did you volunteer your time during your exchange?

Yes No

| 15

Indicator

Definition Disagg. Target Data Source and Required* Survey Question

Host community - A community or institution that sponsors, supports, or provides exchange programs support or services. Host communities and institutions do not necessarily directly participate in a program. Benefits to this group most often occur through interaction with the participants and/or alumni during or after the program.

Host community members comprise two groups: 1) host families who welcomed exchange visitors into their homes;

and 2) host institutions or businesses who provided professional or academic experiences to exchange visitors.

Count Frequency Each respondent should only be counted once in the reporting year following their program completion.

Calculations To calculate the percentage, the numerator will be the number of respondents who answered “yes” while the denominator will be the total number of survey respondents. Divide the numerator by the denominator and then multiply by 100 for the percent value.

My program did not offer the opportunity to volunteer

E2.0.04:

Number of hours that foreign participants spend volunteering in their host communities

Participant - A person who has directly participated in an ECA exchange program. Note that participants also include ECA’s Office of Private Sector Exchange’s (ECA/EC) “Exchange Visitors.”

Volunteer - To offer one’s time or services to an organization or community effort for free.

Host community - A community or institution that sponsors, supports, or provides exchange programs support or services. Host communities and institutions do not

Gender Age Country of origin

To be completed by the applicant

Survey (self-reported data)

How many hours did you spend volunteering during your exchange program? (Please note that volunteer hours do not include time spent working as part of your professional placement.)

| 16

Indicator

Definition Disagg. Target Data Source and Required* Survey Question

When to Collect necessarily directly participate in a program. Benefits to this group most often occur through interaction with the participants and/or alumni during or after the program.

Host community members comprise two groups: 1) host families who welcomed exchange visitors into their homes;

and 2) host institutions or businesses who provided professional or academic experiences to exchange visitors.

Count Frequency Each respondent should only be counted once in a reporting year following their program completion.

Calculations Participants will complete a survey question that will request the number of hours of time spent volunteering during their exchange program.

The total number of hours for all participants should be reported.

Fill in the blank: __________ [validate as numeric]

ECA Sub-Objective 2.2: Foster participants’ belief that civic engagement benefits communities/countries

E2.2.01:

Percent of participants who have more confidence in their ability to have an impact in their home country (core indicator)

Participant - A person who has directly participated in an ECA exchange program. Note that participants also include ECA’s Office of Private Sector Exchange’s (ECA/EC) “Exchange Visitors.”

Home community - The community that the program participant originates from and will return to at the conclusion of the program.

Home country - The country in which the program participant resides.

Country of origin Home state Race Participant type:

American, foreign, reciprocal

To be completed by the applicant

Survey (self-reported data)

As a result of your program participation, how has your confidence in each of the following changed?

I have the ability to make a difference in the community where I live

| 17

Indicator

Definition Disagg. Target Data Source and Required* Survey Question

When to Collect

Count Frequency Each respondent should only be counted once in the reporting year following their program completion.

Calculation To determine the percent value, numerator is number of respondents that responded “more confident” to the statement about their ability to make a difference in the ‘community where I live’ while the denominator will be the total number of survey respondents. Divide the numerator by the denominator and then multiply by 100 for the percent value.

I have the ability to make a difference in the country where I live

I have the ability to make a difference in the global community

[Scale: I am less confident; No change in confidence, I am more confident]

ECA Objective 3: Strengthen engagement among participants, alumni, beneficiaries, and institutions

E3.0.02:

Percent of foreign participants who report increasing their network of Americans (core indicator)

Participant - A person who has directly participated in an ECA exchange program. Note that participants also include ECA’s Office of Private Sector Exchange’s (ECA/EC) “Exchange Visitors.”

Network - An interconnected or interrelated group or system. In the context of MODE, this refers to an interconnected or interrelated group intentionally convened, sustained, or otherwise facilitated through ECA programming.

Count Frequency Each respondent should only be counted once in the reporting year following their program completion.

Calculations To determine the percent value, numerator is number of respondents who reply “yes” to the first question item

Gender Age Country of origin

To be completed by the applicant

Survey (self-reported data)

Are you still in contact with any Americans you met during your program exchange?

Are the contacts you maintain with Americans personal, professional, or both?

Personal Professional Both personal and professional

| 18

Indicator

Definition Disagg. Target Data Source and Required* Survey Question

When to Collect while the denominator will be the total number of survey respondents. Divide the numerator by the denominator and then multiply by 100 for the percent value.

E3.0.04:

Percent of foreign participants who report increasing their network of third country nationals

Participant - A person or group of persons that has directly participated in an ECA exchange program. Note that participants also include ECA’s Office of Private Sector Exchange’s (ECA/EC) “Exchange Visitors.”

Network- An interconnected or interrelated group or system intentionally convened, sustained, or otherwise facilitated through ECA programming.

Third-country national - A person who does not have the same nationality and is not from the country in which the exchange took place or from the country of a participant of the exchange.

Count Frequency Each respondent should only be counted once in the reporting year following their program completion.

Calculations To determine the percent value, numerator is number of respondents who reply “yes” to the first question item while the denominator will be the total number of survey respondents. Divide the numerator by the denominator and then multiply by 100 for the percent value.

Gender Age Country of origin

To be completed by the applicant

Survey (self-reported data)

Are you still in contact with any individual not from the U.S.

and not from your own country that you met during your program exchange?

•Yes •No

If YES:

Are the contacts you maintain with these individuals personal, professional, or both?

•Personal •Professional •Both personal and professional

Post- Program

E3.0.07:

Percent of participants who identify as a Department of

Participant - A person or group of persons that has directly participated in an ECA exchange program. Note that participants also include ECA’s Office of Private Sector Exchange’s (ECA/EC) “Exchange Visitors.”

Country of origin Home state

To be completed by the applicant

Survey (self-reported data)

Do you consider yourself a participant of a U.S.

Department of State program?

| 19

Indicator

Definition Disagg. Target Data Source and Required* Survey Question

When to Collect

State program participant (core indicator)

Alumni - Program participants who have been exposed to at least 36 hours of an exchange or a cultural exchange element with the possibility of sustained networking.

Count Frequency Each respondent should only be counted once in the reporting year following their program completion.

Calculations To calculate percentage, numerator will be number of participants who answered “yes” in response to the survey question. The denominator will be the total number of survey respondents. Divide the numerator by the denominator and then multiply by 100 for the percent value.

Race Participant type:

American, foreign, reciprocal

Yes No

ECA Sub-Objective 3.1: Alumni subscribe to platforms for resources and information-sharing

E3.1.02: Percent of participants who learned about new resources

Participant - A person who has directly participated in an ECA exchange program. Note that participants also include ECA’s Office of Private Sector Exchange’s (ECA/EC) “Exchange Visitors.”

Resource – Information that is new and potentially useful to the participant. Can include information guidance that is either personal or professional. Does not include persons.

Can be information provided during any portion of the exchange process (pre-departure, during the exchange, or immediately post-exchange – such as a Congress or concluding event).

Age Country of origin Home state Race Participant type:

American, foreign, reciprocal

To be completed by the applicant

Survey (self-reported data)

Through your program experience, did you learn about any of the following Department of State-managed resources? Select all that apply.

• Small grants applications from the U.S. Government [Carry forward selection to next question]

• Alumni Engagement Innovation Fund (AEIF) applications [Carry forward selection to next question]

| 20

Indicator

Definition Disagg. Target Data Source and Required* Survey Question

When to Collect

Each respondent should only be counted once in the reporting year following their program completion.

Calculations To calculate percentage, numerator will be number of respondents who select at least one of the survey question response options EXCEPT “None of the above” to the first question, while the denominator will be the total number of survey respondents. Divide the numerator by the denominator and then multiply by 100 for the percent value.

• Citizen Diplomacy Action Fund applications [ask of American alumni only] [Carry forward selection to next question]

• Project Development toolkits [Carry forward selection to next question]

• Supplies or equipment [Carry forward selection to next question]

• Information on workshops/events hosted by the U.S. Embassy in your home country [ask of Foreign Alumni only] [Carry forward selection to next question]

• Embassy network of experts and leaders [ask of Foreign Alumni only] [Carry forward selection to next question]

• American Spaces (either American Centers, American Corners, or Bi-National Centers) [ask of Foreign Alumni only] [Carry forward selection to next question]

• Career Connections [ask of American alumni only] [Carry forward selection to next question]

• None

| 21

Indicator

Definition Disagg. Target Data Source and Required* Survey Question

For each category the respondent selects, ask:

Was this the first time you heard about this resource?

• Yes

• No

E3.1.05: Percent of participants who are connected to ECA through a U.S. Department of State-managed platform

Participant - A person or group of persons that has directly participated in an ECA exchange program. Note that participants also include ECA’s Office of Private Sector Exchange’s (ECA/EC) “Exchange Visitors.”

Digital platform - A digital place, opportunity, or event for public discussion or interpersonal engagement intentionally convened, sustained, or otherwise facilitated through ECA programming.

Count Frequency Each respondent should only be counted once in the reporting year following their program completion.

Calculations To calculate percentage, numerator will be number of respondents who select “yes” to at least one of the survey question response options while the denominator will be the total number of survey respondents. Divide the numerator by the denominator and then multiply by 100 for the percent value.

Gender Age Country of origin Home state Race Participant type:

American, foreign, reciprocal

To be completed by the applicant

Survey (self-reported data)

Are you connected with the Department of State International Exchange Alumni Network on the following digital platforms?

• Registered on the alumni.state.gov website (https://alumni.state.gov/)

• Follow @Exchangealumni on Twitter,

• Connected with Alumni Affairs on LinkedIn (https://www.linkedin.com/in/al umni-affairs-9b871b103/)

[Scale: Yes/No/I don’t know]

Post-

ECA Objective 4: Strengthen personal, professional, and technical abilities and aptitudes of participants and beneficiaries

| 22

Indicator

Definition Disagg. Target Data Source and Required* Survey Question

When to Collect

E4.0.01: Percent of participants reporting increases in their job skills as a result of their program participation

Participant - A person who has directly participated in an ECA exchange program. Note that participants also include ECA’s Office of Private Sector Exchange’s (ECA/EC) “Exchange Visitors.”

Count Frequency Each respondent should only be counted once in the reporting year following their program completion.

Calculations To determine the percent value, numerator is number of respondents who reply “yes” while the denominator will be the total number of survey respondents. Divide the numerator by the denominator and then multiply by 100 for the percent value.

Gender Age Country of origin Home state Race Participant type:

American, foreign, reciprocal

To be completed by the applicant

Survey (self-reported)

[NOTE: This question is also included in Indicators E4.0.03, and E4.0.05. If these are to be included in a survey, this question can be asked once, with three multiple follow-up questions based on an affirmative response.]

Did you increase your personal and/or professional skills as a result of your participation in this program?

Yes No

Post- Program

E4.0.03: Percent of participants reporting an increase in soft skills as a result of exchange program participation

Soft skills - Qualities and behaviors an individual demonstrates to interact with others effectively. Examples include communication, listening, self-awareness, awareness of others, self-initiative, cognitive competence skills, self-confidence, resourcefulness/resilience, and decision-making and problem-solving skills.

Participant - A person who has directly participated in an ECA exchange program. Note that participants also include ECA’s Office of Private Sector Exchange’s (ECA/EC) “Exchange Visitors.”

Count Frequency Each participant should only be counted once in the reporting year following their program completion.

Calculations

Gender Age Country of origin Home state Race Participant type:

American, foreign, reciprocal

To be completed by the applicant

Survey (self-reported)

NOTE: Question 1 is also included in Indicator 4.0.01 and 4.0.05. If at least two of those questions are to be included in a survey, question 1 can be asked once, with two multiple follow-up questions based on an affirmative response.

Did you increase your personal and/or professional skills as a result of your participation in this program?

Yes

| 23

Indicator

Definition Disagg. Target Data Source and Required* Survey Question

When to Collect

To determine the percent value, numerator is number of respondents who select at least one answer option except for “none of the above skills” in the second question while the denominator will be the total number of survey respondents. Divide the numerator by the denominator and then multiply by 100 for the percent value.

No

If YES:

Please select the personal skills you increased. Select all that apply.

Leadership skills Communication skills Listening skills Self-awareness Self-initiative Self-confidence Resourcefulness Decision-making skills Problem-solving skills Other: ______ [write-in]

None of the above skills E4.0.05: Percent of participants who report an increase in technical skills as a result of exchange program participation

Technical skills - Knowledge and ability to perform specific tasks. Examples of technical skills include project management, project management, entrepreneurship, journalism, teaching instruction, science, technology, engineering and mathematics (STEM) competence.

Participant - a person or group of persons that has directly participated in, and benefits from, an ECA exchange program. Note that participants also include ECA’s Office of Private Sector Exchange’s (ECA/EC) “Exchange Visitors.”

Count Frequency Each respondent should only be counted once in the reporting year following their program completion.

Gender Age Country of origin Home state Race Participant type:

American, foreign, reciprocal

To be completed by the applicant

Survey (self-reported data)

NOTE: Question 1 is also included in Indicator 4.0.01 and 4.0.03. If at least two of these questions are to be included in a survey, question 1can be asked once, with two multiple follow-up questions based on an affirmative response.

Did you increase your personal and/or professional skills as a

| 24

Indicator

Definition Disagg. Target Data Source and Required* Survey Question

Calculations To determine the percent value, numerator is number of respondents who select at least one answer option EXCEPT for “none of the above skills” in the second question while the denominator will be the total number of survey respondents. Divide the numerator by the denominator and then multiply by 100 for the percent value.

result of your participation in this program?

Please select the professional skills you increased. Select all that apply.

Project management skills Business management skills Entrepreneurship skills Journalism skills Teaching skills Science competence Technology competence Engineering competence Mathematics competence Other: ______ [write-in]

None of the above skills E4.0.09: Percent of participants reporting an increase in language skills as a result of exchange program participation

Participant - A person or group of persons that has directly participated in, and benefits from, an ECA exchange program. Note that participants also include ECA’s Office of Private Sector Exchange’s (ECA/EC) “Exchange Visitors.”

Count Frequency Each respondent should only be counted once in the reporting year following their program completion.

Calculations

Gender Age Country of origin Home state Race Participant type:

American, foreign, reciprocal

To be completed by the applicant

Survey (self-reported)

1. Did you improve your skills in another language as a result of your participation in the exchange program

If YES to question 1, ask questions 2 and 3:

| 25

Indicator

Definition Disagg. Target Data Source and Required* Survey Question

When to Collect

To determine the percent value, numerator is number of respondents who answered yes in question 1, while the denominator will be the total number of survey respondents. Divide the numerator by the denominator and then multiply by 100 for the percent value

1. Which of the following best described your ability to speak in that foreign language before your participation in the program exchange?

No ability at all Able to communicate only in a very limited capacity Able to satisfy basic survival needs and minimum courtesy requirements

Able to satisfy routine social demands and limited work requirements

Able to speak with sufficient grammatical accuracy and vocabulary to discuss relevant professional areas

Able to speak fluently and accurately in all situations

Proficiency equivalent to that of a native speaker

[Use display logic to show those choices with abilities greater than the selection in question 2.]

| 26

Indicator

Definition Disagg. Target Data Source and Required* Survey Question

When to Collect

Which of the following best describes your ability to speak in that foreign language after the program exchange?

Able to communicate only in a very limited capacity

Able to satisfy basic survival needs and minimum courtesy requirements

Able to satisfy routine social demands and limited work requirements

Able to speak with sufficient grammatical accuracy and vocabulary to discuss relevant professional areas

Able to speak fluently and accurately in all situations

Proficiency equivalent to that of a native speaker

E4.0.14: Percent of participants reporting new digital skills/competenc ies gained from virtual exchange

(VE)

programming

Participant - A person who has directly participated in an ECA exchange program. Note that participants also include ECA’s Office of Private Sector Exchange’s “Exchange Visitors.”

Virtual Exchange - A sustained, multi-way interaction, that are online, used to build networks and shared knowledge around policy priorities. For a program to be considered a virtual exchange, participants must engage 51% or more virtually. A series of virtual components can be considered an exchange if they are (1) originated as a unit with a collective start and end date and (2) are organized around a unifying theme. For example: a series of MOOCs would be

Gender Age Country of origin Home state Race Participant type:

American, foreign, reciprocal

To be completed by the applicant

Survey (self-reported)

Did you gain new digital skills/competencies as a result of your participation in the exchange program?

Yes No

| 27

Indicator

Definition Disagg. Target Data Source and Required* Survey Question

When to Collect considered an exchange if they are offered as a set (of say six classes) and are centered on a specific topic (for example, media literacy). Offering MOOCs as an online resource is not considered a virtual exchange program.

Virtual Elements - Expand the reach of programs and increase engagement for participants and alumni. Unlike virtual programs, virtual elements are single, stand-alone components, offered either as add-on enrichment to an in-person exchange or as general outreach by ECA. For example, offering single MOOCs to the general public or virtual pre-departure orientation webinars are both considered virtual elements. Additionally, online spaces for participants to communicate (such as a program Facebook page) is considered a virtual element.

Digital skill/competencies - Digital skills are defined as a range of abilities to use digital devices, communication applications, and networks to access and manage information, and to participate in a virtual exchange. Skills include: knowledge about various low and high-tech technologies (such as the internet, digital video, software programs, etc.), knowledge of how various technologies can be used to communicate with others, knowledge required by teachers for integrating technology into their classrooms, and/or creating content for various technologies.

Count Frequency Each respondent should only be counted once in the reporting year following their program completion.

Calculations

| 28

Indicator

Definition Disagg. Target Data Source and Required* Survey Question

When to Collect

To determine the percent value, numerator is number of respondents who respond “yes” to the question, while the denominator will be the total number of survey respondents. Divide the numerator by the denominator and then multiply by 100 for the percent value.

This indicator counts participants involved in virtual exchange programs (see definition above) and WILL NOT count participants ONLY engaging with virtual program elements.

ECA Sub-Objective 4.1: Participants engage in language, academic, professional, and cultural exchange programs

E4.1.01: Total number of participants (core indicator)

Participant - A person who has directly participated in an ECA exchange program. Note that participants also include ECA’s Office of Private Sector Exchange’s (ECA/EC) “Exchange Visitors.”

Counting Frequency Each participant should only be counted once per semi-annual reporting cycle following their program completion.

Gender Age Country of origin Home state Race Participant type:

American, foreign, reciprocal

To be completed by the applicant

Administrative data from award recipients - not collected from the participant…