PWS Attachment 3 REL Stakeholder Feedback Survey Guidance.docx

Overview and Administration of the REL Stakeholder Feedback Surveys

Purpose of the Stakeholder Feedback Survey

The REL Program Stakeholder Feedback Survey (SFS) is intended to capture timely feedback on partnership participation, activities and events offered by the RELs. Data obtained from the survey can be used for program improvement. For example, the data can assist REL staff in understanding the utility of activities, identifying areas of stakeholder need or concern, and identifying future research and technical support activities. In addition, these data can help RELs in determining the extent to which they are achieving the expected outcomes of the REL program (as outlined in the 2017-2022 REL Performance Work Statement).

As mentioned above, the SFS collects specific feedback on activities and events, but RELs also engage in informal conversations to learn the views of stakeholders in their region about products or services received. Data from the SFS might prompt some of these informal conversations. The SFS is designed to be just one component of the REL feedback system.

The SFS survey is to be administered under OMB control number 1880-0542.

SFS Content

The new survey consists of three distinct modules:

1. Research Partnership Participation

2. Training, Coaching, and Dissemination Events

3. Training and Coaching Follow-Up (Optional) Modules 1 and 2 Module 1 is intended to collect information from REL Partnership members once per year. This Module specifically focuses on participation in the research alliance/partnership; it may also be appropriate for partnership members to complete other survey modules for specific TCTS activities they participate in throughout the year.

Module 2 is intended to collect information from people who participate in REL training, coaching, and dissemination events.

Both Modules 1 and 2 contain three similar sets of required items that ask about the information and supports received from the REL. The first set of items ask about the extent to which the respondent agrees or disagrees with statements related to the information and support received. The second set of required items asks about whether the respondent has taken or plans to take various actions based on the information and support. The third set of required items is open-ended and asks about what aspects of the support were most and least helpful. The required items can be supplemented with additional items selected from the lists of optional items.

Module 3 Module 3 is optional and should only be administered to participants who participated in sustained training or coaching activities (i.e. training or coaching activities that included multiple sessions). This module should not be administered to dissemination event or webinar participants.

Each of the modules end with two demographic items about the respondent’s primary occupation and state/territory where their work is based.

SFS Development and Approval Process

Required and Optional Items Each SFS may include up to 15 total items. The bank of optional items is organized into categories to help RELs select items that are most appropriate and of interest; note that the category names should not be included in the administered survey.

Module 1: Research Partnership Participation RELs must include item 1 or 2 (or both items) depending on the focus of the partnership.

· Item 1: I have greater capacity to use research on [topic] to inform decisions about policies or practices in my agency/organization/school.

· Item 2: I have greater capacity to use data on [topic] to inform decisions about policies or practices in my agency/organization/school.

RELs must include item 3 or 4 (or both items) depending on the focus of the partnership.

· Item 3: I have used research presented by the REL to inform decisions in my agency/organization/school.

· Item 4: I have used data in new ways to inform decisions in my agency/organization/school All three open-ended items are required:

· Item 5: What aspects of the partnership support were most helpful and why?

· Item 6: What aspects of the partnership support were least helpful?

· Item 7: How would you suggest improving the partnership support provided by the REL?

RELs can add additional items to the module by selecting from the bank of optional items provided by IES.

Module 2: Training, Coaching and Dissemination Events RELs must include item 1 or 2 (or both items) depending on the focus of the project. These items are not required for dissemination events.

· Item 1: I have greater capacity to use research on [topic] to inform decisions about policies or practices in my agency/organization/school.

· Item 2: I have greater capacity to use data on [topic] to inform decisions about policies or practices in my agency/organization/school.

RELs must include one or more of items 3-5 depending on the focus of the project. These items are not required for dissemination events.

· Item 3: I expect to apply information from the [activity/coaching/support] in my work.

· Item 4: I have used research presented by the REL to inform decisions in my agency/organization/school.

· Item 5: I have used data in new ways to inform decisions in my agency/organization/school All three open-ended items are required:

· Item 6: What aspects of the [activity/coaching/training/support] were most helpful and why?

· Item 7: What aspects of the [activity/coaching/training/support] were least helpful?

· Item 8: How would you suggest improving the [activity/coaching/training/support] provided by the REL?

RELs can add additional items to the module by selecting from the bank of optional items provided by IES.

Module 3: Training and Coaching Follow-Up There are no required items for this module. RELs may select up to 15 of the items from the bank of optional items provided.

Customizing Items The REL should alter the survey instructions to add the missing information where indicated by highlighted placeholder text (such as Partnership Name and REL Region Name).

Many items include words in brackets, such as [topic], [methodology], [activity/coaching/training/support], and [stakeholders]. These items may be customized in one of two ways:

1) The REL can fill in the applicable topic, methodology, type of activity, or type of stakeholders. For example, the item “I have greater capacity to use research on [topic] to inform decisions in my agency/ organization/school” could be customized to read “I have greater capacity to use research on chronic absenteeism to inform decisions in my agency/organization/school.”

2) RELs can slightly revise the item to not include the bracketed portion of the question. For example, the item “I have greater capacity to use research on [topic] to inform decisions in my agency/ organization/school” could be customized to read “I have greater capacity to use research to inform decisions in my agency/organization/school.” Note that many items do not lend themselves to this kind of revision.

COR Approval

RELs should submit a proposed SFS for each partnership and project to their COR, even if the REL only plans to administer the required items. The COR will review and approve each SFS before it is administered.

SFS Administration

Who Should Complete the SFS?

Module 1 should be administered to members of REL partnerships. This module is intended to collect information about how partnership members’ capacity has been built and how they have used the information in their work. Therefore, only partnership members who received direct services from the REL should be surveyed.

Module 2 should be administered to people who participate in REL training, coaching, and dissemination events.

Module 3 is optional and can be administered to the same people who completed Module 2.

Time to Complete the SFS We estimate that each module will take no more than 10 minutes to complete.

Frequency of SFS Administration Module 1 should be administered once per year beginning in fall 2019. RELs should plan to administer this module to partnership members near the end of each contract year, no later than November 30th of each year. Note that Module 1 is for research partnership participation. If a coaching activity or training is delivered specifically for a research partnership, then the associated training/coaching module (Module 2 and possibly Module 3) should also be administered at the completion of that activity.

Module 2 should be administered at the end of an activity or event. For ongoing coaching or training projects with multiple sessions, the SFS can be completed at the end of the final session so that it reflects all the support provided, or it can be administered after a significant milestone. However, if the participants vary from session to session, the SFS should be completed at the last session attended by each participant.

Module 3 is optional. It can be administered any number of weeks or months after a training/coaching project ends or after a significant project milestone occurs. The REL should propose the timeline for administration.

Administration Modes The SFS may be conducted in person, by mail, or online. The survey may be translated into other languages, if necessary, for certain stakeholder populations. The survey is not approved for administration over the telephone. Specific guidance for each method of survey administration is provided below, along with suggestions for increasing response rates.

In person The SFS can be administered in person, via a paper copy. Tips for increasing response rates at an event include:

· Include the SFS in the packet of materials. Print the SFS on different colored paper to differentiate it from other materials in the packet.

· Ask the event facilitator to point out the survey during the agenda overview. If any participants need to leave before the end of the session, ask them to complete the SFS based on their experience at the event before they leave.

· Have a designated place for the SFS to be returned (sign-in table, box, etc.).

· Allow time for completion before the end of the session. (Include this on the agenda to ensure that time is provided.)

· Provide talking points for the facilitator to share prior to completion of the SFS:

· Explain why the SFS is being administered. Be explicit and say that our REL, the REL program, IES, and the U.S. Department of Education value participants’ opinions and suggestions for improvement. Stress that we are collecting data to improve our efforts.

· Note that the SFS is an opportunity for the participants to share their reflections about the content shared during this event.

· Tell participants that if they do not understand the question, they should let you know and you will explain.

· Remind participants that their responses are anonymous.

· If the REL event is being held in conjunction with another event (a session in a larger conference) that has its own feedback form/survey, RELs still need to administer the SFS.

By mail The SFS may be administered via mail. Suggestions for increasing response rates include:

· Include a cover letter that explains why the SFS is being administered:

· Be explicit and say that our REL, the REL program, IES, and the U.S. Department of Education value participants’ opinions and suggestions for improvement. Stress that we are collecting data to improve our efforts.

· Note that the SFS is an opportunity for the participants to share their reflections about an activity, event, training, or workshop.

· Remind participants that their responses are anonymous.

· Provide a deadline for sending back the SFS; a week from receipt of the letter is reasonable.

· Provide a URL to give the respondent an option for completing the SFS online.

· Include a return envelope with postage.

· Send a follow-up postcard to remind respondents to complete the survey after the deadline has passed.

Online The SFS may be administered online. Some suggestions for increasing online response rates are:

· At the end of an online event or webinar, provide an active link that brings the participants to the online SFS; some webinar programs can send participants directly to another website (e.g., the SFS page; the REL home page) at the end of the event.

· Include either a link or a PDF document of the survey as part of a “thank you for attending” email sent to event participants within a day of the event.

· Send a follow-up email to all participants a week after the event, with a link or a PDF attachment of the SFS instrument.

Contact Information for Respondents with Questions Per the information at the bottom of the survey, if respondents have any comments or questions about the SFS, they may contact: Janelle Sands, Institute of Education Sciences, U.S. Department of Education, 550 12th Street, SW, Room 4113, Washington, D.C. 20202; Janelle.Sands@ed.gov.

SFS Data Entry and Submission

RELs should use the provided Excel SFS reporting templates when submitting their SFS summaries to their CORs. SFS reporting templates for each module are available on the REL Intranet. The Excel files contain two worksheets for data entry:

· The first worksheet is for reporting on close-ended items. The top part of the worksheet collects background information about the partnership or project activity, including the date, partnership/activity title, number of participants, and number of surveys returned. The module 2 worksheet includes two questions that ask about the focus of the activity: the primary topic the work is focused on and the primary type of decision/action the work is intended to inform. Please select at least one primary topic and one primary decision/action from the drop-down menus.

The bottom part of the worksheet is the SFS summary table for that module. RELs should enter each item as it was worded in the final survey administered to stakeholders, along with the corresponding IES item number (the number used in the SFS Modules provided by IES). In most cases, the IES item number will differ from the item number in the survey itself. For example, if a REL chooses to use optional item 14, it should be entered into the SFS reporting spreadsheet as item 14, even if it was not #14 on the survey administered to the stakeholders.

For each item, enter the number of responses received per response option. The worksheet will automatically compute response rates and the percentage of Agree or Strongly Agree responses.

Note: Because these reports are not public, RELs are not obliged to suppress cells with small sample sizes.

· The second worksheet is for the open-ended and demographic questions. RELs should enter all responses, with one row per respondent, in the columns corresponding to each question.

To standardize data entry, some responses are to be selected from drop-down menus (REL name on the first worksheet, respondent’s occupation and state/territory/jurisdiction on the second worksheet).

The REL should email the completed SFS reporting Excel file to their COR.

Stakeholder Feedback Survey (SFS) Guidance (February 7, 2020) Page| 2

Regional Educational Laboratories Stakeholder Feedback Survey

Module 1: Research Partnership Participation Please take a few minutes to provide feedback about your experience participating in the <Insert Partnership Name> with Regional Educational Laboratory (REL) <Insert Region Name> over the past year. Your responses are voluntary and will be aggregated to inform future REL activities. Your responses may not be disclosed or used in identifiable form for any other purpose except as required by law (Education Sciences Reform Act of 2002 [ESRA 2002] [Section 9573, 20 U.S. Code]).

Research Partnership: <insert partnership name> Survey Completion Date: _______________

At least one item from each of these groups of items is required to be administered:

· Items 1 and/or 2

· Items 3 and/or 4

Based on the information and support received through your participation in this partnership over the past year, please indicate the extent to which you agree or disagree with the following statements:

1. I have greater capacity to use research on [topic] to inform decisions about policies or practices in my agency/organization/school.

2. I have greater capacity to use data on [topic] to inform decisions about policies or practices in my agency/organization/school.

For the questions below, please indicate whether you have taken or plan to take the following actions based on the information and support received through your participation in this partnership over the past year.

3. I have used research presented by the REL to inform decisions in my agency/organization/school.

4. I have used data in new ways to inform decisions in my agency/organization/school.

Required open-ended items

5. What aspects of the partnership support were most helpful and why?

6. What aspects of the partnership support were least helpful?

7. How would you suggest improving the partnership support provided by the REL?

This module can be supplemented with items selected from the lists of optional items for a total of up to 15 items.

Module 2: Training, Coaching, and Dissemination Events Please take a few minutes to provide feedback based on your experience participating in the training or coaching event or webinar provided by Regional Educational Laboratory (REL) <Insert Region Name> on <Topic>. Your responses are voluntary and will be aggregated to inform future REL activities. Your responses may not be disclosed, or used, in identifiable form for any other purpose except as required by law (Education Sciences Reform Act of 2002 [ESRA 2002] [Section 9573, 20 U.S. Code]).

Activity Title: <insert activity title> Survey Completion Date: _______________

For Training and Coaching projects, at least one item from each of these groups of items is required to be administered (items 1-5 are not required for Dissemination Events):

· Items 1 and/or 2

· Items 3 and/or 4 and/or 5

Based on the information and support received through this activity, please indicate the extent to which you agree or disagree with the following statements:

1. I have greater capacity to use research on [topic] to inform decisions about policies or practices in my agency/organization/school.

2. I have greater capacity to use data on [topic] to inform decisions about policies or practices in my agency/organization/school.

3. I expect to apply information from the [activity/coaching/support] in my work.

For the questions below, please indicate whether you have taken or plan to take the following actions based on the information and support received.

4. I have used research presented by the REL to inform decisions in my agency/organization/school.

5. I have used data in new ways to inform decisions in my agency/organization/school.

Required open-ended items

6. What aspects of the [activity/coaching/training/support] were most helpful and why?

7. What aspects of the [activity/coaching/training/support] were least helpful?

8. How would you suggest improving the [activity/coaching/training/support] provided by the REL?

This module can be supplemented with items selected from the lists of optional items for a total of up to 15 items.

Optional Items for Modules 1 and 2

Category

Category

Optional Open-Ended Items

1. What additional information, support, tools, or activities on this topic would help you apply the information to your own work?

2. What action steps do you plan to take as a result of the [activity/coaching/training/support]?

3. Please list more specific areas of need or interest to your agency/organization/school for future REL support or studies:

Module 3: Training and Coaching Follow-Up (Optional) Please take a few minutes to provide feedback about your experiences since you participated in the following training or coaching support provided by Regional Educational Laboratory (REL) <Insert Region Name> on <Topic>. Your responses are voluntary and will be aggregated to inform future REL activities. Your responses may not be disclosed or used in identifiable form for any other purpose except as required by law (Education Sciences Reform Act of 2002 [ESRA 2002] [Section 9573, 20 U.S. Code]).

Activity Title: <insert activity title> Activity Date: <insert activity date(s)> Survey Completion Date: _______________

Up to 15 items total can be selected from the lists below for this module.

For the questions below, please indicate whether you have taken or plan to take the following actions based on the information and support received through this activity:

Category

Open-Ended Items

1. What aspects of the [activity/coaching/support] were most helpful and why?

2. What decisions have you or your agency/organization/school made, or steps have you taken, as a result of the [activity/coaching/support]?

3. If you were unable to make a decision or take action, why? What additional support would be helpful?

4. What additional information, support, tools, or activities on this topic would help you apply the information to your own work?

Respondent Information A. Which of the following best describes your primary occupation? (Choose all that apply.)

School Level

· Teacher/Educator

· Principal/Vice-Principal

· Instructional coach

· Other school-level administrator

· Librarian

Local Education Agency

· School board member

· School district central office staff – research office

· School district central office staff – program office

· School superintendent/Assistant superintendent

State Education Agency

· State-level education administrator

· State-level advisor or board member

· Other state-level staff (please specify)

State Government

· State legislator or legislative staff member

Tribal Government

· Council member

· Tribal member

· Education specialist

· Other tribal government staff (please specify)

Other

· Adult Education staff

· Community Organization staff

· Higher Education/Postsecondary staff

· Other (please specify) Options for Pacific Region only

· National-level education administrator

· National-level education central office staff

· State-level education administrator

· State-level education central office staff

B. In which State/Territory/Jurisdiction is your work based? ______________________________

Thank you for your feedback.

Paperwork Burden Statement According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless such collection displays a valid OMB control number. The valid OMB control number for this information collection is 1880-0542. The time required to complete this information collection is estimated to average 10 minutes per response, including the time to review instructions, search existing data resources, gather the data needed, and complete and review the information collection. If you have any comments concerning the accuracy of the time estimate(s) or suggestions for improving this form, please write to: U.S. Department of Education, Washington, D.C. 20202-4700. If you have comments or concerns regarding the status of your individual submission of this form, write directly to: Janelle Sands, Institute of Education Sciences, U.S. Department of Education, 550 12th Street, SW, Room 4113, Washington, D.C. 20202; Janelle.Sands@ed.gov.

image1.png