UACED_Technical_Approach__Southeast_Alaska_Cluster_Initiative.pdf

University of Alaska Center for Economic Development US Forest Service, Southeast Cluster Initiative Program Assessment

I. UACED Technical Approach – Southeast Alaska Cluster Initiative – Program Assessment The University of Alaska Center for Economic Development (UACED) is pleased to respond to your Request for a Program Assessment of the US Forest Service’s Southeast Alaska Cluster Initiative. We have a team with the necessary expertise to help you accomplish your program assessment goals.

Proper program assessments provide economic development officials with the ability to assess the direct impacts resulting from their activities. Since 2010, the US Forest Service has invested over $1,000,000 to support diversification of the economy of Southeast Alaska, as part of the transition plan for the Tongass National Forest. This funding went toward a number of initiatives, including cluster development efforts for Southeast Alaska.

Economic development program impacts are often difficult to successfully monitor. Program assessments that focus exclusively on quantitative impacts (unemployment rate, average wages, etc.)

can often miss the causal relationships. These quantitative measures usually fluctuate significantly, and depend on a multitude of factors above and beyond the efforts of program managers. By a similar token, qualitative measures can paint a misleading picture. They may suffer from small sample sizes and a response bias from participants. Making matters worse, without prior evaluation methods and metrics in place, performing a comprehensive and reliable ex post program evaluation can be an arduous, and often imprecise, task.

With these factors in mind, UACED favors adopting a holistic approach that focuses on clearly realized outcomes resulting from the Southeast Cluster Initiative. These outcomes will help to paint the overall picture of the relative success of the initiative, and will provide primary data supporting the efficacy (or lack there of) of the program. This knowledge will then serve to assist both economic development practitioners and funders alike.

Our Experience

UACED has longstanding experience providing direct data collection and analysis services. For instance, in 2014 the UACED led a statewide effort that investigated the reestablishment of the Manufacturing Extension Partnership. This statewide study focused on interviewing over 70 businesses and key stakeholders, as well as the convening of 7 industry or geographic-specific focus groups across the state.

The study was one of the most thorough investigations into Alaska manufacturing to date.

In addition, the UACED was a key partner in the recent Vibes study (Viable Business Enterprises in Rural Alaska). The study looked at the barriers to rural entrepreneurship and involved conducting key informant interviews with dozens of rural small business owners. The study served as a holistic assessment for the many factors involved in rural business formation, and identified areas for improving rural business startups and supporting their success.

Our Approach

Our methodology to evaluate the Southeast Cluster Initiative will be to focus on gathering primary data directly from those involved in the program, as well as those tangentially affected by the program as stakeholders. This multifaceted approach will aim to identify the successes of the program, general attitudes towards the initiative’s success, as well as providing a picture for how the cluster landscape has changed since the project began. This holistic approach will focus on targeting the key individuals and stakeholders involved in the six primary focus clusters of the initiative, including the following:

Identification of accomplishments and challenges within each cluster. Recognizing that not all of the original clusters are still active, the US Forest Service will need to assist in identifying the targets. Original clusters included:

o Ocean Products o Visitor Products o Renewable Energy o Mining Services and Supply o Research o Arts and Culture

Linkages between cluster development initiatives and regional economic development strategies and outcomes

Southeast Cluster Initiative Program Assessment – Data Collection Methods

Focus Groups

A key component of the program assessment will be the convening of applicable focus groups. Utilizing focus groups will allow program assessors the opportunity to gather detailed information about individual perceptions and opinions on the effectiveness of each initiative and its various components.

Focus groups also provide participants with the opportunity to build off of the ideas of other attendees, allowing for meaningful dialog and information exchange.

UACED will convene focus groups representing various key stakeholder groups that have participated in the Southeast Cluster Initiative, or those that have interacted with project leaders. Depending on

Holistic Cluster

Development Assessment

Key Informant Interviews

Focus Groups

Questionnaires participant interest and budget constraints, UACED anticipates convening focus groups covering one or more of the following stakeholder segments:

Cluster initiative working groups Innovation Summit attendees Economic development and other similar professionals involved in overseeing the delivery of project funds

Where it is not practical to organize a focus group, UACED will reach out to relevant stakeholders through key informant interviews. These interviews will allow UACED to gain a greater understanding of the overall effectiveness of the Southeast Cluster Initiative. UACED will also intentionally incorporate key informant interviews with USFS-identified critical stakeholders and others that may not be available to participate in a focus group.

The aim of CED’s three data collection methods (questionnaires, key informant interviews, and focus groups) will be to determine the extent to which the economic development and business communities believe that the Southeast Alaska development project has been effective. Questionnaires will be designed to identify specific impacts resulting from the program. Questionnaires will also seek to identify areas for improvement specific to the program.

Key Informant Interviews

Key informant interviews will serve as the foundation for UACED’s program assessment. Interviews will be conducted with key project participants and stakeholders, and will aim to identify the successes, challenges, and general attitudes toward the Southeast Cluster Initiative. It is anticipated that each interview will take approximately one hour. While interviews will follow a predetermined outline of questioning, CED staff will also use discretion in asking relevant follow up questions, and questions not outlined ahead of time. This way the USFS can benefit from both a standardized line of questioning (allowing for data to be compiled in aggregate) as well as a more holistic line of questioning aimed at providing more specific feedback and attitudes. It is anticipated that key informant interviews will take place via phone, although interviews will be conducted in person when practical.

Questionnaires

Questionnaires may be distributed to relevant stakeholders via an online platform, which will serve both qualitative and quantitative functions. Questionnaires will focus on identifying key success metrics for the Southeast Cluster Initiative, and will be be tailored to each relevant stakeholder group. Targeted groups include the economic development community, cluster initiative working groups, and the business community. While some questions will likely overlap, others will be tailored based on the group being approached. UACED will utilize an adaptive surveying tool, one in which the previous answers of a respondent determine the next questions that they will be asked, allowing UACED to gather data with greater specificity than through a traditional questionnaire.

Questionnaires will be designed to gauge individuals’ perceptions of the overall effectiveness of the Southeast Cluster Initiative. Questions will cover the following topics:

Attitudes regarding cluster development since the Southeast Cluster Initiative began to determine if improved networking or alliances of firms and businesses now exist

Opinions regarding the effectiveness of the Southeast Cluster Initiative – have foundational needs been addressed adequately (human resource training, access to technical assistance, improved access to capital, etc.)

Knowledge of key developments or improved industry competitiveness arising due to the Southeast Cluster Initiative

Questionnaires will be used sparingly in the program assessment process. Questionnaires will allow the UACED to expand its reach to additional businesses and segments above and beyond what it would otherwise have been able to achieve. It is the belief of the UACED that interviews and focus groups provide better data collection, but that questionnaires can provide an additional opportunity for reaching out to potential participants.

Program Budget

Conducting direct, one-on-one interviews is a labor-intensive process. Additionally, the cost of scheduling a focus group can be considerable in terms of planning and preparation. Therefore, in an effort to provide the greatest value to the US Forest Service, UACED proposes the following approach. It should be noted that UACED can work within a maximum value of $25,000, but the cost of each line item within the budget could vary. Projected costs include:

Organizing focus groups to occur consecutively on one UACED trip to Southeast Alaska. It is anticipated that UACED will be able to perform at least two focus groups on one trip. Budget permitting, UACED will organize a second trip to conduct a third and fourth focus group, if practical.

o Estimated budget for focus groups - $6,000 per two focus groups (assuming they are conducted on the same trip) o This cost includes approximately $2,000 for travel to Southeast Alaska, an additional $3,000 for organizing and conducting the two focus groups, and approximately $1,000 for hosting two focus groups.

Questionnaire formation, distribution, and data entry - $2,000 Key informant interviews - $300 per interview o This includes research and identification of target interviewees, the one hour interview, a post interview write up, and data entry into the system for analysis o It is anticipated that UACED will conduct at least 40 interviews as part of this assessment, for a total cost of $12,000

Data analysis of focus group, questionnaire, and interview data - $1,500 o This involves taking the gathered data and turning it into presentable charts and graphs

Final report written analysis - $3,500 o This anticipates a written analysis of approximately 20-30 pages detailing the results of the study. This includes a round of edits with the client, and final editing and presentation of the document.

Total Project Budget - $25,000

I. UACED Technical Approach – Southeast Alaska Cluster Initiative – Program Assessment