Attachment_3._SOW____1_(Collective_Impact_Measurement).pdf

ATTACHMENT 3: SOW # 1 (Collective Impact Measurement)

9/12/2017

BPA TASK ORDER

STATEMENT OF WORK

NEA Arts Education Collective Impact Measurement

SECTION B

SUPPLIES/SERVICES AND PRICES

B.1 GENERAL

As part of its Strategic Plan for 2014-2018, the National Endowment for the Arts (NEA) has pledged to promote public knowledge and understanding about the contributions of the arts. In February 2014, the

NEA released a strategic plan establishing a new set of goals, outcomes, and performance measures for the agency. In support of this goal, the NEA is committed to providing the public with access to information about the value and impact of the arts in society as a whole.

The NEA seeks a contractor to build an evidence-based logic model and measurement model for the

NEA’s Arts Education Collective Impact program. Developing a common measurement system across grant projects is an important goal for the NEA, as it would position the Collective Impact program to use grantee data for an outcome evaluation study in the near future.

B.2 PRICING

ITEM

No.

SUPPLIES OR SERVICES QTY UNIT UNIT PRICE AMOUNT

Collective Impact Logic

Model & Measurement

Model* 1 $ $

*As described in this RFP

FIRM FIXED PRICE GRAND TOTAL*:

(Not to Exceed)

1. All-inclusive figure (including, but not limited to, any and all overhead, related costs, travel, etc.).

2. The NEA is a federal agency and exempt from sales tax.

3. Vendors shall expand on/include additional pricing/related information as needed.

4. Pricing submitted must use unit prices (hourly rates, etc.) provided in response to the BPA solicitation; these unit prices must track back to the overall BPA pricing.

B.3 CONTRACT TYPE & TERM

B.3.1 Task Order Type

Firm-fixed price task order.

B.3.2 Contract Term

Period of Performance: 57 weeks.*

*This period of performance allows for approximately 3 extra months beyond the time-frame specified in deliverables schedule specified (in Section F). The additional time allows for unforeseen delays during the contract, external factors, public holidays, etc.

Location: The majority of work shall be conducted at the contractor’s place of work. Some meetings or work sessions will take place at the National Endowment for the Arts offices (located at 400 7th St. SW, Washington, D.C. 20506.

SECTION C

DESCRIPTION/SPECIFICATIONS/STATEMENT OF WORK

C.1 GENERAL BACKGROUND

The NEA made its first Collective Impact grant awards in 2014. These awards are intended to increase student access to arts education through collective, systemic approaches. Collective Impact projects aim to ensure that all students across entire neighborhoods, schools, school districts, and/or states – in communities of all sizes – participate in the arts over time. In an article titled "Collective Impact" in the

Stanford Social Innovation Review, authors John Kania and Mark Kramer have shown how collective efforts have a greater impact on social change than individual efforts.

Collective Impact grant guidelines require that projects embrace the following principles, which may be ongoing and occur at any point during the project:

Partnership: Cross-sector partners work to determine a common vision, define goals, develop strategies, and identify measurable objectives for arts education. Partners may include arts organizations, units of government, school systems, funders, community organizations, or institutions of higher education. Priority will be given to projects that include a managing partner that is the coordinating entity, and involve at least three cross-sector organizations, one of which is an arts/cultural organization.

Data: Data informs decision making. This may include asset mapping of community resources, collecting student data, or creating new data collection tools.

Planning: A plan outlines system-wide arts education implementation. This should include a description of each partner's role in achieving the common vision, as well as plans for communication among the partners and sustainability.

Programming: Activities support the plan. Programming may include services to students, professional development, curriculum design, or convening stakeholders.

Shared Measurement: A shared measurement system is an evaluation tool that assesses the progress of each project partner's work towards common outcomes—increasing student participation in arts education and, as appropriate, societal well-being for students.

Collective Impact projects are multi-year, ongoing, systemic initiatives. All phases of a project - building partnerships, data collection, planning, programming, and shared measurement - are eligible for support.

Additionally, Collective Impact projects are identified as “emerging” or “sustaining.” “Emerging” projects are in the initial phase of work to establish an arts education plan. Projects may include cultivation of partners, convenings, collection of data, or creation of an arts education plan. “Sustaining” projects have an arts education plan in place. These projects may continue work from the emerging phase, http://www.ssireview.org/articles/entry/collective_impact be in the programming and evaluation stage, or be scaling up proven efforts to increase arts education access. These projects must demonstrate how they are disseminating project information to the fields of arts education, public education, and beyond.

Building on the NEA’s 2015 Arts Education Convening on Shared Measurement, the objective of this task order is to facilitate the development of an evidence-based logic model and measurement model in close consultation with NEA Arts Education Collective Impact grantees. Developing a common measurement system across grant projects is an important goal for the NEA, as it would position the

Collective Impact program to use grantee data for an outcome evaluation study in the near future. This information will also provide the NEA with valuable information to 1) inform guidelines revision

(including refining, if needed, the program model) and 2) document grantee successes, challenges, and successful practices for sharing with the grantee community.

C.2 PROJECT SCOPE

The contractor shall perform the below listed duties:

Attend a project kick-off meeting with the project director;

Prepare a detailed timeline and work plan, including regular communications with NEA project director;

Prepare a brief literature scan of key works on measurement and evaluation of Collective Impact projects in education and youth development, and, as appropriate, the broader social services sector;

Describe the nature and impact of NEA Arts Education Collective Impact projects and develop a draft program logic model using grant data (grant applications, final descriptive reports) and other grant documentation from the program’s inception; and consultation with NEA staff and grant project directors/evaluators;

Test and refine the NEA Arts Education Collective Impact logic model and solicit ideas about viable measurement indicators and methods through consultation with grantees;

Prepare a final report proposing a NEA Arts Education Collective Impact logic model and measurement model.

C.3 DETAILED SPECIFICATIONS

The contractor shall perform the following tasks:

C.3.1 Project Kick-off Meeting

The contractor shall meet with the NEA project director to review project tasks and co-plan the project.

At this meeting, the NEA will present, in electronic format to the contractor data, documentation, and resources that will support in part the completion of task C.3.4. Meeting time will not exceed two (2) hours. This meeting may take place on-site or via videoconference or teleconference. Within a week, the contractor shall prepare a memorandum summarizing the discussion and making note of decisions made.

C.3.2 Timeline/Work Plan/Communications

The contractor shall prepare a detailed timeline and work plan for accomplishing the remaining tasks of the contract, incorporating input from the kick-off meeting. The contractor shall first submit a draft timeline and work plan for review by the project director following the kick-off meeting. Following the receipt of feedback from the project director, the contractor shall prepare and submit a final timeline and work plan, which will include a detailed description of all analytical methods, data sources, and consultation methods to be used for this project, proposed table of contents for the interim and final reports, meeting schedule, and schedule for all deliverables inclusive of multiple drafts and deliverable review periods. The work plan shall include participation in regular teleconferences with the project director and preparation of minutes documenting such meetings.

C.3.3 Literature Review on Collective Impact Project Measurement in Education

The contractor shall identify and review key works of the academic, theoretical, and “grey” research and evaluation literature on Collective Impact projects in education and youth development and, as appropriate, the broader social services sector. The intent of this literature scan is to situate the development of a logic model and measurement model of the NEA Arts Education Collective Impact program in the larger context of the research and evaluation of other Collective Impact and/or complex programs in the United States. The literature review shall review no fewer than 25 but not exceeding 50 high quality works from the relevant research and evaluation literature. The deliverable associated with this task will be a brief white paper (not to exceed 6 pages, excluding title page) with a synthesis of high-level findings relevant to this project and a detailed literature review summary table (no page limit) presented in the appendix. At the minimum, the literature review summary table shall include each work’s citation, brief description, relevant findings, and programmatic or other unique characteristics that inform the findings, including program intensity and duration, populations studied, etc.

C.3.4 NEA Arts Education Collective Impact Descriptive Analysis and Draft Logic Model Development

The contractor shall conduct a descriptive analysis describing the nature and impact of the NEA

Collective Impact program and prepare a draft logic model based on this analysis. In addition to identifying common inputs, activities, outputs, and intended outcomes (short- and long-term) across projects, the contractor shall examine the funding streams and partnerships and effective practices that characterize Collective Impact projects as well as common measurement practices and indicators used by individual projects. The task deliverable shall be an interim report that summarizes findings and presents a draft logic model developed in consultation with NEA staff.

In support of this task, the contractor shall conduct the following data collection and/or analyses:

C.3.4.a Review and analyze Collective Impact grant documentation and data. The contractor shall review and analyze all Collective Impact grant documentation and data from the program’s inception in FY 2014 to include the following: application guidelines, grant application data associated with Collective Impact awards (as of Aug. 21, 2017, 39 grant awards have been made), grantee final descriptive report data (as of Aug. 21, 2017, 14 reports have been received), and related documentation from NEA-funded projects (e.g., grantee reports, measurement models, websites). Grant application guidance is available on the NEA website:

https://www.arts.gov/grants-organizations/art-works/arts-education. Final reporting requirements, including copies of final descriptive report forms for this three-year time period, can be viewed on the NEA website: https://www.arts.gov/grants/manage-your-award. Please note that reporting requirements for Arts Education grants, including Collective Impact, changed between FY 2014 and FY 2015.

The contractor is expected to use appropriate quantitative and qualitative analytical techniques for this analysis. Results in the interim report shall be presented separately for “emerging” and

“sustaining” project types.

C.3.4.b Consult with Arts Education Collective Impact project directors and/or evaluators. For

Collective Impact project directors and/or evaluators, the contractor shall develop an interviewee https://www.arts.gov/grants-organizations/art-works/arts-education https://www.arts.gov/grants/manage-your-award list and an interview protocol for NEA input and approval. No more than nine individuals external to Federal government shall be interviewed by the contractor to ensure compliance with the Paperwork Reduction Act. The contractor shall be responsible for contacting interviewees and for scheduling and coordinating the interviews following the NEA’s approval of the interviewee list. The NEA shall provide an introductory letter to interviewees that the contractor shall use to introduce the purpose of, and establish the legitimacy of, the data collection. The NEA reserves the right to observe selected interviews as part of its oversight of this contract with the explicit permission of the interviewee. The contractor shall provide de-identified interview transcripts in electronic format to the NEA at the conclusion of this task.

C.3.4.c Consult with NEA Staff. Following the analysis of data from the preceding two sub-tasks, the contractor shall prepare and deliver a presentation to the NEA project team, composed primarily of Arts Education and Research & Analysis staff, summarizing the findings to date. The contractor shall then facilitate a logic modeling session with staff to gather input for the logic model. Both activities shall take place at the NEA office in Washington, DC, during a half-day meeting scheduled at least a month in advance. Based on input collected from NEA staff at the logic modeling session, the contractor shall generate the first draft of a logic model for NEA review. NEA staff will then review and provide feedback on this first draft. Based on NEA feedback, the contractor shall prepare a second draft of the logic model, which will be tested and refined with grantees in task C.3.5.

C.3.5 Logic Model Refinement through Grantee Consultation

Following the NEA’s acceptance of the report, the contractor shall facilitate face-to-face or online discussions with all current and previous NEA Arts Education Collective Impact grantees (26) to 1) test and refine the logic model and 2) solicit ideas about viable measurement indicators and methods.

Grantees must receive (in advance of the discussion) copies of the literature review and the interim report, along with a pre-assignment to test the logic model using their knowledge of their own Collective Impact program and identify existing measurement strategies. Discussions will be held with small groups of grantees with at least two events anticipated in order to accommodate a large sample of grantees. The contractor is responsible for all technical logistics associated with the discussions, including scheduling and coordinating webinars or on-site meeting space; an NEA staff member will be present to introduce the consultant and to observe each discussion. The contractor shall produce a transcript of each event for delivery to the NEA.

C.3.5 Final Logic and Measurement Model Development

The contractor shall revise the interim report incorporating findings from the consultation with grantees.

The final logic model and measurement model, which identifies appropriate high-level indicators and methods of measurement to collect common data on outputs and outcomes from Collective Impact grantees, will form the basis for the final report. The final report also will include recommendations for guidelines revision (including refining, if needed, the program model) and improvements to performance data collection. The contractor shall expect up to three rounds of feedback from the project director and other NEA staff on the final report. The contractor may opt for a work session with NEA staff instead of a round of feedback.

C.4 BPA Terms and Conditions

All terms, conditions, guidance, and clauses included in the BPA solicitation apply to this task order, including (but not limited to) those for the following areas:

Inspection & Acceptance

Travel

Data Ownerships / Permissions / Etc.

Information Technology / Software Requirements

SECTION F

DELIVERIES OR PERFORMANCE

F.1 Delivery Schedule

Project Task Deliverable

Due Date (Weeks after award)

C.3.1 Project Kick-off Meeting

Memorandum summarizing the discussion and making note of decisions made 3 weeks

C.3.2. Timeline/Work

Plan/Communications Detailed timeline and work plan 5 weeks

Minutes of regularly scheduled teleconferences

Monthly or more frequently

C.3.3 Literature Review on

Collective Impact Project

Measurement in Education

White paper and detailed literature review summary table 13 weeks

C.3.4 NEA Arts Education

Collective Impact Descriptive

Analysis and Draft Logic Model

Development

Interim report 29 weeks

De-identified interview transcripts 29 weeks

C.3.5 Logic Model Refinement through Grantee Consultation Transcripts of events 35 weeks

C.3.5 Final Logic and

Measurement Model

Development Final report 45 weeks

The contractor shall provide for up to three (3) rounds of feedback from the Project Director on all draft deliverables, if needed.

SECTION L

INSTRUCTIONS, CONDITIONS AND NOTICES TO OFFERORS

1. General

The NEA encourages proposals that demonstrate expertise in program evaluation, the development of logic models and measurement models, and specifically in the measurement and evaluation of Collective

Impact projects. Expertise in facilitation of stakeholder participation in logic model development is also required. Expertise in arts education programs is preferred. Offerors are required to provide a sample report demonstrating expertise in the measurement and evaluation of Collective Impact projects.

2. Proposals

Responses to this SOW must be separate from responses to the overall BPA solicitation or any other SOW under the BPA. Responses must be CLEARLY MARKED to indicate which document it is a response to.

The proposal shall be no more than 8 pages, single spaced, in 12-point font. This page limit is exclusive of resumes, appendices, and all other attachments and supplemental materials. The NEA is open to alternative approaches to that proposed in this solicitation. Submissions shall include at a minimum each of the following:

The offeror’s understanding of the project through a discussion of the rationale and purpose of the project.

The offeror’s technical approach to completing the tasks and deliverables associated with this project, including a description of intended work for each task and a work schedule by task including timeline for deliverables.

Contractor’s relevant experience and past performance and qualifications for proposed personnel, including subcontractors. Resumes for all key personnel must be submitted as an appendix. The contractor’s past performance must be verified by the provision of three references, including phone number and email address.

Pricing: Include a detailed cost breakdown table that identifies the labor hours by personnel for each task, total cost by task, and total cost for the contract. The cost breakdown provided must be sufficient to allow the Government to evaluate the price and individual cost components. Pricing submitted must use unit prices (hourly rates, etc.) provided in response to the BPA solicitation; these unit prices must track back to the overall BPA pricing.

3. Budget

The budget for this project is $55,000. Proposals over this amount will NOT be considered.

SECTION M

EVALUATION FACTORS FOR AWARD

For each evaluation factor, a point scoring system will be used to rank all proposals, using a rating scale of 100 points. The evaluation will include the following four (4) major areas:

1. Understanding of the Project (10 points)

2. Technical Approach (35 points)

3. Personnel Qualifications and Experience (Past Performance) (35 points)

4. Price (20 points)

In order to select the winning offeror:

Proposals will be evaluated by a technical evaluation team in accordance with a 'best value' evaluation method.

Each proposal evaluated will be given a score for each evaluation criterion listed above, ranging from 0 to the maximum allowable for each category (with a maximum of 100 points for all categories).

Category scores will be summed up to receive a total score for each proposal to determine the proposal providing the best value.

As reflected in the weighted scoring stated above, the three (3) technical factors combined are more important than price.

Additionally, the agency may rank offerors from best to worst by making paired comparisons, trading off the marginal differences in capability and price. If one offeror has both the better capability and the lower price, then that offeror will be the better value. If one offeror has the better capability and the higher price, then the technical evaluation panel will decide whether the marginal difference in capability is worth the marginal difference in price. If the technical evaluation panel considers the better capability to be worth the higher price, then the more capable, higher-priced offeror will be the better value. If not, then the less capable, lower-priced offeror will be the better value.

EVALUATION CRITERIA

1. Understanding of the Project (10 points)

The contractor demonstrates a thorough understanding of the project, as shown through discussion of the rationale and purpose for the project. The contractor provides evidence of a clear understanding of the development of logic models and measurement models, and specifically in the measurement and evaluation of complex programs in education and youth development. The contractor does not simply repeat the statement of work, but shows an independent understanding of the issues and context inherent to the development of logic models and shared measurement models for complex programs in education and youth development.

2. Technical Approach (35 points)

The contractor’s approach is succinct and logical in format and fully addresses the tasks described in the statement of work. The proposed technical approach clearly specifies and describes the intended work under each task and presents a work schedule by task that includes a timetable for deliverables. The proposal discusses the contractor’s proposed approach to all tasks and deliverables, including grant data analysis, literature review, consultation and facilitation methods, logic model development, and measurement model development. The contractor identifies potential challenges to the project’s success and presents strategies for addressing them. Overall, the technical approach is of a high quality and demonstrates a thorough technical proficiency with logic model and shared measurement model development for complex education and youth development programs.

3. Personnel Qualifications and Experience (Past Performance) (35 points)

The contractor provides evidence of having conducted tasks similar to the work outlined in the RFP (with examples cited). The contractor demonstrates their personnel qualifications in educational program evaluation, the development of logic models and measurement models, and specifically in the measurement and evaluation of complex programs in education and youth development and/or, as appropriate, the social services sector, and facilitation of stakeholder participation in logic model development must also be demonstrated. The contractor provides a sample report demonstrating expertise in the measurement and evaluation of complex programs in education and youth development. The proposed project manager has demonstrated experience in managing projects of comparable scale and complexity (examples are cited). Resumes/qualifications for all proposed personnel (including subcontractors) are provided.

4. Price (20 points)

The proposed budget is appropriate to the administration of the project. The Government will evaluate the contractor’s proposed prices to make a determination that the costs are fair and reasonable in relation to the services provided. The contractor shall provide a reasonable breakdown of their costs to allow the

Government to make an assessment of the various components of the overall price. A detailed cost breakdown table that identifies the labor hours by personnel for each task, total cost by task, and total cost for the contract is provided. The breakdown also shall clearly identify the breakdown between labor costs and other direct costs.

END OF DOCUMENT