SIPPRA-NOFA 2024.pdf

The formatting of this Notice of Funding has been slightly modified for visual and printing purposes. The Notice of Funding published in the Federal Register is the official document.

United States Department of the Treasury

Office of Economic Policy

Notice of Funding Availability (NOFA): Social Impact Partnerships to Pay for Results Act (SIPPRA) Projects

Announcement Type: Initial announcement

Funding Opportunity Number: UST-SIPPRA-2024-002

Catalog of Federal Domestic Assistance (CDFA) Number: 21.017

Application Deadline: Applications under this NOFA must be submitted no earlier than February 12, 2024 and no later than 11:59 p.m. Eastern Time April 15, 2024 electronically via www.Grants.gov.

Funding Ceiling: $47 million ($40.9 million for social impact projects, $6.1 million for evaluations)

Period of Performance: Expected 48-60 months but project dependent.

Anticipated Time to Awards: October 15, 2024. There will not be a rolling review.

For More Information: Potential applicants are advised to review the Federal Register Notices for previous awards and other materials at https://home.treasury.gov/services/social-impact-partnerships/sippra-pay-for-results. Questions may be directed to Matthew Cook, SIPPRA Director, at SIPPRA@treasury.gov.

Summary: The Department of the Treasury (Treasury) is issuing this Notice of Funding Availability (NOFA) to invite applications from State and local governments for awards under the Social Impact Partnerships to Pay for Results Act (the “Act”). An award recipient will receive payment if a specified outcome of the social impact partnership project is achieved as determined by the project’s independent evaluator. The payment to the Awardee must be less than or equal to the value of the outcome to the federal government over a period not exceeding ten years from the date implementation commences. Awards made under this NOFA will be administered by Treasury or by another federal agency with expertise in the social benefits addressed in the proposed project. Treasury expects to award up to approximately $40.9 million in such competitive project grants under this NOFA. Treasury will prioritize projects that directly benefit children in order to meet the statutory threshold that 50 percent of awarded funds be used on awards that directly benefit children. In addition, State and local governments receiving project grants will be eligible to receive a grant for up to 15 percent of the project grant amount to pay for all or a portion of the cost of a statutorily required independent evaluation, which will https://www.federalregister.gov/documents/2023/11/30/2023-26174/social-impact-partnerships-to-pay-for-results-act-projects http://www.grants.gov/ https://home.treasury.gov/services/social-impact-partnerships/sippra-pay-for-results https://home.treasury.gov/services/social-impact-partnerships/sippra-pay-for-results mailto:SIPPRA@treasury.gov be paid regardless of whether outcomes have been met. Treasury expects up to approximately $6.1 million to be available to pay for the costs of independent evaluations under this NOFA.

Table of Contents A. Program Description 1

1. Program Purpose and Authorizing Legislation 1

2. Funding Type 1

3. Limitations 2

4. Pay for Results Framework 3

5. Outcome Valuation Methodology 6

6. Independent Evaluation 11

B. Federal Award Information 15

C. Eligibility Information 15

D. Application and Submission Information 16

E. Application Review Information 25

F. Federal Award Administration Information 32

G. Federal Awarding Agency Contact 39

H. Other Information 39

I. Appendix I: Example of Outcome Valuation Process 40

J. Appendix II: Integration of Managed Care Information/Data 43

K. Appendix III: Benefit-Cost Analysis Tools 44

A. Program Description

1. Program Purpose and Authorizing Legislation

In 2018, Congress appropriated $100 million to Treasury to implement the Social Impact Partnership to Pay for Results Act (the “Act”), which established a new grant demonstration program to encourage funding social programs that achieve results (the “SIPPRA program”).1 Under this NOFA, Treasury announces the availability of approximately $40.9 million for payments for successful outcomes of social impact partnership projects through grants to State and local governments, and, for required project evaluations, the availability of up to approximately $6.1 million. All awards provided through this NOFA are subject to funding availability.

As stated in the Act, the purposes of the SIPPRA program are

(1) To improve the lives of families and individuals in need;

(2) To redirect funds away from programs that, based on objective data, are ineffective, and into programs that achieve demonstrable, measurable results;

(3) To ensure federal funds are used effectively on social services to produce positive outcomes for both service recipients and taxpayers;

(4) To establish the use of social impact partnerships to address some of the Nation’s most pressing problems;

(5) To facilitate the creation of public-private partnerships that bundle philanthropic or other private resources with existing public spending to scale up effective social interventions already being implemented;

(6) To bring pay for performance to the social sector, allowing the United States to improve the impact and effectiveness of vital social services programs while redirecting inefficient or duplicative spending; and

(7) To incorporate outcomes measurement and randomized controlled trials or other rigorous methodologies for assessing program impact.2

2. Funding Type

The Act provides funds for two types of awards: (1) social impact partnership project grants, including grants to pay for independent evaluations for such projects, and (2) feasibility study grants. This NOFA only relates to funds for social impact partnership project grants and funds for the cost of a grantee’s independent evaluation. An awardee under this NOFA will receive a disbursement only if the awardee achieves one or more outcomes specified in the award agreement and if such outcomes are validated by an independent evaluation. The federal payment to the awardee for each specified outcome must be less than or equal to the value of the outcome to the federal government over a period not exceeding ten years from the date implementation commences. Payment for the independent evaluation will be made regardless of

1 For more information, please see the program webpage at https://home.treasury.gov/services/social-impact-partnerships/sippra-pay-for-results.

2 See 42 U.S.C. 1397n.

whether outcomes have been met.

3. Limitations

a. Treasury Discretion to Make Awards

Treasury may make awards to all, some, or none of the applicants under this NOFA and may make awards for amounts less than the amounts requested by applicants. Treasury is placing an upper limit on the amount of each project award—not including the associated independent evaluation—of $10 million.

b. Savings to the Federal, State, or Local Government

According to the Act, projects may only be awarded if they produce savings to the federal, State, or local government, as defined in Section A.5 Outcome Valuation Methodology.

c. Positive Benefit Cost Analysis (BCA)

For this NOFA, Treasury will only consider applications that have a positive Benefit-Cost Analysis (BCA), as explained in Section A.5 Outcome Valuation Methodology.

d. Directly Benefit Children

The Act requires that “[n]ot less than 50 percent of all Federal payments made to carry out agreements under this section shall be used for initiatives that directly benefit children.”3 Treasury will prioritize the funds available under this NOFA for projects designed to directly benefit children in order to meet the 50 percent threshold laid out in statute. To meet this threshold, taking into account the composition of the awards issued under the previous NOFA, 65 percent of the total possible award amount under this competition will be reserved for projects that directly benefit children. Other projects will be considered as long as Treasury reaches 50 percent of the overall available funding with its awards. Treasury will consider a project to “directly benefit children” if 1) the target population is children (aged 0-19 at the beginning of the intervention); or 2) the target population is parents of children. If the project benefits parents, in order to be considered a project that directly benefits children, the application must present strong evidence demonstrating a close logical, causal, and consequential relationship between the project’s effect on parents and the resulting positive effect on the parents’ children, and being a parent must be a part of the intervention’s eligibility criteria. Portions of projects can directly benefit children without having the entire project directly benefit children. Treasury welcomes questions regarding whether a specific project concept would qualify as a project that directly benefits children.

3 See 42 U.S.C. 1397n-2(f).

4. Pay for Results Framework

This section provides an overview of the main features of the SIPPRA program’s social impact partnerships: the pay for results model, outcomes, outcome payments, partnership formation, and the independent evaluations. Social impact partnerships are part of a pay for results model where groups of stakeholders including state and local governments, service providers, philanthropy, intermediaries, or others seek to produce outcomes that result in social benefit and federal, State, or local savings. Treasury, the Commission (Section E.2.c Phase 3: Consistency Review and Commission Recommendation), and the Interagency Council (Section E.2.d Phase 4:

Interagency Council Certification and Treasury Determination) expect that approaches to these partnerships will differ across applications.

Applications under this NOFA must provide all required application elements set out in the Act at 42 U.S.C. § 1397n-1(c)(1)-(24).

a. The Pay for Results Model

The pay for results model mandated by the Act differs from many other federal grant programs, in which the federal government funds the cost of programs and services prior to implementation of the programs. Under the pay for results model (also referred to as the “pay for success” model), the federal government agrees to make payments only if specific, pre-determined, measurable outcomes are achieved. If the intervention does not achieve the pre-determined outcomes, then the federal government will not make an outcome payment. The Act provides that the federal government’s payment for an outcome must be less than or equal to the value of the outcome to the federal government over a period not exceeding ten years from the date implementation commences. Value to the federal government in this NOFA is defined as the net benefits from a BCA. For additional information, see Section A.5.a Federal Value for the SIPPRA Program.

b. Outcomes

The Act requires that the social impact partnership “produce one or more measurable, clearly defined outcomes that result in social benefit and federal, State, or local savings.”4 An outcome is a positive impact on a target population that an Applicant expects to achieve as a result of an intervention over the duration of a project. The partnership’s ability to identify, achieve, and agree upon suitable outcomes is a key determinant of whether pay for results is the appropriate funding instrument for addressing the identified social issue. The statute identifies the following outcomes:

1) Increasing work and earnings by individuals in the United States who are unemployed for more than 6 consecutive months.

2) Increasing employment and earnings of individuals who have attained 16 years of age but not 25 years of age.

3) Increasing employment among individuals receiving Federal disability benefits.

4) Reducing the dependence of low-income families on Federal means-tested benefits.

4 See 42 U.S.C. 1397n-1(b).

5) Improving rates of high school graduation.

6) Reducing teen and unplanned pregnancies.

7) Improving birth outcomes and early childhood health and development among low-income families and individuals.

8) Reducing rates of asthma, diabetes, or other preventable diseases among low-income families and individuals to reduce the utilization of emergency and other high-cost care.

9) Increasing the proportion of children living in two-parent families.

10) Reducing incidences and adverse consequences of child abuse and neglect.

11) Reducing the number of youth in foster care by increasing adoptions, permanent guardianship arrangements, reunifications, or placements with a fit and willing relative, or by avoiding placing children in foster care by ensuring they can be cared for safely in their own homes.

12) Reducing the number of children and youth in foster care residing in group homes, child care institutions, agency-operated foster homes, or other non-family foster homes, unless it is determined that it is in the interest of the child’s long-term health, safety, or psychological well-being to not be placed in a family foster home.

13) Reducing the number of children returning to foster care.

14) Reducing recidivism among juvenile offenders, individuals released from prison, or other high-risk populations.

15) Reducing the rate of homelessness among our most vulnerable populations.

16) Improving the health and well-being of those with mental, emotional, and behavioral health needs.

17) Improving the educational outcomes of special-needs or low-income children.

18) Improving the employment and well-being of returning United States military members.5

19) Increasing the financial stability of low-income families.

20) Increasing the independence and employability of individuals who are physically or mentally disabled.

21) Other measurable outcomes defined by the State or local government that result in positive social outcomes and Federal savings.6

An outcome is measured by one or more indicators that are specific, unambiguous, and observable. The outcomes must be measured for the duration of the intervention period.7 These outcomes must result in social benefit and savings to the federal, State, or local government.

Outcome measurements are used to calculate the value to the federal government as discussed in Section A.5 Outcome Valuation Methodology.

c. Outcome Payments

The federal government will only make a payment if the evaluation successfully shows, using the outcome valuation methodology described in Section A.5 Outcome Valuation Methodology, that an Awardee achieved the pre-determined outcome levels as a result of the intervention. To

5 This may include improving the employment and well-being of United States military members as they transition to civilian status either as non-activated members of the National Guard or Reserves or as they become Veterans of the Armed Forces.

6 See 42 U.S.C. 1397n-1(b).

7 The duration of a SIPPRA project may not exceed 10 years. 42 U.S.C. 1397n-2(c)(1)(C).

qualify for an outcome payment, an Awardee’s project must meet one or more positive outcome targets.8 An outcome payment must be less than or equal to the value of the outcome to the federal government over a period not exceeding ten years from the date implementation commences, and for projects under this NOFA, Treasury is capping outcome payments at $10 million. Under this NOFA, an applicant may propose one or multiple project outcomes and receive separate payments at separate points in time for each outcome achieved depending on how the partnership designs the intervention.

d. Partnership Formation

The State or local government as the eligible applicant may work with other entities, referred to as “partners,” to successfully achieve the outcomes. In addition to the Applicant, the partnership may include investors, service providers, and intermediaries. The Applicant also may fulfill one or more of these roles. Below are examples of possible partners:

• Investor(s) are entities that, if the Applicant is not doing so, provide the funding for the social service interventions. Investors may be not-for-profit or for-profit entities or public sector funds. They accept the risk that they will not be repaid in the event that the target outcome(s) are not achieved as a result of the intervention.

• Intermediary(ies) may be selected by the Applicant to coordinate the pay for results arrangement. The role of the intermediary may include (1) being responsible for achieving the negotiated outcome(s) for the target population by contracting with service providers; (2) raising funds from investors (if applicable) to cover the operating costs of implementing the services or programs; (3) changing or modifying service delivery methods and providers, with concurrence of the other partners, including the independent evaluator and, if applicable, investors; and (4) if outcome target(s) are met, receiving outcome payments from the Awardee and making payments to the investors, if applicable. The partnership is not required to include an intermediary organization, and a service provider, described below, may also serve as an intermediary.

• Service provider(s) deliver the intervention designed to achieve the outcomes sought in a pay for results partnership agreement. An applicant, or, where applicable, an intermediary arranges with a service provider to provide services and/or administer the interventions.

Note that a service provider may be a State or local government agency.

e. Independent Evaluations

The Applicant must contract with an independent evaluator to determine if the project achieved the pre-determined outcome levels as outlined in the project agreement. To ensure the objectivity of evaluations and to preserve the independence of evaluators, the statute requires that the federal government enter an agreement separate from the project grant to recipients exclusively to fund an evaluator’s work on the project. State and local governments receiving project grants will be eligible to receive up to 15 percent of the project grant to pay for all or a portion of the cost of a statutorily required independent evaluator.9 Treasury will make the payment for the independent

8 See 42 U.S.C. 1397n-2(c).

9 See 42 U.S.C. 1397n-4(a) and (f).

evaluator regardless of whether outcomes have been met. This separate grant may not be used to pay for other project expenses or for fees associated with project stakeholder participation in the project. The independent evaluator must not have a financial or other stake in the project that would undermine its objectivity, and the Applicant must avoid the selection of an independent evaluator whose objectivity might be impaired. See Section D.2.a.g.4 Independent evaluator qualifications for the independent evaluator’s required qualifications.

The independent evaluator must determine whether the intervention achieved the expected outcome(s) following the evaluation design plan. If successful, the federal government will then make a payment or payments to the Awardee based on the agreed upon payment schedule. See Section A.6. Independent Evaluation for more information on the requirements for the independent evaluation.

5. Outcome Valuation Methodology

Applications for social impact partnership projects must describe one or more outcome goals for the project, and then determine the value of each outcome to the federal government using outcome valuation. Outcome valuation is the process, at the application stage, for rigorously laying out the evidence and data used to determine the value to the federal government, and thus the appropriate payment from the federal government, for the improved outcomes resulting from project interventions. For projects under this NOFA, value to the federal government is defined as the net benefits derived from a benefit-cost analysis (BCA) over a period not exceeding ten years from the date implementation commences.

As explained in detail below, the Applicant must first show that, as a result of the anticipated outcome of the project intervention, there will be savings to the federal, State, or local government. Savings is defined as reductions in governmental outlays that are directly the result of the project intervention net of the project’s cost. Increased revenues as a result of the intervention are not considered savings. There must be savings for a project to be funded through the SIPPRA program.10

Applicants must then perform a BCA that will be used to determine the value to the federal government. The Applicant may use savings to the federal, State, or local governments, calculated in the previous step, as one of the benefits used in the calculation.

Using the BCA process, the Applicant will then determine the net benefits of a project outcome, which is the monetized value of the benefits minus the costs. If this number is greater than zero (i.e., benefits exceed costs), then there is a positive value to the federal government. If the net benefit is not greater than zero, there is not a positive value to the federal government, and therefore the project is not eligible for payment. The federal payment to an Awardee must be less than or equal to the value of the outcome to the federal government over a period not exceeding ten years from the date implementation commences. Treasury is placing an upper limit of $10 million on the amount of each project award (not including amounts for the associated independent evaluation).

10 See 42 U.S.C. 1397n-1(b); 42 U.S.C 1397n-5(a)(8).

a. Federal Value for the SIPPRA Program

Applicants must use benefit-cost analysis (BCA) to determine the value to the federal government, which is the maximum amount that the Applicant can receive as an outcome payment. BCA is a systematic process for identifying, quantifying, and comparing expected benefits and costs of a potential project, policy, or action to society. In executing the BCA, Applicants must account for both social benefits (including savings to a State or local government or to the federal government) that provide positive value to the federal government, and costs, which result in negative value, to determine the net value to the federal government.

The rest of this section provides a recommended guide for calculating benefits and costs through BCA to determine the value to the federal government for the purposes of the SIPPRA program.11 Applicants may consult OMB Circulars A-4 and A-94 for additional guidance.

Step 1. Demonstrate savings to the federal, State, or local government

Over the course of the period of performance of a project, interventions must provide savings to the State or local government or to the federal government, in the form of reduced outlays as described below. This step is a threshold requirement and must be presented as a separate calculation in the application, prior to those savings being incorporated into the BCA as a social benefit of the project outcome.

The federal, State, and local savings calculation analysis requires estimating the savings— reductions in outlays12—that accrue to the federal, State, and local governments that are the result of the intervention, over the period of performance of the project. The savings calculation must incorporate increases in costs due to intended or unintended impacts of the intervention. In some cases (particularly where there are complex program interactions), it may be necessary to estimate baseline outlays and compare outlays under the intervention to arrive at an appropriate estimate of savings. The application must provide sufficient information (e.g., all data sources, related literature, assumptions, and justifications) to show how the Applicant estimated savings that occur as a direct result of the proposed intervention. Applicants must document and submit their estimates of changes to outlays as a direct result of each proposed intervention such that these analyses can be replicated. Only Applicants with federal, State, or local savings will be considered for the SIPPRA program.13 Savings to each level of government should be presented separately to show how outlays are changing at each level of government as a result of the intervention.

The Applicant must carefully consider how the project intervention may cause the substitution of benefits delivered through one social program for another. Specifically, the Applicant must consider how the intervention will affect eligibility for other federal programs and how this will affect the change in outlays. For example, an intervention that increases employment could decrease participation in government assistance programs while increasing eligibility for

11 This guide is not intended to be a general guide for BCA and is for the purposes of the SIPPRA program only.

12 Applicants proposing to generate value to the federal government only through reductions in federal administrative expenses will not be considered eligible.

13 See 42 U.S.C. 1397n-1(b); 42 U.S.C 1397n-5(a)(8).

https://www.whitehouse.gov/wp-content/uploads/legacy_drupal_files/omb/circulars/A4/a-4.pdf https://www.whitehouse.gov/wp-content/uploads/2023/11/CircularA-94.pdf reimbursable employment-based tax credits. Both the decrease in assistance outlays and the increase in refundable tax credit expenditures are changes in government outlays resulting from the project intervention and must be taken into account in the savings calculation.

In estimating the project intervention’s effect on the outlays of a government program, the Applicant must carefully consider the funding structure of the program and whether or not the program is oversubscribed, i.e., the program has more eligible individuals than funding available for services, such that when one individual is removed from the program another eligible individual replaces that individual.

For applicants who plan to use savings from Medicaid or CHIP, see Section J. Appendix II:

Integration of Managed Care Information/Data for the integration of managed care information/data. This information is required to certify such savings.

Step 2. Assess costs and benefits from the intervention’s effect to the “target” population for each time period

The next step of the outcome valuation process is to use BCA to assess the costs and benefits of the intervention on its target population. BCA is a systematic process for identifying, quantifying, and comparing expected benefits and costs to society of a potential project, policy, or action.

Estimated benefits are based on the projected social impacts of the project, valued in monetary terms. There are a wide range of benefits that can be included in a BCA, and which ones to include will be heavily dependent on the type of intervention that is designed. For example, if the program seeks to increase economic opportunity through a job training program, it might be expected to result in increased wages, increased revenues to the federal, State, and local government, and decreased outlays on programs like SNAP or Medicaid.

The savings calculated in Step 1 must be included in the BCA. Such benefits must be adjusted by the Marginal Cost of Public Funds, a cost adjustment which accounts for the distortion effect of taxes on the cost-benefit tradeoff of actions (this effect is referred to as Dead Weight Loss).

Because of Dead Weight Loss, the cost of every dollar of public funds (the Marginal Cost of Public Funds) is greater than $1. For the purposes of consistency within the SIPPRA program, all benefits from government savings must be multiplied by the Marginal Cost of Public Funds of $1.25.14 Similarly, any increases in revenues to any level of government must be adjusted by the Marginal Cost of Public Funds. See Appendix I for an example of how to apply the Marginal Cost of Public Funds.

Costs include the resources required to develop the project and the costs to facilitate the project over time. Costs associated with impacted federal, State, and local programs must be included in the estimated cost of the program, and these should also be multiplied by the Marginal Cost of Public Funds.

Applicants are encouraged to use existing research, incorporating analytical tools grounded in

14 For further explanation of these principles, see OMB Circular A-94, pg. 17.

microeconomic theory, to quantify the costs and benefits of their expected outcomes. For a greater discussion of analytical tools for BCA see Section K. Appendix III: Benefit-Cost Analysis Tools. When possible, stated preference (for example, surveys of how much an individual values a particular good or service) should be avoided in arriving at any of the core assumptions of the BCA.

Step 3. Assess external costs and benefits

Applicants’ BCA must also consider the effects of interventions that extend beyond the target population. In particular, some interventions may generate positive or negative unpriced external effects, known as externalities. For example, when a person consumes a gallon of gasoline, they pay a price, and receive a benefit. However, that gallon of gasoline also produces air pollutants, both in its production and final consumption. Therefore, when the consumer uses the gallon of gasoline, air pollution is a negative externality of that purchase. Similar externalities, whether positive or negative, must be considered in the Applicant’s BCA.

Additionally, when considering external costs and benefits, applicants must guard against double-counting, since some benefits or costs are embedded in other broader measures. To balance this goal with concerns about under-counting meaningful effects by excluding potentially overlapping benefits or costs, it may be helpful to include a range—with the lower-bound estimate prioritizing the avoidance of double-counting and the upper-bound estimate prioritizing avoidance of omitted categories of impacts. See OMB Circular A-4 for additional guidance.

Step 4. Sum costs and benefits by time period

As illustrated in the example provided in Appendix I: Example of Outcome Valuation Process, for each time period in the analysis, sum the costs and benefits calculated in Steps 1-3. Calculate the net benefits for each time period by subtracting the costs from the benefits.

Step 5. Appropriately account for inflation and sum across time periods

In order to ensure a meaningful comparison between benefits and costs, it is important that all monetized values used in a BCA be expressed in common terms. Data obtained for use in BCAs is sometimes expressed in nominal dollars from several different years. Nominal dollars reflect the effects of inflation over time and are sometimes also called current or year of expenditure (YOE) dollars. Such values must be converted to real dollars (also referred to as constant dollars), using a common base year, to net out the effects of inflation. Applicants must use the Consumer Price Index (All Urban) from the FY 2024 President’s Budget Mid-Session Review for all inflation adjustments.15 Projects that have benefits and costs beyond 2033 should assume an inflation rate of 2.3%.

In some benefit-cost calculations, after netting out the effects of inflation, a second distinct adjustment, called discounting, is made to account for the time value of money. However, the SIPPRA program operates over a relatively short period of time, lowering the impact of

15 https://www.whitehouse.gov/wp-content/uploads/2023/07/msr_fy2024.pdf, p. 6.

https://www.whitehouse.gov/wp-content/uploads/2023/07/msr_fy2024.pdf discounting on value calculations. Therefore, do not discount costs and benefits.

For the BCA used in SIPPRA, a 10-year time period is allowed in which to accrue benefits.16 Once each time period is adjusted for inflation, sum across the time periods—up to 10 years.17 Also, calculate the benefit-cost ratio by dividing the benefits by the costs. Only projects with a benefit-cost ratio greater than one will be considered for SIPPRA.

For an example of how to apply these steps, see Section I. Appendix I: Example of Outcome Valuation Process.

b. “Tips” for Conducting BCA for SIPPRA Program Projects

The following recommendations may be helpful to applicants in conducting BCA for SIPPRA projects.

Tip #1. Avoid any effects of your program to the general economy.

Some changes have big enough impacts to change the prices of goods or services in a market.

For example, a national change to the replacement rates for unemployment insurance will have large enough effects to adjust wages for everyone, not just individuals in a program (known as general equilibrium effects). However, programs and policy changes of the size that are eligible under SIPPRA are not large enough to affect prices. For example, a small job training program for unskilled workers is unlikely to move the market price for unskilled labor. Therefore, when assessing the benefits of the program, the BCA analyst must only consider the additional wages the worker receives and not consider any benefit to employers.

Tip #2. Do not give different weights to different groups or populations of people.

Some BCAs use distributional weights, an approach in which different weights are applied to costs and benefits for different groups. For the purposes of SIPPRA, Treasury is not considering distributional weights.

Tip #3. Do not use discounting for time preference.

When performing forward-looking BCA, future costs and benefits are sometimes discounted.

However, given that the SIPPRA program operates over a relatively short period of time, costs and benefits will not be discounted for time preference. As explained above, however, adjusting for inflation is required.

16 See 42 U.S.C. 1397n-2(c)(1)(B).

17 As described above, for purposes of meeting the SIPPRA statutory requirement that a project provide savings to the State or local government, or to the federal government, savings must be achieved by the time of project completion. See 42 U.S.C 1397n-5(a)(9). However, for purposes of counting savings as benefits towards the BCA calculation, savings may be calculated up to a 10-year time period like other benefits.

6. Independent Evaluation

This section addresses post-award independent evaluations, including evaluation design, research methodologies, and expected coordination of activities.

a. Overview

By statute, SIPPRA program projects must have evaluations conducted by independent evaluators.18 Awardees can expect to commit significant time and resources to the formal evaluations of their project. All applicants are eligible to receive evaluation funding to support post-award evaluation costs, regardless of whether outcomes are met.

The federal government will fund up to 15 percent of the amount of the estimated project award (not including the cost of the evaluation) for an independent evaluation of the project.19 The federal government will base its maximum award of funds for the grantee’s cost of an independent evaluator on the amount of the top tier outcome payment. The federal government will fund only completed post-award evaluation work. The federal government will not pay for pre-award costs or the portion of an evaluator’s contract contemplating evaluation work that is not completed in the event a project terminates earlier than expected.

b. Evaluation Requirements

The Act requires projects to establish that the outcomes “have been achieved as a result of the intervention.”20 The evaluation used to determine whether a State or local government will receive outcome payments under SIPPRA shall use experimental designs with random assignment or other reliable evidence-based research methodologies that, as certified by the Interagency Council, allow for the strongest possible causal inferences when random assignment is not feasible.21 The project’s independent evaluation must be designed to assess the strength of the causal evidence, i.e., the degree to which the evaluation establishes the causal impact of the intervention on the outcomes of interest not due to other factors.22

Randomized controlled trials (RCTs) are generally considered to be the most rigorous type of experimental design. In RCTs, a sample is randomly split into two groups– treatment and control. One will receive the intervention and the other will continue as normal. These studies are designed to minimize the chance that the observed difference in outcomes is due to an alternative explanation.

Treasury will also accept other reliable, evidence-based research methodologies commonly known as quasi-experimental design studies. These are studies with an evaluation design in

18 See 42 U.S.C. 1397n-1(c)(22); 42 U.S.C. 1397n-4(b).

19 See 42 U.S.C. 1397n-4(a) and (f).

20 See 42 U.S.C. 1397n-1(c)(7).

21 See 42 U.S.C. 1397n-4(c).

22 More information on evidence standards in the context of Federal program evaluations can be found at https://home.treasury.gov/system/files/136/SLFRF-Compliance-and-Reporting-Guidance.pdf.

https://home.treasury.gov/system/files/136/SLFRF-Compliance-and-Reporting-Guidance.pdf which outcomes for the treatment group, or a broader target population that includes both the treatment group and those outside the treatment group, are measured relative to a comparison group. Such a design attempts to approximate an experimental design and can support causal conclusions, without random assignment. Sophisticated analytic techniques are used to control for factors that might be associated with the outcome being analyzed. Applicants that cannot implement an RCT study will not be deemed less competitive or penalized for implementing a quasi-experimental design. Applicants using a quasi-experimental design must address other possible causes of the outcomes, such as selection, other policies, economic conditions, and other confounding factors. This should include a description of the contrast in services that the comparison and treatment group will receive during the project period.

A part of this evaluation will be a statistical significance requirement where the coefficient on the treatment variable is statistically significant (supporting rejection of the null hypothesis of no impact). For purposes of the SIPPRA program, the coefficient will be considered statistically significant if the null hypothesis falls outside of the 80 percent confidence interval. The choice of how to best calculate standard errors and confidence intervals is left to the independent evaluator, who must follow best practices based on the identification strategy. The power calculation (see guideline #12 on pg. 20) required in the evaluation design plan will be a critical input for Treasury to consider when evaluating the application.

Applicants may use classical statistical analysis or Bayesian statistical analysis. For applicants using Bayesian statistical analysis, the appropriate Bayesian tests must be used to show the equivalent of classical statistical significance at the 80 percent level. Additionally, applicants using Bayesian statistical analysis must conduct prior sensitivity analysis to ensure any causal result is not due only to a dominant prior. Applicants using this approach must use high-quality experimental or quasi-experimental evidence to justify the prior distribution.

c. Evaluation Design Plan

The Applicant must provide an evaluation design plan that includes a range of information related to design, implementation, statistics, and data. The full list of requirements is available in Section D.2.a.(g)5 Evaluation design plan.

The design plan may evolve during a project’s early implementation period (approximately the first 6–12 months) to ensure proper measurement of project outcomes. However, outcome goals may not change without prior approval from Treasury or the administering federal agency.

Grantees must submit the design plan to Treasury or the administering federal agency once it is finalized. Elements of the evaluation design plan may be posted on the Federal Interagency Council on Social Impact Partnerships (Interagency Council) website.23

d. Evidence Standard

The Act requires Treasury to take into consideration the likelihood, based on evidence provided in the application and other evidence, that the State or local government in collaboration with the

23 See 42 U.S.C. 1397n-10(3)(J).

intermediary and the service providers will achieve the specified outcomes.24 The evidence base should consist of well-designed and well-implemented experimental studies or quasi-experimental studies that support the effectiveness of the practice, strategy, or program; and/or large, well-designed, and well-implemented randomized controlled, multi-site trials that support the effectiveness of the practice, strategy, or program. The magnitude of the impact assumed for the SIPPRA project must be derived from this evidence base.

For each project application, the Subject Matter Expert Panel (see Section E.2.b) will determine the strength of the evidence provided, as described further below. Projects with strong or moderate evidence are most likely the best candidates for the SIPPRA program, but all projects will be considered.

• Strong evidence means that the evidence base can support causal conclusions for the specific program proposed by the applicant with the highest level of confidence. The evidence must support causal conclusions (i.e., studies with high internal validity) and include enough of the range of participants and settings to support scaling up to the state, regional, or national level (i.e., studies with high external validity). The following are examples of strong evidence: 1) More than one well-designed and well-implemented experimental study or well-designed and well-implemented quasi-experimental study that supports the effectiveness of the practice, strategy, or program; or 2) one large, well-designed and well-implemented randomized controlled, multi-site trial that supports the effectiveness of the practice, strategy, or program.

• Moderate evidence means that there is a reasonably developed evidence base that can support causal conclusions. Evidence from previous studies on the program, the designs of which can support causal conclusions (i.e., studies with high internal validity) but have limited generalizability (i.e., moderate external validity). This also can include studies for which the reverse is true— studies that only support moderate causal conclusions but have broad general applicability. The following would constitute moderate evidence: 1) At least one well-designed and well-implemented experimental or quasi-experimental study supporting the effectiveness of the practice strategy, or program, with small sample sizes or other conditions of implementation or analysis that limit generalizability; 2) at least one well-designed and well-implemented experimental or quasi-experimental study that does not demonstrate equivalence between the intervention and comparison groups at program entry but that has no other major flaws related to internal validity; or 3) correlational research with strong statistical controls for selection bias and for discerning the influence of internal factors.

• Preliminary evidence means that the evidence base can support conclusions about the program’s contribution to observed outcomes. The evidence base consists of at least one non-experimental study. A study that demonstrates improvement in program beneficiaries over time on one or more intended outcomes OR an implementation (process evaluation) study used to learn about and improve program operations would constitute preliminary evidence. Examples of research that meet the standards include: 1) outcome studies that track program beneficiaries through a service pipeline and measure beneficiaries’ responses at the end of the program; and 2) pre- and post-test research that determines whether beneficiaries have improved on an intended outcome.

24 See 42 U.S.C. 1397n-2(b)(3).

The project narrative must include a theory of change and a logic model that builds from this evidence base. A theory of change must inform the intervention design by reflecting the logical (and evidence-informed) reasoning that supports the expectation the actions taken will lead to the intended outcomes. The logic model builds off this theory of change. A logic model provides a bridge between project design and the evaluation by clarifying the inputs, activities, outputs, outcomes, and impacts that can help to crystalize how each of those things can be measured and tracked.

e. Evaluation Facilitation

The Applicant is expected to participate in and manage several activities to ensure the successful independent evaluation of demonstration projects. These activities include:

• Working with the independent evaluator to facilitate the execution of the overall evaluation strategy and to ensure the intervention is performed according to the evaluation design plan described above;

• Reporting progress and final evaluation results to Treasury and/or the relevant federal agency on schedule;

• Over the course of the performance period, working with the independent evaluator to ensure that project randomization procedures and other evaluation processes are adhered to;

• Working with the independent evaluator to modify evaluation plans, as appropriate.

• Ensuring that the independent evaluator can collect all relevant data and has access to needed datasets.

f. Agreement with Independent Evaluator

Because the evaluation findings provide the basis for pay for results payments to the grantee, the agreement each applicant enters into with an independent evaluator must require an agreed-upon evaluation design and methodology, observed outcome measure(s), and findings regarding outcome targets. The agreement must address the following:

• Plan to obtain relevant datasets from various sources, for example, local agencies, state agencies, or other federal agencies, including the responsibilities of the grantee and evaluator in accomplishing this task;

• Design and coding of a management information system, as needed, that is tailored for research or evaluation, to track participants and obtain individual level data;

• Collection or assessment of individual-level data. The independent evaluator must work directly with the Applicant and other organizations to enter into one or more agreements for the access and use of the data. These agreements must include assuring data quality and adherence to all federal and state data privacy statutes and policies and data security standards;

• Institutional Review Board (IRB) approval or a plan to get IRB approval to ensure the protection of human subjects, to the extent applicable; and

• Submission of progress reports to Treasury, the Interagency Council, and the head of the relevant agency in accordance with the reporting requirements described in Section F.3.b Evaluation Progress Reports and Section F.3.c Evaluation Final Reports.

If the Applicant is unable to execute an agreement prior to the application deadline, Treasury will accept a draft agreement containing these elements.

B. Federal Award Information

1. Type of Federal Award

Treasury expects to award up to $40.9 million to fund projects under this NOFA, with an additional amount up to $6.1 million available to fund the independent evaluations. The total amount awarded under this NOFA will be determined based on the number and strength of applications for projects received and other programmatic considerations. Treasury reserves the right to make no awards or to make awards for amounts less than the amounts requested by applicants. As stated above, Treasury is placing an upper limit on the amount of each project award – not including the associated independent evaluation – of $10 million.

As noted above, for projects funded under this NOFA, the federal government, under separate agreements with grantees, will also make available up to 15 percent of the project award amount (not including the cost of the evaluation) for the cost of an independent evaluation. These agreements to pay for evaluations will provide for payment regardless of outcomes, but the agreements will limit payments to evaluation work performed.

2. Project Period

SIPPRA funds must be liquidated by September 2033. Therefore, the period of performance for SIPPRA project awards must end by September 2032, to allow for up to six months for final measurement, analysis, evaluation, submission of the independent evaluator’s final report, and submission of payment requests to the federal government.25 Applicants should carefully construct their project timeline to allow sufficient time for all required activities. Treasury expects the period of performance to generally be about 48-60 months, but this will be heavily dependent on the nature of the project interventions. Applicants must specify the intervention period and explain the basis for specifying such period. Requests to extend the period of performance after an agreement is awarded will not be considered.

C. Eligibility Information

1. Eligible Applicants

Only States or local governments are eligible applicants; applications from any other entities will

25 The Act provides that the period of performance under the award agreements may not exceed 10 years. See 42 U.S.C. 1397n-2(c)(1)(C). Treasury will strive to maximize use of the amounts Congress appropriated to make awards and outcome payments. The Act appropriates funds that are available for ten years to make awards. See 42 U.S.C. 1397n-9 and 1397n-13. Federal law generally provides that disbursements of funds awarded within the SIPPRA program 10-year window (e.g., outcome payments) must occur within five years after that 10-year window closes. See 31 U.S.C. 1552(a).

not be reviewed. The Act defines the term “State” to mean each State of the United States, the District of Columbia, each commonwealth, territory, or possession of the United States, and each federally recognized Indian tribe.