TPP23 Tier2 RI QA - 03142023_final.pdf

Updated March 14, 2023

HHS Office of Population Affairs Web: opa.hhs.gov | Email: opa@hhs.gov

Twitter: @HHSPopAffairs | YouTube: HHSOfficeofPopulationAffairs

October 2022

Teen Pregnancy Prevention Program (TPP) Tier 2 Rigorous Evaluation Cooperative Agreement

AH-TP2-23-001

QUESTIONS & ANSWERS

file://///ct-files/g-drive/Fertility%20and%20Family%20Structure/OPA%20Communications/Option%20Year%202/1-%20Strategic%20Communications/1.8%20Graphic%20Design/opa.hhs.gov

Contents General Programmatic Budget

General

1. When are applications due for Opportunity Number: AH-TP2-23-001?

Applications are due on Wednesday, May 17, 2023, by 6pm ET

2. Can the same organization submit applications to the Rigorous Evaluation NOFO, TPP Tier 1 Advancing Equity in Adolescent Health through Replication of Evidence- Based TPP and/ or TPP Tier 2 Adolescent Sexual Health Innovation Hubs?

Yes, an organization can submit an application to more than one NOFO. Each NOFO is different, and organizations should check the eligibility and expectations of each individual NOFO to determine whether or not to apply. HHS/OASH/GAM will deem each application submitted by an entity eligible according to the eligibility information included in the specific

NOFO.

3. I am wondering if this is a single submission opportunity. Can we submit more than one application for this grant if the applications are for two separate projects? I am seeking to clarify the language on page 17 of the application that reads as follows, “If you successfully submit multiple applications from the same organization for the same project, we will only review the last application received by the deadline.”

Yes. You may submit more than one application if each is for a different project. As you noted, page 17 of the NOFO states, “If you successfully submit multiple applications from the same organization for the same project, we will only review the last application received by the deadline.”

4. Can an individual submit a grant application?

Grants are awarded to organizations rather than individuals. An application may be submitted by an individual authorized to act/sign for an organization and to assume the obligations imposed by the grant and any additional conditions of the grant. However, the award will not go directly to an individual but to the organization which the individual represents.

5. What is page limit for the application?

Your total application (i.e., the Project Narrative plus Appendices) must not exceed 100 pages. The following items do not count toward the Project Narrative page limit: all required forms, including SF-424, SF-424A, SF-LLL, Project Abstract Summary, and Budget Narrative (including budget tables) (Section C.4) (p.17).

6. What is page limit for the project narrative?

Your Project Narrative must not exceed 50 pages. The following items do not count toward the Project Narrative page limit: all required forms, including SF-424, SF-424A, SF-LLL, Project Abstract Summary, and Budget Narrative (including budget tables) (Section C.4) (p.17).

7. Do the appendices count toward the 50 pages allowed for the project narrative?

No, but the Appendices do count toward the page limit for your total application. Your total application (i.e., the Project Narrative plus Appendices) must not exceed 100 pages. The following items do not count toward the Project Narrative page limit: all required forms, including SF-424, SF-424A, SF-LLL, Project Abstract Summary, and Budget Narrative (including budget tables) (Section C.4) (p.17).

8. Will OPA advise or consult with our organization about potential ideas for my proposal before submission of the application?

No. Since this is a competitive funding announcement, OPA cannot provide input regarding the content of individual grant applications and is therefore unable to answer questions specific to the content of an application.

Applicants are encouraged to carefully review the NOFO and should put forward the best proposal the organization can create based on application and evaluation criteria published in the announcement.

9. What organizations are eligible to apply for the funding?

Applicants eligible to apply for this notice of funding opportunity are any public or private (profit or nonprofit) entity located in a State (which includes one of the 50 United States, District of Columbia, Commonwealth of Puerto Rico, U.S. Virgin Islands, Commonwealth of the Northern Mariana Islands, American Samoa, Guam, Republic of Palau, Federated States of Micronesia, and the Republic of the Marshall Islands).

Faith-based organizations and American Indian/Alaskan Native/Native American (AI/AN/NA) organizations that are public or private entities are eligible to apply. Public or private community-based organizations are eligible to apply (p.15-16).

Please reference the Eligibility Information in the NOFO beginning on page 16 for additional examples of eligible Organizations.

10. Any FTE expectation or requirement for the PI/PD position?

There is no expectation or requirement regarding the FTE level of the PI/PD position. The NOFO does note that only one Project Director/Principal Investigator (PD/PI) will be named on any resulting award. You should clearly identify the individual in that role in your application. This individual should be the person who will be responsible for the programmatic aspects of the project if an award is made (p. 18).

See Sections D.3.a.4. Organizational Capability, Partnerships, and Staffing (p.26) and D.3.a.5. Project Management and Work Plan (p.27) for more information on what to include in your application as it relates to the PI/PD role. Section D.3.b - Budget Narrative Content (p.28-34) also provides guidance on what information to include as it relates to personnel and staffing.

11. What's allowable for images/graphics in the narrative?

Any files uploaded or attached to the Grants.gov application must be Adobe PDF, Microsoft Word, or image formats (JPG, GIF, TIFF, or BMP only) and must contain a valid file format extension in the filename. We will not accept Microsoft Excel files.

In addition, the use of compressed file formats such as ZIP, RAR, or Adobe Portfolio will not be accepted. We will not contact you for resubmission of uncompressed versions of files.

Compressed files in the application will not be forwarded to the independent merit review panel for consideration.

We strongly recommend that electronic applications be uploaded as Adobe PDF. If you convert to PDF prior to submission, you may prevent any unintentional formatting that might occur with submission of an editable document. Although Grants.gov allows you to attach any file format as part of your application, we restrict this practice and only accept the file formats identified above for compatibility with our other systems. Any file submitted as part of the Grants.gov application that is not in a file format listed above will not be accepted for processing and will be excluded from the application during the review process (p.41).

12. Do you have to be a OPA previously funded grantee to apply for this funding?

No. Eligibility is not based on previous OPA funding. Any public or private entity, including faith-based, community-based, and Tribal organizations, is eligible to apply for an award under this NOFO. (p. 15-16).

13. Should the applicant organization be the evaluator or the program implementer or the intervention developer?

Any eligible organization may apply for funding under this NOFO (p.15-16). Either the evaluator’s organization, the program implementer’s organization, or the intervention developer’s organization may apply as the lead applicant, provided that the organization is an eligible entity. The NOFO does not specify what types of organizations should be the lead agency or which ones should be the partners. Information regarding eligibility pertains only to applicants. We expect recipients to use partnerships to fulfill capacity needs and successfully complete the project. Recipients and key partners should have the collective expertise necessary to successfully accomplish the goals and objectives for the overall project (p. 11). For information on partnerships, please review Sections D.3.e - Collaborate with Partners and Maintain Organizational Capacity (p.11) and D.3.a.4. Organizational Capability, Partnerships, and Staffing (p.26-27) for information on what should be included in your application to describe your partnership efforts and those with whom you may want to partner. We recommend you also review Section E.1 - Criteria. There you will find the criteria that will be used by federal staff and an independent review panel to assess all eligible applications (p.42-45).

14. I am attempting to apply to the NOFO (AH-TP2-23-001) - on Grants.gov. However, when I log into my account - the "Apply" button is not enabled. I will also need access to create a Workspace - as I am the Executive Director for the 501c3 agency applying.

Please contact Grants.gov directly for any issues with their system. Below is the contact information for Grants.gov:

• Email: support@grants.gov

• Telephone: 800-518-4726

Programmatic

1. Does OPA conduct a separate medical accuracy review in addition to the grantee-led review by subject-matter experts?

Yes. We expect recipients to demonstrate that subject matter experts have reviewed the materials, in order to ensure that all materials delivered to study participants within the funded project are medically accurate, age appropriate, culturally, and linguistically appropriate, trauma-informed, user-centered, and inclusive of all youth. We also expect recipients to submit all program materials to OPA for a medical accuracy review. Recipients may not begin implementation with the materials until after OPA has approved the use of the materials. See Section A.2.d – Complete OPA Material Review (p.10-11) for more information.

2. Please confirm the ages of youth eligible for an intervention.

Primary participants receiving the intervention evaluated under an award should be adolescents and youth, their caregivers, or youth-serving professionals (p.6). For purposes of this NOFO, OPA is defining adolescents and youth to be individuals between the ages of 10-24.

3. How many research questions should an applicant propose?

There is no specified minimum or maximum number of research questions mentioned within the NOFO. Federal staff and an independent review panel will assess all eligible applications on the extent to which (p.43):

The proposed research questions are well-reasoned, focused, well-aligned with the intervention’s theory of change, and clearly address the reduction of unintended teen pregnancy, behavioral risk factors underlying unintended teen pregnancy, or other associated risk factors for unintended teen pregnancy. At least one research question assesses the intervention’s impact on a sexual risk behavior associated with unintended teen pregnancy. At least one research question assesses the intervention’s impact on a sexual risk behavior associated with unintended teen pregnancy. Behavioral risk factors may include sexual activity, number of sexual partners, contraceptive use, sexually transmitted infections (STIs), etc. Other risk factors may include mental health, educational attainment, healthy relationships, intimate partner violence, substance use, etc.

mailto:support@grants.gov

Since this is a competitive funding announcement, OPA cannot provide input regarding the content of individual grant applications and is therefore unable to answer questions specific to the content of an application. Applicants are encouraged to carefully review the NOFO and should put forward the best proposal the organization can create based on application and evaluation criteria published in the announcement.

4. What does Cooperative Agreement mean?

Awards made under this NOFO will be cooperative agreements. A cooperative agreement is a form of assistance that allows for substantial involvement by OPA. Substantial involvement is in addition to the usual monitoring and technical assistance provided under a grant.

Section B.3 - Federal Agency Substantial Involvement (p.15) outlines what may be included as “substantial programmatic involvement.”

5. To be considered a “robust design”, how large does sample size need to be?

There are no specified sample sizes; your team should propose a sample based on power calculations and justify it within the evaluation design section of your application. The design should be feasible for the intervention in the setting and with the intended population.

Smaller sample sizes may be acceptable if the proposed design and analysis has sufficient rigor and is capable of producing evidence of effectiveness. Please see Section D.3.a.2.h - Impact Evaluation Design - Sample Size (p.24) for more information. When feasible, we expect the evaluation design to meet the criteria for the quality of an evaluation study per the criteria established in the HHS Teen Pregnancy Prevention Evidence Review (TPPER), version 6.0. (https://tppevidencereview.youth.gov/ReviewProtocol.aspx) (p. 8).

Since this is a competitive funding announcement, OPA cannot provide input regarding the content of individual grant applications and is therefore unable to answer questions specific to the content of an application. Applicants are encouraged to carefully review the NOFO and should put forward the best proposal the organization can create based on application and evaluation criteria published in the announcement.

6. What is an OPA-directed Federal evaluation?

An OPA-directed Federal evaluation is an evaluation conducted by an OPA-funded contractor. Examples of previous federal studies conducted by OPA include a cost study, a meta-analysis, effectiveness studies, and implementation studies. As a condition of the award, we expect recipients to participate in in an OPA-directed Federal evaluation, if selected and funding for such an evaluation becomes available (p.9).

7. Do we need or are we expected to have Institutional Review Board (IRB) approval at the start of the grant?

You are not required to have IRB approval at the start of the grant. However, you are expected to have approval from an IRB by the end of the planning period (p.10).

In your application, include a proposed timeline for acquiring the approval of the IRB for the proposed activities. Any award based on your application does not supersede the need for https://tppevidencereview.youth.gov/ReviewProtocol.aspx

IRB review, approval, or determination of an exemption. Do not include in your application the package prepared for your IRB review (p.23).

8. What does OPA mean by expecting “rigorous impact evaluations to have the most robust possible design that is feasible for the intervention in the setting and with the intended population”?

OPA defines rigorous impact evaluation as “either a randomized control trial (RCT) or a quasi-experimental design (QED) with counterfactual condition; the most robust possible design that is feasible for the intervention (p.66).” Please see the Glossary of Selected Terms and Definitions (starting on p.62) for more information on how OPA defines randomized control trial (RCT) and quasi-experimental design (QED). The definition for QED also includes examples.

You should propose an appropriate study and justify the rigor by describing your processes for forming study groups, collecting data, etc. You are encouraged to incorporate analyses for small sample sizes such as Bayesian methods. Please refer to the evaluation design section of the NOFO for more information (Section D.3.a.2 – Impact Evaluation Design, p.21-25).

9. What sorts of interventions are eligible to be evaluated under this funding opportunity?

We are seeking evaluations of interventions that address key adolescent health disparities and have the potential to make a significant contribution to preventing unintended teen pregnancy. We anticipate funding a wide range of adolescent health innovations under this funding announcement; the eligible interventions may include, but are not limited to, programs, curricula, clinical interventions, technological interventions, systems-level interventions, community-level approaches, and others. Interventions may work directly with adolescents, their caregivers, and/or with youth-serving professionals to address adolescent health behaviors relevant to teen pregnancy prevention. We expect any intervention proposed for evaluation under this initiative to already have compelling, positive preliminary evidence from previous formative evaluation and prior research, including documented effects on intended outcomes. We expect the intervention to have demonstrated support from previous participants and communities. The intervention should fit the population and setting proposed for the application. Interventions should have a clearly described theory of change and logic model (See Section 2.a.) (p.6).

10. What interventions are not appropriate to be evaluated under this funding?

TPP Interventions rigorously evaluated previously under federal funding are not appropriate for this funding unless significant changes have been made to the intervention to address findings from the previous evaluation results. See page 7 of the NOFO for examples of previous federal funding.

Interventions identified as effective by the HHS Teen Pregnancy Prevention Evidence Review (TPPER) are also not appropriate for this funding (https://tppevidencereview.youth.gov/FindAProgram.aspx), unless significant adaptations are proposed to the intervention, ultimately creating a new intervention (p.7).

https://tppevidencereview.youth.gov/FindAProgram.aspx

11. Do we need to include all Memoranda of Agreement (MOAs) in our application?

If available at the time of submission, signed Memoranda of Agreement (MOAs) or signed Letters of Commitment (LOCs) may be submitted for each partner (or one signed MOA with all partners) and include specific roles, responsibilities, resources, and contributions of partner(s) to the project. Submit a signed MOA or LOC from the intervention developer/purveyor/copyright holder indicating that you may use the materials as described within the application in the project (p.35).

The signed LOCs must detail the specific role and resources that will be provided, or activities that will be undertaken, in support of the applicant. The organization’s expertise, experience, and access to the targeted population(s) should also be described in the LOC.

Fully-executed MOAs may be required within the first 30 days following the start of the project of any award made under this announcement (p.36).

Please note that Federal staff and an independent review panel will assess all eligible applications on the extent to which “Signed Memoranda of Agreement (MOAs) or Letters of Commitment (LOCs) within the appendices support the partnership and demonstrate the level of commitment needed for the project to be successful.” (p.44)

Budget

1. Are renovations allowed under this funding opportunity?

No.

2. The NOFO lists an award floor of $500,000 and an award ceiling of $1,000,000 are these totals referring to the total grant or annual amounts?

Those totals are annual amounts for each year which is a budget period.

3. Is there a maximum percentage of the annual budget that is allowable for evaluation expenses?

No. The NOFO does not state a minimum or maximum for the evaluation expenses.

However, recipients will be required to provide a summary narrative and line-item budget for each year beyond the first. For categories or items that differ significantly from the first budget year, provide a detailed justification explaining these changes (p.28).

4. Where can I find information regarding unallowable costs?

Please be sure to carefully review Section D.7 Funding Restrictions (beginning on p.39) for specific information regarding allowable, unallowable, and restricted costs. You can also look under the guidance within GENERAL PROVISIONS FOR SELECTED ITEMS OF COST and through the following link: https://www.ecfr.gov/current/title-45/subtitle- A/subchapter-A/part-75/subpart-E/subject-group-ECFR5d90ba314caea08 https://www.ecfr.gov/current/title-45/subtitle-A/subchapter-A/part-75/subpart-E/subject-group-ECFR5d90ba314caea08 https://www.ecfr.gov/current/title-45/subtitle-A/subchapter-A/part-75/subpart-E/subject-group-ECFR5d90ba314caea08

5. Can an applicant include in their budget request incentives for participants to participate in the evaluation (whether control or intervention)?

Yes, participants can be provided incentives for evaluation participation. The program participation cost and or activities must be in support of the goals and objectives of the NOFO. Please see 45 CFR §75.456 - Participant support costs. Participant support costs are defined as direct costs for items such as stipends or subsistence allowances, travel allowances, and registration fees paid to or on behalf of participants or trainees (but not employees) in connection with conferences, or training projects.

https://www.ecfr.gov/current/title-45/subtitle-A/subchapter-A/part-75/subpart-E/subject-group-ECFR5d90ba314caea08/section-75.456

If including incentives as part of your budget request, you must include a plan for oversight of federal award funds. The plan must describe for any program incentives proposed, the specific internal controls that will be used to ensure only qualified participants will receive them and how they will be tracked.

6. Can youth who participate on an advisory board for the grant project or who serve as advisors for the grant project be provided with stipends?

Stipends for participants of an advisory board/council (such as a youth advisory board or community advisory board) and other costs associated with the advisory board/council may be allocable and charged directly to the TPP award if the aim of the advisory board/council is to meet the expectations of the NOFO. This includes providing guidance on the design, implementation, and monitoring of the project. The scope of work and related budget narrative must describe the youth advisory board and/or community advisory board’s membership, functions, and costs and explain why the advisory board is necessary to carry out the TPP funded project.

7. Does the application require cost sharing or match funds?

No, it is not a requirement (p.16).

8. Is Budget and Budget Narrative required for each year or just Year 1?

Provide a budget justification, which includes explanatory text and line-item detail, for the entire first year of the proposed project. For subsequent budget years in an anticipated multi-year project, provide a summary narrative and line-item budget for each year beyond the first. For categories or items that differ significantly from the first budget year, provide a detailed justification explaining these changes (p.28).

Funding for all approved budget periods beyond the first is generally level with the initial award amount and is contingent upon the availability of funds, satisfactory progress of the project, adequate stewardship of Federal funds, and the best interests of the Government (p.14).

9. Clarifying that the 100-page limit does not include budget and budget narrative. Only narrative and appendices.

https://www.ecfr.gov/current/title-45/subtitle-A/subchapter-A/part-75/subpart-E/subject-group-ECFR5d90ba314caea08/section-75.456 https://www.ecfr.gov/current/title-45/subtitle-A/subchapter-A/part-75/subpart-E/subject-group-ECFR5d90ba314caea08/section-75.456

Your total application (i.e., the Project Narrative plus Appendices) must not exceed 100 pages. The following items do not count toward the Project Narrative page limit:

• All required forms, including SF-424, SF-424A, SF-LLL

• Project Abstract Summary

• Budget Narrative (including budget tables)(Section D.2.a) (p.17).

10. Are indirect costs limited to 8% of MTDC for all applicants even if the applicant’s organization has an approved rate that is higher?

Page 32 of the NOFO indicates that the indirect cost category has one of two methods that you may select. You may only select one and must clearly identify that selection in your submitted budget. These methods are as follows:

• Your organization currently has an indirect cost rate approved by the Department of Health and Human Services (HHS) or another cognizant federal agency. You should enclose a copy of the current approved rate agreement in your Budget Narrative file.

If you request a rate that is less than allowed, your authorized representative must submit a signed acknowledgement that the organization is accepting a lower rate than allowed.

• Per 45 C.F.R. § 75.414 (f) Indirect (F&A) costs, “any non-Federal entity [i.e., applicant] that has never received a negotiated indirect cost rate, … may elect to charge a de minimis rate of 10% of modified total direct costs (MTDC) which may be used indefinitely. As described in § 75.403, costs must be consistently charged as either indirect or direct costs but may not be double charged or inconsistently charged as both. If chosen, this methodology once elected must be used consistently for all Federal awards until such time as a non-Federal entity chooses to negotiate for a rate, which the non-Federal entity may apply to do at any time.”

Note that the de minimis rate method only applies if you have never received an approved negotiated indirect cost rate from HHS or another cognizant federal agency. If you are waiting for approval of an indirect cost rate, you may request the 10% de minimis rate. If you choose this method, costs included in the indirect cost pool must not be charged as direct costs to the award. Indirect costs on Federal awards for training are limited to a fixed rate of eight percent of MTDC exclusive of tuition and related fees, direct expenditures for equipment, and subawards in excess of $25,000 (45 C.F.R. § 75.414 (c)(1)(i)).

Page 32 also explains how applicants should provide justification for and the calculation of their indirect costs total.