DD24-0051 NOFO_CHDSTAR2_InfoWebinar_3.18.24_NS.pdf

Centers for Disease Control and Prevention

Notice of Funding Opportunity Informational Webinar

Opportunity Name: Surveillance of Congenital Heart Defects Among Children, Adolescents, and Adults

Opportunity Number: CDC-RFA-DD-24-0051

Federal Assistance Listing: 93.073

National Center on Birth Defects and Developmental Disabilities

Division of Birth Defects and Infant Disorders

Centers for Disease Control (CDC)

March 27, 2024

CHDSurveillance@cdc.gov

Today's agenda

Participants can follow along using the PDF version of the NOFO, available at www.grants.gov.

2. Approach o Component A o Component B

1. Overview

4. Award and Eligibility Info

5. Submission Procedures

6. Review and Selection Process

7. Key dates and questions

3. Evaluation and Work Plans

Overview

Pages 4 - 5

CHD STAR 2

Background

CHDSurveillance@cdc.gov

CHD affects 1 in 100 livebirths

CHD is the leading cause of infant illness and death resulting from birth defects

About 2 million persons in the United States are living with a CHD

There is limited population-based data on children, adolescents, and adults with CHD

Pilot

#DD12-1207

Lifespan

#DD15-1506

CHD STAR* A

#DD19-1902

Ages 11-64y 1-64y 0-55y

Surveillance 2008-2010 2011-2013 2010-2019

*Congenital Heart Defects Surveillance across Time And Regions (CHD STAR)

Related Work

CHD STAR* B

#DD19-1902

*Congenital Heart Defects Surveillance across Time And Regions (CHD STAR)

Related Work

• Validate billing codes for CHD in healthcare claims data

• Determine best ways to use healthcare claims for CHD surveillance across the lifespan, using machine learning and natural language processing.

Goal: conduct population-based surveillance of congenital heart defects (CHD) to describe health outcomes including health equity, with the goal of identifying opportunities to improve the health of all people living with CHD

Congenital Heart Defects Surveillance across Time And Regions (CHD STAR) 2

Component A -required-

Identify CHD cases; Collect, manage, and analyze data, submit data to CDC; and disseminate information to partners

Component B -optional-

Develop and apply machine learning model to data collected through Component A

Approach

Pages 6 - 18

Component A

Congenital Heart Defects Surveillance across Time And Regions (CHD STAR) 2

Goal: conduct population-based surveillance of congenital heart defects (CHD) to describe health outcomes including health equity, with the goal of identifying opportunities to improve the health of all people living with CHD

CHDSurveillance@cdc.gov

Component A -required-

Identify CHD cases; Collect, manage, and analyze data, submit data to CDC; and disseminate information to partners

Component B -optional-

Develop and apply machine learning model to data collected

Logic Model Strategies and Activities Short-term

Outcomes Intermediate Outcomes

Long-term Outcomes

Program Planning -Identify/engage partner -Establish partnership collaborations -Prioritize program goals -Define source population -Identify data sources

Project Management -Develop management and staffing plan -Obtain or demonstrate legal authority to access data

Data Reporting -Collect/store standardized data, submit individual-level data to CDC Same as Component A

-Improved knowledge about the characteristics of individuals identified from multiple data sources

-Improved understanding of:

• age-specific mortality

• healthcare use, comorbidities, outcomes

• health equity

• racial, ethnic, socioeconomic patterns in healthcare use and outcomes

• respiratory viruses— healthcare use during COVID-19 pandemic

• potential impact of state and national healthcare policies

-Improved ability to develop surveillance programs

-Improved use of surveillance data to understand the outcomes of individuals with CHDs

-Improved ability of policy-makers to make data-informed decisions

-Improved ability to develop effective secondary prevention strategies to reduce the public health impact of CHD

-Improved health among individuals of all ages with

CHD

-Improved quality of life of individuals of all ages with

CHD

-Improved continuity of care of individuals of all ages with

CHD

Outcomes Intermediate Outcomes

Long-term Outcomes

Case Identification -Use all available data sources (birth defects registry data, clinical and administrative data) to identify eligible cases with CHD

Data collection -Establish vital status of identified cases, link identified cases to other data sources -Geocode address, link healthcare and census data, create standardized dataset

Data Assessment -Describe population with CHD -Validate 200-300 randomly selected cases per site (if funded for DD19-1902) -Assess quality and accuracy of data

-Improved knowledge about the characteristics of individuals identified from multiple data sources

-Improved understanding of:

• age-specific mortality

• healthcare use, comorbidities, outcomes

• health equity

• racial, ethnic, socioeconomic patterns in healthcare use and outcomes

• respiratory viruses— healthcare use during COVID-19 pandemic

• potential impact of state

-Improved use of surveillance data to understand the outcomes of individuals with CHDs

-Improved ability of policy-makers to make data-informed decisions

-Improved ability to develop effective secondary prevention strategies to reduce the public health impact of CHD

-Improved health among individuals of all ages with

CHD

-Improved quality of life of individuals of all ages with

CHD

-Improved continuity of care of individuals of all ages with

Outcomes Intermediate Outcomes

Long-term Outcomes

Dissemination -Share information, communicate results -Inform stakeholders on healthcare utilization, comorbidities, and outcomes of individuals with CHD -Work with local/state partners to use site-specific CHD data

-Improved knowledge about the characteristics of individuals identified from multiple data sources

-Improved understanding of:

• age-specific mortality

• healthcare use, comorbidities, outcomes

• health equity

• racial, ethnic, socioeconomic patterns in healthcare use and outcomes

• respiratory viruses— healthcare use during COVID-19 pandemic

• potential impact of state

-Improved use of surveillance data to understand the outcomes of individuals with CHDs

-Improved ability of policy-makers to make data-informed decisions

-Improved ability to develop effective secondary prevention strategies to reduce the public health impact of CHD

-Improved health among individuals of all ages with

CHD

-Improved quality of life of individuals of all ages with

CHD

-Improved continuity of care of individuals of all ages with

CHD STAR 2 Component A Strategies/Activities

▪ Program Planning

– Identify and engage partners, establish partnership collaborations

– Demonstrate effective relationships and cooperation with appropriate partners (letters of support)

– Identify and prioritize program goals

– Define and describe the source population (≥ 1 million/year)

– Identify data sources (clinic data, vital records, immunization systems, hospital discharge, payers, birth defect registries)

▪ Project Management

– Develop a management and staffing plan for implementation of this program

– Provide evidence of legal authority to access data

– Describe how information will be disseminated to stakeholders

– Participate on all cooperative agreement calls and on-site meetings

– Update CDC regularly on project status

▪ Case Identification

– Use all available data sources (birth defects registry data, clinical and administrative data sources) to identify eligible cases

• 0-64 years at any time between 2021-2023

• Reside in the source population geographic area at any time during 2021-2023

• Have one or more ICD-10-CM (or equivalent ICD-9-CM or BPA) CHD codes in a case finding data source (birth defects registry data, clinical and administrative data); see NOFO page 12-13

– Applicants should demonstrate, through memorandums of understanding and/or letters of support, the ability to access and share the data per the NOFO parameters.

– Demonstrate capacity to access, link, and use data to identify individuals with one or more healthcare encounters during 2021-2023.

▪ Data collection

– Demonstrate capacity to access, link and analyze data

– Identify and describe any data use agreement (DUA) or IRB data transmission restrictions

– Establish vital status of identified cases

– Geocode address to assess census tract-level demographic and socioeconomic status indicators agreed upon by CDC and all program recipients

– Create standardized diagnostic, morbidity, and healthcare resource utilization groupings as agreed upon by CDC and all program recipients

▪ Data Reporting

– Collect/store standardized data: Collect and store data from all data sources in a standardized manner with common data elements

• Refer to Appendix for required data elements

– Submit de-identified, de-duplicated individual level data, including standardized data elements using a common database to CDC in a secure manner in a timeline established by CDC

– Component A recipients will have final dataset submitted to CDC by end of project Year 3 (September 2027)

▪ Data Assessment

– Improve knowledge of characteristics of individuals with CHD

– Describe population of individuals with CHD

– Assess quality of data sources for completeness of data

– If recipient was funded for CHD STAR 1 (NOFO #DD19-1902): validate 200-300 in Year 1

▪ Dissemination

– Communicate project information and results to healthcare providers, patients, families, stakeholders, the public

– Work with state/local partners to use site-specific CHD data to identify potential areas for systems-based improvements

– Participate in collaborative projects/manuscripts

– Lead one or more collaborative manuscripts using multi-site data and submit to CDC clearance within 12 months of data finalization

Outcomes Intermediate Outcomes

Long-term Outcomes

Dissemination -Share information, communicate results -Inform stakeholders on healthcare utilization, comorbidities, and outcomes of individuals with CHD -Work with local/state partners to use site-specific CHD data

-Improved knowledge about the characteristics of individuals identified from multiple data sources

-Improved understanding of:

• age-specific mortality

• healthcare use, comorbidities, outcomes

• health equity

• racial, ethnic, socioeconomic patterns in healthcare use and outcomes

• respiratory viruses— healthcare use during COVID-19 pandemic

• potential impact of state

-Improved use of surveillance data to understand the outcomes of individuals with CHDs

-Improved ability of policy-makers to make data-informed decisions

-Improved ability to develop effective secondary prevention strategies to reduce the public health impact of CHD

-Improved health among individuals of all ages with

CHD

-Improved quality of life of individuals of all ages with

CHD

-Improved continuity of care of individuals of all ages with

Short-Term Outcomes

▪ Improved knowledge about the characteristics of individuals with CHD identified from multiple data sources

▪ Improved understanding of:

• age-specific mortality

• healthcare use, comorbidities, other outcomes

• health equity

• racial, ethnic, and socioeconomic patterns in healthcare use and long-term outcomes

• Respiratory viruses (including COVID-19, influenza, and RSV) and immunizations

• healthcare use during the COVID-19 pandemic

• the potential impact of state and national healthcare policies

▪ Increased awareness among stakeholders of nationwide CHD issues

Intermediate Outcomes

▪ Improved ability to develop surveillance programs

▪ Improved use of surveillance data to understand the outcomes of individuals with CHDs

▪ Improved ability of policy-makers to make data-informed decisions

▪ Improved ability to develop effective secondary prevention strategies to reduce the public health impact of CHD

Long-Term Outcomes

▪ Improved health among individuals of all ages with CHD

▪ Improved quality of life of individuals of all ages with CHD

▪ Improved continuity of care of individuals of all ages with CHD

Organizational Capacity

▪ An applicant’s narrative should elaborate on their ability to either create a new or enhance an existing CHD surveillance system, ensuring:

– Individuals aged 0-64 years with CHD are included

– A geographically representative source population

– Ability to begin immediate implementation of the NOFO activities

– Appropriate staff expertise (program/performance management, CHD clinical expertise, data analysis, data linkage, epidemiologic/surveillance expertise, communication/material development, informatics, and financial reporting)

– CVs of existing staff are included

▪ An applicant’s narrative should elaborate on their ability to either create a new or enhance an existing CHD surveillance system, ensuring:

– Legal authority to access and share individual level, de-duplicated, de-identified data with CDC and other recipients is included in application

– Applicants must describe required DUAs to access and share data with CDC and other recipients

– Applicants must indicate any restrictions (e.g., IRB, HIPAA) to provide all required data elements as indicated in Appendix

– Capacity to lead and publish original manuscripts in a timely manner, as evidenced by citations of recent first- or senior-author publications

Component B

Goal: conduct population-based surveillance of congenital heart defects (CHD) to describe health outcomes including health equity, with the goal of identifying opportunities to improve the health of all people living with CHD

Congenital Heart Defects Surveillance across Time And Regions (CHD STAR) 2

CHDSurveillance@cdc.gov

Component A -required-

Identify CHD cases; Collect, manage, and analyze data, submit data to CDC; and disseminate information to partners

Component B -optional-

Develop and apply machine learning model to data collected

Related work

▪ CHD STAR (#DD19-1902) Component B

– Validated billing codes for congenital heart defects in healthcare claims data

– Determined best ways to use healthcare claims for CHD surveillance across the lifespan, using machine learning and natural language processing

Outcomes

Intermediate Outcomes

Long-term Outcomes

Program Planning Project Management Data Reporting Same as Component A Dissemination

Data Assessment -Develop and refine a machine learning model for the surveillance data collected through Component A -Apply this machine learning model to data collected from all recipients funded through Component A

Data Reporting -Document the process from raw data (from Component A) to final classification for complete reproducibility -Submit pre-processed dataset and model via standardized format, model code and documentation, and results (i.e., model’s classification for each individual person) to CDC

-Improved understanding of how machine learning algorithms and models can be applied to surveillance data to identify individuals with

CHD

-Improved ability to develop surveillance programs

-Improved use of surveillance data to understand the outcomes of individuals with CHDs

-Improved ability of policy-makers to make data-informed decisions

-Improved ability to develop effective secondary prevention strategies to reduce the public health impact of CHD

-Improved accuracy of surveillance of CHD

-Improved health among individuals of all ages with

CHD

-Improved quality of life of individuals of all ages with

CHD

-Improved continuity of care of individuals of all ages with

Outcomes

Intermediate Outcomes

Long-term Outcomes

Program Planning Project Management Data Reporting Dissemination

Data Assessment -Develop and refine a machine learning model for the surveillance data collected through Component A -Apply this machine learning model to data collected from all recipients funded through Component A

Data Reporting -Document the process from raw data (from Component A) to final classification for complete reproducibility -Submit pre-processed dataset and model via standardized format, model code and documentation, and results (i.e., model’s classification for each individual person) to CDC

-Improved understanding of how machine learning algorithms and models can be applied to surveillance data to identify individuals with

CHD

-Improved ability to develop surveillance programs

-Improved use of surveillance data to understand the outcomes of individuals with CHDs

-Improved ability of policy-makers to make data-informed decisions

-Improved ability to develop effective secondary prevention strategies to reduce the public health impact of CHD

-Improved accuracy of surveillance of CHD

-Improved health among individuals of all ages with

CHD

-Improved quality of life of individuals of all ages with

CHD

-Improved continuity of care of individuals of all ages with

CHD

Same as Component A

CHD STAR 2 Component B

▪ Program Planning

– Identify and engage partners, establish partnership collaborations

– Identify and prioritize program goals

▪ Project Management

– Develop a management and staffing plan for implementation of this program. Need staff/partners with experience/expertise to:

• train machine learning models for healthcare data with administrative diagnosis codes

• understand existing machine learning models

• implement transfer learning of pre-trained models to new data, and

• adjust parameters and thresholds of the machine learning pipeline or otherwise refine pre-trained models as needed for proper application to new data

▪ Data Assessment

– Develop and refine a machine learning model for the surveillance data collected through Component A:

• Refer to materials and references provided by CDC for descriptions of existing machine learning models (pre-trained on similar data) to be used as the base for this component of the current NOFO.

• Evaluate the annotated dataset of validated cases from Component A

• Use the annotated data to further develop and refine the pre-trained models provided by CDC via transfer learning

• Recipients must apply the refined machine learning models to data collected from all recipients funded through Component A.

▪ Data Reporting

– Document the end-to-end process from raw data (from Component A) to final classification with transparency for complete reproducibility and accurate use.

– Submit all datasets (including raw and preprocessed datasets), all source code to train the model, the final trained model (via a standardized format), documentation, and results (i.e., model’s classification for each individual person) to CDC in a secure manner.

• CDC has a secure network that will be used for this project and recipients will need to obtain and maintain access.

▪ Dissemination

– Add the source code(s), trained model(s), and corresponding documentation on a publicly available platform (e.g., GitHub)

– Work collaboratively with all recipients and CDC using the multi-site data to communicate project information and results to healthcare providers, patients, families, stakeholders, the general public and other target audiences via publications, presentations, and other methods.

• The multi-site data should include the model’s classification for each individual person.

– Participate in collaborative projects/manuscripts using multi-site data whenever possible.

Outcomes

Intermediate Outcomes

Long-term Outcomes

Program Planning Data Reporting Same as Component A Dissemination

Data Assessment -Develop and refine a machine learning model for the surveillance data collected through Component A -Apply this machine learning model to data collected from all recipients funded through Component A

Data Reporting -Document the process from raw data (from Component A) to final classification for complete reproducibility -Submit pre-processed dataset and model via standardized format, model code and documentation, and results (i.e., model’s classification for each individual person) to CDC

-Improved understanding of how machine learning algorithms and models can be applied to surveillance data to identify individuals with

CHD

-Improved ability to develop surveillance programs

-Improved use of surveillance data to understand the outcomes of individuals with CHDs

-Improved ability of policy-makers to make data-informed decisions

-Improved ability to develop effective secondary prevention strategies to reduce the public health impact of CHD

-Improved accuracy of surveillance of CHD

-Improved health among individuals of all ages with

CHD

-Improved quality of life of individuals of all ages with

CHD

-Improved continuity of care of individuals of all ages with

Outcomes

▪ Short-term

– Improved understanding of how machine learning algorithms and models can be applied to surveillance data to identify individuals with CHD

▪ Intermediate

– Improved ability to develop surveillance programs

– Improved use of surveillance data to understand the outcomes of individuals with CHDs

– Improved ability of policy-makers to make data-informed decisions

– Improved ability to develop effective secondary prevention strategies to reduce the public health impact of CHD

– Improved accuracy of surveillance of CHD

▪ Short-term

– Improved understanding of how machine learning algorithms and models can be applied to surveillance data to identify individuals with CHD

▪ Intermediate

– Improved ability to develop surveillance programs

– Improved use of surveillance data to understand the outcomes of individuals with CHDs

– Improved ability of policy-makers to make data-informed decisions

– Improved ability to develop effective secondary prevention strategies to reduce the public health impact of CHD

▪ Short-term

– Improved understanding of how machine learning algorithms and models can be applied to surveillance data to identify individuals with CHD

▪ Intermediate

– Improved ability to develop surveillance programs

– Improved use of surveillance data to understand the outcomes of individuals with CHDs

– Improved ability of policy-makers to make data-informed decisions

– Improved ability to develop effective secondary prevention strategies to reduce the public health impact of CHD

▪ Long-term

– Improved health among individuals of all ages with CHD

– Improved quality of life of individuals of all ages with CHD

– Improved continuity of care of individuals of all ages with CHD

▪ An applicant’s project narrative should elaborate on:

– Staff with skill sets to implement this award

– Their experience and expertise in:

• using machine learning to improve the validity of case identification using administrative healthcare data

• performing transfer learning

• publishing original manuscripts on machine learning efforts to improve the validity of healthcare data in a timely manner, as evidenced by citations of recent publications

• posting source code(s), trained model(s), and corresponding documentation on a publicly available platform in a timely manner, as evidenced by hyperlinks to previous releases of machine learning models and documentation.

– Their ability to begin immediate implementation of the NOFO activities.

Evaluation and Work Plans

Pages 25-26; template on 39

▪ A logic model is not required

▪ An evaluation plan is required and should demonstrate how the program will fulfill requirements in the CDC Evaluation and Performance Measurement and Project Description sections of the NOFO

– Collect the performance measures, respond to the evaluation questions, and use evaluation findings for continuous program quality improvement.

– How key program partners will participate in the evaluation processes

– Plans for updating the Data Management Plan

Evaluation and Performance Measurement Plan

Work Plan

▪ A detailed work plan for Year 1 of the project is required

▪ Applicants do not have a required format for submitting the work plan

▪ An example work plan template has been provided in the NOFO

– See page 39

▪ Work plans should include:

– A detail of the first year of the project that aligns with the strategies, activities, and evaluation and performance measurement section

– A high-level description of plans for years 2-5 activities

▪ The work plan is included in the 15-page limit for the project narrative section

Award and Eligibility Information

Pages 29 – 31

Expected Awards / Performance Periods

Component A

Award up to 7 recipients for

5-year period of performance

(five 12-month budget periods)

Component B

Award 1 recipient for

5-year period of performance

(five 12-month budget periods)

Expected Awards / Budgets

Component A

Expected average award amount per budget period:

$500,000 (range: $400,000 - $500,000)

Expected total program funding over the performance period:

$15,500,000

Component B

Expected average award amount per budget period:

$300,000 ($50,000 for Year 1; $300,000/year Years 2-5)

Eligibility: Component A & B

• List of organizations eligible to apply on page 31

• Only one application per organization will be accepted.

• Component A is required. Component B is optional.

• Organizations applying for both Component A and Component B will submit a single application. Within the application, each component will be evaluated separately.

Submission Procedures

Pages 32 – 44

▪ Table of contents (no page limit- not included in project narrative page limit)

▪ Project Abstract Summary

▪ Project Narrative

– max 15 pages, single-spaced, 12 pt font, 1-inch margins, number all pages (includes the work plan)

– Component B: additional 4 pages for Project Narrative

– Include:

• Background

• Approach: Purpose, Outcomes, Strategies & Activities, Collaborations, Population(s) of Focus and Health Disparities

• Applicant Evaluation and Performance Measurement Plan (including Data Management Plan)

• Organizational Capacity of Applicants to Implement the Approach

• Work Plan

▪ Budget Narrative

▪ Attachments

Application Submission Guidance

Review and Selection Process

Pages 44 – 51

▪ Phase I Review

– Office of Grants Services initially reviews for eligibility and completeness

– Non-responsive applicants will not advance to Phase II review

▪ Phase II Review

– Review panel convenes to evaluate complete, eligible applications

– Refer to specifics on review criteria for each of the key areas of the NOFO and their corresponding score points

• Key areas: Approach (40 points), Evaluation and Performance Management (20 points), and Organizational Capacity to Implement Approach (40 points)

• Budget not scored, but evaluated

Review and Selection Process

▪ Phase III Review

– Applications for each component will be rank-ordered

– CDC may fund out of the rank order determined by the review panel.

The following factors may affect the funding decision:

• Diversity of population

• Size of the population under surveillance and the ability to obtain data that is population-based

• Geographic location and diversity

• Availability of funds and program priorities.

Review and Selection Process (cont.)

Key Dates and Questions

CHD STAR 2 Component A & B Key Dates

• Application deadline: May 14, 2024 (Electronically submitted applications must be submitted no later than 11:59 pm ET)

• Optional letter of intent deadline: April 12, 2024

• Expected Award Date: September 1, 2024

• Expected Start Date: September 30, 2024

• Expected Project End date: September 29, 2029

For more information, contact CDC

1-800-CDC-INFO (232-4636)

TTY: 1-888-232-6348 www.cdc.gov

The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention.

Questions:

Questions accepted through May 3, 2024