PKG00257724-instructions.pdf

Centers for Disease Control and Prevention

National Center on Birth Defects and Developmental Disabilities

Improving Timely Documentation, Reporting, and Analysis of Diagnostic and Intervention

Data through Optimization of EHDI Surveillance Practices and Information Systems

CDC-RFA-DD20-2006

Application Due Date: 03/16/2020

Improving Timely Documentation, Reporting, and Analysis of Diagnostic and Intervention

Data through Optimization of EHDI Surveillance Practices and Information Systems

CDC-RFA-DD20-2006

TABLE OF CONTENTS

Part I. Overview Information A. Federal Agency Name B. Funding Opportunity Title C. Announcement Type D. Agency Funding Opportunity Number E. Assistance Listings (CFDA) Number F. Dates G. Executive Summary

Part II. Full Text A. Funding Opportunity Description B. Award Information C. Eligibility Information D. Application and Submission Information E. Review and Selection Process F. Award Administration Information G. Agency Contacts H. Other Information I. Glossary

Part I. Overview Information Applicants must go to the synopsis page of this announcement at www.grants.gov and click on the "Subscribe" button link to ensure they receive notifications of any changes to CDC-RFA- DD20-2006. Applicants also must provide an e-mail address to www.grants.gov to receive notifications of changes.

A. Federal Agency Name:

Centers for Disease Control and Prevention (CDC) / Agency for Toxic Substances and Disease Registry (ATSDR) B. Notice of Funding Opportunity (NOFO) Title:

Improving Timely Documentation, Reporting, and Analysis of Diagnostic and Intervention Data through Optimization of EHDI Surveillance Practices and Information Systems C. Announcement Type: New - Type 1 This announcement is only for non-research activities supported by CDC. If research is proposed, the application will not be considered. For this purpose, research is defined at https ://www.gpo.gov/fdsys/pkg/CFR-2007-title42-vol1/pdf/CFR-2007-title42-vol1-sec52-2.pdf.

Guidance on how CDC interprets the definition of research in the context of public health can be found at https://www.hhs.gov/ohrp/regulations-and-policy/regulations/45-cfr-46/index.html (See section 45 CFR 46.102(d)).

D. Agency Notice of Funding Opportunity Number:

CDC-RFA-DD20-2006

E. Assistance Listings (CFDA) Number:

93.314 F. Dates:

1. Due Date for Letter of Intent (LOI): 02/06/2020

2. Due Date for Applications: 03/16/2020, 11:59 p.m. U.S. Eastern

Standard Time, at www.grants.gov.

3. Date for Informational Conference Call:

01/23/2020 2:00pm, EST Join Zoom Meeting https://zoom.us/j/314597149 Meeting ID: 314 597 149 One tap mobile +1 929 205 6099 / 314597149# US (New York) +1 669 900 6833 / 314597149# US (San Jose) Dial by your location +1 929 205 6099 US (New York) +1 669 900 6833 US (San Jose) Meeting ID: 314 597 149 Find your local number: https://zoom.us/u/aGOXDp8Fb https://www.grants.gov https://www.grants.gov https://www.gpo.gov/fdsys/pkg/CFR-2007-title42-vol1/pdf/CFR-2007-title42-vol1-sec52-2.pdf https://www.gpo.gov/fdsys/pkg/CFR-2007-title42-vol1/pdf/CFR-2007-title42-vol1-sec52-2.pdf https://www.hhs.gov/ohrp/regulations-and-policy/regulations/45-cfr-46/index.html https://www.grants.gov https://zoom.us/j/314597149 https://zoom.us/u/aGOXDp8Fb

G. Executive Summary:

1. Summary Paragraph:

The Centers for Disease Control and Prevention (CDC) announces availability of funds, for notice of funding opportunity (NOFO), Improving Timely Documentation, Reporting, and Analysis of Diagnostic and Intervention Data through Optimization of EHDI Surveillance. The timely receipt of diagnostic and intervention data is made possible in part by jurisdictions using their CDC-funded EHDI Information Systems (EHDI-IS) to actively track infants to ensure they receive essential screening, diagnostic, and intervention services. This NOFO aims to advance this capacity by optimizing jurisdictions’ existing EHDI-IS and expanding their ability to report, analyze, and use patient-level data. This will involve four strategies: 1) Enhance EHDI-IS through system optimization; 2) Engage stakeholders in the tracking and reporting processes and provide technical assistance during the reporting process; 3) Maintain quality of the data and lead strategic actions for continuous program improvement through data analysis and evaluation; and 4) Develop data reports and disseminate information among internal and external stakeholders. The expected outcomes during the award are: 1) Use of timely and complete patient-level data among EHDI programs for tracking and to inform decision making;

2) Increased number of infants who receive a diagnosis no later than 3 months of age; 3) Increased enrollment in intervention services no later than 6 months of age, for infants who are deaf or hard of hearing.

a. Eligible Applicants: Open Competition

b. NOFO Type: Cooperative Agreement

c. Approximate Number of Awards: 35

d. Total Period of Performance Funding: $28,000,000

e. Average One Year Award Amount: $160,000

f. Total Period of Performance Length: 4

g. Estimated Award Date: 06/01/2020

h. Cost Sharing and / or Matching Requirements: N Cost sharing or matching funds are not required for this program. Although no statutory matching requirement for this NOFO exist, leveraging other resources and related ongoing efforts to promote sustainability is strongly encouraged.

Part II. Full Text A. Funding Opportunity Description

Part II. Full Text

1. Background

a. Overview Congenital hearing loss affects nearly one in every 500 live-born infants and has been referred to as a neuro-developmental emergency. When left undetected, congenital hearing loss can negatively impact children through delays in their speech, language, social, and emotional development. In response, jurisdictions have established Early Hearing Detection and Intervention (EHDI) programs to ensure all infants who are deaf and hard of hearing (D/HH) are identified early and receive intervention services. National benchmarks call for hearing screening no later than 1 month of age, diagnosis no later than 3 months of age for infants not passing the screening, and enrollment in intervention no later than 6 months of age for those identified as D/HH, and form what is known as the EHDI 1-3-6 plan.

The CDC has provided cooperative agreements for the development of jurisdictional EHDI Information Systems (EHDI-IS). Jurisdictions use their EHDI-IS to collect and capture real-time data, which enables active tracking. This tracking helps ensure each child who is at risk for permanent hearing loss or who is diagnosed as D/HH receives a timely diagnosis and intervention services. This is crucial to minimizing the risks of developmental delays from undetected or late identified hearing loss.

The most recent Notice of Funding Opportunity (NOFO) DD17-1701 marked a shift in focus areas by CDC. The first area involved moving from supporting hearing screening related activities to focusing on diagnosis and enrollment in intervention services. This has supported increased capacity among funded jurisdictions to help ensure and document the delivery of clinically recommended diagnostic and intervention services before the three and six month benchmarks, respectively. The second area involved further enhancing the capacity for active tracking and timely surveillance by funding 9 states to report patient-level data to CDC (i.e., Expanded Activities).

The successful reporting and analysis of relevant and standardized sets of patient-level data have enabled CDC to begin moving beyond the limitations of aggregate data. CDC, in collaboration with the 9 states, is conducting detailed analyses to better understand why some infants are not receiving follow-up services, identify areas for improvement in service delivery and tracking, inform CDC’s technical assistance, and answer questions of interest from stakeholders (e.g., average age of hearing loss identification; indicators for loss to follow-up).

The long-term outcomes of this NOFO include: Use of timely and complete patient-level data among EHDI programs for tracking and to inform decision making; Increased number of infants who receive a diagnosis no later than 3 months of age; Increased enrollment in intervention services no later than 6 months of age, for infants who are D/HH.

This includes supporting all funded jurisdictions and their primary stakeholders (e.g.

audiologists) to:

1. Enhance jurisdictional EHDI-IS that conforms to CDC EHDI Functional Standards [cdc.gov /ncbddd/hearingloss/ehdi-is-functional-standards.html] and strengthen data linkages through EHDI-IS optimization.

2. Promote and support collaboration with stakeholders around tracking and surveillance activities within the jurisdiction through stakeholder engagement and providing technical assistance during the reporting process

3. Maintain data quality and lead strategic actions for continuous program improvement through http://www.cdc.gov/ncbddd/hearingloss/ehdi-is-functional-standards.html ongoing data analysis

4. Develop detailed data reports and disseminate relevant information among internal and external stakeholders

b. Statutory Authorities Section 399M (b) (1) of the Public Health Service Act, [42 U.S.C. 280g-1(b)(1)

c. Healthy People 2030 This program addresses the "Healthy People 2030" focus area(s) of Hearing and other Sensory or Communication Disorders (ENT-VSL 1):

Increase the proportion of newborns who are screened for hearing loss by no later than age 1 month, have audiologic evaluation no later than 3 months of age, and are enrolled in appropriate intervention services no later than 6 months of age.

Note: This objective is scheduled to be included in HP 2030. HP 2030 is expected to be launched in the spring of 2020.

d. Other National Public Health Priorities and Strategies This NOFO aligns with CDC strategic directions to strengthen tracking and surveillance capacity of public health programs and collaboration and data exchange between health departments and health care services. It also aligns with the CDC's priorities of early identification and ensuring receipt of recommended services. In addition, it aligns with the 2007 Joint Committee on Infant Hearing Position Statement, which recommends jurisdictions implement data management and tracking systems. These systems are to monitor the quality of services and provide a link between health and education professionals and help determine the outcomes of children who are D/HH (jcih.org/posstatemts.htm).

e. Relevant Work The following NOFOs have supported jurisdictions in implementing a broad range of EHDI activities. These include 1) developing an EHDI-IS and supporting integration with other child health data systems 2) Enhancing the EHDI-IS to support documentation and use of follow-up diagnostic and intervention services data, and 3) Reporting patient-level data from jurisdictional EHDI-IS to CDC to strengthen EHDI surveillance. These efforts have helped jurisdictions ensure over 95% of newborns are screened and led to over 64,000 infants who are D/HH being diagnosed early between 2005 and 2017.

DD08-803 Early Hearing Detection and Intervention (EHDI) Tracking, Surveillance and Integration DD09-903 Early Hearing Detection and Intervention (EHDI) Tracking, Surveillance and Integration DD11-1101 Development, Maintenance and Enhancement of Early Hearing, Detection and Intervention Information System (EHDI-IS) Surveillance Programs DD17-1701 Documentation and Use of Follow-up Diagnostic and Intervention Services Data http://www.jcih.org/posstatemts.htm through the Maintenance and Enhancement of the Early Hearing Detection and Intervention Information System

2. CDC Project Description

a. Approach

Bold indicates period of performance outcome.

CDC-RFA-DD20-2006

Improving Timely Documentation, Reporting, and Analysis of Diagnostic and Intervention Data through Optimization of EHDI Surveillance Bold indicates period of performance outcome

Strategies and Activities

Short-term Outcomes Intermediate Outcomes Long-Term Outcomes

EHDI-IS

optimization Enhance jurisdictional EHDI information system that conforms to CDC EHDI Functional Standard and strengthen data linkages through EHDI-IS optimization

Engaging stakeholders in the Follow-up tracking and reporting Promote and support collaboration with stakeholders around tracking and surveillance activities within the jurisdiction through stakeholder engagement and providing technical assistance during the reporting process

Recipients meet the “Shall” requirements of the CDC EHDI Functional Standards

Improved collaboration between audiologists and jurisdictional EHDI program

Increased knowledge and skills among audiologists in the jurisdiction about reporting hearing results

Improved collaboration between Early Intervention programs and jurisdictional EHDI program

More audiologists in the jurisdiction report high quality diagnostic hearing results

Improved accuracy, completeness and timeliness of follow-up and enrollment intervention data

Increased targeted dissemination of information among internal and external stakeholders

Increased timely submission of standardized patient-level datasets to CDC (excluding PII)

Increased

Use of timely and complete patient-level data among EHDI programs for tracking and to inform decision making

Increased number of infants who receive a diagnosis no later than 3 months of age

Increased enrollment in Early Intervention (EI) services no later than 6 months of age, for infants who are deaf or hard of hearing

Data Analysis And Evaluation Maintain data quality and lead strategic actions for continuous program improvement through ongoing data analysis

Data Submission and dissemination Develop detailed data reports and disseminate relevant information among internal and external stakeholders

Increased knowledge among program staff and stakeholders of current data issues, including loss to follow-up and loss to documentation

Increased understanding by jurisdictional EHDI program and stakeholders of strengths and weaknesses of the system standardization and comparability of EHDI follow-up data by CDC at national level.

i. Purpose The purpose of this funding is to support jurisdictions that have routinely identified infants who are D/HH, and are working to provide and document recommended follow-up testing and intervention services. This award will support 1) Optimization of jurisdictional EHDI tracking and surveillance process and information systems, and 2) Standardized reporting, analysis and use of timely patient-level data to ensure infants receive timely diagnostic and intervention services.

ii. Outcomes Short-term Outcomes

Recipients meet the “Shall” requirements of the CDC EHDI Functional Standards Improved collaboration between audiologists and jurisdictional EHDI program Increased knowledge and skills among audiologists in the jurisdiction about reporting hearing results

Improved collaboration between early intervention programs and jurisdictional EHDI program

Increased knowledge among program staff and stakeholders of current data issues, including loss to follow-up and loss to documentation

Increased understanding by jurisdictional EHDI program and stakeholders of strengths and weaknesses of the system

Intermediate Outcomes

More audiologists in the jurisdiction report high quality diagnostic hearing results Improved the accuracy, completeness and timeliness of follow-up and intervention enrollment data Increased targeted dissemination of information among internal and external stakeholders Increased the timely submission of standardized patient-level datasets to CDC

(excluding PII) Increased standardization and comparability of EHDI follow-up data by CDC at national level

Long term Outcomes

Use of timely patient-level data among EHDI programs for tracking and to inform decision making

Increased the number of infants who receive a diagnosis no later than 3 months of age Increased enrollment in Early Intervention services no later than 6 months of age, for infants who are D/HH.

iii. Strategies and Activities Recipients are required to complete ALL required activities from each of the 4 focus areas and at least one additional activity from each of EHDI-IS Optimization and Stakeholder Engagement and Follow- up focus area as defined below:

EHDI-IS Optimization Strategy: Enhance jurisdictional EHDI Information System that conforms to CDC EHDI Functional Standard and Strengthen data linkages through EHDI-IS optimization.

This will include activities that focus on optimizing the existing EHDI-IS infrastructure and functionalities to improve the collection, management and efficient use of diagnostic and intervention data, in accordance with the EHDI-IS Functional Standards. www.cdc.gov/ncbddd /hearingloss/ehdi-is-functional-standards.html. Recipients will work towards meeting all “SHALL” requirements under Goals 2-8.

EHDI-IS Optimization Activities:

i) Required: All recipients must maintain the collection and management of the hearing screening data, and the electronic exchange of demographic data between EHDI-IS and Vital

Records in order to maintain all the “SHALL” requirements of Goal 1 of the CDC EHDI-IS Functional Standards, and all the functional standards under Goals 2-8 that were previously met.

ii) Required: Enhance EHDI-IS capacity to meet the SHALL requirements under Goals 2-8 in the EHDI-IS Functional Standards that are currently not met. This would include collection and reporting of Tier 2 patient-level data items that are not previously available to the recipient (www.cdc.gov/ncbddd/hearingloss/ehdi-programs.html).

iii) Required: Enhance EHDI-IS capacity to support preparation of patient-level dataset: All recipients are required to extract data from their EHDI-IS and construct a patient-level limited* dataset according to given standard and format. Recipients are required to develop and implement certain pseudonymization** functionality and protocol to remove direct identifiers from each patient in the system.

*Information in a limited dataset is not directly identifiable and excludes direct identifiers such as names or social security numbers of the individual or relatives, employers, or household numbers of the individual.

**Each child record in the dataset has a unique, randomly generated, artificial identifier, known as the pseudonym. The length and format of the pseudonym must conform to a given standard and must not contain, and not a part of any of the child’s personal identifiable information such as name, social security number, medical record number, street address (excluding zip code). A mechanism must be in place to allow the developer to link the pseudonym back to the original child record for which it was generated, and the pseudonymization procedure is repeatable for the same child, and can be used for multiple datasets (e.g. updates) that include data about the same child, while the child’s original pseudonym is unchanged across subsequent updates.

iv) Additional: Upgrade the audiology reporting modules in EHDI-IS. Recipients choosing to implement this activity should first complete an evaluation on the usability of their EHDI-IS audiology reporting module and upgrade the system to address issues identified in the evaluation.

v) Additional: Develop and implement data linkages to enable data sharing between jurisdictional EHDI-IS and other public health/intervention information systems to improve timeliness and completeness of follow-up diagnostic and intervention information. Examples include but not limited to: state Medicaid system, WIC, early intervention systems.

vi) Additional: Enhanced data reports: Expand EHDI-IS’ capacity to generate automated, on-demand data reports. This should include the ability to generate, present and transmit standard and/or custom-defined reports (e.g., CDC Hearing Screening and Follow-up Survey (HSFS) survey, loss to follow-up, and other relevant indicators) for a range of stakeholders without assistance from system vendor or IT personnel.

Stakeholder Engagement for Follow-up Tracking and Reporting Strategy: Promote and support collaboration with stakeholders around tracking and surveillance activities within the jurisdiction through stakeholder engagement, and providing technical assistance during the reporting process.

Key partners and stakeholders include but not limited to: other state and territories, intervention programs, birthing hospitals, home birth facilities, audiologists, pediatric clinicians, health care http://www.cdc.gov/ncbddd/hearingloss/ehdi-programs.html systems and healthcare provider groups, and other health and social service organizations.

Stakeholder Engagement Activities:

i) Required: Obtain number of audiology diagnostic facilities in the jurisdiction. Recipients should maintain an up-to-date list of audiology diagnostic facilities in the jurisdiction and how they comply with established reporting protocol.

ii) Required: Maintain/Improve connection and collaboration with audiologists. Examples include: work with audiologists to incorporate data reporting in their regular clinical practice to reduce reporting burden; collaborate with audiologists to review and address key data needs and gaps.

iii) Required: Maintain/Improve connection and collaboration with intervention programs.

Examples include: Collaborate with state and local intervention programs to review and address key data needs and gaps, improve data exchange methods and protocols, etc.

iv) Required: All recipients are required to attend and participate in an in-person kick-off and planning meeting in Atlanta for the purpose of sharing the latest information and collaborating with other experts in the field of early hearing loss on best practices in early hearing detection and surveillance.

v) Additional: Maintain/Improve connection and collaboration with other stakeholders.

Examples include: participate in EHDI advisory group activities; increase awareness of EHDI and reporting protocols for other providers such as pediatricians, ENT doctors, etc.

vi) Additional: Strengthen audiologist reporting protocols. Recipients choosing to implement this activity in the first year should have completed an evaluation activity to identify challenges/gaps in their audiologist reporting protocol in the past and update the reporting protocol to address issues identified in the evaluation.

vii) Additional: Conduct training and education of audiologists on EHDI and reporting protocols to improve compliance with established reporting protocols.

viii) Additional: Recipients can attend the annual National EHDI conference for the purpose of sharing the latest information and collaborating with other experts in the field of early hearing loss on best practices in early hearing detection and surveillance.

Analysis and Evaluation Strategy: Maintain data quality and lead strategic actions for continuous program improvement ongoing data analysis.

Activities in this focus area will lead to strategic actions for program improvement (e.g., processes to reduce errors, eliminate duplicate records, and ensure accuracy and validity of the data reported). The recipients are required to collaborate with CDC on developing and implementing a data analysis plan (trend analysis, analysis to identify demographic groups that should be targeted to reduce Lost to follow-up and/or lost to documentation (LFU/LTD), or similar analysis).

Analysis and Evaluation Activities:

i) Required: Assess quality of data collected in the EHDI-IS in accordance with the EHDI-IS Functional Standards 5.1 “Regularly evaluate incoming and existing patient records to identify, prevent, and resolve duplicate and fragmented records”, and 5.5 “Analyze information with respect to data quality”.

ii) Required: Assess program progress towards stated outcomes using data collected in the

EHDI-IS.

iii) Required: Collaborate with CDC to conduct detailed analysis on patient-level data to identify trends and gaps.

Data Submission and Dissemination Strategy: Develop detailed data reports and disseminate relevant information among internal and external stakeholders.

Activities in this focus area will lead to increased knowledge and understanding of EHDI data and system among stakeholders, and increased standardization and comparability of EHDI data at national level.

Data Submission and Dissemination Activities:

i) Required: Strategic communication and dissemination of findings of data analysis and evaluation to key stakeholders and partners. Findings and lessons may be shared through various communication modes, such as reports, presentations, webinars and websites. In addition, recipients are expected to improve EHDI-IS visibility (e.g. publish and disseminate fact sheets, reports, briefs, newsletters, or other materials as needed to support project activities). Recipients should consider presenting at state, regional, and national meetings and conferences, such as those held by the American Academy of Audiology, American Speech-language-hearing Association, and the National EHDI Meeting.

ii) Required: Data submission to CDC. This includes two parts:

1) Submission of HSFS survey: Recipients not funded under the DD17-1701 Expanded Activities must submit the annual HSFS survey during the first two budget years after funds are awarded. This requirement is waived for states that were previously funded under the DD17- 1701 Expanded Activities.

2) Submission of patient-level dataset to CDC: Recipients must submit standardized patient-level limited datasets that include EHDI information on all occurrent births of a specified calendar year to CDC throughout the period of performance. All recipients are required to:

a. Collaborate with CDC to develop and sign a Data Use Agreement (DUA) within the first 6 months of the award (by 12/31/2020), if applicable.

b. Develop and implement a protocol for the pseudonymization of individual level EHDI data that will be transmitted to CDC (Please also see EHDI-IS optimization activity iii)

c. Construct, validate, and submit datasets to CDC in accordance with established timeline and data specification.

d. Identify methods to improve data quality and the reporting process

Jurisdictions funded under DD17-1701 Expanded Activities

Jurisdictions not funded under DD17-1701 Expanded Activities

Year 1 07/20-12/20 Submit 2019 data Review data items and definitions, submit HSFS

Year 1 01/21-06/21 Submit updated 2019 data EHDI-IS optimization

Year 2 07/21 – 12/21 Submit 2020 data Submit test data (2020), submit

HSFS

Year 2 01/22 – 06/22 Submit updated 2020 data Submit 2020 data

Year 3 07/22-12/22 Submit 2021 data Submit 2021 data

Year 3 01/23 – 06/23 Submit updated 2021 data, submit 2022 data

Submit updated 2021 data, submit 2022 data

Year 4 07/23-06/24 Submit updated 2022 data, submit 2023 data

Submit updated 2022 data, submit 2023 data

Recipients are expected to develop and submit a plan within 6 months of the award for sustaining the project after the period of performance. Additionally, individuals identified in the Leadership, Surveillance, and Informatics capacity (described under the organizational capacity of this NOFO, maximum 3 people) are required to attend and participate in an in-person kick-off and planning meeting in Atlanta, GA.

1. Collaborations

a. With other CDC programs and CDC-funded organizations:

Recipients are required to collaborate with CDC and other jurisdictional EHDI programs, if applicable, to achieve the outcomes of this NOFO. Recipients should explore improved connections and collaborations with other CDC funded jurisdictional public health programs.

b. With organizations not funded by CDC:

Recipients should collaborate with other programs as needed, to collect and use complete and accurate data. Collaborations with strategic partners who are essential to the effective implementation of their selected strategies will facilitate strengthening information exchange between health information systems. Potential organizations for collaboration include: Vital records, Early Intervention (EI) programs, audiologists, clinicians, health care systems and healthcare provider groups, and other organizations to aid in minimizing loss to follow-up and loss to documentation; reducing delays in receiving diagnostic evaluation for infants not passing the hearing screening; and increasing enrollment in intervention services for infants confirmed to have a hearing loss. Letters of support from state vital records and early intervention programs describing what will be done as part of the collaboration must be included in the application package. MOUs/MOAs, if applicable, can also be submitted in support of the application. Name the file according to the document type and upload it as a PDF file at www.grants.gov. For sample, if document is a letter of support from vital records name the file

Vital Records LOS.

2. Target Populations The target populations for the proposed project are as follows:

All occurrent births in the jurisdiction All infants in the jurisdiction referred for audiologic diagnostic testing and medical evaluation All infants in the jurisdiction identified as D/HH and referred for early intervention

The applicants are expected to identify populations at greatest risk through evaluation and data analysis activities, and describe actions to target specific populations

a. Health Disparities Disparities by race, ethnicity, socioeconomic status, disability status, access, primary language, health literacy, education, and other relevant dimensions (e.g. rurality, tribal communities) should be considered.

iv. Funding Strategy

NA

b. Evaluation and Performance Measurement

i. CDC Evaluation and Performance Measurement Strategy Evaluation and performance measurements help demonstrate achievement of project outcomes;

build a stronger practice base for specific project strategies; clarify applicability of the evidence-based interventions to different populations, settings, and contexts; and support continuous program improvement. The EHDI-IS evaluation strategies can be supported by the following resources: EHDI Guidance Manual - Chapter 6: Monitoring and Evaluation, available at: www.cdc.gov/ncbddd/hearingloss/guidancemanual/chapter6.html the document: Planning an Evaluation of an EHDI Information System available at www.cdc.gov/ncbddd/hearingloss /documents/planning-an-evaluation.pdf and the “Updated Guidelines for Evaluating Public health Surveillance Systems” July 27, 2001/50(RR13); 1-35 available at: www.cdc.gov/mmwr /preview/mmwrhtml/rr5013a1.htm Over the course of this cooperative agreement, CDC will:

Work individually and collectively with each of the recipients to evaluate its own EHDI- IS assessing the following EHDI –IS system attributes:

o Usability of the audiology reporting module/functions in the EHDI-IS: this evaluation will focuses on describing how well users can learn and use the EHDI-IS audiology reporting module to achieve intended goals and how satisfied users are with that process.

o Usefulness of the EHDI-IS: this evaluation will focus on assessing and reporting http://www.cdc.gov/ncbddd/hearingloss/guidancemanual/chapter6.html http://www.cdc.gov/ncbddd/hearingloss/documents/planning-an-evaluation.pdf http://www.cdc.gov/ncbddd/hearingloss/documents/planning-an-evaluation.pdf http://www.cdc.gov/mmwr/preview/mmwrhtml/rr5013a1.htm http://www.cdc.gov/mmwr/preview/mmwrhtml/rr5013a1.htm the level of usefulness of the EHDI-IS by describing:

The actions taken to improve EHDI tracking and surveillance as a result of analysis and interpretation of the data.

The entities that have used the data to make decisions and take actions.

The instances of use of EHDI surveillance data leading to improve public health (e.g making referrals to Part C).

Monitor and assess the processes and outcomes of the project using performance measures. Recipients are required to: 1) Develop project-specific Process Measures in the Work Plan and use them to monitor the progress of proposed activities. Examples of process measures including: 1) Number of new EHDI-IS modules/functions added to generate patient-level data file, number of trainings on diagnostic reporting conducted, number of presentations/manuscripts produced based on analysis findings (including collaborative work), etc. 2) Use the Outcome Measures defined below to assess the degree to which short-term, mid-term, and long-term outcomes for the logic model have been achieved 3) Collect and report process and outcome measurements to CDC at the end of each budget period using the Annual Performance Measurement Report. The format will be specified in post-award guidance.

SHORT-TERM OUTCOMES PERFORMANCE MEASURES

·Recipients meet the “SHALL” requirements of the CDC EHDI Functional Standards

Number of SHALL requirements met before and after the performance period

·Improved collaboration between audiologists and jurisdictional EHDI program

Document engagement strategy to improve/maintain collaborations between audiologists and jurisdictional EHDI program

·Increased knowledge and skills among audiologists in the jurisdiction about reporting hearing results

Document training sessions completed among audiologists

·Improved collaboration between early intervention programs and jurisdictional EHDI program

Number of MOU/MOA signed /renewed between EI and EHDI programs

·Increased knowledge among program staff and stakeholders of current data issues, including loss to follow-up and loss to documentation

Description of actions taken to summarize and share information/data about identified challenges (e.g., loss to follow-up and loss documentation) among program staff and stakeholders based on the data analysis activities

·Increased understanding by jurisdictional EHDI program and stakeholders of strengths and weaknesses of the system

Description of actions taken to summarize and share information among program staff and stakeholders based on the evaluation activities.

INTERMEDIATE OUTCOMES PERFORMANCE MEASURES

·More audiologists in the jurisdiction report Total number of audiologists/audiology high quality diagnostic hearing results diagnostic facilities in the jurisdiction that see infants Total number of audiologists/audiology diagnostic facilities reporting follow-up re-rescreening and diagnostic data over time

·Improved the accuracy, completeness and timeliness of follow-up data

Average and range of diagnostic reporting time Diagnostic reporting time = Number of days between the patient’s diagnostic visit and data about this visit is reported to EHDI Completeness of follow-up data is measured by:

Percent of documented diagnostic follow up = (number of children with documented diagnostic visit + number of children with documented reason of not receiving diagnostic follow-up)/total number of children did not pass final hearing screening Percent of documented EI follow up = (number of children with documented EI enrollment (i.e.

IFSP date) + number of children with documented reason of no EI enrollment) / total number of children diagnosed with permanent hearing loss

·Increase targeted dissemination of information among internal and external stakeholders

Total of presentations; data reports;

publications disseminated among stakeholders about work to improve the timely documentation, reporting, and analysis of diagnostic and intervention data

·Increase the timely submission of standardized patient-level datasets to CDC

Successful completion, validation, and submission of patient-level data file according to established timeline, standard and protocol

LONG-TERM OUTCOMES PERFORMANCE MEASURES

·Use of timely patient-level data among EHDI programs for tracking and to inform decision making

New responses (e.g. process change, policy report, etc.) as a result of data utilization and analysis of patient-level data

LONG-TERM OUTCOMES OUTCOME MEASURES

·Increased the number of infants who receive a diagnosis no later than 3 months of age

Percent of children diagnosed overall = Total number of children with completed diagnostic assessment (no hearing loss + permanent hearing loss) / Total that did not pass hearing screen Percent of children diagnosed by 3 months of age = Total number of children diagnosed by 3 months of age / Total that did not pass hearing screen

· Baseline: 2018 data

· Target: Increase by 30% from reported baseline or achieve a minimum diagnosis rate of 85%

·Increased enrollment in early intervention services no later than 6 months of age, for infants who are deaf and hard of hearing

Percent of children enrolled in early intervention services overall = Total number of children enrolled in part C or non-Part C EI services/Total diagnosed with permanent hearing loss Percent of children enrolled in early intervention services by 6 months of age = Total number of children enrolled in early intervention services (including Part C and non-Part C) by 6 months of age/Total number* of children diagnosed with permanent hearing loss through the EHDI process

· Baseline: 2018 data

· Target: Increase by 30% from reported baseline or achieve a minimum enrollment rate of 80% the number of infants identified to be D/HH that are enrolled in EI services no later than 6 months of age.

* Diagnosed at any age

ii. Applicant Evaluation and Performance Measurement Plan Applicants must provide an evaluation and performance measurement plan that demonstrates how the recipient will fulfill the requirements described in the CDC Evaluation and Performance Measurement and Project Description sections of this NOFO. At a minimum, the plan must describe:

How applicant will collect the performance measures, respond to the evaluation questions, and use evaluation findings for continuous program quality improvement.

How key program partners will participate in the evaluation and performance measurement planning processes.

Available data sources, feasibility of collecting appropriate evaluation and performance data, and other relevant data information (e.g., performance measures proposed by the applicant)

Plans for updating the Data Management Plan (DMP), if applicable, for accuracy throughout the lifecycle of the project. The DMP should provide a description of the data that will be produced using these NOFO funds; access to data; data standards ensuring released data have documentation describing methods of collection, what the data represent, and data limitations; and archival and long-term data preservation plans.

For more information about CDC’s policy on the DMP, see https://www.cdc.gov/grants/additionalrequirements/ar-25.html.

Where the applicant chooses to, or is expected to, take on specific evaluation studies, they should be directed to:

Describe the type of evaluations (i.e., process, outcome, or both).

Describe key evaluation questions to be addressed by these evaluations.

Describe other information (e.g., measures, data sources).

Recipients will be required to submit a more detailed Evaluation and Performance Measurement plan, including a DMP, if applicable, within the first 6 months of award, as described in the Reporting Section of this NOFO.

Recipients will provide a performance measurement plan and a draft evaluation plan as part of this application to demonstrate how they will fulfill the requirements described in the CDC Evaluation and Performance Measurement and Project Description sections of this NOFO. At a minimum, the plan must describe:

How applicant will collect the performance measures, including process and outcome measures specified by CDC in the project description and presented by the applicant in their Work Plan

Mechanisms to collect data on evaluating usability and usefulness of the EHDI-IS How the applicant will use evaluation findings for continuous program quality improvement How key program partners will participate in the evaluation and performance measurement planning processes. Available data sources, feasibility of collecting appropriate evaluation and performance data, and other relevant data information (e.g., performance measures proposed by the applicant)

Plans for updating the Data Management Plan (DMP), if applicable, for accuracy throughout the lifecycle of the project. The DMP should provide a description of the data that will be produced using these NOFO funds; access to data; data standards https://www.cdc.gov/grants/additionalrequirements/ar-25.html ensuring released data have documentation describing methods of collection, what the data represent, and data limitations; and archival and long-term data preservation plans.

For more information about CDC’s policy on the DMP, see www.cdc.gov/grants/addit ionalrequirements/ar-25.html

If selected for funding, CDC EHDI will work with each recipient within six months of award to finalize project-specific performance measurements and evaluation plan. The targets are to be achieved by recipients on or before the completion of this award.

c. Organizational Capacity of Recipients to Implement the Approach Applicants must demonstrate that they have the following capacity to meet the project requirement and successfully complete the activities defined in this CDC project description section.

Leadership Capacity

Project Director/Principal Investigator (PI/PD): A PI/PD is the primary individual responsible for the scientific, technical or programmatic aspects of the CDC cooperative agreement. The PI/PD serves a key role in providing administrative oversight, budget management, and meeting project goals. This individual should have project management and supervisory experience.

Surveillance Capacity*

Surveillance: The surveillance lead performs a variety of tasks to assure the quality and integrity of EHDI follow-up data, conducts data analysis, disseminates findings with the team and relevant stakeholders, and support evaluation activities. His/her primary responsibility is to improve the documentation and use of diagnostic and early intervention data. This includes: 1) monitoring the EHDI follow-up data collection processes and providing interim and summary reports to improve tracking and surveillance activities 2) using epidemiological and statistical methods to answer key questions related to the EHDI tracking surveillance process and outcomes 3) determining the strengths and weaknesses of the EHDI-IS to establish priorities for updates and improvements 4) supporting evaluation of the effectiveness of the EHDI-IS

5) disseminating relevant, detailed EHDI data to stakeholders to support and improve diagnostic and intervention data collection and use 6) providing technical assistance or training, design and compile summary statistical report on diagnostic and intervention data. This individual should have knowledge and skills in the follow areas: research, epidemiologic methods, data collection, management, analysis, and skills in presenting technical and non-technical information to diverse audiences.

Evaluation and Performance Monitoring Capacity

Evaluation and performance monitoring: The evaluator is responsible for conducting evaluation activities, including the development, coordination, and implementation of an EHDI-IS evaluation plan in close collaboration with stakeholders, and using evaluation findings to support program improvements. This individual should be familiar with http://www.cdc.gov/grants/additionalrequirements/ar-25.html http://www.cdc.gov/grants/additionalrequirements/ar-25.html evaluation methods and have trainings and/or prior experience conducting an evaluation.

He/she should also be able to present evaluation findings and conclusions to a variety of audiences.

Health Informatics Capacity

Health informatics: The health informatician is responsible for assisting EHDI-IS optimization and evaluation, data management and quality assurance, identifying data elements affecting data quality and ways to improve data collection, ensuring standardized data definitions are applied, preparing and validation of data files submitted to CDC, and identifying and resolving any data errors. The person serving in this role should have experience working on public health information systems and large public health datasets.

Notes

1. A resume/CV that reflects the necessary training and experience for each individual that will serve in the above defined roles is required. Applicant must name the file "CVs/Resumes, save the file as a pdf, and upload at www.grants.gov

2. *In addition to the PI/PD and Authorized Organization Representative (AOR), the Surveillance Lead is designated as key personnel under this NOFO. Prompt notification to the Office of Grant Services (OGS) about changes to individuals serving in these positions is required.

d. Work Plan Applicants must submit a detailed work plan for the first year of the project and a high-level work plan for years 2-4. Required elements in the detailed year 1 work plan include for each strategy:

Performance measures (both process and outcome), 2018 data and targets for the performance measures, Activities (if EHDI-IS enhancements are proposed please list which functional standard the enhancement maps to for strategy 1 only), Responsible program staff, and proposed time frame for completion.

For the years 2-4 work plan include the following elements for each strategy: activities, time frame, and Responsible program staff for each year.

A sample work plan template is available upon request by sending an email to ehdico-op@cdc.gov. While applicants are not required to use the sample template they are required to submit a work plan with the application that includes all of the bulleted elements listed above” Applicants must name this file “Work Plan” and upload it as a PDF file @ www.grants.gov.

e. CDC Monitoring and Accountability Approach Monitoring activities include routine and ongoing communication between CDC and recipients, http://www.grants.gov http://www.grants.gov/ site visits, and recipient reporting (including work plans, performance, and financial reporting).

Consistent with applicable grants regulations and policies, CDC expects the following to be included in post-award monitoring for grants and cooperative agreements:

Tracking recipient progress in achieving the desired outcomes.

Ensuring the adequacy of recipient systems that underlie and generate data reports.

Creating an environment that fosters integrity in program performance and results.

Monitoring may also include the following activities deemed necessary to monitor the award:

Ensuring that work plans are feasible based on the budget and consistent with the intent of the award.

Ensuring that recipients are performing at a sufficient level to achieve outcomes within stated timeframes.

Working with recipients on adjusting the work plan based on achievement of outcomes, evaluation results and changing budgets.

Monitoring performance measures (both programmatic and financial) to assure satisfactory performance levels.

Monitoring and reporting activities that assist grants management staff (e.g., grants management officers and specialists, and project officers) in the identification, notification, and management of high-risk recipients.

The recipient and CDC staff will work collaboratively to assess and monitor progress on strategies and make adjustments as needed to ensure overall program objectives are met.

Immediate post-award collaboration and monitoring between CDC and the recipient will include at a minimum working to:

Revise the technical assistance/program implementation plan, as needed Revise the evaluation plan, as needed Review the work plan timeline to ensure it is feasible based on the budget and consistent with the intent of the award

Monitoring activities will include routine and ongoing communication between CDC and the recipient site via virtual and in-person meetings, conference calls, site visits, and recipient reporting (including work plans, process and outcome performance measures, monthly summary reports, and financial reporting). Consistent with applicable grants regulations and policies, CDC expects the following to be included in post-award monitoring for grants and cooperative agreements:

Tracking recipient progress in achieving the desired outcomes.

Ensuring the adequacy of recipient systems that underlie and generate data reports.

Creating an environment that fosters integrity in program performance and results.

Monitoring may also include the following activities deemed necessary to monitor the award:

Ensuring that work plans are feasible based on the budget and consistent with the intent of the award.

Ensuring that recipients are performing at a sufficient level to achieve outcomes within the stated timeframes.

Working with recipients on revising the work plan based on achievement of outcomes, evaluation results and changing budgets.

Monitoring performance measures (both programmatic and financial) to assure satisfactory performance levels. The findings from the performance measure will be used to identify areas of program improvement for broader technical assistance needs.

Monitoring implementation of Data Access and Management Plans.

f. CDC Program Support to Recipients (THIS SECTION APPLIES ONLY TO

COOPERATIVE AGREEMENTS)

CDC staff will be substantially involved beyond site visits and regular performance and financial monitoring during the project period of this cooperative agreement. Substantial involvement means that recipient can expect federal programmatic partnership in carrying out the efforts under the award. The CDC program will work in partnership with the recipient to ensure the success of the cooperative agreement by:

Supporting recipient in implementing cooperative agreement requirements and advancing program activities to meet outcomes.

Providing technical assistance to revise annual work plans and budgets.

Collaboration on enhancing and expanding outcomes surveillance activities, including the collection, management, analysis, and dissemination of EHDI data.

Collaborating with recipient to develop and implement strategies and evaluation plans and use evaluation findings.

Providing technical assistance to define and operationalize performance measures and implement…