Appendix 8_Additional Standards Guidelines and Best Practices for Data Submission Management Analytics and Reporting DRAFT 10.4.2023.docx

Appendix 8 Additional Standards, Guidelines and Best Practices for Data Submission, Management, Analytics, and Reporting

Data Submission and Documentation Practices:

For all types of data submissions, to help ensure and improve the quality of data collected and utilized, QIOs shall:

follow all CMS and PMEC provided instructions for data submission to ensure proper data standardization work with CMS and the PMEC to develop standardized approaches to measure aggregation ensure that they comply with all applicable laws, including HIPAA, regarding data used or disclosed and ensure providers are able to comply with these laws Collaborate with the partners and interested parties s improve the usefulness and usability of the data.

Streamline data access and reporting Collaborate with providers to assess their understanding of the data reported and provide ongoing training of partners and interested parties s to ensure proper submission and interpretation of the data reported.

Provide regular support to partners and interested parties s to achieve success in data collection and reporting for improvement in quality of care.

When selecting outcomes to be included in data analysis or when collecting new data, QIN-QIOs must adhere to the following guidelines:

Data are to be selected based on goals of the program and the measures used to achieve those goals Data are selected using methodology and data that are valid, relevant, and appropriate, without favoritism or prejudice, and in a manner intended to produce valid, interpretable, and reproducible results.

Selection should be consistent across time periods unless and until an issue is cited with the current methodology. If such a change is to be presented, the same criteria as above will be applied.

Data are transparent and reproducible. CMS's guidelines call for identification and documentation of data sets used in producing estimates and projections, and for clear descriptions of the methods used.

CMS will require industry standard measures and avoid customization of measures beyond specifications provided by CMS. Any use of non-standard measures, must be approved by the COR.

Data Management and Ethical Practices

When managing, analysing, visualizing, and reporting data, :

QIOs must ensure data are processed and exchanged using the CMS-designated workflows and other deliverables using multiple data formats per CMS request and that data file formats are machine readable.

operational mechanisms for reporting will be incorporated rapidly, flexibly and reliably to report progress on the aims and minimize inefficiency.

data must be transparent and reproducible using clear descriptions of data sources and methods used.

reported data shall be selected based on goals of the program and the measures used to achieve those goals.

data will be selected using methodologies that are valid, relevant, and appropriate, without favoritism or prejudice, and in a manner intended to produce valid, interpretable, and reproducible results. Selection must be consistent across time periods unless and until an issue is cited with the current methodology. If such a change is to be presented, the same criteria as above will be applied.

QIOs must contribute to creation and maintenance of data dictionaries and other documentation to all users within CMS, including standardized definitions for measures, conditions, interventions and outcomes.

QIOs may be asked to create dashboards as specified by CMS within the CCSQ IT environment that can easily be interpreted by executive leadership and provide transparent results for future decision and quality improvement efforts. Flexibility will be added in data reporting to allow for changes in requirements based on the changing/evolving priorities, to be determined by CMS.

QIOs must be able to share information across different contractors to promote best practices.

QIOs must also ensure that:

Analyses are aligned with any relevant CMS-identified data management plan.

Staff involved in producing dashboards and statistical publications are knowledgeable about the content, structure and limitations of the administrative data files and maintain working relations with staff who create, update and maintain these files, and will comply with publication standards and procedures as directed by CMS where applicable.

Where administrative files are linked to surveys for analysis, sound procedures for extracting and linking data from external sources are employed, based on a thorough understanding of the relevant components of the data sources.

Computer programs for sampling, extracting and linking external data files are developed and tested by technically qualified staff.

Samples from administrative files are evaluated to ensure that samples are representative of the underlying administrative data files. Estimates prepared from external data sources are reviewed to ensure that the data extraction and linkage processes were implemented correctly.

Intellectual property rights to all data and data products (e.g., analytic code, reports, associated data dictionaries, etc.) produced under CMS programs is retained by CMS, except where rights are retained by partnering government agencies.

QIOs must also abide by the following Data Validity and Ethical Practices:

Make every effort not to predetermine or influence the results of statistical practices and/or selectively interpret data.

Disclose and manage any conflicts of interest relating to the data sources Take appropriate action to notify CMS when aware of deviations from set guidelines as set forth in the contract.

Conduct statistical practice that is objective and transparent, and not allow organizational objectives or expectations to encourage unethical statistical practice by its employees.

Be transparent about assumptions made in the execution and interpretation of statistical practices, including methods used, limitations, possible sources of error, and algorithmic biases.

QIOs must also make operational improvements as applicable to their work under this contract:

Use librarian and/or informatics principles to help drive standardization of measures, outcomes and conditions but also incorporate a human-centered approach to information and data delivery.

Communicate and document data sources and fitness for use, including data generation and collection processes and known biases.

Communicate and document data processing and transformation procedures, including missing data handling.

Create data visualization tools based on purpose and audience Provide data dictionaries for any source of data produced for CMS