AMENDMENT 0003 - 75FCMC25RJ018_QA.pdf

Amendment 0003 -05/30/2025 Research Measurement Assessment Design Analysis (RMADA) 3 Questions and Responses

Category Question

No.

Section & Title

Page Number

Question CMS Response

GENERAL 1 Attachment J.5 - Background Information for Section C/Statement of Work

1 The text in Attachment J.5 indicates "Provided below are examples of types of work that may be performed at the task order level and the skillset necessary to complete the tasks: " ....Capacity to conduct qualitative and quantitative analyses, including complex multivariate analysis, using software packages such as SAS and STATA." The text in Section C however indicates the contractor will use " Python, R, Scala, and SQL". QUESTION: Will the Government please confirm whether or not SAS and/or Stata are potentially relevant to this IDIQ?

Yes, SAS and STATA are still potentially relavent. See SOW Section C.2 for further clarification.

GENERAL 2 C.2.ii. Design of Materials 10 In this section, one of the types of deliverable formats is described as "mobile-friendly".

Could CMS provide additional clarification regarding the "mobile-friendly" deliverables, including the target audiences and the uses of these 'mobile-friendly' materials? Similarly, could CMS elaborated on the target audiences and objectives of using podcasts and social media?

The Offeror will need to demonstrate their capability in performing this requirement.

GENERAL 3 Section C.3.ii.1.16: Develop, implement, and manage stakeholder engagement, model recruitment, and model launch

22 Could CMS elaborate and provide additional information and definition to “model footprint analyses" within the context of learning system activities?

The Learning and Diffusion Group conducts model footprint analyses using data sources including Medicare claims to measure factors contributing to market climate nationally, at regional, state and metropolitan levels. These factors may include state-level (such as legislation and policy climate), health system level, practice or provider level, and beneficiary level analyses of the model footprint.

GENERAL 4 Section C.3.ii.2.3.a. 26 Could CMS please clarify and confirm if there are specific expectations, requirements, or limitations on sources of secondary data?

There are no specific expectations, requirements, or limitations for sources of secondary data. At a task order level, the evaluation criteria will vary depending on the model under review, and the choice of secondary data to incorporate will be determined by the contractor's proposed approach.

GENERAL 5 C.3.ii.2.7.b. Primary Data Collection 28 This section states "A plan to determine when and how to collect data concerning the research questions, potential participants, and potential topics for discussion." and "A plan description to address any expected issues identifying populations or gaining consent, topics for discussion, compensation, budget implications, and strengths and limitations of the planned approach and analysis plan." Could CMS please provide any specific requirements, guidance, or expectations regarding this plan?

The Offeror will need to demonstrate their capability in performing this requirement.

GENERAL 6 H.4 CMS Information Security 69 Please confirm that CMS will be providing government furnished equipment for contractors/subcontractors to provide services for Federal information/Data and thus, FISMA/FedRamp requirements will not apply.

No, the government will not be providing GFE; CMS will provide access and use of data and systems as addressed in the solicitation.

Any equipment provided by the vendor shall be FISMA/FedRamp compliant.

GENERAL 7 C.2 Information Technology Systems, Data Aggregation Requirements C.2 Information Technology Systems, Data Analytics Requirements

7, 9, & 13 Solicitation states that the 3rd Party Data Aggregator will (2) Establish the necessary agreements with external entities to obtain the data and (3) Collect and aggregate data from internal and data sources. The solicitation also states that the Implementation contractor can utilize Model Space to "Add third-party data and link it to CMS data". Can CMS clarify if the Implementation Contractor is able to add new third-party data directly into Model Space (Data Bricks) or can only the 3rd party data aggregator collect data from any external entities which is what is stated on page 9 under Data Aggregation Requirements (3) bullet.

The third-party aggregator will establish agreements with external entities, such as patient registries, for the necessary external data for analytics. The Model Team / Implementation Contractor will provide requirements for third-party aggregation data needs, as well as CMS data and other data not requiring an aggregator. The third-party contractor will make the required data available in Model Space (Databricks).

GENERAL 8 C.2 Information Technology Systems, Data Aggregation Requirements C.2 Information Technology Systems, Data Analytics Requirements

13 Solicitation states under (i) Pre-Implementation Phase, (2) Support Data Sharing and (3) Distribute data files discuss that that the contractor is responsible for data file sharing/transfer and that the distribution process may require extensive resources and IT capabilities from the Contractor. Can CMS clarify the role of the 3rd party database, Model Space and other technologies involved in file transfer, storage and analytics and what the RMADA contractor is responsible for versus the 3rd party Contractor?

Model Space (Databricks) provides Model Implementation Contractors, CMS Evaluation Teams, etc. with an analytics and visualization platform. It supplies claims and other required data to the Model Implementation Contractors.

GENERAL 9 C.2 Information Technology Systems, Data Analytics Requirements (MODEL

SPACE)

7 Solicitation states that it typically takes 2 weeks to set up a new workspace in Model Space. Are there any SLAs or requirements for the 3rd Party Contractor who manages Model Space with regards to turn-around time for requests for a new workspace and of requests such as processing of new data sources into Model Space for any given task order/model needs?

There are no specific SLA requirements for the third-party contractor who manages Model Space.

GENERAL 10 C.2.i - Data

L.10.1.A.v -Implementation > Monitoring Activities

6 & 169 Section C states "The Contractor shall not develop new IT systems such as data collection websites and online participant dashboards." Can CMS confirm that the scope of the Implementation Contractor is to provide business requirements to with the ISP who will develop dashboards using eDFR or another existing tool? If yes, are there SLAs that govern timeliness?

The timelines will be determined at the task order level.

GENERAL 11 iii Model Characteristics 11 Solicitation requires approaches which must also account for the need to gather, coordinate, and analyze private payers and other data sources, as appropriate using Model Space. Section 6, page 18 includes the requirement for the Contractor to create/maintain processes for collecting performance data; obtain appropriate claims, surveys, and clinical data for measurement; conduct necessary training for participants; and use existing IT platforms and software as appropriate. However page 26 states that the Contractor will work primarily in the Virtual Resource Data Center (VRDC). Can CMS clarify the distinction between the Contractor working primarily in the VRDC versus Model Space and the distinction between the two in terms of functionality and use?

See updated SOW section C.2.i.

Virtual Research Data Center (VRDC) and Model Space (Databricks) are mutually exclusive. VRDC is a SAS-based, secure cloud-based data access system that CMS provides for researchers to analyze Medicare and Medicaid data.

The purpose of Model Space (Databricks) is to provide Model Implementation Contractors, CMS Evalutation Teams, etc. with an analytics and visualization platform. Model Space (Databricks) will provide claims and other required data to the Model Implementation Contractor.

GENERAL 12 i Pre-Implementation Phase, (6) Conduct Routine and Ad-Hoc Program Monitoring Activities (b)

14 (6) Conduct Routine and Ad-Hoc Program Monitoring Activities (b) references that the contractor will provide technical assistance including help desk responses to participants.

Can CMS clarify which help desk this is and which systems are required for tracking help desk tickets and responses and if that is the RMADA contractor or another CMS contractor who runs the help desk? Similarly on page 15 under (1) Implementation Phase (1) Provide orientation and ongoing technical assistance to model participants there is a requirement for the contractor to establish an inquiry system/process and help desk to respond to inquiries. Is the RMADA contractor to leverage a CMS system and technologies for the help desk function, or another CMS contractor, or shall the RMADA offeror propose a separate help desk for the entire RMADA IDIQ or will that be task order specific and separate?

Typically, the RMADA contractor does not develop or maintain a separate help desk system. Instead the RMADA contractor works collaboratively with the existing CMMI help desk. The CMMI help desk serves as the first point of contact and then escalates specific technical questions or issues to the appropriate Model team or RMADA contractor for assistance. In certain cases task orders may require the RMADA contractor to establish a dedicated help desk for a specific model. These requirements will be defined within the specific task order SOW.

GENERAL 13 i Pre-Implementation Phase, (6) Conduct Routine and Ad-Hoc Program Monitoring Activities (b)

18 Solicitation (12-Develop a viable process for responding to beneficiary/provider inquiries) includes an interface with 1-800-Medicare, states etc. to monitor beneficiary concerns relating to the administration of models including developing scripts etc. Can CMS clarify the loop between the RMADA contractor and 1-800-Medicare and if the contractor will have access to NGD or 1-800-Medicare operations to know what concerns have come in and how we should plan to coordinate with CCO to provide scripts and gather information on concerns that come in regarding the model via 1-800-Medicare.

Model team staff and their RMADA contractors work collaboratively with the CMS staff overseeing 1-800 Medicare to provide scripts and training to the 1-800 operators, usually when a new model is launching. A communication plan is established between the two groups to ensure that concerns are addressed promptly and revisions to scripts are made as needed.

GENERAL 14 5. Providing enrollment support to CMS and model participants

16 Regarding the requirement for the contractor to prepare and maintain current lists of model participants, considering potential overlaps with other programs- can CMS clarify if all the participant rosters are available in the MDM of other GFE provided the Contractor for comparison across models and if the RMADA contractor will have access to all the rosters within the MDM or Model Space?

The RMADA contractor is responsible for maintaining a roster of model participants and uploading that to MDM or another CMS system. Overlap analyis responsibilities will vary by model and therefore will be specified in individual task orders. If overlap analysis is identified as a requirement, the contractor will have access to the applicable rosters to complete the task.

GENERAL 15 6. Develop and model-specific and cross model quality strategy

18 Solicitation states that information regarding nuances of Medicare claims data is available through Research Data Assistance Center, Resdac: http://www.resdac.org/. Can CMS clarify which type of entity/RESDAC entity the contractor is considered (e.g., Qualified Entity) for registration?

The Contractor should select the entity type that matches their organization's status or reach out directly to ResDAC for guidance.

GENERAL 16 6. Develop and model-specific and cross model quality strategy

18 Solicitation states that information regarding nuances of Medicare claims data is available through Research Data Assistance Center, Resdac: http://www.resdac.org/. Does CMS require that the contractor be registered through the Qualified Entity Certification Program (QECP) prior to proposal submission?

No, the offeror is not required to be registered with ResDac as a qualified entity (QE) prior to proposal submission.

GENERAL 17 6. Develop and model-specific and cross model quality strategy

18 Solicitation states that information regarding nuances of Medicare claims data is available through Research Data Assistance Center, Resdac: http://www.resdac.org/. Can CMS provide instruction on how the Contractor should propose costs associated with the VRDC fees for research data requests in the IDIQ or is that to be priced into the individual model task orders in the future?

VRDC fees should be priced at the task order level.

GENERAL 18 3. Learning, Improvement, and Diffusion, Sections 2. and 3.

32-33 We understand knowledge management and collaboration capabilities (Sections 2 & 3) are provided via Connect, but can CMS clarify whether 1) a separate contractor provisions dedicated space within Connect to a given model for the Implementation and Learning System contractors to use based on requirements we provide? Or, 2) is it in scope for the Learning System contractor to staff the engineers responsible for provisioning a space dedicated to the model(s) we are supporting?

The Learning contractor has a dedicated space for each of the models. The setup and maintenance processes are simple and straightforward. There is additional support through the Connect contractor.

GENERAL 19 iii. Scale & Close Out, DUA Closure 39 States that CMS does not accept electronic/digital signatures on DUA forms. Can CMS clarify if the Contractor will utilize another CMS contractor for physical mail/mailroom for all DUAs (initiation and closure) with participants or should the Contractor propose a separate mail fulfillment function outside of CMS operations.

The SOW section has been updated.

Contractors should process DUA closures in the Enterprise Privacy Policy Engine (EPPE) by following the Requester (Contractor) Training titled “Closing a Contractor DUA.” However, if closing a DUA is the only remaining DUA action and there is no other DUAs in EPPE, a completed CMS DUA Certificate of Disposition (COD form) can be sent to DataUseAgreement@cms.hhs.gov with the appropriate CMS contact(s) copied on the email. The CMS DUA Team will receive the form and close the DUA for the Contractor.

For questions about digitally signing the COD form contact DataUseAgreements@cms.hhs.gov or refer to https://www.cms.gov/data-research/files-for-order/data-disclosures-and-data-use-agreements-duas.

GENERAL 20 SECTION C –

DESCRIPTION/SPECIFICATIONS/

STATEMENT OF WORK - C.2

Information Technology Systems / Data

6 This section mentioned that "Analytical code includes but is not limited to Python and SQL programs". Can CMS clarify if there be any legacy SAS system available?

There will not be any legacy SAS or STATA systems available. See SOW Section C.2 for further clarification.

GENERAL 21 SECTION C –

DESCRIPTION/SPECIFICATIONS/

STATEMENT OF WORK - C.2

Information Technology Systems / Data

6 Will there be limitation on how many people on the implementation team gets access to the Model Space?

There will be no limitations on how many staff on the implementation team gets access to Model Space.

GENERAL 22 SECTION C –

DESCRIPTION/SPECIFICATIONS/

STATEMENT OF WORK - C.2

Information Technology Systems / Data

6 Can CMS please confirm the Model Space is fully functional at this time? The Model Space is fully functional at this time.

GENERAL 23 SECTION C –

DESCRIPTION/SPECIFICATIONS/

STATEMENT OF WORK - C.2

Information Technology Systems / Data

6 If Model Space is not fully functional at the start of a Task Order award, will CMS provide alternative solutions such as CCW or allow contractor to use their data infrastructure to perform the work?

The Model Space is fully functional at this time.

GENERAL 24 SECTION C –

DESCRIPTION/SPECIFICATIONS/

STATEMENT OF WORK - C.2

Information Technology Systems / Data

6 Can CMS please confirm if Model Space is being used for all CMMI models? Yes. See updates to SOW section C.2.i for further clarification.

GENERAL 25 SECTION C –

DESCRIPTION/SPECIFICATIONS/

STATEMENT OF WORK - C.2

Information Technology Systems / Data

6 What is the process for getting access to Model Space? Will that be the responsibility of the RMADA contractor or MS contractor?

The process for getting access to Model Space will be determined at the task order level.

GENERAL 26 SECTION C –

DESCRIPTION/SPECIFICATIONS/

STATEMENT OF WORK - C.2

Information Technology Systems / Data

6 Can CMS please provide the current name of the Model Space contractor? As of the time of this solicitation Amendment 0003, the current Model Space contractor is Global Alliant.

GENERAL 27 SECTION C –

DESCRIPTION/SPECIFICATIONS/

STATEMENT OF WORK - C.2

Information Technology Systems / Data Analytics Requirements

7, 8 "Implementation contractor shall use Model Space to complete the data analysis." and "CMS will supply direct access to CCW or IDR-C if the Implementation Contractor has an exceptional need and provides a justification to the COR that Model Space is not a suitable solution."

Does CMS require contractors to use CCW for non-implementation contracts? If so, will CMS provide CCW licenses to the contractor?

CMS does not require contractors to use CCW for non-implementation contracts. CMS will not provide CCW licenses. All CMMI analytic users, including Model Implementation Contractors, should leverage CCW data and/or IDR-C data via Model Space (Databricks) and not through direct access to CCW or IDR.

GENERAL 28 SECTION C –

DESCRIPTION/SPECIFICATIONS/

STATEMENT OF WORK - C.2

Information Technology Systems / Data Aggregation Requirements

9 How does the 3rd party contractor's role relate to the Data Aggregation Contractor (DAC) and Innovation Support Platform (ISP) Contractor?

The third-party aggregator will establish agreements with external entities, such as patient registries, for the necessary external data for analytics based on requirements provided by the Model Team / Implementation Contractor. The ISP contractor will be responsible for technical integration between the external entity and Model Space (Databricks) and data transfers. The third-party aggregator will then link and aggregate data for use by Model Team / Implementation Contractor.

GENERAL 29 SECTION C –

DESCRIPTION/SPECIFICATIONS/

STATEMENT OF WORK - C.2

Information Technology Systems / Data Aggregation Requirements

9 What specific responsibilities does the Implementation Contractor have in the external data pipeline compared to the Data Aggregation Contractor?

The Implementation Contractor provides the third-party data aggregation contractor with data requirements from external sources (e.g., patient registries) . The Implementation Contractor also validates the transferred and aggregated data.

GENERAL 30 SECTION C –

DESCRIPTION/SPECIFICATIONS/

STATEMENT OF WORK - C.2

Information Technology Systems / Data Aggregation Requirements

9 Will data distribution occur via existing CMS IT systems, the Implementation Contractor, or other CMMI contractors?

Data distribution/availability will occur per Model Team / Implementation Contractor requirements (e.g., Implementation Contractor needs the data for analysis, Model Team requires the data for participant feedback reports (eDFR)). ISP will provide the technical support to make the data available per requirements.

GENERAL 31 SECTION C –

DESCRIPTION/SPECIFICATIONS/

STATEMENT OF WORK - C.2

Information Technology Systems / Data Aggregation Requirements

9 What are the ISP Contractor's specific responsibilities for data acquisition and distribution with external and internal partners?

ISP will establish technical integration with the required external data source and transfer the data to Model Space (Databricks) for third party data aggregator contractor to aggregate. ISP will then make aggregated data available to the Model Team / Implementation Contractor , ISP, etc. as required.

GENERAL 32 SECTION C –

DESCRIPTION/SPECIFICATIONS/

STATEMENT OF WORK - C.2

Information Technology Systems / Data Aggregation Requirements

9 How are responsibilities clearly delineated between the Implementation Contractor, DAC, and ISP contractor to prevent duplication?

Responsibilities will be clearly delineated at the task order level.

GENERAL 33 SECTION C –

DESCRIPTION/SPECIFICATIONS/

STATEMENT OF WORK - C.2

Information Technology Systems / Data Aggregation Requirements

9 "The contractor shall obtain Federal, state, and private healthcare sector data as needed or required". On page 9, CMS states, data aggregation contractor will provide the data. Please confirm that this data aggregation contractor will be responsible for the acquisition and distribution of data payloads such as, but not limited to, all APCD claims and USCDI data.

Are we to assume any payloads from external entities such as other federal, state, and private healthcare sections are going to be acquired and distribute via data aggregation contractor?

Section C.3.i.2 of the SOW has been updated to clarify the Contractor shall identify this data as needed or required.

GENERAL 34 SECTION C –

STATEMENT OF WORK - C.2

Information Technology Systems / Data Aggregation Requirements

9 Could CMS please clarify the specific responsibilities and boundaries between the Implementation Contractor and the Data Aggregation Contractor regarding data acquisition, processing, and distribution? We seek to understand the Implementation Contractor's precise role in the data pipeline, including any responsibilities for data preparation, validation, or integration that complement rather than duplicate the Data Aggregation Contractor's functions.

While the thrid party data aggregation contractor is responsible for drafting and managing complex data-sharing agreements, agreements with other federal and state agencies may not require such rigor and may fall outside the aggregator’s responsibility. There should not be duplication of effort between the Implementation Contractor and the Data Aggregation Contractor's functions.

GENERAL 35 SECTION C –

DESCRIPTION/SPECIFICATIONS/

STATEMENT OF WORK - C.2.vi Information Technology Systems/Engagement

12 Does CMS expect the RMADA contractor to recruit external experts for consultation, or will CMS recruit them to engage with the RMADA contractor?

The Offeror will need to demonstrate their capability in performing this requirement.

GENERAL 36 SECTION C –

DESCRIPTION/SPECIFICATIONS/

STATEMENT OF WORK - C.2

Information Technology Systems/Data (Federal, state, health care sector)

12 Would CMS please provide detailed information about the Innovation Support Platform Contractor's scope of work regarding data exchange with both external and internal partners? Specifically, we need clarification on this contractor's responsibilities for developing and maintaining data exchange protocols, managing API access for innovation partners, and how these responsibilities align with or differ from the data management functions performed by the Data Aggregation Contractor

There should not be duplication of effort between any of the contractors.

GENERAL 37 SECTION C –

DESCRIPTION/SPECIFICATIONS/

STATEMENT OF WORK - C.3.

Requirements / During the Implementation Phase

15 Per the requirement: "Where applicable, the Contractor shall provide technical assistance for the design of Awardees’ self-evaluations."

Could CMS provide more information on parameters for self evaluation of the awardees?

The Offeror will need to demonstrate their capability in performing this requirement.

GENERAL 38 SECTION C –

DESCRIPTION/SPECIFICATIONS/

STATEMENT OF WORK - C.3

Requirements/During the Implementation Phase

15 Would the inquiry system and help desk of the RMADA contractor need to be integrated with an existing CMS help desk? If that is the case, will the existing CMS help desk will be the first tier that triages inquires in their system.

Typically, the RMADA contractor does not develop or maintain a separate help desk system. Instead the RMADA contractor works collaboratively with the existing CMMI help desk. The CMMI help desk serves as the first point of contact and then escalates specific technical questions or issues to the appropriate Model team or RMADA contractor for assistance. In certain cases task orders may require the RMADA contractor to establish a dedicated help desk for a specific model. These requirements will be defined within the specific task order SOW.

GENERAL 39 SECTION C –

DESCRIPTION/SPECIFICATIONS/

STATEMENT OF WORK - C.3

Requirements/During the Implementation Phase / 6. Develop a model-specific and cross-model quality strategy

18 On Page 18, SOW states "The Contractor shall develop a limited number of new measures that fill identified gap areas in performance measurement for alternative payment models and re- specify existing standards to meet model needs". Can CMS also provide information on the type of measures to be developed such as claims based, eCQM, PROMs etc.?

The Offeror will need to demonstrate their capability in performing this requirement.

GENERAL 40 SECTION C –

DESCRIPTION/SPECIFICATIONS/

STATEMENT OF WORK - C.3

Requirements / 1. During the Implementation Phase / 8. Provide quality improvement reviews

19 SOW states "The quality improvement review may be done annually for each model and include a center-wide report that compares all models against preset strategic objectives."

Can CMS please specify if this task will be done by implementation contractor/ monitoring/compliance contractor or a different contractor?

The quality improvement review would be performed by the Implementation Contractor.

GENERAL 41 SECTION C –

DESCRIPTION/SPECIFICATIONS/

STATEMENT OF WORK - C.3

Requirements /1. During the Implementation Phase / 8. Provide quality improvement reviews

19 SOW states "The quality improvement review may be done annually for each model and include a center-wide report that compares all models against preset strategic objectives.".

The requirements asks for a "center-wide report", can CMS please confirm if this will be a Center wide contract performed by a single contractor or model specific?

This could be Center wide or model specific and will be determined at the task order level.

GENERAL 42 SECTION C –

STATEMENT OF WORK - C.3

Requirements / 2. During the Evaluation Phase / 12. Production of analytic files and documentation for records retention

29 Can CMS specify its preferred methods for transferring evaluation data to CMS? For now, CMS prefers data be transferred via the VRDC archive procedure until its planned sunset per Section C.2.i.7. After that, the preferred method for transferring evaluation data to CMS will be Model Space.

RFP/ PROPOSAL INSTRUCTIONS 43 Information and Technology Systems C.2.i

5-7 CMS notes that the "Contractor shall execute a Joint Operating Agreement (JOA) with CMS-designated Contractors when necessary." Will RMADA 3 contractors be required to execute a JOA with the Data Aggregation / Model-Space third-party Contractor?

Yes, the RMADA 3 contractors will be required to execute a JOA with the Data Aggregation/Model Space third-party contractor.

RFP/ PROPOSAL INSTRUCTIONS 44 Information and Technology Systems C.2.i

6 Throughout this section, reference is made to the “Implementation Contractor.” This seems to refer to the “Contractor” referenced in the rest of the document; is this correct?

See updated SOW Section C.2.i for clarification.

RFP/ PROPOSAL INSTRUCTIONS 45 C.2.i 6 The “third-party contractor” referenced in this section appears to be responsible for linkage of Medicare claims, assessments, Medicaid encounters, CAHPS surveys, and so on

– duplicating the efforts of the CCW contractor. Is this correct and who is the contracting organization?

There should be no duplication of effort. For models or other entites in CMMI who need access to Medicare claims, assessments, Medicaid encounters, etc. they should be using Model Space (Databricks) and/or eDFR. Both applications fall under the ISP contract which is currently awarded to Global Alliant.

RFP/ PROPOSAL INSTRUCTIONS 46 C.2.i 6 The described Model Space connection with the CCW is “coming later.” Does CMS have a sense of when that might be?

Model Space (Databricks) connectivity will be provided upon request by the Model Team or Implementation Contractor if IDR-C data does not meet their data requirements.

RFP/ PROPOSAL INSTRUCTIONS 47 C.2.i 6 The SOW seems to draw a distinction between analysis of a data set and creation of that data set, with the latter function assigned to the “third-party contractor” based on specifications drawn up by the “Implementation Contractor.” That is, there is no role for the Implementation Contractor in the creation of the analytic file. Is this correct? We note that this arrangement is likely to slow significantly any exploratory/pre-implementation work under the IDIQ.

For models or other entites in CMMI who need access to Medicare claims, assessments, Medicaid encounters, etc. they should be using Model Space (Databricks) and/or eDFR. Both applications fall under the ISP contract which is currently awarded to Global Alliant.

RFP/ PROPOSAL INSTRUCTIONS 48 C.2.i 8 CMS notes that "Model Space makes Python, R, Scala, and SQL available to the Implementation Contractor for analysis." Does this mean that SAS will not be supported in the Model Space or that acquiring and maintaining SAS licenses will be the responsibility of the IC?

Acquiring or maintaing licenses will be determined at the task order level.

RFP/ PROPOSAL INSTRUCTIONS 49 C.2.ii 9 Can CMS explain in more detail the reasoning behind the statement that “the contractor shall be able to staff multiple larger-scale CMS models simultaneously adequately” in the context of a multiple-award IDIQ contract? It would seem that this capacity would be assessed on a task-by-task basis

The Offeror will need to demonstrate their capability in performing this requirement.

RFP/ PROPOSAL INSTRUCTIONS 50 C.2.v 12 The SOW describes a need for familiarity with “claims available through the CMS …VRDC,” but previous sections require analysis in the Model Space. Can CMS reconcile these two requirements?

The SOW has been restructured and updated. See section C.3.i.2 for clarification.

RFP/ PROPOSAL INSTRUCTIONS 51 C.2.v 12 The SOW states that “Contractors shall be knowledgeable and required to work in CMS environments, including the VRDC, Integrated Data Repository (IDR), and the mainframe.”

This seems inconsistent with section C.2.i above

The SOW has been updated.

VOLUME I: IDIQ TECHNICAL

PROPOSAL

52 C.2.i Data Analytic Requirements Page 8 of

It states that "The Implementation Contractor shall use Model Space to complete calculations and analysis required by the Contract". Will the Evaluation Contractor use the CCW VRDC for analysis?

See Section C.2.i in the updated Statement of Work. The Evaluation Contractor will use the CCW VRDC until the transition to Model Space.

VOLUME I: IDIQ TECHNICAL

PROPOSAL

53 C.3 Requirements Page 34 of 187

In RFP Section C.3, the section titled, "Performance Requirements, Coordination, and Reporting," is unnumbered. Can CMS clarify whether this section (along with the following three (3) sub-sections 1. Strategy Planning, 2. Coordination with CMS and other Contractors, 3. Reporting) falls under RFP section, "3. Learning, Improvement, and Diffusion" or if this should be listed as Section 4 in the RFP?

The SOW has been restructured. See Section C.3.iii iv, and v for updates.

VOLUME I: IDIQ TECHNICAL

PROPOSAL

54 C.3.ii.2.13.b 30 Since the hours and percent time committed would depend upon individual task orders, what should offerors provide for the "breakdown of hours and percent time for those committed to work under the RMADA 3 vehicle?"

The SOW has been updated and this particular reference has been removed.

VOLUME I: IDIQ TECHNICAL

PROPOSAL

55 C3 Requirements, 3. Learning, Improvement and Diffusion, Page 32

35-36 In the SOW task for Learning, Improvement and Diffusion and under "Performance Requirements, Coordination, and Reporting" there are reporting requirements for the following : 1. Monthly Invoice Report; 2. Quarterly Progress Reports; 3. Annual Reports.

Are these project management reports to be completed only for Learning, Improvement and Diffusion task activities or are they to be completed for ALL contract activities performed on a task order? If the latter, its not clear why these work statements are included within the Learning, Improvement and Diffusion task.

Note the SOW has been restructured. See Section C.3.iii iv, and v for updates. Reporting will be determined based on the task order SOW.

VOLUME I: IDIQ TECHNICAL

PROPOSAL

56 C.3 Requirements; ii.2.13 Personnel 30 In the description of evaluation tasks, the RFP instructs the offeror to provide "…information for all identified vital personnel…" and to "provide a breakdown of hours and percent time for those persons committed to work on the RMADA 3 vehicle." This requirement is not referenced in Section L. Instructions. Given that this is an IDIQ without a sample task order and thus no level of effort information is needed for proposed staff, will CMS consider deleting this requirement from Section C?

The SOW has been updated and this particular reference has been removed.

VOLUME I: IDIQ TECHNICAL

PROPOSAL

C.2.i - Data

6 The RFP in item 3 states "CMS' Technical Review Architecture (TRA) includes a Data Stewardship and Governance policy that all CMS data remain with CMS authorization boundaries (i.e., data should remain in CMS)." QUESTION: Given CMS' interest in primary data collection and patient perspectives, would CMS identify the tools for qualitative data analysis (e.g., NVivo, Atlas.ti) that are available within the CMS authorization boundaries? If such analysis tools are not available within the CMS authorization boundaries, can CMS confirm that contractors may continue to gather, store, and/or analyze primary data (e.g., interviews, focus groups, surveys/questionnaires) in secure environments outside CMS?

No. See updated SOW Section C.2.i for further clarification.

VOLUME I: IDIQ TECHNICAL

PROPOSAL

58 C.2.v - Data (Federal, state, health care sector) C.3.ii.2.3.b - Secondary data collection

12, 26 The RFP says, "Contractors shall be knowledgeable and required to work in CMS environments, including the VRDC, Integrated Data Repository (IDR), and the mainframe."

This seems to contradict the requirement to perform data analysis in the Model Space as specified the Data Analytics Requirements on pages 7-9, unless there is an exceptional need with justification to the COR if the Model Space is not suitable. QUESTION: Will the Government please confirm that contractors will be permitted to use Model Space, VRDC, IDRC and/or the Mainframe for analytic work, depending on the requirements of a specific

TORP?

See updated SOW Section C.2.i for further clarification.

VOLUME I: IDIQ TECHNICAL

PROPOSAL

59 L10.1.B.iv. Cross-Model Considerations

170 By "evaluating shadow bundles data provision", does the government mean to evaluate whether providing such data support access to high-quality, integrated care or to evaluate a demonstration model using shadow bundles data?

The Offeror will need to demonstrate their capability in performing this requirement.

VOLUME I: IDIQ TECHNICAL

PROPOSAL

60 Section C.3.ii.2.13. Personnel 30 In the Evaluation section of the Statement of Work, CMS requests that the contractor provides "information for all identified vital personnel..." and "...a breakdown of hours and percent time for those persons committed to work under the RMADA 3 vehicle.” Given the difficulty with committing staff to work under a contract vehicle in the absence of any task orders, and noting that this paragraph was removed from the RMADA 2 SOW in an amendment, would CMS consider similarly removing the requirement from this RMADA 3

SOW?

The SOW has been updated and this particular reference has been removed.

VOLUME I: IDIQ TECHNICAL

PROPOSAL

61 Section C.3.ii.3.Performance Requirements, Coordination, and Reporting.-- item 3.Reporting; Sub-item 3b. "Reports".

35 Can CMS clarify whether the reports described in this section (i.e., monthly invoice report, quarterly progress reports, annual reports, final report, and ad hoc reports) only apply to Learning, Improvement, and Diffusion activities, or do they also apply to Implementation or Evaluation contracts?

Note the SOW has been restructured. See Section C.3.iii iv, and v for updates. Reporting will be determined based on the task order SOW.

VOLUME I: IDIQ TECHNICAL

PROPOSAL

62 Section C.3.ii.3.Performance Requirements, Coordination, and Reporting.-- item 3.Reporting

35-37 The phrase "Practice Network" is used three times in this section, first with respect to the "Practice Network Contractor" (p. 35) , then referencing "Practice Network results" (p. 36), and then "Practice Network Activities" (p. 37). "Practice Network" is not mentioed elsewhere in the RFP. Can CMS please clarify: (1) Is the "Practice Network Contractor" the Learning/Diffusion Contractor for a given CMMI model? And (2) do "Practice Network results"and "Practice Network activities" refer to the results and activities of the Learning contractor's Learning Network and other support activities for model participants?

1. Yes, the Practice Network Contractor is the Learning/Diffusion Contractor for a given CMMI model.

2. Yes, "Practice Network results"and "Practice Network activities" refers to the results and activities of the Learning Contractor's Learning Network and other support activities for model participants.

VOLUME I: IDIQ TECHNICAL

PROPOSAL

63 C.2 Information Technology Systems 6 Under i. Data #3, the RFP mentions CMS' Technical Review Architecture (TRA). Please clarify if this is different than CMS Technical Reference Architecture (https://www.cms.gov/tra/Home/Home.htm).

The SOW Section C.2.i.3 has been updated to correct the meaning of the TRA acronym.

64 C.2 Information Technology Systems 7 Under Data Analytic Requirements, the RFP states "the Implementation Contractor can write code to analyze the data [in the Model Space]". Please clarify if the Evaluator Contractor can also perform these activities.

See Section C.2.i in the updated SOW. The Evaluation Contractor will use the CCW VRDC until the transition to Model Space.

VOLUME I: IDIQ TECHNICAL

PROPOSAL

65 C.3. Requirements 34 Please confirm that the Performance Requirements, Coordination, and Reporting subsection (that is, p.34-37 up to but not including the paragraph that begins, The Contractor shall follow the Standard Operating Procedures (SOPs) for the scaling and closing-out of CMMI models. ) is a subsection of 3. Learning, Improvement, and Diffusion that starts on p.30, as opposed to a new section such as the Scale and Closeout section that follows.

The SOW has been restructured. See Section C.3.iii iv, and v for updates.

VOLUME I: IDIQ TECHNICAL

PROPOSAL

66 C.3. Requirements 34 Please confirm we do not need to address the performance requirements, coordination, and reporting activities on pages 34-37 in our discussions of pre-implementation, implementation, or evaluation.

The SOW has been restructured. See Section C.3.iii iv, and v for updates.

VOLUME I: IDIQ TECHNICAL

PROPOSAL

67 C.3.i.4 "Provide technical assistance to model sites" paragraph 1

14 Can the government please define "operating sites"? Operating sites are defined as model sites.

VOLUME I: IDIQ TECHNICAL

PROPOSAL

68 C.3.i.4 "Provide technical assistance to model sites" paragraph 1

14 Can the government please confirm that this requirement applies to technical assistance provided to participants following application and selection and prior to implementation?

The RMADA 3 IDIQ Contractor will not assist in application support.

VOLUME I: IDIQ TECHNICAL

PROPOSAL

69 C.3.ii.3 "Performance Requirements, Coordination, and Reporting"

34-37 Please confirm that requirements related to Strategy Planning, Coordination with CMS and other Contractors, Reporting have been correctly included as a requirement for IDIQ RFP responses.

The SOW has been restructured. See Section C.3.iii iv, and v for updates.

VOLUME I: IDIQ TECHNICAL

PROPOSAL

70 C.3.ii.3 "Performance Requirements, Coordination, and Reporting"

34-37 If these requirements are correctly included as a requirement for IDIQ RFP responses, please confirm that these requirements apply only to Learning, Improvement, and Diffusion, and not to other tasks.

The SOW has been restructured. See Section C.3.iii iv, and v for updates.

VOLUME I: IDIQ TECHNICAL

PROPOSAL

71 SOW 24 When registries are used for secondary data, does CMS anticipate that RMADA3 contractors would need to sign a data use agreement with the registry owner to obtain the patient-level data? Does CMS have a role in ensuring that the registry owner will share the data related to key model measures with the relevant model contractor?

This will be determined at the task order level.

VOLUME I: IDIQ TECHNICAL

PROPOSAL

72 Section C.1, "Supporting all aspects of model design and operations (except information technology)" bullet at the bottom of the page

4 On the page 4 in the listed bullets, the first bullet states, "Supporting all aspects of model design and operations (except information technology)." The remainder of the draft statement of work does not include tasks specific to model design. Please include clarification to the extent to which RMADA3 primes will be expected to conduct model design activities.

The SOW has been updated and this requirement has been removed.

VOLUME I: IDIQ TECHNICAL

PROPOSAL

73 Section C.1, "Provide data analysis/ integration support" bullet at the top of the page

5 Can CMS provide more context and/or examples of the task indicated by "provide data analysis/ integration support"? For example, is this "integration support" referencing the same integration requirement as the "data integration" reference on p. 18?

The Offeror will need to demonstrate their capability in performing this requirement.

VOLUME I: IDIQ TECHNICAL

PROPOSAL

74 C.1 Background 5 What are examples of “integration support” the vendor should provide? The Offeror will need to demonstrate their capability in performing this requirement.

VOLUME I: IDIQ TECHNICAL

PROPOSAL

75 C.2.i. Data 6 As the vendor will interface with CMS systems within the CMS boundary, will CMS provide laptops or CMS VDIs to access the environment?

GFE is not anticipated; however, GFI (such as data) is anticipated under the RMADA3.

VOLUME I: IDIQ TECHNICAL

PROPOSAL

76 C.2.i. Data 6 The RFP states that "The Contractor shall use CMS data in CMS environments.” Will CMS provide GFE?

GFE is not anticipated; however, GFI (such as data) is anticipated under the RMADA3.

VOLUME I: IDIQ TECHNICAL

PROPOSAL

77 Section C.1.i. Data Aggregation Requirements heading

9 Will the third-party data aggregation contractor be the signatory on the DUAs with external parties (including state APCDs) or will the respective contractor?

For questions about DUAs, contact DataUseAgreements@cms.hhs.gov or refer to https://www.cms.gov/data-research/files-for-order/data-disclosures-and-data-use-agreements-duas.

VOLUME I: IDIQ TECHNICAL

PROPOSAL

78 Section C.1.i. Data, Data Aggregation Requirements heading and Section C.1.v. Data (Federal, state, health care section)

9 and 12 It appears that data acquisition areas of the draft statement of work conflict. Please provide clarity on the data acquisition requirements of the respective contractors. For example, the following text on Page 9 reads, “The third-party data aggregation contractor will collect, link, and aggregate the data.” The text on Page 12 reads, "The Contractor shall obtain Federal, state, and private healthcare sector data as needed or requested."

This is correct. In most cases the third party contractor will do this work. In the event that the third party contractor is unable to perfrom the task, we will ask the Implementation Contractor or Evaluation Contractor to collect as needed.

VOLUME I: IDIQ TECHNICAL

PROPOSAL

79 Section C.1.iii. Model Characteristics 11 For the "Analytic designs that incorporate regression-based approaches as well as actuarial approaches", does CMS wish to include documentation of actuarial judgment where applicable, or is CMS primarily focused on incorporating actuarial modeling approaches?

The Offeror will need to demonstrate their capability in performing this requirement.

80 C.3. ii. Design , Implement & Evaluate,

3. Reporting (b)1 Monthly Invoice report

13 The RFP states "At the time of monthly invoice submission, the Contractor shall submit an accompanying analysis of invoice costs that, at a minimum, includes expenditures broken down by task and by workstream by month, to-date expenditures by year and total over time, and significant deviations from planned expenses and a description of labor hours expended and associated cost for each deliverable as determined by the COR." Please confirm if the requirements in this section are limited to other than FFP type of awards.

Yes, the SOW has been updated to clarify this is only required for award types other than FFP.

VOLUME I: IDIQ TECHNICAL

PROPOSAL

81 C3. Requirements, Pre-Implementation Phase

13 Please provide detail on existing systems or tools that need to be used for maintaining the master enrollment list and records on participation status, participant attribute data, and performance scores data, if applicable.

The RMADA contractor is responsible for maintaining a roster of model participants and uploading that to MDM or another CMS system.

VOLUME I: IDIQ TECHNICAL

PROPOSAL

82 RFP C.3.i.1.3 13 At the bottom of the page, there is an incomplete sentence that reads, "can be provided to CMS after the model ends for Records Retention purposes." Can the government clarify?

The SOW has been updated to clarify that all documentation and data shall be provided to CMS after the model ends for Records Retention purposes.

VOLUME I: IDIQ TECHNICAL

PROPOSAL

83 C3. Requirements, Design, Implement & Evaluate

16 How frequently will the data need to be synthesized and made available to other CMS contractors? What are the expected formats for data sharing?

Data frequency for synthesization and availability depend on the model type and technical requirements/approach, which will be specified in each task order.

VOLUME I: IDIQ TECHNICAL

PROPOSAL

84 Section C.3, Subsection ii. Design, Implement, & Evaluation, Task 1.

During the Implementation phase, Subtask 6. Develop a model-specific and cross-model quality strategy, bullet referencing "CMMI Quality Innovation Council (QIC)"

17 Would CMS provide clarity on the CMMI Quality Innovation Council? The Innovation Center's recently released Quality Pathway and other publicly released materials on the Innovation Center's quality activities do not include a reference to the Quality Innovation Council.

The Quality Innovation Council is an internal workgroup at CMMI that helps determine the proper measures for a given model.

VOLUME I: IDIQ TECHNICAL

PROPOSAL

85 C.3. iii. Design, Implement, & Evaluation 10. Payment Operations and administrative activities

19-20 Can CMS clarify the roles and responsibilities of the IPC vs the Contractor? Is the Contractor expected to carry out the services listed within this task or use the IPC for these services? Is there a cost to the Contractor for using the IPC?

The Contractor is expected to work with the IPC contractor as necessary. Any cost associated with the IPC contractor would be identified in the task order SOW.

VOLUME I: IDIQ TECHNICAL

PROPOSAL

86 Section C.3, Subsection ii. Design, Implement, & Evaluation, Task 2.

During the Evaluation Phase, Subtask

3. Secondary data collection

26 CMS outlines that it is the evaluation contractor's responsibility to obtain and identify needed data sources. Will CMS provide clarity on if the third-party aggregator will be responsible to obtaining the required necessary data sources or if this is the responsibility of the evaluation contractor?

The Evaluation Contractor is expected to obtain required secondary data and should not rely on the third-party data aggregator.

VOLUME I: IDIQ TECHNICAL

PROPOSAL

87 Section C.3, Subsection ii. Design, Implement, & Evaluation, Task 2.

During the Evaluation Phase, Subtask

3. Secondary data collection

26 Which contractor is responsible for obtaining the data needed to work in the VRDC/CCW/IDR (e.g., evaluation contractor, third-party data aggregator, implementation contractor)?

See updated SOW Section C.3.i.2.

VOLUME I: IDIQ TECHNICAL

PROPOSAL

88 Section C.3, Subsection ii. Design, Implement, & Evaluation, Task 2.

During the Evaluation Phase, Subtask

13. Personnel

30 The evaluation phase includes a task related to personnel. A similar personnel task is not included in "during the implementation phase" or "learning, improvement, and diffusion".

Would CMS consider moving this subtask to the instructions for the staffing section of the

IDIQ?

No, these requirements remain unchanged in the SOW.

VOLUME I: IDIQ TECHNICAL

PROPOSAL

89 Text under Section C.3, Subsection ii.

Design, Implement, & Evaluation, Task

3. Learning, Improvement, and Diffusion, "Performance Requirements, Coordination, and Reporting" heading

34 Will CMS confirm that the sections under this header (i.e., strategy planning, coordination with CMS and other contractors, and reporting)are meant to be included in as part of the "learning, improvement, and diffusion" section?

The SOW has been restructured. See Section C.3.iii iv, and v for updates.

VOLUME I: IDIQ TECHNICAL

PROPOSAL

90 RFP C.3.ii.3 34 Is the text immediately under the heading "Performance Requirements, Coordination, and Reporting" meant to be included as a concluding paragraph in the Learning Improvement, and Diffusion section?

The SOW has been restructured. See Section C.3.iii iv, and v for updates.

VOLUME I: IDIQ TECHNICAL

PROPOSAL

91 C.3. ii. Design , Implement & Evaluate,

3. Reporting…