07-05-17_IDIQ_TO-_Q_and_A.pdf

HHSM-500-2017-00060I 7/05/2017

Request for Proposal for IDIQ (HHSM-500-2017-00060I) – National Consensus Development and Strategic Planning for Healthcare Quality Measurement

SOW/SOD

Section Title of Section Question CMS Response

IDIQ Proposal Section C.II.A.6

(p. 15)

SOW, Task Order Domains , Ad Hoc Consensus Projects

Many of the task orders (for example, the three attached to the IDIQ RFP) awarded under this contract are likely to be fixed-price TOs, for which the contractor assumes the risk based on budgeted costs, and does not need to submit incurred costs on an item by item basis with its invoices. Yet this section states that the Contractor must identify costs associated with in-person and web meetings and include them in monthly invoices. Does this requirement apply to all task orders awarded under this IDIQ, or only to those awarded on a time and materials basis?

This language has been removed from Section C.II.A.6. (p.15).

Section L13, a. (p.

120)

Proposal Organization, Page Size, Typing, Spacing Numbering

Can HHS confirm that the specified page limit for the technical proposal of 50 single-sided pages does NOT include either the three task order proposals (Tab 6) or the resumes of key personnel?

The page limit does not include the TO proposals nor the resumes.

Attachment J.5, Small Business Subcontracting Plan

Can HHS confirm when the small business subcontracting plan needs to be submitted—e.g., with the proposal, or upon award?

The small business plan is to be submitted with the proposal.

TO 1, Consensus-based Endorsement and Maintenance of Performance Measures II.A.3. Technical Assistance and

Education Requirements The SOW indicates that the contractor should host “up to two measure developer workshops” on page 2. On page 10, of the SOW, it states that the contractor “shall also conduct up to two measure developer workshops every contract year.” Can HHS provide clarification on the number of measure developer workshops? Does HHS want the contractor to host two measure developer workshops throughout the period of performance, or does HHS want NQF to host up to two measure developer workshops every contract year?

HHS requires 2 workshops per contract year; 10 for the entire 5-year performance period.

II.B. Task 1 Endorsement/Maintenance:

Process

The SOW as currently written states that the contractor should provide measure stewards and developers with advance notice of providing responses to the public comments received prior to the Standing Committee reviewing and voting on the measures. As a part of NQF’s current consensus development process (CDP), NQF does provide measure stewards and developers the opportunity to

This is acceptable if the offeror directly notifies all the stewards/developers via email.

Title of Section Question CMS Response respond to all public comments received following the 30-day public comment period, prior to the Committee deliberation. Additionally, any comments that are submitted prior to the comment period deadline are made available to measure stewards and developers.

Can HHS confirm this approach is still acceptable?

II.B. Task 1 Endorsement/Maintenance:

Process

The SOW as currently written states that the contractor should notify CMS and measure stewards and developers “in writing of the decisions of the board/committee responsible for adjudicating appeals. Voting outcomes will be relayed to measure developers and measure stewards expeditiously so that they have the maximum time to prepare for the appeals process, if needed.” Currently as a part of its consensus development process, NQF notifies the measure stewards and developers of the Consensus Standards Approval Committee’s (CSAC) endorsement decision prior to the start of the 30-day appeals period. Following the close of the appeals period, NQF notifies measure stewards and developers if an appeal was submitted on their measure(s). Can HHS confirm this approach is still acceptable?

This is acceptable if the offeror confirms that they directly notifies, in writing (i.e. via email), the stewards/developers as soon as the appeals period ends (within 3 business days).

II.B. Task 1 Endorsement/Maintenance:

Process

The SOW indicates that the contractor cannot implement changes to the consensus development process (CDP) once the call for measures have opened. Could changes be implemented at the start of each task order contract year, if necessary?

Changes in criteria or processes should not be introduced or implemented to a project after the project has already started. For example, there should be no changes to the endorsement criteria (by the offeror or any project committees) to a project once stewards or developers have submitted a measure to that project based on current guidelines. Changes to the process are acceptable only after they have been communicated to all stakeholders and the offeror has solicited and considered public comments on these changes.

Title of Section Question CMS Response

II.B. Task 1 Measures The SOW indicates that the contractor “shall give priority to all HHS developed measures and measures that are in CMS programs.”

Can HHS provide clarity on this expectation?

CMS requests that HHS measures be given priority when the number of measures submitted to a project for review exceeds the number of measures that the committee for that project has the resources to review. For example, if 20 measures have been submitted for the Renal Project, but the committee can only handle up to 12 measures, then the HHS measures among the 20 measures have the priority to be included in the project.

II.B. Task 1 Conflict of Interest The SOW as currently written indicates that the contractor “shall ensure that internal staff who also work with any measure developers are not involved in any CDP or MAP processes in any capacity, regardless of whether these staff members are financially compensated by the measure developers.” Can HHS provide clarification on this statement? Currently, as the incumbent contractor, NQF staff may serve as a member of the project team for both CDP and MAP given their expertise. Additionally, all contractor staff are involved in process and LEAN improvement activities to enhance our CDP and MAP process.

This refers to the offeror’s employees or subcontractors/consultants who are involved in or working with measure developers to develop measures. These individuals should not be involved in the CDP or MAP due to potential conflict of interest.

II.B. Task 1.1 Quality of Care Projects The SOW as currently written states that the contractor “shall conduct up to 120 CDP projects between the base year and the last option year of this Task Order.” Based on the outputs from the 2017 Kaizen event, NQF as a potential contractor would offer two measure submission opportunities for each topic area each year. Due to the anticipated increase frequency of submissions, NQF has consolidated the 22 measure review topical areas to 15-16 topical areas. As proposed, the total possible number of CDP projects between the base year and the last option year is 160. Does HHS concur with up to 160 CDP projects?

Concurred.

II.B. Task 1.2 Cost and Resource Use Projects

The SOW indicates that the contractor can “convene up to 46 hours of meetings for each project.” However, on page 7 of the SOW, it

Note that the up to 40 hours is for quality care projects and the up to

Title of Section Question CMS Response indicates that NQF can “convene up to 40 hours of meetings for each project.” NQF proposes convening up to 46 hours of meetings to include an increase in web meetings for each project in order to reduce the number of in-person meetings for each project. Can HHS confirm that NQF can convene up to 46 hours of meetings for each project?

46 hours is for CRU projects.

CMS recognizes that CRU measures usually require a higher level of review efforts.

II.B. Task 1.3 Project Reports The SOW states that each project written report should include the “identification of potential high-priority gap areas for measure developers to consider for future development.” Could the contractor remove the section on gaps in measurement from each project report and instead develop an annual cross-cutting report that would summarize endorsement activities and identify prioritized gaps in measurement across all of the topic areas? Seeing all the gap areas together in a single report may provide more value.

CMS agrees with this approach.

II.B. Task 2.1 Education Materials The SOW indicates that the education materials “may also be made available to the public, pending approval of the IDIQ and Task Order CORs.” As the incumbent, NQF publicizes all education materials for everyone (including non CMS/HHS measure leads and developers) to use as resources for learning about NQF and the endorsement maintenance process. Can HHS clarify why approval to be transparent with the education materials is needed?

CMS is requesting the opportunity to review all the deliverables before they become publicly available. CMS appreciates the past history of making these materials available to the general public. However, we have received feedback from developers that some of these material were not always clear or concise. An example is the CDP—developers are sometimes confused about next steps and what they can expect from each step of the process. Others have indicated that the information is not easily accessible, but rather piecemeal and tend to scatter across various locations of the the offeror’s website. CMS believes that having the COR(s) review and provide feedback on these

Title of Section Question CMS Response materials before public dissemination may help to clarify information for developers and reduce the efforts of the offeror’s staff to answer similar inquiries repeatedly.

II.B. Task 2.1 Education Materials The SOW as currently written states that following the measure developer webinars, the contractor should post all meeting materials on the contractor’s website. Currently, NQF does not post the measure developer webinar meeting materials on the website.

Would HHS be amenable to the contractor sending the meeting materials to measure developers electronically instead of posting then? This approach has worked for the incumbent contractor.

No, CMS sees these webinars as educational and believe that stakeholders would benefit from being able to access and review them whenever they need. Not all developers are able to attend all the webinars when they are available for the first time, and there are new developers who may not yet be aware of these webinars. Making these webinars archived and permanently available for later viewing, and having a dedicated link to these webinars would allow them to be great resources for all interested stakeholders.

II.B. Task 3.1. Kick-off Meeting:

Deliverable #8

The SOD states that HHS will provide feedback on the kick-off meeting materials. Can HHS clarify its proposed timing for providing the contractor with this feedback? Does HHS plan to provide feedback on the materials prior to the kick-off call or during the kick-off call?

CMS will provide feedback on the materials prior to and during the Kick-off meeting/call. Thus, the offeror shall provide kick-off meeting materials at least 48 hours prior to the kick-off meeting date.

II.B. Task 3.2.1. Project Management Design: Deliverable #9

The SOD states that HHS will provide feedback on the project management design. Can HHS clarify the timing in which the contractor would receive the feedback? Does HHS plan to provide feedback on the materials prior to the kick-off call or during the kick-off call?

CMS will provide feedback prior to and during the Kick-off as needed.

Title of Section Question CMS Response

II.B. Task 3.2.2. Risk Mitigation Plan:

Deliverable #10

The SOD states that HHS will provide feedback on the risk mitigation plan. Can HHS clarify is proposed timing for providing the contractor with this feedback? Does HHS plan to provide feedback on the materials prior to the kick-off call or during the kick-off call?

CMS will provide feedback prior to and during the Kick-off as

II.B. Task 3.4

Weekly Conference Calls

The SOW as currently written states the NQF should have weekly IDIQ and Task Order COR conference calls. Under the current IDIQ, the incumbent contractor meets with the IDIQ and Task Order COR bi-weekly. Does HHS concur with bi-weekly conference calls for this task order?

Weekly is preferred, with opportunity to go to bi-weekly.

II.B. Task 3.5 Standing and Coordinating Committee Meeting Summaries

The SOW states that the contractor should sent a meeting summary for each committee webinar, conference call or in-person meeting in which measures under consideration are discussed. The SOW indicates that the meeting summaries should be submitted within three business days of the webinar or conference call and five business days of the in-person meeting. In order to allow enough time to draft the meeting summary and have it professionally edited, would it be acceptable to submit the meeting summary within seven business days of the committee meeting?

Five (5) business days would be acceptable for all committee meeting types. Please refer to the revised SOD.

II.B. Task 3.6 Quarterly Debriefings on Consensus Development Process (CDP)

The SOW as currently written states that the contractor should have four quarterly conference calls with HHS and their measure developers regarding their experience as it relates to the endorsement maintenance process. Can HHS clarify if this is a separate requirement from the current quarterly contractor/CMS contract in-person meetings? If not, would HHS be amenable to adding this requirement as an agenda item for the contractor/CMS quarterly contract in-person meetings?

No, these are separate meetings with different purposes and attendees.

II.B. Task 3.6 Quarterly Debriefings on Consensus Development Process (CDP): Deliverable #17

The SOD states that the description of deliverable # 17 is “two days before each conference call—meeting minutes.” Can HHS provide clarification on the deliverable description? As currently written, the description seems to reflect a delivery date and not an actual task. Is HHS requiring a meeting summary of the quarterly debriefings with HHS and their measure developers?

Quarterly Debriefings on CDP meeting summaries/minutes are to be submitted NLT 2 days after the meeting.

II.B. Task 3.7 CDP-MAP Integration The SOD states that HHS will provide feedback on the MAP-CDP integration plan. Can HHS clarify the timing in which the contractor

CMS will provide feedback prior to and during the Kick-off as

Title of Section Question CMS Response would receive the feedback? Does HHS plan to provide feedback on the materials prior to the kick-off call or during the kick-off call?

II.B. Task 4 Annual Report Can HHS provide a delivery due date for the outline of the annual report to the IDIQ and Task Order CORs to review?

Fifteen (15) days before the end of the OY, the outline is due to CMS for review and comment.

II.B. Task 4.2 Task Order Final Report Can HHS provide a delivery due date for the outline of the task order final report to the IDIQ and Task Order CORs to review?

Fifteen (15) days before the end of the OY, the outline is due to CMS for review and comment.

TO 3, Multistakeholder Input on the Selection of Quality and Efficiency Measures as part of the Pre-Rulemaking Process under Section 1890A III.A. Tasks To Be Performed:

Implement a consensus-based process to elicit multi-stakeholder input on Measures to be implemented through the federal rule rulemaking process

The SOW as currently written states that the contractor should “submit a plan to CMS to describe how the plan could improve the representation of these stakeholder groups.” Would CMS be amenable to having this as a discussion during one of the Task Order COR/GTL weekly calls?

No. The offeror’s proposal shall describe how they plan to sustain adequate stakeholder representation and increase underrepresented stakeholders thorough the life of the contract.

CMS anticipates receiving a detailed plan within the offeror’s proposal response that describes how this task will be achieved for each stakeholder group (e.g., patients and families, population health organizations, etc).

III.A. Tasks To Be Performed:

Implement a consensus-based process to elicit multi-stakeholder input on Measures to be implemented through the federal rule rulemaking process

The SOW as currently written states that a “MAP nominee cannot be formally or informally affiliated with or have a financial interest in the organization that develops the measure which the individual is assigned to review”. Would the incumbent contractor’s approach of vetting any financial interests of subject matter experts as they sit on the Workgroups/Committee as individuals still be acceptable?

No. All parties named in the SOW shall disclose all known as well as perceived financial interests. CMS anticipates receiving a detailed plan within the offeror’s proposal response.

III.A. Tasks To Be Performed:

Implement a consensus-based process to elicit multi-stakeholder input on Measures to be implemented through the

NQF’s current MAP Conflict of Interest Policy requires subject matter experts to undergo a stringent COI analysis. NQF does not require this analysis for organizational members because these representatives sit at the table with a particular point of view and, as CMS mentions in the SOW, MAP is required to include a broad range of stakeholders that have an interest in, or are affected by, the

Quorum per se may not be sufficient to withstand public scrutiny if conflict of interest is not fully disclosed. The Contractor should require MAP Workgroups and Coordinating

Title of Section Question CMS Response federal rule rulemaking process use of measures. Applying the principle that every MAP member must recuse if it has any interest in the measure being considered appears to contradict the required composition of MAP and would likely result in losing quorum. Is it acceptable for NQF to apply the COI analysis stated to subject-matter experts only to preserve the valuable multi-stakeholder input of the MAP Workgroups/Committees?

Committee members as well as other commenters to disclose any affiliation with developer of a competing measure that is also on the MUC list and is competing for inclusion in the same program.

III.A. Tasks To Be Performed:

Implement a consensus-based process to elicit multi-stakeholder input on Measures to be implemented through the federal rule rulemaking process

The SOW as currently written states the use of “professional, experienced moderators… and highly trained”. Could you please clarify the requirement for moderators to be professional and experienced? Could you please clarify what is meant by being highly trained?

This referred to professional, experienced, highly trained moderators with formalized certification, accreditation and/or job related work moderation experience in a similar setting.

III.A. Tasks To Be Performed:

Implement a consensus-based process to elicit multi-stakeholder input on Measures to be implemented through the federal rule rulemaking process

Can CMS confirm that they will identify the measures to be reviewed through the feedback loop process?

The measures to be reviewed via the feedback loop process will be identified by CMS.

III.A. Tasks To Be Performed:

Implement a consensus-based process to elicit multi-stakeholder input on Measures to be implemented through the federal rule rulemaking process

Given the timing of the implementation of the MAP Rural Health Task Force, would a phased approach be acceptable to integrate members into MAP workgroup pre-rulemaking activities?

Yes, a phased approach would be acceptable to CMS.

III.B.

Tasks To Be Performed:

Meetings: Deliverable #1

In previous years, the incumbent contractor has hosted up to two all- MAP membership web meetings for upcoming pre-rulemaking activities. Could CMS please clarify the intent of a one-day in-person planning meetings for stakeholders and Workgroup Committee members?

It is for situations like CMS bringing PAC/LTC measure updates to the MAP. It is informational rather than for further review.

Title of Section Question CMS Response

Meetings: Deliverable #4

The SOD as currently written states four two-day in-person meetings. In previous years, the incumbent contractor has only convened the MAP Clinician, Hospital, Post-Acute and Long-Term Care Workgroups. Is there an expectation to also convene the Dual Eligible Workgroup for an in-person meeting under this Task Order?

CMS will revise the requirement to read three (3) two-day in-person meetings.

Meetings

During the previous task order, the Coordinating Committee held a one two-day in-person meeting in January. The current SOW specifies two one-day in-person meetings. Given the potential increase in travel and meeting costs to hold two one-day meetings, could one two-day meeting can meet CMS’ requirements?

Yes, two (2) one-day in person meetings is the same as one two-day in-person meetings.

Can CMS provide clarification on the purpose of the six conference calls stated in the SOW? Is the intent to utilize these calls as preparatory meetings with the MAP co-chairs? If so, would CMS be amenable to a proposal of adding nine conference calls (for a total of

15) for this purpose?

We want at least one webinar in Spring to run changes like those for the decision categories by the MAP WGs and CC before the upcoming MAP cycle.

In previous years, there have been delays with receiving the MUC list far in advance prior to the Workgroup in-person meetings. Given the short time period to conduct the in-person meetings, it would be difficult to submit the preliminary analysis for each candidate measure to CMS fifteen (15) business days prior to each workgroup in-person meeting. Would it be sufficient to update CMS verbally on the preliminary analyses of any measure(s) intended to be controversial during the required COR/GTL briefing conference call ahead of the in-person Workgroup meetings?

No. CMS requires written preliminary analysis for each candidate measure from the offeror (7) days prior to the December workgroup meetings.

CMS will edit the preliminary analysis accordingly and return to the offeror for distribution to the Workgroup committee members.

Meetings: Deliverable #9

Can CMS please clarify the expectation for the publication of the measures under consideration (MUC) list measure specifications?

Historically, the MUC list has been published in the format it is received from CMS, given time constraints within the MUC clearance process and the desire to share the MUC list with the public as quickly as possible. Due to timing of its release, any necessary updates are incorporated into the final meeting materials.

Would this approach still be acceptable?

Concurred.

Meetings: Deliverable #12

Can CMS please clarify their expectation to submit a list of potentially controversial measures as a formal contractual deliverable? Would it be sufficient to update CMS verbally on

No. The offeror shall furnish CMS with a written report of any known or anticipated barriers or

Title of Section Question CMS Response controversial measures as identified by NQF during the required GTL/ COR briefing conference call ahead of the in-person Workgroup meetings?

problems that may arise during the course of the in-person meetings. The report shall be delivered 7 days prior to each workgroup meeting.

Meetings: Deliverable #12

Can CMS please clarify their expectations for tracking negative coverage of measures in the press and on social media? Would it be sufficient to update CMS verbally on any identified coverage during the required GTL/ COR briefing conference call ahead of the in-person Workgroup meetings?

CMS requires written notification on negative coverage on any measures on the MUC list or Federal quality reporting or value-based purchasing programs discussed in the MAP meetings, as indicated in the SOW.

The SOW as currently written states that the Contractor should “review previous final MAP reports, lessons learned and programmatic objectives to identify topics to review with CMS staff”. Could HHS please provide clarity on the expectations to identify topics and a summary for each topic area? Would an agenda outlining the topics for discussion be acceptable?

In preparation for all meetings, the offeror’s due diligence is required to identify potential topics for discussion with CMS.

MAP reports, lessons learned, etc.

are sample topic areas to explore when developing meeting agenda’s.

Meetings: Deliverable #16

Can CMS clarify the expectation for a communication plan and its contents as a formal contractual deliverable?

The purpose of creating and implementing a communication plan strategy is to increase the attendance of non-CMS measure stewards at the December MAP meetings.

Meetings: Deliverable #16

Can CMS please clarify how deliverables #18 and #19 (measure specifications on the MUC list and preliminary analyses) should be used as discussion materials for deliverable #16?

Providing the measure specifications as well as the information in the preliminary analysis and giving the non-CMS measure stewards an opportunity to red-line the documents will improve the accuracy of the data for MAP committee members.

III.D. Tasks To Be Performed:

Written Reports

The SOW indicates that the Contractor should submit the draft recommendations to CMS for review and feedback. As a part of the

No. The Contractor is required to submit draft reports to CMS for

Title of Section Question CMS Response incumbent’s standard commenting period, the contractor posts the recommendations to the NQF website for a public commenting period. During this time, members of the public may submit comments. Following the commenting period, the contractor adjudicates all submitted comments prior to finalizing the recommendations. Members of the federal government are encouraged to submit feedback and comments this period as an equally important stakeholder in Task Order supported efforts.

Given this process, is CMS’ review of the draft recommendations as part of the public comment period an acceptable means of providing feedback?

review and comments prior to public commenting periods.

Specifically, the Contractor is required to incorporate CMS comments and edits before publicly releasing a public commenting version. CMS has the legal expertise to ensure the accuracy of the information related to statutes and programs.

The Contractor can propose due dates by which CMS send comments and edits in time for them to be incorporated into a public commenting version.

III.D. Tasks To Be Performed:

Written Reports

Would CMS prefer that the PAC/LTC and Hospitals Reports are combined or submitted as separate documents? (See deliverables #21, #22, and #24).

The PAC/LTC Programs Report and the Hospital Programs Report should be two separate documents. The Hospital Report is deliverable # 21 in the SOD;

the PAC/LTC Report deliverable # 22. The Report on Strategic Guidance for Hospital and PAC/LTC Programs is deliverable # 24.

III.D. Tasks To Be Performed:

Written Reports

In previous years, the incumbent contractor has developed four written reports. Three reports discussed multi-stakeholder input and strategic guidance on measures challenges for hospital, clinician and post-acute/long-term care programs. Additionally, the incumbent has developed a cross-cutting report outlining strategic challenges as identified by the Coordinating Committee across all workgroups.

Can HHS confirm this cross-cutting report is no longer a formal deliverable ?

Confirmed. CMS has already sponsored elective task orders to support several well-focused HHS cross-cutting efforts. Another cross-cutting report may increase resource burden without providing added-value.

III.F. Tasks To Be Performed:

Meetings

Given the lessons learned from each pre-rulemaking process, NQF intends to update guidebook and educational materials on the pre-rulemaking process on an annual basis. Is this approach sufficient?

Using lessons learned as one mechanism to update guidebook and educational materials is

Title of Section Question CMS Response appropriate. The offeror is welcome to propose additional mechanisms fulfill the requirement.

III.F. Tasks To Be Performed:

Meetings

Is the ‘Certificate of Completion’ for workgroup members that review educational materials and voting procedures considered a formal deliverable? If so, what is the expectation of this deliverable?

Yes, a reporting output of workgroup member’s certificate of completion assessment shall be included.

III.J.

Deliverable # 36

Can CMS please provide more information regarding the desired format and content of the Risk Assessment Report (Deliverable #36)?

The Contractor is required to 1) identify potential risks; 1) analyze the root cause of each risk; 3) discuss in details the pros and cons of various options in addressing each risk and recommending the best option;

and 4) describe the strategies for monitoring and evaluating the actions for addressing each risk.

For # 2, the Contractor shall indicate whether the risk could impact schedule, costs, or quality.

The Contractor shall also rate the likelihood of each risk (e.g. high, medium, low). For # 3, the Contractor shall provide the rationale for selecting the recommended option. The Contractor is welcome to propose the format for the Risk Assessment Report.