Final_Q A_HHSM-500-2016-RFP-0037_1.pdf

No. Document Name Page Section Question/Comment/Suggestion Response

General FedBizOpps states that “This acquisition is being conducted under the procedures in accordance with FAR 13. Simplified Acquisition Methods.”. Can the Government please confirm that the size of the procurement for this effort, for all base and option periods, is under the Simplified Acquisition threshold of $150,000?

Subpart 13.5 – Simplified Procedures for Certain Commercial Items(a) This subpart authorizes the use of simplified procedures for the acquisition of supplies and services in amounts greater than the simplified acquisition threshold but not exceeding $7 million ($13 million for acquisitions as described in 13.500(c)), including options, if the contracting officer reasonably expects, based on the nature of the supplies or services sought, and on market research, that offers will include only commercial items. Therefore the size of this procurement will be above the Simplified Acquisition threshold of $150K.

General The title of this procurement as stated on FedBizOpps is “Agile development for Merit-based Incentive Payment System (MIPS)” and the project title as stated in the Statement of Objectives is “Performance Category Calculation Architecture with Merit-Based Incentives Payment System”. Please clarify the title for this procurement.

The objective of the contract is to utilize the Agile development process to establish the architecture for MIPS but the actual title for project is "Performance Category Calculation Architecture with Merit- Based Incentives Payment System"

General Which program office within CMS does this effort support? This effort is being sponsored by the Center for Clinical Standars and

Quality

General The RFP states that this engagement is to create the framework to wrap around the background analytics, and that a web user interface (UI) is not part of this engagement. Can CMS explain the need for submitting a Section 508 PAT?

508 is applicable any Federal EIT contract that develops, procures, maintains, or uses information and communications technology so would apply in this case.

General The RFP cites FAR Part 12 - Commercial Items. Is the Government looking for a commercial off-the-shelf (COTS) product?

The Government is procuring agile services under this solicitation.

General The RFP cites FAR Part 13 - Simplified Acquisitions. Does this mean that the COTS product and all the Agile Development for the Merit-Based Incentives Payment System (MIPS) is under the simplified acquisition threshold

Please see the response to question one (1) above.

General Can the government clarify the definition of "iteration" for pricing? We understand this to be a single sprint for code delivery. Is this correct, or does it also include any time to promote the code through multiple environments? Yes, this is a single sprint for code delivery.

General Will the government be providing an extension for responding to this solicitation? Answers to vendor questions may have a material impact to our responses, requiring content and pricing adjustments accordingly.

The proposal due is extended t Wednesday, August 18, 2016

General It appears that the effort has a twelve month period of performance consisting of a base of 6 months, starting on the award date and option of 6 months. Please confirm.

This is correct.

10 General Can CMS provide an estimated start date for pricing purposes? Monday, September 12, 2016

General FedBizOps Notice – Per the notice, offeror understands that this is a “combined synopsis/solicitation for commercial items prepared in accordance with the format in Subpart 12.6, as supplemented with additional information included in this notice”. : Does CMS expect offerors to use GSA Schedule pricing to develop their T&M labor rates since the GSA Schedule represents commercial pricing?

Offerors should use the appropriate tools to provide their T&M pricing.

This is not a GSA procurement.

General FedBizOps Notice - Per the notice, offeror understands that “this acquisition is being conducted under the procedures in accordance with FAR 13. Simplified Acquisition Methods”.

Q: Does the maximum contract value found in FAR 13.5, Simplified Procedures for Certain Commercial Items apply to this opportunity?

Yes, Subpart 13.5 – Simplified Procedures for Certain Commercial Items(a) This subpart authorizes the use of simplified procedures for the acquisition of supplies and services in amounts greater than the simplified acquisition threshold but not exceeding $7 million ($13 million for acquisitions as described in 13.500(c)), including options, if the contracting officer reasonably expects, based on the nature of the supplies or services sought, and on market research, that offers will include only commercial items.

Instructions to Offerors 1,7

Technical Assumptions, Conditions, or Exceptions

Please confirm that "Technical Assumptions, Conditions, or Exceptions" topic is a not part of Factor 3 Notional QCP and it should be a separate section with no page limit Correct. The technical assumptions, conditions or exceptions is not part of Factor 3 Notitonal QCP. Document corrected to remove.

Instructions to Offerors Can the Government clarify what it is expecting from Offerors in Section I Technical Proposal Factor 6 – Section 508 Product Accessibility Template?

Offerors should provide information supporting compliance with 508

MIPS/APM Questions HHSM-500-2016-RFP-0037

Instructions to Offerors What is the difference in scope of work between labor services provided in CLIN 0001 and CLIN 0002? (Instructions to Offerors Pg 4, Section II) Looks like the instructions table don't align with what we had discussed on the 29th of July. I've included revised tables in the email with this attachment.

Instructions to Offerors CLIN 0002 states that it is part of the Base Period of 6 months, and specifies 3 iterations with a variable iteration length. Would these iterations occur after the 6 month base period CLIN 0001 (i.e., March 2017 and onward), or within the first 6 months of the Base Period (i.e., September 2016 - March 2017)? (Instructions to Offers Pg 4) The vendor should propose what they determine to be the most suitable entire iteration structure for each CLIN.

Instructions to Offerors

Instructions to Offerors Can the Government confirm that for CLIN 0001, it expects the Offeror to state its expected team size, labor categories, and estimated hours per iteration in order to quote a price per iteration? Correct.

Instructions to Offerors CLIN 1001 suggests 6 iterations of two weeks per iteration (total of 12 weeks, or 3 months), but the heading suggests a option period length of 6 months. Can the government clarify the length of CLIN 1001? (Instructions to Offers Pg 5)

Please seeresponses to number 13 and number 16. The period of performance for the base period is 6 months.

Instructions to Offerors Can the Government provide the number of FTEs or LOE for the incumbent effort? There is no incubment; vendors should propose using their determination of the scope of work the appropriate LOE/FTE

Instructions to Offerors Given the range of 2 to 5 days per week for travel, is it sufficient to choose a sample number of days of travel as an assumption, or does CMS have an anticipated number of days per week of travel we can use for our travel cost estimates? (Statement of Objective Pg 11, 5.0)

The government does not anticipate over $3,000 for travel if any at all as contractors will not be compensated for travel within a 50 mile radius and in accordance with the JTR/FTR.

Instructions to Offerors Can the Government clarify if cover letters are allowed, and if so, whether they contribute to the page length of Submission Volumes I or II?

Cover pages are allowed and is limited to two pages. The two pages do not count toward the the volume 1 page limit.

Instructions to Offerors Are resumes needed at the time of submission? If so, are there any special requirements and where should they be included in the proposal package? There are no key personnel requirements for this acquisition.

However, Offerors can name key personnel if applicable to their technical solution. No resumes are needed at the time of submission.

Instructions to Offerors Are key personnel needed to be named for the contract in the proposal? If so, are there any special requirements and where should they be included in the proposal package?

There are no key personnel requirements for this acquisition.

However, Offerors can name key personnel if applicable to their technical solution.

Instructions to Offerors Can headers and footers be within the 1” top and bottom margins? Per the soliciaiton the submision Formatting states Page size 8.5 by

11 inches with 1 inch margins.

Instructions to Offerors Can we submit our proposal (including both Technical and Cost/Price) in PDF format? PDF format is accepted for the technical submision. However the the

Cost/Price should be in excel.

Instructions to Offerors The PWS has a maximum page length of 5 pages. Can the government define the areas they would like the PWS to address in these 5 pages? Is there an outline or template available? The contractor must be able to address all objectives within the solicitation. Please allow for no more than 15 pages for the PWS.

Instructions to Offerors Section 508 - Since CMS will be developing the GUI; what will this apply to? Will the System performance or Admin/management interface have a GUI written by the successful proposer?

508 is applicable any Federal EIT contract that develops, procures, maintains, or uses information and communications technology so would apply in this case.

Instructions to Offerors Potential OCI Does CMS have any concerns about organizational conflicts of interest if the contractor is supporting task orders and the ADELE-QRI BPA while also performing work for this procurement? No

Statement of Objective The primary deliverable for this solicitation is a backend architectural API. In addition, per the solicitation, this contract does not include building a front-end, graphical user interface for submitting data and sample input data will be either provided by the government or generated by the vendor. Given those parameters can the government define the services required in sections 1.4 and 1.5 or confirm that there will be no services for these sections of the solicitation.

Section 1.4 and 1.5 are general agile pratices. In an agile delivery environment, requirements will change rapidly according to sprint planning. Needs cannot be finalized in a waterfall fashion before the contract begins.

Statement of Objective

1.5 Address the whole

experience, from start to finish

There is a reference to a "Government Roles" section in SOO 1.5. Please provide the location of that Government Roles section.

The reference will be deleted in the SOO at contract award.

32 Statement of Objective 12 5.0 Place of

Performance Please clarify whether CMS providing a colocation space for this work as GFE?

Contractors should assume CMS will not be providing a colocation space for this work at this time. CMS is actively seeking space for this effort.

33 Statement of Objective 2 Factor 4 Past

Performance

Item #2 states "all" relevant contracts/orders, which could be a sizeable number for some companies. Please confirm that providing 3 of the most relevant non-CMS contracts / orders is desirable, and not all.

Each Offeror needs to determine which relevant contract or orders they want to provide. There is not minimum or maxium limit.

Statement of Objective 9 4.5 Collaboration

Environment Please confirm that the government will provide as GFE licensing for Confluence, JIRA and HipChat collaboration tools stated in SOO Section 4.5. The Government plans to give contractor access to these tools.

Statement of Objective 10 4.6 Transition Activities Can CMS please clarify which CLIN is used to provide the cost for the Transition Activities defined in SOO Section 4.6.

Transition should be included in the applicable iteration team and the corresponding CLIN

Statement of Objective The Statement of Objectives skips over item 2.2 between pages 2 and 3. Please confirm that this is intentional.

2.3 should be corrected to 2.2 and also correct 2.4 to 2.3. The SOO will corrected at contract award.

Statement of Objective Page 8 of the SOO states “Because CMS’s direction may change in an agile environment, CMS seeks to purchase development capacity for goals similar to those stated above.” Can the Government please explain and provide clarifications for this statement?

In an agile delivery environment, requirements will change rapidly according to sprint planning. Needs cannot be finalized in a waterfall fashion before the contract begins.

Statement of Objective Please provide confirmation on whether there will be workstations available for the contractors at CMS’ “co-loaction space”, per page 11 of the SOO. Please see number 24

Statement of Objective Section 2.4 talks about monitoring tools; however, Section 3.0 references Amazon Web Services (AWS) as the infrastructure platform. Are there monitoring tools CMS is already using within AWS for current Quality Programs?

The application team is still responsible for setting up and integrating monitoring tools such as Splunk and New Relic, which are currently monitored by a central infrastructure security team by CMS.

Statement of Objective Are there any requirements for data from one category to be used in the calculation of the score for another category, or are all Merit-Based Incentives Payment System (MIPS) categories isolated at the category score level?

All categories are isolated from each other according to the proposed rule, although this may change.

Statement of Objective Is there an expectation to use any of the existing HCQIS Software (COTS or custom vendor developed) or Hardware? No.

Statement of Objective Are the Quality Reporting Document Architecture (QRDA-3), qualified clinical data registry (QCDR), and Registry XML specifications the only specifications for data submission?

The specifications may change as conditions evolve, but we are pushing towards a small number (or a single) unified XML specification.

Statement of Objective How does the Quality Payment Program (QPP) currently provide feedback to the user (is there an application, tool, or service that will be provided)? There is no current feedback provided to users.

Statement of Objective The Quality, ACI, and CPIA data is all coming from the XML files submitted, but where does the Resource Use (RU) data come from? Is it an existing system?

Resource use data comes from claims analysis based on CMS claims warehouses.

Statement of Objective Please confirm that bidders do not need to include pricing for the cloud infrastructure (SOO, p.4), but should price any other software or infrastructure ODCs Yes, bidders do not need to include pricing for the cloud infrastructure.

46 Statement of Objective Will monitoring and feedback need a GUI for users? No, monitoring and feeback will not need a GUI for users.

Statement of Objective The SOO, Appendix, references the system containing SBU/PII. Will the system also need to handle and process PHI (e.g., claims-based data for resource use)? Yes

Statement of Objective Can the government provide any additional details on the data format, type, or size (e.g., structured, delimited, fixed-width data) for each of the for input sources listed in the SOO?

Data formats will vary throughout the development cycle.

Statement of Objective How will access and authorization work? Will there be a single sign-on to integrate with or is this application responsible for user management and authorization? Section 4.2 Page 6 and 8.

Identity and authentication is expected to be handled by a CMS system that exposes a RESTful API for identity and authentication services.

Statement of Objective Section 4.2 Page 8 - "The system is expected to store data on behalf of at least 1.2 million clinicians." What is the estimated total annual data submission size in GB? What is the expected Peak data submission size in GB?

As the total disk usage is not available at this time (Spread across multiple CMS systems), With the exception of claims based measures data submitted will be at an aggregated level. This greatly reduces the file size and storage requirements when compared to the PQRS program.

Statement of Objective Administrative users (section 4.2, page 8) - Are they intending this to be a web UI? Restful API? Command line utilities? What type of users are these administrators? Business operations? Technical software/datacenter staff?

Initially we expect this to be an exposed restful API that would eventually have a front-end built on top of it. It would be operated by business operators.

Statement of Objective Network configuration - How will requirements or authorization be different for batch data loaded from internal CMS datacenters or private clouds versus data coming from the public Internet?

CMS uses Enterprise File Transfer for internal data movement, which has the ability to expose an "Internet Server" for access via SFTP.

Statement of Objective Section 4.2 page 5 states "In this Contract, CMS seeks to build a proof-of-concept API and platform that serves to integrate measure calculation and results across these four categories." - Is this contract limited to proof-of-concept, is the application built going to be used in production or will that be a separate solicitation?

This is a proof of concept and the application should be built as if going to production. CMS will make a decision about the appropriateness about using the system for production use.

Statement of Objective Will CMS award this proof-of-concept to multiple vendors and then select the best application for production. If so, how many awards does CMS plan to make? No, this is awarded to one vendor.

Statement of Objective Section 4.2 - "The CMS/CCSQ/ISG seeks the skills to build software for some or all of the following needs. " Is the contract going to be for some or all? If for some, then please identify the specific needs.

In an agile delivery environment, requirements will change rapidly according to sprint planning. Needs cannot be finalized in a waterfall fashion before the contract begins.

Statement of Objective Section 4.2 - When does CMS need to start providing the performance measures based on CPIA, ACI, PQRS and RU? Is the completion date of this contract based on when CMS needs to provide these performance measures? If so, then what is the desired completion date?

The period of performance is six months with a six month option.

Statement of Objective Section 4.2 - Can the government please provide a rough estimate of the “maximum expected volume from real users”?

For the PQRS program alone, the peak month in 2015 had 2 million files submitted by 4000 users. We expect submissions to increase dramatically.

Statement of Objective Section 4.2 In regard to QRDA and other complex data submission specifications, states that these specifications “can be used”. Does this mean that the contractor could alternatively base their pricing on the assumed use of new, simpler non-standard formats?

The specifications may change as conditions evolve, but we are pushing towards a small number (or a single) unified XML specification.

Statement of Objective Section 4.2, states “As possible, develop a working scoring engine”. For pricing purposes, can the expression “as possible” be interpreted to mean that the working scoring engine is not a requirement?

We intend to purchase agile development capacity, where priorities are determined every iteration based on sprint planning.

Statement of Objective Will the vendor need to provide development, testing and production environments for this system? Yes.

Statement of Objective Statement of Objectives, page 4 – The Statement of Objectives says that “The Contractor shall support multiple CMS environments and other contractor environments, including sandbox environments, as directed by CMS. CMS environments may include, but are not limited to, the following: development, test, implementation (staging/pre-prod environment) and production”. Q. Can CMS provide detailed information regarding what other CMS and Contractor environments may need to be supported under this project? Q. Will the contractor be required to supply additional hardware, software, testing for these environments? If so, how should the additional hardware, software and testing for these environments be priced?

This development will follow a DevOps model. Part of the effort may include the building of automated test scripts using established QPP tools.

Statement of Objective Statement of Objectives, Section 4.2, Objectives page 6. The Statement of Objectives states that “Resource Use requires claims-based data as an input”. Q: Does CMS already process claims data for costs per beneficiary, by provider for other programs such as the Shared Savings Program? Q: If so, then can the service created under this contract access and encapsulate that output to reduce redundant processing? Q: If not, for pricing purposes, can CMS provide an estimate of claims volumes that need to be processed to address Resource Use requirements?

CMS currently uses our Integrated Data Repository for internal claim analytics (Teradata). Alternative solutions for claim analytics may be evaluated.

Statement of Objective Can they government supply a sample calculation for scoring that best identifies the complexity of the algorithms needed? A sample calculation is not available to be provided at this time.

Statement of Objective Will users, when querying their already submitted data, or check status or giving feedback on submissions, use their own or another interface to the software developed in this effort, or is the successful submitter expected to have a GUI for those purposes only to coexist with the

CMS GUI?

The APIs built would be hosted by the same application, and could thus be consumed by another single GUI application.

Statement of Objective Can the government specify the length of the period of performance for providers/users identified in the SOO? Jan 1 to Dec 31 (With current 90 Claims rollout)

Statement of Objective What is the total throughput or connections anticipated for the final implementation? For the PQRS program alone, the peak month in 2015 had 2 million files submitted by 4000 users. We expect submissions to increase dramatically.

Statement of Objective Can the government confirm that the project of 6-month base plus 6-month option period is for the POC, or is the option anticipated to include Pilot or Production implementation of some functionality?

Integration with existing UI or development of Administrative UI may be required.

Statement of Objective Statement of Objectives, section 2.3, page 3

References U.S. Web Design standards implying there is an UI. However ,this is in conflict with page 6 bullet: “this contract does not include building a front-end, graphical user interface for submitting data.” Can CMS clarify what is needed? There is no UI planned for this procurement.

Statement of Objective Statement of Objectives, section 4.4, page 9

Bullet 3 mentions “supporting streaming measure calculation to calculate Quality Payment Program”.

Based on the size and frequency of data submitted, CMS expects the vendor to determine the best practice for expedient processing of the data -- as close to real-time as possible.

Statement of Objective Can CMS clarify which part of the processing requires streaming: Based on the size and frequency of data submitted, CMS expects the vendor to determine the best practice for expedient processing of the data -- as close to real-time as possible. In an agile delivery environment, requirements will change rapidly according to sprint planning.

Statement of Objective Files based Incoming submission data via users for single provider Development of data submission solutions TBD as part of award. API or HTTPS file transmission standard

Statement of Objective File based Incoming submission from third party (e.g. registries) that contains measures data for multiple providers? If yes, can these be received by SFTP or does the submission has to be streamed while the file is getting transmitted?.

Development of data submission solutions TBD as part of award. API or HTTPS file transmission standard.

Statement of Objective After receiving measures data does the category calculation have to be streamed and processed in memory?

Real time or Near Real time feedback on submitted data will be a requirement/goal.

Statement of Objective After the category score calculation, does the CPS have to be processed in stream and in memory

CPS should be determined in Real time or Near Real time as submission/data is received. This is not a onetime process but updated with each new data entry.

Statement of Objective After the CPS calculation, does the Threshold CPS calculation (that depends on submission from other providers than the one in question) need to be streamed and processed in memory? Threshold is determined before submissions

Statement of Objective The Statement of Objectives, page 5, 4.2 Objectives, Functional goals, paragraph 2 reads, “… CMS seeks to build a proof-of-concept API and platform to serve to integrate measure calculation and results across these four categories.” Should this read, “… CMS seeks to build a proof-of-concept API and platform to serve to integrate and measure calculations and results across these four categories.”

"Measure calculation" refers to the process of calculation an individual's score on a measure.

Statement of Objective 13 10.0 Product Owner

Please clarify whether CMS is providing a Product Owner, or is the Contractor providing the Product Owner/Project Manager/Federal Solution Architect role described in SOO section 10.0? CMS will have a product owner.

Statement of Objective

14-15 12.0 APPENDIX.

Technical Overview Please confirm that the government will provide as GFE licensing for the items listed in the HCQIS Hardware/Software Reference Guide table See response to questions number 43 and 47.

Terms and conditions 5; B. Block 19, 20, 21, 22, 23 and 24;

The CLIN definition provided in the Terms and Condition on page 5, and the Instructions to Offerors on pages 4 and 5 do not match. Please correct the discrepancies.

Please see the updated Instructions to Offerors. The terms and conditions will be corrrect with the contract award.

Terms and conditions

5;

B. Block 19, 20, 21, 22, 23 and 24; and Section II Cost and Price Proposal

The Section II - Cost and Price Proposal instructions on page 4 state that for Time and Material CLINs to Provide all labor categories, hours, rates and supporting documentation and rational for labor categories, rates and hours; but the CLIN Description for CLIN 0001 says it is Time and Material, but follows the structure defined in the instructions for Firm Fixed Price CLINs (Iteration period, price per Iteration, Number of Iterations, number of teams).

Please clarify what is requested for Time and Material CLINs such as CLIN 0001.

This will be a Hybrid contact whereas only CLIN 0001 and CLIN 1002 will be T&M and CLIN 0002 and CLIN 1001 Firm Fixed Price.

Terms and conditions

5;

B. Block 19, 20, 21, 22, 23 and 24; and Section II Cost and Price Proposal

Please Clarify the period of performance and the number of iterations requested for CLIN 0002. The Instructions state 6 months, but the Number of Iterations is 3.

The Offerors should provide the number of iterations for the the CLINS. Please see the updated instructions to Offerors. The terms and conditions will be correct with the contract award.

Terms and conditions For clarification, is the work in the Option Period intentionally split between CLIN 1001, Agile Services and CLIN 1002, Incidental Products? If so, does this mean that during the Option Period there are no “additional agile services capacity in accordance with the Performance Work Statement (PWS)” as defined in CLIN 0002? Correct

Terms and conditions The time period for CLIN 0002 is "TBD" in Terms and Conditions page 15, B but is defined as 6 months in Instructions to Offerors page 4. Can the Government please clarify what the length is for CLIN 0002? (Terms and Conditions Pg 15, B) period of performance for CLIN 0002 is 6 months; the date of award is TBD but anticipated date of award is before 9/16/16.

Terms and conditions The Government has incorporated the Service Contract Labor Standards clause FAR 52.222- 41 into the solicitation; however, no wage determination schedules have been included.

Flexion maintains that the Statement of Work mandates that the services to be performed are deemed to be professional services which under 29 CRF Part 541 meets the exemption and is not covered under the FAR Clause 52.222-41 As a result, we respectfully request that the Government remove the requirement for 52.222-41 or acknowledged that the exemption and confirm that no Wage Determinations will be issued under an awarded contract.

The Government acknowledges that the exemption applies to this requirement and no wage seterminations will be issued under the awarded contract.

Instructions to Offerors Does the Product Roadmap need to show just the iterations that are part of the Base

CLIN0001 and CLIN0003, or should it include optional CLINs CLIN1001 and CLIN1002? Optional CLIN iteration associated with the optional task also.

General Is there a preferred programming language for development, or any open source language/framework would be fine?

The vendor should propose what they determine to be the most suitable based off of their interpretion of the requirment of this solicitation.

Statement of Objective Does the user experience team that the contractor will provide focus on user stories that are specific to interacting with the API, or also the user interface needs even though building the user interface specifically is out of scope for this PWS? Does the user experience team need to develop information architecture and wireframes even though the Graphical portion of the site will be built separately by CMS?

There is no UI currently planned yet in an agile delivery environment, requirements will change rapidly according to sprint planning. Needs cannot be finalized in a waterfall fashion before the contract begins.

Sheet1