36C10X18Q9150-0001001.pdf

36C10X18Q9150 – Access to Care Mystery Shopper Assessment

Questions and Answers

Q1. Will there be multiple awards made for this project?

A1. No, this is a single award. The Prime Contractor is allowed to subcontract as they deem necessary and as long as it is within the terms and conditions of the solicitation.

Q2. Can a small company only bid the people it has in house? 2x Project managers, and the SME (clinical consultant- dr. Mitchell)?

A2. No, the vendor responding to the solicitation must submit their quote as if they are bidding on the requirement in its entirety. The company is allowed and encouraged to network with other potential bidders to give the Government, and themselves, the best outcome possible.

Q3. Is there an incumbent contractor providing Mystery Shopper services to the VA? If so, who are they and what was the award amount on the contract?

A3. No, there is no incumbent. This is a new requirement.

Q4. Are the intercepts only in Washington DC, or are they nationwide?

A4. The majority of the tasks under this PWS shall take place at VA medical facilities selected for this assessment and/or authorized locations utilized by the contractor to enable mystery shopper Veteran participants to place calls nationwide. Assessment planning and preparation will take place at the contractor’s offices. Occasionally, in-person meetings within the metro Washington DC area will be required at the PE program manager’s discretion. These meetings may take place at either contractor or government facilities.

Q5. Since they are at VA centers, does the mystery shopper have to be a veteran? Or can they be civilian?

A5. This requirement calls for all mystery shoppers to be a Veteran that has current access to the VA Health Care system.

Q6. How many mystery shops are being conducted in total? How long should each mystery shop take?

A6. Please refer to section 2. CONTRACT SCOPE, subsection: Phased Assessment for information regarding what is being requested as part of this solicitation. The length of mystery shops will vary based on geographic location, number of Veteran mystery shoppers, and other factors that the contractor may include as part of a proposal.

Q7. Will you be providing the questions and script to follow for the shoppers?

A7. No, the contractor shall develop a detailed evaluation plan that will include detailed patient pathway scenarios, scripts, and anticipated outcomes. However, the program office will collaborate with the contractor and provide subject matter expertise in creating the questions and scripts for this requirement.

Q8. If answers to questions are coming at a later date, “does that mean that the submission date will be extended as well?”

A8. The Government will not be extending the response date. The due time is extended to 3pm, EST, Friday, August 31, 2018.

Q9. On p's. 6-15, Section B - Price/Cost Schedule, the table includes item numbers for each deliverable, however does not include an item number for the actual mystery shopping field work. Where and how would the client like to see the price for performing the mystery shopping field work? Since the exact volume of assessments is undefined, would the client want to see a "per-shop" price?

A9. Additional CLINs will be added to the solicitation to account for the field work.

Q10. On p.21/22, Section D - Travel Requirements, the government estimates the following number of trips shall occur during the performance of this task order: 80 trips, 2 personnel per trip, 5 days per trip. Do these numbers of trips, personnel, and length of trip represent travel required to perform the mystery shops or travel required to deliver the reports and deliverables listed in the RFQ?

A10. Yes, the Travel Requirements (numbers of trips, personnel, and length of trip) are intended to be used for travel required to perform the mystery shops. However, they may be used for delivering reports and presentations based on the concurrence of the program office and the Contracting Officer’s Representative (COR).

Q11. As a follow-on to Question #2, regarding the travel trips, p.21/22, Section D – Travel, does the government believe it would be feasible to identify and contract with a group of Veteran mystery shoppers that both have required healthcare needs in multiple care delivery areas, and are currently enrolled in up to 80 VHA care delivery locations? To ensure this coverage would require an unreasonable and/or unmanageable number of contracted Veterans.

A11.The Government is not familiar with such a group of Veteran mystery shoppers that both have required healthcare needs in multiple care delivery areas, and are currently enrolled in up to 80 VHA care delivery locations. However, as long as the requirements of the solicitation are met, the contractor may propose any innovative approach that they deem feasible.

Q12. As a further follow-on question, ideally the Veteran mystery shoppers would be evaluating their care within the VHA facilities in which they are enrolled. What was the reasoning for allotting 5 days per site visit on p.22 Section D – Travel?

A12. The site visits are intended for the contractor’s program management, subject matter experts, and/or other staff who will assist with the execution of the contract.

Q13. On p.17, Performance Work Statement, Section 2 Scope, Objective #1, the RFQ describes assessing the process for patients with referrals from primary care, it would seem the shopper would have to have an appointment with their primary care physician first to get the desired referral OR is the idea that we would recruit veterans who already have a pending referral?

A13. Either option is feasible, just please keep in mind that Veteran participants in the assessment who attend appointments should only do so if the service is within their normal clinical care.

Q14. On p.20, Phase 1 the client identifies (5) specialties to be included in evaluation (Primary Care, Mental Health, Women’s Health, Physical Therapy, Oncology) – can you provide some idea of how that will expand in Phase 2 and 3 as that directly impacts sampling plan?

A14. Other clinic types may be included based on VHA access initiatives. For example, more clinics/services are being offered to Veterans on a self-referral direct scheduling basis such as audiology.

Q15A. Assuming the Base year includes Phase 1, Phase 2 and Phase 3 as outlined in the RFP, what is the scope of Option Years 1 and 2? Is it 65 centers or more?

Q15B. In the Base Year, are Phase 1, Phase 2 and Phase 3 cumulative? In other words, during Phase 3 are we evaluating the 5 centers from Phase 1, the 30 centers from Phase 2 and the 30 centers from Phase 3?

A15A. Options years 1 and 2 are intended to expand the assessment to additional VA medical facilities that were not part of the base year. However, for trending purposes, the option years may include a sampling of sites from the base year.

A15B. No, the base year sites are not intended to be cumulative (i.e., phase 1, 2 and 3 sites are intended to be different). However, the Government reserves the right to re-sample any site it deems necessary.

Q16. In some cases, if the Veteran “mystery shopper” is scheduling an appointment, but is not intending to attend the appointment, this can negatively impact care delivery for other Veterans. Does the government have a response for this obvious conflict?

A16. Yes, once the appointment is scheduled, and if the service is not medically necessary, or within the Veteran’s normal clinical care, then the expectation is to have the Veteran mystery shopper call back the VA and cancel the appointment so that other Veterans are not impacted.

Q17. Does the government anticipate adding a significant number of deliverables to cover the hours required to perform each site visit, which includes prep work, travel time, onsite presence and post site-visit data aggregation?

A17. The Government will take into evaluate the proposals and make adjustments to the deliverable schedule, if necessary.

Q18. What deliverable is associated with the substantial amount of education and training for the Veteran mystery shoppers, required to prepare each Veteran prior to any onsite experience and/or data collection?

A18. Additional CLINs will be added to the solicitation to account for the field work.

Q19. Within the deliverable descriptions, how do the data collected during the VHA mystery shopper care experiences relate to the business process workflows briefly described?

A19. The data collected should help validate how closely business process workflows are being followed at the VA medical facility.

Q20. On p. 18, Section 2 Contract Scope, the government states: “This engagement requires the contractor to utilize Veterans, who are at a minimum, eligible for VHA medical services, but preferable are receiving health care at the VHA. These Veterans shall follow various scenarios developed by the contractor (in collaboration with the PE program office and any applicable VA subject matter experts) to either schedule appointments, or to attend appointments. Veteran participants in the assessment who attend appointments should only do so if the service is within their normal clinical care”. Can the government please explain how it intends to address potential delays and/or an inability to execute the requisite number of enterprise-wide mystery shops due to a lack of willing (and eligible) veteran participation?

A20. It is understood that the recruitment of Veterans will be challenging. Because of this challenge, the number of sites included in phases 2 and 3 will have to be reviewed prior to the initiation of the respective phases to ensure that the number of sites are realistic.

Q21. On p. 19, Section 2 Contract Scope, Analysis of Findings, the government states:

“Findings are to be presented iteratively at pre-defined intervals, to be identified in coordination with the PE program manager” and on p. 22, Section 3 – Specific Mandatory Tasks and Associated Deliverables, the government states: “Some deliverables are considered to be “living” documents, which shall be created based on a timeline established in the project management plan, and then updated and delivered again towards the end of a phase, when applicable. For example, meeting minutes will be compiled into a single artifact that will continuously be updated throughout the course of the assessment and submitted as a formal deliverable as identified in the project management plan for such living documents, the intent is to maintain a single deliverable over the life of the project rather than having documents recreated for each sub-phase.” Requirements such as these are typically set forth under Time and Materials or Cost Plus agreements. Can the government please explain how vendors are to accurately price such requirements under a Firm Fixed Price Agreement?

A21.Many of these deliverables have multiple quantities associated with them. Please refer to the detailed deliverable descriptions for more information. For example - Deliverable PM.15- Meeting Minutes: This deliverable…is a “Roll Up” deliverable repeated throughout the engagement...The most current version of the document is to be submitted as a deliverable on the last business day of every month.

Q22. On p.21/22, Section D - Travel Requirements, the government estimates the following number of trips shall occur during the performance of this task order: 80 trips, 2 personnel per trip, 5 days per trip. Can the government provide a list of locations for these trips to ensure accurate and comparable pricing, or would the government prefer to issue a lump-sum travel amount?

A22. The contractor should estimate a blended travel amount that includes travel to the northeast, southeast, midwest, southwest, and northwest.

Q23. On p. 22, Section 3 – Specific Mandatory Tasks and Associated Deliverables, Assignment of Contract Phases and Sub-Phases to ACMSA Engagements, the government states: “Each Task Phase will be assigned by the PE PM individually. The contractor shall confirm engagement stakeholders; scope, business value and assessment approach strategy with the PM and receive approval prior to commencing any Task Phase under the contract”. Can the government please clarify the intent of this clause? The black ink interpretation is that all 138 of the listed Deliverable Items will be released in three distinct stages -Phase 1, Phase 2, and Phase 3, and that vendors will subsequently receive payment upon completion of each deliverable.

A23. Within each phase, deliverables will be assigned by the PE PM in coordination with the contractor to ensure that there is a clear understanding of what work is being conducted per the project plan. The intent is to keep the contractor and the PE PM aligned and mission focused.

Q24. Reference page 42, Section 7.0, Organizational Conflict of Interest. The Government includes a restriction on future contractor activity if the contractor “participated in the analysis and recommendation leading to [an] acquisition decision to acquire those services.”

(emphasis added). Participating in analysis and recommendation can be a very broad standard, and seems to be a significant expansion of the FAR 9.5 OCI concern regarding Biased Ground Rules. As written, this RFQ seems to anticipate some level of recommendation and analysis work, meaning this RFQ could be read to significantly constrain future contractor activities. For example, mystery shopping work and subsequent observations or recommendations regarding the necessary level of access may impair a contractor’s ability to pursue future patient access implementation work. Such a broad interpretation of this “participation” standard could be read to implicate a broad number of opportunities, and out of an abundance of caution to avoid this unusually broad OCI description contractors may forego bidding on certain opportunities. In order to assure maximum contractor participation in this RFQ could the government please clarify the anticipated extent of future prohibition on future activity and if it was intended to expand the scope of the well-established Biased Ground Rules OCI of FAR 9.5.

A24. The Government’s intention is not to limit future activities but to only bring to the Mystery Shopper contractor’s attention to future conflict issues. If there are any future concerns of OCI issues, those would be evaluated at that future time.

Q25. Reference page 66, “Note: Evaluations will be completed as a two-part evaluation approach (Phase 1 and Phase 2). If the vendor does not meet the criteria for Phase 1 (Technical Capability and Past Performance), then that vendor will be immediately eliminated from competition. All vendors that receive an acceptable in Phase 1 will then be reviewed for Phase 2 (Veterans Involvement, Price and Subcontracting Plan, if applicable) criteria. If the vendor does not meet the criteria for Phase 2 (Veterans Involvement, Price, and Subcontracting Plan, if applicable), then that vendor will be immediately eliminated from competition.” Should Offerors submit the Subcontracting Plan during the initial proposal submission for evaluation, or submit it after Phase 1’s determination of meeting the criteria;

if not, at what point should it be submitted?

A25. The Offeror shall submit all Proposal Volumes for the initial proposal submission including the Subcontracting Plan, if applicable.

Q26. Reference page 73, the solicitation states: “……The targeted small business subcontracting goals/percentages are based on a percentage of total subcontracted value:”

However, on page 74, it states: “ (d) Dollar value and percentage of the total subcontracted value of subcontractors planned for;” however the chart indicates percentage of total contract value. How does the VA expect the dollars and percentages be calculated, based on total contract value or total subcontracting dollars?

A26. The targeted small business subcontracting goals/percentages should be based on the total subcontracting dollars. The chart language has been revised.

Q27. Section 8.3.b and Section 12.2 of the RFP both address information and media storage requirements; however, the language is not consistent between these two sections. The language in Section 12.2 is consistent with the VA Handbook 6500. Thus, could the government please confirm that the language in Section 12.2 is applicable to this requirement?

A27. Section 12.2 is applicable to this requirement. Section 8.3.b has been revised accordingly.

Q28. The Subcontracting Plan template provided under Section E.2.2.d of the RFP includes small business subcontracting goal percentages. Could the government please confirm if the provided goal percentages are for illustrative purposes only?

A28. The subcontracting goals are to be met, when applicable and to the best of the ability of the prime contractor. The subcontracting plan and goals are only applicable to those vendors that identify as a large business under the solicitation NAICS code.

Q29. Could the government please confirm if the representations and certifications included under Sections E.3 through E.5 should be completed and included in the Offeror’s proposal submission, or if they should be completed after contract award? If they should be included in the Offeror’s proposal response, could you please clarify which proposal Volume should include this information?

A29. Reps and Certs will be confirmed through the System for Award Management (SAM.gov) for the intended awardee prior to award. The awardee must be registered and current in SAM prior to award.

Q30. Under Section 4. Volume IV – Price, the RFP states that “any Contractor-imposed terms and conditions and/or assumptions which deviate from the Government’s material terms and conditions established by the Solicitation, may render the Contractor’s quote Unacceptable”.

However, since this project is a firm-fixed price requirement, it is customary and best practice for Offeror’s to include detailed assumptions off which the proposed pricing is predicated, in order to provide transparency, clarity and a mutual understanding between the parties.

Could the government please confirm these types of assumptions, which do not contradict the terms and conditions of the RFP, but rather clarify specific details, are allowable?

A30. Confirmed. Assumptions, which do not contradict the terms and conditions of the RFP, but rather clarify specific details, are allowable.

Q31. On page 20 under the paragraph for Phase 1, there are comments on assessing a VAMC.

Could the government please confirm if these comments are referring to the parent facility with all of the children facilities combined, or the outpatient clinic portion of the VAMC housed at the location?

A31. Phase 1 includes three to five VA health care systems, each consisting of one main facility [also referred to as a VA Medical Center, or VAMC] and at least one community based outpatient clinic.

Q32. In reference to Deliverable PM.1: Weekly Status Reports (pg. 24), is there a minimum number of shopping scenarios per facility to report each week?

A32. No. The weekly status reports are to be prepared weekly; including during the initiation and planning portions of the contract when no shopping scenarios are being actively executed.

Q33. On page 18, Data Collection Considerations, the RFP refers to subject matter experts who will provide input on the sites to be assessed. Can you provide some clarity on who these subject matter experts are? Will they be experts recruited within the VA or experts that the contractor will be expected to identify externally?

A33.The subject matter experts referenced in this particular section will be recruited by the PE program office.

Q34. Section E.1.1.d (at the top of page 67 of 81) requires “submission of a sample presentation on a topic relevant to the PWS and an executive summary of no more than five pages each.” Can the Government confirm that these 10 required pages are excluded from the overall 20-page limit for the technical volume?

A34. Yes, the sample presentation and same executive summary are excluded from the 20-page limit.

Q35. Eighty trips have been identified as likely to be executed. Eighty trips spread over three phases does not create a large sample. Is there a ratio of “local” Veterans that would be expected to participate at local facility that other Veterans would travel to?

A35.Yes, Veterans are generally enrolled at their local VA medical center. They may be enrolled at another VAMC (e.g., traveling for a holiday); however, it is not common for Veterans to be enrolled at multiple VAMCs.

Q36. Are the contracted Veterans to participate in all three phases of the assessment?

A36. No, the Veteran mystery shoppers are not anticipated to participate in all three phases of the assessment.

Q37. Pg. 16. Who will be responsible for VAMC/CBOC coordination logistics (on-site POC, facility access, etc.)?

A37. The solicitation does not require VAMC/CBOC access. If such access is deemed necessary, the PE program office will provide coordination support.

Q38. Pg. 18. Veteran participants in the assessment who attend appointments should only do so if the service is within their normal clinical care. For example, if the scenario is to include physical therapy (PT), the Veteran should need PT in their normal course of care.

Please provide clarification as to whether the Veteran “requires” care or that is only in their “normal course of care”.

A38.The primary objective of this assessment is to evaluate, from a Veterans perspective, access to VA clinical services and availability of appointments for both primary and specialty care services. The intent is not to take away appointments from Veterans who need them.

Therefore, if an appointment is made for physical therapy by a Veteran mystery shopper who does not need physical therapy, the requirement is for that specific Veteran mystery shopper to call back and cancel the appointment.

Q39. Pg. 18. Sites to be assessed will be selected by the PE program office with input from subject matter experts and the contractor. Scenarios developed for the assessment shall address a mix of Veteran demographic criteria (e.g., age, gender, race) and clinical settings that will be determined in collaboration with the PE program office. Assuming Veterans will need to travel to selected site – When a contract Veteran attempts to schedule an appointment at a site that requires travel, what is an acceptable window for scheduling and traveling to the facility?

A39.It is anticipated that Veteran mystery shoppers will not need to travel outside the boundaries of their local VA medical center. They may be enrolled at another VAMC (e.g., traveling for a holiday); however, it is not common for Veterans to be enrolled at multiple VAMCs.

Q40. Pg. 20. Results shall be compared to any similar assessments conducted in the private, state, or federal health care sectors (e.g., VA Survey of Healthcare Experience of Patients (SHEP)). It is understood that this type of comparison will only be used for reference purposes since assessment objectives, methodologies, and sample sizes will most likely differ.

A40. Correct. Similar assessments will be used only for reference purposes; primarily to understand if there are any correlations.

Q41. Is there a minimal survey size necessary to have the results to have statistical significance?

A41.Due to time, resource, and financial considerations, the solicitation does not require the survey size to be statistically significant. However, the contractor is requested to identify a viable sample size with supporting rationale as part of an evaluation plan to meet assessment objectives.

Q42. Pg. 22. Is the assumption that the contractor will utilize the Microsoft Office suite of products and are there existing templates or artifacts that Government can share?

A42.The contractor is not limited to only using Microsoft Office products as long as Government personnel can access the artifact(s) created. There will be instances where a template may be necessary (e.g., PowerPoint presentation template); however, any such templates will be distributed post-award.

Q43. Pg. 23. As the deliverables are produced, is the audience for these products VA Executive Level Leadership or VAMC level?

A43.Per the solicitation, these documents need to be written at the executive level that could be presented to VA senior program leaders, top level agency leadership and members of Congress.

Q44 Pg. 25. The Contractor will update the assessment evaluation plan to reflect any added, deleted or updated benefit(s), measurement grouping(s) or metrics needed to successfully measure the business value impact of the change from the baseline/initial measurement in the post implementation measurement. The objective of the Post Implementation Evaluation Plan is to describe and demonstrate all quantitative and qualitative measure(s), measurement grouping(s), and metrics to include data definition(s), a detailed and comprehensive rationale, sample size, data collection timeframe, data collection plan, known benchmarks, desired outcomes and validated data sources. Is the Evaluation Plan Update the same as the Post Implementation Evaluation Plan?

A44.Yes, the Evaluation Plan Update is the same as the Post Implementation Evaluation Plan.

The objective of the Evaluation Plan Update deliverable is to optimize the evaluation plan by keeping it current and relevant based on project execution experiences, lessons, and/or findings.

Q45. Pg. 39. For the Key Personnel positions that require PMP certifications, will the Government accept years of demonstrable program and/or project management experience in lieu of the PMP certification? If so, will this be evaluated as equally acceptable?

A45.Yes. Yes.

Q46. Ref: p18 of 81 “Data Collection Considerations”. Given the requirement that scripts be standardized and utilized at all sites, and that the mystery shoppers must be actual Veterans using their real identities, the use of fictitious symptoms such as very high blood pressure may lead to complications worth consideration. The fictitious information may be entered into the actor’s actual record of care, may lead to unnecessary clinical assessments, potential patient harm and clinical waste. Given that the government wants information to assess timeliness of getting needed care, can you address the limits within which the fictitious encounter is to be played out in the interest of avoiding patient harm or caregiver waste?

A46.All scenarios will be reviewed with subject matter experts recruited by the PE program office prior to data collection. The subject matter experts will consist of medical doctors as well as VA staff who are familiar with VA access, scheduling, and operational processes. They will also be requested to take into consideration data that may be entered into the mystery shoppers’ actual patient record, or may lead to unnecessary clinical assessment.

Q47. Ref: p17 of 81, “Objectives” “Evaluate the appropriateness of VA staff triaging”. Under the conditions described for assessment of specialty referrals it seems clear that the patient actor will have to suggest that they have the condition, such as skin cancer, yet the assessment by the referring physician will rule out skin cancer, hence rule out patient referral. Can you clarify the conditions which you expect patient actors to mimic that are not easily clinically screened out, and hence wasted effort, for assessing lag time for specialty appointments? It is not clear how “referral from primary care” for patient actors can be assessed for most specialty referrals.

A47. All scenarios will be reviewed with subject matter experts recruited by the PE program office prior to data collection. The subject matter experts will consist of medical doctors as well as VA staff who are familiar with VA access, scheduling, and operational processes. However, Veteran participants in the assessment who attend appointments should only do so if the service is within their normal clinical care. The primary objective of this assessment is to evaluate, from a Veterans perspective, access to VA clinical services and availability of appointments for both primary and specialty care services. The intent is not to take away appointments from Veterans who need them.

Q48. Ref: p19 of 81 “Adherence to government policies”. This project will require soliciting standardized information requests from a substantial number of Veterans in order to recruit patient actors with specific conditions and health care needs (e.g. patients need physical therapy, or seeking oncologic care etc…), a process that will require surveying. This will require OMB approval in accordance with the PRA. The process should be expected to take between 9-12 months. Does the solicitation take into account that phase 1 cannot begin until PRA approval is granted?

A48. As indicated, since this assessment will require participation from existing Veterans who are eligible and enrolled in VA health care services, careful considerations will need to be made to ensure that all federal government policies and procedures are followed. However, the primary objective of the assessment is to evaluate availability to same day and future medical services. Since any Veteran eligible for health care services should be able to support this objective, it is anticipated that surveying will not be necessary. Per the solicitation, it is anticipated that there will be Veteran mystery shoppers participating in the assessment who may have a need for certain specialty services. Only these Veteran mystery shoppers should be utilized for more complicated scenarios. Lastly, the clinic types are subject to change based on the availability of Veteran mystery shoppers participating.

Q49. Can the government provide some guidance on level of effort or cost for developing this firm fixed price budget?

A49. The quotes developed by the vendor should be consistent with the industry standard. It is the vendors responsibility to submit a fair and reasonable quote.

Q50. Page 4, B.1(3) indicates invoices are to be submitted monthly, but the RFP also indicates that this is a Firm Fixed Price project with milestones based on specific deliverables. Please confirm that invoicing will be based on established milestones rather than on a monthly basis for costs incurred.

A50.Invoices are to be submitted monthly for the specific deliverables that have been completed by the time of the invoice and not previously invoiced.

Q51. FAR 52.222-41, Service Contract Labor Standards, is incorporated by reference in the solicitation, but no wage determination is included. The SOW seems to constitute bona fide professional services which may be considered exempt under the Service Contract Labor Standards. Would the government be open exempting this project a bona fide professional services?

A51. Yes, the Government shall remove the Service Contract Labor Standards clause if determined applicable.

Q52. Page 35 indicates project deliverables must be written from a “business value perspective.” Is it possible to provide a definition for the term ‘business value perspective’?

A52. Business value in this solicitation refers to topics that may ultimately result in actionable recommendations. For example, this may include observations and recommendations regarding process flow inefficiencies, barriers to obtaining access to care, customer service.

Q53. Page 36 in the deliverable schedule it says PM.17 the Lessons Learned Report is due three business days after start of phase. Could the government please confirm the timing for this deliverable?

A53.The timing for this deliverable was incorrectly indicated. It will be updated to “as defined in the project management plan in collaboration with the PE Program Office.”

Q54. While the Government has identified 541910 as the NAICS code for this work, would you consider the use of additional NAICS under 541? While 541910 is also accurate, the scope of this work largely includes project management (inclusive of hiring, communications, coordination, quality management, scope management etc), research/analytics, and comprehensive reporting to the Client. Would the government consider utilizing 541990 which encompass Professional, Scientific and Technical Consulting Services or 541611(Admin Management and General Management Consulting Services) in addition to the current

NAICS?

A54. Yes, the Government would consider utilizing 541990 or 541611 in addition to the current 541910.

Q55. Page 21 of 81 of the solicitation Section D. Travel Requirements states “General and Administrative (G&A) expenses are prohibited and will not be reimbursed.” This is in conflict with FAR part 31.203(d) and FAR part 42.7. Will the government reconsider this prohibition in light of the referenced FAR citations as well as relevant case law regarding John Bransby Productions, Ltd., ASBCA 26913, 82-2 BCA P 16,050, affirmed on reconsideration 83-1 BCA P 16, 375?

A55. We have determined that FAR 31.203 (d) is not applicable to this specific contract;

however, the Government will remove that section of the solicitation and allow G&A expenses.

Profit on G&A and travel is strictly prohibited.

Q56. Pg. 17 What is the difference between the two-bullet statement listed on page 17 (see below) that addresses wait times?

• Assess wait times if services were not offered for dates/times requested (i.e. how many days/weeks after the requested date/time was the appointment scheduled)

• Obtain sampling of Veterans’ wait times to get routine, same day and urgent appointment requests scheduled

A56. The first bullet is a sub-set of the second bullet.

Q57. Pg. 17. Reference the bullet statement on page 17 (see below), what staff will be evaluated? Is this at each site? If so, how will this impact the concept of mystery shopper?

• “Evaluate the appropriateness of VA staff triaging and referring patients to specialty services based on need, …”

A57. Generally, the VA staff triaging and referring patients will be administrative (e.g., VA scheduling staff) or clinical (e.g., RN) personnel who are authorized to work with patients regarding access to medical services.

Q58. Pg. 17 Reference the bullet statement on page 17 (see below), is the Government looking for Veterans with the same types of appointment needs? If not, explain how many scenarios will need to be defined and for how many Veterans will it apply.

• Define various clinical scenarios for an appropriate demographic mix of Veterans

A58.Yes, to obtain a reasonable data set, the number of scenarios will be limited to similar appointment needs. The scenarios will be reviewed and finalized with the PE program office and panel of VA subject matter experts recruited by the PE program.

Q59. Pg. 17 Reference the bullet statement on page 17 (see below), can the Government be more specific? Who will define what should be observed?

• Identify best practices observed (i.e., what can we learn from specific VA medical facilities?)

A59. The data collection will be conducted by the Veteran mystery shoppers; therefore, in this example, the Veteran mystery shoppers may identify a positive experience at a VA medical facility that the contractor program management team will document as a best practice.

Q60. Pg. 17 Reference the bullet statement on page 17 (see below), does each VAMC have its own set of appointment policies or is this referring to enterprise “VA” policies?

• Assess variations from existing VA appointment scheduling policies

A60.This is in reference to local variations from enterprise VA policies.

Q61. Pg. 17 Reference the bullet statement on page 17 (see below), is this routing automatic or does the Veteran need to ask for this?

• Understand the Veteran experience when routed to community care provider services, when applicable

A61. The routing is determined by local VA medical center staff; however, the panel of subject matter experts recruited by the PE program office will assist with defining scenarios that have a high likelihood of being routed to community care.

Q62. Pg. 17 Reference the bullet statement on page 17 (see below), how will the “impacted staff” be identified if the contractor is not allowed to document staff information? Is there a requirement to survey the staff? Are there deliverables associated with this objective?

• Identify impacted staff, medical facilities, and stakeholders as well as level of effort (e.g., short-term, long-term, requires additional resources to implement, needs policy changes)

A62. Impacted staff in this bullet indicates a role, or function. For example, call center personnel picking up phones, clerk at a primary care clinic, etc.

Q63. Pg. 18 Reference the statement on page 18 (see below), how many different scenarios is the Government expecting and for what type of clinic or medical appointment?

“…, if the scenario is to include physical therapy (PT), the Veteran should need PT in their normal course of care.”

A63. Clinics of interest are listed in the solicitation. The number of scenarios will be limited to similar appointment needs. The scenarios will be reviewed and finalized with the PE program office and panel of VA subject matter experts recruited by the PE program.

Q64. Pg. 19 Reference the bullet statement on page 19 (see below), what does this data look like? Will the contractor have to do any data cleansing?

• Common scenarios that apply to most patients as they make their appointments shall be developed based on internal VA access to care data

A64. The data will be provided by the PE program office, it may require filtering, but no data cleansing is anticipated.

Q65. Pg. 19 Reference the bullet statement on page 19 (see below), what does the Government mean by this statement “utilizing different vendor”?

• Phone calls and in-person visits will be required multiple times per scenario (utilizing different vendors) for each clinic/call center to obtain better sampling data and assess site variability

A65. This was in error. It should read: Phone calls and in-person visits will be required multiple times per scenario (utilizing different Veteran mystery shoppers) for each clinic/call center to obtain better sampling data and assess site variability

Q66. Pg. 19 Reference the requirements listed on page 19 (see below), how does these requirements compare to objective #5 on page 17?

2. Anonymity of VHA staff will be maintained

• This is a quality assessment of the structural variation in the processes followed within the VHA system to gain a deeper understanding of what is working well and what issues need improvement, it is not an inspection of individual staff performance

• No identifying information of VHA employees who come into contact with mystery shoppers shall be documented

A66. Objective #5 is in reference to staff groups or roles (e.g., call center clerks, RNs, clinic clerks)

Q67. Pg. 19 Reference the bullet statement on page 19 (see below), will the Government add an addendum that list the paperwork and reference required? In addition, is there an agency within the VA that will need to be a part of this process?

• The Paperwork Reduction Act (PRA) of 1995, which gives the Office of Management and. Budget (OMB) authority over the collection of certain information by Federal agencies

A67. The PE program office will facilitate any communication with the Office of Management and Budget (OMB) as well as any paperwork required.

Q68. Pg. 19 Reference the bullet statement on page 19 (see below), will the Government add an addendum that list the paperwork and reference required? Will the approval from employee unions be done at the VA headquarters level or at each VAMC site? Will the government provide the process for seeking those approvals (at the VA headquarters and local site level)?

• Notification and communication of assessment to applicable federal employee unions.

Although the PE program office will facilitate all communications with national and local unions, the contractor will be required to complete all necessary documentation to support the notification process

A68. The PE program office will facilitate all communications with the federal employee unions.

Q69. Pg. 19 Reference the bullet statement on page 19 (see below), does this have to get approved by the VA? If so, with which agency, and how long will this process take?

• Rules, restrictions, and any applicable release forms needing to be completed regarding the hiring and/or compensation of Veterans

A69. The contractor is expected to ensure that all federal government policies and procedures are followed in hiring and/or compensating Veterans. However, please read the Office of Management and Budget (OMB) guidance regarding compensation of Veterans who are not employees of a company to ensure compliance.

Q70. Pg. 20 Reference the statement on page 20 (see below), is this comparison at the enterprise or local level? Will the Government provide the polies and rules (at the enterprise and local level) for this comparison? Is the contractor required to assess the organizational training for each site, effectiveness, change management, etc.?

“… compared to internal business policies, practices and business rules to identify inconsistencies in operational effectiveness, training, organizational change management and optimization of the business process, expectations, measurements, and successes. Although many of these policies and business rules have already been collected by the PE program office, the contractor shall obtain these artifacts from VA medical facility and central office staff (with the assistance of the PE program manager) as needed and identified.”

A70.The PE program office will provide the national level guidance and artifacts, as applicable.

The program office will also obtain local level artifacts, when available and applicable.

Q71. Pg. 20 Reference the statement on page 20 (see below), what is the Government’s expectation for the duration of each phase (start to finish)?

“The ACMSA contract shall consist of three phases of data collection; each phase may include sub-phases with defined deliverables.”

A71.The Government has not set a specific duration for each phase, only that the three phases are completed during the period of performance.

Q72. How do we capture the price for travel? There is no CLIN listed in the RFP for travel.

A72. Travel costs are included as CLINS 0001-0003, including respective option years. Locations and number of trips will be specified in coordination with the PE Program Manager. Travel shall be in accordance with Federal Acquisition Regulations (FAR) 31.205-46 as well as Federal Travel Regulations. Travel must be pre-approved by the Contracting Officer’s Representative (COR) five business days in advance. Each Contractor invoice shall include copies of all receipts that support the travel costs claimed in the invoice. Trip Reports shall be submitted to the COR within five business days after trip completion. General and Administrative (G&A) expenses are allowable; however, profit shall not be included with G&A and travel.

Q73. Is the Government willing to extend the due date for the submission of proposals?

A73. The Government will not be extending the response date. The due time is extended to 3pm EST, Friday, August 31, 2018.

Q74. How many Veterans are the VA expecting to participate at each site?

A74. Due to time, resource, and financial considerations, the solicitation does not require the survey size to be statistically significant. However, the contractor is requested to identify a viable sample size with supporting rationale as part of an evaluation plan to meet assessment objectives.

Q75. Will key personnel have to work on site?

A75.Some, but not all key personnel will most likely need to be onsite to coordinate data collection activities by Veteran mystery shoppers. However, the specifics of how this will be conducted should be part of the contractor’s proposal.

Q76. Will the work be performed out of Frederick Maryland, or Washington, DC?

A76. The majority of the data collection tasks under this PWS shall take place at VA medical facilities selected for this assessment and/or authorized locations utilized by the contractor to enable mystery shopper Veteran participants to place calls. Assessment planning and preparation will take place at the contractor’s offices. Occasionally, in-person meetings within the metro Washington DC area will be required at the PE program manager’s discretion. These meetings may take place at either contractor or government facilities

Q77. What is the estimated budget for this effort?

A77.Due to the competitive procurement environment, the estimated budget cannot be revealed at this time.

Q78. Will key personnel have to be cleared before beginning work?

A78. Key personnel will not have to be cleared but must go through a background check.

Q79. The PWS estimates approximately 400 days of travel; which is longer than a normal Base Year. Can you clarify?

A79. The travel does not need to be sequential, i.e., multiple teams may travel at the same time in the same week.

Q80. Will we be able to access recorded phone calls through your automated phone systems at the individual facilities?

A80. The PE program office will make every effort to obtain any relevant and/or complimentary data, as available. However, considerations will be made adhere to the Health Insurance Portability and Accountability Act of 1996 that provides data privacy and security provisions for safeguarding patient medical information.

Q81. When veterans make mystery shopping calls, are you expecting those calls to be recorded?

A81. The specifics to how contractor calls are documented have not been included in the solicitation. The specifics of how data collection will be conducted and documented should be part of the contractor’s proposal.

Q82. For Phase 1, have you decided the location of the 3-5 facilities to be evaluated? If not, who will decide those locations and when?

A82. The locations will be finalized at, or near, the contract award date by the PE program office from input by VA subject matter experts. The contractor may also provide any recommendations for consideration.

Q83. Is there an expectation that all three phases will begin during the Base Year?

A83. Yes.

Q84. On page 21 of the solicitation it indicates travel costs are included as a separate line item, but the line item is not included in section b Price/Cost Schedule. Please clarify where you would like us to include the information within the price volume.

A84. Travel CLINs has been included with the updated PWS. Vendors shall utilize CLINS 0001 – 0003, including respective option years, as travel.

Q85. Page 60 of 81 of the Solicitation, Part B. Factors to Be Evaluated, e. states:

e. Performance ratings in PPIRS and submitted Past Performance Questionnaires of not less than Satisfactory for awards having the same NAICS code as appears in the SF1449 during the past three years of the date of solicitation issuance.

Can the Government please provide the Questionnaire template as none was provided with the solicitation?

A85. The amendment will include a standardized PPQ for your reference. The Government will only accept PPQ’s from the referenced contracting office on the PPQ. PPQ’s will not be accepted when submitted by the vendor.

Q86. Page 19 of 81, Data Collection Considerations, Adherence to Government Policies.

“Rules, restrictions, and any applicable release forms needing to be completed regarding the hiring and/or compensation of Veterans.” We understand the Veterans previously utilized to conduct similar types of initiatives which were on a voluntary basis at a single facility. Is there a limit on how much the successful offeror will be permitted to compensate Veteran mystery shoppers participating in this assessment? For example, is it permissible to compensate the Veteran Mystery Shoppers with gift cards?

A86. The contractor is expected to ensure that all federal government policies and procedures are followed in hiring and/or compensating Veterans. However, please read the Office of Management and Budget (OMB) guidance regarding compensation of Veterans who are not employees of a company to ensure compliance. Generally, per OMB guidelines, it is permissible to compensate Veteran mystery shoppers with gift cards, if the compensation is not excessive.

However, the guidance is a bit vague.

Q87. Page 20 of 81, Phased Assessment Phase 1. The PWS states: It is anticipated that appointments scheduled during this phase will be cancelled, unless the assessment participant has a need to go to the appointment. If an appointment is cancelled, is the cancellation recorded in the Veteran mystery shopper’s records at the VAMC?

A87. The cancellation is recorded in the VistA scheduling system. However, the cancelations are recorded in the scheduling system only, but not added to the medical record. So, the cancellation is captured in VistA, but not part of the Computerized Patient Record System

(CPRS).

Q88. Page 25 of 81, Deliverable PM.5. Steering Committee and Working Group Presentations shall contain the information needed to initiate and guide discussions with SME’s and the Working Group. Is there a preferred format for the “Presentations”?

A88. Generally, these will be created using PowerPoint. The PE Program manager will work with the contractor to establish an outline of how the contents are to be presented.

Q89. Page 39 of 81, Key Personnel. The PWS states, “All Personnel proposed under this contract shall be considered Key Personnel”. If the offeror intents to leverage additional labor categories, such as a Technical Writer, is it the Government’s intention the additional labor categories be considered Key Personnel or is it the Government’s intention the 7 positions listed in the RFQ be considered Key Personnel?

A89. The seven positions listed in the RFQ are to be considered Key Personnel

Q90. Page 22 of 81, Travel Requirements. The solicitation indicates the estimated number of…