ODDS DDCM RFP S-DASOBO-00009374 Amd 4 - Rnd 2 Completion Rnd 3 Commence vFinalPost~1.docx

ODDS Case Management System Request for Proposal (“RFP”)

S-DASOBO-00009374

Amendment #4 July 3, 2024 Round 2 Completion, Competitive Range Notice, and Round 3 Commencement The purpose of this Amendment No. 4 (“RFP Amendment 4”) to the ODDS Case Management System RFP is to:

1. Announce completion of the Round 2 scored evaluations and Competitive Range Proposers selected for the next round of Evaluations (“Round 3”),

2. Disclose the final Round 2 scores and Proposer ranking,

3. Establish the Competitive Range of Proposers for Round 3 participation and the Competitive Range Protest Period, and

4. Release initial Round 3 Submission Requirements and Related Evaluation Criteria.

1. Declare completion of the second round of scored evaluation (“Round 2”).

After completion of Round 2 evaluations, all Round 2 Competitive Range Proposers are selected for further participation in a subsequent round of scored evaluation (“Round 3”) (listed in descending order from highest to lowest overall rank after Round 2):

a. RSM Public Service

b. Visionary Integration

c. Guidehouse

d. Ahead

2. RFP final Round 2 scores and Overall Proposer ranking. Four Proposals were evaluated in Round 2 on Demonstrations, Interview, and previously submitted Price Proposal. Overall and Evaluation Round scores for each Proposal are summarized in the table below (listed in order of highest to lowest Total Average Points score).

TABLE 1 – Round 2 Overall Ranking

Average Points Round 2 Average Points Round 1 Average Points

See RFP Amendment 4, Attachment 1 for detailed scoring by scored component.

3. Establish the Competitive Range for Round 3.

a. DAS and the Oregon Department of Developmental Disabilities (“ODDS”) (together the “State Parties”) find value in continuing with scored competition among all Round 2 Competitive Range Proposers in a Round 3.

b. Since there is no exclusion from the Round 3 Competitive Range, there is no opportunity for protesting the establishment of the Competitive Range.

4. Round 3 Evaluation.

a. Methodology Description

1) RFP Section 4.15 provides examples of components for subsequent Rounds of evaluation that the State Parties may require for Competitive Range Proposers and describe via an RFP amendment. The State Parties create Round 3 as a round of evaluation during which they may release submission requirements and evaluation criteria for selected components at different times during the Round.

2) The State Parties currently intend to leverage the following options for Round 3:

a) Additional User-focused Demonstrations and Interviews (Scored) (See RFP Amendment 4, Attachment 3)

b) Revised Price Proposals (Scored) (See RFP Amendment 4, Attachment 4)

c) Proposer Reference Check Surveys (Scored) (See RFP Amendment 4, Attachment 5)

3) To gain additional insight of the Proposed solutions, the Round 3 Evaluation team and scoring structure is changed from prior rounds.

4) Round 3 evaluation will not conclude until all required evaluation components have been received and evaluated according to requirements in this RFP Amendment 4 or subsequent RFP amendments.

5) To remain competitive for possible Contract award, remaining Competitive Range Proposers must participate in any required Round 3 event. Proposer may, however, exercise its discretion to withdraw from further participation in the procurement at any time by written notification to the procurement’s SPC.

b. Submitting Round 3 Documentation.

1) Detailed submission requirements and deadlines are explained in RFP Amendment 4, Attachment 2.

2) Preference: emailing submissions to the SPC is preferred, and provide the following information:

a) Email Subject Line: S-DASOBO-00009374 ODDS CMS RFP Round 3 Demonstration Submission;

b) Company Name; and

c) Name and Contact Information for each company representative participating.

3) Submission size limits:

a) The email system available to the SPC will not accept a single email transmission larger than 25 MB. Submissions exceeding that limit should be separated into two or more transmissions.

b) Submissions requiring multiple email transmissions should also be identified in the subject line with their position in the total number of submissions (e.g., “1 of 3”).

c) The SPC will acknowledge receipt of submissions by email and the SPC’s email system does not alert email rejections. Therefore, Proposers should follow up with the SPC if they do not receive acknowledgement of their submission.

4) Redacted Submissions. If Proposer believes any of its submissions are exempt from disclosure under Oregon Public Records Law (ORS 192.311 through 192.478), Proposer shall use the Disclosure Exemption Affidavit, RFP Attachment B, to identify the desired exclusions. Proposer shall also submit a fully redacted version of such submissions, clearly identified as redacted versions.

5) Each Competitive Range Proposer is solely responsible for ensuring their submission(s) and all requested documents for each round event are received by the SPC by the required deadlines. The State Parties are not responsible for any delays in transmission errors or delays, or for any mis-delivery for any reason.

c. Round 3 Scoring and Evaluation Criteria

a. User-focused Demonstrations and Interview Score. See RFP Amendment 4, Attachment 3 for detailed submission requirements and evaluation criteria, related to Round 3 Demonstrations and Interviews.

b. Revised Price Proposal Score.

See RFP Amendment 4, Attachment 4 for detailed submission requirements of a Revised Price Proposal. Revised Price Proposals received in response to this RFP Amendment 4 will be subjectively scored by comparing and evaluating the following aspects, including but not limited to:

1) How well the Revised Price Proposer’s Pricing Assumptions align with the Price Proposal submission requirements and provide a transparent understanding of the costs.

2) Breakdown of costs included with the proposed solution.

3) Breakdown of costs not included with the proposed solution, (e.g., added features, 3rd party licenses, etc.).

4) Completeness, understanding of project complexity.

5) Alignment with the ODDS and ODHS/OHA goals identified in the RFP.

c. Reference Check Survey. See RFP Amendment 4, Attachment 5 for detailed submission requirements and evaluation criteria, related to Round 3 Proposer Reference Check Surveys.

5. Round 3 Evaluation Score and Ranking.

a. The User-focused Demonstrations and Interview, Revised Price Proposal, and Evaluation Round 2 events will be evaluated and scored using the criteria and process of RFP Section 4.11.2 to determine the points awarded.

b. There is a total of 25,000 total points available for Round 3, allocated in proportion to three different Evaluation elements as shown in Table 2 below.

TABLE 2 –

Round 3 Evaluation Scoring Point Allocation

Each Evaluation Element will be independently evaluated by members of an Evaluation Committee. Evaluators will assign a score for each Evaluation Element up to the maximum points available for that Evaluation Element.

The Evaluator’s point score for each Evaluation Element will be averaged by the SPC. The three averaged Evaluation Element scores will then be summed to produce the Proposer’s Round 3 Score (rounded to nearest whole number).

c. Each Competitive Ranges Proposer’s Round 3 scores will be carried over and summed with its prior round scores for a cumulative total score.

d. The cumulative total score will be used to rank the Competitive Range Proposers from highest to lowest.

-Attachment 1 begins on next page-

RFP Amendment No. 4 to RFP S-DASBO-00009374 ODDS Case Management System RFP Round 2 Completion, Competitive Range Notice, and Round 3 Commencement.

RFP Amendment 4, Page 1 of 4

RFP S-DASBO-00009374 ODDS CASE MANAGEMENT SYSTEM RFP AMENDMENT NO. 4

ATTACHMENT NO. 1

ROUND 2 Evaluation Detailed Scoring Breakdown

-Attachment 2 begins on next page-

Round 2 Completion, Competitive Range Notice, and Round 3 Commencement.

RFP Amendment 4, Attachment 1, Page 2 of 1

RFP S-DASBO-00009374 ODDS CASE MANAGEMENT SYSTEM RFP AMENDMENT NO. 4

ATTACHMENT NO. 2

ROUND 3 Evaluation Schedule and Material Submission Requirements This RFP Amendment 4, Attachment 2 is designed to guide Competitive Range Proposers for the Evaluation Round 3 Milestones and Scheduling component. Other Round 3 evaluation components are addressed is separate attachments.

Evaluation Round 3 Milestones and Scheduling (all times Pacific Time Zone)

July 18, 2024, 3:05 pm – 5:00 pm

July 19, 2024, 8:00 am – 9:55 am

July 19, 2024, 10:05 am – 12:00 pm

The Competitive Range Proposers are encouraged to commence planning for their respective solution demonstrations now. It is the desire of the State Parties for all Competitive Range Proposers to each conduct their demonstration during one of the Round 3 Demonstration Time Slots.

1. Round 3 Demonstration Submission Deadlines.

a. Schedule Coordination. By 5:00 p.m. Pacific Time on July 12, 2024, each Competitive Range Proposer submits to the SPC via email a list of the Round 3 Demonstration Time Slots in descending order of their most to least desired time slot for presenting their Round 3 demonstration or indicate they have no preference for any of the time slots The SPC will attempt to accommodate preferences as possible, but will have final discretion on assigning time slots and respond to each Competitive Range Proposer with the time slot selected for their demonstration.

b. Webinar Information. By 5:00 p.m. Pacific Time on July 16, 2024, each Competitive Range Proposer must submit its Demonstration webinar connection details to the SPC via email.

c. Submission Deadline and Deductions.

1) Demonstration Day Materials: All Competitive Range Proposers must submit to the SPC their Round 3 Demonstration Day Materials set forth below by the common Demonstration Materials Submission Deadline. Failure to provide the items by the deadline may result in elimination of the Proposer from further consideration for this procurement. The following items make up the Round 3 Demonstration Day Materials:

a) Demonstration Day Agenda, Schedule, and identification of participating personnel by name and position.

b) Revised Price Proposal

2) Reference Check Surveys: Reference Check Surveys must be received by the SPC no later than the Reference Surveys Due Date/Time.

2. Round 3 Demonstration Presentation Management. The State Parties currently plan to set aside one four-hour time slot for each Competitive Range Proposer during the Round 3 Demonstration Schedule Period.

a) Each Competitive Range Proposer is responsible for setting up, hosting, conducting, and managing its Demonstration as a webinar.

b) Timing: On its scheduled Demonstration Day, a Competitive Range Proposer Demonstration will be limited to 1 hour and 55 minutes, inclusive of up to 10-minutes of break time. Please plan accordingly to ensure coverage of the key user stories and evaluation criteria.

c) Demonstrations will not be recorded. However, as determined by the features and capabilities of the webinar platform provided by the Proposer,

-Attachment 3 begins on next page-

RFP Amendment 4, Attachment 2, Page 2 of 2

RFP S-DASBO-00009374 ODDS CASE MANAGEMENT SYSTEM RFP AMENDMENT NO. 4

ATTACHMENT NO. 3

ROUND 3 Demonstration and Interview Requirements This RFP Amendment 4, Attachment 2 is designed to guide Competitive Range Proposers for the Round 3 Demonstration and Interview Evaluation component. Other Round 3 evaluation components are addressed is separate attachment. This section offers a framework for proposers to demonstrate the functionality of their proposed solution effectively. It includes the User Stories detailing the priority scenarios that the Agency wishes to be demonstrated, outlining specific use cases and requirements.

1. Evaluation Overview Demonstrations and Interviews will be evaluated and scored by the Agency’s evaluation committee. An important focus in Round 3 will be evaluating the performance of the Competitive Range Proposers’ respective solutions by the way of User Stories in two overarching scenarios:

Round 3 Demonstrations should focus specifically on the user interface and user experience, highlighting the following aspects:

· Real-World Scenario: Where and when possible, the demonstration should exhibit use of the Proposed Solution in a scenario representative of real-world case management.

· User Experience and Interactions: Clearly showcase and highlight the interactions and experiences of different roles working together to assist an individual in acquiring I/DD services.

· Easy and Intuitive Navigation: Demonstrate how users can easily and intuitively navigate the system to find key features without extensive training.

In addition to the User Stories evaluation criteria, Round 3 Demonstrations will be evaluated for the following aspects:

· How closely or clearly aligned the Round 3 demonstrations are to the requested Round 3 user scenarios?

· How well the demonstration showcases the user interface and experience with the Solution in an interactive production-like environment, similar to what users will experience in Oregon?

· How well the Demonstration showcases the Proposed Solution in an interactive production-like environment, limiting the use of static images or presentation slides?

· How well does the Demonstration incorporate or cross-check with ODDS specific terminology when configuring these demonstrations (see ODDS Terms and Definitions at the end of this Attachment 3)?

· How well do the Proposer’s Key Persons (Project Manager, Solution Architect, etc.) demonstrate subject matter experience and expertise via lead the demonstration of the proposed Solution vs. relying on presentation from a non-Key Person, such as a third-party partner?

· How well does the Demonstration showcase how users can effectively accomplish the User Stories using the Solution?

2. Demonstration Scenarios SCENARIO 1: Applicant Applying for and Receiving Services Applying for and receiving I/DD Case Management services in Oregon involves a multi-step process which requires various roles to support the end-to-end experience. This process involves up to six phases including intake, eligibility, entry, assessment, Individual Support Plan (ISP) development, and monitoring. The primary roles that support this end-to-end experience include but are not limited to the applicant, eligibility specialist, supervisor, assessor, and case manager.

For this scenario, showcase the end-to-end experience of an applicant navigating the process to apply for and receive services. Showcase how each role would interact and navigate this process within your proposed solution. Consider highlighting the full experience for each role involved, including accessing dashboards, navigating workflows, and collaborating with different users.

User Stories for Applicant Applying for and Receiving Services

· Search: Given that I have received an applicant’s prime number, when I search the prime number within the Case Management System, then I must have the ability to transfer the applicant’s data from ONE Eligibility System.

· Scheduling: Given that the applicant needs assistance with the application process, when the applicant requests a meeting to discuss the process and next steps, then I must have the ability to schedule a meeting with the applicant.

· Reminder: Given that I am meeting with the applicant to discuss the application process and next steps, when it is two days from the meeting, then a reminder must be sent to me and the applicant.

Step 2: Application and Information Gathering

· Data Collection: Given that I need to collect demographic, contact information, language preference, race and ethnicity, diagnosis disabilities, and related information, when I access the Case Management System to enter this data, then I must be able to do so using an online form.

· Error Handling: Given that I am required to enter an applicant’s data within a webform, when enter a value that is not supported within a particular field, then an error must appear indicating that I have made a mistake.

· Signatures: Given that I am required to access documentation supporting the applicant disability diagnosis, when I need to request a release of information, then I must be able to easily request a digital signature from the applicant to access their records.

· Uploading Documents: Given that I have acquired documentation on the applicant’s disability diagnosis, when I access the applicants record, then I should be able to upload and associate documents to the applicant’s record.

Step 3: Review and Eligibility Determination

· Determining Eligibility: Given that I have received all the information I need to determine the applicant’s eligibility, when I submit the application, then the Case Management System must return an eligibility result with next steps.

Step 4: Eligibility Notification

· Collaborate with Supervisor: Given that I need to quickly identify a Case Manager to assign to the applicant, when I access the chat feature, then I must be able to communicate with my Supervisor to identify a Case Manager to assign in real-time.

· Triggering Notice: Given that the applicant received an eligibility determination and has an assigned Case Manager, when I trigger a notice, then the notice must include the eligibility decision, Case Manager contact information, and next steps.

· Case Note: Given that I have completed all of my tasks in the process, when I go to record a Case Note, then I should be able easily create a Case Note and assign it to the applicant’s record.

· Collaborate with Eligibility Specialist: Given that I have received a request from an Eligibility Specialist to identify a Case Manager, when I access my dashboard, then I must l be able to easily identify and communicate the assigned Case Manager via chat.

· View Workload Capacity: Given that the Case Managers I support have their own workloads, when I access my dashboard, then I must be able to compare the workloads of each Case Manager to determine which one has capacity to support a new applicant.

Step 5: Entry Meeting

· Collaborate with Case Manager: Given that I have received a request from a Case Manager to identify an Assessor, when I access my dashboard, then I must be able to easily identify and communicate the assigned Assessor via chat.

· View Workload Capacity: Given that the Assessors I support have their own workloads, when I access my dashboard, then I must be able to compare the workloads of each Assessors to determine which one has capacity to support a new applicant.

· Collaborate with Supervisor: Given that I have received a new applicant and need to identify an Assessor, when I access the chat feature, then I must be able to communicate with my Supervisor to identify an Assessor in real-time.

· Schedule Assessment: Given that an Assessor has been assigned, when I go to access the calendar to schedule a meeting, then I must be able to see the Assessors availability so I can schedule a meeting for an assessment.

Note: For this demonstration, skip the act of completing an assessment. Assessment data will be pulled in from another system via an integration following the initial go-live.

Step 6: ISP Development

· Schedule ISP meeting: Given that the assessment has been completed, when I access the calendar feature, then I must be able to schedule a meeting with the applicant to finalize the ISP.

· Documenting ISP: Given that I have received the information regarding an individual’s goals and desired services, when I access the Case Management System, then I must be able to capture this information within a webform.

· Searching and Identifying Providers: Given that I have confirmed the services an individual desires to receive, when I access the Case Management System, then I must be able to search for available providers.

· Service Agreements: Given that I have confirmed an available provider, when I go to request for signatures from the provider and the applicant, then I must be able to easily request for a digital signature for the service agreement.

· Case Note: Given that I have completed all of my tasks in the process, when I go to record a Case Note, then I should be able easily create a Case Note and assign it to the applicants record.

Note: For this demonstration, skip the act of authorizing services. ISP data will be pushed to another system to authorize these services.

Step 7: Monitoring

· Due Dates for Monitoring: Given that the applicant has successfully had their ISP implemented, when I go to access my dashboard, then I must be able to easily identify upcoming due dates for the applicant such as their next eligibility determination, ISP due date, assessment due date, and any site monitoring due dates.

· Contact Attempt Case Note: Given that I have just attempted to contact an applicant regarding their services, when I go to access my dashboard, then I must be able to easily create a Case Note and assign it to the applicant’s record.

· Contact Success Offline Case Note: Given that I have contacted the individual in-person where there was no service, when I go to access the Case Management System offline, then I must be able to easily create a Case Note, assign it to the applicant’s record, and have the data sync to the system once I have access to the internet again.

· Ease-of-Use Given that I am applying for services, when I go to interact with the representatives helping me with the process, then I must be able to easily complete my tasks through each phase without confusion or misunderstanding.

· Collaboration: Given that I need to communicate with a representative that is helping me with the process, when I access my dashboard, then I must be able to easily communicate with them in real-time to address any questions and/or concerns.

· Preferred Language: Given that my preferred language is Spanish, when I access my dashboard, then I must have the ability to view information in Spanish.

· Dashboard: Given that I have accessed my dashboard, when I go to view where I am at in the process for applying for services, then I must be able to easily identify the process steps that have been completed, that are remaining, and any next steps with detailed information on what to expect.

· Notification: Given that notifications and reminders have been sent, when I go to access my dashboard, then I must be able to easily identify any actions that require my attention related to the notification/reminders.

SCENARIO 2: Program Management and Reporting Managing program outcomes is a multifaceted effort involving the Centers for Medicare and Medicaid Services (CMS), Office of Developmental Disability Services (ODDS), and Case Management Entities (CME). Each of these entities play a crucial role in ensuring that individuals and families receive the appropriate services for their unique situation. At the center of this coordinated effort, lies the importance of a Case Management System that provides the ability for roles such as eligibility specialists, case managers, supervisors, quality assurance analysts, etc., to have access real-time data on the entire I/DD population. Doing so will empower these entities to make data-driven decisions that improve program outcomes, meet compliance requirements, and ultimately ensure the wellbeing of individuals and their families.

For this scenario, showcase the end-to-end experience for an I/DD representative accessing the Case Management System interested in understanding the characteristics of a distinct population. Showcase how the role would interact and navigate this process within your proposed solution. Consider highlighting the full experience for each role involved, including accessing dashboards, analyzing data, and exporting results.

User Stories for Program Management and Reporting

· Metrics: Given that I need to measure the success of my team, when I go to access my dashboard, then I must have the ability to easily analyze the case load, time spent per case, service plan completion, client contact rate, and case transfer rate of my team.

· Analysis: Given that I am interested in analyzing trends and reporting on outcomes for my team, when I access my dashboard, then I must have the ability to easily aggregate and/or dissect the workload data generated by my team.

Process

· Metrics: Given that I need to measure the success of my team, when I go to access my dashboard, then I must have the ability to easily analyze the eligibility determination time, process cycle time, follow-up compliance, and compliance rate.

· Analysis: Given that I am interested in analyzing trends and reporting on outcomes for my team, when I access my dashboard, then I must have the ability to easily aggregate and/or dissect the process data generated by my team.

Note: For definitions on the metrics listed in this story, reference section 6.

· Quality of Care: Given that I need to measure the success of the organization, when I go to access my dashboard, then I must have the ability to easily analyze client satisfaction, ISP goal achievement, timeliness of service, waitlist times, incident reporting, etc.

· Case Notes: Given that I want to analyze metadata on Case Notes that were submitted late, when I access my dashboard, then I must be able to easily analyze this data including identifying the number of case notes submitted late, assigned case manager per case note, created date, required by date, case note type (e.g., billable vs nonbillable), county of origin, etc.

Note: For definitions on the metrics listed in this story, reference section 6.

3. ODDS Terms and Definitions

This section provides details on common terms used to describe the characteristics of I/DD Case Management in Oregon.

-Attachment 4 begins on next page-

RFP Amendment 4, Attachment 3, Page 1 of 8

RFP S-DASBO-00009374 ODDS CASE MANAGEMENT SYSTEM RFP AMENDMENT NO. 4

ATTACHMENT NO. 4

ROUND 3 Revised Price Proposal Process and Submission Requirements This RFP Amendment 4, Attachment 4 is designed to guide Competitive Range Proposers for the Round 3 Revised Price Proposal Evaluation component. Other Round 3 evaluation components are addressed is separate attachments. This section offers a framework for Proposers to propose an estimated of cost associated with their solution, detailing their assumptions, conditions, and other information that could be considered relevant to understanding the calculation and factors that affect their proposed price.

1. Proposer shall complete and submit the Revised Price Proposal Worksheet at the end of this attachment.

a. Proposer may add additional rows and columns to increase granularity or clarity of their proposal.

b. Proposer shall list any assumptions that informed its pricing approach to implementing and maintaining the Solution, along with providing any relevant assumptions for the Agency to consider during its evaluation of the Revised Price Proposal Form.

2. Agency anticipates making fixed monthly payments for the Solution and fixed prices for each Deliverable completed, delivered to, and Accepted by Agency, up to a Maximum Not-To-Exceed compensation amount to be negotiated with the successful Proposer.

3. The Revised Price Proposal for Round 3 must clearly convey and distinguish the following cost parameters:

a. Implementation Cost: Proposer shall provide its estimated one-time cost for the implementation of the solution, which must be inclusive of the professional consulting and implementation services as specified in RFP, Section 2.4.2.1.5.

b. Proposer may provide a narrative of its pricing approach to providing the professional consulting and implementation services as reflected in its Implementation Cost Items to provide context and key descriptors that would support understanding the Proposer’s pricing approach. Licensing Cost(s): Proposer shall provide its estimated cost for ongoing licensing of the solution, inclusive of all related fees, for an estimated 3,000 end users of the Solution and any necessary administrative, customer service, or other licenses to fully utilize the Solution as described in the Proposer’s submittal.

If the Proposer’s Solution does not directly include licensing, and licensing will be provided through a reseller for the Agency, the Proposer must still provide an estimate of the licensing cost to the Agency based on sample quotes or prior experience – assumptions must be stated accordingly.

c. Hosting Cost: Proposer shall provide its estimated cost for ongoing hosting of the Solution, inclusive of all related fees.

If the Proposer’s Solution does not include a hosting fee, or if it is inclusive in the estimated licensing cost, the Proposer must include this statement to provide clarity to the Agency.

d. Maintenance & Support: Proposer shall provide its estimate for ongoing maintenance and support of the Solution following the successful completion of Implementation activities. Provide estimated costs through a 10-year service period.

The Proposer shall include cost that reflects its proposed approach to supporting the Agency to close out the project activities, transition to maintenance and operations post go-live, and provide the necessary levels of ongoing maintenance and support – assumptions must be stated accordingly.

e. Other One-Time Costs: Proposer shall provide any estimated one-time costs not included in the Cost Item categories above.

f. TOTAL: Proposer shall consolidate costs into a total cost to the Agency, organized by year and inclusive of an overall total projected over ten years.

g. Contingency Tasks: Proposer shall provide role titles and hourly rates for any contingency tasks outside of the current scope of the Case Management System project, as defined in the Scope of Services (S-DASOBO-00009374 DDCMS RFP, Section 2.4.2, page 15).

RFP Amendment 4, Attachment 4, Page 1of 3

RFP S-DASBO-00009374 ODDS CASE MANAGEMENT SYSTEM

Round 3 Revised Price Proposal Worksheet

Professional Consulting and Implementation Services Cost Narrative

Assumptions

-Attachment 5 begins on next page-

RFP Amendment 4, Attachment 4, Page 3 of 3

RFP S-DASBO-00009374 ODDS CASE MANAGEMENT SYSTEM RFP AMENDMENT NO. 4

ATTACHMENT NO. 5

ROUND 3 Proposer Reference Check Survey Evaluation Process and Submission Requirements This RFP Amendment 4, Attachment 4 is designed to guide Competitive Range Proposers for the Round 3 Reference Check Survey Evaluation component. Other Round 3 evaluation components are addressed is separate attachments. This section offers a framework for Proposers to provide their references with an opportunity to provide more detailed feedback to Evaluators on the Proposer’s past performance.

1. Each Proposer must submit the name and contact information for three client references to the SPC. The client references must be able to verify the quality of the Proposer’s previous work. These client references:

a. Should have direct knowledge of the project or engagement which the Proposer delivers or delivered services to that client,

b. Should be for projects that are similar in scope to the ODDS Case management system that is the subject of this RFP S-DASOBO-0000934,

c. May be the same, in addition to, or be alternates of the client references previously submitted in response to the RFP.

2. Each Proposer will send the Proposer Reference Check Survey (“Survey”) to their named client references, and request the client:

a. Provide written responses to Survey questions, or otherwise provide information in writing or via follow up discussion, concerning aspects of its relationship with Proposer,

b. Send their completed Survey to the SPC no later than the deadline provided in the RFP Amendment 4.

The Proposer Reference Check Survey is included as an embedded object in this document, and available as a separate download from OregonBuys.

3. DAS PS and ODDS may, at their discretion, request Proposer send a Survey to its clients that were not submitted by Proposer in response to the RFP or this RFP Amendment 4.

4. Survey Description

a. The survey asks a total of thirty-two questions grouped into six areas:

1) Section 1, System Quality: Related to the client’s overall satisfaction with the Proposer’s quality of the system.

2) Section 2, Service Quality: Related to the client’s experience with the Proposer relating to quality of the services.

3) Section 3, Convenience of Operation: Related to the client’s experience with the convenience of operating the system.

4) Section 4, Long-term Reliability: Related to the overall experience and satisfaction of the long-term reliability of Proposer’s performance.

5) Section 5, Transparency of Cost: Related to the client’s experience with Proposer’s transparency regarding cost of services.

6) Section 6, Contract: Related to the client’s experience with negotiation and administration of their contract with the Proposer.

b. Proposer Client References are asked to complete a brief description of their project, and for each question rate the Proposer’s performance using a drop-down menu and provide comments the reference deems appropriate to clarify or support their rating.

5. The Evaluation Committee will consider the following factors in scoring the Survey:

a. Information provided in Survey responses, and other information provided by client references that respond,

b. Degree to which the Survey responses present an engagement comparable in size, scope, and complexity to the project described in the RFP.

c. Relevant information from other client checks (if performed).

6. While ideally the Evaluation Committed would receive at least three Surveys for each Proposer, the Surveys are to be evaluated as a collective of those Surveys received by the deadline (i.e., receiving a higher number of Surveys is not in and of itself a factor resulting in a higher score). However, if no Surveys are received by the deadline, the Evaluation committee will only consider the information provided by the reference checks submitted with the initial Proposal.

-End of RFP Attachment 5, and RFP Amendment 4 -

RFP Amendment No. 4 to RFP S-DASBO-00009374 ODDS Case Management System RFP Round 2 Completion, Competitive Range Notice, and Round 3 Commencement.

RFP Amendment 4, Attachment 5, Page 1 of 2 image1.png image2.png image3.emf

ODDS%20DDCM%20RFP%20S-DASOBO-0000934%20Rnd%203%20P

COVER & INSTRUCTIONS

RFP S-DASBO-00009374 ODDS CASE MANAGEMENT SYSTEM RFP

Evaluation Round 3 Proposer Reference Survey

Proposer:

Proposer Client Reference Name:

Proposer Client Reference Contact Information (phone/email):

Proposer Client Reference's Project Name and Description:

Purpose:

This document serves as a guide for the collection of pertinent information as part of the RFP S-DASBO-00009374 ODDS CASE MANAGEMENT SYSTEM RFP, for Round 3 Evaluations. There are a total of 32 questions across 6 categories.

Completion Instructions:

In the Project Name and Description above, the client provides a brief description of the project the Proposer has performed work for which the client has direct knowledge regarding the Proposer's performance. When possible, please ensure to describe the scope, perceived complexity, estimated cost, and size of the user base for the project. If the Proposer (Contractor) has performed more than one project, please include that information and rate the Proposer's performance as an average.

The Survey Question tab has questions divided into six areas sections as follows:

Section 1, System Quality: There are 6 questions related to the client’s satisfaction with the Proposer's quality of the system.

Section 2, Service Quality: There are 6 questions related to the client’s experience with the Proposer relating to quality of services.

Section 3, Convenience of Operation: There are 8 questions related to the client’s experience with the convenience of operating the system.

Section 4, Long-term Reliability: There are 6 questions related to the overall experience and satisfaction of the long-term reliability of Proposer's performance.

Section 5, Transparency of Cost: There are 3 questions related to the client’s experience with Proposer's transparency regarding cost of services.

Section 6, Contract: There are 3 questions related to the client’s experience with the Proposer relating to contract negotiation and contract administration.

Each Section is made up of four columns: Question ID#, Question, Response, and Comments. For each Question, please:

Use the drop down menu to select a response that best represents your experience with the Proposer;

Add comments as you beleive appropriate to clarify or support your response.

&P

Survey Questions

Section 1, System Quality

ID Question Response (Please select from drop-down menu) Comments

SY.1 Are the number of system defects ("bugs") lower than expected given the complexity of the system, about what you expected given the complexity of the system, or higher than expected given the complexity of the system? Please describe the number of system defects and whether the number of defects is trending down, remaining fairly constant, or trending up.

SY.2 Describe Proposer's approach to testing for changes to system functionality ("enhancements") and how they coordinate and look across functional track areas ("effect on SNAP and Medical, Reports and Dashboards, Rule Sheets and Notices).

SY.3 Describe Proposer's approach to testing for technology changes and how items are coordinated across functional and technological tracks including other partner interfaces, reporting, disaster recovery, and various environments.

SY.4 Is Proposer's testing effective for identifying defects prior to implementation into the production environment and maintaining a stable system?

SY.5 Does the Proposer meet their contractual service level agreements (SLAs)?

SY.6 Is your system available as scheduled?

Section 2, Service Quality

ID Question Response (Please select from drop-down menu) Comments

SE.1 Describe the skill level and knowledge of human services programs (e.g., Medicaid, SNAP, TANF, intellectual/developmental disabilities, employment-related services, housing) demonstrated by the Proposer's staff.

tc={C9DC15DD-EBD2-408E-9233-AEAB53D7F63E}: [Threaded comment]

Your version of Excel allows you to read this threaded comment; however, any edits to it will get removed if the file is opened in a newer version of Excel. Learn more: https://go.microsoft.com/fwlink/?linkid=870924

Comment:

This is relevant, so I have removed the highlighting and added "intellectual/developmental disabilities" as an example.

SE.2 Describe the skill level and knowledge of technology components (e.g., interfaces, configuration management, data management, patch management, server configuration) demonstrated by the Proposer's staff.

SE.3 Describe the Proposer's collaboration and interaction with stakeholders (e.g., staff, leadership, outside entities). How did Proposer modify its communication style and methodology to interact with different people?

SE.4 Conflict resolution is a necessary skillset for successful completion of intricate work efforts. Please describe how the Proposer handled conflict situations. Was the conflict resolved in a manner that assisted the project. Please give examples.

SE.5 What was the quality of advice that the Proposer provided throughout your project? Did the Proposer suggest meaningful improvement to design, business processes, or other relevant topics? Did the Proposer provide examples from other states or other applicable systems to improve your overall experience or assist with your decision making?

SE.6 Did the Proposer provide highly skilled staff for your engagement? How did the Proposer communicate any staffing changes?

Section 3, Convenience of Operation

ID Question Response (Please select from drop-down menu) Comments

CO.1 Is the system relatively easy to use ("user friendly")? Is the system efficient for accomplishing the work?

CO.2 What do system end users like best about the system and what would they like to improve? How does this vary across different types of end users? How did the proposer take feedback from users?

CO.3 Is the system technology and infrastructe reliable and relatively easy to operate and maintain? (E.g. are batches successfully processed on schedule or is intervention required on a regular basis, are patches applied on a reasonable schedule, is data loaded into non-production environments on a reasonable schedule, are non-production environments maintained and usable, is disaster recovery testing conducted on a regular basis)

CO.4 What do people responsible for managing the technology, infrastructure, and security aspects of the system like best about the system and what would they like to improve?

CO.5 Have you been able to modify system functionality to quickly respond to changes in processes, information, federal requirements, and legislative changes?

CO.6 Have you been able to modify system configuration easily to respond to technology changes (e.g., end-of-support for COTS products / third party software, adding or removing non-production system environments)

CO.7 Describe the documentation provided by the Proposer to assist end users (e.g., training materials, frequently asked questions, tips and tricks).

CO.8 Describe the documentation provided by the Proposer for system operations, configuration management, and technology management.

Section 4, Long-term Reliability

ID Question Response (Please select from drop-down menu) Comments

LT.1 Has the Proposer's organizational stucture been reasonably stable over the course of your work with them? How has the Proposer communicated organizational changes, if appliable, over that time?

LT.2 Did the Proposer consistently provide highly skilled and highly experienced persons for key roles within the project ("Key Persons")? When Key Persons needed to be replaced, how did the Proposer communicate those changes and were the suggested replacement persons of equal or better skill and experience?

LT.3 Did the Proposer consistently provide skilled and experienced subject matter experts for non-Key Person roles?

LT.4 Did the Proposer consistently provide…