Att 3 eCAS Reports and Plans~1.docx

ATTACHMENT 3: E-CAS REPORTS AND PLANS

Contents

Attachment 3, UCA System Reports and Plans 2 The Bidder will provide reports including:

· User reports will support assessment operations and program management, including monitoring of assessment workflows, operations and quality, Assessor performance, provider performance, and assessed individuals’ clinical needs for use in program integrity initiatives, population health analytics and policy development. Reports will be available as part of each user’s dashboard and will include standard programmatic reports and the capacity for users to customize reports based on a set of predetermined parameters, as well as selected data sets for download. Reports will be populated with data available based on each user’s authorized access (i.e. based on user role, Provider organization, Assessment Team, and Release of Information). See the User Reports section below for the types of standard reports and proposed elements the Selected Bidder will develop for users.

· CMS reports to support EOHHS performance reporting to the Centers for Medicare and Medicaid Services (CMS). CMS reports will be available in a form and format and on a cadence to be specified by EOHHS.

· Project Deliverables/reports will allow EOHHS to monitor Selected Bidder project progress. The Project Deliverables Table below includes the Deliverables/Reports/Plans from Section 3.2 Implementation Tasks and Deliverables and Section 3.4 Maintenance and Operations Support Services Tasks and Deliverables. Plans are identified in bold font.

· System performance reports will support EOHHS with monitoring ongoing system performance. See the System Performance Reports Table below for details.

User Reports User reports will be developed for each Assessment Tool according to the implementation process outlined in RFR Section 3.2 and will meet requirements described in RFR Section 1.4.1.1.10.

The table below outlines report example users and potential use cases:

· Information about behavioral health needs, functional needs, program eligibility, demographics, MassHealth coverage type, number of assessments per year, assessment Outputs, no-show rates, etc.

· Data from a single assessment or information from multiple assessments to show trends over time.

Help clinicians gather information, prioritize needs for intervention and examine improvement or challenges over time.

Facilitate options counseling and referrals to specific services that the individual has been determined eligible for. For example, the UCA may indicate that a Member is eligible for several LTSS programs, and the report may be used to facilitate discussion of how best to meet the Member’s needs.

· Information about behavioral health needs, functional needs, program eligibility, demographics, MassHealth coverage type, number of assessments per year, assessment Outputs, no-show rates, etc.

· Data from a single assessment or information from multiple assessments to show trends over time.

· Data from a single assessed individual or groups of assessed individuals (i.e. Assessor’s entire caseload).

Individual or summary information about Assessors and their work. Reports may include:

· Assessment Outputs, answers to high priority assessment questions, excerpts from narrative sections, etc.

· Assessor productivity (e.g., timeliness to schedule and complete assessments, number of assessments completed) and performance (e.g., outcome of quality reviews, Appeals).

· Data from a single assessment or information from multiple assessments to show trends over time.

Help supervisors examine case mix of caseloads for equity of case assignment.

Help supervisors examine accuracy of CANS ratings to inform supervision.

Help supervisors provide quality supervision around assessment and case conceptualization as well as intervention choices.

Monitor interrater reliability (IRR) and provide opportunities for coaching and training of individual Assessors.

· Information about behavioral health needs, functional needs, program eligibility, demographics, MassHealth coverage type, number of assessments per year, assessment Outputs, no-show rates, etc.

· Data from multiple assessments to show trends over time.

· Data from groups of assessed individuals (e.g., all individuals at an outpatient clinic).

· Combined assessment data with claims, Prior Authorization, or other service utilization data.

Monitor population characteristics including acuity, presenting problems and demographic information.

Support new program development, grant writing, or quality improvement initiatives.

Monitor population health trends over time, including outcomes of services.

Provide information about case mix by program type (for instance in-home therapy vs. outpatient).

Calculate performance metrics or quality measures.

· Information about behavioral health needs, functional needs, program eligibility, demographics, MassHealth coverage type, number of assessments per year, assessment Outputs, no-show rates, etc.

· Data from multiple assessments to show trends over time.

· Data from groups of assessed individuals (e.g., all individuals at an outpatient clinic).

· Combined assessment data with claims, Prior Authorization, or other service utilization data.

Information about specific Assessment Tool workflow steps. Reports may include:

· Quality review statistics, including productivity and performance (e.g., number of assessments selected for quality review, outcomes of quality review such as changes made to assessments).

· Appeals statistics, including productivity and performance (e.g., withdrawals prior to Appeal, BOH decisions (upheld, partially upheld, overturned, new information provided during hearing).

· Data from a single assessment or information from multiple assessments to show trends over time.

· Data from a single assessed individual or groups of assessed individuals (i.e. Assessor’s entire caseload).

· Data associated with a single or multiple Assessors, Assessment Teams, Providers, programs, services, or state agencies.

Examine population health trends, such as changes in functional and clinical needs over time. For example, CANS data can be linked to service utilization data to analyze the relationship between population characteristics, clinical needs, service utilization, and outcomes.

Assess availability of needed service providers by geographic region.

Examine predictors and outcomes of service utilization.

Monitor Provider and Health Plan compliance and performance, including calculating performance metrics or quality measures.

Support new program or policy development.

Support contract management, program integrity initiatives, and process improvement efforts (e.g., improving the way Appeals are managed).

· Assessment Outputs, answers to high priority assessment questions, excerpts from narrative sections, etc.

· Assessor productivity (e.g., timeliness to schedule and complete assessments, number of assessments completed) and performance (e.g., outcome of quality reviews, Appeals).

· Data from a single assessment or information from multiple assessments to show trends over time.

Monitor interrater reliability (IRR) and provide opportunities for coaching and training of individual Assessors.

Examine trends in quality of Assessors’ practice in order to identify continued training needs and develop future trainings.

Examine data quality.

· Information about behavioral health needs, functional needs, program eligibility, demographics, MassHealth coverage type, number of assessments per year, assessment Outputs, no-show rates, etc.

· Data from multiple assessments to show trends over time.

· Data from groups of assessed individuals (e.g., all individuals at an outpatient clinic).

· Combined assessment data with claims, Prior Authorization, or other service utilization data.

Summary information about Providers. Reports may include:

· Assessor, Assessment Team, or Provider productivity (e.g., timeliness to schedule and complete assessments, number of assessments completed) and performance (e.g., outcome of quality reviews, Appeals).

Examine population health trends, such as changes in functional and clinical needs over time. For example, CANS data can be linked to service utilization data to analyze the relationship between population characteristics, clinical needs, service utilization, and outcomes.

Assess availability of needed service providers by geographic region.

Examine predictors and outcomes of service utilization.

Monitor Provider compliance and performance, including calculating performance metrics or quality measures.

CMS Reports

· e-CAS usage/access by EOHHS staff users

· Number of assessments for the same member for different programs or services

· Spend on assessments for the same member for different programs or services

· Number of members receiving services who have e-CAS assessment data

· Program Integrity analyses conducted using e-CAS data

Project Deliverables/Reports

There are a number of Tasks and Deliverables specified in the RFR and listed in this Attachment.

There are a number of Tasks and Deliverables specified in the RFR and listed in this Attachment.

There are a number of Tasks and Deliverables specified in the RFR and listed in this Attachment.

There are a number of Tasks and Deliverables specified in the RFR and listed in this Attachment.

· Scope: High-level definition of the scope of functionality that the e-CAS will have, including the timing of Phases 1-2 of the e-CAS implementation and which programs, services, and functional Modules will be included in each implementation stage.

· Project timeline and overview: A three-level work breakdown structure; project timeline; project Milestones; and Deliverables. For each milestone and Deliverable, the Selected Bidder will describe exactly what will be provided to meet the Milestone or Deliverable, metrics used to determine when the Milestone or Deliverable has been met, and project schedule associated with the Milestone or Deliverable. If applicable, Milestones and Deliverables should be included for the implementation of each stage of the project.

· Project stages: Describes the Selected Bidder’s process to complete each major project stage (i.e., Project Initiation; Requirements, Development, Testing, and Pilot; Knowledge Transfer and Training; Data Conversion and Migration; Deployment and Acceptance).

· Work management and execution guidance: Includes description of how the Selected Bidder will ensure the project team is completing work according to project timelines and at a high level of quality. Must include proposed meetings and meeting cadence, a description of the environment that the Selected Bidder will set up to manage the project and project information and to facilitate collaboration among project team members (this environment can take the form of an online “project portal”), and any other specifics related to work completion and work processes that the Selected Bidder deems important.

· Division of project tasks: Describes how Selected Bidder will utilize its resources for certain project activities and when the team will rely on EOHHS resources for certain project activities. Includes high-level roles and responsibilities, Selected Bidder staffing plan and organizational chart, and any other important information related to division of labor. Additionally, the Selected Bidder must clearly specify deliverables and dates that require EOHHS’s involvement for technical setup and project environments and the involvement of EOHHS staff in implementation activities.

· Documentation of key project information and management of project documentation: Describes how the Selected Bidder will record assumptions, decisions, risks, and other key pieces of project information. Describes how documents will be maintained, including document owners and update protocols. Must also detail how the “project portal” will manage the flow of project-related documents.

· Stakeholder management: Describes how the Selected Bidder will engage various project stakeholders throughout the life of the project to ensure all impacted parties are aware of project progress and are consulted as needed. The Selected Bidder will address EOHHS stakeholders as well as Providers and Health Plans.

· Project controls: Describes how the Selected Bidder will facilitate key management review as to project content, scope, and schedule.

· Any other sections that the Selected Bidder deems necessary and are approved by EOHHS.

The Selected Bidder’s Implementation Plan shall be constructed in accordance with industry standards, accepted project management principles outlined in the Project Management Body of Knowledge (PMBOK) from the Project Management Institute, or acceptable equivalent. The Selected Bidder must build, produce and maintain the Implementation Plan in Microsoft Project or comparable project management system approved by EOHHS.

· How the Selected Bidder has already identified (based on prior experience and organizational experience) and will, during the course of the implementation, identify issues that could affect the achievement of project objectives;

· How the Selected Bidder will systematically assess and rank the risk associated with these issues;

· How the Selected Bidder is already pursuing or would rapidly develop and implement mitigation strategies that effectively address the risks associated with the aforementioned issues; and

· The tools the Selected Bidder will use for tracking internal (Selected Bidder) and external (EOHHS, Provider, Health Plan) issues and related risks, including both technical and non-technical issues that could affect the Deliverables, schedule and/or budget.

EOHHS expects that, at a minimum, the plan will contain the following sections:

· Risk Management Approach – This section of the Risk Management Plan must provide a general description for the approach the Selected Bidder will take to identify and manage the risks associated with the project.

· Risk Identification – This section of the Risk Management Plan must explain the process by which the risks associated with the e-CAS will be identified. It must describe the method(s) for how the Selected Bidder identifies the risks, the format in which risks are recorded, and the forum in which this process will be conducted.

· Risk Qualification and Prioritization – This section of the Risk Management Plan must describe the Selected Bidder’s approach to determining the probability and impact of each risk to allow the Project Manager to prioritize the risk avoidance and mitigation strategy. This is usually done with a risk matrix. This section must also explain how risks will be qualified and prioritized for this project.

· Risk Monitoring – This section of the Risk Management Plan must discuss how the risks in the project will be actively monitored, ensuring that risks are identified and monitored throughout the life of the project and that each risk has trigger conditions identified. There must be thorough documentation of the risk monitoring process.

· Risk Mitigation and Avoidance – Once risks have been qualified, determination must be made on how to address those risks which have the greatest potential probability and impact on the e-CAS project. This section of the Risk Management Plan must explain the considerations which must be made and the options available to the Project Manager in managing these risks.

Requirement Elaboration and Specification Definition

· Documenting business requirements;

· Providing configuration and, if applicable, customization instructions to developers;

· Providing a basis for estimating configuration/customization level of effort and work duration;

· Establishing agreement with EOHHS on exactly what will be built; and

· Providing a point of synchronization for the whole project team.

The Functional Specification Documents will also be inputs to the Implementation and Deployment and Acceptance Plans. After functional specifications are reviewed and approved, changes to the specifications will require EOHHS’ prior written approval.

Section 3.2.2, Requirement Elaboration and Specification Definition

· Goals and business objectives;

· Background information, including relevant as-is documentation;

· Assumptions;

· User Stories and/or Use Cases;

· Acceptance criteria;

· User interaction and user design;

· Questions and answers tracking; and

· Future work planned, including work that has been identified for future e-CAS phases.

Section 3.2.2,

· Overview and background;

· Architecture and design principles;

· Connection to User Stories and/or Use Cases;

· Solution details; and

· Diagrams of the solution.

Section 3.2.2,

· The concept of operations for each interface;

· Definitions of the message structure and protocols that govern the interchange of data;

· Identification of the communication paths along which the project team expects data to flow;

· A description of the data exchange format and protocol for exchange;

· Documentation of exception/error handling process, including detailed definition of Bad Data and how it will be corrected;

· A general description of each interface;

· Data dictionaries;

· Assumptions where appropriate; and

· Estimated size and frequency of data exchange.

The Data Integration/Interface Specifications Document will be completed in accordance with the dates set forth in the approved Implementation Plan.

Section 3.2.2,

Along with EOHHS’s Project Manager, and in accordance with the Implementation Plan, the Selected Bidder will participate in project briefings and supply content for communications materials that convey project status and progress to executive sponsors and key stakeholders.

Section 3.2.3, Build: Configuration, Customization and Integration

· Scope;

· Objective;

· Roles and Responsibilities;

· Test Schedule;

· Test Execution Protocol/Workflow;

· Assumptions for Test Execution;

· Constraints for Test Execution;

· Test Scripts – these must be tied to functional requirements;

· Test Data Requirements;

· Test Resource Requirements;

· Expected Results;

· Acceptance Criteria (including item pass/fail criteria);

· Issue Tracking;

· Issue Reporting;

· Testing Status Reports;

· Stage Completion Reports;

· Test Final Report Sign-Off;

· Risk Mitigation;

· Testing Facilities;

· Testing Tools;

· Issue Tracking Tools;

· Issue Severity and Priority Definition;

· Issue Reporting; and

· Remediation Process 3.2.4, Testing

For each audience (system administrators and end users), the Knowledge Transfer and Training Plan shall include at a minimum:

· A recommended approach or approaches to knowledge transfer (system administrators) or training (end users); the approaches should be designed for adult learners possessing a variety of backgrounds, experiences, and learning styles;

· The population size and types of roles that need training;

· A recommended approach to acquired skills assessment;

· An inventory of tasks, deliverables, and resources necessary to complete the knowledge transfer or training effort, including tools and documentation necessary to support the proposed effort; and

· For each method or course:

· A course description;

· The target audience (system administrator, end user or other);

· Proposed training goals;

· Proposed training standards;

· The specific plan for training relevant personnel, including EOHHS input;

· The delivery timeframe (by phase, implementation step) with a strategy for providing training early in the project to allow the training goals to be implemented throughout the project life cycle;

· A description of training deliverables and format (i.e., online, written documentation, course materials); and

· A description of skill sets achieved at the end of training and how training effectiveness will be measured and addressed.

The Selected Bidder’s EOHHS-approved training schedule must be closely coordinated with EOHHS staff. Upon acceptance by the EOHHS Project Manager, the Selected Bidder shall implement the approved Plan.

· The scope of data conversion/migration activities;

· An inventory/catalog and profile of data to be converted, if applicable;

· The approach to be followed for all data conversion and migration activities for the e-CAS functional areas, including a detailing of specific activities, their durations and applicable dependencies;

· A list of applications impacted;

· A list of conversion and reconciliation tools to be employed;

· An outline of conversion roles and responsibilities;

· A description of conversion resource requirements;

· The approach and methodology to data transformations;

· The approach to be followed for data cleanup;

· The approach and methodology for data classification (i.e., organization and/or tagging);

· The approach to be followed for methodology for conversion testing/validation; and

· An outline of acceptance criteria.

It is expected that the Selected Bidder’s approach to the completion of the following tasks will be addressed in the Data Conversion and Migration Plan:

· Ensure database backups are in place;

· Execute data conversion routines/packages;

· Validate converted data to confirm success;

· Revert to backup if conversion fails;

· Provide EOHHS with the results of the conversion and any exceptions;

· Work with EOHHS to resolve nulls and non-converted data; and

· Provide post conversion support through requested ad-hoc reporting and provision of access to the pre-and post- converted data for EOHHS confirmation analysis.

The Deployment and Acceptance Plan must address, at a minimum:

· Deployment activities, including deployment of user documentation and online help;

· Sequencing of all deployment events;

· Deployment schedule;

· Go/No-Go Decision Points;

· Incident and Defect Resolution documentation and tracking, including Defect classification as described in Attachment 5, EOHHS Key Performance Indicators;

· Cut-off schedule for legacy applications; and

· Each item listed above should be provided for each deployment phase (i.e., Phases 1-2).

· Recording and codification of incidents and/or problems with the solution that compromise its availability or performance: At a minimum the following information shall be captured for each incident: incident’s reporting origin, a description of the incident including the incident’s impact, and the codification of the incident based on potential cause(s), magnitude and resolution priority.

· Recording and codification of Defects and/or problems with the solution that limit or otherwise adversely impact its functionality: At a minimum the following information shall be captured for each Defect: Defect’s reporting origin, a description of the Defect including the Defect’s scope and functionality affected, and the classification of the Defect based on potential cause(s), magnitude and resolution priority.

· Resolution of incidents and Defects: At a minimum, the log/report shall capture, irrespective of reporting origin, interim and final measures taken by the Selected Bidder (and, if applicable, EOHHS) to resolve incidents and Defects.

· Date/time stamps for events associated with all the above.

· Ensuring the Selected Bidder's solution continually meets EOHHS requirements and is updated with new or revised functionality (i.e., via development and configuration) to support new requirements as a result of programmatic, operational, organizational, legal or regulatory changes that are agreed upon via the Change Management process;

· Accommodating changes to regulations, standards, and EOHHS organizational processes through operational and system changes that do not require significant system functional changes (i.e., development or configuration);

· Making improvements or changes in response to feedback collected from e-CAS users via regular surveys;

· Ensuring data interfaces are adequately supported and maintained to support EOHHS and Provider and plan data exchange partners;

· Ensuring e-CAS data is high quality, including a plan for correcting Bad Data;

· Ensuring that solution maintenance windows do not interfere with EOHHS business or occur during business hours;

· Providing regular and periodic maintenance to the solution on a schedule agreed upon by the Selected Bidder and EOHHS; and

· Ensuring that solution training materials, manuals and reference guides, and technical documentation are kept up to date throughout the life of the contract.

The M&S plan shall incorporate the following elements:

· Scope;

· Resource Roles and Responsibilities;

· Configuration Management Protocol;

· Change Management Protocol, including Change Request Form as defined in Section 7 of Attachment 1, Statement of Work;

· Levels of Support;

· Key Performance Indicators; and

· Reporting and Management Protocols.

Annual updates due at cadence agreed upon by the Parties; payment incorporated into monthly invoices billed by Vendor