Attachment A Appendix H OTS Summary Scope__20220209.pdf

Massachusetts Health Insurance Exchange Program, Project Management and Testing Services Request for Responses 1 Attachment A, Model Contract Appendix H. OTS Summary of Scope

Massachusetts Health Insurance Exchange Program Project Management and Testing Services

Request for Responses

Appendix H

Operations and Technical Services

Summary of Scope

Massachusetts Health Insurance Exchange Program, Project Management and Testing Services Request for Responses 2

Operations and Technical Services

Summary of Scope

General

The OTS vendor will have primary responsibility for implementing new releases, providing technical support, maintaining and operating the MA HIX Application, operating and maintaining the MA HIX/IES System monitoring and operations center, and supporting the interfaces between the MA HIX/IES System and other systems.

Methodology

The OTS Vendor will utilize the EOHHS-provided tools as described in RFR Appendix C. The OTS Vendor will follow all industry standards and additional legal and contractual standards and requirements pertaining to their scope of work.

Deliverables and Documentation

The OTS Vendor is responsible for implementation and technical services, and for ongoing operation of the MA HIX/IES System. The OTS vendor shall provide documentation and reports which allow for the tracking and monitoring of the MA HIX Application.

Implementation and Technical Services

The OTS Vendor shall, for implementation and technical services, provide the necessary information to allow the PMT Vendor to monitor and report on the actions taken by the OTS Vendor, the Infrastructure Vendor, or the MA HIX Application Vendor, as appropriate, and:

• Conduct Non-Production Application Environment Operations, including: executing ad hoc or scheduled batches, loading production-like environments for production-like testing, loading or refreshing non-production environments to a designated data baseline, managing vendor software and enterprise service bus updates and patching in all non-production environments, troubleshooting and resolving intra-system connectivity and data discrepancy issues, and operating and maintaining the non-production environments as required to support the System and related project activities;

• Support reporting and queries, including: developing and executing scheduled and ad hoc queries in all non-production environments, and designing, developing, deploying and supporting Commonwealth- and HIX Vendor-authored queries, reports, and data extracts (both scheduled and ad hoc) for production;

• Execute data extracts and reports against production data sources and perform related supporting analysis;

• Coordinate with the MA HIX Application Vendor and Infrastructure Vendor to support updates to the Daily Data Extract as initiated through any of the MA HIX SDLC processes;

Massachusetts Health Insurance Exchange Program, Project Management and Testing Services Request for Responses 3

• Provide support for Commonwealth UAT testing activities by changing and reverting system configurations as required by test plans, troubleshooting unanticipated system responses, and assisting the PMT vendor and the Commonwealth in developing test approaches, as needed;

• Collaborate with the MA HIX Infrastructure Vendor to maintain and enhance the QA Tool;

• Execute ad-hoc batch and query-driven projects, e.g., initial runbook preparation for batch processing;

• Coordinate with the Commonwealth’s batch technology vendor (currently Argent) including creating and maintaining batch job scheduler and alerts;

• Maintain and support middleware technologies (e.g., API Gateway, File Transfer, MQ, Axway);

• Perform deployment of code into production for all MA HIX releases including the distribution of updates/upgrades (e.g., new releases, versions, service packs, E-patches);

• Adhere to an established release and distribution processes so that any Service change to the MA HIX is controlled, tested, authorized and implemented in a structured, consistent and repeatable manner; and

• Baseline each release against the previous production release to identify any negative performance impacts (excluding performance testing execution activities).

The OTS Vendor shall, for Security-related activities, provide the necessary information to allow the PMT Vendor to monitor and report on the actions taken by the OTS Vendor, the Infrastructure Vendor, or the MA HIX Application Vendor, as appropriate, and in conjunction with the MA HIX Application Vendor and Infrastructure Vendor:

• Provide technical services in support of the MA HIX/IES security management program, including risk, and compliance support activities;

• Provide ongoing technical services to the MA HIX Application Vendor and MA HIX Infrastructure Vendor in support of FTI Data Protection activities. Facilitate and ensure FTI data protections are implemented;

• Provide technical support for security testing of release changes performed by the MA HIX Application Vendor as required;

• Provide technical support for bi-annual penetration testing as directed by the Commonwealth and in collaboration with the MA HIX Infrastructure Vendor;

• Support the Commonwealth’s vulnerability triage activities; and

• Support inflight activities including but not limited to access management reporting enhancements, database activity monitoring (DAM) enhancements, & operational security issues.

Operations

• Monitor for MA HIX Application health and availability as agreed to by the Parties;

Massachusetts Health Insurance Exchange Program, Project Management and Testing Services Request for Responses 4

• Adhere to the approved detailed written descriptions of alerting, notification and response procedures as defined in the Operations Run Book and other related documentation;

• Provide 24 x 7 Level 2 Support and Level 3 Support for Incidents in accordance with PMT Vendor responsibilities, provided that Level 3 Support shall be provided on an on-call basis outside of business hours only for Priority Level 1 and 2 Incidents. Business hours are 7:00 AM to 7:00 PM EST;

• Provide troubleshooting, analysis and resolution of production issues and Incidents;

respond to business queries and ad hoc requests, in accordance with PMT Vendor responsibilities;

• Follow Problem Management practices for all reported defects, perform Root Cause Analysis for all P1 & P2 incidents, support and review Root Cause Analysis findings from 3rd party vendors, and provide results from all to EOHHS for review;

• Perform prompt and frequent progress reporting to EOHHS in accordance with Project Management and SLA standards;

• Monitor and analyze trends in performance and reliability of the MA HIX to identify potential Problems and recommend appropriate actions for EOHHS approval; and

• Utilize applicable ticket management system (currently JIRA).

The following are the tasks that the OTS Vendor will perform and the information that the OTS Vendor will provide to the PMO in connection with the OTS Vendor’s monitoring and management of the MA HIX/IES System.

Corrective and Emergency Maintenance Support

The OTS Vendor shall, for Corrective and Emergency Maintenance efforts, provide the necessary information to allow the PMT Vendor to monitor and report on the actions taken by the OTS Vendor, the Infrastructure Vendor, or the MA HIX Application Vendor, as appropriate, and:

• Work cooperatively and collaboratively with HIX Program Entities to perform scheduled maintenance and emergency changes, including the break/fix activities that enable the MA HIX to provide the required functionality, availability and performance to meet applicable service levels and to comply with applicable Security Requirements;

• Follow Problem Management processes, work cooperatively and collaboratively with the MA HIX Application Vendor to perform root cause analysis for Priority Level 1 and Priority Level 2 Incidents and provide the results to EOHHS to review; and

• In accordance with the PMT Vendor’s responsibilities (including compliance with applicable service levels), work cooperatively and collaboratively with the MA HIX Application Vendor to resolve Incidents affecting the MA HIX Application that require modifications to the database, MA HIX Application code, interface, and/or software configuration.

Preventive Maintenance Support

Massachusetts Health Insurance Exchange Program, Project Management and Testing Services Request for Responses 5

The OTS Vendor shall, for preventive maintenance support, provide the necessary information to allow the PMT Vendor to monitor and report on the actions taken by the OTS Vendor, the MA HIX Infrastructure Vendor, or the MA HIX Application Vendor, as appropriate, and:

• Work cooperatively and collaboratively with HIX Program Entities to perform scheduled maintenance and emergency changes, including the break/fix activities that enable the MA HIX/IES System to provide the required functionality, availability and performance to meet applicable service levels and to comply with applicable Security Requirements;

• Work cooperatively and collaboratively with MA HIX Application Vendor and the MA HIX Infrastructure Vendor to perform preventive maintenance to improve the efficiency and reliability of MA HIX/IES System and minimize ongoing maintenance requirements;

and

• Monitor and analyze trends in performance and reliability of the MA HIX/IES System to identify potential problems and recommend appropriate actions.

Adaptive Maintenance Support

Adaptive maintenance provides support required to adapt the system to a change in the current hardware, operating environment, or volume where no new or changed functionality is required.

The OTS Vendor shall provide the necessary information to allow the PMT Vendor to monitor and report on the actions taken by the OTS Vendor, the MA HIX Infrastructure Vendor, or the MA HIX Application Vendor, as appropriate, and:

• Work cooperatively and collaboratively with MA HIX Application Vendor and Infrastructure Vendor to perform adaptive maintenance for the MA HIX, including identifying, developing, testing and implementing modifications to the MA HIX to maintain usability. This work should be in accordance with the applicable change order process; and

• Perform testing to determine whether performance of the MA HIX/IES System has been affected by security system or policy changes, new or upgrades to existing operating system or third-party software releases, new or changed equipment as required for PMT Vendor to perform the Services.

Application Tuning Support

The OTS Vendor shall, for Application Tuning Support, provide the necessary information to allow the PMT Vendor to monitor and report on the actions taken by the OTS Vendor, the MA HIX Infrastructure Vendor, or the MA HIX Application Vendor, as appropriate, and:

• Work cooperatively and collaboratively with MA HIX Application Vendor to support Application Tuning to the MA HIX Application to improve and maintain performance or maintainability;

• Proactively evaluate, identify and recommend updates to the MA HIX Application and/or operational procedures to enhance performance of the MA HIX Application; and, Massachusetts Health Insurance Exchange Program, Project Management and Testing Services Request for Responses 6

• Suggest, whenever possible, automated supports to replace or augment manual efforts.

Application Quality Assurance Support

The OTS Vendor shall, for Application Quality Assurance Support, provide the necessary information to allow the PMT Vendor to monitor and report on the actions taken by the OTS Vendor, the Infrastructure Vendor, or the MA HIX Application Vendor, as appropriate, and:

• Develop, document, implement and manage QA processes and procedures for the services within the PMT scope as defined in this RFR;

• Work cooperatively and collaboratively with HIX Program Entities to provide, document and perform changes to interfaces between the MA HIX Application and other systems and regularly maintain an updated listing of certified interfaces; and

• Test all MA HIX Application interface changes, and resolve, track and report on compatibility issues.

Interface and Integration Support

The OTS Vendor shall, for Interface and Integration Support, provide the necessary information to allow the PMT Vendor to monitor and report on the actions taken by the OTS Vendor, the Infrastructure Vendor, or the MA HIX Application Vendor, as appropriate and:

• Work cooperatively and collaboratively with HIX Program Entities to maintain integration of the MA HIX with EOHHS equipment and other vendor-, EOHHS- or third party-provided software and resolve compatibility issues, in accordance with PMT Vendor’s responsibilities;

• Work cooperatively and collaboratively to provide and document, and perform changes to, interfaces between the MA HIX System and other systems, in accordance with PMT Vendor’s responsibilities for the MA HIX System. Provide updated listing of certified interfaces;

• Test all MA HIX/IES System interface changes, resolve compatibility issues, and track and report on compatibility issue resolution; and

• Monitor MA HIX/IES System interfaces as necessary to confirm that data transmissions complete successfully; perform data verification and reconciliation as required for troubleshooting reported Incidents.

Systems Operations Support

The OTS Vendor shall, for System Operation Support, provide the necessary information to allow the PMT Vendor to monitor and report on the actions taken by the OTS Vendor, the Infrastructure Vendor, or the MA HIX Application Vendor, as appropriate and:

• Work cooperatively and collaboratively with the Infrastructure Vendor to ensure that infrastructure performance meets or exceeds all applicable performance standards or other requirements;

Massachusetts Health Insurance Exchange Program, Project Management and Testing Services Request for Responses 7

• Work cooperatively and collaboratively with the MA HIX Application Vendor to ensure that the MA HIX Application performance meets or exceeds all applicable performance standards or other requirements;

• Review initially, prior to commencing operations, and semi-annually thereafter during the life of the OTS Contract, the MA HIX/IES System infrastructure and make configuration recommendations to EOHHS so that the MA HIX Application performance meets or exceeds all applicable performance standards or other requirements;

• Define scheduling requirements, interdependencies, EOHHS and Infrastructure Vendor contacts, and rerun requirements for all production jobs in agreement with the EOHHS and Infrastructure Vendor standards and requirements;

• Using the Commonwealth’s scheduling tool (currently Argent, see Appendix C), provide job scheduling, job execution, reporting and resolution, including:

o Daily;

o Weekly;

o Monthly;

o Quarterly;

o Annually; and o Special (this may also include any special runs that need to be coordinated with the

Infrastructure Vendor);

• Implement job scheduling requirements, interdependencies, EOHHS and Infrastructure

Vendor contacts, and rerun requirements for all production jobs;

• Define batch scheduling requirements and job run parameters in agreement with EOHHS and Infrastructure Vendor standards and requirements;

• Prepare batch jobs for execution;

• Execute/Monitor production batch jobs, in accordance with the schedules;

• Provide quality control for reprocessing activities, such as batch reruns;

• Monitor performance of batch cycles, on-lines, and portals to ensure performance standards or other requirements are met (where there are issues, develop and present a corrective action plan to EOHHS);

• Provide security and password support to the end user community; and

• Ensure Backups are properly scheduled and run.

Database Administration Support

The OTS Vendor shall, for Database Administration Support, provide the necessary information to allow the PMT Vendor to monitor and report on the actions taken by the OTS Vendor, the Infrastructure Vendor, or the MA HIX Application Vendor, as appropriate, and:

• As needed for compliance with service levels, work cooperatively and collaboratively with MA HIX Application Vendor and the Infrastructure Vendor to provide database administration and support for the MA HIX/IES System, including: monitoring and

Massachusetts Health Insurance Exchange Program, Project Management and Testing Services Request for Responses 8 analyzing database activity; database performance tuning; maintaining test and production databases; documenting database-related settings, processes and procedures for EOHHS system personnel; and certifying patches and advising whether they are required for EOHHS installations.

Configuration Management Support

The OTS Vendor shall, for Configuration Management Support, provide the necessary information to allow the PMT Vendor to monitor and report on the actions taken by the OTS Vendor, the Infrastructure Vendor, or the MA HIX Application Vendor, as appropriate, and:

• Work cooperatively and collaboratively with the MA HIX Application Vendor and the Infrastructure Vendor to perform configuration management, including the identification, documentation, control, maintenance and verification of configuration items, maintain the configuration management database and report on configuration change.

Root Cause Remediation Support

The OTS Vendor shall, for root cause remediation support, provide the necessary information to allow the PMT Vendor to monitor and report on the actions taken by the OTS Vendor, the Infrastructure Vendor, or the MA HIX Application Vendor, as appropriate to:

• Support root cause analysis;

• Support creation of analysis documentation including what happened and why it happened, and identifying what changes need to be made to prevent it from happening again; and

• Support root cause remediation plan submissions to EOHHS and receive approval from EOHHS before commencing remediations to the system.