05 - Final PASA SLAs .pdf

Service Level Agreements

Transition Phase Performance Metric Performance

Target Definition Calculation Review

Frequency Service Credit

Transition Phase Work Plan

99% Plan must include how the selected Offeror will transition the current supplier tasks to selected Offeror, how selected Offeror will engage business partners, stakeholders, etc. without disruption of services.

Transition Phase Work plan must be provided within ten business days of after receipt of the fully executed contract.

Ten days after receipt of fully executed contract

Service credit will be calculated at 10% of the deliverable for each day the services are delayed.

Transition Services 99% Transition services within the Transition Phase Work Plan are initiated within one month and fully operational within three months per the deliverables schedule.

Services begin on agreed upon due date.

Review is based on approved timeline.

Service credit will be calculated at 10% of deliverable for each day the services are delayed, beginning the day after the acceptance of the transition plan and agreed due date.

Alignment Studies and Crosswalks

99% The selected Offeror shall complete alignment studies of the content of the pre-established, ready to implement assessment system solution to the .

The content of the assessment solution must be aligned to PA standards in ELA, Math, and Science The selected Offeror shall develop crosswalks to show the alignment between the assessment system solution content and PA core standards for ELA and Math, and PA STEELS (Science, Technology, Engineering, and Environmental Literacy) for Science.

Services begin on agreed upon due date.

Review is based on approved timeline

Service credit will be calculated at 10% of the deliverable for each day the services are delayed, starting from the agreed-upon service commencement date.

Item & Test Development, Scoring and Reporting

Target Definition Calculation Review

Frequency Service Credit

Documentation of Item Characteristics

99% Documentation must show items follow principals of Universal Design and provide broad range of accommodations. Items must be shown to align to the PA Core Standards in ELA and Math and STEELS in Science. In addition, items should vary in difficulty and depth of knowledge, and items sets should be computer adaptive based on the student’s performance. The item bank must be available starting in year one of the contract with adequate coverage and established technical quality of items. The development of items in the existing item bank and in the future must adhere to a strict plan of development and review consistent with principles of technically sound item development. Ongoing test development activities must be documented and described.

Services begin on agreed upon due date.

Review is based on approved timeline.

Service credit will be calculated at 10% of the deliverable for each day the services are delayed, starting

Scoring of Assessment

99.9% Documentation must describe how assessments employ automated scoring.

Documentation must also describe the Diagnostic Classification model and how performance levels and standards setting are derived.

Services begin on annual agreed upon due date.

Review is based on annual approved timeline.

Service credit will be calculated at 10% of the deliverable for each day the services are delayed, starting

Reporting of Results 99.9% An electronic file of assessment results must be provided to the State for review and corrections withina timeline that aligns with the state’s accountability reporting deadlines and an agreed upon date.

Final electronic score reports will be available within three weeks after State approval of the electronic file format of assessment results.

Printed score reports must be provided one month from the delivery of the final electronic file of assessment results and shall be to the educational entities agreed up in the Project Plan. Project Plan to include the quantity and format of the score reports (i.e., black/white or color and district/service provider).

Services begin on annual agreed upon due date

Review is based on annual approved timeline

Service credit will be calculated at 10% of the deliverable for each day the services are delayed, starting

Website, Application Programming Interface, and System Service Level Agreements

Target Definition Calculation Review

Frequency Service Credit

System Availability 99.9% The percentage of time full system functionality is available during any quarter. System availability excludes approved maintenance time.

A=(T-M-D)/(T-M) x 100% A = Availability T = Total Quarterly Minutes M = Approved Quarterly Maintenance Time D = Downtime

Quarterly For System Availability between 99.89% - 99.85% - the Service Credit will be 2.5% of the quarterly SaaS subscription.

For System Availability between 99.84% - 99.80% - the Service Credit will be 5% of the quarterly SaaS subscription.

For System Availability <99.80% - the Service Credit will be 10% of the quarterly SaaS subscription.

Non-Degradation of System

98% The percentage of time the System is non-degraded. Degradation means the System is fully operational but functioning below the baseline established during acceptance testing or using the baselines below. This includes, but is not limited to, slow performance or intermittent system errors or both.

Degraded System means a system that falls below the following targets, excluding approved maintenance time:

• System response time – the time required to complete an action within the application (e.g. load a page, save a record). The standard shall be set during baseline acceptance testing.

N=(T-M-S)/(T-M) x 100% N = Non-degradation T = Total quarterly minutes M = Approved maintenance time S = Time service is degraded

Quarterly The Service Credit will be 5% of the quarterly SaaS subscription when the Service performance standard is not met.

Target Definition Calculation Review

Frequency Service Credit

• Reports response time – the time required to generate a completed report. The standard shall be set during baseline acceptance.

• Concurrent use – the system sustains full functionality with simultaneous access by at least 40% of all registered users.

Incident Management - Notification Timeliness

100% of the incidents reported within 30 minutes

An Incident notification shall be considered timely if the notification was provided within 30 minutes from the time of discovery of Degradation or an Outage to the system.

= ( Total number of incidents requiring notification - Total number of incidents not provided timely ) / Total number of incidents requiring notification )

Continual The Service Credit will be 2.5% of the quarterly SaaS subscription per incident.

Incident Management – Resolution Communication Timeliness

1-Urgent Priority within 2 hours 2-High Priority less than 4 hours 3-Standard Priority less than 8 hours 4-Low Priority less than 24 hours

When an incident occurs, the Contractor must present an issue mitigation strategy to PDE within the Performance Target time specified by the priority of the problem

Time from incident occurrence until incident reporting.

Continual The Service Credit will be 2.5% of the quarterly SaaS subscription per incident.

Target Definition Calculation Review

Frequency Service Credit

Incident Management - Resolution Timeliness

1-Urgent Priority within 2 days 2-High Priority within 4 days 3-Standard Priority within 7 business days 4-Low Priority within week

The time required for circumvention or solution after an incident is reported to Contractor based on priority level. This service level measures the time to resolve each Incident expressed.

Time from incident occurrence until incident is circumvented.

Continual The Service Credit will be 2.5% of the quarterly SaaS subscription per incident.

System Unavailability Notification within two (2) hours

Notification to PDE, of any non-scheduled system unavailability within two (2) hours of incident occurrence.

Time from incident occurrence until system unavailability notification is sent to

PDE.

Continual The Service Credit will be 2.5% of the quarterly SaaS subscription per incident.

Scheduled Maintenance Notification

100% Notification of scheduled maintenance to PDE no less than 14 days prior to scheduled maintenance window.

Number of days prior to scheduled maintenance that notification was sent to

PDE.

Quarterly The Service Credit will be 0.5% of the quarterly SaaS subscription per incident.

System Support Availability

99% Application Desk Support must be available during the following times:

• Monday through Friday 8:00 AM to 6:00 PM Eastern Time (EST or EDT as applicable)

• Technical support includes full management on behalf of PDE Technical support must assist users in ensuring that students are fully enrolled, and educators have access to training materials, resources, and

Total time Application Desk Support unavailable during operating hours / Total operating hours commitment

Quarterly

The Service Credit will be 5% of the quarterly SaaS subscription when the Service performance standard is not met.

Target Definition Calculation Review

Frequency Service Credit tests. Technical support will include completing actions within the platform on behalf of the user as appropriate.

System Support Response to Methods of Contact Available

80% of contacts responded to in 15 minutes or less

Documentation must provide for the nature of support requested, measure of average length of time from receipt of email, voicemail, or other method of contact (excluding phone call) to the time a responding email is sent or return call is placed.

Responses to requests for assistance must be answered during within one business day.

N=(T-M-D)/(T-M) x 100% N= Non-degradation T= Total quarterly minutes M= Approved maintenance time D= Time service is degraded

Quarterly The Service Credit will be calculated at 1% of the quarterly SaaS subscription per incident.

Data Quality and Transfer

99.9% The data system must allow for data uploads and corrections and must have on-demand quality and status reports available for download.

Monthly status reports Monthly Service credit will be calculated at 2% of the deliverable for each day the services are delayed.

Training, Support, and Resources

Target Definition Calculation Review

Frequency Service Credit

Professional Development Training

99.9% Professional training and supporting resources developed for educators and parents to understand the assessment design, instruction, and test administration procedures.

Training must be provided in an accessible format, allow for in person and/or virtual access, and include recorded demonstrations.

The system must allow for tracking and reporting of educator participation in required trainings.

The system must provide opportunities for students and teachers to practice administration in advance of testing.

Professional development and supporting resources should allow for review by the State prior to posting.

All training and resources necessary to fulfill full test administration implementation requirements must be available electronically from the start of the contract.

Additional training and support, including online and or/on-site will be determined between PDE and vendor based upon need.

4 weeks prior to all trainings

Service Credit will be calculated at 2.5% for each day past the scheduled review frequency.

Supporting Resources and Interpretive Guide

98.0% The Offeror must provide supporting resources including manuals, recorded demonstrations, interpretive guide to score reports, etc. made accessible and available on the public website.

Resources must be made available for review by PDE prior to posting.

Ongoing available resources on the public website with annual updates made as necessary.

Quarterly Service Credit is calculated at 2.5% for each day past the scheduled review frequency.

Online Resources 98.0% The Offeror must provide online resources that help parents and

Ongoing available online resources on the public

At least annually or

Service Credit is calculated at 2.5% for each day past

Training, Support, and Resources

Target Definition Calculation Review

Frequency Service Credit educators understand the purpose, design and scoring of the assessment.

The online resources must be made available for review by PDE prior to posting.

website with annual updates made as necessary.

as agreed upon by

PDE.

the scheduled review

Psychometric Documentation and Services

Target Definition Calculation Review

Frequency Service Credit

Psychometric Services/Documentation

98.0% The selected Offeror must provide PDE with technical documentation as part of the test administration and evaluation process. Technical documentation should include, but is not limited to supported purposes, a description of the psychometric background of the diagnostic classification model, test development procedures, validity, reliability, accommodations, and testing of students with the most significant cognitive disabilities in the areas of ELA, Mathematics and Science including data associated with the federal one percent threshold requirement.

Specifically, the selected Offeror shall propose an annual technical manual/reports of state level participation and performance data to be delivered by a date agreed upon by PDE and the selected Offeror each year that is additive in nature. In its first-year documentation should include the psychometric and technical qualities and characteristics of each assessment from item development

Ongoing available resources on the public website with annual updates made as necessary

Annual Service Credit is calculated at 2.5% for each day past

Project Management Performance Metric

Performance Target

Definition Calculation Frequency of Review

Service Credit

Project Plan 99% The Project Plan contains the details and schedule for full project management, development, and implementation of the assessment

Continuously and per approved Project Plan.

Within 30 calendar days of the

Continuous and per approved Project Plan

Service credit will be calculated at 10% of the deliverable for each day the project plan is to administration and scoring/reporting. This technical manual/reports will consist of multiple volumes and various formats, will be posted publicly, and will meet the requirements of any PDE peer review submission. In subsequent years it shall provide the information necessary to build a clear understanding of changes in the testing program across years, making the technical manual information actionable from year to year and useful in making improvements to PASA testing over time.

Technical Advisory Committee (TAC)

The Offeror must provide for a TAC meeting of national experts on measurement and alternate assessment at least one time per year. In addition, the Offeror must participate in a PA specific TAC meeting annually or more frequently if necessary.

Presentations for PA TAC meetings submitted at least 4 weeks in advance of meeting.

Annually Service Credit is calculated at 1%for each week past frequency

Metric

Performance Target

Definition Calculation Frequency of Review

Service Credit system including procedures, training schedule, technology supports, deliverables, data exchange timeline, quality assurances, communication plans, data flow diagrams and security profile and project management team.

Project Management Team documentation shall include project manager and additional team roles and responsibilities. Responsibilities of the Project Manager include providing annual detailed project plan and schedule, project deliverables, quality assurance and communication plans, data flow diagrams and security profile, proposed training schedule, monthly status reports, ongoing communication with PDE, at least biweekly conference calls as scheduled with PDE monitoring of timelines and subcontractors and technical assistance to PDE as requested.

contract award, the selected Offeror must be in contact with PDE leadership to establish a meeting date(s) for short and long-term planning purposes.

Within 45 days the Project Plan must be developed, provided to PDE for review and finalized.

delayed.

Change Order Response Time

10 days The number of days from the time a Contractor receives a change order until the full Contractor response is received by the PDE.

Number of business days greater than 10 for change order Response

Quarterly .5% of Quarterly SaaS subscription per day past 10 business days (maximum of 3% quarterly SaaS Subscription per quarter.)

Metric

Performance Target

Definition Calculation Frequency of Review

Service Credit

Change Order Cost Breakdown

Every change order

All change orders must include an itemized price structure.

Number of change orders denied without a cost breakdown.

Quarterly The Service Credit will be 1% of the quarterly SaaS subscription for each change order proposed without an itemized price structure. The maximum Service Credit for this SLA is 3% of the quarterly SaaS Subscription per quarter.

Definitions

1. Corrective Action Plan: If any of the SLA metrics defined in this document are not met, PDE, at its sole discretion may require the Contractor to develop a Corrective Action Plan (CAP).

2. Incident: An unplanned interruption to a service or a reduction in the quality of service, i.e., a page not rendering properly or allowing a user access to a page that should be inaccessible.

3. Maximum at Risk Amount: a defined percentage of monthly service fees or a total dollar value which is not exceeded when assessing service credits.

Maximum at risk amount is 20% of the quarterly invoice amount.

4. Problem: A cause or potential cause of one or more incidents.

5. Service Credits: means the financial credits earned by the Commonwealth for Contractor Service Level Defaults for Performance Targets. Service

Credits will be assessed in accordance with Paragraph 64. Liquidated Damages of Appendix A, IT Contract Terms and Conditions, except that the Parties agree the Service Credits are not susceptible to recoupment as outlined in that Paragraph 64. With the exception of the deliverables outlined in this Appendix C, all deliverables specified in the RFP are considered “Major Deliverables” susceptible to liquidated damages in accordance with Paragraph 64.

6. SLA Verification: The Department has the right to audit or have an independent entity validate operations and SLAs at least annually but may initiate an audit at any time. The Contractor shall cooperate with the audit and make all documentation available to the auditing entity. The Contractor shall provide the Department with system access to view all information scanned and attached to the application records. The service levels herein shall be measured (and credits applied, to the extent applicable) per calendar quarter.

7. SRS: is the State Routing Service Software provided by Tetrus, the Contractor for the RxCheck hub. This software is used to encrypt and route the PDMP request to the proper state being requested.

8. System: The entire SaaS solution including hardware, software and network infrastructure. SaaS is defined as Software as a Service.

9. Workaround: A solution that reduces or eliminates the impact of an incident or problem for which a full resolution is not yet available. Some workarounds reduce the likelihood of incidents.

Notes Priorities are defined as follows:

1 - Urgent Priority = Hosted system or major functionality of system is not available or operational for all users 2 - High Priority = Hosted system or major functionality of system is not available or operational for multiple users 3 - Standard Priority = Minor function of hosted system is not operational for multiple users, but all other functionality is available and operational for users 4 - Low Priority = Minor function of hosted system is not operational for single user, but all other functionality is available and operational for users

Quarters are defined as follows:

• January 1 through March 31

• April 1 through June 30

• July 1 through September 30

• October 1 through December 31