Attachment 12 - Application Management SLAs _20110215_.pdf

H92222-10-R-0046 Attachment 12

Performance Measurement SLAs

15 Feb 2011

Introduction

The Customer will evaluate contractor performance in accordance with the Service Level Agreements (SLA) contained in Appendix A to this attachment. The Customer’s evaluation will be used to calculate the Contractor’s eligibility for option year Period of Performance (PoP).

Performance Incentive Calculation

The Customer will measure and assess Contractor performance each month to determine the service level achievements performance against the Minimal Acceptable SLA performance criteria, assign a score to each SLA, and develop a monthly performance score by applying weights to each SLA score. The Customer will use the cumulative total of all months for the evaluation period to determine the Contractor’s eligibility for option year Period of Performance (PoP).

If the Contractor exceeds the Target Criteria level of performance stated in block 4B of the SLA the Customer will assign a score “1” for the SLA. If the Contractor meets the Minimum Acceptable level in Block 4C of performance the Customer will assign a score of “0” for the SLA. If the Contractor fails to meet the Minimum Acceptable level of performance the Customer will assign a score of “-1” for the SLA.

After the Customer has completed the monthly assessment of the SLAs the Customer will determine SLA Area scores and monthly performance score using the weights in Appendix B. The SLA Area Score will be the sum of each SLA score within the SLA Area multiplied by the weights assign to the respective SLAs. The monthly performance score will be the sum of each SLA Area score multiplied by the weights assign to the respective SLA Areas.

For each evaluation period, the Customer will assess the Contractor’s SLA performance achievement using the monthly performance scores determined. Each Evaluation Period Score will be the sum of the monthly performance scores for all months within the evaluation period.

F-1 Application Management – Service Improvement Plan

1. SLA SUMMARY

1A. TASK AREA Application Management 1B.

PERFORMANCE

CATEGORY

Quality

1C. SLA # F-1 1D. SLA NAME Service Improvement Plan

2. SLA OVERVIEW

2A. SLA DESCRIPTION IT service performance must be maintained at prescribed levels.

Unsatisfactory levels of performance or identified opportunities for improvement require the development and implementation of Service Improvement Plans (SIPs). This metric monitors the effectiveness of the Application Management Contractor in keeping relevant services optimized.

Service Improvement areas are identified and Improvement Plans are developed, which must include the performance measurement method and target criteria and submitted to the ITMO Lead for approval.

SIPs can also be used as a means to evolve service delivery concepts in the case of insertion of new best practices, new external standards releases, as a result of research or other lessons learned from external projects and experiences. SIPs are also used as the vehicle for major coordination of changes that are the results of proactive problem management identified solutions that must be coordinated across IT service providers.

Government-approved SIPs are implemented and managed as a project for risk, cost, schedule, progress towards service improvements, and achieving SIP defined metric goals according to schedule and plan. The SIP mechanism is used whenever service improvements or corrective modifications to services are needed. Examples include improving process compliance by modifying process to handle reoccurring exceptions or removing impediments to service area provider process compliance when in the best interest of USSOCOM as demonstrated by SIP plan approval.

2B. PERFORMANCE PERIOD This SLA is in effect on a continuous basis without interruption throughout the Contract Period of Performance.

3. SLA METRICS

3A. MEASUREMENT

INTERVAL

The Measurement Interval is 1month.

3B. MEASUREMENT PERIOD The Measurement Period begins at 00:01 on the first day of the month and ends at 24:00 on the last day of the month.

3C. SOURCE OF

MEASUREMENT DATA

Project status and review documents for SIPs.

3D. METHOD OF

SURVEILLANCE

100 Percent Inspection. Monthly report on SIPs.

3E. TIMING OF

MEASUREMENT

Measurements are taken within 5 calendar days of the end of the Measurement Period.

3F. EXCEPTIONS Field left blank intentionally.

4. SLA MEASUREMENT

4A. CALCULATION a) Number of open SIPs meeting review criteria reviewed in the last review period must be greater than zero.

b) Percentage of Open SIPs Reviewed = Number of open SIPs meeting the specified review criteria that are reviewed in the last three review periods divided by total number of open SIPs meeting the specified review criteria.

c) When SIP action item criteria exceed a +\- 10% variance from the criteria specified in the approved SIP the days until escalation is calculated as the elapsed time in days from the Julian date the variance is escalated to ITMO leadership minus the Julian date the review concludes.

4B. TARGET CRITERIA 100% of SIP Action Items and Milestones are reviewed for costs, schedule and risks. Those where cost/resources, or the targeted service level benefits according to SIP achievement schedule are outside of 10% variance and/or at risk of failure, are escalated within the ITMO for action within 2 business days of review.

4C. MINIMUM ACCEPTABLE 90% of SIP Action Items and Milestones are reviewed for costs, schedule and risks. Those where cost/resources, or the targeted service level benefits according to SIP achievement schedule are outside of 10% variance and/or at risk of failure, are escalated within the ITMO for action within 5 business days of review.

4D. DEFINITIONS None

5. SLA ADMINISTRATION

5A. REPORTING FREQUENCY 1. Reporting to commence upon Contract Award and continue throughout the Period of Performance.

2. Monthly reporting of SLA attainment.

5B. NOTES AND COMMENTS None

F-2 Application Management – Accuracy of Configuration / Asset Management Records

1A. TASK

AREA

Application Management 1B. PERFORMANCE

CATEGORY

Accuracy & Quality

1C. SLA # F-2 1D. SLA NAME Configuration, Asset, and Reported Accuracy

2. SLA OVERVIEW

2A. SLA DESCRIPTION IT Service Area Contractor monitored changes to IT Services supporting asset and configuration item (CI) attributes are controlled and tracked, with all system configuration and asset data promptly and accurately recorded reflecting the as-installed configuration of the infrastructure CIs within each IT Service Area provider’s area and its associated lifecycle (asset) data.

For each IT Service Area provider, validation checks are performed on a randomly selected set of CIs recorded in the Configuration Management Database (CMDB) of record and compared to the actual as-installed configuration of each CI in the randomly selected data set. Both CI and Asset attributes are validated.

Asset and configuration accuracy reports are generated and reviewed by the ITMO with the Contractor to identify where and how the configuration management process failed and determine corrective action to be taken.

2B. PERFORMANCE

PERIOD

This SLA is in effect on a continuous basis without interruption throughout the Contract Period of Performance for all baselined CIs and assets.

3. SLA METRICS

3A. MEASUREMENT

INTERVAL

The Measurement Interval is a continuous process with CI accuracy validation checks conducted at ITMO-selected periodic intervals within the monthly measurement period, as well as conducted whenever it is determined that the root cause of implementation failure of an authorized Request for Change (RFC) failed.

3B. MEASUREMENT

PERIOD

The Measurement Period begins at 00:01 on the first day of the month and ends at 24:00 on the last day of the month.

3C. SOURCE OF

MEASUREMENT

DATA

BMC CMDB, BMC Configuration Exception and Reconciliation Reports, Microsoft SCCM Scan Reports, Change and Release records.

3D. METHOD OF

SURVEILLANCE

Random Sampling. Sampling will occur by ITMO of Application Management Contractor configuration and asset accuracy reporting. The ITSM Contractor will review the accuracy of Application Management Contractor managed configurations and assets using reconciliation and physical inventory. The ITMO will develop and assign SIP actions to the ITSM Contractor if accuracy of reporting falls below the minimum accuracy used in this SLA or to the Application Management Contractor if relevant asset or CMDB accuracy falls below Minimum Acceptable levels.

A performance level baseline will be used by CI and asset type. At USSOCOM’s sole discretion, a determine when the data regarding a service area, grouping of CIs or assets within a service area is stable enough to be baselined and the responsible IT Service Area provider’s ability to maintain configuration and asset accuracy be measured. After a service or system transition, USSOCOM may or may not remove baseline status at their discretion for items until data or process to establish baseline or measure is stable. (Examples: During initial operating capability (IOC), after a major COOP event, or during re-locations).

When a new asset is acquired or a change is made to an existing asset, the IT service provider performing this work shall be required to enter or update the asset and configuration records with accurate data for each CI, which will become the baseline for that asset. Until a CI or asset is baselined, SLA achievement will consider the minimum being met for that CI or asset or it will be excluded from the SLA calculation.

3E. TIMING OF

MEASUREMENT

Performed on an ongoing basis throughout the contract term once items are baselined.

3F. EXCEPTIONS Field left blank intentionally.

4. SLA MEASUREMENT

4A. CALCULATION For IT Service Provider accuracy of CI and Assets:

• Quantity of reviewed CIs having inaccurate, incomplete, or improperly documented CI or asset attributes divided by the total number of CIs baselined by IT service provider area.

4B. TARGET

CRITERIA

The following targets are established for a CI and for asset accuracy.

IT Service Provider Within 12 Mo of Initial Baseline

Within 24 Mo of Initial Baseline

>24 Mo after Initial Baseline

Networks, Data Center, Distributed Computing, Applications

<=15% CI

inaccuracy

<=10% CI

inaccuracy

<= 5% CI

inaccuracy

<=15% asset inaccuracy

<=10% asset inaccuracy

<= 5% asset inaccuracy

4C. MINIMUM

ACCEPTABLE

The following minimums are established for a CI and for asset accuracy.

IT Service Provider Within 12 Mo of Initial Baseline

Within 24 Mo of Initial Baseline

>24 Mo after Initial Baseline

Networks, Data Center, Distributed Computing, Applications

<=25% CI

inaccuracy

<=15% CI

inaccuracy

<= 10% CI

inaccuracy

<=25% asset inaccuracy

<=15% asset inaccuracy

<= 10% asset inaccuracy

4D. DEFINITIONS Asset allocation usage is defined as the total number of discrete hardware items, software licenses, user accounts, etc. that form the basis of determining total unit counts pertaining to overall resource utilization within each IT Service Area (e.g., number of valid individual personnel Active Directory user accounts, number of installed servers, amount in gigabytes of installed storage, number of routers, firewalls, number of software licenses procured and in use, etc.).

5. SLA ADMINISTRATION

5A. REPORTING

FREQUENCY

1. Reporting to commence upon Contract Award and continue throughout the Period of Performance.

2. Monthly reporting of SLA attainment.

5B. NOTES AND

COMMENTS

The CI attributes that are validated should include at a minimum: Name, Description, Custodian/Steward, Criticality, Type, Manufacturer, Serial number, Support Provider, License information, Version, History, Date of Purchase, Date of Install, Current status, Relationships to IT Services and CIs, how used, Relationships to other service management data objects (incident, problems, changes, etc.), documentation location (operating, contractual, User, COOP, etc.)

Once an enterprise asset management process is established, lifecycle data is captured during procurement/ installation activities (IMAC) and maintained for at least the following: asset location, asset tag, asset serial number, inventory quantity, purchase date, purchase date, cost and warranty/ contract expiration dates.

These are then sampled from the data stores that comprise the CMDB and asset databases and compared to discovery and audit data.

Method of reconciliation is defined as when a sampling criterion is determined and BMC scan runs against SCCM formerly SMS are other discovery/ scanning data sources. Data is parsed by the tool into two records and compared line for line in the CMDB. The exceptions are then presented in a report/ screen. On the screen Contractor or IT service provider will need to check which fields to use – current CMDB or scan value. It forces reconciliation via the procedures. Note this is the tool the IT Service Area providers will use to clean their data. Not every run of this process is an SLA report, only a monthly run. IT Service Area providers are going to likely run this weekly and will need a day or so prior to SLA report run to clean up where they can. Also, this procedure is used to identify “leaks” where configuration drift is occurring and fix root cause of inaccuracy.

These are then sampled from the data stores that comprise the CMDB and asset databases and compared to discovery and audit data.

The method of reconciliation is defined as when a sampling criterion is determined and BMC scan runs against SCCM formerly SMS are other discovery/ scanning data sources. Data is parsed by the tool into two records and compared line for line in the

F-3 Application Management – Changes and Releases without Incident

CMDB. The exceptions are then presented in a report/ screen. On the screen the Contractor or IT service provider will need to check which fields to use – current CMDB or scan value. It forces reconciliation via the procedures. Note that this is the tool the IT service providers will use to clean their data. Not every run of this process is in a SLA report, only a monthly run. IT service providers are going to likely run this weekly and a day or so prior to SLA report run to clean up where they can. This procedure is also used to identify “leaks” where configuration drift is occurring and fix root cause of inaccuracy.

These targets take into account the maturity and ability to capture configuration and asset information by area.

1. SLA SUMMARY

1A. TASK AREA Application Management 1B.

PERFORMANCE

CATEGORY

Quality and Accuracy

1C. SLA # F-3 1D. SLA NAME Changes / Releases without Incidents

2. SLA OVERVIEW

2A. SLA DESCRIPTION Changes and Releases must be introduced into the infrastructure without unscheduled service disruption. The Incident Management system will be scanned to identify all Incident tickets classified as Priority #1 and #2 that have occurred within the Measurement Interval. Details of each selected Incident ticket will be reviewed and compared to RFC Release Management tracking records to determine if there is any direct correlation between any changes introduced up until the previous release cycle or 30 calendar days (whichever is shorter in duration) prior to an incident occurrence. Each such event correlation will be considered as a deficiency in the Change and Release Management processes and included as a “Failed Change” for SLA calculation. Also, the Incident will be escalated to Problem Management to perform Root Cause Analysis (RCA) to determine the underlying cause and recommend corrective action.

2B. PERFORMANCE PERIOD This SLA is in effect on a continuous basis without interruption throughout the Contract Period of Performance.

3. SLA METRICS

3A. MEASUREMENT

INTERVAL

The Measurement Interval is weekly.

3B. MEASUREMENT PERIOD The Measurement Period begins at 00:01 on the first day of the month and ends at 24:00 on the last day of the month.

3C. SOURCE OF

MEASUREMENT DATA

Measurement data will be sourced from the Service Desk Incident Management system, the Change Management request for change (RFC) record repository, and the Release Management record repository.

3D. METHOD OF

SURVEILLANCE

Periodic Inspection. Monthly reporting of SLA attainment providing the results of failed change analysis. The Government ITMO personnel will periodically inspect service desk data for a reporting period(s) to ensure / validate accurate reporting.

3E. TIMING OF

MEASUREMENT

Measurements are taken within 7 calendar days of the end of the Measurement Period.

3F. EXCEPTIONS Field left blank intentionally.

4. SLA MEASUREMENT

4A. CALCULATION The number of Failed Changes divided by the total number of approved RFCs introduced into a production system within the measurement period.

4B. TARGET CRITERIA 0% of introduced RFCs are determined to have caused a Priority 1 or Priority 2 Incident.

4C. MINIMUM ACCEPTABLE ≤ 5% of all introduced RFCs are determined to have caused a Priority 1 or Priority 2 Incident.

4D. DEFINITIONS None.

5. SLA ADMINISTRATION

5A. REPORTING FREQUENCY 1. Reporting to commence upon Contract Award and continue throughout the Period of Performance.

2. Monthly reporting of SLA attainment.

5B. NOTES AND COMMENTS None.

F-4 Application Management – Staffing Management Timeliness

1A. TASK AREA Application Management 1B.

PERFORMANCE

CATEGORY

Quality / Availability

1C. SLA # F-4 1D. SLA NAME Staffing Management Timeliness

2. SLA OVERVIEW

2A. SLA DESCRIPTION The Contractor staffs all key personnel, as identified in the Contractor contract organization documents, as may be amended from time to time by customer, with personnel having appropriate skills, training, and experience to meet the job requirements in a timely manner.

2B. PERFORMANCE PERIOD This SLA is in effect on a continuous basis without interruption throughout the term of the task order agreement.

3. SLA METRICS

3A. MEASUREMENT

INTERVAL

Measurements to be taken on a rolling daily calendar day basis for each unfilled position.

3B. MEASUREMENT PERIOD The Measurement Period begins at 00:01 on the first calendar day of the month and ends at 24:00 on the last calendar day of the month.

3C. SOURCE OF

MEASUREMENT DATA

The customer will create, maintain, and update a key personnel log where the Contractor will, at minimum, record and track key position titles and responsibilities, date that the position was initially established, date that the key position was staffed, name of the individuals filling the key position at any given period of time, the date that (for any reason) the key position vacated, and the cumulative elapsed time the position is vacant.

3D. METHOD OF

SURVEILLANCE

100 Percent Inspection. Monthly report on SLA attainment providing length of elapsed time to fill vacant key positions with qualified and trained staff from date of initial vacancy. The Customer will conduct full reviews of all Contractor key personnel vacancies and the timely filling of open key personnel positions.

3E. TIMING OF

MEASUREMENT

Measurements are taken monthly on a per-position vacancy basis commencing no more than 24 hours following the date a position is created and unfilled or vacated by Contractor staff member.

3F. EXCEPTIONS Field left blank intentionally.

4. SLA MEASUREMENT

4A. CALCULATION Total elapsed time in calendar days to fill an open key position from date position is initially vacated until qualified appropriately cleared personnel are working onsite is calculated as:

• The Minimum Service Level shall be obtained if the total of calendar days to fill position <= 28 calendar days ; and

• The Target shall be obtained if the total of calendar days to fill position <= 14 calendar days applicable to all open key positions during the measurement period.

4B. TARGET CRITERIA Total elapsed time of not more than 4 calendar weeks (i.e., 14 calendar days) to fill any given key position vacancy with appropriately cleared qualified personnel.

4C. MINIMUM ACCEPTABLE Total elapsed time of not more than 8 calendar weeks (i.e., 28 calendar days) to fill any given key position vacancy with qualified and appropriately cleared qualified personnel.

4D. DEFINITIONS None.

5. SLA ADMINISTRATION

5A. REPORTING FREQUENCY 1. Reporting to commence upon Task Order Award and continue throughout the Period of Performance.

2. Monthly reporting of SLA attainment with report provided to the Customer within 5 working days of the end of each calendar month reporting period.

5B. NOTES AND COMMENTS None.