Atch 5 - GSM-O QASP 02-09-2011.pdf

HC1028-10-R-2000

Attachment 05

Amendment 0001

Global Information Grid Services Management - Operations (GSM-O)

Quality Assurance Surveillance Plan (QASP)

09 February 2011

Amendment 0001

1 INTRODUCTION

The Global Information Grid (GIG) Services Management (GSM) – Operations (GSM-O) Quality Assurance Surveillance Plan (QASP) will evaluate the performance of the GSM- O contractor in meeting the program’s requirements as described in the GSM-O Performance Work Statement (PWS). The contractor, and not the Government, is responsible for management and quality control actions to meet the terms of the contract. The Government will perform quality assurance and surveillance to verify contract performance standards are achieved.

2 PURPOSE

a. This QASP provides a quality surveillance strategy for the GSM-O contract. The primary intent of the plan is to provide a basis for the Government to evaluate performance quality. Oversight of contractor performance will assure quality performance. The plan will also allow the Contracting Officer and Contracting Officer’s Representative a mechanism to preclude major deficiencies in performance, provide input for annual contractor past performance evaluations, and assist in the decisions pertaining to the exercise of contract options.

b. The GSM QASP is intended to accomplish the following:

• Discuss the process of performance measurement;

• Describe the incentive award plan;

• Establish a governance structure for monitoring performance, comparing actual performance against the performance standards, and authorizing the invoicing (payments) for earned incentives (or the credits to the Government); and,

• Identify the key performance standards and acceptable performance levels that will be used as both a measure of success and as a basis for incentive awards.

This initial draft of the QASP will contain a set of performance standards that will be reconciled with the contractor’s proposed performance standards and subject to mutual agreement between the parties. The consolidated, negotiated standards and measures will be included in the final QASP.

3 GSM-O EXECUTION CONTEXT

a. See Government Furnished Information (GFI) provided as supplemental information with the GSM-O RFP for general background information

b. The Contractor’s work performance shall be monitored and evaluated in accordance with the contract (including the PWS) and task order (TO) requirements. In case of conflict between the contract/PWS and a task order the contract/PWS shall govern. The controls applicable to each active Task Order will be contained in the QASP included within that task order.

c. The GSM-O contract has an ID/IQ structure with a PWS that describes primary requirement areas. Each GSM-O TO specifies the scope and requirements of the PWS that are to be provided, including the specific performance measures and a separate tailored QASP

Amendment 0001

d. For the GSM-O contract, QASP procedures will be applicable at the task order level.

For each task order issued by the Government during each execution year of the contract, the task order will contain its own draft QASP to be used by the Contractor in proposing a responding Quality Control Plan (QCP).

e. PWS Appendix C provides a table of Acceptable Levels of Performance (ALPs) that measure the performance of specific PWS activities. Each ALP establishes the following color-coded performance levels:

• Blue = Performance is Above-Satisfactory to a degree that merits Incentive

• Green = Performance is Satisfactory (no incentive is applicable)

• Yellow = Performance is not yet Satisfactory, but no credit to the Government is applicable

• Red = Performance is sufficiently below the Satisfactory level to justify a credit to the Government

f. Each ALP contains the following structural elements:

• Reference to PWS requirement being evaluated by the ALP

• Performance objective (e.g. “adhere to security policies and procedures to minimize classified “spillage” incidents”)

• Performance Standard for measurement (e.g. “zero spillage incidents per period”)

• Acceptable Level of Performance (e.g. “There shall be zero classified “spillage” incidents.”)

• Performance Ranges (e.g. “Blue = not applicable; Green = zero incidents; Yellow = one incident; Red = two or more incidents”)

• Surveillance Method (e.g. “Frequency: monthly; Surveillance Type: 100% monitoring; surveillance method: data analysis”)

g. The PWS table of ALPs is included in this QASP in paragraph 8 without change as the primary basis for QASP performance evaluation. Each TO may include additional appropriate measures/metrics/incentives and methods of surveillance within its respective QASP

4 WORK REQUIRING SURVEILLANCE

In the QASP for each specific TO, this section will provide applicable information about the tasks of that TO that are subject to incentives management under that QASP.

5 PRIMARY METHODS OF SURVEILLANCE

a. As outlined in FAR Part 46.4, Government Contract Quality Assurance, the QASP specifies the method and location of surveillance to ensure that the services conform to contract requirements. The place of performance will be as designated in the applicable Task Order.

Amendment 0001

b. The primary methods of quality surveillance are addressed below. This list may be revised based on the contractor’s recommendations in a final negotiated QASP.

(1) 100 Percent Data Capture. If the performance of a PWS requirement (such as logging of help desk calls or resolving confirmed troubles) is documented via automated systems that capture descriptive and status data, that data can be used for QASP assessment purposes.

(2) Random or Periodic Sampling or Data Capture. The Government PMO can direct that data associated with the performance of a specific requirement be collected for purposes of QASP assessment, on either a random or periodic basis.

(3) Government Operator/Customer input. The Government can request input pertaining to perceived contractor performance from Government operations staff and from DISN end-users if/when there is direct interaction with the contractor such as in Help Desk calls or in contractor-provided assistance to resolve troubles at an end-user location. (Most GSM-O services are provided to the Government DISN PMO not to DISN end-users per se.)

6 GOVERNANCE

6.1 Roles and Responsibilities

The following roles and responsibilities will be incorporated in the QASP process as documented in the final QASP after award.

6.1.1 Contractor

The contractor will perform the work as described in each GSM-O Task Order and as requested by the Government, provide source data pertaining to task activities needed for evaluation by Government QARs for QASP related procedures.

6.1.2 Government Contracting Officer (KO)

The Government contracting officer will ensure that QASP-related processes, to include the payment of incentive payments as approved by the Incentives Determining Official (IDO) are executed in conformance with the FAR, the terms of the contract, and the final approved GSM-O QASP.

6.1.3 Government Contracting Officer’s Representative (COR)

The COR will act under Contracting Officer direction to assist in the administration of the contract at the contract level, and will coordinate with Task Monitors for each task order as applicable. The COR receives a completed incentive fee recommendation from each operative task order TM, checks that the TM’s recommendation is complete, and ensures that the recommendation is processed appropriately through the KO and IDO.

6.1.4 Government Quality Assurance Reviewer (QAR)

a. For QASP-related processes, the Government will appoint functionally qualified field staff to perform "quality assurance reviewer” (QAR) duties for each task order where QASP incentives are applicable. If a task order is divided for execution on a geographic region basis, the regional Field Commander will appoint functionally qualified field staff to perform QAR duties for QASP incentive actions in that region. Further references in

Amendment 0001 this QASP to a “QAR” shall be interpreted to mean, a functionally qualified field staff appointed to perform QAR duties.

b. QARs will have responsibility for monitoring Contractor performance supporting the applicable regional Field Command, or globally if applicable, collecting applicable ALP source data from the Contractor and other sources,and interpreting that data to compute the performance color level at the end of the evaluation period for each ALP. The QAR also provides additional information related to contractor performance to the TM for consideration in the incentive fee decision process. The QARs are aligned with the TO structure to ensure that contractor performance is monitored and rewarded by Government staff who are best positioned to make the recommended incentive decisions.)

6.1.5 Government Task Monitor (TM)

a. Government policy requires that each task order must have a Task Monitor (TM) who verifies that the requirements of a given specific GSM-O Task Order are being satisfied.

b. In the GSM-O context where multiple TOs are operative, each TM is responsible for coordinating, as may be needed, with other TMs in the same region/Field Command to ensure that QASP procedures are applied properly. However, each TO has its own QASP; incentive fee awards are earned and approved on a by-TO basis and are not rolled up at the contract level.

c. For QASP management purposes, the TM receives QASP inputs from the QARs and applies the decision criteria of the Incentives Table of RFP Section H to recommend incentive fee awards, if earned, or credit to the Government, as applicable, for decision by the IDO. (Incentive related data is recorded continuously, extracted monthly, evaluated for payment monthly, for invoicing on a quarterly basis.) See RFP Section H Tables 1 and 2 for details of application of the QASP metrics.

6.1.6 QASP Incentive Determining Official (IDO)

For GSM-O task orders, the DISA Network Services Division (NSD) Director will be the Incentive Determining Official (IDO) and in that role is the final approving authority for incentive fee payments earned, and credit to the Government to be applied, as applicable. The IDO receives recommendations from the COR and provides decision notifications to the KO and COR.

6.2 Finalizing the QASP

a. This draft QASP provides standards and assessment metrics consistent with the GSM-O PWS. After award, the draft QASP and the contractor’s QCP will be reconciled through negotiation to result in a revised QASP that provides the Government the means to assess performance and award incentives and credits to the Government, and allows the contractor to monitor the quality of its performance.

b. The Government can change QASP standards unilaterally at its discretion reflecting changes in priorities, Government requirements, and other considerations. PWS-level QASP performance standards may be reviewed and updated by mutual agreement.

Negotiated updates to the QASP will be added to the contract in a bi-lateral contract modification. Reviewing trends and discussing corrective actions and performance successes regularly will allow the QASP to remain relevant and ensure the focus is always to measure and reward the performance that leads to service quality above satisfactory levels.

6.3 Using the QASP to Monitor Task Order Performance

Within the overall GSM-O framework provided by the PWS and the final approved QASP, specific GSM-O Task Orders will use subsets of the PWS standards according to the scope of requirements of each TO. Applying the QASP to the performance of each awarded task order will follow a 5 step process as described below.

1) The process begins with the contractor performing the required work and delivering the required results. This includes, as applicable, performing data inputs into Government OSS tools during the course of task activities, including data related to status of request fulfillment, configuration management, trouble management, security, and performance management activities. As explained in the PWS (paragraph 3.7) these data inputs are not deliverables in the formal sense but are artifacts of ongoing task activity that are used by the Government as part of the management overhead and capabilities of the DISN network. For QASP execution purposes, the Contractor will propose, in its Quality Control Plan (QCP) responding to the QASP, its procedures for collecting and providing activity data to enable evaluation of its performance under the ALPs of the TO.

The Government QARs will also use Government data from the OSS and other sources to confirm and supplement the source data provided by the Contractor.

2) This performance is evaluated by the Government QAR who applies the specific metrics of each applicable ALP from the TO. The QAR also receives and interprets, as applicable, other information pertaining to contractor performance from Government officials including the applicable TM or field command personnel

3) The QAR will then apply the criteria of an ALP (i.e. the Performance Ranges column in each ALP structure) to determine the performance level of that ALP.

(For example, in ALP 8 pertaining to request fulfillment order processing within a specific geographic region, if there were 120 customer requests processed during the evaluation month, and 117 of them were processed within the time frames provided in the ALP, the performance level would be computed as 117/120 = 97.5%, which falls into the Satisfactory (Green) performance range for that ALP. The QAR documents this conclusion for each of the ALPs of the TO in his/her responsibility, and forwards that information to the TM for further processing. Note that, as described in detail in RFP Section H, some incentive fees are earned only after a set of evaluated ALPs have been considered. The QAR also forwards to the TM any non-ALP findings or materials that are recommended for consideration during the specific evaluation cycle.

4) The TM will receive QASP evaluation inputs from all the applicable QARs, and using the applicable table from the Incentive Plan in RFP Section H, will determine whether or not an incentive fee has been earned in each of the task areas of the TO for that period, and if so what the incentive amount is according to the criteria of the table from the Incentive Plan in RFP Section H. The TM also considers any non-ALP performance information submitted by the QARs in making each incentive determination. The TM then forwards his/her recommendation, with the supporting data, to the COR who determines Incentive payments earned/recommended and the forwards to the Incentive Determination Official (IDO) for decision. The TM’s recommendation also includes any credit to

Amendment 0001 the Government that are recommended to be imposed, based on the table for that purpose from the Incentive Plan in RFP Section H.

5) The IDO will then act on the TM’s recommendation pertaining to incentive fee payments earned, and/or credit to the Government to be imposed, for the evaluation period. The IDO will notify the KO and the TM of this decision. The KO will either process a modification to the contract to add the fee, or (if fee is already obligated) then notifies the Contractor and the TM that the appropriate invoice should be submitted (via the Government Wide Area Workflow (WAWF) system to the WAWF Acceptor for payment.

7 PERFORMANCE-BASED INCENTIVE STRUCTURE

a. Performance-based incentive contracts are intended to provide motivation for the contractor to perform at levels above the required acceptable level, and to be appropriately rewarded when it does so. When a task order that is subject to performance incentives is awarded, an incentive CLIN will be included in the award and will be funded as described in the task order, either at a stated dollar amount or as a stated percentage of the value of the parent task order.

b. The primary basis for implementation of performance based principles in the GSM-O contract is the set of ALPs which are derived from industry best practices pertaining to a given specific task requirement in the PWS. The PWS released with the GSM-O has a table of 35 ALPs that map to PWS requirements in paragraphs 4, 5, 6, and 7. If the Government issues task orders for capabilities under PWS paragraphs 8, 9, or 10, the table of ALPs in each such task order(s) will be expanded to include additional ALPs beyond those in the PWS. (All QASP incentive processes are executed at the task order level in the GSM-O contract environment.)

c. Each ALP contains a set of structural elements as described in paragraph 3f above.

These structural elements are sufficient to enable both the Contractor and the Government to understand what the minimum satisfactory performance results are for that task activity, as measured by the ALP, and specifically how contractor performance in that ALP activity will be measured and evaluated.

d. As described in paragraph 3e above, most ALPs identify four performance regions by color (Blue, Green, Yellow, and Red) that are determined by interpretation of quantitative data collected over the period of evaluation for that ALP by the applicable QAR (see paragraph 6.3 above).

e. An ALP is evaluated either on a Global basis (i.e. the designated Quality Assurance Reviewer in one geographic region will collect and evaluate all of the data pertaining to this ALP), or on a By-Region basis (i.e. the given ALP will be evaluated separately by the QAR in each of the four GSM-O regions, with a separate color evaluation; this prevents a high-scoring region from masking lower performance in another region). By-region ALP evaluations are not rolled up but are each considered separately.

f. Paragraph 6 above describes what the QAR does as the first part of applying the incentive structure for QASP purposes. When the QAR has completed his/her process of collecting (from the Contractor, per its QCP, and from other sources), evaluating, and scoring the data for a specific ALP for the evaluation period, those results (including the QAR’s determination, by use of the criteria in the ALP, of the color performance level attained for the period) are forwarded to the TM. The TM then interprets the inputs from

Amendment 0001 all responding QARs and applies the criteria of Table 1 of the RFP Section H Incentive Plan, to determine whether or not an incentive fee has been earned for each separate task area of Table 1 for that task order. As is shown in the Incentive Plan, in some task orders (including TO-1 and TO-2) the performance levels of multiple ALPs are considered (but not combined per se) to determine whether or not the incentive has been earned – which result is only visible at the TM level after receipt of all QAR inputs.

g. The Section H Incentive Plan Table of Incentives shows the mapping relationship between the set of ALPs applicable to a given task order, and the number of possible incentive fee awards that are possible. For example, for TO-1 for year-1, there are 19 individual ALPs that are evaluated for QASP purposes; those are grouped into seven (7) task areas for purposes of determining whether or not an incentive fee has been earned.

This approach, selected by the Government after consideration of industry practices, is midway between one extreme of requiring Blue level performance for all 19 ALPs before any incentive has been earned, and the other extreme of parceling out the total incentive fee pool into 19 small slices. Grouping them causes the Contractor to focus on the collective effects of the tasks being performed, and to work to raise the level of those areas whose performance could still be improved. Thus the incentive fee for a given task area, supported by multiple contributing ALPs, is an all-or-nothing determination for that particular task area.

h. In the DISN networking environment where the availability of a service is as aspect of the environment, how the Government addresses topics such as outages not attributable to the contractor affects how incentives are earned. In the GSM-O context, due to scale and complexity of Government-provided DISN services, there is no single overall measure or metric that deals with this complex issue. Therefore, specific ALPs have been developed, based on the requirements of the PWS, to deal with availability and responsibility issues. For a given type of task activity, the ALP that measures that activity specifies, within the ALP, how conditions beyond the GSM-O Contractor’s responsibility are handled in the collection and evaluation of QASP data. For activities that are timed, such as response to outages, this leads to specific conditional rules, within the ALP, that determine when “the clock is stopped” for ALP measurement purposes. These conditions are typical and inherent in network availability situations, and the ALP deals with them so the GSM-O Contractor is evaluated on activities over which it has sufficient control and responsibility to be held accountable through ALP metrics and measures.

These conditional provisions are not generalized either in the Incentive Plan or in this QASP because they must be applied at the level of each ALP.

i. When the Government notifies the Contractor of the results of QASP evaluation for a given period, if there are any ALPs that were rated below the Satisfactory (Green) level, (i.e. were rated in the Yellow or Red ranges), the Government will instruct the contractor to identify corrective actions for any performance area with a Red or Yellow indicator.

The contractor will then be required to actively monitor the trend for any performance area with a Yellow indicator and implement corrective actions as necessary to resolve performance shortfalls.

8 QASP MATRIX

The QASP Matrix in the table below is an exact copy of the Table of ALPs from PWS Appewndix C,and provides detailed performance standards for each task of the PWS.

The table in this initial draft represents the Government’s expectations. The final QASP

Amendment 0001 matrix will be based upon mutually agreed upon performance standards, measures and matrix, surveillance methods, and associated incentives. It includes descriptions of the performance standards for each PWS requirement, and the acceptable quality level associated with that standard. The table also establishes the surveillance method for each standard, the frequency of inspection, and the inspection process. Finally, the table identifies the incentive metrics and scoring ranges for performance evaluation.

GSM-O QASP Acceptable Levels of Performance (from PWS Appendix C)

PWS

Requirement Objective Standard

Acceptable Level of Performance

Performance Ranges

Surveillance Method/ Measure

Note: in the rows of the table, column 6 from the left identifies up to four regions of performance for incentive purposes:

A Blue region, where performance is Above-Satisfactory to a degree that merits Incentive;

A Green region, where performance is Satisfactory;

A Yellow region, where performance is not yet Satisfactory, but no credit to the Government is applicable;

A Red region, where performance is sufficiently below the Satisfactory level to justify a credit to the Government.

The “Standard” of satisfactory performance is minimally attained at the lower value of the Green/Satisfactory range.

In the Performance Ranges column, the notation “NR” means that a 100% value must be attained without rounding-up of a computed fraction (e.g. where applicable, the value 0.9999 does not round up to 1.00 to determine Blue status).

An example of these relationships is as follows, using Row 5 values:

0% -- [Red: credit to govt applicable] – 80% -- [Yellow: not yet Satisfactory] – 90% -- [Green: Satisfactory] – 99.8% -- [Blue: Above-Satisfactory, Incentive earned] – 100%

PWS 4.2 Security Requirements

Contractor shall adhere to security policies and procedures to minimize classified “spillage” incidents.

Zero spillage incidents per period

There shall be zero classified “spillage” incidents.

0= Blue 1 =Green n/a = Yellow 2 = Red

Frequency— Monthly, Surveillance Type—100% monitoring;

Method of surveillance— Data analysis.

Amendment 0001

Objective Standard

Acceptable Level of Performance

Performance Ranges

Surveillance Method/

PWS 4.2 Security Requirements

Implement security measures with minimum incidents

One incident per period

Security plan procedures are followed properly with no more than one incident per quarter.

0 = Blue 1 =Green 2 = Yellow 3 = Red

Surveillance Type—100% monitoring;

Method of surveillance—

PWS 4.4a Cost Management and Reporting

Submit invoices that are complete and timely

100% Invoices shall be submitted to the Government on a CLIN by CLIN basis within 30 days of the end of each monthly billing period with all costs included during that monthly period.

100% NR

= Blue 99.0% - 99.9% = Green 95-99%= Yellow <95% = Red

Surveillance Type—100% monitoring;

Method of surveillance—

PWS 4.5a Business-Related Metrics:

Business related activities as measured by business metrics should show improvement

3-4 or more areas each period; but no credit to the Government if all metrics are already being fully satisfied.

The Contractor’s performance as measured by the business metrics of PWS 4.5 shall show improvement over the previous period in at least 3 of the 5 areas

5 = Blue 3-4 =Green 1-2 = Yellow 0 = Red

Surveillance Type—100% monitoring;

Method of

Amendment 0001

Objective Standard

Acceptable Level of Performance

Performance Ranges

Surveillance Method/

PWS 5.3

Transition Execution

Execute transition activities effectively so that risks and costs of this initial period are mitigated effectively

90-99.8% On a regional basis where applicable, Transition Plan objectives shall be attained successfully for 90% of the activities in the applicable region within 5 days of the original scheduled target for completion.

99.9% = Blue 90-99.8% =Green 80-89% =

<80% = Red

Surveillance Type—100% monitoring;

Method of

PWS 6b Network Operation, Maintenance, Administration

Upon Government direction, execute roll-back actions effectively to ensure that a former state can be reinstated, per planning data.

98-100% Roll-back actions are effective 100% of the time

100% NR

= Blue 98-99.9%= Green N/A= Yellow <98%= Red

Monthly;

Surveillance Type— TBD;

Method of surveillance -

Row 7 not used

Amendment 0001

Objective Standard

Acceptable Level of Performance

Performance Ranges

Surveillance Method/

PWS 6.2.1a Order Processing & Request Fulfillment, Customer Support and Validation

Seamless and efficient processing of customer orders. Note: if a circuit requires C&A or crypto actions by user, Contractor is not responsible for that time period. Also, if a new third party access circuit is required, time attributable to DITCO action to establish the lease with the circuit provider is not counted in the periods being measured.

Time attributable to Government coordination with the user to obtain user concurrence and funding does not count against the 7-day period for completion of the

TSO.

The request fulfillment timeline ends with recording of data related to Data Capture item B005 in the OSS tool.

97%

Provide services sufficient to satisfy the DISN SLA within the following timeframes:

not more than seven days from original request to final approval decision for the Telecomm Service Order, followed by not more than--seven days when access bandwidth and ports are available; or 14 days when only cards and other minor hardware are needed;

or 21 days when added DISA provider edge access is needed.

[see next cell for continued text]

98% = Blue 97% = Green 80-96% = Yellow <80% = Red + $1000 credit to the govt/month for any order not fulfilled within 120 days of approval.

[continuation of text from ALP cell:] In all other cases, have ready for customer end-to-end activation within 21 days of Government acceptance of approved upgrades and requests for needed equipment chassis, bays, cards and minor hardware

Frequency— Monthly, Surveillance Type—100% monitoring;

Method of

Amendment 0001

Objective Standard

Acceptable Level of Performance

Performance Ranges

Surveillance Method/

PWS 6.2.4a(2)

& Request Fulfillment, Customer Request Processing

Maximize availability of the order entry system to customers

99.5-99.8% 99.5% availability of the order entry system to customers, where availability accounts for time under contractor’s control, and excludes time attributable to other entities

99.9-100% = Blue 99.5-99.8% =Green 95-99.4% = Yellow <95%= Red

Frequency— Monthly, Surveillance Type—100% monitoring;

Method of surveillance— Data analysis

PWS 6.2.1a

& Request Fulfillment, Customer Request Processing

Maximize quality of data entered in the order entry system

99.5-99.8% 99.5% accuracy of the information contained in the order entry system

99.9%= Blue 99.5-99.8% =Green 95-99.4% = Yellow <95% = Red

Frequency— Monthly, Surveillance Type—100% monitoring;

Method of

Amendment 0001

Objective Standard

Acceptable Level of Performance

Performance Ranges

Surveillance Method/

PWS 6.2.2

Trouble Management, Help Desk Support

Within 90 days of task order start, achieve and maintain established service levels for each of the following:

Call Abandonment Rate, and Average Answer Delay, and Customer Satisfaction.

65-79% of the sampled period where each rate is equal or better than the stated standard

Call abandonment rate:

= or less than 3%;

Average answer delay:

= or less than 11 seconds;

Customer satisfaction:

= or greater than 90%

80-100% =

65-79%= Green 50-64%Yellow <50% = Red

Surveillance Type—100% monitoring;

Method of

Management, Alarm Surveillance/ Reporting

Alarm Surveillance:

Document trouble events that are projected to require labor expenditure. All labor expended should be noted in ticket.

100%

Recognition within a five minute period that an alarm needs response action.

100% NR

= Blue 99.0-99.9% = Green 98.0-98.9% = Yellow <98% = Red

Surveillance Type—Audited Sample; Method of

Amendment 0001

Objective Standard

Acceptable Level of Performance

Performance Ranges

Surveillance Method/ Measure

13 PWS 6.2.2

Trouble Management, Alarm

Data in trouble ticket to be accurate.

99.5-99.7% Trouble Ticket filled out as required by

TMS SOP

99.8% = Blue 99.5-99.7%= Green 95-99.5 %=

Surveillance Type—Audited Sample; Method of

14 PWS 6.2.2

Management, Alarm

Provide timely, accurate and relevant incident management documentation. This includes description of incident, root cause of incident, current status/actions to resolve and final corrective action(s) (if known).

99.5 - 100%

Open TMS tickets IAW GFI (CCIR’s) for network alarms or reported incident/event.

Update of trouble ticket within 15 minutes of a change in status/actions to resolve.

100% NR

= Blue 99.5-99.9%= Green 95-99.4% = Yellow <95% = Red Credit to Government of $1,000 for every hour late

Surveillance Type—Audited Sample; Method of

Amendment 0001

Objective Standard

Acceptable Level of Performance

Performance Ranges

Surveillance Method/

Management, Alarm

Reporting: The Contractor’s staff will utilize GSM-OSS tools to notify Government representatives of identified incidents

98-100%

In accordance with GFI “CCIR” reporting criteria

98 - 99.9% = Green 95-97% = Yellow <95% = Red credit to Government of $1,000 for every hour late

Surveillance Type—Audited Sample; Method of surveillance—

Management, Fault Isolation and Correction

When fault analysis isolation confirms that the trouble is attributable to infrastructure under GSM-O control: Restore failed services promptly

99.90-99.96%

At minimum:

Average of 18 hours for all network service failure; 24 hours Maximum for any failure, 12 hours for fiber cuts.

In accordance with GFI SLAs reporting criteria

99.97%

99.90-99.96% = Green 99.0-99.9% = Yellow <99% = Red

Frequency-

Surveillance Type—Audited Sample; Method of surveillance—

Amendment 0001

Objective Standard

Acceptable Level of Performance

Performance Ranges

Surveillance Method/

Management, Fault Isolation and Correction

When fault analysis isolation confirms that the trouble is attributable to infrastructure not under GSM-O control:

Coordinate with applicable provider to ensure appropriate response. Notify Government if further action is required.

Engage Provider within 5 minutes after confirming that outage is in other Provider’s infrastructure; and coordinate with Government within 15 minutes if there are issues related to engaging Provider.

99.0-99.9% = Green 95-99% =

Frequency-

Surveillance Type—Audited Sample; Method of surveillance—

Amendment 0001

Objective Standard

Acceptable Level of Performance

Performance Ranges

Surveillance Method/

PWS 6.2.3

Configuration Management

(CM):

Configuration Management Database (CMDB) Accuracy

Show positive quarterly trends in the accuracy of existing configuration data-base items

Year-1: n/a

Year-2:

80-89%

Year-3:

90-94%

From an initial assessment of CMDB accuracy, demonstrate gradual improvements in accuracy of existing CMBD data, toward a target of >90% accuracy of this data within 3 years

Year-1: n/a

Year-2:

90% = Blue 80-89% = Green <80% = Yellow n/a = Red

Year-3:

95% = Blue 90-94% = Green 85-89% = Yellow <85% = Red

Surveillance Type—audited sampling;

Method of

Management

(CM):

Configuration Management Database (CMDB) Accuracy

For configuration changes added by the Contractor, information shall be accurate and complete

99-99.6%

Contractor-executed CMDB transactions will be at 99%

99.7% = Blue 99-99.6% = Green 80-98% = Yellow <80% = Red

Surveillance Type—audited sampling;

Method of

Amendment 0001

Objective Standard

Acceptable Level of Performance

Performance Ranges

Surveillance Method/

Management

(CM):

Each CM change to be submitted in a timely manner

90-94% Change to be submitted within the following timeframe:

90% within 24 hours

95% = Blue 90-94%= Green 80-89% =

<80%= Red

Surveillance Type—audited sampling;

Method of surveillance—

PWS 6.2.4

GSM-OSS

operation, Server/Application Administration and Maintenance

Provide uninterrupted operational LAN availability.

99.5-99.9%

LAN availability of 99.5% where availability accounts for time under Contractor’s control, and excludes time attributable to other entities.

Coordinated and approved downtime is excluded for purposes of calculating availability.

100.0% NR

99.5-99.9%= Green 95-99.4% =

<95%= Red

Surveillance Type—100% monitoring;

Method of

Amendment 0001

Objective Standard

Acceptable Level of Performance

Performance Ranges

Surveillance Method/

PWS 6.2.4

GSM-OSS

application operation, Maintenance

Provide uninterrupted GSM-OSS availability

99.7-99.9% Operational GSM- OSS availability of 99.7% where availability accounts for time under contractor’s control, and excludes time attributable to other entities. Coordinated and approved downtime is excluded for purposes of calculating availability.

100.0% NR

99.7-99.9%= Green 95-99.6% = Yellow <95%= Red

Frequency— Monthly, Surveillance Type—100% monitoring;

Method of surveillance—

Row 23 not used

PWS 6.2.6

Maintenance, Operational Engineering, Administration and Security:

Ensure the DISN and associated networks and software are changed/upgraded and updated as necessary.

99.5-99.8%

Actions are executed in accordance with procedures/directives.

99.9%= Blue 99.5-99.8%= Green 95-99.4%=

Surveillance Type— TBD;

Method of surveillance -

Amendment 0001

Objective Standard

Acceptable Level of Performance

Performance Ranges

Surveillance Method/

PWS 6.2.6

Maintenance, Operational Engineering, Administration and Security:

Execution of hardware and software patches as approved by applicable Change Advisory Board

(CAB).

100% Approved hardware and software patches shall be completed no later than 'due date' outlined in Government Notice; (if no due date, within 30 days of receipt of notice and approval by applicable authority).

99.5-99.9% = Green N/A = Yellow <99.5% = Red

Surveillance Type— TBD;

Method of surveillance -

PWS 6.2.7

Program Integration Services

Provide seamless integration of Government-approved deliverables or release packages into the supported Networks and Network support

95-100% Changes are incorporated within 72 hours of implementation, after applicable test/accept checks have been done, with no unintended degradation of performance and results in ability to meet relevant Network SLAs and Performance Measures

100% NR

= Blue 95-99.9% = Green 85-94%=

<85% = Red

Surveillance Type— TBD;

Method of

Amendment 0001

Objective Standard

Acceptable Level of Performance

Performance Ranges

Surveillance Method/ Measure

27a PWS 6.3.2 Contingency/ Emergency Response

Seamless and efficient response to Contingency/Emergency directive(s)

97-98% Upon receipt of a Government approved action, present proposed plan/action within 24 hours to mitigate/support the required action

99-100% = Blue 97-98% = Green 90-96%=

<90 %= Red

Frequency—Per Incidence, Surveillance Type—100% monitoring;

Method of surveillance— Data analysis.

27b PWS 6.3.2 Contingency/ Emergency

Seamless and efficient response to Contingency/Emergency directive(s)

97-98% Upon Government approval of the proposed plan of action, implement the required actions according to the schedule contained in the approved plan.

99-100% = Blue 97-98% = Green 90-96%=

<90 %= Red

Frequency—Per Incidence, Surveillance Type—100% monitoring;

Method of surveillance— Data analysis.

28 PWS 6.4

GSM-O COOP

Ensure Continuity of Operations for the

DISN.

100% Complete handover of mission within 30 minutes of failure, with no loss of network control directly attributable to GSM-O contractor performance.

100% NR

= Blue 98-99.9% = Green 97-98% =

<97% = Red

Surveillance Type—100% monitoring;

Method of

Amendment 0001

Objective Standard

Acceptable Level of Performance

Performance Ranges

Surveillance Method/

PWS 7.1

Computer Network Defense -Protect Activities

Timely Dissemination of Information Assurance Vulnerability Alerts (IAVAs

97-98% Disseminate Information Assurance Vulnerability Alerts (IAVAs) within 24 hours of release by the Department of Defense

99% = Blue 97-98%= Green 90-96%=

<90% = Red

Surveillance Type— TBD;

Method of surveillance -

PWS 7.2

Network Defense Detect-Activities

Monitor and detect all incidents and events in accordance with the

CJSCI 6510.01E

100% Detect all incidents and events in the network under their control.

100% NR=

99-99.9%= Green 97-98% =

<97%= Red

Surveillance Type— TBD;

Method of surveillance -

PWS 7.3

Network Defense

Activities

Effective reporting of events

95-98% 95% of Events reported within 2 hours timelines as outlined in DoD 6510.01E [directive does not state timelines per se]

99% = Blue 95-98%= Green 90-95% =

<90% = Red

Surveillance Type— TBD;

Method of

Amendment 0001

Objective Standard

Acceptable Level of Performance

Performance Ranges

Surveillance Method/

PWS 7.3

Network Defense

Effective reporting of incidents

97-98% 97% of Incidents reported IAW CJCSM 6510.01E Chg 3 Encl B timelines as outlined in DoD 6510.01E [directive does not state timelines per se]

99% = Blue 97-98%= Green 95-96% =

Surveillance Type— TBD;

Method of surveillance -

PWS 7.4

Network Defense Sustainment

Maintain and protect applicable network and systems configurations by preventing intrusions in networks under Contractor control in accordance with local, DoD Component, and DoD classification guidance.

100% of the reporting period with no successful intrusions that are attributed to the NetOps Center in the military domain in the networks under their control.

NR=Blue 99-99.9% = Green 98-98.9% =

Surveillance Type— TBD;

Method of

Amendment 0001

Objective Standard

Acceptable Level of Performance

Performance Ranges

Surveillance Method/

Network Defense

Provide information related to events/incidents to the Vulnerability Management System (VMS) for tracking and status checking to ensure it is maintained Maintain and protect applicable network and systems configurations in accordance with local, DoD Component, and DoD classification

Data and updates to be input into the VMS within 2 hours of being verified Provide information to the Vulnerability Management System (VMS) for tracking and status checking to ensure it is maintained

100% NR=Blue 99-99.9% = Green 98-98.9% =

Surveillance Type— TBD;

Method of surveillance -

Network Defense

Achieve at least satisfactory ratings during security reviews to be performed by the Designated Approving Authority.

Maintain and protect applicable network and systems configurations in accordance with local, DoD Component, and DoD classification

Achieve at least a Satisfactory rating 100% on all DAA reviews Achieve at least satisfactory ratings during security reviews to be performed by the Designated Approving Authority.

100% NR=Blue 99-99.9% = Green 98-98.9% =

Surveillance Type— TBD;

Method of

Amendment 0001

Objective Standard

Acceptable Level of Performance

Performance Ranges

Surveillance Method/ Measure

36-

Rows 36-46 not used

PWS 4.5b Business Metrics

Reduction in OCONUS Staffing footprint

Annual Reduction of

OCONUS

staffing level

After a year-one baseline is established by Government, GSM- O Contractor shall share 5-10% of the savings realized from any annual reduction in OCONUS staffing.

Not applicable No credit to the Government is computed from actual realized savings

Frequency— Annually;

Surveillance Type— TBD;

Method of surveillance – Data and billing analysis