Section_J _Attachment_7_QASP_v7.1_25_Feb_13.docx

FA4600-12-R-0013 Page 21 of 21 Section J, Attachment 7

QUALITY ASSURANCE SURVEILLANCE PLAN

for

HQ AIR FORCE WEATHER AGENCY SYSTEMS ENGINEERING, MANAGEMENT and SUSTAINMENT III (SEMS III) 24 Sep 2012

System Engineering, Management and Sustainment III (SEMS III)

Quality Assurance Surveillance Plan

Multi-Functional Team Members:

____________________________________ Date: ___________ Functional Commander/Director

____________________________________ Date: ___________ Contracting Officer’s Representative

____________________________________ Date: ___________ Contract Administrator

___________________________________ Date: ___________ Quality Assurance Program Coordinator

___________________________________ Date: ___________ Contracting Officer

___________________________________ Date: ___________ Contractor (After Award)

TABLE OF CONTENTS

Page

TABLE:

Table 1 -- Services Summary and Surveillance Method

ATTACHMENT:

Attachment 1 - COR Monthly Report Attachment 2 - Corrective Action Report

1.0 Purpose

1.1. This Quality Assurance Surveillance Plan (QASP) provides the methods and procedures the Government will use to ensure it receives the services under the Systems Engineering, Management and Sustainment (SEMS) III contract as identified in the Performance Work Statement (PWS). This QASP describes a systematic approach for conducting contract performance surveillance. Contracting Officer’s Representatives (CORs) will use the QASP to ensure the objectives of the contract are met. The key contract performance objectives will be evaluated using the contract Services Summary. This surveillance method will adequately assure that the Contractor’s performance is acceptable.

1.2. The QASP is based on the premise that the Contractor is responsible for management and quality control actions to meet the terms of the contract. The performance requirements recognize the Contractor is not a perfect manager, and that unforeseen and uncontrollable situations do occur. Good management and use of an adequate quality control plan will allow the Contractor to operate within specified performance requirements. The role of Government quality assurance is to ensure contract objectives are achieved. The COR must be objective, fair, and consistent in evaluating Contractor’s performance against the objectives and contract documents.

1.3. The QASP describes the objective in having the service provided and the results the multi-functional team (MFT) is striving to achieve in managing the acquisition. The QASP also describes the methods and tools the MFT will use to assess the Contractor’s performance against the performance thresholds identified in the contract.

2.0 Description of Services

2.1. This contract provides systems engineering, program management/systems management and systems sustainment services required to support Air Force Weather (AFW). It provides system upgrades, refreshes, technology insertions and sustainment. The Government’s focus is outcome based and the Contractor has the flexibility to continuously improve and innovate over the course of the contract as long as the critical outcomes expected are being achieved.

2.2. The primary objective of the contract is that AFW systems deliver an increasingly higher level of effectiveness, performance (technically, functionally and in terms of customer support) and consistently high operational readiness while reducing ownership costs. The Contractor is to provide world-class, industry standard information technology services. The design, development and delivery of efficient and reliable systems to reduce ownership costs while improving system performance, combined with strong and up-to-date knowledge of technology solutions are key components of this service. Additionally, the Contractor is expected to be a partner with AFW personnel and a proactive catalyst in continuously improving the effectiveness of AFW systems and the products these systems provide to AFW customers in supporting current and emerging missions. The Contractor is expected to bring the tools, experience, practices and leadership to AFW to move its information technology environment into a proactive customer responsive system following best business practices.

3.0 MFT Responsibilities

3.1. The MFT will plan and manage this service contract throughout the life of the requirement. The MFT shall:

3.1.1. Develop a business/acquisition strategy that will meet customer requirements and foster partnerships with industry

3.1.2. Develop, implement, and execute performance based acquisition strategy

3.1.3. Perform market research to ensure best commercial practices

3.1.4. Conduct source selection which ensures selection based on requirements

3.1.5. Contract performance management, to include collection of past performance information

3.1.6. Provide support to leadership, as required, to ensure dissemination of acquisition information and to ensure involvement on services/mission risks

3.1.7. Ensure funding is adequate and within approved funding baselines

3.1.8. Manage risk to ensure mission performance is within cost and schedule constraints

3.1.9. Identify opportunities to improve performance throughout the life of the acquisition and recommend changes to the QASP

3.2. The MFT members work together to ensure success. Together, the team builds the acquisition strategy, solicitation, and executes performance management. The MFT membership will be individuals holding the positions listed herein.

3.2.1. SEMS III Program Manager (PM)

3.2.2. Contracting Officer (CO)

3.2.3. Quality Assurance Program Coordinator

3.2.4. Contract Administrator

3.2.5. Lead COR (A8P)

3.2.6. COR in the functional area

3.2.7. Environmental, Legal, Manpower, Finance, Records Management and Safety (Optional. These members may be called to help research or provide guidance on an issue that arises during the acquisition process.)

3.2.8. Contractor (Becomes a member after award of contract.)

4.0 Surveillance Procedures

4.1. The COR will observe Contractor’s performance and use the Services Summary to evaluate that performance. The method of surveillance is identified in the Table 1 below. The COR will utilize two methods of inspection; either periodic or 100 percent, as identified in the Surveillance Method column in Table 1 below, to assess the applicable performance objectives (POs). The COR will assess the Contractor’s performance by watching actual performance, physically checking an attribute of the completed task, or inspecting the task or its results to determine whether or not the performance meets the threshold. The COR will perform surveillance and complete the SEMS III COR Monthly Report which will be tailored for each Task Order (Attachment 1). Although the SEMS III COR Monthly Report will be utilized monthly, the COR must maintain a daily working knowledge of the Contractor’s performance.

4.2. The 100 percent inspection method requires complete inspection of a contract requirement and will be used for requirements that are especially critical or where there is some reason for suspecting that the performance objective is not being met and therefore, should be more closely monitored. All POs requiring a 100 percent evaluation e.g. milestones/test type events/certain deliverables which require Government acceptance/permission to proceed will be complied with when the event is scheduled or upon delivery.

4.2.1. Evaluation procedures: Observed defects for a service monitored by 100 percent inspection are compared to the applicable performance objectives threshold column in the Services Summary (Table 1 below).

4.3. The periodic inspection provides a systematic way of looking at service outputs and forming conclusions about the contractor's level of performance in accordance with Performance Objective identified in the Services Summary (Table 1). Evaluation by periodic inspection is designed to inspect, at varying intervals of time, some or all of the products and services being monitored.

4.3.1. Evaluation procedures: The levels of evaluation appropriate for periodic inspection are subjective. In order to perform trend analysis from periodic inspection, criteria for sample selection should be applied consistently from period to period. To ensure valid results, the COR will use the SEMS III COR Monthly Report (Attachment 1) and follow a detailed inspection schedule. Schedules may be developed monthly to coincide with the Contractor's monthly schedule of work, and regularly updated after receiving the contractor’s definitive weekly schedule. Observed defects for services monitored by periodic inspection will be totaled at the end of each month. For each service, the total number of defects will be compared to the Services Summary performance objective threshold.

4.4. The COR must survey performance by viewing the Contractor perform the tasks in the PWS to determine whether or not the performance meets the threshold standards identified in the Services Summary. When a surveillance observation results in a rating that meets threshold performance, the COR will record it in the SEMS III COR Monthly Report. When a surveillance observation results in an unacceptable rating (less than performance threshold), the COR will record it in the SEMS III COR Monthly Report and complete a Corrective Action Report (CAR) (Attachment 2). The specific reason for the unacceptable performance must be recorded in the SEMS III COR Monthly Report and in the CAR.

Table 1 Services Summary and Surveillance Method

100% without failures Periodic

100% submitted before or on due date Periodic

.90<SPI<1.10

Cumulative to date Cost Performance Index (CPI) Schedule Performance Index (SPI) Evaluated once per month using the Contract Performance Report. Outside of threshold no more than 25% of period of performance.

100% monthly

1 Reference TO 00-35D-54, Table 1-2 2 “Actively used” means that the Contractor reviews risk mitigation steps monthly and reschedules dates that are past due or completes each step before its due date.

Operational Readiness* (Firm Fixed Price Tasks)

4.1.4, 4.1.5 4.1.6

Production operational readiness shall be evaluated monthly and be at least 99.5 percent.

Mean Down Time for the production system shall not exceed 4 hours monthly Periodic

PO-17 Software sustainment and license management services for HQ AFWA SWAFS.

PO-19 On-call response

For “duty hour”** support, greater than 95% of duty hours, on-call requests are responded to within 20 minutes of original call. When on-call support personnel are offsite during duty hours greater than 95% of on-site arrivals are within 60 minutes of determination that on-site support is required.

For “best effort” support, on-call requests are responded to as resources become available during duty hours.

Periodic

*NOTE: Operational readiness is the probability that the system is operating satisfactorily at any point in time excluding scheduled maintenance and training. (Air Force Technical Order 00-33A-1001, para 13.2.2.1.1.b, page 13-1) **NOTE: Duty Hours are 0730-1630 Mon-Fri excluding Federal Holidays.

5.0 Performance Assessment Documentation

5.1. The COR evaluates, reports, and records the Contractor's performance using the SEMS III COR Monthly Report. This documentation provides the CO with Contractor status as it applies to the performance criteria. Every effort should be made to make this report as accurate as possible. The COR are required to provide their monthly reports to A8P no later than the 3rd workday of each month detailing the results of critical requirement verifications and reporting all discrepancies associated with the performance thresholds. This includes deficiencies identified by the Contractor. This information will be used as a basis for the monthly feedback to the Contractor, annual Contractor Performance Assessment Report prepared for all task orders (in aggregate) completing their period of performance during the contract period and for determining the overall Quality Performance Indicators on all completed Cost Plus Incentive Fee (CPIF) Task Orders (TOs).

5.2. Corrective Action Report (CAR): Used to document contract requirement findings and subsequent corrective actions to be taken. See attached CAR (Attachment 2) and instructions for filling it out.

5.2.1. The COR identifying the discrepancy is responsible for completing the CAR utilizing the instructions attached. If there are other questions as to how the CAR is completed, that individual should consult the Lead COR.

5.2.2. Consult the assigned Contract Administrator (CA) to give them the opportunity to assist and advise how the CAR is to be worded. Forward a copy of the initiated CAR to the CO and CA. The CO will forward the CAR to the contractor.

5.2.3. After the contractor takes action on the assigned CAR the applicable COR will validate by commenting in block 10 and will then either accept or reject the contractor’s re-performance by completing block 11. The Lead COR will then forward the CAR to the CO/CA.

5.2.4. Categories of Findings: A finding is any Government-identified noncompliance with contract requirements that specifies that an activity or action did not take place or was not performed to the quality or timeliness standard required by the contract. A finding can be categorized as either major or minor. Findings will be documented on a CAR and must also be tracked by the Lead COR. Major/Minor Findings are defined as follows:

5.2.4.1. Major Finding: A conclusion that identifies a condition having a significant adverse effect on the quality of the activity under review such as:

5.2.4.1.1. Failure to meet a Performance Threshold

5.2.4.1.2. Significant mission degradation in any Contractor operated function

5.2.4.1.3. A situation that is likely to result in hazardous or unsafe conditions for individuals/equipment coming into contact with the work

5.2.4.1.4. Failure to provide adequate corrective action to preclude reoccurrence of Government identified findings

5.2.4.1.5. Failure to provide corrective action to deficiencies identified by the Government within a prescribed suspense period

5.2.4.1.6. Any failure to adhere to security regulations that results in a security incident

5.2.4.2. Minor Finding: A departure from established standards having little bearing on service provided. However, if the same minor finding is repeatedly identified, it may be an indication that a Major Finding (5.2.4.1.4) is occurring or has occurred because the Contractor’s quality control system has not acted to prevent recurrence. In this case, the Minor Finding should be elevated to the level of a Major Finding and a new CAR initiated.

5.3. The COR is required to maintain accurate records of the Contractor's performance and keep the CO informed of all data pertaining to Contractor status and maintain the COR records in a continuity book that includes:

5.3.1. A copy of the contract/order and all modifications; COR designation memorandum; correspondence between the COR and the Contractor; copies of correspondence to or from the Contracting Officer; records of all inspections performed and the results; and all other documentation of actions taken by the COR.

5.3.2. The COR is required to maintain accurate records of the Contractor's performance and keep the CO informed of all data pertaining to Contractor status. The following items may be readily accessed via a central database, e.g. Contractor database or Wide Area Work Flow; COR records minutes of all meetings; copies of all invoices submitted and paid; copies of all Contractor data submittals.

5.4. Each COR is responsible to review the Incentive Fee Plan (IFP) associated with this contract.

5.4.1. Each COR is responsible to draft and route a final task order assessment letter at the completion of their assigned task order (CPIF TOs only). An assessment letter template will be provided by the Lead COR or PM.

6.0 Unacceptable Performance

6.1. If at any time during the surveillance period, the results of surveillance show the Contractor did not meet threshold requirements for any performance objective, and the COR determines it is not Government caused, the COR initiates a CAR as soon as possible. The COR completes blocks 1 through 8 of the form and brings it to the Lead COR for signature. The Lead COR evaluates the CAR and notifies the SEMS III Program Manager. If deemed valid by the Government SEMS III PM, the CAR will be sent to the CO/CA who will notify the contractor. The Contractor’s Representative must respond with a written corrective action plan that details the corrective action of the cited deficiency, the cause of the deficiency, and actions taken to prevent recurrence by Suspense Date in Block 5 of the CAR. Upon receipt of the Contractor’s response the COR will forward to the CO. The CO in consultation with the COR must evaluate the Contractor’s response and take the appropriate action. If deduction from payment is deemed the appropriate action, the deduction will be made on the following invoice after the decision is made or in the case of final payment, deduction will be made from the final invoice.

7.0 Certification of Services

7.1. The COR will certify receipt of acceptable Contractor fixed priced task order services each month. The COR will certify that acceptable services were received, in accordance with the terms and conditions of the contract. The COR will certify receipt of Contractor services by email to HQ AFWA/FM when they provide the invoice. Certification of services shall be accomplished within 7 days of receipt of the invoice or as specified by HQ AFWA/FM to avoid payment of interest penalties.

8.0 Revision of QASP

8.1. Revisions to this plan are the joint responsibility of the MFT. The COR should provide recommendations for improving the PWS and QASP through to the CO and/or MFT for review. This is a living document that may be changed at any time; however, if changes are made to the QASP or to the PWS, the COR/CO must ensure that the QASP corresponds with the requirements of the PWS throughout the life of the contract.

ATTACHMENT 1

SEMS III COR Monthly Report

PWS ref: 2.1.1, 2.1.2, 2.1.3, 2.1.4, 2.1.7, 2.1.8, 2.3.2.6, 2.3.2.8, 2.3.2.18, 2.3.2.20, 2.3.2.22, 2.3.3

PWS ref: 2.1.1, 2.1.2, 2.1.4, 2.2.1, 2.3.2.6, 2.3.2.8, 2.3.2.18, 2.3.2.20, 2.3.2.22, 2.3.3

PWS ref: 2.1.5, 2.1.6, 2.1.7, 2.3.2.6, 2.3.2.8, 2.3.2.18, 2.3.2.20, 2.3.2.22, 2.3.3

PWS ref : 2.1.5, 2.3.3

100% without failures

PWS ref. 2.1.1, 2.1.3, 2.1.5, 2.1.8, 2.3.2.6, 2.3.2.8, 2.3.2.18, 2.3.2.20, 2.3.2.22, 2.3.3

100% submitted before or on due date

PWS ref: 2.2.4

.90<SPI<1.10

Cumulative to date Cost Performance Index (CPI) Schedule Performance Index (SPI) Evaluated once per month using the Contract Performance Report. Outside of threshold no more than 25% of period of performance.

PWS ref: 2.2.1, 2.2.2

PWS ref: 2.2.6

PWS ref: 2.2.1, 2.2.2

PWS ref: 2.3.4

PO 11

Mean Down Time for the production system shall not exceed 4 hours monthly.

PWS ref: 2.3.2, 2.3.5, 2.3.6, 2.3.7, 2.3.8, 2.3.9, 4.1.4, 4.1.5 4.1.6

PWS ref: 2.3.2, 2.3.5, 2.3.6, 2.3.7, 2.3.8, 2.3.9, 4.1.4, 4.1.5, 4.1.6

P O Hardware maintenance, system administration, property management and license management services for Oracle (Sun) Solaris Production.

PWS ref: 2.3.2, 2.3.5, 2.3.6, 2.3.7, 2.3.8, 2.3.9, 4.1.4, 4.1.5, 4.1.6 Production operational readiness shall be evaluated monthly and be at least 99.5 percent. Mean Down Time for the production system shall not exceed 4 hours monthly.

PWS ref: 2.3.2, 2.3.5, 2.3.6, 2.3.7, 2.3.8, 2.3.9, 4.1.4, 4.1.5, 4.1.6

PWS ref: 2.3.2, 2.3.5

Maintenance will include remedial and preventive maintenance.

UPS operational readiness shall be evaluated monthly and be at least 99.9 percent monthly.

PWS ref: 2.3.2, 2.3.5, 2.3.6, 2.3.7, 2.3.8, 2.3.9, 4.1.6

PWS ref: 2.3.2, 2.3.5

PO 18

PWS ref: 2.3.2, 2.3.5

For 24x7 support, greater than 95% of on-call requests are responded to within 20 minutes of original call and greater than 95% of on-site arrivals are within 60 minutes of determination that on-site support is required.

For “duty hour” support, greater than 95% of duty hours on-call requests are responded to within 20 minutes of original call. When on-call support personnel are offsite during duty hours greater than 95% of on-site arrivals are within 60 minutes of determination that on-site support is required.

For “best effort” support, on-call requests are responded to as resources become available during duty hours.

PWS ref: 2.3.2

ATTACHMENT 2

CORRECTIVE ACTION REPORT (CAR)

(If more space is needed, use reverse and identify by number)

1. CONTRACTOR

2. CONTRACT NUMBER

3. TYPE OF SERVICES

4. FUNCTIONAL AREA

5. SUSPENSE DATE

6. CONTROL NUMBER

7. DEFICIENCY |_| MAJOR |_| MINOR

FINDING:

FINDING IMPACT:

Please respond with a written corrective action plan that details the corrective action of the cited deficiency, the cause of the deficiency, and actions taken to prevent recurrence by Suspense Date in Block 5. If date was not entered in Block 5, the contractor is not required to provide a response.

8. CONTRACTING OFFICER REPRESENTATIVE (COR)

TYPED NAME AND GRADE

SIGNATURE AND DATE

9. ISSUING AUTHORITY

TYPED NAME AND GRADE

SIGNATURE AND DATE

10. COR RESPONSE TO CONTRACTOR CORRECTIVE ACTION AND ACTION TAKEN TO PREVENT RECURRENCE

11. COR DETERMINATION

|_| ACCEPTED |_| REJECTED

12. CLOSE DATE

Corrective Action Report (CAR) Template, Feb 05 (SAF/AQCP)

CORRECTIVE ACTION REPORT (CAR)

Continuation Sheet (Number to correspond with applicable Item Number on reverse)

Corrective Action Report (CAR) Template, Feb 05 (SAF/AQCP)

Corrective Action Report (CAR) Instructions

Block 1. Enter Contractor Name.

Block 2. Enter Contract Number.

Block 3. Enter Contract Name or Type of Services.

Block 4. Enter Functional Area of the Contract.

Block 5. Enter the assigned suspense date given the contractor to provide a response to the CAR. A date must be entered for a Major CAR. A date is optional at the discretion of the CONTRACTING OFFICER’S REPRESENTATIVE (COR) initiating the CAR if the finding is Minor.

Block 6. All CARs will be tracked with a Control Number. The functional commander/functional director is the Issuing Authority for his/her contract COR generating CARs. By providing the control number to the COR originating the CAR, the Issuing Authority demonstrates concurrence with the finding. The control number will be made up of the first two letters of the name of the site or contract, the last two digits of the calendar year, and a three digit number starting with 001 and progressing upward throughout the calendar year (e.g., WP04-001).

Block 7.

1. Check the block that indicates whether the identified deficiency is assigned as a Major or Minor finding. See Performance Plan (PP), paragraph 6.1.4, for the definitions of Major and Minor findings and associated explanations.

2. Finding: Clearly state the details of the finding followed by a reference to the stated contractual requirement. The reference must state the portion of the contract, part, section, paragraph and subparagraph and must make a complete brief quotation of the contract reference.

3. State the impact that the finding has or could have on the accomplishment of the mission that the contract provides.

4. When determining how long to give the contractor to respond to the identified finding, the normal is 10 working days. The suspense date should reflect this unless the finding requires a greater amount of time to come to solution. Major findings require a contractor response. Minor findings may require a contractor response at the discretion of the COR initiating the CAR and/or the Issuing Authority.

Blocks 8 and 9. Contract COR initiating a CAR must sign in Block 8 and the Issuing Authority signs in Block 9. Each annotates the date at the time of signature.

Block 10. Upon review of the contractor’s reply, the originator of the CAR will enter comments regarding acceptance or rejection of the contractor’s response. This block may also contain any comments regarding follow-on inspections conducted or needed at a later date to validate that the finding has not reoccurred.

Block 11. The COR selects Accept or Reject after reviewing the contractor’s response.

Block 12. The COR enters a close date after advising the Issuing Authority that the contractor’s response is acceptable and the Issuing Authority concurs. The Issuing Authority will then forward the closed CAR to the contractor and report accordingly in the monthly Certificate of Service (COS) or other applicable contracting officer services report. This routing remains the same if the contractor response is unacceptable and the CAR remains open until an acceptable response is received except that the Issuing Authority forwards the open CAR back to the contractor for additional information.

Continuation Sheet. The Continuation Sheet is to be used to expand on any information that will not fit in the applicable numbered block.

SEMS III QASP v7.1