Attachment_1_Executive_Leadership_QASP_Final.doc

Quality Assurance Surveillance Plan

Quality Assurance Surveillance Plan (QASP) Under Secretary of the Air Force, Office of Business Transformation and

Deputy Chief Management Officer

Executive Education Continuous Process Improvement (CPI) Executive Leader Training

Solicitation No. FA7014-16-R-5007 Contract No. TBD

COMPLIANCE WITH THIS DOCUMENT IS MANDATORY

OPR: SAF/MGS

Reviewed by: SAF/MGS Ms. Penny O’Neil GS-15, Mr. J.R. Tillery Contracting Officer Representative Deputy Director Reviewed by: AFDW/PKQ Accepted by: AFDW/PKS Mrs. MARY LAWRENCE Mr. Bumjoon J. Pak Quality Assurance Program Coordinator Contracting Officer

Pages: 20

TABLE OF CONTENTS

1.0

SCOPE

1.1 Purpose

1.2 Application

2.0

MANAGEMENT LOCATIONS AND KEY PERSONNEL

2.1 Contract Management Locations

2.2 Contract Management Team

2.3 Contractor Key Personnel

3.0

CONTRACT SURVEILLANCE

3.1 Surveillance Methods

3.2 Surveillance Documents

3.3 Assessing Performance

4.0

WIDE AREA WORK FLOW (WAWF)

4.1 Validating Receiving Reports

4.2 Certifying Receiving Reports

4.3 Rejecting Receiving Reports

5.0

CONTRACTING OFFICER REPRESENTATIVE (COR) FOLDER

TABLES

Deliverables

Services Summary

Non-Conformance Notifications

Quarterly Performance Assessment Report Evaluation Categories

Performance Rating Definitions

COR Folder Tabs and Contents

ATTACHMENTS

First Notice Template

Corrective Action Request (CAR) Template

Quarterly Performance Assessment Report (PAR) Template

1.0 SCOPE.

The Quality Assurance Surveillance Plan (QASP) is applicable to personnel performing contract surveillance audits in accordance with (IAW) Federal Acquisition Regulation (FAR) Part 46, Quality Assurance; Department of Defense FAR Supplement (DFARS) Subpart and PGI 201.602-2, Contracting Officer Responsibilities; Air Force FAR Supplement (AFFARS) Mandatory Procedures (MP) 5346-103, Contracting Office Responsibilities, The Quality Assurance Program; Air Force Instruction (AFI) 63-501, Air Force Acquisition Quality Program; and organizational policy. It is designed to provide the Contracting Officer Representative (COR) a systematic surveillance method for each service identified in the Performance Work Statement (PWS) for each contract. The owning Functional Director (FD) and COR develop the methods for administering and evaluating contract performance. The Quality Assurance Program Coordinator (QAPC) and Program Manager (PM) or Contracting Manager (CM), will review these methods, which are then accepted or rejected by the Contracting Officer (CO).

1.1 Purpose. This document defines the surveillance policy, procedures, and associated methods used by CORs and Program Managers (PM) for planning, preparing, performing, analyzing, and reporting Contractor performance. This QASP provides a systematic surveillance method to evaluate the services received from the Contractor, and NOT to address the details of how the Contractor accomplishes the work.

1.2 Application: This QASP is used by all parties involved with assessing Contractor performance and is for the use of FDs, CORs and CO. The Contractor may receive a courtesy copy in order to further enhance communications with the Contractor, but shall not be considered as part of the contract.

2.0 MANAGEMENT LOCATIONS AND KEY PERSONNEL.

2.1 AFDW/PKS: The address for Contracting Management is:

1500 W. Perimeter Rd Suite 2750

Joint Base Andrews, MD 20762

2.2 Contracting Management Team.

Contracting Officer (CO) – Mr. Bumjoon J. Pak: CO duties are detailed in FAR subpart 1.602-2. The CO is the only Government agent authorized to award or modify contracts; therefore, the CO is the only person authorized to contractually obligate the Government. The CO is also the individual who will sign any notifications to Contractors regarding performance issues and accepts Government surveillance methods utilized for each TO. The CO approves/appoints all CORs assigned by the Functional Director/Functional Commander.

The CO can be reached at DSN: 612-2995 or Commercial 240-612-2995.

2.2.1 Contract Manager (CM) – Ms. Cynthia I Harper (CTR): The CM is the Government agent within the contracting office who performs day-to-day administration of the contract.

The CM can be reached at DSN: 612-2996 or Commercial 240-612-2996.

2.2.2 Program Manager (PM) – Ms. Penny O’Neil: The PM assists customers with requirements development and definition, PWS modifications, deficiency resolution, and portfolio management.

The PM can be reached at DSN: 256 0444 or Commercial 571 256 0444

2.2.3 Quality Assurance Program Coordinator (QAPC)-Mary Lawrence: The QAPC assists the CO and QAM in managing the overall QA program and supports customers in developing and adhering to TO-specific QASPs.

The QAPC can be reached at DSN: 612-6140 or Commercial 312-612-6040

2.2.4 Functional Director (FD)/Functional Commander (FC). The FC/FD Identifies mission essential services, reviews contract performance documentation, by the COR, on a regular basis to ensure performance is compatible with contract mission objectives and nominates COR(s) for contract oversight. The FC/FD receives contract specific training by the QAPC.

2.2.5 Contracting Officer Representative (COR) Ms. Penny O’Neil: The COR is ultimately responsible for implementing all QA matters and procedures. COR(s) should be at a level where other duties don’t prevent proper oversight of Contractor performance. The COR must maintain both technical competency and evaluation proficiency in Contractor surveillance procedures. The COR must also have enough knowledge of Contractor activities to properly review and disposition monthly invoices. The COR cannot assume duties until assigned by the requiring agency (FC/FD), completed COR required training (CO/QAPC and CO led training), and appointed by the CO.

2.2.6 Contracting Officer Representative (COR) Alternate Mr. Jordan Wescott: The COR is ultimately responsible for implementing all QA matters and procedures. COR(s) should be at a level where other duties don’t prevent proper oversight of Contractor performance. The COR must maintain both technical competency and evaluation proficiency in Contractor surveillance procedures. The COR must also have enough knowledge of Contractor activities to properly review and disposition monthly invoices. The COR cannot assume duties until assigned by the requiring agency (FC/FD), completed COR required training (CO/QAPC and CO led training), and appointed by the CO.

2.3 Contractor Key Personnel. TBD

2.3.1 Project Manager. The Contractor’s Project Manager is responsible for the overall execution of the task order. This individual will serve as the focal point for the PM/COR assigned to the contract and may interface with the customer, as needed.

2.3.2 Project Manager. The Project/Task Lead is responsible for technical or quality control oversight; this individual may be the Project Manager.

3.0 CONTRACT SURVEILLANCE. The COR function is responsible for a wide range of surveillance requirements that effectively measure and evaluate Contractor performance. The Contractor, not the Government, is responsible for task order management and Quality Control (QC) actions to meet the terms of the contract. However, because of the non-commercial nature of the work performed by Contractors, the Government cannot rely solely on the Contractor’s quality program but must have a method to oversee the entire contracting process. The Government does this through the QASP and can inspect or test all services called for by the contract, to the extent practicable, at all times and places, during the term of the contract. The COR contribution is comprised of professional, non-adversarial relationships, which include positive and open communications with the CO, PM, QAPC, and the Contractor. The foundation of this relationship is built upon regular, objective, fair, and consistent COR evaluations of Contractor performance against contract requirements, and the ability to discuss inspection results, trends, and items of mutual interest with the Contractor. The COR uses the methods in this QASP and the PWS to achieve this goal.

3.1 Surveillance Methods. Three primary methods of surveillance are normally used: one hundred percent inspection, periodic audits and customer complaints. Other methods of surveillance must be fully documented and approved by the CO and QAPC before implementation. Information obtained from these activities may be used by the FD/FC to ascertain whether or not Contractor performance is compatible with contract and mission objectives and by the CO for actions relating to the Contractor.

3.1.1 One Hundred Percent Inspection. This method will be used for Contractor submissions, products, and deliverables (tangible items). CORs will use the One Hundred Percent Inspection method to evaluate all reports delivered under Section 4.10 the PWS. The COR, with assistance from the FD and CO if necessary, accepts or rejects these products based on the criteria and performance standards as outlined in the PWS. The COR also documents all results of inspections actions, and subsequent Contractor re-performance taskings, if applicable.

3.1.2 Periodic Audit. This type of surveillance consists of evaluating products/services not surveyed via one hundred percent inspection or by customer survey, and is implemented at a pre-defined frequency. An example of periodic audit is performing monthly inspections of class training conducted by the Contractor or weekly inspection of a Contractor’s process in developing training manuals.

3.1.3 Customer Survey or Customer Complaint. Although usually not a primary method, this is a valuable supplement to other, more systematic methods of surveillance. For example, in a case where random sampling indicates unsatisfactory service, customer complaints can be used as substantiating evidence. In certain situations, where customers complain consistently when the quality of performance is poor (e.g., dining facilities, custodial services), customer surveys and customer complaints may be a primary surveillance method, and customer satisfaction an appropriate performance standard. In all cases, complaints will be documented, preferably on a standard form. All customer complaints shall be forwarded to the COR. Information obtained from customer complaint activities may be used by the CO for actions relating to the Contractor.

3.1.4 The Contractor shall also ensure that each of their staff completes the annual training per the direction of the Government Security Manager to include the Combat Trafficking in Persons Requirement that IAW PGI 222.17. The Government the COR(s) along with the Office of Special Investigations (OSI) will conduct an Annual Surveillance on the Contractor's performance concerning Trafficking in Persons. This Annual Surveillance may involve an examination of the Contractor(s) and Sub-Contractor(s) business practices to include the following:

a.

All paperwork pertaining to the Contractor(s) and Sub-Contractor(s) activities.

b.

All electronic (e.g. e-mail, facsimile, text messaging, and telephonic) traffic pertaining to the Contractor(s) and Sub-Contractor(s) activities.

c.

Facilities pertaining to the Contractor(s) and Sub-Contractor(s) activities.

d.

Interviews with Contractor(s) and Sub-Contractor(s) personnel.

3.2 Surveillance Documents.

3.2.1 Deliverables. All deliverables shall be reviewed for timeliness, accuracy and format. If a deliverable due date falls on a weekend or holiday, the Contractor shall submit the deliverable on the work day following the due date. 95% of deliverables must be received in format/by due date in accordance with the PWS. Remaining 5% or any documents needing correction must be submitted within three (3) business days.

Table 1 Deliverable Schedule:

Government COR/Actual documents by via electronic format.

Government COR/Microsoft via electronic format.

3.2.2 Services Summary. The Services Summary is a list of critical tasks that must be performed by the Contractor, the performance threshold, and the method of surveillance the COR will use to validate/inspect these tasks.

Table 2 Services Summary:

Student Feedback and Periodic Audit.

The contractor shall submit a Curriculum Vitae (CV) of contractor personnel delivering the course content.

The contractor shall administer a Participant Course Evaluation survey to all attendees and provide the evaluations to the Government.

Contractor shall ensure adequate lodging is available for up to 27 executive participants.

Student Feedback and Periodic Audit.

3.2.3 Surveillance Checklist (Attachment 3). The COR is responsible for developing standard procedures for Surveillance Checklists and may request QAPC assistance; the Surveillance Checklist serves as the guideline for all inspections. As a minimum, the Surveillance Checklist will include the items in the Services Summary and Deliverables Tables. The COR may also add other inspection activities/surveillance items to the Surveillance Checklist, with applicable PWS references, and obtain permission from the QAPC. These items may fluctuate throughout the course of the contract based on customer needs, identified risks, or issues related to Contractor performance. The COR will document all inspections conducted using the Surveillance Checklist and maintain them in the COR folder.

3.3 Assessing Performance. CORs must document Contractor performance on each contract, as well as the scope and purpose of any inspections. Documentation will be maintained in the COR’s folder. CORs will also assess and rate Contractor performance as it relates to contract requirements. Performance Assessment may take the form of notes, assessment results from the Surveillance Checklist, non-conformance notifications, quarterly Performance Assessment Reports (PAR) and annual Contract Performance Assessment Reports (CPAR). Each builds on the other and provides a thorough documented history of Contractor performance.

3.3.1 Inspections. The COR may perform inspection/surveillance at any time during contract performance IAW FAR 252.246-2 through 5, as applicable. The COR will retain all audit and inspection documentation until contract closeout, unless otherwise approved by the CO.

3.3.2 Non-conformances and Deficiencies. Audit or Inspection results that fall below contractual standards shall be identified as non-conformances or deficiencies. The COR will notify the contactor IAW para 3.3.3.2 and document all non-conformances, referencing the specific PWS requirement that the Contractor did not meet, document the name of the individual contacted, and date and time of notification. If the Contractor fails to take corrective actions in a reasonable time-frame, the COR will notify the CM/CO and QAPC for further actions.

3.3.2.1 Types of Non-conformance.

3.3.2.1.1 Minor Non-conformance: A minor non-conformance is a non-conformance, which by itself does not adversely impact mission, safety of personnel and/or equipment, performance (quality), schedule (delivery), or cost. A minor non-conformance normally does not increase risk to the Government.

3.3.2.1.2 Major Non-conformance: A major non-conformance is a non-conformance that adversely impacts mission, safety of personnel and/or equipment, performance (quality), schedule (delivery), or cost. A major non-conformance also increases risk to the Government. An example of increasing risk would be recurring non-conformances, which indicates inadequate preventive measures or actions, thus lowering the Government’s confidence that quality services will be provided on time and at cost.

3.3.2.2 Non-conformance Notification. If a deficiency is found during inspection of the Contractor’s performance, the COR will determine the type of notification based on the category of the non-conformance (see Table 3) and pursue non-conformance resolution per paragraph 3.4.

TABLE 3 – NON-CONFORMANCE NOTIFICATIONS.

Minor

Step 1: First Notice

Step 2: Corrective Action Request (CAR)

3.3.2.2.1 First or Second Notices (see Attachment 1). As shown in Table 3, First and Second Notices provide written notification from the Government to the Contractor of minor non-conformances. Initial minor non-conformances should be addressed with a First Notice to the applicable Contractor representative. The COR will notify both the CM/CO and the QAPC of the issue, and prepare the notice for CO signature. The COR will provide a final, signed copy to the CM/CO and QAPC. The CO will provide the First notice to the contractor with a suspense date for the contractors reply. If the Contractor fails to respond by the specified due date, the COR will notify the CO/QAPC for action and annotate the Contractor’s non-responsiveness in the quarterly PAR. If the Contractor fails to resolve the issue as documented in the Notice, the COR should elevate the issue to the next level (CAR). Corrective/preventive actions initiated and/or completed by the Contractor will be verified by CORs on the scheduled inspection and documented accordingly.

3.3.2.2.2 Corrective Action Requests (CAR) (see Attachment 2). As shown in Table 3, a CAR provides written notification from the Government to the Contractor of significant performance discrepancies. The COR will recommend to the CM/CO and QAPC that a CAR be issued when previous notifications failed to result in corrective actions or when severe deficiencies exist. If a CAR is deemed necessary, the QAPC will assist the COR in filling out the CAR. The draft CAR will be routed NLT ten duty days after the event to the CO for review and signature, as the CO is the only person who has authority to issue a CAR to the Contractor.

3.3.2.2.2.1 To satisfactorily close-out a CAR, the following criteria must be met: the Contractor must meet the suspense; the corrective action must have already begun; and the Contractor's actions must correct the deficiency and if applicable, be within specified standards.

3.3.2.2.2.2 If the corrective action is deemed UNSATISFACTORY by the COR, annotate the CAR accordingly and forward it to the CO with a brief explanation, including an estimate of the Government's loss due to the deficient performance. The estimate may account for loss or destruction of property, supplies, or equipment, and hours lost to re-perform the task. If the task cannot be re-performed, indicate the number of hours it ordinarily would have taken to do the task.

3.3.2.2.3 The COR will maintain a non-conformance log documenting all First Notices and CARs. The log will include the date sent to the Contractor, issue, due date for response and actual response date. If a detailed response is required, include a link to the response and include the response in the applicable COR folder tab.

3.3.3 Quarterly Services Summary Reporting – CORs will perform services summary inspections (table 2) and document in PAR.

3.3.4 Quarterly Performance Assessment Report (PAR) (Attachment 4). The COR will accomplish quarterly PARs documenting Contractor performance for the preceding quarterly performance period. This is a critical document giving the CO performance oversight on all services and ensures all products during the quarterly period were provided to the Government IAW all contract and TO requirements. The QAPC will notify/suspense the COR for providing information on the Contractor’s performance. The COR will provide the both QAPC and FD a copy of the completed report.

3.3.4.1 Quarterly PAR Evaluation Categories. The quarterly PAR will also provide a brief summary of Contractor performance covering, as applicable, the categories in Table 4 below.

TABLE 4 – PERFORMANCE ASSESSMENT REPORT (PAR) EVALUATION CATEGORIES

3.3.4.2 Performance Rating Definitions. The COR rates each of the quarterly PAR categories, as applicable, based on the Contractor’s actual performance during the reporting period (see Table 5 below). These ratings can also serve as a basis for CPARs. Ratings will not be arbitrary or reflect an emotional assessment of the Contractor’s performance. Any ratings other than ‘Meets Standards’ must be appropriately justified. For instance, if the COR rates the Contractor’s performance as ‘Exceeds Standards,’ the COR must show what PWS was exceeded and how that benefitted the Government.

TABLE 5 – PERFORMANCE RATING DEFINITIONS

3.3.5 Annual Contract Performance Assessment Reporting System (CPARS) –

The Contractor Performance Assessment Reporting System (CPARS) http://www.cpars.csd.disa.mil/cparsmain.htm ) is the Department of Defense (DoD) Enterprise Solution for collection of Contractor Past Performance Information (PPI) as required by the Federal Acquisition Regulation (FAR). CPARS is a web-enabled application that collects and manages a library of automated Contractor report cards. An annual CPAR will be written on all contracts that exceed $1M total contract value. The COR (or PM) will be designated as the Assessing Official Representative (AOR) in CPARS; the written assessment in CPARS will begin annually from the start date of the contract and forwarded to the Contract Manager within 60 days of the contract start date anniversary. The AOR will use objective evidence from previous quarterly PARS, monthly Services Summary, inspection logs, 1st notices, CARS, and correspondence letters for CPARs assessment. A CPARS quality checklist is available at the following link. http://www.cpars.csd.disa.mil/cparsfiles/pdfs/CPARSQualityChecklist.pdf.

4.0 WIDE AREA WORK FLOW (WAWF).

The COR will accept or reject all Receiving Reports submitted by the Contactor in WAWF within 15 days of submittal by the Contractor when possible. To access the receiving report, log into WAWF, select the Government, Acceptor, Acceptance Folder tab; then type in the contract and delivery order (task order) number. The Shipment Number column in WAWF is the Receiving Report number

4.1 Validating Receiving Reports. The COR will review and validate all WAWF receiving reports (Shipment Number in WAWF) ensuring that:

· The Contractor submitted a combination invoice/receiving report

· The report reflects the correct contract

· Beginning and ending dates being billed are stated and accurate

· The correct CLINs are charged for billing period

· A description of the services is provided

· The correct amount is charged, to include applicable handling rates

· Invoiced travel was pre-approved

· Back-up documentation is attached that includes a Contractor invoice with current and cumulative hours (for FFP LOE contracts) and amounts, as applicable

· A breakout of any other charges, as agreed to by the CO or required by the contract, is attached in WAWF or has been provided to the COR

4.2 Certifying and Accepting Receiving Reports.

4.2.1 Certification of services is ALWAYS performed by the appointed COR. After validation of services is performed by the COR per paragraph 4.1 above, the COR will select the “CQA” (Contract Quality Assurance) button to verify that the invoice was accurate, this step will normally select the Acceptance: button also. If anyone other than the COR (typically the Resource Advisor (RA)) is accepting services for the task order, the COR MUST certify in writing to the acceptor that services have been inspected/validated, the acceptor must then state in the comments block (under Misc. Info tab) that the COR has reviewed, validated and certified services/invoices. In all cases, the COR will retain documentation (electronic or hard copy) showing COR validation and certification, as well as final acceptance of Contractor invoices.

4.2.2 COR Expenditure Log. After certification of an invoice in WAWF, the Primary COR will update the expenditure log documenting/tracking certified invoice expenditures against funds allocated on the contract against each CLIN. The QAPC will provide a template, if requested. Funding belongs to the organization where the COR resides; the COR expenditure log allows the COR to ensure that funds are available for the work performed, to include travel and materials.

4.3 Rejecting Receiving Reports.

4.3.1 If a receiving report is rejected in WAWF, the COR will go to the Miscellaneous Info tab to complete the comments block as follows:

· List details regarding all unsatisfactory services or incorrect invoice information

AND

· Instruct the Contractor to contact DFAS to have the invoice rejected back to the Contractor; this will allow the Contractor to make the same correction on both the receiving report and invoice, since combination invoices/receiving reports are required.

4.3.2 The rejecter will then return to the Header tab and click the “Reject to Initiator” block. Anyone other than the COR may reject receiving reports in WAWF, but only when services/invoices have been reviewed and rejected by the COR in writing per the paragraph above. After rejecting the receiving report, e-mail the CM/CO the contract number, receiving report number, and reason for rejection.

5.0 COR FOLDER.

The Primary COR will establish and maintain a COR folder for each task order. The COR folder must contain the tabs and contents outlined below in Table 6. ALL surveillance records are considered FOR OFFICIAL USE ONLY documents, and must be marked and safeguarded as such. Documentation is kept for the life of the contract and provides necessary continuity should COR duties transition during the course of the contract. The FD/FC and COR will receive instructions from the CO on disposition or archival procedures upon contract completion.

TABLE 6 – COR FOLDER TABS & CONTENTS

(January 15, 2016)

MEMORANDUM FOR (TBD)

FROM:

(CO’s Name and Office Symbol)

SUBJECT:

First/Second Notice for Contract for Business Engineering & Technical Assistance on – contract (#TBD)

1. Contracting Officer Representative: (Ms. Penny O’Neil)

2. Office: (SAF/US(M)S)

3. Date response due from Contractor: (dd-mmm-yyyy)

4. Requirement Reference: (PWS section and paragraph; Deliverable, Services Summary (SS))

5. Performance Requirement: (State what action the Contractor should have performed IAW the reference above)

6. Description of Non-Conformity: (provide objective evidence of the Contractor’s performance discrepancy(ies))

(CO signature block))

CORRECTIVE ACTION REPORT (CAR)

(If more space is needed, use reverse and identify by number)

1. CONTRACTOR

2. CONTRACT NUMBER

3. TYPE OF SERVICES

4. FUNCTIONAL AREA

5. SUSPENSE DATE

6. CONTROL NUMBER

7. DEFICIENCY MAJOR MINOR

FINDING:

FINDING IMPACT:

Please respond with a written corrective action plan that details the corrective action of the cited deficiency, the cause of the deficiency, and actions taken to prevent recurrence by Suspense Date in Block 5. If date was not entered in Block 5, the contractor is not required to provide a response.

8. QUALITY ASSURANCE PERSONNEL (QAP)

TYPED NAME AND GRADE

SIGNATURE AND DATE

9. ISSUING AUTHORITY

TYPED NAME AND GRADE

SIGNATURE AND DATE

10. QAP RESPONSE TO CONTRACTOR CORRECTIVE ACTION AND ACTION TAKEN TO PREVENT RECURRENCE

11. QAP DETERMINATION

ACCEPTED REJECTED

12. CLOSE DATE

QUARTERLY PERFORMANCE ASSESSMENT REPORT (PAR)

COR:

Report Month:

Organization: HQOSI/XI Date Submitted:

Contract No.:

Contractor:

Contract Title:

Contractor Team Lead:

Method of Performance Evaluation

· At least every month, performance will be evaluated by using one or more of the methods checked below. Check all that apply or may be used if deeper levels of scrutiny is required.

RATING: Exceptional Very Good Satisfactory Marginal Unsatisfactory TRAVEL: Date______ Destination________________

(COR signature)

PERFORMANCE SUMMARY

CRITERIA:

a. Quality –

b. Schedule –

c. Cost Control – N/A

d. Business Relations –

e. Management –

f. Small Business –

g. Other Areas –

a. Personnel Attraction/Retention:

b. Security

TECHNICAL PERFORMANCE:

DEFICIENCIES:

AREAS FOR IMPROVEMENT:

ADDITIONAL COMMENTS: