ATTACHMENT 5 TO RFQ SP330020Q0108 QASP.pdf

Quality Assurance Surveillance Plan (QASP) (revised September 2019) Page | 1

Quality Assurance Surveillance Plan (QASP)

For

Maintenance Support Services for Lenel Electronic Security System (ESS) and PELCO Closed Circuit television (CCTV) at DLA

Distribution Norfolk, Virginia

SP3300-20-D-500X, 00XX / SP3300-XX-X-500X

(Issue Date/Revision Date)

1. SCOPE

DLA Distribution Norfolk, Virginia has a requirement for support services for the Lenel Electronic Security Sustem (ESS) and the PELCO Closed Circuit Television (CCTV).

2. PURPOSE

The QASP represents the Government's acceptance and inspection program for performance requirements, and agencies must ensure Government Quality Assurance (QA) is conducted before Contractor products or services are accepted by or under the direction of Government personnel.

The QASP documents this program to provide a measure of the quality and timeliness of products and services provided by the Contractor. The Government, as the recipient of the Contractor’s products and services, retains the responsibility for developing and implementing QA. Implementation of the QASP assists in providing validation that the quantity, quality, and timeliness of products and services received comply with the contract performance requirements, to include DLA Distribution policies and procedures.

This QASP is a “living document” and the Government may review and revise it on a regular basis. However, the Government shall coordinate changes with the Contractor. Updates shall ensure that the QASP remains a valid, useful, and enforceable document. Copies of the original QASP and revisions shall be provided to the Contractor and Government officials implementing surveillance activities.

3. GOVERNMENT ROLES AND RESPONSIBILITIES (AS APPLICABLE)

The following is a listing of responsibilities for the personnel that shall oversee and coordinate surveillance activities. See Attachment 1 for a list of personnel assigned to these roles.

a. CONTRACTING OFFICER (KO) - The KO shall ensure performance of all necessary actions for effective contracting, ensure compliance with the contract terms, and shall safeguard the

ATTACHMENT 5 TO RFQ SP330020Q0108 Page 1 of 11

Quality Assurance Surveillance Plan (QASP) (revised September 2019) Page | 2 interests of the United States in the contractual relationship. The KO shall also assure that the Contractor receives impartial, fair, and equitable treatment under this contract. The KO is ultimately responsible for the final determination of the adequacy of the Contractor’s performance.

b. ACQUISITION SPECIALIST (AS) - The AS acts as an acquisition consultant and serves as liaison between KO, the COR/TPOC/QAE, and the Contractor.

c. CONTRACTING OFFICERS REPRESENTATIVE (COR) - The COR is responsible for technical administration of the contract and shall assure proper Government surveillance of the Contractor’s performance. The COR shall keep a quality assurance file. At the conclusion of the contract or when requested by the KO, the COR shall provide documentation to the KO. The COR is not empowered to make any contractual commitments or to authorize any contractual changes on the Government’s behalf. The Contractor shall refer any changes they deem may affect contract price, terms, or conditions to the KO for action. Responsibilities include, but are not limited to:

• Ensuring COR training and other certification requirements are up to date

• Filing required reports and documents with the AS and KO as required; upload documents to CORT as required

• Preparing and coordinating annual contract performance assessments and reports

• Maintaining file copies of all documents submitted by the Contractor

• Reviewing and accepting the Contractor’s Quality Control Plan; rework as necessary

• Becoming thoroughly familiar with all of the terms and conditions of the contract

• Maintaining a copy of the current QASP; understanding the requirements of the QASP

• Participating in requirement teams and coordinating contract modifications

• Coordinating nonconforming surveillance activities with the Contractor

• Reviewing and accepting Corrective Action Plans (CAPs) and monitor through closure

• Performing quality assurance surveillances as required or necessary (i.e. no TPOC or

QAEs have been assigned)

d. TECHNICAL POINT OF CONTACT (TPOC) – A TPOC is similar in scope and responsibility to a COR but is usually appointed when services are being provided at multiple locations and the COR is located at a central location. The TPOC must meet the same training requirements and standards as a COR. The TPOC reports through the COR to the KO. If assigned, the TPOCs responsibilities include, but are not limited to:

• Ensuring COR training and other certification requirements are up to date

• Filing required reports and documents with the AS and KO as required; upload documents to CORT as required

• Preparing and coordinating annual contract performance assessments and reports

• Maintaining file copies of all documents submitted by the Contractor

• Reviewing and accepting the Contractor’s Quality Control Plan; rework as necessary

• Becoming thoroughly familiar with all of the terms and conditions of the contract

ATTACHMENT 5 TO RFQ SP330020Q0108 Page 2 of 11

Quality Assurance Surveillance Plan (QASP) (revised September 2019) Page | 3

• Maintaining a copy of the current QASP; understanding the requirements of the QASP

• Participating in requirement teams and coordinating contract modifications

• Coordinating nonconforming surveillance activities with the Contractor

• Reviewing and accepting Corrective Action Plans (CAPs) and monitor through closure

• Performing quality assurance surveillances as required or necessary (i.e. no QAEs have been assigned)

e. QUALITY ASSURANCE EVALUATOR (QAE) - A QAE, if assigned, is typically a technical and subject matter expert in the products or services that are being provided and performed by the Contractor. The QAEs primary role is with the inspection, documentation, and reporting of Contractor performance through the TPOC or COR. If assigned, the QAEs responsibilities include, but are not limited to:

• Filing required reports and documents with the COR/TPOC as required

• Reviewing the Contractor’s Quality Control Plan; recommend rework as necessary

• Becoming thoroughly familiar with all of the terms and conditions of the contract

• Participating in requirement teams and coordinating contract modifications

• Coordinating nonconforming surveillance activities with the Contractor

• Performing quality assurance surveillances as assigned

• Maintaining a copy of the current QASP; understanding the requirements of the QASP

• Monitor the Contractor’s performance under a CAP and report both positive and negative findings to the COR/TPOC.

f. QUALITY ASSURANCE TEAM (QAT) – The QAT is comprised of all individuals assigned a role in the quality assurance surveillance and contract oversight process. The size and scope of this team will vary based on the individual characteristics and complexity of the products or services being provided by a contract entity.

4. QUALITY CONTROL AND QUALITY ASSURANCE

a. QUALITY CONTROL - The overall responsibility for performance quality rests with the Contractor. Through their Quality Control Plan (QCP), they are responsible for monitoring, reporting, and correcting performance issues as they are discovered. The QCP is considered a living document (just like this QASP) and should be reviewed and revised as necessary throughout all periods of performance.

b. QUALITY ASSURANCE - The QAT, through assigned QAEs, TPOCs, and COR, are responsible for conducting scheduled and unscheduled performance checks (surveillances) to gauge the effectiveness of the Contractor’s quality program. Findings or nonconformances discovered by the QAT will be shared with the Contractor, as they are discovered, in order to permit the Contractor to both identify corrective actions necessary to improve performance as well as improve their quality control efforts.

5. QUALITY ASSURANCE SURVEILLANCE PROCESS

ATTACHMENT 5 TO RFQ SP330020Q0108 Page 3 of 11

Quality Assurance Surveillance Plan (QASP) (revised September 2019) Page | 4

The foundation to the quality assurance surveillance process is a firm and thorough understanding of the requirements contained in the contract. This includes the Performance Work Statement (PWS) or Statement of Work (SOW), Contract Data Requirements Listings (CDRLs), Technical Exhibits, and any referenced SOPs, Handbooks, Manuals, Instructions, or other guidance cited in the requirement. Additionally, and equally important, a clear understanding of the Contractor’s Quality Control Plan (QCP) is necessary in order to have insight on how the Contractor has proposed to evaluate and report their internal quality efforts.

With this understanding, the individual(s) that will be conducting the quality assurance surveillances can begin their inspection and verification processes. DLA Distribution uses a 7- Step Process to assist in making this easier. The steps are:

• Identify – Precisely identify the objective of the surveillance and the associated standards

• Review – Review the contract, any cited references, and other pertinent documents

(CDRLs, QCP, SOPs, Agreements, etc.)

• Conduct – Conduct the surveillance by paying close attention to the object or processes being surveilled to include environmental factors at the time of the surveillance; Be Observant, Objective, and Document all findings both good and bad

• Document – Document your surveillance results as they were observed

• Analyze – Review your documentation and analyze the results looking for missing data.

Look for the “Root-Cause” to problems by asking yourself “why” repeatedly until a potential root cause is discovered

• Report – Be clear and concise but thorough when reporting your surveillance results

• File – File your surveillance documentation

6. QUALITY ASSURANCE SURVEILLANCE METHODS

Contract requirements can be surveilled using a variety of techniques and methodologies and many requirements can be surveilled using multiple techniques. It is up to the individual conducting the surveillance to determine the most appropriate method for a particular requirement. DLA Distribution utilizes the four methods below.

• DIRECT OBSERVATION – This method should be used to physically observe a process and procedures being performed

• 100% INSPECTION – Generally the most time consuming surveillance method; ideally used when Lot Sizes are relatively small (25 or less)

• RANDOM SAMPLING – Ideal for use with large Lot Sizes; a statistical sample is selected from the Lot. The table below will be used to determine Sample Size based on a range for the Lot. Accept/Reject numbers are included

ATTACHMENT 5 TO RFQ SP330020Q0108 Page 4 of 11

Quality Assurance Surveillance Plan (QASP) (revised September 2019) Page | 5

Figure 1 - Random Sampling Plan

• VALIDATED CUSTOMER COMPLAINT – A customer complaint that has proven to be valid based on subsequent investigation; not all complaints are valid or reasonable unless they can be verified. Customer complaints should be documented in writing and submitting by the customer. The customer may be any stakeholder in the process and not necessarily the end user customer.

7. DOCUMENTING AND REPORTING PERFORMANCE

All surveillance activities must be thoroughly documented. This is equally important for conforming results as it is for nonconforming results. All surveillance documents become an official part of the Contract File and are preserved throughout the periods of performance, contract closeout, and then further for the required document retention period. For this contract, two types of performance reports will be utilized, Non-Acceptable Performance Level (Non- APL) and Acceptable Performance Level (APL). All non-conforming surveillance outcomes shall include a detailed explanation that clearly identifies the non-conformance or failure of Non- APL Performance or APL Performance requirements in the Monthly COR Report. A “COR Report” template is available in the Surveillance and Performance Monitoring Tool (SPM) in the Procurement Integrated Enterprise Environment (PIEE).

• NON-APL PERFORMANCE - The COR/TPOC will utilize the Contract Oversight Surveillance Report (COSR) to document oversight activities and document contractor non-APL performance. Non-APL surveillances should occur multiple time throughout the month.

o SCHEDULED SURVEILLANCE - The COSR consists of a series of questions that are derived directly from the requirements document (PWS/SOW and other contract references). The questions are broadly written to permit the individual conducting the surveillance to inspect any portion of the requirement.

o CHANCE DISCOVERY (Unscheduled or Chance Surveillance) – A Chance Discovery can be used at any time to document a nonconformance for a requirements for which there isn’t a specific surveillance question, for a condition

ATTACHMENT 5 TO RFQ SP330020Q0108 Page 5 of 11

Quality Assurance Surveillance Plan (QASP) (revised September 2019) Page | 6 that is discovered during other than regular scheduled surveillance reporting, or a safety or security issue that needs to be documented. The COSR has blank spaces that can be used to document this Chance Discovery. The process for documenting and methods used are the same as for any scheduled surveillance.

o The COSR is required to be completed on a monthly basis and attached to the “Contracting Officers Representative (COR) Report” (COR Report).

• APL PERFORMANCE – The COR/TPOC will utilize the “APL Report” to document conformance to acceptable performance levels (Conforming/Non-Conforming). The APLs are specific timeliness or quality metrics that are collected and analyzed on a monthly basis to give a general idea of the Contractor’s ability to meet specific measurable targets. APLs are normally only reviewed after the end of the month, as APLs are measured on a full month’s performance.

o The APL Report is required to be completed on a monthly basis and attached to the “COR Report”.

• The QAT is required to provide documentation for both conforming as well as nonconforming performance each month.

o ACCEPTABLE PERFORMANCE (Conforming).

The Government shall document positive performance within the Contract Oversight Surveillance Report and submit this report to the Contracting Office based on Contract Requirements Monthly.

o UNACCEPTABLE PERFORMANCE (Nonconforming).

The QAT shall document all unacceptable performance within the

Contract Oversight Surveillance Report and submit this report to the Contracting Office based on Contract Requirements Monthly. If the COSR is completed by a QAT member other than the COR, the COSR shall be forwarded to the COR for acceptance. The COR shall inform the contractor within one working day of each non-conforming surveillance.

This shall be in writing unless circumstances necessitate verbal communication, which the COR will document. The COR shall retain all documentation regarding surveillance activities and outcomes within the COR file.

Reporting of non-conforming surveillances is used to determine the appropriate course of action following the documentation of the contractor’s non-compliance and convey the significance of the surveillance, the individual requirement, a count of the consecutive nature of the non-conformance and identifiable trends. If the nonconformance has occurred previously or is an ongoing issue, a trend statement must be included to document the numbers of occurrences of the same nonconformance over a quarterly, semi-annual, or annual basis.

For each nonconformance (both Non-APL and APL), the Contractor shall prepare and submit a Corrective Action Plan (CAP) to the COR/TPOC.

ATTACHMENT 5 TO RFQ SP330020Q0108 Page 6 of 11

Quality Assurance Surveillance Plan (QASP) (revised September 2019) Page | 7

The CAP shall provide detailed information regarding an explanation of what the nonconformance was, root-cause analysis to determine the cause of the probable cause of the nonconformance, corrective/preventive measures the Contractor will follow to correct the problem, and roles and responsibilities associated with getting the nonconformance corrected.

The COR/TPOC will evaluate the CAP and accept the CAP if the evaluation determines the CAP should be successful. If the COR determines the CAP will not be successful or there is missing information, the COR/TPOC will return the CAP to the Contractor for re-work. The QAT will conduct surveillance on the accepted and implemented CAP to document the effectiveness of the CAP to correct the non-conformance. If through QA surveillance non-conformances are continue to be identified, the COR/TPOC will discuss with the Contractor and may request a revised CAP or evaluate the non-conformance to the Contracting Officer for other actions.

The “COR Report”, must be completed monthly basis and includes the completed COSR, APL Report and any CAPs that were prepared during the surveillance reporting period as attachments. The COR/TPOC submits the COR Report to the KO and AS for review and file and uploads the report in PIEE. The COR Report must be submitted NLT the 10th day of the month following the reporting month. All document submissions must be electronic with the COR/TPOC retaining a copy for their records and file.

8. WORKFLOW

9. SPECIAL INSTRUCTIONS/CONTRACT SPECIFIC REQUIREMENTS

A Contract Oversight Surveillance Report (COSR) is not required. Task performance requirements will be monitored using the monthly Acceptable Performance Level (APL) Report.

Surveillance Conducted &

Documented on

COSR

Nonconformances Identified?

Provide Contractor with a copy of the Nonconforming

COSR

No

Yes Contractor prepares and submits a CAP

Sign COSR. Attach COSR and the APL Report to the “COR

Report”

Sign COSR. Attach COSR, CAP, and APL Report to the “COR

Report”

ATTACHMENT 5 TO RFQ SP330020Q0108 Page 7 of 11

Quality Assurance Surveillance Plan (QASP) (revised September 2019) Page | 8

Contracting Officer’s Representative (COR)

TPOC

Attachments

1. Point of Contact Listing for this Contract

2. COSR

3. Corrective Action Plan (CAP) - blank

ATTACHMENT 5 TO RFQ SP330020Q0108 Page 8 of 11

Attachment 1: Assigned Personnel:

Assigned KO: Jennifer Fasting

Organization: DLA Distribution J7 Acquisition Operations

Telephone: 717-770-7012

Email: Jennifer.Fasting@dla.mil

Assigned AS: Daniel Herring

Telephone: 717-770-6030

Email: Daniel.Herring@dla.mil

Assigned COR: Frederick Jolin

Telephone: 717-770-2939

Email: Frederick.Jolin@dla.mil

Assigned TPOC: XXX-XXX

Telephone: XXX-XXX-XXXX

Email: XXX.XXX@dla.mil

ATTACHMENT 5 TO RFQ SP330020Q0108 Page 9 of 11 mailto:Daniel.Herring@dla.mil

Electronic Security System

01- Do the SP employees have all the necessary training and certifications IAW contract requirements to work on the systems?

Surveillance Comments

Nonconforming Conforming Requirement

02- Did the SP provide who the acceptable contract manager is to the KP and Contracting Officer's Representative in writing?

Contract Oversight Surveillance Report

1. Contract # / Service 2. Contract Location

3. Contract Performance Period 4. Evaluation Period

From - To -

5. Contract Service Provider

6. Method(s) of Surveillance (check all that apply for this surveillance period)

Direct Observation Random Sampling 100% Inspection Customer Complaint Other (explain)

Surveillance Methods Described -

- Direct Observation is the visual examination of processes and procedures by an evaluator who is physically present and watching individuals actually performing the work to determine whether the processes conform to the requirements.

- Random Sampling is correctly performed by selecting a statistically significant number of samples from a lot where all members of the lot have the same chance of being selected. Random Sampling is used to reduce the time and effort of validating conformance to requirements when a definable lot of significant size exists.

- 100% Inspection method evaluates all outputs of a particular requirement. Every item in a lot is evaluated for all or some of the characteristics in the specification. (Recommend if Lot Size is 25 or less)

- Validated Customer Complaints are customer complaints that prove to be valid upon subsequent investigation. They can be used as a trigger for conducting surveillances, and are included as a part of the surveillance report. When a valid complaint is received and included in a surveillance activity, strongly consider scheduling targeted follow up surveillance to ensure the problem has been corrected. These can prompt corrective actions, preventative action plans, and performance improvement plans.

7. Overall Surveillance Evaluation

Conforming Nonconforming

8. Additional Comments / Notes to the Contracting Officer (Either positive or negative)

9. COR/TPOC Certification - I certify that the supplies or services provided by the Service Provider under the terms and conditions of this contract, have been received and accepted unless otherwise noted on this surveillance report.

TPOC Signature COR Signature

ATTACHMENT 5 TO RFQ SP330020Q0108 Page 10 of 11

04- Did the SP personnel possess and safeguard a DoD Common Access Card (CAC)/Access ID Badge of their own?

05- Is the SP requisting all Information Technology (IT) access needed to preform work fourteen (14) days prior to the work start date?

06- Is the SP providing new parts and components that are equal or better when replacing units, parts, components, and materials to the original equipment?

07- Is the SP adhering ti all work control, work schedule, management and staffing requirements IAW the contract?

Is the SP providing all the required mandatory reports to the COR and/or KO IAW the contract?

Is the SP meeting all the requirements of the functional areas set forth in the PWS?

03- Is the SP complying with all DoD and DLA Safety and Security regulatory guidance set forth within the contract?

Nonconforming Conforming Requirement ATTACHMENT 5 TO RFQ SP330020Q0108 Page 11 of 11