Attachment 8 - ESS VII Quality Assurance Surveillance Plan (QASP).pdf

QUALITY ASSURANCE SURVEILLANCE PLAN

For Electronic Security Systems (ESS) VII

Indefinte Delivery/Indefinite Quantatiy Multiple Award Task Order Contracts (MATOC)

1. PURPOSE.

This Quality Assurance Surveillance Plan (QASP) has been designed as the overarching guide for ESS VII MATOC. This MATOC QASP forms the basis of what will be used by the Contracting Officer's Representative (COR) or Quality Assurance Evaluator (QAE)s for individual Task Orders issued under ESS VII. This will provide guidance and a basis for effective and systematic surveillance of all aspects of the ESS VII MATOC. In accordance with AR 11-2, Internal Control Systems, contract administration and quality assurance functions are subject to the internal control review process. Thorough documentation of services provided under this contract is essential.

Each Task Order QASP will provide a systematic method to evaluate performance for the stated contract and will explain different aspects such as:

What will be monitored.

How monitoring will take place.

Who will conduct the monitoring.

How monitoring efforts and results will be documented.

Each TO QASP will be based on the premise that the Contractor, and not the Government, is responsible for management and quality control actions to meet the terms of the contract. The QASP does not detail how the contractor accomplishes the work. Rather, the QASP is created with the premise that the contractor is responsible for management and quality control actions to meet the terms of the contract. It is the Government’s responsibility to be objective, fair, and consistent in evaluating performance. In addition, the Government has encorporated into the QASP that unforeseen and uncontrollable situations may occur.

The following FAR clauses may apply depending on contract type:

52.246 -4 – Inspection of Services -- Fixed-Price.

52.246-1 – Contractor Inspection Requirements

2. GOVERNMENT ROLES AND RESPONSIBILITIES.

The following personnel shall oversee and coordinate surveillance activities. Each individual task order will specifically identify the employee that will fill the roles identified below:

a. Contracting Officer (KO) - The KO shall ensure performance of all necessary actions for effective contracting, ensure compliance with the contract terms, and shall safeguard the interests of the United States in the contractual relationship. The KO shall also assure that the contractor receives impartial, fair, and equitable treatment under this contract. The KO is ultimately responsible for the final determination of the adequacy of the contractor’s performance.

b. Contract Specialist - The contract specialist acts as an acquisition consultant and serves as liaison between the contracting office and the requesting program office.

c. Contracting Officer’s Representative (COR) - The COR is responsible for technical administration of the contract and shall assure proper Government surveillance of the contractor’s performance. The COR shall keep a quality assurance file in the VCE COR Tool. At the conclusion of the contract or when requested by the PCO, the COR shall provide all documentation to the PCO. The COR is not empowered to make any contractual commitments or to authorize any contractual changes on the Government’s behalf. Any contractor or customer requests which may affect contract price, terms, or conditions shall be referred to the PCO for action.

3. PERFORMANCE STANDARDS

Performance standards for each individual task order will be defined at the task order level. The Performance Requirements Summary (PRS) Matrix for ESS VII has been provided in a separate document.

4. INCENTIVES.

The Government shall NOT provide incentives for this contract.

5. METHODS OF QA SURVEILLANCE.

Various methods exist to monitor performance. The COR shall use the surveillance methods listed below may be used in each individual Task Order QASP.

Regardless of the surveillance method, the COR shall always contact the contractor's task manager or on-site representative when a defect is identified and inform the manager of the specifics of the problem. The COR, with assistance from the Contract Speciliaist, shall be responsible for monitoring the contractor’s performance in meeting a specific performance standard.

a. DIRECT OBSERVATION. (Can be performed periodically or through 100% surveillance.)

b. PERIODIC INSPECTION. (Uses a comprehensive evaluation of selected outputs. Inspections may be scheduled [Daily, Weekly, Monthly, Quarterly, or annually] or unscheduled, as required.)

c. 100% INSPECTION. (Evaluates all outputs. Most applicable to small quantity, but highly important services.

May be used where there are written deliverables and stringent requirements such as tasks required by law, safety, or security.)

g. PERIODIC SAMPLING. (Variation of random sampling. However, sample is only taken when a deficiency is suspected. Good follow-up to MIS analysis. Sample results are applicable only for the specific work inspected.

Since sample is not entirely random, it cannot be applied to total activity performance.)

Surveillance results may be used as the basis for actions (to include payment deductions) against the contractor. In such cases, the Inspection of Services clause in the Contract becomes the basis for the PCO’s actions.

6. GOVERNMENT QUALITY ASSURANCE METHODOLOGY AND TECHNIQUES

Government surveillance will begin with regular planned evaluations or audits. Government surveillance will remain flexible enough to adjust to changes required by the Contractor. The evaluation of Contractor performance may be aided by the use of checklists. Generally, excellent performance requires less Government monitoring and actual performance verification while poor performance requires increased degrees of monitoring and verification.

The Government's primary QA methods are based on monitoring the Contractor's QC Program. QA will be conducted as required, under the terms of the Contract Terms and Conditions - Commercial Items (FAR 52.212-4) clause of the Federal Acquisition Regulation and the contract. There are many other methods of evaluation, which may be used by the Government.

Surveillance Procedures. Various evaluation methods are suitable for monitoring Contractor performance. The frequency of surveillance of Contractor performance will be established based on a number of underlying factors to include: the frequency with which certain tasks are performed; the relative importance of specific tasks; and the type task to be monitored. The Government may modify the type and frequency of inspections at its discretion.

Determination as to type of surveillance method used shall be at the discretion of the COR. The COR will be consistent when choosing the most appropriate surveillance method. As a minimum, consideration will be given to the frequency with which surveillance is to be accomplished and the Contractor's immediate past performance as well as to the underlying factors previously mentioned.

Inspection Schedules:

It is the CORs/QAEs responsibility to develop a monthly schedule for activities based on the scope of the surveillance plan. A monthly schedule (plan) will be completed by the last workday of the preceding month and a copy submitted to the KO. This Monthly Surveillance Schedule will include all areas in the Performance Requirements Summary and other areas considered essential to appropriately monitor Contractor performance. The KO will review the schedules to ensure that all requirements of the contract are periodically surveyed.

To build the inspection schedule, the COR/QAE will select the areas and times for the random sampling using standard random sampling procedures. Then periodic checklist items to be surveyed during the month will be programmed into the schedule and should show what the COR/QAE is monitoring each day. Once completed, the schedule shall be marked "FOR OFFICIAL USE ONLY" and not be shown to the Contractor. Changes will be posted weekly and copies sent to the KO. The reasons for changes will be fully documented and explained. Actual surveillance activity must be comparable to the monthly schedule. The KO must be able to observe the COR's/QAE's performance by using the monthly schedule, from observing the COR/QAE perform sampling or other checklist surveillance, to completion of the random sampling and other checklists.

Basic Surveillance Methods. Five basic surveillance methods are described for potential use within task order QASPs; random sampling, planned sampling, 100% inspection, customer complaints, and unscheduled inspection.

Since planned sampling, customer complaints, and unscheduled inspection may not give unbiased estimates of the amount of work performed satisfactorily, they are not as effective when deducting is a major consideration.

Deductions would be limited to the documented work not performed or performed unsatisfactorily. One hundred percent inspection provides excellent documentation for deductions. Random sampling is also good; however, great care must be taken to insure that sample selections are unbiased and that sample sizes are adequate.

Random Sampling. Random sampling is a QA evaluation method designed to evaluate some part, but not all, of the contract requirement being monitored and then, based on statistical theory, to estimate the Contractor's overall level of performance for a given requirement. Using random sampling, any occurrence of a contract requirement is as likely to be evaluated as any other occurrence. The random sampling procedures utilized in this QASP are based on American National Standard ANSIIASQC Z1.4-1993. They consider the maximum allowable deviation from the required standard acceptable quality limit (AQL), the level (intensity) of the evaluation effort, and the population (lot) size. Random sampling requires that an inspection schedule be prepared prior to evaluation for those services that are scheduled for performance. For the purpose of this QASP, schedules must be developed monthly based on the work expected to be accomplished during the next performance period. Surveillance of unscheduled outputs of service is based on samples drawn from accomplished work. Evaluations are conducted by the QAE. Results are compared to performance standards to check conformance.

Initiation of Inspection. Normal sample size inspections will be used at the start of surveillance unless otherwise directed by the KO. Sample lot sizes for normal surveillance are shown in Table II-A, ANSIIASQC Z-1.5-1993.

Sample selection worksheets will be prepared for the sampling guide and used to record information on observations and nonconformity. Each observation in the sampling will be recorded on the sample selection worksheet. These documents will become a formal record for later reference. The tally of observations and nonconformity will be compared to the acceptable number of nonconformity appearing in the relevant sampling guide at the end of the each month. All nonconformity detected during the period of surveillance, even if not of sufficient degree to render the service unsatisfactory in terms of the AQL parameter, shall require corrective action by the Contractor.

Specifics concerning any nonconformity shall be recorded on the reverse side of the sample selection worksheet.

The COR/QAE will have the Contractor initial to acknowledge nonconformity.

If the observed total number of nonconformity is less than the unsatisfactory level, the Contractor's overall performance for the given service evaluation is satisfactory. When the observed total number of nonconformity is equal to or greater than the unsatisfactory level, the Contractor's overall performance is judged to be unsatisfactory.

Further inspection or corrective action is indicated.

Acceptable Performance. When a Contractor's quality control program works, acceptable performance results. If the results of a COR/QAE 's surveillance show consistently acceptable performance, the amount of surveillance can be decreased.

Reduced Sampling Inspection. Inspections can be reduced from normal size when the following conditions have been met for a sampling guide.

The preceding four lots (i.e., the last 4 months) have all been acceptable.

The number of nonconformity in each of the preceding four lots is 50% or less of the acceptable number. For example, with an AQL of 6.5% and a sample size of 32, the acceptance number is 5. If two or less nonconformity were found in each of the last four lots, reduced inspections could be used.

If the COR and the KO agree to use reduced inspections, Reduced Sample Size and Acceptance or Rejection Numbers will be used. Sample lot sizes for reduced surveillance are shown in Table II-C, ANSI/ASQC Z-1.4-1993.

Reduced inspection decreased the sample size. In addition, the acceptance and rejection numbers will change. To make the changes to existing sampling guide, take the following steps:

a. Make sure that the original sampling guide was using the normal sample size. To determine this, compare the lot size with the sample size in the sampling guide.

b. Find the new sample size by using the reduced surveillance table. Take the lot size and find the new reduced sample size.

c. Use the AQL in the sampling guide and the new reduced sample size for the new acceptance and rejection numbers. Note that there is a gap between the acceptance and rejection numbers (for example, sample size 32 and AQL 6.5%, acceptance is 5, rejection is 8). This means that a lot would not be rejected unless eight nonconformity where found and would be accepted if five or less nonconformity were found. However, a number of nonconformity greater than five will be cause for returning to normal inspection (i.e., return to the sample size and acceptance and rejection numbers used in the original sampling guide).

Returning to Normal Inspection. When reduced inspection is in effect, return to normal inspection the next month under the following conditions:

a. When the number of nonconformity exceeds the acceptance number under reduced sampling, or

b. The COR and the KO deem it necessary to return to normal inspection.

Returning to Reduced Inspection. If during the first month of the return to normal inspection, the number of nonconformity found is again 50% or less of the acceptance level, a return to reduced inspection may be done the next month. If the number of nonconformity found exceeds 50%, then normal sampling shall be accomplished until four months of 50% or less of acceptance level nonconformity are found.

Tightened Inspection. Tightened surveillance should be used for area where the Contractor's performance is less than satisfactory. Sample lot sizes for tightened surveillance are shown in Table II-B, ANSI/ASQC Z-1.4-1993.

NOTE: When using random sampling solely for performance evaluation, a second method of inspection may also be used, such as validated customer complaints. Information collected using other evaluation methods can never be combined with information gathered by random sampling for purposes of performance evaluation. In the example above, if there were 5 observed nonconformity in the lot sampled, the Contractor's performance for the service is satisfactory. Two additional validated customer complaints during the same period do not make the Contractor's overall performance for the service unsatisfactory. Evaluation results collected by other methods serve only as supportive data. However, payment deductions can be made for, nonconformity observed through the secondary method as well as for any of the five observed nonconformity if the Contractor did not satisfactorily rework them.

Planned Sampling. Evaluation by planned sampling, like evaluation by random sampling, is designed to inspect some part but not all of the activities being monitored. Specific occurrences of contract requirements that are to be monitored are selected for evaluation prior to their scheduled accomplishment. Planned sampling differs from random sampling in the way in which samples are selected. Sample selection is based on some subjective rationale and sample size is usually arbitrarily determined. Planned sampling, unlike random sampling, does not provide a means of making comparisons between observed and overall performance, and the Contractor's overall level of performance cannot be determined. Planned sampling provides a systematic way of looking at service output and forming conclusions about the Contractor's level of performance.

Planned sampling is generally used in one of two ways. First, it can provide a one-time subjective evaluation of Contractor performance. Second, it can be used to detect a change in the Contractor's level of performance (i.e., trend analysis). This method requires that the sample selection criteria be well-documented and consistently applied from period to period and that there are no other intervening factors.

One-Time Subjective Evaluation. The magnitude of the nonconformity rate may be sufficient to initiate action in addition to taking any normal deductions. If the nonconformity rate is large and there is no apparent cause, the QAE should investigate the problem.

Trend Analysis. Nonconformity rates are monitored for a period of time to detect changes in Contractor performance level. Evaluation of Contractor performance using nonconformity rate based on planned sampling is not as exact as evaluation using nonconformity rates based on random sampling. It is more a subjective assessment.

AQLs are used as benchmarks for planned sampling results. That is, when the number of nonconformity exceeds some specified number (e.g., three) overall performance is considered unsatisfactory.

To ensure valid results, use planned sampling evaluation sheets and follow a detailed inspection schedule.

Schedules must be developed monthly to coincide with the Contractor's monthly schedule of work, and updated each Friday after receiving the Contractor's more definitive weekly schedule. Next, the criteria to be used for sample selection should be documented. The rationale for criteria and sample size requirements should be stated.

The sample size requirements are based on Contractor performance. A larger sample size should be used for poor performance and a smaller sample size for good performance. Any impact on services that receive little or no evaluation must also be documented.

Observed nonconformity for services monitored by planned sampling will be totaled at the end of each month. For each service, the total number of nonconformity will be compared to the reject level. When the observed total number of nonconformity is equal to or less than the reject level, the Contractor's overall performance for the given service evaluated is satisfactory. When the observed total number of nonconformity is greater than the reject level, the Contractor's overall performance is judged to be unsatisfactory. Further inspection or corrective action is indicated.

One Hundred Percent Inspection. One hundred percent inspection is that evaluation method that requires total, or one hundred percent, inspection of a contract requirement. One hundred percent inspection should be used for standards that are: especially critical; where there is some reason for suspecting that a standard is not being met and, therefore, should be more closely monitored; or for monitoring scheduled contract requirements, such as, one-time deliverables, scheduled submissions, and infrequently identified requirements. Evaluation schedules for one hundred percent inspections will be prepared each month.

Observed nonconformity for a service monitored will be totaled at the end of each month. Total number of nonconformity (i.e., inspections failed) are divided by the population (lot size). This gives the actual nonconformity rate. The nonconformity rate computed will reflect the Contractor's actual level of performance since all work was evaluated. The computed nonconformity rate is compared to the AQL. If the nonconformity rate is greater than the AQL the Contractor's overall level of performance for that contract requirement is classified as unsatisfactory. If the observed nonconformity rate is equal to or less than the AQL, the Contractor's overall level of performance for the item evaluated is classified as satisfactory.

Customer Complaints. Even the best QASP will not allow the QAE to check all aspects of the Contractor's performance. Formal customer complaints are a means of documenting certain kinds of service problems. Where there is a case of poor performance or nonperformance, the QAE is notified. Upon receipt of a complaint, the QAE will investigate the report and, if it is found to be valid, documents it, and determines whether the Contractor is at fault. If the complaint is valid and caused by poor performance or non-performance by the Contractor, the Contractor must be notified to take appropriate corrective action. The numbers of complaints and resulting inspections are dependent upon customer awareness and response. Since QA inspections based on validated customer complaints cannot be scheduled prior to work accomplishment; this method is normally supplemented with other surveillance techniques.

In-depth instructions should be given and explained to every segment of the organization that receives Contractor's services. The instructions should include a detailed description of the Contractor's established performance level for the services, an outline of the customer complaint program, the format and the content of a formal customer complaint, and the action that can be expected for the Government.

The Contractor is required by the solicitation to develop an active customer complaint program as part of the OC Plan. The Government OA evaluation of quality control will regularly audit this program to assure satisfactory maintenance and operation.

Validated complaints are summarized each month by service. These nonconformity are then reviewed. When the PRS in the contract indicates that the method of evaluation is customer complaint, these validated complaints will be counted against the Contractor. When the PRS indicates another method, the results must never be combined with those evaluation results. If combined with random sampled data, statistical assumptions would be invalid. If combined with planned sample data, trend analysis would not be possible. If the assumption is made that customer awareness is similar from month-to-month, trend analysis can be used to test for variation in the number of complaints received each month.

Unscheduled Inspection. Unscheduled inspection is the impromptu evaluation of contract requirements. It enables the Government to monitor and document those Contractor requirements where increased levels of surveillance are thought to be necessary. Unscheduled inspection is also used to monitor those contract requirements with no specified surveillance method. There is no special procedure to be followed to initiate unscheduled inspections.

Whenever an unscheduled inspection is conducted, regardless of the outcome, an Unscheduled Inspection Report is filled out. This form is the official documentation of that evaluation.

Unscheduled inspection results are summarized each month by service. These data are reviewed but never combined with other evaluation results. If combined with random sampled data, statistical assumptions would be invalid. If combined with planned sample or customer complaint data, trend analysis would not be possible.

Unscheduled inspection data are used to support results achieved through other evaluation methods and to provide the data required for the day-to-day administration of the contract.

Regardless of the surveillance method, the COR shall always contact the contractor's task manager or on-site representative when a defect is identified and inform the manager of the specifics of the problem. The COR, with assistance from the COD KS, shall be responsible for monitoring the contractor’s performance in meeting a specific performance standard/AQL.

7. RATINGS.

Metrics and methods are designed to determine if performance exceeds, meets, or does not meet a given standard and acceptable quality level. A rating scale shall be used to determine a positive, neutral, or negative outcome. The following ratings shall be used:

Example 1:

EXCEPTIONAL:

Performance significantly exceeds contract requirements to the Government’s benefit.

SATISFACTORY: Performance meets contractual requirements.

UNSATISFACTORY: Performance does not meet contractual requirements.

8. DOCUMENTING PERFORMANCE.

a. ACCEPTABLE PERFORMANCE.

The Government shall document positive performance. A report template is attached. Any report may become a part of the supporting documentation for fixed fee payments, award fee payments, or other actions.

b. UNACCEPTABLE PERFORMANCE.

When unacceptable performance occurs, the COR shall inform the contractor. This will normally be in writing unless circumstances necessitate verbal communication. In any case the COR shall document the discussion and place it in the COR file.

When the COR determines formal written communication is required, the COR shall prepare a Contract Discrepancy Report (CDR), and present it to the contractor's task manager or on-site representative. A CDR template is attached to this QASP.

The contractor shall acknowledge receipt of the CDR in writing. The CDR will specify if the contractor is required to prepare a corrective action plan to document how the contractor shall correct the unacceptable performance and avoid a recurrence. The CDR will also state how long after receipt the contractor has to present this corrective action plan to the COR. The Government shall review the contractor's corrective action plan to determine acceptability.

Any CDRs may become a part of the supporting documentation for contract payment deductions, fixed fee deductions, award fee nonpayment, or other actions deemed necessary by the PCO.

10. FREQUENCY OF MEASUREMENT.

a. Frequency of Measurement.

For each task order, the COR shall take periodic measurements, as specified in the AQL column of the Performance Standards Summary Matrix, and shall analyze whether the negotiated frequency of measurement is appropriate for the work being performed.