Quality Assurance Surveillance Plan - TQMS - 08 May 25.pdf

Quality Assurance Surveillance Plan (QASP) for

ECM software and support 31 CES TQMS project

08 May 2025

THIS PAGE IS NOT USED.

Table of Contents

1.0 Purpose

2.0 Authority

3.0 Roles and Responsibilities

4.0 Performance Requirements and Method of Surveillance

4.1 Contract Surveillance

4.2 Surveillance Matrix

5.0 Performance Rating Definitions

6.0 Performance Reporting

6.1 Corrective Action Report (CAR)

6.2 Customer Complaint Record

6.3 Performance Assessment Report (PAR)

7.0 Surveillance Matrix for ECM software provider

1.0 Purpose

a. This Quality Assurance Surveillance Plan is a government-developed document used to determine if the contractor’s performance meets the performance standards contained in the contract. The QASP establishes procedures on how this assessment/inspection process will be conducted. It provides the detailed process for a continuous oversight process:

What will be monitored How monitoring will take place Who will conduct the monitoring How monitoring efforts and results will be documented

b. The contractor is responsible for implementing and delivering performance that meets contract standards using its Quality Control Plan. The QASP provides the structure for the government’s surveillance of the contractor’s performance to assure that it meets contract standards. It is the government’s responsibility to be objective, fair, and consistent in evaluating contractor performance.

c. The QASP is not part of the contract nor is it intended to duplicate the contractor’s quality control plan. This QASP is a living document. Flexibility in the QASP is required to allow for an increase or decrease in the level of surveillance necessary based on contractor performance.

d. The government may provide a copy of the QASP to the contractor to facilitate open communication. In addition, the QASP should recognize that unforeseen or uncontrollable circumstances might occur that are outside the control of the contractor.

e. Bottom line--the QASP should ensure early identification and resolution of performance issues to minimize impact on mission performance.

2.0 Authority

Authority for issuance of this QASP is provided under Part 46 of the Federal Acquisition Regulation, Inspection of Services clauses, which provide for inspection, acceptance and documentation of the service called for in the contract or order. This acceptance is to be executed by the contracting officer or a duly authorized representative.

3.0 Roles and Responsibilities

The following personnel shall oversee and coordinate surveillance activities.

a. Program/Project Manager (PM) and IT Specialist. The PM provides primary program oversight, nominates the COR, ensures the COR is trained before performing any COR duties and supports the COR’s performance assessment activities. While the PM may serve as a direct conduit to provide Government guidance and feedback to the Contractor on technical matters, they are not empowered to make any contractual commitments or any contract changes on the government’s behalf.

Assigned PM: Roberto Capovilla Organization or Agency: 31CES/CEIAI Telephone: 00390434305710 Email: roberto.capovilla.it@us.af.mil

b. Contracting Officer (CO). The CO shall ensure performance of all necessary actions for effective contracting, ensure compliance with the contract terms, and shall safeguard the interests of the United States in the contractual relationship. The CO shall also ensure that the contractor receives impartial, fair, and equitable treatment under this contract. Determine the final assessment of the contractor’s performance.

Assigned CO:

Organization or Agency:

Telephone:

Email:

c. Contracting Officer’s Representative (COR). The COR is responsible for providing continuous technical oversight of the contractor’s performance. The COR uses the QASP to conduct the oversight/surveillance process. The COR shall keep a Quality Assurance file that accurately documents the contractor’s actual performance. The purpose is to ensure that the contractor meets the performance standards contained in the contract. The COR is responsible for reporting early identification of performance problems to the CO. The COR is required to provide an annual performance assessment to the CO which will be used in documenting past performance. The QASP is the primary tool for documenting contractor performance. The COR is not empowered to make any contractual commitments or to authorize any contractual change on the Government’s behalf.

Primary COR: Alessandro Stefani Organization or Agency: 31CES/CEOER Telephone: 00390434307741 Email: alessandro.stefani.it@us.af.mil

d. Program/Project IT Specialist (PS). This may include performance monitors, inspectors, technical experts, or others who provide information that helps the COR monitor contractor performance.

Program/Project IT Specialist: Federico Olimpi Organization or Agency: 31CES/CEIAI Telephone: 00390434305710 Email: federico.olimpi.it@us.af.mil

e. Other Key Government Personnel. This may include performance monitors, inspectors, technical experts, or others who provide information that helps the COR monitor contractor performance.

Contract Analyst

Assigned CA:

Organization or Agency:

Email:

Contractor Representatives

The following employees of the contractor serve as the contractor’s Program Manager and Task Manager for this contract. (Complete this section after the contract award)

Program Manager: to be named

Email:

Other Key Contract Personnel – to be named Title:

Telephone:

Email:

4.0 Performance Requirements and Method of Surveillance

The contractor shall meet the following performance requirements in delivering a tailored CMIS-based ECM solution. Detailed specifications for each requirement are further defined in the Performance Work Statement (PWS).

Analysis and Reporting: The contractor shall thoroughly analyze the organization's requirements and provide a detailed report outlining the analysis performed. This includes analyzing documents, workflows, and lifecycles in collaboration with the organization's IT staff and stakeholders. The contractor shall also deliver a detailed implementation plan for the tailored ECM solution.

Deployment and Configuration: The contractor shall deploy and configure the ECM solution and all related services on the organization's servers, working in conjunction with the organization's IT staff (who have sole authority over the Government IT infrastructure).

Security Implementation: The contractor shall implement DoD smart card authentication for the ECM infrastructure and its custom interfaces. Secure access to the application shall be enforced via protocols such as HTTPS and FTPS, with permissions managed through DoD-provided domain security groups. The contractor shall assist and train the organization's IT staff in defining and implementing all required protocols and measures to independently maintain the ECM software environment, including updates, backups, disaster recovery strategies, data restoration, and system redundancy.

Custom Development: The contractor shall develop custom APIs and dedicated web interfaces to effectively manage organizational data.

Training and Knowledge Transfer: The contractor shall conduct in-person training for two IT developers from the organization on modifying existing interfaces, developing new interfaces, and creating custom APIs. The contractor shall also provide end-user training on the implemented TQMS platform, focusing on the use of the functionalities developed.

Licensing: The contractor shall provide 50 annual licenses for the ECM software and all its functionalities.

Methods of surveillance:

Periodic Surveillance (Bi-Weekly Meetings): regular, scheduled meetings (in this case, every two weeks) between the contractor and government representatives (Project Lead, Specialist, and Contracting Officer Representative - COR) to:

o review the contractor's progress against the project schedule, milestones, and deliverables.

o identify and address any problems, risks, or challenges that the contractor is facing.

o assess the contractor's performance against agreed-upon metrics and standards.

o facilitate communication and collaboration between the contractor and the government.

o provide the contractor with guidance and direction from the government.

The contractor typically prepares a report or presentation summarizing their activities, progress, and any issues. The government representatives review the report and ask questions to clarify any points. The meeting is an opportunity for both sides to discuss any concerns and develop solutions.

Action items are assigned and tracked to ensure that issues are resolved. All minutes of the bi-weekly meetings, prepared by the contractor, must be discussed and approved by Government Project Lead, Project Specialist, COR and the contractor.

100% Review by Government Project Lead, Specialist, and COR: a comprehensive review of all deliverables, documentation, or work products submitted by the contractor. The review is conducted by the Government Project Lead, a subject matter Specialist, and the Contracting Officer Representative (COR) to:

o ensure that the contractor's work meets the required quality standards and specifications.

o verify that the contractor is complying with all applicable laws, regulations, and contract terms.

o ensure that the contractor's work is technically accurate.

o identify and mitigate any potential risks associated with the contractor's work.

The contractor submits the deliverables to the government. Each reviewer (Project Lead, Specialist, and COR) independently examines the deliverables against the contract requirements, technical specifications, and relevant standards. Reviewers provide feedback, comments, and suggestions for improvement. The contractor addresses the feedback and resubmits the deliverables, if necessary. This process continues until the government is satisfied that the deliverables meet all requirements.

Performance Requirements Checks: systematic assessment of the contractor's performance against specific performance requirements outlined in the contract.

These requirements can be quantitative (e.g., response time, error rate) or qualitative (e.g., customer satisfaction, quality of service) to:

o provide an objective measure of the contractor's performance.

o identify areas where the contractor needs to improve their performance.

o ensure that the contractor is meeting the performance requirements specified in the contract.

o determine whether the contractor is subject to penalties based on their performance.

The government establishes clear and measurable performance requirements in the contract. The government collects data on the contractor's performance using various methods, such as data analysis, audits, surveys and observations. The government provides feedback to the contractor on their performance and identifies areas for improvement.

Validated Customer Complaints: formal process for collecting, investigating, and resolving customer complaints related to the contractor's performance. Only "validated" complaints are considered, meaning those that have been investigated and confirmed to be legitimate to:

o identify areas where the contractor is failing to meet customer expectations.

o improve customer satisfaction by addressing and resolving complaints.

o hold the contractor accountable for their performance and ensure that they are providing satisfactory service.

o provide the contractor with valuable feedback on their performance from the customer's perspective.

4.1 Contract Surveillance

The goal of the QASP is to ensure that contractor performance is effectively monitored and documented. The COR’s contribution is their professional, non-adversarial relationships with the CO, PM, and the contractor, which enables positive, open, and timely communications. The foundation of this relationship is built upon objective, fair, and consistent COR evaluations of contractor performance against contract requirements. The COR uses the methods contained in this QASP to ensure the contractor is following contract requirements. The COR function is responsible for a wide range of surveillance requirements that effectively measure and evaluate the contractor’s performance. Additionally, this QASP is based on the premise that the contractor, not the government, is responsible for management and QC/QA actions to successfully meet the terms of the contract.

4.2 Surveillance Matrix

a. The Surveillance Matrix (Appendix 1 & 2) is the list of performance objectives and standards that must be performed by the contractor. This matrix details the method of surveillance the COR will use to validate and inspect these performance elements.

Inspection of each element will be documented in the COR file.

b. Performance objectives define the desired outcomes. Performance Standards define the level of service required under the contract to successfully meet the performance objective. The inspection methodology defines how, when, and what will be assessed in measuring performance. The Government performs surveillance, using this QASP, to determine the quality of the contractor’s performance as it relates to the performance element standards. The Service Summary (SS) should be used to form the foundation of the COR’s inspection checklist.

5.0 Performance Rating Definitions

In evaluating the quality of contractor’s performance, the following performance ratings may be used.

Performance

Rating Criteria

Excellent / Outstanding

Performance meets contractual requirements and exceeds many to the government’s benefits. The contractual performance of the element or sub-element being assessed was accomplished with no problems and contractor actions were highly effective.

Very Good Performance meets contractual requirements and exceeds some to the government benefits. The contractual performance was accomplished with few minor problems for which corrective actions taken by the contractor were effective.

Good Performance meets contractual requirements. The contractual performance contains some minor problems for which corrective actions taken by the contractor were satisfactory.

Marginal Performance does not meet some contractual requirements. The contractual performance reflects a serious problem for which the contractor has not yet identified corrective actions. The contractor’s proposed actions appear marginally effective or were not fully implemented.

Unsatisfactory Performance does not meet most contractual requirements and recovery is not likely in a timely manner. The contractual performance contains serious problem(s) for which the contractor’s corrective actions appear or were ineffective.

6.0 Performance Reporting

The below documents are located on the following pages.

6.1 CORRECTIVE ACTION REPORT (CAR)

Adobe Acrobat Document

6.2 CUSTOMER COMPLAINT RECORD (CCR)

6.3 PERFORMANCE ASSESSMENT REPORT (PAR)

6.1 Corrective Action Report (CAR)

Describes how discrepancies are reported and resolved.

CORRECTIVE ACTION REPORT (CAR)

(If more space is needed, use reverse and identify by number)

1. CONTRACTOR 2. CONTRACT NUMBER 3. TYPE OF SERVICES

4. FUNCTIONAL AREA 5. SUSPENSE DATE 6. CONTROL NUMBER

7. DEFICIENCY MAJOR MINOR

FINDING:

FINDING IMPACT:

Please respond with a written corrective action plan that details the corrective action of the cited deficiency, the cause of the deficiency, and actions taken to prevent recurrence by Suspense Date in Block 5. If date was not entered in Block 5, the contractor is not required to provide a response.

8. QUALITY ASSURANCE PERSONNEL (COR)

TYPED NAME AND GRADE

SIGNATURE AND DATE

9. ISSUING AUTHORITY

TYPED NAME AND GRADE

SIGNATURE AND DATE

10. COR RESPONSE TO CONTRACTOR CORRECTIVE ACTION AND ACTION TAKEN TO PREVENT RECURRENCE

11. COR DETERMINATION

ACCEPTED REJECTED

12. CLOSE DATE

6.2 Customer Complaint Record

CUSTOMER COMPLAINT RECORD

DATE/TIME OF COMPLAINT

SOURCE OF COMPLAINT

ORGANIZATION BUILDING NUMBER INDIVIDUAL PHONE NUMBER

NATURE OF COMPLAINT

CONTRACT REFERENCE

VALIDATION

DATE/TIME CONTRACTOR INFORMED OF COMPLAINT

ACTION TAKEN BY CONTRACTOR

RECEIVED/VALIDATED BY

6.3 Performance Assessment Report (PAR)

PERFORMANCE ASSESSMENT REPORT (PAR)

(If more space is needed, use reverse and identify by number)

1. CONTRACT/TASK ORDER NUMBER 2. CONTRACTOR 3. TYPE OF SERVICES

4. QUALITY ASSURANCE PERSONNEL (COR) SIGNATURE AND DATE

5. COR PHONE

6. SUSPENSE DATE

I. PERFORMANCE

7. DEFICIENCY (CHECK ALL BOXES THAT APPLY)

NEW

REPEAT

NO DEFICIENCY NOTED

8. SERVICES SUMMARY or PWS PARAGRAPH ITEM REVIEWED

9. BRIEF DESCRIPTION OF DEFICIENCY (IF DEFICIENCY BOX WAS CHECKED) 10. DETAILED PERFORMANCE ASSESSMENT

II. CONTRACTOR VALIDATION

11. CONTRACTOR REPRESENTATIVE CONCUR NON-CONCUR 12. CORRECTIVE ACTION ESTIMATED COMPLETION DATE

13. CONTRACTOR REPRESENTATIVE CORRECTIVE ACTION AND PREVENTION OF RECURRENCE OR REASON FOR NON-CONCURRENCE OF COR

CITED DEFICIENCY

III. ACTION CORRECTED

14. CONCUR NON-CONCUR COR SIGNATURE AND DATE

15. COR REMARKS (REQUIRED)

6. CONTRACTOR REPRESENTATIVE REMARKS

7.0 Surveillance Matrix for ECM software provider

Description of Service PWS Reference

Performance Threshold

Methods of Surveillance

Analysis of the Organization requirements.

5.1 Good Periodic Surveillance (bi-weekly meetings)

and 100% review by Government Project Lead, COR and contractor.

Assist the organization's IT staff and stakeholders in analyzing documents, workflows, and lifecycles, and translating them into a tailored CMIS-based ECM solution.

5.2 Good Periodic Surveillance (bi-weekly meetings)

and 100% review by Government Project Lead, Specialist and COR.

Provide, deploy and configure the ECM software solution and all related/required services with the support of the organization’s IT staff.

5.3 Good Periodic Surveillance (bi-weekly

meetings), 100% review by Government Project Lead, Specialist and COR, Random Inspections, Performance Requirements checks and Validated Customer Complaints.

Assist and train the organization IT staff in the definition and implementation of all the required protocols and measures to independently maintain the just created ECM environment.

5.4 Good Periodic Surveillance (bi-weekly

meetings), 100% review by Government Project Lead, Specialist and COR, Random Inspections and Performance Requirements checks.

Develop custom APIs and dedicated web interfaces to effectively manage organizational data.

5.5 Good Periodic Surveillance (bi-weekly

meetings), 100% review by Government Project Lead, Specialist and COR, Random Inspections, Performance Requirements checks and Validated Customer Complaints.

Conduct in-person training for two IT developers from the organization on the modification of existing interfaces, the development of new interfaces, and the creation of custom APIs.

5.6 Good Periodic Surveillance (bi-weekly

meetings), 100% review by Government Project Lead, Specialist and COR, Performance Requirements checks and Validated Customer Complaints.

Provide 50 annual licenses. 5.7 Good 100% review by Government Project Lead, Specialist and COR.

Provide end-users’ training on the implemented TQMS platform (on-site).

5.8 Good Periodic Surveillance (bi-weekly

meetings), 100% review by Government Project Lead, Specialist and COR, Performance Requirements checks and Validated Customer Complaints.

Appendix 1.0 Version Change History

Section / Page No. Add / Change /

Delete Change No. / Date Explanation

THIS PAGE IS NOT USED.