Attch_0016_ACWS_Quality_Assurance_Surveillance_Plan_2016_05_31.pdf

Quality Assurance Surveillance Plan

Version 1.02 31 May 2016

Distribution A: Approved for public release; distribution unlimited

ARMY CONTRACT

WRITING SYSTEM

(ACWS)

Quality Assurance Surveillance Plan Version 1.02 31 May 2016

Prepared By:

Product Manager Army Contract Writing System (ACWS)

Quality Assurance Surveillance Plan i

This page intentionally left blank

Quality Assurance Surveillance Plan ii

APPROVAL PAGE

SUBMITTED BY:

LTC Robert J. Wolfe Product Manager Army Contract Writing System (ACWS)

Date

Quality Assurance Surveillance Plan iii

Revision History

Version Date Summary of Changes

Version 1.0 17 December 2015 Final RFP Version

Version 1.01 6 May 2016

Revised Final RFP version based on vendor comments;

Date and version number updated on title page and footers Sections renumbered after Section 5.2.Updated period of performance in Section 1. New Section 5.8 includes Table 2: Surveillance Matrix: Performance Outcomes and Required Services, and Table 3:

Surveillance Matrix: Processes / Products.

Version 1.02 31 May 2017

Revised Table 2: Surveillance Matrix: Performance Outcomes and Required Services – Corrected the content to align with Attachment 0001 SOO, removed references to EVM.

Quality Assurance Surveillance Plan iv

Table of Contents

1.0 Purpose

2.0 Authority

3.0 Scope

3.1 Contract Scope

3.2 Contract Quality Assurance

3.3 Government Quality Assurance Surveillance

4.0 Roles and Responsibilities

4.1 Product Manager (PdM) Army Contract Writing System

4.2 Procuring Contracting Officer

4.3 Primary Contracting Officer Representative

4.4 Contracting Officer Technical Representative

4.5 PdM ACWS Budget Analyst

5.0 Methods of Quality Assurance

5.1 Surveillance Folder

5.2 Evaluation Areas of Contractor Performance Assessment Report

5.3 Contract Manpower Reporting Application

5.4 Performance Standards

5.5 Performance Rating Definitions

5.6 Timely Feedback to Contractor

5.7 Documents / Reporting

5.8 Government Surveillance Matrices

5.9 Remedies for Non-Conformance and Monitoring Forms

5.10 Contract Performance Assessment Reporting System (CPARS)

5.11 Disputes

Appendix A: Performance Assessment Report

Appendix B: Customer Complaint Form

Appendix C: Corrective Action Report

Quality Assurance Surveillance Plan v

List of Tables

Table 1: Evaluation Ratings Definitions

Table 2: Surveillance Matrix: Performance Outcomes and Required Services

Table 3: Surveillance Matrix: Processes / Products

1.0 Purpose

This Quality Assurance Surveillance Plan (QASP) is a Government developed and applied document that describes the systematic quality assurance methods and guidelines used by the Procuring Contracting Officer (PCO) and the Contracting Officer Representative (COR) / Contracting Office Technical Representatives (COTRs) to measure and monitor Contractor performance.

Specifically, this QASP measures the Contractor efforts toward meeting the requirements of the Statement of Objectives (SOO) and the contractual requirements for the Army Contract Writing System (ACWS).

This QASP is designed to define roles and responsibilities, identify the performance objectives, define the methodologies used to monitor and evaluate the Contractor’s performance, describe quality assurance reporting, and describe the analysis of quality assurance monitoring results. The QASP describes the procedures to ensure that the Contractor provides quality products in response to requirements and performs in accordance with the basic SOO and performance standards.

The associated performance-based Contract has a hybrid of Cost Plus Fixed Fee (CPFF), Firm Fixed Price (FFP), and Cost Reimbursable (CR) Contract Line Item Numbers (CLINs). The Period of Performance: an initial twelve (12) month Base Ordering Period, one (1) forty-eight (48) month Option Ordering Period, and one (1) sixty (60) month Option Ordering Period.

2.0 Authority

This surveillance plan is developed in accordance with Federal Acquisition Regulation (FAR) Part 46 Quality Assurance and the relevant Clauses for Inspection of Services and/or Inspection of Supplies to establish the procedures for monitoring, reporting, and evaluating the performance of the Contractor. The Government acceptance of Supplies or Services is to be executed by the PCO or designated COR. The frequency and method of the government’s inspection, review responsibilities, and reporting requirements follow.

3.0 Scope

3.1 Contract Scope

The Statement of Objectives (SOO) for ACWS establishes the requirements for a System Integrator to provide a single enterprise-wide contract writing and management system that will replace the Standard Procurement System (SPS), the Procurement

Automated Document Data System (PADDS), and the Virtual Contracting Enterprise (VCE) suite of tools in order to meet the Army's current and future critical functional requirements.

3.2 Contract Quality Assurance

Per the Defense Federal Acquisition Regulation (DFAR) 246.4 Government Contract Quality Assurance and the contract PWS paragraph 6.1 Program Management, the Government will monitor and record the Contractor performance as stated in this QASP.

PWS paragraph 6.1 Program Management, requires that:

a. The Contractor shall develop a Quality Assurance Program Plan (QAPP) [Contract Data Requirements List (CDRLs) A007].

As corrective actions are identified, the PCO may notify the Contractor of required modifications to the plan during the period of performance.

3.3 Government Quality Assurance Surveillance

The Government will monitor Contractor performance under this contract in accordance with this QASP and the associated reports as Attachments A, B, and C.

Requirements for performance include CDRLs, work products, meetings, monthly Project Management Reviews (PMRs), and Technical Reviews to provide progress of the contracted effort. The Government COR and/or COTRs will monitor performance and review performance reports furnished by the Contractor to determine how the Contractor is performing against communicated and documented performance objectives. The Government will make decisions based on performance measurement metric data and will notify the Contractor of those decisions. The Contractor will be responsible for making required changes in processes and practices to ensure performance is managed effectively.

4.0 Roles and Responsibilities

The following Government personnel positions will oversee and implementation surveillance activities:

4.1 Product Manager (PdM) Army Contract Writing System

The ACWS Product Manager (PdM) provides primary program oversight. While the PdM may serve as a direct conduit to provide Government guidance and feedback to the Contractor on project matters, the PdM is not empowered to make any contractual commitments or any contract changes on behalf of the Government. The PdM will recommend a COR and COTRs to the PCO, ensure the COR / COTRs are trained before performing any delegated duties, and will support COR/ COTR performance assessment activities.

4.2 Procuring Contracting Officer

The Army Contracting Command-Rock Island will assign a PCO, duly appointed with the authority to enter into, administer, or terminate contract actions such as basic awards or modifications, and make related determinations and findings on behalf of the Government. See FAR Part 1.602 for authority and responsibilities.

The PCO will ensure performance of all necessary actions for effective contracting and ensure compliance with the contract terms and conditions. The PCO serves as the official Government agent with authority to modify and interpret the contract with the Contractor. The PCO is also the only Government official who can authorize contract actions that may result from review of Contractor performance. The PCO will also ensure that the Contractor receives impartial, fair, and equitable treatment.

The PCO relies on the COR and other functional COTRs to provide information and recommendations regarding Contractor performance. While the PCO may delegate many of the day-to-day contractual administration duties to the COR, certain contractual actions are retained as the sole purview of the PCO, including: negotiation and issuance of contract modifications, issuance of Contract Deficiency Reports, issuance of “show cause” letters, contract termination, contract closeout, and Contractor claims and disputes resolution. The PCO will designate in writing a COR. The PCO will ensure the COR / COTRs are trained before performing any duties and supports COR performance assessment activities.

The PCO is ultimately responsible for the final determination of adequacy of Contractor performance.

4.3 Primary Contracting Officer Representative

The primary COR will be an individual designated in writing by the PCO to act as the authorized representative to assist in the administration of the contract. The COR is responsible for providing continuous technical oversight of Contractor performance.

The COR is not empowered to make any contractual commitments or to authorize any contractual changes on behalf of the Government. For any changes that may affect contract price, terms, or conditions, the COR will alert the PCO for potential resolution or action. The source of authority for the COR is the PCO delegation memorandum.

See FAR Part 1.604 and DFAR 201.602-2 for definition of responsibilities.

4.4 Contracting Officer Technical Representative

The COTR will be an individual designated in writing by the PCO who is authorized to assist the PCO and COR in providing continuous monitoring and review Contractor technical compliance with the contract requirements. The COTR is not empowered to make any contractual commitments. For any changes that may affect contract price, terms, or conditions, the COTR will refer to the COR for coordination with the PCO for action.

See FAR Part 2.101 for Definition that a COR means an individual, including a contracting COTR, designated and authorized in writing by the PCO to perform specific technical or administrative functions.

4.5 PdM ACWS Budget Analyst

The PdM ACWS Budget Analyst will be an individual designated by PdM ACWS to act as the authorized representative to assist in the administration of the financial management of the contract.

5.0 Methods of Quality Assurance

5.1 Surveillance Folder

A surveillance folder will be used by the primary COR assigned to perform surveillance of Contractor performance for Contractor assigned contract requirements. The folder has the following divisions:

a. COR appointment letter, PCO letter of designation, and Training Record

b. PWS and other pertinent parts of the contract

c. Surveillance Plan

d. Surveillance Reports

e. Correspondence: Customer and user comments

f. Data and technical reports.

5.2 Evaluation Areas of Contractor Performance Assessment Report

The COR will perform an assessment quarterly and will file an annual assessment of the following evaluation factors:

a. Quality of Product or Service (Technical Performance) – Assess the Contractor’s effort to transform operational needs and requirements into an integrated solution. Areas of focus may include the planning and management of program tasks, the quality of support provided throughout all phases of contract execution, and the management of a totally integrated effort of all program management concerns to meet cost, performance, and schedule objectives.

Following the metrics in tables 1 and 2 below, the Contractor will be evaluated monthly as to the quality of the CDRLs and work products. Results will be documented in the COR file. Included in the technical performance is the Contractor’s contribution in meetings and reviews; the quality of the Contractor's technical reports; the Contractor's productivity level, and the overall quality of the technical support provided.

b. Schedule and Timeliness of Deliveries – Assess the Contractor’s adherence to the required delivery schedule by assessing the Contractor’s efforts during the assessment period that contribute to or effect the schedule variance. This will take into consideration the significance of scheduled events, and will also assess the effectiveness of corrective actions taken by the Contractor. The Contractor should be responsive to Government taskings and submit their CDRLs, work products and reports as required by the contract. The Contractor shall conduct scheduled reviews on time and shall submit material for the review per the agreed upon process and associated timelines as required by the contract.

c. Cost / Resource Control – Assess the Contractor’s effectiveness in forecasting, managing, and controlling cost. Assess the causes and Contractor-proposed solutions for the existence of cost overruns. Address underruns in the same manner where performance is adversely impacted. Assess the extent to which the Contractor demonstrates cost responsibility through the efficient use of resources. The Contractor shall use the funding authorized to provide support throughout the period of performance. The Contractor should not exceed the material, travel, or other cost-reimbursable CLINs without prior PCO approval.

The Contractor will be evaluated on the successful control of resources dedicated to PWS tasks. Monthly, the COR will compare the Contractor burn rate of labor and funds usage with those allocated for the effort on the CLIN. Any discrepancies, and when funds reach 25% remaining, will be brought to the attention of the PCO.

d. Business Relations or Management – Assess the turnaround, completeness, and quality of problem identification, corrective action plans, the cooperativeness of the Contractor, and customer satisfaction. Assess the Contractor success with management of subcontracts. Assess Contractor ability to identify and apply resources required to meet schedule requirements, assign responsibility for tasks/actions required by the PWS, and communicate appropriate information to affected program elements in a timely manner. Assess Contractor risk mitigation plans. Identify any other management areas that are applicable to the PWS.

Contractor management should put adequate systems in place to most efficiently perform the contract and protect Government interests. For example, the Contractor should have approved Government property management, quality control, subcontracting, and accounting systems in place.

e. Small Business Participation – Assess the Contractor-submitted Electronic Subcontracting Report System (www.eSRS.gov) Report. Assess compliance with all terms and conditions in the contract/order relating to Small Business participation (including FAR 52.219-8, Utilization of Small Businesses and FAR 52.219-9, Small Business Subcontracting Plan (when required). Assess any small business participation goals which are stated separately in the contract/order. Assess achievement on each individual goal stated within the contract/order or subcontracting plan including good faith effort if the goal was not achieved.

f. Other Areas – Assess additional evaluation areas within the terms and conditions of the contract or unique to the PWS including Paragraph 4. Special Requirements.

5.3 Contract Manpower Reporting Application

The COR is responsible for validating Contractor input into the Contract Manpower Reporting Application (CMRA) website at https://cmra.army.mil. The COR is to obtain a user name and password, and validate that the Contractor has completely filled in required Contractor information at the CMRA web site by October 31 of the first year of the contract and annually by October 31 thereafter for the duration of the contract, or sooner if the contract is closed out. The COR will perform this validation during the month of November each year. The COR will report the CMRA results in the November Monthly COR Report (due in December of each year) to the PCO each year.

5.4 Performance Standards

Performance standards define the desired service levels. The Government performs surveillance to determine if the Contractor exceeds, meets, or does not meet these standards.

The Performance Requirements Matrix below includes the performance standards. The Government will use these standards to determine Contractor performance and will compare Contractor performance to the Acceptable Quality Level (AQL).

5.5 Performance Rating Definitions

The Government ratings and narrative must reflect FAR Part 42.1503, Table 42-1 “Evaluation Ratings Definitions” as depicted in Table 1 below:

Table 1: Evaluation Ratings Definitions

Rating Definition Note

Exceptional – Performance meets contractual requirements and exceeds many to the Government’s benefit. The contractual performance of the element or sub-element being evaluated was accomplished with few minor problems for which corrective actions taken by the Contractor were highly effective.

To justify an Exceptional rating, identify multiple significant events and state how they were of benefit to the Government. A singular benefit, however, could be of such magnitude that it alone constitutes an Exceptional rating. Also, there should not have been any significant weaknesses identified.

Very Good – Performance meets contractual requirements and exceeds some to the Government’s benefit. The contractual performance of the element or sub-element being evaluated was accomplished with some minor problems for which corrective actions taken by the Contractor were effective.

To justify a Very Good rating, identify a significant event and state how it was a benefit to the Government. There should not be any significant weaknesses identified.

Rating Definition Note

Satisfactory – Performance meets contractual requirements. The contractual performance of the element or sub-element contains some minor problems for which corrective actions taken by the Contractor appear or were satisfactory.

To justify a Satisfactory rating, there should have been only minor problems, or major problems the Contractor recovered from without impact to the contract/order.

There should not have been significant weaknesses identified. A fundamental principle of assigning ratings is that Contractors will not be evaluated with a rating lower than Satisfactory solely for not performing beyond the requirements of the contract/order. Marginal – Performance does not meet some contractual requirements. The contractual performance of the element or sub-element being evaluated reflects a serious problem for which the Contractor has not yet identified corrective actions.

The Contractor proposed actions appear only marginally effective or were not fully implemented.

To justify Marginal performance, identify a significant event in each category that the Contractor had trouble overcoming and state how it impacted the Government. A Marginal rating should be supported by referencing the management tool that notified the Contractor of the contractual deficiency (e.g., management, quality, safety, or environmental deficiency report or letter). Unsatisfactory – Performance does not meet most contractual requirements and recovery is not likely in a timely manner. The contractual performance of the element or sub-element contains a serious problem(s) for which Contractor corrective actions appear or were ineffective.

To justify an Unsatisfactory rating, identify multiple significant events in each category that the Contractor had trouble overcoming and state how it impacted the Government. A singular problem, however, could be of such serious magnitude that it alone constitutes an unsatisfactory rating. An Unsatisfactory rating should be supported by referencing the management tools used to notify the Contractor of the contractual deficiencies (e.g., management, quality, safety, or environmental deficiency reports, or letters).

5.6 Timely Feedback to Contractor

Timely feedback to the Contractor on unacceptable performance will be provided by the COR and documented in writing. Prompt feedback is essential so that the Contractor can develop and implement a corrective action plan. The Contractor's corrective action must be reported to the PCO for tracking purposes.

5.7 Documents / Reporting

A copy of the Quarterly Contractor Performance Review shall be provided to the PCO.

The COR will report to the PCO any issues and concerns that cannot be resolved at the working level or which impact contractual requirements for resolution and contractual action as they occur.

5.8 Government Surveillance Matrices

Performance objectives are used to define the desired outcomes. Performance standards are used define the level of service required under the contract to successfully meet the performance objective. These objectives and standards are contained in the Surveillance Matrix in Table 2 along with the required services, acceptance quality, and monitoring methodology. These define how, when, and what will be assessed in measuring performance. The Government performs surveillance, using this QASP, to determine the quality of Contractor performance as it relates to the performance element standards. The Services Summary should be used to form the foundation of the COR inspection checklist.

The Surveillance Matrix: Performance Outcomes & Services (Table 2 below) lists the six

(6) performance objectives and standards that must be performed by the Contractor.

This matrix details the method of surveillance the COR will use to validate and inspect these performance elements. Inspection of each element will be documented in the COR file.

Surveillance Matrix: Processes / Products (Table 3 below) lists the four (4) process / product groups of CDRLs and standards that must be delivered by the Contractor. This matrix details the method of surveillance the COR will use to validate and inspect the CDRLs. Inspection of each element will be documented in the COR file.

Table 2: Surveillance Matrix: Performance Outcomes and Required Services

Desired Performance

Outcome Required Services

Performance Standard

Acceptance Quality Level (AQL)

Monitoring Method

Program Management (SOO 6.1)

1 Program Management activities will span all contract tasks. This includes the management and oversight of all activities performed by Contractor personnel, including subcontractors, to satisfy the requirements identified in the

PWS.

• Integrated Master Schedule (IMS) management

• Project Management Planning and Reporting

• Earned Value Management (EVM) Reporting (Waived)

• Personnel Management

• Communications

• Risk Management

• Quality Management

• Configuration Management

• Contract Funds Management

• Documentation Management

• Audit Readiness Support

• Periodic and Adhoc Meetings and Reporting

• Implementation of the schedule

• Milestones met on time

• Metric reports delivered on time in correct format/content

• Meetings and Reporting provides the information needed

• Meets schedule baseline

• Meets contract cost baseline

• Personnel required to perform tasks are qualified and present

COR and Government Team will review and assess:

• CDRLs

• IMS

• EVM Data

• Staffing Levels and

Quality of performance

• Outcomes performance of regularly scheduled and ad hoc meetings

Desired Performance

Outcome Required Services

Performance Standard

Acceptance Quality Level (AQL)

Monitoring Method

Systems Engineering Management and Planning (SOO 6.2)

2 System Engineering Management and Planning will span across all contract engineering tasks. This includes the technical management required to achieve a balanced set of activities, such as the analysis, design, development, test and support functions, to satisfy ACWS requirements throughout the program life cycle. Document, implement, and maintain a Systems Engineering Management Plan.

• Perform System Engineering Management / Integration

• Manage the development of the Enterprise Architecture viewpoints and artifacts

• Report System Performance and Analysis with metrics

• Conduct Hosting Analysis & monitor the performance

• Support the Operating Environment Infrastructure with configuration and deployment of the ACWS Solution

• Comply with Human Systems Integration

• Conduct Technical Reviews

• Enterprise Architecture compliance with requirements, SI vendor specifications, and engineering practices

• Installation compliance with requirements, vendor specifications, and engineering practices

• Compliance with ACWS Architecture standards (Published after Contract Award)

• Compliance with submitted DISA Service Request Form (SRF) for hosting

• Compliance with systems engineering proposed processes and methodologies

• COR and Government Team will review and assess:

• CDRLs

• System

Performance reports (with metrics)

• Design documents

• Users acceptance per release / implementation/ upgrade

Outcome Required Services

Performance Standard

Acceptance Quality Level (AQL)

Monitoring Method

Solution Design and Development (SOO 6.3)

3 Perform all analysis, design, development, test, and integration activities necessary for the successful delivery of the ACWS Software Solution Builds 1 – N.

• Perform Risk Reduction

• Apply proven standards

• Develop ACWS Solution documentation

• Provide RMF-based secure

Operational Environment

• Implement Tools & Procedures for data access

• Conduct reviews to validate

Requirements, Design, Implementation, and Test Products

• Lead Global Analysis

• Support System Characterization to include Technical Risk & Standards Assessment and Performance and Scalability Assessment

• Define Interfaces

• Develop Architecture(s)

• Perform Requirements analysis

• Plan ACWS Solution Builds

• System/Software Design activities

• Develop / integrate the Software

• Develop / configure Interfaces

• Integrate Software/System Solution

• Configure System parameters

• Perform developmental tests

• Deliver and install ACWS Solution

• Support Government Configuration

Audits

• Provide capabilities to access

ACWS data, Tools, Service & Maintenance

• Satisfy 100 % of

ACWS

requirements

• Task activities defined in PWS completed

• Satisfy 95% of ACWS requirements

COR and Government Team will review and assess:

• Design

• Integration

• Development

• Test

Outcome Required Services

Performance Standard

Acceptance Quality Level (AQL)

Monitoring Method

Test and Evaluation (SOO 6.4)

4 Support Solution Design and Development by serving as a feedback mechanism on iterative ACWS System Engineering and Planning, and on the progress of the design process and product compliance with contractual requirements.

• Conduct Requirements Testability Analysis

• Conduct Test and Evaluation

• Establish a Test Schedule

• Integrated Testing

• Develop Test Tools

• Consider Modeling as T&E Strategy

• Compliant with

ACWS TEMP

• Complete test within time planned on the

IMS

COR and Government Team will review and assess:

• CDRLs

• Test Plans

• Test Reports

• Test Tools

• Test Schedule

• Requirements

Testability Analysis

Training (SOO 6.5)

5 Develop, deliver, and manage a comprehensive training solution and approach that establishes and maintains user proficiency throughout the lifecycle of

ACWS.

• Develop and Implement Training Management Plan

• Provide Training Materials

• Deliver ACWS OOTB training and familiarization

• Perform user training

• Provide Training Augmentation

• Educate users for each ACWS builds

• Develop Management Training

• Make training available for all users at all levels

• Complete required Training for ACWS users

COR and Government Team will review and assess:

• CDRLs

• Training Materials

• Training

Management Plan

• Training Plan for field supervisor

• User Surveys

Outcome Required Services

Performance Standard

Acceptance Quality Level (AQL)

Monitoring Method

Solution Deployment (SOO 6.6)

6 Provide the ACWS user community access to the ACWS solution in accordance with assigned “ACWS User Roles”

Conduct Solution Deployment to include integration of hardware, software, training and organizational change management in a synchronized and efficient manner.

• Develop and implement a Deployment Plan

• Provide the ACWS user community access to the user sites

• Provide technical services/Support

• Perform Deployment/Fielding Tasks

• 100% of users targeted for the deployment have access to the ACWS solution

• 95% of users targeted for the deployment have access to the ACWS solution

COR and Government Team will review and assess:

• Site Visits

Organization Change Management (SOO 6.7)

7 Develop and deliver an introduction of the ACWS Program to the user community and gain solution acceptance by end-users, key-stakeholders, and interface partners.

• Develop and implement a Change Management Plan

• Leverage the existing ACWS Knowledge Management Portal

• Develop and deliver an Army-wide ACWS change management approach

• 100% of users targeted for the deployment have received introduction to the ACWS Program

• 95% of users targeted for the deployment have received introduction to the ACWS Program

COR and Government Team will review and assess:

• Site Visits

Operations and Support (SOO 6.8)

8 Provide operations and support.

Ensure the implementation and management of operations and support activities achieve the required sustainment performance thresholds (HLO-6), and (3) control/minimize costs through innovation.

• Develop and implement a Product Support Plan

• Train the Government Tier I and II helpdesk support

• Document, track, report and resolve ACWS solution incidents and problems

• Support post deployment activities

• 100% of Government Tier I and II personnel trained

• 100% of incidents and problems resolved

• 95% of Government Tier I and II personnel trained

• 95% of incidents and problems resolved

COR and Government Team will review and assess:

• Product Support

• Help Desk Support

• Trouble Ticket

Support

• Deployment

Support

Outcome Required Services

Performance Standard

Acceptance Quality Level (AQL)

Monitoring Method

Selected Special Requirements (SOO 6.3.3 and SOO 6.9)

9 Cybersecurity and Program Protection Support

Compliance with all Cybersecurity and Program Protection Support requirements

100 % compliance with Cybersecurity regulations and policies

This is a Go / No Go criteria.

COR and Government Team will review and assess:

• CDRLs

• Independent

Verification and Validation

10 Provide Licenses for Commercial Software with flexible and effective support over the lifecycle of the ACWS solution - Data Rights Granted

• Issue all software licenses in the name of the U.S Government

• Issue all Software documentation identical to that provided to a customer

• Convey Government’s rights to software

• Issue license terms applicable to any software changes or new builds

• Issue and maintain all necessary commercial, Open Source, and third party software agreements to include licenses, warranties, and/or commercial maintenance agreements

• Transfer as applicable all necessary commercial, Open Source, and Third (3rd) Party software agreements to include licenses, warranties, and/or commercial maintenance agreements

100 % of required licenses (proposed) are accounted for and conveyed to the Government

This is a Go / No Go criteria.

COR and Government Team will review and assess:

• Proposed software licenses of the ACWS Solution

Table 3: Surveillance Matrix: Processes / Products

Surveyed Processes /

Products Review Method Entry Criteria Surveillance Methods

1. Exhibit A – Administrative /

Management Reporting (SOO 6.1) CDRLs A001 thru A016

2. Exhibit B – Systems Engineering

Management Reporting (SOO 6.2 – 6.3) CDRLs B001 thru B026

3. Exhibit C – Test and Evaluation

(Soo 6.4) CDRLs C001 thru C006

4. Exhibit C – Operations and

Support (SOO 6.5) CDRLs D001 thru D005

100% inspection of initial submission of CDRLs (e.g.

Schedule, Plans, Reports) submitted per DD 1423 100% of major revisions Judgmental inspection of minor revisions to deliverables

Delivered on time in the format required per DD 1423

COR and Government Team will provide for each CDRL:

• Comment Matrix

• Documented conversations and/or correspondence regarding issues

COR and Government Team will review:

• Meeting Minutes

• Action Items

Corrective Action / Customer Feedback Reports and Records:

• ACWS Corrective Action Report

• ACWS Customer Complaint

Record

• ACWS Performance Assessment

Report

100% of submitted Reports and Records

Report(s)/Records(s) form submitted by COR or a Customer

COR and Government Team will Report / Record form with corrective actions; plans / tasks will be tracked until completely resolved.

The Government will use generally accepted surveillance methods. These methods include, but are not limited to:

a. 100% Inspection – The Government will evaluate 100% of designated tasks.

This surveillance method is appropriate for tasks where 100% inspection is required and is practical.

b. Partial Inspection – The Government will evaluate less than 100% of designated tasks and on a selective basis. This surveillance method is appropriate for tasks that occur infrequently and where 100% inspection is neither required nor practical.

c. Random Sampling – The Government will evaluate less than 100% of designated tasks on a random basis. This surveillance method is appropriate for tasks that occur infrequently and where 100% inspection is neither required nor practical.

d. Operational Monitoring – The Government will examine reports generated by automated tools and logs and websites.

e. Performance Review Meetings – The Contractor shall attend Contract Performance Reviews as required. These meetings will focus on overall contract performance. Specific dates, locations, agenda, and attendance requirements will be specified by the PCO at least 30 days prior to each meeting.

5.9 Remedies for Non-Conformance and Monitoring Forms

In accordance with FAR 46.407 Nonconforming supplies or services, the PCO should reject supplies or services not conforming in all respects to contract requirements. If inspections indicate an unacceptable product or performance, the COR will notify the Contractor POC of the deficiencies for correction. The PCO ordinarily must give the Contractor an opportunity to correct or replace nonconforming supplies or services when this can be accomplished within the required delivery schedule.

If deficiencies are not corrected within the required time frame, the COR should notify the PCO for action. If the Contractor disagrees with the noted discrepancy and an agreement cannot be reached, the COR will notify the CO for a final decision.

Three forms are provided as attachments to record non-conformance:

Attachment A – Performance Assessment Report (PAR). The PAR will be used to report all minor discrepancies and will be generated by the COR and sent to the Contractor for corrective action. Contractors will be given ten (10) days to correct PARs unless they have requested an extension. If three (3) minor discrepancies are found, combine them and process a Corrective Action Report (CAR).

Attachment B – Customer Complaint Form. The COR must furnish instructions for this form to the customer receiving the Contractor service including the contract requirements for the service or product received. The customer must retain supporting evidence of complaint or feedback. After the COR validates feedback or complaint demonstrating Contractor noncompliance, the COR processes a PAR or CAR to the PCO. All Customer Complaint forms will be validated by the COR prior to notifying the Contractor.

Attachment C – Corrective Action Report (CAR). The CAR will be used to report a situation when there are three (3) or more minor discrepancies or one (1) or more major discrepancies documented. The CAR will be generated by the COR and forwarded to the PCO for action. A copy will be provided to the Contractor representative by the PCO. Most CARs will be corrected within 30 calendar days. Depending on the complexity of the finding, the Contractor may be allowed up to six months (maximum) to correct a CAR. The COR will request that the Contractor provide the status of CARs on a monthly basis. The COR will keep an electronic log of all CARs and will keep a hard copy of CARs in the contract surveillance file.

5.10 Contract Performance Assessment Reporting System (CPARS)

CPARS (www.cpars.gov) will be prepared at least annually, documenting Contractor performance for this specific contract. See FAR Part 42.1502. More frequent updates may be made as required to document a significant improvement or degradation of the level of contract performance.

5.11 Disputes

Every attempt will be made to resolve all disputes arising under this contact/plan using the Alternate Dispute Resolution (ADR) as outlined in FAR 33.214. If resolution cannot be made under ADR, the PCO will render a final decision.

Attachments (Fillable Forms):

A. Performance Assessment Report

B. Customer Complaint Record

C. Corrective Action Report

Appendix A: Performance Assessment Report

Performance Assessment Report (PAR) Template, Feb 05 (SAF/AQCP)

PERFORMANCE ASSESSMENT REPORT (PAR)

(If more space is needed, use reverse and identify by number)

1. CONTRACT/TASK ORDER

NUMBER

2. CONTRACTOR 3. TYPE OF SERVICES

4. QUALITY ASSURANCE PERSONNEL (QAP) SIGNATURE AND DATE

5. QAP PHONE 6. SUSPENSE DATE

I. PERFORMANCE

7. DEFICIENCY (CHECK ALL BOXES THAT

APPLY)

NEW

REPEAT

NO DEFICIENCY NOTED

8. SERVICES SUMMARY or SOW PARAGRAPH ITEM

REVIEWED

9. BRIEF DESCRIPTION OF DEFICIENCY (IF DEFICIENCY

BOX WAS CHECKED)

10. DETAILED PERFORMANCE ASSESSMENT

II. CONTRACTOR VALIDATION

11. CONTRACTOR REPRESENTATIVE CONCUR

NON-CONCUR

12. CORRECTIVE ACTION ESTIMATED COMPLETION

DATE

13. CONTRACTOR REPRESENTATIVE CORRECTIVE ACTION AND PREVENTION OF RECURRENCE OR REASON FOR

NON-CONCURRENCE OF QAP CITED DEFICIENCY

III. ACTION CORRECTED

14. CONCUR NON-CONCUR QAP SIGNATURE AND DATE

15. QAP REMARKS (REQUIRED)

16. CONTRACTOR REPRESENTATIVE REMARKS

PERFORMANCE ASSESSMENT REPORT

Continuation Sheet

(Number to correspond with applicable Item Number on reverse)

Performance Assessment Report (PAR) Template, Feb 05 (SAF/AQC)

Performance Assessment Report (PAR) Instructions

Block 1. Quality Assurance Personnel (QAP) enters contract or task order number.

Block 2. QAP enters contractor name.

Block 3. QAP enters type of services.

Block 4. QAP signs and dates.

Block 5. QAP enters telephone number.

Block 6. QAP assigns suspense date for Contractor Representative validation if a deficiency box was checked in Block 7.

Part I. Contractor Performance

Block 7. QAP checks all boxes that apply.

Block 8. QAP enters Services Summary or Statement of Work (SOW) paragraph item reviewed.

Block 9. QAP enters brief description of deficiency and tracking number if the deficiency box was checked in Block 7. Detailed description will be on Corrective Action Report (CAR).

Block 10. QAP enters a detailed performance assessment if the deficiency box in Block 7 was not checked.

Parts II and III will be used only if a deficiency was documented in Part I. Part II. Contractor Validation Block 11. Contractor Representative checks one box in response to QAP cited deficiency. If Concurrence is indicated, Contractor Representative continues to Block 12. If Non-Concurrence is indicated, continue to Block 13 and state reason(s) for Non-Concurrence with QAP cited deficiency and return PAR to QAP.

Block 12. Contractor Representative enters Estimated Completion Date of Corrective Action and continues to Block 13.

Block 13. Contractor Representative details Corrective Action and the Action Taken to Prevent Recurrence or Reason for Non-Concurrence of QAP cited deficiency.

Part III. Deficiency Corrected

Block 14. QAP checks appropriate block, signs, and dates.

Block 15. (Required Entry) QAP remarks regarding the Corrective Action and the Action Taken to Prevent Recurrence. If Non-Concur was checked, this block should be very detailed to explain why.

Block 16. Contractor Representative enters remarks if further discussion is needed about QAP concurrence with Contractor’s Corrective Action and Action Taken to Prevent Recurrence.

Continuation Sheet. The Continuation Sheet is to be used to expand on any information that will not fit in the applicable numbered block.

Appendix B: Customer Complaint Form

CUSTOMER COMPLAINT RECORD

DATE/TIME OF COMPLAINT

SOURCE OF COMPLAINT

ORGANIZATION

BUILDING NUMBER

INDIVIDUAL

PHONE NUMBER

NATURE OF COMPLAINT

CONTRACT REFERENCE

VALIDATION

DATE/TIME CONTRACTOR INFORMED OF COMPLAINT

ACTION TAKEN BY CONTRACTOR

RECEIVED/VALIDATED BY

Customer Complaint Record Template, Feb 05 (SAF/AQCP) COMPLAINANT - 1

Customer Complaint Record Template, Feb 05 (SAF/AQCP) QAE - 2

Customer Complaint Record Template, Feb 05 (SAF/AQCP) CONTRACTOR - 3

Customer Complaint Record Template, Feb 05 (SAF/AQCP) CONTRACT ADMINISTRATOR - 4

Appendix C: Corrective Action Report

CORRECTIVE ACTION REPORT (CAR)

(If more space is needed, use reverse and identify by number)

1. CONTRACTOR

2. CONTRACT NUMBER

3. TYPE OF SERVICES

4. FUNCTIONAL AREA

5. SUSPENSE DATE

6. CONTROL NUMBER

7. DEFICIENCY MAJOR MINOR

FINDING:

FINDING IMPACT:

Please respond with a written corrective action plan that details the corrective action of the cited deficiency, the cause of the deficiency, and actions taken to prevent recurrence by Suspense Date in Block 5. If date was not entered in Block 5, the contractor is not required to provide a response.

8. QUALITY ASSURANCE PERSONNEL (QAP)

TYPED NAME AND GRADE

SIGNATURE AND DATE

9. ISSUING AUTHORITY

TYPED NAME AND GRADE

SIGNATURE AND DATE

10. QAP RESPONSE TO CONTRACTOR CORRECTIVE ACTION AND ACTION TAKEN TO PREVENT RECURRENCE

11. QAP DETERMINATION

ACCEPTED REJECTED

12. CLOSE DATE

Corrective Action Report (CAR) Template, Feb 05 (SAF/AQCP)

CORRECTIVE ACTION REPORT (CAR)

Continuation Sheet

(Number to correspond with applicable Item Number on reverse)

Corrective Action Report (CAR) Template, Feb 05 (SAF/AQCP)

Corrective Action Report (CAR) Instructions

Block 1. Enter Contractor Name.

Block 2. Enter Contract Number.

Block 3. Enter Contract Name or Type of Services.

Block 4. Enter Functional Area of the Contract.

Block 5. Enter the assigned suspense date given the contractor to provide a response to the CAR. A date must be entered for a Major CAR. A date is optional at the discretion of the Quality Assurance Personnel (QAP) initiating the CAR if the finding is Minor.

Block 6. All CARs will be tracked with a Control Number. The functional commander/functional director is the Issuing Authority for his/her contract Quality Assurance Evaluators (QAPs) generating CARs. By providing the control number to the QAP originating the CAR, the Issuing Authority demonstrates concurrence with the finding. The control number will be made up of the first two letters of the name of the site or contract, the last two digits of the calendar year, and a three-digit number starting with 001 and progressing upward throughout the calendar year (e.g., WP04-001).

Block 7.

1. Check the block that indicates whether the identified deficiency is assigned as a Major or Minor finding.

See Performance Plan (PP), paragraph 6.1.4, for the definitions of Major and Minor findings and associated explanations.

2. Finding: Clearly state the details of the finding followed by a reference to the stated contractual requirement. The reference must state the portion of the contract, part, section, paragraph, and subparagraph and must make a complete brief quotation of the contract reference.

3. State the impact that the finding has or could have on the accomplishment of the mission that the contract provides.

4. When determining how long to give the contractor to respond to the identified finding, the normal is 10 working days. The suspense date should reflect this unless the finding requires a greater amount of time to come to solution. Major findings require a contractor response. Minor findings may require a contractor response at the discretion of the QAP initiating the CAR and/or the Issuing Authority.

Blocks 8 and 9. Contract QAPs initiating a CAR must sign in Block 8 and the Issuing Authority signs in Block 9. Each annotates the date at the time of signature.

Block 10. Upon review of the contractor’s reply, the originator of the CAR will enter comments regarding acceptance or rejection of the contractor’s response. This block may also contain comments regarding follow-on inspections conducted or needed at a later date to validate that the finding has not reoccurred.

Block 11. The QAP selects Accept or Reject after reviewing the contractor’s response.

Block 12. The QAP enters a close date after advising the Issuing Authority that the contractor’s response is acceptable and the Issuing Authority concurs. The Issuing Authority will then forward the closed CAR to the contractor and report accordingly in the monthly Certificate Of Service (COS) or other applicable contracting officer services report. This routing remains the same if the contractor response is unacceptable and the CAR remains open until an acceptable response is received except that the Issuing Authority forwards the open CAR back to the contractor for additional information.

Continuation Sheet. The Continuation Sheet is to be used to expand on any information that will not fit in the applicable numbered block.