DOT COE - Section C - Quality Assurance Surveillance Guide 2.doc

Date July 29, 2008 Department of Transportation Common Operating Environment—Quality Assurance Surveillance Guide

DRAFT

Department of Transportation

Common Operating Environment Quality Assurance Surveillance Guide Date July 29, 2008

This is a component of the RFP for Common Operating Environment services for the Department of Transportation. Unless otherwise expressly defined herein, the capitalized terms used herein shall have the meaning assigned to them in the Attachment A (Definitions) or in the Agreement.

Table of Contents

11.0 Requirements of a Quality Assurance Plan

22.0 Purpose of this Document

33.0 Project Objectives

44.0 Commercial Quality Standards

55.0 Meaningful Measures

66.0 Best Practices For Performance-Based Service Contracting

66.1 Quality Assurance Plan And Surveillance

66.2 Style and Administration

66.3 Methods

76.4 Quality Assurance Resource Requirements

86.5 Surveillance Requirements

86.6 Extent of Surveillance

97.0 Definitions

108.0 Allocation of Inspection Responsibilities

108.1 Contracting Office Responsibilities

108.2 The Role of Project Personnel

118.3 Contractor Responsibilities

129.0 Contract Quality Requirements

129.1 General Requirements

129.2 Reserved Conditions

1310.0 Government Quality Assurance Procedures

1310.1 General

1310.2 Government QA at Source

1410.3 Subcontracts.

1511.0 Nonconforming supplies or services.

1511.1 Ongoing Services

1511.2 Assets

1511.3 Project Work

1511.4 Retesting

1611.5 Disposition

1611.6 Penalties

1812.0 Acceptance

1812.1 General.

1812.2 Place of acceptance.

1812.3 Certificate of conformance.

1812.4 Material Inspection and Receiving Reports

1913.0 Instructions to Offerors

1.0 Requirements of a Quality Assurance Plan

Federal Acquisition Regulation 38.6 requires Federal Agencies and Departments to use measurable performance standards and quality assurance surveillance plans. These plans, described in FAR Part 46 address both the Contractor's role in quality control and the Government's role in quality assurance.

While there are a number of approaches to Quality Management, the preferred approach , under the performance management construct, is to require the Contractor to propose performance metrics and to produce the quality assurance plan (QAP).

http://isb.wa.gov/tools/pmframework/planning.aspxThis is especially suitable when using a Statements of Objectives (but equally effective with detailed work statements) because solutions are not known until proposed. With most acquisitions, Offerors are free to develop their own solutions, so it makes sense for them to develop and propose a QAP that is tailored to their solution and permits the use of their tools and practices. If the service recipient were to develop the QAP, it could very well limit what Contractors can propose and shifts a great deal of responsibility to the government in identifying and aligning tools and processes necessary to managing performance. Thus, the governmental practice is to define requirements and evaluate each Offerors’ capabilities and maturity levels in managing quality and performance.

2.0 Purpose of this Document

As the government team considers what is appropriate and required in a QAP for each service area, it may be useful to consider the necessity for quality control. In short, QAPs are quite necessary when federal acquisitions force innovation or are dominated by low-cost selections. In the first case, the need for high innovation and development sometimes results in the use of ideas that are new and unproven. This emphasizes the need for testing and comes with a high cost. In the second case, low-cost awards or the acquisition of commodity-like services can force contractors to take financial risks simply to win the award. While the belief is that all parties enter into a contract in good faith, low-cost / commodity-like services can lead to contractors protecting profit margins through sub-standard delivery. This often led to the government designing elaborate testing criteria and incurring large costs for staffing quality control teams. However, with best-value selection the emphasis on past-performance and on the evaluation of the contractor’s capabilities, alignment, and application of their quality processes seeks to place appropriate responsibility on the contractor.

The regulations have changed to reflect this reality. FAR 46.102 provides that contracts for commercial items (FAR Part 12) “shall rely on a Contractor's existing quality assurance system as a substitute for compliance with Government inspection and testing before tender for acceptance unless customary market practices for the commercial item being acquired permit in-process inspection." However, even with use of negotiated procurements (FAR Part 15) the concept is the same and can be as equally applicable to complex acquisition projects. For standard service offerings, or where customization relies heavily on contractor processes, the expectation should be that the contractor will have a quality assurance program, maturity, and experience with tools and processes for managing performance to its own solutions.

3.0 Project Objectives

The objective of the ITO project effort is to provide the government with integrated Information Technology services and to improve infrastructure support to DOT locations around the United States and its territories. As part of the contract, it is expected that the Service Provider will implement a quality program that provides visibility into supporting processes and daily operation activities. In addition, it is expected that the Service Provider’s quality program and controls will result in an integrated Quality architecture (e.g. Information Technology Services Management) and in the development of performance based management disciplines. This environment will provide maximum flexibility, allow for innovation, and further promote a value proposition through improved service, visibility, and performance to its customers. Specific objectives of the contractor’s QAP shall demonstrate how the contractor will:

· Maintain alignment of service objectives with the mission of DOT Operating Divisions and with the needs of stakeholders

· Ensure compliance and accountability for service assurance through various incentives and disincentives

· Minimize risk to the Government through quality programs and continuous improvement

· Assign and allocate roles and responsibilities (e.g. project team, back-office support, sub-contractors) in order to achieve results through integrated practices, tools, and proven processes

· Survey, analyze and address user requirements, information infrastructure issues and legacy / migration system architectures specific to these areas of operation.

· Introduce additional capabilities into the DOT environment (e.g., multiple increments, evolutionary deployment, fast-track) given considerations of risk and funding

· Sustain and manage the placement of technical personnel to maintain, support and enhance upon, current and proposed operations for the duration of task efforts

· Maintain compliance to federal policies

4.0 Commercial Quality Standards

Rather than inventing metrics or quality or performance standards, integrated solution / government team will seek to use existing commercial quality standards (identified during market research), such as but not limited to: the Information Technology Infrastructure Library (ITIL), International Standards Organization (ISO) 9000, or the Software Engineering Institute's Capability Maturity Models Integration® (CMMI).

The instructions to Offerors shall require potential contractors to, at a minimum, submit a Quality Assurance Plan outlining their Quality Assurance Programs, application of tools and processes within the project, and method of proposed incentives and disincentives.

5.0 Meaningful Measures

Whether the measures are developed by the proposing Contractor or by the government’s integrated project team, it is important to limit the measures to those that are truly important and directly tied to the program objectives. The Attachment entitled Government Surveillance and Performance Measures contains an initial view of minimum measures and acceptable quality levels. However, the intent is for Contractors to review and propose measures that are selected with consideration to requirements and to their standards and practices; and with consideration and balance to costs. From an evaluation perspective the government team will want to determine that the cost of measurement does not exceed the value of the information and that more expensive means of measurement are used only when necessary to mission-critical requirements.

There are many types of performance measures. Measure can be operational, economic and financial (such as return on investment), or highly quantitative and qualitative. They can range from detailed and specific (e.g. error rates, shipments on time) to the more esoteric (e.g. Customer Value, public image, cost of quality). Regardless of the metrics used the one factor that holds true is that Stakeholders are best served when measures relate directly to the project objectives for which there are clearly defined outcomes.

One important step in ensuring the continued alignment of measures and project objectives is for the project to reserve the right to change metrics and measures. One effective way to do this is for the service recipient and the Contractor to meet regularly to review performance and to adjust measures accordingly. This requires that the contractual documents include provisions for a “balanced scorecard; through the use of other provisions such as value engineering change provisions, share-in-savings options; or simply through a provision preserving the Government's unilateral right to review and revise.

6.0 Best Practices For Performance-Based Service Contracting

6.1 Quality Assurance Plan And Surveillance

This QAP defines what the Government must do to ensure that the Contractor has performed in accordance with the contract and performance standards. This can range from a one-time inspection of a product or service to periodic in-process inspections of on-going product or service delivery, and is needed to ensure the Government receives the quality of services called for under the contract—paying only for the acceptable level of services received.

For the Service Recipient an initial set of quality measures are set forth in the RFP/RFQ (see attachment entitled Performance Measures) and Contractors will be required to submit detailed plans showing their approach for applying controls. Individual Task Order Requests may contain unique instructions and may vary depending on scope and complexity.

A good QAP should include a surveillance schedule and clearly state the surveillance method(s) to be used (proposed by Contractor) and what systems will be employed in the monitoring function. These systems are expected to provide automated feeds to a collective reporting system that will inform management and provide situational awareness. Additionally, the QAP development is intended to establish how resources will be used to manage performance, mitigate risk, and ensure that the Government receives what it is paying for (typically through audit). Development of the QAP also allows the Agency or departments to clearly define the amount of administrative resources needed to ensure contractual compliance.

Principles and Negotiation Objectives: 1) Details in the QAP regarding a particular task should be consonant with the importance of the task. 2) The QAP should first focus on the quality, quantity, and timeliness of performance and the outputs to be delivered by the Contractor and then on the steps or procedures used to provide the product or service.

6.2 Style and Administration

In some cases the QAP may be a stand-alone document or combine the Task Order PWS and QAP into one document. Each project manager should determine the most appropriate means for administration. For example, this latter situation makes it easier for both the Contractor and inspector to understand and administer the contract requirements. Through the process of proposal responses and discussions, the Government will seek to define the most appropriate and efficient method to state quality measures and control processes.

6.3 Methods

Selecting the most appropriate surveillance method for the effort involved is important. The government will take into consideration each task, criticality, task lot size, surveillance period, performance requirements and standards, availability of quality assurance evaluators (QAEs), surveillance value in relation to task cost/criticality, and available resources. Careful selection of appropriate surveillance methods enables the Government to determine the amount of resources and associated costs needed to perform the surveillance task.

Acceptable surveillance methods include:

· 100 Percent Inspection: This is usually only the most appropriate method for infrequent tasks or tasks with stringent performance requirements, e.g., where safety or health is a concern. With this method, performance is inspected/evaluated at each occurrence. One hundred percent inspection is too expensive to be used in most cases.

· Random Sampling: This is usually the most appropriate method for the government in assessing recurring tasks. With random sampling, services are sampled to determine if the level of performance is acceptable and if performance is on target with, or meeting, parameters established within Service metrics or by Service Level Objectives. Random sampling works best when the number of instances of the services being performed is very large or ongoing (e.g. daily operations), and to which a statistically valid sample can be obtained. The government may determine whether to establish their own system for monitoring or to monitor through use of the contractor’s systems and sampling procedures.

· Periodic Inspection: This method, sometimes called "planned sampling," consists of the evaluation of tasks selected on other than a 100 percent or random basis. It may be appropriate for tasks that occur infrequently, and where 100 percent inspection is neither required nor practicable. A predetermined plan for inspecting part of the work is established using subjective judgment and analysis by the recipient’s resources in deciding what work to inspect and how frequently to inspect it, and then to compare results to Service metrics or to Service Level Objectives. Once again, the government may determine whether to establish their own system for monitoring or to monitor through use of the contractor’s systems and sampling procedures.

· Customer Input: Although usually not a primary method, this is a valuable supplement to more systematic methods. For example, in a case where random sampling indicates unsatisfactory service, customer complaints can be used as substantiating evidence. In certain situations where customers can be relied upon to complain consistently when the quality of performance is poor (e.g., collaborative systems such as email, application availability, or building services) customer surveys and customer complaints may be a primary surveillance method. It is important to identify appropriate elements of customer satisfaction and to establish consistent performance standards. In all cases, complaints and the disposition of each should be documented and communicated to appropriate organizations.

The government will also need to discuss the surveillance methods, to be used, with the Contractor and confirm that they are fully understood. Whatever the final form of surveillance determined, the government project office must take care to ensure that no undue interference with Contractor operations occurs; and to also avoid cumbersome and intrusive process-oriented inspection and oversight programs while assessing Contractor performance. However, at a minimum, surveillance must be performed as stated in the contract QAP.

6.4 Quality Assurance Resource Requirements

It is expected that companies will have existing quality assurance programs for their services or products, and will avoid customization by leveraging existing processes and appropriately skilled personnel. As such, the expectation is that the government’s Quality Assurance resources can be specially trained in the contractor’s QA functions or are Engineers by education. In either case, these personnel must be fully qualified to meet the major responsibilities of the position: maintaining complete and accurate documentation; sustaining a relationship of trust and fairness with the Contractor; and holding a thorough knowledge of the contract and requirements.

Experience and training are essential for effective surveillance. The Quality Assurance Evaluator (QAE) should be identified with a letter of assignment that includes a copy of the contract and negotiated surveillance plan. However, QAEs should be cautioned against making legal interpretations, imposing tasks not included in the contract, supervising Contractor employees, or dismissing (waiving) contract requirements (functions reserved unto the COTR and Contracting Officer). A copy of any such letters must be provided to the Administrative Contracting Officer.

Ideally, QAEs will be dedicated full time to quality assurance activities. If this is not possible, the QAE aspect of the individual's job should be made a critical element in their performance review or appraisal.

6.5 Surveillance Requirements

Reviewing and discussing the Contractor's plan for maintaining an acceptable quality level under the contract is important, and Contractors will be required to submit a Quality Control Plan to the Government as part of their initial technical and management submissions. Such plans are expected to include the use of surveillance checklists, industry techniques for inspection, correspondence reviews, customer surveys, audits, and routine standard reports. As part of implementing or managing ongoing quality processes, both the Government and Contractor teams will be briefed on surveillance requirements and responsibilities at a post-award conference or kick-off. Surveillance plans must be comprehensive, systematic, and well documented with implementation during the Transition period(s).

6.6 Extent of Surveillance

The extent of surveillance will be predefined before award (Contract or Task Order specific) but will be determined only through negotiations and reflected by the surveillance schedule established in the QAP. Surveillance is expected to be systematic and sufficient so as to fairly evaluate the Contractor's total performance throughout the measurement and performance period.

Where surveillance results show good performance consistently, the amount of surveillance should be adjusted accordingly. This saves the Government money, reduces the burden of oversight on the Contractor, recognizes the Contractor's achievement of performance, and promotes trust through controls (not blind trust). In addition, Contractors will be free to propose incentive plans that reward the Contractor for areas of value and where exceptional performance occurs.

When performance is deficient, the COTR will be required to notify the Contracting Officer who, in turn, will promptly notify the Contractor’s project manager and contract administration organization for the need to promptly take and track corrective action. Where appropriate, penalties will be assessed and reflected in the remittance of payment for Contractor invoices.

7.0 Definitions

"Acceptance" means the act of an authorized representative of the Government by which the

Government, for itself or as agent of another, assumes ownership of existing identified supplies tendered or approves specific services rendered as partial or complete performance of the contract.

"Conditional acceptance" means acceptance of supplies or services that do not conform to contract quality requirements, or are otherwise incomplete, that the contractor is required to correct or otherwise complete by a specified date.

"Contract quality requirements" means the technical requirements in the contract relating to the quality of the product or service and those contract clauses prescribing inspection, and other quality controls incumbent on the contractor, to assure that the product or service conforms to the contractual requirements.

"Critical nonconformance" means a nonconformance that is likely to result in hazardous or unsafe conditions for individuals using, maintaining, or depending upon the supplies or services; or is likely to prevent performance of a vital agency mission.

"Government contract quality assurance" means the various functions, including inspection, performed by the Government to determine whether a contractor has fulfilled the contract obligations pertaining to quality and quantity.

"Major nonconformance" means a nonconformance, other than critical, that is likely to result in failure of the supplies or services, or to materially reduce the usability of the supplies or services for their intended purpose.

"Minor nonconformance" means a nonconformance that is not likely to materially reduce the usability of the supplies or services for their intended purpose, or is a departure from established standards having little bearing on the effective use or operation of the supplies or services.

"Off-the-shelf item" means an item produced and placed in stock by a contractor, or stocked by a distributor, before receiving orders or contracts for its sale. The item may be commercial or produced to military or Federal specifications or description.

"Patent defect" means any defect which exists at the time of acceptance and is not a latent defect.

“QAP” means Quality Assurance Plan and is intended to define the application of quality concepts and processes by which the parties will assure quality and performance. It is synonymous with Quality Assurance Surveillance Plan.

"Testing" means that element of inspection that determines the properties or elements, including functional operation of supplies or their components, by the application of established scientific principles, industry standards, and procedures.

8.0 Allocation of Inspection Responsibilities

8.1 Contracting Office Responsibilities

Contracting offices are responsible for oversight of the project by ensuring technical requirements reflect any specifications for inspection, testing, and other contract quality requirements essential to ensure the integrity of the supplies or services. However, the activity responsible for technical requirements is responsible for prescribing contract quality requirements, such as inspection and testing requirements or, for service contracts, a quality assurance surveillance plan.

The Contracting Officer or designee shall verify that the contractor fulfills the contract quality requirements and ensure that nonconforming results are identified and that remedial actions are taken. Verification may be done in participatory meetings (e.g. monthly IPRs) or by receipt of written reports but should give consideration to the significance of the requirement and risk of non-conformance.

The Contracting Officer shall also appropriately communicate requirements for contractor quality of the supplies or services to be acquired, and issue any necessary instructions to the cognizant, contract administration office, and act on recommendations submitted by that office.

8.2 The Role of Project Personnel

The Contractor shall maintain and DOT personnel shall ensure that:

1) Contract performance includes inspection and other quality requirements that ensure compliance to contractual and operational requirements (e.g. management of warranty clauses when appropriate and necessary to protect the Government's interest);

2) Supplies or services tendered by contractors meet standards and requirements;

3) Quality assurance is conducted before acceptance (except as otherwise provided in the QAP, or under the direction of Government personnel);

4) No contract modification or Task Order precludes the Government from performing inspection;

5) Nonconforming services or materials are documented and rejected;

6) Services and Commercial items rely on a contractor's existing quality assurance system as a substitute for compliance with Government inspection and testing before tender for acceptance unless customary market practices for the commercial item being acquired permit in-process inspection

7) The quality assurance and acceptance services of other agencies are used when this will be effective, economical, or otherwise in the Government's interest;

8) A system of efficient procedures for performing Government contract quality assurance is developed and applied;

9) The project team performs all actions necessary to verify whether the supplies or services conform to contract quality requirements;

10) Performance records are maintained; suitable and reflective of:

a) The nature of the quality assurance actions, including, when appropriate, the number of observations made and the number and type of defects;

b) Decisions regarding the acceptability of the services or products, the processes, and the requirements; as well as action to correct defects;

c) Implementing any specific written instructions from the contracting office;

d) Reporting observed defects in design or technical requirements, including contract quality requirements; and

e) Recommendations for changes necessary to the contract, specifications, instructions, or other requirements that will provide more effective operations or eliminate unnecessary costs.

8.3 Contractor Responsibilities

Notwithstanding the other duties and responsibilities outlined in the final QAP the contractor is responsible for carrying out its obligations under the contract by:

1) Controlling the quality of supplies or services;

2) Tendering to the Government for acceptance only those supplies or services that conform to contract requirements;

3) Ensuring that vendors or suppliers of inputs (e.g. materials, parts, subcontracted services) have metrics and are subject to an acceptable quality control system;

4) Maintaining and substantiating evidence, when required by the contract, that the supplies or services conform to contract quality requirements, and are able to furnish such information to the Government as required;

5) Provide and maintain an inspection system and program for quality assurance that can ensure control of quality acceptable to the Government.

The control of quality by the contractor may relate to, but is not limited to:

1) Back-office, support, overhead and functional support processes, manufacturing processes, or any other non-direct organization necessary to meeting the contract's technical requirements;

2) Timely develop and update to drawings, specifications, and engineering changes, to ensure configuration management operational processes meet the contract's technical requirements;

3) Frequent testing and examination to ensure that practices and equipment provide the means for optimum evaluation of the characteristics subject to inspection;

4) Reliability and maintainability assessments (resourcing, sustainability, and continued readiness);

5) Fabrication and delivery of products, to ensure that only conforming products are tendered to the Government;

6) Including adequate technical and management documentation (e.g. diagrams, drawings, specifications, handbooks, manuals, and other technical publications);

7) Appropriate labeling, packaging, packing, and marking; and

8) Establish consistent procedures and processes for services to ensure that services meet contract performance requirements.

9.0 Contract Quality Requirements

9.1 General Requirements

Each solicitation should clearly identify phases (e.g. Transitional or operational), areas of measurement, and the extent of service measures and service level objectives. Where task orders are to be used, the governing contract should include the type and extent of contract quality requirements and make provision for more detailed requirements as needed and described with each task order request. In addition, each service area or “tower of service” may include differing requirements ranging from inspection at time of acceptance to a requirement for the contractor to implement a comprehensive quality program.

Permitted by the FAR solicitations and contracts may provide for alternative, but substantially equivalent, inspection methods to obtain wide competition and low cost. Under the DOT ITO contract it is anticipated that the contract line item types may consist of Firm Fixed Price, Fixed Price Labor Hour, and potentially Cost Reimbursable services and that the appropriate FAR clauses governing inspection will be included.

Unless specified in the contract or within the any Task Order the Government shall rely on the contractor to accomplish all inspection and testing needed to ensure that supplies or services delivered conform to contract quality requirements before they are tendered to the Government for acceptance. However, the Government reserves the right to perform its own inspections and not rely on inspection by the contractor if the contracting officer determines that the Government has a need to test the supplies or services in advance of their tender for acceptance, or to pass judgment upon the adequacy of the contractor's internal work processes. However, in making the determination, the contracting officer should consult with the project team and shall consider:

1) The nature of the supplies and services being purchased and their intended use;

2) The potential losses in the event of defects;

3) The likelihood of uncontested replacement or correction of defective work; and

4) The cost of detailed Government inspection.

9.2 Reserved Conditions

Under this approach to quality and testing, DOT shall permit the contractor to make use of its tools and quality methods, but shall require that any such tools, methods, and systems produce output that is acceptable to the Government. Second, the Government shall have the right to make inspections and tests while work is in process and to require the contractor to keep complete, and make available to the Government, records of its inspection work.

10.0 Government Quality Assurance Procedures

10.1 General

DOT shall monitor and periodically audit the contractor’s quality control procedures for ability to ensure compliance to contractual requirements. DOT may choose to perform quality checks with project personnel or by use of a designated contractor to perform Independent Verification and Validation (IV&V). Such checks shall be performed at such times (including any stage of manufacture or performance of services) and places (including subcontractors' plants) as may be necessary to determine that the supplies or services conform to contract requirements.

This Quality Assurance Surveillance Plan has been prepared in conjunction with the preparation each statement of work for the ITO Towers of Service:

· Seat Management

· Infrastructure Management, and

· Application Hosting The Quality Assurance Plan should specify all work requiring surveillance and the method of surveillance. Each contract shall designate the place or places where the Government reserves the right to perform quality assurance, and if the contract provides for performance of Government quality assurance at source, the place or places of performance may not be changed without the authorization of the contracting officer.

If a contract provides for delivery and acceptance at destination and the Government inspects the supplies at a place other than destination, the supplies shall not ordinarily be re-inspected at destination, but should be examined for quantity, damage in transit, and possible substitution or fraud. In all cases Government inspection shall be performed by or under the direction or supervision of Government personnel with inspections documented on an inspection or receiving report form or commercial shipping document/packing list, under agency procedures.

10.2 Government QA at Source

The FAR requires that Agencies perform contract quality assurance, including inspection, at source if performance at any other place would require uneconomical disassembly or destructive testing. For DOT, the most logical point of inspection is at the final place of installation or delivery (Hardware and Software IMACs). For services inspection is the monitoring and review (reports and periodic audit) of services delivered. These can take various forms (e.g. a helpdesk call or network availability).

Thus, attainment of service level objectives is a key measure of quality and performance, which are tendered for acceptance through monthly reviews and submission of invoices by the contractor.

In addition, there may be situations where the government determines that inspection of supplies or services earlier in the process is more effective and warranted. Examples of these situations would be the need for demonstrations, special instruments, gauges, or facilities that are only available at an upstream source; considerable loss may result from the shipment of unacceptable supplies; or from the delay in making necessary corrections (e.g. image build and test).

10.3 Subcontracts.

Contract quality assurance by the Government on subcontracted supplies or services shall be performed only when required as being in the best interest of the Government. Typically, such situations are only needed to assist the cognizant, government contract administration office having responsibility over a prime contractor's plant in determining the conformance of subcontracted supplies or services with contract requirements. Such should not be the case with DOT with the possible exception of application hosting [to be determined].

However, it is important to note that any such inspection by the government of Subcontractors does not relieve the prime contractor of any responsibilities under the contract. If and when appropriate, the prime contractor shall be requested to arrange for timely Government access to the subcontractor facility. The Government shall then perform quality assurance at the subcontract level only after:

· The Prime makes such arrangements,

· Determination has been made that supplies or services for which certificates, records, reports, or similar evidence of quality are not available, and

· All oral and written statements and contract terms and conditions relating to Government quality assurance actions at the subcontract level shall be worded so as not to:

· Affect the contractual relationship between the prime contractor and the Government, or between the prime contractor and the subcontractor;

· Establish a contractual relationship between the Government and the subcontractor; or

· Constitute a waiver of the Government's right to accept or reject the supplies or services.

11.0 Nonconforming supplies or services.

The contracting officer should reject supplies or services not conforming in all respects to contract requirements (see FAR 46.102) unless, in those instances where deviation from this policy is found to be in the Government's interest, such supplies or services may be accepted.

Ordinarily the Contracting Officer must give the contractor an opportunity to correct or replace nonconforming supplies or services when this can be accomplished within the required delivery schedule. Unless the contract specifies otherwise (as may be the case in some cost-reimbursement contracts), correction or replacement must without additional cost to the Government. In the case of DOT these concepts will hold true but with some variation.

11.1 Ongoing Services

For the most part DOT will be acquiring services under a performance based contract. In an operational context (as opposed to a project) services will be ongoing and are measured against a series of Services Level Objectives (see individual Statements of Work). Where service levels are missed these will be subject to the final agreement on “incentives and disincentives” and the contractor will be expected to take corrective action. Where chronic failures are experienced the Contracting Officer will make determination as to the government’s rights and remedies (e.g. issuing a show cause, etc.).

11.2 Assets

In the case of acquired assets the responsibility for non-conforming supplies will fall to the party owning the asset. That is, for government owned assets the service provider may have responsibility for provisioning (i.e. ordering, receiving, deploying) but the true contractual relationship rests with the seller and the contract holder (in this scenario the government). Thus, the service provider may have responsibility for ordering and processing returns for defects but potentially may be excused from a service level if the contract or task order is not clear on the allocation of responsibility. In cases where assets are directly acquired by the contractor for use or transfer, bundled within a service, or as a delivered supply item then the contractor will have clear accountability for any non-conforming supplies.

11.3 Project Work

Project Work is that which is operationally related but has a finite start and completion date. Project Work should not be confused with routine services such as Pc Installations, Moves, Adds, and Changes (IMACs) which are provided for in within statements of work and covered with contract line item pricing. Acceptance of Project Work shall be made against criteria establish with each Task Order and each respective project.

11.4 Retesting

Paragraph (e)(2) of the clause at 52.2462, Inspection of Supplies-Fixed-Price, reserves to the Government the right to charge the contractor the cost of Government re-inspection and retests because of prior rejection. However, the nature of the scope of work and service activities under the DOT ITO contract are such that inspection and acceptance are expected to become embedded and a significant part of the daily operational activities. For example, Incident Management and Problem Resolution are expected and the cost of retesting or corrective action is part of the ongoing operational state. As such the government should use such rights judiciously and where appropriate (e.g. Rollout and deployment of large scale software upgrade).

11.5 Disposition

In situations not covered by this section, the contracting officer ordinarily must reject supplies or services when the nonconformance is critical or major or the supplies or services are otherwise incomplete. However, there may be circumstances (e.g., reasons of economy or urgency) when the contracting officer determines acceptance or conditional acceptance of supplies or services is in the best interest of the Government. The contracting officer must make this determination based upon:

1) Advice of the technical activity that the item is safe to use and will perform its intended purpose;

2) Information regarding the nature and extent of the nonconformance or otherwise incomplete supplies or services;

3) A request from the contractor for acceptance of the nonconforming or otherwise incomplete supplies or services (if feasible);

4) A recommendation for acceptance, conditional acceptance, or rejection, with supporting rationale; and

5) The contract adjustment considered appropriate, including any adjustment offered by the contractor.

Typically, the cognizant contract administration office, or other Government activity directly involved, must furnish this data to the contracting officer in writing (except in urgent cases when it may be furnished orally and later confirmed in writing). However, for purposes of managing IT operational services it is expected that the government COTR and operations personnel will monitor performance and require frequent reporting (e.g. daily, weekly, monthly as appropriate—but see government surveillance and performance measures attachment) for which the Contracting Officer may participate in meetings or receive reports.

Before making a decision to accept, the contracting officer must obtain the concurrence of the activity responsible for the technical requirements of the contract and, where health factors are involved, of the responsible health official of the agency concerned. If the nonconformance is minor, the cognizant contract administration office may make the determination to accept or reject, except where this authority is withheld by the contracting office of the contracting activity. To assist in making this determination, the contract administration office may establish a joint contractor-contract administrative office review group. Acceptance of supplies and services with critical or major nonconformance is outside the scope of the review group.

11.6 Penalties

The contracting officer must discourage the repeated tender of nonconforming supplies or services, including those with only minor nonconformance, by appropriate action, such as rejection and documenting the contractor's performance record. Typically, when supplies or services are accepted with critical or major nonconformance the contracting officer must modify the contract to provide for an equitable price reduction or other consideration. In the case of DOT ITO Services the contract or task orders will define a system of disincentives that may automatically reduce payment to the contractor.

In the case project work or where conditional acceptance occurs, any amounts withheld from payments should be at least sufficient to cover the estimated cost and related profit to correct deficiencies and complete unfinished work. In such situations the contracting officer must document in the contract file the basis for the amounts withheld.

For services, the contracting officer can consider identifying the value of the individual work requirements or tasks (subdivisions) that may be subject to price or fee reduction. This value may be used to determine an equitable adjustment for nonconforming services. However, when supplies or services involving minor nonconformance are accepted, the contract need not be modified unless it appears that the savings to the contractor in fabricating the nonconforming supplies or performing the nonconforming services will exceed the cost to the Government of processing the modification.

12.0 Acceptance

12.1 General.

Acceptance constitutes acknowledgment that the supplies or services conform to applicable contract quality and quantity requirements, and other terms and conditions of the contract. As outlined above acceptance may take place before delivery, at the time of delivery, or after delivery, depending on the provisions of the terms and conditions of the contract. Supplies or services shall ordinarily not be accepted before completion of Government contract quality assurance actions and is ordinarily evidenced by execution of an acceptance certificate on an inspection or receiving report form or commercial shipping document/packing list. However, in the case of services contracts it is acceptable for the service provider to provide detailed reports and conduct a project review to report the status, progress, and performance of activities. Such reports, if summarized appropriately may be used to support claim for payment to which the government COTR may accept for payment.

Under the DOT ITO contract a determination will need to be made to ascertain the appropriate requirements for inspection and acceptance (e.g. Operation, Project Based, supply item).

12.2 Place of acceptance.

Pursuant to FAR Part 46 each contract shall specify the place of acceptance. Contracts that provide for Government contract quality assurance at source shall ordinarily provide for acceptance at source. Contracts that provide for Government contract quality assurance at destination shall ordinarily provide for acceptance at destination. (For transportation terms, see Subpart 47.3.) Supplies accepted at a place other than destination shall not be re-inspected at destination for acceptance purposes, but should be examined at destination for quantity, damage in transit, and possible substitution or fraud.

DOT ITO will conduct periodic reviews of operational services and any supplies delivered under the contract or resulting task orders. Acceptance shall be made with review and approval of the contractor’s claim for payment (see the following section on Certificate of Conformance and Material Inspection Receiving Reports).

12.3 Certificate of conformance.

Pursuant to FAR Part 46.315 a certificate of conformance may be used in certain instances for supplies and services instead of source inspection (whether the contract calls for acceptance at source or destination) at the discretion of the contracting officer if the following conditions apply:

1) Acceptance on the basis of a contractor's certificate of conformance is in the Government's interest.

2) Small losses would be incurred in the event of a defect; or

3) Because of the contractor's reputation or past performance, it is likely that the supplies or services furnished will be acceptable and any defective work would be replaced, corrected, or repaired without contest. In no case shall the Government's right to inspect supplies under the inspection provisions of the contract be prejudiced.

12.4 Material Inspection and Receiving Reports

Agencies shall prescribe procedures and instructions for the use, preparation, and distribution of material inspection and receiving reports and commercial shipping document/packing lists to evidence Government inspection (see 46.401) and acceptance (see 46.501).

13.0 Instructions to Offerors

ATTACHMENT [insert]: QUALITY ASSURANCE SURVEILLANCE PLAN INSTRUCTIONS

1.0

OVERVIEW

As part of the contract the Offeror shall describe its approach to managing quality and performance at the BPA and Task Order levels. It is expected that quality and performance management will occur primarily at the levels of service delivery (i.e. Task Order and below) but that Quality Assurance and some elements of quality control will occur with managing the contract and customer relationship.

The Offeror’s Quality Assurance Surveillance Plan (QASP), Service Level Performance Management Plan (SLPMP), and Plan for Incentives and Disincentives (PID) will be incorporated into the award as a result of negotiations with the Government. These documents will become part of the Contract Management Plan and Task Order Management Plans. As such, they will serve as living documents that are expected to be modified throughout the term of each Task Order with unique sections, metrics, or information necessary to effectively control performance on each and every Task Order. Updates to the QASP will be made on an annual basis through a governance process (to be defined) and subject to any constraints imposed by the FAR. The Government’s management team will work with the contractor to ensure that updates to the QASP always reflect the key outcomes that are critical to the government.

As requests are made for Task Order proposals the government will provide any available information (e.g. internal service levels, etc.) that Offerors may find useful in determining how services (staffing or managed services) can be met or exceeded. The nature of quality service or assumption of service levels (not intended at this time) will be described by the government with each task order. Offerors shall describe their methods for assuring quality and performance—for which the government and task order contractor will agree on by mutual consent.

In making award the Government will consider the contractor proposed Quality Plan and Plan for Incentives and Disincentives as the Offeror’s minimum performance commitment to fulfilling the objectives of the proposal.

2.0

THE QUALITY ASSURANCE SURVEILLANCE PLAN

The QASP will be used to ensure that systematic quality assurance methods are used in administration of performance-based Task Orders. The Offeror shall propose a QASP that details the application of the Service Provider’s Quality Assurance tools and methods; and how and when the Service Provider will apply quality control to observing, surveying, testing, evaluating, and documenting performance. The QASP shall also detail how the Service Provider will perform quality management and corrective action of the services; and the reporting to be made available to the Government in performing performance management and surveillance. The Offeror shall also describe how the Government will be afforded situational awareness and the manner in which the Offeror will establish transparency in performance visibility provided the Government.

The Quality Plan does not require a complex or administratively burdensome measurement system. DOT prefers an approach that is flexible, transparent, easy to view, and lends itself to clear analysis of performance. However, the plan must be thorough and where possible (e.g. for quantitative performance standards) automated measures are mandatory over manual intervention. As such, the contractor shall describe where the need for low level surveillance or high level surveillance is most effective and appropriate. In all cases the Quality Plan must address a baseline for the “satisfactory” level of performance for each category as identified below.

At a minimum, the Offeror’s proposed Quality Plan should address surveillance and reporting for the following areas:

· Customer Satisfaction—which may include small and large Modes

· Demonstrated ability to meet ITO operational objectives

· IT Service Management processes (e.g. Incident Management, Problem Resolution, Continuous Improvement)

· Ability to work with other contractors in an integrated environment

· Communication with government and other providers

· Collaborative aspects and ability to integrate

· Cooperation with all regardless of roles or position For surveillance of areas shown above, the Offeror should provide:

· Proposed methods for surveillance to be utilized

· How the surveillance…