J_TE11_QASP_1.docx

RFP: 140D0420R0005: Operations and Maintenance Support Services for MRMSS Title: Minerals Revenue Management Support System (MRMSS) - Operations and Support / Development and Enhancement for the Office of Natural Resource Revenue (ONRR)

Section J technical exhibiT(TE11) MRMSS Quality Assurance Surveillance Plan

1.1.1. INTRODUCTION

The Quality Assurance Surveillance Plan (QASP) defines the procedures used by the Office of Natural Resources Revenue (ONRR) to monitor the effectiveness and quality of systems development and operations work performed by contractors.

1.1.1.1. Purpose

The QASP describes the performance management approach used to monitor and evaluate the contractor’s performance to achieve outcomes and performance objectives described in the O&M and DME call orders. The contractor is responsible for quality management necessary to meet the terms of the contract.

The QASP defines:

· Roles and responsibilities for the contractor and TSD Project Managers (PMs)

· Target performance objectives and Key Performance Indicators (KPIs)

· Methodologies used to monitor and evaluate contractor performance

1.1.1.2. Performance Management Approach

The call order structures the acquisition around “what” service or quality level is required, as opposed to “how” the contractor should perform the work. Performance management represents a significant shift from traditional quality assurance (QA) concepts by focusing on outcomes rather than compliance to procedures. The contractor has flexibility on how to perform the work as long as the critical outcomes expected by the QASP are being achieved.

1.1.1.3. Performance Management Strategy

The contractor is responsible for the quality of all work performed. The contractor measures that quality through the Quality Management Plan (QMP). The focus of the QMP is work output, not workers, and therefore includes all work performed under this contract regardless of whether the work is performed by contractor employees or by subcontractors. The QMP sets forth the procedures for self-inspecting the quality, timeliness, responsiveness, customer satisfaction, and other performance requirements. The contractor will develop and implement a performance management system with processes to assess and report its performance to the designated government representative.

The government representative(s) monitor performance and review performance reports furnished by the contractor to monitor the contractor’s performance against communicated performance objectives. The government makes determinations regarding compliance based on performance measurement results. The contractor is responsible for making required changes in processes/procedures/staffing if performance does not meet objectives.

1.1.2. ROLES AND RESPONSIBILITIES

1.1.2.1. The Contracting Officer

The contracting officer (CO) is responsible for monitoring contract compliance, contract administration, and cost control and for resolving any differences between the observations documented by the contracting officer's representative (COR) and the contractor. The CO shall ensure performance of all necessary actions for effective contracting, ensure compliance with the contract terms, and shall safeguard the interests of the United States in the contractual relationship. The CO shall also ensure that the contractor receives impartial, fair, and equitable treatment under this contract.

Assigned CO: TBD

1.1.2.2. The Contracting Officer’s Representative (COR)

The COR provides continuous oversight of the Contractor’s performance and deliveries. COR keeps a Quality Assurance file that accurately documents the contractor’s actual performance. The purpose is to ensure that the contractor meets the performance standards contained in the contract. The COR is responsible for reporting early identification of performance problems to the CO. The COR is required to provide an annual performance assessment to the CO which will be used in documenting past performance. The COR uses the QASP to perform performance oversight. The QASP is a critical tool for documenting contractor performance. The COR is not empowered to make any contractual commitments or to authorize any contractual change on the Government’s behalf.

Assigned COR: TBD

1.1.2.3. Other Key Government Personnel

Other government personnel shall be assigned to assist the COR with contract monitoring. This may include Government Analysts, technical experts, or others who provide information that helps the COR monitor contractor performance.

The following Government personnel shall assist with contract oversight/surveillance activities for this contract but are not empowered to make any contractual commitments or to authorize any contractual change on the Government’s behalf.

Government Quality Assurance Manager (QAM): TBD

1.1.2.4. Contractor Representatives

The following employees of the contractor serve as the contractor’s Project Manager for this contract.

Project Manager :<TBD> - Phone Number: < TBD > - Email: < TBD >

1.1.3. IDENTIFICATION OF REQUIRED PERFORMANCE STANDARDS/QUALITY LEVELS

The required performance standards and/or quality levels are included in QASP Performance Requirements section. If the contractor meets or fails the required performance targets, it may be subject to the incentive/disincentive defined for the objectives.

1.1.3.1. Service Level Agreement (SLA) and Key Performance Indicators (KPIs)

SLA

SLAs define a minimum Acceptable Quality Level (AQL) of service that the contractor will deliver to meet specific outcomes for BSEE. SLAs will be associated with disincentives that may be financial and/or impact contractor performance ratings.

KPI

Key Performance Indicators are metrics that are useful in measuring performance of the project and contract, but definition of target acceptance levels may not be feasible until metrics are collected over a period of time. Such indicators used over time may identify recurring challenges or problems and could result in future SLAs with appropriate target AQLs based upon historical trends and desired outcomes.

KPIs will not have disincentives, but will provide additional factors used to monitor contract performance.

1.1.3.2. Burn-In Period

Recognizing that new contractors who lack familiarity with the BSEE operating environment may require time to understand the nuances of the TIMS operating environment and calibrate their processes to meet target service level objectives, the Government will allow a ninety (90) day burn in period during which SLA and KPIs will be monitored and reviewed, with no assessment of disincentives. Certain target AQLs may be amenable up to 5% AQL adjustments based upon a plan delivered by the contractor to achieve the below listed targets within the first year of performance.

1.1.3.3. Software Defect Severity Classification

This section classifies software defects identified through testing:

· Emergency A defect affecting a production system that is preventing a user from carrying out critical tasks or threatens critical data loss or corruption, and there is no current acceptable work around. Such a problem must be corrected ASAP, usually within 24 hours in production. Examples include a critical report that cannot be generated due to program error, data that cannot be entered into the system due to program error etc.

· Critical A defect that affects critical functionality or critical data. It does not have a feasible work around.

· High A defect that affects major functionality or data but has a work around that is difficult, may not be obvious, or require numerous additional steps to accomplish desired function. Such work arounds would be acceptable only for limited durations.

· Low A defect that affects minor functionality or non-critical data. It has an easy work around, resolution is not time sensitive.

1.1.3.4. PROCEDURES/METHODOLOGIES TO MONITOR PERFORMANCE (Surveillance Techniques) A combination of surveillance methods shall be used by the government to evaluate contractor performance. The methods of surveillance are:

· 100% Inspection

· Government Verification and Validation

· Periodic Inspection

· Random Sampling

· Contractor Self-Reporting 100% Inspection QA method designed to evaluate all outputs of the contract requirement. This method requires a total inspection of Contractor’s performance.

Government Verification and Validation QA method designed to provide an independent review of deliverables.

Periodic Inspection QA method based on a predetermined plan to evaluate part of the contract outputs using subjective judgment and analysis of agency resources to decide what work to inspect and how frequently to inspect it (daily, weekly, monthly, quarterly, etc.) as determined by the COR.

Random Sampling QA method based on sampling deliverables to determine if the level of performance is acceptable.

Contractor Self-Reporting Monthly reports provided by the contractor that identifies problems encountered, risk reporting, defect reports, status reports, etc.

1.1.3.5. Inspection Method Selection

The depth and detail of observations of work shall be based on the relative importance of the items of work under evaluation. The principal method of surveillance shall be Random Sampling; however, any surveillance method can be used at the Government’s discretion. Certain tasks may be so important that nonperformance or poor performance may significantly impact the mission and warrant 100% inspection. Joint inspections may be made by the COR/QAM and the Project Manager and are beneficial when routinely performed. They are vital when any disagreement or misunderstanding occurs.

1.1.3.6. Acceptable Quality Levels

Acceptable Quality Level (AQL) defines the target performance level below which the Government deems the service as not meeting desired objectives. The AQL can be expressed as a number, a percentage, or a quantity per deliverable inspected.

The AQLs for this requirement are included in QASP Performance Requirements section.

1.1.3.7. Quality Ratings

The following ratings shall be used and recorded in the Quality Assurance Monitoring Form to determine whether performance objectives meet or fail the required Acceptable Quality Levels.

ACCEPTABLE:

(SATISFACTORY)

Performance meets contractual requirements.

UNACCEPTABLE:

(UNSATISFACTORY)

Performance does not meet contractual requirements.

1.1.3.8. Unacceptable Performance

When unacceptable performance occurs, the COR shall inform the Contractor Project Manager and/or Contracting Officer and document the results in the Quality Assurance Monitoring Form. The Government works with the Contractor to devise a corrective action plan to improve contractor performance to the required levels.

1.1.4. QUALITY ASSURANCE DOCUMENTATION

1.1.4.1. The Performance Management Feedback Loop

The performance management feedback loop begins with the communication of expected outcomes. Performance standards are expressed and assessed using the performance monitoring techniques shown in QASP Attachment 1 Performance Requirements.

1.1.4.2. Monitoring Form

The government’s QA surveillance accomplished by the QAM or COR, will be reported using the monitoring form in QASP Attachment 2. The form, when completed, will document the government’s assessment of the contractor’s performance under the contract to ensure that the required results are being achieved.

An essential part of any QASP is the Quality Assurance Monitoring Form for the service that will be evaluated. The monitoring form shall contain such information as type of service, inspection results, date, time, and remarks relevant to the inspection. These monitoring forms comprise the basis for all QA evaluations and constitute the documentation which will be used to substantiate determination of unsatisfactory performance should that occur.

1.1.5. ANALYSIS OF QUALITY ASSURANCE ASSESSMENT

1.1.5.1. Determining Performance

The Government shall use the monitoring methods cited to determine whether the performance service levels/AQLs have been met. If the contractor has not met the minimum requirements, it may be asked to develop a corrective action plan to show how and by what date it intends to correct performance to the required levels.

1.1.5.2. Reporting

At the end of each month, the QAM will prepare a written report for the COR summarizing the overall results of the quality assurance surveillance of the contractor’s performance. This written report, and the completed quality assurance monitoring forms (QASP Attachment 2), will become part of the QA documentation. It will enable the government to demonstrate whether the contractor is meeting the stated objectives and/or performance standards.

1.1.5.3. Reviews and Resolution

The COR may require the contractor’s project manager, or a designated alternate, to meet with the QAM and other government personnel as deemed necessary to discuss performance evaluation. The QAM will define a frequency of in-depth reviews with the contractor. The agenda of the reviews may include:

· Monthly performance assessment data and trend analysis

· Issues and concerns of both parties

· Projected outlook for upcoming months and progress against expected trends, including a corrective action plan analysis

· Recommendations for improved efficiency and/or effectiveness

· Issues arising from the performance monitoring processes

The COR must coordinate and communicate with the contractor to resolve issues and concerns regarding marginal or unacceptable performance.

The COR and contractor should jointly formulate tactical and long-term courses of action. Decisions regarding changes to metrics, thresholds, or service levels should be clearly documented. Changes to service levels, procedures, and metrics will be incorporated as a contract modification.