M67854-16-R-4806_Att_4_DRRS_MC_QASP.docx

Product Manager, Total Force Information Technology Systems

Defense Readiness Reporting System – Marine Corps (DRRS-MC)

Quality Assurance Surveillance Plan (QASP)

April 2016

TABLE OF CONTENTS

QUALITY ASSURANCE SURVEILLANCE PLAN

(QASP)

INTRODUCTION

This Quality Assurance Surveillance Plan (QASP) is in support of the Product Manager (PdM) Total Force Information Technology System (TFITS) Defense Readiness Reporting System – Marine Corps (DRRS-MC) Systems Integrator (SI) initiative and pursuant to the requirements listed in the Indefinite Delivery/Indefinite Quantity (ID/IQ) and associated task order Performance Work Statements (PWS). This plan sets forth the procedures and guidelines PdM TFITS will use in ensuring the required performance standards or services levels are achieved by the DRRS-MC SI.

Purpose The purpose of the QASP is to describe the systematic methods used to monitor performance and to identify the required documentation and the resources to be employed. The QASP provides a means for evaluating whether the DRRS-MC SI is meeting the performance standards/quality levels identified in the PWSs and the DRRS-MC SI Quality Assurance Program Plan (QAPP).

This QASP defines the roles and responsibilities of all members of the Integrated Project Team (IPT), identifies where and how the performance objectives are identified, defines the methodologies used to monitor and evaluate the DRRS-MC SI performance, describes quality assurance documentation requirements, and describes the analysis of quality assurance monitoring results.

Performance Management Approach The ID/IQ structures the acquisition around “what” service or quality level is required. This QASP will define the performance management approach taken by PdM TFITS to monitor and manage the DRRS-MC SI performance to ensure the expected outcomes or performance objectives communicated in the PWSs are achieved. The QAPP describes the performance management approach to review and analyze information generated through performance assessment. The ability to make decisions based on the analysis of performance data is the cornerstone of performance management; this analysis yields information that indicates whether expected outcomes for the project are being achieved by the DRRS-MC SI.

Performance management focuses on assessing whether outcomes are being achieved and to what extent. A performance-based approach enables the DRRS-MC SI to play a large role in how the work is performed, as long as the proposed processes are within the stated constraints. The only exceptions to process reviews are those required by law (federal, state, and local) and compelling business situations, such as safety and health. A “results” focus provides the DRRS-MC SI flexibility to continuously improve and innovate over the course of the contract as long as the critical outcomes expected are being achieved and the desired performance levels are being met.

Performance Management Strategy The DRRS-MC SI is responsible for the quality of all work performed. The DRRS-MC SI measures that quality through the DRRS-MC SI’s own quality assurance and control program. The DRRS-MC SI QAPP will set forth the staffing and procedures for self-inspecting the quality, timeliness, responsiveness, customer satisfaction, and other performance requirements. The DRRS-MC SI will develop and implement a performance management system with processes to assess and report its performance to PdM TFITS. This QASP describes how PdM TFITS intends to evaluate DRRS-MC SI quality and performance.

PdM TFITS will monitor performance and review performance reports furnished by the DRRS-MC SI in the Monthly Status Report (MSR) to determine how the DRRS-MC SI is performing against communicated performance objectives. The Government will make determinations regarding future task order actions based on performance measurement metric data and notify the DRRS-MC SI of those decisions. The DRRS-MC SI will be responsible for making required changes in processes and practices to ensure performance is managed effectively.

ROLES AND RESPONSIBILITIES

PdM TFITS PdM TFITS is responsible for the overall performance evaluation and assessment of the DRRS-MC SI performance.

The Contracting Officer The Contracting Officer (KO) is responsible for monitoring contract compliance, contract administration, and cost control. The KO is responsible for resolving any differences between the observations documented by the Performance Evaluation Review Team (PERT) chaired by the Contracting Officer’s Representative (COR).

The Contracting Officer’s Representative The COR is designated in writing by the KO to act as his or her authorized representative to assist in administering a contract. The COR is responsible for technical administration of the project and ensures proper Government surveillance of the DRRS-MC SIs performance. The COR will chair the PERT and be supported by Government Subject Matter Experts (SMEs). The number of SMEs serving as technical inspectors on the PERT depends on the complexity of the services measured, as well as the DRRS-MC SI’s performance. The COR will have the responsibility for establishing and completing Quality Assurance (QA) monitoring via TFITS SharePoint used to document the inspection and evaluation of the DRRS-MC SI’s work performance.

Tier 2 Competency Leads Tier 2 Leads are Senior Competency Managers responsible for multiple systems as well as being well-versed in the project management and acquisition process. During the evaluation and assessment of the DRRS-MC SI’s performance, Tier 2 Leads will represent and discuss concerns relating to the systems they are responsible for managing to the PERT.

Tier 3 Competency Managers The Tier 3 Competency Managers will review the MSR performance report against the ID/IQ performance standards and Acceptable Quality Level (AQL)s and provide their endorsement via the TFITS SharePoint list to the Tier 2 Lead and COR. As needed, the Tier 3 Project Manager (PjM) will brief the PERT to address specific performance issues.

IDENTIFICATION OF REQUIRED PERFORMANCE STANDARDS/QUALITY LEVELS

System performance requirements are provided in the ID/IQ and task order PWSs. The PWSs also contain the minimum performance standards AQLs that the DRRS-MC SI will need to achieve for acceptable accomplishment of the major task requirements in the ID/IQ and associated task orders.

METHODOLOGIES TO MONITOR PERFORMANCE

Active Surveillance Techniques In an effort to minimize the performance management burden, the Government will use surveillance methods to evaluate DRRS-MC SI performance. The primary methods of surveillance are:

· Deliverable review and evaluation: Each Deliverable will be reviewed for content, accuracy, and compliance with contractual requirements. Deliverables are typically accepted, rejected or conditionally accepted by the Government within 10 working days; however, the Program Management Office (PMO) will specify the number of working days , as it should reflect the size and scope of the deliverable (e.g., a Risk Management Plan may take more time then review of a Technical Data Package).

· Rejected Deliverables must be corrected and resubmitted within 10 working days. All Government provided comments will be adjudicated. The Government will review the Deliverable as if it was submitted for the first time. The resubmitted Deliverable can be accepted, rejected or conditionally accepted.

· Conditionally accepted Deliverables will be corrected and resubmitted. The Government will review to ensure comments were adjudicated.

· Performance Evaluation Reviews: The COR will chair Performance Evaluation Reviews (PERs) to evaluate surveillance results. PERs will occur no less than quarterly. At the COR’s discretion, the PER frequency may increase for reasons such as: performance problems resolution is not timely; pending critical milestones require closer performance scrutiny; etc. Upon completion of a PER, a Memorandum for the Record (MFR) and minutes will be signed by the COR and stored on the TFITS SharePoint portal. PER results will be entered into the Contract Performance Assessment Reporting System (CPARS).

Passive Surveillance Techniques The primary objective is the ability to gauge the DRRS-MC SI influence on customer satisfaction ratings. Customer satisfaction is typically in the area of system performance, system usability, system support materials, the number of helpdesk calls, the number of Trouble Tickets and the quality of Helpdesk personnel. Possible tools to support customer satisfaction include:

User Satisfaction Surveys Customer satisfaction feedback may be obtained from the results of customer satisfaction surveys.

User Feedback Customer satisfaction will be gleamed from User feedback provided directly to the Helpdesk or to the PMO.

Performance Standards and Acceptable Quality Levels Performance standards and AQLs are very important in that they identify minimally acceptable performance levels and map directly to the requirements in the PWS. Performance standards and AQLs must be measurable and represent a best value determination between risk and cost.

Table 1: Acceptable Quality Levels

Weekly TIM

MSR

Weekly TIM

MSR

Deliverable Transmittals

Weekly TIIM

MSR

Government review of User Surveys

MSR

Minimize unplanned downtime.

Any down-time on weekends or attributed to the Government will not count as unplanned down-time.

Planned down-time that exceeds the agreed upon window will count as unplanned down-time Help desk ticket logs.

FRACAS

MSR

Government defined operational availability is 99.6% based on an operational time of 24/7/365

3.2.4

FRACAS

MSR

SETR results

GAT Scorecard

QUALITY ASSURANCE DOCUMENTATION

The performance management feedback loop begins with the communication of expected outcomes. Performance standards are expressed in the ID/IQ and assessed by the Tier 3 IPT in the TFITS SharePoint Portal as an endorsement to the MSR. Other Deliverables are reviewed and assessed by the Competency Leads. These reviews occur before the PERT meets in order for the Government to accept Deliverables via the TFITS SharePoint Portal.

ANALYSIS OF QUALITY ASSURANCE ASSESSMENT

Determining Performance The Government PERT will determine whether the performance standards, service levels, AQLs have been met via the MFR, based on the endorsement of the MSR by the Tier 3 PjM. If the DRRS-MC SI has not met the minimum requirements, the DRRS-MC SI may be asked to develop a corrective action plan to show how and by what date it intends to bring performance up to the required levels.

Reporting At least quarterly, the PERT will meet and review the DRRS-MC SI performance. The MSRs and associated Tier 3 Competency Manager’s endorsements, Deliverable assessment, and user feedback from each project are planned to be the primary input to the PERT review. After the PERT review, the COR will prepare a MFR for the Contracting Officer summarizing the overall results of the PERT review on the quality assurance of the DRRS-MC SI’s performance. This written report will become part of the QA documentation. It will enable the Government to demonstrate whether the DRRS-MC SI is meeting the stated performance standards, including cost, technical, scheduling standards and requirements.

Reviews and Resolution The PERT includes the COR as chair along with the DRRS-MC SI Project Manager, or a designated alternate, to discuss performance evaluation at least quarterly. The agenda of the reviews may include:

· Monthly performance assessment data and trend analysis

· Issues and concerns of both parties

· Projected outlook for upcoming months and progress against expected trends, including a corrective action plan analysis

· Recommendations for improved efficiency and effectiveness

The KO will send the MFR to the DRRS-MC SI and work with the COR and DRRS-MC SI to resolve issues and concerns regarding marginal or unacceptable performance.

The COR and DRRS-MC SI should jointly formulate tactical and long-term courses of action. Decisions regarding changes to metrics, thresholds, or service levels should be clearly documented. Changes to service levels, procedures, and metrics may be incorporated after PERT approval as a contract modification at the convenience of the COR and KO.

Contractor Performance Assessment Reporting System During the course of the contract and at the end of the contract, the COR will input results of its quality assessment into the CPARS.