DRAFT_M67854-16-R-4805_QASP.DOCX

Marine Corps Training Information Management System (MCTIMS) Quality Assurance Surveillance Plan March 2016

Product Manager, Total Force IT Systems

Marine Corps Training Information Management System (MCTIMS)

Quality Assurance Surveillance Plan (QASP)

March 2016 ii

TABLE OF CONTENTS

QUALITY ASSURANCE SURVEILLANCE PLAN

(QASP)

INTRODUCTION

This quality assurance surveillance plan (QASP) is in support of the Product Manager’s (PdM’s) Total Force Information Technology Systems (TFITS) MCTIMS Systems Integrator initiative and pursuant to the requirements listed in the Indefinite Delivery/Indefinite Quantity (ID/IQ) and associated task order Performance Work Statement (PWS). This plan sets forth the procedures and guidelines PdM TFITS will use in ensuring the required performance standards or services levels are achieved by the MCTIMS Systems Integrator (SI).

Purpose The purpose of the QASP is to describe the systematic methods used to monitor performance and to identify the required documentation and the resources to be employed. The QASP provides a means for evaluating whether the MCTIMS SI is meeting the performance standards and quality levels identified in the PWS and the MCTIMS SI Quality Assurance Program Plan (QAPP).

This QASP defines the roles and responsibilities of all members of the MCTIMS Integrated Project Team (IPT), identifies where and how the performance objectives are identified, defines the methodologies used to monitor and evaluate the MCTIMS’s SI performance, describes quality assurance documentation requirements, and describes the analysis of quality assurance monitoring results.

Performance Management Approach The ID/IQ structures the acquisition around “what” service or quality level is required. This QASP will define the performance management approach taken by PdM TFITS to monitor and manage the MCTIMS’s SI performance to ensure the expected outcomes or performance objectives communicated in the ID/IQ and the Task Order PWS are achieved. The MCTIMS SI’s QAPP describes the performance management approach to review and analyze information generated through performance assessment. The ability to make decisions based on the analysis of performance data is the cornerstone of performance management. This analysis yields information that indicates whether expected outcomes for the project are being achieved by the MCTIMS SI.

Performance management focuses on assessing whether outcomes are being achieved and to what extent. A performance-based approach enables the MCTIMS SI to play a large role in how the work is performed as long as the proposed processes are within the stated constraints. The only exceptions to process reviews are those required by law (federal, state, and local) and compelling business situations such as safety and health. A “results” focus provides the MCTIMS SI with the flexibility to continuously improve and innovate over the course of the contract as long as the expected critical outcomes are being achieved and the desired performance levels are being met.

Performance Management Strategy The MCTIMS SI is responsible for the quality of all work performed. The MCTIMS SI measures that quality through the MCTIMS SI’s own quality assurance and control program. The MCTIMS SI QAPP will set forth the staffing and procedures for self-inspecting the quality, timeliness, responsiveness, customer satisfaction and other performance requirements. The MCTIMS SI will develop and implement a performance management system with processes to assess and report its performance to PdM TFITS. This QASP describes how PdM TFITS intends to evaluate MCTIMS SI quality and performance.

PdM TFITS will monitor performance and review performance reports furnished by the MCTIMS SI in the Monthly Status Report (MSR) to determine how the MCTIMS SI is performing against communicated performance objectives. The government will make determinations regarding future ordering period options based on performance measurement metric data and notify the MCTIMS SI of those decisions. The MCTIMS SI will be responsible for making required changes in processes and practices to ensure performance is managed effectively.

ROLES AND RESPONSIBILITIES

PdM TFITS Product Manager (PdM) TFITS is responsible for the overall performance evaluation and assessment of the MCTIMS SI’s performance.

The Contracting Officer The Contracting Officer (KO) is responsible for monitoring contract compliance, contract administration, and cost control.

The Contracting Officer’s Representative The COR is designated in writing by the KO to act as his/her authorized representative to assist in administering a contract. The COR is responsible for technical administration of the project and ensures proper government surveillance of the MCTIMS SIs performance. The COR will conduct a periodic performance evaluation review (PER) and be supported by Government Subject Matter Experts (SMEs). The COR will have the responsibility for establishing and completing Quality Assurance (QA) monitoring via TFITS SharePoint used to document the inspection and evaluation of the MCTIMS SI’s work performance.

Tier 2 Competency Leads Tier 2 Leads are Senior Competency Managers responsible for multiple systems as well as being well-versed in the project management and acquisition process. During the evaluation and assessment of the SI’s performance, Tier 2 Leads will provide input to the PER.

Tier 3 Competency Managers The Tier 3 Competency Managers will review the performance metrics reported in the MSR against the PWS performance standards and Acceptable Quality Levels (AQLs). As needed, the Tier 3 PjM will brief the Tier 2 to address specific performance issues.

MCTIMS SI

The MCTIMS SI is responsible for managing the SI’s performance in accordance with the QAPP and the QASP, providing the MSR, and ensuring that performance and deliverables meet the standards established in the PWS and AQLs. The MCTIMS SI is also responsible for providing a representative to attend PERs.

IDENTIFICATION OF REQUIRED PERFORMANCE STANDARDS/QUALITY LEVELS

System performance requirements are provided in the Task Order PWS. The PWS contains the minimum performance standards (the AQLs) that the MCTIMS SI will need to achieve for acceptable accomplishment of the major task requirements in the ID/IQ and associated task orders.

METHODOLOGIES TO MONITOR PERFORMANCE

Active Surveillance Techniques In an effort to minimize the performance management burden, the government will use simplified surveillance methods to evaluate the MCTIMS SIs performance. The primary methods of surveillance are:

· Deliverable review and evaluation: Each Deliverable will be reviewed for content, accuracy and compliance with contractual requirements. Deliverables are typically accepted, rejected or conditionally accepted by the Government within 10 working days. However, the Program Management Office (PMO) will specify the number of working days as it should reflect the size and scope of the deliverable (e.g., a Risk Management Plan may take more time than a review of a Technical Data Package).

· Rejected deliverables must be corrected and resubmitted within 10 working days. All Government provided comments will be adjudicated. The Government will review the Deliverable as if it was submitted for the first time. The resubmitted Deliverable can be accepted, rejected or conditionally accepted.

· Conditionally accepted Deliverables will be corrected and resubmitted. The Government will review to ensure comments were adjudicated.

· Performance Evaluation Reviews: The COR will chair PERs to evaluate surveillance results. PERs will occur on a quarterly basis. At the COR’s discretion, the PER frequency may increase for reasons such as: performance problems, the resolution is not completed in a timely manner, closer performance scrutiny of pending critical milestones, etc. Upon completion of a PER, a Memorandum for the Record (MFR) and minutes will be signed by the COR and stored on the TFITS SharePoint portal and COR document file. PER results will be entered into the Contract Performance Assessment Reporting System (CPARS).

Passive Surveillance Techniques The primary objective is to gauge the MCTIMS SI’s influence on customer satisfaction ratings. Customer satisfaction is typically the user’s input on how well the SI performs in the areas of system performance, system usability, system support, materials, and the number of helpdesk calls, the number of trouble tickets and the quality of Helpdesk personnel. Possible tools to support customer satisfaction include:

· User Satisfaction Surveys: The Government may conduct user satisfaction surveys to solicit Customer satisfaction feedback.

· User Feedback: Customer satisfaction will be gleamed from User feedback provided directly to the Helpdesk or to the PMO.

QUALITY ASSURANCE DOCUMENTATION

The performance management feedback loop begins with the communication of expected outcomes. Performance standards are expressed in the ID/IQ and assessed by the PER. Other deliverables are reviewed and assessed by the Competency Leads. These reviews occur regularly in order for the Government to accept deliverables via the TFITS SharePoint Portal.

ANALYSIS OF QUALITY ASSURANCE ASSESSMENT

Determining Performance The Government PER will determine whether the performance standards, service levels and AQLs have been met via the MFR, based on the endorsement of the MSR by the Tier 3 PjM. If the MCTIMS SI has not met the minimum requirements, the MCTIMS SI may be asked to develop a corrective action plan to show how and by what date it intends to bring performance up to the required levels. The COR will assess performance as either exceptional, very good, satisfactory, marginal, or unsatisfactory.

Reporting At least quarterly, the PER will meet and review the MCTIMS SI’s performance. The MSRs and associated Tier 3 Competency Manager’s endorsements, deliverable assessment, and User feedback are planned to be the primary input to the PER. After the PER, the COR will prepare a MFR for the Contracting Officer summarizing the overall results of the PER on the quality assurance of the MCTIMS SI’s performance. This written report will become part of the QA documentation. It will enable the Government to demonstrate whether the MCTIMS SI is meeting the stated performance standards, including cost, technical, scheduling standards and requirements.

Reviews and Resolution The PER agenda may include:

· Monthly performance assessment data and trend analysis

· Issues and concerns of both parties

· Projected outlook for upcoming months and progress against expected trends, including a corrective action plan analysis

· Recommendations for improved efficiency and effectiveness

The Contracting Officer will send the MFR to the MCTIMS SI and work with the COR and MCTIMS SI to resolve issues and concerns regarding marginal or unacceptable performance.

The COR and MCTIMS SI should jointly formulate tactical and long-term courses of action. Decisions regarding changes to metrics, thresholds or service levels should be clearly documented. Changes to service levels, procedures and metrics may be incorporated after PER approval as a contract modification at the convenience of the COR and Contracting Officer.

Contractor Performance Assessment Reporting System (CPARS) During the course of the contract and at the end of the contract, the COR will input results of its quality assessment into the CPARS.

D-6