A0132_QASP_LDP_01252018.pdf

QUALITY ASSURANCE SURVEILLANCE PLAN (QASP)

For

Leadership Development Program for the U.S. Army Corps of Engineers

Portland, Oregon

19 January 2018

TABLE OF CONTENTS

1 INTRODUCTION

1.1 Purpose

1.2 Performance Management Approach

1.3 Performance Management Strategy

2 GOVERNMENT’S ROLES AND RESPONSIBILITIES

2.1 The Contracting Officer

2.2 The Contracting Officer’s Representative

3 IDENTIFICATION OF REQUIRED PERFORMANCE STANDARDS/QUALITY LEVELS

4 METHODOLOGIES TO MONITOR PERFORMANCE

4.1 Surveillance Techniques

4.2 Customer Feedback

4.3 Acceptable Quality Levels

5 QUALITY ASSURANCE DOCUMENTATION

5.1 The Performance Management Feedback Loop

5.2 Monitoring Forms

6 ANALYSIS OF QUALITY ASSURANCE ASSESSMENT

6.1 Determining Performance

6.2 Reporting

6.3 Reviews and Resolution

6.4 Payment and Corrective Action

ATTACHMENT 1: PERFORMANCE REQUIREMENTS SUMMARY

QUALITY ASSURANCE SURVEILLANCE PLAN

(QASP)

INTRODUCTION

This performance-based Quality Assurance Surveillance Plan (QASP) sets forth the procedures and guidance that the Contract Officer Representative (COR), will use in evaluating the performance of the Contractor in accordance with the terms and conditions of the performance work statement (PWS). The QASP objective is to define Government procedures to be used to verify that appropriate performance and quality assurance methods are used in the management of this performance-based contract.

Purpose This QASP describes the mechanism for documenting noteworthy accomplishments or discrepancies for work performed by the Contractor. Information generated from the Government’s surveillance activities will directly support the overall evaluation of the Contractor’s performance on this contract. The intent is to ensure that the Contractor performs in accordance with performance metrics set forth in the contract documents, the Government receives the quality of services called for in the contract, and the Government only pays for the acceptable level of services received.

The QASP details how and when the Government will monitor, evaluate, and document Contractor performance on the PWS. The QASP is intended to define the role and responsibilities of participating Government representatives, define the key milestones/deliverables that will be assessed, define performance standards for key milestones/deliverables, as well as incentives /disincentives and describe the surveillance methodology that will be employed by the Government in assessing the Contractor’s performance.

Performance Management Approach The PWS structures the acquisition around “what” service or quality level is required, as opposed to “how” the Contractor should perform the work (i.e., results, not compliance). This QASP will define the performance management approach taken by the Portland District Office to monitor and manage the Contractor’s performance to ensure the expected outcomes or performance objectives communicated in the PWS are achieved. Performance management rests on developing a capability to review and analyze information generated through performance assessment. The ability to make decisions based on the analysis of performance data is the cornerstone of performance management; this analysis yields information that indicates whether expected outcomes for the project are being achieved by the Contractor.

Performance management represents a significant shift from the more traditional quality assurance (QA) concepts in several ways. Performance management focuses on assessing whether outcomes are being achieved and to what extent. This approach migrates away from scrutiny of compliance with the processes and practices used to achieve the outcome. A performance-based approach enables the Contractor to play a large role in how the work is performed, as long as the proposed processes are within the stated constraints. The only exceptions to process reviews are those required by law (federal, state, and local) and compelling business situations, such as safety and health. A “results” focus provides the Contractor flexibility to continuously improve and innovate over the course of the contract as long as the critical outcomes expected are being achieved and/or the desired performance levels are being met.

Performance Management Strategy The Government is responsible for administering the contract and ensuring that they receive accurate and appropriate deliverables. The Government is responsible for quality assurance of the Contractor’s work. The COR, will review and comment on written deliverables. It is not the responsibility of the Government to perform quality control for the Contractor or to execute work that the Contractor was contracted to perform.

1.3.2 The Government representative(s) will monitor performance and review performance reports furnished by the Contractor to determine how the Contractor is performing against communicated performance objectives. The Government will make determination regarding incentives based on performance measurement metric data and notify the Contractor of those decisions. The Contractor will be responsible for making required changes in processes and practices to ensure performance is managed effectively.

GOVERNMENT’S ROLES AND RESPONSIBILITIES

The Contracting Officer (CO) The Government Contracting Officer (CO) has overall responsibility for overseeing the Contractor’s performance. The CO is responsible for the day-to-day monitoring of the Contractor’s performance in the areas of contract compliance, and contract administration;

reviewing the COR’s assessment of the Contractor’s performance; and resolving all differences between the COR’s assessment and the Contractor’s assessment of performance.

It is the CO that assures the Contractor receives impartial, fair, and equitable treatment under the contract. The CO is ultimately responsible for the final determination of the adequacy of the Contractor’s performance.

The Contracting Officer’s Representative (COR) The Contracting Officer’s Representative (COR) is designated in writing by the CO to act as his or her authorized representative to assist in administering a contract. COR limitations are contained in the written appointment letter. The COR is responsible for technical administration of the project and ensures proper Government surveillance of the Contractor’s performance.

The COR is not empowered to make any contractual commitments or to authorize any contractual changes on the Government’s behalf. Any changes that the Contractor deems may affect contract price, terms, or conditions shall be referred to the CO for action. The COR will have the responsibility for monitoring, assessing, recording, and reporting on the technical performance of the Contractor on a day-to-day basis and will inform the CO of all performance issues for resolution. Government surveillance may occur in accordance with the Federal Acquisition Regulation Clause 52.246.4 – Inspection of Services.

IDENTIFICATION OF REQUIRED PERFORMANCE STANDARDS/QUALITY LEVELS FOR

KEY MILESTONES/DELIVERABLES

The Contractor’s performance will be evaluated by assessing the key milestones/deliverables described in the PWS and in Attachment 1, “Performance Requirements Summary.” If the Contractor meets the required service or performance level, it will be paid the fixed price amount agreed on in the contract. Failure to meet the required service or performance level will result in a deduction from the amount.

METHODOLOGIES TO MONITOR PERFORMANCE

Surveillance Techniques At the Government’s discretion, periodic inspections may be conducted to evaluate progress toward key milestones and deliverables. The COR may complete a periodic progress inspection if he/she believes that deficiencies exist that must be addressed prior to milestone/deliverable completion. In an effort to minimize the performance management burden, simplified surveillance methods shall be used by the Government to evaluate Contractor performance when appropriate. At the completion of all key milestones and deliverables, performance will be evaluated through 100% inspection.

The primary methods of surveillance conducted by the COR are:

• Random monitoring

• Periodic Inspection

• Formal Customer Satisfaction Survey

Customer Feedback The Contractor is expected to establish and maintain professional communication between its employees and customers. The primary objective of this communication is customer satisfaction. Customer satisfaction is the most significant external indicator of the success and effectiveness of all services provided and can be measured through customer complaints.

Performance management drives the Contractor to be customer focused through initially and internally addressing customer complaints and investigating the issues and/or problems but the customer always has the option to communicate complaints to the COR, as opposed to the Contractor.

Customer complaints, to be considered valid, must set forth clearly and in writing the detailed nature of the complaint, must be signed, and must be forwarded to the CO/COR. The CO/COR will accept those customer complaints and investigate.

Customer feedback may also be obtained either from the results of formal customer satisfaction surveys or from random customer complaints.

Acceptable Quality Levels The acceptable quality levels (AQLs) included in Attachment 1, Performance Requirements Summary Table, for Contractor performance are structured to allow the Contractor to manage how the work is performed while providing negative incentives for performance shortfalls. For certain critical activities, the desired performance level is established at 100 percent. Other levels of performance are keyed to the relative importance of the task to the overall mission performance.

QUALITY ASSURANCE DOCUMENTATION

The COR will use a performance evaluation form to record evaluation of the Contractor’s performance for each milestone and deliverable in accordance with the methodology described in the PWS and QASP. The COR must substantiate, through narratives in the form, all unacceptable ratings. Performance at the acceptable level is expected from the Contractor. At a minimum, the evaluation form will indicate actual and scheduled delivery times and number of reviews required to achieve the final product.

The COR will forward copies of all completed performance evaluation forms to the CO and Contractor within one week of performing the inspection. When a milestone/deliverable receives an overall unacceptable rating, the Contractor will explain, within 15 days, in writing to COR why performance was unacceptable, how performance will be returned to acceptable levels, and how recurrence of the problem will be prevented in the future.

The COR will review each performance evaluation form. When appropriate, the COR and/or CO may investigate further to determine if all the facts and circumstances surrounding the event were outlined on the form. The COR and/or CO will immediately discuss any unacceptable rating with the Contractor to assure that corrective action is promptly initiated.

The COR will maintain a complete QA file. This file will contain copies of all performance evaluation forms and any other related documentation. The COR will forward these records to the CO at completion or termination of the contract.

The Performance Management Feedback Loop The performance management feedback loop begins with the communication of expected outcomes. Performance standards are expressed in the PWS and are assessed using the performance monitoring techniques shown in Attachment 1.

Monitoring Forms The Government’s QA surveillance, accomplished by the COR will be reported using the Army Contracting Officer’s Representative Tool (CORT). This will document, monthly, the Government’s assessment of the Contractor’s performance under the contract to ensure that the required results of are being achieved.

ANALYSIS OF QUALITY ASSURANCE ASSESSMENT

Determining Performance The Government shall use the monitoring methods cited to determine whether the performance standards/service levels/AQLs have been met. If the Contractor has not met the minimum requirements, the Contractor may be asked to develop a corrective action plan to show how and by what date t intends to bring performance up to the required levels.

Reporting At the end of each month, the COR will prepare a written report using the CORT Tool summarizing the overall results of the quality assurance surveillance of the Contractor’s performance. This written report will become part of the QA documentation. It will enable the Government to demonstrate whether the contractor is meeting the stated objectives and/or performance standards, including cost/scheduling objectives.

Reviews and Resolution The COR must coordinate and communicate with the Contractor to resolve issues and concerns regarding marginal or unacceptable performance. The COR will define a frequency of in-depth reviews with the Contractor, including appropriate self-assessments by the Contractor; if the need arises. The Contractor will meet with COR as often as the COR deems necessary or per the Contractor’s request. The agenda of the reviews may include:

• Monthly performance assessment data and trend analysis

• Issues and concerns of both parties

• Projected outlook for upcoming months and progress against expected trends, including a corrective action plan analysis

• Recommendations for improved efficiency and/or effectiveness

• Issues arising from the performance monitoring processes

The COR and Contractor should jointly formulate tactical and long-term courses of action.

Decisions regarding changes to metrics, thresholds, or service levels should be clearly documented. Changes to service levels, procedures, and metrics will be incorporated as a contract modification by CO.

Payment and Corrective Action Full payment for a milestone/deliverable will be provided upon verification of overall acceptable performance. This verification will be recorded in a performance evaluation specifying overall Contractor performance as either acceptable or unacceptable for the milestone/deliverable.

If a milestone/deliverable is unacceptable corrective action must be noted and is required until the milestone/deliverable is deemed acceptable.

ATTACHMENT 1: PERFORMANCE REQUIREMENTS SUMMARY

PERFORMANCE REQUIREMENTS SUMMARY

Performance Objective

Performance Standard

Allowed Number of

Discrepancies

Acceptable Threshold and Method of Surveillance

Organization Capability and Experience 1.5 and Program Overview 1.2

Experience and credentials for delivering leadership and development in public service graduate level academic program curriculum providing three non-concurrent 3-credit master’s level courses over the duration of three consecutive academic quarters (minimum 192 hours class time), for mid-career adults

0% discrepancies allowed

100% inspection of the Contractor’s deliverables or services

Contract Academic Instructor(s)/Fac ulty and Staff 1.6

The Contractor’s Academic Instructor(s) are required to be graduate-level academic instructors possessing proven experience and credibility in the following areas:

1. Teaching leadership development and multi-level policy analysis in the public service and multi-governance arenas

2. Improving student’s speaking and writing skills

3. Facilitating graduate-level academic instruction in a hybrid model format with senior government executives where academic theory is integrated with practicable real-life examples/scenarios

0% discrepancies allowed

100% inspection of the Contractor’s deliverables or services

Program Overview 1.2 and Program Development 1.3

The Leadership Development Program is a jointly facilitated program by the Contractor and NWP.

The Contractor will be responsible for providing graduate level instruction and guidance to the cohort. The Contractor’s course curriculum will lead to graduate level academic degree credits transferable to an academic institution(s) should students wish to pursue further graduate education.

The Contractor and NWP will meet once monthly to discuss the overall Leadership Development Program and solidify the upcoming curriculum.

The Contractor shall shape the academic and governance theory portion of the program and integrate into the program specific Corps examples and leadership themes relevant to NWP. The first meeting is a two-hour Program Planning Meeting held mid-March to discuss the Year-at-a-Glance and set the specific dates for monthly planning meetings, classroom sessions, Fall Foresight, National Policy Process Trip to Washington, DC, Capstone/Graduation, and LDP Survey and Closeout. Monthly pre-plan meetings will start in June and run for the remainder of the program.

Each meeting will average 90 to 120 minutes in duration. Attendance will include the Contractor, Program Manager, and LDP Facilitators. During the pre-planning meetings, the Contractor will provide NWP participants with its planned academic curriculum for the next month, to include class topics and proposed materials to be used.

2 discrepancies allowed

93% or better inspection of the Contractor’s deliverables or services

Pre-Program Kickoff/ Orientation 1.4.4

In July, the Contractor and LDP Facilitators will present a program kick-off/orientation session at NWP Headquarters, which will be presented to the NWP cohort. The Contractor will be responsible for giving an administrative overview of the program elements.

At a minimum, the Contractor must present on its planned program for the following topics:

• Introductions of the Contractor and

Academic Instructors, Program Managers, and students

• Explanations of roles and responsibilities of the Contractor and faculty, Program Manager, LDP Facilitators, and students (both Contractor and LDP Facilitators)

• An administrative overview of the Program, with an LDP Program Overview document that outlines what will be covered throughout the academic year

• Calendar of Events and time commitments

• Dynamics of being a member of the cohort

• Mentoring overview, to include providing the Mentoring Resource Guidebook to the students (as further described below) (both Contractor and LDP Facilitators)

• Question and answer session (both Contractor and LDP Facilitators)

0% discrepancies

100% inspection of the Contractor’s deliverables or services

Classroom Sessions 1.4.5

The Contractor shall provide graduate-level academic classroom instruction for graduate level credit in the following three areas: Leadership Development (Individual leadership), Leading Public Organizations, and the U.S. National policy process (at a minimum, the Federal budget, Congress’s role, the multiple roles and functions of the varied legal constraints that operate on public organizations, the legal constraints and accountability requirements faced by the Corps, and how the larger external policy process affects the mission of the Corps, must be addressed).

Classroom instruction will result in the completion of three graduate-level classes that are at least three credits each for a total of at least nine credits. Each class session is not to exceed two days per month, should be held on Tuesday, Wednesday, and/or Thursday, and should not exceed eight hours per day.

2 discrepancies

93% or better inspection of the Contractor’s deliverables or services

Foresight and Washington D.C Trip 1.4.9 and 1.4.10

The Contractor shall use two full days over a three-day consecutive overnight period (Tuesday noon start with Thursday noon departure) for a Fall Foresight retreat to establish the organizational and thematic framework for the year-long program, with a focus on the results of the leadership assessment and teambuilding. The Foresight will be held at Camp Rilea in Warrenton, Oregon. The Contractor and NWP will be responsible for separate portions of the Fall Foresight, however the majority of the agenda will be the responsibility of the Contractor.

2 discrepancies allowed

93% or better inspection of the Contractor’s

The Contractor shall present on the results of the leadership assessment and include various teambuilding exercises in order build deeper relationships between the students, and leverage identified strengths and weaknesses of the students. The Contractor shall also present on the history of the Corps to the students.

The Washington D.C. trip is to take place in late January/early February of the LDP academic year.

The purpose of this policy-orientated component is to provide the cohort with an in-depth understanding of the larger political, legal, inter-organizational, and inter-jurisdictional environment within which the students undertake leadership initiatives. For all meetings, to include the White House or U.S. Capitol tour, the Contractor shall pre-arrange any necessary requirements to gain access to the site (security, etc), as necessary, to ensure all cohort members and NWP Facilitators can gain access to any tours or meeting locations.

Formal Graduation and Capstone Project 1.4.11 and 1.4.12

The Capstone Project case application will be selected by NWP and assigned to the students.

The Contractor’s Academic Instructor(s) is expected to provide the academic framework for the Capstone project, which must include a written and verbal presentation element. A formal LDP Capstone presentation event gives the cohort an opportunity to present their final Capstone project analysis and results to Senior Leaders, the Contractor’s Academic Instructor(s), and other invited guests. The Contractor will provide the facility/venue for this event and for a practice run-through one day prior to the capstone presentation.

The Capstone Presentation and Graduation event will occur from 0900-1400 hours, with appropriate breaks to include for lunch, and will include each student presenting their Capstone analysis/results, a presentation of Corps-provided plaques, and Contractor-provided certificates. NWP will provide photographic services for the event. The location for the event will need to accommodate up to 45 people which includes the cohort, Facilitators, LDP mentors, the District Corporate Board, and potential Northwest Division Command Staff. The venue must be accessible via public transportation from the Portland District Headquarters.

0% discrepancies

100% inspection of the Contractor’s deliverables or services

Graduation Preparation Day 1.4.13

The Contractor will secure the presentation venue for the Capstone Presentation one day prior to the presentation for a final preparation day that involves the Contractor, Facilitators, Program Manager, and the cohort. This preparation day shall be an all-day event lasting from 0800 to 1700, with appropriate breaks, to include for lunch. The Contractor shall provide appropriate feedback to the students, as appropriate, during the practice-run through of the capstone presentation.

0% discrepancies allowed

100% inspection of the Contractor’s deliverables or services

Coaching Sessions and

The Contractor will provide two specialized rounds of one-on-one coaching sessions, one in the fall

2 discrepancies allowed

95% or better inspection of the

Leadership Assessment

1.4.6 and 1.4.7 and one in the spring, with each student. The Contractor will be responsible for providing the location for the fall coaching session, which must be held in the Portland Metropolitan area and easily accessible via public transit. NWP will provide coaching session meeting space for the spring sessions.

The one-on-one coaching sessions are expected to last for approximately one hour. In each session, the Contractor will discuss leadership development related issues in depth on a face-to-face basis.

The Contractor will cover the individual student’s individual leadership assessment results, review the student’s Leadership Development Portfolio, discuss mentoring relationships, and assist students in applying new leadership development concepts and skills in their work and life settings.

During the course of the program, the contractor shall administer at least one individual leadership assessment, such as the Meyers-Briggs Type Indicator or Strengthfinder, to assess each student’s strengths and/or weaknesses relating to each individual’s emotional intelligence and leadership skills. The Contractor must explain the assessment during one of the initial classroom sessions and require the cohort to complete the assessment prior to the Fall Foresight. The Contractor shall provide the feedback from the assessment at the Fall Foresight. The Contractor will also discuss the results of the assessment with each student as part of the Coaching Sessions.

Contractor’s deliverables or services

Miscellaneous Support – Mentoring 1.4.8

The Contractor will provide the framework for a Mentor Program as described below to pave a path for the mentors and protégés. The LDP Facilitators will match an experienced Corps manager (mentor) with each student (protégé) who will work with them throughout the year of LDP. The framework provided by the Contractor to the mentors and protégés shall include the following:

• Resource Guide for Effective Mentoring, which at a minimum covers the following:

The purpose of mentoring as part of leadership development; roles and responsibilities in mentoring; suggested mentoring activities; tips for successful mentoring; references and recommended reading related to mentoring; and a checklist for developing the mentor relationship. The Contractor shall provide the Resource Guide to the LDP Program Manager during the first monthly planning meeting, which the NWP Facilitators will provide to the mentors.

• Three one-hour Mentor Training Sessions, for the mentors and protégés, provided during the lunch-hour of three classroom training sessions during the program year.

2 discrepancies allowed

95% or better inspection of the Contractor’s

The purpose of the Mentor Training Sessions are to provide training on affective mentoring skills (particularly in a leadership development context), roles and responsibilities in mentoring, and further develop the mentor/protégé relationship.

As part of the Mentor Training Sessions, the Contractor shall explain the Resource Guide for Effective Mentoring, explain the expectations of the mentors and protégés, and include a number of mentor/protégé training activities that will reinforce leadership skill development and the mentor relationship.

In order to ensure that both the mentors and protégés are available to attend and that the classroom sessions are not disrupted, these trainings must occur during the lunch hour of three of the classroom sessions, and the Contractor shall be responsible for providing a meal for each mentor and protégé (for a maximum total of 32 Corps’ employees). The meal must include non-alcoholic beverage options, a vegetarian food option, and must not exceed the meal rate for each individual attending, as specified in the Joint Travel Regulation in effect at the time of each meal.

NWP will provide each protégé with a Protégé Development Plan, to be completed by the protégé with the mentor. The Contractor will not be responsible for the Plan, but is expected to check in with the mentors and protégés on progress.

Closeout Report 1.4.15

The Contractor shall provide the Program Manager with a reflective report evaluating the effectiveness of LDP components, activities, and recommendations for changes and improvements.

The Contractor shall provide the report to NWP within 30 days following the Capstone Presentation. The Contractor is encouraged to be innovative in developing an effective evaluation system for the program that provides relevant information regarding the strengths, weaknesses, and recommendations to improve the program.

0% discrepancies allowed

100% inspection of the Contractor’s

Payment and Corrective Action

ATTACHMENT 1: PERFORMANCE REQUIREMENTS SUMMARY