Attachment B - QASP.docx

11/17/2011 11/17/2011

Department of Homeland Security Federal Emergency Management Agency

DR-1981 North Dakota

Maintenance and Deactivation Requirement Quality Assurance Surveillance Plan (QASP)

December 2011

Table of Contents

I. INTRODUCTION…………………………………………………………………..2

II. SCOPE………………………………………………………………………………3

III. PERFORMANCE CRITERIA……………………………………………………...3

A. Timeliness (SSPA#1)………………………………………………………..3 B. Customer Satisfaction (SSPA#2)……………………………………………6 C. Quality (SSPA#3)……………………………………………………………7 D. Overall Performance Score…………………………………………………..7

IV. PERFORMANCE REQUIREMENTS SUMMARY………………………………..6

EXHIBITS:

1. Customer Satisfaction Surveys (for Routine and Emergency Maintenance {1a} and Deactivations {1b}

2. Maintenance Quality Survey

3. Deactivation Quality Survey

I. INTRODUCTION

Contractor shall perform according to the requirements in the Performance Work Statement (PWS); Section C. Contractor activity will be monitored by the Contacting Officer’s Technical Representative (COTR), with assistance from the Technical Monitor and/or other designated staff to ensure full compliance with contract terms and conditions. The contractor’s performance will be evaluated on a quarterly basis. This Quality Assurance Surveillance Plan (QASP) outlines the procedures that will be used in assessing and measuring the quality of contractor performance. This Quality Assurance Surveillance Plan (QASP) has been developed to support the requirements set forth in the Performance Work Statement (PWS) for Maintenance and Deactivation Services for DR-1981 North Dakota. Due to the criticality and the nature of this contract in providing assistance to victims of disasters, FEMA must monitor and evaluate performance to ensure that quality services, supplies, and work performed are rendered in an expedited manner with minimal setbacks. The role of the government is to perform quality assurance to ensure that contract and task order requirements are appropriately achieved. The purpose of this QASP is to document procedures, guidelines, and evaluation criteria the Government will use to monitor, evaluate, and ensure the contractor provides appropriate technical performance and quality service in a timely manner consistent with the objectives, mission, and performance requirements in the Performance Work Statement via the most cost efficient means available.

A. The COTR is responsible for monitoring Contractor performance on a task order basis. The COTR will provide overall technical management, and monitor contractor performance. The Technical Monitor (or other designated staff member) will document Contractor’s progress and performance, and inform the COTR immediately regarding any potential problems and recommended courses of action. Problems of a contractual nature (as opposed to technical) must be reported to the Contracting Officer immediately.

B. The method of surveillance will involve monitoring Work Order records, Property Transfer Reports (PTR), actual receipt of repairs, call logs, contractor maintenance/deactivation logs and phone or in-person interviews with customers. The contractor shall provide reports to COTR on maintenance performed and deactivations as specified in the PWS.

C. Selected Service Performance Areas (SSPAs) are scored on the basis of achievement under individual task orders (See para IV. Below). There are no financial incentives or disincentives under this contract and QASP.

D. The Contractor, Technical Monitor (or other designated staff member), and COTR will partner throughout the life of each task order and discuss the quality of performance during weekly meetings (conference calls or face-to-face meetings), or more frequently if necessary. Performance results will be documented by the Technical Monitor (or other designated staff member). The Contractor and COTR (or other designated staff member) will affix their signatures to indicate agreement.

E. If the Contractor and COTR are unable to reach agreement on the evaluation results, then the matter shall be elevated to the Contracting Officer. The Contracting Officer will consider any discrepancies between the Contractor and the COTR, and issue a final decision. There is no provision for the Contractor to appeal the Contracting Officer’s final decision. The original evaluation shall be forwarded to the Contracting Officer for retention in the Official Contract File, and one copy each shall be provided to the Contractor and the COTR.

Roles and Responsibilities Contracting Officer (CO) The CO ensures performance of all requirements of the Basic Contract and Task Orders (TO) to ensure compliance with the terms and conditions of the contract and/or TO. The CO safeguards the interests of the government in the contractual relationship. The CO will receive all discrepancy reports and forward them to executive management of the contractor for action upon each concurrence of a noted discrepancy. The CO may monitor, perform surveillance, and evaluate the contractor’s performance at any time during the life of the contract and/or TO.

Contracting Officer’s Technical Representative (COTR) (Task Order/disaster basis).

The COTR will serve as a designated government representative for monitoring, surveillance, evaluating and performing quality assurance activities for the task order. The COTR ensures the contractor quantity, cost effectiveness, timeliness, and performance is monitored, assessed, recorded, and documented. The COTR responsibilities are contained in the written Letter of Designation from the CO.

Technical Monitor (TM)

II. SCOPE

A. The Selected Service Performance Areas (SSPAs) include the following categories:

1. Timeliness

2. Customer Satisfaction

3. Quality, III. PERFORMANCE CRITERIA

A. Timeliness (SSPA #1)

1. Timeliness will be evaluated for Routine Maintenance, Emergency Maintenance and Deactivation.

a. Pursuant to the Performance Work Statement (PWS) Exhibit 1 and Exhibit 2 , the contractor shall:

i. Respond and initiate emergency repairs within six (6) hours of receipt of the emergency request with the tools, equipment and supplies, necessary to make repairs.

ii. Pursuant to PWS, the contractor shall initiate and complete routine maintenance repairs within forty-eight (48) hours after receipt of the request or monthly scheduled maintenance whichever comes first. Contractor shall complete routine maintenance items within seventy-two (72) hours of initial visit.

iii. Pursuant to PWS, the contractor shall imitate and complete routine deactivation requests for manufactured homes (MH) and Park Models (PM) within seven (7) calendar days after receipt of the work order.

iv. MH units must have deactivation initiated on an emergency basis within six (6) hours of receipt of Work Order. The work must be completed the same day as the Work Order is issued.

b. Once per quarter, following the start of the performance period, the COTR, Technical Monitor, or other designated staff member will select for evaluation the work orders issued for the month prior in each of the following three areas of performance: (1) emergency maintenance, (2) routine maintenance and (3) deactivations.

c. The COTR, Technical Monitor, or other designated staff member will receive from the contractor a single Excel spreadsheet of all emergency maintenance, routine maintenance and deactivation work orders issued during the prior month. Each row of data shall include, at a minimum:

i. The type of work order issued (emergency maintenance, preventive, routine maintenance and deactivation);

ii. The occupant’s name;

iii. The occupant’s billing address

iv. The occupant’s contact phone number;

v. The occupant’s Mobile Home bar code;

vi. A description of the work performed;

vii. The date and time the work was requested and completed;

viii. And the status of the work order (if work order is incomplete, state why, what needs to be done, what has been done and who needs to complete the action.)

d. A minimum of 5% of the total number of work orders should be randomly sampled every month. If fewer than twenty five (25) work orders have been issued in a given month for any one of the three areas, then 100% inspection will be performed instead of random sampling.

e. An Excel spreadsheet of the Customer Satisfaction and Quality Survey results of each occupant randomly selected will be maintained on a monthly basis.

f. The COTR, Technical Monitor, or other designated staff member will randomly select at least a 5% sample of work orders and serviced customers for call-outs. IF fewer than twenty five (25) work orders have been issued in a given month, then 100% call-outs will be performed instead of random sampling. Using the Customer Satisfaction Survey, Exhibits 1a and 1b, designated staff will contact the customers, ask them to answer the questions on the survey, and thus determine those customer’s levels of customer satisfaction. Scores will be calculated for each survey performed and recorded on the Excel spreadsheet.

g. The COTR will administer performance surveillance on an on-going, quarterly basis, in order to more frequently update contractors regarding their performance quality levels.

h. For each randomly selected work order, determine whether the emergency maintenance, routine maintenance, or deactivation was accomplished within the timeframes in III.A.1.a. Maintain a tally of those randomly selected work orders performed in a timely fashion versus those work orders performed in a tardy fashion.

2. This Timeliness performance will be factored within two other percentage sub-scores, one for Customer Satisfaction and one for Quality, to arrive at one Overall Performance percentage score.

B. Customer Satisfaction (SSPA #2)

1. Customer Satisfaction is evaluated for Routine Maintenance, Emergency Maintenance, and Deactivations.

a. Pursuant to the Performance Work Statement, the contractor shall provide maintenance and deactivation services and activities in a professional manner, incorporating valuable customer service, performance and quality assurance measures into the task, to ensure that disaster impacted individuals and households occupying the temporary housing units are treated and served with courteous and timely services.

b. During each calendar month, the COTR, Technical Monitor, or other designated staff member will randomly select a statistically significant sample of the work orders issued in each of the following three areas of performance: (1) emergency maintenance (2) routine maintenance, and (3) deactivation. The samples to be used will be the same sample identified and collated in Steps III.A.1.b.-III.A.2. listed above.

c. The COTR, Technical Monitor, or other designated staff members will designate the randomly selected sample of work orders and the serviced customers, to be called. Using the Customer Satisfaction Survey, Exhibits 1a and 1b, designated staff will contact the customers, ask them to answer the questions on the survey, and thus determine those customer’s levels of customer satisfaction. Scores will be calculated for each survey performed and put on an Excel spreadsheet.

2. The Customer Satisfaction sub-score will be calculated as the average score for all surveys performed.

Example: If 10 surveys are performed and the contractor received the following customer satisfaction scores:

C. Quality (SSPA#3)

1. Quality will be evaluated for Routine/Emergency Maintenance, Deactivation and Group Site Maintenance.

a. The contractor shall ensure that work is performed pursuant to the standards listed in the Statement of Work.

b. During each quarter, the COTR, Technical Monitor, and other designated staff member will select a sample of the work orders issued in each of the following three areas of performance: (1) emergency maintenance (2) routine maintenance, and (3) deactivation. The sample to be used will be the same sample identified and collated in Steps III.A.1.a-III.A.2 listed above.

c. The COTR, Technical Monitor, or other designated staff members will utilize Exhibit 2, Maintenance Quality Survey, to evaluate Emergency and Routine Maintenance, and Exhibit 3, Deactivation Quality Survey, to evaluate Deactivations. Scores will be calculated for each survey performed for each of the randomly selected work orders.

2. The Quality sub-score will be calculated as the average score for all surveys performed. The formula is the same as used in above noted III.B.2. This Quality percentage sub-score will be averaged as one of the two percentage sub-scores including sub-score for Customer Satisfaction, to arrive at one Overall Performance percentage score.

D. Overall Performance Score

1. The Customer Satisfaction, and Quality (with Timeliness factored in) sub-scores will be averaged to obtain a single Overall Performance score.

2. Rating Scores:

Overall Performance score of 69.99% or lower: Unsatisfactory

Overall Performance score of 70% -79.99%: Marginal

Overall Performance score of 80%-89.99%: Satisfactory

Overall Performance score 90%-94.99%: Excellent

Overall Performance score of 95% or higher: Outstanding

3. There are no financial incentives or disincentives under this contract or QASP.

IV. PERFORMANCE REQUIREMENTS SUMMARY

A. The Performance Requirements Summary (PRS), which follows, summarizes (in chart form):

1. The Selected Service Performance Areas (SSPAs) critical to successful performance,

2. The minimum standard,

3. The paragraph cited in the Performance Work Statement where a description of the exact requirement can be found,

4. The method of surveillance and,

5. The scoring and measurement system to be applied to each SSPA.

PERFORMANCE REQUIREMENTS SUMMARY (PRS)

SSPA#1 Timeliness

a. Emergency Maintenance

a. Initiate and complete 100% emergency repair requests within six (6) hours of receipt of emergency request 1.a. 69.99% and below of the maintenance calls and deactivation requests were responded to within the timeframe: Unsatisfactory 1.b. 70%-79.99% of the maintenance calls and deactivation requests were responded to in the appropriate timeframe: Marginal

1. c. 80%-89.99% of the maintenance calls and deactivation requests were responded to in the appropriate timeframe. Satisfactory 1.d. 90%-94.99% of the maintenance calls and deactivation requests were responded to in the appropriate timeframe: Excellent 1.e. 95% and above of the maintenance calls and deactivation requests were responded to in the appropriate timeframe: Outstanding

Random sampling Minimum 5% up to 100% surveillance

SSPA#2

Customer Satisfaction

a. Emergency Maintenance Customer Satisfaction

b. Routine Maintenance Customer Satisfaction

c. Deactivation Customer Satisfaction

The Contractor will be determined to have met the minimum requirement for customer satisfaction when the average Customer Satisfaction score tallied from all surveys is 80% or higher (Exhibit 1)

1. a. Average Customer Satisfaction score is 69.99% or lower. Unsatisfactory

1. b. Average Customer Satisfaction score is between 70%-79.99%. Marginal

1. c. Average Customer Satisfaction score is between 80%-89.99%. Satisfactory 1.d. Average Customer Satisfaction score is between 90%-94.99% Excellent

1. e. Average Customer Satisfaction Score is 95% or higher. Outstanding

SSPA#3 Quality

a. Emergency Maintenance Quality: PWS and QASP

The contractor will be determined to have met the minimum requirement for quality when the average Quality score tallied from all surveys is 80% or higher.

1.a. Average Quality score is 69.99% or lower: Unsatisfactory

1. b. Average Quality score is between 70%-79.99%. Marginal 1.c. Average Quality score is between 80%-89.99%: Satisfactory

1. d. Average Quality score is between 90%-94.99. Excellent

1. e. Average Quality score is 95% or higher. Outstanding

Random sampling Minimum 5% up to 100% surveillance

b. Routine Maintenance Quality: PWS and QASP

Overall Performance

The Contractor will be determined to have met the minimum requirement for overall performance when the average Overall Performance score is 80% or higher.

An Overall Performance score of less than 80% is unacceptable; a score of 90% or over is excellent or outstanding.

1. a. Overall Performance score is 69.99% or lower. Unsatisfactory

1. b. Overall Performance score is between 70%-79%. Marginal

1. C. Overall Performance score is between 80%-89.99%. Satisfactory

1.d. Overall Performance score is between 90%-94.99% Excellent

1. e. Overall Performance score is 95% or higher. Outstanding

1.a. Unsatisfactory 1.b. Marginal 1.c. Satisfactory 1.d. Excellent 1.e. Outstanding

Invoice changes will not be required under this QASP

Exhibit 1a Customer Satisfaction Survey (Routine and Emergency Maintenance)

FEMA-DR-1981

Use the following scale to answer the satisfaction questions.

1 = Completely Inappropriate 4 = Very Appropriate (Very Good) 2 = Somewhat Inappropriate 5 = Completely Appropriate (outstanding) 3 = Appropriate (Minimum Requirement)

Did the contractor perform in an acceptable manner (attitude, professional attire, badge displayed, and professionalism)?

Did the contractor respond in a timely manner?

Did the contractor address (examine and repair) the problem?

Did contractor’s arrival match the scheduled time, and did he/she provide information for future monthly maintenance?

Did contractor restore the site and not leave any safety hazards?

Observation Score Total

Total raw score:

Divide by .25 to obtain Customer Satisfaction Rating

Registration # or ID

Address

Phone

Date

Interview

Remarks

Exhibit 1b Customer Satisfaction Survey (Deactivations)

FEMA-DR-1981

Use the following scale to answer the satisfaction questions.

1 = Completely Inappropriate 4 = Very Appropriate (Very Good) 2 = Somewhat Inappropriate 5 = Completely Appropriate (outstanding) 3 = Appropriate (Minimum Requirement)

Did the contractor perform in an acceptable manner (attitude, professional attire, badge displayed, and professionalism)?

Did the contractor respond in a timely manner?

(ask in the event of a reinstallation or repositioning) Was the deactivation/reinstallation or deactivation/repositioning completed within a twelve hour timeframe, so that the applicant was not displaced overnight?

Did contractor’s arrival match the scheduled time?

Did contractor restore the site and not leave any safety hazards?

Observation Score Total

Total raw score:

Registration # or ID

Address

Phone

Date

Interview

Remarks

Quality Assurance Surveillance Plan 11/17/2011

Exhibit 2

MAINTENANCE QUALITY SURVEY

Description of Standard

Maintenance (100% maximum score)

1. Contractor performed monthly preventative maintenance inspection for unit whose work order is being surveyed, either for month of survey or previous month (previous month if surveyed work order is for routine or emergency maintenance call which originated from applicant and monthly preventative maintenance inspection is scheduled for later in the month). (10 percentage points)

2. Contractor provided appropriate FEMA reports detailing the surveyed work order in a timely manner as required. (10 percentage points)

3. Contractor left the occupant a summary of the findings and work performed for any of visit and service call (preventative maintenance, routine maintenance, or emergency maintenance (10 percentage points)

4. Did contractor perform contractor responsibilities according to the SOW? To include phone service, hours of operation, facilities, identification, resources, warranty, and reporting problems? (20 percentage points)

5. Contractor reported maintenance problems covered by warranty. (10 percentage points)

6. Qualified Contractor personnel performed the maintenance work. (10 percentage points)

7. Contractor did all things necessary to maintain/repair the unit to a safe, working, livable and comfortable condition, with the exception of those things that the occupants could reasonably maintain/repair. (20 percentage points)

8. Contractor used new materials of a quality equal to or better than the materials that were replaced. (10 percentage points)

Total Percentage Points: _____________________% Work Order Number: __________________________ Quality Survey Performed By: ___________________

Exhibit 3

DEACTIVATION QUALITY SURVEY

Description of Standard

Deactivations (100% maximum score)

1. Did contractor perform Deactivation Preparation according to the SOW? (10 percentage points)

2. Did the contractor secure the interior fixtures, equipment, and furnishings of the unit according to the SOW? (10 percentage points)

3. Did the contractor secure the exterior according to the SOW? (10 percentage points)

4. Did contractor perform the cleaning and fumigating according to the SOW? (10 percentage points)

5. Did contractor perform the winterization according to the SOW? (10 percentage points)

6. Did contractor remove the debris associated with the deactivation in accordance with the SOW? (20 percentage points)

7. Did the contractor use the appropriate equipment resources to perform deactivation (10 percentage points)

8. Did contractor at any time leave unattended or otherwise unprotected, any excavation or safety hazard that might cause injury to any person? (20 percentage points)

Total Percentage Points: _____________________% Work Order Number: __________________________ Quality Survey Performed By: ___________________

Exhibits Page 18 Final-DRAFT image1.jpeg