Attachment_1_HMTAP_QASP_v4.pdf

Federal Insurance and Mitigation Administration

Hazard Mitigation Technical Assistance Program

Indefinite Delivery/Indefinite Quantity

Quality Assurance Surveillance Plan

U.S Department of Homeland Security

Federal Emergency Management Agency

Federal Insurance and Mitigation Administration

Risk Reduction Division

500 C Street

Washington, DC

August 14, 2014

Attachment 1

TABLE OF CONTENTS

1. INTRODUCTION

2. ROLES AND RESPONSIBILITIES

3. QUALITY CONTROL PLAN

4. METHODS OF SURVEILLANCE

5. ACCEPTABLE QUALITY LEVELS

6. QUALITY ASSURANCE DOCUMENTATION

7. ANALYSIS OF QUALITY ASSURANCE ASSESSMENT

Appendix A: Performance Requirements Summary Table

Appendix B: Site Visit Quality Assurance Monitoring Form

Appendix C: Customer Service Feedback Investigation Form

Appendix D: Task Order Performance Evaluation Form

1. INTRODUCTION

The Government’s Quality Assurance Surveillance Plan (QASP) is based on the premise that the service provider, and not the Government, is responsible for the day-to-day performance of the work and all the management and quality control actions required to meet the terms of the contract. The role of the Government in quality assurance is to ensure performance standards are achieved and maintained. The service provider is required to develop a comprehensive program of inspections and monitoring actions, and to document its approach in a Quality Control Plan (QCP). The service provider’s QCP, upon approval by the Government, will be made a part of the resulting contractual agreement.

This QASP is designed to provide an effective surveillance method to monitor the service provider’s performance relative to the requirements listed in the agreement. The QASP illustrates the systematic method the Government (or its designated representative) will use to evaluate the services the service provider is required to furnish.

This QASP is based on the premise that the Government must validate that the service provider is complying with Government-mandated quality standards in all aspects of the services provided. Performance standards address all facets of operations, including quality; accuracy; completeness; timeliness; safety and health; conformance with statutory, regulatory, and policy guidance; appropriate use and care of Government furnished property, information, or facilities; data and records management; etc. Good management by the service provider and use of an approved QCP will ensure that the services are provided within acceptable quality levels.

2. ROLES AND RESPONSIBILITIES

Contracting Officer The Contracting Officer (CO) is responsible for monitoring contract compliance, contract administration, and cost control and for resolving any differences between the observations documented by the Contracting Officer’s Representative (COR), the HMTAP Program Manager and the contractor. The CO will designate one full-time COR as the Government authority for performance management. The number of additional representatives serving as project managers and technical monitors depends on the complexity of the services measured, as well as the contractor’s performance, and will be identified and designated by the CO for each task/delivery order.

Contracting Officer’s Representative The Contracting Officer’s Representative (COR) is designated in writing by the CO to act as his or her authorized representative to assist in administering a contract. COR responsibilities are contained in the written appointment letter. The COR is responsible for day to day administration of the program and ensures proper Government surveillance of the contractor’s performance. The COR is not empowered to make any contractual commitments or to authorize any contractual changes on the Government’s behalf.

Any changes discussed by the COR that the contractor deems may affect contract price, terms, or conditions shall be referred to the CO for action. The COR will have the responsibility for completing quality assurance (QA) monitoring forms used to document the inspection and evaluation of the contractor’s work performance.

3. QUALITY CONTROL PLAN

As a part of its agreement with the Government, the service provider is required to develop, implement, and maintain a QCP that illustrates the methods it will use to review its performance to ensure it conforms to the performance requirements (Appendix A:Performance Requirements Summary Table). Such reviews are performed by the service provider in order to validate its operations and assure the Government that the services meet the performance standards.

The service provider’s QCP should include monitoring methods that ensure and demonstrate its compliance with the performance standards. This includes inspection methods and schedules that are consistent with the regular reviews conducted by the Government. The reports and other results generated by the service provider’s QCP activities should be provided to the COR as requested.

The frequency and type of the service provider’s reviews should be consistent with what is necessary in order to ensure compliance with the performance standards, but no less frequent than what is described in the Government’s monitoring instrument/worksheets (See Appendix A).

The service provider is encouraged not to limit its inspection to only the processes outlined in the Government’s standard, however, certain key documents must be provided by the contractor to assure the Government that the services meet the performance standards. Some of the documentation that must be generated and made available to the COR for inspection is listed below:

QCP Reports

Written policies and procedures to implement and assess operational requirements of the standard

Documentation and recording-keeping that ensure ongoing operational compliance with the standards (e.g., monthly reports, quarterly reports, invoices, etc.)

Staff training records

The list is intended as illustrative and is not all-inclusive. The service provider must develop and implement a program that addresses the specific requirement of each standard and the means it will use to document compliance.

4. METHODS OF SURVEILLANCE

In an effort to minimize the performance management burden, simplified surveillance methods shall be used by the Government to evaluate contractor performance whenever possible. The primary methods of surveillance are:

4.1 Site Visits: Site visits will be random. They can be scheduled or ad-hoc. Site visits are used to observe actual performance to determine the extent of compliance with performance standards, and to ensure any noted deficiencies are effectively addressed and corrected as quickly as possible.

Scheduled site visits will be performed no less than one (1) per contract period as funding allows. Scheduled site visits may involve direct observation of contractor personnel performing tasks, interacting with Government staff or the public. Site visits may be performed by the COR or by other parties designated as representatives of FEMA. The COR or other designated representative of FEMA will record all findings on the Site Visit Quality Assurance Monitoring Form (Appendix B).

Ad-hoc site visits are unscheduled and will be conducted as a result of special interests arising from routine monitoring of the service provider’s QCP, an unusual occurrence pertaining to the agreement, or other FEMA concerns.

When visiting a site, either the COR or a designated representative of FEMA may conduct their own inspections of contractor performance activities, or accompany the service provider’s designated Quality Control Inspector (QCI) on scheduled inspections.

The COR and QCI may inspect the same event simultaneously but independently and compare results to determine if any surveillance areas were overlooked. The COR or other designated representative of FEMA will record all findings on the Site Visit Quality Assurance Monitoring Form (Appendix B). Site visit findings will be provided to the contractor personnel as appropriate.

4.2 Customer Feedback:

The contractor is expected to establish and maintain professional communication between its employees and customers. The primary objective of this communication is to determine the level of customer satisfaction. Customer satisfaction is the most significant external indicator of the success and effectiveness of all services provided and can be measured through customer compliments and/or complaints. The COR will accept those customer compliments and complaints and investigate using the Customer Service Feedback Investigation Form, identified in Appendix C.

Performance management drives the contractor to be customer-focused by initially and internally addressing customer complaints and investigating the issues and/or problems.

The customer always has the option to communicate complaints to the CO and COR, rather than the contractor.

Customer complaints, to be considered valid, must be set forth clearly and in writing with the detailed nature of the complaint, must be signed, and must be forwarded to the

COR.

4.3 Other Feedback:

The COR or other designated representative of FEMA will complete a performance evaluation as part of the closeout of each task order to ascertain the extent of compliance with the performance standards. The COR or other designated representative of FEMA will record all findings on the Task Order Performance Evaluation Form (Appendix D).

5. ACCEPTABLE QUALITY LEVELS

The acceptable quality levels (AQLs) included in Appendix A are structured to allow the contractor to manage how the work is performed while providing negative incentives for performance shortfalls. For certain critical activities that may be identified in a task/delivery order, the desired performance level maybe established at 100 percent.

Other levels of performance are keyed to the relative importance of the task to the overall mission performance as indicated in the task/delivery order.

6. QUALITY ASSURANCE DOCUMENTATION

6.1 Performance Management Feedback Loop

The performance management feedback loop begins with the communication of expected outcomes. Performance standards are expressed in the task/delivery order and are assessed using the methods of surveillance shown in Appendix A.

6.2 Monitoring Forms

The Government’s quality assurance surveillance will be reported using the specific monitoring tool identified in Section Four (4) of this QASP. The forms, when completed, will document the Government’s assessment of the contractor’s performance under the contract to ensure that the required results identified in the task order are being achieved.

7. ANALYSIS OF QUALITY ASSURANCE ASSESSMENT

7.1 Determining Performance

The Government shall use the monitoring methods cited herein to determine whether the performance standards/service levels/AQLs have been met. If the contractor has not met the minimum requirements, it may be asked to develop a process improvement plan or a corrective action plan to show how and by what date it intends to bring performance up to the required levels. Any mandatory deduction for performance requirements will be addressed in each individual task order (fixed-price only). In addition, failure to meet the AQL may result in a less than favorable rating in the

Contract Performance Assessment Reporting System (CPARS) as indicated in Appendix A.

7.2 Reporting

At the end of each quarter, the Contractor will prepare a written report for the COR summarizing the overall results of the quality assurance surveillance of the contractor’s performance. This written report, which includes any of the aforementioned completed quality assurance monitoring forms, will become part of the QA documentation. It will enable the Government to demonstrate whether the contractor is meeting the stated objectives and/or performance standards, including cost/technical/scheduling objectives.

7.3 Reviews and Resolution

The CO or COR may require the contractor’s Program Manager, Task Order Manager, or a designated alternate, to meet with Government personnel as deemed necessary to discuss the performance evaluation. The CO and COR will define a frequency of reviews with the contractor, including appropriate self-assessments by the contractor;

however, if the need arises, the contractor will meet with the CO and COR as often as required or per the contractor’s request. The agenda of the reviews may include:

Monthly performance assessment data and trend analysis

Issues and concerns of both parties

Projected outlook for upcoming months and progress against expected trends, including a corrective action plan analysis

Recommendations for improved efficiency and/or effectiveness

Issues arising from the performance monitoring processes

The COR must coordinate and communicate with the contractor to resolve issues and concerns regarding unacceptable performance.

The CO and contractor should jointly formulate tactical and long-term courses of action.

Decisions regarding changes to metrics, thresholds, or service levels should be clearly documented. Changes to service levels, procedures, and metrics will be incorporated as a contract modification at the convenience of the CO.

Appendix A: Performance Requirements Summary Table

Required Service Performance

Standard Acceptance

Level Methods of

Surveillance Incentive

Administrative Performance Requirements

Administrative paperwork (such as monthly invoices, monthly reports, quarterly reports, task order closeout reports, etc…) complete, accurate, and timely

Administrative report accurately depict current status 100% of the time

95%

File reviews, periodic inspections, random observation, and customer compliments and/or complaints

Less than favorable

CPARS

Rating

Administer quality control program including subcontractor management in accordance with Quality Control Plan

Contractor is in compliance with Quality Control Plan 100% of the time

95%

File reviews, periodic inspections, random observation, and customer compliments and/or complaints

Less than favorable

CPARS

Rating

Contract Objectives Performance Requirements

Ensure a high quality of product or service for the technical assistance provided to the HMA (Hazard Mitigation Assistance) Grant Programs

Quality assurance ratings have a satisfactory rating or higher 100% of the time

95% HMTAP Performance

Evaluation Form

Less than favorable

CPARS

Rating

Ensure a high quality of product or service for all components of environmental activities

Quality assurance ratings have a satisfactory rating or higher 100% of the time

95% HMTAP Performance

Evaluation Form

Less than favorable

CPARS

Rating

Ensure a high quality of product or service for all post-event assessments

Quality assurance ratings have a satisfactory rating or higher 100% of the time

95% HMTAP Performance

Evaluation Form

Less than favorable

CPARS

Rating

Ensure a high quality of product or service for all technical assistance and interagency coordination

Quality assurance ratings have a satisfactory rating or higher 100% of the time

95% HMTAP Performance

Evaluation Form

Less than favorable

CPARS

Rating

Ensure a high quality of product or service for all non- A&E publication technical assistance activities

Quality assurance ratings have a satisfactory rating or higher 100% of the time

95% HMTAP Performance

Evaluation Form

Less than favorable

CPARS

Rating

Contract Management Performance Requirements

Ensure satisfactory or better cost efficiency for all task orders

Quality assurance ratings have a satisfactory rating or higher 100% of the time

95% HMTAP Performance

Evaluation Form

Less than favorable

CPARS

Rating

Ensure satisfactory or better business relations for all task orders and general contract activities

Quality assurance ratings have a satisfactory rating or higher 100% of the time

95% HMTAP Performance

Evaluation Form

Less than favorable

CPARS

Rating

Ensure satisfactory or better key personnel management for all task orders and general contract activities

Quality assurance ratings have a satisfactory rating or higher 100% of the time

95% HMTAP Performance

Evaluation Form

Less than favorable

CPARS

Rating

*NOTE: A satisfactory CPARS rating is considered favorable and higher ratings require better than favorable performance.

Appendix B: Site Visit Quality Assurance Monitoring Form

Appendix B: Site Visit Quality Assurance Monitoring Form

Government Evaluator Name:

Government Evaluator Title:

Evaluation Date:

Task Order Number:

Project Title:

Observation Location:

Activity Being Observed:

Check the box that most closely indicates your level of agreement with each statement.

Use the scale of 1 (strongly disagree) through 5 (strongly agree). The Government Evaluator must provide detailed site visit notes with clear examples that support the ratings for each evaluation criteria.

Evaluation Criteria

Strongly Disagree

2 3 4

Strongly Agree

General Observations

The Contractor demonstrated acceptable interaction with Government field staff.

The Contractor was able to clearly state the objectives of the field work.

The Contractor was demonstrated knowledge of the project.

The Contractor demonstrated knowledge of FEMA-related programs.

Specific Observations: The Government will identify observations that are specific to the project type.

Site Visit Notes:

Appendix C: Customer Service Feedback Investigation Form

Memorandum for: _______, Contracting Officer

Memorandum From: __________, Contractor Officer’s Representative

Date:

Subject: Customer Service Feedback Investigation

General Information

1. Contractor:

2. Contract Number:

3. Task Order Number:

4. Project Title:

5. Evaluation Date:

6. Government Evaluator Name:

7. Government Evaluator Title:

Customer Service Feedback Information

8. Name of Customer:

9. Date of Customer Service Feedback:

10. Customer Service Feedback specifics (Describe in detail and reference task order proposal…Include any backup documentation provided by complainant.):

Contractor Response to Customer Service Feedback

11. Narrative of Contractor Response:

Government Analysis of Customer Service Feedback

12. Is Government surveillance investigation necessary? _______ (If so, complete questions 13 – 17.)

13. Surveillance Type:

14. Date of Surveillance:

15. Narrative of Performance during Survey Period:

16. Government Evaluation of Contractor Response/Resolution:

17. Recommended Government Action:

By signing below, the COR confirms the information contained herein for the Contracting Officer.

COR Signature Date

Printed Name – FEMA COR

Appendix D: Task Order Performance Evaluation Form

HMTAP

PERFORMANCE EVALUATION

Contractor

Contract Number Project Title

Task Order Number Not Applicable Evaluation Date

FEMA COR

FEMA PM

Contractor TOM Period of Performance through start date end date

PURPOSE OF PROJECT (This should be taken from SOO):

* All ratings must include supporting comments.

Upon opening this document you will need to disable the security warning by clicking on the OPTIONS button at the top of the document. When the Security Alert-Macro Warning comes up, select the Help protect me from unknown content (recommended) option. This will allow you to make changes to the document.

To complete this form, fill out each of the five following evaluation comments and the rating recommendations section.

For each evaluation factor, select the appropriate rating from the drop-down menu in the left column. In the right column, provide supporting comments that substantiate that rating. The evaluation comments must directly apply to the performance requirements in the agreed upon proposal. These comments are required in order for the evaluation to be accepted for review by the Contracting Officers Representative.

QUALITY OF PRODUCT OR SERVICE

(use below requirements guidelines for quality rating)

Exceptional No quality issues; Performance is “Outstanding”; Deliverables are precedent-setting; Reports are concise, thorough, and easy to understand; Deliverables document innovative or creative approaches; The explanation must include an example of a significant event that lead to the rating; No weaknesses were noted.

Very Good Minimal quality issues; More than one performance work statement revision required; Technical analysis incomplete; A few recommendations not accepted;

Reports adequately written, need minimal rework; Data accurate; The explanation must include an example of what was done to meet the requirements and what was done to exceed them in some “minor” way; No significant weaknesses.

Satisfactory Meets the requirements of the Performance Work Statement; If a correction was required, the explanation should include examples of what was done that needed correction and that the correction was done.

Unsatisfactory Serious quality issues; Technical analysis incomplete; Recommendations ineffective; Reports not usable; Some data flawed; Reports poorly written, not usable, and ineffective; The explanation must include an example of what was not done correctly and that it was not corrected or repeated; The explanation must state what magnitude of the contractor’s failure to the government was addressed.

Quality of Product or Service Evaluation Comments

(select a rating)

COST CONTROL/EFFICIENCY

(use below requirements guidelines for quality rating)

Exceptional No cost issues; Work assignment completed within budget in spite of extraordinary circumstances; Significant cost avoidance measures implemented with excellent documentation; Cost avoidance measures exemplary; The explanation must include an example of a significant event that lead to the rating;

No weaknesses were noted.

Very Good Rare cost issues; Rare failure to provide satisfactory documentation for cost increase; Cost avoidance efforts are demonstrated; The explanation must include an example of what was done to meet the requirements and what was done to exceed them in some “minor” way; No significant weaknesses.

Satisfactory Met the requirements of the agreed too cost schedule; If a correction was required, the explanation should include examples of what was done that needed correction and that the correction was done.

Unsatisfactory Cost issues; Failure to manage costs; Budgets exceeded without adequate warning or justification; Failure to provide documentation for cost increases;

Requirements for additional funding to complete the project; The explanation must include an example of what was not done correctly and that it was not corrected or repeated; The explanation must state what magnitude of the contractor’s failure to the government was addressed.

Neutral Rating of neutral is used for fixed price projects.

Cost Control/Efficiency Evaluation Comments

SCHEDULE

Exceptional No timeliness issues; All deliverables submitted ahead of schedule; Precedent-setting creative and innovation means to shorten timeframes; Proactive in identifying issues and in responding to project demands; The explanation must include an example of a significant event that lead to the rating; No weaknesses were noted.

Very Good Rare timeliness issues; All deliverables are submitted on time; Adequate documentation for requests to extend due dates; Efforts taken to reduce time well documented; Consistent response to issues and requests; The explanation must include an example of what was done to meet the requirements and what was done to exceed them in some “minor” way; No significant weaknesses.

Satisfactory Meets the schedule agreed upon in the Performance Work Statement; If a correction was required, the explanation should include examples of what was done that needed correction and that the correction was done.

Unsatisfactory Serious timeliness issues; Frequent slippage of schedule without adequate warning or justification; Frequent requests to extend due dates as a result of contractor caused issues; Failure to respond to issues and requests; The explanation must include an example of what was not done correctly and that it was not corrected or repeated; The explanation must state what magnitude of the contractor’s failure to the government was addressed.

Schedule Evaluation Comments

BUSINESS RELATIONS (Contract Requirements)

Business Relations Evaluation Comments

MANAGEMENT OF KEY PERSONNEL

Management of Key Personnel Evaluation Comments

CONSIDERATIONS AND RECOMMENDATIONS FOR FUTURE PROJECTS OF THIS TYPE

Were “Official” corrective actions required?* (If yes, corrective action documents must be attached)

Yes if yes, continue to answer the questions below.

No