SAQMMA11R0001 Att 2 DS CTO Award Fee Plan.pdf

Award Fee Plan for IT Support Services

Bureau of Diplomatic Security Office of the Chief Technology Officer

September 2010

TABLE OF CONTENTS

1.0 INTRODUCTION

2.0 ROLES AND RESPONSIBILITIES

2.1 Award Fee Determining Official (AFDO)

2.2 Award Fee Evaluation Board (AFEB) Members

2.3 Performance Monitors

3.0 AWARD FEE PROVISIONS AND EVALUATION PERIODS

3.1 Fixed Price

3.2 Total Available Award Fee

3.3 Evaluation Period

4.0 EVALUATION FACTORS, CRITERIA, WEIGHTS AND RATINGS

4.1 Evaluation Factors and Criteria

4.2 Evaluation Ratings and Weights

5.0 EVALUATION PROCEDURES

5.1 Performance Evaluation

5.2 Reporting and record keeping procedures

5.3 Changes/Modifications

APPENDIX 1 – LIST OF ACRONYMS

APPENDIX 2 – AWARD FEE EVALUATION PROCESS

APPENDIX 3 – PERFORMANCE MONITOR CHECKLIST

APPENDIX 4 – RECOMMENDATIONS FOR THE AFEB

APPENDIX 5 – EVALUATION DURING CONTRACT TRANSITION PERIOD . 17

1.0 INTRODUCTION

The purpose of this Plan is to document the award fee approach for the Bureau of Diplomatic Security’s (DS) Office of the Chief Technology Officer (CTO) IT Support Services Program. In addition to the approach, the Government will define the roles and responsibilities associated with determining the fee to be periodically awarded to the contractor, the procedures for assessing the contractor’s performance, change requests and evaluation periods. With the selection of a fixed price, award fee contract (FPAF), the Department of State (hereafter referred to as the Department or DoS) is provided with the flexibility to judgmentally evaluate the contractor’s performance at specified intervals throughout the life of the contract (including option years) and, if necessary, make adjustments to reflect changes in Department and/or Bureau priorities.

The main objectives of negotiating an award-fee arrangement with the contractor are to:

Achieve effective communication between Government and contractor personnel;

Meet desired results;

Share risks between the Government and contractor;

Define and plan for areas deemed as critical to Program success; and

Encourage continual improvement in all aspects of the contract.

Those areas deemed as critical must align with performance evaluation areas in this Plan. The Award Fee Plan has been organized into the following four sections:

1. Roles and Responsibilities;

2. Award Fee Provisions and Evaluation Periods;

3. Evaluation Factors, Criteria, Weights, and Ratings; and

4. Evaluation Procedures.

Each of these sections is discussed in detail below. This Plan is subject to change based on negotiations between the Government and the contractor.

2.0 ROLES AND RESPONSIBILITIES

The following organizational structure has been established to administer the award fee provisions of the contract. Positions include:

Award Fee Determining Official (AFDO), Award Fee Evaluation Board (AFEB) Members, and

Performance Monitors.

Each of these positions is described in detail below.

2.1 Award Fee Determining Official (AFDO)

The AFDO has been designated as a DS senior executive who is not involved directly in the performance evaluation of the contractor. By not having a daily interface with the contractor, this individual is better able to maintain a broader perspective on the contractor’s performance. The primary responsibilities of the AFDO include, but are not limited to:

Making the final determination regarding the award fee earned and payable for each evaluation period in accordance with the procedures outlined in this Plan;

Ensuring the award-fee process integrity is maintained throughout the life of the Program; and

Instituting changes to any portion of the award fee structure as appropriate.

The AFDO has been identified in the following table:

Role Position Office

AFDO Chief Technology Officer DS/EX/CTO

Table 1 – Award Fee Determining Official

2.2 Award Fee Evaluation Board (AFEB) Members

The AFEB consists of DS representatives who have been selected to evaluate the contractor’s performance with respect to the evaluation factors and criteria discussed in Section 4 of this Plan. These individuals have been selected based on their knowledge of DS and the Office of the CTO, expertise in one of the contractor’s performance areas to be assessed, and commitment to the Department overall. The Chair of the AFEB has been designated as the individual most familiar with the functional areas as a whole, namely the Contracting Officer’s Representative (COR). At the conclusion of each evaluation period, the AFEB will convene a meeting to discuss and evaluate the contractor’s performance. Once a decision has been reached, the Board prepares a written report detailing the performance ratings of the contractor and recommendations to the AFDO. The Award Board is composed of four members as identified in the following table:

Table 2 – Award Board Members

Quorum. A quorum is four AFEB members, or their designated alternates. All four members or their alternates must be present in order to have a meeting.

Alternates. In the event that an AFEB member is unable to attend a Board meeting, the principal may designate an alternate.

Subject Matter Expert(s). As determined by the Chair, additional participants, such as operating unit project managers, may be invited to attend and advise the AFEB.

Role Position Office

AFEB Member - Chair CTO Branch Chief/COR DS/CTO

AFEB Member CTO Branch Chief DS/CTO/SIB

AFEB Member CTO Branch Chief DS/CTO/OPS

AFEB Member CTO Branch Chief DS/CTO/SEC

AFEB Member CTO Branch Chief DS/CTO/SFW

AFEB Member Contracting Officer A/LM/AQM/WWD

2.3 Performance Monitors

The Government will monitor the performance metrics in accordance with the Quality Control Plan (QCP) put forth by the contractor during contract solicitation.

If employed by the AFEB, Performance Monitors (PM) will collect contractor performance information during each evaluation period. Each PM will be responsible for working with the contractor to collect, evaluate and assess performance in accordance with the evaluation factors outlined in the Plan.

The AFEB Chair will designate PMs from within the DS CTO’s government staff based on technical expertise, knowledge of the contract, and freedom from conflicts of interest.

3.0 AWARD FEE PROVISIONS AND EVALUATION PERIODS

The fee established in an FPAF contract consists of two parts; they include:

Fixed Price amount for base period (1 year) and option years (4), and

Award Fee amount for base period (1 year) and option years (4).

The fixed price is paid for satisfactory performance; the award fee, if any, is earned for performance beyond that required.

3.1 Fixed Price

Fixed prices for the DS CTO IT Support Services has been approved at (Insert Dollar Amount Here after negotiations) for the entire contract period, which includes the base period (1 year) with four additional one-year option periods. This price has been competitively negotiated and mutually agreed to by the Government and the contractor with amounts not subject to economic adjustments without modification to the contract.

Fixed price funds have been budgeted and approved by the DS CTO.

3.2 Total Available Award Fee

The available award fee has been approved at (Insert Dollar Amount Here after negotiations) yearly for the base and option periods. This amount is 5% of the yearly fixed price and is equally distributed between the evaluation periods and reviewed by the AFEB using defined criteria. The maximum negotiated fees (fixed and award) will be budgeted as part of the total contract budget.

Once the AFDO has reached a decision, payment of the award fee earned should be made promptly after the fee determination has been made. This helps ensure that the award fee functions as it was designed, as an incentive to motivate the contractor to exceed performance objectives. Any portion of the award fee not awarded for an evaluation period may not be transferred to another evaluation period.

3.3 Evaluation Period

With the IT Support Services Program, the DS CTO has elected to vary the length of the evaluations period during the life of the contract. A tri-annual award fee evaluation will be used during the first year of the contract and then semi-annual evaluations thereafter, until the closeout of the contract. Altering the tri-annual evaluation period after the first year is based on the assumption that the Government will need to evaluate the contractor more frequently during the first year of providing these services, while out years will require less oversight due to the contractor’s increased knowledge of the Program. In addition, the Department has selected this award fee model to:

1) Allow the contractor to provide performance information over a sufficient period of time,

2) Provide adequate time for the contractor and the Government to make adjustments to work activities and/or the Award Fee Plan, and

3) Reduce the time requirement for all stakeholders involved in the collection and review of information.

The Evaluation Periods are as follows:

Table 3 – Evaluation Periods (including option years)

Year Evaluation

Periods Description

Base Years

1 1-3 Yearly Award Fee Amount - $TBD Tri-Annual Evaluation Award Fee Pool - $TBD per period

Option Years

2 4-5 Yearly Award Fee Amount - $TBD Semi-Annual Evaluation Award Fee Pool - $TBD per period

3 6-7 Yearly Award Fee Amount - $TBD

4 8-9

5 10-11

The Evaluation Periods have been scheduled as follows:

Evaluation Period Schedule

Evaluation Time Period

Start Date Finish Date Amount

1st Base Year

1 TBD

2nd Base Year

3rd Base Year

1st Option Year

2nd Option Year

Total

Table 4 – Evaluation Period Schedule (including option years)

Payment of the award fee will correspond to the evaluation period and will be invoiced separately to maintain a record of payment (specific to the award fee).

Procedures for award fee changes can be found in Section 5.3 of this Plan. While the Government may unilaterally revise the distribution of the award fee, the total award fee may not be altered unless a contract modification occurs.

4.0 EVALUATION FACTORS, CRITERIA, WEIGHTS AND RATINGS

4.1 Evaluation Factors and Criteria

The DS CTO has elected to assess the contractor’s performance against general factors that have then been divided into sub-factors as needed. These factors have been selected to support the Bureau’s priorities. Only the award fee evaluation factors set forth in this Plan shall be used to determine the contractor’s award fee scores unless otherwise mutually agreed to by both parties. See Attachment 5 regarding transition period evaluation criteria.

The factors include:

Business Management

Technical Performance

Each of these factors and their corresponding high-level criteria and weightings (%) are discussed and examined in the sub-sections below. As described in Section 5.0, the Government and contractor will meet prior to the start of each evaluation period to further decompose criteria to be used for evaluation purposes and to agree upon the method for collecting data.

4.1.1 Business Management

Overall business management will be evaluated on such criteria as project management, subcontractor management, quality control and schedule. Cost control has not been included as a sub-factor due to the fixed priced nature of the contract.

However, if cost overruns occur with regard to any contract line item numbers (CLINs), then the Government reserves the right to add cost control as a sub-factor. The business management factor corresponds to 50% of the award fee.

4.1.1.1 Project Management (20%)

The contractor will be evaluated on its ability to manage operations, both locally and remotely, and to manage the contract. The evaluation will include a review of the contractor’s effectiveness in planning and managing tasks across the CTO divisions, assessing and mitigating risk that pertains to the Program, meeting delivery schedules for products/deliverables, adequately staffing (24 hours per day, 7 days a week, 365 days per year) for requirements including surges, meeting all contract administration requirements, providing timely status updates on all contract tasks, responding to the Government Project Manager’s requests, and maintaining regulatory standards of employee conduct (security, etc).

In addition, the Government will assess the contractor’s ability to adapt to changes in the Department and/or Bureau’s priorities.

4.1.1.2 Subcontractor Management (10%)

The contractor will be evaluated on its ability to adequately implement and manage subcontracting plans. This evaluation will include the business relationship between the prime and sub(s) and the quality of services provided by both parties as a team.

4.1.1.3 Quality Control (20%)

The contractor will be evaluated on the quality of its work including the accuracy (both with regard to Department and technical information), internal review process, coordination with DoS quality assurance (QA) entities, and overall compliance with the contractor’s quality assurance plan (as included in the contract).

4.1.2 Technical Performance

Overall technical performance will be evaluated on such criteria as performance against agreed upon standards and technical knowledge & innovation.

The technical performance factor corresponds to 50% of the award fee.

4.1.2.1 Performance Against Agreed Upon Standards (40%)

The contractor will be evaluated on its ability to meet specified performance standards as defined in the contract’s Performance Work Statement (PWS) and any Service Level Agreements (SLAs) that the Government and contractor have agreed upon. Possible areas for review include, but are not limited to, customer satisfaction, help desk metrics (tickets open/closed, response time, etc.)

reliability, total network uptime/downtime, patch management metrics, network statistics (bandwidth consumption, firewall status, etc.), and security status (# of active users, account requests, intrusions, etc.).

All standards and metrics will be discussed with the contractor prior to each evaluation period.

4.1.2.2 Technical Knowledge & Innovation (10%)

The Bureau shall evaluate the contractor on its ability to adapt to new technologies and techniques (as they pertain to this contract) and to develop innovative designs, concepts, processes, etc., that will result in operational efficiencies, performance improvements, and/or cost savings.

4.2 Evaluation Ratings and Weights

The contractor will be judged on how well it performs in relation to the criteria set forth in this Plan. An award fee (of any amount) will not be provided for less than satisfactory overall performance. In addition, if the contractor achieves an unsatisfactory rating in any category, no award fee will be made available for that evaluation period.

Using Government best practices1, the Award Fee evaluation ratings have defined as follows:

The Award Fee Rating Table and Rating Conversion Chart have been constructed using best practices as defined in NASA contract guidance and processes.

Award Fee Ratings

Adjective Rating

Definition

Excellent

Of exceptional merit

Exemplary performance relating to all evaluation factors in a timely, efficient, and economical manner

Very minor (if any) deficiencies with no adverse effects on overall performance

Very Good

Very effective performance

Exemplary performance relating to all evaluation factors in a timely, efficient, and economical manner

Minor deficiencies with no adverse effects on overall performance

Good

Effective performance

Fully responsive to contract requirements and Department priorities

Deficiencies with minor impact on overall performance

Satisfactory

Meets or slightly exceeds minimum standards provided in the contract

Adequate results

Notable deficiencies with slight impact on overall performance

Poor/Unsatisfactory Does not meet minimum standards

Deficiencies with major impact on overall performance

Remedial action required to improve performance

Table 5 – Award Fee Ratings

Rating Conversion Chart

Numeric Rating

Exceptional 71-100

Very Good 31-70

Good 1-30

Satisfactory and Below 0

Table 6 – Award Fee Rating Conversion Chart

During the award fee evaluation, the AFEB scores each sub-factor from 0 to 100 and then assigns the corresponding adjectival rating. For example, if the AFEB members rate the contractor’s performance in relation to project management as effective and fully responsive to contract requirements, but with one or two minor deficiencies then a score between 1 and 30 and a ‘Good’ rating is assigned. The numerical score, in this case 1 to 15, is then multiplied by the pre-determined weighting (20%), to determine the weighted score (15 x .20 = 3) for project management. Once all of the scores have been determined, the AFEB members tally them to determine the percentage of the available award fee that will be paid.

5.0 EVALUATION PROCEDURES

5.1 Performance Evaluation

The performance evaluation for the DS CTO IT Support Services Program consists of nine steps for each evaluation period. It should be noted that Step 5 is the first step in the process where the AFEB and contractor may communicate regarding evaluation period performance due to conflict of interest concerns. The performance evaluation steps include:

1. Prior to the commencement of each evaluation period, the Government and contractor will meet to review DS priorities (relating to the contract) and discuss the performance metrics and other pertinent information to be used for the contractor’s performance assessment at the conclusion of the evaluation period, including distribution of the award fee between factors.

2. During the evaluation period, the contractor works with the DS CTO Program

Office to collect and report required metrics and other pertinent information in accordance with the QA plan.

3. At the conclusion of each evaluation period, the DS CTO Program Office will provide the contractor’s performance information to the AFEB members. A meeting will then be convened to discuss and evaluate the contractor’s performance. The AFEB will discuss this information only as it relates to the criteria defined in Section 4.1 of this Plan and determine a rating for the contractor.

4. Once a decision has been reached, the Board prepares the Award Fee

Evaluation Report, a (draft) written report detailing the performance ratings of the contractor, a recommendation of the award fee amount, and any other recommendations relating to the contractor’s performance or the Award Fee Plan to the AFDO. The Award Fee Evaluation Report will be developed not later than five (5) working days after the conclusion of the AFEB meeting.

5. The Government shall provide the contractor with an opportunity to respond to the AFEB’s findings through an oral self-assessment (not to exceed one hour) and a written assessment (not to exceed 20 pages) by the contractor assessing their performance during the evaluation period. The written assessment must be submitted to the COR within five (5) working days after the AFEB has developed its written recommendations and the oral presentation shall occur two to three working days after written assessment submission, subject to Award Board member availability.

6. The AFEB prepares its final report to the AFDO with a courtesy copy to the contractor. The AFEB will develop and deliver the final Award Fee Evaluation Report to the AFDO within two (2) working days of the contractor’s oral presentation.

7. A determination of the final award fee earned for each evaluation period shall be made by the AFDO within 45 calendar days after the end of the period.

8. The AFDO will provide the final award fee amount for the evaluation period to the contracting officer and contractor in writing. If an award fee has been selected, then the Government will provide payment to the contractor in a timely fashion or not later than 60 calendar days after the conclusion of the evaluation period.

9. A briefing will be offered to the contractor regarding their performance during the evaluation period.

5.2 Reporting and record keeping procedures

The contractor, in accordance with its approved QA plan, shall perform all reporting and record keeping. The Government will work with the contractor to collect this information for evaluation purposes per the evaluation period schedule. All information shall be collected and stored in a central repository.

5.3 Changes/Modifications

The Government reserves the right to unilaterally change any matters pertaining to this Plan not otherwise requiring mutual agreement not later than thirty calendar days prior to the beginning of an evaluation period. Both parties shall agree to any changes to the award fee plan that take effect in the current evaluation period. All changes, albeit proposed by the Government or contractor, shall be coordinated with the AFEB with the AFDO having final approval authority. FAR 52.249-2 applies if the determination is made to terminate the contract for the Government’s convenience.

APPENDIX 1 – LIST OF ACRONYMS

AFDO – Award Fee Determining Official

AFEB – Award Fee Evaluation Board

AQM – Acquisitions Management

CLIN – Contract Line Item Number

COR – Contracting Officer’s Representative

CTO – Chief Technology Officer

DoS – Department of State

DS – Diplomatic Security

FAR – Federal Acquisition Regulation

FPAF – Fixed Price, Award Fee

IT – Information Technology

PM – Performance Monitor

PWS – Performance Work Statement

QA – Quality Assurance

QASP- Quality Assurance and Surveillance Plan

SLA – Service Level Agreement

SOP – Standard Operating Procedure

APPENDIX 2 – AWARD FEE EVALUATION PROCESS

1.0

Review DS priorities with contractor and decide on evaluation criteria

Performance Monitors

2.0

Collect metrics & other pertinent information

Contractor Performance

Information

AFEB

3.0

Review Contractor

Performance Information

Determine preliminary score

4.0

Prepare draft Award Fee

Evaluation Report

5.0

Review Contractor's Self

Assessment Reports

Determine final score

6.0

Prepare final Award Fee

Evaluation Report

Final Award Fee Evaluation

Report

AFDO

7.0

Review Award Fee

Evaluation Report

Determine award

8.0

Forward award fee decision and documentation to CO for appropriate action

Official

Contract

Records

9.0

Provide briefing to contractor

APPENDIX 3 – PERFORMANCE MONITOR CHECKLIST

While preparing their contractor performance reports, the PM should consider several questions. These questions can prove helpful in assuring that the evaluation data are complete and pertinent and accurately assess how well the contractor performed in the Monitors' respective assigned areas during the period.

a. What (in the monitor's area) was the contractor supposed to do during the period? What was actually accomplished?

b. How critical are the efforts accomplished, or not accomplished, by the contractor?

c. What was the impact of any efforts completed early or late? How critical was the time frame involved?

d. How well did the contractor perform the tasks that were accomplished?

e. What are the major strengths and weaknesses (in sufficient detail to discuss with the contractor)?

f. Did any major obstacles arise which impacted contractor performance? Were these obstacles caused by events/decisions beyond the contractor’s control?

What corrective actions were implemented? How effective were they?

g. Has the contractor efficiently and effectively used available resources (e.g., personnel and facilities) to improve its performance (including cost control as it relates to surge requirements)?

h. Has the contractor's performance been clearly assessed in regard to all tasks and specific objectives?

APPENDIX 4 – RECOMMENDATIONS FOR THE AFEB

The Board should be established in sufficient time so it can ensure development and distribution of the approved plan before the start of the first evaluation period.

The award fee determination process must contain appropriate checks and balances for maintaining its impartiality.

The DS CTO PMO should draft templates for the evaluation process including the initial fee recommendations.

People knowledgeable of the contract requirements should perform or oversee the monitoring, assessing and reporting of the contractor’s performance.

Officials at the management level who are not involved in daily, on-line operational interface with the contractor should conduct their own evaluation.

A charter outlining the AFEB members, their goals, processes, etc. should be developed.

DS CTO should draft confirmation letters for each AFEB member outlining their roles and responsibilities. This letter allows personnel overseeing the award fee process to make sure only appropriate personnel are evaluating performance and that these individuals understand their position and expectations placed upon them.

All AFEB members should be trained on their roles and responsibilities.

PMs should give specific examples to support their conclusions.

When documents aren’t sufficient, the evaluators need to find out why and correct the cause of the problem.

When preparing written evaluations, evaluators shall provide detailed examples supporting their specific ratings. In addition, evaluators are required to identify areas where improvement is necessary or has already occurred.

The AFEB should work with the contractor to identify dispute resolution and mediation resources/processes prior to beginning the contract activities.

The AFEB should develop an Award Fee change process prior to the start of the contract.

APPENDIX 5 – EVALUATION DURING CONTRACT TRANSITION

PERIOD

A5.1 Introduction In accordance with the tri-annual Award Fee evaluation periods established for the first year of contract performance, the Government has elected to implement a four (4) month transition schedule to provide ample time for an effective relationship with the contractor to be established and to ensure that a true performance-based environment is achieved. The purpose of this appendix, which replaces Section 4.1 of the Award Fee Plan during the first evaluation period, is to outline how the Government will rate the contractor during this time period.

The goals of this transition period are as follows:

Allow the contractor to collect/establish baseline metrics that will support and validate the performance measures outlined in the PWS and any SLAs;

Allow for the modification of any SLAs, in the event that the performance metrics are not aligned with realistic expectations given environmental conditions at

DS/CTO;

Validate contractor management processes, to include configuration management and ability to meet requirements for staffing surges as they arise;

and

Facilitate communications between the Government and contractor and integrate the contractor staff into the DS/CTO organizational culture within a finite period of time.

A5.2 Evaluation Factors The evaluation of the contractor during the transition period will be based broadly on two factors: 1) compliance with the transition schedule (provided in the contractor’s proposal), and 2) quality/relevance of deliverables. Innovation and Adaptability will also be evaluated as they are reflected in the two factors.

Transition Schedule (20%) <Insert schedule after contract negotiations>

Deliverables (Total weight - 60%) Contractor deliverables required during the transition period will be listed below and referenced in the transition schedule. Unless outlined below, the standards of acceptance will be in accordance with Section F of this contract. It should be noted, however, that Government acceptance of a deliverable does not guarantee a rating higher than satisfactory, in accordance with section A5.4 of this document.

Each deliverable specified in the Transition Schedule will be given a raw weight, represented as a percentage of the total award fee. The sum of these weighted values will equal 60% of the total award fee available during the transition period.

<Insert list of deliverables/standards/weights after negotiations>

Innovation and Adaptability (20%) In addition to those standards of acceptance outlined in Section F of this contract, the contractor will be evaluated on its ability to adapt to changing environmental conditions within DS/CTO and to implement innovative solutions to problems and challenges affecting service implementation.

A5.3 Evaluation Procedures Evaluation procedures for determination of the award fee amount will remain the same as Section 5.0 of the Award Fee Plan.

A5.4 Evaluation Ratings and Weights The contractor’s deliverables will be evaluated adjectivally, based on the descriptions below. The ratings below apply specifically to deliverables and will replace Table 5 of the Award Fee Plan only in the evaluation of those deliverables agreed upon during negotiations. Other procedures outlined in Section 4.2 of the Award Fee Plan will remain in effect.

As previously noted, an award fee (of any amount) will not be provided for less than satisfactory overall performance. In addition, if the contractor achieves an unsatisfactory rating in any category, no award fee will be made available for that evaluation period.

Award Fee Ratings -- Deliverables

Definition

Excellent

Of exceptional merit;

Exemplary product, demonstrates thorough understanding of CTO objectives and provides keen insight for current and future operations;

Delivered in a timely and efficient manner;

Very minor (if any) deficiencies with no adverse effects on overall performance.

Very Good

Very effective communication;

Exemplary product that demonstrates complete understanding of CTO objectives;

Delivered in a timely and efficient manner;

Minor deficiencies with no adverse effects on overall contract performance.

Good

Effective communication of results;

Fully responsive to contract requirements and CTO priorities;

Deficiencies with minor impact on overall contract performance.

Satisfactory

Meets or slightly exceeds minimum standards provided in the contract;

Adequate compliance with CTO objectives;

Notable deficiencies with slight impact on overall contract performance.

Poor/Unsatisfactory

Does not meet minimum standards;

Deficiencies with major impact on overall contract performance;

Major structural/content changes need to be made to comply with contract requirements.