15F06718R0000093_Award_Fee_Plan.pdf

FEDERAL BUREAU OF INVESTIGATION

Information Assurance Section Cybersecurity Section

Cybersecurity Operations Unit (COU)

Award Fee Plan

(Performance Evaluation Plan)

Red and Blue

Contract No: TBD

CONCURRENCE

SIGNATURE PAGE

This Award Fee Plan as prepared in accordance with FAR 16.401 under Contract No. TNBD has been coordinated and approved, as follows:

Concur

Contracting Officer Date

Concur:

Procurement Section Unit Chief Date

Program Manager Date

ASU Unit Chief Date

1. Introduction

a. This award-fee plan is the basis for the Red and Blue (REBL) Program’s subjective evaluation of the Contractor’s performance. It assists the decision of the Fee Determining Official (FDO) by presenting a structure to that assessment. The plan describes a clear process and specific criteria that will be used to evaluate the Contractor’s performance and, in doing so, to determine the amount of fee that will be awarded to the Contractor. The award-fee determinations and the methodology for making these determinations are unilateral decisions, made solely at the discretion of the Government. The methodology has been formulated to provide proper motivation for excellence in such areas as quality, timeliness, technical ingenuity, and cost-effective management.

b. The award free will be provided to the Contractor through contract modifications and is in addition to any fee associated with the Cost Reimbursement provisions of the contract. The award fee earned and payable will be determined by the FDO, based upon review of the Contractor’s performance against the criteria set forth in this plan. The Government has the flexibility to consider both the Contractor's performance levels and the conditions under which these levels were achieved during the evaluation period.

c. The FDO may unilaterally change this plan prior to the beginning of any evaluation period. The Contractor will be notified of changes to the plan by the Contracting Officer, in writing, before the start of the affected evaluation period. Changes to this plan that are applicable to a current evaluation period will be incorporated by mutual consent of both parties. Unless the Contracting Officer gives the contractor specific written notice of any changes to evaluation areas fifteen (15) days prior to the start of a new evaluation period, the same evaluation criteria and weights listed for the preceding period will be used in subsequent periods.

2. Organization

The award-fee organization consists of: the Fee Determining Official (FDO); an Award Fee Review Board (AFRB), which consists of a Chairperson, the Contracting Officer, a Recorder, Performance Monitors, and other functional area participants/advisors. The FDO, AFRB members, and performance monitors/advisors are listed in Annex 1.

3. Responsibilities

a. Fee Determining Official. The FDO approves the award-fee plan and any significant changes.

AFRB members are approved by the FDO. The FDO reviews the recommendation(s) of the AFRB, considers all pertinent data, and unilaterally determines the earned award-fee amount for each evaluation period.

b. Award Fee Review Board. AFRB members review Performance Monitors evaluation of the contractor’s performance, consider all information from pertinent sources, prepare interim performance reports, and arrive at an earned award-fee recommendation to be presented to the FDO. The AFRB may also recommend changes to the award fee plan. If the Contractor provides written or verbal self-assessment of its performance, the AFRB must consider this assessment when developing the earned award-fee recommendation to the FDO.

c. AFRB Recorder. The AFRB Recorder is responsible for coordinating the administrative actions required by the Performance Monitors, the AFRB and the FDO, including: 1) receipt, processing and distribution of evaluation reports from all required sources; 2) scheduling and assisting with internal evaluation milestones, such as briefings; 3) accomplishing other actions required to ensure the smooth operation of the award fee; and 4) scheduling FDO determination debriefing with the contractor.

d. Contracting Officer. The Contracting Officer is the liaison between Contractor and Government personnel.

e. Performance Monitors. Performance Monitors maintain written records of the contractor’s performance in their assigned evaluation area(s), so that a fair and accurate evaluation is obtained. Performance Monitors also prepare interim and end-of-period evaluation reports as directed by the AFRB.

4. Award Fee Processes

a. Available Award-Fee Amount. The available award fee for each evaluation period is shown in Annex 2. The award fee earned will be paid based on the Contractors’ performance during each evaluation period.

b. Evaluation Criteria. If the Contracting Officer does not provide specific notice in writing to the Contractor of any change to the evaluation criteria prior to the start of a new evaluation period, then the same criteria listed for the preceding period will be used in the subsequent award-fee evaluation period. The menu of possible criteria applicable to any evaluation period are delineated at Annex 3. The criteria chosen for an evaluation period will be commensurate with the phase of the REBL Program’s life-cycle.

c. Special Interest Item. During the life of the contract, the Government may want the Contractor to devote special emphasis to subjects of interest to the Government. Contractor performance in these areas would be subject to award-fee consideration; however no additional award fee will be added to the pool to cover these items.

d. Interim Evaluation Process. The AFRB Recorder will notify each AFRB member and Performance Monitor before the midpoint of the evaluation period. Performance Monitors will submit their evaluation reports to the AFRB. The AFRB will determine the interim evaluation results and notify the Contractor if deemed, by the Government, necessary to highlight area of Government concern.

e. End-of-Period. The AFRB Recorder will notify each AFRB member and performance monitor before the end of the evaluation period. Performance monitors will submit their evaluation reports to the AFRB after the end of the evaluation period. The AFRB will prepare its evaluation report and recommendation of earned award fee. The AFRB will brief the evaluation report and recommendation to the FDO. At this time, the AFRB may also recommend any significant changes to the award-fee plan for FDO approval. The FDO will determine the overall grade and earned award-fee amount for the evaluation period after each evaluation period. The FDO’s letter informs the Contractor of the earned award-fee amount. The Contracting Officer will issue a contract modification after the FDO’s decision is made authorizing payment of the earned-award-fee amount.

f. Contractor’s Self-Assessment. When the Contractor chooses to submit a self-assessment, it must be submitted to the Contracting Officer within five working days after the conclusion of the evaluation period. This written assessment of the Contractor’s performance throughout the evaluation period may contain any information that may be reasonably expected to assist the AFRB in evaluating the Contractor’s performance. The Contractor’s self-assessment may not exceed 10 pages, in total.

5. Award-Fee Plan Change Procedure

All significant changes to the plan, including methodology and performance criteria will be approved by the FDO; the AFRB Chairperson will be the approving official for all other changes. Examples of significant changes include changing evaluation criteria, adjusting weights to redirect the Contractor’s emphasis to area needing improvement, and revising the distribution of the award-fee dollars. The Contractor may recommend changes to the Contracting Officer no-later-than thirty-days (30) prior to the beginning of the new evaluation period. The Contracting Officer will notify the Contractor in writing of any changes(s). Unilateral changes may be made to the award-fee plan, if the Contractor is provided written notification by the Contracting Officer before the start of the upcoming evaluation period. Changes affecting the current evaluation period must be by mutual agreement of both parties.

6. Contract Termination

If the contract is terminated for the convenience of the Government after the start of an award-fee evaluation period, the award fee deemed earned for the period shall be determined by the FDO using the normal award-fee evaluation process. After termination for convenience, the remaining award-fee amounts allocated to all subsequent award-fee evaluation periods cannot be earned by the Contractor and, therefore, shall not be paid.

5 Annexes

1. Award-Fee Organization

2. Award-Fee Allocation by Evaluation Periods

3. Rating Definition and Evaluation Criteria

4. Point Distribution and Weights

5. List of Acronyms

Annex 1 – Award Fee Organization

Members:

Fee-Determining Official: (Position Title)

Award-Fee Review Board Chairperson: (Position Title)

Award-Fee Review Board Members:

Program Manager

Contracting Officer (Mandatory)

Recorder (Mandatory)

Office of General Council

Financial Management Representative

Performance Monitors:

Area of Evaluation Position Title

Program Management

Subcontract Management

Quality Assurance

Configuration Management

Engineering and Test Management

Cost and Schedule Management

Logistics

Annex 2 – Award-Fee Allocation by Evaluation Periods

The award fee earned by the Contractor will be determined at the completion of evaluation periods shown below. The percentage and dollars shown corresponding to each period is the maximum available-award-fee amount that can be earned during that particular period.

Evaluation Period* From To Available Award Fee**

First Through

Last Period

Total 100%

*The Government may unilaterally revise the distribution of the remaining award-fee dollars among subsequent periods. The Contractor will be notified of such changes, if any, in writing by the Contracting Officer before the relevant period is started and the award-fee plan will be modified accordingly. Subsequent to the commencement of a period, changes may only be made by mutual agreement of the parties.

** Will be computed in and expressed in dollars at conclusion of negotiation or in proposal and Final Price Revision using percentage shown.

Annex 3 – Rating Definition and Evaluation Criteria

Unsatisfactory Performance: Contractor’s performance relative to most Meets Criteria is inadequate and inconsistent. Quality, Responsiveness, and Timeliness on many areas require attention and action.

Corrective actions have not been taken or are ineffective.

Satisfactory Performance: Contractor’s performance relative to most Meets Criteria is adequate with few tangible benefits to the Government due to contractor’s effort or initiative. Although there are areas of good or better performance, these are more or less offset by lower-rated performance in other areas.

Good Performance: Contractor’s performance relative to most Meets Criteria is better than adequate and provides some tangible benefits to the Government in several significant areas. While the remainder of the contractor’s effort generally conform to the Meets Criteria, areas requiring improvement are more than offset by the better performance in other areas.

Very Good Performance: Most of the Contractor’s performance consistently meets the Exceeds Criteria, providing numerous significant tangible and intangible benefits to the Government (e.g., improved quality, responsiveness, increased timeliness, or generally enhanced effectiveness of operations).

Although some areas may require improvement; these areas are minor and are more than offset by better performance in other areas. Few, if any, recurring problems have been noted, and contractor takes satisfactory corrective action.

Excellent Performance: Virtually all of the Contractor’s performance consistently meets the Exceeds Criteria and provides numerous significant, tangible or intangible, benefits to the Government. The few area for improvement are all minor. There are no recurring problems. Contactor’s management initiates effective corrective action whenever needed.

SOO Paragraph

Performance Measure Criteria

4.1 General Contract Functions

4.1.1 Expert Knowledge of Cyber Threats.

Contractor personnel and written products demonstrate an expert knowledge of the threats that seek to target FBI missions, systems, networks, and information. The Contractor effectively and efficiently leverages this expertise to continuously improve services and to make relevant and practical recommendations to identified risk.

Meets Criteria: Customers, partners and stakeholders recognize the expertise of Contractor personnel with respect to both insider and external threats, to include the TTPs utilized by a threat to achieve an objective. The Contractor effectively and efficiently leverages its knowledge of threats to deliver high quality, relevant, and comprehensive cybersecurity services.

Exceeds Criteria: Contractor personnel leverage their threat expertise to continuously seek improvement to the Contractor, improve the tools and techniques used to deliver cybersecurity services, and to provide creative solutions to security challenges.

4.1.2 Applicability to FBI.

The Contractors personnel and written products demonstrate knowledge of the FBI. The Contractor effectively and efficiently leverages its knowledge of the FBI to continuously improve the Unit’s cybersecurity services and to make relevant and practical recommendations to identified risks.

Meets Criteria: Customers, partners, and stakeholders recognize Contractor personnel as knowledgeable with regard to the FBI technology environment and the mission of the FBI.

Exceeds Criteria: Contractor personnel rapidly develop their institutional knowledge of the FBI, using historical data, newly created data, and internal partnerships to create effective working relationships with key areas of the FBI that ensure their services remain relevant and valuable.

4.1.3 Technical & Security Recommendations.

Contractor personnel demonstrate and written products reflect an expert knowledge of the Meets Criteria: Customer, partners, and stakeholders have confidence in the technical and security expertise of technologies used to build, operate, secure and monitor complex global classified and unclassified networks.

Contractor personnel, in their recommendations, and in how issues identified by the Contractor impact an FBI mission.

Exceeds Criteria: Specialized skillsets are secured as needed to effectively and efficiently meet unique or critical technical or security challenges.

4.1.4 Initial and Full Operating Capabilities.

The Contractor coordinates, plans, and delivers an agreed upon set of capabilities and deliverables in an agreed upon timeframe, demonstrating sound project and Contractor management principles.

Meets Criteria: The Contractor reaches its operational goals and is ready to provide a full set of services to customers, within budget and at the agreed upon time.

Exceeds Criteria: The Contractor is able to rapidly offer a limited set of services as an initial operating capability and achieves full operating capability ahead of schedule and within budget.

4.1.5 Consistency of Operations.

The Contractor demonstrates a consistent quality of service, maintaining or improving upon capabilities over time.

Meets Criteria: The Contractor recruits, trains, and retains a diverse team of individuals capable of meeting the Contractors objectives. Contractor capabilities are enhanced overtime, ensuring the FBI’s security needs are being met on an ongoing basis.

Exceeds Criteria: Federal oversight and individual team members recognize the Contractor as a superior team, exceeding expectations for America’s premier security and crime-fighting force.

4.2 Red Team Functions

4.2.1 Threat Emulations.

Red Team leadership and operators demonstrate an expert knowledge of offensive tools, tactics, techniques, and procedures relative to the threats

Meets Criteria: Customers, partners, and stakeholders have a high degree of confidence in the quality of services provided and that there are no significant false being emulated and the environment in which the operations are being conducted.

positives or false negatives following a Red Team operation. Targeted operations are realistic emulations of threats, take into consideration the environment in which the operations are being undertaken, FBI mission and information likely targeted, and likely TTPs used by threats within this environment.

Exceeds Criteria: The Contractor establishes and leverages internal partnerships to provide a realistic emulation of threats and uses its knowledge of threats and the FBI to recommend creative approaches to delivering useful Red Team services.

4.2.2 Cohabitation and Incident Response.

Operations effectively integrate threat hunting measures to identify threat activity and assess cohabitation, i.e. the existence of an actual threat on a system targeted during a Red Team operation.

Meets Criteria: Threat hunting techniques are efficiently and effectively incorporated into Red Team operations, where appropriate. Red Team personnel have a clear understanding of incident response and forensics best practices to ensure potential evidence is not contaminated and proper incident handling procedures are followed when/if threat activity is discovered.

Exceeds Criteria: Red Team operators implement creative hunting solutions that do not compromise the overall objective of the operation.

4.2.3 Continuous Operations.

The Contractor operates a continuous Red Team function that includes general operations, which seek to continuously identify high-risk vulnerabilities across the FBI, and tailored operations, which are true emulations of threats.

Meets Criteria: The Contractor is effective and efficient at providing continuous Red Team operations across the FBI, considering both insider and external threats.

Contractor personnel provide expert input and advice on approaches and objectives for general operations, which are operations conceived of and initiated by the Contractor in coordination with FBI leadership. The performance of tailored operations, those initiated by FBI leadership, are measured under 3.2.1. The Contractor is effective at communicating time-sensitive countermeasure recommendations to the appropriate area of the FBI.

Exceeds Criteria: The Contractor is able to rapidly adjust to changes in technology or threat landscape, and is seen as a leader an innovator by customers, stakeholders, and partners in the delivery of Red Team operations.

4.3 Blue Team Functions

4.3.1 Comprehensive Assessment.

Blue Team leadership and operators demonstrate expertise in effectively and efficiently identifying vulnerabilities on complex, global networks that operate at all levels of classification; recommended mitigations strategies reflect a solid understanding of the threats, the operating environment, and the risk to mission.

Meets Criteria: Customers, partners, and stakeholders are confident that Blue Team operations are comprehensive in achieving their objectives;

• Fully utilizing direct access to systems and personnel,

• Leveraging multiple subject matter experts to ensure all aspects of a system are assessed,

• Use the best tool or technique for the task at hand, effectively and efficiently assessing the technologies in use.

• Demonstrate a solid understanding of customers’ mission and threats to that mission.

Exceeds Criteria: The services and written products provided by the Blue Team are recognized by Federal oversight as superior in quality and serve as a benchmark among the Federal Government and Intelligence Community for the performance of operational assessments.

4.3.2 Incident Response and Threat Hunting.

The Blue Team demonstrates a solid understanding of incident response best practices and incident response at the FBI; supporting, facilitating, or performing incident response activities to help investigate and resolve security events.

Meets Criteria: Customers, partners and stakeholders are confident in the skills and abilities of Blue Team operators deployed in response to an incident. The Contractor effectively leverages existing capabilities within the FBI, such as forensics and security monitoring capabilities, to facilitate, support and/or perform incident response activities.

Exceeds Criteria: The incident response services provided by the Blue Team are recognized by Federal oversight as superior in quality, meeting expectations for one of the world’s premier security and crime-fighting force.

Further, Blue Team support is sought in support of counter-cyber operations.

The Blue Team effectively and efficiently incorporates threat hunting tools and techniques into operational security assessments.

Meets Criteria: Threat hunting techniques are efficiently and effectively incorporated into Blue Team operations, where appropriate.

Exceeds Criteria: Federal oversight recognizes the superior quality of Blue Team assessments; identifying not only risk but recognizing when a system has been potentially compromised by often highly sophisticated threats.

4.4

Mitigations and Support Functions

4.4.1 Cybersecurity Remote Operations Center and Flyaway Kits.

The CyROC enables the Unit to effectively and efficiently provide remote Blue and Red Team services at all classification levels.

Meets Criteria: The CyROC and flyaway kits are designed, developed, documented, and operated in a manner that effectively and efficiently enables the Contractor’s remote and deployed cybersecurity operations and services. All equipment is operated in a secure and efficient manner, adhering to FBI security requirements and security best practices. The Contractor maintains a robust toolbox of customized, open-source, commercial and GOTs tools and technologies.

Exceeds Criteria: The CyROC and flyaway kits are recognized by Federal oversight as a superior example for enabling efficient and effective remote and deployed Red and Blue Team services.

4.4.2 Customized Research.

Individuals collecting information during operations and conducting customized research provide up-to-date, accurate, and relevant information and analysis in support of the Contractor.

Meets Criteria: The Contractor uses all resources at its disposal, including classified and unclassified sources.

Researchers actively participate in Red and Blue Team operations to document findings, research potential mitigation strategies as operations are ongoing, and follow-up with customers to support mitigation efforts.

Customers, partners, and stakeholders have confidence in the research performed.

Exceeds Criteria: The research and analysis services provided to the Contractor are recognized as a superior example, supporting efficient and effective cybersecurity services.

4.4.3 Unit Operations/Coordination.

The Contractor coordinates all requests for services in a professional and organized manner, maintaining and overseeing individual project schedules, and advocating for FBI customers throughout their interaction with the Contractor.

Meets Criteria: Contractor’s customers, partners, and stakeholders are satisfied with the quality of service they receive from the Contractor and their questions, concerns, and comments are being addressed in a timely, and professional manner.

Exceeds Criteria: Customers, stakeholders, and partners recognize the Contractors operations management and customer service as a superior.

4.4.4 Documentation, Program Media, and Written Products.

The Contractor clearly and effectively documents standard operating procedures, business processes, and methodologies; all documentation is maintained over time. Written products, including documentation and other written materials, effectively and efficiently satisfy the requirements of the target audience.

Meets Criteria: Contractor internal documents are concise, clear, professional, and accurately reflect the Contractor’s operations. SOPs enable personnel to carry out operations in a consistent manner, following a defined set of requirements and standards. Contractor products are delivered in a timely manner, are professional, free of spelling and grammar errors;

customers, partners, and stakeholders are satisfied with the quality of the material. Products use an appropriate mixture of media, visual aids, data, and prose to ensure information is communicated effectively to the target audience.

Exceeds Criteria: Federal oversight recognizes the superior quality of the Contractors documentation, media, and written products, which portray the Contractor in professional manner.

4.5

Enterprise Compliance and Continuous Monitoring Functions

4.5.1 Security Assessment and Authorization (SA&A) and Information Assurance Support.

Contract personnel demonstrate an expert knowledge of all relevant security policies and regulations to include DOJ, FISMA, and ICD 503 policies; and provide clear, consistent, efficient and accurate advice and services with regard to assessing a systems compliance with these policies.

Meets Criteria: Advice and assistance is timely, professional, comprehensive, relevant, and accurate.

Customers, partners, and stakeholders are confident in the advice and assistance they receive from the Contractor.

Exceeds Criteria: The services and written products provided are recognized by Federal oversight as superior in quality and serve as a benchmark among the Federal Government and Intelligence Community.

4.5.2 Technical Security Assessment.

Contract personnel demonstrate expertise in applicable security controls, are effectively and efficiently performing technical security assessments using a standard set of tools, and are documenting and recording their results in a manner that meets the customer’s needs.

Meets Criteria: Assessments are professional, timely, accurate, efficient, and meet the customer’s needs.

Customers, partners, and other stakeholders are confident in the results and advice received from the Contractor.

Exceeds Criteria: The services and written products provided are recognized by Federal oversight as superior in quality and serve as a benchmark among the Federal Government and Intelligence Community.

4.5.3 Enterprise Vulnerability Scanning & Reporting.

Contract personnel demonstrate expertise in providing automated and manual enterprise vulnerability assessment services to the enterprise in support compliance and vulnerability management.

Meets Criteria: Assessments are professional, timely, accurate, efficient, and meet the customer’s needs.

Customers, partners, and other stakeholders are confident in the results and advice received from the Contractor.

Exceeds Criteria: The services and written products provided are recognized by Federal oversight as superior in quality and serve as a benchmark among the Federal Government and Intelligence Community.

4.5.4 Enterprise Vulnerability Scanning System.

The solution developed meets the FBI’s enterprise vulnerability scanning needs, is designed to be adaptable to shifts in technical and threat environments, and is cost-effective to build, operate and maintain over time.

Meets Criteria: The solution is designed, developed, documented, and operated in a manner that effectively and efficiently enables the FBI to conduct regular vulnerability scans and basic security hygiene across the enterprise. All equipment is operated in a secure and efficient manner, adhering to FBI security requirements and security best practices. The solution takes advantage of existing FBI/DOJ/ODNI licenses, infrastructure, and resources where appropriate and is integrated into security reporting/tracking systems.

Exceeds Criteria: The solution is recognized by Federal oversight as a superior example for enabling efficient and effective enterprise vulnerability assessments, basic security hygiene, and continuous diagnostics and mitigation.

Annex 4 – Point Distribution and Weights

Sub-Area Weight Monitor Rating Recommended Point Score Technical 50% Area 1 45% Satisfactory Area 2 35% Excellent Area 3 20% Good Management 30% Area 1 50% Very Good Area 2 50% Good Cost 15% Satisfactory Special Interest

5% Excellent

Standard Description

Range of Rating Points Percentage Award Fee

Excellent 91-100 91-100% Very Good 76-90 76-90% Good 61-75 61-75% Satisfactory 51-60 0 Unsatisfactory 1-50 0

Annex 5 – List of Acronyms

AFRB Award-Fee Review Board

AP Acquisition Plan

CO Contracting Officer

CPAF Cost-Plus-Award-Fee

FAR Federal Acquisition Regulation

FDO Fee-Determining Official

PM Performance Monitor

PWS Performance Work Statement

QAP Quality Assurance Personnel

R&M Reliability and Maintainability