RFP_Attachment_4_FA2_PWS_0001.pdf

RFP Attachment 4 – FA2 PWS

RFP # HQ0034-16-R-0258_0001

Department of Defense

Crowdsourced Vulnerability Discovery & Disclosure Services

Functional Area 2

Performance Work Statement

August 9, 2016

TOC

TABLE OF CONTENTS

1. Project Title

2. Background

3. Purpose & Objective

4. Scope

5. General Requirements

5.1. Subcontract & Researcher Management

5.2. Place of Performance

6. Performance Requirements

6.1. Transition Activities

6.2. Vulnerability Discovery & Disclosure Platform

6.3. Experience

6.4. Task Execution

6.5. Preparation Phase

6.6. Challenge Phase

6.7. Post-Challenge Phase

6.8. Duration of All Phases

6.9. Key personnel

7. SPECIAL REQUIREMENTS

7.1. Government Furnished Materials

7.2. Security

7.3. Documentation

7.4. DoD Intellectual Property

7.5. Background Checks

7.6. Deliverables

8. Performance Requirements Summary

RFP # HQ0034-16-R-0258 Page 1

PERFORMANCE WORK STATEMENT

Functional Area (FA) 2

1. PROJECT TITLE

This document provides the Performance Work Statement (PWS) for Functional Area (FA) 2 of the Indefinite Delivery/Indefinite Quantity (IDIQ) contract vehicle for Department of Defense (DoD) Crowdsourced Vulnerability Discovery & Disclosure Services.

2. BACKGROUND

The DoD’s computer networks and systems support the nation’s defense and are critical both for daily business operations and mission-critical activities. Maintaining the security and integrity of the DoD’s networks and systems is a matter of national security and requires the continuous identification and remediation of vulnerabilities that can be exploited by malicious cyber actors.

As part of its responsibility to the public at large, DoD is constantly considering innovative and diverse approaches to meet this goal. To support DoD’s continual efforts to remain at the forefront of rapidly evolving technologies, and to maintain the highest levels of integrity and security required of its IT infrastructure, DoD has identified an emerging need to leverage a diverse pool of innovative information security researchers (herein referred to as “researcher”), via crowdsourcing, for vulnerability discovery, coordination, and disclosure activities.

Crowdsourcing is a modern business practice that, as of 2010, the Federal government has employed to obtain needed services, ideas, or content by soliciting contributions from a large group of people rather than from traditional employees or suppliers. Crowdsourcing incentivizes innovation in solving mission-centric problems. Remaining ahead of present and emerging cyber threats is a significant responsibility in any environment. For DoD the responsibility is amplified as the repercussions associated with security failure are severe.

3. PURPOSE & OBJECTIVE

With the assistance of the private sector, the Government intends to use existing commercial crowdsourcing expertise and best practices, tailored for the Government’s use, to support the DoD in the application of the crowdsourcing methodology to enhance its information security.

In support of this objective, the DoD intends to partner with a commercial firm that has extensive experience with administering crowdsourced vulnerability discovery and disclosure activities as a service. Under the resulting task order(s), the firm will host crowdsourced security activities on behalf of the DoD.

4. SCOPE

The first FA2 Task Order will be to conduct private crowdsourced vulnerability discovery and disclosure activities against the source code and operational instantiation of one or more modules in a DoD file transfer capability. Although the solution includes dozens of distinct components, the scope of the first task order would include 1) approximately 200,000-500,000 lines of DoD contractor developed and maintained code owned by DoD, and 2) a live internal DoD application, accessible via the DoD Intranet. It is expected that DoD code will be hosted in the contractor’s platform, accessible only through DoD-specified authentication methods, and

RFP # HQ0034-16-R-0258 Page 2 available only to vetted researchers, triage personnel, and approved asset-owner personnel. The contractor will develop rules of engagement for conducting all testing activities.

Both the proprietary code and the software are sensitive Government assets. Therefore, the FA2 contractor will be required to maintain a private community of skilled and trusted researchers, diverse in skillset, and able to conduct both deep binary hacking, web-based attacks, reverse engineering, and network and system exploitation. The challenge phase itself will last three (3) weeks, and the total period of performance of the task order will not exceed six (6) months.

Any applicable unclassified vulnerability assessments and code reviews for the specific asset for this task order will be provided to the contractor by the DoD upon task order award. The code and access to the asset will be provided to the contractor upon task order award.

5. GENERAL REQUIREMENTS

This section describes the general requirements for this effort. The following sub-sections provide details of various considerations.

5.1. Subcontract & Researcher Management

Contractors are encouraged to subcontract with entities to ensure full support of all required services as outlined in this PWS, if necessary. This could include additional triage capacity and security consultants for vulnerability validation, strategic communications, and penetration testing. Additionally, it is expected that individuals conducting the vulnerability discovery and disclosure are independent persons and entities.

Any firm with whom the Government contracts will be the sole point of contact between third party entities and the Government unless otherwise specified by the COR. The contractor will be responsible for the supervision of the vulnerability disclosure, and assessment process, unless otherwise specified by the Government. Under no circumstances may the contractor represent to the independent third parties that the independent parties are working on behalf of DoD or that the independent third parties have privity of contract with the DoD.

The contractor shall be responsible for any subcontract management necessary to integrate work performed on this requirement and shall be responsible and accountable for subcontractor performance on this requirement. The prime contractor will manage work distribution to ensure there are no Organizational Conflict of Interest (OCI) considerations, as defined in FAR 9.5.

Researchers will be used to identify and submit vulnerability reports for the Government under this program. The contractor will manage the researchers and all communications necessary regarding vulnerability specifics between the Government and the contractor.

Certain DoD employees and other Federal Government employees are prohibited by law from receiving any financial reward. The contractor shall ensure no prohibited individuals, including these employees, receive financial compensation. A list of restrictions shall be provided by the Contracting Officer’s Representative (COR) to the contractor for inclusion into the Security Page for this effort.

5.2. Place of Performance

All work will be performed at the contractor or subcontractor facilities. Work will not be performed at DoD facilities.

RFP # HQ0034-16-R-0258 Page 3

6. PERFORMANCE REQUIREMENTS

6.1. Transition Activities

Transition-In: At the commencement of the period of performance, the contractor is responsible during the PREPARATION phase of the process, for the following transition-in activities:

o Identify appropriate triage team, and technical lead for the designated challenge.

o Identify researchers that may be best suited with adequate skillset, especially for source code challenge, to recruit.

Transition-Out: At the end of the task order period of performance, the contractor is responsible for the following transition-out activities:

o Deliver a final report that rolls up all data from end of each phase status reports, including activity metrics and coverage analytics, researcher vulnerability discovery and remediation metrics, vulnerability submission volume, triage speed, signal-to-noise ratio, and patch effectiveness and lessons learned from the challenge.

o Delete all vulnerability reports from the platform on the last day of each task order’s period of performance.

6.2. Vulnerability Discovery & Disclosure Platform

The contractor must own and maintain a platform with the technical capability to conduct full packet capture through its secure portal, and have access to a diverse “crowd” of participants to facilitate vulnerability discovery and disclosure activities.

6.3. Experience

The contractor is expected to have at least two (2) years proven experience hosting crowdsourced vulnerability discovery and disclosure programs for large enterprises; have an existing platform user base diverse in skillset; have an ability to reach out to individual researchers based on their specific expertise to personally invite them to specific challenges; have an existing user-friendly platform with automated services; experience in managing disclosure of third-party vulnerabilities, administering vulnerability discovery and disclosure challenges and awarding payments via the crowdsourcing techniques; must have an existing, robust triage team capable of triaging and prioritizing reports on behalf of the Government within 48 hours of receipt; must be capable of adhering to ISO 29147 and ISO 30111.

6.4. Task Execution

For this effort, the challenge phase itself will be a three (3) week duration. The total period of performance for this effort will not exceed six (6) months. All unclassified vulnerability assessments for the specific asset relative to this effort will be provided to the contractor by the DoD.

This requirement falls under the domain of Functional Area 2 (FA2) of the DoD Crowdsourced Vulnerability Discovery & Disclosure Services IDIQ Contract and involves enabling researchers

RFP # HQ0034-16-R-0258 Page 4 to conduct crowdsourced vulnerability discovery and disclosure activities against sensitive assets. The platform must include a secure portal for full packet capture capabilities to enable auditability and continuous monitoring of researcher activities.

The contractor’s platform will be the mechanism for:

Providing comprehensive vulnerability report triaging, validation, and prioritization within 48 hours of submission, and reporting to the DoD Remediation Teams to ensure it can patch the vulnerability as soon as possible;

Providing the secure portal through which all testing occurs with full packet capture capabilities to enable continuous monitoring and auditability;

Researchers to submit all vulnerability reports;

Participants to apply to participate in the challenge and to submit vulnerability reports;

Conducting all management and coordination with researcher community and project management and coordination with DoD Remediation Teams;

Coordinating the disclosure of vulnerabilities affecting third party organizations/vendors.

This requirement will be “closed” by invite only. It will be limited to U.S. persons, as defined by the Internal Revenue Code Section 7701(a)(30), who are not on the U.S. Department of Treasury’s Specially Designated National’s List, who have passed criminal background checks.

It is the sole responsibility of the contractor to communicate and coordinate with the researchers.

For this FA2 task order, the Government expects researcher quality over quantity. Therefore, it is expected that researcher capacity will be limited to no more than 50 of the most highly-skilled researchers. The contractor is expected to recruit researchers with extensive experience against these types of assets (e.g. static and dynamic testing and computer network exploitation).

For purposes of this requirement, the challenge will be divided between three (3) distinct phases.

The phases, PREPARATION, CHALLENGE, and POST-CHALLENGE, will vary in length as described below.

1. Preparation Phase in which the contractor conducts activities to tailor their existing platform for the challenge, invite and vet researchers, per DoD’s requirement, define and then publish the challenge scope, handle press associated with each specific challenge, and define challenge rules and requirements. DoD expects the PREPARATION PHASE not to exceed four (4) weeks.

2. Challenge Phase in which the contractor receives vulnerability reports that researchers submit to the platform, filters out duplicate and out of scope vulnerability reports, triages incoming reports for severity, assists the DoD remediation team in validating reports, and manages the coordination of third party disclosures. DoD expects the CHALLENGE PHASE not to exceed three (3) weeks.

3. Post Challenge Phase in which the contractor coordinates with researchers and the government to ensure vulnerabilities have been closed, manages the compensation, both financial and non-financial, to the researchers, and provides any final deliverables/metrics to the government on challenge performance. DoD expects that POST CHALLENGE PHASE not to exceed four (4) weeks.

RFP # HQ0034-16-R-0258 Page 5

6.5. Preparation Phase

Strategically recruit the best-suited researchers based on their proven experience, and their known skillset, given the scope of the challenge (source code, operational functionality).

Conduct criminal background checks on all researchers before granting them access to any DoD information.

Working with the DoD Asset Owner, and other Tech Stakeholders, develop the asset scope of the challenge.

Working with DoD Asset Owner, and other Tech Stakeholders, develop the specific technical parameters for the scope of the challenge.

Determine payment amounts for researchers based on contractor’s prior experience, and industry best practices. Socialize payment amounts to the researchers.

Once finalized, distribute scope of the challenge and the technical parameters (i.e., rules and restrictions) to the researchers.

Tailor existing platform to the specific challenge environment for researchers to report vulnerabilities, for triage personnel to coordinate and communicate with the researchers, and for the remediation team to view vulnerability report details.

6.6. Challenge Phase

Communicate vulnerability discovery and disclosure rules of engagement and legal parameters to researchers.

Communicate vulnerability reporting standards and requirements to researchers.

Conduct full packet capture of all researcher activities.

Integrate appropriate controls over researcher traffic, include a secure portal for full packet capture capabilities to enable auditability and continuous monitoring of researcher activities.

Flag improper, suspicious, or out-of-scope testing conducted by researchers for DoD.

Use existing platform to receive and aggregate vulnerabilities identified by researchers, and ensure vulnerability reports are of high-qualify enabling efficient remediation efforts.

Complement researcher efforts with automated testing tools for source code analysis, host and application scanning, and vulnerability analysis, if applicable.

Ensure subcontractors and security researchers adhere to rules and restrictions as consented to prior to registration and throughout the whole challenge.

Triage incoming vulnerability reports through both automated and manual techniques based on severity to identify submissions most impactful to the DoD asset owner and communicate and assign those vulnerabilities to the DoD Remediation Team based upon mutually agreed upon escalation policies.

RFP # HQ0034-16-R-0258 Page 6

Identify duplicate vulnerability reports, and filter out other reports that are ineligible or out of scope, preferably utilizing existing automation tools.

Ensure submitted vulnerability reports are complete, and contain a severity assessment, description, as many detailed reproductive steps as possible, and recommended remediation fix so DoD can remediate the vulnerability as soon as it is reported.

Engage with personnel responsible for operating, securing and defending the DoD asset on discovered vulnerabilities and facilitate communications between DoD personnel and subcontractors, independent persons or entities, and researchers.

Ensure all identified vulnerabilities are communicated securely to DoD, adhering to common international standards for the secure transmission of sensitive security data.

Have the technical capability (i.e., an Application Program Interface (API)) to export vulnerability reports into a variety of systems (i.e., JIRA), or other dedicated vulnerability management or ticketing system.

Assist the designated DoD Remediation Team with validating vulnerability reports.

Coordinate the disclosure of vulnerabilities that have a multivendor/multiparty impact, as necessary. Third-party suppliers of software or hardware technology that may be impacted by the disclosure of certain vulnerabilities may be common and require unique expertise.

6.7. Post-Challenge Phase

Coordinate with researchers and the designated DoD Remediation Team to ensure open vulnerability reports are adjudicated and closed out to the level of satisfaction of DoD personnel.

As appropriate, provide packet capture and other logs to DoD.

Write a final report as described in section 6.1 of this PWS.

Manage and facilitate the secure, legal payment of monetary and non-monetary awards to researchers for validated and qualifying vulnerability reports.

6.8. Duration of All Phases

The contractor will effectively communicate and coordinate with prospective as well as current researchers to ensure smooth user experience.

Communicate electronically with researchers at each stage of the vulnerability lifecycle, including initial receipt, remediation, and acknowledgement/reward.

Securely manage the storage and distribution of credentials to researchers to enable remote vulnerability discovery and disclosure activities against assets that require trusted relationships/connections.

Ensure that the vulnerability discovery and disclosure process is capable of adhering to common international standards for handling vulnerability data, such as ISO 29147 and

ISO 30111.

RFP # HQ0034-16-R-0258 Page 7

Deliver status reports at the end of the PREPARATION, CHALLENGE, and POST- CHALLENGE Phases.

6.9. Key personnel

The Program Manager must, at a minimum, be eligible to receive a U.S. Government-issued Secret security clearance. Although an existing clearance is preferred, it is not a requirement.

7. SPECIAL REQUIREMENTS

This section describes the special requirements for this effort. The following sub-sections provide details of various considerations on this effort.

7.1. Government Furnished Materials

The Government will provide technical information, material and forms unique to the Government for supporting the task. The DoD will provide the results of the most recent penetration test or security asset to the contractor prior to the Challenge Phase as available.

Government unique information, including software, source code, system configuration files, IP ranges, and other Government unique information related to this requirement, which is necessary for contractor performance, will be made available to the contractor on the first day of each task order period of performance. If access to services on any DoD internal networks (i.e., NIPRNet) is required, the Government will ensure any access is remotely available to the contractor in order to make the challenge successful. The COR will be the point of contact for identification of any required information to be supplied by the Government. Government Furnished Materials also includes any information received during the challenge from Government employees.

7.2. Security

The contractor will have escorted access to DoD facilities for any physical meetings required and will complete Non-Disclosure Agreements (NDAs) for all contractor employees and subcontractor employees with access to vulnerability information.

Although it is not likely that any classified material or assets will be tested during this contract, it may be useful for the Program Manager to understand background information that may be classified. The Program Manager, therefore, must, at a minimum, be eligible to receive a U.S.

Government-issued Secret security clearance. Although an existing clearance is preferred, it is not a requirement. No facility clearance is necessary for this contract.

7.3. Documentation

Documents, data files, reports, correspondence, and all other documents and writings, regardless of the medium (or media) by which they were produced, preserved, stored, or created in connection with or for purpose(s) of work performed under this PWS and contract, shall be delivered to the COR promptly, upon request, either via email or directly within the vulnerability reporting platform.

The contractor agrees to assume responsibility for protecting the confidentiality of Government records, which is not considered public information. Each contractor or employee of the Contractor to whom information may be made available or disclosed shall be notified in writing by the contractor that such information may be disclosed only for purposes and to the extent authorized herein. The contractor shall not release any information related to this contract to the

RFP # HQ0034-16-R-0258 Page 8 public, media or other unauthorized persons or organizations unless the government has conducted the appropriate security review and granted written approval.

7.4. DoD Intellectual Property

The contractor may be granted limited authority to use the official seal of the Department of Defense (DoD), or the official seals of all DoD Components for limited purposes in a task order.

If granted, use of all logos must be discontinued at the conclusion of each task order or the master IDIQ contract, unless provided advanced written approval by the Contracting Officer.

Without written approval, no contractor, or subcontractor, is authorized to use the official seal of DoD for any other purpose.

7.5. Background Checks

Conduct or confirm that criminal background checks on all researchers before granting them access to any DoD information. Additionally, have the capability to restrict researcher participation based on criteria set by DoD. This may include ensuring that researchers are U.S.

persons; are not felons; are not known terrorists, or an associate of a known terrorist or terrorist organization; are not listed on the U.S. Department of Treasury’s Specially Designated Nationals List; or otherwise ineligible to conduct work for DoD.

7.6. Deliverables

Deliverables of the FA2 task order shall be submitted to the DoD Technical Monitor designated in the contract by the Contractor Program Manager. All deliverables shall be submitted using Microsoft Office suite of tools (e.g. MS Word, MS Excel, MS PowerPoint), or Adobe PDF format, unless otherwise specified by the Technical Monitor. Electronic submission shall be made via email, unless otherwise agreed to by the Contracting Officer’s Representative and/or the Technical Monitor.

The Contracting Officer’s Representative has the right to reject or require correction of any deficiencies found in the deliverables. In the event of a rejected deliverable, the contractor will be notified in writing by the Contracting Officer’s Representative of the specific reasons for rejection.

The following table specifies the deliverables for this requirement:

RFP # HQ0034-16-R-0258 Page 9

PWS

Reference

Deliverable Title Delivery Time Frequency

5.4.

Attendance by the Program Manager and task order Technical Lead at a one day meeting with DoD Tech Team and Asset Owner at the DoD asset owner’s facility

During first week of contract award

1 time

6.5.

With DoD’s assistance, develop the asset scope of the challenge

Within two weeks of task order award

1 time

6.5.

With DoD’s assistance, develop challenge technical parameters of the challenge (i.e., rules and restrictions)

Within two weeks of task order award

1 time

6.5.

Develop financial award amounts, following industry standard

Within two weeks of task order award

1 time

N/A Platform accounts developed for DoD Remediation Team, Technical Monitor, COR, and designated DoD Tech personnel

Within three weeks of task order award

1 time

6.5.

Researchers recruited and vetted per DoD requirements before formal invitation

Within three weeks of task order award

1 time

6.5.

Rules and restrictions well communicated to registered researchers

24 hours prior to start of, and through the duration of the challenge phase

1 time

6.5 List of user names of accepted researchers for the challenge

No later than 24 hours before the Challenge Phase begins

1 time

6.6.

Triaged, validated, and prioritized vulnerability reports assigned to DoD Remediation Team in platform within 48 hours of receipt

Duration of challenge phase

Continually

6.6.

Close out all vulnerability reports following successful DoD remediation within 72 hours

Duration of three week challenge phase

Continually

6.7. Pay out all eligible financial rewards

Not more than three weeks following end of challenge phase

First 3 weeks of post-challenge phase

RFP # HQ0034-16-R-0258 Page 10

N/A

End of each phase status report to include any deliverables submitted during the phase, any metrics (i.e., how many registered researchers, vulnerability reports submitted, remediation metrics, signal-to-noise ratio, etc.) and any lessons learned from the phase necessary for DoD to know before moving on to next phase

Within five days of end of each phase

1 time per phase

6.7. Logs of researcher activities (e.g. PCAP)

Not more than one week following end of challenge phase

1 time

6.1.

Final Report that includes findings from automated testing; researcher testing activity metrics and coverage analytics;

researcher vulnerability discovery and remediation metrics; and vulnerability triage metrics, to include, vulnerability submission volumes, triage speed, triage accuracy and signal-to-noise ratio;

researchers that received an award; and lessons learned from challenge

Final day of task order’s period of performance

1 time

8. PERFORMANCE REQUIREMENTS SUMMARY

Performance metrics for the FA2 PWS are detailed in the table below:

Task Requirement

Performance Standard

Acceptable Quality Level Surveillance

Method

Designated Program Manager

Contractor has one single point of contact for Government’s engagement for duration of the program

Contractor has designated one single point of contact to discuss deliverables, deadlines, and to establish the right team Contractor team for each specific task order

Staffing absence longer than two continuous weeks are filled with alternate assignment to cover workload.

COR

review/government personnel feedback

Provide support personnel to meet

PWS

requirements

All contractor personnel possess the skills needed to perform the required tasks as specified in the PWS

All contractor personnel possess the needed skills before contract award, including technical and triage team, strategic communication team, and program manager

COR

review/government personnel feedback

RFP # HQ0034-16-R-0258 Page 11

The contractor attracts and maintains a stable researcher base that effectively supports the task order requirements

The contractor is able to attract the minimum number of researchers needed for each task order at least 24 hours prior to each challenge commencing

If security requirements apply to the task order, all documentation is submitted to the government in a timely manner so that no contractor caused delays are apparent in commencing a FA2 challenge

The contractor’s deliverables are complete and on time

Security of vulnerability disclosure

Vulnerabilities are securely disclosed to and managed through the platform

Contractor adheres to international standards for the security and transmission of vulnerability information, such as ISO 27001 and 29147

COR

review/government personnel feedback

Monitoring of researcher activities

Contractor provides the ability to monitor the activities of researchers

Contractor provides a secure portal with full-packet capture of researcher testing activities.

Contractor provides tools to identify specific behaviors

COR

review/government personnel feedback

Auditability of testing activities

Contractor provides the ability to audit the testing activities conducted through the TASKORD

Testing activities are conducted through a secure portal provided by the Contractor that logs researcher activities.

Contractor provides logs of activities after the Challenge phase, as well as metrics to provide insights into the testing activities

COR

review/government personnel feedback

Cost Effectiveness

Awards are paid to researchers at a level that ensures effective and consistent participation

Costs are in line with industry standard and are deemed reasonable on the task order cost proposal

Task Order cost proposal

Triage Capabilities

Ability to triage and prioritize reports within 48 hours of receipt

The Contractor’s personnel use a combination of automated and manual methods to triage vulnerability reports within 48 hours of receipt.

COR

review/government personnel feedback

RFP # HQ0034-16-R-0258 Page 12

Services and Deliverables

The Contractor provides the full range of technical and manual services required to support the requirements addressed in section 4 and 6.6 of the

PWS.

The Contractor provides competent expertise and analysis that is consistent with the PWS

Written deliverables are timely delivered, factual, well-written, 99% free of grammatical errors or misspellings

Random review of work products by the COR and Technical Monitor