RFP_Attachment_3_FA1_PWS.pdf

RFP Attachment 3 – FA1 PWS

RFP # HQ0034-16-R-0258

Department of Defense

Crowdsourced Vulnerability Discovery & Disclosure Services

Functional Area 1

Performance Work Statement

August 9, 2016

TOC

TABLE OF CONTENTS

1. Project Title

2. Background

3. Purpose & Objective

4. Scope

5. General Requirements

5.1. Subcontract & Researcher Management

5.2. Place of Performance

6. Performance Requirements

6.1. Transition Activities

6.2. Public Vulnerability Discovery & Disclosure Platform

6.3. Host DoD’s Public Vulnerability Disclosure Policy

6.4. Experience

6.5. Task Execution

6.6. Preparation Phase

6.7. Challenge Phase

6.8. Post-Challenge Phase

6.9. Duration of All Phases

7. SPECIAL REQUIREMENTS

7.1. Government Furnished Materials

7.2. Security

7.3. Documentation

7.4. DoD Intellectual Property

7.5. Background Checks

7.6. Deliverables

RFP # HQ0034-16-R-0258 Page 1

PERFORMANCE WORK STATEMENT

Functional Area (FA) 1

1. PROJECT TITLE

This document provides the Performance Work Statement (PWS) for Functional Area (FA) 1 of the Indefinite Delivery/Indefinite Quantity (IDIQ) contract vehicle for Department of Defense

Crowdsourced Vulnerability Discovery & Disclosure Services.

2. BACKGROUND

The DoD’s computer networks and systems support the nation’s defense and are crucial both for daily business operations and mission-critical activities. Maintaining the security and integrity of the DoD’s networks and systems is a matter of national security and requires the continuous identification and remediation of vulnerabilities that can be exploited by malicious cyber actors.

As part of its responsibility to the public at large, DoD is constantly considering innovative and diverse approaches to meet this goal. To support DoD’s continual efforts to remain at the forefront of rapidly evolving technologies, and to maintain the highest levels of integrity and security required of its IT infrastructure, DoD has identified an emerging need to leverage a diverse pool of innovative information security researchers (herein referred to as “researcher”), via crowdsourcing, for vulnerability discovery, coordination, and disclosure activities.

Crowdsourcing is a modern business practice that, as of 2010, the Federal government has employed to obtain needed services, ideas, or content by soliciting contributions from a large group of people rather than from traditional employees or suppliers. Crowdsourcing incentivizes innovation in solving mission-centric problems. Remaining ahead of present and emerging cyber threats is a significant responsibility in any environment. For DoD the responsibility is amplified as the repercussions associated with security failure are severe.

3. PURPOSE & OBJECTIVE

With the assistance of the private sector, the Government intends to use existing commercial crowdsourcing expertise and best practices, tailored for the Government’s use, to support the

DoD in the application of the crowdsourcing methodology to enhance its information security.

In support of this objective, the DoD intends to partner with a commercial firm that has extensive experience with administering crowdsourced vulnerability discovery and disclosure activities as a service. Under the resulting task order, the firm will host DoD bug bounties and/or other crowdsourced security activities on behalf of the DoD.

4. SCOPE

For this first challenge, the Office of the Secretary of Defense intends to conduct crowdsourced vulnerability discovery and disclosure services against several websites and their subdomains, owned by one of the Military Departments that has both internal and external components. The website has static content and dynamic fields with Human Resource data in the backend (as well as other interactive customer engagement features (i.e., chat, user accounts). The challenge will be opened to all U.S. persons but limited to 400 registrants, preferably recruited based upon expertise by the contractor. Additional details about the specific domain will be provided to the

IDIQ FA1 awardee upon task order award.

RFP # HQ0034-16-R-0258 Page 2

5. GENERAL REQUIREMENTS

This section describes the general requirements for this effort. The following sub-sections provide details of various considerations.

5.1. Subcontract & Researcher Management

Contractors are encouraged to subcontract with entities to ensure full support of all required services as outlined in this PWS, if necessary. This could include additional triage capacity and security consultants for vulnerability validation, strategic communications, and penetration testing. Additionally, it is expected that individuals conducting the vulnerability discovery and disclosure are independent persons and entities.

Any firm with whom the Government contracts will be the sole point of contact between third party entities and the Government unless otherwise specified by the COR. The contractor will be responsible for the supervision of the vulnerability disclosure and assessment process, unless otherwise specified by the Government. Under no circumstances may the contractor represent to the independent third parties that the independent parties are working on behalf of DoD or that the independent third parties have privity of contract with the DoD.

The contractor shall be responsible for any subcontract management necessary to integrate work performed on this requirement and shall be responsible and accountable for subcontractor performance on this requirement. The prime contractor will manage work distribution to ensure there are no Organizational Conflict of Interest (OCI) considerations, as defined in FAR 9.5.

Researchers will be used to identify and submit vulnerability reports for the Government under this program. The contractor will manage the researchers and all communications necessary regarding vulnerability specifics between the Government and the contractor.

Certain DoD employees and other Federal Government employees are prohibited by law from receiving any financial reward. The contractor shall ensure no prohibited individuals, including these employees, receive financial compensation. A list of restrictions shall be provided by the

COR to the contractor for inclusion into the Security Page for this effort.

5.2. Place of Performance

All work will be performed at the contractor or subcontractor facilities. Work will not be performed at DoD facilities.

6. PERFORMANCE REQUIREMENTS

6.1. Transition Activities

Transition-In: At the commencement of the period of performance, the contractor is responsible, during the PREPARATION phase of the process, for the following transition-in activities:

o Identify appropriate triage team, and technical lead for the designated challenge.

Transition-Out: At the end of the task order period of performance, the contractor is responsible for the following transition-out activities:

RFP # HQ0034-16-R-0258 Page 3 o Deliver a final report that rolls up all data from end of phase status reports, including activity metrics and coverage analytics, researcher vulnerability discovery and remediation metrics, vulnerability submission volume, triage speed, signal-to-noise ratio, and patch efficiency.

o Delete all challenge-pertinent data, including vulnerability reports, eligibility criteria, and the challenge page, the last day of each task order’s period of performance.

6.2. Public Vulnerability Discovery & Disclosure Platform

The contractor must own and maintain a commercially available platform and have access to a diverse “crowd” of participants to facilitate vulnerability discovery and disclosure activities.

6.3. Host DoD’s Public Vulnerability Disclosure Policy

The contractor is expected to host the DoD’s public vulnerability disclosure policy, separate from any specific challenges conducted under FA1. Hosting the policy may involve use of the contractor platform to receive vulnerability reports, and may require use of an Application

Program Interface to export vulnerability reports received through the platform to other systems

(i.e., JIRA) or other dedicated vulnerability management or ticketing system. Triage personnel, or any other tools/services of the contractor, will not be required.

6.4. Experience

The contractor is expected to have at least two (2) years proven experience hosting crowdsourced vulnerability discovery and disclosure programs for large enterprises; have an extensive existing platform user base; have an ability to reach out to individual researchers based on their specific expertise to personally invite them to specific challenges; a robust public affairs and community outreach capability to conduct public outreach, manage the public perception of each challenge, conduct sensitive coordination with researchers, and answer queries in social media; have an existing user-friendly platform with automated services; experience in managing the coordination of vulnerability disclosure affecting third party organizations/vendors;

administering vulnerability discovery and disclosure challenges and awarding payments; and must have an existing, robust triage team capable of triaging and prioritizing reports on behalf of the Government within 48 hours of receipt.

6.5. Task Execution

For this first effort, the challenge phase itself will be a three (3) week duration. The total period of performance for this effort should not exceed three (3) months. All recent vulnerability assessments for the specific asset relative to this effort will be provided to the contractor by the

DoD.

This requirement falls under the domain of Functional Area 1 (FA1) of the DoD Crowdsourced

Vulnerability Discovery & Disclosure Services IDIQ Contract and involves enabling researchers to conduct remote, over the Internet crowdsourced vulnerability discovery and disclosure services against several Internet-accessible public-facing websites. The contractor’s platform will be the mechanism for:

Participants to apply to participate in the challenge and to submit vulnerability reports;

All communication between the contractor and participants;

The contractor to triage the reports and submit to the Government remediation team; and

RFP # HQ0034-16-R-0258 Page 4

The Government remediation team to communicate and coordinate with the contractor’s triage team.

This requirement will be “open” to U.S. persons who are not on the U.S. Department of

Treasury’s Specially Designated National’s List. That means that the contractor may open up the challenge to any interested researcher who is a U.S. person as defined by the Internal

Revenue Code Section 7701(a)(30). The identities of the individuals and the specific techniques used will be unknown to the Government. The contractor would be solely responsible for communicating and coordinating with the participants. The contractor would be expected to contract a third party commercial criminal background check firm to conduct background checks against eligible bounty winners before being awarded a bounty, per the Government’s requirements. At most, this will entail a commercial criminal background check, but will be confirmed during the PREPARATION PHASE of the task order’s period of performance.

For this FA1 task order, the Government expects researcher quality over quantity. Therefore, it is expected that the challenge will be time-boxed to three (3) weeks in total, and be capped at

400 researchers. The contractor is expected to recruit researchers with extensive experience and proven success in hacking websites and web applications.

For purposes of this requirement, the challenge will be divided between three (3) distinct phases.

The phases, PREPARATION, CHALLENGE, and POST-CHALLENGE, will vary in length as described below.

1. Preparation Phase in which the contractor conducts activities to tailor their existing platform for the challenge, invite and ensure researchers meet DoD’s eligibility criteria, define and then publish the challenge scope, handle press associated with each specific challenge, and define challenge rules and requirements. DoD expects the

PREPARATION PHASE not to exceed four (4) weeks.

2. Challenge Phase in which the contractor receives vulnerability reports that researchers submit to the platform, filters out duplicate and out of scope vulnerability reports, triages incoming reports for severity, assists the DoD remediation team in validating reports, and manages the coordination of third party disclosures. DoD expects the CHALLENGE

PHASE not to exceed three (3) weeks.

3. Post Challenge Phase in which the contractor coordinates with researchers and the government to ensure vulnerabilities have been closed, develops and posts a “Hall of

Fame” page, conducts a criminal background check, if required by DoD, manages the compensation, both financial and non-financial, to the vetted eligible researchers, and provides any final deliverables/metrics to the government on challenge performance.

DoD expects that POST CHALLENGE PHASE not to exceed four (4) weeks.

6.6. Preparation Phase

Publicize the challenge to ensure the best-suited researcher participation, given the limitations criteria, including only being open to U.S. persons, and limited to only 400 total researchers.

Develop a “registration page” hosted on the platform for interested researchers to apply to participate.

RFP # HQ0034-16-R-0258 Page 5

Invite and vet researchers, per DoD’s requirements (i.e., that they are not on the U.S.

Department of Treasury’s Specially Designated National’s List) including strategically recruiting specific researchers based on the necessary known skillsets.

Work with DoD Asset Owner, and other Tech Stakeholders, define the scope of the challenge.

Work with DoD Asset Owner, and other Tech Stakeholders, develop the rules and restrictions of each challenge.

Determine bounty amounts per vulnerability based on prior experience, industry best practices, and the previous red team results of the asset specified in the task order. While not required, it is anticipated that for most task orders, that approximately half of the task order price will pay financial rewards to the researchers..

Once finalized, publish the eligibility criteria, scope of the challenge, the rules and restrictions, and bounty amounts on the platform challenge specific page.

Tailor existing platform to the specific challenge environment as specified by the government.

Ensure adequate operational procedures are in place for triage personnel to coordinate and communicate with the researchers; for triage personnel to coordinate with various government remediation personnel; and for the remediation personnel to view and pull vulnerability report details.

6.7. Challenge Phase

Communicate vulnerability discovery and disclosure rules of engagement and legal parameters to participants.

Communicate vulnerability reporting standards and requirements to participants.

Use existing commercially available web platform to receive and aggregate vulnerabilities identified by researchers.

Ensure subcontractors and researchers adhere to rules and restrictions as consented to prior to registration and throughout the whole challenge.

Triage incoming vulnerability reports through both automated and manual techniques based on severity to identify submissions most impactful to the DoD asset owner and communicate and assign those vulnerabilities to the DoD remediation team based upon mutually agreed upon escalation policies.

Identify duplicate vulnerability reports, and filter out other reports that are ineligible or out of scope, preferably utilizing existing automation tools.

Engage with personnel responsible for operating, securing and defending the DoD asset on discovered vulnerabilities and facilitate communications between DoD personnel and subcontractors, independent persons or entities, and researchers.

Ensure that identified vulnerabilities are communicated securely to DoD, adhering to common international standards for the secure transmission of sensitive security data.

Have the technical capability (i.e., an Application Program Interface (API)) to export vulnerability reports received through the platform to other systems (i.e., JIRA) or other dedicated vulnerability management or ticketing system.

RFP # HQ0034-16-R-0258 Page 6

Assist the designated DoD remediation team with validating vulnerability reports.

Coordinate with third-party organizations that may be impacted by the disclosure of vulnerabilities.

Have the technical capability to pass the names of participants who identified vulnerabilities to a commercial firm to conduct a background check verifying the participant is not a convicted felon, terrorist, or otherwise ineligible to receive an award.

Coordinate with researchers and the designated DoD remediation team to ensure open vulnerability reports are adjudicated and closed out to the level of satisfaction of DoD personnel.

Ensure submitted vulnerability reports are complete, and contain a severity assessment, description, as many detailed reproductive steps as possible, and recommended remediation fix so DoD can remediate the vulnerability as soon as it is reported.

6.8. Post-Challenge Phase

Coordinate with researchers and the designated DoD remediation team to ensure open vulnerability reports are adjudicated and closed out to the level of satisfaction of DoD personnel.

Work with DoD to develop a final report specifying best practices and recommendations to inform future activities.

Manage and facilitate the secure, legal payment of monetary and non-monetary awards, to researchers for validated and qualifying vulnerability reports, leveraging industry standards.

Update the Hall of Fame page - thanking and ranking researchers for the vulnerabilities they submitted and/or comparable acknowledgement, which is contractor’s standard practice.

Collect and share relevant metrics from the challenge related to vulnerabilities as specified by the DoD Component; metrics may include types and volume of vulnerabilities reported; response/remediation times; or other metrics that may help improve the Component’s situational awareness and security. Compare metrics from this challenge to those of other challenges.

6.9. Duration of All Phases

The contractor will effectively communicate and coordinate with prospective as well as current researchers to ensure smooth user experience.

Communicate electronically with researchers at each stage of the vulnerability lifecycle, including initial receipt, remediation, and acknowledgement/reward.

Securely manage the storage and distribution of credentials to researchers to enable remote vulnerability discovery and disclosure activities against assets that require trusted relationships/connections.

Contractor will handle strategic messaging. No more than 24 hours shall pass, prior to relaying social media responses, for 95% of all challenge-related inquiries and/or negative posts from the researcher community.

RFP # HQ0034-16-R-0258 Page 7

Ensure that the vulnerability discovery and disclosure process is capable of adhering to common international standards for handling vulnerability data, such as ISO 29147 and

ISO 30111.

Deliver status reports at the end of the PREPARATION, CHALLENGE, AND POST-

CHALLENGE Phases.

7. SPECIAL REQUIREMENTS

This section describes the special requirements for this effort. The following sub-sections provide details of various considerations on this effort.

7.1. Government Furnished Materials

The Government will provide technical information, material and forms unique to the

Government for supporting the task. The DoD will provide the results of the most recent penetration test to the contractor prior to the Challenge Phase as available. Government unique information related to this requirement, which is necessary for contractor performance, will be made available to the contractor. The COR will be the point of contact for identification of any required information to be supplied by the Government. Government Furnished Materials also includes any information received during the challenge from Government employees.

7.2. Security

The contractor will have escorted access to DoD facilities for any physical meetings required and will complete Non-Disclosure Agreements (NDAs) for all contractor employees and subcontractor employees with access to vulnerability information. No additional security requirements apply to this effort.

7.3. Documentation

Documents, data files, reports, correspondence, and all other documents and writings, regardless of the medium (or media) by which they were produced, preserved, stored, or created in connection with or for purpose(s) of work performed under this PWS and task order, shall be delivered to the COR promptly, upon request, either via email or directly within the vulnerability reporting platform.

The contractor agrees to assume responsibility for protecting the confidentiality of Government records, which is not considered public information. Each contractor or employee of the contractor to whom information may be made available or disclosed shall be notified in writing by the contractor that such information may be disclosed only for purposes and to the extent authorized herein. The contractor shall not release any information related to this task order to the public, media or other unauthorized persons or organizations unless the government has conducted the appropriate security review and granted written approval.

7.4. DoD Intellectual Property

The contractor may be granted limited authority to use the official seal of the Department of

Defense (DoD), and the official seals of all DoD Components for limited purposes. These purposes include advertising the challenge on the registration and challenge pages in the platform. Use of all logos must be discontinued at the conclusion of each task order or the duration of the contract, unless provided with advanced written approval by the Contracting

RFP # HQ0034-16-R-0258 Page 8

Officer. Any contractor or subcontractor is not authorized to use the official seal of DoD and/or other DoD seals without written pre-approval for any purpose. The contractor is not authorized to create, publish, or distribute any physical or electronic material that contains both the official seal of DoD and the contractor’s name and/or logo.

7.5. Background Checks

The contractor will subcontract to a third party commercial firm to conduct criminal background checks, if required by the task order, in advance of bounty payment. Suitable commercial background check services must be able to identify any researcher convicted of a felony, or known terrorists.

7.6. Deliverables

Deliverables of the FA1 task order shall be submitted to the DoD Technical Monitor designated in the contract by the contractor Program Manager. All deliverables shall be submitted using

Microsoft Office suite of tools (e.g. MS Word, MS Excel, MS PowerPoint), or Adobe PDF format, unless otherwise specified by the Technical Monitor. Electronic submission shall be made via email, unless otherwise agreed to by the Contracting Officer’s Representative and the

Technical Monitor.

The COR has the right to reject or require correction of any deficiencies found in the deliverables. In the event of a rejected deliverable, the contractor will be notified in writing by the COR of the specific reasons for rejection.

The following table specifies the deliverables required for this effort:

PWS

Reference

Deliverable Title Delivery Time Frequency

5.7. Attendance by the Program Manager and

Technical Lead for a one day Kick-Off with DoD Tech Team and Asset Owner

During first week of the task order’s period of performance

1 time

6.6. With DoD’s assistance, develop the

registration page which includes eligibility criteria for registration and legal consent to participate

Immediately following Tech

Meeting

1 time

6.6. With DoD’s assistance, develop the scope

(all specific domains and subdomains) and rules and restrictions of the challenge for the challenge page

Immediately following Tech

Meeting

1 time

6.6. Develop bounty amounts, following

industry standard

No more than two weeks following kick-off

1 time

N/A Platform accounts developed for DoD

Remediation Team, Technical Monitor, COR, and designated DoD Tech personnel

Once mock-up of challenge page in platform is developed

1 time

6.6. Registration page posted Not more than three

weeks following kick-off meeting

1 time

RFP # HQ0034-16-R-0258 Page 9

6.6. Public statement (press release or blog)

with social media push to advertise challenge to community, if advisable to recruit sufficient researchers

In concert with registration page posted

1 time

6.6. Ensure all researchers meet DoD’s

requirements (i.e., are U.S. persons and not on the Specially Designated

National’s List) before the researcher may officially register. Turn away any researcher that does not pass vetting against eligibility criteria

No later than five business days before

Challenge Phase begins

1 time

6.6. List of usernames of accepted researchers

for the challenge

No later than 24 hours before

Challenge Phase begins

1 time

6.6. Challenge page posted and visible to

registered researchers

24 Hours before

Challenge Phase begins

1 time

6.7. Triaged vulnerability reports assigned to

DoD remediation team in platform within

48 hours of receipt

Duration of

Challenge Phase

Continually throughout the

Challenge

Phase

6.7. Within 72 hours, close out all

vulnerability reports following successful

DoD remediation

Duration of challenge phase

Continually throughout the

Challenge and

Post-Challenge

Phases

6.8. Develop and distribute any non-monetary

reward for participants

Not more than three weeks following end of challenge phase

First 3 weeks of post-challenge phase

6.8. Pay out all eligible monetary rewards Not more than three weeks following end of challenge phase

First 3 weeks of post-challenge phase

6.8. List of usernames that received a bounty

and how much they received for individual reports and in total if submitted multiple reports

Day after final financial rewards have been paid out

1 time

6.8. Update the “Hall of Fame” for all public

thanks to researchers who submit valid vulnerability reports for the duration of all task orders and post publically on platform

Day after final financial rewards have been paid out

1 time

6.9. End of each phase status report to include

any deliverables submitted during the phase, any metrics (i.e., how many registered researchers, vulnerability reports submitted, remediation metrics, Within five days of end of each phase

1 time per phase

RFP # HQ0034-16-R-0258 Page 10 signal-to-noise ratio, etc.) and any lessons learned from the phase necessary for DoD to know before moving on to next phase

6.8. Final Report that includes researcher testing

activity metrics; researcher vulnerability discovery and remediation metrics; and vulnerability triage metrics, to include, vulnerability submission volumes, triage speed, triage accuracy and signal-to-noise ratio; researchers that received a bounty;

and lessons learned from challenge

Final day of task order’s period of performance

1 Time

6.3. Host the DoD’s public vulnerability

disclosure policy

Within first week of contract and continuous throughout contract

Continually

8. PERFORMANCE REQUIREMENTS SUMMARY

Performance metrics for the FA1 PWS are detailed in the table below:

Task

Requirement Performance

Standard Acceptable Quality Level

Surveillance

Method

Designated

Program

Manager

Contractor has one single point of contact for

Government’s engagement for duration of the program

Contractor has designated one single point of contact to discuss deliverables, deadlines, and to establish the right team Contractor team for each specific task order

Staffing absence longer than two continuous weeks are filled with alternate assignment to cover workload

COR

review/government personnel feedback

Provide support personnel to meet each task order

PWS

requirements

All contractor personnel possess the skills needed to perform the required tasks as specified in the PWS

The contractor attracts and maintains a stable researcher base that effectively supports the task order requirements

All contractor personnel possess the needed skills before task order award, including technical, triage team, and strategic communications team, and program manager

The contractor is able to attract the minimum number of researchers needed for each task order at least 24

COR review / government personnel feedback

RFP # HQ0034-16-R-0258 Page 11 hours prior to each challenge commencing

If security requirements apply to the task order (i.e., signed NDAs), all documentation is submitted to the government in a timely manner so that no contract caused delays are apparent in commencing an FA1 challenge

The contractor's deliverables are complete and on time

Cost Effectiveness

Awards are paid to researchers at a level that ensures effective and consistent participation

Costs are in line with industry standard and are deemed reasonable on the task order cost proposal

COR review / government personnel and random review of researchers social media responses regarding this issue

Triage

Capabilities

Ability to triage and prioritize reports within 48 hours of receipt

The contractor’s personnel use a combination of automated and manual methods to triage vulnerability reports within

48 hours of receipt

COR review / government personnel feedback

Security of vulnerability disclosure

Vulnerabilities are securely disclosed to and managed through the platform

Contractor adheres to international standards for the security and transmission of vulnerability information, such as ISO 27001 and

29147

COR review / government personnel feedback

Strategic communications and ability to handle public relations

Contractor has experienced personnel capable of handling both discreet communications on sensitive issues and public communications

Contractor has experienced personnel that handle strategic messaging. No more than 24 hours pass prior to social media responses for 95% of all inquiries about challenge and negative posts from researcher community

COR review / government personnel feedback

Services and

Deliverables

The contractor provides the full range of technical and manual services required to support the requirements addressed in Section

The contractor provides competent expertise and analysis that is consistent with the PWS

Written deliverables are timely delivered, factual, well-written, 99% free of

Random review of work products by the

COR and Technical

Monitor

RFP # HQ0034-16-R-0258 Page 12

6 and 7.8. of the

PWS

grammatical errors or misspellings