RFP_Attachment_2_IDIQ_PWS.pdf

RFP Attachment 2 – IDIQ PWS

RFP # HQ0034-16-R-0258

Department of Defense (DoD)

Crowdsourced Vulnerability Discovery & Disclosure (CVDD) Services

Indefinite Delivery Indefinite Quantity (IDIQ)

Performance Work Statement

August 9, 2016

RFP # HQ0034-16-R-0258 Page 1

TABLE OF CONTENTS

1. Project Title

2. Purpose & Objective

3. Background

4. Scope

5. Transition Activities

6. General IDIQ Requirements

6.1. Non-Personal Services

6.2. Inherently Governmental Functions

6.3. Business Relations

6.4. Contract Management

6.5. Contract Administration

6.6. Subcontract and Independent Persons and Independent Persons/Entities Management.. 7

6.7. Contractor Personnel, Disciplines and Specialties

6.8. Travel

6.9. Place of Performance

6.10. Vulnerability Discovery & Disclosure Platform

6.11. Experience

6.12. Managing the Contract & Task Orders

6.13. Managing Researchers

6.14. Key Personnel

6.15. Host DoD’s Public Vulnerability Disclosure Policy (FA1)

6.16. Identifying Contractor Employees

7. Special Requirements

7.1. Government Furnished Materials

7.2. Security

7.3. Documentation

7.4. DoD Intellectual Property

7.5. Background Checks

7.6. 508 Compliance

8. Deliverables

9. Performance Requirements Summary

RFP # HQ0034-16-R-0258 Page 2

IDIQ

PERFORMANCE WORK STATEMENT

1. PROJECT TITLE

This document provides the Performance Work Statement (PWS) for the Indefinite Delivery/

Indefinite Quantity (IDIQ) contract vehicle for Department of Defense (DoD) Crowdsourced

Vulnerability Discovery & Disclosure Services.

2. PURPOSE & OBJECTIVE

The purpose of this procurement is to support the DoD in its efforts to leverage expertise and best practices from the private sector for utilizing the crowdsourcing methodology to enhance information security. The objective is, with the assistance of the private sector, to tailor existing commercial crowdsourcing practices for the Government’s use vice developing, internally, new

Government procedures. In support of this objective, the DoD intends to partner with commercial firms that have extensive experience with administering crowdsourced vulnerability discovery and disclosure activities as a service. Under the resulting contract(s), the successful firm will host DoD bug bounties and other crowdsourced security activities on behalf of the

DoD. The DoD seeks to use this contract vehicle(s) to conduct vulnerability discovery and disclosure activities against the broadest possible range of DoD assets. Activities under the resulting contract(s) may include testing of web applications, networks and systems, both internal and external-facing, including the full stack of associated devices and software, software-embedded devices, and platform information technology.

3. BACKGROUND

The DoD’s computer networks and systems support the nation’s defense and are crucial both for daily business operations and mission-critical activities. Maintaining the security and integrity of the DoD’s networks and systems is a matter of national security and requires the continuous identification and remediation of vulnerabilities that can be exploited by malicious cyber actors.

As part of its responsibility to the public at large, DoD is constantly considering innovative and diverse approaches to meet this goal. To support DoD’s continual efforts to remain at the forefront of rapidly evolving technologies, and to maintain the highest levels of integrity and security required of its IT infrastructure, DoD has identified an emerging need to leverage a diverse pool of innovative information security researchers (herein referred to as “researcher”), via crowdsourcing, for vulnerability discovery, coordination, and disclosure activities.

Crowdsourcing is a modern business practice that, as of 2010, the Federal Government has employed to obtain needed services, ideas, or content by soliciting contributions from a large group of people rather than from traditional employees or suppliers. Crowdsourcing incentivizes innovation in solving mission-centric problems. Remaining ahead of present and emerging cyber threats is a significant responsibility in any environment. For DoD the responsibility is amplified as the repercussions associated with security failure are severe.

Hack the Pentagon Pilot

The Government’s first bug bounty, “Hack the Pentagon,” was launched on April 18, 2016 and ran for 24 days under a test pilot program. Through this initiative, the researchers were provided with legal authority to use specific hacking techniques against DoD websites, and received financial awards for successfully submitting qualifying vulnerability reports. The pilot yielded

RFP # HQ0034-16-R-0258 Page 3 impressive results that greatly exceeded expectations. As such, the DoD's intent is to utilize the impending contract vehicle to capitalize on its success and continue to evolve the way it secures

DoD networks, systems, and information. Through the use of a firm that offers vulnerability discovery and disclosure as a commercial service, the Hack the Pentagon challenge was conducted against five websites and most of their sub-domains: defense.gov, dodlive.mil, dvidshub.net, myafn.net, and dimoc.mil. Each of the websites are operated by the Defense

Media Activity (DMA), who is responsible for approximately 700 DoD websites. The pilot utilized the firm’s web-based platform and services which automated vulnerability triage by identifying the reports of greatest concern and weeding out the "noise," i.e. findings that were less substantive or irrelevant, including duplicative reports. Historically, these functions represented thousands of labor hours.

Statistics

During the pilot effort, the firm assisted in recruiting 1,410 hackers for the Hack the Pentagon challenge. Over 250 of the recruited hackers submitted vulnerability reports. Ultimately, 138 reports were deemed valid security vulnerabilities, and 58 hackers were paid for their efforts.

4. SCOPE

The scope of work, under the resulting IDIQ contract vehicle, is to conduct crowdsourced vulnerability discovery and disclosure (CVDD) services across the full range of networks, systems, and information, including web applications, software, source code, and software-embedded devices across the whole Department of Defense. Work performed under the resulting IDIQ contract vehicle will be categorized under two functional areas as detailed below.

Functional Area 1 (FA1) activities may involve:

Although not relevant to hosting paid challenges, the contractor is expected to host

DoD’s public vulnerability disclosure policy;

Enabling researchers to conduct remote, Internet-based, crowdsourced vulnerability discovery and disclosure services against Internet-accessible assets, including public-facing websites, networks, systems, and applications that are Internet connected;

Small scope time-boxed crowdsourced efforts with less than 100 total participants or larger scaled crowdsourced efforts with up to 1,500 total participants;

Global, open-ended crowdsourced efforts open to anyone willing to participate; and

A robust public affairs and community outreach capability to conduct public outreach and sensitive coordination with researchers.

Under FA1, it is expected that there will be approximately eight (8) crowdsourced challenges over the course of the one (1) year period of performance (and can add up to twelve (12) over the next two years if option is exercised). There may be FA1 task orders with overlapping periods of performance, however, it is expected that there will only be one “challenge phase” occurring at a time. The period of performance is expected to vary per task order with an average duration of three (3) to six (6) months.

Functional Area 2 (FA2) activities may involve:

(Given the sensitive nature of the potential assets, participation would be invite-only and feature more limited participation than activities associated with FA1.)

RFP # HQ0034-16-R-0258 Page 4

Researchers conducting auditable crowdsourced vulnerability discovery and disclosure activities through a secure portal on the contractor’s platform against a variety of sensitive but Internet-connected assets, as well as non-Internet-connected assets.

Assets could include closed networks, software-embedded devices, proprietary source code, or other private or internal systems not generally accessible via the public

Internet;

Testing content hosted on contractor’s infrastructure, through its secure platform in a controlled environment or repository;

Testing content hosted by DoD through a secure contractor portal where IP addresses are logged and/or directly provisioned by the contractor and other data, potentially including keystrokes, are captured;

The platform must have a secure portal capable of full packet capture to enable auditability and continuous monitoring of researcher activities;

Small scoped and time-boxed crowdsourced efforts with less than 25 total participants, and larger crowdsourced efforts with closer to 200 total participants; and

Most activities under FA2 will include background check requirements, which will be provisioned by the contractor.

It is expected that there will be approximately six (6) challenges for FA2 during the first year of the program, and up to ten (10) challenges if the option year is exercised. Although there may be FA2 task orders with overlapping periods of performance, it is expected that there will only be one “challenge phase” occurring at a time. The period of performance is expected to vary per task order with an average duration of three (3) to eight (8) months.

Overlapping activities:

FA1 and FA2 have several overlapping activities that may involve:

Generation of high-quality vulnerability reports that enable DoD to efficiently remediate asset vulnerabilities;

Ability to create customized vulnerability workflow management and track vulnerabilities throughout the remediation lifecycle;

Assist the DoD Asset Owner in identifying and developing mitigation approaches for discovered vulnerabilities;

Provide comprehensive vulnerability triaging, validation, and prioritization within 48 hours of submission, and reporting to the DoD Asset Owner to ensure it can patch vulnerability as soon as feasible;

Ability to provide a means to easily export vulnerability reports to other systems

(JIRA, etc.) and synchronize vulnerability remediation statuses between multiple systems; and, Conduct all management and coordination with researcher community, and project management and coordination with DoD Remediation Teams.

Upon contract award, there will be a three week “Transition Phase” where the contractor implements key personnel, builds the initial triage team, and sets up billing and invoice accounts.

Immediately following the Transition Phase, the first challenges for FA1 and FA2 are anticipated to begin. Each challenge will be divided between three (3) distinct phases. The phases, PREPARATION, CHALLENGE, and POST-CHALLENGE, will vary in length and are

RFP # HQ0034-16-R-0258 Page 5 dependent on the scope of each challenge. Further details, specific to each phase, are listed in this section of this PWS.

Below, organized by phase, are contractor requirements relative to the crowdsourced services provided. Although the contractor may move an activity to different phases shall include:

1. PREPARATION PHASE. The contractor must:

Conduct activities to tailor their existing platform for the ensuing challenge, Conduct background checks, if necessary and indicated in each task order, Invite researchers per DoD’s requirement, Assist in defining and then publish the challenge scope, and

Assist in defining and then publish the challenge rules and requirements.

2. CHALLENGE PHASE. The contractor must:

Receive vulnerability reports that researchers submit to the platform, Filters out duplicate and out of scope vulnerability reports, Triage incoming reports for severity, Assist the DoD remediation team in validating reports, and

Manage the coordination of vulnerability disclosures that impact third party organizations or vendors.

3. POST CHALLENGE PHASE. The contractor must:

Coordinate with researchers and the designated DoD Remediation Team to ensure open vulnerability reports are adjudicated and closed out to the level of satisfaction of

DoD personnel, Conduct background checks*, if necessary and indicated in each task order, Manage the compensation, both financial and non-financial, to the researchers, and

Provide any final deliverables/metrics to the government on challenge performance.

4. DURATION OF ALL PHASES. The contractor must:

During the course of the whole period of performance, the contractor will effectively communicate and coordinate with prospective as well as current researchers to ensure smooth user experience, Communicate electronically with researchers at each stage of the vulnerability lifecycle, including initial receipt, remediation, and acknowledgement/reward, Securely manage the storage and distribution of credentials to researchers to enable remote vulnerability discovery and disclosure activities against assets that require trusted relationships/connections, Ensure that the vulnerability discovery and disclosure process is capable of adhering to common international standards for handling vulnerability data, such as ISO 29147 and

ISO 30111, and

RFP # HQ0034-16-R-0258 Page 6

Deliver status reports at the end of the PREPARATION, CHALLENGE, and POST-

CHALLENGE Phases. Deliver final report at the end of the task order.

*Background checks, if required by the task order, will only be necessary one time (in the Pre or

Post Challenge Phase, but not both)

5. TRANSITION ACTIVITIES

Transition (in and out) activities will be specified at the task order level. They will consist of activities that the contractor is expected to do in order to prepare for beginning work in the

PREPARATION PHASE and activities that the contractor is expected to do before ending performance under a task order.

6. GENERAL IDIQ REQUIREMENTS

This section describes the general requirements for this effort. The following sub-sections provide details of various considerations.

6.1. Non-Personal Services

In accordance with FAR 37.101, this contract is a non-personal services contract. Contractor personnel rendering the services shall not be subject, either by the contract's terms or by the manner of its administration, to the continuous supervision and control of a Government officer or employee. The ontractor shall immediately notify the Contracting Officer’s Representative and the Contracting Officer if, through contract administration, the actions of a Government employee will result in the performance of a personal services contract.

6.2. Inherently Governmental Functions

No inherently governmental functions as defined in FAR 2.101 and FAR 7.5 shall be performed by the contractor under this contract. Contractor employees shall not participate in any deliberations or meetings intended to exercise an inherently governmental function. All final determinations such as binding the United States to take or not to take some action, selecting program priorities, and providing direction to Federal employees shall be made by the government. The contractor shall immediately notify the Contracting Officer’s Representative and the Contracting Officer if performance of an activity would result in the performance of an inherently governmental function.

6.3. Business Relations

The contractor shall successfully integrate and coordinate all activity needed to execute the requirements specified herein and at the task order level. The contractor shall manage the timeliness, completeness, and quality of the contract and task order deliverables. The contractor shall provide corrective action plans, proposal submittals, timely identification of issues, and effective management of all contractor personnel (including subcontractors). The contractor shall seek to ensure customer satisfaction and professional and ethical behavior of all contractor personnel.

6.4. Contract Management

The contractor shall establish clear organizational lines of authority and responsibility to ensure effective management of the program and the resources assigned to this requirement. The

RFP # HQ0034-16-R-0258 Page 7 contractor must maintain continuity between the support operations at DoD and the contractor's corporate offices.

6.5. Contract Administration

The contractor shall establish processes and assign appropriate resources to effectively administer the IDIQ contract and each subsequent task order. The contractor shall respond to

Government requests for contractual actions in a timely fashion. The contractor shall have a single point of contact (herein referred to as the “Program Manager”) between the Government and contractor personnel assigned to support this contract. Additional detail in Section 6.14.

“Key Personnel” of this PWS.

6.6. Subcontract and Independent Persons and Independent Persons/Entities Management

Contractors are encouraged to subcontract with entities to ensure full support of all required services as outlined in this master PWS.

Any firm with whom the Government contracts will be the sole point of contact between third party entities and the Government unless otherwise specified by the Contracting Officer’s

Representative (COR). The contractor will be responsible for the supervision of the vulnerability disclosure, and assessment process, unless otherwise specified by the Government. Under no circumstances may the contractor represent to the independent third parties that the independent parties are working on behalf of DoD or that the independent third parties have privity of contract with the DoD.

The contractor shall be responsible for any subcontract management necessary to integrate work performed on this requirement and shall be responsible and accountable for subcontractor performance on this requirement. The prime contractor will manage work distribution to ensure there are no Organizational Conflict of Interest (OCI) considerations.

6.7. Contractor Personnel, Disciplines and Specialties

Successful performance under each task order will include the responsiveness of contractor personnel in the day-to-day output of work products. While the end product is vital to successful performance, day-to-day oversight also includes client interaction and responsiveness.

Accordingly, the contractor is required to proactively maintain assigned tasks, and be responsive to all entities with professional business relative to the assigned tasks.

The contractor must at all times maintain an adequate work force for the uninterrupted performance of all tasks defined within the respective task order PWS. When hiring personnel, the contractor shall keep in mind that the stability and continuity of the workforce are essential.

6.8. Travel

Work required by this PWS shall primarily take place remotely at the contractor’s facilities.

Travel to the DoD asset owner’s facility during the PREPARATION PHASE of each challenge during the duration of the IDIQ contract is necessary for a one (1) full day workshop with DoD

Tech Stakeholders to determine and finalize the scope of the challenge. This facility will be located within the continental United States. Travel shall be approved in advance by the

Contracting Officer’s Representative. Travel shall not commence until written approval is received.

RFP # HQ0034-16-R-0258 Page 8

The contractor shall be reimbursed for actual allowable, allocable, and reasonable travel costs incurred during performance of this effort in accordance with the FAR 31.205-46 “Travel

Costs.” Requests for approval of costs in excess of maximum per diem rates in accordance with the procedures contained in FAR 31.205-46(a)(3) must be submitted to the Contracting Officer for final approval prior to commencement of travel.

6.9. Place of Performance

All work will be performed at the contractor or subcontractor facilities. Work will not be performed at DoD facilities.

6.10. Vulnerability Discovery & Disclosure Platform

The contractor must own and maintain a platform to facilitate vulnerability discovery and disclosure activities relative to FA1 and FA2 . Platform requirements, respective to FA1 and FA2 are stipulated below:

FA1 Platform Requirements

Platforms, conducting work under FA1, must have:

The capability to accept vulnerability reports from researchers.

The capability to apply tools and processes, automated as well as manual, to triage reports for the Government. This includes de-duplication of reports within 48 hours of receipt.

The capability to ensure that vulnerability reports, transmitted to Government remediators, are clear and of high quality. This will ensure that Government personnel can immediately remediate identified vulnerabilities.

The capability to facilitate effective communication between the triage team and researchers and between the triage team and Government remediators. This may include corresponding, separately, with multiple teams.

The capability to facilitate the secure transmission and storage, of vulnerability information, and adhere to ISO standards.

The capability to host the DoD's public vulnerability disclosure policy, separate from any specific challenges, relative to FA1. Hosting the policy may involve use of the contractor’s platform to receive vulnerability reports. It may also require use, of an

Application Program Interface, to export vulnerability reports, received through the platform, to other systems, like JIRA, or to a dedicated vulnerability management or ticketing system (Triage personnel, or any other tools/services of the contract, will not be required for this capability).

RFP # HQ0034-16-R-0258 Page 9

FA2 Platform Requirements

Platforms, conducting work under FA2, must have:

The capability to accept vulnerability reports from researchers.

The capability to apply tools and processes, automated as well as manual, to triage reports for the Government. This includes de-duplication of reports within 48 hours of receipt.

The capability to ensure that vulnerability reports, transmitted to Government remediators, are clear and of high quality. This will ensure that Government personnel can immediately remediate identified vulnerabilities.

The capability to facilitate effective communication between the triage team and researchers and between the triage team and Government remediators. This may include corresponding, separately, with multiple teams.

The capability to facilitate the secure transmission and storage, of vulnerability information, and adhere to ISO standards.

The capability to implement continuous monitoring as well as auditing tools, to monitor and assess, researcher behavior. This will require the Contractor to deliver logs, of all testing activity, to the Government during the post-challenge phase.

The capability to capture and inspect encrypted researcher traffic, such as, through an

SSL interception proxy.

The capability to function as a secure portal that is capable, of full packet capture to enable auditability and continuous monitoring, of researcher activities.

6.11. Experience

The contractor must have at least 2 years of experience in managing challenges, via the use of crowdsourcing techniques, for the purposes of vulnerability discovery and disclosure. The contractor’s experience must include providing vulnerability discovery and disclosure services on behalf of major corporations and for enterprise-scale networks and systems, and globally deployed software and hardware products.

The contractor’s experience must also include proven experience in triaging vulnerability reports based on industry-standard escalation policies within 48 hours of receipt, experience in managing the coordination of vulnerability disclosure affecting third-party organizations and vendors, administering vulnerability discovery and disclosure challenges and awarding payments, and have an existing pool of researchers that is large and diverse enough to support the efforts detailed in each of the task order PWS’.

6.12. Managing the Contract & Task Orders

The contractor maintains a single point of contact (POC or Program Manager) between itself and the Government for the management and maintenance of the IDIQ contract. Individual POC requirements will be detailed in the PWS at the task order level for subsequent task orders.

RFP # HQ0034-16-R-0258 Page 10

For each task order, researchers will be used to identify and submit vulnerability reports for the

Government. The contractor will manage the researchers and all communications necessary regarding vulnerability specifics between the Government and the contractor.

Certain DoD employees and other Federal Government employees are prohibited by law from receiving any financial reward. The contractor shall ensure that no prohibited individuals, including Government employees, receives financial compensation.

6.13. Managing Researchers

When managing Researchers under the performance of task orders the contractor(s) shall:

Effectively communicate and coordinate with current and prospective researchers to ensure a smooth experience for participants over the full duration of the respective task order period of performance.

Communicate electronically with researchers at each stage of the vulnerability lifecycle, including initial receipt, remediation, and acknowledgement/reward.

Securely manage the storage and distribution of credentials to researchers to enable remote vulnerability discovery and disclosure activities against assets that require trusted relationships/connections.

Ensure that the vulnerability discovery and disclosure process is capable of adhering to common international standards for handling vulnerability data, such as ISO 29147 and ISO 30111.

Identify appropriate bounty amounts, pay researchers, and provide appropriate tax documentation for any monetary and/or nonmonetary bounties paid to researchers for each task order.

6.14. Key Personnel

The contractor shall designate a “Program Manager” as a single point of contact to the DoD with at least one year experience as an effective client manager. Before removing or replacing the

Program Manager, the contractor shall (1) notify the Contracting Officer reasonably in advance and (2) submit justification (including proposed substitutions) in sufficient detail to permit evaluation of the impact on this contract. The contractor shall make no diversion without the

Contracting Officer's (CO) written consent; provided that the CO may ratify in writing the proposed change, and that ratification shall constitute the CO's consent required by this document.

6.15. Host DoD’s Public Vulnerability Disclosure Policy (FA1)

The contractor is expected to host the DoD’s public vulnerability disclosure policy, separate from any specific challenges conducted under FA1. Hosting the policy may involve use of the contractor platform to receive vulnerability reports, and may require use of an Application

Program Interface to export vulnerability reports received through the platform to other systems

(i.e., JIRA) or other dedicated vulnerability management or ticketing system. Triage personnel, or any other tools/services of the contractor, will not be required. This requirement is applicable to FA1 only.

RFP # HQ0034-16-R-0258 Page 11

6.16. Identifying Contractor Employees

All contract personnel attending meetings and working in other situations where their contractor status is not obvious to third parties are required to identify themselves as such to avoid creating an impression in the minds of members of the public that they are Government officials. They must also ensure that all documents or reports produced by contractors are suitably marked as contractor products or that contractor participation is appropriately disclosed.

7. SPECIAL REQUIREMENTS

This section describes the special requirements for this effort. The following sub-sections provide details of various considerations on this effort.

7.1. Government Furnished Materials

The Government will provide technical information, material and forms, and, as necessary, hardware and software unique to the Government for supporting the task orders. The DoD will provide any results of the most recent penetration test, if applicable, to the contractor prior to each Challenge Phase. Government unique information related to this requirement, which is necessary for contractor performance, will be made available to the contractor. The COR will be the point of contact for identification of any required information to be supplied by the

Government. Government Furnished Materials also includes any information received during the challenge from Government employees. Additional detail is included in each Functional

Area PWS.

7.2. Security

If required, the contractor will have escorted access to DoD facilities for any physical meetings required. A Non-Disclosure Agreement (NDA) for all contractor employees and subcontractor employees with access to vulnerability information must be provided at the commencement of each task order. Additional security requirements may be required and identified in each respective task order.

7.3. Documentation

Documents, data files, reports, correspondence, and all other documents and writings, regardless of the medium (or media) by which they were produced, preserved, stored, or created in connection with or for purpose(s) of work performed under this PWS and contract, shall be delivered directly within the vulnerability reporting platform. The contractor shall not deliver outside the platform unless provided written request from the COR.

The contractor agrees to assume responsibility for protecting the confidentiality of Government records, which is not considered public information. Each contractor or employee of the contractor to whom information may be made available or disclosed shall be notified in writing by the contractor that such information may be disclosed only for purposes and to the extent authorized herein. The contractor shall not release any information related to this contract to the public, media or other unauthorized persons or organizations unless the government has conducted the appropriate security review and granted written approval.

RFP # HQ0034-16-R-0258 Page 12

7.4. DoD Intellectual Property

The contractor may be written granted limited authority to use the official seal of the Department of Defense (DoD), and the official seals of all DoD Components (Other DoD Seals) for limited purposes. These purposes could include advertising the challenge on the challenge page in the platform, such as the eligibility page and the rules and the Challenge page. Use of all logos must be discontinued at the conclusion of each task order or the duration of the contract, with the

Contracting Officer’s Representative approval. Any contractor or subcontractor is not authorized to use the official seal of DoD and/or other DoD Seals without written pre-approval for any purpose. The contractor is not authorized to create, publish, or distribute any physical or electronic material that contains both the official seal of DoD and the contractor’s name and/or logo.

7.5. Background Checks

The contractor must have the ability to conduct criminal background checks for all registered participants. Although not a requirement for each FA1 task order, the contractor may be required to contract a third party commercial firm to conduct a commercial background check to ensure DoD resources do not pay bounties to felons or terrorists. The FA2 contractor must conduct thorough background checks of every researcher before they are invited to participate in every/any challenge.

7.6. 508 Compliance

Any/all Electronic and Information Technology (EIT) developed, maintained, or used through this effort must meet the applicable accessibility standards at 36 CFR 1194. 36 CFR 1194 implements Section 508 of the Rehabilitation Act of 1973, as amended, and is viewable at http://www.section508.gov . The contractor shall comply with Section 508 of the Rehabilitation

Act of 1973 and support the Government to ensure compliance with Section 508 throughout the development and implementation of this requirement.

8. DELIVERABLES

Deliverables of the Master IDIQ shall be submitted to the DoD COR designated in the contract.

All deliverables shall be submitted using Microsoft Office suite of tools (for example, MS Word, MS Excel, MS PowerPoint), or Adobe PDF format, unless otherwise specified by the COR.

Electronic submission shall be made via email, unless otherwise agreed to by the COR.

The COR has the right to reject or require correction of any deficiencies found in the deliverables. In the event of a rejected deliverable, the contractor will be notified in writing by the COR of the specific reasons for rejection.

The following table specifies the deliverables for this requirement:

PWS

Reference

Deliverable Title Delivery Time Frequency

6.5. &

6.12.

Contractor designated one single point of contact as the official Program Manager

1 week after contract award

Updated as necessary

4 Contractor develops any necessary materials for purposes of public relations, including

As often as required Continually

RFP # HQ0034-16-R-0258 Page 13 program logo for website and “swag” for researchers, and blogs, or other digital media

4 Final Report following each task order identifying what worked, what did not work, and ways to improve the scope of challenges for future task orders

At the end of each task order

At the end of each task order

4 FA1 only

Develop a “Hall of Fame” page for all public thanks to researchers who submit valid vulnerability reports for the duration of all task orders and post publically on platform

Within six weeks of each Challenge completion

1 time

4 FA1 only

Host the DoD’s public vulnerability disclosure policy

Within first week of contract and continuous throughout contract

Continually

9. PERFORMANCE REQUIREMENTS SUMMARY

Performance metrics are detailed in the table below:

Requirement Performance Standard Acceptable Quality Level Surveillance

Method

Designated

Program

Manager Per

Section 6.5..

Contractor has one single point of contact for

Government engagement for the duration of the program

Contractor has designated one single point of contact to discuss deliverables, deadlines, and to establish the right contractor team for each specific task order

Staffing absence longer than two continuous weeks are filled with alternate assignment to cover workload

COR review/ government personnel feedback

Provide support personnel to meet each task order

PWS requirement

All contractor personnel possess the skills needed to perform the required tasks as specified in the

PWS

The contractor attracts and maintains a stable researcher base that effectively supports the task order requirements

All contractor personnel and independent researchers possess the needed skills before contract award, including technical and triage team, strategic communications team, and program manager

The contractor is able to attract the minimum number of researchers needed for each task order at least

24 hours prior to each challenge commencing

If security requirements apply to the task order, all documentation is submitted to the government in a

COR review/ government personnel feedback

RFP # HQ0034-16-R-0258 Page 14 timely manner so that no contractor caused delays are apparent in commencing a FA2 challenge

The contractor's deliverables are complete and on time

Triage Personnel

& Capabilities to meet each task order PWS requirement

Contractor has extensive and experienced personnel on hand as well as proprietary automated tools

The contractor’s personnel use a combination of automated and manual methods to triage vulnerability reports within 48 hours of receipt

COR review/ government personnel feedback

Strategic communications and ability to handle public relations

Contractor has experienced personnel capable of handling both discreet communications on sensitive issues and public communications

Contractor has experienced personnel that handle strategic messaging. No more than 24 hours pass prior to social media responses issues for 95% of negative hacker posts

COR review/ government personnel feedback