RFQ_Attachment_2_PWS.pdf

PERFORMANCE WORK STATEMENT

DEFENSE DIGITAL SERVICES

Development of CyberStakes Capture-the-Flag Problem Set

10 July 2018

1.0. BACKGROUND AND INTRODUCTION

Defense Digital Service(DDS) intends to leverage the Army Cyber Institute(ACI) CyberStakes competition to identify, as well as, train top technical talent in the Army. CyberStakes is an annual

Capture-the-Flag (CTF) competition. Competitions typically span 10 days and are run in a virtual environment using a “Jeopardy” style format. The difficulty of the challenge generally starts at the level of the novice participant and progresses in difficulty level to keep those with advanced problem-solving skills engaged in the competition. These competitions are open to anyone in the

Army, Cyber National Mission Force, anyone from the five Service Academies, and ROTC cadets. This event is intended to reach a broad scope of technical talent throughout the Army and other DoD constituents. Winners are recognized at the International Conference on Cyber Conflict by the Commander of Army Cyber Command.

2.0. OBJECTIVE

DDS has a need for a contractor to develop a set of 70 problems for DDS to use on ACI’s

PicoCTF platform to run CTF exercises. The picoCTF platform has two main components: the shell servers and the web server. Refer to the documentation online at https://github.com/picoCTF/picoCTF/wiki and the information provided in Appendix B concerning the challenge format. The Wiki mainly provides information on setting up the platform and adding your own challenges.

3.0. SCOPE

The general nature of the work is for the contractor to develop and create a problem set to use in

CTF exercises. Problem set categories shall include binary exploitation, reverse engineering, web security, forsencis, and crytography.

4.0. GENERAL REQUIREMENTS

4.1. NON PERSONAL SERVICES

The services and performance required under this contract are determined to be not inherently governmental. The contractor shall not perform or give the appearance of performing inherently governmental functions as described in FAR Part 2 and Subpart 7.5, and all DoD policy. The Government shall neither supervise contractor employees nor control the method by which the contractor performs the required tasks. All contractor personnel shall identify themselves as contracted support personnel in all forms of communication with all entities with whom the OUSD(C) has business dealings. The contractor shall manage its employees and guard against any actions that are of the nature of personal services, or give the perception of personal services. If the contractor believes that any actions constitute, or are perceived to constitute personal services, the contractor shall notify the Contracting Officer or Contracting Officer Representative (COR) immediately.

4.2. BUSINESS RELATIONS

An integral part of successful performance under this contract is not only the production of quality products but the responsiveness of contracted personnel in the day-to-day output of work products. While the end product or deliverable is vital to successful performance, https://github.com/picoCTF/picoCTF/wiki https://github.com/picoCTF/picoCTF/wiki/Set-Up https://github.com/picoCTF/picoCTF/wiki/Set-Up https://github.com/picoCTF/picoCTF/wiki/Adding-Your-Own-Content day-to-day success also includes client interaction, continual performance and process improvement, and responsiveness. The contractor is required to proactively maintain assigned tasks and respond to all entities with professional business dealings related to the assigned tasks.

The contractor shall integrate and coordinate all activity to execute the requirements specified. The contractor shall manage the timeliness, completeness, and quality of the contract deliverables. The contractor shall provide effective management of all contracted personnel (including subcontractors), timely identification of issues and corrective action plans. The contractor shall seek to ensure customer satisfaction and professional and ethical behavior of all contractor personnel.

4.3 CONTRACT MANAGEMENT

The contractor shall establish clear organizational lines of authority and responsibility to ensure effective management of the resources assigned to this requirement. The contractor must maintain continuity between the support operations at the Pentagon and the contractor's corporate offices. The contractor shall establish processes and assign resources to effectively administer this contract. The contractor shall respond to Government requests for contractual actions promptly. The contractor shall have a single point of contact between the government and contracted personnel assigned to support this contract. The contractor shall assign work effort and maintain proper and accurate time keeping records of personnel assigned to work on this requirement.

4.4 SUBCONTRACT MANAGEMENT

The contractor shall be responsible for any subcontract management to seamlessly integrate work performed on this requirement and shall be responsible and accountable for subcontractor performance on this requirement. The prime contractor will manage work distribution to ensure there are no Organizational Conflict of Interest (OCI) considerations.

4.5. LOCATION

The work will be primarily performed at the contractor facilities.

4.6. PRIVACY ACT AND PERSONNALLY IDENTIFIABLE INFORMATION

Contractor personnel will have requirements to access Privacy Act and Personally

Identifiable Information in performance of this contract and will have to maintain all measures required to appropriately protect the information. Contractors must complete annual Government Privacy Act and Personally Identifiable Information training and maintain training records.

4.7. TRAVEL

Travel outside the National Capital Region will not be required. When movement of contractor personnel is required locally (e.g., Metro), travel costs will be borne by the contractor. Contracted personnel may not operate but may travel in government furnished transports with proper identification.

4.8. ORGANIZATIONAL CONFLICT OF INTEREST (OCI)

Personnel performing work under this contract may receive or have access to proprietary information (e.g., cost or pricing information, budget information or analyses, specifications or work statements, etc.) which may create a current or later OCI as defined in FAR Subpart 9.5. The contractor shall notify the contracting officer immediately whenever it learns such access of participation may cause any actual or potential OCI and shall promptly submit a plan to the contracting officer to avoid or mitigate any such OCI.

The contractor’s mitigation plan will be determined to be acceptable solely at the discretion of the contracting officer and if the contracting officer unilaterally determines that any such OCI cannot be satisfactorily avoided or mitigated, the contracting officer may affect other remedies as he/she deems necessary, including prohibiting the contractor from participation in subsequent contracted requirements which may be affected by the

OCI.

4.9. DOCUMENTATION AND DISCLOSURES

Documents, data files, reports, correspondence, and all other documents and writings, with any charts, graphs, tables, illustrations, photographs, images, and other illustrative, explanatory, historical documents related thereto or independent thereof, regardless of the medium (or media) by which they were produced, preserved, stored, or created for purpose(s) of work performed under this PWS and contract, are property of the government and shall be delivered to the COR promptly, upon request. All challenges delivered for this contract become the property of DDS and ACI and cannot be re-used for other efforts on the part of the contractor.

The contractor agrees to assume responsibility for protecting the confidentiality of

Government records, which is not considered public information. Each contractor or employee of the Contractor to whom information may be provided or disclosed shall be notified in writing by the contractor that such information may be disclosed only for purposes and to the extent authorized. The contractor shall not release any information related to this contract to the public, media or other unauthorized persons or organizations unless the government has conducted the appropriate security review and granted written approval (e.g. posting information to a public website).

The contractor obtains all non-disclosure agreements with all applicable corporate, supplier and subcontractors with proprietary, restricted, competition sensitive, or any other restricted (e.g. non-foreign disclosure due to public law) data used or accessed during the duration of this contract. The contractor shall complete a Non-Disclosure Agreement

(NDA) at contract award.

4.10. POST AWARD CONFERENCE / PERIODIC PROGRESS MEETINGS

The contractor agrees to attend any post award conference convened by the contracting activity or contract administration office. The contracting officer, contracting officer representative (COR), and other government personnel may meet periodically with the contractor to review the contractor’s performance. At these meetings the government will apprise the contractor of how its views the contractor’s performance and the contractor will apprise the government of problems being experienced. These meetings shall be at no additional cost to the government.

5.0. PERFORMANCE REQUIREMENTS

The performance requirements will be met through Sprints. Sprints shall be conducted in 2 week intervals to complete questions using feedback from DDS and ACI.

5.1. CHALLENGES

The contractor shall create 70 CTF challenges. Fifty-eight (58) of the questions shall be allocated to the following categories at the minimum quantity listed below. The remaining 12 questions can be allocated to one of the named categories or added to a miscellaneous category.

Binary exploitation: 12

Reverse engineering: 12

Web security: 12

Forensics: 12

Cryptography: 10

Refer to Appendix A for a description of the challenge categories.

5.2 SPRINTS: The contractor shall develop and create the challenge problem set outlined in PWS

5.1 through the use of sprints.

SPRINT 1: The contractor shall start Sprint 1 at contract award. During this sprint, the contractor shall provide a draft list of challenges to DDS and ACI for review and feedback. The contractor shall make adjustments as requested by DDS and ACI. The list of challenges will include:

- Name;

- Category;

- Intended point value; and

- Short Description and learning objective.

SPRINT 2: The contractor shall provide at least five challenges per category for review. ACI will validate that the challenges deploy and function correctly on the PicoCTF platform.

Sprint 3: The contractor shall deliver 50% of the challenges for testing and evaluation.

Sprint 4: The contractor shall provide all 70 challenges per the requirements.

Sprint 5: The contractor shall be available to troubleshoot any issues with deploying the challenges until the end of the period of performance.

6.0. PERFORMANCE STANDARDS SUMMARY

The government shall evaluate the contractor’s performance under this contract in accordance with the Performance Requirements Summary. This summary is primarily focused on what the

Government must do to ensure that the contractor has performed in accordance with the contract general, performance and special requirements. It defines how the performance for requirements will be applied, the minimum Acceptable Quality Level (AQL), the frequency and method of surveillance.

Incentives/Disincentives: Consistent positive performance will reflect on past performance report and exercising of options. Failure to meet AQL(s) may cause the government seeking an equitable financial adjustment, may influence the decision to exercise options and will reflect negatively on past performance.

Performance

Objective Performance Standard Acceptable Quality Level

Method of

Surveillance

PWS section

5.1

The contractor shall create 70 CTF challenges. Fifty-eight

(58) of the questions shall be allocated to the following categories at the minimum quantity listed below. The remaining 12 questions can be allocated to one of the named categories or added to a miscellaneous category.

Binary exploitation: 12

Reverse engineering: 12

Web security: 12

Forensics: 12

Cryptography: 10

No deviation

Customer comments/report, COR review

PWS Section

5.2

SPRINT 1: The contractor shall start

Sprint 1 at contract award. During this sprint, the contractor shall provide a draft list of challenges to DDS and ACI for review and feedback. The contractor shall make adjustments as requested by DDS and ACI. The list of challenges will include:

- Name;

- Category;

- Intended point value; and

- Short

Description and learning objective.

90%

Comments

PWS Section

5.2 SPRINT 2: The

contractor shall provide No deviation

Comments at least five challenges per category for review.

ACI will validate that the challenges deploy and function correctly on the PicoCTF platform.

PWS Section

5.3

Sprint 3: The contractor shall deliver 50% of the challenges for testing and evaluation.

No deviation

Comments

PWS Section

5.4

Sprint 4: The contractor shall provide all 70 challenges per the requirements.

No deviation Customer

Comments

PWS Section

5.5

Sprint 5: The contractor shall be available to troubleshoot any issues with deploying the challenges until the end of the period of performance.

No deviation

Comments

7.0. DELIVERABLE FORMAT

Requirement deliverables shall be submitted to the contracting officer and COR within the time frames identified in the performance standards summary.

The contractor shall deliver the following deliverables:

70 CTF challenges with the full source code that is compatible with the PicoCTF challenge format. Each challenge must include the following:

1. README.md

2. problem.json

3. challenge.py

4. Source code required to compile, generate, or host the challenge.

5. A solve script (bash, python) which demonstrates how to solve the challenge given a specific generated instance.

The challenges within each category must deliberately adhere to a progression of increasing difficulty.

Challenges must be creative, novel, and designed with a specific learning objective compatible with its category.

Challenges delivered for this contract become the property of DDS and ACI and cannot be re-used for other efforts on the part of the contractor.

Refer to the documentation online at (https://github.com/picoCTF/picoCTF/wiki) and the information provided in Appendix B concerning the challenge format.

https://github.com/picoCTF/picoCTF/wiki

The contracting officer or the COR may reject or require correction of any deficiencies in the deliverables. In the event of a rejected deliverable, the contractor will be notified by the contracting officer or the COR of the specific reasons for the rejection.

The following table specifies the deliverables for this requirement:

PWS

Reference

Deliverable Due Format

4.9 Non-Disclosure Agreement NLT 14 days after award; and

upon replacement of personnel

Microsoft Word with original employee signature

5.1 70 Challenges End of Sprint 5 Microsoft Word

Appendix A

Definition of Terms

Jeopardy Style Capture-the-Flag (CTF)

A traditional jeopardy style CTF is a time limited competition that involves individuals or teams solving challenges that are grouped into categories which generally follow a progression of increasing difficulty within each category. Challenges are discreet problems to be solved, which illustrate a specific topic/flaw/security principle, with objective success criteria through the acquisition and submission of a "flag". For additional information, please refer to the following documents:

Trail of Bits CTF Field Guide (https://trailofbits.github.io/ctf/)

The Many Maxims of Effective CTFs (http://captf.com/maxims.html)

CTF-Time Frequently Asked Questions (FAQ) (https://ctftime.org/ctf-wtf/)

Plaid Parliament of Pwning Guide (https://git.io/vpuei)

Binary Exploitation Category

Binary exploitation challenges typically run as a service that participants connect to and exploit after discovering the vulnerability. Challenges can also be run on a server with SSH access, SUID permissions on the binary, and a flag in the challenge directory.

Challenges in this category help participants demonstrate their ability to exploit stack-based buffer overflows, heap based buffer overflows, type confusion, integer errors, format string vulnerabilities, use after free, and other common vulnerabilities. These challenges also incorporate varying levels of software projections to include stack canaries, no-execute stack, address space layout randomization, and position independent executables.

Reverse Engineering Category

Reverse engineering challenges cover a wide variety of reverse engineering tasks and can include compiled software for various target computer architectures and operating systems.

Challenges in this category can also include techniques used in the wild to obfuscate malware, javascript, or powershell, but more commonly represent license checks that require manual reverse engineering of compiled binaries.

Web Security Category

Web security challenges or web exploitation challenges focus on vulnerabilities that occur in web technologies. Good challenges in this category distill real world vulnerabilities from the last couple years into something that can be solved in a few hours. The web security category includes technologies such as PHP, Flask, and dotNet, the backend database vulnerabilities in

SQL and no-SQL systems, and new technologies such as WASM.

https://trailofbits.github.io/ctf/ http://captf.com/maxims.html https://ctftime.org/ctf-wtf/ https://git.io/vpuei

Forensics Category

Forensics challenges include network forensics (PCAP analysis), host forensics (file recovery), various file formats and protocols. Challenges in this category can be derived from the real world incident response (IR) activities, but large artifact size (full disk images) needs to be considered and avoided if possible.

Cryptography Category

In cryptography challenges participants attack poorly implemented crypto, outdated crypto, or use well-known vulnerabilities to attack encrypted messages.

Miscellaneous Category

Miscellaneous challenges include everything that doesn’t fit nicely elsewhere. In the past challenges in this category have been linux/unix security related challenges, but this category provides a “creative” outlet for new challenges that would expose a participant to a new topic area they may be unfamiliar with. Examples of challenges that would not be suitable are recon, trivia, or multiple choice style questions. All challenges should have either a security relevant focus, support baseline technical knowledge, or capture an unexpected insight.

Appendix B

PicoCTF Challenge Format Specifications

Additional details about the PicoCTF challenge format can be found at https://github.com/picoCTF/picoCTF/wiki

Problem.json

Every problem must contain a problem.json. This file provides metadata about the problem through a series of mandatory and optional fields.

Field Data type

Req uire d

Description author Strin g

Yes Author of the problem.

score Int Yes Points gained for solving the problem.

name Strin g

Yes The title of the problem on the competition page.

description Strin g

Tem plate

Yes Description for the problem. jinja2 template.

category Strin g

Yes Category of the problem.

t List[

Strin g]

Yes Additional help to the problem.

version Strin g

No Version string for the problem.

Defaults to "1.0.0".

tags List[

Strin g]

No Minor descriptors for the problem.

organization Strin g

No Organization that the problem identifies with.

pkg_architec ture

Strin g

No Compatible architectures. Defaults to

"all".

pkg_descript ion

Strin g

No Description for the deb package.

Defaults to desc.

pkg_name Strin g

No Optional name for the problem package. Defaults to name. Check here for the naming policy.

https://github.com/picoCTF/picoCTF/wiki https://www.debian.org/doc/debian-policy/ch-controlfields.html#s-f-Architecture https://www.debian.org/doc/debian-policy/ch-controlfields.html#s-f-Source https://www.debian.org/doc/debian-policy/ch-controlfields.html#s-f-Source pkg_depend encies

List[

Strin g]

No List of package dependencies.

Defaults to none.

pip_require ments

List[

Strin g]

No List of pip requirements for the challenge. Defaults to none. Can alternatively include a requirements.txt in your problem directory.

challenge.py

The challenge.py script is responsible for problem generation and deployment. Refer to the documentation online for examples of use.

README.md

The README provides an overview of the challenge with a description and the desired learning objective. The overview should include a description of the intended path to a solution.

Solve script (solve.py, solve.sh)

The solve script contains a solution to a specific challenge, generally written in bash or python, but could also include some manual steps. As challenges should be written with a portion of randomness when possible, the solve script may not be able to always solve a generated instance of the challenge. Additionally, some challenges require manual analysis that cannot be replicated in a simple script. In these cases, the solve scripts should include comments for actions that need to be taken

Challenge source files

The challenges source files that are required to generate a new instance of the problem. This includes C source files, Make files, scripts to generate artifacts, http root source files, etc. These may be organized in any manner that makes sense.

https://github.com/picoCTF/picoCTF/wiki/Challenge.py