Q-As -Cyber TRIDENT W900KK-20-R-0001 27 Mar 2020.pdf

Date Question # Subject &/or SOW Reference Questions Gov't Answer

3/17/2020 1

PCTE L and M v11 - Technical Management. 1.1.1.A.

Approach to improve and innovate the current PCTE agile development process. Please describe the current process.

Platform Development Kit details current process. Provide specific questions on any of the current process documentation. Furthermore, the PCTE Industry Day on 11 Jun 2019, the program office covered on slides 6-11 & 14-15 our agile development processes, construct, and tooling as well.

3/17/2020 2

PCTE L and M v11 - Technical Management.

1.1.1.A.1. Agile Development Operations (DevOps).

Improvement of offeror backlog task completion rate (velocity) and reduction of incomplete tasks (user story under-delivery) at sprint reviews

Please describe current velocity and quantity of incomplete tasks (user story under-delivery).

Platform Development Kit details current process. Provide specific questions on any of the current process documentation. Furthermore, the PCTE Industry Day on 11 Jun 2019, the program office covered on slides 6-11 & 14-15 our agile development processes, construct, and tooling as well.

3/17/2020 3

PCTE L and M v11 - Technical Management.

Infrastructure / Platform Operations. Infrastructure and configuration to support rapid CI/CD, automatic scaling and load balancing of existing PCTE contractors software across RCS instances

The scope associated with this criteria includes the deployment of platform training applications. Is the actual tooling and processes associated with deploying exercise-specific emulated networks to the platform part of the TRIDENT scope, or is it capability provided by 3rd party applications.

This sub-element was removed from Section L and M to minimize complexity associated with criteria. As part of execution of DO1-2-3, it is expected prospective bidders are to integrate capabilities within a multi-vendor environment. To verify the agile integration in an incremental and iterative fashion, the ability to test acros the plan-prepare-execute-assess training lifecycle of components delivered by multiple vendors is critical. This is further reinforced with DO2 providing an 3rd party orchestration role. Any sofware integration challenges would need to be mediated amongst the Government, TRIDENT, and 3rd party applications - but would require overall lead effort by TRIDENT in the collaboration via the integration factory and CI/CD processes.

3/17/2020 4

CYBER TRIDENT DO 1 SOW v8 -11-25-19. 3.2.

Operations. The contractor shall manage PCTE platform operations to ensure a system availability (Ao) of 95% for a high quality, persistent user experience.

How is system availability (Ao, e.g., 95%) for a "high quality, persistent user experience" determined? Will the Government provide this metric? Is the requirement " The contractor shall manage PCTE platform operations to ensure a system availability (Ao) of 95% for a high quality, persistent user experience".

95% system availability in discussion internally for calculation. Final number will come with Draft RFP. With PCTE capabilities being provided by multiple vendor, it is expected that TRIDENT bidders would have full cognizance over platform infrastructure layers across hardware, software virtualization/middleware, and core enterprise services. The system is expected to maintain this availability number even through basic troubleshooting steps provided and maintained by the vendor (to include those provided by 3rd parties).

3/17/2020 5

CYBER TRIDENT DO 1 SOW v8 -11-25-20. 3.2.1.

PCTE Maintenance and Support. The contractor shall manage the maintenance for the PCTE infrastructure to include all RCS Nodes.

updated to contain complete RCS Infrstructure (Hardware and Software)to include model numbers, actual quantities, spares, etc. in order to provide the offeror with enough information to provide a Fixed-Price bid? Yes, planning to provide HW/SW List

3/17/2020 6

CYBER TRIDENT DO 1 SOW v8 -11-25-21. 3.2.2.

PCTE Maintenance and Support. The contractor shall manage the maintenance for the PCTE infrastructure to include all RCS Nodes.

Will the Government provide a roll-out schedule for the RCS deployments or will the offeror just assume all RCS are active on Day 1 of the contract. For the purpose of DO 1, the assumption should be made all RCS are active on day one.

3/17/2020 7

CYBER TRIDENT DO 1 SOW v8 -11-25-21. 3.2.2.

PCTE Maintenance and Support. The contractor shall manage the maintenance for the PCTE infrastructure to include all RCS Nodes.

Are there six or seven RCS Locations, there is a descrrepancy between the Base SOW (Vicksburg) and DO 1 which has six and does not include Vicksburg. How many nodes will there be at each location?

Will provide updated RCS information for clarity at Draft RFP. At present there is anticipated to be 7 RCS (1 DevRCS and 6 production nodes - of which are at multiple, separate classification levels).

3/17/2020 8

CYBER TRIDENT DO 1 SOW v8 -11-25-19. 3.2.1.

PCTE Maintenance and Support. Assist, maintain, and update Memorandums of Agreement (MOAs) and Inter-Agency Service Agreements (ISAs)

Are MOAs currently in place? Will the Government initially establish these agreements?

Yes, MOAs in place and/or Government will do initial establishment. Government has lead on MOAs, and contractor in assist role.

3/17/2020 9

CYBER TRIDENT DO 1 SOW v8 -11-25-19. 3.2.1.

PCTE Maintenance and Support. Perform on-going test and performance vaidation of the PCTE RCS at designated sites

Can the Government define the performance requirements for the RCS so the offeror can define the required performance tests?

The Government is assembling a number of RCS related documents to be provided as part of the bidders library.

Questions-Answers on Cyber TRIDENT Solicitation

Please beware questions and answers will be posted on solicitation page on beta.SAM website unless proprietary data is included.

3/17/2020 10

CYBER TRIDENT DO 1 SOW v8 -11-25-19. 3.2.2.

Infrastructure Tier 3 Technical Support. The Tier 3 Technical Support function shall maintain support communications through a phone line, dedicated email address, channels on the PCTE platform's onboard chat capability, and leverage the existing enterprise-wide Trouble Ticketing/Problem Reporting System (TTPRS) accessible from the PCTE PLatform.

Is the trouble ticket application that is accessible from the PCTE platform GOTS or is it to be provided by the TRIDENT contractor?

As part of the Cyber Innovation Challenge (CIC) 3, a Technical Operations Dashboard was awarded that provided JIRA Service Desk for ticketing from users, and the DI2E instance of JIRA provides ticketing for the development team. Both applications are intended to stay in place after TRIDENT contract is awarded.

3/17/2020 11

CYBER TRIDENT DO 1 SOW v8 -11-25-19. 3.3.2.

Infrastructure Tier 3 Technical Support. The Tier 3 Technical Support function shall maintain support communications through a phone line, dedicated email address, channels on the PCTE platform's onboard chat capability, and leverage the existing enterprise-wide Trouble Ticketing/Problem Reporting System (TTPRS) accessible from the PCTE PLatform.

Does the Tier 3 Contractor Technical Support route Tier 3 tickets to the CIC Vendors?

Tier 2 internal PM help desk routes to Tier 3 and CIC vendors as per slide 19 of the Industry Day slides from I/ITSEC 2019.

3/17/2020 12

CYBER TRIDENT DO 1 SOW v8 -11-25-19. 3.4.1.

Maintenance Support. The contractor shall be responsible for all repairs and coordinate with vendors to keep the capabilities fully operational.

In this context, are vendors outside the set of 3rd party vendors performing under the CICs or other contracts, or are CIC COTS providers included in this requirement?

The contractor is responsible for all parts under attachment 10 (GP). The GP listing does not include parts/equipment from CIC vendors, but does include parts/equipment from other contracts.

3/17/2020 13

CYBER TRIDENT DO 2 SOW v9 -11-25-19. 3.

Multiple. The DO #2 SOW makes numerous references to traditional waterfall DoD 5000 CDRLs such as System/Subsystem Specifications, Software Requirements Specifications, etc.

Since these documents do not typically apply to Agile/DevOps programs will these CDRLs be deleted or allowed to be tailored/replaced by the offerors based on their proposed solution?

The program office is relooking each of the CDRLs. However, we intend to tailor traditional CDRLs to be updated to fit the Agile nature of PCTE. The program office is not looking to develop a SSS, SRS, CRS centric requirement trace but to align them within a agile approach that could include decomposition and alignment of program office requirements to agile backlog, epics, user stories and themes evolved over time. Further review by the Government will better elaborate this.

3/17/2020 14

CYBER TRIDENT DO 2 SOW v9 -11-25-19. 3.2.1.

Capability Development, Integration, Delivery and Sustainment. The Agile Systems Engineering process shall include traditional Government ceremonies such as but not limited to Systems Requirement Review (SRR), Preliminary Design Review (PDR), Critical Design Review (CDR), Integration, Verification and Validation (IV&V)

Since these events do not typically apply to Agile/DevOps programs will these events be deleted or allowed to be tailored/replaced by the offerors based on their proposed solution?

The offeror may recommend tailoring or removal of these events in their proposed solution. No traditional SETR reviews are not anticipated but agile reviews corresponding to them are requested to be offered for substitution.

3/17/2020 15

CYBER TRIDENT DO 2 SOW v9 -11-25-19. 3.2.2.

Integration Factory. The contractor shall perform integration factory activities to include recommending improvements to the current integration factory development environment to increase the quality of the software and release frequency.

Does the offeror make its recommendations against the information solely contained in the PDK, or are recommendations to consider current prototype activities and capabilities?

The current integration factory development environment is documented in the PDK. We have asked for innovations in Section L&M for the current status documented in the PDK. Additional information has also been furnished wihtin the PCTE Jun 2019 Industry Day and I/ITSEC Presentations published.

3/17/2020 16

CYBER TRIDENT DO 2 SOW v9 -11-25-19. 3.6.2.

Development Testing. The Contractor shall for each software release within the CI/CD pipeline conduct PCTE software release testing (SRT). SRT testing consist of a variety of test including: performance or scalability testing, end-to-end integration, and smoke testing, which are essential for formal qualification and government acceptance of the PCTE capability. The contractor shall maintain a Requirements Traceability Matrix (RTM) which shall be provided to the government at the TRR and Post SRT test results brief.

Do these exist today? Do we have to create from scratch?

Does a current RTM/RVTM exist today? Will the bidder's library contain the current prototype RVTM, test reports, test artifacts, test logs, etc.?

Regarding T&E, an RTM is available linking requirements to capabilities to test cases. The RTM will need to be modified and managed by the TRIDENT vendor upon contract award.

The RTM, Test reports and artifacts will be made available in the bidders library.

3/17/2020 17

CYBER TRIDENT DO 2 SOW v9 -11-25-19. 3.6.4.

Defect Management and Resolution. The contractor shall document all defect discrepancies for the capabilities and track the failure analysis and corrective action for each discrepancy until correction and regression test are successfully completed.

Any effective failure analysis effort seems to have dependency on 3rd Party Software, and 3rd Party processes, in order to identify potential root causes and resolutions. Do CIC vendors have the requirements to support this activity, or will their software, development processes, architecture artifacts, etc. be provided in the bidder's library?

CIC vendors have the requirement to support bugs and defects, and currently this activity is supported in the Government managed JIRA ticketing system. As part of leading DO2 efforts, it is expected TRIDENT bidder would be leading the orchestration and collaboration of this multi-vendor integration factory and managing the allocaiton, tracking, and satisfacotry resolution of defects provided by 3rd party vendors.

3/17/2020 18

CYBER TRIDENT DO 2 SOW v9 -11-25-19. 3.2.2.

Integration. The contractor shall ensure each vendor provides updates to the documentation for each component of the system.

Reword or remove the contractor's responsibility to ensure these updates, to read that the contractor can report status on the updates to documentation.

Yes, we will reword in the SOW with the below:

The contractor shall integrate 3rd party vendor contributions

3/17/2020 19

CYBER TRIDENT DO 2 SOW v9 -11-25-19. 3.2.3.8.

Configuration Management (CM). The contractor's CM efforts shall: Identify the configuration items, components, and related work products that will be placed under configuration management…

Does this requirement apply to new platform capability developed under the TRIDENT contract or apply to existing potential Configuration Items? Does this requirement apply to 3rd Party provided applications and products?

Yes, applies to 3rd party provided applications and products. The core crux of DO2 is an integration factory that needs to track, orchestrate, integrate, and deliver capabilities via a mature CI/CD process - a cornerstone of that is robust CM processes and tools that reflect the total capabilities that amount to the final PCTE platform, development tooling, lab configuration and other items necessary to routinely and rapidly produce "shippable products" out of the integration factory.

3/17/2020 20

CYBER TRIDENT DO 2 SOW v9 -11-25-19. 3.4.1.

Security Engineering. The DevOps process shall emphasize cyber security teams collaborating with all teams to secure the PCTE platform throughout the lifecycle of the system.

Remove requirement for a separate cyber security teams and let the offeror propose how cybersecurity activites are performed. Acknowledged, and will take under consideration.

3/17/2020 21

CYBER TRIDENT DO 2 SOW v9 -11-25-19. 3.5.

Training. The contractor shall provide training courseware related to the capability for operators, maintainers, users, and other potential audiences to facilitate smooth system and user operations.

Does this requirement apply to 3rd party training applications, and if so, will 3rd party vendors provide that training content, or will the offeror provide a solution to obtain this critical input?

Yes, applies to 3rd party provided applications and products. Vendors will provide training data and information, and offeror will package into a releasable training product.

3/17/2020 22

CYBER TRIDENT DO 3 SOW v10 -11-25-19. 3.2.6.

Requirements. Based upon analysis of system requirements, system design, and other considerations, the contractor shall manage and record the software requirements to be met by each software item, the methods to be used to ensure that each requirement has been met, and the traceability between the software item requirements and system requirements.

The scope of this section seems to heavily utilize terminology of a traditional DoD 5000 waterfall program. Will these requirements be removed based on the Agile/DevOps requirements elsewhere in the SOWs, or can the offeror tailor this approach in their offering? Offeror can tailor in their approach/proposal. See line item 13 for similar response.

3/17/2020 23

CYBER TRIDENT DO 3 SOW v10 -11-25-19. 3.2.7.

Architecture. The contractor shall conduct architecture evaluations, including stakeholders external to the contractor's organization, for each iteration of PCTE.

funded by the Cyber TRIDENT contract? Can the definition of "External" be clarified - is this outside of the contractor's program staff, within its parent company and partners, or is completely external to those entities? Can "each iteration of PCTE" be clarified since it indicates how often this event

Currently that would include core program office, TCM, and USCC identified representatives.

Currently the program office holds architecture working groups with representatives from PM and vendors with gov't holding final approval.

3/17/2020 24

CYBER TRIDENT DO 3 SOW v10 -11-25-19.

3.2.13. Metrics. The contractor shall execute the

capture and reporting of the Operational Effectiveness and Suitability, Usability, Agile Development, technical performance (e.g., performance scaling, pub-sub, serialization, and API conformance), ….

Will the definitions of these metric areas be provided by the Government or left to the offerors to develop their own definitions which may vary significantly. Are these items to be provided by a DID/CDRL?

The Government is seeking metrics from industry to assess agile performance from offerors.

The data will be submitted on a CDRL when requested.

3/17/2020 25

CYBER TRIDENT DO 1 SOW v8 -11-25-19. PCTE Maintenance and Support. Regularly apply software patches and updates for any COTS/GOTS products utilized on the PCTE RCS platform.

Government or left to the offerors to develop their own definitions which may vary significantly. Are these items to be provided by a DID/CDRL?

Software patching and updating is a routine task across all DoD programs with periodic and recurring patch cycles developed and executed by the contractor with Government oversight.

The possiblity of a critical, out of cycle patch will happen and should be planned for based on the offeror's experience with critical COTS patching (i.e.: Microsoft updates, etc)

3/17/2020 26

CYBER TRIDENT DO 2 SOW v9 -11-25-19. 3.3.1.

Event and Exercise Support. The contractor shall participate alongside the Government in the coordination and management of user interviews, audits, and surveys in order to regularly capture user feedback.

Will the Government provide how many events will take place, their duration, how many users will attend, or how taxing they will be on the system in order for the offeror to bid this effort?

The current vendor pool as denoted on slide 13 of the PCTE Tech State of the Union slides held on 11 JUN 2019 show the current vendors that will be analogous in size and scale to the number moving forward while factoring in vendor turnover.

3/17/2020 27

CYBER TRIDENT DO 2 SOW v9 -11-25-19. 3.2.3.1.

Third Party Onboarding and Orchestration. The contractor shall manage, execute and host as well as participate in third party vendors interaction and handover meetings.

Given that we don't know (right now) how many future CICs there will be or the number of future CIC vendors, how can we bid onboarding and orchestrating an arbitrary amount of third party vendors?

The current vendor pool as denoted on slide 13 of the PCTE Tech State of the Union slides held on 11 JUN 2019 show the current vendors that will be analogous in size and scale to the number moving forward while factoring in vendor turnover.

3/17/2020 28

CYBER TRIDENT Base SOW v20 -11-25-19.

3.1.9.1. Transtion In (Initial Operations). Test plan to

validate Contractor’s Transition-In Plan.

Please clarify all components that are expected to be tested as part of transition-in.

Base SOW will be updated as part of Draft RFP to show all elements that will need to included in the Contractor's Transition In Test plan.

3/17/2020 29

CYBER TRIDENT Base SOW v20 -11-25-19. 3.6.1.

Maintenance Support. As specified by the individual DOs, the contractor shall provide maintenance and support services to ensure the continued uninterrupted operation of systems or products capabilities

Please clarify this support is limited to platform infrastructure and does not include Training Applications developed by 3rd Party providers.

Yes - the maintenance support does not include training applications developed by 3rd party providers.

3/17/2020 30

CYBER TRIDENT DO 1 SOW v8 -11-25-19. 3.4.1.

Maintenance Support. The contractor shall be responsible for the end-to-end lifecycle maintenance and support of the PCTE

Please clarify this support is limited to platform infrastructure and does not include Training Applications developed by 3rd Party providers.

CIC vendor product maintenance is not included however maintenance on enterprise COTS products is included (ie: Windows updates, Red Hat updates,etc)

3/17/2020 31

CYBER TRIDENT DO 1 SOW v8 -11-25-19. 3.4.2.

Property Accountabilty and Management. The contractor shall ensure 100% accountability for all government property to include supplies, GFP, and maintenance parts. The Contractor shall establish and implement a Property Management System consist with FAR Clause 52.245-1 requirements for all Government Furnished Property (GFP) provided under this contract.

Please specify and quantify supplies. Or is the requirement for the Contractor to ensure 100% accountability for Government property to be transferred to the Contractor(e.g.

GFP maintenance parts).

Contractor will be responsible for 100% accountability of all equipment, not just GFP.

Complete list will be available shortly.

3/17/2020 32

CYBER TRIDENT DO 1 SOW v8 -11-25-19. 3.4.3.

Schedule Management. The contractor shall establish and maintain an automated supply system that accounts for PEO STRI organizational and installation supplies and equipment. The contractor’s automated supply system shall include policies and procedures for the reception, inspection, inventory, loading and unloading, storing, issuing, and delivery of supplies and equipment

PEO STRI Organizational and installation supplies is confusing. Is the requirement "The contractor shall establish and maintain an automated supply system that accounts for Government property to be transferred to the Contractor(e.g. GFP maintenance parts).

Will clarify. The intention is the contractor will have responsbility for PCTE property accountabilty of all PCTE equipment, not just GFP.

3/17/2020 33

CYBER TRIDENT DO 3 SOW v10 -11-25-19. 3.1.3.

Schedule. The contractor shall plan, execute and deliver contributions within the Government agile scrum processes implemented within Delivery Order 2.

We are not familiar with the current Government agile scrum process. Is the requriement? The contractor shall plan, execute and deliver contributions within their implementation of an agile scrum processes implemented within Delivery Order 2.

See Platform Development Kit (PDK) for all necessary information on Government agile scrum process. This process will be the starting point for the offeror, however, the Government will be open to proposed improvements and changes based on the offeror's feedback to innovate the process.

3/17/2020 34

CYBER TRIDENT DO 3 SOW v10 -11-25-19. 3.4.1.

The contractor shall provide installation, implementation and training for each software license.

Please clarify - Is this requirement restricted to vender platform licenses managed by the Contractor and excludes CIC application licenses acquired by the Government?

The vendor shall be the single point contact regarding education and training of licensed COTS and GOTS hardware / software. The vendor shall be the coordinator for new equipment training and new software training to include all initial purchases, fielding and sustainment of education / training with regards to major revisions / versions changes. This includes COTS, GOTS and proprietary hardware or software.

3/26/2020 35

"The contractor shall maintain and enhance the current implementation of the CI/CD pipelines documented in the PCTE PDK as directed by the Government team."

- In the interest of improving the efficiency and collaboration of the entire SDLC, would the government be open to migrating their CI/CD pipelines to a new platform? Also, we only saw that CI/CD was called out, does the government expect to maintain other tools/processes for source code management or issue tracking for example?

As stated in Section L Technical/Management Approach factor, the Government is looking for innovative, mature and robust means to establish an integration factory of which CI/CD is core.

The Government intends to evaluate offeror solutions on these fronts and would be open to understanding alternative tooling as long as it aligns to commercial best practices, industry trends, and modern tooling to rapidly deliver platform capabilities on-demand. The PCTE strategy is to maximize utilization of COTS components and with that model we do not expect to do much customer software development or coding. See PDK for current tools used by

PCTE.

3/26/2020 36

We saw that SAST was called out in Security Requirements in draft of SOW for DO 2, but didn't see any mention of DAST, dependency scanning, container scanning or license compliance checks. Are those also required?

The program office is looking for a CI/CD process that implements variety of automated checks, tests, and compliance gates of which cybersecurity is one such lane (others include regression, scalability, loading, new feature, API contract testing, etc). It is expected SAST will be of limited value to the program office due to the nature of heavy COTS components for which we don’t own the source code for. There will be instances of open source capabilities integrated into the platform to enable integration such as RabbitMQ, OpenShift, API Gateway, NGINX, and Infrastructure as Code scripts for which source is available but which is integrated into the platform.

3/26/2020 37

How many users does PCTE expect to be involved in the total development process: developers, program management, security/IA, infrastructure

The program office expects at least 100-120 personnel responsible for development activities across Government lanes, vendors, COTS providers, and end users (as part of the DevOps integration).

3/26/2020 38

Would the government be open to using a single platform that supported the entire DevSecOps lifecycle as opposed to using multiple point solutions that only address pieces of the Software Development lifecycle?

IAW Section L, the Government would be willing to consider the merits of any offeror’s DevSecOps process and the innovation that it bring forth. The Government is highly seeking robust, mature and commercial best practices to establish this integration factory utilizing DevOps, CI/CD and agile processes as the core tenets for its integration factory.