Attach2_DAVE_Cap_Sust_PWS_Final.docx

PERFORMANCE WORK STATEMENT

DAVE Capability Sustainment

1. DEFINITIONS. Agile software development is a collection of software development methods within a broad philosophy that vary in terminology, technique, execution. This Performance Work Statement (PWS) and these definitions assume an understanding of agile methods by the contractor and are intended to clarify unique aspects regarding execution of this requirement.

1.1 “Capability” is defined for this effort as a software-enabled function, process, feature, application, system, or service that exposes, displays, or processes acquisition data to allow users to access, view, aggregate, analyze, report, or otherwise employ this data to support acquisition decision-making across the Defense Acquisition Community.

1.2 “Capability Delivery” is the planning, design, prototyping, agile development, testing (unit, integrated, and user), deployment on government-furnished environment(s), user training, technical documentation, system administration, help desk, applicable licensing, DoD information system security compliance, and major enhancement of an acquisition Capability. Capability Delivery may also include associated administrative, management, and technical support necessary to successfully implement and manage Capabilities.

1.3 “Capability Sustainment” is a subset of Capability Delivery that encompasses software maintenance (ensuring the software continues to function in the host environments) technical refresh to ensure delivered Capabilities remain fully functional, bug fixes, updates to comply with additional cyber security policies, and address security vulnerabilities, maintaining compliance with Section 508 of the Rehabilitation Act of 1973, and minor enhancements.

1.4 “Minor Enhancement” is a minor software release, update, modification, which does not provide materially new functionality. Minor enhancements include incremental improvements that may appear as elements of a software release from version 3.3 to 3.4 or scheduled quarterly releases, for example. Minor enhancements include improvements to existing features or functions in the Capability, vs. introduction of a new Capability.

1.5 “Sprint” is defined as a three-week development period of prioritized user stories pulled from the backlog, bracketed by Sprint planning and Sprint review. Although user stories vary in complexity, functionality, and required experience to develop, a generally accepted standard is that one developer can deliver on average 12-14 story points per three-week Sprint.

2. BACKGROUND.

2.1 The Organization. The Office of the Under Secretary of Defense (Acquisition and Sustainment) (OUSD(A&S)) oversees Department of Defense (DoD) acquisition and establishes DoD-wide acquisition policies. The effective oversight of, insight into, and innovation of acquisition processes and outcomes requires quality, timely, and accurate acquisition data. Empowered with access to authoritative acquisition data and insights from analyses of that data, DoD leadership can better report, analyze, and make informed decisions on the Department’s complex, $2 trillion acquisition portfolio.

2.2 The Acquisition Policy & Innovation (API) Directorate is an organization within OUSD(A&S) subordinate to the Assistant Secretary of Defense (Acquisition) (ASD(A)) and the Executive Director, Defense Pricing, Contracting, and Acquisition Policy. API delivers timely access to accurate, authoritative, and reliable acquisition data to the Defense Acquisition Community, the USD(A&S), the Defense Acquisition Executive and Component Acquisition Executives and leaders, and OUSD(A&S), Services, Agency, and Component analysts, facilitating program oversight, accountability, and decision making for the DoD’s portfolio of Acquisition Category (ACAT) I through IV programs. API’s acquisition data facilitates the statutory and regulatory reporting, assessment, and oversight requirements of the DoD for program data across all Acquisition Pathways.

2.3 The Defense Acquisition Visibility Environment (DAVE). DAVE is an API-first platform for managing authoritative and trusted acquisition data. It serves as a single access point for Capabilities developed internal and external to DAVE and is architected on top of a data framework that governs all API capabilities. Architected with a Java back-end, DAVE is integrated with several systems across the Department that leverage DAVE’s APIs capabilities for consuming and exposing data. In addition to its data sharing capabilities, DAVE features a front-end developed in Angular that enables DoD Component users to view, modify, and report on the data. DAVE is transitioning from Initial Operating Capability to Full Operating Capability. See the DAVE architecture and technology stack documents attached with this RFI.

2.4 The Acquisition Information Repository (AIR). DoD Instruction 5000.02 establishes milestones that programs must achieve to continue progressing through the acquisition lifecycle. Documentation required at these milestones is stored in AIR, a standalone DAVE Capability. AIR’s unstructured data is also a source of information to support analysis and decision making. AIR is currently a Confluence-based capability that is in the process of transitioning from Confluence to a stand-alone Java and Angular based application. Management of the AIR software and hosting is performed by a government office. The API Directorate augments the government office with a software developer.

2.5 The Requirement. API requires a contractor to employ agile methodologies to sustain, maintain, and enhance DAVE Capabilities to maintain currency with changes in policy or statute as well as meet new information requirements or changes in information security requirements. API will continue to receive data from existing sources (e.g., the Military Departments and Agencies) and will integrate new data sources into Capabilities to meet customer requirements. Requirements also include functional and integrated testing of Capabilities, and Agile development, security, and operations (DevSecOps) support.

3. SCOPE. Using Agile methods, the contractor shall sustain and perform minor enhancement of DAVE Capabilities to support acquisition decision-making and achieve the DAVE Product Vision and Product Road Map. If required, by contract modifications exercising optional tasks, the contractor shall deliver new DAVE Capabilities and systems engineering support to DAVE.

The DAVE Product Vision:

DAVE provides the DoD timely access to accurate, authoritative, and reliable acquisition data to support insight, analysis, and decision-making. It provides easily accessible front-end applications, data repositories, capabilities, system to system integration, and a well-defined and managed data framework to offer perspective and context for the use of Acquisition data. DAVE supports the DoD Acquisition community and partners, providing an integrated foundation for the DoD Acquisition community, delivering improved security, performance, reusability, interoperability, scalability, and maintainability of Acquisition data.

4. AGILE PERFORMANCE REQUIREMENTS.

4.1 Agile Requirements Process. The government will create and maintain the DAVE product vision, system roadmaps, project plans, and product and release backlogs. These become the basis for the contractor’s work. The government’s designated Product Owner will specify high-level requirements to the Agile team. Using Scrum-based Agile processes, the Product Owner will work together with the contractor to develop and estimate user stories and establish acceptance criteria. Acceptance criteria will specify expected functionality for a user story, as well as any non-functional requirements that must be met in the development of the story. The Product Owner determines if acceptance criteria have been satisfied during Sprint review.

4.2 Tools and Standards. The contractor shall utilize the DAVE tools, technologies, and frameworks (the technology stack), which are outlined at Attachment 5. Ensure Capabilities conform to API’s Best Practices and Implementation Criteria (BPICs). Adhere to Test-Driven Development techniques and Department of Defense information security guidance.

4.3 Agile Team. The contractor shall perform as an integrated member of the API team, working cooperatively with other contractor firms and contractor employees, as well as government officials and stakeholders, to deliver capabilities on schedule. To enable story point estimation, the contractor shall engage with the integrated team to understand the intent of each ticket and its contribution to the overall Capability. The contractor shall not over-estimate, or give the appearance of over-estimating, the number of story points or “T-shirt” sizes required to complete the user stories assigned to a Sprint (sand-bagging). The contractor shall make every effort to incrementally deliver completed functionality for testing during each Sprint rather than hand a large quantity of stories over for testing at the end of the Sprint. A completed user story is delivery of a Sprint-specific expected functionality for a user story that meets the DAVE definition of “Done” in the “Agile Processes” BPIC, as adjusted for each particular Sprint. With the approval of the PM/COR, specialized training, retrospectives, and other administrative requirements may be allocated story points when these requirements contribute to broader API objectives.

5. TASK ONE. DAVE Capability Sustainment (Internal Use Software (IUS) Expense). The contractor shall, employing Agile methodologies, provide the Front End and Back End (Full Stack) technical expertise to sustain, unit test, document, and deliver DAVE minor enhancements. Provide the appropriate blend of technical expertise to deliver thin-client applications, capabilities, and features. Provide the appropriate blend of technical expertise to develop extract transform load (ETL) functions, including retrieving data from external data storage or transmission sources; transforming data into a format to meet operational needs; and transmitting and loading data. The contractor shall maintain delivered Capabilities and perform technical refresh to ensure delivered Capabilities remain fully functional. The contractor will support or complete information security updates as required, within the deadlines required by the Defense Information Systems Agency Joint Service Provider (JSP), Data Services, or designated network representatives under the guidance of the DevSecOps team, and provide technical support to fielded Capabilities. Provide technical support to DAVE Tier 1 and Tier 2 Help Desk requirements when needed. Provide Tier 3 Help Desk technical support and provide technical support to DAVE training and training documentation. The government’s objective Sprint velocity is 80 to 90 completed Story Points per three-week Sprint. The government’s quarterly development standard is no less than 320 completed and tested (passing) Story Points per quarter that comply with the DAVE definition of Done. More or fewer than 80-90 story points may be delivered during any given three-week Sprint.

6. TASK TWO. AIR Sustainment (IUS Expense). The contractor shall support the Defense Technical Information Center (DTIC) and API in these offices’ efforts to sustain and maintain the AIR Capability, which is built on Confluence. Support the transition of AIR from Confluence to a stand-alone Java and Angular based application. Perform information security compliance requirements. Provide bug fixes. Maintain the AIR Registration Database to ensure continued interoperability with evolving environments. Update code and code documents and unit tests to support system updates based on required new releases of underlying technologies and software. Conduct unit testing and error resolution. Provide technical support to AIR Tier 1 and Tier 2 Help Desk requirements when needed. Provide Tier 3 Help Desk technical support to AIR. Provide technical support to AIR training and training documentation. The government’s objective development velocity is a level of effort equivalent to 20 or more completed Story Points per three-week Sprint. The government’s quarterly development velocity objective is no less than 60 completed Story Points per quarter. AIR velocities are objectives only due to the contractor’s reliance on DTIC staff.

7. TASK THREE. Functional and Integrated Testing of Capabilities (IUS Expense). The contractor shall provide an independent test team with a combination of the relevant domain knowledge required to test DAVE and AIR Capabilities. The contractor shall perform independent functional and integrated tests and perform API validation and verification to ensure developed and maintained Capabilities have the required functionality and produce accurate results. Record functional test results at the story/ticket level in Jira in accordance with acceptance criteria. The independent test team shall test the functionality of each user story/task/bug in accordance with acceptance criteria and BPICs. Assist with User Acceptance Testing, including executing automated and manual tests and performing regression testing as needed to ensure the capability is working as expected and to help inform stakeholders of known issues. The contractor shall perform regression testing, including automated and manual testing, as capabilities are developed to minimize regressions prior to capability deployment. The independent test team will also assist with the smoke testing associated with releases. The contractor’s effective functional tests facilitate early problem detection and ensures the Capability can process required tasks in real-world scenarios.

7.1 Testing Requirements. Design testing scenarios, conduct the testing, analyze the results, and document the results. Interact with functional, data, technical teams, and stakeholders to better understand the requirements. Verify that the Capability provides correct and timely output required and properly applies data definitions and implements business rules. Compare known valid/invalid data inputs and outputs to generated outputs to verify Capability functionality. Assess usability and user experience. Ensure Capability navigational functions work as required. Test error conditions – input invalid data and determine if correct error messages are displayed. Verify that related systems maintain data integrity and can operate in coordination with other systems in the same environment. For each Sprint, outline the functions of the Capability and how each function will be tested and document test steps (including positive/negative and happy/non-happy path scenarios and test input data. Execute tests and document test results.

7.2 Automated Testing. The contractor shall develop automated tests, such as automated black box tests assist with functional testing, regression testing, smoke testing, and support user acceptance testing. Any user story, task, or bug that has a user-facing or integrator-facing feature is testable through the UI or API and therefore should have an automated test to support it. Perform tasks required for automated test implementation, including reviewing acceptance criteria, creating integration/regression test scenarios, implementing data management strategy, and generating required reports. Prepare and execute Sprint test plans using software test engineering best practices and principles. Review technical requirements, design documents, data definitions, and business rules to enhance test plans. Create, update, and maintain use cases, test cases, test steps, and test input data to include positive/negative and/or happy/non-happy path scenarios. Create, update, and maintain automated tests to remain in sync with the test cases. Collaborate with the application developers to plan and update testing processes and to ensure development is providing the hooks the independent test team needs to automate. Store automated test cases and results in Jira as Test Tickets, ensuring the tests are linked to the original ticket and functionality. Design performance load scenarios based on business requirements and real-world application usage patterns. Perform load testing and stress testing to ensure stability in normal computing environments. Conduct performance testing to ensure Capabilities are functioning optimally. Identify data bottlenecks, interoperability issues, and assess performance against real-time peak data loads. Support system integration testing to verify that related systems maintain data integrity and can operate in coordination with other systems in the same environment. Validate data integrity and data flow between the sub-components that make up system. Analyze test results and deliver detailed reports that accurately isolate problems. Report Capability defects, define each defect’s severity, and recommend the priority to address each defect.

8. TASK FOUR. Development, Security, & Operations Support (DevSecOps) (IUS Expense)

8.1 Systems Administration. Perform database administration, system administration and Information Assurance/Vulnerability Assessment support for API developmental and production systems, instances, and applications in hosted and cloud environments including Amazon Web Services and MilCloud, using Microsoft Azure, Linux, or other software tools. Support 15 – 20 environments including 1) lower environments in and/or Azure for development, testing, integration, staging, mirroring, user testing, and demonstrations, 2) NIPR and SIPR production environments in MilCloud, Amazon Web Services (AWS), and/or Azure. Employ API’s BPIC guidance where applicable. This task is covered by DoD Directive 8140.01, Cyberspace Workforce Management. contractor employees performing this support must possess and maintain a “Security+” Information Assurance Technical (IAT) Level I certification. contractor systems administrator(s) must obtain and maintain an “AWS SysOps Administrator” certification and Microsoft Azure Administer Associate certification.

8.1.1 Support the planning, coordination, installation, testing, operation, troubleshooting, and maintenance of hardware and software information systems. Responsible for effective provisioning, installation/configuration, operation, and maintenance of systems and software related infrastructure. Provide technical support for both system and software issues, including planned downtime and unscheduled outages, ensuring data protection and restoration of performance with minimal impact to API operations. Upgrade systems and processes as required for enhanced functionality and security issue resolution. Ensure infrastructure administration, including databases, software protection and other processes. Troubleshoot problems and make recommendations for future upgrades and system developments. Support necessary updates upon release of new operating systems, changes in hosting or changes in software. Monitor changes in DoD, OSD, and Agency information security policies and ensure contractor performance and API capabilities remain in compliance. Implement security policies.

8.1.2 Plan and schedule the integration and/or installation of new or modified hardware and operating systems and applications software. Manage accounts, network rights, and access to systems and equipment. Manage and monitor systems resources including performance, capacity, availability, serviceability, and recoverability. Identify hardware/software interface and interoperability problems and recommend solutions. Support system and application availability, reliability, functionality, integrity, and efficiency. Maintain systems configuration. Manage the installation and integration of systems fixes, updates, and enhancements.

8.2 Systems and Capability Security. Understand, adhere to, comply with, and implement security requirements as required per the environment administered, according to policy where available and best commercial practices. Integrate systems and network security requirements into the agile development of capabilities. Support and/or implement security procedures and tools including planning and execution of OSD CIO and/or DISA Information Assurance and Vulnerability Assessment and Computer Network Directive protections, fixes, patches, and procedures; apply mitigations in the timeframes expressed in the tasking, advisory, or bulletin and report compliance status if required via the DoD Vulnerability Management System (DoD Instruction 8531.01, DoD Vulnerability Management). Understand, apply, and comply with applicable DISA Security Technical Implementation Guides including the DISA Cloud Computing Security Requirements Guide and Defense Federal Acquisition Regulation Supplement Clause 252.239-7010, Cloud Computing Services, which outlines specific requirements regarding data disclosure, cyber incident reporting, malicious software, and spillage, among other requirements.

8.3 Database Administration. Support the creation, maintenance, backups, querying, tuning, user rights assignment, and security of API databases using commercial relational database management systems such as Microsoft's SQL Server, Oracle DB, MySQL or IBM's DB2. Maintain servers, perform backups, security, patches, and replication. Build SQL queries, stored procedures, etc., to meet API needs. Support the DAVE data architecture: design schemas, build tables indexes, data structures and relationships. Develop and implement cost effective backup and recovery processes that protect data. Create and maintain logical storage management as well as Continuity of Operations plans in case of outages or data loss. contractors supporting this task must obtain and maintain Azure Database Administrator Associate certification.

8.4 Azure Migration Support. Rearchitect the current DAVE infrastructure to support the conversion and sustainment of DAVE in both the NIPR and SIPR environments from MilCloud to Microsoft Azure. Rearchitecting for Azure must take place while maintaining the SIPR MilCloud instance of DAVE. Rearchitecting tasks using Kubernetes, Amazon Web Services, and Azure include mapping existing code that must be re-written, in-line parameter and encapsulation refactoring, revising conditional logic and data handling rules, refactoring microservices where applicable, unit and integration testing, and tuning.

8.5 DevSecOps Support. Perform DevSecOps to sustain a repeatable and automated continuous integration pipeline of secure API Capabilities. Using provisioning tools and hierarchical configuration data, support the development effort by automating manual builds and provisioning of infrastructure for delivered code. Integrate capability security with development operations. Create and maintain documentation to support the continuous integration process to enable developers to push frequent updates to an environment set up to run automated tests, without requiring manual supervision. Monitor and log disabled or malfunctioning scripts and infrastructure such that future problems can be addressed proactively, reducing impacts. contractor employees performing DevSecOps must obtain and maintain AWS Solutions Architect and/or AWS Developer certification, as well as obtain maintain Microsoft Azure DevSecOps Engineer certification including prerequisites.

9. TASK FIVE. OPTIONAL Systems Engineering and Development Lead Support. (IUS Expense). The contractor shall provide Surge Systems Engineering and Scrum Master/Development Lead support within 30 calendar days of a contract modification exercising this surge requirement.

9.1 Architecture. The contractor must lead the design and support the development of the DAVE technical architecture as well as the technical integration of legacy capabilities into DAVE, utilizing an API First strategy. The contractor shall provide information systems engineering expertise to support the design, development, testing, and evaluation of API capabilities, architectures, and frameworks. Employing systems engineering expertise and an understanding of DoD 8000-Series information security requirements, the contractor shall analyze requirements and strategies to develop high level architectures and technical requirements. Assist API leadership, requirements teams, customers, and capability delivery teams with formulating technical requirements, identifying alternative approaches. Provide technical leadership for the integration of requirements, design, and technology. Support API technical risk identification, mitigation, and monitoring. Assess the quality, viability, thoroughness, and completeness of requirements, identifying technical and schedule risks. Identify systems engineering technical considerations and associated problems, issues, or conflicts, evaluate current capabilities for effectiveness, and make recommendations regarding scalability. Establish and maintain a Code Repository to house API capabilities’ production code.

9.2 Technical Documentation. Recommend and prepare technical BPICs for the DAVE architecture and DAVE capabilities including RESTful design principles, 508 Compliance, and Coding Standards. Update API’s current BPICs to maintain currency with strategy, policy, and best practices. Review functional requirements documents to ensure the documents are technically accurate and compatible with technical environments. Provide input to the AV Data Framework (AVDF) architecture. Identify methods to obtain and disseminate data and provide technical enablement for those methods, e.g., data schemas, XML layouts, APIs, etc. Assist with the technical implementation strategy for data releasability controls and security marking. Accreditation and Certification and Authority to Operate efforts. Prepare and maintain the systems architecture artifacts, database design/schema, and diagrams for both NIPR and SIPR, As-Is and To-Be, in accordance with API’s Systems Architecture Framework. Address the System/Service Architecture, Capability Architectures, Technical Requirements, and Information Assurance Documentation. Minimum required artifacts include an API High-Level Cartoon, High-Level Interoperability, and High-Level Capabilities Overview; the DAVE Infrastructure View and DAVE MilCloud Infrastructure; AIR, and Analytics Layer Architectures, and API’s Capability Taxonomy. Maintain artifacts in Confluence or other designated government digital tool or site. Update artifacts within 30 calendar days after significant changes, major releases, and no less than annually.

9.3 Develop and maintain a written Agile Systems Engineering Management Framework for AV Capabilities. The Framework should address configuration management, code management, technical requirements management, use case management, architecture and design processes, technology stack, testing, DAVE interfaces, etc. Do not recreate a Waterfall-style Systems Engineering Management Plan, but provide a written Framework to document processes, identify required artifacts and artifact update frequencies, and promote continuity. Maintain the Framework in Confluence. Update the AV Agile Systems Engineering Management Framework within 30 calendar days after a major release or approval of a significant engineering/development strategy change, or no less than annually. Include the Framework as a BPIC.

9.4 Agile Test Management Strategy. Develop and implement an agile test management strategy BPIC for DAVE capabilities. The strategy shall provide standards, guidelines, templates, and methodologies used for testing. Develop interoperability test cases and automated test suites, and conduct interoperability testing. Assist with the testing efforts across all API teams including ensuring entry/exit criteria are met, test guidelines, templates, and methodologies are followed, test deliverables are completed on time, and conflicts are resolved. Emphasize Test-driven Development practices or other cutting edge testing methodologies to speed the delivery of functional capabilities. Update the API Agile Test Management Strategy BPIC within 30 calendar days after approval of a significant test strategy change, or no less than annually.

9.5 Technology Planning. Provide guidance to the Government PM/COR on commercial and government best practices in information technology applicable to API’s Capabilities. Support capability integration across API, to include existing systems, Component systems, and new systems. Provide guidance for Application integration, hosting, hosting agreements, service-level agreements, standards definition/oversight. Support the development and implementation of usage metrics and performance indicators. Develop strategies and recommendations, including cost-benefit analyses, risks for transition from current state, and impacts for any changes. Develop a Cyber Incident Response/Security/Spillage response plan for the organization. Rehearse the plan with the AV organization annually. Capture lessons learned from the rehearsal and update the plan annually. Update the plan within 30 calendar days after approval of a significant change and no less than annually.

9.6 Configuration Management. Perform configuration management of technical baseline standards and architectures and clearly identify proposed changes to the Government. Support Configuration Control Boards. Assess DAVE Capability usage individually and in relation to other Capabilities and develop an implementation plan to ensure that addition of a new Capability does not conflict technically or functionally with existing Capabilities or disrupt the overall integrity of DAVE.

9.7 Scrum Master / Development Lead Support. The contractor shall provide Scrum Master / Development Lead support to ensure quality coded Capability deliveries from API developers. Provide overarching technical guidance, cross-team arbitration, development methodology arbitration, and facilitation of quick decision-making on development questions including code organization, code framework usage and implementation strategies. Manage and maintain the code repository and maintain the reserved branches in accordance with the "Source Control (git)"; BPIC. Ensure configuration control of the DAVE codebase; review and accept/reject all pull requests and merges of code to ensure performance, reusability, interoperability, scalability, and maintainability (PRISM); ensure code coverage and unit testing best practices are adhered to, including no failing tests (in reserved branches) and 80% code coverage; design the physical database schema; make quick non-cost decisions on development/code-level implementation options to ensure quality while removing development roadblocks (particularly arbitration of team disagreements); assess – and recommend to the PM/COR for decision - development/code-level implementation options when the recommendation may increase cost or delay schedule; collaborate with development teams and individual developers for continuous improvement of the coded products; support capability development teams with planning and understanding existing code to ensure builds are compliant with BPICs, consistent in architecture, and support configuration management; mentor and provide technical guidance to scrum teams/ development teams; perform code reviews; develop release plans and package code for release candidates; and support Sprint planning.

10. TASK SIX. OPTIONAL Capability Delivery Surge (IUS Capitalize). The contractor shall provide Surge Capability Delivery support within 30 calendar days of a contract modification exercising this surge requirement. Provide additional support to develop, modify, test, or enhance Capabilities or integrate partner Capabilities from the Services, Components, and the 4th Estate with DAVE, and/or integrate DAVE with partner Capabilities. Develop enhanced Capabilities to make acquisition data readily available to facilitate Defense-wide analysis, portfolio management, or the digitization of acquisition to improve acquisition outcomes. Provide additional development, testing, and DevSecOps resources as necessary to increase Development Velocity by 45 Story Points per three-week Sprint.

11. PERFORMANCE METRICS. Report the following metrics quarterly to the COR:

11.1 Development Metrics. For DAVE and AIR Developers, the annual Sprint requirement is 16 three-week Sprints (Base Period: 9 Sprints). For DAVE, the government’s objective Sprint velocity is 80 to 90 completed Story Points per three-week Sprint. The government’s quarterly development standard is no less than 320 completed and tested (passing) Story Points per quarter that comply with the DAVE definition of Done. More or fewer than 80-90 story points may be delivered during any given three-week Sprint. For AIR, the government’s objective development velocity is a level of effort equivalent to 20 or more completed Story Points per three-week Sprint. For AIR, the government’s quarterly development velocity objective is no less than 60 completed Story Points per quarter. AIR velocities are objectives only due to the contractor’s reliance on DTIC staff.

11.2 DAVE Availability Metric. Purpose: Ensure DAVE capabilities are available for use during typical government hours for offices across the Continental United States. Standard: DAVE and DAVE applications (except AIR) maintain a 99.20% up-time percentage (not including scheduled down-time) during the hours of 8:00 a.m. Eastern time to 8:00 p.m. Eastern Time (5 p.m. Pacific) Monday - Friday, not including federal holidays. Methodology: contractor monitoring and tracking of the Production environment. Format: Total number of hours of downtime / total number of core hours during the quarter x 100%.

11.3 Test Case Automation. Purpose: Automated testing ensures testing keeps pace with delivery and is less prone to error. Standard: Measured quarterly, the objective automated test coverage is 90%; the minimum threshold is 75% automation. Methodology: Test ticket tracking. Exclude user stories/tasks/bugs that do not have a user facing or integrator facing feature from the automated test requirement. Format: Test tickets completed by automated testing / total test tickets (after exclusions) x 100%.

11.4 Mean Time to Recover. Purpose: To assess the DevSecOps team’s ability to maintain configuration control. Standard: Measured quarterly, the average time it takes to recover from an unplanned Production outage must be less than 30 minutes per incident during core hours (Monday – Friday from 8am-8pm). Methodology: contractor monitoring and tracking of the Production application. Format: Total Number of Minutes of Unplanned Outage / Total Number of Unplanned Outages equals Average Outage Duration.

12. GENERAL QUALITY STANDARDS. Written deliverables must be professional and meet the requirements set forth in this PWS. Upon delivery, deliverables become the property of the Government including all rights as specified in the contract’s data rights clauses. No contractor proprietary data is to be included in deliverables. Unless otherwise specified, the following performance standards apply to written deliverables:

12.1 Accuracy - Deliverables must be accurate in presentation, technical content, and adherence to accepted elements of style. Analysis must be sound, supported by research, well documented, and reflect the best commercial and DoD practices and standards.

12.2 Recommendations must be sound, compliant with policy and regulation, practical and actionable, and reflect the best commercial and DoD practices and standards. A recommendation must meet, as applicable, the following quality criteria:

· Suitable - Does it solve the problem?

· Feasible - Does it fit within available resources?

· Acceptable - Is it worth the cost or risk?

· Distinguishable - Does it differ significantly from other recommendations?

· Complete - Does it solve the problem from start to finish?

12.3 Documents for Government signature must be formatted as specified by the Government, to include the DoD Manual for Correspondence Management and must be error free in DoD format, style, grammar, and spelling to the maximum extent practicable. Non-signature documents must be generally free of grammar and spelling errors (errors do not present a distraction to the reader or hinder understanding of content).

12.4 Appearance - All deliverables must be neat and templated to reflect an OUSD(A&S) work product.

12.5 Clarity - Deliverables must be clear and concise, and easily understood in a single, rapid reading. Diagrams must be easy to understand and be relevant to the supporting narrative.

12.6 File Editing - Text and diagrams must be editable by the Government using standard DoD-licensed software. Submit Deliverables electronically. Submit in hard copy by exception only, as mutually agreed upon prior to submission. Store a digital copy of all deliverables (other than proprietary contract performance reports) on shared drives or other locations universally accessible to the supported Government staff.

12.7 Timeliness - Submit Deliverables on or before the due date specified by the Government. If a deliverable will miss a suspense date, the contractor must notify the Government COR prior to the suspense date, advise when the item will be completed, and request an extension. Short extensions may be granted at the discretion of the Government.

13. DELIVERABLES AND PERFORMANCE STANDARDS

THE FOLLOWING DELIVERABLES ARE DUE ONLY IF OPTIONAL TASKS ARE EXERCISED:

13.1 Quarterly Report. Reports are not required monthly due to the frequency of Sprint reviews. Submit a performance report quarterly, no later than five business days after the completion of each quarter, as measured from the contract performance period (PoP) start date. For the Base period, submit an initial abbreviated report regarding onboarding and transition, 30 calendar days after PoP start, and quarterly reports thereafter.

QUARTERLY PERFORMANCE REPORT

Provide up to five bullets per performance area below. Where possible, use “CAR” statements. C=Conflict. Describe the problem/issue/situation. A=Action. Describe the actions taken to resolve the problem. R=Result. Describe the benefit to the Government. Example: When AIR Refactor development velocity fell below the threshold for two Sprints (C), we adjusted team structure to optimize developer front end skills (A) exceeding threshold velocity and meeting objective velocity for subsequent Sprints (R).

Performance Quality

· Address Sprint velocity, capability delivery, and testing success, efforts to reduce re-work, fulfillment of PWS requirements, etc.

· Report each of the four performance metrics for the quarter noted in Paragraph 11. These metrics are not counted as part of the five bullets.

Schedule

· Address on-time delivery of completed stories and capabilities, meeting release milestones, efforts that helped API meet scheduled release deadlines, etc.

Cost Control

· ODCs anticipated next quarter; efforts taken to control ODC costs.

Management

· Address items such as team cooperation, responsiveness, sub-contractor management, program management, team personnel and key personnel management, vacancies, turnover reductions, etc.

· Address any required assistance from the PM/COR

Regulatory Compliance

· Address items such as contract compliance, STIG & BPIC compliance, security procedure compliance, etc.

13.2 Inspection and Acceptance of Deliverables. Unless otherwise specified by the COR for a particular deliverable, the Government has a maximum of ten business days from the day a deliverable is received to review the deliverable(s), provide comments back to the contractor, and accept or reject the deliverable(s). Unless the Government informs the contractor before ten business days has elapsed that additional time for review is necessary, deliverables will be considered accepted by default after ten business days even if the Government takes no action/provides no feedback. However, if a deliverable is accepted by default, the Government may still direct changes at no cost. Late Government-directed changes will not impact the contractor’s ability to invoice or impact assessments of timeliness of delivery. Unless otherwise specified by the COR for a particular deliverable, the contractor will have ten business days from the day comments are received to incorporate changes and submit the revised deliverable to the Government.

13.3 Quality Control. The contractor must establish and maintain an effective quality control program to ensure ordered services are of acceptable quality. The contractor must establish and implement procedures to identify, prevent, and ensure non-recurrence of defective services. The contractor’s quality control program ensures work complies with the requirement of the contract. As a minimum, the contractor must establish and document quality control procedures that address 1) the General Performance Standards listed above, and 2) include controls to ensure adherence to performance plans and schedules. The contractor’s must submit its quality control plan to the Contracting Officer no later than 45 days after contract award and update it annually.

14. KEY PERSONNEL REQUIREMENTS. Key Personnel provide program continuity and assure the Government that appropriately qualified replacements are provided for essential positions. The contractor may replace a key person any time after the first 90 days, provided that the résumé of a similarly qualified replacement has been reviewed and accepted by the AV Program Manager in advance. If a résumé review is insufficient to verify a replacement’s qualifications, the Government may require an in-person meeting.

14.1 Program Manager (PM). The contractor PM provides leadership, direction, and management Support for contract and contractor team. The PM must have managerial, technical, and business knowledge, and experience relating to software systems. The PM must have a demonstrated ability to lead and solve complex issues involving staff and resources of sizable magnitude. The PM must be capable of thinking strategically to address issues and implement plans for software programs. Responsible for overall contract performance. The PM must have at least three years of hands-on software development experience and at least five additional years of software project and program management experience (total of eight or more years of experience). Must possess a DoD Secret Clearance.

14.2 Lead Application Developer (Key). The Lead Application Developer is responsible for planning and coordination of the Development Team’s support. The Lead Application Developer is a working lead, develops code and instructs, assigns, directs, and checks the work of other team members. The individual must be proficient in front-end/back-end programming and have four years or more of experience using more than half of the tools of the DAVE technology stack. Must have passed a National Agency Check with Inquiries. Must be DoD Secret clearable.

14.3 AIR Sustainment Developer (Key). The individual must be proficient in front-end/back-end programming and have two years or more of experience with Confluence AND four years of experience with Angular. Must have passed a National Agency Check with Inquiries. Must be DoD Secret clearable.

14.4 Lead Tester (Key). The Lead Tester is a working team leader, and has at least four years or more of experience testing capabilities developed using more than half of the tools of the DAVE technology stack. Must have passed a National Agency Check with Inquiries. Must be DoD Secret clearable.

14.5 DevSecOps Lead (Key). The DevSecOps Lead requires broad experience across multiple systems administration disciplines. Eight or more years total of DevSecOps experience is required. The DevSecOps Lead must have a well-rounded blend of skills and experience including Configuration Management, Risk Management Framework, Enterprise Architecture, and Network Specialist experience. Must possess a DoD Secret Clearance.

14.6 OPTIONAL Development Lead (Key). Must have full stack development experience using Java and Angular, and eight or more years of experience with multi-team Agile Development. Must possess a DoD Secret Clearance. Key Person required only upon Option Exercise.

14.7 OPTIONAL Lead Systems Engineer (Key). The Lead Systems Engineer must have eight or more years of experience leading and integrating disparate development teams, resolving conflicts between technical strategies, integrating, and coordinating the work of multiple application development teams, and architecting new capability driven designs. The Lead Engineer must have a working knowledge of Java, JavaScript, Angular, and Atlassian products. The Lead Engineer must have demonstrated experience employing API-First strategies. Must possess a DoD Secret Clearance. Key Person required only upon Option Exercise.

15. GOVERNMENT FURNISHED SUPPORT

15.1 Government-Furnished Information (GFI). The Government will provide access as needed to acquisition documentation, acquisition data, acquisition policies, information technologies and information assurance, DoD Instructions, Directive-type Memoranda, Security Technical Implementation Guides (STIGs), programs, processes, and schedules to facilitate the contractor’s review and analysis, as required. Information will also include the AV Data Matrix/AV Dat Framework, BPICs, data, artifacts, requirements documents, process maps, acquisition databases, and other reference material. The Government will provide access to unclassified Government networks and email.

15.2 Government-Furnished Equipment (GFE). Common Access Cards are GFE that must be used, protected, and accounted for. It is the responsibility of the contractor to have a system of internal controls to account for, preserve, and protect GFE/GFI in its possession. The Government will supply access to existing licensed software and may supply additional software as GFE. The Government will provide limited access to hotel office space in the Mark Center Suite 15F09-02 as needed and as available for Key Personnel on an ad hoc basis. Hotel spaces will include phones, desktop workstations, OSD Network access, and fax and copy machines. The Government will issue government laptops for personnel requiring access to Controlled Unclassified Information and/or government network access. Government laptops lack the computing power to support development.

15.3 Return of GFE/GFI. At the request of the Government, or at completion of this effort, the contractor will immediately return any Government-provided property and information, including any equipment, electronic media, specialized or off-the-shelf software, licenses, maintenance agreements, service agreements, warranties, product keys, API-unique customer account codes, and all other property provided by the Government or charged to the Government, for the contractor’s use, to complete this effort.

16. CONTRACTOR FURNISHED SUPPORT

16.1 Contractor Furnished Software.