Sustainment.Development PWS RFI_07.31.24.pdf

Section 2: PERFORMANCE WORK STATEMENT

2.1 BACKGROUND

The goal of this requirement is to sustain and enhance the Defense Logistics Agency’s (DLA’s) modern e-Commerce system. This system is for Federal, DOD, state and authorized local agencies customers. DLA has developed a single Government Electronic Mall (i.e., FedMall) to provide one-stop shopping for customers. The system is configured on a state-of-the-art, Commercial off the Shelf (COTS) e-commerce platform comprised of a hybrid solution of an HCL e-commerce suite of applications and Amazon microservice applications: HCL (previously IBM WebSphere) Commerce (runs FedMall front end shopping environment and custom applications), FedMall Enhanced Data System (FEDS) (Manages supplier, user, and program data for FedMall), Application Connect Enterprise (ACE) (interfaces FedMall with internal and external systems/components), FedMall Supplier Portal (runs the environment for suppliers to register, upload and manage catalogs, and view purchase orders), and FedMall Custom Utilities. FedMall is virtualized using RHEL 8, AWS EKS, and other AWS services to meet current and future system needs. The program is broken up into three tiers: application, database, and a web DMZ of servers clustered behind load balancers. There are approximately 32 servers and an EKS Cluster. All operating environments are hosted in Amazon Web Services (AWS) West GovCloud in the Procurement Integrated Enterprise Environment

(PIEE).

Starting in 2020, FedMall underwent a system upgrade to replace the monolithic IBM products and architecture to a microservices architecture built with open-source products. This decision was made to allow more flexibility for system updates, deployments, and scale to meet demand for specific functions of FedMall users. Additionally, the FedMall PMO is committed to an agile software development using DevSecOps and a continuous integration/continuous development (CI/CD) pipeline.

Figure 1: FedMall Application Architecture (09/2021 –Present)

DLA’s e-commerce system brings FedMall customers who need commercial and military products and services and DOD approved suppliers together in an efficient electronic marketplace. FedMall users have access to over 76 million individual items of supply, from centrally managed DOD and GSA assets to commercial products, with over 1,800 commercial contracts in place: generally, over 70% small business. The system contains over 35,000 active users (up to 2500 concurrent users) and about 700 suppliers, and processed roughly 350,000 transactions, amounting to over $300 million in sales. The application predominantly supports EDI 850s and 511Rs, batch MILSTRIP transactions (up to 50,000), and single line MILSTRIP transactions. FedMall business partners include Defense Automatic Addressing System (DAAS), Defense Information Systems Agency (DISA), Pay.gov, SAM.gov, National Geospatial-Intelligence Agency (NGA), and other DLA systems. The functionally is similar to commercial e-commerce solutions, with tailored capability to meet the unique functional requirements of the Federal and DOD ordering environments. This strategy enables the DOD to streamline the ordering process for commercial items and eliminates the traditional labor-intensive process of identifying sources of supply through supplier brochures, paper catalogs, phone book searches, or a series of Internet catalog searches. FedMall adds value for the user by making ordering more effective and efficient, while maintaining high standards of regulatory and reporting compliance across customer-connected systems.

Additionally, the e-commerce system provides various transactional data to Government oversight organizations, to include interactions with legacy financial systems within the DOD, streamlining the web ordering process for DOD.

The e-commerce system accepts Military Standard Requisitioning & Issue Procedures (MILSTRIP), Government Purchase Card (GPC), and Federal Standard Requisitioning & Issue Procedures (FEDSTRIP), payment methods, which supports the requirement for Payment Card Industry Data Security Standards (PCI-DSS) compliance.

2.2 SCOPE

The contractor shall be required to provide all labor, tools, materials, licenses and maintenance, and other services (and optionally additional hardware and software) necessary to fulfill the requirements as set forth in this PWS.

FedMall is currently hosted in AWS West GovCloud. Based on current contracting actions, the cloud environment could change during late FY25. An optional CLIN may be added for additional hardware and software support. This PWS will provide the foundation and outline the activities that will be required to develop and sustain FedMall. In addition, the contractor will be required to manage the modernization and rationalization of the Subsistence Total Order and Receipting System (STORES) application onto the FedMall platform, as outlined in section 2.3.3.1.

• The contractor will operate, maintain, and sustain FedMall as specified in this PWS, allowing for both contractor and Government sustainment for some parts of the system. The current environments will be supported, as well as other environments/hosting facilities that may be added in the future.

• The contractor should be prepared to start on day 1 of the contract.

• The present sustainment team would be on a transition out period not to exceed 90 days for knowledge transfer to the awarded sustainment team.

• The contractor shall work with the hosting provider as required.

• The contractor shall provide project management oversight for all the activities detailed in this PWS

• The contractor shall ensure STIG, Auditing, and Security Compliance is achieved during the development and delivery of all functionalities (new or sustainment).

• The contractor shall ensure Web inspect vulnerability remediation for all active findings test and production url's.

• The contractor will maintain separate sustainment and development teams.

• The contractor will develop and manage a DEVSECOPS pipeline to deploy continuous releases, monthly or at least quarterly, of FedMall in accordance with a recognized Software Development Life Cycle (SDLC) process (i.e. Agile CI/CD), during off-peak hours.

• The contractor shall be required to provide training on the use and administration of the system to various Government personnel, contractors, and vendors.

This task order will be evaluated in accordance with Tradeoff source selection procedures. Award of this task order will be made on a competitive best value basis, using “tradeoff” procedures between non-price and price. In evaluating task order proposals, the non-price factor(s) is/are considered significantly more important than price. The Government may consider award to other than the lowest priced Contractors or other than the highest technically rated Contractors.

The criteria used to select the contractor for award of this task order will be based on an evaluation of the contractor’s proposed labor mix and allocation along with its proposed price as compared to the government’s estimate of the work to be performed.

Non-Price Factor – Labor Mix: The contractor’s quality of proposed personnel will be evaluated based on an assessment of the level of experience and skill necessary to minimize Government risk and successfully complete the technical requirements outlined in the task order PWS. Proposed labor mix and allocation will be evaluated on experience, skills and qualifications as they relate to the requirements of the task order PWS. Contractors are required to submit copies of resumes for all key personnel (and key personnel substitutions) proposed in response to this task order PWS.

Non-Price Factor – Technical Approach: Describe in detail your approach to develop the specified deliverables and perform each tasking in the PWS. The Contractor must provide evidence demonstrating a clear understanding of the scope and complexity of the requirements set forth in the PWS. The proposed approach must be well-defined and logical and include the methodologies and processes that can successfully meet DLA objectives and mitigate risk.

Non-Price Factor – Past Performance: The Government will consider the contractor’s past performance/experience under U.S. Government contracts or other contracts/projects within 3 years of the initial closing date of this RFQ. Any contracts/projects submitted in reference to past performance that are outside of the 3-year recency period will not be considered for evaluation. Contractors shall submit at least one past performance example, but not more than two examples. Contractors may submit JETS task orders as past performance examples.

Note: In the case of a contractor without a record of relevant past performance or for whom information on past performance is not available, the contractor will not be evaluated favorably or unfavorably on past performance and will receive a Neutral rating.

In evaluation of this factor, the following aspects will be considered:

Aspect One: Relevancy of Prior Contracts - The relevancy of contractor’s prior projects to the current PWS requirements will be evaluated based on the similarity to the service/support, the complexity, and dollar value. The Government will evaluate the contractor’s demonstrated evidence of experience in performing contracts with a similar size, scope, and complexity to this task order requirement.

Aspect Two: Quality of Past Performance (i.e., how well the contractor performed) - The contractor’s history of success in completion of projects, history of producing high-quality reports and deliverables, and commitment to customer satisfaction and history of staying on schedule and within budget will be evaluated via CPARS/PPIRS; information provided in the contractor’s offer;

and/or with the Government’s independent contact with the contractor’s prior customers. Degree of conformance to contractual requirements, responsiveness, and effectiveness in addressing issues, quality of customer service, and/or conformance to cost/price requirements or goals, will be evaluated in the qualitative past performance ratings.

Price: The proposed task order pricing will be evaluated to determine it is in accordance with the IDIQ.

Pricing will be evaluated to ensure completeness and reasonableness.

2.3 TASK REQUIREMENTS

The Contractor will be required to perform tasks such as the following:

2.3.1 TASK 1 – Non-Capital Project Management (Reference IDIQ RFP SP4701-15-R-0030 Task Area 10)

The Contractor Project Manager shall oversee all aspects of the sustainment, including planning, organizing, staffing, directing, integrating, and controlling activities necessary for all tasks. To meet cost, schedule, and performance requirements, the Project Manager must provide diligent, responsive, and timely performance reporting to the Government Program Management Office (PMO). Coordination with the stakeholders, partners, and hosting activities will also be necessary. The Project Manager may be required to support meetings with the functional community to understand requirements and issues.

Specific Tasks for Project Management - The contractor shall:

2.3.1.1 Task Order Kickoff

Coordinate, schedule, participate in, and document meeting minutes/action items for the Task Order kickoff meeting. Non-Disclosure Agreements shall be submitted prior to the Kickoff meeting for all Contractor personnel attending and, for all others, within 5 business days of starting on the Task Order.

Deliverables:

• Task Order Kick Off Meeting, Minutes/Action Items

• Non-Disclosure Agreements

2.3.1.2 Personnel Resource/Training Spreadsheet

Develop and maintain a Personnel Resource/Training Spreadsheet to identify contractor personnel resources and training information. The contractor shall submit the initial Personnel Resource/Training Spreadsheet within five (5) days of award. The contractor shall provide an updated electronic spreadsheet to the Contracting Officer Representative (COR) within one (1) business day of the arrival or departure of any personnel working on the contract. The spreadsheet shall include, at a minimum:

• Contract number

• Employee name

• Identification of whether the employee is considered key personnel or non-key personnel

• Company name (working for prime or sub-Contractor)

• Work site (city & state)

• Work phone

• Work e-mail

• Physical/Offsite location (city & state)

• Job Title

• Labor category

• Functional area

• Incoming Date

• Outgoing Date

• CAC information (EDIPI, status, status date, MPAS)

• System Access(es)

• Technical and/or management IA categories

• IA waivers (start and expiration dates)

• Security Clearance / IT Level

• Date of investigation

• 8570/8140 and/or other Certifications, i.e., certification(s) required of position, certification(s) held in position, certification expiration dates, certification renewal dates.

• Name and completion date of required training

The contractor shall maintain copies of IA certifications or waivers, and training records, designation letters, and certificates on site and make them available for the Government to review upon request. The training records shall include, at a minimum, the name of the employee, the name of the course, the source of the training, a description of the training provided, and the date the employee successfully completed the training.

To ensure that all employees know and understand regulations and policy pertaining to physical, information, operations, and personnel security, contractor personnel shall, at a minimum, complete the following training (typically online) annually to comply with DOD regulatory requirements:

• Cyber Awareness Challenge

• Personally Identifiable Awareness Training

• Antiterrorism/Force Protection Level One training

• Operations Security training

• Counterintelligence Awareness training

• Combating Trafficking in Persons training

• Audit Readiness training

The contractor shall be notified by the COR of any new/additional/supplemental DLA mandatory training requirements. Upon receipt of the training notice, the contractor shall submit weekly training status updates to the COR until all designated resources have completed the training.

Deliverable:

• Personnel Resource/Training Spreadsheet

2.3.1.3 Contract Work Breakdown Structure (CWBS) and Subcontracting Plan and Reports

The prime contractor shall manage and administer a system of internal controls for all management, administrative, and system engineering tasks. Management of these operations shall include integrating common systems of internal controls across the sub-contractor’s performance under this task order, and implementing processes that are risk-based, cross-functional, and cost effective to optimize and streamline operations. This includes:

Perform overall integrated planning, management, sustainment, software development, integration, project management, testing, training, and support of FedMall.

Deliverable:

• Contract Work Breakdown Structure (CWBS)

2.3.1.4 Project Management Plan (PMP)

Submit (15 days after award) and maintain a Project Management Plan (PMP), to include transition-in activities with the present sustainment team transitioning out, schedule, performance, quality, risk, and personnel management.

• Project Management Plan (PMP) in Microsoft Word format (.doc/.docx)

2.3.1.5 Integrated Master Schedule (IMS)

Submit (within 30 days of award) and maintain a complete and detailed Integrated Master Schedule (IMS) covering all task areas under 2.3 Task Requirements. Link the IMS to as many sub-project schedules as represented in the IMS.

The IMS shall contain a single baseline schedule which shall be approved by the PMO. The updated IMS should clearly identify schedule variance to the baseline and should provide resource loading to identify potential conflicts or schedule impact.

The Contractor should update the IMS to be current as of the Friday preceding delivery to the FedMall PMO. In updating the schedules, the Contractor shall minimally update the Percent Complete, Actual Start, and Actual Finish attributes for activities in progress. The Contractor shall revise planned start and finish dates for all activities that have not met their scheduled start or finish dates.

The IMS delivered shall contain the Contractor’s current, best estimate of planned start and finished dates of all unfinished activities. Finished activities shall include their actual start and finish dates.

Deliverables:

• Integrated Master Schedule (IMS) in Microsoft Project (.mpp) or compatible software where the deliverable can be opened in Microsoft Project, as well as a copy in .PDF format.

2.3.1.6 Weekly Status Meeting

Participate in and document a Weekly Status Meeting. Contractor shall prepare and provide the Weekly Status Meeting briefing materials/slide deck to the PMO at least two (2) days prior to the meeting and provide the minutes/action items within 5 days of each meeting.

Deliverables:

• Weekly Status Meeting briefing materials/slide deck, Minutes/Action Items

2.3.1.7 Monthly Status Reports (MSRs)

The contractor shall include a written performance self-assessment as part of the MSR. The self-assessment will be all encompassing to give a holistic understanding of contractor performance and development delivery and should include pertinent information related to each PWS task area, and other items as requested by the PMO.

Deliverable:

• Monthly Status Report (MSR)

2.3.1.8 In-Process Reviews (IPRs)

Participate in and document monthly IPRs. Contractor shall prepare and provide the IPR briefing materials/slide deck to the PMO at least two (2) days prior to the meeting and provide the minutes/action items within 5 days of each meeting.

Deliverable:

• IPR briefing materials; meeting minutes/action items

2.3.1.9 Contractor labor hour reporting

Report all contractor labor hours (including subcontractor labor hours) required for performance of services provided under this contract for the Defense Logistics Agency via the Enterprise-Wide Contractor Manpower Reporting Application (ECMRA). The ECMRA can be accessed at the following web address: http://www.ecmra.mil/.

Reporting will be for the labor executed during the applicable Government fiscal year (FY), which runs October 1 through September 30. The contractor shall report all labor data no later than October 31 each year for the reporting period ending September 30 of that calendar year. Labor hours maybe input at any time during the FY. Contractors may direct questions to the help desk at: http://www.ecmra.mil/.

Reference: DLAD 37.103-90 Contracting Officer Responsibility.

Deliverable:

• ECMRA Reporting

2.3.1.10 Monthly 8570 / 8140 Report

The contractor shall provide a monthly 8570/8140 deliverable report to the COR/COTR that includes a list of applicable 8570/8140 key personnel. The following information shall be provided in the report, due by the 5th calendar day of each month:

http://www.ecmra.mil/ http://www.ecmra.mil/

• Change Status (No Change, Update, New/Add, Remove)

• Name

• IA Category

• IA Level

• Company

• Contract Name/Description

• COR

• Contract # / Task Order #

• Assigned DLA Location

• Onsite/Offsite

• Primary/Baseline IA Certification Held

• Date of Primary/Baseline IA Certification

• Date Primary/Baseline IA Certification Expires

• Primary CE Certification Held

• Date Primary CE Certification Received

• Primary IA Certification Status (Current or Past Due)

• Primary CE Certification Status (Current or Past Due)

8570/8140 contractor personnel identified on this contract shall have their required DoD 8570/8140 security and industry competency certifications (per below) prior to starting work and maintain their certifications throughout the life of this task order. Industry certifications are used to meet Computing Environment certification requirements. Refer to the DOD 8570 CE IA matrix for more information.

http://iase.disa.mil/iawip/Pages/iabaseline.aspx. Certification requirements are dependent on the level of access to the component (e.g. Red Hat certification for Red Hat Admin access). The first report shall be a comprehensive report covering all 8570/8140 personnel. The contractor is required to report updates on a monthly basis thereafter (or report no changes), unless an 8570/8140 resource leaves the project, the contractor shall provide notice within one (1) business day of the departure. See Attachment X for the 8570/8140 Deliverable Report template. If a certification becomes obsolete during the life of the contract, the Contractor shall be responsible for ensuring applicable personnel obtain an acceptable replacement certification prior to the IT product’s end of life.

• Monthly 8570/8140 Report

2.3.2 TASK 2 – Non-Capital Operations, Maintenance, and Sustainment (Reference IDIQ RFP SP4701-15-R-0030 Task Area 4)

The Contractor shall operate, sustain, and maintain FedMall. The Contractor shall use best practice procedures for software configuration, release management, secure coding, and IT services delivery. The Contractor shall use established Government tools to track and document changes when making system modifications or adding functionality (e.g. ServiceNow and/or Atlassian tools). Issue tracking for bug/break/fixes are managed in Service Now (ie. SNOW). Where applicable, the Contractor shall coordinate, organize, schedule, and facilitate agile technical design reviews, to include Page Wireframes (mock-ups), prototypes and demonstrations. Prior to each release, The Contractor shall notify the PMO if any DLA Enterprise Architecture documents will require change during the release. The Contractor shall http://iase.disa.mil/iawip/Pages/iabaseline.aspx update all applicable DLA Enterprise Architecture documents and systems within 15 days of any release that would alter them.

The Contractor shall maintain a team for Operations, Maintenance and Sustainment. The government anticipates this team will be led by the Operational Technology Specialist - Senior who shall share the backlog of Sustainment/ O&M requirements with the PMO Sustainment Manager to ensure prioritization of fixes matches PMO expectations. At the end of each development Sprint, enhanced functionality and/or bug fixes shall be demonstrated to the PMO prior to PMO testing and delivery, ensuring the requirement meets/exceeds Stakeholder(s)/PMO expectations.

The Contractor shall provide product expertise and support, to include the technical architecture/infrastructure hardware and software, and application, system and database administration, security, and related management, oversight, and support to the Government at all FedMall hosting sites, and if environments are moved between locations. The Contractor shall include an improvement to O&M automation, where applicable, to include the following:

• Log monitoring, file exchanges, reporting, system health and activities

• Auto-detecting connectivity to dependent external services

• Auto-detecting interface issues with external applications supporting FedMall

The requirements should be presented to the PMO for consideration, and upon approval, be added to the FedMall Sustainment Product Backlog in JIRA for development and deployment prioritization.

The Contractor shall maintain the application, database, and data environments software, ensuring that all environments remain consistent and compliant with directives. The Contractor shall refresh all FedMall environments ensuring that they are kept in a state that they can be reconstituted or restored when needed.

The Contractor shall be responsible for providing on-call support for resolving Severity 1 incidents, 24x7x365. The Contractor shall be responsible for resolving issues of the day-to-day operations of the system and interfaces. The contractor shall review and analyze incidents and implement fixes for Severity 1 and Severity 2 defects that meet the following criteria:

Severity Severity Description

Severity Characteristics

1 Emergency Application is down or is otherwise unavailable to entire customer base. Serious application component failure that must be resolved immediately.

2 Critical Key application functionality is down or unusable with a high number of customers affected.

3 Major An error that does not significantly impact using the system. A flaw in the requirements or functional specification that will annoy most users. Major loss of service; the impact is restricted to few accounts.

4 Normal An error that will distress most users. It has no design documentation, requirements or specifications documenting the process. No loss of service; issues include incorrect message wording, formatting, or minor system errors. This would also include enhancement requests

The contractor shall draft a Plan of Action & Milestones (POA&M) and/or Authorizing Official Risk Acceptance (AORA) when Severity 1 and Severity 2 fixes cannot be implemented immediately. The contractor shall implement fixes for Severity 3 and Severity 4 defects that meet the criteria above. The Contractor shall coordinate and perform all application and database CM and restoration operations. The contractor shall perform security scans and analyses on source code and remediate findings. The contractor shall review logs related to the Audit Log Review Supplemental Implementation Guide (ALRSIG) weekly and provide reports. The contractor shall document their key management process and provide a Key Management Plan (KMP) to the PMO. The contractor shall deliver all documents in DLA specified formats unless otherwise agreed to by the PMO. The contractor shall provide support for annual penetration testing, incident response, and Continuity of Operations (COOP) exercises. The contractor shall proactively monitor the system for intrusions to detect any unauthorized changes and will alert the FedMall PMO immediately during confirmed or suspected events. The contractor shall subscribe to vulnerability alert channels to receive notices of current security weaknesses to maintain situational awareness with respect to products being used by the information system. The contractor shall deliver operational, user friendly software that sufficiently meets the defined requirements, to include all security and section 508 requirements, in agreed upon Sprint cycle(s).

If upgrades or changes to the system must be done, every effort should be made to make these updates during off peak hours. Off peak hours as defined under the resulting contract are as follows:

• Monday through Thursday - 1800 hours through 0600 hours (the following day)

• Weekends – Friday beginning no earlier than 1800 hours through Monday 0600 hours

All work under this contract shall comply with the latest version of all applicable Government regulations/standards to include, but not limited to: Federal and DOD Regulations (FAR, DFARS, etc.), Instructions and Manuals; Acquisition Bulletins (AB); American National Standards Institute (ANSI) standards; and National Institute of Standards and Technology (NIST) standards (e.g. Special Publication 800-53), including Federal Information Processing Standards (FIPS) publications (e.g. 140 compliant encryption), Federal Risk and Authorization Management Program (FedRAMP) and PDS.

The contractor shall setup all applicable environments, at least two test (pre-production) environments and Production, with the ability to stand up a COOP environment as needed. The contractor shall establish development code, properties, and configuration structure, code and configure the deployment process.

PMO/the Government will be provisioned access to all cloud-hosted environments (FIT, SITA & B, COOP, PROD). The contractor will maintain their own environment for development; development will not be done in the government-funded FIT environments.

2.3.2.1 Business Requirements Analysis (Reference IDIQ RFP SP4701-15-R-0030 Task Area 5)

Upon reception of a Business Requirement, the Contractor shall take a systematic approach to identify, document, and control all updates to the Functional Baseline and System Requirements throughout the project. The contractor shall work with the FedMall PMO to ensure capture and eventual translation of all business and functional requirements into appropriate system specifications. The contractor shall manage these requirements using the Atlassian Product Suite (to include Jira & Confluence). Atlassian products should be configured to maximize PMO collaboration, including providing PMO edit/write/view access.

Requirements will be captured as User Stories that are independent, valuable, small, testable, and are able to be point estimated. Depending on the size and scope, this would begin by the assigned Business Analyst eliciting and analyzing stakeholder feedback to ensure functional requirements, User Stories, and Use Cases are well-defined, fully understood, and agreed-upon by the Stakeholder(s), the FedMall Requirements Manager, the UI/UX Designer, and other FedMall personnel as necessary. Each Business Requirement must have an assigned Technical Lead to oversee the project from start to finish, consulting as a technical SME during business requirements analysis.

The Business Analyst is responsible for any necessary specifications, and the creation of (or updates to) Business Process Mappings (BPMs) and other relevant workflow diagrams. Changes to the baseline will be scoped, and mark-up of BPMs and/or DODAAF artifacts will be performed to demonstrate the change in scope and for a collective understanding of the change(s). All relevant documentation will be created and maintained in Confluence for direct integration/relationship to JIRA Epics/Stories. The Government, Stakeholder(s), and the contractor will work collaboratively to decide on the requirements’ acceptance criteria, also known as the “definition of done.” The contractor requires FedMall PMO approval to begin any development work with the absence of documented JIRA Epic(s)/Stories or existing Epic(s)/Stories with incomplete success criteria.

In addition to system design and specification documentation, the contractor shall begin drafting a Requirements Traceability Matrix (RTM) based on identified business and functional requirements. The RTM will have backward traceability to all business requirements and forward traceability to all test cases.

As requirements are iteratively identified, documented, and agreed-upon, the UI/UX Designer will create mock-ups and/or prototypes of the work for presentation and approval from both a design and functional requirements implementation perspective. After PMO and Stakeholder(s) UI/UX approval, the requirement is Backlog Ready and will be sent for a deeper technical analysis and peer-reviewed technical approach.

Deliverables:

• JIRA Product Backlog

• Business Requirements Analysis Documentation to include white papers, change requests, and functional requirements descriptions in Confluence to include Use Cases and User Stories walkthrough

• BPMs, Workflows, DODAAF Analysis/Draft and Final Changes

• Testable User Stories, Use Cases, and Acceptance Criteria in JIRA

• Requirements Traceability Matrix (backward and forward traceability user stories)

2.3.2.2 Technical Requirements Analysis (Reference IDIQ RFP SP4701-15-R-0030 Task Area 16)

Further to the details outlined in section 2.3.2.1 Business Requirements Analysis, throughout the requirements gathering process and requirement scoping, the Solution Architect shall ensure requirements are befitting and well-suited into the overall technical architecture. In coordination with the Business Analyst(s), they shall consistently review for overlapping and/or conflicting business/functional/technical/system requirements. The Solution Architect will also analyze the proposed technical solution for more effective and/or efficient implementation methodologies and/or technical solutions. For example, “a COTS product suffices for X% of the requirements set at a fraction of the resources expected to expend”.

The assigned Development Team will perform further Technical Analysis, verifying the alignment of the technical aspects of the project as well as technical feasibility, complexity, and a peer-review of the technical approach. Once a technical approach is agreed upon – which as a default shall consist of the least complex and the least resource intensive technical solution – an Agile Scrum Refinement shall ensue. All meeting minutes for Scrum and business/technical requirement meetings is the responsibility of, and will be captured by, the Contractor and posted in JIRA/Confluence and provided per the deliverable requirements outlined in section 2.5. In addition, the technical approach must also be documented, generally prior to development work being performed. The contractor requires FedMall PMO approval to begin development work with the absence of a documented technical approach.

The contractor shall define the Sprint scope and objectives based on the functional, technical, and system requirements. The contractor shall work with the PMO to ensure translation of requirements into appropriate system specifications. As in 2.3.2.1 Business Requirements Analysis, any changes to the baseline will be scoped, and mark-up of BPMs and/or DODAAF artifacts will be drafted to demonstrate the change in scope and for a collective understanding of these change(s).

The Solution Architect, the lead Technical Advisor, and the Development Team shall identify and document project and technical risks during both the Business Requirements Analysis and Technical Requirements Analysis to prevent rework, reduce downstream testing issues, and decrease changes and/or the impact(s) from any forthcoming changes.

Throughout all parts of this systematic process, the team shall ensure the functional requirements and technical solutions match not only the business goals, but also meet/exceed customer expectations as laid out within the initial business requirement and throughout any/all periodic stakeholder reviews.

Deliverables:

• Technical Analysis and solution for functional requirements

• Jira Sustainment Product Backlog to include how O&M automation requirements will be captured, developed, and deployed

• BPMs, Workflows, DODAAF Analysis/Draft and Final Changes

• Meeting minutes

• Documented technical solutions in Jira and Confluence

• Documented project and technical risks

2.3.2.3 Monitoring and Incident Support (Reference IDIQ RFP SP4701-15-R-0030 Task Areas 3)

The Contractor Sustainment Team shall be able to perform the following functions for FedMall support:

• Respond to system outage notifications, system alerts, etc. and provide root cause analysis

• Analyze all incidents, perform troubleshooting, and recommend appropriate resolutions

• Correct or otherwise resolve incidents

• Support restarts

• Provide system monitoring

• Validate/analyze product performance

• Recommend and document process/technical improvements/lessons learned and implement when/if approved

• Proactively monitor system for unauthorized changes or intrusions

• Weekly review of audit logs with a report to the FEDMALL ISSM and PMO

Actual Log Reviews are as follows: Daily for SA Logs and Weekly at a minimum for Application or as adjusted by updates to the DLA Audit Implementation Guide.

Audit Review, Analysis, and Reporting:

(1) Frequency of log review:

a. Server event logs will be reviewed daily for evidence of abnormal or malicious activity.

b. Segregation Of Duties Controller will review elevated privilege job role use audit logs within five business days.

c. At a minimum, all other audit logs will be reviewed weekly for activity of abnormal or malicious activity.

Deliverables:

• Incident response and resolutions with root cause and lessons learned identified, system restarts and monitoring

• POA&Ms

• Weekly audit log review and report

2.3.2.4 Support calls and meetings (Reference IDIQ RFP SP4701-15-R-0030 Task Areas 10)

Contractor personnel shall be prepared to actively participate in weekly system support calls by having status updates ready and be available for any questions from the customer. The contractor shall participate in weekly Configuration Control Working Group (CCWG) and monthly meetings, providing functional/technical support. The contractor shall participate in the quarterly Executive Configuration Control Board (ECCB) and monthly Operations Requirements Committee (ORC) by providing technical support at ECCB and ORC meetings as requested.

• System support, CCWG, and CCWB calls/meetings; and functional/technical support

2.3.2.5 Problem Reports (PRs) (Reference IDIQ RFP SP4701-15-R-0030 Task Areas 4)

Problem reports will include compliance requirements that are current as of the start of the period of performance, documentation, bug/break fixes, fixing broken links, missing data, error message handling, formatting corrections, modifying mandatory fields on existing interfaces or modules, adding data elements, disabling code, applying software security patches and updates, or addressing other vulnerabilities issues, etc. The Government or contractor may identify problems with the system. The contractor may need to perform knowledge transfer with personnel administering the hosting of the system prior to integration of new or modified software or code.

• Provide feasibility and performance/security impact analysis and schedule estimates related to defect resolution

• Test, implement, and support the resolution of problems across all environments, roles, and access methods. Minimize disruption to the production environment when releasing packages;

coordinate testing and synchronization of releases; install, test, and validate proposed releases prior to implementation (see specific testing requirements below).

• Problem Report (PR) analyses and schedule estimates

• Test & implement resolutions

2.3.2.6 Test Management (Reference IDIQ RFP SP4701-15-R-0030 Task Areas 19) The contractor shall develop a Test Management Plan (TMP) to be delivered within 30 days of contract start. This plan requires approval by the PMO and will be a commitment regarding how all aspects of testing (functional unit testing, regression testing, system integration testing, smoke testing, user acceptance testing, etc.) will be conducted. This plan shall be inclusive of testing activities for both the Sustainment and the Development Teams. The plan should specifically state how the contractor’s approach will minimize post-release defects and customer service impacts, and how the contractor team will partner with the PMO to resolve unexpected post-release issues.

The PMO will work with the contract team to devise a test plan that offers flexibility and adequately prepares the government and JITC for test events. However, the following should be considered non-negotiable:

1. The contractor will create / maintain a test environment which shall contain a representative subset of the data needed to test a wide range of system functionality / products / user types / payment methods.

a. The data will be a subset of actual, aged production data. (Fabricated test data / products will be permitted in the system, but for demo and development purposes only. This data will not be accepted for any formal testing and shall not be included in any test scripts provided to the PMO or JITC for test events.)

b. The scope / mix of data will be maintained and validated by the contractor team prior to each test event to ensure that each requirement can be thoroughly tested.

c. The contractor will ensure the test environments remains operational / available during standard business hours and contain aged production data sets equal to roughly 15% of the current production environment with enough variety of data to allow for on demand end to end full regression testing.

d. The contractor shall support PMO use of the test environments during test events and as ongoing tools for help desk troubleshooting and new requirements research.

2. The contractor will be responsible for delivering detailed UAT test scripts with each release but should also expect to support testing beyond the specific examples provided in the scripts. Each

FedMall release should be progressive not regressive and to ensure that the PMO and other skilled testers may test outside of the boundaries of the test scripts provided to ensure the highest quality releases possible will reach the production environment.

a. A proposed list / scope of test scripts will be required at the close of the requirements determination for PMO / JITC / user review. Adjustments will be made based on feedback.

Typically, test script will be required for any release content which alters the current production environments UI/UX, and other front-end testable changes that users may encounter.

b. The contractor shall work with the PMO and JITC to develop an ongoing test & release schedule with meetings as needed to determine release content viability, known risks going into each test event, known test environmental or data set issues, and any risks to established schedules once they are determined.

c. When possible draft test scripts should be provided to PMO and JITC for testing prior to the completion of each sprint to allow PMO and JITC testers to report bugs during the development cycle and so issues can be resolved in the same or subsequent sprints via the backlog.

d. Compiled draft test scripts will be delivered to the PMO and JITC 21 days prior to each UAT event to allow for JITC and PMO to conduct SIT testing and to review the draft test scripts for any errors or clarification needed. Fabricated or altered data will not be referenced in test scripts. If existing test data cannot be found to support a scenario, aged production data must be added to the test environment by the contractor team.

e. Finalized test scripts will be delivered at least 14 days prior to each UAT event. Any content intended to be added to the release after finalized test scripts have been provided must be accepted by the PMO or moved to a later release.

f. The contractor will provide demos of functionality and walkthroughs of changes / testing if requested as part of test events. The contractor team will support researching and resolving PMO / JITC / UAT test issues and concerns during all test events. The contractor shall review issues reported by the PMO and/or JITC related to each release and create JIRA tickets as appropriate to track bugs, 508 reported issue, security scan results, and emerging requirements. The contractor shall report to the PMO and JITC the JIRA numbers of tickets created for tracking and oversight purposes.

3. The contractor shall develop Test Scripts on an iterative basis to ensure the full scope of requirements are met. The contractor shall ensure testing is interwoven with all development phases (requirements, design, code development, test case generation etc.), and that testing takes place simultaneously throughout the Software Development Lifecycle.

a. The contractor shall encourage testers to engage in the requirements process to advocate for sensible changes to customer requirements to enable more secure, user friendly, and streamlined experiences.

b. Test scripts shall be based on inputs to JIRA (user story and the PMO/developer agreed to acceptance criteria and definition of done).

c. Agile Testing involves all members of the project team, with special expertise contributed by testers. Testing is not a separate phase and is interwoven with all the development phases such as requirements, design and coding and test case generation. Testing takes place simultaneously through the Development Life Cycle.

4. The contractor shall report and deliver test results from all tests completed by their test team to the government.

5. Test Results

a. Formal testing documentation will be required for back end / database changes, dependent upon the system transparency / access provided to the PMO. Data updates to correct product data elements and contract data elements do not require documentation but must be coordinated with the PMO prior to the change being made and approved in writing.

i. Development changes that are determined to be ‘database’ or ‘back-end-only’ changes (those affecting database/system elements that fall within that restricted access) will require a formal deliverable via the appropriate Enterprise architecture view that describes the functionality affected and how it will change, plus the testing approach taken with documented results including print screens / errors logged / errors resolved.

ii. The preferred approach would be for the contractor team to share read-only admin access with the PMO to the extent that the PMO can see the results of the changes / tests to avoid these additional documentation requirements.

6. Testing Feedback Loop. Since the scope / quality of testing often correlates to the volume of help desk incidents after a release, the PMO and the contractor team will share the responsibility of analyzing and resolving release-related incidents (PMO and user submitted) after each release.

a. The PMO/contractor team will collaborate through the PMO help desk to resolve these issues, documenting root causes.

b. High impact / high volume issues will be analyzed to see if they could have been prevented with a different testing approach / scope.

c. Lessons learned will be applied to future releases, and the expectation is that the team will adjust the testing approach as needed. Metrics will be developed and maintained to understand this trend over time.

• A fully developed Test Management Plan (TMP) within 30 days of award.

• Test Environments

• Test Environment data to support test cases and on demand end-to-end regression testing

• Test and Release Schedule

• Test Scripts:

1. Delivered during sprints when possible

2. Drafts delivered 21 days prior to test event

3. Finalized and delivered 14 days prior to test event

• Demos as requested

• Ticket numbers for issues related to releases when reported by the PMO or JITC

• Test Results / Testing Documentation

• Metrics to track post production release issues over time and identify lessons learned to apply to future releases and adjust the testing approach as needed.

2.3.2.7 Training Requirements (Reference IDIQ RFP SP4701-15-R-0030 Task Areas 11)

The Contractor shall provide live training (either web-based or on site) to Government staff/administrators on the use of the system for all user roles at least 1 week prior to each release. Provide training and materials/job aids to Government staff/administrators on the use of the e-commerce system for all user roles. The Contractor shall update, test, and field training tools/materials/job aids (e.g., online videos or webinars), and/or communications for customers and vendors/suppliers on the use of the system for their roles.

Deliverable:

• Role-Based Training and materials/job aids/tools/communications

2.3.2.8 Section 508 Compliance (Reference IDIQ RFP SP4701-15-R-0030 Task Areas 4)

DLA has determined that the following accessibility standards (36 CFR Part 1194) apply to this system under Section 508 of the Rehabilitation Act of 1973:

i 1194.21 - Software Applications and Operating Systems ii 1194.22 - Web Based Intranet and Internet Information and Applications iii iii 1 1 9 4 . 3 1 - Functional Performance Criteria iv Section 508 1194.41(Documentation and Support) should be included in every solicitation since deliverables are in electronic format even if it is not considered an EIT contract.

v the contractor shall be responsible for making documents such as pdf's and media files such as videos compliant with accessibility standards.

a. The Vendor shall deliver products and/or services that provide the features described in the above standards, or equivalent salient characteristics. Additionally, the Vendor shall provide product support documentation and services in alternate formats in order to accommodate end users with disabilities.

b. The Vendor must ensure Federal employees with disabilities have access to and use of information and data that is comparable to the access and use by Federal employees who are not individuals with disabilities.

c. The Vendor must provide their own 508 Compliance testing software that has the same functionality as the DLA 508 testing tool. DLA currently uses JAWS, Zoom Text, and Dragon Naturally Speaking 508 Compliance Software.

d. As per DODMANUAL 8400.01 ACCESSIBILITY OF INFORMATION AND COMMUNICATIONS TECHNOLOGY Section 3.6. WEB AND SOFTWARE

DEVELOPERS.

i. DoD web and software application developers must ensure their applications are planned and developed in conformance with U.S. Access Board standards, described in Paragraph 5.2., for Section 508.

Further information on Section 508 is available via the Internet at the following address: http://www.section508.gov.

• 508-Compliant changes, testing support, and documentation to include, but not limited to, the

Government Product Accessibility Template (GPAT).

http://www.section508.gov/

2.3.2.9 Cybersecurity and the Risk Management Framework (RMF) Process (Reference IDIQ RFP SP4701-15-R-0030 Task Areas 6)

The contractor shall ensure that the information system complies with all applicable Laws, Regulations, and Policies, to include the RMF Process for NIPRNET (https://rmfks.osd.mil), commensurate with the system’s security categorization for Confidentiality, Integrity, and Availability (C-I-A) (FEDMALL is Moderate, Moderate, Low under RMF, STORES is Moderate, Moderate, Moderate. The Modernized solution will need to meet the Moderate, Moderate, Moderate C-I-A, with 12 hours recovery time). The contractor shall assist in the assessment of the security posture of the information system and shall provide the necessary artifacts needed for the RMF process (e.g., Systems Security Plan/Concept of Operations (CONOPS), Hardware/Software Listing, Listing of Ports, Protocols, and Services, Firewall Rules, etc.). The contractor shall incorporate and document the security practices used in the administration of the information system and changes to the information system.