BES RFI QA_Final.docx

BES RFI Questions, Answers and Recommendations

1. The PWS provides adequate documentation for scoping the work performed during execution of the task order apart from Transition-In planning. As identified in paragraph h of this response, we agree with the VA’s decision to include a 180-day transition in period between current operations and the BES performance approach. As we noted, we believe the VA should require a draft transition in plan as a component of proposals for evaluation.

To deliver a well-formulated and low risk plan we believe the VA should provide additional information on current strategic plans, operations metrics, product backlogs, etc. as part of an Electronic Reading Room or with the Final Request for Proposal. These documents will afford the VA adequate competition and lower risk approaches to improve the evaluation and selection of the most capable vendor.

ANSWER 1: VA will consider the recommendation regarding use of transition plans as part of the evaluation approach and the final decision will be posted with solicitation. Strategic objectives are included in the PWS, and attachments and operations metrics are provided in the SLA. VA will not provide product backlogs as they contain sensitive information.

2. Provide composition of the 15 system teams and 230 Agile development teams.

ANSWER 2: The PWS provides the requirements regarding System and Agile development team performance. Offerors will be required to provide their own proposed composition of teams consistent with their technical approach to the PWS requirements.

For example, Systems Teams could be made up of the following:

Extra Small Teams – VETSNET Small Team – CAPRI Medium Team – CRP Large team – MBS

3. General - Will the government consider a two-step evaluation process wherein Offerors submit case studies as a Technical Factor in Phase 1? This will enable the government to validate Offerors’ experience of successfully executing complex and multiple tasks comparable to the BES PWS and limit the government’s administrative burden of reviewing proposal submissions for Offerors who may not be qualified to execute this work, as those Offerors would not proceed to Phase 2.

ANSWER 3: The Government will consider a two-step evaluation approach and the final decision will be posted with solicitation.

4. General - In (Company)’s experience, legacy Basis of Awards methods such as “Best Value” incentivize Offerors to “game” their price proposal using tactics that increase performance risk and delay critical solutions. These tactics include proposing unrealistically low rates and less qualified personnel, severely underpricing optional tasks and option periods, and not proposing cost of living/inflation increases for the option periods. These tactics are especially prevalent for Best Value Firm-Fixed Price (FFP) and hybrid FFP/Time and Materials (T&M) type contracts, where the government bears the performance impact of not selecting the higher technically rated/higher cost Offeror. An Offeror’s low price triggers their inability to recruit and retain qualified personnel, the delivery of resources without substantive skills and expertise, and an inability to meet requirements. In these instances, data in the Federal Procurement Data System (FPDS) indicates that the government often chooses not to exercise option years and recompetes the solicitation early, significantly delaying the solutions Veterans, caregivers, and VA employees need. (Company) recommends that VA will benefit from a Highest Technically Rated Reasonably Priced (HTRRP) source selection approach that prioritizes technical superiority and expertise over price, ultimately providing a greater value to the government than an underpriced bid. With this approach, IDIQ, BPA, contract, or task order price negotiations are conducted only with the Offeror with the highest technical rating, and VA is ultimately selecting a trusted partner who provides the highest quality solutions, services, and work products needed to successfully support Veterans, caregivers, and VA. In the alternative, should VA choose to perform a Best Value Trade Off approach for BES, we recommend that VA reserve the option to perform a price realism analysis as an additional tool for price evaluation and/or a technical risk assessment. This will put Offerors on notice that VA expects pricing that reflects sufficient incentives for the contractor to execute PWS requirements with the required quality level, along with pricing which is commensurate with the proposed solution and provides the government confidence in the Offeror’s ability to execute its solution. Sample price realism language from other solicitations which the GAO has found to provide industry with sufficient notice is as follows: “The Government reserves the right to reject any proposal that reflects an inherent lack of technical competence or a failure to comprehend the complexity and risks required to perform RFP requirements due to submission of a proposal that is unrealistically high or low in price and/or unrealistic in terms of technical or schedule commitments.

ANSWER 4: The Government will consider these recommendations regarding its evaluation approach and the final decision will be posted with the solicitation. However, vendors are advised that the Government will be pursuing increased efficiency and efficacy with the BES procurement with an expectation of seeing corresponding cost/price savings in Offerors’ proposals.

5. As an additional alternative to traditional Best Value Basis of Award acquisitions, would the government consider a comparative analysis approach? This approach provides the government the ability to use a streamlined evaluation and decision document that enables comparison of Offerors without ratings and weighting of factors. The comparative analysis approach has been used by contracting teams within the Technology Acquisition Center for potential FAR Part 8.4 acquisitions.

ANSWER 5: VA will consider as part of the evaluation approach and the final decision will be posted with solicitation.

6. (Company) recommends the government add Shared Services teams to the PWS. Shared Services are an important component of Product Line Management to promote efficiency across products. While we believe that the addition of new CLIN types will reduce the number of Shared Services teams exercised, Shared Services teams provide the VA with the flexibility to support development with specialty expertise in systems or solutions that require unique knowledge and skills that cannot be anticipated within the scope of the other defined BES CLINS. Our current BID work on cloud migration efforts, eventing, and ML-MATS are good examples for the need of Shared Services teams.

ANSWER 6: Shared Service team requirements will not be added to the PWS. Any scope of work that does not align with the BES PWS requirements for Systems Teams, Agile Teams, Training and Helpdesk Support will be competed as a separate procurement.

7. PWS 5.2, pg. 26 - Please provide a comprehensive list of all supporting environments that should be treated with production Service Level Targets during active UAT.

ANSWER 7: The Service level targets table has been revised to note “*UAT environments shall adhere to the Production Service Level Targets. “

8. PWS 5.2, pg. 26 Will dedicated System Teams be exercised for each environment, similar to the Demo ACAD System Team today?

ANSWER 8: No. Primarily, systems team will be supporting products not individual environments.

9. PWS 5.2, pg. 26 To properly scope the necessary support for these Core Hours, we recommend the language be updated to reflect Core Hours support for all products. For example, “Core Hours for all Products are Mon – Sat 6 AM to 10 PM Eastern, and Sun 6 AM to 3 PM, except for Federal Holidays”.

ANSWER 9: PWS Section 5.2 System Teams has been updated to define the Core hours applicable to the Service Level Targets for Non-Production environments.

10. PWS 5.2.1, pg. 32 The terminology of “checking out” and “checking in” code, data, and artifacts is applicable to older version control systems such as Subversion and CVS. Modern workflows, based on Git, which forms the basis of VA-approved tools like GitHub, have evolved this terminology to “cloning” a repository (the equivalent of “checking out”) and “committing” and “pushing” to a repository (the equivalent of “checking in”) code, data, and artifacts. (Company) recommends that the terminology in this requirement be updated to “The Contractor shall commit and push code to VA source control repositories on a daily basis”.

ANSWER 10: The PWS has been updated to revise the terminology to “commit and push code”.

11. PWS5.2.1.1, pg. 28 VAEC recently shifted from Disaster Recovery Plans (DRPs) to Incident Response Plans (IRPs). (Company) recommends this requirement be updated to reflect IRPs (specifically IRP Appendix A for all applications hosted within the VAEC) versus DRPs.

ANSWER 11: The PWS has been updated to revise Disaster Recover Plans to Incident Response Plans.

12. PWS 5.3.5, pg. 44 (Company) recommends the government remove PWS 5.3.5 from the Agile Development Teams’ scope and include it as scope for Shared Services teams or as its own Data Analytics team. The requirements under PWS 5.3.5 necessitate a labor mix that is specialized and focused on analysis of data (e.g., Data Scientist, Data Visualization Specialist, Data Engineer - BIG Data, Advance Data Scientist/Researcher, Data Architect) which is very different from the labor mix needed for Agile Development Teams focused on software development services (e.g., Scrum Masters, Requirements Analysts, Software Developers, Quality Assurance/Testing Specialist, Release Managers). By realigning the Data Analytics scope under a separate team, the government will get a proposal package with the correct representation of labor mix and fair pricing to meet and exceed the requirements specified under 5.3.5, thus lowering the risk to delivery. 5.3.7, pg. 46 (Company) recommends the government remove PWS 5.3.7 from the Agile Development Teams’ scope and include it as scope for Shared Services teams or as its own Innovation Architecture and Engineering team. The requirements under PWS 5.3.7 necessitate a labor mix that is specialized and focused on architecting and designing innovative solutions (e.g., Architect, Business Process Specialist, Cloud Computing Infrastructure Architect, Cloud Security Architect, Digital Engineer, DevOps Engineer, Enterprise Architect, AI Solution Architect) which is a very different composition than Agile Development Teams focused on software development services (e.g., Scrum Masters, Requirements Analysts, Software Developers, Quality Assurance/Testing Specialists, Release Managers). By realigning Innovation Architecture and Engineering scope of work under a separate team, the government will get a proposal package with a correct representation of labor mix and fair pricing to meet and exceed the requirements specified under 5.3.7, thus lowering the risk to delivery.

ANSWER 12: The PWS has been updated to further define the requirements around data analytics at the overall portfolio level for management. See updated PWS section 5.1.10 Data Analytics.

13. PWS 5.8, pg. 49 PWS 5.8 allows for additional Product System Teams to perform the tasks identified in PWS 5.2.1, including all subtasks. However, this Optional Task does not allow for additional BIP System Teams to perform the tasks identified in PWS 5.2.2. We recommend the government either expand the scope of PWS 5.8 to cover additional Product System Teams and BIP System Teams or add an additional Optional Task that would allow for additional BIP System teams.

ANSWER 13: PWS Section 5.8 has been updated to clarify Product System Teams are required to only perform the requirements of 5.2.1 under the Optional Task. The requirements of 5.2.2 are not applicable to the Product System Teams that will be exercised under the optional task. Section 5.2.2 defines the requirements of the specific BIP System team that performs as part of the Base and each Option Period. VA does not intend to exercise the Optional Task for any additional BIP System teams to perform the 5.2.2 requirements.

14. PWS B.7 (i.-k.), pg. 113 Can the government confirm that this section does not supersede the vulnerability management language in PWS 5.2.1.3 and 5.3.6: “Coordinate the remediation of vulnerabilities identified in scans, quality checks, or reviews/audits within the required time frames based on severity”? If so, (Company) recommends the severity levels and remediation timelines be clearly defined in the PWS as follows: “The Contractor shall notify VA within 24 hours of the discovery or disclosure of successful exploits of the vulnerability which can compromise the security of the systems (including the confidentiality or integrity of its data and operations, or the availability of the system). Such issues shall be remediated as quickly as is practical but in no event longer than the guidelines below: Critical – Must be remediated immediately; High – Must be remediated within 60 days; Medium – Must be remediated within 90 days; Low – Must be remediated according to the timeframe established by the system owner, typically in the next release or no longer than 120 days.”

ANSWER 14: PWS Section 5.2.1.3 and 5.3.6 have been updated to be consistent with the remediation timeline requirements specified in Addendum B – VA Information and Information System Security/Privacy Language, Section B.7 Information System Hosting, Operation, Maintenance or Use.

15. Section 5.1.4.1 How are software licenses currently managed? Are there dashboards and an associated database that help effectively manage licenses nearing expiration or is the information located in disparate systems/places?

ANSWER 15: I5.1.4.1 (h)The software licenses are currently managed in a spreadsheet. No dashboards are currently being used. Non-standard VA software that will be used by contractor developers will be the responsibility of the contracting company.

16. Section 5.1.6 for PI Planning, what agile software is currently being utilized to maintain the team’s PI roadmaps, tasks/activities, risks, etc.?

ANSWER 16: Currently BAM is using JIRA.

17. Section 5.2 What is the current scope of the AI/ML Operations? What kind of work have they done to date?

ANSWER 17: VA is in the process of exploring AI use cases including opportunities to enhance and maintain infrastructure and applications including

· Designing, developing, and deploying an agentic reusable framework for AI.

· Developing and integrating MCP servers and clients.

VA endeavors to implement AI and ML Ops to enhance the efficiency, reliability, and scalability of VA IT services, ensuring a better experience for users and more robust support for systems.

18. Section 5.2.1.1 How many critical/bedrock systems are maintained under this contract and which BAM portfolio product lines owns them, ANSWER 18:

RECOMMENDATIONS

1. Carving out specific product lines under the VA’s Benefits Enterprise Services contract into a set-aside for Service-Disabled Veteran-Owned Small Businesses (SDVOSBs).

RESPONSE: BAM will not carve out specific product lines under the VA’s Benefits Enterprise Services contract into a set-aside for Service-Disabled Veteran-Owned Small Businesses (SDVOSBs). The requirements allow for portfolio management and vision driven execution goals of the VA. This allows VA to implement a strategy to re-use and share capabilities across products to realize the BIP capabilities and economies of scale and develop and grow portfolio-wide benefits, appeals and memorials data and services that support Veterans benefits claims processing. This contractor will be responsible for identifying, enabling, and implementing expansion and increased utilization of portfolio enterprise common services and solutions.

2. While the Draft PWS provides a comprehensive overview of software development practices and team structure, it lacks crucial details regarding Amazon Web Services (AWS) cost analysis and scaling requirements. Recommend including Labor Category (LCAT) roles specifically dedicated to AWS cost modeling, capacity planning, and scaling analysis (Program Manager, Site Reliability Engineer, Senior, Systems Engineer). This expertise will be essential for making informed decisions about resource allocation, cost optimization strategies, and infrastructure scaling to ensure both technical and financial efficiency of the Benefits Integration Platform (BIP) platform.

On the software side, we recommend Github projects for organizing issues and maintaining a central repository of ongoing Sprint work. GitHub Projects is particularly valuable because developers build it for use by developers and with an Agile development cycle in mind. With an Identity and Access Management (IAM) system like Okta or Google Cloud Identity, GitHub is a secure and feature-rich way to log Sprint work, collaborate across teams, and maintain a paper trail of past, present, and future projects. Similarly, we recommend using Docker and Jenkins to manage the reverse proxy and workspaces. These tools allow developers to test their code without posting it to production. Like GitHub projects, Docker and Jenkins are open source, government-compliant, and industry-standard, so many developers will already be familiar with these products, reducing training time and related overhead.

The proposed Systems Development Life Cycle (SDLC) will require several standard teams, including Backend Development, DevOps, Quality Assurance, and an Analytics team. In addition, we recommend a Tier 1 and Tier 2 Support Engineer team that allows one team to handle more straightforward short-term tasks and free up bandwidth for the other team to handle complex, multi-Sprint support needs. Finally, we recommend adopting a Governance Team Collaboration Cycle, similar to the team at VA.gov. The Governance Team, comprised of specialists in QA, IA, Accessibility, Design, and Content, maintains a series of checkpoints throughout the SDLC, offering support and guidance to teams as they build new pages and features for the project. Maintaining this process ensures that diverse teams can still create a cohesive final product.

RESPONSE: BES is not specifying LCATs as part of the requirement. As an Offeror you will be responsible for providing the labor mix appropriate to your technical approach. The BES requirement is for portfolio-level technical management and coordination to further develop and provide secure, BAM architecture, engineering, solutions, services, and data. The Contractor shall provide portfolio and product line technical management and maintain continuous strategy-to-execution alignment in partnership with stakeholders to maximize outcomes. The Offeror should propose in accordance with the solicitation requirements and PWS.

3. Acquisition Strategy Recommendation (COMPANY) recommends Veterans Affairs procure BES utilizing the GSA MAS 54151S SIN under NAICS 541512 utilizing a multi-phase Advisory Down Select. Doing so will limit competition and possibly protests and allow the government to solicit from only the most qualified companies. (COMPANY) recommends the first phase utilize a pass/fail down-select process. We recommend a past performance/case study approach for Phase 1, requiring at least three referenceable past performances over $100M in total contract value within the prior three years with NAICS 541512, where the bidding entity for BES must be the prime for all three. By doing so, VA will have confidence that all companies entering Phase 2 have the capabilities and experience to BES staff appropriately. The government can also have a quick turn for Phase 1 responses to create speed for the overall procurement. (COMPANY) recommends Phase 2 consisting of a management methodology and the technical approach evaluation including the price volume. (COMPANY) believes this method will help the government receive only the most viable and capable companies, saving the government time and reducing the risks of protests. GSA MAS 54151S gives the VA the flexibility to execute the advisory down select methodology for procurement.

RESPONSE: VA is currently contemplating the use of Alliant 2 and FAR Part 16.5 procedures for the BES procurement. The evaluation approach will be posted with the solicitation.

4. Breaking Out Data Fabric Components for Enterprise Correspondence Management (COMPANY) recommends that the VA consider separating the Data Fabric components supporting Enterprise Correspondence Management (ECM) from the broader BES initiative. While the modernization of BAM systems requires seamless interoperability and integration, the unique operational and technical requirements of ECM merit focused attention as a standalone effort.

RESPONSE: The Government is considering ECM as a requirement separate from BES.

5. We respectfully recommend that the TAC consider removing all modernization scope from the PWS, and address modernization through other means. Our recommendation recognizes a fundamental difference between “modernization” (i.e., replacing legacy systems with modern technology, architecture, and infrastructure) versus “enhancements” which involve improving the functionality and performance of legacy application systems For these reasons, (Company) respectfully recommends that the VA consider focusing the BES program PWS exclusively on operating, maintaining, and enhancing the BAM portfolio of systems in the scope.

RESPONSE: At this time the Government will be keeping the modernization scope as part of the BES requirements.