Draft RFP Q_A.pdf

Department of Veterans Affairs Independent Enterprise Testing Support Services

Questions and Answers November 13, 2024

Note: The answers provided below are for informational purposes, to assist potential Offerors in preparing proposals. To the extent that any answers provided below conflict in any way with the RFP, the final RFP takes precedence.

# Reference Question/Feedback/

Recommendation Answer

1. Draft RFP; E.15

SUBMISSION

INSTRUCTIONS,

2. PROPOSAL

FILES, b. Content Requirements;

Would the government remove the page count limitation for Volume VI:

Solicitation, Offer & Award Documents, Certifications & Representations or note that signed SF1449, SF30 and completed relevant Reps and Certs do not count toward page count?

Page limitations removed for other volumes.

2. Draft RFP; C.14

ORGANIZATION

AL CONFLICT

OF INTEREST;

With respect to Draft Solicitation Section C.14, Organizational Conflict of Interest, please explain why the Contracting Officer determined that "significant organizational conflicts of interest exist in the performance of independent testing for Contractors currently participating as a prime or subcontractor on VA IT project management services contracts or software development services contracts for VA/GOTS software". (See also Draft Solicitation Section E.14, Evaluation Approach.)

Because of the size, complexity, and dynamic nature of VA’s IT infrastructure, a maximum degree of technical, managerial, and financial independence are all essential for testing efforts of software systems to ensure complete and accurate results are provided to the decision maker without bias, intentional or otherwise.

Due to the expansive testing requirements for which these IV&V testing services are being procured the exclusion must extend to any other project within the department that may interact with or otherwise provide services to IETSS or to the Department.

3. Draft RFP; C.14

ORGANIZATION

AL CONFLICT

OF INTEREST,

e); Page 100 of

Please explain the reason for the requirement for the contractor's employees to execute a "Contractor- Employee Personal Financial Interest/Protection of Sensitive Information" Agreement as appropriate".

This is required to ensure the Contractor employee(s) do not have a financial interest, personal activity, or relationship that could impair the employee’s ability to act impartially and in the best interest of the Government when performing under the contract.

4. Draft RFP; C.14

ORGANIZATION

AL CONFLICT

OF INTEREST, j);

Notwithstanding the stringent nature of the independence requirement and the prohibition on participation in this acquisition, it is noted that paragraph

(j) allows a contractor to resolve an OCI that is discovered after award

If an OCI exists as described by the independence requirement, it cannot be mitigated.

through avoidance or mitigation: "The Contractor agrees that if after award it discovers a potential organizational conflict of interest, a prompt and full disclosure shall be made in writing to the Contracting Officer. This disclosure shall include a description of the actions the Contractor has taken or proposes to take, to avoid or mitigate such conflicts."

Accordingly, please explain why the avoidance or mitigation approach is not deemed to be an equally viable baseline for the competitors for this acquisition.

5. Draft RFP; 4.1

PERFORMANCE

PERIOD; Page 28 of 138

PWS Section 4.1, Performance Period, states that "the contract includes one optional task to provide Transition Support. If exercised, the period of performance of the optional task will be a total of 180 days". Please verify the necessity for the inclusion of a six-month Transition Support requirement.

The Optional Task is for transition Support up to/not to exceed 180 days.

It is possible the actual transition support services is exercised for less than 180 days.

6. Draft RFP; C.15

SMALL

BUSINESS

PARTICIPATION

REQUIREMENTS

; Pages 100-101 of

Draft Solicitation Section C.15, Small Business Participation Requirements, states that "[t]he prime contractor shall meet the Government's overall small business participation requirement for this procurement. In addition, the prime contractor shall make a good faith effort to meet its proposed small business participation percentages.

The prime contractor is cautioned that failure to make a good faith effort to meet the overall small business participation requirement or percentages for this contract or to provide detailed supporting documentation and/or rationale when it falls short of meeting this contract's small business participation requirement or percentages may result in adverse past performance or termination for default. The Government small business participation requirement and the proposed small business participation percentages as set forth in your Small Business Participation Commitment

Adherence to subcontracting the overall small business participation requirement and percentages is intended to align with the statutory requirements of compliance with the small Business Subcontracting Program which states that, “any contractor or subcontractor failing to comply in good faith with the requirements of the subcontracting plan is in material breach of its contract.” FAR 19.702(c).

(SBPC) proposal shall become a requirement of this contract.

a) Please explain why this provision states that "the prime contractor shall meet the Government's overall small business participation requirement for this procurement", but also states that "the prime contractor is cautioned about the "failure to make a good faith effort to meet the overall small business participation requirement.". Since the overall Small Business Participation requirement is a contractual requirement, why will the contractor only be required to make "a good faith effort" to meet that requirement?

b) If the participation percentages set forth in the offeror's Small Business Participation Commitment (SBPC) proposal "shall become a requirement of this contract", why then is the prime contractor only required to "make a good faith effort to meet" its proposed small business participation percentages?

7. Draft RFP; E.14

EVALUATION

APPROACH, 3.

VETERANS

INVOLVEMENT

EVALUATION

APPROACH Page

Draft Solicitation Section E.14, Evaluation Approach, paragraph 3, Veterans Involvement Evaluation Approach, states that "non- SDVOSB/VOSB Offerors proposing to use SDVOSBs or VOSBs as subcontractors will receive some consideration under this evaluation Factor". Please indicate the extent of SDVOSB/VOSB subcontracting that is required in order for a non- SDVOSB/VOSB offeror to receive "some consideration" for this Factor.

Any proposed use of SDVOSB/VOSB will received a rating of partial credit.

8. Draft RFP; E.14

EVALUATION

APPROACH, 4.

SMALL

BUSINESS

PARTICIPATION

COMMITMENT

EVALUATION

Draft Solicitation Section E.14, Evaluation Approach, paragraph 4, Small Business Participation Commitment Evaluation Approach, paragraphs (a) and (b) (page 132), state that "a realism analysis of the proposed goals will not be conducted. The percentages proposed for the goals will be incorporated into

The Government does intend to evaluate the participation percentages proposed by the offerors as well as whether the offerors meet the Small Business Participation requirement.

The proposal will be evaluated to determine the extent to which the Offeror demonstrates a commitment to meeting or exceeding the Small

APPROACH;

any resulting contract. The proposal will also be evaluated to determine whether the Offeror has met the overall Small Business Participation Requirement for this procurement, which is 25% of the total contract value. The Offeror shall meet the Small Business Participation Requirement in order to be found acceptable under this factor". Please explain why the Government does not intend to evaluate the participation percentages proposed by the offerors but will evaluate whether the offerors meet the Small Business Participation requirement.

Business Goals. The proposal will also be evaluated to determine whether the Offeror has met the overall Small Business Participation Requirement for this procurement which is 25% of the total contract value. Meeting the 25% small business goal is a condition for award.

9. 5.2.3 Could the VA provide historical data on common failure points or high-risk areas identified in previous Electronic Health Record (EHR) testing cycles, specifically concerning system interoperability and data migration?

This information would aid in refining the Criticality Analysis and Risk Assessment (CARA) process by prioritizing known vulnerabilities and ensuring a focused testing approach for critical components.

No, Offerors shall provide their approach to meeting the solicitation requirements set forth in section E.15(2)(c)(i).

10. 5.4 Could the VA clarify the specific configurations and technical requirements for the Test Environment Logical Requirements Document, particularly regarding virtualized infrastructure and network segmentation?

This information would help ensure that Test Systems Engineering and Implementation (TSEI) support aligns with the VA’s existing IT architecture, especially in optimizing system performance and security during integrated testing.

Additional information is provided in Attachment 002 – IETSS Infrastructure and Processes

11. 5.2.5.1 Could the VA provide detailed quality benchmarks and historical performance metrics from previous testing phases, specifically for defect identification and resolution times?

No, benchmarks and performance metrics vary based on the specific systems being tested.

Understanding these metrics will allow the Continuous Quality Improvement Process Plan (CQIPP) to set realistic targets for quality enhancement and establish precise measures for tracking improvements over time.

12. 5.2.5.6 Could the VA provide specific examples of key performance indicators (KPIs) and metrics that have been most valuable in previous projects for assessing test effectiveness and system readiness?

Additionally, are there specific metrics or reporting formats mandated by VA standards that should be incorporated into the Measurement and Metric Plan to ensure consistent evaluation and compliance?

No, KPIs and metrics vary by the systems being tested. Additional information is provided in Attachment 002 – IETSS Infrastructure and Processes

13. 5.2.7 Could the VA provide guidance on specific criteria or standards used for test result evaluation and acceptance, particularly concerning interoperability, security, and performance benchmarks?

Additionally, are there any preferred formats or analytical tools the VA requires for the Test Analysis Summary to facilitate consistency in evaluating test outcomes across different project phases?

No, criteria and standards vary by the systems being tested. Additional information is provided in Attachment 002 – IETSS Infrastructure and Processes

14. 5.2.8 Could the VA clarify the prioritization criteria and risk thresholds for documenting and escalating patient safety-related defects? Additionally, are there specific testing methodologies or reporting formats mandated for capturing the severity and potential impact of these issues to ensure alignment with VA’s patient safety protocols?

Patient safety prioritization and impact is established by the VA Patient Safety Office.

15. 5.2.9 Could the VA provide detailed guidelines on defect categorization and severity levels, as well as required response times for defect resolution across different categories?

Additionally, is there a preferred defect tracking system or integration requirement with VA’s existing systems to ensure consistency in

Additional information is provided in Attachment 002 – IETSS Infrastructure and Processes managing and reporting defect status throughout the project lifecycle?

16. 5.3 Could the VA specify the technical requirements for virtualized test environment provisioning, including resource allocation thresholds, compatibility constraints with VA's legacy systems (e.g., database schemas, middleware versions), and required compliance with FIPS 140-3 encryption standards? Additionally, are there specific orchestration tools (such as Kubernetes or Ansible) and configuration management databases (CMDBs) that VA mandates for version control, environment snapshots, and configuration baselines to support rigorous compliance with VA’s cybersecurity protocols?

No. Technical requirements vary by the system to be tested and are created by the IETSS contractor. Offerors shall provide their approach to meeting the solicitation requirements set forth in section E.15(2)(c)(i).

Additional information is provided in Attachment 002 – IETSS Infrastructure and Processes

17. 5.4.1 Could the VA provide detailed specifications for compliance with network security and data isolation protocols within the Test Center environment, particularly regarding any required integration with VA’s secure network zones, data encryption standards (e.g., FIPS 140-3), and user access control policies? Additionally, does the VA mandate periodic third-party audits or vulnerability assessments for architecture updates, as these would significantly influence resource allocation and cost projections for maintaining compliance?

See Attachment 001 - IETSS Excel Price Evaluation Spreadsheet for labor category descriptions and allocation of labor hours for evaluation purposes.

Additional information is provided in

18. 5.4.2 Could the VA specify detailed operational requirements for maintaining Test Center environments, particularly in relation to automated patch management, disaster recovery protocols, and network redundancy standards? Additionally, are there specific tools or infrastructure components (e.g., load balancers, backup systems, and monitoring software like Nagios or Splunk) mandated by the VA for maintaining high availability and rapid fault detection in line with VA’s IT resilience and compliance standards?

See Attachment 001 - IETSS Excel Price Evaluation Spreadsheet for labor category descriptions and allocation of labor hours for evaluation purposes.

Additional information is provided in

Understanding these details would significantly assist in accurately estimating resource allocation and associated costs.

19. 5.4.3 Could the VA provide detailed specifications for required configuration management practices within Test Center Operations, including expected frequency of system audits, baseline version control for both software and hardware components, and any mandatory compliance with VA-specific change management protocols? Additionally, are there preferred tools or integrations (e.g., ServiceNow CMDB or Git-based repositories) that must be used to track configuration changes and align with VA’s IT asset management and documentation standards?

Additional information is provided in

20. 5.5.1 Could the VA provide benchmarks or historical metrics on defect occurrence rates, mean time to resolution (MTTR), and user-reported issues from prior testing cycles for similar systems? Additionally, are there specific quality metrics (such as defect density or requirements traceability matrix coverage) that VA mandates for monitoring improvements in test quality and process efficiency, and should these be tracked using a particular tool or integrated into VA's quality management systems to ensure consistent reporting and evaluation?

No, benchmarks and metrics vary by the systems being tested.

Additional information is provided in

21. 5.5.3 Could the VA provide details on the expected volume and complexity of artifacts to be reviewed per reporting period, along with any specific criteria or standards (e.g., IEEE 730 or ISO/IEC 25010) for evaluating compliance and quality? Additionally, are there specific review and automation tools required or recommended by VA (e.g., JIRA for tracking, or automated quality assessment tools) to streamline artifact

Estimated volumes are provided in the Performance Work Statement.

Additional information is provided in analysis and ensure consistent documentation across reports?

22. 5.5.6 Could the VA provide specific data retention and processing requirements for metrics reporting, particularly regarding storage solutions, data refresh rates, and access controls aligned with VA’s cybersecurity and compliance standards? Additionally, are there preferred analytics platforms (such as Power BI or Tableau) and automated reporting tools that the VA mandates for real-time metric visualization and historical data analysis to support continuous quality improvement?

Data retention and processing requirements are provided in the Performance Work Statement.

Additional information is provided in

23. 5.5.11 Could the VA provide specific data validation criteria, including acceptable error thresholds, data transformation requirements, and compatibility standards for migrated data within the EHR system?

Additionally, are there mandated data migration tools or scripts, as well as expected performance metrics (e.g., migration speed, data integrity checks) that the VA requires to meet its compliance and operational benchmarks for secure, large-scale data migration?

No, data validation criteria requirements and compatibility standards vary by the systems being tested.

24. OCI We believe that the structure of this provision is overbroad and will effectively prohibit offerors with necessary VA IT experience from being able to participate in this acquisition. Please explain the Government’s rationale for precluding offerors with test and evaluation, test systems engineering and implementation, test process, and test quality management support experience and expertise within the VA from competing for this procurement, and why it believes that limiting the competition to offerors without VA experience and expertise would be in the best interests of the Government.

The independence requirement precludes offerors that are currently participating as a prime or subcontractor on VA IT project management services contracts or software development services contracts for VA/GOTS/COTS software. The requirement does not preclude offerors with test and evaluation, test systems engineering and implementation, test process, and test quality management support experience and expertise.

25. Is the list of tools provided on page 47 a comprehensive list of all tools used, The list is for current tools being utilized in VA environments and is and will there be room for the contractor to adjust innovatively (with government approval) for efficiency and cost savings?

subject to change. The IETSS Contractor is required to assess these tools.

26. Does the government have a preferred methodology (e.g., Agile, DevOps) for testing and support activities?

There is no preferred methodology.

27. Will the contractor directly access end-users or stakeholders to gather feedback and insights during testing?

No, the contractor will not directly access end-users or stakeholders to gather feedback and insights during testing.

28. For clarification regarding

OPTIONAL TASK—TRANSITION

SUPPORT (T&M), is this a new requirement, or is there an incumbent with whom the awardee will need to collaborate with for Transition Support?

This optional task is for transition services at the end of this contract, should the incumbent vendor not be selected for award of the follow-on effort.

29. Does the government have an anticipated time frame for transition activities to be completed?

The time frame for transition is unknown at this time.

30. Is the transition period included or in addition to the 12-month PoP?

The Transition Support Optional Task may be exercised anytime, in whole or in part, within the Base Period or Option Periods 1 through 4.

31. Can the government provide an anticipated or historical Level of Effort?

See Attachment 001 - IETSS Excel Price Evaluation Spreadsheet

32. Can the government confirm the Period of Performance understood as one 12-month Base Period and four 12-month Option Periods and contextualize that PoP by providing an anticipated start date?

Anticipated period of performance start date is February 15, 2025.

33. B.5 Performance Work Statement;

pg 29; paragraph

4.2 and various sub-sections in 5.4

Section states that the offeror will propose a place of performance.

There is also a statement that the site in Bay Pines, FL will require on-site work of 2-6 contractors, and that those positions should be prioritized. 5.4.1 further states that "Some Contractor(s) shall be physically located onsite at designated workspaces assigned at the Bay Pines offices to support the TC."

Can the government clarify whether the on-site requirement is for the duration of the contract?

It is not anticipated that testers are on-site for the duration of the contract.

34. B.5 PWS; pg 31;

paragraph 5.1.4

Section states, "Additionally, throughout the period of performance

The Prime Contractor and its subcontractors are required to of this contract, the Contractor shall immediately notify the Contracting Officer, in writing, if or its subcontractors intends on pursuing VA business opportunities for software development or IT project management." Can government confirm that offeror and teammates are intended to be "independent" throughout the duration of the contract?

maintain their independence throughout the duration of the contract.

35. RFP section E.12;

pg 130

Section states: To receive consideration for award, a rating of no less than "Acceptable" must be achieved for the Technical Factor and the SBPC Factor. What is the definition of "Acceptable" for the SBPC Factor?

The Offeror shall meet the Small Business Participation Requirement of 25% to be found acceptable under this factor as set forth in Section E.12 Basis of Award and Section E.14(4)(b) Small Business participation Commitment Evaluation Approach.

36. RFP; pg 35-39;

PWS paragraphs 5.22.5.1-5.2.9

What is the average number of hours per month for executing in-scope testing scripts?

See Attachment 001 - IETSS Excel Price Evaluation Spreadsheet

37. General How many separate instances of Cerner need to be or will need to be tested?

Cerner “Oracle Health” testing requirements are dependent upon the deployment schedule.

38. Draft PWS; pg 39- 40; section 5.1.2

Please Clarify: Unable to replace key personnel without break in performance may provide grounds for termination.

Termination refers to termination of the contract not of contractor personnel.

39. Draft PWS; pg 76;

section 6.2

GFE must be provided by the government in some reasonable timeframe to assure contractor performance. Please provide a timeframe related to Government's ability to timely provide GFE.

The Government will furnish GFE in a timely manner.

40. Draft PWS; pg 31;

section 5.1.4

Please Clarify: Additionally, throughout the period of performance of this contract, the Contractor shall immediately notify the Contracting Officer, in writing, if or its subcontractors intends on pursuing VA business opportunities for software development or IT project management. Please provide a minimum notification period (days) necessary for notification.

When the prime contractor or subcontractor intends on pursuing an opportunity for software development or IT project management it must immediately notify the Contracting Officer.

41. Draft PWS; pg 149; E-15 – Submission Instructions

Please Clarify: Offerors shall submit a Memorandum of Independence Confirmation (Attachment 004) demonstrating that IETSS project

Not at this time as sustainment is already within the definition of software development.

team proposed, including prime and subcontractors is not currently providing IT project management or software development services to VA.

Will the government please add Operations and Maintenance of VA software systems to the Independence definition?

42. Draft PWS; pg

102; C.7 VAAR

852.209-70

The Government cannot require any Offeror's team to be "Independent" if they are able to provide "Mitigation Plans" to address any perceived or in fact Organizational Conflict of Interest (OCI). Please delete C.7 in total.

The OCI Clause has been updated to state that the independence requirement cannot be mitigated.

43. General The criticality of this work and the high trust necessary of any successful offeror and its teaming partners to consistently perform with high quality and distinction. We recommend the Government add FAR Section 52.203- 13 Contractor Code of Business Ethics and Conduct and require disclosure by the Offeror of any violation (active or settled) within the last 3 years of issuance of the final PWS.

FAR Clause 52.203-13 is not required for contracts for commercial services.

44. Draft RFP; pg 135;

E.15.2

Offerors shall submit a list of all contracts (including Federal, State and local government and private) (prime contracts, task/delivery orders and/or subcontracts) in performance at any point during the three years immediately prior to the proposal submission date, which are relevant to the efforts required by this solicitation.

Please confirm that the statement "three years immediately prior to the proposal submission date" equates to the originally established proposal submission date (i.e., not inclusive of any proposal submission date extensions established via any subsequently issued solicitation Amendments)

Proposal submission dated is the date the actual proposals are submitted.

45. Draft RFP; pg 131;

E.14.2

The Government will conduct a performance risk assessment based on the quality of the Offeror's recent and relevant past performance, as well as that of its major subcontractors, as it

The Government will issue the Past Performance Questionnaires directly to the POCs provided.

relates to the probability of successful accomplishment of the required effort.

Offerors are cautioned that the Government will review available past performance data available in the Contractor Performance Assessment Reporting System (CPARS).

Please confirm that the government will provide a Past Performance Questionnaire (PPQ) template and that the government will accept PPQs from the offeror and major subcontractors in the proposal submission.

46. Draft PWS; pg 31- 40; 5.2

Due to the sensitivity of data stored in VA systems, we recommend adding a task in PWS section 5.2 for Cybersecurity Testing:

5.2.11 Cybersecurity Testing for

Software Applications. The contractor shall conduct thorough cybersecurity testing for all software applications developed or utilized by the U.S.

Department of Veterans Affairs (VA).

This will include a comprehensive assessment of software security, encompassing threat modeling, static and dynamic code analysis, and penetration testing. The contractor will identify vulnerabilities in the software and assess their potential impact on the confidentiality, integrity, and availability of sensitive VA data. All testing will align with established standards such as NIST SP 800-53 and Open Worldwide Application Security Project (OWASP) Top Ten and FISMA guidelines. The contractor will provide detailed reports outlining identified vulnerabilities, risk assessments, and prioritized remediation recommendations. Ongoing collaboration with VA security teams will ensure that testing protocols are integrated with the VA's cybersecurity framework, thereby enhancing the overall security posture of VA software applications.

The IETSS Contractor is not responsible for Cybersecurity testing.

47. Government Furnished Property

Paragraph 6.6 Government Furnished Property states in part, “the Government estimates that the following GFE will be required by this effort:

Recommend the VA provide more detail as to how many minimal personnel are required upon award and what and how many resources are required in each functional area described in Paragraph 5 (all sections).

Recommend the VA evaluate what the “potential work increases” are and provide specifics around those “potential” requirements. Is the increased workload anticipated and does the VA intent to expand this contract to include all or, at a minimum, more of the VA development efforts for required use of this new contract for testing? It is a big unknown what the intent is and recommend clarification.

Finally, there are no optional tasks for this “potential work increases” component. If the VA intends to add additional work at a later date, recommend that optional tasks be included with the expected labor categories and number of FTEs be added to the RFP, including the pricing table and Section B.

Incremental increases could be accomplished with Optional CLINs that can be exercised multiple times.

This requirement includes a Time and Materials (T&M) aspect which allows for flexibility in uncertain workload fluctuations.

See Attachment 001 - IETSS Excel Price Evaluation Spreadsheet for Labor Category Descriptions and allocation of labor hours for evaluation purposes.

48. Draft RFP E.15 Submission Instructions – Proposal Files, b.

Content Requirements, (ii)

VOLUME III –

Past Performance Files

Could the government please confirm whether the narrative requested in Section 2 of the Past Performance Files (Volume III) should address all listed contracts performed over the past three years, or if it should focus solely on the two specific instances provided for the prime contractor and each major subcontractor?

Volume III files shall include instances which are relevant as defined in the solicitation.