VA-25-00075960 - Performance Work Statement Template - AI Medical Coding_RFI.docx
DOCX document 143 KB Posted
- Attached to
- 7A20--AI Medical Coding
- Solicitation number
- 36C10B26Q0028
About this file
This Performance Work Statement (PWS) details a Department of Veterans Affairs (VA) requirement for an AI-powered medical coding solution for the National Center for Collaborative Healthcare Innovation. The project aims to develop a browser-based AI chatbot that enhances clinical documentation and coding accuracy, supporting Current Procedural Terminology (CPT), International Classification of Diseases (ICD-10), and Diagnostic Related Groups (DRG) coding. The solution will be hosted within the VA Enterprise Cloud, capable of processing clinical notes up to 2000 words with response times under five seconds, and will support concurrent use across multiple VA facilities.
Key technical requirements include developing a large language model fine-tuned for VA-specific medical terminology, implementing continuous monitoring for performance and bias detection, and creating human-in-the-loop review workflows. The project has a 12-month performance period, with phases covering planning, development, testing, deployment, training, and ongoing improvement. The contractor will be responsible for stakeholder engagement, dataset acquisition, model training, beta testing in clinical settings, and pilot deployment across the VA Palo Alto network, with strict compliance requirements around federal cybersecurity, privacy, and information system standards.
View the file
Other files
| File | Type | Posted |
|---|---|---|
| RFI-36C10B26Q0028_Extended.docx | DOCX document | |
| 36C10B26Q0028.docx | DOCX document |
On GovTribe
Work with this file
- Download the original file
- Contacts named in this file
- Similar government files
- Ask GovTribe AI about this file
Text version
AI Medical Coding
VA-25-00075960
PERFORMANCE WORK STATEMENT (PWS)
DEPARTMENT OF VETERANS AFFAIRS
National Center for Collaborative Healthcare Innovation (NCCHI)
AI Medical Coding
Date: 12/03/2025
VA-25-00075960
PWS Version Number: 2.0
BACKGROUND
The Department of Veterans Affairs (VA), National Center for Collaborative Healthcare Innovation (NCCHI) strives to provide high quality, effective, and efficient Information Technology (IT) services to those responsible for providing care to the Veterans at the point-of-care as well as throughout all the points of the Veterans’ health care in an effective, timely and compassionate manner. In accordance with Executive Order 13960, which mandates the adoption of trustworthy artificial intelligence (AI) to enhance government efficiency and effectiveness, this project's Performance Work Statement outlines our commitment to leveraging AI innovation. The order underscores the importance of acquiring AI capabilities that are both cost-effective and timely while remaining secure and reliable.
Our strategy aligns with overarching federal guidance advocating for a competitive American AI marketplace, ensuring that government acquisitions not only benefit from cutting-edge solutions but also support interoperability and vendor diversity. We prioritize safeguarding taxpayer resources by closely monitoring AI performance and managing associated risks.
The project will engage cross-functional teams early in the acquisition process to address unique challenges presented by AI technologies. By doing so, we aim to streamline procurement procedures and optimize outcomes in alignment with the principles set forth in OMB Memorandum M-25-22—Driving Efficient Acquisition of Artificial Intelligence in Government.
APPLICABLE DOCUMENTS
In the performance of the tasks associated with this Performance Work Statement, the Contractor shall comply with the following:
1. “Federal Information Security Modernization Act of 2014”
2. Federal Information Processing Standards (FIPS) Publication 140-3, “Security Requirements for Cryptographic Modules”, March 22, 2019
3. FIPS Pub 199. “Standards for Security Categorization of Federal Information and Information Systems,” February 2004
4. FIPS Pub 200, “Minimum Security Requirements for Federal Information and Information Systems,” March 2006
5. FIPS Pub 201-3, “Personal Identity Verification of Federal Employees and Contractors,” January 2022
6. 10 U.S.C. § 2224, "Defense Information Assurance Program", as amended
7. 5 U.S.C. § 552a, as amended, “The Privacy Act of 1974”
8. Public Law 109-461 (P.L. 109-461), Veterans Benefits, Health Care, and Information Technology Act of 2006, as amended, Title IX, Information Security Matters
9. 42 U.S.C. § 2000d “Title VI of the Civil Rights Act of 1964”, as amended
10. VA Directive 0710, “Personnel Security and Suitability Program,” June 4, 2010, https://www.va.gov/vapubs/index.cfm
11. VA Handbook 0710, “Personnel Security and Suitability Program,” May 2, 2016, https://www.va.gov/vapubs/index.cfm
12. VA Directive 6102, “Internet/Intranet Services,” August 5, 2019
13. 36 C.F.R. Part 1194 “Information and Communication Technology Standards and Guidelines,” as amended
14. Office of Management and Budget (OMB) Circular A-130, “Managing Federal Information as a Strategic Resource,” July 28, 2016
15. 32 C.F.R. Part 199, “Civilian Health and Medical Program of the Uniformed Services (CHAMPUS)”, as amended
16. NIST SP 800-66 Rev. 2, “Implementing the Health Insurance Portability and Accountability Act (HIPAA) Security Rule: A Cybersecurity Resource Guide,” February 2024
17. 45 C.F.R Parts 160 and 164, Subparts A and E, the Standards for Privacy of Individually Identifiable Health Information (“Privacy Rule”); and 45 C.F.R. Parts 160 and 164, Subparts A and C, the Security Standard (“Security Rule”); as amended
18. Sections 504 and 508 of the Rehabilitation Act (29 U.S.C. § 794d), as amended
19. Homeland Security Presidential Directive (12) (HSPD-12), August 27, 2004
20. VA Directive 6500, “VA Cybersecurity Program,” February 24, 2021 (see VA NOTICE 24-18, dated September 27, 2024, “Update to VA Directive 6500 Cybersecurity Program”, and VA Notice 25-07, dated 3/18/25, “Amending VA Directive 6500 to incorporate M-24-15”)
21. VA Handbook 6500, “Risk Management Framework for VA Information Systems VA Information Security Program,” February 24, 2021 (see VA Notice 25-01, dated October 15, 2024, “Update to VA Handbook 6500 Risk Management Framework for VA Information Systems VA Information Security Program”, and VA Notice 25-06, dated 3/18/25, “Amending VA Directive 6500 to incorporate M-24-15”)
22. VA Handbook 6500.2, “Management of Breaches Involving Sensitive Personal Information (SPI),” June 30, 2023
23. VA Handbook 6500.6, “Contract Security,” March 12, 2010 (see VA Notice 24-12, dated April 22, 2024, “Update to VA Handbook 6500.6, Contract Security, Appendix C VA Information and Information System Security/Privacy Language For Inclusion Into Contracts, As Appropriate”)
24. VA Handbook 6500.8, “Information System Contingency Planning,” March 25, 2025
25. VA Handbook 6500.10, “Mobile Device Security Policy,” February 15, 2018
26. VA Handbook 6500.11, “VA Firewall Configuration,” August 22, 2017
27. OIT Process Asset Library (PAL), OIT Process Asset Library.
28. One-VA Technical Reference Model (TRM) (reference at https://www.va.gov/trm/TRMHomePage.aspx)
29. VA Directive 6508, “Implementation of Privacy Threshold Analysis and Privacy Impact Assessment,” October 15, 2014
30. VA Directive 6510, “VA Identity, Credential and Access Management,” September 3, 2024
31. VA Handbook 6510, “VA Identity, Credential and Access Management,” September 27, 2024
32. VA Directive and Handbook 6513, “Secure External Connections,” October 12, 2017
33. VA Directive 6300, “Records and Information Management,” September 21, 2018
34. VA Handbook, 6300.1, “Records Management Procedures,“ March 24, 2010
35. NIST SP 800-37 Rev 2, “Risk Management Framework for Information Systems and Organizations: A System Life Cycle Approach for Security and Privacy,” December 2018
36. NIST SP 800-53 Rev. 5, “Security and Privacy Controls for Federal Information Systems and Organizations,” September 23, 2020 (includes updates as of 12/10/2020)
37. VA Directive 0735, “Homeland Security Presidential Directive 12 (HSPD-12) Program,” October 26, 2015
38. VA Handbook 0735, “Homeland Security Presidential Directive 12 (HSPD-12) Program,” March 24, 2014
39. OMB Memorandum 05-24, “Implementation of Homeland Security Presidential Directive (HSPD) 12 – Policy for a Common Identification Standard for Federal Employees and Contractors,” August 5, 2005
40. OMB Memorandum M-19-17, “Enabling Mission Delivery Through Improved Identity, Credential, and Access Management,” May 21, 2019
41. OMB Memorandum, “Guidance for Homeland Security Presidential Directive (HSPD) 12 Implementation,” May 23, 2008
42. Federal Identity, Credential, and Access Management (FICAM) Architecture, June 30, 2023 (FICAM Architecture (idmanagement.gov))
43. NIST SP 800-116 Rev 1, “Guidelines for the Use of Personal Identity Verification (PIV) Credentials in Facility Access,“ June 2018
44. NIST SP 800-63-3, 800-63A, 800-63B, 800-63C, “Digital Identity Guidelines,” updated March 02, 2020
45. NIST SP 800-157, “Guidelines for Derived PIV Credentials,” December 2014
46. VA Memorandum, VAIQ #7100147, “Continued Implementation of Homeland Security Presidential Directive 12 (HSPD-12),” April 29, 2011 (reference https://www.voa.va.gov/documentlistpublic.aspx?NodeID=514)
47. IAM Identity Management Business Requirements Guidance document, May 2013, (reference Enterprise Architecture Section, PIV/IAM (reference https://www.voa.va.gov/documentlistpublic.aspx?NodeID=514)
48. VA Memorandum “Personal Identity Verification (PIV) Logical Access Policy Clarification,” July 17, 2019, https://www.voa.va.gov/DocumentView.aspx?DocumentID=4896
49. Trusted Internet Connections (TIC) 3.0 Core Guidance Documents, https://www.cisa.gov/publication/tic-30-core-guidance-documents
50. OMB Memorandum M-19-26, “Update to the Trusted Internet Connections (TIC) Initiative,” September 12, 2019
51. OMB Memorandum M-08-23, “Securing the Federal Government’s Domain Name System Infrastructure,” August 22, 2008
52. Office of Information Security (OIS) VAIQ #7424808 Memorandum, “Remote Access,” January 15, 2014, https://www.voa.va.gov/DocumentListPublic.aspx?NodeId=28
53. Clinger-Cohen Act of 1996, 40 U.S.C. §11101 and §11103
54. “Veteran Focused Integration Process (VIP) Guide 4.0,” March 2021, https://www.voa.va.gov/DocumentView.aspx?DocumentID=4371
55. VA Memorandum “Proper Use of Email and Other Messaging Services,” January 2, 2018, https://www.voa.va.gov/DocumentListPublic.aspx?NodeId=28
56. “Product Line Management Transformation Playbook” version 3.1, August 2023, https://www.voa.va.gov/DocumentView.aspx?DocumentID=4946
57. NIST SP 500-267B Revision 1, “USGv6 Profile,” November 2020
58. OMB Memorandum M-21-07, “Completing the Transition to Internet Protocol Version 6 (IPv6),” November 19, 2020
59. Social Security Number (SSN) Fraud Prevention Act of 2017
60. Section 240 of the Consolidated Appropriations Act (CAA) 2018, March 23, 2018
61. C.F.R Title 15 Part 7, “Securing the Information and Communications Technology and Services (ICTS) Supply Chain”, as amended
62. Executive Order 14028, “Executive Order on Improving the Nation's Cybersecurity,” May 12, 2021, https://bidenwhitehouse.archives.gov/briefing-room/presidential-actions/2025/01/16/executive-order-on-strengthening-and-promoting-innovation-in-the-nations-cybersecurity/
63. OMB Memorandum M-22-09, “Moving the U.S. Government Toward Zero Trust Cybersecurity Principles”, January 26, 2022, https://www.whitehouse.gov/wp-content/uploads/2022/01/M-22-09.pdf.
64. NIST SP 800-52 Revision 2, “Guidelines for the Selection, Configuration, and Use of Transport Layer Security (TLS) Implementations
65. OMB M-22-18, “Enhancing the Security of the Software Supply Chain through Secure Software Development Practices,” September 14, 2022
66. OMB M-23-16, “Update to Memorandum M-22-18,” June 9, 2023
67. OMB M-21-31, “Improving the Federal Government’s Investigative and Remediation Capabilities Related to Cybersecurity Incidents,” August 27, 2021
68. NIST SP 800-88 Revision 1 “Guidelines for Media Sanitization,” December 2014
69. CISA Binding Operational Directive (BOD) 19-02: “Vulnerability Remediation Requirements of Internet-Accessible Systems,” April 29, 2019, BOD 19-02: Vulnerability Remediation Requirements for Internet-Accessible Systems | CISA
70. CISA BOD 22-01: “Reducing the Significant Risk of Known Exploited Vulnerabilities,” November 3, 2021, BOD 22-01: Reducing the Significant Risk of Known Exploited Vulnerabilities | CISA
71. CISA BOD 23-01: Improving Asset Visibility and Vulnerability Detection on Federal Networks.” Cybersecurity & Infrastructure Security Agency (CISA), BOD 23-01: Improving Asset Visibility and Vulnerability Detection on Federal Networks | CISA
72. VA Directive 6550, Pre-Procurement Assessment and Implementation of Medical Devices/Systems, June 03, 2019
73. VA Memorandum, “VA Security Controls,” January 17, 2025, https://www.voa.va.gov/DocumentView.aspx?DocumentID=5010
SCOPE OF WORK
The Contractor shall design, develop, deploy, and support a real-time, browser-based AI chatbot that enhances clinical documentation and coding accuracy for Current Procedural Terminology (CPT), International Classification of Diseases, 10th Revision (ICD-10), Comorbid Condition/ Major Comorbid Condition (CC/MCC), and Diagnostic Related Groups (DRG)-like groupings. The solution shall be hosted within the VA Enterprise Cloud (VAEC), support rapid response times of five seconds or less for standard clinical notes up to 2000 words and enable concurrent use across multiple VA users and facilities. The solution shall allow clinicians and coders to copy and paste clinical notes into the interface for analysis by a coding-specific large language model (LLM) capable of highlighting key clinical terms, recommending clearer or more complete terminology, providing rationale for suggestions, supporting structured note creation, and enabling version control with approval or rejection of edits. The solution shall display assigned and suggested CPT/ICD codes through roll-over indicators and generate lists of current and additional recommended codes based on documentation improvements, while supporting VA terminology, reimbursement hierarchies, CC/MCC logic, DRG-like groupings, and multiple clinical disciplines. The LLM shall be fine-tuned for VA-specific terminology and updated regularly to reflect evolving VA policies, clinical practices, and reimbursement guidance, while maintaining performance for long or complex clinical notes. The Contractor shall implement continuous monitoring to detect performance degradation or bias, support human-in-the-loop review and feedback workflows, and track utilization, text edits, and coding suggestions for auditability and ongoing model improvement. While full Computerized Patient Record System (CPRS) integration is not required during the first year, the system shall be architected to support future capabilities including retrieval, reformatting, and reinsertion of clinical notes into CPRS as VA technical constraints evolve. The Contractor shall also develop and provide training modules, quick reference guides, and a clear plan defining accountability for updates, model retraining cycles, and feedback management to support successful adoption and sustainment of the solution.
PERFORMANCE DETAILS
PERFORMANCE PERIOD
The Period of Performance shall be 12-months from date of award.
Any work at the Government site shall not take place on Federal holidays or weekends unless directed by the Contracting Officer (CO).
There are 11 Federal holidays set by law (USC Title 5 Section 6103) that VA follows:
Under current definitions, five are set by date:
| New Year's Day | January 1 | |
| Juneteenth | June 19 | |
| Independence Day | July 4 | |
| Veterans Day | November 11 | |
| Christmas Day | December 25 |
If any of the above falls on a Saturday, then Friday shall be observed as a holiday. Similarly, if one falls on a Sunday, then Monday shall be observed as a holiday.
The other six are set by a day of the week and month:
| Martin Luther King's Birthday | Third Monday in January |
| Washington's Birthday | Third Monday in February |
| Memorial Day | Last Monday in May |
| Labor Day | First Monday in September |
| Columbus Day | Second Monday in October |
| Thanksgiving | Fourth Thursday in November |
PLACE OF PERFORMANCE
Tasks under this PWS shall be performed at the Contractor. Any work at the Government site shall not take place on Federal holidays or weekends unless authorized and/or directed by the Contracting Officer (CO). Federal holidays are set by law (USC Title 5 Section 6103). The contractor shall observe the Federal holidays observed during the period of performance. Observation of Federal holidays can be found at www.opm.gov, and the contract shall observe the Federal holidays as indicated for each year the contract is in place.
TRAVEL
There is no anticipated travel associated with this requirement.
SPECIFIC TASKS AND DELIVERABLES
The Contractor shall perform the following:
PROJECT MANAGEMENT
| Phase |
| Key Activities |
| Timeline |
Phase 1: Planning & Design
| Requirements gathering, architecture design, LLM selection, VA terminology mapping |
| Month 2 |
| Phase 2: Development |
| Build chatbot interface, integrate LLM, implement coding logic, version control, and UI features |
| Month 6 |
Phase 3: Testing & Validation
| Performance testing (≤5s response), bias detection, usability testing, VA terminology validation |
| Month 8 |
Phase 4: Deployment
| VAEC hosting, multi-user support, rollout to pilot VA facilities |
| Month 9 |
Phase 5: Training & Documentation
| Develop training modules, quick reference guides, feedback workflows |
| Month 10 |
Phase 6: Monitoring & Improvement
| Continuous monitoring, feedback loop, model retraining, audit tracking |
| Month 12 |
REPORTING REQUIREMENTS
The Contractor shall provide the Contracting Officer Representative (COR) with monthly Progress Report in electronic form in Microsoft Word and Project formats. The report shall include detailed instructions/explanations for each required data element, to ensure that data is accurate and consistent. These reports shall reflect data as of the last day of the preceding monthly.
The monthly Progress Report shall cover all work completed during the reporting period and work planned for the subsequent reporting period. The report shall also identify any problems that arose and a description of how the problems were resolved. If problems have not been completely resolved, the Contractor shall provide an explanation including their plan and timeframe for resolving the issue. The report shall also include an itemized list of all Information and Communication Technology (ICT) deliverables and their current Section 508 conformance status. It is expected that the Contractor will keep in communication with VA accordingly so that issues that arise are transparent to both parties to prevent escalation of outstanding issues.
Deliverable:
A. Monthly Progress Report
PLANNING AND REQUIREMENTS ANALYSIS
The Contractor shall engage stakeholders—including physicians, medical coders, IT personnel, and other relevant users—to collect input that informs system functionality and design. This process shall focus on identifying practical requirements, workflow considerations, and user expectations to ensure the solution supports clinical and operational needs effectively. Additionally, the Contractor shall define comprehensive system requirements and specifications that address the identified needs from the assessment. The Contractor shall prepare and deliver a comprehensive Systems Report that documents all summaries, analysis results, and recommendations from the planning and requirements analysis. The Systems report shall serve as a foundational reference for system design, development, testing, and implementation activities. See Table 1.0 Systems Report.
The Contractor shall develop a comprehensive project plan that clearly outlines the timelines, milestones, and resource allocation necessary for the successful completion of the project. This plan should provide a structured roadmap that ensures all phases of the project are effectively managed and executed in a timely manner, with appropriate resources allocated to each task to achieve the project's objectives. The plan should reflect a detailed understanding of project scope, critical milestones, and key deliverables, facilitating clear tracking and management of progress throughout the project lifecycle.
Table 1.0 Systems Report
| Section Title |
| Description |
| Executive Summary |
| Overview of the assessment process, key findings of needs assessment with stakeholders, and a summary of major system and data requirements. |
| Stakeholder Needs Assessment Summary |
| Detailed account of stakeholder engagement activities, including identified user needs, pain points, workflow challenges, and system expectations gathered from physicians, coders, IT personnel, and other stakeholders. |
| Functional and Technical Requirements |
| Clearly defined system specifications and requirements, categorized by functional, technical, and performance criteria. This section should include any data integration, security, compliance (e.g., HIPAA), and interoperability with existing VA systems and data exchange standards (e.g., HL7/FHIR). |
| Gap Analysis |
| Evaluation of current system capabilities versus required capabilities, highlighting deficiencies, risks, and potential solutions. |
| System Architecture Overview |
| Conceptual description of the proposed system architecture, including data flow diagrams, integration points with VA systems, hardware/software environments, and major system components. |
| Project Plan Summary |
| High-level overview of project timelines, milestones, resource requirements, dependencies, and risk management considerations. |
| Validation and Evaluation Plan |
| Outline of proposed metrics and evaluation methods to measure success during system testing and implementation (e.g., coding accuracy, discrepancy rates, throughput). |
| Recommendations and Next Steps |
| Summary of prioritized actions, decisions, or additional analyses required prior to system development or procurement. |
| Frequency and Timeline |
| An initial draft Systems Report shall be submitted within 30 calendar days following completion of the stakeholder needs assessment. |
The final Systems Report shall be submitted within 10 business days after receipt of Government feedback on the draft.
Additional update reports or addenda may be requested by the Government on a quarterly basis or as major changes to requirements are identified during project execution.
Deliverables :
A. Systems report B. Project Plan
AI MODEL DEVELOPMENT & TRAINING
The Contractor shall acquire de-identified clinical datasets from clinical champion for the purpose of model training. These datasets shall maintain patient confidentiality and comply with all relevant data privacy regulations. The datasets should be robust and representative, ensuring that the AI model is trained on accurate and comprehensive clinical information. The Contractor shall prepare and deliver a Data Acquisition Report documenting the processes, sources, and standards applied in obtaining de-identified clinical datasets for AI model training. The report shall ensure full transparency and traceability of data collection and preprocessing activities and confirm compliance with all applicable privacy, security, and data governance regulations. Data Acquisition Report see Table 2.0
The Contractor shall select appropriate AI algorithms specifically tailored for medical coding. These algorithms should be chosen based on their suitability to handle the complexities and nuances of medical data, ensuring they can accurately and efficiently process clinical information. The selected algorithms shall demonstrate the capability to integrate seamlessly into the existing system while enhancing the accuracy and reliability of the coding process.
The Contractor shall train the initial AI model using historical data. This data should be comprehensive and representative of the scenarios the AI model will encounter in real-world applications. The training process shall ensure that the model accurately learns from past examples to develop patterns and insights necessary for effective medical coding. The Contractor shall deliver a fully trained and validated AI model package designed for medical coding applications. The model shall demonstrate operational readiness, reproducibility, and scalability for integration into the existing VA systems. The deliverable shall include all necessary artifacts, documentation, and performance metrics to allow the Government to evaluate, reproduce, and deploy the model within secure VA environments. Trained AI Model see Table 3.0.
Table 2.0 Data Acquisition Report
| Section Title |
| Description |
| Data Sources and Provenance |
| Description of all data sources used, including dataset origin, ownership, custodianship, and justification for selection. The Contractor shall identify any data refresh frequency, access permissions, and limitations of use. |
| De-identification and Privacy Compliance |
| Detailed explanation of the methods used to de-identify patient data, including applied and data minimization practices. |
| Data Summary and Structure |
| Overview summary of data types, volume, and distribution across clinical domains (e.g., diagnoses, procedures, notes, imaging). |
| Data Quality Assessment |
| Quantitative assessment of data completeness, accuracy and consistency. Include identification of missing values, outliers, or potential biases that might affect model performance. |
| Data Preprocessing and Normalization |
| Description of all preprocessing steps including data cleaning, formatting, annotation, labeling, and normalization procedures. |
| Security and Storage Controls |
| Overview of where and how data are stored and processed, encryption methods, and access control measures. |
| Ethical and Regulatory Compliance Statement |
| Confirmation that all data use conforms with relevant laws, institutional policies, and ethical standards (e.g., HIPAA, VA Handbook 6500, etc.). |
| Data Governance and Documentation |
| Description of dataset versioning, documentation of transformations, and procedures to ensure reproducibility and auditability of all data handling steps. |
Table 3.0 Trained AI Model
| Section Title |
| Description |
| Model Specifications |
| The trained model weights, model files, and architecture definition (e.g., configuration files, layer specifications, or model card). Specifications shall include version identifiers and checksum validation to ensure reproducibility. |
| Model Documentation |
| Comprehensive documentation describing model objectives, intended use, input/output schemas, data preprocessing requirements, and dependencies. Include step-by-step documentation on how to load, execute, and integrate the model into the target system, including any configuration files and API endpoints. |
| Training Configuration and Environment Summary |
| Detailed description of training data used (referencing the Data Acquisition Report), data splits (train/validation/test), hyperparameter settings, model version, and hardware/software environment. Include notes on any frameworks or libraries used (e.g., PyTorch, TensorFlow). |
| Evaluation and Validation Results |
| Quantitative performance metrics (e.g., accuracy, precision, recall, F1-score, confusion matrix), validation methodology, cross-validation approach, and summary of test datasets used. |
| Bias and Fairness Assessment |
| Evaluation of potential biases in model predictions across demographic or clinical subgroups; mitigation strategies if applicable. |
| Deployment Package |
| Model packaged for deployment, including APIs, scripts, or containerized environments as applicable to the project’s operational framework. |
Deliverables:
A. Data Acquisition Report B. Trained AI Model
BETA TESTING IN CLINICAL SETTINGS
The Contractor shall implement the AI model in surgical and primary care settings for testing. During this testing phase, the Contractor shall closely monitor the AI model's performance and meticulously document any discrepancies that arise between AI-generated codes and those produced by human coders. This careful monitoring and documentation are essential to identify areas where the AI model may need further refinement and to ensure that its coding accuracy meets or exceeds the standards set by human coders. The Contractor shall deliver a comprehensive Beta Testing Implementation Plan that outlines the strategy, methods, and procedures for testing AI models within surgical and primary care environments. The plan shall ensure structured and measurable testing activities, aligned with clinical workflows and VA technical structure. Beta Testing Implementation Plan see table 4.0.
The Contractor shall prepare a detailed report summarizing the finding of the beta testing phase, focusing on differences between AI-generated codes and those produced by human coders. This report should provide actionable insights for model refinement, improvement of coding accuracy, workflow optimization, and ensure production readiness. Discrepancy Analysis Report see table 5.0.
Table 4.0 Beta Testing Implementation Plan
| Section Title |
| Description |
| Objectives and Scope |
| Description of specific goals of beta test, including validation of medical coding accuracy, reliability, workflow integration, and user acceptance. |
| Test Environments and Site Selection |
| Identification of participating VA clinical sites, along with justification for selection based on representativeness and technical readiness. |
| Integration and System Interoperability |
| Description of how the trained AI model will be technically integrated and tested within existing VA systems based on the system architecture and deployment specifications defined in Sections 5.2 and 5.3. Include details on data exchange methods, APIs, and validation of output integration within clinical workflows. |
| Participant Roles/Responsibilities |
| Identification of all personnel involved in testing along with respective roles and responsibilities |
| Testing Methods |
| Detailed testing approach, including AI model execution procedures, human coder comparisons, and how discrepancies will be documented and analyzed. |
| Performance Metrics/Criteria |
| Quantitative and qualitative measures for evaluating AI performance (e.g., accuracy, precision, recall, efficiency, workflow impact, user feedback). |
| Feedback/Improvement Process |
| Description of how user and coder feedback will be collected, analyzed, and incorporated into model refinement. |
| Schedule/Milestones |
| Detailed timeline for testing phase with anticipated completion dates. |
| Risk Management Plan |
| Identification of potential risks (e.g., workflow disruptions, data access issues, integration failures) along with mitigation strategies. |
Table 5.0 Discrepancy Analysis Report
| Section Title |
| Description |
| Executive summary |
| Overview of beta testing phase (e.g., duration, cases evaluated, high-level outcomes) |
| Comparison Results |
| Statistical summary of coding discrepancies (e.g., total matches, partial matches, mismatches, and unclassified outputs) |
| Error Categorization |
| Classification of discrepancies by type (e.g., omission, incorrect modifier), with root-cause evaluation. |
| Impact Assessment |
| Analysis of potential clinical, operational, or billing implications. |
| Corrective Action |
| – Guidance/recommendations for retraining, parameter tuning, or rule adjustments to mitigate identified errors. |
| Model Integration Performance |
| Summary of how model outputs performed when integrated into VA systems and workflows, referencing interoperability and testing parameters defined in Section 5.4.A. Include any technical or workflow challenges observed during result ingestion, display, or feedback capture. |
| Bias and Fairness Review |
| Evaluation of discrepancies or performance differentials across demographic or clinical subgroups; note any corrective steps taken. |
| Lessons Learned and Next Steps |
| Documentation of key findings, process improvements, and recommendations for subsequent iterations or deployment. |
Deliverables:
A. Beta Testing Implementation Plan B. Discrepancy Analysis Report
PILOT DEPLOYMENT
The Contractor shall deploy refined model across pilot sites within VA Palo Alto network.. This deployment aims to evaluate the model's effectiveness and scalability in varied clinical settings. Offerors must ensure that the model is seamlessly integrated into these additional sites, maintaining consistent performance and reliability. The deployment across multiple locations will provide valuable insights and further validate the AI model's capability to support medical coding processes across the VA Palo Alto network.
GENERAL REQUIREMENTS
ENTERPRISE AND IT FRAMEWORK
VA TECHNICAL REFERENCE MODEL
The Contractor shall comply with the VA OIT Technical Reference Model (VA TRM). Compliance with the VA TRM is achieved by using only technologies and standards that are listed as approved for use in the VA TRM. The Contractor shall provide all necessary information requested by VA to ensure TRM approval is obtained prior to use on VA’s network.
ZERO TRUST – VA CRITICAL SECURITY CONTROLS
VA has established minimum mandatory security requirements and requires that any network connected software system or service must meet the VA Critical Security Controls as outlined in the VA Memorandum, “VA Security Controls”, https://www.voa.va.gov/DocumentView.aspx?DocumentID=5010. VA Critical Security Controls identify the minimum mandatory requirements that must be implemented across all VA enterprise infrastructure, cloud computing environments, information systems, networks, and specialized devices (medical devices/systems, special-purpose systems, and research scientific computing devices) that process, store, and/or transmit VA data. Effective July 1, 2025, the Contractor shall implement these VA Critical Security Controls, within any network connected software system or service prior to being authorized for use in the VA. This functional requirement is not negotiable, and Plan of Action & Milestones (POAM) will not be accepted in the event these controls cannot be implemented for new systems. Critical Security Controls are intended to increase VA’s security posture and provide security and privacy risk visibility into the VA network and is not a new requirement. The Contractor’s failure to maintain these VA Critical Controls after implementation will result in VA discontinuing the use of the system.
FEDERAL IDENTITY, CREDENTIAL, AND ACCESS MANAGEMENT (FICAM)
The Contractor shall ensure Commercial Off-The-Shelf (COTS) product(s), software configuration and customization, and/or new software are Personal Identity Verification (PIV) card-enabled by accepting HSPD-12 PIV credentials using VA Enterprise Architecture (EA), and VA Identity and Access Management (IAM) approved Enterprise Technology Guidelines (ETG). The Contractor shall ensure all Contractor delivered applications and systems comply with the VA Identity, Credential, and Access Management policies and guidelines set forth in Executive Order 14028, VA Handbook 6510 VA Identity and Access Management, VA Handbook 0735 Homeland Security Presidential Directive 12 (HSPD-12) Program, CISA Binding Operational Directive 23-01, and align with the Federal Identity, Credential, and Access Management Roadmap and Implementation Guidance v2.0.
The Contractor shall ensure all Contractor delivered applications and systems provide user authentication services compliant with the National Institute of Standards and Technology (NIST) Special Publication (SP) 800-63-3, VA Handbook 6500, and VA IAM enterprise requirements for direct, assertion based authentication, and/or trust based authentication, as determined by the ETGs. Direct authentication at a minimum must include Public Key Infrastructure (PKI) based authentication supportive of PIV card and/or Common Access Card (CAC), as determined by the business need.
The Contractor shall ensure all Contractor delivered applications and systems conform to the specific Identity and Access Management PIV requirements set forth in the Office of Management and Budget (OMB) Memoranda M-05-24, M-19-17, M-22-09, and NIST Federal Information Processing Standard (FIPS) 201-3. Contractor delivered applications and systems shall be on the FIPS 201 Approved Product List (APL). If the Contractor delivered application and system is not on the APL, the Contractor shall be responsible for taking the application and system through the FIPS 201 Evaluation Program.
The Contractor shall ensure all Contractor delivered applications and systems support:
1. Automated provisioning and are able to use enterprise provisioning service.
2. Interfacing with VA’s Master Person Index (MPI) to provision identity attributes, if the solution relies on VA user identities. MPI is the authoritative source for VA user identity data.
3. The VA defined unique identity (Secure Identifier [SEC ID] / Integrated Control Number [ICN]).
4. Multiple authenticators for a given identity and authenticators at every Authenticator Assurance Level (AAL) appropriate for the solution.
5. Identity proofing for each Identity Assurance Level (IAL) appropriate for the solution.
6. Federation for each Federation Assurance Level (FAL) appropriate for the solution, if applicable.
7. Two-factor authentication (2FA) through an applicable Technology Guideline as outlined in VA ETG.
8. A Security Assertion Markup Language (SAML) implementation if the solution relies on assertion-based authentication. Additional assertion implementations, besides the required SAML assertion, may be provided as long as they are compliant with NIST SP 800-63-3 guidelines.
9. Authentication/account binding based on trusted Hypertext Transfer Protocol (HTTP) headers if the solution relies on Trust based authentication.
10. Role Based Access Control.
11. Auditing and reporting capabilities.
12. Compliance with VIEWS 00155984, PIV Logical Access Policy Clarification https://www.voa.va.gov/DocumentView.aspx?DocumentID=4896.
The required Assurance Levels for this specific effort are Identity Assurance Level 3, Authenticator Assurance Level 3, and Federation Assurance Level 3.
INTERNET PROTOCOL VERSION 6 (IPv6) The Contractor solution shall support IPv6-Only based upon the memo issued by the Office of Management and Budget (OMB) on November 19, 2020 (https://www.whitehouse.gov/wp-content/uploads/2020/11/M-21-07.pdf). Which defines IPv6-only as the state of an operational system or service when IPv4 protocol functions (addressing, packet forwarding) are not in use. The NIST USGv6 profile defines technical requirements for a product to be capable of operating in IPv6-Only environments. IPv6-Only technology, in accordance with the USGv6 Program (https://www.nist.gov/programs-projects/usgv6-program/usgv6-revision-1), NIST Special Publication (SP) 500-267B Revision 1 “USGv6 Profile” (https://doi.org/10.6028/NIST.SP.500-267Br1), and NIST SP 800-119 “Guidelines for the Secure Deployment of IPv6” (https://doi.org/10.6028/NIST.SP.800-119), compliance shall be included in all IT infrastructures, application designs, application development, operational systems and sub-systems, and their integration. In addition to the above requirements, all devices, applications, and systems shall support all applicable functionality on native IPv6-Only as well as dual stack (IPv6 / IPv4) connectivity without additional memory or other resources being provided by the Government, so that they can function in a mixed environment. All public/external facing servers and services (e.g., web, email, DNS, ISP services, etc.) shall support native IPv6-Only and dual stack (IPv6 / IPv4) users and all internal infrastructure and applications shall support using native IPv6-Only and dual stack (IPv6 / IPv4) for all functionality and operations.
TRUSTED INTERNET CONNECTION (TIC)
The Contractor solution shall meet the requirements outlined in Office of Management and Budget Memorandum M-19-26, “Update to the Trusted Internet Connections (TIC) Initiative“ (https://www.whitehouse.gov/wp-content/uploads/2019/09/M-19-26.pdf), VA Directive 6513 “Secure External Connections”, and shall comply with the TIC 3.0 Core Guidance Documents, including all Volumes and TIC Use Cases, found at the Cybersecurity & Infrastructure Security Agency (CISA) (https://www.cisa.gov/publication/tic-30-core-guidance-documents).
STANDARD COMPUTER CONFIGURATION
The Contractor IT end user solution that is developed for use on standard VA computers shall be compatible with and be supported on the standard VA operating system, Windows 10 (64bit) and Windows 11, Edge (Chromium based), and 365 Apps for enterprise. Applications delivered to VA and intended to be deployed to Windows 10 and or Windows 11 workstations shall be delivered as a signed .msi package with switches for silent and unattended installation and updates shall be delivered in signed .msp file formats for easy deployment using Microsoft Endpoint Configuration Manager (CM) VA’s current desktop application deployment tool. Signing of the software code shall be through a vendor provided certificate that is trusted by VA using a code signing authority such as Verizon/Cybertrust or Symantec/VeriSign. The Contractor shall also ensure and certify that their solution functions as expected when used from a standard VA computer, with non-admin, standard user rights that have been configured using the Defense Information Systems Agency (DISA) Secure Technical Implementation Guide (STIG) specific to the particular client operating system being used.
VETERAN FOCUSED INTEGRATION PROCESS (VIP) AND PRODUCT LINE MANAGEMENT (PLM)
The Contractor shall support VA efforts IAW the Veteran-Focused Integration Process (VIP) Guide and Product Line Management (PLM) Playbook. The major focus of VIP is on Enterprise Oversight, Governance, reporting, and establishing practices that focus on outcomes and continuously delivering sustainable IT capabilities to benefit the Veteran. PLM is a framework that focuses on delivering functional products that provide the highest priority work to customers while delivering simplified, reliable, and practical solutions to the business, medical staff, and our Veterans. The VIP Guide is a companion guide to the PLM Transformation Playbook and can be found at: https://www.voa.va.gov/DocumentView.aspx?DocumentID=4371 and the PLM Transformation Playbook can be found at https://www.voa.va.gov/DocumentView.aspx?DocumentID=4946. VIP and PLM are the authoritative processes that IT projects must follow to ensure development and delivery of IT products.
The VIP Guide provides guidance for transitioning from project-centric to product-centric delivery. The VIP guide applies to all OIT Office of Management and Budget (OMB) reportable products that are required to collect and report costs, schedules, risks, and other valuable information in VA PARS (whether products are procured or developed by VA) and any VA Information Technology project effort in Software Product Management (SPM) and Infrastructure Operations (IO), or their successors, that touch the VA network regardless of IT organizational alignment (whether it spends government funding from VA's Congressional IT Appropriation or any other appropriation).
The PLM Transformation Playbook details how OIT leadership expects Product Lines (PLs) to implement PLM, DevOps, and Lean-Agile principles, methods, practices, and techniques through levels of maturity. The PLM Transformation Playbook applies to all Information Technology (IT) products aligned to PLs in the Software Product Management, Infrastructure Operations and Product Engineering organizations within the OIT organization.
Others outside the referenced groups or organizations may use the VIP Guide and PLM Transformation Playbook for situational awareness as needed.
PROCESS ASSET LIBRARY (PAL)
The Contractor shall perform their duties consistent with the processes defined in the OIT Process Asset Library (PAL). The PAL includes a spectrum of OIT functions and activities, such as project management, operations, service delivery, communications, acquisition, and resource management. PAL serves as an authoritative and informative repository of searchable processes, activities or tasks, roles, artifacts, tools and applicable standards and guides to assist the OIT workforce, Government and Contractor personnel. OIT utilizes PAL processes to ensure compliance with policies and regulations and to meet VA quality standards. The PAL includes the contractor onboarding process consistent with Section 6.2.2 and can be found at process_CONB_ext.docx (live.com). The main PAL can be accessed at www.va.gov/process.
AUTHORITATIVE DATA SOURCES
The VA Enterprise Architecture Repository (VEAR) is one component within the overall EA that establishes the common framework for data taxonomy for describing the data architecture used to develop, operate, and maintain enterprise applications. The Contractor shall comply with the department’s Authoritative Data Source (ADS) requirement that VA systems, services, and processes throughout the enterprise shall access VA data solely through official VA ADSs where applicable, see below. The Information Classes which compose each ADS are located in the VEAR, in the Data & Information domain. The Contractor shall ensure that all delivered applications and system solutions support:
1. Interfacing with VA’s Master Person Index (MPI) (formerly the Master Veteran Index (MVI)) to provision identity attributes, if the solution relies on VA user identities. MPI is the authoritative source for VA user identity data.
2. Interfacing with Capital Asset Inventory (CAI) to conduct real property record management actions, if the solution relies on real property records data. CAI is the authoritative source for VA real property record management data.
3. Interfacing with electronic Contract Management System (eCMS) for access to contract, contract line item, purchase requisition, offering vendor, and solicitation information above the micro-purchase threshold, if the solution relies on procurement data. ECMS is the authoritative source for VA procurement actions data.
4. Interfacing with HRSmart Human Resources Information System to conduct personnel action processing, on-boarding, benefits management, and compensation management, if the solution relies on personnel data. HRSmart is the authoritative source for VA personnel information data.
5. Interfacing with VA Profile to access personal contact information, if the solution relies on VA Veteran personal contact information data. VA Profile is the authoritative source for VA Veteran Personal Contact Data.
6. Interfacing with VA/Department of Defense (DoD) Identity Repository (VADIR) for determining eligibility for VA benefits under Title 38, if the solution relies on qualifying active duty military service data. VADIR is the authoritative source for qualifying active duty military service in VA.
7. Interfacing with VA Systems Inventory (VASI) to access VA systems information, if the solution relies on VA system information. VASI is the authoritative source for VA Information Technology systems.
8. Interfacing with Enterprise Mission Assurance Support Service (eMASS) to access the Authority to Operate (ATO) status of all VA Information Systems. eMASS contains the official Authority to Operate (ATO) status of all VA Information Systems.
SOCIAL SECURITY NUMBER (SSN) REDUCTION
The Contractor solution shall support the Social Security Number (SSN) Fraud Prevention Act (FPA) of 2017 which prohibits the inclusion of SSNs on any document sent by mail. The Contractor support shall also be performed in accordance with Section 240 of the Consolidated Appropriations Act (CAA) 2018, enacted March 23, 2018, which mandates VA to discontinue using SSNs to identify individuals in all VA information systems as the Primary Identifier. The Contractor shall ensure that any new IT solution discontinues the use of SSN as the Primary Identifier to replace the SSN with the ICN in all VA information systems for all individuals. The Contractor shall ensure that all Contractor delivered applications and systems integrate with the VA MPI for identity traits to include the use of the ICN as the Primary Identifier. The Contractor solution may only use a Social Security Number to identify an individual in an information system if and only if the use of such number is required to obtain information VA requires from an information system that is not under the jurisdiction of VA.
SOFTWARE AND LICENSING REQUIREMENTS
The Contractor shall be responsible for the provision of all software licenses and any associated licensing maintenance required for any development, delivery, integration, operation, and/or maintenance associated with its proposed application(s), software products, software solution, and/or system including, but not limited to, any and all application(s), software and/or software products that comprise, are a part of, or integrate with the Contractor’s proposed application(s), software products, software solution, and/or system for the life of any resulting contract.
SECURITY AND PRIVACY REQUIREMENTS
It has been determined that protected health information may be disclosed or accessed and a signed Business Associate Agreement (BAA) shall be required. The Contractor shall adhere to the requirements set forth within the BAA, referenced in Section D of the contract, and shall comply with VA Directive 6066.
POSITION/TASK RISK DESIGNATION LEVEL(S)
In accordance with VA Handbook 0710, Personnel Security and Suitability Program, the position sensitivity and the level of background investigation commensurate with the required level of access for the following tasks within the PWS are:
Position Sensitivity and Background Investigation Requirements by Task
| Task Number |
| Tier1 / Low Risk |
| Tier 2 / Moderate Risk |
| Tier 4 / High Risk |
| 5.1 | ||
| ☒ | ||
| ☐ | ☐ | |
| 5.2 | ||
| ☒ | ☐ | ☐ |
| 5.3 | ||
| ☒ | ☐ | ☐ |
| 5.4 | ||
| ☒ | ☐ | ☐ |
| 5.5 | ||
| ☒ | ☐ | ☐ |
The Tasks identified above and the resulting Position Sensitivity and Background Investigation requirements identify, in effect, the Background Investigation requirements for Contractor individuals, based upon the tasks the particular Contractor individual will be working. The submitted Contractor Staff Roster must indicate the required Background Investigation Level for each Contractor individual based upon the tasks the Contractor individual will be working, in accordance with their submitted proposal.
METHOD AND DISTRIBUTION OF DELIVERABLES
The Contractor shall deliver documentation in electronic format, unless otherwise directed in Section B of the solicitation/contract. Acceptable electronic media include: Microsoft 365 suite, MS Word, MS Excel, MS PowerPoint, MS Project, MS Access, MS Visio, AutoCAD, and Adobe Postscript Data Format (PDF).
PERFORMANCE METRICS
| Performance Objective |
| Performance Standard |
| Acceptable Levels of Performance |
| A. Technical / Quality of Product or Service |
| 1. Demonstrates understanding of requirements |
2. Efficient and effective in meeting requirements
3. Meets technical needs and mission requirements
4. Provides quality services/products
5. Incorporates “ease of use” Human Centered Design principles in any software developed.
Satisfactory or higher
| B. Project Milestones and Schedule |
| 1. Established milestones and project dates are met |
2. Products completed, reviewed, delivered in accordance with the established schedule
3. Notifies customer in advance of potential problems Satisfactory or higher
| C. Cost & Staffing |
| 1. Currency of expertise and staffing levels appropriate to perform tasks required |
2. Personnel possess necessary knowledge, skills, and abilities to perform tasks Satisfactory or higher
| D. Management |
| 1. Integration and coordination of all activities to execute effort |
| Satisfactory or higher |
The COR will utilize a Quality Assurance Surveillance Plan (QASP) throughout the life of the contract to ensure that the…
This is the start of the file's text. The full file is on GovTribe.
File details come from the government source that posted it. Updated .