Attachment A PWS.docx

PERFORMANCE WORK STATEMENT (PWS)

DEPARTMENT OF VETERANS AFFAIRS

Digital Health Office (DHO) Office of Health Informatics (OHI) Office of Clinical Informatics (OCI) Human Systems Integration (HSI)

Human Systems Services (HSS) Electronic Health Record Modernization (EHRM)

Update: September 15, 2025

Table of Contents

BACKGROUND

The Office of Health Informatics (OHI) is integral to Veterans Health Administration (VHA) programs that provide patient-centered health care to Veterans, offering advanced and secure enterprise data systems, sophisticated analytic and measurement solutions, decision support, business intelligence and web communications. OHI facilitates evidence-based decisions for individual Veterans and their families, patient populations, clinicians, and those managing health care delivery systems. The Human Systems Integration (HSI) directorate exists within the Department of Veterans Affairs (VA), VHA, OHI, Office of Clinical Informatics (OCI). The Human Systems Integration (HSI) Division seeks to improve usability, safety, and supportability of VA systems through tailored analysis and timely insight into design tradeoffs related to human performance, staffing, training and systems safety, with the goal of maximizing system performance and minimizing lifecycle costs. HSI applies agile best practices to provide value incrementally to sponsors and business owners and works collaboratively within multi-disciplinary teams.

HSI employs a holistic systems view for understanding the human component of healthcare systems, and offers a framework for understanding how the configuration of human healthcare elements shape healthcare delivery. The division helps make design tradeoffs related to humans explicit, so decisions can be made intentionally.

HSI encourages and offers human-centered design (HCD) services (including task analysis, evaluation of context of use, workflow assessments, training needs analysis, training effectiveness, and user interface design and usability assessments) for a variety of VA-developed and acquired products and services. HSI requires multi-disciplinary engineering, management, and technical support for numerous projects to carry out their mission.

As a key domain of HSI, the HSI division accepts "human factors" as the scientific discipline concerned with understanding the interactions among humans and other elements of a system (Work system design for patient safety: the Systems Engineering Initiative for Patient Safety (SEIPS) model), and human factors engineering is the profession that applies theory, principles, data and methods to design in order to optimize human well-being and overall system performance (International Ergonomics Association).

VHA requires a rigorous human centered, total systems approach in operational settings to ensure that HIT (acquired or developed) and corresponding elements of the health care socio-technical system (STS) are jointly optimized for performance and safer operations while providing a satisfying user experience. The HSI office optimizes both employee (e.g., clinicians, administrators) and Veteran/Caregiver user experience with VHA systems, processes, and products, including HIT, medical devices and consumer products (e.g., tablets, smartphones, wearables) via an iterative HCD approach offered in the International Organization for Standardization (ISO) 9241: plan the process, understand context of use, specify requirements, design, and test user interfaces to ensure they improve user effectiveness, minimize mental workload, support decision making, maximize clinical workflow efficiency, reduce system error and latent design issues, improve patient safety, and enhance quality of care. HSI collaborates with other administration and departmental resources to meet this enterprise objective of adoption of HCD, such as ERMIO and the Office of Healthcare Transformation (OHT), Office of Healthcare Innovation and Learning (OHIL), the Veterans Experience Office (VEO), the Office of Research and Development (ORD), and Office of Quality and Patient Safety (QPS)- which includes the National Center for Patient Safety (NCPS).

While all work undertaken applies human factors methods and techniques, projects vary significantly in terms of the role of end users, the number and type of HIT systems included, and the level of effort required to complete. For example, projects range from literature reviews performed by human factors experts only, to a training needs analysis to understand and document specific learning objectives by user role that may require observing and interviewing dozens of clinical staff members.

ReFERENCE DOCUMENTS

The Contractor shall follow guidance in the following documents, in addition to the documents in PWS 9.0, in the performance of this effort:

A. ISO Standard: Ergonomics of human-system interaction -- Usability: Definitions and concepts (ISO 9241-11:2018) B. ISO Standard: Human-centered design processes for interactive systems (ISO 9241-210:2009) C. ISO Standard: Ergonomics of human-system interaction -- Usability methods supporting human-centered design (ISO/TR 16982:2002) D. ISO Standard: Ergonomics — General approach, principles and concepts (ISO 26800:2011) E. ISO Standard: Ergonomics of human-system interaction—Principles for the presentation of information (ISO 9241-112:2017) F. ISO Standard: Ergonomics of human-system interaction—Dialogue principles (ISO 9241-110:2008) G. ISO Standard: Ergonomics of human-system interaction— Guidance on visual user-interface elements (ISO 9241-161:2016) H. ISO Standard: Ergonomics of human-system interaction— Ergonomic principles for the design and evaluation of environments of interactive systems (ISO 9241-500:2018) I. ISO Standard: Ergonomics principles in the design of work systems (ISO 6385:2016) J. ISO Standard: Ergonomics of human-system interaction— Forms (ISO 9241-143:2012) K. ISO Standard: Human resource management – Learning and development (ISO 30422:2022) L. International Electrotechnical Commission (IEC) Medical device software — Software life cycle processes (IEC 62304:2006) M. ISO/IEC Standard: Systems and software engineering -- Software product Quality Requirements and Evaluation (SQuaRE) -- Common Industry Format (CIF) for usability: User needs report (ISO/IEC 25064:2013) N. ISO/IEC Standard: Systems and software engineering -- Systems and SQuaRE -- CIF for usability: Context of use description (ISO/IEC 25063:2014) National Institute of Standards and Technology (NIST) Interagency/Internal Report (NISTIR) 7742: Customized Common Industry Format Template for Electronic Health Record Usability Testing Office of the National Coordinator Health Information Technology (ONC) 2015 Edition of Safety-enhanced Design (URL)

SCOPE OF WORK

The purpose of this requirement is to enable HSI to expand services and execute key program areas, while also increasing consistency within performance and customer service. Success would include clear guidance and resources for HSI staff as well as adopted enterprise standards and policies to improve systems development and maximize performance of the HIT within the enterprise, while improving user experience. The Contractor shall provide overall program management services with specified deliverables and shall provide specified services performed by qualified resources to support key program Areas.

Dedicated Resources are individuals that provide at least a 60% full-time employee equivalent (FTEE), where they are available for team ceremonies and collaborative sessions Monday-Friday, between 6 AM Eastern Standard Time (EST) and 6 PM EST, and other contiguous US time zones as applicable for nationally resourced teams. The contractor may be required to provide back-up coverage during periods of extended absences of assigned contractor personnel (five or more workdays) to ensure continuity of services.

1. Core Requirements

a. Service Expectations

i. The Contractor shall identify all resources capable of being tasked, in alignment with Key Personnel (marked by *) for their technical skillsets, for specific roles required in execution of the requirements in their Contractor Project Management Plan (CPMP). The Contractor shall provide the Key Personnel as Dedicated Resources that provide Full-time support during regular business hours from any contiguous time zone within the US, for continuity purposes.

ii. The Contractor shall support the HSI office in the provision of Human Factors Engineering and Human Systems Engineering Services to the numerous customers and sponsors engaged with OHI that aim to optimize overall system performance. The Contractor shall provide knowledgeable resources that shall have appropriate training, education, expertise and credentials to conduct the studies in accordance with HSI and VA policies and procedures. This includes supporting the usability, macro-ergonomics, human factors, instructional systems design, systems safety, applied analysis, learning engineering cognitive engineering and interaction design functions in collaboration with HSI to perform studies for assessments, evaluations, and analyses to provide recommendations for interactive systems to improve their usefulness and usability.

iii. The Contractor shall support the analysis, design, development, testing, and implementation of various clinician-facing and Veteran/Caregiver-facing health IT applications and associated training materials.

iv. The Contractor shall provide support services across the project lifecycle utilizing the HCD approach, and at times provide services that spread across numerous activities concurrently.

v. The Contractor shall develop, submit, and execute scientifically valid, repeatable studies that include objectives, background, methodology, data analysis and results discussion that is consistent with the nature of the study. The Contractor shall apply human factors methods and analysis, mastering the fundamental concepts of observational, experimental, and correlational research methods that include both qualitative and quantitative data.

vi. Below are representative methods that the Contractor shall conduct in accordance with published human factors engineering practices:

1. Literature reviews

2. Clinical observation studies

3. Time and motion studies

4. Contextual inquiries

5. Structured, semi-structured, and unstructured interviews

6. Questionnaires

7. Focus groups

8. Diary studies

9. GOMS (Goals, Operators, Methods, and Selection) modeling

10. Job and work analysis

11. Hierarchical task analysis

12. Cognitive task analyses

13. Affinity diagramming

14. Qualitative and quantitative data collection and analysis

15. Design thinking

16. Brainstorm sessions

17. Patient experience mapping

18. Story mapping

19. Conceptual modeling

20. Workflow reengineering / cognitive system reengineering

21. Storyboarding

22. Participatory design

23. Parallel (A/B alternative) design

24. User interface design (wireframes, functional prototypes)

25. Information architecture (Card sorting, information design)

26. HIS configuration

27. Survey development

28. Rapid ethnography

29. Thematic Analysis

30. User Performance Monitoring and Data Analytics

31. Training needs analysis, training effectiveness evaluation

vii. The Contractor shall be able to recommend and execute usability studies, to include:

1. Usability inspections/ design audit

2. Usability walkthroughs

3. Cognitive walkthroughs

4. Formative usability testing

5. Keystroke level modeling

6. Summative usability testing

7. In-situ usability testing

8. Usability observations

9. Usability and design inspections

viii. The contractor shall have experience conducting training needs analyses and training effectiveness studies; the contractor shall have experience producing and delivering content that prepares clinical staff to carry out HCD methodologies.

ix. The Contractor shall have expertise and skill with data analyses, to include parametric and nonparametric statistical analysis, meta analyses, and qualitative data coding.

x. The Contractor shall have experience in administering the NASA TLX, SUS, PSSUQ, Health-ITUEs and other standard usability and satisfaction instruments. They shall also advise on the best tool(s) to meet Sponsors’ desires for data.

xi. The Contractor shall execute all activities in accordance with approved HSI office methods, processes, and report formats as documented in the HSI Program SharePoint and following guidance in the following documents, in the performance of this effort: See Applicable Resources.

xii. The contractor shall have experience in capturing workflows to be used for various purposes including re-engineering and optimization.

xiii. The contractor shall have knowledge in the application of physical and cognitive ergonomic principals.

b. Roles

i. The Contractor shall collaborate with other Resources on each assigned study activity, which may include Government staff, other contractors, and Intergovernmental Personal Assignment (IPA) staff.

ii. The Contractor shall follow, support, and endorse the application of HCD practices within HSI and VA.

iii. The Contractor shall ensure that personnel are reliable/have experience in a domain or work with the government to change personnel if the methods change.

c. Work Execution

i. The Contractor shall perform study kick-off meetings to document the context of need for a study request for any assigned study activity and load the materials as a Working Document on the Effort Tracker.

ii. The Contractor shall propose use of an industry standard, or minimally, a recommended approach supported by published evidence.

iii. The Contractor shall acquire agreement from the Government HSI Lead and Sponsor (Business Owner Point of Contact) on format prior to execution and identify and communicate the key characteristics required to meet each stakeholder’s objectives for the study.

iv. The Contractor shall identify all key stakeholders for executed studies.

v. The Contractor shall deliver the Government HSI Lead approved proposal and study plan to the Sponsor upon approval.

vi. The Contractor shall perform work as part of a Study and deliver study packages- this includes a study proposal, study plan, and study report. Study Reports and Study Plans are publicly accessible on the VA Network, while Working Products (e.g., drafts, After Action Reviews [AARs], study kick-offs, study proposals) are only accessible to HSI Resources on the Effort Tracker.

vii. The Contractor shall benchmark user performance measures on the as-is system and iterate the design and configuration of any new UI and work system as appropriate.

viii. The Contractor shall lead, manage, monitor, control and execute all aspects of a study; define test goals, recruit, and screen participants, establish test plans and scripts, conduct assessments, perform data analysis, create reports, communicate findings, and make design and functionality recommendations to improve performance and system reliability.

ix. The Contractor shall identify, outreach and consult with users, business leads, user experience, user interface and agile team members to lead, plan and execute a variety of studies and assessments applying the methods listed in Section 3.0.1.a.v.

x. The Contractor shall identify and define the gaps, risks, limitations, capabilities, system constraints, functionalities, affordances, findings, lessons, and factors contributing to the efficiency, effectiveness, comfort, motivation, safety and satisfaction of users.

xi. The Contractor shall apply a multi-disciplinary approach to human factors testing, with knowledge from HF, HCI, HSI, usability, ethnography, and cognitive, social and organizational psychology to produce assessments and reports, and other elements as appropriate to specified study objectives.

xii. Most studies shall be conducted virtually using the VA network and internet resources, although some studies shall be conducted at VA Medical Centers (VAMCs) or Community-Based Outpatient Clinics (CBOCs) in-situ, simulated in a lab, or in other face-to-face settings.

xiii. The Contractor shall provide expertise for evaluating user experience with HIS and to assess and validate the usability of a VA application or system in the context of its intended use in a production environment in accordance with usability testing standards in the International Organization for Standardization (ISO) standard 9241 part 11 (ISO 9241-11) and HF practices and procedures.

xiv. The Contractor shall coordinate the appropriate access requests and complete any necessary training to effectively conduct testing on the targeted systems for any study.

xv. The Contractor shall plan, prepare, conduct, analyze, report, and debrief the results of assessments and HF evaluations.

xvi. The Contractor shall plan for and provide the appropriate study logistics and coordinate the necessary data collection requirements in advance, such as recording voice, video, screen capture, keystroke monitoring, facial expressions, or eye-tracking.

xvii. The Contractor shall follow VA and HSI practices and appropriate regulatory policies, such as Paperwork Reduction Act (PRA), regarding usability testing, interviews, questionnaires, and consent for use of picture and/or voice.

xviii. The Contractor shall coordinate studies with appropriate facility approvals, including any potential union approvals that require advanced notice and responding to revisions.

xix. For both formative and summative usability tests, the Contractor shall provide a Study Report containing Findings which shall be written using the HSI report template which adheres to NIST Interagency/Internal Report (NISTIR) 7742 Common Industry Format (CIF), an established standard format for reporting the results of summative usability tests.

xx. The Contractor shall include the test script, details regarding the number and types of participants, task success criteria, and other project-specific parameters (e.g., test objectives, test procedures, and scenarios) as part of the study plan; the raw data and preliminary findings (e.g., time on task measurements, mouse clicks, keystrokes) and findings such as themes produced from an initial analysis of the data, with a preliminary review of the data as part of the study report; and any updates based on the disposition with the customer for the final study report.

xxi. The Contractor shall also perform an After-Action Review (AAR) and submit entries to the Lessons Learned Journal (LLJ) per HSI standard operating procedure.

xxii. The Contractor shall attend meetings as members of project or study teams and coordinate the discovery (scope definition), preparation (kick-off), execution (status), disposition, and closeout briefings for HSI studies during the project lifecycle.

xxiii. The Contractor shall facilitate sprint retrospectives and/or AARs per HSI guidelines and also if requested by HSI or a customer.

xxiv. The Contractor shall work with each customer on preferred meeting and communication cadence.

xxv. The Contractor shall recommend standards to which projects should adhere, as part of the baseline metric discussions.

d. Work Products

i. Regardless of format, the Contractor shall provide deliverables that are 508 Accessibility compliant and text editable by the Government, including reusable components for future work.

ii. All materials, including written and visual products produced by the Contractor shall be owned by the Government and be submitted using approved templates or branding.

iii. The Contractor shall create study packages thatincludes a proposal (concurred on by the HSI Government Lead and sponsor before study work can begin), a study plan, and the study report for each study-related task performed, each of which the Contractor shall upload to the HSI Effort Tracker SharePoint (VA Network URL provided upon award, referred to as Effort Tracker throughout) as created.

iv. The Study Proposal shall minimally include:

1. Introduction

2. Provide an overview of the study being proposed. Shall include the intent of the study.

3. Objectives

4. Describe the purpose of the study.

5. Mention what is not in scope or evaluated

6. Study Design

7. Include the study method used.

8. If possible, provide a link to the method details.

9. Provide a brief description of the test data being used, test measures, and study participants.

10. Describe the high-level expected results of the study.

11. Schedule (High Level)

12. Create the high-level schedule and include the logical phases.

13. Describe the duration of the project and any schedule constraints (i.e. holidays or other events)

14. Assumptions/Dependencies

15. Describe any organizational or study dependencies.

16. Consider any developer or contractor requirements for supporting study efforts.

17. Identify what assumptions are being made about the study and supporting elements.

18. Concurrence

19. Collected via email, with signatures and date.

20. Concurrence indicates that both HSI and the sponsor agree on the objectives of the study before work begins to ensure that the results of the study address program office needs, including timeline.

21. As the Contractor shall collaborate with other HSI Resources, some studies may be executed where the Study Proposal was crafted by another HSI Resource, but all related study materials shall be loaded to the same Effort Tracker entry.

v. The Study Plan shall minimally include:

1. Purpose of Study

2. Study Objectives

3. Applicable Methods, tools, and resources

4. Session script(s) or questions for users (questionnaire)

5. Target number and types of participants (for various user roles)

6. Data collection and analysis approach

7. Task success criteria if performing tasks

8. Project Timeline and Milestones

9. Sponsor connection points to review progress, risks, and responsibilities

10. Other project-specific parameters, such as:

11. Test procedures

12. Scenarios

13. Personas

14. The Study Report format (visual or written) shall vary depending on the study type (such as wireframes for design related studies) and customer preferences and can range from an Adobe InDesign, Adobe PDF, Excel workbook, a PowerPoint presentation, a Word document, diagrams, web content or other visual format.

vi. The HSI minimum expectation for written reports shall follow this structure, adapted from the Human Factors and Ergonomics Society (HFES) conference proposal requirements (http://cms.HSIs.org/Events/Annual-Meeting/Call-for-Proposals.aspx#Forms), and are expected to be associated with a Word document:

1. Introduction: General statement to orient the reader to the specific problem and context, study personnel, background information, and relevant literature.

2. Method: How the study was conducted, including participants, apparatus, and procedure (e.g., any tasks completed by participants).

3. Results: Summarize the data collected, list any findings1, and/or include visual products2 such as mock-ups

4. Discussion: Describe your analysis of the results, the implications, pertinent evidence-based recommendations, and study limitations and next steps.

5. References: Provide full references for all citations used in the proposal in current edition American Psychological Association (APA) style.

vii. The Contractor shall produce other products, to include briefings, note summaries, Visio models, Excel workbooks, findings/process summary reports, patient journey maps, health service blueprints, personas, value stream maps, and user stories.

viii. The Contractor shall create design patterns, guidelines, and similar resources to be used to promote evidence-based products, quality, and consistency.

ix. For support roles filled by Dedicated Resources, the Contractor shall submit meeting summary notes in tables, with actionable outcomes, key decisions, assumptions, upcoming milestones and deliverables, and any other takeaways and actions that are critical to communicate to HSI government staff.

x. The Contractor shall provide analysis of meeting tone and style (e.g., frustration, confusion, dominated by one organization) along with notes when meeting attendance and report back is tasked.

xi. The Contractor shall provide evidence that their solutions meet approved acceptance criteria and/or the “definition of done” or explain deviance.

e. Tracking and Coordination

i. The Contractor shall update and maintain the VA Program Manager (PM) approved CPMP throughout the life of the award.

ii. The Contractor shall identify the primary contact point for all programmatic issues/concerns/status.

iii. The Contractor shall complete a Problem-Issue + Resolution (PI+R) document or a decisions, expectations (assumptions), risks, actions, and issues log (DERAIL) document to track decisions for each effort. See Applicable Resources. The DERAIL or Problem-Issue tracker and Communication Schedule shall be maintained on the Effort Tracker by the Contractor as a Working Product.

iv. The Contractor shall document and communicate weekly the PI+R or DERAIL aligned with each activity during study planning, execution, and closing phases.

v. The Contractor shall establish and maintain a recurring Communication Schedule with stakeholders aligned with each assigned activity’s objectives.

vi. The Contractor shall document all initiated work, provide updates, and load the final Study materials in the HSI Effort Tracker SharePoint and the Project Management (PM) Tracker SharePoint.

vii. The Contractor shall be responsible for ensuring all materials are loaded to the Effort Tracker and the LLJ, and shall notify the Government HSI Lead of known gaps of content, sourced or attributed as work products from other entities.

viii. Microsoft Teams is VA’s official collaboration tool and is used extensively for VA-initiated meetings and is available on the VA Network and shall be the primary teleconferencing tool used during performance of studies.

ix. The Contractor shall perform and document White Glove (WG) reviews and formal Peer Reviews (PR) on assigned work activities and submit as a Working Product to the relevant Effort Tracker entry.

x. The Contractor shall provide project coordination services across multiple, simultaneous activities of varying complexity.

xi. The Contractor shall perform VHA employee (such as clinicians, not Veterans) participant recruitment and scheduling coordination, technology set-up for virtual studies, logistics planning for in-person or remote sessions, data recording, coding and analysis, report writing, technical editing/proofreading for formal PR and WG, meeting scheduling and agenda preparation, study recommendations disposition tracking, engagement summaries preparation, publishing support, and graphical design.

xii. The Contractor shall take notes that will include key information discussed, decisions and their justification, outstanding questions, and action items. If all parties agree, the call may be recorded, but a call transcript is not a suitable substitution for this notes document, which shall include appropriate grammatical standards for secondary use.

xiii. On calls they lead or represent HSI on, the Contractor shall track action items, discussion points, and decisions and review them prior to the call ending for concurrence.

xiv. The Contractor shall create agendas and facilitate calls for work they lead.

xv. The Contractor shall monitor their action items for completion and report status to the government.

xvi. When reviewing deliverables, the Contractor shall make notes of government requested changes, note how each was met (e.g., in a table or in MS Word comments) and submit that prior to next review, to include rationale for changes.

xvii. The Contractor shall use data (e.g., Jira reports or adherence to an agreed-on schedule) to make recommendations for resource changes to achieve project or sprint objectives.

xviii. The Contractor shall obtain government approval before proposing any specific work to customers.

f. Operational Support

i. The Contractor shall offer guidance and materials to HSI on how to align their processes with industry best practices.

ii. The Contractor shall offer guidance and materials to HSI on promoting the science of HF within VA.

iii. The Contractor shall support the continued creation, publication, and curation of HSI and usability.

iv. The Contractor shall support the management and enhancement of knowledge materials on sharing portals, such as SharePoint.

Capability Areas

a. General Requirements

Contractor personnel shall have the level of experience necessary to accomplish the requirements of this PWS. Requirements of this PWS require highly complex healthcare experience. Highly complex healthcare requirements may include the involvement of multiple Functional Categories and several program offices and/or departments e.g., DOD and VHA. The scope of these requirements may include the integration and management of various interrelated workstreams for a strategic, resource intensive, time constrained projects.

In addition, contractor personnel shall be acceptable to the Government in terms of personal and professional conduct, and in technical knowledge. Furthermore, contractor personnel are expected to be proficient in using office automation equipment and software, which includes a requirement to leverage specific healthcare and/or Government technologies and have sufficient written and verbal communication skills to support VHA program offices, their customers and any other VHA organizations. Should any contractor personnel be determined to be unacceptable in terms of technical competency or unacceptable conduct or behavior while on-site or while working on contract activities, the contractor shall immediately remove and replace the unacceptable on-site personnel at no additional cost to the Government. Contractor personnel are to serve in a support role; therefore, final decisions regarding inherently governmental functions will always be made by Government personnel.

b. Service Requirements Services performed shall be of a non-personal nature. VHA organizations will not provide supervision of contractor personnel. Contractor personnel shall at no time allow an employer-employee relationship to develop with VHA organizations or their staff. VHA organizations will refrain from any activities that create the appearance of such a relationship.

Contractors shall not perform inherently governmental functions including decision-making, supervision of Government employees, supervision of other contractors on other contracts, and activities that create the appearance of performing such functions.

c. Capability Areas The Contractor shall provide services, labor categories, and solutions necessary to fulfill the scope of all the below Capability Areas.

Work may require expert level resources, specialized labor categories, rapid response, an extensive resource pool, enterprise-wide engagement and support, implementation and sustainment of complex initiatives and high-level integration.

Capability Areas:

1) Program & Project Management- The contractor will provide support to assist the Government in implementing disciplined, comprehensive, and flexible program and project management processes, including monitoring of project metrics, rigorous risk management, and prompt reporting on Government-approved cost, schedule, performance, and risk baseline. When needed the approach must incorporate large integrated (to include governmental) healthcare project management best practices. Program and project management must integrate all functions required for a success.

2) Program Evaluation- The contractor will employ a proven systematic method for collecting, analyzing, and leveraging information to evaluate healthcare projects, policies and programs for effectiveness and efficiency, including risk assessments and mitigation strategies.

3) Performance Measurement- The contractor shall provide support of strategic performance measurement development, including consideration of risks, based upon healthcare industry standards, including improved linkage between VA-wide strategic healthcare goals and programmatic outcomes; and implementation of the Government Performance and Results Act (GPRA) Modernization Act (GPRAMA).

4) Informatics Risk Analysis & Mitigation- The contractor shall conduct risk analysis applying a risk stratification framework using risk management standards and methods to provide assessments to document potential issues and risks associated with existing or emerging technologies, tools, or methods. Services shall include providing recommendations to maximize effectiveness and reduce risks and associated costs. The contractor shall propose strategies to reduce the likelihood or severity of harm through error-tolerant design and mitigation implementation on risks or perceived issues. The contractor shall identify potential issues and risks with decisions made, with a particular focus on any usability, information security and safety impacts. The contractor shall identify and summarize workarounds established by users that mitigate potential issues, including potential policy adaptations.

5) Solution Engineering & Issue Resolution- The contractor shall lead business process reviews and recommend enhancements to elements of the socio-technical system (STS). The contractor shall assess deployment of health information systems (HIS) and post-implementation status to determine the impact on ongoing operations, including progress assessments related to defined key performance indicators (KPI). The contractor shall perform analysis efforts that inform candidate solution development resulting in recommendations for changes to the system design or functionality or other optimal mitigation strategies to an identified issue to enhance policy, procedures, organizational / leadership, education / training, manpower / personnel / staffing, or some combination of those listed. The contractor shall perform workflow evaluations that apply best practices or standardized procedures to improve service delivery with measurable health outcomes, or craft new methods based on literature. The contractor may support activities that explore the variance and design constraints of workflow adoption issues. The contractor shall validate workflows, suggest enhancements, enable performance optimization, and identify latent risks and design inefficiencies, challenges, and opportunities for system enhancement. The contractor shall identify HIS deployment issues that can be explored and quantified so alternative STS design options can be considered, including design recommendations or proposed process redesigns. The contractor may measure performance changes and design enhancements that informs future trade off analyses. The contractor shall perform analysis that identifies sub optimal workflows and assess user reported interface and functionality issues as appropriate. The contractor shall perform and have working knowledge and specialized experience in analytics related to processes (e.g., operations research, clinical pathways, Business Process Modeling and Notation (BPMN+), causal inference, machine learning and artificial intelligence, and master data management in context of healthcare data and electronic health records (EHR) data. The contractor shall perform operations research, decision support, and analysis related to diagnostic reasoning, clinical practice understanding, and guideline application.

6) Simulation- The contractor shall perform simulation studies that delineate the current performance from the potential system performance with identified variable factors for short-term solutions and longer-term solutions to inform operational and strategic planning. The contractor shall identify the system components, flow, and interactions as appropriate for the model’s aims to predict and subsequently validate the model's performance. The contractor shall provide simulation to support predictive data and information on operational performance issues and define functions and tasks to enable assessing workload, evaluating alternative configurations, assessing operating procedures, workflow integration, and identifying accident or error-provocative situations.

7) Systems Testing- The contractor shall perform pre- and post-implementation assessments in-situ and virtually that analyze Key Performance Indicators (KPI) and identify established workarounds or pain points from user-discovered usability issues. The contractor shall establish appropriate usability objectives and KPIs for targeted user roles in specified domains for common, core, and safety critical tasks. The contractor shall provide performance-based assessments that provide clarity on the operational impact of a system design and qualitative testing that identifies underlying usability concerns and workflow integration challenges. The contractor shall establish performance baseline usability metrics that provide a reference point to determine quantitative impact of applied interventions.

8) Human Centered Design & Human Systems Services- The contractor shall support a systems-based approach that aims to improve usability, information security, safety, and overall user experience with VA services and Human Systems Integration (HSI) in alignment with existing federal government policies, regulations, and international standards, while developing new techniques and approaches for VA to efficiently adopt and demonstrate the value of involving users throughout the systems development lifecycle. The contractor shall support the design of products, processes, and tools to optimize overall system performance, by employing methods with demonstrable success in reducing development costs, improving overall system efficiency, heightened user satisfaction, increased safety, and improved employee experience. The contractor shall apply human factors methods and analysis, mastering the fundamental concepts of observational, experimental, and correlational research methods that include both qualitative and quantitative data. The contractor shall have expertise and skill applying statistical analysis, interviews, contextual inquires, focus groups, inspections, evaluations, survey development, rapid ethnography, systems design, data coding and literature reviews. The contractor shall have experience in administering the NASA Task Load Index (TLX), System Usability Scale (SUS), Post-Study Usability Questionnaire (PSSUQ), Health Information Technology Usability Evaluation Scale (ITUEs) and other standard usability and satisfaction instruments. The contractor shall benchmark user performance measures on the as-is system and iterate the design and configuration of any new UI and work system as appropriate. The contractor shall identify and define the gaps, risks, limitations, capabilities, system constraints, functionalities, affordances, findings, lessons, and factors contributing to the productivity, comfort, motivation, safety, and satisfaction of users.

9) Agile Development Methodology- The contractor shall provide consultation on the effective use of agile methodologies including but not limited to the use of Scrum, Kanban, and Scaled Agile Framework for solution development. The contract shall enhance team agility and maturity by developing coaching plans and conducting maturity assessments. The Contractor shall facilitate the organization’s transition to a continuous delivery model and adoption of an agile mindset across stakeholders by using change management techniques to generate organizational buy-in. The Contractor shall participate in development and application of guiding principles and standards to guide product management practices. The Contractor shall develop ecosystem and business capability models, value stream and customer journey maps to guide product teams in developing their minimum marketable and minimum viable products, value propositions and drive business outcomes.

PERFORMANCE DETAILS

PERFORMANCE PERIOD

The Period of Performance (PoP): A 10-month base period, followed by two optional periods (30 days each). An additional optional task for a Phase-Out-Plan with 150-day period may be executed by the Government during the base period.

Any work that takes place at a Government site shall not take place on Federal holidays or weekends unless directed by the Contracting Officer (CO).

Federal holidays set by law (USC Title 5 Section 6103) that VA follows are outlined here: https://www.opm.gov/policy-data-oversight/pay-leave/federal-holidays/#url=Overview.

PLACE OF PERFORMANCE

Efforts under this contract shall be performed at Contractor facilities. The Contractor shall identify the Contractor’s place of performance in their Task Execution Plan (TEP) submission.

TRAVEL OR SPECIAL REQUIREMENTS

The Government anticipates travel under this effort to perform the tasks associated with the effort, as well as to attend program-related meetings or conferences through the life of the award. Travel shall be Cost Reimbursable with an NTE ceiling on travel expenses. Travel expenses shall be invoiced and paid against the travel CLIN, which will have an NTE price specified.

Travel shall be in accordance with the Federal Travel Regulations (FTR) and requires advance concurrence by the Contracting Officer’s Representative (COR). Each Contractor invoice must include copies of all receipts that support the travel costs claimed in the invoice. Contractor travel within the local commuting area will not be reimbursed. General and Administrative expenses will not be reimbursed. These costs will be considered to be reasonable and allowable only to the extent that they do not exceed the maximum per diem rates in effect at the time of travel as set forth in the FTR.

The FTR can be referenced at:

https://www.gsa.gov/policy-regulations/regulations/federal-travel-regulation

Travel may occur but is not limited to the below locations for performance of activities detailed in the PWS.

SPECIFIC TASKS AND DELIVERABLES

PROJECT MANAGEMENT

The Contractor shall summarize all their planned sprint activities, including planned study packages and completed study packages for the previous month, for each Area below for each team, every month in a Monthly Execution Plan (MEP). The Contractor shall provide invoices covering each MEP timeframe that will align with Quality Assurance Surveillance Plan (QASP) measures and include % Commitments Completed and Planned Effort / Performed Effort Ratio. The Contractor shall deliver the CPMP, and update the document as needed as an appendix to the MEP. The Contractor shall deliver a Summary Package that includes the final update for performance to the last MEP and the transition materials for each task.

The Contractor shall provide reports or meetings outlined below for each Area as needed by the VA Government Program Manager (GPM). The Contractor shall provide the Decisions, Expectations/Assumptions, Risks, Actions, Issues Log (DERAIL) or a Problem-Issue Resolution document on the VA Effort Tracker as a working document that is kept up to date for all efforts. The Contractor shall upload deliverables and related components to the HSI Effort Tracker SharePoint as created. The Effort Tracker is the single authoritative project and study deliverables and working documents repository. The Project Management (PM) Tracker is the task listing where all existing commitments and planned efforts are tracked with assigned resources. The Contractor shall provide weekly status updates on the PM Tracker. The Contractor shall support effective communication of commitments through shared project workspaces when collaborating with other interdisciplinary teams, such as Jira. The Contractor shall support material sharing on the Teams channel for collaboration. The Contractor shall apply agile principles to the work effort and be familiar with Scaled Agile Framework techniques. The Contractor shall support product increment planning efforts as appropriate to the sprint, epic, and feature objectives.

The Contractor shall prepare and deliver a CPMP that lays out the Contractor’s approach, timeline and tools to be used in execution of each task. The CPMP shall also include how the Contractor shall coordinate and execute planned, routine, and ad hoc data collection reporting requests as identified within each task.

The Contractor must have VA network access to complete this task (Effort and PM Trackers). The Contractor shall expedite onboarding in accordance with VHA practices. The Contractor shall expeditiously undergo sufficient background checks and complete required training and forms in order to be granted access to VA’s network and key resources used and managed by the HSI team within three (3) business days of receiving the notice of award. The Contractor shall communicate on an on-going manner the current status of contracting on-boarding including; last actions taken, next steps, and anticipated completion dates.

The initial baseline CPMP shall be concurred upon and shall include a labor category staff roster identifying the following:

1. Employee Name

2. Personnel PIV Expiration

3. Personnel clearance level and adjudication date

4. Role

5. Project affiliation and assigned activities log

6. Personnel duty location address, alternate email, phone number

The Contractor shall support sprint (2-week work cycle) planning that detail commitments for the coming 2 weeks, across all the Areas, and detail them to the GPM on a Workload Assessment Meeting (WAM). The Contractor shall review the backlog by Area to inform commitments. The Contractor shall summarize engagements / activities and delivered study packages via the MEP as well as specify key performance indicators (KPIs) and intended outcomes for each project. At the end of each sprint, the Contractor shall document the percentage done.

The Contractor shall be paid for the percentage of work that has been accepted by the Government from each MEP period (i.e., monthly).

Work submitted shall follow VA, VHA, DHO, OHI, OCI, and HSI’s processes and procedures, such as AAR and LLJ.

The Contractor shall continuously monitor performance and report any deviation to the COR and VA GPM. The Contractor shall lead a WAM with the VA GPM every two weeks for project summary and status discussions in preparation for the MEP submission.

The WAM shall include the following information:

1. Overall high-level assessment of progress

2. Works in progress (WIP) with planned deliverables and working documents

3. Activities recently completed since the last meeting

4. Planned projects to begin in the next two (2) weeks, when applicable

5. Barriers to progress or inefficiencies experienced

6. New assignments and opportunities

The MEP shall incorporate the prior WAM details for the prior sprint’s efforts, including corresponding changes made during the current reporting period, such as “Original target date for study report MM/DD/YYYY revised to MM/DD/YYYY on MM/DD/YYYY because of … concurred upon by HSI Government Lead”. Government feedback on prior reports shall be addressed in follow-on deliveries, with rationale provided and details about the corrective actions taken.

The Contractor shall hold a technical kickoff meeting within 10 business days after award to summarize the CPMP and execution strategy. The Contractor shall present, for review and approval by the Government, the details of the intended approach, work plan, and project schedule for each task.

The Contractor shall specify kickoff dates, location (often virtual), agenda (shall be provided to all attendees at least five (5) calendar days prior to the meeting), and meeting minutes including all presented materials (shall be provided to all attendees within three (3) business days after the meeting).

The Contractor shall invite the Contracting Officer (CO), Contract Specialist (CS), COR, the VA GPM, and the HSI Government Lead.

The Contractor shall acquire tooling necessary to complete the taskings below, making full use of the VHA software center tooling already available, and only with software that is…