S02 Notice of Intent to Sole Source_CVP.pdf

B.2 PERFORMANCE WORK STATEMENT

1.0 BACKGROUND

As an innovation engine within the Department of Veterans Affairs (VA), the vision of the Veterans Health Administration’s Innovation Ecosystem (VHAIE) is continuously keeping innovation at the forefront when it comes to science and research, service delivery and implementation of solutions, as well as employee empowerment. VHAIE leads this vision by fostering organizational capability, delivering operational and clinical breakthroughs, and driving futures. VHAIE is committed to developing and employing agile mechanisms that allow VA to source incremental and transformational innovations to best serve Veterans and their families.

The Broad Agency Announcement (BAA) for suicide prevention is structured to allow for a competition cycle, which consists of submission of concept paper followed by an invitation only proposal. The reviews of these proposals are completed by subject matter experts within Office of Suicide Prevention (OSP), Office of Mental Health, and Office of Healthcare Innovation and Learning (OHIL) for scoping and impact.

This BAA opportunity seeks to source and fund early-stage research, development, prototyping, field testing, and implementation piloting with an overall goal of moving forward the state of the art.

Through this BAA, VHAIE invites all potential offerors (including private sector companies, non-profits, and institutions of higher learning) to contribute ideas for innovations in suicide prevention, care coordination, and treatment that significantly increase Veteran access to services, reduce or control costs of delivering those services, enhance the performance of VA operations, and improve the quality of service that Veterans and their families receive.

The significant and unprecedented challenges Veterans face, fuel the continued call to action related to suicide prevention. Suicide is a complex problem which requires a full public health approach, involving community prevention as well as clinical intervention.

VA services are a critical part of this public health approach.

The data spanning 20 years reveals that Veterans engaged in VHA care have shown a less sharp rise in suicide rates, underscoring the importance of VHA care. Over 20 years of Veteran suicide, data also reveal a substantial reduction in suicide rates, specifically for recent Veteran VHA Users with mental health or substance use disorder diagnoses (77.8 per 100,000 in 2001 to 58.2 per 100,000 in 2021), falling 32.9% for Veterans with depression, 27.6% for those with posttraumatic stress disorder, 26.9% for those with anxiety and 40.4% for those with sedative use disorder.

Comparing Veterans with recent VHA use to other Veterans, we also find notable trends. While overall rates of Veteran suicide rose across the 20 years, age-adjusted suicide rates rose 24.5% for male Veterans with recent VHA use compared to 62.6% for male Veterans without recent VHA use. Although less notable for women Veterans, the age-adjusted suicide rates rose 87.1% for female Veterans with recent VHA use and 93.7% for female Veterans without recent VHA use. Likewise, when looking more specifically across 2020 and 2021, we find the greatest increase in unadjusted rates for

Veterans who were neither engaged with VHA nor with Veterans Benefits Administration (VBA).

From 2020 to 2021, there were also notable decreases for particular subpopulations of Veterans with recent VHA use, including those between ages 55- and 74-years-old (overall suicide rate -2.2%, -0.6% for men, -24.9% for women), males between ages 18-and 34-years-old (overall suicide rate -1.9%) and males aged 75-years-old and older (overall suicide rate -8.6%). These findings underscore the importance of continuing to expand access to, and engagement of, Veterans in VHA and VBA services, as over 50% of Veterans who died by suicide in 2021 had not been engaged in either service.

Yet, in order to address the complex interweaving of individual, relational, community and societal risks, VA must continue to fully engage with other federal agencies; public-private partnerships; government at the local, state national levels; Veteran Service Organizations (VSOs); and local communities to reach all Veterans to support the implementation of a full public health approach, as outlined in the White House Strategy Reducing Military and Veteran Suicide (2021) and VA’s National Strategy for Preventing Veteran Suicide (2018). As noted above, Veterans receiving VHA care show evidence of higher risk with being more likely to have lower annual incomes, poorer self-reported health status, more chronic medical conditions, and self-reported disability due to physical or mental health factors, greater depression and anxiety, and greater reporting of trauma, lifetime psychopathology, and current suicidality.

These guiding documents have been operationalized through Suicide Prevention (SP) 2.0; Suicide Prevention Now initiative (SP Now); new laws, including the 2020 Commander John Scott Hannon Veterans Mental Health Care Improvement Act, the Veterans Comprehensive Prevention, Access to Care and Treatment Act (COMPACT) of 2020, the National Suicide Hotline Designation Act of 2020, and emerging innovations combined with research and program evaluation.

As 2021 has again shown, this public health approach must include both community-based prevention and clinical interventions to reduce suicide in the Veteran population.

As we reflect on the core of what we learned about Veteran suicide in 2021, 7 themes emerge for our call to action (see summary listing and description below).

While no one solution can address the complexity of all factors involved in suicide, the data clearly outlines that significant reductions in Veteran suicide will not occur without meaningful focused effort to address Veteran firearm suicide. While we vigorously pursue enhanced policies, research, and programs to effectively address the broader socioecological and individual risk and protective factors, which speak to “why” a Veteran may consider suicide, we must address directly the “how” of Veteran suicide. It is inescapable that the “how” in 72% of Veteran suicide deaths is firearm compared to 52% of non-Veteran U.S. adult suicides.

VHAIE, therefore, begins our call to action with a focus on primary topic areas that take into account the many facets of suicide prevention, including topics like the “how” of suicide, the importance of a community-led approach to preventing suicide, improved training, or increased access to care. To address the need for innovation in Suicide Prevention, VA seeks innovations across 7 primary topic areas:

• Promote firearm secure storage for Veteran suicide prevention.

• Implement and sustain community collaborations focused upon community-specific Veteran suicide prevention plans.

• Continue expansion of readily accessible crisis intervention services.

• Improve tailoring of prevention and intervention services to the needs, issues, and resources unique to Veteran subpopulations.

• Advance suicide prevention meaningfully into non-clinical support and intervention services, including financial, occupational, legal, and social domains.

• Increase access to and utilization of mental health across a full continuum of care.

• Integrate suicide prevention within medical settings to reach all Veterans.

2.0 SCOPE OF WORK

This contract and proposed solution focus on the following topic:

• Integrate suicide prevention within medical settings to reach all Veterans.

Topic Detail: Data revealed that a significant percentage of VHA Veterans who died by suicide did not have a VHA mental health or substance use disorder diagnosis (see section 3.1). A creative implementation to address the needs of Veterans at risk on never seeking mental health services and having other risk factors outside of mental health (e.g., pain, cancer, sleep disturbance), is required through expansion of suicide screening, assessment, and safety planning into all medical settings, within VHA and within community care.

2.1 SPECIFIC PROBLEM TO ADDRESS WITH THIS SOLUTION

The 2023 National Veteran Suicide Prevention Annual Report reveals that 6,392 Veterans died by suicide in 2021, an increase of 114 from the previous year. Suicide is a national healthcare concern, with Veterans facing higher rates than civilians. Many non-mental health staff lack proper training to recognize suicide warning signs.

Traditional training methods, often using role-playing exercises, have proven ineffective, with participants reporting them as unrealistic and awkward. This poses a challenge in preparing healthcare providers to identify and address suicide risk effectively.

2.2 PROPOSED SOLUTION

The contractor shall design, develop, test, and validate a virtual, immersive environment that enables training related to the Suicide Prevention for VA employees. Work will include the development of a working prototype – including functionality across multiple head-mounted displays– to validate the virtual and immersive experience, done so in collaboration with VA clinical experts who will determine utility and feasibility of this solution.

The required outcomes for the proposed Immersive Technology solution are:

1. Customizable experiences considerate of factors related to social determinants of health, geographic region, and appropriate suicide prevention-related considerations

2. Hardware and software compatibility capable of screen-casting, both eye-gaze and hand controller manipulation, embedded data collection (such as, but not limited to, perceptions of the immersive experience, pre- and post- assessments, etc.)

3. Functionality for individual and/or group education and training sessions

4. Leverage Immersive Technology to advance the utilization of augmented, mixed, or virtual reality as a tool for VA employee training related to prevention of suicide.

5. Provide a more distributive and virtual experience, by which a VA employee and/or patient, in concert with clinical staff as appropriate, may utilize the technology, without having to be physically present at a VA facility.

The proposed solution is intended to empower VA practitioners to recognize suicidality signs and determine necessary actions, regardless of who the Veteran sees. Contractor shall accomplish this by creating immersive and personalized training that goes beyond traditional methods, combining virtual reality with live, one-on-one coaching with Artificial Intelligence (AI). The solution will provide this hands-on practice in speaking with and identifying suicide risk in a safe, realistic environment. The Virtual Reality (VR) for suicide prevention training project (The Project) shall use a human-centered design approach, including iterative processes, beginning with extensive user research, rapid prototyping, and then rigorous user testing. This cycle of prototyping, testing, and refining is repeated, incorporating immersive artwork to enhance authenticity and achieve maximum immersion.

To transform how healthcare providers identify and intervene in potential suicide risk scenarios, Contractor shall complete a pilot that includes the following:

1. Develop the VR training application.

2. Recruit 100 VA licensed medical providers.

3. Administer a pre-test survey on preparedness for dealing with suicidal ideation.

4. Assign 50 providers to a control group and 50 to receive our VR training.

5. After 8 weeks, conduct a post-test survey on preparedness and VR technique application and compare the results.

For this suicide prevention training program, the Contractor shall develop the front-end using the Unity Integrated Development Environment (IDE), a platform they have successfully utilized in creating other immersive applications for the VA. The Contractor shall also develop an alternative version ("build") of the software that will run on a computer. While the computer-based version would provide a 2D experience rather than the immersive 3-Dimensional (3D) environment of VR, it would still deliver the core training content and interactive elements. The Contractor shall deploy the app on Meta Quest headsets, or a similar product approved by VA.

The Contractor shall use a development process that integrates agile methodologies with Human Centered Design (HCD) principles. The Contractor shall conduct informational interviews with VA Subject Matter Experts (SMEs), collaborate closely with stakeholders, and involve VA Project Management (PM) and associated project team in frequent reviews. This approach ensures that the solution remains focused on end-user needs and aligns with VA objectives. The Contractor shall host the system’s back end in a secure Amazon Web Services (AWS) cloud environment, leveraging AI capabilities to assess learner progress in real-time. By using a Large Language Model (LLM) for real-time evaluation against an expert-designed rubric, the solution will provide personalized feedback based on VA-recommended practices. The Contractor team shall adhere to secure coding practices throughout the development lifecycle, storing code in GitHub and other assets in a dedicated AWS account. The solution will utilize an auto-update capability to ensure ease of pushing new content or iterations to devices.

The solution shall be designed with specific special Veteran populations such as geographically remote, white males over 55, women Veterans under 35, enlisted service members under 30, survivors of military sexual trauma, and/or American Indian/Alaska Native Veterans.

2.2.1 PROJECT METRICS

The Project will measure success through a combination of metrics, co-designed with and approved by VA. A two-phased approach will be used, inclusive of a prototyping phase and then a field-testing phase. The REAIM framework shall be used to evaluate impact in both phases. The below provides an example of how metrics may be tracked using the REAIM framework:

Reach • Track number of user testing and field-testing participants

• Track pertinent aggregate information about participants such as clinical discipline, site, years of experience, or prior immersive experience.

• Track number of sites participating and number of end users providing feedback at each

Effectiveness • Assess and compare baseline knowledge and post-training knowledge

• Assess for any differences between traditional training and VR-based training

• Assess realism and sense of presence

• Assess accuracy of training based on existing curriculum and trainings

Adoption • Track ease of use (e.g. System Usability Scale)

• Track incidence of side effects

• Track number of participants and number of sites

• Track number of headsets being utilized and cost associated

Implementation • Track UX improvements made based on feedback.

• Feedback on ease of use

• Evaluate needed workflows for integration of training into current practices

• Evaluate value of mobility of solution

• Test and evaluate solution compared to standard

Maintenance • Evaluate perception of operational integration of solution into existing practices and systems including starting process of solution integration into VA systems if appropriate

• Successful development of long-term scaling and sustainability plan for solution

2.2.1.1 PHASE 1: PROTOTYPING/TESTING

Phase 1 (initial 12 months) the Contractor shall focus on initial prototype development, including conducting informational interviews with VA SMEs, collaborating closely with stakeholders, and involving VA PM and associated project team in frequent reviews as prototype development progresses following an agile development process. Phase 1 shall further include extensive user testing to gather feedback and iteratively refine the solution. The Contractor shall work with VA PM and other VA SMEs to determine which pre-existing curriculum(s) shall be used to develop the solution as well as primary training focus of the prototype solution.

During this phase, Contractor shall work with VA experts to:

1. Identify key learning objectives and essential skills to be translated into the VR format

2. Develop realistic scenarios that align with these objectives

3. Structure the curriculum to best support the desired learning outcomes

4. Design interactive exercises that reinforce critical information and skills

Phase 1 shall further include development of an AI grading system to grade end users in real time and provide constructive feedback on:

• Active listening skills

• Empathetic validation of concerns

• Use of supportive statements

• Ability to restate and rephrase

• Provision of warm referrals

• Identification of need for further assistance (i.e., same-day mental health appointments, in-depth assessment, or additional screening).

2.2.1.2 PHASE 2: FIELD TEST/PILOT

2.3 INTENDED BENEFITS OF THE SOLUTION

By adapting established curricula to a VR platform, the Contractor shall create an immersive and interactive learning experience that enhances the effectiveness of suicide awareness training. This approach allows medical staff to practice critical skills in a safe, controlled environment, improving their ability to identify and respond to potential suicide risks. The VR format enables participants to engage with realistic scenarios, fostering deeper understanding and retention of key concepts while providing opportunities for immediate feedback and skill refinement. The interactive nature of VR allows trainees to develop crucial empathy and communication skills through one-on-one role-playing exercises.

This approach enhances engagement and retention, enabling staff to better understand the complexities of identifying suicide risks. This solution intends to complement existing educational tools by offering a unique, experiential learning opportunity that builds confidence and competence in this critical priority area of healthcare. By implementing VR training for management of suicide risk, the VA can equip a wider range of staff with the skills and confidence needed to intervene early. This proactive approach has the potential to significantly impact suicide prevention efforts among the Veteran population while also being adaptable to other modalities (e.g., telehealth.).

Key intended benefits include:

Benefit Description

Enhanced Realism • VR simulations can replicate nuances like body language, tone, and environments.

• VR training enables role-playing scenarios for special populations at VA (Figure 1).

Reduced Anxiety and Increased Engagement

• VR environments offer a low-stakes practice space, reducing trainee anxiety.

• Increased comfort leads to higher engagement and more diverse approach strategies.

Consistency and Scalability

• Standardized, replicable VR scenarios ensure consistent training quality.

• VR provides a scalable solution ideal for large organizations like the VA.

Immediate Feedback and Iterative Learning

• AI systems provide instant performance feedback, offering immediate correction and improvement opportunities.

• Repeatable scenarios with variations reinforce learning and adaptation.

Cost-Effectiveness • Reduces expenses for in-person trainers, travel, and physical spaces.

• VR headsets cost less than single overnight training trips.

3.0 PERFORMANCE DETAILS

3.1 PERFORMANCE PERIOD

The period of performance (PoP) shall consist of a 12-month base period followed by one 12-month option year.

Estimated PoP: Base Period: September 2, 2025 – September 1, 2026 Option Period 1: September 2, 2026 – September 1, 2027

Any work at the Government site shall not take place on Federal holidays or weekends unless directed by the Contracting Officer Representative (COR).

There are 11 Federal holidays set by law (USC Title 5 Section 6103) that VA follows:

Under current definitions, five are set by date:

New Year's Day January 1 Juneteenth June 19 Independence Day July 4 Veterans Day November 11 Christmas Day December 25

If any of the above falls on a Saturday, then Friday shall be observed as a holiday.

Similarly, if one falls on a Sunday, then Monday shall be observed as a holiday.

The other six are set by a day of the week and month:

Martin Luther King's Birthday Third Monday in January Washington's Birthday Third Monday in February Memorial Day Last Monday in May Labor Day First Monday in September Columbus Day Second Monday in October Thanksgiving Fourth Thursday in November

3.2 PLACE OF PERFORMANCE

Tasks under this PWS shall be performed in VA facilities located at:

VA Center Address

Atlanta VA Medical Center

1670 Clairmont Rd.

Decatur, GA. 30033

Western North Carolina VA Healthcare System

1100 Tunnel Rd.

Asheville, NC. 28805

Richmond VA Medical Center

1201 Broad Rock Blvd.

Richmond, VA. 23249

Work may be performed at remote locations with prior concurrence from the Contracting Officer’s Representative (COR).

Additional tasks under this PWS, other than user and field-testing site visits, shall be performed at Contractor facilities. The Contractor shall identify the Contractor’s place of performance in their proposal and as stated below:

Customer Value Partners, LLC 805 Fifteenth St, Suite 650 Washington, D.C. 20005

4.0 SPECIFIC TASKS AND DELIVERABLES

4.1 PROJECT MANAGEMENT

4.1.1 CONTRACTOR PROJECT MANAGEMENT PLAN

The Contractor shall deliver a Contractor Project Management Plan (CPMP) that lays out the Contractor’s approach, timeline and tools to be used in execution of the contract. The CPMP should take the form of both a narrative and graphic format that displays the schedule, milestones, risks and resource support. The CPMP shall be in electronic form in Microsoft Word and Excel or Project formats. The CPMP shall also include how the Contractor shall coordinate and execute planned, routine, and ad hoc data collection reporting requests as identified within the PWS. The initial baseline CPMP shall be concurred upon and updated in accordance with the task 6.0 Delivery Schedule of the contract. The Contractor shall update and maintain the VA PM approved CPMP throughout the PoP.

The plan should include detailed tasks and deliverables with specified duration, objective, approach, potential challenges and mitigation, plan for measuring success or outcomes, completion criteria, deliverables, and responsible organization.

Deliverable:

A. Contractor Project Management Plan

4.1.2 REPORTING REQUIREMENTS

The Contractor shall provide the COR with Monthly Progress Reports in electronic form in Microsoft Word and Excel or Project formats. The report shall include detailed instructions/explanations for each required data element, to ensure that data is accurate and consistent. These reports shall reflect data as of the last day of the preceding Month.

The Monthly Progress Reports shall cover all work cumulatively completed during prior reporting periods, the current reporting period, and work planned for the subsequent reporting period, to include a summary of the progress made, project milestone schedule, challenges, successes, proposed changes, and next steps. The report shall also identify any problems that arose and a description of how the problems were resolved. If problems have not been completely resolved, the Contractor shall provide an explanation including their plan and timeframe for resolving the issue. The report shall also include an itemized list of all Electronic and Information Technology (EIT) deliverables, as applicable, and their current Section 508 conformance status. The Contractor shall monitor performance against the CPMP and report any deviations. It is expected that the Contractor will keep in communication with VA accordingly so that issues that arise are transparent to both parties to prevent escalation of outstanding issues.

The Contractor shall attend an up-to-weekly teleconference meeting, a cadence to be determined by the VA Program Manager, to be held at a time convenient for both the government and the Contractor. The Contractor shall provide Teleconference Progress Meeting Minutes within two business days after the teleconference meeting. The

Contractor shall provide weekly emails with the progress, issues, and mitigations to the VA Program Manager and additional VA team members as designated by the VA PM.

This team will be introduced in the kickoff meeting.

Deliverable:

A. Cumulative Monthly Progress Report B. Email Weekly to VA PM

4.2 PHASE 1 PROTOTYPING

4.2.1 PROTOTYPE SCENARIO DESIGN

The Contractor shall, in collaboration with VA PM and identified VA SMEs, review existing VA-approved language, suicide prevention training materials and curriculum.

The Contractor shall conduct meetings with identified VA SMEs and other stakeholders to understand requirements and current VA approaches for training and then, based on information gathered, develop scenario outlines and scripts focused on suicide prevention, leveraging existing VA resources as approved by VA. The Contractor shall provide those scripts to the VA PM and other SMEs for review and approval.

The Contractor shall then create user stories based on those approved scripts and any applicable VA guidelines and will submit those to VA PM for review and approval.

Deliverables:

A. Initial Scenario Outlines and Scripting B. Prototype User Stories

4.2.2 INITIAL PROTOTYPE DEVELOPMENT

Using approved prototype user stories and associated scripting, the Contractor shall develop the initial basic VR prototype for role-playing exercises. This shall include the implementation of core functionality and basic user interface in a VR environment and a

2-D interface. Development shall include only VA-approved language and approaches into the VR scenarios.

Deliverable:

A. Basic VR Prototype for Role-Playing Exercises

4.2.3 DEVELOPMENT OF AI GRADING FUNCTIONALITY

The Contractor shall develop, following VA policies and guidelines, an AI-enabled software system to embed in the solution to enable real-time evaluation of end-user responses to scenarios developed compared to the VA curriculum and best practices.

This functionality shall be added to existing VR prototype for review by VA PM and SMEs for user testing.

A. AI Grading Prototype Functionality

4.2.4 SUSTAINABLE DATA ACCESS AND ANALYSIS

The Contractor shall coordinate with VA Program Manager to determine metrics associated with project goals and specify the means of capturing required information in a data collection and analytics plan. Data collection and analytics plan will also include co-development of pre- and post-testing survey questions.

The Contractor shall provide aggregated, deidentified data showing impact of solution including both quantitative and qualitative metrics as well as develop a means of completing and visualizing data analytics accessible by VA. The Contractor shall use the REAIM evaluation framework to evaluate impact, value, and all associated processes and outcomes, in all phases of this project. Final plan for data collection and analysis, including metrics, shall be presented to the VA Program Manager for approval before developing a data analytics visualization solution.

The Contractor shall develop a means of accessing data metrics and analytics determined and approved by VA PM that will be accessible to VA. Once the Contractor has developed a resource for providing real-time data analytics visualization, VA PM must be provided access for review and approval. Resource may be tested and iterated upon in Phase 1 and Phase 2.

A. Data Collection and Analysis Plan B. Project Data Access Resource with Real-time Data Analytics Visualization

4.2.5 PHASE 1 USER TESTING AND PRODUCT ITERATION

Prior to beginning user testing, the Contractor shall develop a Phase 1 User Testing Plan and provide to the VA PM for review and approval. Then, to enable participation from site-level providers and other SMEs, the Contractor shall develop and coordinate the signing of Memorandums Of Understanding (MOUs) between participating sites, OHIL, and OSP. Phase 1 User Testing shall be conducted with a minimum of 3-5 sites, 10 users at each site.

The Contractor shall conduct initial user testing with VA SMEs and other VA-designated site-based participants. The Contractor shall gather feedback on VR experience, content accuracy, and effectiveness of AI grading functionality using the agreed upon

Data Collection and Analysis Plan. Once a user testing session(s) is complete, the

Contractor shall analyze and synthesize feedback, provide the results to the VA PM as a User Testing Feedback Report, including plans for iteratively refining content and approach prior to next User Testing session.

The Contractor will then update user stories and create a refined product backlog based on feedback received and will develop updated VR and 2D prototypes with that feedback to create a Final Prototype ready for additional User Testing and initial Field

Testing in Phase 2.

A. Phase 1 User Testing Plan B. MOU Coordination for at least 5 Sites C. Phase 1 User Testing Feedback Report (5)

4.2.6 FINAL PHASE 1 REPORT

At the completion of Phase 1, the Contractor shall provide the VA Program Manager with a Phase Summary Report, inclusive of all work completed during Phase 1 and all data analytics required to meet project goals and determination of innovation value.

A. Final Phase 1 Summary Report

4.3 OPTIONAL TASK – PHASE 2 FIELD TESTING

If VA exercises the Optional Task, the Contractor shall perform tasks identified in Sections 4.3.1 and all subsections. The period of performance for this optional task is 12 months. In reference to the deliverables in these sections, if the task has generated a document during the base period, the Contractor shall provide updates only. If VA exercises the Option Period, the Contractor shall also perform the following:

4.3.1 FIELD TESTING AND PRODUCT ITERATION

Prior to beginning field testing, the Contractor shall develop a Phase 2 Field Testing Plan, including all pre-determined metrics and evaluation criteria from the Data Collection and Analysis Plan, and provide to the VA PM for review and approval. Field testing shall include a collection of data pre-testing, immediately following testing, and at 8 weeks post-testing. To enable participation from site-level providers and other subject matter experts, the Contractor shall develop and coordinate the signing of MOUs between additional participating sites, OHIL, and the OSP. Phase 2 Field Testing evaluation shall be conducted with a minimum of 5-10 sites, minimum 10-20 participants at each site, in a configuration that ensures at least 100 total participants in Phase 2. 5- 10 individuals at each site will participate in VR training with pre-determined pre- and post-questions, and 5-10 will only complete existing training but will complete the same pre- and post-questions, with a goal of having 50 participants in the VR group and 50 in the control group. The Contractor shall coordinate dates for on-site visits to each participating site, prepare for and travel to sites, set up equipment for multiple testing sessions, instruct users how to use the solution (if in VR group), document all feedback, and facilitate completion of pre-questions and immediate post-questions with all participants at site. The Contractor will then follow-up with participants 8 weeks later to request completion of additional post-questions. The Contractor shall provide 5 VR headsets to the three sites and sites will retain ownership of following field-testing completion.

Once a field-testing site visit is complete, the Contractor shall aggregate quantitative and qualitative feedback and data from the field site and provide the results to the VA

PM as a field-testing report, including plans for iteratively refining content and approach prior to next field-testing site visit.

The Contractor will update VR and 2D solutions using Field Testing feedback, if changes approved by VA PM, to create a final product ready for operationalizing. This may include updating user stories, imagery, functionalities, or any necessary and feasible changes to ensure a satisfactory product. Application shall be converted from prototype status to one ready for initial operational capability. The Contractor shall create “how to” guides and any resources required by VA prior to performance period completion. The Contractor shall enable the auto-update features so headsets can be updated remotely while in the field. All files and resources to allow VA to continue iteration and further development of the solution post-performance period shall be provide to the VA PM prior to contract expiration – transferring ownership of the solution to the VA.

A. Phase 2 Field Testing Plan B. MOU Coordination for at least 10 Sites C. Phase 2 Field Testing Report (10) D. Final Product, both VR and 2D versions, including necessary APK files/source code

4.3.2 OPERATIONAL PLAYBOOK FOR SUSTAINABILITY AND SCALABILITY

Based on results of all User and Field testing, and with any additional feedback and insights from VA participants and SMEs, the Contractor shall develop a uniform process via an Operational Playbook for Sustainability and Scalability for implementing the solution with additional sites in the future and plan for integration of solution with VA’s Talent Management System (TMS) in a manner to allow tracking of training completion.

A. Operational Playbook for Sustainability and Scalability

4.3.3 PHASE 1 AND PHASE 2 COMBINED FINAL REPORT

At the completion of Phase 2, the Contractor shall provide to the VA Program Manager a Project Summary Report, inclusive of all work completed during Phase 1 and 2 and all data analytics (initial data analyses and final data analyses of all data captured during all periods of performance) required to meet project goals and determination of innovation value. Summary Report will also include detailing the process and outcomes, including recommendations for additional testing, scalability, and/or integration with other VA Resources.

A. Phase 1 and Phase 2 Final Project Report

5.0 SCHEDULE OF DELIVERABLES

Task Item Description Quantity Due Date (Calendar Days)

4.1.1 CONTRACTOR

PROJECT

MANAGEMENT PLAN

Contractor Project

Management Plan

1 Thirty (30) days after contract

(DAC) and updated monthly thereafter

4.1.2 REPORTING

REQUIREMENTS

Cumulative Monthly

Progress Report

12 5th Day of each month throughout the POP

4.1.2 REPORTING

REQUIREMENTS

Email Weekly to VA PM 52 Due every Friday after kickoff

4.2.1 PROTOTYPE

SCENARIO DESIGN

Initial Scenario Outlines and Scripting

1 Thirty (30) days after contract award

4.2.1 PROTOTYPE

SCENARIO DESIGN

Prototype User Stories 1 Sixty (60) days after contract award

4.2.2 INITIAL

PROTOTYPE

DEVELOPMENT

Basic VR Prototype for

Role-Playing Exercises

11 Ninety (90) days after contract award

4.2.3 DEVELOPMENT

OF AI GRADING

FUNCTIONALITY

AI Grading Prototype

Functionality

1 Ninety (90) days after contract award

4.2.4 SUSTAINABLE

DATA ACCESS AND

ANALYSIS

Data Collection and

Analysis Plan

1 One hundred and twenty (120) days after contract award

4.2.4 SUSTAINABLE

DATA ACCESS AND

ANALYSIS

Project Data Access

Resource with Real-time Data Analytics

Visualization

1 One hundred and twenty (120) days after contract award

4.2.5 PHASE 1 USER

TESTING AND

PRODUCT

ITERATION

Phase 1 User Testing

Plan

1 Ninety (90) days after contract award

4.2.5 PHASE 1 USER

TESTING AND

PRODUCT

ITERATION

MOU Coordination for at least 5 Sites

1 One hundred and twenty (120) days after contract award

4.2.5 PHASE 1 USER

TESTING AND

PRODUCT

ITERATION

Phase 1 User Testing

Feedback Report (5)

1 Three hundred

(300) days after contract award

4.2.6 FINAL PHASE 1

REPORT

Final Phase 1 Summary

Report

1 Three-hundred and twenty-five (325) days after contract award

4.3.1 FIELD TESTING

AND PRODUCT

ITERATION

Phase 2 Field Testing

Plan

1 Thirty (30) days after the award of optional period

4.3.1 FIELD TESTING

MOU Coordination for at least 10 Sites

1 Forty-five (45) days after the award of optional period

4.3.1 FIELD TESTING

Phase 2 Field Testing

Report (10)

1 Two hundred (200) days after exercise of optional period

4.3.1 FIELD TESTING

AND PRODUCT

ITERATION

Final Product, both VR and 2D versions, including necessary

APK files/source code

1 Two hundred (200) days after exercise of optional period

4.3.2 OPERATIONAL

PLAYBOOK FOR

SUSTAINABILITY

AND SCALABILITY

Operational Playbook for Sustainability and

Scalability

1 Three hundred

(300) days after exercise of optional period

4.3.3 PHASE 1 AND

PHASE 2 COMBINED

FINAL REPORT

Phase 1 and Phase 2

Final Project Report

1 Three hundred and fifty (350) days after exercise of optional period

6.0 GENERAL REQUIREMENTS

6.1 SECURITY AND PRIVACY REQUIREMENTS

6.1.1 POSITION/TASK RISK DESIGNATION LEVEL(S)

In accordance with VA Handbook 0710, Personnel Security and Suitability Program, the position sensitivity and the level of background investigation commensurate with the required level of access for the following tasks within the PWS are:

Position Sensitivity and Background Investigation Requirements by Task

Task Number Tier1 / Low Risk Tier 2 / Moderate Risk

Tier 4 / High Risk

6.1

6.2

6.3

The Tasks identified above, and the resulting Position Sensitivity and Background Investigation requirements identify, in effect, the Background Investigation requirements for Contractor individuals, based upon the tasks the particular Contractor individual will be working. The submitted Contractor Staff Roster must indicate the required Background Investigation Level for each Contractor individual based upon the tasks the Contractor individual will be working, in accordance with their submitted proposal.

6.1.2 CONTRACTOR PERSONNEL SECURITY REQUIREMENTS

Contractor Responsibilities:

a. The Contractor shall prescreen all personnel requiring access to the computer systems to ensure they maintain the appropriate Background Investigation, and are able to read, write, speak and understand the English language.

b. Within 3 business days after award, the Contractor shall provide a roster of Contractor and Subcontractor employees to COR to begin their background investigations in accordance with the PAL template artifact. The Contractor Staff Roster shall contain the Contractor’s Full Name, Date of Birth, Place of Birth, individual background investigation level requirement (based upon Section 6.2 Tasks), etc. The Contractor shall submit full Social Security Numbers either within the Contractor Staff Roster or under separate cover to the COR. The Contractor Staff Roster shall be updated and provided to VA within 1 day of any changes in employee status, training certification completion status, Background Investigation level status, additions/removal of employees, etc. throughout the Period of Performance. The Contractor Staff Roster shall remain a historical document indicating all past information and the Contractor shall indicate in the Comment field, employees no longer supporting this contract. The preferred method to send the Contractor Staff Roster or Social Security Number is by encrypted e-mail. If unable to send encrypted e-mail, other methods which comply with FIPS 140-2 are to encrypt the file, use a secure fax, or use a traceable mail service.

c. The Contractor should coordinate with the location of the nearest VA fingerprinting office through the COR. Only electronic fingerprints are authorized. The Contractor shall bring their completed Security and Investigations Center (SIC) Fingerprint request form with them (see paragraph d.4. below) when getting fingerprints taken.

d. The Contractor shall ensure the following required forms are submitted to the COR within 5 days after contract award:

1) Optional Form 306

2) Self-Certification of Continuous Service

3) VA Form 0710

4) Completed SIC Fingerprint Request Form

e. The Contractor personnel shall submit all required information related to their background investigations (completion of the investigation documents (SF85, SF85P, or SF 86) utilizing the Office of Personnel Management’s (OPM) Electronic Questionnaire for Investigations Processing (e-QIP) after receiving an email notification from the Security and Investigation Center (SIC).

f. The Contractor employee shall certify and release the e-QIP document, print and sign the signature pages, and send them encrypted to the COR for electronic submission to the SIC. These documents shall be submitted to the COR within 3 business days of receipt of the e-QIP notification email. (Note: OPM is moving towards a “click to sign” process. If click to sign is used, the Contractor employee should notify the COR within 3 business days that documents were signed via e-

QIP).

g. The Contractor shall be responsible for the actions of all personnel provided to work for VA under this contract. In the event that damages arise from work performed by the Contractor provided personnel, under the auspices of this contract, the Contractor shall be responsible for all resources necessary to remedy the incident.

h. A Contractor may be granted unescorted access to VA facilities and/or access to VA Information Technology resources (network and/or protected data) with a favorably adjudicated Special Agreement Check (SAC), completed training delineated in VA Handbook 6500.6 (Appendix C, Section 9), signed “Contractor Rules of Behavior”, and with a valid, operational PIV credential for PIV-only logical access to VA’s network. A PIV card credential can be issued once your SAC has been favorably adjudicated and your background investigation has been scheduled by OPM.

However, the Contractor will be responsible for the actions of the Contractor personnel they provide to perform work for VA. The investigative history for Contractor personnel working under this contract must be maintained in the database of OPM.

i. The Contractor, when notified of an unfavorably adjudicated background investigation on a Contractor employee as determined by the Government, shall withdraw the employee from consideration in working under the contract.

j. Failure to comply with the Contractor personnel security investigative requirements may result in loss of physical and/or logical access to VA facilities and systems by Contractor and Subcontractor employees and/or termination of the contract for default.

k. Identity Credential Holders must follow all HSPD-12 policies and procedures as well as use and protect their assigned identity credentials in accordance with VA policies and procedures, always displaying their badges, and returning the identity credentials upon termination of their relationship with VA.

6.2 METHOD AND DISTRIBUTION OF DELIVERABLES

The Contractor shall deliver documentation in electronic format of the solicitation/contract. Acceptable electronic media include MS Word 2000/2003/2007/2010, MS Excel 2000/2003/2007/2010, MS PowerPoint 2000/2003/2007/2010, MS Project 2000/2003/2007/2010, MS Access 2000/2003/2007/2010, MS Visio 2000/2002/2003/2007/2010, AutoCAD 2002/2004/2007/2010, and Adobe Postscript Data Format (PDF).

6.3 QUALITY ASSURANCE SURVEILLANCE PLAN (QASP)

The COR will utilize a Quality Assurance Surveillance Plan (QASP) throughout the life of the contract to ensure that the Contractor is performing the services required by this PWS in an acceptable level of performance. The Government reserves the right to alter or change the surveillance methods in the QASP at its own discretion. A Performance Based Service Assessment will be used by the COR in accordance with the QASP to assess Contractor performance.

The Government shall evaluate the Contractor's performance in accordance with the Quality Assurance Surveillance Plan (QASP). This plan is primarily focused on what the Government must do to ensure the Contractor has performed in accordance with the performance standards. It defines how the performance standards will be applied, the frequency of surveillance, and the minimum acceptable defect rates.

Performance Standards define required performance for specific tasks. The Government performs surveillance to determine if the Contractor exceeds, meets or does not meet these standards. Various methods exist to monitor performance. The COR shall use any or several of the surveillance methods listed below in the administration of this QASP.

i. 100% Inspection (evaluates all outcomes) - Each month, the COR shall review all the Contractor’s performance/generated documentation and document results accordingly. This assessment shall be placed in the COR’s Quality Assurance File.

ii. Random Sample - The COR shall review a random sampling of the Contractor’s performance/generated documentation and document your results accordingly.

This assessment shall be placed in the COR’s QA file. Validated Customer Complaint or Validated Below Average ALP in a Specific Area - Each month, the COR shall review the Contractor’s performance/generated documentation corresponding to a validated customer complaint or validated inability to perform in accordance with the ALP in a specific area and document your results accordingly. This assessment shall be placed in the COR’s QA file.

6.4 PERFORMANCE METRICS

The table below defines the Performance Standards and Acceptable Levels of Performance associated with this effort.

Performance Objective

Performance Standard Acceptable Levels of Performance

A. Technical / Quality of Product or Service

1. Demonstrates understanding of requirements

2. Efficient and effective in meeting requirements

3. Meets technical needs and mission requirements

4. Provides quality services/products

Satisfactory or higher

B. Project Milestones and Schedule

1. Established milestones and project dates are met

2. Products completed, reviewed, delivered in accordance with the established schedule

3. Notifies customer in advance of potential problems

Satisfactory or higher

C. Cost & Staffing 1. Currency of expertise and staffing levels appropriate

2. Personnel possess the necessary knowledge, skills and abilities to perform tasks

Satisfactory or higher

D. Management 1. Integration and coordination of all activities to execute effort

Satisfactory or higher

The COR will utilize a Quality Assurance Surveillance Plan (QASP) throughout the life of the contract to ensure that the Contractor is performing the services required by this PWS in an acceptable level of performance. The Government reserves the right to alter or change the surveillance methods in the QASP at its own discretion.

6.5 FACILITY/RESOURCE PROVISIONS

The Government may provide office space, telephone service and system access when authorized contract staff work at a government location as required to accomplish the Tasks associated with this PWS if available. All procedural guides, reference materials, and program documentation for the project and other Government applications will also be provided on an as-needed basis.

The Contractor shall request other Government documentation deemed pertinent to the work accomplishment directly from the Government officials with whom the Contractor has contact. The Contractor shall consider the COR as the final source for needed

Government documentation when the Contractor fails to secure the documents by other means. The Contractor is expected to use common knowledge and resourcefulness in securing all other reference materials, standard industry publications, and related materials that are pertinent to the work.