JA - TAC-20 - Splunk Cloud SaaS License - Subscription - Redacted.pdf

Splunk Cloud License/Subscription Control Number: VA-20-00034083

JUSTIFICATION

FOR AN EXCEPTION TO FAIR OPPORTUNITY

1. Contracting Activity: Department of Veterans Affairs (VA) Office of Procurement, Acquisition and Logistics Technology Acquisition Center 23 Christopher Way Eatontown, New Jersey 07724

2. Description of Action: The proposed action is for the award of a firm-fixed-price delivery order to be issued under the National Aeronautics and Space Administration (NASA) Solutions for Enterprise-Wide Procurement (SEWP) V Governmentwide Acquisition Contract (GWAC), for brand name Splunk Cloud License/Subscription in support of the VA Office of Electronic Health Record Modernization (OEHRM) Performance Monitoring project and COVID-19 support for increased telehealth and telework.

3. Description of the Supplies or Services: VA Office of Information and Technology (OIT), Enterprise Command Operations (ECO) and OEHRM have a requirement for procurement of two annual brand name Splunk Cloud Software as a Service (SaaS) Licenses/Subscription, as well as brand name Splunk professional support services, in order to support emergent mission requirements related to COVID-19 and management/support of OEHRM applications. The required SaaS licenses will allow VA ECO the ability to monitor the availability and performance of the VA infrastructure and applications that utilize data feeds from Computer Associates application performance monitoring, Solarwinds, Dynatrace and AppDynamics products. Within the proposed Splunk Cloud SaaS, Splunk Core serves as the log aggregation engine for machine data generated by VA information technology (IT) systems within physical, virtual, and cloud infrastructures, and provides a means to capture data generated by VA’s IT infrastructure and applications. Splunk’s Core log management tool will provide VA ECO’s infrastructure and application analysis teams with a definitive record of all key performance indicators (KPIs) for the VA IT infrastructure and application environment and other activity in a raw data form that can be stored indefinitely. Another component of the Splunk Cloud SaaS solution is the Splunk IT Service Intelligence (ITSI) module which provides the VA ECO the capability to develop glass tables/dashboards to display the services and KPIs within the VA environment in support of the VA OEHRM program.

VA ECO will utilize these glass tables and dashboards to provide rapid incident response, event correlation, and in-depth monitoring across all data sources and external partners, monitor equipment uptime, and proactively identify, pinpoint, and resolve performance problems across the OEHRM program. The Splunk Cloud instance will be further utilized to support the emerging need for monitoring of the application performance of the telehealth systems. Additionally, VA Cyber Security will utilize the platform as a pilot to monitor the active directory systems and the increase in telework activity associated with COVID-19.

Also included in this effort are Splunk Professional Services required for cloud instance setup and configuration, as well as on-site and offsite support services for the management, training, and administrative services related to the deployment of the Splunk SaaS solution.

The period of performance is 12 months from date of contract award with two 12-month option periods, as well as Optional Tasks for 50 optional units for additional data storage (one unit equates to 500 gigabytes of additional data storage) and one (1) additional Splunk Professional Services line item for additional management, training, and administrative services per contract period.

4. Statutory Authority: Pursuant to Federal Acquisition Regulation (FAR) 16.505(a)(4), the following brand name requirement is being justified and approved as an exception to fair opportunity under the statutory authority of Section 41 U.S.C. 4106(c)(2) as implemented by FAR 16.505(b)(2)(i)(B), entitled “Only one awardee is capable of providing the supplies or services required at the level of quality required because the supplies or services ordered are unique or highly specialized.”

5. Rationale Supporting Use of Authority Cited Above: Based on market research, as described in section 8 of this justification, it was determined that only the Splunk brand name SaaS solution is capable of supporting the COVID-19 monitoring requirements and the requirements of the OEHRM program.

In Fiscal Year 2015, the VA competitively awarded a contract for the deployment of Splunk Enterprise solution in order to satisfy the VA ECO monitoring requirements. Due to the requirement to expand the VA ECO’s ability to provide continuous monitoring during the COVID-19 pandemic, and VA’s increased telework and telehealth platforms as a result of this pandemic, VA must pivot to a cloud-based solution in order to meet capacity and ongoing monitoring requirements. The existing solution, while previously able to provide these monitoring requirements, cannot continue to properly function without significant enhancement of the existing infrastructure to accommodate this additional strain to the system. VA estimates that these infrastructure improvement costs would exceed $6 Million in hardware/software augmentations per year, not including the additional required VA labor and configuration, and take a minimum of six months to implement. These figures are based on historical information for previous implementations.

Further, based on VA’s Market Research it was determined that no solution, other than the Splunk SaaS solution, can provide the required functional requirement of an intuitive single pane of glass. Specifically, this unique feature allows VA Leadership from different organizations such as Veterans Health Administration (VHA) , OIT, and OEHRM the ability to view the functionality of the system in real time, which will directly provide leadership data increasing its ability to make informed decisions and report more accurately on potential system downtime. For instance, this function would allow VHA the ability to verify status of the functionality of its telehealth systems, which are critical to providing direct care for the Veteran and reducing strain on the VA Medical Centers. Failure to obtain this function would result in VA’s impaired ability to provide Veteran patient care. Current monitoring tools do not provide the single pane dashboarding capability needed to rapidly identify and diagnose issues as well as provide VHA, OIT and OEHRM leadership the visibility required to manage the program.

ITSI can link the Splunk SaaS Core instance with other monitoring tools as well as Splunk Core products from disparate partners in the OEHRM environment such as DoD and Cerner. Further, it correlates data feeds from the current monitoring tools (Computer Associates Intrascope, Solarwinds, Dynatrace and AppDynamics) into a single presentation of data. Failure to provide this functional requirement will exacerbate inefficiencies, resulting in VA leadership having to review multiple webpages from multiple tools which would have to be reviewed and analyzed manually to assess the overall system health which effectively decreases visibility and increases time to resolution. As such, the Splunk product provides End-to-End visibility of VA’s environment from a single point, and increases VA’s ability to rapidly respond to issues within the network.

Additionally, OEHRM has strict compatibility requirements to integrate with external partners. Specifically, partners such as the Department of Defense (DoD) and Cerner are both utilizing the Splunk Core log aggregation tool in its environments. The proprietary language utilized by Splunk is not compatible with any other known brand name products. This ability to integrate with VA’s partners is a core requirement of the Electronic Health Record Modernization (EHRM) effort, as VA positions itself to have a single Electronic Health Record (EHR) with DoD. Any other potential product would therefore limit the data feeds VA could receive from DoD and Cerner partners thereby preventing full End-to-End visibility of the shared Federal EHR system. Furthermore, the on-premise Splunk Enterprise version currently in use by Cyber Security is only compatible with brand name Splunk products. A pilot with any other product would not yield like-for-like results for determining the viability of transitioning all the cyber security Splunk products to the cloud offering.

Further, Splunk is the only solution that currently has a FedRAMP Medium certified environment that meets the VA Authority to Operate (ATO) security requirements. No other solution would be able to meet the timeliness requirements of providing an immediate Cloud based solution in order to provide relief. Specifically, hypothetically if an alternative SaaS/Cloud solution were available that could meet the functional requirements, it would still have to receive an ATO within the VA environment which would take approximately three to six months.

Also included in this effort are brand name professional support services that will provide VA with the implementation and configuration of this product, as well as the training of the solution. No source, other than Splunk or its authorized resellers, can provide these services due to the requirement of access to the proprietary code of the solution. Additionally, no source, other than Splunk or its authorized resellers, have the requisite and intimate knowledge of the product to provide VA the required support and ensure optimal operation of the solution.

Failure to procure this solution and the associated services increases the risk to VA that it will be impaired in providing its mission, which is to care for the Veteran. Specifically, the inability to provide real time status of the infrastructure and ability to view the system from End-to-End, could have disastrous impact on VA’s ability to provide telehealth services, which would prove especially harmful during the COVID-19 pandemic.

6. Efforts to Obtain Competition: Market research was conducted, details of which are in Section 8 of this justification. This effort did not yield any additional sources that can meet the Government's requirements. However, it was determined that limited competition is available among authorized resellers for brand name Splunk Licenses. In accordance with FAR 16.505(a)(4)(iii)(A)(1), this justification will be provided with the solicitation to all appropriate NASA SEWP V GWAC holders. Furthermore, in accordance with FAR 5.301, 16.505(b)(2)(ii)(D), and 16.505(a)(4)(iii), the award notice for this action will be synopsized on the Federal Business Opportunities website and this justification will be made publicly available within 14 days of award.

7. Actions to Increase Competition: In order to remove or overcome barriers to competition in future acquisitions for this requirement, the Government will continue to perform market research to determine if there are any new products or maintenance services available in the marketplace that would enable future actions to be fully competed.

8. Market Research: Market Research was conducted in January and February of 2020 to evaluate the industry leading log file and data aggregation products. VA technical experts identified two potential solutions. The first is a tool suite comprised of Elasticsearch, Logstash, Kibana (ELK) and the second is the proposed Splunk SaaS.

VA is currently utilizing both Splunk Enterprise (on-premise) and the ELK tool suite.

The ELK tool suite, while a cloud based technology that was piloted to determine suitability is not capable of providing the functional requirement of an intuitive single pane of glass view, it is not a FedRAMP approved environment or a SaaS, nor does it integrate with the required VA Partners as noted above.

Additional market research was conducted to determine if limited competition is available among authorized resellers. The NASA SEWP V GWAC Provider Lookup Tools was also utilized on April 1, 2020 to search for Splunk resellers. The search yielded 83 resellers including 17 Service-Disabled Veteran Owned Small Business, five of which are authorized resellers. There is reasonable expectation that limited competition exists on the NASA SEWP V GWAC for Splunk SaaS.

9. Other Facts: None