P03_JA_IOSS Extension_Final_Redacted.pdf

Control Number VA-21-00057209

JUSTIFICATION

FOR AN EXCEPTION TO FAIR OPPORTUNITY

1. Contracting Activity: Department of Veterans Affairs (VA) Office of Procurement, Acquisition, and Logistics

Technology Acquisition Center 23 Christopher Way Eatontown, NJ 07724

2. Description of Action: The proposed sole source modification is for a continuation of Infrastructure Operations Services Support (IOSS) services against Transformation Twenty-One Total Technology Next Generation (T4NG),Contract VA118-16-D-1004 Task Order 36C10B18N10040007 with Systems Made Simple, Inc. (SMS).

3. Description of the Supplies or Services: On March 6, 2017, T4NG Task Order (TO) VA118-16-D-1004 36C10B18N10040007 was awarded to SMS using competitive procedures for management, operations, logistics and maintenance and technical support services in support of VA, Office of Information and Technology, Information Technology (IT) Operations and Services (ITOPS), Infrastructure Operations. The current period of performance ends on June 6, 2021. This will impact Veterans that rely on these services if the applications and/or infrastructure become inoperable. IOSS employs a contractor staff of approximately 700 which provides support for 90 VA Project Managers across eight different VA organizations (Enterprise Project Management Office, Infrastructure Operations (IO), Electronic Health Record Modernization, Information Technology Resource Management (ITRM), Service Management and Planning, Enterprise Command Operations, End User Operations, Technology and Platform Services. Under this extension, IOSS’ Contractor Staff will continue to deliver mission critical IT support services including operational support for VA’s data centers, application management for delivery of healthcare and benefits to Veterans, IT systems security monitoring and remediation, and IT incident response coordination to all VA hospitals, benefits offices and VA cemeteries to perform core mission functions for over 17,000 servers and 400 major applications. Critical areas of the IOSS task order include Network Management supporting infrastructure for all VA datacenters, Database System Administration supporting all VA databases, Technical Cybersecurity and Security Support which monitors and protects VA data and applications to include VA patient information, and Application Performance and Availability Monitoring Administration/Architecture. A more detailed description of these IOSS requirements is provided in Section 5 of this Justification. While the task order is a hybrid of Time-and-Materials (T&M) and Firm-Fixed Price (FFP), this Justification is for the FFP support and the extension of the T&M line items as there is sufficient ceiling for the T&M support. The period of performance is a four-month base period with two one-month option periods. The total estimated value of this proposed acquisition is for the FFP portion for the four-month extension and two one-month options.

4. Statutory Authority: The statutory authority permitting an exception to fair opportunity is Section 41 U.S.C. 4106(c)(2) as implemented by the Federal Acquisition

IOSS Task Order Extension Control Number VA-21-00057209

Regulation (FAR) Subpart 16.505(b)(2)(i)(B), entitled “Only one awardee is capable of providing the supplies or services required at the level of quality required because the supplies or services ordered are unique or highly specialized.”

5. Rationale Supporting Use of Authority Cited Above: The proposed source for this action is SMS, 149 Northern Concourse, Syracuse, New York 13212. Based on market research, which can be found in Section 8 of this document, it was determined that SMS remains the only responsible source capable of satisfying VA’s IOSS requirements in the required timeframe. No other source can provide the aforementioned services on day one of task order extension. The amount of time to on-board and recruit an individual resource is six to eight weeks. This timeframe includes fingerprint capturing, security clearances for badging, physical, logical, and elevated system access. Existing IOSS Contractor Staff can commence performance immediately upon award of this extension as there is no learning curve or on-boarding requirements. VA anticipates the award of a competitive TO under the T4NG contract vehicle on or about July 30, 2021 which would encompass the services detailed in Section 3. There were unanticipated delays that occurred from November 2019 to April 2021 which delayed award of this competitive TO. In this timeframe, VA migrated from Information Technology Operations and Services (ITOPS) to Development, Security, and Operations (DevSecOps). In addition to the continued support, VA is seeking a new requirement for modernization and innovation that had not been accomplished on previous TO efforts. Extensive market research was conducted and a Request for Information (RFI) was published on December 21, 2020 which resulted in over 300 industry questions, specifically, VA received 189 questions submitted by Industry in January 2021. VA assessed the questions and made adjustments to multiple sections within the PWS based on questions and input from Industry. In March 2021, VA, unsolicited, received an additional 146 questions specific to PWS Section 5.8, Data Center IT Facilities and Connectivity. This section was previously T&M and migrated to FFP. Based on Industry questions and input, the PWS was further revised. While this requirement is for continued services, the approach to the competitive award required an extensive amount of rewrite and update to the requirements. VA had to better assess its requirements and needed additional input due to the complexity of the requirements.

VA experienced delays in better defining requirements that transitioned from a T&M price structure to FFP. It was determined, as result of market research, the timeframe for the Request for Task Execution Plan needed to be extended.

Due to the high number of full time employees needed, only SMS possesses all resources ready with the necessary security clearances to include physical, logical and elevated system access, no learning curve for any new resources required, and retainment of knowledge of Contractor staff and can commence performance immediately upon contract award. Only SMS can provide the essential services without a significant interruption and minimal turnover which is critical to Veterans, caregivers, and families. SMS can provide the required services immediately upon contract award with no ramp-up time required. Also, no other contractor can provide the required services because they could not onboard the proper personnel immediately. This transition will require the coordination and facilitation of approximately 700 plus contractor resources to be recruited, accepted, physical, logical, and elevated access established, background investigation submitted, and Government furnished equipment (GFE) issued.

Each accepted contractor would require National Criminal Fingerprint for an initial criminal background check that is favorable. In addition to a favorable Fingerprint check, a background investigation request would have to be submitted to VA Security and Investigation Center for review and concurrence and then submitted to Office of Personal Management (OPM) for investigation. Once received by OPM, the PIV process would involve each contractor to be scheduled, registered, fingerprint capture, badge creation, issuance, and desktops support (to register PIV Certs). This would all be required prior to the issuance of a PIV and elevated system access being granted.

Due to the multiple dependencies to complete these actions a vendor would have to supply approximately 700 plus contractor personnel who have a current adjudicated background investigation, current PIV card, existing accesses and already issued GFE.

This on-boarding process can range from two to six weeks per resource.

VA requires essential IOSS services immediately and it is critical to obtain these services. Specifically, the Contractor shall provide Network Management Support for all VA Data Center and Tier 3 Field Operations to over 1,800 VA facilities, 33,000 network devices, and 150,000 wireless access points. Services include 24x7x365 support in the area of connectivity, interoperability with the various computer platforms.

Routers and switches must be installed, configured, secured, backed up, tuned and monitored daily. The Contractor must provide 24x7x365 availability. Network support shall require troubleshooting to a wide range of individuals including customers, end users, programmers and analysts, Database Administrators, and other System and Network Administrators. When network connectivity is interrupted, Veterans are unable to access critical resources such as; the Veterans Crisis Line, health and benefits services, electronic health records, telehealth, online pharmacy refills, diagnostic imaging and medicine home loan applications. The Veterans Crisis Line connects Veterans in crisis and their families and friends with qualified, mental health professionals through a confidential toll-free hotline, online chat, or text. The Veterans Crisis Line receives approximately 2,000 calls per day. If the infrastructure supporting this hotline goes down, VA will not have the IOSS TO in place to fix it, which would render the hotline inoperable and could immediately and directly result in increased suicides amongst the Veteran population.

In addition, the Contractor shall provide 24x7x365 Database System Administration support to include availability, troubleshooting, and tuning to ensure acceptable response times. The Contractor shall install/tailor new releases, upgrades or patches to associated database management product software, diagnose and correct problems that occur within databases and application interfaces. One example is the Veterans Health Administration Medication Order Check Healthcare Application (MOCHA).

MOCHA provides drug order checking to address critical patient safety issues. When a VA provider orders a drug, MOCHA performs order checks on that drug and alerts the provider if that drug has anomalies. Examples could include: an interaction with other drugs the patient is taking; prescriptions being in the same Therapeutic Class as other drugs the patient is taking; or prescription of a drug in a dose that is incompatible with patient factors such as age, weight and body surface area. If MOCHA becomes inoperable, VA will not have the IOSS in place to fix it. When MOCHA becomes unavailable, physicians are unable to cross check Veterans’ prescriptions allowing the possibility of negative interactions between existing and new prescriptions. The resulting inability of medical personnel to cross-check prescriptions could directly and immediately result in serious harm or the loss of Veteran life.

Also, the Contractor shall provide 24x7x365 Technical Security Support to manage the security tools and the growing technical security responsibilities across ITOPS IO in order to secure Veterans’ data and important VA systems. Security Threat Monitoring responds to an average of 85 daily confirmed cyber security threat responses. This very real security problem threatens loss of Veterans’ data and significant application outages across all applications in VA. The support services required include analyzing current security tools and datacenter environments to ensure all cybersecurity environments meet Federal Information Security Modernization Act System, National Institute of Standards and Technology controls and ensures the Government Accountability Office, Office of Inspector General, Internal Revenue Services and Office of Management and Budget get the support required per Federal and VA policy and customization needed with the different applications or datacenter solutions. IOSS provides after-hour cybersecurity support for the immediate customer, ISO and VA Incident Response via the Office of Information Security Cybersecurity Operations Center requirements. Without these contractor resources on staff, the security posture of IO would be severely diminished, putting Veterans’ personal health information (PHI), personally identifiable information (PII) and VA systems at critical risk of exposure.

Beyond the threat to Veterans’ PHI/PII exposure, there is an economic impact to VA anytime our Veterans’ data is compromised. Charges to prevent and cover possible losses from the data theft can total 40 million to 500 million dollars per significant incident. This does not address the undocumented costs of diminished VA reputation and the negative impact to our veterans’ physical and mental health and quality of life.

The Contractor shall provide Application Performance and Availability Monitoring Administration/Architecture support on a 24x7x365 basis. Perform discovery and requirements analysis for new system features, analysis of current system capabilities and limitations and design/engineer/integrate new system capabilities to meet requirements. Additionally, the Contractor shall provide Subject Matter Expert level support on application performance, desktop performance, and application availability and desktop availability monitoring and reporting using current VA monitoring tools such ServiceNow AppDynamics monitoring tools, Arcturus Applicare, Microsoft Bluestripe Factfinder, XPOlog Management, Aternity Desktop monitoring, and other performance monitoring tools as required. The Contractor shall also customize the monitoring interface to provide dashboards, monitoring and alerts for all monitored applications, document and test the customizations, and use monitoring diagnostic tools to find the root cause of performance issues. In addition, the Contractor shall assist Government and Contractor helpdesk staff on the use of monitoring diagnostic tools to pinpoint production issues, assist Government and Contractor staff on Root Cause Analysis based on data reported from the monitoring system, create and tune performance monitoring and availability alerts and, recommend policies and procedures on actions to take based on the alerts generated. Moreover, the contractor shall develop standards, policies and procedures around the use of application performance monitoring and availability in the pre-production and production environments, and provide all recommendations to the COR/VA PM(s) for approval prior to any implementation or changes; Participate in tuning and troubleshooting efforts; Work with customers to resolve performance and availability issues; Monitor subsystems using customized scripting appropriate to the monitoring tool being used; and, document operational steps and practices necessary to perform the assigned administrative duties.

The Contractor shall provide 24x7x365 SiteMinder Administration/Architecture support with the architecture, development, and integration of ServiceNow for the ITOPS IO customer. The Contractor conducts architectural analysis and review, application integration, provide guidance to developers, and ensure adherence to best practices as well as the customer's business objectives. IT infrastructure supports benefits systems for disability, education, and housing programs that process and disburse over $10B of monthly disability and education payments to Veterans and their families. Veterans Benefit Administration (VBA) IT core infrastructure systems are supported by approximately 50% IOSS contract staff, with some key areas of support being 100% provided by contract staff. Examples of benefits systems IOSS support include, but are not limited to, OpenAM and Siteminder. OpenAM is an access management system, which limits and allows which users may log into the system. If OpenAM is unavailable, users cannot log in to the benefits systems. Siteminder is a similar tool which limits and allows internal users to log in and troubleshoot systems. If Siteminder is unavailable, internal users cannot log in to the benefits systems and will not be able to access VA portals in order to: apply/manage benefits; renew prescriptions; file claims; and make necessary medical appointments.

Veterans Benefits Management System (VBMS), a critical VBA system, results in approximately $1M in lost overtime payments for each hour of downtime. In the event of an outage, 10,000 Regional Office staff would be unable to service Veterans.

If IOSS contractors are unavailable to support VBA systems the ability to respond to disruptions would be severely degraded and outage times would be extended. For each hour that the system is down, VA Claims Processors would be unable to process and determine the rate of disability for Veterans. If the VBMS system is unavailable, Veterans disability claims will not be processed resulting in the loss of service related disability income.

If there is a break in service for all IOSS support, there will be an impact to Veterans that rely on these services if the applications and/or servers become inoperable. This daily risk grows exponentially as the break in service continues.

VA is requiring a four-month extension until the October 6, 2021. However, VA is also including two, one-month option periods. VA requires the two, one-month option periods in the event the Government incurs delays with awarding the competitive TO and onboarding. The competitive TO includes management, operations, logistics, and technical support services for VA OI&T ITOPS. Task Execution Plans (TEPs) and the subsequent evaluation of these TEPs for such a large, enterprise-wide effort will be detailed and highly complex. TEPs will require multiple detailed technical reviews, and it is likely offeror clarifications/corrections will be required to ensure minimum requirements are met. In addition, in order to successfully transfer the magnitude of resources to support VA IO Critical Operations, on day one, would require the recruitment of approximately 700 plus contractors already interviewed and then technically accepted by VA IO. Recruitment effort can range from two to eight weeks per resource with more technical skillsets having extended timeframes beyond eight weeks. If recruiting all 700 resources simultaneously, VA would expect an extended recruitment timeframe due to the high volume and limited resource pool. If on-boarding a high volume of resources simultaneously, VA would expect this timeframe to be extended beyond the six weeks identified above. Due to Covid-19 restrictions, facility access is limited to accommodate fingerprint and PIV badging.

6. Efforts to Obtain Competition: Market research was conducted on December 21, 2020, details of which are in the market research section of this document. This effort did not yield additional sources that can meet the Government’s requirements and required timeframe. In accordance with FAR 5.301 and 16.505(b)(2)(ii)(D), the award notice for this action will be synopsized on the Federal Business Opportunities Page (FBO) and the justification will be made publicly available within 14 days of award.

7. Actions to Increase Competition: VA anticipates awarding a competitive TO on or about July 30, 2021 under the T4NG contract vehicle. Based on the complexities of this requirement as outlined in Section 3 and 5 of this Justification, VA has included option periods that would provide seamless support in the event that the July 30, 2021 award date cannot be met. However, if the aforementioned management, operations, logistics and maintenance and technical support services in support of VA, OI&T, ITOPS can be fully provided from the awardee of the competitive TO prior to exercising the option periods for seamless support, VA will not exercise the option periods and will transition to the competitive TO.

8. Market Research: The Government’s technical experts conducted market research November 2019 through April 2021 to ascertain the ability of any source to provide IOSS services, to include within in the required timeframe. On December 21, 2020, the Government posted a RFI on the Contract Opportunities site via beta.SAM.gov, requesting technical information necessary to determine capable vendors of providing the required services. In addition, Prime Contractors under T4NG were also notified of the RFI on beta.SAM.gov. On January 11, 2021, responses were received from 11 vendors to include nine large businesses: Amazon Web Services; Booz Allen Hamilton;

Cognosante MVH, LLC; GovernmentCIO, LLC; International Business Machine U.S Federal; ManTech MGS, Inc.; Real World Technologies, Inc; SRA International, Inc.;

Systems Made Simple, Inc.; DK Shin Laboratories; and one service disabled Veteran-owned small businesses (SDVOSB): Sierra7 Inc.

The responses were reviewed by VA subject matter experts from the OI&T, ITOPS, IO.

While sources were found that possess the general capabilities, no sources were found that can provide the support services in the required timeframe as outlined in Section 5 of this justification.

9. Other Facts: None.

10. Technical and Requirements Certification: I certify that the supporting data under my cognizance, which are included in this justification, are accurate and complete to the best of my knowledge and belief.

Date: see digital signature

Signature:

11. Fair and Reasonable Cost Determination: I hereby determine that the anticipated price to the Government for this contract action will be fair and reasonable based on a comparison with the Independent Government Cost Estimate and prices obtained through market research on same or similar services conducted by Government Technical experts.

Signature:

12. Procuring Contracting Officer Certification: I certify that this justification is accurate and complete to the best of my knowledge and belief.

Signature: _

Approval

In my role as procuring activity Advocate for Competition, based on the foregoing justification, I hereby approve the acquisition of IOSS on an other than fair opportunity basis pursuant to the authority cited in Section 4 above, subject to availability of funds, and provided that the property and services herein described have otherwise been authorized for acquisition.

Date: see digital signature Signature:_

Advocate for Competition Technology Acquisition Center

Office of Procurement, Acquisition, and Logistics