EOSDIS EED-3 Source Selection Statement 4-26-21 Final signed.pdf

Source Selection Statement For

Earth Observing System Data and Information System (EOSDIS) Evolution and Development (EED)-3

Solicitation Number 80GSFC20R0026

On March 29, 2021, I, along with senior officials from National Aeronautics and Space Administration’s (NASA) Goddard Space Flight Center (GSFC), met with members of the Source Evaluation Board (SEB) to hear findings based on the evaluation of the proposals for the Earth Observing System Data and Information System (EOSDIS) Evolution and Development (EED)-3 procurement. I held a follow-up meeting on April 1, 2021, with the same individuals to conclude the review of the findings.

PROCUREMENT DESCRIPTION

The EOSDIS EED-3 requirement was issued as a full and open competitive procurement to acquire services for the Earth Science Data Information System (ESDIS) Project. The ESDIS Project is responsible for the development, sustaining engineering, and operations of NASA’s

EOSDIS.

The EOSDIS ingests, processes, archives, preserves and distributes data from NASA’s Earth remote sensing satellites, as well as data from other government agencies and international partners. EOSDIS is one of the largest public science data information systems in the world, as well as one of the most heavily utilized. It is a comprehensive, distributed system. In addition to satellite data, it supports field campaign measurements, airborne data, in situ data, model data, ancillary products used for processing, and other related datasets. The system ingests, processes, and archives 32+ terabytes per day, and distributes over 1.8 billion products to over 4 million users per year. The ever-growing archive includes land, ocean, atmosphere, cryosphere, human dimensions, and calibrated radiance and solar radiance data types.

Pursuant to task orders issued by the Contracting Officer, the EED-3 contractor will provide services for the continued development and sustaining engineering of software, hardware, and cloud-based systems that provide science data management for the ESDIS Project as specified in the EED-3 Statement of Work (SOW).

The Request for Proposal (RFP) was released on September 11, 2020. Two amendments were issued to the RFP:

• Amendment 1 was issued on October 1, 2020, to revise solicitation clauses G.2 1852.216-76 Award Fee for Service Contracts (JUN 2018) and 52.204-24 Representation Regarding Certain Telecommunications and Video Surveillance Services or Equipment (AUG 2020); to provide clarification on the Proposal Content and Page Limitations section of the Past Performance Volume; and to revise Exhibit 13 EED-3 Past Performance Questionnaire.

• Amendment 2 was issued on February 19, 2021, to revise solicitation provision L.16 GSFC 52.215-223 COST VOLUME INSTRUCTIONS (JUL 2020), (b) Cost Volume Proposal Format.

The resultant contract will be a Cost-Plus-Award-Fee (CPAF) Indefinite-Delivery/Indefinite- Quantity (IDIQ) task order contract with an effective ordering period of five (5) years, with no option periods. A 90-day phase-in period will be accomplished under a separate Firm-Fixed- Price contract vehicle issued at contract award.

The North American Industry Classification System (NAICS) code for this acquisition is 541512 and the small business size standard is $30.0M in average annual receipts. The maximum ordering value inclusive of fee for this procurement is $275M.

PROPOSALS SUBMITTED

The following two (2) companies submitted timely proposals by October 14, 2020:

Science Applications International Corporation (SAIC) Raytheon Company, Raytheon Intelligence & Space (Raytheon)

The RFP stated that the Government intended to evaluate proposals and award the contract without discussions with Offerors. The RFP also stated that the Government reserved the right to conduct discussions if the Contracting Officer (CO) later determined them to be necessary. After evaluating proposals, the CO found that establishment of a competitive range was not necessary, a determination with which I concurred.

EVALUATION PROCEDURES

As NASA’s Source Selection Authority (SSA) for this procurement, I appointed the SEB, which included a team of technical and business members and consultants from appropriate disciplines, to assist in proposal evaluations.

The EED-3 SEB evaluated proposals in accordance with the source selection procedures identified in Federal Acquisition Regulation (FAR) part 15.3 "Source Selection" and NASA FAR Supplement (NFS) 1815.3. The SEB procedures at NFS 1815.370, NASA Source Evaluation Boards, were applied.

The RFP listed three (3) evaluation factors for award: Mission Suitability, Cost, and Past Performance. The RFP specified the relative order of importance of the evaluation factors as follows:

The Cost Factor is significantly less important than the combined importance of the Mission Suitability Factor and the Past Performance Factor. As individua l factors, the Cost Factor is less important than the Mission Suitability Factor but more important than the Past Performance Factor.

The RFP established that only the Mission Suitability Factor would be point scored in the evaluation process. The Mission Suitability Factor consisted of the following three (3) subfactors with assigned points as indicated.

Mission Suitability Factor Points

Subfactor A Technical Approach 500

Subfactor B Management Approach 400

Subfactor C Small Business Utilization (SBU) 100 Total 1000

The Mission Suitability Factor was evaluated using the adjectival ratings, definitions and percentile ranges in NFS 1815.305(a)(3)(A). The SEB first reviewed each proposal and generated consensus findings (i.e., significant weaknesses, weaknesses, strengths, and significant strengths) within each subfactor. Next, the SEB considered the findings under each subfactor and applied the adjectival rating definitions set forth in NFS 1815.305(a)(3)(A) to assign a rating based on the merit of the proposal within that subfactor. After assigning the adjectival rating for the subfactor, the SEB determined the associated percentile score with the range set forth in NFS 1815.305(a)(3)(A). The maximum points available for each subfactor were then multiplied by the assigned percent for each subfactor to derive the score for the particular subfactor. For example, if a subfactor has a possible 200 points and receives a percent rating of 80, then the score of that subfactor would be 160 points.

Regarding the Cost Factor, the proposed costs of the Government Pricing Model (GPM), and the rates proposed in Attachment B, DIRECT LABOR RATES, INDIRECT RATES AND FEE MATRICES, were assessed to determine reasonableness and cost realism. As stated in the RFP, the cost evaluation was conducted in accordance with FAR 15.305(a)(1) and NFS 1815.305(a)(1)(B). Offerors were referred to FAR 2.101(b) for a definition of “cost realism” and to FAR 15.404-1(d) for a discussion of “cost realism analysis” and “probable cost.” The proposed costs, including direct labor, other direct costs, and indirect rates, were assessed to determine reasonableness and cost realism. Both the “proposed and probable” costs reflected the Offeror’s proposed fee amount. Proposed fee was not adjusted in the probable cost assessment.

The proposed and probable costs were presented to the SSA, along with any issues and risks associated with the proposed costs.

For the Past Performance Factor, the Past Performance evaluation was conducted in accordance with FAR Part 15 and provision M.5 of the solicitation. Each Offeror’s contract references were evaluated to determine the degree of relevance based on size, content, and/or complexity. In evaluating Past Performance, the SEB relied on the Government-wide Contractor Performance Assessment Reporting System (CPARS) database and customer feedback, in addition to the narrative on relevant past/current contracts provided by the Offerors. The Past Performance factor was not point scored, but was assigned an adjectival rating of “Very High Level of Confidence,” “High Level of Confidence,” “Moderate Level of Confidence,” “Low Level of Confidence,” “Very Low Level of Confidence,” or “Neutral.”

MISSION SUITABILITY EVALUATION

The Mission Suitability factor has three subfactors as follows: Subfactor A, Technical Approach;

Subfactor B, Management Approach; and Subfactor C, Small Business Utilization.

Each subfactor was evaluated in accordance with the weights delineated in the RFP. The table below provides the adjectival ratings assigned in each Mission Suitability subfactor for the two

(2) EED-3 proposals.

Offeror Subfactor A Subfactor B Subfactor C Mission Suitability Points

SAIC Good Good Excellent 667

Raytheon Excellent Good Excellent 849

The substance of the SEB’s evaluation of Mission Suitability for all Offerors is presented below.

SAIC

Under Subfactor A, Technical Approach, SAIC received an adjectival rating of “Good,” with no Significant Strengths, four (4) Strengths, three (3) Weaknesses, no Significant Weaknesses, and no Deficiencies.

SAIC received four (4) Strengths in the following areas: (1) a holistic approach to data disaster recovery and data access optimization that could result in a more effective and efficient use of cloud storage; (2) proposing an innovative use of employing and harnessing the power of Machine Learning (ML) in the creation and operation of an intelligent multi-cloud data lake; (3) recognizing the primary importance of NASA compliance, security hardening and governance provided by the NASA-compliant General Application Platform (NGAP) for Amazon Web Services (AWS), and determining the NGAP equivalent compliance, governance, and security functionality needed for YouCloud; and (4) proposing the use of computer-based, remotely-deliverable, customized and highly focused Microlearning modules that could prove useful in helping the EOSDIS end-user community learn and assimilate new tools, services, and methods of interacting with data in the cloud. These Strengths enhance the potential for successful contract performance.

SAIC received three (3) Weaknesses in the following areas: (1) creating a new set of common Application Programming Interfaces (APIs) in order to integrate YouCloud into the Earthdata Cloud (EDC) introduces complexity and increases the risk of an inefficient solution; (2) proposing solutions addressing the challenge of avoiding/minimizing inter-cloud egress costs without adequately detailing or discussing: how implementation approaches minimize impacts to existing EDC components, an assessment of key risks and mitigation strategies, and the specifics of the proposed future architecture; and (3) proposing technical approaches for four of the six identified biggest challenges to EOSDIS end users, without providing adequate detail to fully assess the potential impacts, effectiveness, reasonableness or efficiency. These Weaknesses increase the risk of unsuccessful contract performance.

Under Subfactor B, Management Approach, SAIC received an adjectival rating of “Good,” with no Significant Strengths, two (2) Strengths, no Weaknesses, no Significant Weaknesses, and no Deficiencies.

SAIC received two (2) Strengths in the following areas: (1) proposing an effective, detailed, well developed, Phase-In Plan that would provide an orderly transition of ongoing EED-2 tasks and personnel, processes, as well as relocation of the EOSDIS Core System (ECS) Development Facility (EDF) data center; and (2) integrating Large Solution SAFe into its management organization in a manner that increases effectiveness of Mission X development in the new multi-cloud environment by increasing efficiency of coordination, sequencing, and timely integration of work from multiple sources into solution capabilities through the creation of cross-functional teams. These Strengths enhance the potential for successful contract performance.

Under Subfactor C, Small Business Utilization, SAIC received an adjectival rating of “Excellent,” with one (1) Significant Strength, no Strengths, no Weaknesses, no Significant Weaknesses, and no Deficiencies.

SAIC received one (1) Significant Strength for proposing an excellent, comprehensive Small Business Subcontracting Plan that exceeds government recommended goals (GRGs) for all individual Small Business socioeconomic categories in the subcontracting plan. Additionally, SAIC is committed to providing high technology work via enforceable agreements, and proposes to informally mentor one of its proposed team members. The Small Business Utilization approach of SAIC greatly enhances the potential for successful contract performance.

RAYTHEON

Under Subfactor A, Technical Approach, Raytheon received an adjectival rating of “Excellent,” with two (2) Significant Strengths, nine (9) Strengths, one (1) Weakness, no Significant Weaknesses, and no Deficiencies.

Raytheon received two (2) Significant Strengths in the following areas: (1) proposing an exceptional approach detailing a comprehensive architecture that includes necessary future functionality (new and extended), components, primary interfaces and data/control flows, as well as how external entities (users, Distributed Active Archive Centers (DAACs), Mission Systems and other external systems and services) interface with the overall architecture while ensuring that existing Amazon Web Services (AWS) cloud applications, interfaces, tools and procedures are minimally impacted by the addition of a new data lake from any cloud vendor which significantly reduces implementation risk; and (2) proposing an excellent, holistic approach to cloud controls to ensure that all operational use of cloud resources is actively monitored and constrained in accordance with policy and cloud resource usage plan (budgetary) thresholds established by NASA and ESDIS in a manner that significantly reduces the risk of utilizing cloud resources beyond established usage thresholds. These Significant Strengths greatly enhance the potential for successful performance.

Raytheon received nine (9) Strengths in the following areas: (1) proposing an effective, efficient and reasonable approach to cloud-agnostic development and deployment that utilizes proven technology and would allow data ingest, data archive, data distribution, data transformation and data egress monitoring/throttling capabilities to be implemented in new Cloud Service Providers (CSPs) without interruption to any current users/missions relying on AWS; (2) proposing to utilize the power of Artificial Intelligence and Machine Learning (AI/ML), an innovative and fundamental technology with numerous applications and benefits; (3) proposing an approach to streamlining multi-vendor cloud cost modeling that results in uniform formatting, increased data accuracy, and cloud-specific reports that support the budgetary planning needs of NASA financial managers while also reducing the burden on DAACs and other cloud account developers; (4) proposing a comprehensive set of recommendations to take advantage of opportunities provided by a multi-cloud environment to optimize data access and Data Disaster Recovery (DDR) operations beyond what is possible in a single cloud environment; (5) identifying an automated scalable cyber security compliance testing approach to ensure secure deployments from DAAC Continuous Integration (CI)/Continuous Delivery (CD) systems across multiple cloud environments, while providing efficient and effective security testing and minimizing the risk of potential security threats and vulnerabilities during deployment; (6) proposing an approach to monitor, assess and visually illustrate several emerging technology trends, topically grouped and ranked by Technology Readiness Level (TRL), that clearly lays out low-risk near-term technologies to consider adopting, and higher-risk mid-term opportunities to pursue; (7) utilizing quantitative and qualitative methods to measure, track, and evaluate the effectiveness of their technical approaches to addressing the biggest current challenges facing EOSDIS end users and application developers; (8) proposing the use of a database which provides an efficient solution to the challenges application developers, as well as EOSDIS end users, routinely face with data access, data preparation, and applications development; and (9) identifying top end-user challenges and proposing concise and well-articulated approaches to addressing these challenges in a manner that prioritizes optimizing functional and performance benefits, reduces inefficiencies and operational costs, and improves the overall end user experience. These Strengths enhance the potential for successful contract performance.

Raytheon received a Weakness for identifying development tools and documentation as one of the biggest current challenges to application developers leveraging EOSDIS without adequately describing how their approach addresses the long-term maintenance and management of existing and proposed guides, documentation, templates, and code examples. This Weakness increases the risk of unsuccessful contract performance.

Under Subfactor B, Management Approach, Raytheon received an adjectival rating of “Good,” with no Significant Strengths, two (2) Strengths, no Weaknesses, no Significant Weaknesses, and no Deficiencies.

Raytheon received two (2) Strengths in the following areas: (1) proposing an effective, detailed, well developed, Phase-In Plan, that would provide an orderly transition of ongoing EED-2 tasks and personnel in less than the required 90 days and ensure the continuity of current EED tasks;

and (2) building its program management and organizational structure centered around a tailored variation of the SAFe 5.0 Large Solution process in a manner that leverages the SAFe structure to foster more effective and efficient communication, collaboration, and subcontractor integration across multiple diverse organizations to ensure successful identification and execution of Mission X requirements and priorities and minimizes potential risks while maintaining schedule and performance. These Strengths enhance the potential for successful contract performance.

Under Subfactor C, Small Business Utilization, Raytheon received an adjectival rating of “Excellent,” with one (1) Significant Strength, no Strengths, no Weaknesses, no Significant Weaknesses, and no Deficiencies.

Raytheon received one (1) Significant Strength for proposing an excellent, comprehensive Small Business Subcontracting Plan that significantly exceeds GRGs for all individual Small Business socioeconomic categories in the subcontracting plan. Additionally, Raytheon is committed to providing high technology work via enforceable agreements, and is an active participant in the NASA Mentor-Protégé program and plans to continue the participation in the program with a new mentee. The Small Business Utilization approach of Raytheon greatly enhances the potential for successful contract performance.

COST EVALUATION

The SEB evaluated the total proposed cost and determined the Government’s probable cost for both Offerors. The total proposed cost of Raytheon ($213,552,906) was slightly lower than the total proposed cost of SAIC. The SEB made the following probable cost adjustments: 1) For the SAIC proposal, the SEB adjusted the direct labor rates for three (3) labor categories resulting in an upward probable cost adjustment; and 2) For the Raytheon proposal, the SEB adjusted the direct labor rates for three (3) prime and three (3) subcontractor labor categories resulting in an upward probable cost adjustment. Based on the SEB’s evaluation results, Raytheon had the lowest total probable cost ($216,617,843), which was very slightly lower (approximately 0.05%) than the total probable cost for SAIC.

PAST PERFORMANCE EVALUATION

In evaluating the Past Performance of the Offerors, the SEB determined the Overall Past Performance Level of Confidence Rating for SAIC to be "Very High" based on the combination of “Very High” Overall Relevance and “Very High” Overall Performance. Raytheon was determined to have an Overall Past Performance Level of Confidence Rating of "High" based on the combination of “Very High” Overall Relevance and “High” Overall Performance.

SELECTION DECISION

As the SSA for this procurement, I carefully reviewed the SEB's documentation entitled "Earth Observing System Data and Information System (EOSDIS) Evolution and Development (EED)-3 Procurement Presentation to the Source Selection Authority” and the associated Cost Report. On March 29, 2021, I met with the EOSDIS EED-3 SEB, along with several ex-officio members, to receive the findings of the SEB. I held a follow-up meeting on April 1, 2021, with the same individuals.

I determined that the findings presented by the SEB, as documented in its presentation and the accompanying "EOSDIS EED-3 Cost Evaluation Report" were detailed and consistent with the evaluation criteria in the EED-3 RFP, and provided a clear description of the merits of each proposal. I discussed with the SEB its rationale for the findings, adjectival ratings and scores for the Mission Suitability Subfactors, and also discussed the rationale for the evaluation of Cost and Past Performance. Further, I solicited the views of ex-officio members in their areas of expertise.

I determined that the findings were reasonable and valid for the purpose of making a selection decision based on initial proposals. Thus, I accept the SEB's findings and concur with the Contracting Officer that a competitive range and discussions are not necessary.

In determining which proposal offered the best value to NASA, I referred to the relative order of importance of the three evaluation factors as specified in the RFP:

The Cost Factor is significantly less important than the combined importance of the Mission Suitability Factor and the Past Performance Factor. As individua l factors, the Cost Factor is less important than the Mission Suitability Factor but more important than the Past Performance Factor.

My selection was based on the comparative assessment of each proposal against each of the RFP evaluation factors to determine which proposal represented the best value.

Regarding the Mission Suitability factor overall, I found a significant discriminator in Subfactor A, Technical Approach, as Raytheon received an Excellent rating while SAIC received a Good rating. Adjectival ratings (Good) and scores for Subfactor B, Management Approach, were equal for each Offeror. Both Offerors received Excellent adjectival ratings for Subfactor C, Small Business Utilization, with nearly equal point scores.

Within Mission Suitability Subfactor A, Technical Approach, I noted that Raytheon had two (2) Significant Strengths, nine (9) Strengths, and one (1) Weakness. SAIC had no Significant Strengths, four (4) Strengths, and three (3) Weaknesses.

I noted that Raytheon received its first Significant Strength for proposing an extremely comprehensive architecture that included necessary future functionality (new and extended), components, primary interfaces and data/control flows while only minimally impacting existing Amazon Web Services (AWS) cloud applications, interfaces, tools and procedures. I found that Raytheon’s overall excellent approach to developing its architecture significantly decreases the potential for technical risk and dependencies on vendor-specific cloud services, which greatly enhances the potential for successful performance. Raytheon received its second Significant Strength for its outstanding, holistic approach to cloud controls while ensuring that all operational use of cloud resources is actively monitored and constrained in accordance with policy and cloud resource usage thresholds established by NASA and ESDIS. I found that Raytheon’s proposed approach to address cloud security and operational cloud resource usage control challenges has the potential to lead to more efficient long-term system/software maintenance/enhancement activities with lower time, personnel, and cost implications. In addition to the two Significant Strengths, I noted that Raytheon received noteworthy Strengths for its effective, efficient and reasonable approach to cloud-agnostic development and deployment in response to Scenario A.1; and for identifying top end-user challenges and proposing concise and well-articulated approaches to addressing these challenges in a detailed and comprehensive manner. I found these Strengths will potentially reduce inefficiencies and improve operational performance, reduce end-user workloads, and provide a more consistent and improved end-user experience, thereby enhancing the potential for successful performance.

These features noted above will contribute toward exceeding or greatly exceeding the contract requirements in a manner that provides additional value to the Government. I acknowledge Raytheon’s Weakness for not adequately describing an approach for addressing the long-term maintenance and management of existing and proposed guides, documentation, templates, and code examples. However, I did not find this Weakness to significantly detract from its overall excellent technical approach in Subfactor A.

I noted that SAIC received a “Good” rating, with no Significant Strengths, four (4) Strengths, and three (3) Weaknesses. I noted that SAIC received Strengths for 1) data disaster recovery; 2) use of Machine Learning; 3) YouCloud NGAP compliance; and 4) Microlearning. While I found that these Strengths will enhance the potential for successful performance, I noted that SAIC’s Weaknesses for 1) unnecessary API changes; 2) inadequate approach for avoiding/minimizing inter cloud egress costs and proposed architecture; and 3) inadequate detail addressing four of the six identified biggest challenges to EOSDIS end users, detracted from its overall response to Subfactor A.

Overall for Subfactor A, Technical Approach, I found a clear and distinct advantage in favor of Raytheon with its “Excellent” adjectival rating compared to the “Good” adjectival rating of SAIC. In comparing the findings for SAIC and Raytheon, I found meaningful discriminators with the two Significant Strengths assigned to Raytheon, whereas SAIC received no Significant Strengths. Specifically, Raytheon’s excellent response detailing a comprehensive EED-3 architecture and a holistic approach to cloud controls; in addition to its approach to cloud-agnostic development and deployment, and identifying top end-user technical challenges was extremely beneficial and demonstrates the ability of Raytheon to successfully conduct and perform the work of the development and sustaining engineering of hardware, software, and cloud based systems that comprise the EOSDIS. By comparison, SAIC received Weaknesses for its proposed EED-3 architecture and approach to addressing end-user challenges with no finding for cloud-agnostic development and deployment. Therefore, overall for Subfactor A, I found that the advantages of the Raytheon proposal, particularly the two Significant Strengths and Strengths noted above, provided significantly more value than SAIC’s proposal.

Within Mission Suitability Subfactor B, Management Approach, I noted that both SAIC and Raytheon received two (2) Strengths. I found that each Offeror received similar Strengths for the integration of SAFe into its program management and organization structure, and for comprehensive and well-developed Phase-In Plans. Overall, I found these Strengths to be equally beneficial and therefore found no meaningful discriminator between SAIC and Raytheon in this subfactor.

Within Mission Suitability Subfactor C, Small Business Utilization, I noted that SAIC and Raytheon each received an “Excellent” adjectival rating for this subfactor with each having one Significant Strength. I found these Significant Strengths to be equally beneficial and therefore found no meaningful discriminator between SAIC and Raytheon in this subfactor.