CBL-N2_ConOps_Final_05222024.docx

DOCX document 994 KB Posted

Attached to
FS3i-RFI
Solicitation number
FS3i-TBD
Issued by

About this file

This document is a Concept of Operations (ConOps) for the N2 modernized hybrid architecture, which is a mission-critical system for the Department of Defense (DoD) and Intelligence Community (IC) that provides for the composition, compilation, maintenance, hosting, and dissemination of Maritime Safety of Navigation Publications.

The ConOps outlines the high-level functional requirements and specifications for transitioning the legacy NSS system to a more modernized solution with process management automation. Key details include the system architecture, development approach, and integration of components such as FastAPI, Kong, Helm Charts, GEOAxIS authentication, KeyCloak, OAuth, and PostgreSQL database. The document also covers the cloud deployment strategy, software baselines, and interfaces with external systems like the Enterprise Engine (E2) and Maritime Safety Information (MSI) Portal. The final versions of the N2 ConOps, Requirements Document, User Guide, and Maintenance Guide are scheduled for delivery on May 22, 2024.

View the file

Other files

File Type Posted
RFI_QA_20241003.xlsx XLSX spreadsheet
CBL-SFN_Overview_Trifold_FINAL_v3.pdf PDF
CBL-FS2_TanDEM_X_Addendum_V1.docx DOCX document
CBL-IPF_S3_Object_Lifecycle_Management_(2023-05-17).docx DOCX document
CBL-IPF_EBS_Encryption_(2023-08).docx DOCX document
Science_Task_Order_20240917.docx DOCX document
CBL-SMAPS_V2.pptx PPTX presentation
CBL-Maritime_Pub_SMAPS_QASP.docx DOCX document
CBL-IPF_Developer_Environment_Guidance.docx DOCX document
CBL-IPF_EBS_Provisioned_IOPS_and_Throughput_(2023-06-22).docx DOCX document
CBL-SFN_APNTt_Placemats_FINAL_v2.pdf PDF
CBL-The_NGA_Software_Way_2022.pdf PDF
CBL-GNSS_As-Is_and_Aspirational_Endstate.pdf PDF
CBL-GGEPS_As-Is_and_Aspirational_Endstate.pdf PDF
CBL-GGPM SvcV-2 Diagram.jpg JPG image
CBL-IPF_Orphaned_EBS_Volume_Management_(2023-06-22).docx DOCX document
CBL-NGA_Technology_Strategy_APR_20-512.pdf PDF
CBL-IPF_Capability_Delivery_SOP--Oct2021.docx DOCX document
CBL-J-03_FS2_EM_TO_Geomatics_VDA_Addendum.docx DOCX document
CBL-Data_Conditioning_SvcV-2_Diagram.jpg JPG image
CBL-IPF_Tagging_Standards_(2023-05-17).docx DOCX document
CBL-GSS_As-Is_Architecture.png PNG image
Sea_Task_Order_Rev1_20240829.docx DOCX document
Base_PWS_Rev1.3_20240903.docx DOCX document
FS3i_RFI_20240917.docx DOCX document
Show all 25

On GovTribe

Work with this file

  • Download the original file
  • Contacts named in this file
  • Similar government files
  • Ask GovTribe AI about this file

Text version

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

7500 GEOINT DRIVE

SPRINGFIELD, VA 22150

Sustainment for Content Services (SCS)

N2 Concept of Operations (ConOps)

Contract HM0476-19-C-0035

22 May 2024

Revision History

Date
Description
Version
Author
July 2022
Initial Draft
1
Charles Thilman
Oct 2022
Review and Annotation
2
Charles Thilman
Nov 2022
Scrub and Update
3
Charles Thilman
Dec 2022
SCS Draft Review
4
Charles Thilman
Jan 2023
Incremental Delivery 1
5
Mathais R. Rivera
May 2023
Scrub and Update
6
Charles Thilman
Jun 2023
SCS Draft Review
7
Charles Thilman
Jun 2023
Incremental Delivery 2
8
Mathais R. Rivera
Oct 2023
Scrub and Update
9
Charles Thilman
Oct 2023
SCS Draft Review
10
Charles Thilman
Nov 2023
Incremental Delivery 3
11
Mathais R. Rivera
Feb 2024
Scrub and Update
12
Charles Thilman
Mar 2024
SCS Draft Review
13
Charles Thilman
May 2024
Final Review
14
Mathais R. Rivera
May 2024
Final Delivery
15
Joseph Dominguez

UNCLASSIFIED

**NOTE: All sections are in a non-static state and will be revised as systems mature**

Table of Contents

1.Documentation Overview6
1.1.General Overview6
1.2.Introduction6
1.3.Purpose6
1.4.Mission Overview7
1.5.System Overview8
2.N2 Process Overview9
2.1.Architectural Overview9
2.2.N2 Development Overview10
2.3.N2 Openshift Architecture Overview10
2.4.N2 Componentry11
2.4.1.CORE Red Hat OpenShift11
2.4.2.Fast API11
2.4.3.Kong11
2.4.4.Helm Charts11
2.4.4.1.Chart YAML12
2.4.4.2.Values YAML12
2.4.4.3.Charts Directory12
2.4.4.4.Templates12
2.4.5.GEOAxIS Authentication12
2.4.6.KeyCloak12
2.4.7.OAuth12
2.4.8.Database Components13
2.4.8.1.Database Overview13
2.4.8.2.Database Transformation Workflow13
2.4.8.3.Extract Transform Load (ETL) Database Process14
2.4.8.4.Supporting Dual Operations via Continuous ETL15
2.4.8.5.PostgreSQL17
2.4.8.6.Crossplane17
2.4.8.7.PostGIS Plug-In17
2.4.8.8.pgAdmin18
2.4.9.Appian Workflows18
2.4.9.1.Training18
2.4.9.2.Development18
2.4.10.FrameMaker Server19
3.Cloud Deployments19
3.1.1.N2 Instances by Domain19
3.1.2.Uranium to Titanium Transfer19
3.1.3.End of Development Transfer20
3.1.4.System Configuration20
3.1.5.Network Diagram20
4.Software Baseline21
5.Interfaces and Services23
5.1.Enterprise Engine (E2)23
5.2.Martitime Safety Information (MSI) Portal24
6.Notes (AV-2 Integrated Dictionary)24
6.1.1.Glossary24
6.1.2.Acronyms and Abbreviations25
6.1.3.Endnote Links26

List of Tables and Figures

Table 1: Information System Primaries9
Figure 1: N2 Developmental Breakout10
Figure 2: N2 Phase II OpenShift Architecture10
Figure 3: High level authentication overview13
Figure 4: Detailed N2 Database component interaction13
Figure 5: High level ETL Process viewpoint14
Figure 6: Transitory ETL configuration within N2 environment15
Figure 7: Post Transition Architecture20
Table 2a: (U) N2 Desktop Baseline21
Table 2b: (U) N2 Server Baseline21
Table 2c: (U) N2 Microservices Part 121
Table 2d: (U) N2 Microservices Part 222
Table 3: Proposed N2 Win10 software baseline (as of 29 Jan 2024)23
Figure 8: Publications Dissemination to MSI workflow24

1. Documentation Overview General Overview The purpose of this document is to outline and define the high-level functional requirements and specifications of the N2 modernized hybrid architecture as developed in the NGA Common Operations Release Environment[endnoteRef:1] (CORE) and IC Commercial Cloud Services cloud environments. [1: Building better, faster software at the CORE of this name change | NGA Today]

Items which exceed classification limitations of CUI[endnoteRef:2] systems will be flagged on their paragraph header with the annotation HSA or high-side addendum. The appropriate documentation will be stored in the classified SCS document repository located in the NGANet enclave. [2: Handling information marked CUI | NGA Today]

This document shares its basic outline with other Maritime related system ConOps documents listed within IPF’s Confluence webpages for SF systems and subsystems. Where at all possible, these future documents will adhere to this base structure as closely as possible within reason to maintain logical and representational consistency between relative systems.

SCS CDRLs Final Versions N2 Concept of Operations (ConOps): May 22, 2024 N2 Maritime Requirements/Description Document: May 22, 2024 N2 User Guide: May 22, 2024 N2 Maintenance Guide: May 22, 2024

Introduction The legacy NSS system was systematically and sequentially enhanced starting with the client-side services being transitioned from a monolithic desktop application to containerized microservices deployed in the CORE KubIC environments. The backend functions were redeveloped as decoupled services that integrate with the enhanced front end and a newly implemented workflow manager framework to support functional consolidation and service automation.

Purpose SCS was tasked with enhancing the legacy on-premises NSS system from a non-compliant, at-risk 32-bit architecture to a more modernized solution with process management automation. This was achieved as a phased engineering effort by deploying a newly developed 64-bit codebase as microservices in the cloud to maximize scalability, redundancy, and maintainability through ongoing operations and sustainment efforts. The Appian BPM was implemented to facilitate role-based publication processing, workflow automation, and endpoint/service integration to leverage ESS components such as GEOAxIS EMS enterprise services. Cost efficiencies were achieved through customized open source and ESS component conversions to include Oracle to PostgreSQL database, Esri ArcSDE to PostGIS geospatial data, and the functional replacement of Oracle Fusion Middleware with enterprise licensed services.

The first steps in moving towards an enhanced environment was to identify relevant existing legacy programming and convert the necessary components as specified by the Maritime Safety Office, SFH. During the first phase, the legacy client-side services were rewritten and deployed as decoupled, modular microservices to more easily update, modernize, and streamline utilization and maintenance of these programs going forward. This effort was initially deployed in the IC Titanium cloud environment. Additional phase I efforts included upgrading the client-side code to support the Windows 10 (Win10) operating system, an upgrade to FrameMaker 2019, and latest versions of all COTS software required for the N2 baseline image.

The second phase involved the conversion of the monolithic backend to a multifaceted solution including microservice development and deployment, Appian BPM workflow development and service implementation, ESS component integration, and 32-bit to 64-bit software library and plugin updates. N2 microservices are now deployed in the CORE as part of the agency’s strategic goals to standardize software development and integration – The NGA Software Way.

Mission Overview N2 is a mission critical system available to the Department of Defense (DoD) and Intelligence Community (IC). NSS is maintained by the Integrated Program Office – Foundation (IPF) on behalf of the NGA Maritime Safety Office (SFH). N2 provides for the composition, compilation, maintenance, hosting, and dissemination of Maritime Safety of Navigation Publications.

A weekly unclassified publication since 1869, Notice to Mariners (NtM) provides critical SON updates to all NGA, NOAA, and USGC SNC and publications to include a weekly list of lights update, DLA catalog corrections, and a monthly classified version. Each weekly NtM includes three primary sections providing timely marine safety information for the correction of all US Government navigation charts and publications from a wide variety of sources, both foreign and domestic to ensure the safety of life at sea (SOLAS).

The US Notice to Mariners corrects NGA and National Ocean Service (NOS) charts using information collected from many sources, among them the Local Notice to Mariners published by the nine US Coast Guard Districts. The US Notice to Mariners will contain only those chart corrections of interest to ocean-going vessels. It may also be referred to as Marine Information Reports and is a part of surface navigation mission subset.

In addition, the Classified NtM (CNtM) provides timely marine safety information for updating all classified USG navigation charts and publications. The CNtM is composed monthly from various sources on a monthly basis and provides critical updates for the correction of nautical charts and the DLA Catalog of Hydrographic Products.

When producing an NtM, the following portions need to be accounted for with NSS and its components. Along with Front / Back Matter, Front Cover, Important Information, Hydrogram, Explanation of Contents, Marine Information Report and Suggestion Sheet, Geographic Locator, and Back Cover, the typical NtM is broken into the following sections:

Section I Chart Corrections: I-1.1, Chartlets / Depth Tabulations / Notes: I-2.1, Charts Affected by Notice to Mariners: I-3.1

Section II NGA/DLIS Catalog Corrections: II-1.1, Navigation Publications Corrections: II-2.1, USCG Light List Corrections / Radiobeacon Corrections: II-3.1, NGA List of Lights / Radiobeacon / DGPS Corrections: II-4.1, Publications Affected by Notice to Mariners: II-5.1

Section III Broadcast Warnings: III-1.1, Marine Information: III-2.1

In addition, there are five categories of Publications. A given document may be constructed from one of these five categories and include:

Category 1 – Content that is automatically generated based on Hydrographic Feature Layer (HFL) extractions, such as the tables in the List of Lights.

Category 2 – Content that is feature supported, where features in the HFL are mentioned in the publication, but the text will not be automatically generated, such as the Sailing Directions.

Category 3 – Content that is supported by databases other than the HFL, such as the World Port Index.

Category 4 – Content that is algorithmically derived, such as the Sight Reduction Tables

Category 5 – Content not tied to specific features in the HFL, such as the American Practical Navigator.

N2 also generates other products including:

Quarterly Publication DVD

Broadcast Warning Messages (NAVAREA IV, NAVAREA Xll, HYDROARC, HYDROLANT)

Daily Memorandum

Worldwide Threats to Shipping (via MSI website only)

Related miscellaneous data / products (via MSI website only) System Overview

(U) The following individuals are identified as the N2 system owners or functional proponent/advocate for this system as documented during the registration process for both test and operations accounts:

Table 1: Information System Primaries N2 Process Overview Architectural Overview Phase I architecture was originally developed in an AWS hosted framework. In an effort to save costs and more properly align our modernization efforts with the over-arching aims of NGA’s Strategy 2025 and The NGA Software Way all N2 custom microservices are now hosted in the CORE KubIC OpenShift environment.

One area which was addressed to do this was the reconfiguring of microservices away from Chalice and Lambda and into the OpenShift environment. As a framework for writing serverless applications, Chalice certainly met the need under AWS, but CORE’s deployment model provides several advantages and efficiencies for seamless pipeline builds, integration, test, and deployment.

The initial N2 enhancement involved the migration of client-side code execution as a desktop application to a modern cloud-based microservices suite. The microservices are fronted by web-accessible RESTful APIs deployed in the CORE environment. Integrating the N2 front-end with the enhanced cloud architecture eliminates most of the software dependencies presently hindering routine OS, security, and COTS application upgrades on the N2 Workstations. The modernized N2 system is engineered in accordance with intelligence oversight, records management, security compliance, ICD 503, and other required and recommended NGA policies, procedures, and best practices.

N2 Development Overview

Figure 1: N2 Developmental Breakout N2 Openshift Architecture Overview

Figure 2: N2 Phase II OpenShift Architecture

1. Developer accounts are required to create, modify, or delete the publication workflows.

2. User accounts provide read, write, and execute permissions for each role-based function.

3. Kong provides API gateway management and routing for the Appian WFM UI.

4. GEOAxIS provides authentication and access management EMS services

5. The workflow manager (WFM) UI will call out to the microservices using internal OpenShift routes. These routes will not be callable via Kong.

6. The FrameMaker microservices are deployed on a Windows VM as python microservices that interact directly with FrameMaker to create a frame file convert frame to PDF and convert frame to post script space.

7. The microservices deployed as these pods will do everything that does not require FrameMaker.

8. PostgreSQL will be deployed and will manage the database while providing the high availability, disaster recovery, and monitoring.

9. The microservice pods will query a PostgreSQL database using internal routes to grab the information needed to generate the necessary SGML documents.

10. Database developers and administrators utilize pgAdmin to call on PostgreSQL within OpenShift Environment (OSE). OAuth by way of CAC provides the necessary authorizations to access.

11. PostgreSQL will be deployed via CORE provided PostgreSQL. This will be an AWS RDS with PostGIS extension. PostgreSQL reaches back to Amazon RDS services (and vice versa) using Crossplane.

N2 Componentry CORE Red Hat OpenShift Within the Phase I structure Chalice was used to create a lambda function for the environment. Now, going forward with Phase II, Fast API is being utilized by SCS to implement the microservice framework.

Fast API FastAPI is a python library the SCS development team is using to create a Rest API on our behalf so the team does not waste work cycles recreating solutions that already pre-exist. This allows the developers to focus the majority of their efforts on writing new and refined code that is specific to the N2 environment.

Kong Kong Enterprise is deployed with a Workspaces implementation. Workspaces enable admins to easily organize their teams, services, and environments into groups. Administrators can then implement policies in their Workspaces which span across singular or multiple teams. Admins can also ensure compliance by restricting or granting access to specified services and data. Additionally, admins can view analytics for the workspace to isolate any issues.

Helm Charts The files and directories of a Helm chart each have a specific function. There are two top-level files and two supplementary sub-directories. The four main components that are required for a Helm Chart to be executed are as follows: a Chart.yaml, a Values.yaml, a Charts Directory, and Templates (Directory).

Chart YAML Your Chart.yaml has required information describing what the Chart is about, with a name and version. There is more information that can be included in the Chart.yaml, but that is the minimum.

Values YAML The Values.yaml file is where you define values to be injected/interpreted by the templates. Imagine this as a properties file. In combination with the templates, the values will be used to create Kubernetes manifests.

Charts Directory The charts directory holds other Helm charts which this one depends on. It's used to configure complex parent-child chart relationships.

Templates Templates are integral to using Helm. The Template Directory is where existing Kubernetes manifests can leverage the values in Values.yaml and if resources need to be generated, additional YAMLs can be generated with Templates.

GEOAxIS Authentication GEOAxIS has been integrated with the N2 Appian frontend for all Identity, Credential and Access Management (ICAM) and Entitlement Management Service (EMS) functions as the primary means of access and management of N2 analysts, users, and groups. Kong is used to provide OAuth2 authentication via GEOAxIS and routing to the Appian endpoint.

KeyCloak The means of authenticating access for the database administrator via CAC.

OAuth One of the most common use cases in modern web-based applications or services is the need for user authentication and authorization.

Authentication - identifying who the client/requester is (who are you?)

Authorization - defining and enforcing what permissions user has (what are you allowed to do?)

Although there are many different approaches for handling ICAM functions, an enterprise system such as NGA CORE benefits from having a singular, approved pattern for users and developers to leverage to avoid unnecessary effort, technical debt and a ballooning tech stack to manage and maintain.

The pattern that NGA CORE uses and encourages is a Single Sign-On approach that relies on OAuth 2.0 possibly coupled with a lightweight protocol like OpenID connect (OIDC). The identity broker that most often facilitates this pattern for NGA users is GEOAxIS.

Figure 3: High level authentication overview Even though we now settle upon a fairly unified pattern of OIDC and OAuth 2.0 with GEOAxIS as the identity broker, it does not completely solve two primary problems that we face in a multi-tenant environment such as NGA CORE OpenShift (KubIC).

Here a simple pattern for tenants to get going quickly with SSO using an OAuth Client to connect to OpenShift's OAuth server which uses GEOAxIS as an OpenID Connect identity provider

Rather than asking tenants to introduce OAuth clients into their applications and code bases, KubIC recommends leveraging the sidecar pattern to add an OAuth proxy container into application pods. This allows tenants to use an existing, tested OAuth proxy solution in their deployments without requiring custom code.

Database Components Database Overview

Figure 4: Detailed N2 Database component interaction Database Transformation Workflow In the effort to convert the legacy database into the model desired for CORE operations, the following steps were utilized to achieve a desired database end state.

1. Tailoring the data model and correcting legacy model issues

2. Configuring the "work rig" which is a powerful laptop running OEL V8 to process the database from Oracle to PostgreSQLQL. This included the tool ora2pg config. That ora2pg tool is the thing that does the actual ETL. All these conversions need to be checked for errors, consistency, and compatibility with the microservice code.

3. Pushing out the new database as a PostgreSQL dump file.

4. Provisioning and building out the infrastructure.

5. Loading the database.

6. Regression testing to ensure quality, consistency, and that all data is free from errors and ready for use with the microservice model.

SCS database developers repeated these steps multiple times to fix any discovered issues and to dial-in steps 1-6. After this point, SCS will have full, detailed vetted procedures.

Figure 5: High level ETL Process viewpoint In addition, SCS developers are culling the database to remove all tables referencing ArcSDE as these previous functions will be performed by developed microservices.

Extract Transform Load (ETL) Database Process

Figure 6: Transitory ETL configuration within N2 environment Supporting Dual Operations via Continuous ETL

The ETL synchronization method will facilitate a dual operations cutover when properly configured. The synchronization flags were inconsistent during initial testing and preparation for the originally planned cutover. It was therefore determined that dual operations would not be a feasible cutover methodology due to schedule constraints. This section provides an overview of the intended dual operations approach for guidance and reference.

Proposed State One - Users operate the same microservice at both the legacy Oracle system and new environment PostgreSQL system. Operating includes inserting, updating and deleting data. The same authoritative microservice records exist in both systems and must be logically reconciled between both systems.

Proposed State Two - Users operate a microservice that has been cutover from the legacy Oracle system to the new PostgreSQL system. Operating includes inserting, updating and deleting data. The authoritative records related to the microservice exists in the new PostgreSQL system, with some exceptions for authoritative records shared between microservices. All microservice functionality is shut down after the cutover.

The methodology to perform this is as follows. The Database is broken apart by microservice blocks. A first-step is to assign database structure participants to their respective microservice block. The database structure participants are primary tables but may include F and S geospatial tables and other associated child tables.

For each microservice model the data is prepped for the ETL process in the Oracle ETL database. Data is then pushed thru the ETL process and is rendered as PostgreSQL database insert statements. These records are subsequently inserted into the PostgreSQL database in logical order.

While performing these operations the following considerations must be regarded:

All tables in the Microservice block need to be identified. As such, these table parent-child relationships need to be fully documented.

Primary Key conflicts can arise for sequence generated values. Specifically, post initial migration, PostgreSQL users will allocate new sequences to add microservice records.

Concurrently, legacy users of the same microservice in Oracle will allocate those same sequence numbers to add new records. On subsequent ETL loads, those sequences will conflict with those that are already allocated via the PostgreSQL sequence generation process.

For primary table inserts without child records:

A possible remedy would be to remove the Oracle generated sequence on insert. In order to do this, an Oracle ETL database must be set up to perform this pre-transformational work. In addition to that, there must be confirmation that the PostgreSQL sequence will assign a new sequence on insert independent of the method of adding the record.

For primary table inserts with child records:

This process is more complicated. Since the sequence number generated in Oracle also identifies any child record as belonging to the parent, removing the sequence record in both places will destroy the relationship between the parent and child. A way around this is to capture the last sequence generated in PostgreSQL and use that as a starting point to re-assign sequence numbers in the Oracle ETL database which will not conflict with PostgreSQL on insert. PostgreSQL sequence generation for these records is disabled on insert.

There can be shared records between microservices. This is always reconciled at the database as it is the authoritative codes for the system. However, with two databases, this record must be reconciled at both databases as long as dual-operations are in place.

In short, where a shared record exists both microservices need to be migrated at the same time.

Administrative direction of work

Specifically, accounting must be done for the entirety of all business operations when assigning migration order, with emphasis on the continuous ETL order path least disruptive to business operations.

The middle-tier migration activities must be aware of and coordinated with the data migration order and timing.

Overlapping Activities exist in business operations and can reveal themselves within microservice architecture, database programming and data structures. These must be identified, documented and accommodated within the middle-tier to database level.

Oracle can break apart the database to just move those tables we identify as belonging to a microservice we want to move. However, we need to identify any shared tables between microservices. Once we move a microservice it should be permanent one-way transfer to the new PostgreSQL db.

PostgreSQL In as far as the operational environment for N2 is aligned, a quick discussion of the deployment is as follows: PostgreSQL is deployed via AWS RDS and is made available via OpenShift through Crossplane. The database developer/administrator can access PostgreSQL and control it via pgAdmin while being authorized access through Keycloak https://www.siriusopensource.com/en-us/blog/Oracle-PostgreSQL-migration-using-ora2pg

The above link was used to configure the N2 database and push it thru this ETL tool to get it into PostgreSQL with no errors.

Crossplane Crossplane[endnoteRef:3] builds an AWS RDS resource physically outside the OpenShift cluster but it is considered as the same infrastructure for ATO. Crossplane uses Kubernetes as the underlying platform to expose these features and workflows to manage cloud infrastructure outside of Kubernetes, such as data centers, the cloud or the edge. [3: Step-by-step guide to working with Crossplane and Kubernetes | TechTarget]

It installs a set of components called providers by extending Kubernetes to manage resources from cloud providers. Providers build on the Kubernetes API, along with the custom resources and controllers, and expose declarative APIs to manage cloud provider resources.

Crossplane is geared toward platform teams and operators looking to build abstractions and easier-to-consume infrastructure components based on an organization's needs. It has a few concepts, such as CompositeResourceDefinition and Composition, that enable teams to extend and build on top of the default resources exposed for a cloud provider.

PostGIS Plug-In The PostGIS[endnoteRef:4] extension enables geospatial datatypes and queries and is a very common utilization of this resource. PostGIS extends the capabilities of the PostgreSQL relational database by adding support storing, indexing and querying geographic data. PostGIS features include: [4: PostGIS]

Spatial Data Storage: Store different types of spatial data such as points, lines, polygons, and multi-geometries, in both 2D and 3D data.

Spatial Indexing: Quickly search and retrieve spatial data based on its location.

Spatial Functions: A wide range of spatial functions that allow you to filter and analyze spatial data, measuring distances and areas, intersecting geometries, buffering, and more.

Geometry Processing: Tools for processing and manipulating geometry data, such as simplification, conversion, and generalization.

Raster Data Support: Storage and processing of raster data, such as elevation data and weather data.

Geocoding and Reverse Geocoding: Functions for geocoding and reverse geocoding.

Integration: Access and work with PostGIS using third party tools such as QGIS, GeoServer, MapServer, ArcGIS, Tableau.

pgAdmin pgAdmin[endnoteRef:5] is a management tool for PostgreSQL and derivative relational databases such as EnterpriseDB's EDB Advanced Server. It may be run either as a web or desktop application. It is utilized to allow database administrators and developers to interact with the PostgreSQL database through Crossplane. [5: pgAdmin - PostgreSQLQLQL Tools]

Appian Workflows Training Training and familiarization was conducted by both SCS developers and operations support personnel. These resources can be found at https://community.appian.com/learn/courses and Appian administrators are available to create accounts for concerned parties.

All SCS developers supporting N2 participated in online (self-paced) Appian Developer courses (Under the "Builder" section). Primary developers were placed in Instructor led training sessions.

SCS Operations personnel supporting N2 were allowed to take the online Administrator course (Under the "Administrator" section). Once again, primary operations support personnel sat in on the Instructor led sessions.

Other operations personnel and in fact the entirety of SCS were encouraged to create an Appian community account and conduct familiarization training wherever possible.

Development Appian resources included a grouping of Appian images that were utilized initially. SCS grabbed the web app image and appian-operator image from this site. Deployment in CORE was completed in mid-2023 following completion of security scans and approvals. During the same timeframe, SCS installed Appian in DeepSky to facilitate initial workflow development efforts.

Appian has since been deployed within OpenShift. It has access to the other microservices and can call out to their services. The Appian pod calls out to the backend microservices. It is worth noting that the Appian process is the only N2 feature available through Kong.

FrameMaker Server This section referenced is still being engineered and information pertaining will be provided in the final documentation.

Cloud Deployments The following deployments are currently approved for N2 utilization.

N2 SPIDs:

65200 - Uranium Navigation Safety System (Uranium N2) 61135 - Titanium Navigation Safety System (Titanium N2)

Decommissioned SPIDs:

81004 - Uranium Navigation Safety System OpenShift (Uranium N2 OpenShift) --> 81004 was absorbed by 65200 85006 - Titanium Navigation Safety System OpenShift (Titanium N2 OpenShift) --> 85006 was absorbed by 61135

N2 Instances by Domain

(U) SCS has set up mirroring in DeepSky GitLab. Any of the variables that could change (database host, database user, database password, etc.) are extracted out as environment variables. The code is then committed to GitLab and is mirrored to the other low side repositories.

(U) The last stages of our Uranium pipeline push our code and our images to an ECDS server that moves the code & images up to a Titanium ECDS server. We then have hourly cron jobs running in Titanium that checks for new code & images and pushes them into Titanium GitLab. In short, the cron syncs Uranium & Titanium.

Uranium to Titanium Transfer

(U) Uranium will export to Titanium upon each new commit; all of the code is developed on the low side and moved up high. No development is needed in Titanium. In the Phase I iteration SCS utilized lambda functions which deploys the code via chalice (none of which requires any docker images to be built). Code was to be moved up to the high side via NGA's Project Wormhole.

(U) SCS engineered a better solution (replacing lambda) but this would have required SCS to build out a Titanium pipeline that built the same images that were being built and scanned in SBU since Wormhole only moves code, not docker images. This was doable but labor intensive and still dependent on Wormhole successfully moving the code up so updated images could be built in Titanium.

(U) The images are now compressed into a single TAR file and moved to Titanium via an Enterprise Cross Domain Solution (ECDS) share. A cron copies the image into the Titanium Container Registry. No building of the image happens on the high side with the current pipeline. ECDS moves the code and the docker image up to Titanium. On Titanium, instead of rebuilding the image, we run a single job that will scan our specific folder on the share and import both the code and the image into Titanium GitLab where it can then be deployed.

End of Development Transfer

(U) The specific server we use for ECDS via GitLab is locked down to just GDIT. FS2 will need to set up their own GitLab runner on Gold that will pull data to a local share, and work with a DTO to push it up. They will then need to create a GitLab runner on Titanium and once the DTO pushes it up, the current image sync and git sync scripts can pull it into GitLab. Once they set those up, they should just need to update the CI/CD variables to point to the new location to sync the files to/from.

System Configuration Architectural viewpoints and documents containing N2 logical topology and information will be maintained within an SCS N2 COE share as needed.

Network Diagram

Figure 7: Post Transition Architecture

Software Baseline As of this time, SCS operations personnel are working directly with UDS and SFH personnel to configure and finalize the N2 Win10 baseline to include all software loads, configurations and environmental variables. A detailed list of these items will be documented when the process has completely matured and the build state is finalized and approved by the USG.

Table 2a: (U) N2 Desktop Baseline

Table 2b: (U) N2 Server Baseline

Table 2c: (U) N2 Microservices Part 1

Table 2d: (U) N2 Microservices Part 2 This software baseline comprises a large portion of the entire functional baseline as noted in section 1.8 along with the W10 desktop systems that were deployed within Phase I.

The initial N2 software baseline snapshot is displayed below.

Table 3: Proposed N2 Win10 software baseline (as of 29 Jan 2024) Interfaces and Services Enterprise Engine (E2)

Enterprise Engine (E2) Notice to Mariners (NtM) also known as E2-NtM is required to interface with N2 and has been re-engineered to account for the PostgreSQL database compatibility requirements. Instead of Oracle binary formats previously delivered on the legacy NSS system the E2 export and ingest process will transition to compatible SQL statements once N2 operational cutover is complete.

This integration will need to support the ingest of Electronic Chart Corrections (ECCs) into the new N2 operational environment.

Martitime Safety Information (MSI) Portal N2 Publications transferred to the Cloud are provisioned with the tools to compose, compile, maintain, host, and disseminate the completed products to the Maritime Safety Information (MSI) portal on all networks.

All available publications can be found at https://msi.nga.mil/Publications

The workflow for dissemination of Publications is shown in Figure 8. A detailed version of the dissemination process is provided in the N2 Maintenance Guide, and the following is a high-level summary for reference as outlined in Figure 8.

Upon SFH closeout of the weekly Notice to Mariners the O&S staff will then initiate the Comm Manager Replication Code for dissemination and update to E2-NtM and the MSI websites. The Comm Manager contains sensitivity filters that use a two-step filter attribute function to remove all classified rows and produce a sanitized, unclassified set of files that are compressed/TARed and transferred for unclassified delivery endpoints including the MSI website updates and external recipients.

Figure 8: Publications Dissemination to MSI workflow Notes (AV-2 Integrated Dictionary) Please note that a master AV-2 is being compiled that covers all glossary terms, acronyms/abbreviations, and endnote links contained within the four N2 documentation deliverables.

Glossary Application Programming Interface – a way for two or more computer programs to communicate with each other. It is a type of software interface, offering a service to other pieces of software. It connects computers or pieces of software to each other.

Argo CD - an open-source GitOps continuous delivery tool. It monitors your cluster and your declaratively-defined infrastructure stored in a Git repository and resolves differences between the two — effectively automating an application deployment.

CORE - where NGA builds, releases, and operates software to deliver on the Moonshot Helm Charts - packages of pre-configured Kubernetes resources that can be deployed quickly and easily.

Non-person Entity – A service account OpenShift - a cloud development Platform as a Service (PaaS) hosted by Red Hat. It’s an open source cloud-based user-friendly platform used to create, test, and run applications, and finally deploy them on cloud.

Operator - constructs defined within programming languages which behave generally like functions, but which differ syntactically or semantically. Languages usually define a set of built-in operators, and in some cases allow users to add new meanings to existing operators or even define completely new operators.

pgAdmin - pgAdmin is the most popular and feature rich Open Source administration and development platform for PostgreSQL PostgreSQL - PostgreSQL is an advanced, enterprise class open source relational database that supports both SQL (relational) and JSON (non-relational) querying.

Acronyms and Abbreviations ACS – Advanced Security Cluster API – Application Programming Interface CORE – Common Operations Release Environment CDTN – Chartlets, Depth Tabulations, and Notes DD – Description Document DLA – Defense Logistics Agency EC2 – Elastic Cloud Computing EOL – End of Life ETL – Extract, Transform, Load FRD – Functional Requirements Document IL – Impact Level KubIC – Kubernetes for the Intelligence Community MSI - Maritime Safety Information Portal MWaaS - Middleware as a Service N2-EE – N2 Enhanced Environment NOS – National Ocean Services NPE – Non-person Entity NRAI – NSG Requirements for Applications and Infrastructure NSG - National System for Geospatial-Intelligence NTO - NGA Testing Office PSC – Persistent Storage Claim PVC- Persistent Volume Claim SPID – Security Plan Identification UI – User Interface USG – United States Government

Endnote Links image1.png image2.png image3.png image4.png image5.png image6.png image7.png image8.png image9.png image10.png image11.png image12.png image13.png image14.png image15.png

File details come from the government source that posted it. Updated .