C_2019_SPIE_11015-5_SDR_r4.pdf

Modular Communications and Software-Defined Networks for

Aerospace Applications James Lykea, Christian Petersa, Khanh Phama, Don Fronterhouseb, Matt Hannonb

Air Force Research Laboratory, 3550 Aberdeen Ave SE, Kirtland AFB, NM, USA 87117-5776;

COSMIAC/University of New Mexico, 2350 Alamo Ave SE, Albuquerque, NM, USA 87106.

ABSTRACT

This paper describes a modular strategy for communications in which each platform (being a satellite, airplane, missile/munition, even soldiers) carries a hub-and-spoke internal network integrating a variety of modems and other information devices. These embedded networks, referred to as “global network access terminals” (GNATs), are in fact information convergence devices that seek connection with other peers (subject to constraints, such as energy and security) to form basic “substrate networks” (a type of mobile ad hoc networks). Overlays are formed on top of these often-unreliable substrates, permitting the virtualization of networks that can appear to be more reliable, hence a type of software-defined network (SDN). The concepts for this type of SDN is drawn from the rich base of SDN and network functional virtualization (NFV) concepts used in cloud computing , except that they are must be optimized for the heterogeneous and rapidly changing nature of the basic substrate network formed opportunistically by GNATs..

Keywords: satellite communications, software-defined networks, network functional virtualization, modular open systems architecture.

1. INTRODUCTION

Traditional satellite communications architectures involve a constellation of satellites, user terminals, and ground management segments that control both the space vehicles and payloads. Space vehicle management is concerned with control and maintenance of the spacecraft in their orbits, whereas payload management deals with the provisioning of the

“communications as a service” to the users. In other cases, satellites can carry sensors to conduct a non-communications mission, but require specialty communications to offload/downlink the data generated. The Department of Defense has traditionally built its own systems, and these are referred to as “purpose-built” satellites. When built for communications, the associated service provided is literally called “military satellite communications” (MILSATCOM). However, the demand for communications by the military has far outpaced the capacity of these purpose-built systems, resulting in the fact that the majority of bandwidth in recent conflicts have been supplied from commercial sources1. Fortunately, the availability of commercially-based satellite communications has exploded, and the trend is further accelerating with the

“megaconstellations”, which seek to blanket the earth with more internet delivery services2. Commercial satellite communications (COMSATCOM) pervasively augment MILSATCOM to provide warfighters the amalgam of connectivity needed to conduct operations3.

At first blush, it would seem the problem is solved, but the solution is far from perfect. In the first case, the amalgam creates a complexity of multiple networks that must be set up, with different footprints of equipment and associated logistics and of course different provisioning, security architectures, and interfaces. They are not usually seamlessly integrated, making it difficult to move across networks when one is not available (due to coverage, contract limitations, jamming, weather, etc). In the fog and friction of conflict, these difficulties place a heavy burden on preserving the connectivity necessary for military operations.

The explosion of options for communications, though none perfect, combined with the increasing complexity have led to a seemingly reasonable desire to unify the communications apparatus as a whole. Is there a way we might establish a unified framework that allows the communications options to be added, removed, changed in a transparent way so that purpose-built and commercial approaches alike can be comingled without breaching security, without large gaps and disruptions in operations? Can steps be taken to reverse the trend of apparent complexity and sprawling footprints of equipment and associated headaches of logistics and recapitalization of dozens of different communications terminals?

This paper introduces a modular communications framework that provides a possible solution by enabling in the near-term a method for combining disparate communications options. In a crude analogy, we consider each communications option like a pipe in which, like a fluid, the bits of communications flow. Each communications option, whether a point-to-point link or a complex, opaque COMSATCOM enterprise service, is simply treated as a different pipe. We employ combiners on each end for the pipes, which are added in parallel, as many as we can find to connect. As in the case of fluid flow, so long as one pipe is “clear”, fluid can flow, and if more pipes are clear, the flow can be better. We call this

“pipe combiner” a global network access terminal (GNAT), and this paper discusses space-based and terrestrial reference architectures.

Even by this description of the GNAT, we seem to be solving one problem (the problem of “seams”), but not effectively addressing the sprawl of equipment, and perhaps exacerbating the problem by adding more “things”. In the near-term, we believe the gains more than offset the disadvantage, since unified solutions allow for competition and improved solutions over time and in recapitalization activities. We suggest that several technologies, such as cognitive radio and reconfigurable antennas, can provide a longer-range solution by replacing boxes at first and eventually entire suites of equipment with a single, configurable system. Such a system would eventually mimic any communications architecture through software virtualization, analogous to the way virtual machines in data centers can mimic any processor (through the principle of equivalence in Turing machines) and any network through network virtualization. In time, legacy systems would be replaceable by bit patterns that could be installed in a single black box reconfigurable transceiver, replacing large accumulations of equipment (i.e, palettes of boxes). In this approach, the full power of the “parallel pipes” can be brought to bear, and, as in the case of data centers, the need to maintain dozens of legacy equipment clusters will fade over time.

This paper is organized as follows. In the next section, we introduce the GNAT reference architecture as a first-principles concept for a specific point-to-point communications problem, namely connecting to the internet in very remote locations.

We then address scaling this concept to larger numbers of users. We then describe more sophisticated network embodiments, in which concepts of software-defined (overlay) networks are employed to allow extension to a connected cloud. We then detail work on space-based and terrestrial embodiments of the GNAT concept and consider the open challenges and limitations of the idea. We finally describe the role of technology to provide a long-range solution to equipment sprawl and enhanced versatility of the GNAT concept.

2. GLOBAL NETWORK ACCESS TERMINAL

In this section, we discuss the concept of a global network access terminal (GNAT) as an evolution of basic communication concepts

2.1 Point-to-Point Communications

In the most basic case, we wish to establish communication between two separated things (nodes). We can do this using a perfect wire (Figure 1), and the signal introduced on one end appears at the other end. Abstractly, this can be a conduit of not only electrons (as would be conveyed in the conductor of the perfect wire), but it could also be, sound, fluid, etc. We might denote this is an “infinity” cable.

Figure 1. Communications between nodes A and B with a perfect “infinity” cable.

There will always be a physical limit. In contemporary communications, these include the limitations of electronic equipment, the ambient physical medium, the geometries of communications, and the barriers/impairments between communications modes. The signals between A and B are often facilitated through waveform and signaling conventions that are implemented by modems (Figure 2a). These modems are intended to make best use of the limited physical channel, often incorporating error correction and adaptive protocols that were to achieve the most efficient and reliable approximation of the “infinity cable” possible. In the present state of the art, the closest approximation of an infinity cable might be a dense wavelength division multiplexing (DWDM) system (Figure 2b) impressed onto a single mode fiber optic strand, which can support (at the time of this writing) as much as 20 terabits/sec4. Even so, this feed would be difficult to realize with a single electrical channel (DWDM employs nearly 200 parallel channels to achieve this effective bandwidth rate).

Figure 2. Emulation of perfect connection with imperfect channels. (a) Modems and free-space channel between A and B.

(b) Best approximation of “infinity” using dense wavelength division multiplexing. (c) Implementation through two soft flow paths (r – router, p – control processor, m – modem)

To add capacity and resilience (to step more in the direction of the infinity cable), we can add several copies of the Figure

2a modem chain, as suggested in Figure 3a. To simplify the splitting of a single signal into two channels in dealing with the differences in reliability, latency, and data rate, we implement an intelligent manager in the control processor (p) that ideally manages the different connections seamlessly.

Figure 3. Extended example of A-B connection using multiple soft flows. (i) Point-to-point radio frequency or optical link.

(ii) Use of a cloud-like (proprietary) SATCOM service. (iii) As edge nodes of a mobile ad hoc network. (iv) Depiction of delayed, hopping network.

In principle, should be able to accommodate any number of these parallel chains which we shall call soft flows. The concept is analogous to multi-path TCP5, but the concept is not limited to TCP. The soft flows can be fundamentally different in nature, as suggested in figure 3b, with some being based on a line of sight connection (i), or formed from two different taps in a proprietary cloud network (ii) (such as a commodity satellite communications (SATCOM) service), as edge nodes in a mobile ad hoc network (iii), or even as hops in a temporally disjoint transport concept (iv)(such as an accumulated batch transmission in a delay tolerant network (DTN) 6).

2.2 Defining the Global Network Access Terminal (GNAT)

We now define a basic reference model for the Global Network Access Terminal (GNAT) in Figure 4. In this model, the user connects to a router through a host interface. This router also connects to one or more communications devices, which are referred to as physical layer devices or “PHYs”. The PHYs represent the possible conduits to soft flows. The PHYs and routers are managed (configured and operated) through an intelligent controller referred to as the GNAT Control

Processor (GCP). We combine the control processor, the router, and associated PHY into an enclave that forms the GNAT.

This basic structure will be further elaborated throughout this paper. For now, it will suffice to indicate that each GNAT attempts to optimize its connection with others, through any means possible (i.e., through one or more of the available soft flows).

Figure 4. Global Network Access Terminal (GNAT).

2.3 Service Qualities

The qualities of different soft flows such as utility, availability, and acceptability will be situation dependent. When real-time transmissions are required, a delay-tolerant soft flow cannot be employed, but other soft flow options may be if they are present. It is the primary role of the intelligent manager in the GCP to broker/mediate the establishment and use of some, none, or all of the possibilities, singly or in combination, according to the qualities of service required in the desired connection between A and B. We can consider notionally a multidimensional representation of several of the qualities of service: three are depicted in Figure 5. The first of these is latency, the amount of time needed to transport information

(shown as reciprocal latency on the x-axis in Figure 5, so that lower latency has a higher value). Most of the latency is set by physics and geometry relating to time of flight, but channel disruptions and congestion will also cause delay and/or loss of information. Communications bandwidth is depicted as a second axis (y-axis), and the jitter or variability of latency is depicted as third axis (reciprocal jitter as z-axis). Simplistically, we might conclude that in Figure 5, point Q is a better situation than point P, but not all applications require the same QoS, and imposing arbitrary thresholds across all applications will result in poor overall network utilization. Different types of communications might represent distinct points or regions in such a multidimensional quality of service (QoS) space. For example, file transfer (and email) benefits more from bandwidth than latency or jitter, and reliability is most important. Full motion video benefits from lower variability in jitter and high-bandwidth, though latency is sometimes tolerable and reliability (which might be considered a fourth axis), while important, may be lower than in file transfer (i.e. it may be acceptable to have occasional frame dropouts). Critical event notifications tend to emphasize reduced latency and jitter over bandwidth. Control systems similarly place a premium on determinism, especially in terms of jitter, which can be more difficult to manage than a fixed

(nonzero) latency. Interactive communications can be delay tolerant (e.g., text messages) or “intolerant” (voice).

Figure 5. Multidimensional representation of several qualities of service

Example implementations of quality of service involving multiple soft flows is depicted in Figure 6. When multiple soft flows are available, the basic expectation is that a connection can be maintained so long as at least one soft flow path exists between the two nodes, as suggested in Figure 6a. In fact, it is expected that the GCPs at each side of the intended virtual link maintain a dialog in which relevant metadata, including that relating to quality of service, is updated. This concept is notionally depicted in Figure 6b, in which the different qualities of service in three soft flows are shown. Depending on the circumstance, one or more of the soft flows having a QoS consistent with that desired by the application can be employed. We shall not detail the heuristics for algorithms for best QoS, but certainly the decisions of which to use involve a number of factors, including latency, bandwidth available for a particular soft flow, dependability of the service or channels, etc.

Figure 6. QoS considerations for connections having multiple soft flows. (a) Use of two soft flows, where one is blocked.

(b) Depiction of diagnostic display of QoS multiple soft flow channels, which can drive flow selection heuristics.

2.4 Concepts for control plane, GNAT-GNAT, and GNAT-PHY protocols

In the Figure 1 abstraction of a pure connection between two points with a perfect cable, the only protocols involved are those impressed upon the wire or channel. Any bits impressed upon one are conveyed to the other end after a time of flight delay. There is no notion of a control plane, negotiation for provisioning access to the pool of bandwidth, etc. In almost every other case, additional protocols are required to achieve effective connectivity across physical channels, a particular network or service, and the combination of all networks and services. If we consider the scaled implementation involving thousands of copies of GNATs, then we furthermore need to consider the type of management (centralized, distributed, or some combination) or provisioning, since limited resources must be efficiently brokered to the individual

GNATs.

Even in the case of a single modem (Figure 2a), the free space channel often requires adaptive modulation, forward error correction, and other mechanisms to achieve reliable operation. Military communications, as well as contemporary commercial ones, can employ sophisticated frequency hopping and spread spectrum techniques. The cues for which adjustments to be made are usually baked into the protocol design, though in some cases adjustable “knobs” are exposed that can be managed during the communications process. These amount to a rudimentary control plane established over the air between the modems, and usually the user does not have to be extensively involved, and these control plane functions did not appear in the data sent from A to B.

As we extend the complexity to the GNAT-based communications concept, more control plane complexities are introduced. Some of the key linkages are depicted in Figure 7. For the present discussion, we shall separate these categories into “types” of internal communications. They are effectively meta-channels, perfectly isolated from each other, meaning that the bits that flow in channels of “Type 1” will not be appear in the bits that flow in “Type 3”.

Furthermore, there can be several channels the same type, and these are also isolated from each other.

The scenario we shall focus on involves the communications between two GNAT enclaves (Figure 7a) involving three different modem interactions. One modem (X) involves a satellite communications (SATCOM) service (depicted as cloud) in which each GNAT enclave employs a modem (“X”, connect to the router). Another modem (Y) involves a point-to-point laser communications link. A third modem (Z) involves a connection to a different SATCOM service in which one

GNAT enclave employs a SATCOM modem, whereas the second GNAT enclave is already connected to the Internet

(cloud) through some traditional commodity router. In this case, the satcom service provides an Internet point of presence through a teleport gateway. It is necessary for second GNAT to employ a SATCOM modem for service Z. This scenario is sufficiently detailed to permit discussion of the four classes of communication transactions that take place in the GNAT construct.

Type 1. Type 1 channels refers to the desired primary desired communications channels, the only ones that would appear when an electrical signal bearing information is impressed upon a physical cable (i.e., the Figure 1 case). In the case of an actual physical electrical cable, there can only be a single Type 1 channel. It is possible to establish conventions for multiple Type 1 channels in this physical medium, and these correspond to the concepts of time division multiple access

(TDMA), frequency division multiple access (FDMA), and code division multiple access (CDMA) as examples. In this case, it is always necessary to apply additional concepts that allow the superimposed individual (virtual) channels to be combined in the physical medium and effectively separated. For example, in the case of TDMA, bits from different Type

1 channels are time sequenced the same physical medium, and conventions must be agreed upon to correctly separate particular bits as belonging to particular Type 1 channels. In other cases, such as wavelength division multiplexing

(WDM), channels are treated as effectively being a series of spatially separated channels, as though there were multiple physical cables.

Figure 7. Depiction of GNAT multi-modem interactions and different types of communications that can take place in

GNATs and PHYs. (a) Three-modem interaction with 2 GNAT enclaves. (b) Four classes of communications.

Type 2. - Service endpoints. Type 2 communications refer to the entire category of platform-internal communications, in particular, those channels within the platform between a GNAT Control Processor (GCP) and a PHY device that do not directly appear over the medium negotiated by compatible PHYs on two different platforms. This is a very important category of communication, associated with the “care and feeding” of PHYs, and there may be a rich variety of Type 2 messages. Examples include:

• Wholesale provisioning and configuration of a PHY

• Access to measurands (e.g., temperatures, current/voltage, error rates)

• Operations necessary to establish pointing, channel selection

• Security-related preparation (e.g. key exchange, security protocol selection)

Type 2b. We might differentiate Type 2 communications to denote hierarchy within a PHY. For example, an optical modem may have a gimbling system, steerable mirrors, etc. that can be managed by commands from the GNAT brain.

Type 3. Channel management. Type 3 communications pertain to the meta-exchanges and/or “baked in” conventions necessary for PHY-PHY communications. The information in Type 3 communications do not “escape” the PHYs, meaning that for example the meta-data passed between PHYs do not appear in the user’s expected (Type 1) data stream.

Examples of “baked-in” conventions include rote frequency hopping patterns of a particular protocol that do not change.

More sophisticated meta-exchanges include the means by which adaptive protocols negotiate settings, such as bit rate, modulation time, dynamically-changing frequency hopping patterns, etc. in a complex dense wavelength division multiplexing (DWDM) scheme, a Type 3 communication might be used to explain the wavelength plan as PHYs come into contact. For cognitive radios, Type 3 communications might be essential for coordinating changes in different operating parameters.

Type 4. GNAT-to-GNAT. Type 4 communications refer to GNAT-GNAT interactions. They are especially important for GNATs having multiple soft flows, but they also are essential in establishing, for example, concepts of namespace. It is through this type of communications, that for example would permit an exchange of QoS information depicted in Figure

5b.

Type 4b. GNAT-to-Global (hierarchy-spanning). Type 4 interactions are “egalitarian”, whereas Type 4b interactions take place between nodes that are subordinated to some notion of a hierarchy / control plane above the level of particular

GNATs. For example, if we are concerned with provisioning many GNATs, we would need to construct something like a network operations center (NOC), to manage entire groups of GNATs in a uniform, scalable way. Type 4b communications are simply a specialized form of Type 4 communications. For example, a NOC would likely appear to be a well-known destination (e.g., IP address, phone number, interest topic, etc) to GNATs, but the mechanism for access/interaction between hierarchical levels might otherwise be indistinguishable from the egalitarian case.

Interfaces for Physical vs. Virtual Communications – What can and cannot be abstracted

It is easy to be confused by the mixture of virtualized communications channels with physical (real world channels). For example, between two PHYs (each being on a separate GNAT), we can over the same physical channel pass several Type

1, Type 3, Type 4(4b) communications streams. Furthermore, in the case of soft flows, we might distribute Type 1 and

Type 4 flows in some mixture across different soft flows (but Type 3 information flows are confined to particular soft flows).

Some things cannot be readily virtualized. For example, precision timing transfer between a GNAT and PHY might require more precise determinism than is available over a traditional Ethernet connection. Analog signals, such as those involve antenna remoting, involve a single Type 1 “flow” (in this case an analog/microwave continuously varying signal) that cannot intermingle control plane / metadata traffic.

2.5 Network Virtualization (Substrates and Overlays)

We next discuss the network relationships between GNATs. As in the case of soft flows, in which we virtualize a single channel from several physical channels, we can consider networks as extensions of the two-GNAT problems.

In this discussion, each node is a GNAT enclave, consisting of a GNAT Control Processor, Router, and its PHYs. In the homogeneous case, GNAT enclaves are identical, meaning each would have the same quantity of PHYs, and each PHY would be identical. In general, however we consider the enclaves to be heterogeneous. The existence of a connection from

PHYs between GNAT enclaves is expressed as an edge connection. The edges are assumed to be bidirectional (full duplex) and symmetric unless denoted by arrows. From these basic ideas, we can express actual and desired connectivity between nodes as graphs. The actual connectivity present between GNATs we call a "supply graph", whereas the connectivity we desire is called a demand graph. In Figure 8a, for example, we desire a complete graph between four GNATs {A, B, C, D}. This supply graph can be satisfied in theory by the supply graphs corresponding to different cases depicted in Figures

8b-d. In Figure 8b, we depicted two additional nodes (we will call these non-terminal nodes (NTNs)). These nodes do not appear in the demand graph but are necessary in the supply graph to satisfy the desired connectivity. In Figure 8c, we have a degenerate supply graph that is still theoretically capable of satisfying the Figure 8a demand graph. Figure 8d is another example of a more complex supply graph with many NTNs that can in theory satisfy the same demand graph.

Figure 8. Demand and supply graphs. (a) Demand graph. (b) Supply graph with non-terminal nodes (NTNs). (c) Degenerate supply graph. (d) Second example of supply graph with NTNs.

To relay the diversity that may exist in a GNAT-based network, Figure 9 depicts how the Figure 8c supply graph might be implemented with a heterogeneous GNAT set that employ different numbers and types of PHYs. The nodes are sources and sinks of Type 1 flows. Node A has a connection to Node C involving the linkage between individual PHYs having a

Ku-band radio-frequency (RF) link. Node C in turn connects to node D with two different physical channels. One PHY pair communicates over an X-band RF link, whereas a second involves an optical 1550nm link. Node D connects to Node

B also using an optical crosslink.

Figure 9. An example implementation of the Figure 12c supply graph using a set of GNATs having different degree

(numbers of PHYs) and PHY types (different physical channel capabilities).

A second example (Figure 10) depicts GNAT connectivity in the presence of nodes considered to be non-terminal (i.e., not a part of) in the demand graph (Figure 8a). In this case, we consider the α nodes to be part of a notional space-based laser communications network where the nodes are also equipped with E-band communications links. These links connect to edge nodes B, C, and D (the context of “edge” in this case simply means that the nodes are not part of the α network, and other parts of the α network are not depicted) through either optical or E-band links (or both). The A node is bridged from a Ka-band facility that is present on both nodes A and C.

Figure 10. Second supply graph GNAT example involving non-terminal nodes.

We next consider the consequences of flow implementations in virtual demand graphs when implemented on the physical

(supply) graphs. Figure 11 shows annotated graphs that indicate the nodes needing to be traversed on physical graphs to implement the example (Figure 8a) demand graph. Figure 11a is a redrawing of Figure 8b that depicts the overlay of demand (Figure 8a) on the Figure 8b supply graph. Implementing a flow between node A and node D, viewed as a single hop connection in Figure 8a, in fact requires the traversal of node C as well, and so the real world path A-C-D is required.

Figure 11b is a redrawing of Figure 8c, reflecting the paths that must cross NTNs α1, α2, in the supply graph to implement flows in the demand graph. Finally, Figure 11c depicts the redrawing of Figure 8d. In this case we see the effects of the actual path redundancy to the demand graph. The desired connection for example, between nodes B and C can be implemented by at least three paths. In this example, some of these paths involve the same nodes, meaning that the disruption of one node can actually disrupt several of the possible paths. In other words, path cardinality is not necessarily correlated with additional robustness. While such diagrams may provide insight in these simple examples, it is straightforward to contrive examples that have an explosively high path cardinality while providing little actual benefit to resiliency.

Figure 21. Redrawings of supply graphs to reflect the apparent structure of the Figure 8a demand graph. (a) Redrawing of

Figure 8(b). (b) Redrawing of Figure 8(c). (c) Redrawing of Figure 8(d).

The implementation of multiple simultaneous flows when implemented in supply graphs obviously can lead to path overloading of nodes and edges (and underutilization in cases where large oversupply exists in the supply graph). Figure

12a depicts the implementation of six simultaneous flows. Figure 12b represents a breakout of these flows on the Figure

8c supply graph. If the flows were in fact analogous to fluid flow (or circuit-switched pathways, which would be consistent with supporting multiple simultaneous dedicated paths for antenna remoting), such a drawing would represent the minimum number of actual paths required for non-congested flow. Figure 12c depicts analogously the overloading required in a packet-switched implementation. Figure 12d summarizes the comparison to node degree in the actual supply graph (Figure 8c) and that required in the circuit-switched and packet switched implementations, respectively.

Figure 12. Consequences of multiple flows. Example 8a with six flows (f1- f6). (b) Paths required for non-congested flows.

(c) overloading for packet-switched case. (d) Comparison of node degrees.

Figure 13 illustrates the consequences of different flow policies when there are multiple connection options in the supply graph. Figure 13a illustrates a case where the node pairs C,α1 and α2,B each have two connection options. In this case the flows passing through C,α1 are equally divided, as is the case with flows passing between α2,B. A different policy is depicted in Figure 13b. Consider flows involving C,α1 . Flow f1 is sent through a particular edge (PHY pair between C and α1), flows f3, f6 are sent through a different edge, and flow f5 is split between both edges. We refer to this latter case as “blended roaming”, corresponding to the analogy that in a multi-modem (multi-PHY) situation we have a choice of using one flow “service” or another (“roaming”), but could also have a case where we have simultaneous use of multiple services. Hence by policies decided in GNAT-GNAT interactions, flows can be separated and blended in many different ways.

Figure 33. Effects of flow management policies. (a) Even splitting. (b) Differential policies.

3. SOFTWARE DEFINED NETWORKS AND MULTIPLE SECURITY DOMAINS

The GNAT approach can in principle support multiple virtual networks. In this security, we discuss how these virtual networks can be isolated in separate security domains.

3.1 Basic security and domain separation concept

In brute-force security concepts, communications channels are separated with the highest level of isolation possible, often through the use of distinct physical channels. This concept is depicted in Figure 14a with an example involving three different classification levels. If nodes A and B were people, they would conduct unclassified discussions in one environment (depicted by the green “G” wire), but would enter separate environments to have classified discussions, and two different classifications are depicted with the red (R) and blue (B) wires. In office settings, a single computer monitor/keyboard/mouse are connected to multiple isolated networks (each having a different security classification), through KVM switches, which implement the concept equivalent to Figure 14a (only with an entire computer and its associated network instead of just a single wire).

An elementary concept in modern networks is that of virtualization of connections, also known as network function virtualization (NFV), which has emerged from the significant work in software defined networks (SDN) 7. In its most basic form, the actual communications channel can support several virtualized channel connections. These virtual channels are ideally secured through strong encryption, such that the bits of one are opaque to all others. With strong encryption, it is possible to intermingle communications of several virtual channels over a single actual channel. Figure 14b implements the equivalent multiplex connections. The black boxes at each end (which we shall call “interface PHYs”) support three physically distinct connections, which enforce the equivalent physical security (e.g., connecting to the red interface will result in communications that only appear at the red interface on the other side, etc.). Inside the black boxes are properly isolated encryption/decryption hardware, which combine “protected bits” (post encryption) into a common single physical connection. Of course, in the multi-channel soft flow concept, the single physical channel is itself virtualized over the collection of soft flows. Hence the post encrypted traffic is distributed based on suitable heuristics implemented by the GNAT brain processors.

Figure 14. Depiction the multiple security domains in point-to-point connection. (a) Use of physically distinct channels. (b) virtualization distinct channels using encrypted virtual flows with physically distinct endpoint connections (one for each security domain).

3.2 Separated overlay networks

Figure 15a depicts two virtual networks, each having a different security domain (“unclassified” (U) and “classified” (X)), that we wish to implement using the Figure 12b physical network. We enforce the connection to these security domains through distinguished physical connections on “interface PHYs” present with the user located at each node. Figure 15b depicts a possible implementation. Of course, in order to implement the actual security protocols, it is necessary to enforce strong encryption conventions within each of the “interface PHY” black boxes, as the bitstreams for the different security domains are then sent about the network potentially subject to interception, especially if the links are sent over free space.

The detailed implementations of the security protocols, key exchange/distribution amongst the users of each network, etc.

are not further discussed here.

Figure 15. Separation of security domains in overlay networks. (a) Unclassified overlay network (classification “U”). (b)

Classified overlay network (classification “X”). (c) Implementation in distributed GNAT network, enforcement through physical endpoint connections.

4. TIMING TRANSFER CONSIDERATIONS

With a communications system, we are concerned with the movement of information across collections of participants.

But we can consider the movement of other “things”, and we have already mentioned a fluid flow analogy (see Figure

12b). Those things could indeed be fluid (within a contiguous physical system, as to our knowledge free space fluidic transfer has not been considered as a network problem, notwithstanding fire hoses and water pistols), but also include light, sound, heat, and time as examples. Time is another type of information, but whereas a collection of bits can be abstracted from a temporal context (e.g. as the bits on a piece of paper have none), time has a special binding to the physical world. Temporal events can be inferred sometimes by the sequential nature of communications, and when order is unambiguous across the network, we can at least establish timestamps in the sense of Lamport clocks8, which may tell us which events happened before other events. When we need highly precise time knowledge across a network, however, we must establish determinism through mechanisms that are either inherent in the communications waveforms or through special dedicated clocks whose synchronization can be managed by messages in the network. While a detailed discussion is beyond the scope of this paper, we identify some simple principles that can be considered in GNAT-based network implementations.

There are at least two levels of consideration for timing transfer in GNATs. The first of these is the synchronization of a

GNAT enclave within a platform. The second is between nodes in distributed network.

At the level of the GNAT enclave, time transfer is concerned with synchronization within a platform, and the concepts here loosely follow some of the conventions established in modular spacecraft9. For in-platform synchronization, it is necessary to either employ a communications protocol with deterministic structure (such as time-triggered protocols10 or more broadly, TDMA approaches that very good timing control from a bus master) or to combine loosely coupled communications approaches with precise timing interfaces that are managed through control messages. This can be done by employing precision-timing interfaces in tandem with those for communications and a “timing device”, as shown in

Figure 16a. Here, we depict a GNAT enclave with five PHYs (a, b, c, d, and a host interface, which is an “interface PHY”).

A “timing device” is also shown. The data interfaces are depicted on black-box connections as a rectangle on interfaces between the PHY and the router. Timing interfaces are depicted by triangles; these connect between black boxes and the timing device (which itself has a data interface to the router). The triangles denote a time-sensitive interface, and common examples include the one pulse-per-second (1PPS) and 10 MHz pulsed interfaces. For greater precision, other choices are possible (e.g., frequency combs11). Whereas every black box has a data connection to the router, not every PHY has a time-sensitive interface to the timing device. This notional construct reflects that some parts of the GNAT enclave either do not support precision timing or may not require precision timing by design. Such devices cannot directly participate in the precision timing distribution mechanism.

In order to effect synchronization in the Figure 16a GNAT enclave, at least two other features are essential. The first of these is a disambiguation protocol in the form of messaging convention that precedes events related to the timing device and the chosen timing interface. An example of this is the “time at the tone” message, in which a future definition of a timing event is announced asynchronous through the data interface. A crude implementation of this concept is shown in

Figure 16b. Each black box a circuit such as this. This example circuit contains a free-running oscillator and counter, which maintains a private estimation of internal timing. If the timing interface is a 1PPS signal, the “time at the tone” message may announce that the falling edge of the 1PPS tone corresponds to a precise time value (specified by the message, decoded by finite state machine (FSM) 1, and loaded into a register). When the falling edge of the next 1PPS signal occurs, which is decoded by FSM 2, the register is copied over the free-running counter. If the black box timing is perfectly synchronized, the values should be identical, and if not, the drift is corrected.

Figure 16. Timing transfer within GNAT enclave. (a) Synchronization concept within GNAT enclave. Rectangles indicate data interfaces (e.g., Ethernet), diamonds indicate time-precise interfaces (e.g., 10 MHz and/or 1 Hz synchronization pulses). (b) Simplified state machine for timing transfer.

The second necessary feature is the notion of a timing master. The presumption is that one PHY has a connection to the

“best time” source. Again, through the data interface, the timing device can be controlled to establish the sources and sinks to timing masters and slaves, respectively..

At the network level, where a distributed collection of GNATs participates in timing transfer, analogous concepts are employed, as suggested in Figure 18. In this case, some GNAT is expected to have the “best time” and works to distribute it across the network. In Figure 18, node C has the best time, and propagates it, ostensibly through a spanning tree where node distance can be determined by a simple (e.g., Dijkstra’s12) algorithm. Even though simple, in these examples, we pre-suppose that the PHY-PHY interfaces themselves have excellent time synchronization, which is a non-trivial assumption. We may find, for example, two nearest-neighbor GNATs (such as α1 and B) have the choice of two or more

PHY-PHY interfaces, and in these cases, it is necessary to understand and select the interface providing the best time transfer option (or implement a heuristic that computes a timing estimate from all of the available sources of time).

Figure 47. Timing transfer across GNAT network. Starred node has best time knowledge; transfer follows dashed line path.

5. OTHER EXTENSIONS OF THE MODULAR ROUTING AND NETWORK CONCEPTS

While the emphasis of the present paper is on communications, the relationship of communications and information sometimes involves other functions, such as data generation, processing, and storage. These functions can have a profound impact on communications, since they can reduce as well as exacerbate the problem of information movement. Sensor sources on board a spacecraft act to increase the flow of information. On-board processing can reduce the flow of information from the spacecraft by transforming large amounts of it into more compact representations (through compression or through algorithms that extract mission-relevant insights, alleviating the need to send more voluminous raw versions). Storage can provide a way to bank information temporally, holding it for a time more conducive to effectively routing it (when timeliness permits) or in some cases to discard it (in the cases where it is advantageous to hold information conditionally until a decision point is reached). Further, in information-centric networking (ICN) approaches, such as named data networking (NDN) 13, storing data on intermediate nodes can allow more effective network utilization by alleviating the need for repetitive re-distribution from a distant source node to many destinations. These possibilities are reflected in the Figure 18 diagram of an extended GNAT enclave containing examples of these functions. The traditional “communications” PHYs (RF and optical) and “interface PHYs” can be augmented with a variety of other PHY types, to include “sensor PHYs”, “processor PHYs”, and “storage PHYs”. Electrically and logically, these appear within the enclave as simply other sources of information to be managed and routed across a network of GNATs.

Figure 18. Diagram of an extended GNAT enclave containing examples of data generation, processing, and storage

The modularity of GNAT enclaves is another straightforward extension principle. In this paper so far, a GNAT enclave has been depicted as a “hub and spoke” architecture, but in effect, the routers and PHYs can be readily composed into other topologies, and some of these possibilities are shown in Figure 19. There are several benefits to modularity and composability. The first of these is the construction of larger networks, as suggested in Figure 19a. In this sense, modularity allows GNAT enclaves to have dozens of PHYs instead of the limits imposed by the number of ports in a single router. Beyond scalability, the second important benefit is the ability to effect redundancy. One can achieve dual redundancy for example, by implementing dually redundant endpoint connections on each PHY and employing two routers, as depicted in Figure 19b. The notions of redundancy are not limited to this simple type of “A-B” redundancy, but rather, the redundancy can be shaped in an a la carte fashion. Some parts of a system can be simplex (no redundancy), whereas other parts can be triplicated, but these notions only hint of a much broader set of possibilities.

Figure 19. Extended modularity concepts for the GNAT. (a) Scalability. (b) Redundancy.

A third benefit of the GNAT approaches discussed in this paper is the modularity of security that can be implemented within a GNAT enclave. A three-level security concept is suggested in Figure 21. A “red” (R) security domain is defined for the host, processor, and one storage PHY. Meanwhile, a “green” (G) security domain is defined for the host (on a physical separate connection) and a separate, second storage PHY. Finally, a “blue” (B) security domain is defined for a host and two communications PHYs. The presumption here is that a much more extended network exists where these security concepts are mapped, but even within a single platform/GNAT enclave, it is possible to operate multiple, separated security domains.

Figure 20. Operation of multiple security domains within GNAT enclave.

5.1 Extended router concepts

In the implementation of a GNAT router, we have mostly been concerned about managing digital information, but it is possible to consider other extensions. Digital routers, especially Ethernet based routers are limited to managing information that can be rendered into packetized form. Not all information lends itself to this treatment, such as analog information for antenna remoting, or optical DWDM bundles. The use of a timing device, crudely a “timing router” has already been discussed as, in effect, an extension of the packetized router. We suggest a generalization of this concept in the form of a generalized GNAT router, shown in Figure 21. Such a router is a type of sandwich, where each layer of the sandwich has a hub and spoke structure. The primary layer is the basic router that has been discussed throughout this paper. The other sandwich layers correspond to specialty routing structures that are optionally present, and whose port connections are optionally mapped to PHYs. Each layer would support a control point connection to the primary router, which provides control access by the GNAT brain. Excepting the primary layer, the port connections for the sandwich layers would be optional for any particular PHY device.

Figure 21. Layered “sandwich” model of an extended GNAT router.

• Primary Layer. The primary GNAT router. Control plane and configuration plane are distinct service endpoints that manage “care and feeding” / “health status”, low level configuration/programming of router plane functions.

If not a separate interface, these service endpoints are accessed through packet routing operations on the high-speed digital layer.

• Power management and distribution layer. As an alternative to external power distribution approaches (where the GNAT and PHY components receive separate power connections), a power distribution “router” can be implemented. Such a “power router” would present a single power supply connection to the host and would distribute power to PHY devices. Through a control plane connection to the primary layer, current thresholds can be established and monitored, and other advanced features can be implemented.

• High-speed digital layer. Usually high-speed digital traffic would be subsumed by primary layer. Space-based implementations, however, the lack of suitable high-speed physical layer interface components (i.e., radiation tolerant) leads to the possibility of a separate high-speed digital interface. For most applications, this will be an externally scheduled switch (based on control plane signals sent to the GCP), but it is possible that the high-speed digital layer operates as a packet-routing system (not cued by control plane). This could be a single lane or group of high-speed lines, aggregated as required to achieve the scale bandwidth necessary.

• Analog layer. A layer that would be in effect a circuit-switch persistent connection between two ports supporting continuously variable small signal analog and/or microwave electronic signals. Like the high-speed digital layer, the analog layer could be a “vector” of parallel conductors on each port, feeding a analog matrix switch array

(managed by GNAT brain through the common control connection).

• Optical layer. The optical layer could be a passive, switched passive, or active optical network, the latter cases being analogous to a switched fabric (managed by GNAT brain).

• Timing layer. As previously discussed, the timing layer precision timing sourcing and sinking / distribution through a timing device.

6. SUMMARY

This paper has outlined a modular distributed network architecture that seeks to bind existing and emerging communications architectures in a unified form. It is neither the first nor last to make the claim. The global network access terminal (GNAT) concept attempts to bridge many possible networks, using them as opportunistic tunnels. The idea of soft flow is a general one, one that can be applied to wires, laser-based links, internet protocol networks, and even enterprise SATCOM systems. Inasmuch as possible, GNATs attempt to avoid inflicting new concepts in legacy systems.

It is necessary to introduce “shims”, adapters, bridges, and encapsulating concepts at some level to work, and we do not see an alternative to this for the time being. But when applied carefully, it is possible to approximate the creation of flows that take advantage of any and all available channels that might exist between two points. We assert that all communications are flows, whether between two chips within the same circuit board, or two spacecraft, we view them as black-box (from a content standpoint) movements of information. Effective security constructs are fundamentally necessary for this, and this is an area where excessive layering can render much of these concepts ineffective.