A03-ATT0009 2.5 item n DISN-EN ICAN-DI Standards Specifications v1_09f FINAL.pdf

For Official Use Only (FOUO)

STANDARDS

SPECIFICATIONS

FOR THE

DISN Enterprise Network (DISN-EN)

INSTALLATION CAMPUS AREA NETWORK

DESIGN & IMPLEMENTATION (ICAN-DI)

FOR THE

IMPLEMENTERS OF THE DISN-EN INFRASTRUCTURE

Artifact Version 1.09f FINAL

(5 December 2016)

DISN-EN ICAN-DI DEVELOPED BY USARMY PEO-EIS

V1.09F FINAL UPDATED: 5 DECEMBER 2016

Summary Requirements & Technical Specifications | DISN-EN Page For Official Use Only

UNCLASSIFIED

The DISN-EN (Enterprise Network) ICAN-DI (Design and Implementation)

Standards & Specifications facilitate the systematic, stable and sustainable IT

Infrastructure instantiation of Base/Post/Camp/Stations that enables the Joint

Enterprise Vision. This artifact lays out the ICAN technology portion/segment of the overall DISN-EN Architecture for implementers. This artifact supersedes all previously issued ICAN implementation standards and specifications (i.e.: TR No.

ELIE-ICE-TI 10-001 and DISN-EN ICAN-DI Standards and Specifications v1.08).

Adoption of these standards and specifications is done on a service agency-by-service agency basis in support of a unified whole, and is currently adopted as the

ICAN Implementation Standards and Specifications by:

ROBERT S. FERRELL

Lieutenant General, USA

ARMY CHIEF INFORMATION OFFICER/G-6

ARMY CIO/G6

GARRETT S. YEE

Major General, USA

ARMY LEAD FOR NETWORK MODERNIZATON

ARMY CIO/G6

DOCUMENT REVISION TRACKING ______________________________________________________ 6

1.0 SCOPE _______________________________________________________________________ 8

2.0 INTRODUCTION _______________________________________________________________ 8

2.1 Installation Campus Area Networks _____________________________________________________9

2.2 DISN-EN MPLS Service Network ________________________________________________________9

2.3 DISN-EN Joint Regional Security Stacks __________________________________________________9

2.4 DISN-EN Joint Management Capability __________________________________________________9

3.0 INSTALLATION CAMPUS AREA NETWORKS ________________________________________ 10

3.1 Introduction _____________________________________________________________________ 10

3.2 Description of Intent _______________________________________________________________ 11

3.3 Physical Connectivity Standards______________________________________________________ 11

4.0 CONFIGURATION SPECIFICATIONS _______________________________________________ 18

4.1 Device Host Naming Standards ______________________________________________________ 18

4.1.1 Site-ID Abbreviations _______________________________________________________________________ 19

4.1.2 EAS (Access Switch) Hostnames ______________________________________________________________ 19

4.1.3 ADS (Distribution Switch) Hostnames __________________________________________________________ 20

4.1.4 ACS (Core Switch) Hostnames ________________________________________________________________ 20

4.1.5 L2BS (Layer 2 Bridging Switch) Hostnames ______________________________________________________ 21

4.2 Virtual Local Area Network Numbering Schema _________________________________________ 22

4.3 Virtual Local Area Network Naming Schema ____________________________________________ 24

4.4 Base Executive Agent VLAN Subnet Guidelines __________________________________________ 25

4.5 Access Switch, Port Configuration Standards ___________________________________________ 26

4.5.1 Access Ports: (Downlinks) ___________________________________________________________________ 26

4.5.2 ADS and ACS Facing Ports: (Uplinks) ___________________________________________________________ 27

4.5.3 Management Port: (Console and MGMT IP Address) ______________________________________________ 28

4.6 Distribution Switch, Port Configuration Standards _______________________________________ 28

4.6.1 EAS Facing Ports: (Downlinks) ________________________________________________________________ 28

4.6.2 ACS and JB-CE Facing Ports: (Crosslinks/Uplinks) _________________________________________________ 29

4.6.3 Management Port: (Console and MGMT IP Address) ______________________________________________ 29

4.7 Layer 2 Extensions ________________________________________________________________ 29

4.8 Data Link Layer Encryption Standards _________________________________________________ 30

4.9 Root Bridge/Spanning Tree Standards _________________________________________________ 31

4.9.1 MST Regions ______________________________________________________________________________ 32

4.9.2 MST Groups ______________________________________________________________________________ 32

4.9.3 MST Naming Standard: _____________________________________________________________________ 32

4.9.4 Root Bridges ______________________________________________________________________________ 33

4.10 Port Description Standards __________________________________________________________ 33

4.11 Telecommunications Room Patch Cable Labeling Standards _______________________________ 34

4.12 ICAN Class of Service (CoS) Standards _________________________________________________ 34

4.12.1 Layer 2 Ingress CoS and DSCP Trust Model ___________________________________________________ 34

4.12.2 Layer 2 Egress Queuing ___________________________________________________________________ 34

4.12.3 Other Considerations/Comments ___________________________________________________________ 35

4.13 ICAN Device Management Standards _________________________________________________ 35

5.0 APPLICABLE DOCUMENTS ______________________________________________________ 38

5.1 Government Document ____________________________________________________________ 38

5.2 Non-Government Document ________________________________________________________ 38

6.0 GLOSSARY/ACRONYMS ________________________________________________________ 40

7.0 APPENDIX ___________________________________________________________________ 2

7.1 Overarching Specifications and Standards Requirements (Applicable to all ICAN devices) _________2

7.1.1 Technology Refresh: _________________________________________________________________________ 2

7.1.2 IPV6 Compliance ___________________________________________________________________________ 2

7.1.3 Manufacturing Requirements: ________________________________________________________________ 2

7.1.4 IEEE Standards: _____________________________________________________________________________ 2

7.2 Hardware Requirements: _____________________________________________________________3

7.2.1 For Chassis Based Systems: ___________________________________________________________________ 3

7.2.2 For All Systems: ____________________________________________________________________________ 3

7.3 Software Requirements: ______________________________________________________________3

7.4 Network Management Standards: ______________________________________________________3

7.5 Security Standards: __________________________________________________________________4

7.6 QoS (Quality of Service) Standards: _____________________________________________________4

7.7 IPv6 Standards: _____________________________________________________________________4

7.8 EAS (Edge Access Switch) Requirements _________________________________________________4

7.8.1 Standard Edge Access Switch __________________________________________________________________ 5

7.8.1.1 Standard Edge Access Switch General System Requirements ______________________________________ 5

7.8.1.2 Standard Edge Access Switch Uplink Ports: _____________________________________________________ 5

7.8.1.3 Standard Edge Access Switch Downlink Port Requirements: _______________________________________ 5

7.8.2 Small Form Factor EAS _______________________________________________________________________ 6

7.8.2.1 Small Form Factor EAS General System Requirements ____________________________________________ 6

7.8.2.2 Small Form Factor EAS Downlink Port Requirements: ____________________________________________ 6

7.9 ADS (Area Distribution Switch) Requirements ____________________________________________7

7.9.1 General Requirements _______________________________________________________________________ 7

7.9.2 Uplink Port Requirements: ___________________________________________________________________ 7

7.9.3 Downlink Port Requirements (10GbE to EASs) ____________________________________________________ 7

7.10 ACS Switch Requirements ____________________________________________________________8

7.10.1 General Requirements: ____________________________________________________________________ 8

7.10.2 Uplink Port Requirements: _________________________________________________________________ 8

7.10.3 Downlink Port (10GbE ports to EAS or ADS): ___________________________________________________ 8

7.11 L2BS Switch Requirements ____________________________________________________________9

7.11.1 General Requirements_____________________________________________________________________ 9

7.11.2 Uplink Port Requirements: _________________________________________________________________ 9

7.11.3 Downlink Port Requirements (10GbE to EASs) __________________________________________________ 9

7.11.4 10GBASE-LR, SFP+ Optics (LC) _______________________________________________________________ 9

DOCUMENT REVISION TRACKING

Disclaimer: The user of this document is responsible for obtaining the latest document revision released for use on the DISN-EN. Documents released for use are maintained under configuration management control and are archived in the configuration management library. Associated document revisions will be made in accordance with the appropriate Configuration and Quality Management processes. Configuration management control will be maintained until the document is downloaded from the configuration management library and either stored locally to a personal computer (PC), printed, or copied. Documents that are no longer under configuration management control are to be treated as For Official Use Only (FOUO) in accordance with DISN-EN security procedures. Documents superseded by later revisions are designated obsolete and are no longer appropriate for use on the DISN-EN.

CR # REVISION Summary of Change

Version 1.00 2/11/2014: First Non-Draft Release

Version 1.01 4/18/2014: RFI Response, Market Research, and Collaborations with I3MP and P2E to modify Appendix-A

Remove Next-Day Replacement Requirement (moves to SOW)

Remove ESA Exemption (moves to SOW)

Modify Phrase “Industry Standard CLI” to simply “CLI”

Modify ACS Max Port Count 10G Line Rate (330 to 256)

Exchange the phrase “oversubscription” to “non-blocking”

Remove the MACsec requirements from ACS and ADS Devices

Version 1.02 4/23/2014: Market Research; Updates to Appendix A

Modify 1Gbps/10Gbps Dual-Speed Requirement (Only 10Gbps Required)

P2E Recommended Nomenclature Changes (Cosmetic)

Update terminology of “E-PE” to “JB-CE” (Sync up with MPLS Standards)

Version 1.03 5/5/2014: Revised Signature Page (From Mr. Bezwada to Mr. Blohm)

Version 1.04 6/25/2014: NETMOD-C Surveys

Add Fiber SFP-based Switches to EAS Section of Appendix A

Separate Appendix A from the Main ICAN-DI to allow easy reference

Version 1.05 7/28/2014: Draft RFQ Responses

Limit 802.1AE MACsec requirement for downlink EAS ports only

Change ADS certification to ASLAN-Certified

Version 1.06 8/18/2014: RFQ Response

Remove MACsec requirement from L2-BS switch specifications

Version 1.07 8/20/2014: Rephrases section 1.5.2.1 (applicable to min required ports only)

Version 1.08 8/28/2014: Modified section 1.7.2 (Mgmt. Port Requirement set to 10/100 Mbps)

CR # REVISION Summary of Change

Version 1.09 6/10/2016: Updated Artifact with PEO EIS CIO ERB Change Requests

Version Revision & Outcome from PEO EIS CIO ERB/CCB Meeting

In light of Tier-1 vendors now being able to support hostnames longer than 32 characters, Hostname Max Length and Structure has been made less Restrictive.

Legitimized pervasively implemented but not previously approved use of native vendor First Hop Gateway Redundancy protocols instead of strict use of VRRP.

Legitimized pervasively implemented but not previously approved use of “L2BS” instead of “b01” in hostnames of L2 Bridging Switches.

Legitimized pervasively implemented but not previously approved use of “.agency.mil” suffix to the names of the MST instances, designating the executive agent of the B/P/C/S.

Legitimized pervasively implemented but not previously approved out of order use of port descriptions. The order has been changed to fit what is in the field.

Clarified hostname suffix, (-1 vs. -1s1)

Changed Multicast Ingress Port Policy Limit from 20% to 5%

Removed Q-in-Q as a solution to L2 tunneling (scalability problem identified)

Updated Sections & Diagrams – Reflective of Most Recent Naming/Concepts

Updated ICAN Switch Technical Requirements (added SDN Compatibility)

Added Numbering to Sections, Diagrams and Appendix for Easier Reference

Updated Glossary and Acronym List

Version 1.09c 9/7/2016: Updated Artifact with NETCOM Provided Change Requests:

Added language to reinforce the Joint nature of the document.

Expanded Do’s/Don’ts section to further clarify intended architecture.

Updated VLAN Numbering Schema to reflect more granular details.

Updated Port Configuration Standards to capture most recent STIG requirements.

Refined Layer 2 Extension language to enhance clarity.

Enhanced guidance on labeling standards.

Updated Appendix to reflect main document changes.

Version 1.09d 9/20/2016: Removed MACsec requirement for EAS downlink ports in Appendix.

Version 1.09e 10/20/2016: Updated per vendor questions. VLAN limit of 4094, Minimum Jumbo Frame size of 9000bytes

Added Small Form Factor EAS requirements in 7.8

Version 1.09f 11/22/2016: Updated hostname guidance to reflect decision by NETCOM to precede the hostname with the device make and model.

Added HAIPE/CT to VLAN chart per request.

Revision Policy: This document is revised and updated as needed through the collective agreement of the Service Agencies affected by these standards.

1.0 SCOPE

The scope of this document is focused on providing relevant guidelines, standards and specifications as they pertain to the Defense Information Systems Network - Enterprise Network (DISN-EN) Architecture. This artifact is intended to service Army, Air Force, Marines and the Navy by providing a scalable means of standardizing Base/Post/Camp/Station infrastructure and infrastructure configurations. This artifact is a key component in collectively reaching Enterprise maturity under a unified management framework and thereby the pathway toward achieving the Joint Information Environment (JIE). This artifact is intended to be a living document with need for periodic updates, i.e.: when lifecycle induced technological leaps are introduced into the architecture.

DISN-EN

Standards & Specifications...

DISN-EN MPLS

Network

DISN-EN Datacenter Services

DISN-EN JRSS/JMS

Services

Device Host Naming Standard

VLAN Termination Standard

CE Router Port Configuration Standard

VRRP Standard

VRF Types

VRF Naming Standard

IGP Standards

MPLS Signaling Standards

Site to Site VPN Standard Collection Router Standard

Route Distinguisher / Target Standard

Enterprise VRF Routing Standard

JRSS Capability Specs

Device Host Naming Standard

Firewall Standards

IDS Standards

Client Access Function Standards

Rule-Set Standards

IPS Standards

Routing/Switching Standards

Physical Connectivity Standards Device Host Naming Standard

VLAN Numbering Standard

VLAN Naming Standard

VLAN Subnet Guidelines

Access Switch, Port Configuration Standards

Distribution Switch, Port Configuration Standards

Root Bridge / Spanning Tree

Layer 2 Extensions

Physical Connectivity Standards

DISN-EN ICAN-DI

(Intra Base / Campus Area Network)

Port Description Standards DISN-EN Mgmt Network

Connectivity Standards

Physical Connectivity Standards

MGMT Capability Specs

MPLS Quality of Service

Multicast Services

Covered in this Artifact

Data Link Layer Encryption Standards (MACSEC)

Class of Service / Quality of Service Standards

ICAN Device Management Standards

Applicable Documents

L2BS (L2 Bridging Switch) Product Specs

ACS (Area Core Switch) Product Specs ADS (Area Distribution Switch) Product Specs EAS (Edge Access Switch) Product Specs

Applicable Documents

Collection Router Specs

Category A-C Router Capability Specs

AAA / TACACS Permissions

DHCP Forwarding

CE Router Port Description Standard

Campus Intra VRF Routing Standard

Multicast Standard

Applicable Documents

Covered in Distinct Artifact Covered in Distinct Artifact Covered in Distinct Artifact

Base Abbreviations Standard

Patch Cable Labeling Standards

Appendix-A Device Requirements Specifications

Physical Connectivity Standards

Device Host Naming Standard

Rack Refercing Standard Rack Refercing Standard

Device Labeling Standard

Cabling Standards

Type of Switching Fabrics

Server Virtualization

Types of Datacenters

Logical Connectivity Standards

Link Bundling/Aggregation

Applicable Documents

Datacenter Switch Specs

Datacenter Server Specs

Other Items…

Other Items…

2.0 INTRODUCTION

The DISN-EN is a multi-agency solution that creates an Enterprise Information Technology (IT) Environment across all participating agencies relying on a standardized core Internet Protocol (IP) transport service network (Multi-Protocol Label Switching (MPLS) Service Network), with regionally collapsed security stack services (Joint Regional Security Stacks (JRSS)) and a standardized Base/Post/Camp/Station (B/P/C/S) campus area distribution network (Installation Campus Area Network (ICAN)). The DISN-EN capability infrastructure advocates for a Joint Management solution (DISN-EN Management) that delivers true multi-tenancy in a shared operating environment. All of these components are intended to work together in a dovetailed manner in order to realize the Enterprise vision (active steps toward the Joint Information Environment (JIE)).

Diagram 1.0 – ICAN-DI Contents

2.1 Installation Campus Area Networks

The ICANs provide high speed local IP switched transport services within the B/P/C/S, from the Top Tier Base Nodes (TTBNs - first hop routers) down to the user access devices like desktops and printers. The ICAN infrastructure follows a classic framework of Core, Distribution and Access and is therefore comprised of Area Core Switches (ACS), Area Distribution Switches (ADS) and Edge Access Switches (EAS). The DISN-EN architecture observes a highly stable systematic hierarchy for physical and logical interconnectivity between ACS’, ADS’ and EAS’. Each ACS Pair creates a physically distinct Layer 2 Domain that services the layer 2 production VLANs within that network segment or region. B/P/C/S’ may have as many ACS Pairs as they reasonably need to meet the density demands of the ICAN. In instances where a single VLAN must be shared across multiple L2 Domains for whatever reason, a tightly controlled Layer 2 Bridging Switch (L2BS) device is to be used to ensure that the network remains loop free and stable. The standards and technical specifications as it relates to the “ICAN” are captured in this artifact.

2.2 DISN-EN MPLS Service Network

The DISN-EN MPLS Service Network provides unclassified Wide Area Network (WAN) label switched transport connectivity within and between service regions across the globe. IP Transport within and between service regions utilize a layered MPLS solution in order to preserve the Defense Information Systems Agency (DISA) circuit provider transport backbone (Inner MPLS Core) and keep it logically and administratively distinct from the Common Services Transport Backbone (Outer MPLS Core). The Common Services Transport Backbone is where the ICAN traffic, Datacenter traffic, the JRSS and Joint Management System (JMS) services reside... building toward the Joint "Common" Information Environment. The "Inner Core" is primarily comprised of DISA Provider nodes (P-nodes) and DISA IP Transport Provider Edge (IPT-PE) routers interconnected through an extensive fiber optic/Dense Wavelength Division Multiplexing (DWDM) infrastructure. The "Outer Core" is primarily comprised of the Joint Regional Customer Edge (JR-CE) routers, the Joint Base Customer Edge (JB-CE) routers and the Joint Base Collection Router (JB-CR) routers. The standards and technical specifications as it relates to the "Outer Core" are captured in a distinct artifact.

2.3 DISN-EN Joint Regional Security Stacks

The JRSS’ provide network traffic intrusion detection, intrusion prevention, multi-tier firewall services, content filtering, logging, packet capture services and inter Virtual Routing and Forwarding (VRF) policy filtered routing services for all participating B/P/C/S. The JRSS has been designed to provide these security services to all participating Department of Defense (DoD) Agencies no matter what B/P/C/S they reside on within a JRSS’ regional boundary. The JRSS has been accepted by the DoD - Chief Information Officer (CIO) and DISA as a significant part of the instantiation of the Single Security Architecture envisioned to realize the JIE. Inter-regional IP traffic traverses JRSS control points enabling United States Cyber Command (USCYBERCOM) end-to-end cybersecurity visibility and control. In relation to the ICAN, the JRSS is where logically separated security zones are permitted to change security contexts (be routed together), assuming the individual security zone policies permit such activity. The intent of the JRSS’ is to enable enterprise security services for DoD and de-duplicate locally operated security stacks across DoD (previously identified as Top Level Aggregation (TLA) stacks). The standards and technical specifications as it relates to the “JRSS” are captured in a distinct artifact.

2.4 DISN-EN Joint Management Capability

The JMS provides DISN-EN Fault, Configuration, Accounting, Performance, Security and Capacity Management.

It is the first instantiation of the envisioned JIE Enterprise end-to-end management solution that relies on shared management platforms through virtualization and enterprise level role based access (moving away from duplicative local management systems). JMS reaches its managed elements via the Joint Management Network

(JMN) which is a hybrid in-band/out-of-band transport network augmenting the DISN-EN, ensuring critical items are always accessible to its operators. Managed elements that are configured with in-band only management connectivity (i.e.: an access switch), accept the risk of rare but possible temporary loss of management connectivity by design due to a cost benefit analysis that indicates that reserving or pulling separate fiber to such a device is cost prohibitive. JMS provides a portal based interface to enable management views from anywhere to authorized operators and provides delegated access to trusted operators on devices they are functionally authorized to operate. The standards and technical specifications as it relates to the “JMS” and “JMN” are captured in a distinct artifact.

3.0 INSTALLATION CAMPUS AREA NETWORKS

3.1 Introduction

The Installation Campus Area Networks (ICANs) provide the campus distribution IP connectivity from the TTBNs down to the user access devices like desktops and printers. The ICAN infrastructure is comprised of Area Core

Switches (ACS), Area Distribution Switches (ADS), where they are deemed appropriate by the Executive Agent of the base, and Edge Access Switches (EAS) directly servicing users, printers and other networked devices. The

DISN-EN architecture observes a systematic hierarchy for interconnectivity between ACS’, ADS’ and EAS’. Each

ACS Pair will create a physically separate Layer 2 Domain, where inter-area layer 2 VLAN extensions would be handled through a tightly controlled L2BS device.

Bridging

L2BS

ACS

EAS

ADS

Core

Distribution

Access

Industry Standard Function

Layer 2 Bridging Switch

Area Core Switch

Area Distribution Switch

Edge Access Switch

Joint Base Customer Edge

Layer 3

Layer 2

Base Executive Collection Router

JB-CE

BE-CR

Diagram 3.1 – ICAN Acronyms

3.2 Description of Intent

The intent of the ICAN is to establish a clear understanding of the planned campus area delivery network infrastructure on B/P/C/S through the use of L2BS, ACS, ADS and EAS devices, to include any remote instantiations of ACS’ and EAS’ logically attached to a local ICAN, framed within defined device requirements and configuration standards. A remote ACS follows the same hierarchical physical and logical architecture as a typical ACS, however it is physically outside the boundaries of the normal B/P/C/S infrastructure yet it is physically connected back to the JB-CE routers at the parent B/P/C/S for Layer 3 services. A remote ACS offers infrastructure flexibility to implementers so that they don’t have to have a set of Layer 3 aggregation devices at a remote (small/medium) size site. External ACS locations are locations that service a single L2 Domain, but do not have a JB-CE pair of routers at that location. A good example of this would be sites that are NOT already on the DISA backbone and connect to a router on another installation for access to the DISN-EN. A remote EAS is also provided as an option for mini remote sites, comprised of a single switch or stack of switches. Remote EAS devices uplink to the Base Executive Collection System for their first hop routing. Please note that all interconnectivity between a remote and its parent site, must adhere to relevant transport security (encryption) standards.

JB-CE JB-CE

B/P/C/S

Routed VLAN Interfaces

DISA MPLSDISA MPLS

100 GB

10GB

ACS ACS ACS ACS

1 Virtual Switch (Stacked)

1 Virtual Switch (Stacked)

48 Port POE SW-1

48 Port POE SW-2

48 Port POE SW-1

48 Port POE SW-2

48 Port POE SW-3

1 Physical Switch (Not-Stacked)

48 Port POE SW-1

EAS 1 Member EAS 2 Member EAS 3 Member

B ac kp la n e

In te rc o n n e ct C ab le

B ac kp la rc o n n e ct C ab le

100GB 100GB

ACS ACS

100 GB

100GB 100GB

Detailed Look at Typical EAS Connectivity Mini Remote L2 Only Site etc… Access Switches

Access Switches

Access Switch Access Switches Access Switches

ADS EAS

ADS ADS

Access Switches Access Switches

EASEASEAS

Access Switches

DISN-EN MPLS

DISN-EN CAMPUS NET

L2 Only

Label Switched

L2BS

ADS only used where constraints on passive infrastructure prevent direct EAS to ACS connectivity.

Second ADS if passive infrastructure supports it.

Second link if passive infrastructure supports it.

L2 Domain / Area 001 (example only) L2 Domain / Area 002 (example only)L2 Domain / Area 003 (example only)

L2BS used for Inter MST Region VLAN Bridging Only

Base Executive Collection System

EAS

Encryption

Encryption

Used for external locations that require no more than one switch stack worth of service ports. A remote EAS does NOT have any other switches or hubs hanging off of it, only directly wired/ attached end node devices.

Options:

MACSEC

APPLIANCE BASED

OTHER

Small / Medium Remote L2 Only Site

L3 Only

EncryptionEncryption

Encryption Encryption

Encryption

10GB 10GB 10GB 10GB 10GB

Etc… up to 7

Encryption Traffic for L2BS Only (if needed)

Data & In-Band Mgmt Traffic

Remote Small / Medium sites without a local set of routers for first hop routing may be configured as remote ACS Pairs with their own ACS, ADS and EAS devices. These remote ACS Pairs would be treated just like every other

ACS Pair supported by the Parent Base/Post/Camp/Station.

Options:

MACSEC

APPLIANCE BASED

OTHER

Note on Encryption

In CONUS, encryption is required on links between B/P/C/S where DISA’s infrastructure is NOT utilized (i.e. leased circuits), otherwise it is a DISA responsibility/provided service. In OCONUS, encryption is required on all links between B/P/C/S.

EAS EAS

See Collection Architecture for Details

Diagram 3.2 – Overall Installation Campus Area Network (ICAN) Diagram

3.3 Physical Connectivity Standards

The ICAN is comprised of not only the active (switches, routers and encryption) components but also the existing physical cable plant available to the local B/P/C/S. It is recognized that the ICAN inherits its existing B/P/C/S’ current level of physical cable plant, space and power limitations. Due to this known set of limitations, the ICAN administrators will require flexibility in how their networks are to be implemented.

The table below provides the maximum number of recommended EAS switches in a given switch stack. Both an availability impact calculation as well as the target bandwidth availability calculation per user point to the same recommended maximum number, mainly that there ought to be no more than seven (7) 48 port switches stacked in any given stack. While most vendors allow more switches to be added to the stack from a technical perspective, it undermines availability impact risk management and per user target bandwidth availability when the recommended maximum number is exceeded.

Diagram 3.3.1 – Stacking Limit, Availability & Bandwidth Analysis

The Do and Don’ts of Physical Connectivity

Diagram 3.3.2 – Example of what NOT to do (EAS Uplinks to Different L2 Domain ACS Switches)

Please note that by design an ACS pair defines a Layer-2 Area. Each Layer-2 Area hosts Base Unique as well as L2 Area unique VLANS. If an EAS is connected to more than a single Layer-2 Area, not only will spanning tree not function as planned but you may also run into problems with VLAN tag reuse. As a point of architecture, EAS devices are only authorized to connect to ADS and ACS devices within their own Layer-2 Area.

Uplink Bandwidth of Stack: 10 Gigabits Uplink Bandwidth of Stack: 10 Gigabits

Ports per Switch in Stack: 48 Ports Ports per Switch in Stack: 48 Ports

Number of Switches in Stack: 7 Switches Number of Switches in Stack: 7 Switches

Target Availability: 99.99% UCR Target Bandwidth/User: 30 Mbps

Avg User Ports on Base: 22,633 People HD Video Stream: 8.38 Mbps

Total Avg User Hours/Yr: 198,265,080 Hours Phone Traffic: 0.10 Mbps

Per Person Allowed Outage/Yr: 52.56 Minutes User Data Transfere Traffic: 10.00 Mbps

Total Outage Time Permited/Yr: 19,827 Hours Total Bandwidth Needed /User: 18.48 Mbps

Estimated Avg Outage Time: 30 Minutes (UCR) Possible Number of Users/Stack: 336 Possible Users

Possible Number of Users/Stack: 336 Possible Users Total Uplink Bandwidth Target: 9.84 Gigabits

Outage Impact: 168 Hours Total Uplink Bandwidth Needed/Stack: 6.07 Gigabits

Per Incident Impact to Outage Time Permitted/Yr: 0.84735% EAS Bandwidth Utilization (Daily Use): 6.56250%

Percent of EAS' that may experience Outage/Yr: 0.34 Target is no less than 1/3 of stacks EAS Bandwidth Utilization (Max Use): 60.65063%

Availability Impact Analysis Bandwidth Impact Analysis

Diagram 3.3.3 – Example of what NOT to do (ACS Interconnectivity)

Diagram 3.3.4 – Example of what NOT to do (ACS to JB-CE Interconnectivity)

In JRSS locations, it is possible to have both a JB-CE (serving the user community) and a JR-CE (serving the JRSS stack and regional datacenters) on the same B/P/C/S. In these rare cases, be sure to connect your ACS devices to the JB-CE set of MPLS routers.

Diagram 3.3.5 – Example of what NOT to do (ACS Dual CE Uplink Connectivity)

JB-CE JB-CE

B/P/C/S Routed VLAN Interfaces

DISA MPLSDISA MPLS

100 GB

ACS ACS

100 GB

General Look

Access Switch Stack

EAS EAS

L2 Domain / Area 001 (example only)

Access Switch Stack

ACS ACS

General Look

ACS ACS

General Look

L2 Domain / Area 002 (example only) L2 Domain / Area 003 (example only)

ADS

Access Switch Stack

EAS EAS

Access Switch Stack

ADS

General Look

Access Switch Stack

EAS EAS

Access Switch Stack

Diagram 3.3.6 – Example of what NOT to do (ADS used as a fiber coupler)

B/P/C/S Routed VLAN Interfaces

DISA MPLSDISA MPLS

100 GB

ACS ACS

100 GB

General Look

Access Switch Stack

EAS EAS

L2 Domain / Area 001 (example only)

Access Switch Stack

ACS ACS ACS ACS

General Look

L2 Domain / Area 003 (example only)

ADS

General Look

Access Switch Stack

EAS EAS

Access Switch Stack

General Look

L2 Domain / Area 003 (example only)

General Look

EAS EAS

Access Switch Stack

ADS

Diagram 3.3.7 – Example of what NOT to do (EAS uplinked via ACS and ADS at the same time)

The architecture does not legitimize “daisy chaining” of EAS devices, however it does recognize that in some locations there exists only limited amount of fiber to designated service areas. In the event that there is no other option but to daisy chain, the deviation needs to be clearly documented and provided in Army’s case to NETCOM and in Air Force’s case to AFSPC for awareness / risk management.

B/P/C/S Routed VLAN Interfaces

DISA MPLSDISA MPLS

100 GB

ACS ACS

100 GB

General Look

Access Switch Stack

EAS

EAS

L2 Domain / Area 001 (example only)

ACS ACS ACS ACS

General Look

L2 Domain / Area 003 (example only)

General Look

Access Switch Stack

EAS EAS

Access Switch Stack

General Look

L2 Domain / Area 003 (example only)

General Look

EAS EAS

Access Switch StackAccess Switch Stack

EAS

Access Switch Stack

EAS

Diagram 3.3.8 – Example of what NOT to do (Directly interconnecting or daisy chaining EAS’)

The Do’s and Don’ts – A Summary

Permissible:

EAS’ will only connect to a single L2 Domain/Area (via an ADS or an ACS Pair)1.

EAS’ will rely on switch-stacking as much as possible, up to 7 edge switches maximum per stack (creating virtual edge switches) to minimize uplink requirements.

o Please note that if a vendor’s equipment has a lower limit, that limit MUST be followed.

o Please note that stacking implies the use of built in stacking ports connected to the stacking backplane.

Remote EAS devices apart from being part of a remote ACS Pair, may only connect to the Base Executive

Agent Collection System and rely on link encryption due to being external to the local ICAN (unless waivered).

ADS devices do not directly connect to the JB-CE routers; they connect only to ACS devices.

o Proper ADS/ACS relationship: within the economy of the same L2 Domain.

ADS devices may only connect to a single L2 Domain/Area (ACS Pair)1.

Switch interconnections should be made utilizing Single Mode Fiber where possible. If not possible, it should be documented for remediation and reviewed annually.

ACS’ uplink to its upstream/host set of JB-CE routers.

The two ACS aggregation switches within a given L2 Domain will be interconnected using link aggregation of two physical ports on each switch (from different modules)2.

o If a single service module exists in an ACS due to scarce port density requirements in a L2

Domain/Area, a non-redundant inter-connection to the peer ACS is sufficient because the ACS that loses its service module is automatically removed as a valid path to the Next Hope Redundant Routing Protocol defined Route Point.

ACS switches from different L2 Domains/Areas shall NEVER be directly interconnected3.

o Each ACS within an L2 Domain/Area will only uplink to its designated JB-CE router and port; no cross-connects from an ACS to both JB-CE routers in the JB-CE pair.

Please note that the Base Executive Agent can co-host ACS devices in the same facility as the JB-CE routers to aggregate ADS and EAS devices that are wired to the JB-CE rooms.

The L2BS device singly attaches to any ACS Pair, this lack of connectivity redundancy is done on purpose/by design (it is not a flaw), in order to prevent accidental loops between L2 Domains4.

The L2BS device connects to each JB-CE router, using sub-interfaces on those physical ports on the JB-CE to service L2BS VLANs. L2BS VLANS are not bridged from ACS’ to JB-CE routers.

Using Remote ACS Pairs (with redundant uplinks) o A remote site may be treated as a single or as multiple ACS pairs (independent L2 Domain), uplinked to a parent B/P/C/S JB-CE set of routers for route points (L3 Services).

1 Please note that each ACS Pair represents the top aggregation nodes of all the underlying subordinate switches (potentially multiple ADS and a host of EAS devices) and are treated as belonging to the same Layer 2 Domain.

2 This is required to avoid “split brain routing” with whichever Next Hope Redundancy Protocol is used.

3 Each ACS Pair is representative of its own Layer 2 Domain/Area within the overall Campus Network, this allows for reuse of VLAN tags within a campus and it significantly stabilizes STP. Please see the Layer 2 Extension section to see how the architecture gracefully handles the interconnectivity of nodes that must be bridged across the campus.

4 This solution was developed and successfully used by the Pentagon to limit cross-wedge broadcast storms/service impacts. The likelihood that a misconfiguration will occur over time is high, minimizing the chance of loops between areas manages both the risk and the potential impacts.

o These remote sites should only connect to a single parent site pair of JB-CE routers and it needs to be able to do so in a redundant manner (at least a pair of fiber to each JB-CE router).

o Because the local and the parent sites are geographically distinct, please remember that links leaving a B/P/C/S must be encrypted according to transport security laws currently in place.

Tying in a collection of remote switches from a site without redundant uplinks o In the event that a remote site is comprised of a collection of switches but its aggregation switch only has a single pair of fiber to a parent site, it will need to aggregate to the Base Executive Collection System of the parent site.

o The aggregation switch in this instance is treated as an ADS switch.

o Because the local and the parent sites are geographically distinct, please remember that links leaving a B/P/C/S must be encrypted according to transport security laws currently in place.

Using a Remote Edge Access Switch (EAS) o For truly small sites, a single EAS (which may be stacked) may be used. Remote EAS switches aggregate to the Base Executive Collection System of the parent site.

o Because the local and the parent sites are geographically distinct, please remember that links leaving a B/P/C/S must be encrypted according to transport security laws currently in place.

In all cases, when an exception is taken to the model architecture it must be documented and reported to the Base Executive Agent Director so that an exception report may be recorded and filed for risk assessment and possible future correction.

Impermissible:

EAS’ connected to two L2 Domain/Areas creating a L2 bridge.

EAS’ utilizing any other ports other than stacking ports to stack switches.

ADS devices directly connected to the JB-CE routers rather than ACS devices.

ADS devices bridging L2 areas.

L2BS switches not redundantly uplinked to JB-CE routers.

L2BS switches redundantly paired to both ACS’ in an ACS Pair to bridge between L2 areas.

The connection of ACS’s in two separate L2 areas should occur only through a L2BS or via the JB-

CE routers. They should not be directly interconnected.

The “daisy-chaining” of EAS’ is not acceptable. EAS’ should only connect to ACS or ADS infrastructure.

4.0 CONFIGURATION SPECIFICATIONS

4.1 Device Host Naming Standards

The DISN-EN maximum recommended switching platform hostname length is up to 63 characters5. Although the same format is applied to all switches within the enterprise environment (EAS, ADS, ACS, and L2BS’s), their format between (Edge/Distribution) vs. (Core) slightly varies to account for the relevant information needed by operations. Please note that the JB-CE hostname will adopt elements of the switch hostname format explained below, making all campus network devices follow a similar logic in naming convention.

The Site Identification (ID) is a globally unique, alpha/numeric shorthand for the name of the B/P/C/S6. The globally unique Site-IDs are DISA assigned (to include identifying Site-IDs for non DISN resident sites), currently handled and issued through the Joint Migration Team a subcomponent of DISA Global (DISA’s JMT in support of infrastructure migration and modernization). See section 4.1.1 for more details on Side-ID Abbreviation.

The hostname relies on human-friendly naming down to the room level. More specific location information may be included in the executive banner to help locate a device. The fully qualified location of the device therefore is accessible upon login, via the executive banner7 (see below) as well as the Simple Network Management Protocol (SNMP) System Location variables stored on the local switch in question.

Diagram 4.1 – Banner and SNMP Diagram

5 Contemporary switches can have hostnames up to 255 characters, however DISN-EN device hostnames shall not exceed 63 characters in length.

6 Keeping the name humanly recognizable offers operations personnel clarity while effecting changes and troubleshooting.

7 This banner is not displayed until AFTER login, thus not revealing any data to those without login credentials.

4.1.1 Site-ID Abbreviations

Within the ICAN, Site-ID abbreviations (a.k.a Base abbreviations) are included in L2 Device Hostnames and VLAN names. Site-IDs are comprised of five characters, the first three being specific to the Site Name and the last two designating geographic locations on the globe. These same exact Site-ID Abbreviations are also employed systematically on the Customer Edge Routers, Collection Routers, VRF Names under certain conditions and other Site Related Active IT configurations or device hostnames. Using ONE agreed-upon globally unique Site-ID enables enterprise operators to quickly find all site related items through this shared identifier. A list of pre-defined and approved Enterprise Site-ID abbreviations is available through DISA Global’s JMT maintained JMS secure wiki site (https://wiki.jmn.mil) accessible only from within the secure Joint Management Network (JMN) which can be reached through secure VPN (access must be requested/approved through the JMT); you may inquire about or submit a change/new Site-ID request by contacting the JMT/SMT or the JMS Team.

Remote devices, such as ACS, ADS and EAS utilize their own sites names even though they are serviced by a parent B/P/C/S, in order to provide clarity on where the device is physically located.

4.1.2 EAS (Access Switch) Hostnames

Number of Characters Reserved: up to 63 characters, with dash delimiters between data fields

Diagram 4.1.2 – EAS Hostnames

Hostname indicates:

c3750x - Identifies the vendor (C-Cisco) and the model number (3750x) of the device.

a01 - Specifies that the EAS device is an (Access) Switch in L2 Domain 01.

hootx - Located in Fort Hood, TX (identifies specific B/P/C/S) derived from JMT Site-ID list 4197 - Identifies building 4197 as the host building for this device.

arnsth - Identifies room “arnsth” within the previously mentioned building as the location of the device 0103 - Identifies device in Rack Row 01 Rack 03.

1s3 - Identifies Device Instance 1, which is a stack made up of 3 switches. Please note that devices that are not comprised of a stack of switches are designated as 1s1.

The classification of the device, the theater it belongs to and the region in which the device resides in will be captured in the Domain Name System (DNS) Suffix and therefore are not necessary in the device hostname itself.

Dynamic Length 1 1 2 1 1 Dynamic Length 1 Dynamic Length 1 Dynamic Length 1 1 1 1

Vendor/Model Spacer Function L2 Area Spacer Spacer Building Spacer Room Spacer RackRow / Rack Spacer

Device

Instance

Stack

Designator

# of

Switches in Stack

Alpha/Numeric - A (Access) 01 - - Alpha/Numeric - Alpha/Numeric - None - 1 s 1

Alpha/Numeric - A (Access) 02 - - Alpha/Numeric - Alpha/Numeric - Desk - 2 s 2

Alpha/Numeric - A (Access) 03 - - Alpha/Numeric - Alpha/Numeric - 0102 (RR01.R02) - 3 s 3

Alpha/Numeric - A (Access) 04 - - Alpha/Numeric - Alpha/Numeric - … - 4 s 4

Alpha/Numeric - A (Access) … - - Alpha/Numeric - Alpha/Numeric - … - … … …

Alpha/Numeric - A (Access) 99 - - Alpha/Numeric - Alpha/Numeric - .. - 9 s 7

Up to 63 Characters up to 5

Site ID

Alpha/Numeric

Alpha/Numeric

Alpha/Numeric

Alpha/Numeric

Example: c3750x-a01-hootx-4197-arnsth-0103-1s3

Alpha/Numeric

Alpha/Numeric https://wiki.jmn.mil/

4.1.3 ADS (Distribution Switch) Hostnames

Diagram 4.1.3 – ADS Hostnames c3750x - Identifies the vendor (C-Cisco) and the model number (3750x) of the device.

d01 - Specifies that the ADS device is a (Distribution) Switch in L2 Domain 01 on the local B/P/C/S.

hootx - Located in Fort Hood, TX (identifies specific B/P/C/S).

1531 - Identifies building 1531 as the host building for this device.

120B - Identifies room 120B within the previously mentioned building as the location of the device.

0103 - Device is in Rack Row 01 and Rack 03.

1 - Identifies Device Instance.

The classification of the device, the theater it belongs to and the region in which the device resides in will be captured in the DNS Suffix and therefore are not necessary in the device hostname itself.

4.1.4 ACS (Core Switch) Hostnames

Diagram 4.1.4 – ACS Hostname c3750x - Identifies the vendor (C-Cisco) and the model number (3750x) of the device.

c03a - Is an ACS device belonging to L2 Domain/Area 03, attaching to the campus JB-CE Router 1 (A).

The A & B designation is important in helping operators be clear on which device to terminate L2BS connections on and which devices host which set of VLANs as the active MST root.

hootx - Located in Fort Hood, TX (identifies specific B/P/C/S).

1733 - Identifies building 1733 as the host building for this device.

130 - Identifies room 130 within the previously mentioned building as the location of the device.

0301 - Device is in Rack Row 03 and Rack 01.

1 - Identifies Device Instance.

Dynamic Length 1 1 2 1 1 Dynamic Length 1 Dynamic Length 1 Dynamic Length 1 1

Vendor/Model Spacer Function L2 Area Spacer Spacer Building Spacer Room Spacer RackRow / Rack Spacer

Device

Instance

Alpha/Numeric - D (Distribution) 01 - - Alpha/Numeric - Alpha/Numeric - None - 1

Alpha/Numeric - D (Distribution) 02 - - Alpha/Numeric - Alpha/Numeric - Desk - 2

Alpha/Numeric - D (Distribution) 03 - - Alpha/Numeric - Alpha/Numeric - 0102 (RR01.R02) - 3

Alpha/Numeric - D (Distribution) 04 - - Alpha/Numeric - Alpha/Numeric - … - 4

Alpha/Numeric - D (Distribution) … - - Alpha/Numeric - Alpha/Numeric - … - …

Alpha/Numeric - D (Distribution) 99 - - Alpha/Numeric - Alpha/Numeric - .. - 9

Up to 63 CharactersExample: c3750x-d01-hootx-1531-120b-0103-1 up to 5

Site ID

Alpha/Numeric

Alpha/Numeric

Alpha/Numeric

Alpha/Numeric

The classification of the device, the theater it belongs to and the region in which the device resides in will be captured in the DNS Suffix and therefore are not necessary in the device hostname itself.

4.1.5 L2BS (Layer 2 Bridging Switch) Hostnames

Number of Characters Reserved: Up to 63 characters, with dash delimiters between data fields

Diagram 4.1.5 – L2BS Hostname c3750x - Identifies the vendor (C-Cisco) and the model number (3750x) of the device.

l2bs - Signifies that this is the L2BS (Bridging Switch) for the Base.

hootx - Located in Fort Hood, TX (identifies specific B/P/C/S).

1733 - Identifies building 1733 as the host building for this device.

130 - Identifies room 130 within the previously mentioned building as the location of the device.

0103 - Device is in Rack Row 01 and Rack 03.

1 - Identifies Device Instance.

The classification of the device, the theater it belongs to and the region in which the device resides in will be captured in the DNS Suffix and therefore are not necessary in the device hostname itself.

Dynamic Length 1 1 1 1 1 Dynamic Length 1 Dynamic Length 1 1

Vendor/Model Spacer Function Spacer Spacer Spacer Room Spacer RackRow / Rack Spacer Device Instance

Alpha/Numeric - L2BS - - - Alpha/Numeric - None - 1

Alpha/Numeric - L2BS - - - Alpha/Numeric - Desk - 2

Alpha/Numeric - L2BS - - - Alpha/Numeric - 0102 (RR01.R02) - …

Alpha/Numeric - L2BS - - - Alpha/Numeric - … -

Alpha/Numeric - L2BS - - - Alpha/Numeric - … -

Alpha/Numeric - L2BS - - - Alpha/Numeric - .. -

Alpha/Numeric Alpha/Numeric

Alpha/Numeric Alpha/Numeric

Alpha/Numeric Alpha/Numeric

Site ID Building

Alpha/Numeric Alpha/Numeric

Alpha/Numeric Alpha/Numeric

Example: c3750x-l2bs-hootx-1773-130-0103-1 Upto 63 Characters up to 5 Dynamic Length

Alpha/NumericAlpha/Numeric

4.2 Virtual Local Area Network Numbering Schema

The Institute for Electrical and Electronics Engineers (IEEE) 802.1Q networking standard enables support for VLANs and the multiplexing of said VLANs across shared physical links through VLAN tagging. The IEEE 802.1Q standard is applicable to both IPv4 and IPv6 and utilizes a 12-bit field specifying the VLAN to which the Ethernet frame belongs, providing for 4096 possible VLANs (0000-4095). The DISN-EN architecture lends itself to permitting local administrators to identify and utilize VLAN tags as either Base Unique (tag is unique across the entire base) and/or L2 Domain/Area Unique (tag is unique only to the L2 Domain that is using it). B/P/C/S user populations may range from just a few personnel to anywhere over 70,000+ personnel, the architectural distinction is found in the number of L2 Domains/Areas used to service the final campus network.8 As such, L2 Domains/Areas represent physically distinct networks that are interconnected by the JB-CE routers via a Layer 3 protocol. This permits the reuse of VLAN tags across different L2 Domains/Areas and the scalability needed to account for B/P/C/S of any size. The DISN-EN L2 Domain/Area architecture also relies on Multiple Spanning Tree (MST) regional groupings, with a single L2 Domain/Area belonging to a single region. Each region may have multiple MST instances as is required. This enables load balancing of VLAN traffic across the ACS devices and minimizes network re-convergence impacts. The VLAN Numbering Schema presented here accounts for this the architecture, by systematically creating the two MST domain VLAN tag groupings (i.e.: Air Force MST Group-1 is 2000 - 2149 while Air Force MST Group-2 is 3000 - 3149, see matrix below).

DISN-EN Participating Base Executive Agent Administrators will assign VLAN Tags for their particular tenants from within the range of VLANs provided. Please note that when the VLAN tags already in use do not line up with the matrix provided below, the existing VLAN numbers must be renumbered in order to be compliant with this artifact. Various management and provisioning tools utilize the VLAN Names (which includes the VLAN Tag), VLAN Matrix compliance is required for these tools to work effectively in the Enterprise.