MISTE_TNM_DO_EPG_Performance_Specification.pdf

DISTRIBUTION STATEMENT A. Approved for public release.

SYSTEM PERFORMANCE SPECIFICATIONS

FOR

MODERNIZING INSTRUMENTATION SOLUTIONS FOR TEST AND EVALUATION (MISTE)

Delivery Order: 0002

ELECTRONIC PROVING GROUND (EPG) TEST NETWORK MODERNIZATION

Test Instrumentation Systems Network - Enterprise Network (TISN-EN)

U.S. Army Program Executive Office for Simulation, Training and Instrumentation 12211 Science Drive

Orlando, FL 32826-3276

Engineering Prepared By: ____________________ Concurrence: ____________________ Erick Serrano, PE Derek Vick Lead Systems Engineer, IMO Chief Engineer, PM ITTS IMO Date: _________ Date: _________

PRF-PT-709

V1.0, 1 November 2018

1.0 SCOPE ________________________________________________________ 5

1.1 Identification ____________________________________________________ 5

2.0 APPLICABLE DOCUMENTS _______________________________________ 6

2.1 Government Document ____________________________________________ 6

2.2 Non-Government Document ________________________________________ 6

2.3 Order of Precedence ______________________________________________ 7

3.0 REQUIREMENTS ________________________________________________ 7

3.1 Performance Requirements _________________________________________ 8

3.1.1 Bandwidth ______________________________________________________ 8

3.1.2 Latency ________________________________________________________ 9

3.1.3 Availability ______________________________________________________ 9

3.2 TISN-EN Security Stacks __________________________________________ 9

3.3 Physical Connectivity ______________________________________________ 9

3.4 Configuration Specifications _______________________________________ 15

3.5 Layer 3 JT-CE Router and Layer 2 Switch(s) Specifications _______________ 15

3.5.1 Overarching Specifications ________________________________________ 16

Technology Refresh: _____________________________________________ 16 IPV6 Compliance ________________________________________________ 16 Manufacturing Requirements: ______________________________________ 16 IEEE Standards: ________________________________________________ 16

3.5.2 Hardware Requirements: __________________________________________ 17 For Chassis Based Systems: ______________________________________ 17 For All Systems: ________________________________________________ 17

3.5.3 Software Requirements: __________________________________________ 17

3.5.4 Network Management Standards: ___________________________________ 17

3.5.5 Security Standards: ______________________________________________ 18

3.5.6 QoS (Quality of Service) Standards: _________________________________ 18

3.5.7 IPv6 Standards: _________________________________________________ 18

3.5.8 Routers Requirements ____________________________________________ 19

General Requirements: ___________________________________________ 19 Port Requirements: ______________________________________________ 19

3.5.9 EAS (Edge Access Switch) Requirements ____________________________ 19 Standard Edge Access Switch _____________________________________ 20

3.5.9.1.1 Standard Edge Access Switch General System Requirements ____________ 20

3.5.9.1.2 Standard Edge Access Switch Uplink Ports: ___________________________ 20

3.5.9.1.3 Standard Edge Access Switch Downlink Port Requirements: ______________ 20

Small Form Factor EAS ___________________________________________ 21

3.5.9.2.1 Small Form Factor EAS General System Requirements __________________ 21

3.5.9.2.2 Small Form Factor EAS Downlink Port Requirements: ___________________ 21

3.5.10 ADS (Area Distribution Switch) Requirements _________________________ 22

General Requirements ___________________________________________ 22 Uplink Port Requirements: _________________________________________ 22 Downlink Port Requirements (10GbE to EASs) ________________________ 22

3.5.11 ACS Switch Requirements ________________________________________ 23 General Requirements: ___________________________________________ 23 Uplink Port Requirements: _________________________________________ 23 Downlink Port (10GbE ports to EAS or ADS): __________________________ 23

3.5.12 L2BS Switch Requirements ________________________________________ 24 General Requirements ___________________________________________ 24 Uplink Port Requirements: _________________________________________ 24 Downlink Port Requirements (100GbE to EASs) _______________________ 24 100GBASE-LR, SFP+ Optics (LC) __________________________________ 24

3.6 Network Management System (NMS) ________________________________ 24

3.7 Cable and Equipment Labeling _____________________________________ 25

4.0 Verification _____________________________________________________ 25

4.1 Methods of Verification ___________________________________________ 25

4.2 Classes of Verification ____________________________________________ 26

4.3 Inspections and Tests ____________________________________________ 26

4.3.1 Visual Inspections _______________________________________________ 26

4.3.2 Functional Tests ________________________________________________ 26

4.3.3 Network Performance Test/On-Site __________________________________ 26

4.3.4 Non-Functional Examinations ______________________________________ 27

5.0 GLOSSARY/ACRONYMS _________________________________________ 28

6.0 APPENDICE – B _________________________________________________ 1

Revision History

Revision Number Date Log of Changes Made and

Description of Reason Changes Approved By

V1.0 11/01/2018 Draft Issued to Industry for Comments and Feedback Erick Serrano

1.0 SCOPE

As part of the Test Network Modernization (TNM) Program, the U.S. Army Program Executive Office Simulation, Training and Instrumentation (PEO STRI), Project Manager for Instrumentation, Targets and Threat Simulators and Special Forces Operations (SOF) Training Systems (PM ITTS), Instrumentation Management Office (IMO) has a requirement from the U.S. Army Test and Evaluation Command (ATEC) to modernize the Test Network at Electronic Proving Ground (EPG), in Ft. Huachuca, AZ in support of Testing Instrumentation. It is ATEC’s mission and vision to establish Enterprise based solutions for all existing test centers instrumentation networks to efficiently support sustainment, training, operations, and future upgrades to these critical instrumentation capabilities. The end result is to effectively modernize existing ATEC networks to be more secure, reliable, and efficient.

This performance specification establishes the network system technical requirements for the design, configuration, implementation, integration, installation, demonstration, testing, verification, and training of the Test Instrumentation System Network - Enterprise Network (TISN-EN) Architecture for EPG. This specification is intended to serve Army Test Centers by providing a scalable means of standardizing Test Center architecture and infrastructure configurations in the future. The hardware specifications, protocols, naming conventions, network topologies, and network architectures are vendor-agnostic which have been implemented previously in Department of Defense (DoD) theater networks and management domains and shall be used as applicable, for this network modernization at EPG. The TISN- EN network solution is intended to be designed to be ready to interconnect to the Defense Information Systems Agency, Defense Information Systems Network-Enterprise Network (DISN-EN) to achieve intra-Test Center connectivity.

1.1 Identification

The existing EPG TNM network includes an Enterprise Campus Network supported by a Data Center. The Government understands that there may be limitations with the existing fiber plant; for this reason, other Government projects will be responsible for fiber modernization and Data Center upgrades.

The EPG TNM network consists of three Core and three distribution locations. Core locations are labeled Electromagnetic Environments Test Facility (EMETF), Bannister and C4I. The 3 distribution nodes are Node A (Bldg. 86810), Node F (Bldg. 70301), and Node H (Bldg. 12808) all connected to the Bannister Core location; all Edge locations not connected to the Distribution nodes are connected into the Core routers at EMETF and C4I. The existing EPG Test Network physical topology diagram is included in Appendix B (see separate PDF file provided for ease of readability). This transport network supports data, voice, and video and supports all of EPG’s test instrumentation facilities/capabilities.

The TNM Program shall deliver the following modernized turnkey network system in compliance with these specifications:

1) EPG Test Instrumentation System Network-Enterprise Network (TISN-EN)

2.0 APPLICABLE DOCUMENTS

The following documents of the exact issue listed form a part of this specification to the extent specified herein. This specification is designated as a first tier document. The documents referenced directly in this specification are designated as second tier documents and are considered part of this specification to the extent specified herein. Documents referenced in successive tiers are not part of this specification and are to be used for guidance only.

2.1 Government Document

• AR 25-2 Information Management Information Assurance

• DISR DoD IPv6 Standard Profiles, for IPv6 Capable Products, Version 2.0

• JTA DoD Joint Technical Architecture

• JTA-A Department of the Army Joint Technical Architecture

• UCR 2013 DoD Unified Capabilities Requirements

• UC APL Unified Capabilities Approved Products List

• STIG Information Assurance Support Environment STIGS

• Labeling Technical Criteria for the Installation Information Infrastructure

Architecture

2.2 Non-Government Document

• National Electrical Code (NEC)

• National Fire Protection Association (NFPA)

• American National Standards Institute (ANSI)

• Telecommunications Industry Association/Electronic Industries Association (TIA/EIA)

• Administration Standards for the Telecommunications Infrastructure of Commercial

Buildings (TIA/EIA) 569A

• IEEE 802.1p Ethernet QoS

• IEEE 802.1q VLAN Tagging

• IEEE 802.1s Multiple Spanning Tree

• IEEE 802.1x Port Security

• IEEE 802.1ab Link Layer Discovery Protocol (LLDP)

• IEEE 802.1ae MACsec

• IEEE 802.3ad Link Aggregation

• IEEE 802.3z 1GB Ethernet

• IEEE 802.3ae 10GB Ethernet

• IEEE 802.3at Power over Ethernet +

• ITU Y.1564 Ethernet Service Activation Test Methodology

• RFC 2475 Architecture for Differentiated Services

• RFC 2474 DIFFSERV Field in the IPv4 and IPv6 Headers

• RFC 1757 Remote Network Monitoring MIB

• RFC 1907 Simple Network Management Protocol version 2 http://www-jta.itsi.disa.mil/ http://arch-odisc4.army.mil/jtaa/documents.asp http://iase.disa.mil/stigs/a-z.html

• RFC 3411 Simple Network Management Protocol version 3

• RFC 3246 Expedited Forwarding PHB

• RFC 2597 Assured Forwarding PHB Group

• RFC 2460 IPv6 Specification

• RFC 2461 IPv6 Neighbor Discovery

• RFC 2462 IPv6 Stateless Address Auto-Configuration

• RFC 2463 ICMPv6

• RFC 4291 IPv6 Addressing Architecture

• RFC 3587 IPv6 Global Unicast Address Format

• RFC 2375 IPv6 Multicast Address Assignment

• RFC 2464 Transmission of IPv6 over Ethernet Networks

• RFC 3596 DNS Support

• RFC 2893 Transition Mechanisms for IPv6 Hosts and Routers

• RFC 2328 OSPFv2 (with IPv4 Support Only)

• RFC 5340 OSPFv3 (with IPv6 Support)

• RFC 2338 Virtual Router Redundancy Protocol

• RFC 1541 Dynamics Host Configuration Protocol

• RFC 1654 Border Gateway Protocol version 4

• RFC 1825 Security Architecture for the Internet Protocol

• RFC 1828 IP Authentication using Keyed MD5

• RFC 7426 Software-Defined Networking

• STD-6 User Datagram Protocol

• STD-7 Transmission Control Protocol

• STD 13 Domain Name Systems

• SDT 15 Simple Network Management Protocol

2.3 Order of Precedence

In the event of a conflict between the text of this specification and the references cited herein, the text of this specification takes precedence. Nothing in this specification, however, supersedes applicable laws and regulations unless a specific exemption has been obtained.

3.0 REQUIREMENTS

The TISN-EN solution shall create an Enterprise Information Technology (IT) Environment across all Army Test Evaluation Command (ATEC) Test Centers relying on a standardized core Internet Protocol (IP) transport service network (Multi-Protocol Label Switching (MPLS) Service Network with security stack services and a standardized Test Center area distribution network (Test Center Area Network (TCAN)). It is intended for the TCAN to interconnect existing ATEC Test Center via the DISN-EN transport network to promote joint collaboration and data sharing. The MPLS is a type of data-carrying technique for high-performance telecommunication networks; this network protocol has been selected to achieve ATEC’s Test Center’s interoperability to meet the high performance demands of military test and evaluation system requirements.

The Architecture of the EPG TISN-EN network shall be as per these specifications. The new EPG TISN-EN shall be equipped with two MPLS-capable Layer 3 devices with identical hardware and software configurations, for redundancy, serving as Joint Test Center Customer Edge routers (JT-CE). This will provide redundancy at the Layer 3 level. EPG TISN-EN shall be equipped with two ACS’ (Area Core Switches) per area with identical hardware and software configurations for redundancy.

The EPG Test Center Area Network (TCAN) shall provide high speed local IP switched transport services within the JT-CE first hop routers down to the user access devices. The EPG TCAN infrastructure shall have a framework of Core, Distribution and Access and is therefore comprised of Area architecture that observes a highly stable systematic hierarchy for physical and logical interconnectivity between the routers, ACS’ (Area Core Switches), ADS’ (Area Distribution Switches-Intermediate aggregation switch that collects EAS devices before unlinking into the ACS Device) and EAS’ (Edge Access Switches). Each ACS Pair creates a physically distinct Layer 2 Domain that services the layer 2 production VLANs within that network segment or region. EPG shall have as many ACS Pairs as reasonably needed to meet the density demands of the TCAN. In instances where a single VLAN shall be shared across multiple L2 Domains for whatever reason, a tightly controlled Layer 2 Bridging Switch (L2BS) device shall be used to ensure that the network remains loop free and stable.

Figure 3-0.1 TCAN Concept Diagram The following sections describe the network system requirements for the EPG TISN-EN.

3.1 Performance Requirements

3.1.1 Bandwidth

The TISN-EN Network shall have a bandwidth of: 100Gb at the Layer 3 Core Routers (JT-CE), 40/100Gb at the Layer 2 Area Core and Area Distribution Switches and 1/10GB at the Access Edge Switches.

ADS

DISN-EN

ACS

JT-CE

ACS

ADS

AES AES AES AES

EPG

ADS

ACS

JT-CE

ACS

ADS

AES AES AES AES

Other Test Center

3.1.2 Latency

The TISN-EN Network shall have the following latency regardless of physical routing protocol: 5 milliseconds Objective, 50 milliseconds Threshold.

3.1.3 Availability

TISN-EN architecture availability (Excerpt from Unified Capabilities Requirements (UCR)) for EPG shall be four nines.

3.2 TISN-EN Security Stacks

The Contractor shall provide the TISN security stacks comprised of network traffic intrusion detection, intrusion prevention, multi-tier firewall services, content filtering, logging, packet capture services and inter Virtual Routing and Forwarding (VRF) policy filtered routing services.

3.3 Physical Connectivity

The Government understands that there may be limitations with the existing fiber plant; for this reason, other Government projects will be responsible for fiber modernization upgrades.

However, this Contractor shall be responsible for identifying and documenting all required outside plant fiber/copper upgrades as part of this modernization effort. At this point the Contractor is required to use existing outside fiber and cable plant to effectively execute this modernization. Inside building fiber and cable plant already exists, cable connections between routers, and switches within close proximity (same rack or adjacent rack is the responsibility of the Contractor). Each JT-CE in a JT-CE pair shall be hosted in physically separate facilities at EPG and shall be a minimum of 1476 feet apart to ensure survivability in the event of natural disaster or IED attack. Additionally, the hosting facilities shall use diverse physical pathways from each other in order to eliminate single points of failure in the form of accidental or malicious cable/fiber cuts.

The physical fiber connection between equipment or approved patch-panel shall be made with one individual fiber cable. “Barrel-connecting” two shorter fiber cables to create a longer fiber-cable or change the cable termination type shall not be used.

Figure 3-3.1 Proper and Improper use of Fiber Cables There shall be no more than seven (7) 48 port switches stacked in any given stack. While most vendors allow more switches to be added to the stack from a technical perspective, it undermines availability impact risk management and per user target bandwidth availability when the recommended maximum number is exceeded.

Diagram 3.3.2 – Stacking Limit, Availability & Bandwidth Analysis

Uplink Bandwidth of Stack: 10 Gigabits Uplink Bandwidth of Stack: 10 Gigabits Ports per Switch in Stack: 48 Ports Ports per Switch in Stack: 48 Ports

Number of Switches in Stack: 7 Switches Number of Switches in Stack: 7 Switches

Target Availability: 99.99% UCR Target Bandwidth/User: 30 Mbps Avg User Ports on Base: 22,633 People HD Video Stream: 8.38 Mbps

Total Avg User Hours/Yr: 198,265,080 Hours Phone Traffic: 0.10 Mbps Per Person Allowed Outage/Yr: 52.56 Minutes User Data Transfere Traffic: 10.00 Mbps

Total Outage Time Permited/Yr: 19,827 Hours Total Bandwidth Needed /User: 18.48 Mbps

Estimated Avg Outage Time: 30 Minutes (UCR) Possible Number of Users/Stack: 336 Possible Users Possible Number of Users/Stack: 336 Possible Users Total Uplink Bandwidth Target: 9.84 Gigabits

Outage Impact: 168 Hours Total Uplink Bandwidth Needed/Stack: 6.07 Gigabits

Per Incident Impact to Outage Time Permitted/Yr: 0.84735% EAS Bandwidth Utilization (Daily Use): 6.56250% Percent of EAS' that may experience Outage/Yr: 0.34 Target is no less than 1/3 of stacks EAS Bandwidth Utilization (Max Use): 60.65063%

Availability Impact Analysis Bandwidth Impact Analysis

Physical Connectivity architecture shall be as follows:

Diagram 3.4.2 – EAS Uplinks to Different L2 Domain ACS Switches

EAS devices are only authorized to connect to ADS and ACS devices within their own Layer-2 Area.

Diagram 3.4.3 –ACS Interconnectivity

JT-CE JT-CE

Test Center

Routed VLAN Interfaces

Diagram 3.4.3 – Example of what NOT to do (ACS Interconnectivity)

Diagram 3.4.4 – ACS to JT-CE Interconnectivity

Diagram 3.4.5 –ADS used as a fiber coupler

ACS switches should NOT cross connect to each JT-CE, rather each ACS connects to only its corresponding JT-CE router.

Diagram 3.4.6 –EAS uplinked via ACS and ADS at the same time

The architecture shall not allow “daisy chaining” of EAS devices.

Diagram 3.4.7 –Directly interconnecting or daisy chaining EAS’

JB-CE JB-CE

B/P/C/S

Routed VLAN Interfaces

DISA MPLSDISA MPLS

100 GB

ACS ACS

100 GB

General Look

Access Switch Stack

EAS

EAS

L2 Domain / Area 001 (example only)

ACS ACS ACS ACS

General Look

L2 Domain / Area 003 (example only) General Look

Access Switch Stack

EAS EAS

Access Switch Stack

General Look

L2 Domain / Area 003 (example only) General Look

EAS EAS

Access Switch StackAccess Switch Stack

EAS

Access Switch Stack

EAS

Access Switch Stack

Diagram 3.4.8 – Use of L2BS Switches

Additional physical connectivity requirements:

EAS’ shall only connect to a single L2 Domain/Area (via an ADS or an ACS Pair)1.

EAS’ shall rely on switch-stacking as much as possible, up to 7 edge switches maximum per stack (creating virtual edge switches) to minimize uplink requirements.

o Please note that if a vendor’s equipment has a lower limit, that limit SHALL be followed.

o Please note that stacking implies the use of built in stacking ports connected to the stacking backplane.

ADS devices do not directly connect to the JT-CE routers; they connect only to ACS devices.

o Proper ADS/ACS relationship: within the economy of the same L2 Domain.

ADS devices shall only connect to a single L2 Domain/Area (ACS Pair)1.

Switch interconnections shall be made utilizing Single Mode Fiber.

ACS’ uplink to its upstream/host set of JT-CE routers.

The two ACS aggregation switches within a given L2 Domain shall be interconnected using link aggregation of two physical ports on each switch (from different modules)2.

o If a single service module exists in an ACS due to scarce port density requirements in a L2 Domain/Area, a non-redundant inter-connection to the peer ACS is sufficient because the ACS that loses its service module is automatically removed as a valid path to the Next Hope Redundant Routing Protocol defined Route Point.

1 Please note that each ACS Pair represents the top aggregation nodes of all the underlying subordinate switches (potentially multiple ADS and a host of EAS devices) and are treated as belonging to the same Layer 2 Domain.

2 This is required to avoid “split brain routing” with whichever Next Hope Redundancy Protocol is used.

TISN-EN Area NET

TISN MPLS

L3 Only

L2 Only

Test Center Routed VLAN

Interfaces

ACS switches from different L2 Domains/Areas shall NEVER be directly interconnected3.

o Each ACS within an L2 Domain/Area shall only uplink to its designated JT-CE router and port; no cross-connects from an ACS to both JT-CE routers in the JT- CE pair.

The L2BS device singly attaches to any ACS Pair, this lack of connectivity redundancy is done on purpose/by design (it is not a flaw), in order to prevent accidental loops between L2 Domains4.

The L2BS device connects to each JT-CE router, using sub-interfaces on those physical ports on the JT-CE to service L2BS VLANs. L2BS VLANS are not bridged from ACS’ to JT-CE routers.

In all cases, when an exception is taken to the model architecture it shall be documented and approved by EPG prior to implementation.

3.4 Configuration Specifications

For layer 3 devices (JT-CE Routers), configuration requirements (host names, VLANs, first hop redundancy protocol standard (FHRPS), interface descriptions, virtual routing and forwarding construct, traffic flow, management services) shall be per current Army Network standards. In all cases, when an exception is taken to the model configuration it shall be documented and approved by EPG prior to implementation.

For layer 2 switch devices (ACS, ADS, EAS), configuration requirements (device host names, VLANs, port configurations, extensions, encryption, root bridge/spanning tree, patch cable labeling, class of service) shall be per current Army Network standards. In all cases, when an exception is taken to the model configuration it shall be documented and approved by EPG prior to implementation.

3.5 Layer 3 JT-CE Router and Layer 2 Switch(s) Specifications This section identifies the minimum, Routers, L2BS, ACS, ADS and EAS device technical specifications to ensure that the architecture can be instantiated as architected through a known and common set of baseline technical capabilities. Adherence to this section enables architecture compatibility and sustainability. All layer 3 and layer 2 devices shall be provided from a single vendor for commonality, standardization, maintainability and cost effective life cycle management.

3 Each ACS Pair is representative of its own Layer 2 Domain/Area within the overall Campus Network, this allows for reuse of VLAN tags within a campus and it significantly stabilizes STP. Please see the Layer 2 Extension section to see how the architecture gracefully handles the interconnectivity of nodes that shall be bridged across the campus.

4 This solution was developed and successfully used by the Pentagon to limit cross-wedge broadcast storms/service impacts. The likelihood that a misconfiguration will occur over time is high, minimizing the chance of loops between areas manages both the risk and the potential impacts.

3.5.1 Overarching Specifications

Technology Refresh:

Equipment shall be delivered from Original Equipment Manufacturer (OEM) with the latest Unified Capabilities Approved Products List (UC APL) approved technology (e.g.

latest memory standard) and software/firmware revisions installed.

IPV6 Compliance All hardware and associated software shall be IPv6 capable. An IPv6 capable system or product shall be capable of receiving, processing, transmitting and forwarding IPv6 packets and/or interfacing with other systems and protocols in a manner similar to that of IPv4. Specific criteria to be deemed IPv6 capable are:

Maintenance of interoperability with IPv4. Systems being developed, procured or acquired shall maintain interoperability with IPv4 systems/capabilities.

Systems shall implement IPv4/IPv6 dual-stack and should also be built to determine which protocol layer to use depending on the destination host it is attempting to communicate with or establish a socket with.

Availability of IPv6 technical support for system development, implementation and management.

Manufacturing Requirements:

Minimum of 8 years of OEM support before end of life (3 years manufacturing and 5 years support).

Certified as genuine by manufacturer's representative time of receipt by government On DoD UC APL at time of award Shall be Assured Services Local Area Network (ASLAN)-certified

IEEE Standards:

IEEE 802.1AB LLDP.

IEEE 802.1p Ethernet CoS.

IEEE 802.1q VLAN Tagging.

IEEE 802.1aq (Shortest Path Bridging) IETF RFC 1112 Multicast IETF RFC 2328 OSPFv2 (IPv6 not supported) IETF RFC 5340 OSPFv3 (supports IPv6) IEEE 802.1s Multiple Spanning Tree.

IEEE 802.1x Port Security with Certificates IEEE 802.3z 1GB Ethernet.

IEEE 802.3ad LACP.

IEEE 802.3ba 40/100 GB Ethernet IEEE 802.3ae 10 GB Ethernet

IETF RFC 5798 VRRP

IETF RFC 4760 BGP

IETF RFC 3411 SNMP Management Frameworks IEEE 802.3ae 10GB Ethernet (Small Form-factor Pluggable (SFP+) Long Reach (LR) optical transceivers).

IEEE 802.1ae Media Access Control (MAC) Security.

IEEE 802.3ba 40/100GB Ethernet (Backbone/Uplink Ports) IEEE 802.1p Ethernet QOS

3.5.2 Hardware Requirements:

For Chassis Based Systems:

Control plane and data forwarding modules shall be physically separated.

Distributed forwarding capability shall be performed by the Input / Output (I/O) module.

For All Systems:

Shall be capable of non-blocking as specified in UCR-2013 section 7.2.1 for maximum performance levels for the specified minimum number of ports of each of the switch platforms as stated in their respective sections of this document. Ports in excess of the minimum required need not be non-blocking.

All devices shall have a CLI-based console port for local administration (2FA) Power supplies shall be configured to maximum redundancy (N+1 at a minimum).

All other internal components (fans, etc.) shall be redundant (N + M) and load sharing

(where applicable) to maximum available. N + M redundancy is defined as followed: N is the number of active components required to operate a system without redundancy;

M is the number of “spares” specified to provide the requisite level of system availability.

All fans and power supply internal components, shall be field replaceable non-disruptively, and shall support Online Insertion and Removal (OIR).

3.5.3 Software Requirements:

The acquired systems shall be supported through software, documentation, licenses, version upgrades, patch upgrades, and help desk support for all equipment described above for a period of five years after the date of installation of the equipment.

The software shall be an included component of the equipment purchase.

A single software license model that includes all required features.

Capable of Open Flow (Version 1.3 minimum) as an industry-standard Software Defined

Network (SDN) protocol that facilitates communication between the forwarding plane of a network switch or router and the controller that enables network-wide flow control.

Shall support a minimum hostname length of 63 characters.

3.5.4 Network Management Standards:

Command Line Interface (CLI).

Management via a single interface.

Dedicated out-of-band management port using 10/100 Mbps minimum.

Dedicated out-of-band management port using CLI-based console port.

Management via a single IP interface per device (a stack is treated as one device).

Simple Network Management Protocol (SNMP) Version 3 w/AES 256 Encryption

(minimum) and TLS support.

IPv6 Management.

Authentication, Authorization and Accounting (AAA).

RADIUS.

TACACS+.

Secure Shell (SSH v2).

Secure Copy (SCP v2).

Hyper Text Transfer Protocol Secure (HTTPS).

Username/Password (Challenge and Response).

Bi-level Access Mode (Standard and Privileged Level).

Access Control Lists on Console and VTY ports.

NTP with Message Digest Algorithm (MD5) authentication. Capability SHA1

SNMP MIB II.

Remote Monitoring (RMON).

3.5.5 Security Standards:

The security of ports subject to IA requirements is governed by DISA STIG guidelines. All switches connected to the TISN-EN are bound by these guidelines. For a complete listing of STIGs please refer to the Information Assurance Support Environment (IASE) DISA website, http://iase.disa.mil/stigs/Pages/index.aspx.

Protection for Denial of Service protection against Transmission Control Protocol Synchronous (TCP SYN) or Smurf Attacks.

Shall support Access Control Lists (ACL) on Console, VTY, a per subnet and per VLAN (VLAN ACL (VACL) basis.

AAA.

RADIUS.

TACACS+.

Secure Shell (SSH v2).

Secure Copy (SCP v2).

Hyper Text Transfer Protocol Secure (HTTPS).

Username/Password (Challenge and Response).

Bi-level Access Mode (Standard and Privileged Level).

3.5.6 QoS (Quality of Service) Standards:

RFC 2474 DIFFSERV Field in the IPv4 and IPv6 Headers.

RFC 2475 An Architecture for Differentiated Services.

RFC 3246 An Expedited Forwarding PHB.

RFC 2597 Assured Forwarding PHB Group.

3.5.7 IPv6 Standards:

RFC 8200 IPv6 Specification.

RFC 4861 IPv6 Neighbor Discovery.

RFC 4862 IPv6 Stateless Address Auto-Configuration.

RFC 4443 ICMPv6.

RFC 2460 IPv6 Specification.

RFC 2461 IPv6 Neighbor Discovery.

RFC 2462 IPv6 Stateless Address Auto-Configuration.

RFC 2463 ICMPv6.

RFC 4291 IPv6 Addressing Architecture.

RFC 3587 IPv6 Global Unicast Address Format.

RFC 2375 IPv6 Multicast Address Assignments.

RFC 2464 Transmission of IPv6 over Ethernet Networks.

http://iase.disa.mil/stigs/Pages/index.aspx

RFC 3596 DNS support.

RFC 2893 Transition Mechanisms for IPv6 Hosts and Routers.

3.5.8 Routers Requirements

This section identifies JT-CE technical device specifications required to ensure that the architecture can be instantiated as architected through a known and common set of baseline technical capabilities.

General Requirements:

Shall be a modular chassis device.

All modules/components shall be interchangeable between Type 1, 2 and 3 router chassis.

Shall have capability for “in-service” software upgrade without traffic interruption or loss.

Capable of assigning up to 4094 VLAN Tags.

Supports minimum Jumbo Ethernet frame size of 9126 bytes (IETF RFC4638).

Shall support at a minimum eight (8) queues for traffic prioritization.

Shall be able to assign traffic to queues.

• By VLAN tags, types of traffic or DIFFSERV Code Point.

• Set QoS by DIFFSERV Code Point based on type of traffic (e.g.: VoIP).

Shall have redundant routing engines/modules.

Capable of supporting input voltages of both 120/220 VAC.

Shall be able to accept AC or DC power (customer’s choice at time of purchase).

Shall have N+1 redundant and hot-swappable power supplies.

Shall support Multiprotocol Label Switching (MPLS) Shall support MP-BGP Shall support Open Shortest Path First Protocol version 3 (OSPFv3) Shall support Intermediate System to Intermediate System Protocol (IS-IS) Shall support Carrier Supporting Carrier Protocol (CSC) Shall support Virtual Routing Functions (VRFs)

Port Requirements:

Chassis shall support up to and including a 50% extra capacity for expandability.

Chassis shall support OC192 port modules Chassis shall support 100GB Ethernet port modules Chassis shall support 10GB Ethernet port modules Chassis shall support 1GB Ethernet port modules Chassis shall provide the following minimum ports in a non-blocking configuration (line rate):

• 8 100GbE SFP+ interface ports

• 12 10GbE SFP+ interface ports

• 12 ports of 1Gbps Ethernet

3.5.9 EAS (Edge Access Switch) Requirements

There are two types of EAS devices, the ones that support copper ports and ones that support fiber ports. The copper port switches may be one of two types, a traditional telecommunications rack mounted model and a small form factor model to be installed in special use considerations. The type of switches that will need to be installed will be dependent on the physical infrastructure available EPG. All EAS shall support the following requirements without the addition of hardware, software, firmware or licensing:

Standard Edge Access Switch

3.5.9.1.1 Standard Edge Access Switch General System Requirements One rack unit (RU) in height (1RU=1.75inch).

Shall be available to purchase in the following configurations:

o Minimum Quantity of 48, 10/100/1000 BaseT copper user/downlink ports.

o Minimum Quantity of 24 100/1000 Base-X SFP fiber user/downlink ports.

Input voltage capable of both 120/220 Volts Alternating Current (VAC).

Redundant power supplies (N+1 Redundancy).

Hot-swappable power supplies.

Redundant and Hot-swappable fans.

Shall support stacking (supporting a minimum of 240 ports in a stack). Stacking hardware shall be included in proposed solution.

Shall maintain power redundancy, whether 110 VAC or 220 VAC, for control/data functions and simultaneous class-3 Power Over Ethernet (POE) on every client facing copper edge port in the event of a single power supply failure either via individual switch’s power redundancy or power redundancy within a switch stack configuration.

Shall be able to change, remove or disable the default VLAN from trunks.

Shall support up to 4094 VLAN Tags.

Shall support at a minimum eight (8) queues for traffic prioritization.

Shall be able to assign traffic to queues.

o By VLAN tags, types of traffic or DIFFSERV Code Point.

o Set Quality of Service (QoS) by DIFFSERV Code Point based on type of traffic (e.g.

VoIP).

Shall support IGMP Snooping.

Shall support minimum Jumbo Ethernet frame size of 9000 bytes.

Capable of filtering spanning-tree BPDU.

802.1x - Port-Based Network Access Control supporting Extensible Authentication

Protocol-Top Level Access (EAP-TLS) using DoD PKI Machine Identity Certificates.

o 802.1x – MAC Authentication Bypass (MAB) for non-802.1x capable hosts.

802.1x - Multi-Domain support for separation of voice and data.

802.1x - Fail Open (for migrating to 802.1x).

802.1AB – Link Layer Discovery Protocol (LLDP)/LLDP-Media Endpoint Discovery

(MED).

Support IEEE 802.3ba 40/100GB Ethernet (Backbone/Uplink Ports) for future-proofing.

3.5.9.1.2 Standard Edge Access Switch Uplink Ports:

Minimum of two 100GbE with 40GbE backwards compatibility.

Capable of full line rate (e.g. 2 x 100GB/s Uplink) when fully loaded.

Shall be MACSec capable.

3.5.9.1.3 Standard Edge Access Switch Downlink Port Requirements:

All edge ports (user ports) shall support the following without the addition of hardware, software, firmware or licensing:

48 ports of PoE+ (IEEE 802.3at-2009) with 25.5 watts of power/port.

Spanning-tree BPDU detection and protection.

Time Domain Reflectometry (TDR) diagnostic testing capability.

Shall support auto Media Dependent Interface/Crossover (MDI/MDIX).

Shall support Source Guard (STIG).

Shall support Unknown Unicast Flood Block (UUFB) (STIG) Shall support Root Guard (STIG)

Small Form Factor EAS

3.5.9.2.1 Small Form Factor EAS General System Requirements

One rack unit (RU) in height (1RU=1.75inch).

Shall be available to purchase in the following configuration:

o Minimum Quantity of 12 10/100/1000 BaseT copper user/downlink ports.

o Shall be capable of full line rate speed on all 10/100/1000BaseT downlink ports.

Input voltage capable of both 120/220 Volts Alternating Current (VAC).

Shall be able to change, remove or disable the default VLAN from trunks.

Shall support up to 4094 VLAN Tags.

Shall support at a minimum eight (8) queues for traffic prioritization.

Shall be able to assign traffic to queues.

o By VLAN tags, types of traffic or DIFFSERV Code Point.

o Set Quality of Service (QoS) by DIFFSERV Code Point based on type of traffic (e.g.

VoIP).

Shall support IGMP Snooping.

Shall support minimum Jumbo Ethernet frame size of 9000 bytes.

Capable of filtering spanning-tree BPDU.

802.1x - Port-Based Network Access Control supporting Extensible Authentication

Protocol-Top Level Access (EAP-TLS) using DoD PKI Machine Identity Certificates.

o 802.1x – MAC Authentication Bypass (MAB) for non-802.1x capable hosts.

802.1x - Multi-Domain support for separation of voice and data.

802.1x - Fail Open (for migrating to 802.1x).

802.1AB – Link Layer Discovery Protocol (LLDP)/LLDP-Media Endpoint Discovery

(MED). Small Form Factor EAS Uplink Ports:

Minimum of two 1GbE SFP or SFP+ based uplink ports.

Capable of full line rate (e.g. 2 x 100GB/s Uplink) when fully loaded.

3.5.9.2.2 Small Form Factor EAS Downlink Port Requirements:

All edge ports (user ports) shall support the following without the addition of hardware, software, firmware or licensing:

Minimum four (4) ports of PoE+ (IEEE 802.3at-2009) with 25.5 watts of power/port.

Spanning-tree BPDU detection and protection.

Time Domain Reflectometry (TDR) diagnostic testing capability.

Shall support auto Media Dependent Interface/Crossover (MDI/MDIX).

Shall support Source Guard (STIG).

Shall support Unknown Unicast Flood Block (UUFB) (STIG).

Shall support Root Guard (STIG).

3.5.10 ADS (Area Distribution Switch) Requirements

ADS shall be used by the United States Government (USG) to aggregate multiple switches geographically dispersed in a building prior to reducing the number of uplinks needed between an ACS and EAS. A common use may be in a main telecommunication room (MTR) to aggregate the traffic from switches in secondary TRs. Where outside plant infrastructure exists, the preferred solution shall be to utilize the MTR as a fiber distribution node (FDN) to connect EAS’ back to the appropriate ACS. An ADS capability shall only be used when the outside plant cannot support EAS direct uplinks to the ACS.

All ADS shall support the following requirements without the addition of hardware, software, firmware or licensing:

General Requirements Shall be ASLAN-Certified.

Input voltage capable of both 120/220 VAC or 48 VDC.

Redundant power supplies (N+1 Redundancy).

Hot-swappable Power Supplies.

Capable of assigning up to 4094 VLAN Tags (Trunking).

Supports minimum Jumbo Ethernet frame size of 9000 bytes.

Shall be able to change, remove or disable the default VLAN from trunks.

Shall support at a minimum eight (8) queues for traffic prioritization.

Shall be able to assign traffic to queues.

By VLAN tags, types of traffic or DIFFSERV Code Point.

Set QoS by DIFFSERV Code Point based on type of traffic (e.g.: VoIP).

Redundant Fans (N+1 Redundancy).

Hot-swappable Fans.

Shall have a backplane that is non-blocking.

Line-rate throughput on all ports at full capacity.

Shall be same OEM vendor as EAS devices for link layer integrity.

Capable of filtering spanning-tree BPDU.

Support IEEE 802.3ba 40/100GB Ethernet (Backbone/Uplink Ports) for future-proofing.

Uplink Port Requirements:

A minimum of 2 each 40/100GBASE-X SFP+.

Shall be MACSec capable.

Downlink Port Requirements (10GbE to EASs) Shall support a minimum of 24 port 40/100GBASE-X SFP+ (dual-speed 1/100GbE).

Shall be MACSec capable.

3.5.11 ACS Switch Requirements

ACS shall be used by the USG to form the core backbone and Layer 2 domains of the TCAN.

There shall be two (2) types of ACS, a high-density and a low-density one; based on the maximum of ports that each type can support.

All ACS shall support the following requirements without the addition of hardware, software, firmware or licensing:

General Requirements:

Shall be a modular chassis type device.

All modules/components shall be interchangeable between high-density ACS and low-density ACS chassis.

Shall be able to change, remove or disable the default VLAN from trunks.

Capable of assigning up to 4094 VLAN Tags (Trunking).

Supports minimum Jumbo Ethernet frame size of 9000 bytes.

Control and data forwarding plane modules shall be physically separate.

Shall support at a minimum eight (8) queues for traffic prioritization.

Shall be able to assign traffic to queues.

By VLAN tags, types of traffic or DIFFSERV Code Point.

Set QoS by DIFFSERV Code Point based on type of traffic (e.g.: VoIP).

Shall be flexible in compatibility.

Module available to provide 100GB uplink/crosslink for future environments (IEEE

802.3ba).

Module available to provide 48 port 1GbE-SFP downlinks for legacy environments.

Internal components, other than I/O modules, shall be field replaceable non-disruptively, and shall support OIR.

Shall have redundant switching engines/modules.

Capable of supporting input voltages of both 120/220 VAC or 48VDC.

Shall have N+1 redundant and hot-swappable power supplies.

Shall have N+1 redundant and hot-swappable fans.

Capable of filtering spanning-tree BPDU.

Capable of supporting MACSec encryption as per IEEE 802.1AE Security standard.

Uplink Port Requirements:

A minimum of 2 each 100GbE ports.

Crosslink Port (for interconnecting ACS switches):

A minimum of 2 each 100GbE ports for inter-connecting ACS switches.

Downlink Port (10GbE ports to EAS or ADS):

High-Density ACS chassis shall support up to and including a 50% capacity for expandability 100GbE SFP+ interface ports – in non-blocking configuration (Line Rate and Full Layer 2 Functionality).

Low-Density ACS chassis shall support up to and including a 50% capacity for expandability 100GbE SFP+ interface ports – in non-blocking configuration (Line Rate and Full Layer 2 Functionality).

3.5.12 L2BS Switch Requirements

L2BS shall be used by the USG to form controlled interconnects between multiple Layer 2 Domains (Areas).

All L2BS devices shall support the following requirements without the addition of hardware, software, firmware or licensing:

General Requirements Shall be ASLAN-Certified as Layer 2 Distribution Switch.

Input voltage capable of both 120/220 VAC or 48 VDC.

Redundant power supplies (N+1 Redundancy).

Hot-swappable Power Supplies.

Capable of assigning up to 4094 VLAN Tags (Trunking).

Supports minimum Jumbo Ethernet frame size of 9000 bytes.

Shall be able to remove or disable the default VLAN from trunks.

Shall support at a minimum eight (8) queues for traffic prioritization.

Shall be able to assign traffic to queues.

By VLAN tags, types of traffic or DIFFSERV Code Point.

Set QoS by DIFFSERV Code Point based on type of traffic (e.g. VoIP).

Redundant Fans (N+1 Redundancy).

Hot-swappable Fans.

Capable of filtering spanning-tree BPDU.

Uplink Port Requirements:

A minimum of 2 each 40/100GBASE-X SFP+.

Shall be MACSec capable.

Downlink Port Requirements (100GbE to EASs) Shall support a minimum of 24 port 40/100GBASE-X SFP+ (dual-speed 40/100GbE).

Shall be MACSec capable.

100GBASE-LR, SFP+ Optics (LC) System shall support following SFP+ type optics.

100GBASE-LR, SFP+ optic (LC), rated for 10km over Single Mode Fiber.

100GBASE-ER, SFP+ optic (LC), rated for 40km over Single Mode Fiber.

100GBASE-ZR, SFP+ optic (LC), rated for 80km over Single Mode Fiber.

100GBASE-SX, SFP optic (LC), for multimode fiber transmissions.

100GBASE-LX, SFP optic (LC), for single mode fiber transmissions.

3.6 Network Management System (NMS)

NMS is a critical tool for the EPT TISN-EN network and shall allow operators to monitor the health of the entire network, facilitate troubleshooting, and ensure operation and integrity of the network. NMS shall allow for detection of fault, availability and performance of devices connected to the network to aid in detection, diagnosis, and resolution of network connectivity issues. It also addresses Maximum Transmission Unit (MTU) issues across the network. The NMS shall have the following capabilities:

Monitor and manage network device configurations, and receive real-time change alerts.

Automatically discover and map Layer 2 and Layer 3 network topology, switch ports, Virtual Local Area Networks (VLANs), subnets, and inventory.

Automate network configuration backups, comparisons, and rollback.

Analyze network traffic, bandwidth performance, and traffic patterns by monitoring flow data.

Quickly identify network and application reliability and performance issues with deep packet inspection and analysis.

Improve network reliability with switch, port, and endpoint monitoring.

Discover, monitor, configure, and diagnose network infrastructure issues.

Enhance network security by quickly finding missing and rogue devices, as well as receiving alerts when a user or device connects to the network.

Monitor business critical application performance and ensure optimal network utilization and bandwidth usage.

Identify bandwidth hogs and see which endpoints, enclaves, and applications are using the most bandwidth.

Eliminate IP address conflicts with centralized IP infrastructure management, monitoring, and alerting reporting.

Monitor and troubleshoot critical network path performance issues.

Improve security and reduce downtime with real-time configuration change alerts and compliance management.

Server and application monitoring with remote and local notification.

Support multiple classifications and isolated secured networks.

Shall be Army compliant with Cybersecurity Guidelines

3.7 Cable and Equipment Labeling

Cable and Equipment Labeling shall be per current Army Network standards In all cases, when an exception is taken to the model labeling it shall be documented and approved by EPG prior to implementation.

4.0 Verification

All hardware/software, the NMS and complete EPG TISN-EN Network shall be subject to the following types of verification for compliance with performance requirements. A complete test plan approach and verification methodology shall be developed and implemented to verify traceability to all requirements contained herein this specification.

4.1 Methods of Verification

The following methods shall be utilized to accomplish the verification of requirements:

1) Analyses

2) Demonstrations

3) Examinations

4) Tests

4.2 Classes of Verification

The EPG TISN-EN Network shall be subjected to the following verification classes:

1) Functional Verification Demonstration Test

2) Network Performance Tests/On-Site

The inspections and tests are specified in Section 4.3, below.

4.3 Inspections and Tests

4.3.1 Visual Inspections

The visual inspection shall consist of verification of the availability of the EPG TISN-EN documentation relative to testing requirements, a walk through inspection to determine compliance with requirements, and familiarization with networking procedures.

4.3.2 Functional Tests

Perform basic power-up and green light testing of hardware prior to integration.

Perform basic ping testing after integration for connectivity verification.

4.3.3 Network Performance Test/On-Site

This test shall be used to quantitatively and qualitatively measure the performance of the network and the quality of service. It shall test the uplink and downlink speed of the network.

It shall defines how quick and responsive a network is to user/data communication. It shall be done by uploading and downloading a data object from the network and measuring both upload and download speeds, throughput, successful message delivery rate and more. Utilize signal generators to accomplish this load testing. Perform the following at a minimum for network performance testing:

Download/upload speed test Penetration test (security) Network load test Redundancy Availability

Metrics used in the network performance test shall be but not limited to:

Network speed Throughput Download/upload speed Jitter Latency/delays

Utilize industry standards such as ITU Y.1564 to verify compliance with the EPG TISN-EN Specification requirements.

4.3.4 Non-Functional Examinations

The EPG TISN-EN shall be subject to the non-functional examinations performed to demonstrate that the as-built design satisfies the specified requirements. The examinations shall encompass all areas, major assemblies, and subassemblies of the areas of the completely assembled system. The examinations shall verify conformance to the requirements for:

Hardware Design Assembly and fit Labeling of cable and hardware Workmanship

5.0 GLOSSARY/ACRONYMS

AAA - Authentication, Authorization and Accounting - a security architecture for distributed systems, which enables control over which users are allowed access to which services, and how much of the resources they have used

ACL – Access Control List

ACS – Area Core Switch

ADS – Area Distribution Switch – Intermediate aggregation switch that collects EAS devices before unlinking into the ACS devices.

AES – Advanced Encryption Standard

ANSI - American National Standards Institute

ARP - Address Resolution Protocol - a telecommunications protocol used for resolution of network layer addresses into link layer addresses.

ASLAN - Assured Services Local Area Network

ATEC – Army Test and Evaluation Command

Authenticator - Software or service running on a wireless access point or switch that manages the authentication process between the supplicant and authentication server.

Authentication server - Software or service that provides authentication services to the authenticator. Using the credentials provided by the supplicant, the authentication server controls whether the supplicant is authorized to access the services provided on the authentication server’s protected network.

Joint Test Center Collection Router – A Joint Test Center Collection Router (JT-CR) is an ATEC owned/life-cycled set of routers

BPDU - Bridge Protocol Data Units - data frames used by a bridge to exchange information about bridge IDs and root path costs.

CDP - Cisco Discovery Protocol - network protocol used to discover and share information between connected Cisco equipment.

CLI - Command Line Interface

CNAME – Canonical Name (domain name system record)

CoS – Class of Service

CPU – Computer Processing Unit

C2 – Command and Control

DAI – Dynamic ARP Inspection

DHCP - Dynamic…