Infrastructure Standard for Telecommunications Spaces.pdf

OFFICE OF

INFORMATION

AND TECHNOLOGY

Infrastructure Standards for Telecommunications Spaces Version 2.2

February 19, 2019 | Solution Delivery Data Center Engineering

OFFICE OF INFORMATION AND TECHNOLOGY

INFRASTRUCTURE STANDARDS FOR TELECOMMUNICATIONS SPACES| i

Table 1: Version Changes

Date Reason for Changes Version Author 2007 Original 0.0 Michael Julian, RCDD 2009 Revision 0.1 Michael Julian, RCDD 2012 Revision 1.0 Michael Julian, RCDD 12/07/2016 EDICT Revision 1.1 EDICT Team 2/05/2018 -Expanded revision including the addition of

Administrative and Operations Standards.

-Name changed to include IT Support Infrastructure to reflect broader scope.

1.2 EDICT Team

6/11/2018 -Minor update to allow for deeper network racks in cooperation with IO’s request.

-Expanded allowance for fiber cabinets, cassettes, patch panels, and racks.

-Changed NDCOL facility ID numbers to VA Record ID numbers.

-Clarified reference to OIT responsibility for the enterprise-wide technical framework and IT architecture services.

2.1 EDICT Team

2/19/2019 -Minor editorial changes resulting from initial review by the Office of Construction and Facilities Management (CFM) Facilities Standards Service (FSS) for publication in CFM’s Technical Information Library (TIL).

-Select operational information in Section 3.3 not required for building planning and design removed and will be published as part of the VA Operations and Maintenance Standard for Telecommunications Spaces. In the interim, refer to the June 11th, 2018 version 2.1 of The Standards. VA employees and contractors with VA intranet access will find it on the Baseline Configuration Management Standards site VA Enterprise Facility IT Support Infrastructure Standards. All others must submit a request to EDICT using the contact information in Section 1.3.

-Name updated from VA Enterprise Facility IT Support Infrastructure Standards to Infrastructure Standards for Telecommunications Spaces.

-508 compliance updates.

2.2 Bryan Unger, CFM

Dave Tash, CFM EDICT Team https://vaww.vashare.oit.va.gov/sites/itops/svcs/sma/BCM/Standards Lists/VA Enterprise Facility IT Support Infrastructure Standard_Final Approved_5 Feb 18.pdf https://vaww.vashare.oit.va.gov/sites/itops/svcs/sma/BCM/Standards Lists/VA Enterprise Facility IT Support Infrastructure Standard_Final Approved_5 Feb 18.pdf

INFRASTRUCTURE STANDARDS FOR TELECOMMUNICATIONS SPACES| ii

Table of Contents Infrastructure Standards for Telecommunications Spaces........................................................i Table of Contents ........................................................................................................................... ii Figures .......................................................................................................................................... iv Tables ............................................................................................................................................ v 1 Introduction

1.1 Purpose

1.2 Scope

1.3 Administration

1.4 Standards Overview

1.5 Background

1.6 Objectives

1.7 Authority

2 Data Center Classification

2.1 Core Data Centers (CDC)

2.2 Mission Support Centers (MSC)

2.3 Campus Support Centers (CSC)

2.4 Network Support Center (NSC)

3 Standards

3.1 Infrastructure Standard

3.1.1 Structured Cabling

3.1.1.1 Unshielded Twisted Pair (UTP)

3.1.1.2 UTP Patch Panel Standards

3.1.1.3 UTP Patch Cord

3.1.1.4 Fiber Optic Cable

3.1.1.5 Fiber Distribution Cassettes

3.1.1.6 Fiber Patch Cords

3.1.1.7 Fiber Distribution Panel/Cabinet

3.1.1.8 Cable Support Infrastructure

3.1.1.9 Work Area Outlets (WAO)

3.1.1.10 Horizontal Distribution (HD)

3.1.1.11 Vertical (Backbone) Distribution

3.1.2 Telecommunications Space Design

3.1.2.1 Building Specifications for Common Telecommunications Spaces

3.1.2.2 Data Center Layout Standard

3.1.2.3 Data Center Planned Redundancy Levels

3.1.2.4 Telecommunications Room (TR) Layout

3.1.2.5 Entrance Room (ER) Layout

3.1.2.6 Network Equipment Racks

3.1.2.7 Equipment Cabinets

3.1.3 Telecommunications Space Environment Conditioning

3.1.3.1 Data Center Facility Environment Conditioning Standards (CDC, MSC, and CSC)

3.1.3.2 Data Center Facility Environmental Standards – Exceptions

3.1.3.3 Environmental Control Equipment Requirements

3.1.3.4 Airflow Control

3.1.3.5 HVAC Environmental Control Equipment

3.1.3.6 Computer Room Air Conditioners (CRAC) Units

3.1.3.7 Split System Air Conditioners

3.1.3.8 Self-Contained Air Conditioners

INFRASTRUCTURE STANDARDS FOR TELECOMMUNICATIONS SPACES| iii

3.1.3.9 Close-Coupled Air-Conditioners

3.1.3.10 Humidity Control Equipment

3.1.3.11 Monitoring

3.1.4 Facility IT Support Power Distribution

3.1.4.1 Topology for Power Distribution Standard (CDC, CSC, MSC)

3.1.4.2 Branch Power Circuit to Rack/Cabinet

3.1.4.3 Zone PDUs

3.1.4.4 Rack Mounted UPS

3.1.4.5 Vertical Rack PDUs

3.1.4.6 Cable Distribution

3.1.4.7 Metering

3.2 Administration Standard

3.2.1 Data Center Position Identification

3.2.2 Cross-Reference to Other Identification Systems

3.2.3 Standardized Data Center Facility Type Identification

3.2.4 Standardized Data Center Naming Convention

3.2.5 Color Coded Identification

3.2.5.1 Data Communication Cabling Color Coding

3.2.5.2 Power Distribution Color Coding

3.2.6 ITS Equipment and Component Labeling

3.2.6.1 Label Materials

3.2.6.2 Support Infrastructure Identification

3.2.6.3 IT Equipment Cabinet/Rack Location Identification

3.2.6.4 Transport Media and Interface Identification

3.2.6.5 Power Distribution Identification

3.2.6.6 Data Communications Cabling Labeling

3.2.6.7 IT Equipment Rack/Cabinet Labeling

3.2.6.8 IT Equipment In-Rack/In-Cabinet Power Distribution Labeling

3.2.6.9 Power Distribution to IT Labeling

3.2.6.10 Physical Label Format

3.3 Management Standard

3.3.1 Management of Telecommunications Cabling

3.3.1.1 Installation Guidance

3.3.1.2 Cable Bundling Guidance

3.3.1.3 UTP, Fiber, and Power Cabling Guidance

3.3.2 Management of Airflow

Appendix A: Definitions Appendix B: References Appendix C: Acronyms and Initialisms Appendix D: Contributors

INFRASTRUCTURE STANDARDS FOR TELECOMMUNICATIONS SPACES| iv

Figures Figure 1: Method A Polarity of Horizontal Pre-terminated MPO-based Fiber Optic Cabling

Figure 2: Polarity of Fiber Optic Patch Cable

Figure 3: Typical Standard-Density WAO

Figure 4: Typical High-Density WAO

Figure 5: Hot Aisle/Cold Aisle Approach

Figure 6: Generic Redundancy Level Standards for Physical Plants, Facilities, and Infrastructure Systems

Figure 7: 2011 ASHRAE Environmental Classes for Data Center Applications

Figure 8: Blanking Panel Application

Figure 9: Brushed Floor Grommet

Figure 10: Air Dam Foam

Figure 11: Topology with Site UPS Backup (CDC, CSC, MSC)

Figure 12: TR Topology with In-Rack UPS Backup (NSC)

Figure 13: Power Utilization Effectiveness (PUE)

Figure 14: Typical Position Identification Examples

Figure 15: Example of Best-Practice Differentiated Power Cord Coloration Implementation

Figure 16: Rack Unit (RU) Identification Example

Figure 17: Example of Communications Cable Labeling

Figure 18: Detailed Example of Communications Cable Labeling

Figure 19: Power Whip Label Example

Figure 20: Equipment-Specific Cable Management Requirements

Figure 21: Acceptable Cable Management Results

Figure 22: Unacceptable Cable Management Results

Figure 23: Cable Labels in Bundles

Z:\vacowernaj 6 Aug 18\A_Data Center Reference\Standard_V2.2_Draft\ST-DC-V2_2-Infrastructure Standard for Telecommunications Spaces_19 Feb 19_Draft_Before 508.docx Z:\vacowernaj 6 Aug 18\A_Data Center Reference\Standard_V2.2_Draft\ST-DC-V2_2-Infrastructure Standard for Telecommunications Spaces_19 Feb 19_Draft_Before 508.docx

INFRASTRUCTURE STANDARDS FOR TELECOMMUNICATIONS SPACES| v

Tables Table 1: Version Changes ................................................................................................................. i

Table 2: Data Center Classification Standard

Table 3: UTP Standard

Table 4: UTP Patch Panel Standards

Table 5: UTP Patch Cord Standards

Table 6: Fiber Optic Cable Standards

Table 7: Fiber Distribution Cassettes

Table 8: Fiber Patch Cord Standards

Table 9: Fiber Distribution Panel/Cabinet Standards

Table 10: Cable Support Infrastructure Standards

Table 11: Building Specifications

Table 12: Data Center Floor Layout Standard

Table 13: Data Center Planned Redundancy Levels

Table 14: Core Data Center (CDC) and Campus Support Center (CSC) Planned Redundancy Levels

Table 15: Mission Support Center (MSC) Planned Redundancy Levels

Table 16: Network Support Center (NSC) Planned Redundancy Levels

Table 17: Network Equipment Racks

Table 18: IT Equipment Cabinet Standards

Table 19: ASHRAE Class B Environmental Requirements

Table 20: Data Center Facility Environment Conditioning Standards (CDC, MSC, and CSC)

Table 21: CDC Facility Environmental Requirements (Exceptions)

Table 22: MSC Facility Environmental Requirements (Exceptions)

Table 23: CSC Facility Environmental Requirements (Exceptions)

Table 24: NSC Facility Environmental Requirements (Exceptions)

Table 25: Environmental Control Equipment

INFRASTRUCTURE STANDARDS FOR TELECOMMUNICATIONS SPACES| vi

Table 26: Monitored Conditions

Table 27: Branch Power Circuit to Rack/Cabinet Standard

Table 28: Zone PDU Standard

Table 29: Rack Mounted UPS Standard

Table 30: Vertical Rack PDU Standard

Table 31: Cable Distribution Standards

Table 32: Metered Energy Consumption Data

Table 33: Energy Consumption Performance Metrics

Table 34: Data center Position Identification Conventions

Table 35: Standard FACILITYTYPE Naming Conventions

Table 36: Color Identification Conventions

Table 38: Support Infrastructure Identification Convention

Table 42: Contributors

INFRASTRUCTURE STANDARDS FOR TELECOMMUNICATIONS SPACES| 1

1 Introduction

1.1 Purpose

The Infrastructure Standards for Telecommunications Spaces (The Standards) serves as the master reference document of criteria for Department of Veterans Affairs (VA) facility Information Technology (IT) support infrastructure. It defines the technical requirements necessary to maintain optimum reliability and efficiency within VA facilities and computing centers.

1.2 Scope

The Standards shall be applied to all aspects of IT support infrastructure at all VA facilities. All aspects” includes, but is not limited to: planning, design, construction, sustainment (operations, maintenance, repair), restoration, modernization, and administration. ”All VA facilities” includes but is not limited to VA Central Office (VACO) facilities and all field facilities managed by VA Administrations (Veterans Health Administration (VHA), Veterans Benefits Administration (VBA), and the National Cemetery Administration (NCA)).

IT Support Infrastructure is defined as all passive telecommunications and information technology equipment, and supporting physical space and equipment, supporting:

· Information Transport Systems (ITS) (e.g. copper and fiber cabling);

· IT equipment and systems (servers, storage, network switching and distribution, etc.);

· Telecommunications spaces (telecommunications rooms, entrance facilities, data centers, horizontal distribution pathways, vertical distribution pathways, etc.);

· Telecommunications equipment and systems (local and wide area network, telephone, cable television, physical security IT system, etc.);

· Physical infrastructure equipment and systems necessary to support active equipment operations at the required availability and sustainability levels (power distribution, environmental conditioning, monitoring and metering equipment, grounding, etc.).

INFRASTRUCTURE STANDARDS FOR TELECOMMUNICATIONS SPACES| 2

1.3 Administration

The Standards shall be implemented without modification, except when required by an outside Authority Having Jurisdiction (AHJ). AHJ requirements shall be provided to the Enterprise Data Center Infrastructure Collaboration Team (EDICT) for evaluation of project application and amending The Standards.

Projects in the planning, design, or implementation phases shall contact EDICT for assistance in ITS, data center, and telecommunications space design and standards compliance.

Existing IT support infrastructure that doesn’t conform to The Standards shall be brought into compliance during routine tech refresh, lifecycle replacement, upgrades, new installations, or renovations of existing space.

Facility managers should take every opportunity to bring telecommunications spaces into compliance with The Standards through incremental changes between scheduled sustainment activities.

Area managers shall ensure that all contracting officers, IT personnel, and engineering staff are aware and make use of The Standards.

Requests for variances from or changes to The Standards shall be submitted to EDICT for consideration. If approved by EDICT, proposed change to The Standards will be ratified through the Systems Engineering Design Review (SEDR) process, and submitted to the Enterprise Change Management Datacenter Change Advisory Board for publication.

Transmit all communications to EDICT via e-mail at VAEDICT@va.gov.

1.4 Standards Overview

The Standards include standard specifications, decisions supporting the standard specifications, guidelines or recommendations for implementing the standard specifications, and supplemental factors to consider when evaluating subject components.

The Infrastructure Standard and the Administration Standard define the topologies and specifications for facilities and systems in the VA enterprise.

· The Infrastructure Standard also provides guidance on procuring components that meet the standard specifications, guidance on integrating them with existing components, and explanation of how the subject components fit into the VA Architecture.

· The Administration Standard also lists the acceptable and recommended specifications for Inside Plant naming conventions, including racks, equipment enclosures, fiber and Unshielded Twisted Pair (UTP) transport media, power distribution for active and passive elements, cable plant, and data center naming conventions.

The Management Standard describes facility computer room operation specifications, management of telecommunications cabling, and management of airflow.

mailto:VAEDICT@va.gov

INFRASTRUCTURE STANDARDS FOR TELECOMMUNICATIONS SPACES| 3

1.5 Background

A standard is a set of rules or requirements that are determined by a consensus opinion of subject matter experts and prescribe criteria for a product, process, test, or procedure. The general benefits of a standard are quality, interchangeability of parts or systems, consistency, and reduction of lifecycle costs.

IT standards are based on business needs provided through or supported by IT services. IT services are designed to support business processes and are constructed from software, hardware and infrastructure components.

Establishing and enforcing standards for the selection and configuration of these supporting components improves the capacity, sustainability, maintainability, reliability, and availability of IT services within projected economic constraints in alignment with business needs.

1.6 Objectives

This standard provides recommended specifications across all VA facilities to support:

• Solution evaluation

• Requirement evaluation

• Solution design

• Solution procurement and bid evaluation

• Evaluation of architectural specifications

• Standardization of passive Inside Plant infrastructure

• Standardization of design criteria for telecommunications spaces, facilities physical infrastructure systems, and operation and maintenance of systems supporting VA services

• Standardization of naming and identification conventions for VA data centers

• Standardization of Inside Plant and rack power infrastructure naming conventions

• Standardization of labeling and identification conventions for physical plant systems and components

• Standardization of operations

• Operational solution procurement and bid evaluation

• Evaluation of maintenance, repair, and replacement specifications

• Standardization of lifecycle sustainment activities across all VA data centers

• Standardization of the operational, organizational, and aesthetic results of Sustainment, Maintenance, and Repair (SM&R) activities

INFRASTRUCTURE STANDARDS FOR TELECOMMUNICATIONS SPACES| 4

1.7 Authority

OIT is responsible for the enterprise-wide technical framework and IT architecture services for VA systems and projects. Centralized responsibility promotes one technology vision across VA, which supports system optimization, integration, and interoperability throughout the enterprise.

One way in which OIT executes this responsibility is determining technical standards for deployed infrastructure technologies. These standards are also known as configuration baselines. The authority of The Standards derives from the approved enterprise configuration baselines.

INFRASTRUCTURE STANDARDS FOR TELECOMMUNICATIONS SPACES| 5

2 Data Center Classification VA has four segregated data center types to support the enterprise distributed computing environment.

· Core Data Centers (CDC) provide enterprise-level support services for VA-wide and cross-component systems and applications.

· Mission Support Centers (MSC) are highly specialized data centers that provide unique, focused services that are best provided from smaller, mission-focused facilities that cannot or should not be supported by CDCs or CSCs due to MSCs’ association with mission specific infrastructure or equipment.

· Campus Support Centers (CSC) operate local data centers to support VA tenants located on a VA campus and within geographically supported areas. CSCs provide services that cannot be provided (technically or economically) from a CDC. Only one CSC is authorized per VA campus.

· Network Support Centers (NSC), provide lower-level application and IT services distribution across the geographic breadth of the VA’s computing enterprise, enabling special functions and distributed users to satisfy the specific Agency mission (Note: Telecommunications Rooms (TR) generally follow NSC rating classification and redundancy requirements).

2.1 Core Data Centers (CDC)

CDCs are facilities that provide shared and distributed enterprise services to supported sites.

CDCs host cloud services (IaaS, PaaS, SaaS) to provide complete end-to-end IT services (including but not limited to application, backup, and storage) to the supported VA sites.

The CDC architecture is considered mission-essential and is designed, operated, and maintained to provide availability consistent with current VA national data center standards for supported IT applications. CDCs are the target architecture component for IT services that span multiple Department components and user groups (services and applications used by many or all users, such as messaging, productivity, and collaboration).

CDCs may be VA-owned, leased, or government or commercial outsourced (external cloud) data centers. CDCs meet (or are planned to meet) all published CDC physical infrastructure requirements to ensure enterprise reliability, redundancy, High Availability (HA), and Disaster Recovery (DR) requirements.

CDCs may also provide services designated to be provided by NSC if no local NSC to provide those services is available. CDCs may also provide MSC-type application support.

2.2 Mission Support Centers (MSC)

MSCs provide element-specific and single-instance special enterprise services that may be inappropriate for consolidation to CDC facilities. MSCs are the target architecture for specialized-function systems and applications, particularly those that support a specific but

INFRASTRUCTURE STANDARDS FOR TELECOMMUNICATIONS SPACES| 6

limited and distributed user group (services allowing the Department to function but with a limited user base such as internal financial management, or applications requiring centralized processing for a small number of user facilities, such as consolidated pharmacies).

MSC physical requirements supporting the computing environment may be more lenient than those for CDCs. Specific requirements will vary based on the supported functions, criticality of the mission, and similar criteria. For example, MSCs designated as enterprise Test/Development (Test/Dev) environments will not require the same level of high-availability physical and network redundancy as a mission-critical production environment that directly supports patient care.

MSCs may also provide services designated to be provided by an NSC if no local NSC to provide those services is available.

2.3 Campus Support Centers (CSC)

CSCs provide geographically-specific, operational IT services in support of campus services to Veterans and VA employees that cannot be effectively consolidated or provided over cloud architectures to the campus. CSCs may also provide services designated to be provided by an NSC if no local NSC to provide those services is available.

IT services at CSCs are provided by multiple organizations, including Clinical Engineering (CE) a.k.a. biomed, health informatics, Veterans Affairs Medical Centers (VAMC), Veterans Benefits Administration (VBA) Regional Offices, Office of Information Field Office (OIFO), and Research & Development (R&D), etc.

Future data center architecture models for CSCs are intended to provide for collocation of IT equipment and systems from all provider organizations into the minimum number of operational spaces on a campus; that is, consolidated data centers providing environmental and area network support appropriate to all IT equipment needed by tenant organizations on a particular campus to perform their missions.

2.4 Network Support Center (NSC)

NSCs provide local IT services that cannot be effectively consolidated or provided over a network-supported distributed architecture to the local operational site. Local means to the immediate facility and to facilities within the local commuting/networking area for the purposes of this architecture; NSCs may provide application support for all VA facilities in a metropolitan area, for example.

Where application architecture and network requirements require a closer data center point of presence to enable end users effective, efficient system access, an NSC is the appropriate target environment.

INFRASTRUCTURE STANDARDS FOR TELECOMMUNICATIONS SPACES| 7

Table 2: Data Center Classification Standard

ID Primary Attribute Secondary Attribute Specification

1 Core Data Center

(CDC)

Description

CDCs are centralized data centers that provide enterprise-level services to geographically-distributed VA organizations to support business functions. CDCs maintain complete High Availability (HA) and Disaster Recovery (DR)

Planned ANSI/TIA 942 Rating 3

Examples Information Technology Center (ITC) Regional Data Center (RDC)

2 Mission Support Center (MSC)

Description MSCs are stand-alone data centers that provide specific enterprise IT functionality to VA organizations and business functions

Planned ANSI/TIA 942 Rating 2 or 3

Examples

· Test/Dev data centers at Rating 2

· Financial Services Center (FSC)

· Centralized Mail Outpatient Pharmacy

(CMOP)

· Consolidated Patient Account Center

(CPAC)

· Health Administration Center (HAC)

3 Campus Support Center (CSC)

Description

CSCs are stand-alone data centers that provide specific IT functionality to a VA campus supporting all VA tenants on that campus and in the geographically supported area

Planned ANSI/TIA 942 Rating 3

Examples VA Medical Center (VAMC)

4 Network Support Center (NSC)

Description NSCs are local data centers that provide local IT application and network support to local operating locations

Planned ANSI/TIA 942 Rating 1

Examples

· Enterprise Test/Development data center

· Telecommunications Room (TR)

· Other small server rooms (not “shallow rooms”)

INFRASTRUCTURE STANDARDS FOR TELECOMMUNICATIONS SPACES| 8

Evaluation Factors

· Data center facility appropriately and centrally categorized by function and usage.

· Capability gaps between existing and planned EIA/TIA 942 Rating requirements identified.

INFRASTRUCTURE STANDARDS FOR TELECOMMUNICATIONS SPACES| 9

3 Standards

3.1 Infrastructure Standard

3.1.1 Structured Cabling

This standard supports structured cabling as a vendor-neutral, extensible approach to information transport system design in VA facilities, not limited to data center applications. The specifications for horizontal cabling for LAN applications and vertical cabling for intra and interbuilding backbone are laser-enhanced fiber optics and high-bandwidth Unshielded Twisted Pair (UTP). The minimum bandwidth requirement for fiber is 40 Gbe and 10 Gbe on UTP. All local and campus area networks shall be installed in a structured topology with patch panels serving transition points. Individual “home run” cables are not permitted in this standard for LAN applications, including data centers.

Both Horizontal Distribution Areas (HDA), which serve as the distribution point for horizontal cabling and houses Horizontal Cross-connects (HC) and equipment for connecting to the equipment distribution area, and the Main Distribution Areas (MDA) where the main cross-connect is located as defined by ANSI/TIA 942 specifications should be located adjacent to switching components within the rack to minimize patch cord length and cable management requirements. This approach is consistent with ITS industry standards, and it supports the SD LAN and data center network design baselines.

Work-Area Outlets (WAO), media, media connectors, and patch panels shall support the minimum performance and capacity requirements of this standard.

This standard has been harmonized with OIT LAN Baselines.

3.1.1.1 Unshielded Twisted Pair (UTP)

This standard specifies pre-terminated Category 6A (Cat 6A) UTP for horizontal distribution.

This performance category is applicable to field-terminated or factory pre-terminated applications. The preferred termination method is factory pre-terminated cabling for UTP with each cable terminating in an individual connector. While pre-terminated cables require that distances between termination points are known this approach results in the rapid installation of horizontal cabling without risks associated with field termination including wire fragments, impaired cabling performance due to poor installation techniques, faulty terminations, etc.

Spiral wrapping pre-terminated bundles is an effective way to maintain the bundle of six cables without impacting ease of installation.

Factory pre-termination is the preferred termination method but field-termination is acceptable for work-area distribution applications and where pre-termination is impractical.

Cat 6A cabling is specified in standard ANSI/TIA-1179 Healthcare Facility Telecommunications Infrastructure for new installations. Variances will have to be requested and granted by EDICT.

INFRASTRUCTURE STANDARDS FOR TELECOMMUNICATIONS SPACES| 10

Table 3: UTP Standard

ID Primary Attribute Secondary Attribute Specification

Performance Characteristics (Horizontal and First Level Backbone)

Performance Category Cat 6A (10 Gigabit Ethernet (GbE)) Cat 6 (grandfathered and authorized variances only)

Performance Specifications Meets or exceeds TIA-EIA-568-C.2-10, TSB-155

Wire Size 26 AWG Maximum

Smoke Rating

· Riser cable for vertical runs through floors

· Plenum rated for plenum spaces

· Not rated for other applications

· Or as per AHJ requirement

Jacket Color Blue (to match 606C HC patching field designation)

Termination Method

· Pre-terminated, 8P8C termination method preferred OR

· Field-terminated to WAOs only where distances cannot be precisely calculated

Media Connector Pre-terminated with split 8P8C preferred

Bundling Multiple cable harness

Evaluation Factors

· Performance characteristics

· Termination (factory or field)

· Power over Ethernet (PoE)

· Compatibility with specified patch panels

· Smoke rating

Implementation Guidance Horizontal cabling, including fiber and UTP, shall be installed in a structured topology with patch panels serving as at all vertical and horizontal transition points including Equipment Distributors (ED), Horizontal Distributors (HD), and throughout the MDA for all backbone levels.

The patch panels at the HC and MDA shall be located adjacent to switching components to minimize patch cord length and cable management requirements.

Where installed, the ED patch panels will be installed in the top five (5) Rack Units (RU) of each enclosure or within small (five (5) RUs or less) racks mounted on cable tray above the associated equipment cabinet.

See Section 3.3 on cable routing and implementation.

INFRASTRUCTURE STANDARDS FOR TELECOMMUNICATIONS SPACES| 11

Pre-termination requires that distances between termination points are known; however, it assures maximum performance and rapid installation. Only use field-terminations to WAOs as required where distances can’t be precisely calculated.

3.1.1.2 UTP Patch Panel Standards

This standard anticipates the bandwidth demands of virtualization by specifying Cat 6A UTP in production and test facilities. This standard can be applied to field-terminated or factory pre-terminated applications. The preferred termination method is factory pre-terminated cabling for UTP. Cat 6A cabling is specified in standard ANSI/TIA-1179 Healthcare Facility Telecommunications Infrastructure for new installations.

Modularity of structured cabling components in the TRs, ERs, and data centers improves capacity planning and move/add/change outcomes. This standard specifies a one (1) RU high, 24 or 48 position patch panel that is performance tuned to the UTP. Angled panels eliminate the need for horizontal cable management in the rack and are the typical configuration. No horizontal cable management is required when specifying angled panels.

Table 4: UTP Patch Panel Standards

ID Primary Attribute Secondary Attribute Specification

1 Performance Characteristics

Performance Category Cat 6A (10 Gigabit Ethernet (GbE)) Cat 6 (grandfathered and authorized exceptions variances only)

Position Count 24/48

Form Factor

· One (1) RU

· MDA- angled

· HDA- angled (end of row or TR)

· ED

§ Flat in cabinet § or angled overhead

Color Coding Black

Evaluation Factors

· Performance category

· Form factor

· Compatibility with pre-terminated cable interfaces

Implementation Guidance See Operational Standards section on cable routing and implementation.

Horizontal cabling, including fiber and UTP, shall be installed in a structured topology with patch panels serving as at all vertical and horizontal transition points including EDs, HDs, and

INFRASTRUCTURE STANDARDS FOR TELECOMMUNICATIONS SPACES| 12

throughout the MDA for all backbone levels. The HC shall be located adjacent to switching components to minimize patch cord length and cable management requirements.

Where installed, the ED patch panels will be installed in the top five (5) RUs of each enclosure or within small (five (5) RUs or less) racks mounted on cable tray above the associated equipment cabinet.

3.1.1.3 UTP Patch Cord

This standard specifies center-tuned Cat 6a or Cat 6 grandfathered and authorized variances only patch cords. This standard does not specify color-coding for patch cable jackets. Based on the observations of hundreds of VA facilities, color-coded patch cables provide little or no utility and create inventory management issues. Effective color coding has been observed for in-rack, dual-homed server/appliance applications using a simple white/black scheme. Patch cables shall be factory pre-terminated and shall match the category of the associated patch panel and connected UTP.

Table 5: UTP Patch Cord Standards

ID Primary Attribute Secondary Attribute Specification

1 Performance Characteristics

Performance Category

· Category 6a (Cat 6 grandfathered and authorized variances only)

· Stranded

· 26-gauge

Performance Specifications Center tuned to horizontal media

Jacket Color Not specified

Termination Method Factory pre-terminated

Evaluation Factors

· Performance characteristics

· Center-tuned to installed patch panels

Implementation Guidance See Operational Standard section on cable routing and implementation.

Use most appropriate length to meet cable management standards.

Inside Plant Design best practices dictate that component-level compliance to a performance standard must be maintained across the entire horizontal link. The quality of patch cords and adherence to performance standards is critical to the reliable operation of a high-speed LAN.

3.1.1.4 Fiber Optic Cable

This standard specifies laser-enhanced 50/125 OM4-rated multimode fiber-optic cable.

INFRASTRUCTURE STANDARDS FOR TELECOMMUNICATIONS SPACES| 13

Pre-termination requires that distances between termination points are known; however, it assures maximum performance and rapid installation. In backbone and data center applications, 12 or 24-strand loose tube cable assemblies using Multi-fiber Push On (MPO) connectors is specified.

If fiber media is required for horizontal distribution to WOAs, a structured approach is required.

A stated need for fiber to WAOs should be scrutinized by the planner and justified by sound technical requirements.

Table 6: Fiber Optic Cable Standards

ID Primary Attribute Secondary Attribute Specification

1 Fiber (Horizontal and First Level Backbone)

Performance Category OM4 Laser Enhanced to 40 GbE

Performance Specifications Laser Optimized 50/125 µm fibers with effective modal bandwidth of at least 4,700 MHz∙km at 850 nm

Mode Multimode

Smoke Rating

· Riser cable for vertical runs through floors

· Plenum rated for plenum spaces

· Not rated for other applications

· Or as per AHJ requirement

Jacket Color Aqua

Termination Method Factory pre-terminated

Media Connector Pre-terminated with MPO

Strand Count 12 or 24 per assembly

Bundling Loose Tube

Polarity Straight (or Type A-Key up one end & key down on the other)

Evaluation Factors

· Warranty

· OM4 performance characteristics for 40 GbE

· Smoke rating requirements

· Technical requirements for inclusion in horizontal distribution to the WAO.

Implementation Guidance See Section 3.3 on cable routing and implementation

Pre-termination requires that distances between termination points are known; however, it assures maximum performance and rapid installation.

INFRASTRUCTURE STANDARDS FOR TELECOMMUNICATIONS SPACES| 14

This standard specifies the polarity of horizontal pre-terminated MPO-based fiber optic cabling as Type A or “straight-through.” This will ensure that position one (1) will correspond to position one on both ends of the cable. If a polarity flip for the channel is necessary, use a Type A/A patch cable on one end and a Type A/B patch cable on the other end. In some cases, an A/B cord can be converted to an A/A by switching the connector position on one end of the patch cord.

3.1.1.5 Fiber Distribution Cassettes

This standard specifies the transition component of the structured cabling system for fiber optic cabling in telecommunication spaces. It is the fiber equivalent to UTP patch panels. This standard specifies 12/24-strand, OM4-rated cassettes with duplex LC interfaces or MPO

Figure 1: Method A Polarity of Horizontal Pre-terminated MPO-based Fiber Optic Cabling

INFRASTRUCTURE STANDARDS FOR TELECOMMUNICATIONS SPACES| 15

interfaces. The cassette will be equipped with dual, 12-strand or single 24-strand MPO interfaces as the backbone or horizontal interface.

INFRASTRUCTURE STANDARDS FOR TELECOMMUNICATIONS SPACES| 16

Table 7: Fiber Distribution Cassettes

ID Primary Attribute Secondary Attribute Specification

1 Performance Characteristics

Form Factor One (1) RU

Capacity 3-12 cassettes

User Interfaces LC connectors (6/12 duplex per cassette) MPO connectors (8 fibers)

Backbone Interfaces 12-strand or 24-strand MPO (50 micron)

Type OM4 Laser Enhanced 40 GbE 50/125 Multimode

Evaluation Factors

· Performance characteristics

· Form factor

· Interfaces

Implementation Guidance This standard specifies Optical Multimode-4 (OM4)-rated (40 GbE) 850 nm laser-enhanced multi-cable fiber cable assemblies pre-terminated with MPO connectors. One cassette will be used to provide LC interfaces at both ends of the horizontal or backbone link and will support one or two 12-strand cable assemblies or one 24-strand cable assembly for a total capacity of 6 or 12 duplex LC interfaces. Cassettes can also be used to breakout 24-strand MPO backbone trunks into 3 MPO connections of 8 fiber strands, each providing 40GbE parallel transmission.

3.1.1.6 Fiber Patch Cords

This standard specifies high-bandwidth, factory pre-terminated fiber patch cords. The standard media interface for fiber patch cords is duplex LC. Patch cord polarity requirements can be accommodated by specifying a polarity-reversible duplex interface. When polarity requirements are not known, the polarity-reversible duplex LC interface should be specified.

Table 8: Fiber Patch Cord Standards

ID Primary Attribute Secondary Attribute Specification

1 Performance Characteristics

Performance Category 40 Gigabit OM4

Performance Specifications Laser enhanced, 50/125 multimode fiber, Duplex LC

Jacket Color Aqua

Termination Method Factory pre-terminated

Polarity

· Type A to B (Crossover)– ED

· Type A to A (Straight) – HDA and MDA

Evaluation Factors

· Polarity reversibility

INFRASTRUCTURE STANDARDS FOR TELECOMMUNICATIONS SPACES| 17

· Performance specifications

Implementation Guidance

Inside Plant design best practices dictate that component-level compliance to a performance standard must be maintained across the entire horizontal link. The quality of patch cords and adherence to performance standards is critical to the reliable operation of a high-speed LAN or

SAN.

The polarity of the fiber optic channel may need to be flipped depending on the application.

This will ensure that the equipment sending or transmitting will be connected to the receiving port of the equipment at the other end. If typical A/B fiber patch cords are used on both sides of the channel, continuity may be lost. As shown in Figure 5, this is corrected by either switching the connector position of both strands at one end of the patch cord or by using a Type A/A patch cable on only one end of the channel.

Pre-termination requires that distances between termination points are known; however, it assures maximum performance and rapid installation.

See Section 3.3 on cable routing and implementation.

Figure 2: Polarity of Fiber Optic Patch Cable

INFRASTRUCTURE STANDARDS FOR TELECOMMUNICATIONS SPACES| 18

3.1.1.7 Fiber Distribution Panel/Cabinet

This standard specifies a one (1) RU panel for mounting fiber cassettes. The smallest form-factor has been selected for application flexibility.

Table 9: Fiber Distribution Panel/Cabinet Standards

ID Primary Attribute Secondary Attribute Specification

1 Performance Characteristics

Form Factor One (1) RU

Cassette Capacity 3-12 cassettes

Type

· Cabinet (network rack installation or lock requirements)

· Panel (server cabinet with vertical exhaust, no security requirement for locks)

Evaluation Factors

· Form factor

· Cassette capacity

· Locking requirements

Implementation Guidance ED Fiber Distribution Panels will be installed in RU45 of each vertical exhaust duct cabinet where required. Fiber distribution panels shall be used in all other type implementations.

3.1.1.8 Cable Support Infrastructure

This standard creates uniform high-capacity horizontal and vertical cable management for Cat 6A patch cords.

Table 10: Cable Support Infrastructure Standards

ID Primary Attribute Secondary Attribute Specification

1 Horizontal Cable Management Panels

Form Factor

· One (1) RU (up to 50 Cat 6a patch cords)

· Two (2) RU (up to 100 Cat 6a patch cords)

Cover Double-hinged

Finger Spacing To meet requirements; typically 6-port spacing (five fingers)

Dimensions 6 in. to 16 in.

2 Vertical Cable Management Panels Door Double-hinged

Evaluation Factors

· RU Finger spacing

INFRASTRUCTURE STANDARDS FOR TELECOMMUNICATIONS SPACES| 19

· Double-hinged cover capacity

Implementation Guidance Vertical cable management will be utilized for the HDA and the MDA. Horizontal cable management within the rack will be utilized as necessary to encourage defined and traceable routing of individual patch cords from switches to the vertical cable managers.

3.1.1.9 Work Area Outlets (WAO)

WAOs are user interfaces to the cabling system within a workspace, floor, or building. This standard specifies the typical quantity and type of faceplate, wall box, conduit, and outlets for typical administrative work areas. However, this guidance does not preclude variations based on atypical capacity requirements nor does it supersede guidance or requirements set forth by The Office of Construction and Facilities Management (CFM) concerning Work Area Outlet specifications for specialized spaces.

Typical in-wall workboxes shall be a double-gang telecommunications-rated work box with rigid conduit extending to the horizontal pathway.

Floor boxes shall be dual-gang or “poke-thru” type. Cable that extends through a slab penetration shall terminate in the nearest practicable TR on the same serving floor.

The typical (standard density) WAO will consist of two RJ45 interfaces. This provides connectivity for one (1) IP telephone and one (1) workstation. Some IP phone base units have a data port. This allows the workstation to interface with the network via the phone base unit to leave one (1) RJ45 at the faceplate available for future use.

WAO media interfaces shall be tuned to the horizontal media, specified to the same performance rating, and installed according to best practices.

No “phone outlets” are specified in this standard. Legacy equipment, such as fax machines and analog phones, will require a variance from this standard via the variance request process.

Single-outlet posted faceplates shall be configured with one (1) RJ45 interface.

https://www.cfm.va.gov/TIL/

INFRASTRUCTURE STANDARDS FOR TELECOMMUNICATIONS SPACES| 20

Figure 3: Typical Standard-Density WAO

The high-density WAO configuration consists of four (4) RJ45 interfaces. This configuration is typically used for desktop support and other technical staff or in cases where multiple devices need network access from the same WAO.

INFRASTRUCTURE STANDARDS FOR TELECOMMUNICATIONS SPACES| 21

Figure 4: Typical High-Density WAO

Evaluation Factors

· Serving zone capacity requirements

· Special safety or medical device requirements

3.1.1.10 Horizontal Distribution (HD)

HD is defined as structured cabling that connects the WAO or the ED to a HD. The HD will be located in the end of row HDA or in the TR depending on the application.

Horizontal media shall be specified per section 3.1.1.1 Unshielded Twisted Pair. If the application, such as a data center, requires 40 GbE fiber, specify the media per section 3.1.1.4 Fiber Optic Cable.

3.1.1.11 Vertical (Backbone) Distribution

Vertical Distribution is defined as structured cabling that connects the HD to the Main Distributor (MD). The MDA will be located in a centralized location within the building, data center, or LAN serving area. Typically, this will be a Campus Support Center (CSC) or a data center.

The MDA shall be a separate and distinct telecommunications space within the LAN serving area. The MDA shall not be located in a Telecommunications Room.

INFRASTRUCTURE STANDARDS FOR TELECOMMUNICATIONS SPACES| 22

This standard only recognizes 50/125 Laser-Enhanced Multimode (OM4) as an acceptable backbone media (see 3.1.1.4). UTP shall be utilized only if fiber media cannot serve specialized applications such as legacy phones, fax machines, or security panels. Where copper backbone is required, this standard specifies quantity two (2) riser-rated Cat 6, 25-pair cables to be installed between the Telecommunications Room HDA and the Campus Support Center MDA. This approach will support twelve (12) patch panel positions. One 12-position Cat 6 patch panel shall be installed in the HDA and the MDA. Individual 4-pair Cat 6 UTP riser-rated cables may be used, but this approach is not recommended and will require technical review by OIT.

INFRASTRUCTURE STANDARDS FOR TELECOMMUNICATIONS SPACES| 23

3.1.2 Telecommunications Space Design

VA telecommunications spaces, including ERs, TRs, and data center facilities shall be designed and operated in accordance with the EIA/TIA-942 Ratings for each data center classification, except as detailed specifications are provided in this and other VA data center facilities standards.

3.1.2.1 Building Specifications for Common Telecommunications Spaces

Table 11: Building Specifications

ID Primary Attribute Secondary Attribute Specification

1 Shell

Security Via combination of manual and electronic control or minimum electronic control

Access Route Leads to location outside computer room for access control

Egress Observes life safety code

Door Height (Computer Room)

8 ft. minimum

Door Height (Mechanical Room)

9 ft. minimum

Door Width

· CSC, CDC, MSCs- 6 Ft.

· NSC- 3 ft.

Slab to Floor Above 16 ft. minimum

Raised Floor to Ceiling 12 ft. minimum

Raised Floor in new data center construction (used only if CFD analysis determines raised floor is the best cooling solution)

· 18 in. minimum above slab

· 24 in. fully loaded racks

· 36 in. high density

2 Floor Height

Slab to Floor Above 16 ft. minimum

Raised Floor to Ceiling 12 ft. minimum

Raised Floor in new data center construction (used only if CFD analysis determines raised floor is the best cooling solution)

· 18 in. minimum above slab

· 24 in. fully loaded racks

· 36 in. high density

INFRASTRUCTURE STANDARDS FOR TELECOMMUNICATIONS SPACES| 24

ID Primary Attribute Secondary Attribute Specification

Floor Tile

Perforated Tile

· Only positioned at the base of the cold aisle racks

· Positioned so that they can be removed without interference from racks

Dimensions 24 in. by 24 in.

3.1.2.2 Data Center Layout Standard

Data centers will be designed in accordance with the specifications below, VA standards, and industry best practices referenced in Appendix B.

Computational Fluid Dynamics (CFD) modeling shall be used to determine the most effective and cost-effective layout and containment solution for a given space (including new design/build, capability expansion, and lifecycle replacement). All data center design for new data centers, expansion, or significant modification of existing data centers will be approved through OIT Solution Delivery Data Center and Cloud Engineering (VAITESEDatacenterEngineering2@va.gov).

Airflow direction shall be a prime consideration when designing the floor layout and when selecting and procuring IT equipment. Equipment shall be specified, procured, configured, and installed so that the equipment draws cool air from the front face of the rack and flows from the front of the rack to the rear and the hot exhaust exiting the chassis at the rear face of the rack. Do not create conditions for exhaust to be recirculated back into cold air supply spaces.

Table 12: Data Center Floor Layout Standard

ID Primary Attribute Secondary Attribute Specification

1 Aisles

Orientation

· Front of racks will be oriented toward cold aisle as in Figure 2

· Front of racks will be aligned with each other to create a continuous linear aisle

· If a raised access floor is utilized, front of racks will be flush with edge of floor tiles

Width

· 4 ft. minimum in cold aisles

· 3 ft. minimum in hot aisles (4 ft.

recommended)

Cold Aisle Power cables if run in an access floor will be placed in cable trays in cold aisles mailto:VAITESEDatacenterEngineering2@va.gov

INFRASTRUCTURE STANDARDS FOR TELECOMMUNICATIONS SPACES| 25

ID Primary Attribute Secondary Attribute Specification

Hot Aisle Data cables if run in an access floor will be placed in cable trays in hot aisles

Containment

Hot or cold aisle containment will be determined based on data center layout, Heating, Ventilation, and Air Conditioning (HVAC) configuration, and analysis

2 Support Equipment

Computer Room Air Condition

(CRAC)

· Placed along the perimeter of the data center or in a dedicated mechanical room

· Centered on Hot aisles

· Exact placement determined using CFD analysis

PDUs

· Not to be placed within cold aisles

· “A” side PDU and “B” side PDU will be aligned in a regular order to indicate power source.

Remote Power Panels (RPP)

· Not to be placed within cold aisles

· “A” side RPPs aligned in a row

· “B” side RPPs aligned in a row

UPSs

· CDCs and MSCs UPSs shall be in separate dedicated spaces

· For CSCs placed along the perimeter of the data center in accordance with CFD analysis or dedicated space

3 Equipment Rows

Length No more than 14 racks/cabinets per pod

Orientation No rows that terminate along a wall

Cabinet/Rack Placement

· End of row racks shall be used for the HDA to accommodate structured cabling

· Gaps between adjacent cabinets within each rack are closed to eliminate internal paths of bypass and recirculation of air flows

· Equipment cabinets will be used within the rows as specified in corresponding tables below

Clearance

· 3 ft. minimum distance from walls

· 4 ft. minimum distance from air

INFRASTRUCTURE STANDARDS FOR TELECOMMUNICATIONS SPACES| 26

ID Primary Attribute Secondary Attribute Specification conditioning and power distribution equipment (6 ft. recommended)

Figure 5: Hot Aisle/Cold Aisle Approach

Implementation Guidance Determination and maintenance of categorization of enterprise data center facilities shall be conducted by National Data Center Operations & Logistics (NDCOL) in alignment with their mission to report Department data center information to the OMB.

These standards apply to new data center and telecommunications space construction as well as expansion, modification, and operation of existing facilities.

Note: VA shall not design or build “shallow rooms” or similar closets. Shallow rooms are intended for access to vertical chases and riser cables installed there only. Where a distribution TR is necessary in a location (i.e. the third floor of the west wing of a hospital), VA shall construct and operate out of a standardized TR.

INFRASTRUCTURE STANDARDS FOR TELECOMMUNICATIONS SPACES| 27

3.1.2.3 Data Center Planned Redundancy Levels

VA data center facilities shall be designed and operated in accordance with the EIA/TIA-942 Ratings for each data center type, except as detailed specifications are provided in this and other VA data center facilities standards.

These specifications define the minimum system redundancy levels for the physical plants and facilities physical infrastructure systems (“system”) that support…