36C26319R0118-003.pdf

OFFICE OF

INFORMATION

AND TECHNOLOGY

VA Enterprise Facility IT Support Infrastructure Standards Version 2.1

June 11, 2018 | Solution Delivery Data Center Engineering

OFFICE OF INFORMATION AND TECHNOLOGY

VA Enterprise Facility IT Support Infrastructure Standards | i

Table 1: Version Changes

Date Reason for Changes Version Author 2007 Original 0.0 Michael Julian, RCDD 2009 Revision 0.1 Michael Julian, RCDD 2012 Revision 1.0 Michael Julian, RCDD 12/07/2016 EDICT Revision 1.1 EDICT Team 05/09/2017 Expanded revision including the addition of Administrative and Operations Standards.

Name changed to include IT Support Infrastructure to reflect broader scope.

1.2 EDICT Team

1/31/2018 Ratified by SEDR 2.0 SEG 2/05/2018 SD Change Order

Approval

2.0 CAB

6/11/2018 Minor update to allow for deeper network racks in cooperation with IO’s request.

Expanded allowance for fiber cabinets, cassettes, patch panels, and racks.

Changed NDCOL facility ID numbers to VA Record ID numbers.

Clarified reference to OIT responsibility for the enterprise-wide technical framework and IT architecture services.

2.1 EDICT Team

VA Enterprise Facility IT Support Infrastructure Standards | ii

Table of Contents VA Enterprise Facility IT Support Infrastructure Standards ................................................. i Table of Contents ............................................................................................................................. ii Figures ............................................................................................................................................ iv Tables .............................................................................................................................................. v 1 Introduction

1.1 Authority of This Standard

1.2 Scope

1.3 Purpose

1.4 Objectives

1.5 Theory of Operation

2 Data Center Classification

2.1 Core Data Centers (CDC)

2.2 Mission Support Centers (MSC)

2.3 Campus Support Centers (CSC)

2.4 Network Support Center (NSC)

3 Standards

3.1 Facility IT Support Infrastructure Standard

Structured Cabling Unshielded Twisted Pair (UTP) UTP Patch Panel Standards UTP Patch Cord Fiber Optic Cable Fiber Distribution Cassettes Fiber Patch Cords Fiber Distribution Panel/Cabinet Cable Support Infrastructure Work Area Outlets (WAOs)

Horizontal Distribution Vertical (Backbone) Distribution

Telecommunications Space Design Building Specifications for Common Telecommunications Spaces Data Center Layout Standard Data Center Planned Redundancy Levels Telecommunications Room Layout Entrance Facility Network Equipment Racks Equipment Cabinets

Telecommunications Space Environment Conditioning Data Center Facility Environment Conditioning Standards (CDC, MSC, and CSC) Data Center Facility Environmental Standards – Exceptions Environmental Control Equipment Requirements Airflow Control HVAC Environmental Control Equipment Computer Room Air Conditioners (CRAC) Units Split System Air Conditioners Self-Contained Air Conditioners Close-Coupled Air-Conditioners

Humidity Control Equipment

VA Enterprise Facility IT Support Infrastructure Standards | iii

Monitoring Facility IT Support Power Distribution

Topology for Power Distribution Standard (CDC, CSC, MSC) Branch Power Circuit to Rack/Cabinet Zone PDUs Rack Mounted UPS Vertical Rack PDUs Cable Distribution Metering

3.2 Facility IT Support Administration Standard

Data Center Position Identification Cross-Reference to Other Identification Systems Standardized Data Center Facility Type Identification Standardized Data Center Naming Convention Color Coded Identification

Data Communication Cabling Color Coding Power Distribution Color Coding

ITS Equipment and Component Labeling Label Materials Support Infrastructure Identification IT Equipment Cabinet/Rack Location Identification Transport Media and Interface Identification Power Distribution Identification Data Communications Cabling Labeling IT Equipment Rack/Cabinet Labeling IT Equipment In-Rack/In-Cabinet Power Distribution Labeling Power Distribution to Information Technology Labeling

Physical Label Format

3.3 Facility IT Support Operations Standard

Facilities Physical Support Equipment Sustainment Facilities Physical Support Equipment Maintenance & Repair Data Center IT Support Operations Standard

Telecommunications Rooms (TRs) IT Equipment Management of Telecommunications Cabling Installation Guidance Cable Bundling Guidance UTP, Fiber, and Power Cabling Guidance

Data Center Facility Management Standard Computer Room Operation Specifications Recommended Tools & Bench Stock for Data Centers

Data Center Facility Engineering Standard Appendix A: Definitions Appendix B: References Appendix C: Baseline and Configuration Management Appendix E: Acronyms Appendix F: Contributors

VA Enterprise Facility IT Support Infrastructure Standards | iv

Figures Figure 1: Method A Polarity of Horizontal Pre-terminated MPO-based Fiber Optic Cabling

Figure 2: Polarity of Fiber Optic Patch Cable

Figure 3: Typical Standard-Density WAO

Figure 4: Typical High-Density WAO

Figure 5: Hot Aisle/Cold Aisle Approach

Figure 6: Generic Redundancy Level Standards for Physical Plants, Facilities, and Infrastructure Systems

Figure 7: 2011 ASHRAE Environmental Classes for Data Center Applications

Figure 8: Blanking Panel Application

Figure 9: Brushed Floor Grommet

Figure 10: Air Dam Foam

Figure 11: Topology with Site UPS Backup (CDC, CSC, MSC)

Figure 12: TR Topology with In-Rack UPS Backup (NSC)

Figure 13: Power Utilization Effectiveness (PUE)

Figure 14: Typical Position Identification Examples

Figure 15: Example of Best-Practice Differentiated Power Cord Coloration Implementation

Figure 16: Rack Unit (RU) Identification Example

Figure 17: Example of Communications Cable Labeling

Figure 18: Detailed Example of Communications Cable Labeling

Figure 19: Power Whip Label Example

Figure 20: Equipment-Specific Cable Management Requirements

Figure 21: Acceptable Cable Management Results

Figure 22: Unacceptable Cable Management Results

Figure 23: Cable Labels in Bundles

VA Enterprise Facility IT Support Infrastructure Standards | v

Tables Table 1: Version Changes ................................................................................................................. i

Table 2: Data Center Classification Standard

Table 3: UTP Standard

Table 4: UTP Patch Panel Standards

Table 5: UTP Patch Cord Standards

Table 6: Fiber Optic Cable Standards

Table 7: Fiber Distribution Cassettes

Table 8: Fiber Patch Cord Standards

Table 9: Fiber Distribution Panel/Cabinet Standards

Table 10: Cable Support Infrastructure Standards

Table 11: Building Specifications

Table 12: Data Center Floor Layout Standard

Table 13: Data Center Planned Redundancy Levels

Table 14: Core Data Center (CDC) and Campus Support Center (CSC) Planned Redundancy Levels

Table 15: Mission Support Center (MSC) Planned Redundancy Levels

Table 16: Network Support Center (NSC) Planned Redundancy Levels

Table 17: Network Equipment Racks

Table 18: IT Equipment Cabinet Standards

Table 19: ASHRAE Class B Environmental Requirements

Table 20: Data Center Facility Environment Conditioning Standards (CDC, MSC, and CSC)

Table 21: CDC Facility Environmental Requirements (Exceptions)

Table 22: MSC Facility Environmental Requirements (Exceptions)

Table 23: CSC Facility Environmental Requirements (Exceptions)

Table 24: NSC Facility Environmental Requirements (Exceptions)

Table 25: Environmental Control Equipment

VA Enterprise Facility IT Support Infrastructure Standards | vi

Table 26: Monitored Conditions

Table 27: Branch Power Circuit to Rack/Cabinet Standard

Table 28: Zone PDU Standard

Table 29: Rack Mounted UPS Standard

Table 30: Vertical Rack PDU Standard

Table 31: Cable Distribution Standards

Table 32: Metered Energy Consumption Data

Table 33: Energy Consumption Performance Metrics

Table 34: Data center Position Identification Conventions

Table 35: Standard FACILITYTYPE Naming Conventions

Table 36: Color Identification Conventions

Table 37: Support Infrastructure Identification Convention

Table 38: NSC Tool List

Table 39: CSC Tool List

Table 40: CDC Tool List

Table 41: Bench Stock

Table 42: Contributors

VA ENTERPRISE FACILITY IT SUPPORT INFRASTRUCTURE STANDARDS| 1

1 Introduction

1.1 Authority of This Standard

The Office of Information and Technology (OIT) VA Enterprise Facility IT Support Infrastructure Standard defines the technical requirements necessary to maintain optimum reliability and efficiency within Department of Veterans Affairs (VA) facilities and computing centers. Primary considerations for the standards are to balance economy, reliability, capacity, sustainability, and redundancy against supported system availability requirements and available resources.

The purpose of this document is to serve as the master reference document for planning and design of facility IT support infrastructure throughout the Department of Veterans Affairs (VA) facilities. IT Support Infrastructure is defined as all passive telecommunications and information technology equipment and supporting rooms and spaces including information transport systems, power distribution, environmental conditioning, monitoring and metering equipment, and grounding required to support active IT and telecommunications equipment in data centers/computer rooms, telecommunication rooms, entrance facilities and the necessary pathways, risers and conduits interconnecting them. Specifications and design criteria in this manual are expected to be followed in the planning of VA facilities. This standard establishes the common architecture and baseline standards for physical design and engineering for the IT support infrastructure throughout the VA enterprise. The topologies and specifications documented in this standard are to be implemented without modification, except when required by outside Authority Having Jurisdiction (AHJ), in which case, those requirements shall be communicated to the Enterprise Data Center Infrastructure Collaboration Team (EDICT) for consideration of inclusion in a future version of the standard. The authority of this standard derives from approved technical standards and enterprise baselines. Office of Information and Technology (OIT) is responsible for the enterprise-wide technical framework and IT architecture services for OIT, VHA, VBA, and NCA systems and projects. OIT determines technical standards, also known as configuration baselines, for infrastructure technologies deployed promoting one technology vision across VA, which supports system optimization, integration, and interoperability throughout the enterprise. OIT owns the enterprise configuration baselines for the technologies they manage and baselines for infrastructure technologies and architecture for the projects they support.

Approved technical standards and enterprise baselines are communicated through ITOPS bulletins and published to the Security Management and Analytics (SMA) portal under the Baseline and Configuration Management Library.

Requests for variances or changes to this standard are to be submitted to the VA IT EDICT distribution list for consideration. If approved by the committee then the standard will be updated, socialized through the Systems Engineering Design Review (SEDR) process, and submitted to the Enterprise Change Management Datacenter Change Advisory Board for ratification.

VA ENTERPRISE FACILITY IT SUPPORT INFRASTRUCTURE STANDARDS| 2

1.2 Scope

IT Support Infrastructure is defined as all passive telecommunications and information technology equipment and supporting rooms and spaces including Information Transport Systems (ITS), power distribution, environmental conditioning, monitoring and metering equipment, and grounding required to support active IT and telecommunications equipment in facility data centers/computer rooms, telecommunication rooms, entrance facilities and the necessary pathways, risers and conduits interconnecting them. The scope of this document includes design, construction, sustainment, maintenance, repair, upgrade, modernization, administration, and operations of the IT support infrastructure.

IT Support infrastructure that doesn’t conform to this standard is considered to have a one-time initial variance. However, non-conforming infrastructure will be brought into compliance during routine tech refresh, lifecycle replacement, upgrades, or new installations. Facility managers shall also implement incremental changes to bring telecommunications spaces up to VA sanctioned standards to gain associated efficiencies and cost savings without waiting for required tech refresh periods.

Area managers are responsible for ensuring that these standards are made available and widely distributed to all contracting officers, IT personnel, and engineering staff as a reference to be used whenever construction or IT project requirements are put together for any VA space. The Area Manager will work with all responsible contracting, IT, and engineering staff to ensure these standards are adhered to moving forward and that any renovation of an existing space or tech refresh of passive IT infrastructure meets these requirements.

All related projects entering or in the design and implementation phases shall contact EDICT at VAEDICT@va.gov for assistance in ITS, data center, and telecommunications space design and standard compliance.

1.3 Purpose

A standard is a set of rules or requirements that are determined by a consensus opinion of subject matter experts and prescribe criteria for a product, process, test, or procedure. The general benefits of a standard are quality, interchangeability of parts or systems, consistency, and reduction of lifecycle costs. Information Technology (IT) standards are based on business needs provided through or supported by IT Services. IT Services are designed to support business processes and are constructed from software, hardware and infrastructure components. Establishing and enforcing standards for the selection and configuration of these supporting components improves the maintainability, reliability, and availability of IT Services within projected economic constraints in alignment with business needs.

The Infrastructure and Administration sections of this standard define the topologies and specifications for facilities and systems in the VA enterprise, and the Operations section defines the how those facilities and systems shall be operated. The Administration section lists the acceptable and recommended specifications for cable plant and data center naming conventions.

mailto:VAEDICT@va.gov mailto:VAEDICT@va.gov

VA ENTERPRISE FACILITY IT SUPPORT INFRASTRUCTURE STANDARDS| 3

The IT Support Infrastructure standard section describes VA’s facility IT support infrastructure and contain sections necessary to support their standardization. Sections include standard specifications for subject components, decisions supporting the standard specifications, guidelines or recommendations for implementing the standard specifications, and supplemental factors to consider when evaluating subject components. Additionally, it provides guidance on procuring components that meet the standard specifications, guidance on integrating them with existing components, and explanation of how the subject components fit into the VA Architecture.

The Facility IT Support Administration standard section lists the acceptable and recommended specifications for Inside Plant naming conventions, including racks, equipment enclosures, fiber and Unshielded Twisted Pair (UTP) transport media, as well as active and passive elements concerning power distribution. Sections include standard specifications for components, decisions supporting the standard specifications, guidelines or recommendations for implementing the standard specifications, and supplemental factors to consider when evaluating subject components. Other supplementary documents will provide guidance on procuring components that meet the standard specifications, guidance on integrating them with existing components, and explanation of how the components fit into the VA architecture.

The Operations standard describes VA’s facility operations and contains sections necessary to support the standardization of those operations. Sections include standard specifications for operations and maintenance work and work results, decisions supporting the standard specifications, guidelines or recommendations for implementing the standard specifications, and supplemental factors to consider when evaluating subject components. Additionally, it provides guidance on procuring services that meet the standard specifications, guidance on integrating them with existing services and organizational responsibilities, and explanation of how the subject components fit into the VA architecture.

1.4 Objectives

This standard provides recommended specifications across all VA facilities to support:

• Solution evaluation

• Requirement evaluation

• Solution design

• Solution procurement and bid evaluation

• Evaluation of architectural specifications

• Standardization of passive Inside Plant infrastructure

• Standardization of design criteria for telecommunications spaces, facilities physical infrastructure systems, and operation and maintenance of systems supporting VA services

• Standardization of naming and identification conventions for VA data centers

• Standardization of Inside Plant and rack power infrastructure naming conventions

• Standardization of labeling and identification conventions for physical plant systems and components

• Standardization of operations

VA ENTERPRISE FACILITY IT SUPPORT INFRASTRUCTURE STANDARDS| 4

• Operational solution procurement and bid evaluation

• Evaluation of maintenance, repair, and replacement specifications

• Standardization of lifecycle sustainment activities across all VA data centers

• Standardization of the operational, organizational, and aesthetic results of Sustainment, Maintenance, and Repair (SM&R) activities

1.5 Theory of Operation

These standards are applicable to the planning, designing, and use of telecommunications spaces across all Department components within VA. This includes physical spaces used for operation of IT equipment and systems (servers, storage, network switching and distribution, et cetera), telecommunications equipment and systems (local and wide area network, telephone, cable television, physical security IT systems, et cetera), and the physical infrastructure equipment (power conditioning and distribution, heat rejection and air conditioning, et cetera) necessary to operate the supported IT equipment and systems at the availability and sustainability levels described. Telecommunications spaces include telecommunications rooms, entrance facilities, data centers, horizontal pathways, vertical pathways, etc.

VA ENTERPRISE FACILITY IT SUPPORT INFRASTRUCTURE STANDARDS| 5

2 Data Center Classification VA has four segregated data center types to support the enterprise distributed computing environment.

• Core Data Centers (CDC) provide enterprise-level support services for VA-wide and cross-component systems and applications.

• Mission Support Centers (MSC) are highly specialized data centers that provide unique, focused services that are best provided from smaller, mission-focused facilities that cannot or should not be supported by CDCs or CSCs due to MSCs’ association with mission specific infrastructure or equipment.

• Campus Support Centers (CSC) operate local data centers to support VA tenants located on a VA campus and within geographically supported areas. CSCs provide services that cannot be provided (technically or economically) from a CDC. Only one CSC is authorized per VA campus.

• Network Support Centers (NSC) provide lower-level application and IT services distribution across the geographic breadth of the VA’s computing enterprise, enabling special functions and distributed users to satisfy the Agency Department mission.

2.1 Core Data Centers (CDC)

CDCs are facilities that provide shared and distributed enterprise services to supported sites.

CDCs host cloud services (IaaS, PaaS, SaaS) to provide complete end-to-end IT services (including but not limited to application, backup, and storage) to the supported VA sites.

The CDC architecture is considered mission-essential and is designed, operated, and maintained to provide availability consistent with current VA national data center standards for supported IT applications. CDCs are the target architecture component for IT services that span multiple Department components and user groups (services and applications used by many or all users, such as messaging, productivity, and collaboration).

CDCs may be VA-owned, leased, or government or commercial outsourced (external cloud) data centers. CDCs meet (or are planned to meet) all published CDC physical infrastructure requirements to ensure enterprise reliability, redundancy, High Availability (HA), and Disaster Recovery (DR) requirements.

CDCs may also provide services designated to be provided by NSC if no local NSC to provide those services is available. CDCs may also provide MSC-type application support.

2.2 Mission Support Centers (MSC)

MSCs provide element-specific and single-instance special enterprise services that may be inappropriate for consolidation to CDC facilities. MSCs are the target architecture for specialized-function systems and applications, particularly those that support a specific but limited and distributed user group (services allowing the Department to function but with a

VA ENTERPRISE FACILITY IT SUPPORT INFRASTRUCTURE STANDARDS| 6

limited user base such as internal financial management, or applications requiring centralized processing for a small number of user facilities, such as consolidated pharmacies).

MSC physical requirements supporting the computing environment may be more lenient than those for CDCs. Specific requirements will vary based on the supported functions, criticality of the mission, and similar criteria. For example, MSCs designated as enterprise Test/Development (Test/Dev) environments will not require the same level of high-availability physical and network redundancy as a mission-critical production environment that directly supports patient care.

MSCs may also provide services designated to be provided by an NSC if no local NSC to provide those services is available.

2.3 Campus Support Centers (CSC)

CSCs provide geographically-specific, operational IT services in support of campus services to Veterans and VA employees that cannot be effectively consolidated or provided over cloud architectures to the campus. CSCs may also provide services designated to be provided by an NSC if no local NSC to provide those services is available.

IT services at CSCs are provided by multiple organizations, including Clinical Engineering (CE) a.k.a. biomed, health informatics, Veterans Affairs Medical Centers (VAMC), Veterans Benefits Administration (VBA) Regional Offices, Office of Information Field Office (OIFO), and Research & Development (R&D), etc.

Future data center architecture models for CSCs are intended to provide for collocation of IT equipment and systems from all provider organizations into the minimum number of operational spaces on a campus; that is, consolidated data centers providing environmental and area network support appropriate to all IT equipment needed by tenant organizations on a particular campus to perform their missions.

2.4 Network Support Center (NSC)

NSCs provide local IT services that cannot be effectively consolidated or provided over a network-supported distributed architecture to the local operational site. Local means to the immediate facility and to facilities within the local commuting/networking area for the purposes of this architecture; NSCs may provide application support for all VA facilities in a metropolitan area, for example.

Where application architecture and network requirements require a closer data center point of presence to enable end users effective, efficient system access, an NSC is the appropriate target environment.

VA ENTERPRISE FACILITY IT SUPPORT INFRASTRUCTURE STANDARDS| 7

Table 2: Data Center Classification Standard

ID Primary Attribute Secondary Attribute Specification

1 Core Data Center

(CDC)

Description

CDCs are centralized data centers that provide enterprise-level services to geographically-distributed VA organizations to support business functions. CDCs maintain complete High Availability (HA) and Disaster Recovery (DR)

Planned ANSI/TIA 942 Rating 3

Examples Information Technology Center (ITC) Regional Data Center (RDC)

2 Mission Support Center (MSC)

Description MSCs are stand-alone data centers that provide specific enterprise IT functionality to VA organizations and business functions

Planned ANSI/TIA 942 Rating 2 or 3

Examples

• Test/Dev data centers at Rating 2

• Financial Services Center (FSC)

• Centralized Mail Outpatient Pharmacy

(CMOP)

• Consolidated Patient Account Center

(CPAC)

• Health Administration Center (HAC)

3 Campus Support Center (CSC)

Description

CSCs are stand-alone data centers that provide specific IT functionality to a VA campus supporting all VA tenants on that campus and in the geographically supported area

Planned ANSI/TIA 942 Rating 3

Examples VA Medical Center (VAMC)

4 Network Support Center (NSC)

Description NSCs are local data centers that provide local IT application and network support to local operating locations

Planned ANSI/TIA 942 Rating 1

Examples

• Enterprise Test/Development data center

• Telecommunications Room (TR)

• Other small server rooms (not “shallow rooms”)

VA ENTERPRISE FACILITY IT SUPPORT INFRASTRUCTURE STANDARDS| 8

Evaluation Factors

• Data center facility appropriately and centrally categorized by function and usage.

• Capability gaps between existing and planned EIA/TIA 942 Rating requirements identified.

VA ENTERPRISE FACILITY IT SUPPORT INFRASTRUCTURE STANDARDS| 9

3 Standards

3.1 Facility IT Support Infrastructure Standard

Structured Cabling This standard supports structured cabling as a vendor-neutral, extensible approach to information transport system design in VA facilities, not limited to data center applications. The specifications for horizontal cabling for LAN applications and vertical cabling for intra and interbuilding backbone are laser-enhanced fiber optics and high-bandwidth Unshielded Twisted Pair (UTP). The minimum bandwidth requirement for fiber is 40 Gbe and 10 Gbe on UTP. All local and campus area networks shall be installed in a structured topology with patch panels serving transition points. Individual “home run” cables are not permitted in this standard for LAN applications, including data centers.

The Horizontal and Main Distribution Areas should be located adjacent to switching components within the rack to minimize patch cord length and cable management requirements. This approach is consistent with ITS industry standards, and it supports the SD LAN and data center network design baselines.

Work-area outlets and Equipment Distributions shall support the minimum performance and capacity requirements of this standard.

This standard has been harmonized with OIT LAN Baselines.

Unshielded Twisted Pair (UTP)

This standard specifies pre-terminated Category 6A UTP for horizontal distribution. This performance category is applicable to field-terminated or factory pre-terminated applications.

The preferred termination method is factory pre-terminated cabling for UTP with each cable terminating in an individual connector. While pre-terminated cables require that distances between termination points are known this approach results in the rapid installation of horizontal cabling without risks associated with field termination including wire fragments, impaired cabling performance due to poor installation techniques, faulty terminations, etc.

Spiral wrapping pre-terminated bundles is an effective way to maintain the bundle of six cables without impacting ease of installation.

Factory pre-termination is the preferred termination method but field-termination is acceptable for work-area distribution applications and where pre-termination is impractical.

Category 6A cabling is specified in standard ANSI/TIA-1179 Healthcare Facility Telecommunications Infrastructure for new installations. Where Category 6A cannot be justified on technical requirements, the minimum acceptable UTP performance category is Category 6.

VA ENTERPRISE FACILITY IT SUPPORT INFRASTRUCTURE STANDARDS| 10

Table 3: UTP Standard

ID Primary Attribute Secondary Attribute Specification

Performance Characteristics (Horizontal and First Level Backbone)

Performance Category Category 6A (10 GbE) Category 6 (grandfathered and exceptions only)

Performance Specifications Meets or exceeds TIA-EIA-568-C.2-10, TSB-155

Wire Size 26 AWG Maximum

Smoke Rating

• Riser cable for vertical runs through floors

• Plenum rated for plenum spaces

• Not rated for other applications

• Or as per AHJ requirement

Jacket Color Blue (to match 606C HC patching field designation)

Termination Method

• Pre-terminated, 8P8C termination method preferred OR

• Field-terminated to work area outlets only where distances cannot be precisely calculated

Media Connector Pre-terminated with split 8P8C preferred

Bundling Multiple cable harness

Evaluation Factors

• Performance characteristics

• Termination (factory or field)

• Power over Ethernet (PoE)

• Compatibility with specified patch panels

• Smoke rating

Implementation Guidance Horizontal cabling, including fiber and UTP, shall be installed in a structured topology with patch panels serving as at all vertical and horizontal transition points including Equipment Distributors (EDs), Horizontal Distributors (HDs), and throughout the Main Distribution Area (MDA) for all backbone levels. The patch panels at the HC and MDA shall be located adjacent to switching components to minimize patch cord length and cable management requirements.

Where installed, the ED patch panels will be installed in the top five (5) RUs of each enclosure or within small (five (5) RUs or less) racks mounted on cable tray above the associated equipment cabinet.

See Section 3.3 on cable routing and implementation.

VA ENTERPRISE FACILITY IT SUPPORT INFRASTRUCTURE STANDARDS| 11

Pre-termination requires that distances between termination points are known; however, it assures maximum performance and rapid installation. Only use field-terminations to work area outlets as required where distances can’t be precisely calculated.

UTP Patch Panel Standards

This standard anticipates the bandwidth demands of virtualization by specifying Category 6A UTP in production and test facilities. This standard can be applied to field-terminated or factory pre-terminated applications. The preferred termination method is factory pre-terminated cabling for UTP. Category 6A cabling is specified in standard ANSI/TIA-1179 Healthcare Facility Telecommunications Infrastructure for new installations.

Modularity of structured cabling components in the telecommunications rooms, entrance facilities, and data centers improves capacity planning and move/add/change outcomes. This standard specifies a one (1) rack-unit high, 24 or 48 position patch panel that is performance tuned to the UTP. Angled panels eliminate the need for horizontal cable management in the rack and are the typical configuration. No horizontal cable management is required when specifying angled panels.

Table 4: UTP Patch Panel Standards

ID Primary Attribute Secondary Attribute Specification

1 Performance Characteristics

Performance Category Category 6A (10 Gigabit Ethernet (GbE)) Category 6 (variance)

Position Count 24/48

Form Factor

• One (1) RU

• MDA- angled

• HDA- angled (end of row or TR)

• ED

Flat in cabinet or angled overhead

Color Coding Black

Evaluation Factors

• Performance category

• Form factor

• Compatibility with pre-terminated cable interfaces

Implementation Guidance See Operational Standards section on cable routing and implementation.

Horizontal cabling, including fiber and UTP, shall be installed in a structured topology with patch panels serving as at all vertical and horizontal transition points including Equipment Distributors (EDs), Horizontal Distributors (HDs), and throughout the Main Distribution Area

VA ENTERPRISE FACILITY IT SUPPORT INFRASTRUCTURE STANDARDS| 12

(MDA) for all backbone levels. The HC shall be located adjacent to switching components to minimize patch cord length and cable management requirements.

Where installed, the ED patch panels will be installed in the top five (5) RUs of each enclosure or within small (five (5) RUs or less) racks mounted on cable tray above the associated equipment cabinet.

UTP Patch Cord

This standard specifies center-tuned Category 6a or Category 6 patch cords. This standard does not specify color-coding for patch cable jackets. Based on the observations of hundreds of VA facilities, color-coded patch cables provide little or no utility and create inventory management issues. Effective color coding has been observed for in-rack, dual-homed server/appliance applications using a simple white/black scheme. Patch cables shall be factory pre-terminated and shall match the category of the associated patch panel and connected UTP.

Table 5: UTP Patch Cord Standards

ID Primary Attribute Secondary Attribute Specification

1 Performance Characteristics

Performance Category

• Category 6A (Cat 6 variance)

• Stranded

• 26-gauge

Performance Specifications Center tuned to horizontal media

Jacket Color Not specified

Termination Method Factory pre-terminated

Evaluation Factors

• Performance characteristics

• Center-tuned to installed patch panels

Implementation Guidance See Operational Standards section on cable routing and implementation.

Use most appropriate length to meet cable management standards.

Inside Plant Design best practices dictate that component-level compliance to a performance standard must be maintained across the entire horizontal link. The quality of patch cords and adherence to performance standards is critical to the reliable operation of a high-speed LAN.

Fiber Optic Cable

This standard specifies laser-enhanced 50/125 OM4-rated multimode fiber-optic cable.

Pre-termination requires that distances between termination points are known; however, it assures maximum performance and rapid installation. In backbone and data center

VA ENTERPRISE FACILITY IT SUPPORT INFRASTRUCTURE STANDARDS| 13

applications, 12 or 24-strand loose tube cable assemblies using Multi-fiber Push On (MPO) connectors is specified.

If fiber media is required for horizontal distribution to work area outlets, a structured approach is required. A stated need for fiber to work area outlets (WAO) should be scrutinized by the planner and justified by sound technical requirements.

Table 6: Fiber Optic Cable Standards

ID Primary Attribute Secondary Attribute Specification

1 Fiber (Horizontal and First Level Backbone)

Performance Category OM4 Laser Enhanced to 40 GbE

Performance Specifications Laser Optimized 50/125 µm fibers with effective modal bandwidth of at least 4,700 MHz∙km at 850 nm

Mode Multimode

Smoke Rating

• Riser cable for vertical runs through floors

• Plenum rated for plenum spaces

• Not rated for other applications

• Or as per AHJ requirement

Jacket Color Aqua

Termination Method Factory pre-terminated

Media Connector Pre-terminated with MPO

Strand Count 12 or 24 per assembly

Bundling Loose Tube

Polarity Straight (or Type A-Key up one end & key down on the other)

Evaluation Factors

• Warranty

• OM4 performance characteristics for 40 GbE

• Smoke rating requirements

• Technical requirements for inclusion in horizontal distribution to the WAO.

Implementation Guidance See Section 3.3 on cable routing and implementation

Pre-termination requires that distances between termination points are known; however, it assures maximum performance and rapid installation.

This standard specifies the polarity of horizontal pre-terminated MPO-based fiber optic cabling as Type A or “straight-through.” This will ensure that position one (1) will correspond to position one on both ends of the cable. If a polarity flip for the channel is necessary, use a Type

VA ENTERPRISE FACILITY IT SUPPORT INFRASTRUCTURE STANDARDS| 14

A/A patch cable on one end and a Type A/B patch cable on the other end. In some cases, an A/B cord can be converted to an A/A by switching the connector position on one end of the patch cord.

Fiber Distribution Cassettes

This standard specifies the transition component of the structured cabling system for fiber optic cabling in telecommunication spaces. It is the fiber equivalent to UTP patch panels. This standard specifies 12/24-strand, OM4-rated cassettes with duplex LC interfaces or MPO interfaces. The cassette will be equipped with dual, 12-strand or single 24-strand MPO interfaces as the backbone or horizontal interface.

Figure 1: Method A Polarity of Horizontal Pre-terminated MPO-based Fiber Optic Cabling

VA ENTERPRISE FACILITY IT SUPPORT INFRASTRUCTURE STANDARDS| 15

Table 7: Fiber Distribution Cassettes

ID Primary Attribute Secondary Attribute Specification

1 Performance Characteristics

Form Factor One (1) RU

Capacity 3-12 cassettes

User Interfaces LC connectors (6/12 duplex per cassette) MPO connectors (8 fibers)

Backbone Interfaces 12-strand or 24-strand MPO (50 micron)

Type OM4 Laser Enhanced 40 GbE 50/125 Multimode

Evaluation Factors

• Performance characteristics

• Form factor

• Interfaces

Implementation Guidance This standard specifies Optical Multimode-4 (OM4)-rated (40 GbE) 850 nm laser-enhanced multi-cable fiber cable assemblies pre-terminated with MPO connectors. One cassette will be used to provide LC interfaces at both ends of the horizontal or backbone link and will support one or two 12-strand cable assemblies or one 24-strand cable assembly for a total capacity of 6 or 12 duplex LC interfaces. Cassettes can also be used to breakout 24-strand MPO backbone trunks into 3 MPO connections of 8 fiber strands, each providing 40GbE parallel transmission.

Fiber Patch Cords

This standard specifies high-bandwidth, factory pre-terminated fiber patch cords. The standard media interface for fiber patch cords is duplex LC. Patch cord polarity requirements can be accommodated by specifying a polarity-reversible duplex interface. When polarity requirements are not known, the polarity-reversible duplex LC interface should be specified.

Table 8: Fiber Patch Cord Standards

ID Primary Attribute Secondary Attribute Specification

1 Performance Characteristics

Performance Category 40 Gigabit OM4

Performance Specifications Laser enhanced, 50/125 multimode fiber, Duplex LC

Jacket Color Aqua

Termination Method Factory pre-terminated

Polarity

• Type A to B (Crossover)– ED

• Type A to A (Straight) – HDA and MDA

• Polarity reversibility

VA ENTERPRISE FACILITY IT SUPPORT INFRASTRUCTURE STANDARDS| 16

• Performance specifications

Implementation Guidance

Inside Plant design best practices dictate that component-level compliance to a performance standard must be maintained across the entire horizontal link. The quality of patch cords and adherence to performance standards is critical to the reliable operation of a high-speed LAN or

SAN.

The polarity of the fiber optic channel may need to be flipped depending on the application.

This will ensure that the equipment sending or transmitting will be connected to the receiving port of the equipment at the other end. If typical A/B fiber patch cords are used on both sides of the channel, continuity may be lost. As shown in Figure 5, this is corrected by either switching the connector position of both strands at one end of the patch cord or by using a Type A/A patch cable on only one end of the channel.

Pre-termination requires that distances between termination points are known; however, it assures maximum performance and rapid installation.

See Section 3.3 on cable routing and implementation.

Figure 2: Polarity of Fiber Optic Patch Cable

VA ENTERPRISE FACILITY IT SUPPORT INFRASTRUCTURE STANDARDS| 17

Fiber Distribution Panel/Cabinet

This standard specifies a one (1) RU panel for mounting fiber cassettes. The smallest form-factor has been selected for application flexibility.

Table 9: Fiber Distribution Panel/Cabinet Standards

ID Primary Attribute Secondary Attribute Specification

1 Performance Characteristics

Form Factor One (1) RU

Cassette Capacity 3-12 cassettes

Type

• Cabinet (network rack installation or lock requirements)

• Panel (server cabinet with vertical exhaust, no security requirement for locks)

Evaluation Factors

• Form factor

• Cassette capacity

• Locking requirements

Implementation Guidance ED Fiber Distribution Panels will be installed in RU 45 of each vertical exhaust duct cabinet where required. Fiber distribution panels shall be used in all other type implementations.

Cable Support Infrastructure

This standard creates uniform high-capacity horizontal and vertical cable management for Category 6A patch cords.

Table 10: Cable Support Infrastructure Standards

ID Primary Attribute Secondary Attribute Specification

1 Horizontal Cable Management Panels

Form Factor

• One (1) RU (up to 50 Cat 6a patch cords)

• Two (2) RU (up to 100 Cat 6a patch cords)

Cover Double-hinged

Finger Spacing To meet requirements; typically 6-port spacing (five fingers)

Dimensions 6 in. to 16 in.

2 Vertical Cable Management Panels Door Double-hinged

• RU Finger spacing

VA ENTERPRISE FACILITY IT SUPPORT INFRASTRUCTURE STANDARDS| 18

• Double-hinged cover capacity

Implementation Guidance Vertical cable management will be utilized for the horizontal distribution area and the main distribution area. Horizontal cable management within the rack will be utilized as necessary to encourage defined and traceable routing of individual patch cords from switches to the vertical cable managers.

Work Area Outlets (WAOs)

Work Area Outlets are user interfaces to the cabling system within a workspace, floor, or building. This standard specifies the typical quantity and type of faceplate, wall box, conduit, and outlets for typical administrative work areas. However, this guidance does not preclude variations based on atypical capacity requirements nor does it supersede guidance or requirements set forth by Office of Construction and Facilities Management (CFM) concerning Work Area Outlet specifications for specialized spaces.

Typical in-wall workboxes shall be a double-gang telecommunications-rated work box with rigid conduit extending to the horizontal pathway.

Floor boxes shall be dual-gang or “poke-thru” type. Cable that extends through a slab penetration shall terminate in the nearest practicable TR on the same serving floor.

The typical (standard density) WAO will consist of two RJ45 interfaces. This provides connectivity for one (1) IP telephone and one (1) workstation. Some IP phone base units have a data port. This allows the workstation to interface with the network via the phone base unit to leave one (1) RJ45 at the faceplate available for future use.

WAO media interfaces shall be tuned to the horizontal media, specified to the same performance rating, and installed according to best practices.

No “phone outlets” are specified in this standard. Legacy equipment, such as fax machines and analog phones, will require a variance from this standard via the variance request process.

Single-outlet posted faceplates shall be configured with one (1) RJ45 interface.

https://www.cfm.va.gov/TIL/

VA ENTERPRISE FACILITY IT SUPPORT INFRASTRUCTURE STANDARDS| 19

Figure 3: Typical Standard-Density WAO

The high-density WAO configuration consists of four (4) RJ45 interfaces. This configuration is typically used for desktop support and other technical staff or in cases where multiple devices need network access from the same WAO.

VA ENTERPRISE FACILITY IT SUPPORT INFRASTRUCTURE STANDARDS| 20

Figure 4: Typical High-Density WAO

• Serving zone capacity requirements

• Special safety or medical device requirements

Horizontal Distribution

Horizontal distribution is defined as structured cabling that connects the Work Area Outlet or the Equipment Distributor to a Horizontal Distributor (HD). The HD will be located in the end of row Horizontal Distribution Area (HDA) or in the Telecommunications Room (TR) depending on the application.

Horizontal media shall be specified per section 3.1.1.1 Unshielded Twisted Pair. If the application, such as a data center, requires 40 GbE fiber, specify the media per section 3.1.1.4 Fiber Optic Cable.

Vertical (Backbone) Distribution

Vertical Distribution is defined as structured cabling that connects the Horizontal Distributor to the Main Distributor (MD). The Main Distribution Area (MDA) will be located in a centralized location within the building, data center, or LAN serving area. Typically, this will be a Campus Support Center or a data center.

VA ENTERPRISE FACILITY IT SUPPORT INFRASTRUCTURE STANDARDS| 21

The MDA shall be a separate and distinct telecommunications space within the LAN serving area. The MDA shall not be located in a Telecommunications Room.

This standard only recognizes 50/125 Laser-Enhanced Multimode (OM4) as an acceptable backbone media (see 3.1.1.4). UTP shall be utilized only if fiber media cannot serve specialized applications such as legacy phones, fax machines, or security panels. Where copper backbone is required, this standard specifies quantity two (2) riser-rated Category 6, 25-pair cables to be installed between the Telecommunications Room HDA and the Campus Support Center MDA.

This approach will support twelve (12) patch panel positions. One 12-position Category 6 patch panel shall be installed in the HDA and the MDA. Individual 4-pair Category 6 UTP riser-rated cables may be used, but this approach is not recommended and will require technical review by

OIT.

VA ENTERPRISE FACILITY IT SUPPORT INFRASTRUCTURE STANDARDS| 22

Telecommunications Space Design VA telecommunications spaces, including entrance facility, telecommunications rooms, and data center facilities shall be designed and operated in accordance with the EIA/TIA-942 Ratings for each data center classification, except as detailed specifications are provided in this and other VA data center facilities standards.

Building Specifications for Common Telecommunications Spaces

Table 11: Building Specifications

ID Primary Attribute Secondary Attribute Specification

1 Shell

Security Via combination of manual and electronic control or minimum electronic control

Access Route Leads to location outside computer room for access control

Egress Observes life safety code

Door Height (Computer Room)

8 ft. minimum

Door Height (Mechanical Room)

9 ft. minimum

Door Width

• CSC, CDC, MSCs- 6 Ft.

• NSC- 3 ft.

Slab to Floor Above 16 ft. minimum

Raised Floor to Ceiling 12 ft. minimum

Raised Floor in new data center construction (used only if CFD analysis determines raised floor is the best cooling solution)

• 18 in. minimum above slab

• 24 in. fully loaded racks

• 36 in. high density

2 Floor Height

Slab to Floor Above 16 ft. minimum

Raised Floor to Ceiling 12 ft. minimum

Raised Floor in new data center construction (used only if CFD analysis determines raised floor is the best cooling solution)

• 18 in. minimum above slab

• 24 in. fully loaded racks

• 36 in. high density

VA ENTERPRISE FACILITY IT SUPPORT INFRASTRUCTURE STANDARDS| 23

ID Primary Attribute Secondary Attribute Specification

Floor Tile Perforated Tile

• Only positioned at the base of the cold aisle racks

• Positioned so that they can be removed without interference from racks

Dimensions 24 in. by 24 in.

Data Center Layout Standard

Data centers will be designed in accordance with the specifications below, VA standards, and industry best practices referenced in Appendix B.

Computational Fluid Dynamics (CFD) modeling shall be used to determine the most effective and cost-effective layout and containment solution for a given space (including new design/build, capability expansion, and lifecycle replacement). All data center design for new data centers, expansion, or significant modification of existing data centers will be approved through OIT Solution Delivery Data Center and Cloud Engineering (VAITESEDatacenterEngineering2@va.gov).

Airflow direction shall be a prime consideration when designing the floor layout and when selecting and procuring IT equipment. Equipment shall be specified, procured, configured, and installed so that the equipment draws cool air from the front face of the rack and flows from the front of the rack to the rear and the hot exhaust exiting the chassis at the rear face of the rack. Do not create conditions for exhaust to be recirculated back into cold air supply spaces.

Table 12: Data Center Floor Layout Standard

ID Primary Attribute Secondary Attribute Specification

1 Aisles

Orientation

• Front of racks will be oriented toward cold aisle as in Figure 2

• Front of racks will be aligned with each other to create a continuous linear aisle

• If a raised access floor is utilized, front of racks will be flush with edge of floor tiles

Width

• 4 ft. minimum in cold aisles

• 3 ft. minimum in hot aisles (4 ft.

recommended)

Cold Aisle Power cables if run in an access floor will be placed in cable trays in cold aisles mailto:VAITESEDatacenterEngineering2@va.gov

VA ENTERPRISE FACILITY IT SUPPORT INFRASTRUCTURE STANDARDS| 24

Hot Aisle Data cables if run in an access floor will be placed in cable trays in hot aisles

Containment

Hot or cold aisle containment will be determined based on data center layout, Heating, Ventilation, and Air Conditioning (HVAC) configuration, and analysis

2 Support Equipment

Computer Room Air Condition

(CRAC)

• Placed along the perimeter of the data center or in a dedicated mechanical room

• Centered on Hot aisles

• Exact placement determined using CFD analysis

PDUs

• Not to be placed within cold aisles

• “A” side PDU and “B” side PDU will be aligned in a regular order to indicate power source.

Remote Power Panels (RPP)

• Not to be placed within cold aisles

• “A” side RPPs aligned in a row

• “B” side RPPs aligned in a row

UPSs

• CDCs and MSCs UPSs shall be in separate dedicated spaces

• For CSCs placed along the perimeter of the data center in accordance with CFD analysis or dedicated space

3 Equipment Rows

Length No more than 14 racks/cabinets per pod

Orientation…