Exhibit B-6 - Basic Guidance - Telecommunications Room Rack Utilization.pdf

Version Date: 3 March 2023 1

White Paper: Telecommunications Room Rack Utilization

This white paper describes the VA’s approach to active and passive equipment deployment in distributed telecommunications spaces supporting administrative and healthcare spaces. The approaches described have been selected from industry best practices and optimized for the VA’s operating environment.

There are on the order of 10,000 distributed telecommunications spaces in the VA enterprise.

A primary objective of VA’s telecommunications standardization is for each of these spaces to be operationally and effectively identical. A technician from one facility should be able to walk into another facility and understand how the network is organized because they are organized the same from site to site, station to station, without variability introduced from “local standards and procedures.”

Due to the large quantity of distributed telecommunications spaces across the enterprise, variability of operational requirements inevitably makes a single, completely rigid standard impossible. This white paper establishes basic guidance, common conventions, and describes the VA enterprise methodology for organizing and operating passive and active equipment in the distributed telecommunications environment.

The guidance in this white paper is mandatory. Alternative methods of deploying active and passive network equipment are not acceptable.

Applicability

The prescribed approach to distributed telecommunications space equipment and cabling management is required of all users of the telecommunications space, regardless of organization (OIT, facilities, CE/biomed, research, partner, et cetera).

OIT Data Center & Infrastructure Engineering is the proponent for this standards guidance and the Authority Having Jurisdiction (AHJ) for consideration of variances for non-standard installations. See guidance in VA’s Infrastructure Standard for Telecommunications Spaces (ISTS) for additional detail as necessary.

The Applications, Cloud, & Edge Systems (ACES) Data Center & Infrastructure Engineering (DCIE) team has developed this guidance as a supplementary narrative to the VA Infrastructure Standard for Telecommunications Spaces to provide additional detail for field OIT and FMS staff, A/Es, and implementing contractors and VA staff. Please contact the Data Center & Infrastructure Engineering team for additional information as https://www.cfm.va.gov/til/dguide/OIT-InfrastrucStdTelecomSpaces.pdf https://www.cfm.va.gov/til/dguide/OIT-InfrastrucStdTelecommSpaces.pdf mailto:%20VAITESEDatacenterEngineering2@va.gov?subject=RE:%20Single-Rack%20TR%20Design%20&%20Outfitting

Version Date: 3 March 2023 2

Training & Certification

All VA staff, contractors, installers, and other persons that may install, uninstall, connect, disconnect, or otherwise modify the telecommunications environment in the VA distributed telecommunications space environment shall be aware of and follow the guidance in this document.

VA staff shall be provided with online training not less than triennially that familiarizes the trainee with the basic requirements of this guidance. VA staff shall self-certify at the completion of this training that they are familiar with and shall follow the requirements when planning or executing work in the VA’s distributed telecommunications space environment.

Prior to being allowed to perform work in any distributed telecommunications space, contractors and other outside staff shall acknowledge in writing that they are familiar with the requirements; this may be accomplished using the same visitor sign-in sheets that are required for access to the space.

A copy of the standard single-rack and two-rack elevation drawings shown below shall be posted in all VA enterprise Telecommunications Rooms not later than 45 days following the first publication of this guidance. This provides an available reference for staff and contractors working in the TR.

Convergence Drives the Shared Environment

For telecommunications purposes, VA facilities are divided into serving zones. All telecommunications services for the serving zone are provided by the Telecommunications Room (TR) for the zone. Other building services that span multiple zones (e.g., access control, fire alarm, paging, cellular reinforcement, et cetera) are appropriate to be located in the TRs, and are expected to be where technically feasible.

Telecommunications Rooms are a shared environment that:

• Serves as the transition point between the horizontal and backbone cabling systems

• Contains all networking, security, video distribution, paging, and any other system that serves the TR zone

There shall only be a single TR per serving zone. Additional TRs (or similar) are not authorized to support specific functions (e.g., FMS or CE/biomed). Likewise, TRs shall not be physically split or divided to separate organizational functions or for physical security purposes and shall not be designed, implemented, or operated. This follows with the converged physical telecommunications infrastructure networking model where backbone is distributed from the data center to TRs and other distributed telecommunications spaces and horizontal distribution supports end-user requirements in the serving zones of these distributed telecommunications spaces. The TR is the only IT support space in the serving zone that is supplied with backbone

Version Date: 3 March 2023 3 media suitable to support IT and IT-like system functions, and designed with the redundancy and resiliency characteristics to support end-user equipment requirements appropriately.

Network convergence will move stand-alone systems that use IP-based traffic and/or standard telecommunications cabling media onto protected virtual LANs (VLANs) on the VA local area network (LAN).

Data Center & Infrastructure Engineering recognizes that EHRM modernization projects may not completely relocate some existing non-OIT systems into replacement TRs at some stations.

The intent of developing standardized TRs is to allow future projects to migrate these residual systems into the common TR environment, particularly at system upgrade or replacement (when many systems are expected to become more IP-based and require network connectivity).

Conventions

Locations in VA distributed telecommunications spaces are not identified using the 2’x2’ grid system described for data centers in the Infrastructure Standard for Telecommunications Spaces. Identification of equipment in these smaller spaces follows these conventions:

• In distributed telecommunications spaces with more than one row of equipment (e.g., generic design Antenna Entrance Rooms), the row to the left when entering the space from the main entrance door if oriented perpendicular to the door or closest to the main entrance door if oriented parallel to the door is row 1, with additional rows increasing cardinally (row 1, row 2, et cetera).

• Enclosures are numbered from left-to-right when facing the fronts of the enclosures, beginning with Rack 1 and increasing to the total number of racks (rack 1, rack 2, et cetera).

• Vertical cable managers are not numbered.

• Equipment positions within the 19” racks in each enclosure range from rack unit (RU) 1 to RU45, beginning at the bottom of the rack.

Version Date: 3 March 2023 4

• In the TR environment, equipment is numbered by type from the top of the rack moving down. These numbers are not associated with the RU position of the equipment. Thus, if there are three (3) horizontal distribution switches in the rack, they will be numbered switch 1, switch 2, and switch 3 from the top-most to the bottom-most. (This differs from how equipment is named/identified in facility computer rooms. In the data center environment a specific location identification system described in the Administration portion of the Infrastructure Standard for Telecommunications Spaces is used.)

• From both the front and rear of each enclosure, the A side is on the left and the B side is on the right. Note that telecommunications installers must keep in mind that, in the TR environment, this will mean that the A and B side terminations to patching equipment will appear reversed (B/A) from the rear of the racks. (In the data center environment, server cabinet Equipment Distributors are mounted in the rear of the cabinets; the opposite guidance for installers will apply.)

• Each enclosure is provided with A/B redundant power; no enclosures, even ones intended for only passive equipment, are not provided with power as the use of the enclosures for equipment and to support moves, adds, and changes over the lifecycle of the telecommunications space cannot be known in advance.

• At the IT equipment power cord level, power distribution shall not cross enclosure boundaries. Equipment in an enclosure may only be powered from the A/B distribution for that enclosure. (Where used, Zone PDUs are specifically designed to power multiple enclosures’ vertical Rack PDUs from a single common location.)

• Wall-mounted equipment is not numbered. Walls are generically identified as one enters the main entrance door to the space as the left, rear, and right walls, with the front wall being the one containing the main entrance door.

• VA follows a rule-of-thirds system for placing equipment in racks in the distributed telecommunications spaces.

o The top third of each rack (RUs 31-45) is designated for passive telecommunications distribution (patch panels).

o The middle third of each rack (RUs 16-30) is designated for active telecommunications distribution (switches) and other active equipment.

o The bottom third of each rack (RUs 1-15) is designated for power distribution equipment (and in smaller TRs, active FMS/biomed equipment).

o The rule of thirds is a guideline; equipment sections may be expanded within reason into neighboring sections. For example, if a switch needs to be installed but all positions in RUs 16-30 are filled, it would be permissible to install the switch (and its associated horizontal cable mailto:https://www.cfm.va.gov/til/dguide/OIT-InfrastrucStdTelecomSpaces.pdf

Version Date: 3 March 2023 5 management) in the top of the section designated for power distribution equipment (i.e., RUs 12-15) if this does not negatively impact space needs for that equipment.

Rack 1 in each Telecommunications Room is the primary rack designated for VA LAN distribution of the campus structured cabling system.

Install campus fiber backbone termination equipment at the top of Rack 1. Due to the use of angled fiber distribution panels, no horizontal cable management shall be used in this section to maximize density.

Install horizontal distribution UTP patch panels below the fiber termination equipment. Due to the use of angled patch panels, no horizontal cable management shall be used in this section to maximize density.

Distribution switches used in VA distributed telecommunications spaces are flat 1RU devices with a 48 port capacity each. Cable management for these devices requires each switch to be sandwiched with 1RU of dedicated horizontal cable management. Where multiple switches are present, the use of 2RU horizontal cable managers between the switches is an acceptable alternative.

The maximum quantity of UTP patch panels that shall be installed in Rack 1 is ten (10). This provides up to 480 passive cable connections to WAOs. The maximum quantity of UTP patch panels that may be installed in subsequent (overflow) racks is ten (10). This provides an additional maximum of 480 passive cable connections to WAOs.

The (typical) design loading of VA LAN switches in a single network rack is five (5), aligning with the standard design of 1/3 of the rack (15RU). The maximum quantity of switches that may be installed in each rack is six (6). If the demand in the serving zone for active switch ports exceeds the 288 ports available, a new switch stack shall be started in Rack 2. The most constraining of the TEAM discipline factors driving this quantity is architecture – the amount of

Version Date: 3 March 2023 6 space able to be physically allocated in each rack. Note that while 15RU (1/3 of the standard 45RU rack) is available, this allows for up to 18RU (extending into the 1/3 of the rack allocated for power distribution and active FMS/biomed equipment) to be installed, intruding maximally to 3RU into that allocation. This protects the electrical distribution for the rack at maximal switch PoE utilization.

Patch cables between switches and UTP patch panel ports leading to WAOs may cross enclosure boundaries. For example, a patch cable is allowed to be plugged into an active port on a switch in Rack 1 and the passive port of a patch panel in Rack 2.

Rack 2 will be used as an overflow rack for the VA LAN campus structured cabling system in the event that more than 288 active switch ports or more than 480 passive WAOs are required in the serving zone. In this case, install equipment in Rack 2 in the same manner as Rack 1; in most cases this will result in a more sparse installation in Rack 2, when it is needed at all.

When Fabric Interconnects (FIs) and Fabric Extenders (FEs) are present, they are placed in the switching section of Rack 2 (unless rack 2 is not present).

Rack 1 is always designated for VA LAN distribution support. Additional racks in the TR, if not required for VA LAN overflow (more than 288 active or passive ports supported), are intended for use in supporting (a) other stakeholder equipment (e.g., distributed equipment from facilities, biomed, security, and others) and (b) moves, adds, and changes particularly associated with network convergence.

If only a single rack is located in a TR (primarily due to serving zone size/density and occupancy classification), other stakeholder equipment may be placed in the spare active equipment

Version Date: 3 March 2023 7 section positions of the rack (RUs 16-30) below the active switches and associated horizontal cable managers.

More detailed guidance for Entrance Room and Antenna Entrance Room may be developed at a later point.

Special Systems Considerations

Multiple types of IP-based special systems have been identified as being IP-based and requiring horizontal distribution from a centralized switching or control system. Some of these systems may eventually be converged onto the VA LAN, while others (due to function or similar) may not, but will use parallel passive media from the telecommunications space infrastructure (especially TRs) to operate independently.

These types of systems are anticipated to proliferate in the future; the VA structured cabling model needs to be designed and implemented in order to support these systems in a standardized manner to reduce cost and complexity and to maximize sustainability of the telecommunications infrastructure.

When horizontal distribution from the Telecommunications Room is planned and installed, all types of uses of the network must be taken into account. Identify connectivity and potential future connectivity requirements for systems that will have data riding on VA LAN horizontal distribution or on a parallel network’s horizontal distribution, and provide cabling media and WAOs for all of these non-traditional special systems.

Example 1 – Intensive Care Unit (ICU) Telemetry

In an example of how non-OIT systems may need to be supported, a legacy telemetry system for a medical center’s ICU collecting patient vital statistics (heart rate, blood pressure, et cetera) may currently be located in a stand-alone rack in the ICU area (not in a centralized TR).

The system requires multiple UTP horizontal distribution cabling runs from the head-end equipment location to each patient care room and to monitoring locations in the supporting nurse’s station. This system connects to a variety of Internet-of-Things (IoT) devices that are connected to a patient in each room, collecting and logging real-time data that is relayed to and cached in the system’s head-end equipment (typically workstations, not servers). This system allows practitioners to view vital statistics data both in real-time and going back in time a certain amount of time (24 hours, 3 days, or however the system is configured) to provide better patient care.

In a typical legacy system, the system is effectively a stand-alone shadow network. Telemetry systems do not typically permanently store this type of vital statistics data to a patient’s electronic health record (EHR) but operate on the same type of UTP network as other systems that are supported by a TR.

Version Date: 3 March 2023 8

It is therefore vital that detailed planning for the distribution of WAO structured cabling to the ICU unit include all necessary horizontal distribution cabling connections in the ICU – meaning potentially multiple WAOs in different locations in each patient room for IoT data collection and monitoring equipment, and a distributed set of WAOs where monitoring screens allow nursing staff to view data for all of the patients in the unit. This distribution will be significantly more dense than the WAO requirements for an administrative or other type of space, where a 2- WAO Standard Density wall outlet for VA LAN connectivity may be typical.

It is an acceptable practice to distribute UTP horizontal distribution from the top 1/3 of the second or other overflow rack in the TR dedicated to the WAOs for the telemetry system, and/or to specify a particular color scheme for patch panel jack (outlets, i.e., keystones), horizontal distribution cabling, patch cord, and WAO outlet coloring to identify these WAOs as being intended solely for the telemetry function.

The telemetry system would not necessarily need to be connected to the VA LAN in the future;

via patching in the TR it could remain a completely air-gapped closed network, running over the single converged campus structured cabling system (horizontal distribution in the TR to the ICU in its serving zone).

Example 2 – Autonomous Mobile Robot Systems

Robotic delivery systems using autonomous robots have been encountered in multiple VA medical centers. Depending on design, this sort of system may require separate wireless access points or similar for inventory, location, routing, and similar functions associated with a logistics and autonomous delivery system. Where a parallel horizontal distribution network for systems of this type is necessary, it should also follow the guidance in the example above, using the second or overflow rack for dedicated WAO distribution and the active equipment section of the rack for any dedicated hardware.

VA LAN Network Management Considerations

Data Center & Infrastructure Engineering expects that the telecommunications infrastructure modernization projects being conducted under EHRM (Electronic Health Records Modernization) will largely relieve outlier situations where more than six switches (288 active ports) are required to be supported out of any given TR. Because replacement and upsized TRs in the medical center environment will largely be 2-rack spaces due to space allocation constraints, projects are explicitly limited as to the quantity of WAOs that can be supported from any TR.

Actual active:passive port ratios and total supported WAO counts following these projects will be reviewed when the first cable plant and TR modernization projects are constructed, and guidance in this white paper updated as necessary.

No more than six (6) 48-port switches may be deployed in a single rack. Switches beyond the first six must be placed in the active equipment section of the overflow rack.

Version Date: 3 March 2023 9

1. The physical size of the active equipment section is finite; six switches require 18RU of space, extending into the power distribution section.

2. Switch stacks are currently limited to six devices. (Current switch hardware stacking limits based on switch design by the manufacturer is eight (8), which cannot be supported by VA’s physical and electrical limitations.)

3. IT equipment power cords crossing from one enclosure to another remains prohibited as it does not address multiple discipline constraints. The maximum power draw from the typical TR switch, fully loaded and operating at maximum PoE, is estimated at 800W.

Only six switches at this power level can be electrically supported by the power distribution and heat rejection of the TR.

VA LAN switch identification for the vast majority of TRs is trivial; as shown in the sensitivity analysis results below, more than 90% of current TRs require six or fewer switches, even when including ports and switches from shadow networks that are expected to be converged in the future.

TR designs with multiple racks specify that other stakeholders’ shadow network switches be located in the active equipment section of racks from right-to-left, where VA LAN equipment is installed from left-to-right. This is intended to minimize the potential for confusion about which switches are currently connected to the VA network.

VA LAN switch identification for TRs where more than six switches are located may be accomplished by continuing the switch count in the overflow rack (with switches numbered continuously with 7, 8, and so on). The schema has the (limited) potential to cause confusion with remote hands staff if other stakeholders’ shadow network switches are not placed properly.

Telecommunications Room Port Density Sensitivity Analysis

Data Center & Infrastructure Engineering has conducted multiple analyses of the quantities of switches, active (utilized) data ports, and passive cable connections to WAOs in Telecommunications Room serving zones to provide insight into the designs presented in this white paper. At the time of publication, high confidence data from 86 TRs representing 15 separate medical centers are included in this sensitivity analysis.

The first field survey conducted specifically looked at maximally-sized Telecommunications Rooms in order to attempt to identify the most-dense possible installations; this resulted in several outlier spaces with high quantities of switches (greater than the standard 6-switch stack). These very-high-density TRs were actively sought out in order to show that they did in fact exist, and that this type of space would require special consideration. Following surveys looked at randomly-selected Telecommunications Rooms to allow the analysis to represent the typical conditions present across the VA enterprise.

Version Date: 3 March 2023 10

Surveys captured the expected switch and port density following convergence of all systems onto the VA LAN; that is, where existing shadow network equipment (networks operating on VA but not yet migrated to the VA LAN, such as security camera networks, guest WiFi networks, stand-alone biomed networks, et cetera) were present, the active and passive switch port counts for these were captured to present a picture of the ultimate density once all systems are converged onto the VA LAN in the future. This approach is expected to result in a more conservative solution to the Telecommunications Room elevation planning questions addressed in this analysis.

The validity of the high confidence data and results of the analysis are expected to represent the general conditions at medical centers across the VA medical center enterprise based on established probability theory. The Central Limit Theorem (CLT) establishes that, in many situations, when independent random variables are summed up, their properly normalized sum tends toward a normal distribution even if the original variables themselves are not normally distributed. The necessary minimum sample size of 30 to allow the CLT to be assumed correct has been achieved in this analysis, with the sample size continuing to approach that of an independent statistical population.

The survey reviewed 86 Telecommunications Rooms from sites across the enterprise, the average TR requires 4 standard network switches to provide 175 active connections across 377 passive WAOs supported by the TR. The percentage of active ports when compared to passive WAOs is 46.4%.

Histograms show the ranges of these data in TRs surveyed.

Based on this data, 75.95% of TRs across VA should be expected to have a total of five (5) or fewer switches supporting all user requirements (not just VA LAN) today, and to retain that total quantity following network convergence. 87.34% should be expected to have six (6) or more in the maximal first rack configuration in the same circumstances, and 94.94% of all TRs will have eight (8) or fewer. Rationalization of TR zone sizing being conducted during EHRM

1 2 3 4 5 6 7 8 9 10 11 12

-1

-2

-2

-3

Fr eq ue nc y

Bin

EDICT Assessment Qty of Switches Required per TR

Version Date: 3 March 2023 11 modernization activities is anticipated to dramatically accelerate these percentages, as the EHRM-allowable TR modernization (2-racks) does not permit serving zones with more than 240 active ports, per the flow chart in the TR Design Checklist. These factors show the need to plan for typically light overflow into a second rack to support transition of access control, fire alarm, paging, and similar systems from non-IP to IP-based in the future.

Similarly, data show that 85% of TRs have an active:passive port ratio of 50% or less, which supports a planning conclusion for how many switches will be needed to support a planned quantity of passive WAOs in a serving zone. This allows refinement of minimum TR sizing to support a serving zone with high confidence.

Standard Density data faceplates are provided with (2) WAOs; these are the interfaces where the end-user typically connects their workstation and VoIP phone. As the VoIP phones are connected in-line between the WAO and the workstation, the null hypothesis prior to performing this analysis was that approximately 50% of passive WAOs would be connected to a switch (made active). The data bears this out.

Conclusions

• Most VA Telecommunications Room serving zones can be served VA LAN services, currently and following network convergence, from the switches and patch panels in a single network channel rack.

• There is a valid use case for Telecommunications Rooms requiring more than 6 switches, requiring development of a technical solution.

• A 50% planning factor for switch requirements based on the total quantity of passive WAOs installed in a serving zone is reasonably conservative.

References

VA OIT Infrastructure Standard for Telecommunications Spaces:

https://www.cfm.va.gov/til/dguide/OIT-InfrastrucStdTelecommSpaces.pdf

0- 5%

6-

-1 5%

-2

0%

-2 5%

-3

0%

-3 5%

-4

0%

-4 5%

-5

0%

-5 5%

-6

0%

-6 5%

-7

0%

-7 5%

-8

0%

-8 5%

-9

0%

-9 5%

-1

Fr eq ue nc y

Bin

EDICT Assessment of Active:Passive Ports per TR https://www.cfm.va.gov/til/dguide/OIT-InfrastrucStdTelecommSpaces.pdf

Version Date: 3 March 2023 12

Position Paper: Converged Physical Telecommunications Infrastructure Planning & Usage on VA Campuses

White Paper: VA Campus Copper Backbone

VA Basic Guidance: TR Design Checklist

VA Basic Guidance: Single-Rack TR Design & Outfitting

Version Date: 3 March 2023 13

Typical One-Rack TR Outfitting Requirements

Version Date: 3 March 2023 14

Typical Two-Rack TR Outfitting Requirements