TIA-942-A-1-2013 Telecommunications Infrastructure Standard for Data Centers Addendum 1- Cabling Guidelines for Data Center Fabrics (Addendum to TIA-942-A).pdf

1、 TIA-942-A-1 (Addendum to TIA-942-A) March 2013Telecommunications Infrastructure Standard for Data Centers Addendum 1- Cabling Guidelines for Data Center Fabrics ANSI/TIA-942-A-1-2013 APPROVED: MARCH 26, 2013 NOTICE TIA Engineering Standards and Publications are designed to serve the public interest

2、 through eliminating misunderstandings between manufacturers and purchasers, facilitating interchangeability and improvement of products, and assisting the purchaser in selecting and obtaining with minimum delay the proper product for their particular need. The existence of such Standards and Public

3、ations shall not in any respect preclude any member or non-member of TIA from manufacturing or selling products not conforming to such Standards and Publications. Neither shall the existence of such Standards and Publications preclude their voluntary use by Non-TIA members, either domestically or in

4、ternationally. Standards and Publications are adopted by TIA in accordance with the American National Standards Institute (ANSI) patent policy. By such action, TIA does not assume any liability to any patent owner, nor does it assume any obligation whatever to parties adopting the Standard or Public

5、ation. This Standard does not purport to address all safety problems associated with its use or all applicable regulatory requirements. It is the responsibility of the user of this Standard to establish appropriate safety and health practices and to determine the applicability of regulatory limitati

6、ons before its use. (From Project No. ANSI/TIA-PN-942-A-1-D1, formulated under the cognizance of the TIA TR-42 Telecommunications Cabling Systems, TR-42.1 Subcommittee on Commercial Building Telecommunications Cabling(568, 942, 862, 1179). Published by TELECOMMUNICATIONS INDUSTRY ASSOCIATION Standar

7、ds and Technology Department 1320 N. Courthouse Road Arlington, VA 22201 U.S.A. PRICE: Please refer to current Catalog of TIA TELECOMMUNICATIONS INDUSTRY ASSOCIATION STANDARDS AND ENGINEERING PUBLICATIONS or call IHS, USA and Canada (1-877-413-5187) International (303-397-2896) or search online at h

8、ttp:/www.tiaonline.org/standards/catalog/ All rights reserved Printed in U.S.A. NOTICE OF COPYRIGHT This document is copyrighted by the TIA. Reproduction of these documents either in hard copy or soft copy (including posting on the web) is prohibited without copyright permission. For copyright permi

9、ssion to reproduce portions of this document, please contact the TIA Standards Department or go to the TIA website (www.tiaonline.org) for details on how to request permission. Details are located at: http:/www.tiaonline.org/standards/catalog/info.cfm#copyright or Telecommunications Industry Associa

10、tion Technology (b) there is no assurance that the Document will be approved by any Committee of TIA or any other body in its present or any other form; (c) the Document may be amended, modified or changed in the standards development or any editing process. The use or practice of contents of this D

11、ocument may involve the use of intellectual property rights (“IPR”), including pending or issued patents, or copyrights, owned by one or more parties. TIA makes no search or investigation for IPR. When IPR consisting of patents and published pending patent applications are claimed and called to TIAs

12、 attention, a statement from the holder thereof is requested, all in accordance with the Manual. TIA takes no position with reference to, and disclaims any obligation to investigate or inquire into, the scope or validity of any claims of IPR. TIA will neither be a party to discussions of any licensi

13、ng terms or conditions, which are instead left to the parties involved, nor will TIA opine or judge whether proposed licensing terms or conditions are reasonable or non-discriminatory. TIA does not warrant or represent that procedures or practices suggested or provided in the Manual have been compli

14、ed with as respects the Document or its contents. If the Document contains one or more Normative References to a document published by another organization (“other SSO”) engaged in the formulation, development or publication of standards (whether designated as a standard, specification, recommendati

15、on or otherwise), whether such reference consists of mandatory, alternate or optional elements (as defined in the TIA Engineering Manual, 4thedition) then (i) TIA disclaims any duty or obligation to search or investigate the records of any other SSO for IPR or letters of assurance relating to any su

16、ch Normative Reference; (ii) TIAs policy of encouragement of voluntary disclosure (see Engineering Manual Section 6.5.1) of Essential Patent(s) and published pending patent applications shall apply; and (iii) Information as to claims of IPR in the records or publications of the other SSO shall not c

17、onstitute identification to TIA of a claim of Essential Patent(s) or published pending patent applications. TIA does not enforce or monitor compliance with the contents of the Document. TIA does not certify, inspect, test or otherwise investigate products, designs or services or any claims of compli

18、ance with the contents of the Document. ALL WARRANTIES, EXPRESS OR IMPLIED, ARE DISCLAIMED, INCLUDING WITHOUT LIMITATION, ANY AND ALL WARRANTIES CONCERNING THE ACCURACY OF THE CONTENTS, ITS FITNESS OR APPROPRIATENESS FOR A PARTICULAR PURPOSE OR USE, ITS MERCHANTABILITY AND ITS NONINFRINGEMENT OF ANY

19、 THIRD PARTYS INTELLECTUAL PROPERTY RIGHTS. TIA EXPRESSLY DISCLAIMS ANY AND ALL RESPONSIBILITIES FOR THE ACCURACY OF THE CONTENTS AND MAKES NO REPRESENTATIONS OR WARRANTIES REGARDING THE CONTENTS COMPLIANCE WITH ANY APPLICABLE STATUTE, RULE OR REGULATION, OR THE SAFETY OR HEALTH EFFECTS OF THE CONTE

20、NTS OR ANY PRODUCT OR SERVICE REFERRED TO IN THE DOCUMENT OR PRODUCED OR RENDERED TO COMPLY WITH THE CONTENTS. TIA SHALL NOT BE LIABLE FOR ANY AND ALL DAMAGES, DIRECT OR INDIRECT, ARISING FROM OR RELATING TO ANY USE OF THE CONTENTS CONTAINED HEREIN, INCLUDING WITHOUT LIMITATION ANY AND ALL INDIRECT,

21、 SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES (INCLUDING DAMAGES FOR LOSS OF BUSINESS, LOSS OF PROFITS, LITIGATION, OR THE LIKE), WHETHER BASED UPON BREACH OF CONTRACT, BREACH OF WARRANTY, TORT (INCLUDING NEGLIGENCE), PRODUCT LIABILITY OR OTHERWISE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGES

22、 THE FOREGOING NEGATION OF DAMAGES IS A FUNDAMENTAL ELEMENT OF THE USE OF THE CONTENTS HEREOF, AND THESE CONTENTS WOULD NOT BE PUBLISHED BY TIA WITHOUT SUCH LIMITATIONS. ANSI/TIA-942-A-1 i TELECOMMUNICATIONS INFRASTRUCTURE STANDARD FOR DATA CENTERS ADDENDUM 1 CABLING GUIDELINES FOR DATA CENTER FABR

23、ICS Table of Contents FOREWORD . ii 1 SCOPE 1 2 DEFINITIONS 1 3 TRADITIONAL SWITCH ARCHITECTURE 2 4 MULTIPLE CONNECTIONS . 4 5 DATA CENTER SWITCH FABRICS . 5 5.1 Data center fabric fat-tree 6 5.2 Data center fabric full-mesh 10 5.3 Data center fabric interconnected meshes 12 5.4 Data center fabric c

24、entralized switch . 13 5.5 Data center fabric virtual switch . 15 List of Figures Figure 1: Example of traditional three-tier data center switch architecture 2 Figure 2: Example of ANSI/TIA-942-A cabling topology 3 Figure 3: Examples of redundant connectivity 4 Figure 4: Fat-tree example . 6 Figure

25、5: Fat-tree with port extenders example . 7 Figure 6: Example of use of structured cabling for fat-tree switch architecture 8 Figure 7: Interconnected fat-tree pods example 9 Figure 8: Full-mesh example . 10 Figure 9: Example of use of structured cabling for full-mesh switch architecture 11 Figure 1

26、0: Interconnected meshes example 12 Figure 11: Example of use of structured cabling for interconnected mesh fabric . 13 Figure 12: Centralized switch example . 13 Figure 13: Example of use of structured cabling for centralized switch fabric 14 Figure 14: Virtual switch example . 15 Figure 15: Exampl

27、e of cabling for fabric virtual switch architecture . 16 ANSI/TIA-942-A-1 iiFOREWORD TIA standards documents are developed within the Technical Committees of the TIA and the standards coordinating committees of the TIA standards board. Members of the committees serve voluntarily and without commissi

28、on. The companies that they represent are not necessari-ly members of the TIA. The standards developed within the TIA represent a consensus of the broad expertise on the subject. This expertise comes from within the TIA as well as those out-side of the TIA that have an expressed interest. The viewpo

29、int expressed at the time that this standard was approved was from the editors experience and the state of the art at that time. Users are encouraged to verify that they have the latest revision of the standard. This Addendum has been prepared by the TR-42.1 Subcommittee and approved by TR-42.1, the

30、 TR-42 Engineering Committee and the American National Standards Institute (ANSI). There are no annexes in this Addendum. ANSI/TIA-942-A-1 1 1 SCOPE This Addendum specifies recommendations for telecommunications cabling to support data center switch fabrics. 2 DEFINITIONS For the purpose of this Add

31、endum the following definitions apply. access switch: A switch used to connect devices, such as servers, to a local area network. aggregation switch: A switch that aggregates network traffic to and from access switches and that may also connect to network service devices (e.g., load balancers, netwo

32、rk appliances, firewalls). blocking switch fabric: A switch fabric that does not have sufficient bandwidth to ensure that any port can communicate with any other port in the switch fabric at the full bandwidth capacity of either port. centralized switch fabric: A data center switch fabric architectu

33、re in which the switch fabric is implemented in a single centralized switch. core switch: A backbone switch at the highest hierarchical layer of a network. fat tree, fabric: A switch connection topology where each access switch is connected to every interconnection switch within the fabric. full-mes

34、h, fabric: A switch connection topology where each switch is directly connected to all other switches within the mesh. interconnected mesh fabric: A switch connection topology in which pods, each containing a full-mesh fabric, are connected using interconnection switches. interconnection switch: A s

35、witch used to connect access switches, or lower level interconnec-tion switches, in a fabric. leaf and spine switch fabric architecture: See fat tree, fabric. leaf switch: An access switch in a leaf and spine switch fabric architecture. non-blocking switch fabric: A switch fabric that has sufficient

36、 bandwidth to ensure that any port can communicate with any other port in the switch fabric at the full bandwidth capacity of either port. over-subscribe (bandwidth): The assignment of more traffic to a link than the bandwidth capaci-ty of the link. pod, data center: A modular subset of the data cen

37、ter. point-to-point cabling: Direct cabling between equipment (e.g., direct connection between two switches using a cord). port extender: A device that provides additional ports to the controlling switch to which it is connected. spanning tree protocol: Link management protocol that provides path re

38、dundancy while pre-venting undesirable loops in the network. spine switch: An interconnection switch in a leaf and spine switch fabric architecture. switch fabric: A network topology where devices connect with one another using network switches over multiple paths. ANSI/TIA-942-A-1 2virtual switch f

39、abric: A switch connection topology in which a switch fabric is formed by inter-connecting multiple switches to form a single large virtual switch. 3 TRADITIONAL SWITCH ARCHITECTURE Figure 1 provides an example of the traditional three-tier data center switch architecture. Figure 1: Example of tradi

40、tional three-tier data center switch architecture Some of the characteristics of the traditional three-tier switch architecture are: Backup connections (the dashed red lines in figure 1) to access switches are not active due to spanning tree protocols used to prevent routing loops (some protocols do

41、 not have this limitation) Connections are typically over-subscribed (i.e., more traffic assigned to the link than the bandwidth capacity of the link) When access switches are located in the EDA (i.e., top of rack), more switch ports may be available in each cabinet than necessary Traffic between tw

42、o access switches may need to traverse as many as three intermedi-ate switches The traditional architecture is well-suited for traffic between servers on the same access switch and from servers to external destinations. However, it isnt suitable for large virtualized data centers where compute and s

43、torage servers may be located anywhere in the data center. All cabling for the traditional architecture follows the hierarchical cabling topology specified in ANSI/TIA-942-A. See figure 2 for an example. ANSI/TIA-942-A-1 3 ENIMCICCPICCPHC HCEO EOEOEOAccess provider or campus cablingHCESMDAIDAIDAHDAH

44、DAZDA ZDATREDA EDA EDA EDAHorizontalcabling for spaces outside computer roomhierarchical backbone cablinghorizontal cablingcross-connectinterconnectionoutletLEGENDTelecom spaceHCHDAEOEO EOEDA EDA EDAHCHDAEOEO EOEDA EDA EDACP consolidation pointEDA equipment distribution areaENI external network inte

45、rfaceEO equipment outletES entrance room or spaceHC horizontal cross-connectHDA horizontal distribution areaIC intermediate cross-connectIDA intermediate distribution areaMC main cross-connectMDA main distribution areaTR telecommunications roomZDA zone distribution areaoptional backbone cablingbetwe

46、en peer level cross-connectsFigure 2: Example of ANSI/TIA-942-A cabling topology ANSI/TIA-942-A-1 44 MULTIPLE CONNECTIONS Compute and storage servers often have multiple connections to provide redundancy, additional bandwidth, or to support different functions. The connections may be to a single swi

47、tch, to mul-tiple switches within the same network, or to multiple switches in different networks. See figure 3. AccessSwitchMultiple connections to the same switchAccessSwitchAccessSwitchNetworkMultiple connections to different switches on the same networkAccessSwitchAccessSwitchMultiple connection

48、s to different switches on different networksServerServerNetworkNetworkNetworkServerFigure 3: Examples of redundant connectivity ANSI/TIA-942-A-1 5 5 DATA CENTER SWITCH FABRICS Data center switch fabric architectures are intended to provide low-latency and high-bandwidth communication between any tw

49、o points in the switch fabric. Data center switch fabric architec-tures typically use only one or two tiers of switches. These switches have large numbers of con-nections to other switches. These connections are all active to provide multiple paths that mini-mize latency and to provide maximum switch fabric bandwidth. Data center switch fabric architectures can be accommodated using the cabling topology speci-fied in ANSI/TIA-942-A. Some data center switch fabric architectures require backbone cabling between peer level distributors, for example bet