1、OR-05-1 2-3 High Density Cooling of Data Centers and Telecom Facilities-Part 2 Donald L. Beaty, PE Member ASHRAE Neil Chauhan Daniel Dyer, PE Associate Member ASHRAE ABSTRACT The rapid growth trend of electronic equipment heat density is causing concern regarding how to cool these high, if not extre
2、me, loads. Successfully cooling these loads is a complex problem that is made even more dicult by the lack ofpracticalpeld experience with high density loads. High density loads amplib the impact on the cooling system of many variables (infrastructure conjgurations, spatial allocations, etc.) thatpr
3、eviously may not have been the focus of the design engineer. The needfor a solution is imme- diate. but (due in part to the dot-com crash) there has been a lack of resources to proactively address the problem. There- fore, the industry has fallen behind and requires accelerated recovery. The compoun
4、ded efect of these issues creates a need for a more holistic approach than thosepreviously utilized with a view to a coordinatedprocess that results in planned solutions meeting the approval of all afected parties. Theprst step toward this holistic approach has been taken by ASHRAE TC9.9 with thepub
5、lication ofrhermal Guidelines for Data Processing Environments. Following on from that publication, this two-part paper examines some of the topics that should be considered as apart of the holistic approach and provides some background on the high density cooling topic in general. INTRODUCTION Over
6、view The topics in this field are not easily related to each other since the typical contributors for the topics are often from different sectors of the industry. Specifically, the following areas will be discussed across the two-part paper: The collaborative process: Compensating for a lack of hist
7、orical data (Part 1) Load calculation at the predesign phase (Part 1) The impact of space planning with respect to cooling difficulty (Part 2) High level presentation of some cooling system choices (Part 2) Retrofitting high density loads into existing facilities (Part 2) This second part of the two
8、-part paper will cover three sections that represent the relative impact of the choices that have to be made with regard to the building structure, cooling system types, and determining the approach for retrofitting existing facilities with computer equipment that is considered to be a high density
9、load. The first section in this paper will discuss the variables involved with the building plan and section designs. This will include both the considerations at a load planning level and the dimensions that can impact the cooling delivery system and performance. The second section will provide a h
10、igh level overview of the different types of cooling systems and delivery options that are considered for modem data centers. The final section in this paper attempts to provide a process for determining the path forward when faced with the challenge ofhaving to retrofit a high density load into an
11、exist- ing data center and will discuss how the process changes with aspects such as the size of the deployment and how more radi- cal changes to the data center may prove to be the more complete solution long term as opposed to cooling through brute force by simply adding capacity. Donald L. Beaty
12、is president and Neil Chauhan and Daniel Dyer are engineers at DLB Associates, Consulting Engineers, Ocean, New Jersey. 932 02005 ASHRAE. Some Definitions AS this two-part paper will show, there is a lot of vague- ness with regard to the thresholds where a load is considered high density. Some secto
13、rs of the industry will focus on kW per rack and other sectors will want to focus on the average watts per square foot (watts per square meter) in a given computer room. For the purposes of this paper, high density cooling will mean loads that have reached a threshold of 5 to 10 kW per rack or an av
14、erage density of 100 to 150 watts per square foot (1 O00 to 1500 watts per square meter) across a computer room. Although the two threshold ranges mentioned are not interchangeably related (i.e., 10 kW per rack represents a much higher load than 150 W/ 1500 W/m2), the values do represent the current
15、 industry perceptions of the range of the high density thresholds. A more tangible method of under- standing the thresholds are to consider that a percentage of higher density racks within a computer room will drive up the average watts per square foot (watts per square meter). The term rack has dif
16、ferent definitions in the telecom industry versus data centers, but for this paper, the broadest definition will be used, which is “an open frame or enclosed cabinet that houses electronic equipment.” This paper is not aimed specifically at any particular type of high density equipment deployment. C
17、ertainly there are many scenarios that could exist, including: New construction versus retrofitting an existing building Retrofit of an existing datacom facility versus retrofit of a building with a different (non-datacom) usage Small-scale deployment versus a large-scale deployment of high density
18、loads (e.g., addition of one to three high density racks versus addition of multiple rows of high density racks) The scenarios listed above all have their own unique chal- lenges and constraints, and in some scenarios, a feasibility study for a high density equipment deployment may result in either
19、extensive infrastructure changes being required or the deployment in a particular scenario may be deemed unfeasi- ble. Do High Density Loads Exist? There are plenty of studies and information showing over- stated loads (i.e., a measured load that is a mere fraction of the design or connected load).
20、There are people that claim they have never seen loads above 25 W/ft2 (250 W/m2) and certainly not above 50 W/fi2 (500 W/m2). These same people would then argue that even if they double or triple that load (which is unlikely) that it does not support the high density levels being projected or claime
21、d. Determining whether high density loads exist based on personal experience or national averages can be very mislead- ing since it could imply that high density loads do not exist. The fact is that high density loads do exist (currently, specific loads of 15 to 25 kW per rack or more have been meas
22、ured). Although high density installations are few, the quantity is increasing. The existence of any high density loads establishes a need to address how to cool those loads. To summarize, the claim is not that all loads or most ofthe loads will be high density, but rather some of the loads will be
23、high density in some of the facilities. Based on the product heat density trend charts, history supports a significant increasing trend of loads at the component, board, and equip- ment level. As a result, the current existence of some high density loads combined with the projected future loads warr
24、ants the development of strategies to effectively handle high density loads. Background on Issues and Influences make this a challenging problem to solve: The following are some of the issues or influences that Within the datacom environment (the term datacom refers to both data centers and telecom
25、facilities), the IT equip- ment loads continue to rise. However, practical field expe- rience and actual measured data for those loads are not readily available for the following reasons: High-density loads (5 to 10 kW per rack or greater) have seldom been experienced in datacom facilities so there
26、is a shortage of empirical data. The speed to construct versus the speed of IT equip- ment load growth creates a lag in completed instal- lations. Consequently, the field data lags behind the IT equipment loads that are typically announced to be shipped within the next 12 months. Due to nondisclosur
27、e agreements (NDA), concern for security, and to keep a competitive edge, often there is a reluctance to share data, especially fail- ures, on the latest technology installed. The functions of IT, facility design and construction, and facility operation and maintenance are often driven by, and repor
28、t to, independent departments or people, with each one placing a different emphasis on any given attribute such as: cost Reliability Energy efficiency Speed to marketldeployment Maintenance The different emphases result in budgets that are indepen- dent of one another, This, in turn, creates or prom
29、otes independence and working in isolation rather than work- ing in a multi-functional, collaborative basis, addressing the project and decisions in a holistic way, including effective balancing of criteria. ASH RAE Transactions: Symposia 933 3. High density cooling is thought of as a challenge in t
30、erms of thermodynamics and physics. As a result, it both attracts and drives us toward focusing solely on technical cooling issues. Although these issues are critical, there are other related technical issues and also some nontechnical consid- erations that are just as critical but have a greater ri
31、sk of receiving little attention. Technical Cooling Issues-These specifically focus on using analytical heat transfer and fluid dynamic study, etc., to address the specific heat load challenge presented and determine the cooling system design. Other Technical Issues-These fall into two main categori
32、es, directly related and indirectly related: Directly related technical issues include con- sidering the influences and constraints that complicate or adversely affect the performance of the cooling system. In other words, instead of prematurely trying to implement a cooling system in the physical e
33、nvironment that has been presented, consideration needs to be given as to whether the environment itself can be modified to better serve the cooling system. For example, this could include increasing the height of the raised floor cavity in the high den- sity load area. Indirectly related technical
34、issues consider the variables affecting the heat loads, including the high density load location and the various IT equipment options (e.g., using fiber versus cop- per connectiviy can impact freedom on dis- tances and congestion). Through optimizing these variables, it may be possible to reduce the
35、 cooling loads before even considering how to successful cool them. Nontechnical Considerations-These include intangibles such as process, procedure, and the fun- damental essence of problem solving. Unique and constantly changing problems such as high density cooling require a heavy focus on proces
36、s to enable effective collaboration and holistic assessment/per- formance. This is counterintuitive to the way the industry and the project participants are currently operating. 4. The legacy methodology used for projecting or calculating the amount of heat dissipation that needs to be cooled is poo
37、rly defined and not optimized for addressing high density heat dissipation loads. A common approach is to use watts per square foot (watts per square meter). Due to the lack of consistent definition of the attributes used for that calculation (discussed in detail later), this number could easily var
38、y by as much as 25% to 50%. Since the loads are rapidly changing, the increases in the watts per square foot (meter) values are made using adjustment factors to the previous industry standard watts per square foot (meter) values. However, the potential variation in the definition makes those origina
39、l numbers an ineffective baseline. THE IMPACT OF SPACE PLANNING WITH RESPECT TO COOLING DIFFICULTY The natural tendency is to focus on the thermal engineer- ing aspect of solving the high density cooling issue. Although the technical cooling issues are important, the problem is really both two-dimen
40、sional and three-dimensional, with a heavy spatial influence. Two-Dimensional Impact with Respect to Cooling Difficulty The first variable that is typically considered when deal- ing with cooling impact on the two-dimensional plan view is the layout of the various components associated with the cool
41、- ing system. The majority of cooling systems within a datacom room are air cooled, and one of the first two-dimensional attributes that is considered is the width of the aisles between the racks. Figure 1 compares a narrow aisle to a wide aisle. The intent of the figure is to highlight the extremes
42、 and is not intended to be representative of current installations. However, it is easy to see the difference in air capacity and flow with a wider aisle. A wider aisle provides the ability for more perforated tiles or grilles. It also can reduce the aisle air change rate and velocity, as well as re
43、duce the density of the load in the space. Another way that the space planning affects the cooling system specifications is with the distribution ofthe load within the rows of racks. In the example studies below, we shall consider two methods used to try to reduce the cooling chal- lenge presented b
44、y the layout of racks of varying densities. Example 1. Reducing the Cooling Challenge by Spreading the Load Table 1 demonstrates that for the same equipment, depending on how it is configured, the maximum load for a given row varies significantly. In this example study there are a total of eighteen
45、racks (six with low density loads, six with medium den- sity loads, and six with high density loads). Option 1 shows dedicated low, medium, and high density rows resulting in the maximum row load (high density row) being 60 kW. Option 2 shows an even distribution of the low, medium, and high den- 9
46、934 ASHRAE Transactions: Symposia sity loads, resulting in the maximum row load being 30 kW. In this particular example, the concentration of load is reduced by 50% simply by evenly distrib- uting the load. Option 2 certainly reduces the cooling problem and potentially can be an effective solution f
47、or certain situations. high density rack. 9 Example 2. Reducing the Cooling Challenge by Allo- cating more Space Table 2 demonstrates that by adding six empty racks and an additional row, the same results can be accomplished as in example 1. This is accom- plished by inserting an empty rack adjacent
48、 to each Figure 1 Comparison of a narrow aisle between racks to a wide aisle between racks (cooling is easier to accomplish with a wide aisle). Table 1 - Example 1 - Reducing the Cooling Challenge by Spreading the Load Rack I Configuration Options This example shows that by evenly distributing the h
49、igh density racks within the room, the maximum load per row is reduced by 50%. Although this is a very attractive way to ease the cooling burden, it may compromise the functionality of the equipment from an operators perspective, wiring limitation I efficiency, etc. This approach should certainly be one of the considerations but a multi-discipline, holistic solution is required. ASHRAE Transactions: Symposia 935 Rack Load Quantity Total Density kW A I B C D=BxC Configuration Options Option 1 Option 2 Row4 Row2 Row3 Row4 Row1 Row2 Row3 Row4 El FI GI H Il JI KI L Low Medium . I 1 6 6 6
