1、AN-04-9-3 Cooling of High Heat Density Rooms Today and in the Future Lennart K. Stahl ABSTRACT Technology advances are driving the power densities of electronics to unprecedented levels. Consequently, thermal density issues for the facilities hosting the electronics have become topics of increasing
2、interest. Trends indicate that room densities for data centers and telecom central ofices willjump om less than 100 WJ to densities of over 500 WJ in the near future. This paper will identiJL several of the drivers of those trends and assess the impact on coolingsystem design for these spaces. The f
3、ocus of this paper is how high heat density rooms can be cooled using current cooling technology while lowering energy cost, and how the development ofnew cooling technology will impact high-density spaces in the future. EVALUATING THE TRENDS Two trends are converging to create a situation where the
4、 density of new equipment going into data centers and telecom central offices is so high that it is creating problems for the facility managers and engineers charged with installing and supporting new systems. Both of these trends are expected to continue in the coming years, forcing significant cha
5、nges in how data centers are cooled. The first trend is one that has marked the evolution of the computer industry since the beginning: the trend toward smaller, faster systems. The current generation of high-perfor- mance servers and switches simply pack more processing power into a smaller footpri
6、nt than previous generation systems. The impact on heat generation is obvious to cooling professionals, but often end users have the misconception that smaller systems generate less heat than larger systems. Of course, what really matters is power consumption, and it is helpful to analyze the factor
7、s that influence system power consumption, especially at the microprocessor level. The switching on and off of transistors consumes power and generates heat. So the number of transistors per processor and processor clock speed have a significant impact on proces- sor power. The third factor that con
8、tributes to density is track spacing, or how many transistors can be packed into a proces- sor without increasing its physical size. It will come as no surprise that each of these technology drivers-transistors per processor, clock speed, and track spacing-are pushing power densities higher, but the
9、 rate at which this change is occurring is startling. However, the microprocessor industry does not expect this trend to level off in the immediate future. Intel has predicted the number of transistors on their processor “should pass 200 million by 2005 and reach well in excess of 1 billion by the e
10、nd of the decade.” Projections like this have spurred the International Technology Roadmap for Semiconductors-a cooperative effort of manufacturers and government organi- zations that provides an ongoing assessment of semiconductor technology requirements-to warn that microprocessor maxi- mum power
11、will reach 170 watts by 2005. A research group dedicated to helping member compa- nies maximize the availability of business critical systems has studied this trend on a system level and documented its find- ings in Heat Density Trends in Data Processing, Computer Systems, and Telecommunications Equ
12、ipment (Uptime Insti- tute 2001). The institute found power densities had increased more than 300% from 1992 to 2002, jumping from an average of 250 WIR2 within the product footprint in 1992 to 1000 W/ fi2 in 2002 (see Figure 1). Significantly, the increase from 2000 to 2001 was 100 W/R2 within the
13、product footprint, and Lennart Stahl is a product manager at Lieben Corporation, Columbus, Ohio. 574 02004 ASHRAE. CMD HOT com yeat of Produci Announcement Figure 1 Product heat density trend chart, projections for information technology products (Uptime Institute 2001). the institute warned that, “
14、in each subsequent year the annual change gets larger.” The second trend driving increased densities has been enabled by the first: the vertical racking of systems. Like the trend toward smaller, more powerful systems, this trend is not necessarily new, but it is now reaching the point where it is t
15、hreatening to limit the adoption of new technology unless the issues of heat generation are addressed. The emergence of blade servers is an example of this. Blade servers take the trend toward miniaturization to the extreme, packing the entire server on a single board. Blade servers are being promot
16、ed for their flexibility and easier management, and IDC has projected sales of blade servers to grow from $341 million in 2003 to $3.7 billion by 2006. However, at the Information Week 2003 Spring Conference, Dell President and Chief Operating Oficer Kevin Rollins, whose company marketed blade serve
17、rs, warned that heat issues may limit the adoption of blade server technology. Kenneth Brill, executive director of the Uptime Institute, agrees, noting that, “the failure rate in the top third of server racks is three times the bottom two-thirds. Blades will further exacerbate this problem.” The co
18、nvergence of these two trends has forced data center designers to shift their focus from watts per square foot to “watts per rack” when creating cooling solutions. This is a significant shift because it represents a change from a room- based view of cooling, where the focus is removing heat from the
19、 room, to a rack-based view, with more focus on ensuring proper cooling of individual racks. OPTIMIZING CURRENT COOLING TECHNOLOGY Over the years, several approaches have been developed to optimize the performance of current cooling technology to enable existing cooling systems to better adapt to th
20、e densities of new rack-based systems. One of the most successful of these has been the “hot aisle/cold aisle” approach. Using this approach, rows of equip- Figure 2 Raised floor with hot aisle/cold aisle configuration. INCORRECT CORRECT SIDE VIEW Figure3 Diagrams of rack airflow showing egect of bl
21、anking panels. ment racks are arranged in alternating “cold aisles” and hot aisles.” A cold aisle has perforated floor tiles that allow cool- ing air to come up from under the raised floor, while a hot aisle has no perforated tiles. Equipment racks are arranged face-to- face so cooling air from the
22、cold aisle is drawn into the front of the computer hardware and exhausted out the back of the equipment rack onto the adjacent hot aisles. The objective is to separate the source of cooling air from hot air discharge, which returns to the computer room cooling unit, ensuring the air going back to th
23、e cooling unit is at a higher temperature than it would be if it was mixed with cool air (see Figure 2). Because the air entering the cooling unit is hotter, the cooling unit can remove more heat. This approach has become accepted, within limits, as a best practice in data centers design and is now
24、commonplace. Blanking panels have also been used successfully to prevent hot air from recirculating through a partially filled rack. Blanking panels fill unused spots in the rack to close what would otherwise be an open path for hot air being exhausted from the rack to pass back through the rack and
25、 recirculate (see Figure 3). Specifications and guidelines regarding cooling compat- ibility, equipment airflow (front intake, rear or top exhaust), room layout, standardized operating environments, and other ASHRAE Transactions: Symposia 575 factors have been developed and published by ASHRAE, Telc
26、ordia (GR-63-CORE, GR-3028-CORE), and others. These specifications, combined with the adoption of the hot aisle/cold approach, have been extremely helpful in allow- ing traditional approaches to cooling meet increasing densities over the last ten years. But these approaches have generally provided s
27、mall, incremental improvements in cooling while densities are rising dramatically, as the Uptime Institute (2001) noted in its paper: _- As the projected trends occur over the next three to six years, air from under the floor by itself will not be suffi- cient to remove the heat being generated. And
28、 at some point in the not-so-distant future, hardware manufactur- ers are going to have consider a return to water cooling or other methods of removing heat from their boxes. DESIGNING THE PERFECT COOLING SYSTEM In considering new approaches to cooling, its important to step back and review the crit
29、eria by which future cooling systems will be evaluated. Low Lifetime Costs Costs will continue to be a major factor in any decision affecting the data center, but the emphasis is shifting toward lifetime costs rather than focusing strictly on initial costs. Lifetime costs take initial costs into con
30、sideration but also factor in all other costs associated with acquiring, deploying, operating, and disposing of a particular system. In the case of cooling systems, one of the key factors impacting lifetime costs is energy use. Energy costs are continuing to rise and may soon outweigh initial cost a
31、s the key economic factor driving cooling system design. As that happens, it is important to consider two components of efficiency: the first is the over- all efficiency of the system itself, and the second is the effi- ciency with which a system can cool extreme temperatures within one zone or sect
32、ion of the data center. A system that has to drive down temperatures across the room to bring temper- atures within one zone within specifications will consume more energy over its life than a system that is able to address hot spots within the room directly. Maintenance and serviceability should al
33、so be factored into lifetime costs. Ideally, the cooling system would be easy to service and maintain, simple to operate, and would provide advance notification of any conditions that could lead to fail- ure. Protection of IT Systems The reason we have cooling systems is to ensure computer systems o
34、perate within design specifications. With the way racks are now used, that means more than just ensur- ing room temperatures are within design specifications. In order to prevent hot air from recirculating through IT systems, the cooling system must support an approach that prevents the mixing of ex
35、haust and supply air. It must also ensure adequate cooling across the entire height of the rack. Servers at the top of a rack currently have a higher failure rate than servers at the bottom of the rack, and this is almost certainly heat related. In an ideal world, servers at the top of the rack woul
36、d operate under the same conditions as those at the bottom of the rack. Higher density systems are also more vulnerable to inter- ruptions in cooling. Without a steady stream ofcool air, internal temperatures will rise faster based on the density of the system. For example, tests have shown that in
37、a typical room with a room heat density of 150 W/ft2, the equipment can operate for about 25 minutes with no cooling before air inlet temperatures reach 104F (40C) (see Figure 4). By contrast, air inlet temperatures to equipment in a room with a heat density of 450 W/f? will rise to 104F in just two
38、 minutes! Reliability has always been an important factor in data center support systems, but high-density systems will mandate that even greater emphasis be placed on cooling system reliability. Finally, the cooling system itself must not put the equip- ment it is protecting at risk. That means if
39、liquid is used to cool high-density systems, it must be inh-oduced into the data center in a way that does not have the potential to leak and damage the equipment within the data center. Flexi bilitylScala biiity Flexibility and scalability will be essential in the data center of the ture. Weve alre
40、ady taken a look at the trends and seen that heat densities are expected to continue to rise at least throughout this decade. As new high-density systems displace older systems, hot spots will begin to appear, and the heat within the entire room will grow. The cooling system of the future must be ab
41、le to easily accommodate new loads. It must also be flexible enough to adapt to changes in the layout of the room. Rack-based systems are more “mobile” than mainframes in that they can be moved and rearranged within the data center as new systems are added. The cooling system must be flexible enough
42、 to address spot heating problems in different room configurations. The ability to easily add cool- ing capacity targeted at specific racks without replacing or shutting down the existing systems will also be key. . SOC 122F 45c . -_ 69F KIF O 5 10 15 20 25 Time (Minutes) Figure 4 Typical rack air i
43、nlet temperature rise at loss of cooling for diferent room heat densities (Stahl and Belady 2001). 576 ASHRAE Transactions: Symposia EVALUATING TECHNOLOGIES Air-Based Technologies The Raised Floor. The raised floor has become the stan- dard for data center cooling. The main advantages of this approa
44、ch are that it does not introduce any liquid into the controlled environment, it provides a reasonable amount of flexibility and scalability, and it delivers room-level air filtra- tion and humidity control. The main disadvantage of the raised floor is the limited possibility for high-heat density c
45、ooling. The maximum capacity of raised floor systems depends on factors such as floor height, the airflow capacity of the floor tiles, the amount of cabling and piping obstructions under the floor, and the heat load distribution in the room. Typically, raised floor systems can be effective up to abo
46、ut 2000 watts per rack; however, once systems exceed that threshold, relying solely on cool air forced through the floor will not adequately protect IT systems, particularly those at the top of the rack, In addition, the raised floor approach cannot efficiently isolate zones within the data center t
47、o address the specific problems of an individual rack. Finally, while the raised floor approach has proved to be rela- tively scalable over the years, in many cases it has reached the limits of its scalability because of either space limitations, which prevent additional cooling units from being add
48、ed, or the physical limits of floor tiles themselves. Recently, ducted cabinets have been introduced as a way to deliver rack-specific cooling within the raised floor envi- ronment. These systems feature air chambers at the bottom front of the cabinet with fans that pull air up from below the floor.
49、 This air is then distributed uniformly across the height of the rack. Fans at the rear top of the cabinet discharge hot air into the room or an overhead plenum. This “closed” approach can increase the capacity of an existing raised floor cooling system, but it leaves high-heat-density equipment subject to the rapid temperature rise noted earlier in the event of the fail- ure of the distribution system. These closed systems cannot effectively use the room as a buffer in the event of cooling system failure, and the additional fans required for the air distribution decrease the ove
