Improving Data Center Air Management
Using the discussed air-management metrics, detailed calculations were made for a mid-range data center with bypass and re-circulation factors of 0.5. Eliminating airflow bypass can reduce CRAC fan energy use by 60kW. By eliminating re-circulation, compressor energy savings of 74kW are possible, due to the evaporator operating at higher temperatures. For a 10,000 square foot data center, this can represent a saving of $90,000 annually. These estimates do not include free cooling.
How can facility managers improve the air management of their data centers? From the practical applications the following is a list of measures that reduce the issues of poor air management. They are listed in descending order of impact and feasibility:
Bypass Air:
- Seal air gaps in the raised floor, particularly cable cut-outs within cabinets at the discharge side of servers.
- Relocate floor air grills so that they supply the inlet of servers — that is, remove floor air grills from hot aisles and other areas where cooling isn't required.
- Ensure air velocities from floor grills are not too high, which can cause air to overshoot the top of the cabinet.
- CRAC units turned off are a source of air bypass; consider air isolation with dampers.
Recirculation Air:
- Fit blanking plates in cabinets where servers are not installed.
- Close gaps between cabinets where warm air can make its way to the server inlet.
- Ensure as much as possible the adequate supply of cold air to the server inlets.
- Ensure air velocities and flows from floor grills are sufficient to reach the top of cabinet servers.
- Review air return path to CRACs and consider return air plenum or ductwork if necessary.
- Consider means to physically isolate the supply (cold) and return (hot) air streams.
- Remove obstructions under the floor that restrict cold air to supply the server inlets.
The effectiveness of those measures can be seen from work done at a 5,000-square-foot data center. During an energy assessment of the facility, measurements were taken from more than 20 percent of the servers present and all of the CRAC units. The general data center floor layout is back-to-front, but some server discharges and intakes are on the same side. The load density was 27 watts per square foot. Recirculation, bypass, and balance were calculated to be 0.61, 0.21, and 0.50, respectively. Recall that the ideal for recirculation and bypass is zero, and for balance it is 1. Clearly there was room for improvement.
The balance ratio of 0.5 indicates that the servers are not getting enough air. That was attributed to the fairly low height (18 inches) of the raised floor with many obstructions. The obstructions resulted in low air velocities from the floor grills, so that cool air didn't reach the servers in the top of the cabinets. Another problem is that many server intakes and discharges are located in the same aisle.
To reduce recirculation and increase balance, the assessment team recommended better management of underfloor obstructions; more efficient layout such as cold/hot aisle; retrofitting the data center with a return air plenum; blanking-off all the unused server locations; fitting blanking panels between cabinets; and fitting brush seals at all cable cutouts. Based on measurements taken from the data center at the same locations after implementing most of the recommendations, new air management metrics were 0.34 for recirculation, 0.18 for bypass and 0.82 for balance, a substantial improvement. However, the facility manager is looking at installing a return air plenum to reduce the recirculation flow ratio even more.
Lessons Learned
The following lessons can serve as guidelines for such analyses.
- For all temperature measurements use a simple, reliable thermometer, not laser guns since they measure surface temperatures and not air "ambient" temperature.
- Measure and average the actual air outlet temperature (supply from CRAC) and air inlet temperature (return to CRAC) in a couple of locations across the duct and not only at a single point. In some cases, stratification may lead to different temperature values.
- The assessment team should sample two to three times as many servers as there are CRAC units, measuring the server air inlet and outlet temperatures.
- The server air inlet temperature should be taken with a thermometer as close as possible to server grill inlet (not at the rack or cabinet inlet) without touching any surfaces. Similarly, server outlet temperature should be taken as close as possible to the server without touching any surfaces.
- Server inlet and outlet temperatures should be taken at the same level for the same server. If temperature is measured at the server inlet located in the middle of the rack, the server outlet temperature should be taken for the same server at the same level.
- Readings should be taken from servers in a mix of locations: beginning of aisle, end of aisle, middle of aisle, bottom of cabinets, middle of cabinets, and top of cabinets.
- Readings should be taken from several aisles and not just one or two aisles.
- Measurements should be taken from several hot spots.
- It is acceptable to use average temperatures rather than weighting them with mass flow rate/size.
Many data center operators take a relaxed attitude toward bypass air, which is produced by the CRACs and returns directly without cooling servers. Bypass air causes a gap between the temperature of air produced by the CRAC units and the temperature of the air at the inlet to the server, although in theory those two should be the same. CRAC units are designed to produce lower temperatures for the sole reason of offsetting the re-circulation of air from the outlet of the server to its inlet. Air bypass also starves servers of air, which increases recirculation.
Better air management within data centers — mainly minimizing bypass and recirculation air flow — has the potential to improve temperature control for more reliable server cooling. It can also save energy. What's more, improved air management is essential to maximizing data center free cooling opportunities.
Munther Salim, PhD, is principal, energy efficiency and sustainability in the Chicago office of HP Critical Facilities Services - EYP MCF. Robert Tozer, PhD, is technical director - HP EMEA.
Download the technical treatment of this article.
Air Management Metrics (PDF)
Related Topics: