A Real-World Data Center Power Trend
Over the last three years we have been trending one client’s data center power consumption from weekly power readings taken by their operations staff. We use the trends to predict failover scenarios, identify capacity constraints and plan infrastructure upgrades. We also use the readings to observe the overall data center power consumption.
Over the past three years the data center saw a steady decline in overall data center power consumption. July of 2015, the total data center consumption was 4.6MW compared to 3.4MW today. The reason for the decline were many, including replacing older less efficient servers with new, virtualization, cloud adoption and moving workloads out. Not surprisingly, as the load decreased the data center’s efficiency decreased. The instantaneous PUE in July 2015 was 1.49, compared to 1.61 today.
The following two charts show the 3-year trends. Chart 1 shows the total data center consumption which includes UPS power, HVAC, lights and administrative office spaces. Chart 1 also shows the total UPS load over the same period and the corresponding percent loading of the UPS systems. During the period, UPS load went from 46% down to 31%. Chart 2 shows the PUE of the data center over the same period, along with the same corresponding percent loading of the UPS. Its clear that as the data center load decreased, the PUE increased. This suggests that the data center is operating less efficient at lower loads.
For anyone who has looked at an efficiency curve for a UPS module, you understand that efficiency is a function of UPS load. So, the trend shown in Chart 2 is not surprising. Other system losses will also affect data center performance as the load decreases. Transformer no load losses play a larger role and variable speed fans may have minimum fan speed settings set too high (based on the original data center load) to name a few factors that would negatively affect a data center’s PUE.
A little information on the subject data center. The building was built in 1987 and has been upgraded over the years to keep pace with changing technologies. The building includes 45,000 square feet of office space for business units that are not related to the data center operations. The IT load for the PUE calculation is measured at the UPS output, not downstream of the PDUs. The UPS systems are 10 years old and are arranged in a 3 to make 2 redundant configuration. This allows us to apply more load than the typical 40-45% loading of a traditional 2N UPS configuration. The UPS capacity was sized based on load projections made back in 2007 when load levels were higher. Notice that when the trends shown above started, the UPS load was 46% of the total installed capacity and we were managing it to be no more than 80% of a single system’s capacity under the worst-case failure condition. This required more attention to loads and failure scenarios than a simpler 2N UPS configuration which we accomplished using the weekly trend reports. The benefit to the client was more efficient use of capital and a higher percent load on the UPS than a 2N alternative.
To illustrate the last point lets look at the UPS percent load of the actual system versus a hypothetical 2N system. The subject data center’s 3 to make 2 redundant architecture used three 2,250kW UPSs to make a 4,500kW UPS system. Total installed UPS capacity was 6,750kW (3 x 2,250kW). If we wanted a redundant 4,500kW UPS system using a 2N architecture we would need 9,000kW of installed UPS capacity. Since more installed capacity is required for a 2N system, that system would operate at 34% load compared to 46% load of the 3 to make 2 data center as we saw in July 2015. Chart 3 illustrates this difference.
In conclusion, the downward power consumption trend we observed at this data center is not uncommon. Some of our clients are looking for ways to improve their operating efficiencies by shutting down excess capacity. These trends are a good reminder of how percent utilization can play a role in overall system efficiency and how a good design should be scalable for both load increases and load decreases.
