Today’s data center power and cooling infrastructure has roughly 3 times more data points / notifications than it did 10 years ago. Traditional data center remote monitoring services have been available for over 10 years but were not designed to support this amount of data monitoring and the associated alarms, let alone extract value from the data. This paper explains how seven trends are defining monitoring service requirements and how this will lead to improvements in data center operations and maintenance.
Data center digital remote monitoring services1 have been around for over 10 years but older offline traditional services are limited compared to new digital2 services available today (see Table 1 for comparison). These new services incorporate technology such as cloud computing, analytics, and mobile apps.
Inside a data center today, a manager has no idea when they should replace a component in their UPS or cooling unit that is about to fail. In contrast, outside the data center, a driver gets an instant notification on their smart phone that their normal route is backed up 20 minutes with a recommended alternate route. This disparity has prompted us to look at how advancements and trends in IT are changing data center monitoring and, in turn, how digital remote monitoring will change data center operations and maintenance.
The general concept of monitoring today is widely understood and anyone with a fitness tracker, continuous glucose monitor, or a learning thermostat has had firsthand experience in how advances in IT have improved their lives. In particular, users benefit from immediate knowledge from their devices (e.g. calories burned, blood sugar level, etc.). However, most data centers today are not benefiting from big data analytics and machine learning. These and five other trends are poised to revolutionize how managers operate and maintain data centers.
This paper explains seven trends that are defining next-generation data center monitoring and its benefits. We describe the requirements to attain these benefits, and describe how data center operations and maintenance will evolve in the future.
Table 1 — Comparison between traditional and digital remote monitoring
A key differentiator between these two types of remote monitoring comes down to the definition of online3 – “connected to a computer, a computer network, or the Internet”
Traditional remote monitoring is not an online service therefore it cannot provide real-time monitoring. Instead it relies on intermittent status updates (usually via email).
Digital remote monitoring is online and connected to a data center (usually through a gateway) which allows for realtime monitoring. In addition it uses IT services such as cloud storage and data analytics.
Monitoring services available 10 years ago were desktop-based, limited in data output, and largely reactionary (i.e. depended on humans to interpret what was wrong). Digital remote monitoring has resolved these limitations through technology, and over the next few years more limitations will be addressed by technology. We see seven technology trends that are influencing data center monitoring.
We briefly describe these trends in this section and, in the next section, we describe the digital remote monitoring requirements needed to comprehend, mitigate, or take advantage of these trends.
Embedded systems are found in nearly all data center devices including cooling units, PDUs, UPS, chillers, etc. and basically control the operation of these devices. Without the outputs from these embedded systems, there would be nothing to monitor. Embedded systems have improved significantly over the years in terms of computing capability, data storage, communications, and pricing. This means data center devices today can provide much more data today than they could 10 years ago. We estimate that the total number of alarms and notifications available from power and cooling devices have increased over 300% over the last ten years. This increase comes from a combination of more sensors, more features, more algorithms, and higher sampling rates. The more data available, the more digital remote monitoring can infer helpful information from data center devices, as we describe later in the paper.
Cyber security is one of the biggest concerns5 among data center managers around the world. Not only are they concerned about IT equipment vulnerability, but also physical infrastructure equipment that has been exploited as “backdoors” into the IT network. Digital remote monitoring, as well as other cloud-based services, must comprehend cyber risks even before the product or service is created. Digital service providers need to demonstrate their secure development lifecycle (SDL) practices and policies. Ask for their SDL policy, and validate that the lifecycle includes phases that focus on training, security requirements, design, development (e.g. coding standards), verification, release, deployment, and response. In terms or architecture, there should be a single point of entry into your network using a gateway (usually software), and all devices communicate with the gateway. Figure 1 illustrates a recommended digital remote monitoring architecture.
There are several other factors that data center managers and security stakeholders must consider when evaluating a vendor and their digital remote monitoring service, therefore we discuss this topic further in White Paper 239, Addressing Cyber Security Concerns of Data Center Remote Monitoring Platforms.
Figure 1 — Recommended digital monitoring architecture
Cloud computing is a highly scalable method of storing data and processing that data. Cloud computing is what enables digital remote monitoring services. IT services such as predictive analytics and machine learning can run on a cloud computing platform to further increase the value of data center monitoring.
Big data analytics may seem far from the mainstream but it applies to activities performed today such as condition based maintenance (also referred to as predictive maintenance) for plane engines and predicting how many products manufacturers make for the holidays. A spreadsheet or database can only go so far to identify patterns in data. Big data analytics is required when6:
Global use of mobile phones to access the internet has grown year over year for the last several years while access through desktops has decreased year over year7. This trend applies also to data center managers who are increasingly asked to do more with fewer resources. Mobile computing helps alleviate this burden by allowing managers to float between locations without being disconnected from daily operations.
Machine learning is related to data analytics in that it uses data to make predictions but it’s different in that it improves the model by using results from previous learning8. Machine learning can be used to drive an autonomous vehicle, recognize speech, recognize images, chose a Netflix movie, or accurately model the PUE of a very complex Goggle data center. In all of these examples, the driving, the recognition, etc. improves over time.
Automation for labor efficiency is not a “hot” trend but it’s particularly relevant to data center managers in an increasingly competitive business environment where they are being asked to do more with less. This is where automation through digital remote monitoring can help.
The first trend in the previous section (Embedded system performance and cost improvements), creates an overarching challenge for data centers. The amount of data to track is increasing, rapidly making it harder for data center managers to interpret what it means and take the right actions. This is unsustainable, especially when you operate a data center that’s already understaffed. Some other challenges managers face include:
A digital remote monitoring service that comprehends, mitigates, or takes advantage of the trends discussed above, can overcome these challenges and provide the following benefits. Digital remote monitoring requirements are provided for each benefit.
A review of downtime events typically reveals a series of state changes that collectively lead to downtime. In other words, a single failure event normally does not result in downtime. The whole point of monitoring data centers is to reduce the risk of downtime by identifying and addressing a state change before others occur. In this context, digital remote monitoring services should meet the following requirements.
The following are requirements that allow a digital remote monitoring service to reduce operations overhead, leaving staff to focus on more important proactive tasks that add value to the business.
Figure 2 — Example of a network operations center (NOC)
Figure 3 — Example of a digital monitoring mobile app
The more devices being monitored, the better the opportunity to improve the data center efficiency. However, to make a useful inference about the data center efficiency, the UPS load (at a minimum) must be measured as a proxy for the total IT load. Without knowing the IT load there is no basis upon which to assess an increase or decrease in power and cooling infrastructure. For example, if chiller energy is trending upward, I won’t know if it’s due to a chiller problem or due to an increasing IT load. With this data, one can compare the power consumption of all the devices in the power and cooling paths and look for anomalies compared to the IT load. However, a more effective method for improving data center efficiency is to measure PUE and compare it to a PUE model in real time.
White Paper 154, Electrical Efficiency Measurement for Data Centers discusses how an energy efficiency model works and describes a system for continuous measurement while assessing the PUE against the model. When properly implemented, electrical efficiency trends can be reported, and alerts generated based on out-ofbounds conditions. Furthermore, an effective system can provide the ability to diagnose the sources of inefficiency and suggest corrective action. This modelbased efficiency solution should also be continuously monitored by NOC personnel.
Scalability is the ability for the digital remote monitoring system to accept additional devices, or nodes, to monitor. Depending on how these systems are designed, monitoring may be limited to a few thousand devices. Scalability isn’t typically a problem for smaller data centers (e.g. 500kW IT load capacity) but is a serious problem for larger data centers. Some data centers can have hundreds of thousands of devices to monitor and require polling every few seconds, therefore, a digital remote monitoring system should be designed using a horizontally-scalable, cloud-based architecture. This means that as more devices are monitored, the cloud service automatically adds more compute nodes to handle the monitoring. Data center managers need to identify their requirements and then understand the capabilities and limitations across the various monitoring services being evaluated.
Use of embedded sensors in clothing, in watches, and in other “wearables” will allow doctors to predict when you’re getting sick or when you are at risk of a heart attack, and numerous other insights. By analyzing fuel consumption data, an airline can adjust its flight procedures like the position of its control surfaces to improve fuel efficiency11. These are examples of the “Internet of Things” (IoT), where devices communicate with each other, through a gateway, micro data center, and or a cloud data center, ultimately adding value to our lives and our businesses.
With this backdrop, it’s easier to see how data centers are fertile ground for improvements, made possible through the trends described in this paper and IoT in general. We see the following evolutions in operations and maintenance occurring over the coming years inside small and large data centers alike.
The term “network effect” gained widespread awareness during the rise of Facebook as a leading social network platform. The term basically means that as more people use a particular product or service, the more value users of that product or service will realize. The telephone is an often-used example of the network effect. If only one person in the world had a phone, there would be no value in it because they couldn’t talk to anyone else. But when millions of people have and use one, it becomes valuable. This is true of digital remote monitoring services.
If only one data center manager used a digital remote monitoring service like the one described in this paper, they wouldn’t get the value of data analytics and condition based maintenance. That value is attained very quickly as more data centers use the service and the collective data is analyzed to provide insights. For example, if 100,000 data centers used the service, a large percentage of these data centers are likely to have an air-cooled packaged chiller cooling architecture. With this amount of data, analytics could suggest changes to their cooling system and the estimated savings these changes will have on the energy bill.
Data centers are on a path to become more reliable and efficient through the use digital remote monitoring and condition based maintenance made possible through technologies like big data and machine learning. However, this can only happen with platforms that take advantage of the data constantly generated by the physical infrastructure in a data center. Data center operators should review the digital remote monitoring requirements provided in this paper as they begin to assess their own data center evolution.
1 APC traditional remote monitoring service have been available since 2000
2 http://esmarchitecture.com/key-concepts/business-it-digital-services.html
3 http://www.merriam-webster.com/dictionary/online
4 Network operations center (NOC) is also referred to as a Service Bureau. It is the centralized function responsible for monitoring data centers.
5 2 of the Top 10 Global (technology) Risks 2015 include: Data fraud/theft and cyber attacks, cyber attacks among most likely high-impact risks (World Economic Forum, Global Risks 2015)
6 https://en.wikipedia.org/wiki/Big_data
7 http://gs.statcounter.com/#desktop+mobile+tablet-comparison-ww-yearly-2010-2016
9 https://medicineforreal.wordpress.com/2013/12/23/hear-no-evil/
10 R. P. Mahajan, Critical incident reporting and learning, p. 69
11 Porter M., Heppelmann J., How Smart, Connected Products Are Transforming Competition, 2014, pg 4
Victor Avelar is the Director and Senior Research Analyst at Schneider Electric’s Data Center Science Center. He is responsible for data center design and operations research, and consults with clients on risk assessment and design practices to optimize the availability and efficiency of their data center environments. Victor holds a bachelor’s degree in mechanical engineering from Rensselaer Polytechnic Institute and an MBA from Babson College. He is a member of AFCOM.
Addressing Cyber Security Concerns of Data Center Remote Monitoring Platforms
White Paper 239
Electrical Efficiency Measurement for Data Centers
White Paper 154
Power and Cooling Capacity Management for Data Centers
White Paper 150
Browse all white papers
whitepapers.apc.com
Browse all TradeOff Tools™
tools.apc.com
For feedback and comments about the content of this white paper:
Data Center Science Center
dcsc@schneider-electric.com
If you are a customer and have questions specific to your data center project:
Contact your Schneider Electric representative at
www.apc.com/support/contact/index.cfm
