Deploying High Density Rack Power to Support High Performance Computing in CityU
City University of Hong Kong (hereafter CityU) aspires to become a leading global university, excelling in research and professional education. To address the growing needs arising from our strategic interdisciplinary research areas as laid out in our Strategic Plan 2020–2025, the Computing Services Centre (CSC) had built a centralised High Performance Computing (HPC) system (named CityU Burgundy) to provide state-of-the-art capability in computationally intensive research areas. It was first introduced in September 2021. Researchers from various disciplines had heavily used the HPC resources since the launch of the facility. In fact, some researchers had even decided to contribute their research funding to further expand the capability of the CityU Burgundy.
The HPC IT workloads that spanned across eight racks with a power consumption of 15kW each (a total of 120kW) were protected by a 200kW Uninterruptible Power Supply (UPS) System. Along with the proliferation of compute-intensive workloads, the rack power requirements have also gone up. In order to fully utilize the precious University Space, the CSC has deployed high-density racks that can support 50kW IT workloads per rack to replace the existing ones. The new high-power racks can support more powerful servers that provide the overall theoretical peak performance up to 1.7PFLOPS.
High-density racks are not common. High power consumption will generate a lot of heat energy and in turn, requires stable cooling facilities. To the best of our knowledge, CityU is the first institution in Hong Kong to build a data centre with 50kW power per rack. The rack was designed to provide electric power with two power feeds, one from Normal and the other from UPS. There are sufficient intelligent PDUs to give the redundancy power supply to each compute HPC node. Each 50kW cabinet is equipped with an internal fire compression unit to protect the whole data centre even if there is a fire incident inside the high-density 50kW rack. The power consumption of each power outlet can be monitored remotely via the management network by the Data Centre Infrastructure Management (DCIM) system. The Operations team in the Network Operations Centre can monitor the system status and receive alerts if there is over-usage on the power consumption.
This is just the beginning. The CSC will continue to optimise its high-density data centre for HPC processing-intensive applications. The key is to evaluate the current infrastructure from time to time and understand which technology will work best for them.