Abstract

Data center cooling has never been more challenging as it is today with the thermal design power of the components rising significantly in the past decade. This power growth can be primarily attributed to advancements in high power CPUs and GPUs to support high power computing applications like AI and machine learning algorithms development. In this presentation, challenges, and opportunities in cooling technologies from evaporative heat exchangers, cold plate based liquid cooling, and immersion cooling will be discussed. At the end, challenges related to new packaging architectures and the impact on data center cooling will be addressed.

Related to air cooling and energy efficiency, we have made significant improvements with the adoption air-side economization and evaporative cooling. However, it is imperative to intensify efforts for widespread adoption of such alternatives to mechanical refrigeration systems. This calls for the development of diverse predictive and prognostic models, based on the principles of thermodynamics, transport phenomena and machine learning. The data center of tomorrow calls for a holistic integration of multi-scale thermal design of reliable and energy efficient electronic systems to keep with the ever-changing data center landscape.

Thermal conduction module, one of the earliest applications of liquid cooling can be found in the IBM 3081 mainframe computer, announced in 1982, which incorporated indirect liquid cooling using water-cooled cold plates on a multi-chip module (MCM). Since then, liquid cooling has only been used in few niche applications mostly related to cost and perceived complexities in implementation. Liquid cooling, however, is starting to gain traction. Companies like Google have employed water cooling for their high-performance AI processors and Nvidia is looking into liquid cooling for their high-end GPUs and Switches. A key challenge in cooling of electronics is addressing the issue of non-uniform power and corresponding temperature distribution at the package, server, and rack levels. This can be alleviated through the introduction of a “dynamic” cold plate design. Through implementation of sensing and control, the solution can distribute the coolant based on local cooling requirements.

Single-phase immersion provides simplicity in terms of thermal infrastructure, PUEs as low as 1.07, and reduction in CAPEX equal to or greater than 50%. Immersion cooling may also prove to be an efficient solution for 3D stacked and heterogeneously packaged components because of its inherent ability to provide more uniform temperature profiles. Immersion cooling also enhances system reliability by protecting the IT equipment from the harsh environmental effects of high temperature, dust, vibrations, and corrosive gases. Enhanced equipment reliability and ease of implementation, especially for edge data centers, also make single-phase immersion cooling an attractive alternative for data centers. Material compatibility of the immersion fluids with IT components, however, still remains a challenge.

A complement to Moore’s law is Heterogeneous Integration using 2.5D and 3D packaging which enables continued system performance by allowing cost effective, high bandwidth and low power integration of diverse IP build on different nodes as no single transistor node is optimal across all design points. This also presents a cooling challenge due to the chip to chip thermal resistance and corresponding hotspots which makes it almost impossible to use air cooling. Most data centers are not prepared to address this question.

Biography

Dereje Agonafer is a Presidential Distinguished Professor in MAE at University of Texas at Arlington (UTA) where he heads two centers: Site Director of NSF I/UCRC in Energy Efficient Systems and Director of Electronic Packaging. After receiving his PhD at Howard University, he worked for 15 years at IBM. In 1991, his work was recognized by being awarded the "IBM Outstanding Technical Achievement Award in Appreciation for Computer Aided Thermal Modeling.” Since joining UTA in 1999, he has graduated 230 graduate students (a record for the University) including 25 PhDs and currently advising 16 PhDs and 13 MS students. His former students are making significant contributions in many technology companies such as Facebook, Intel, 3M, Microsoft, and Amazon. His new initiative is to start a new center called RAMPES (Center for Reliability Assessment in Micro and Power Electronic Systems) for which he has received significant funding including $1.3M for new equipment, 3000 sq ft of new lab space, Assistant and Associate Professor openings to work with him, and research engineer among others. For his contributions, he has received numerous awards including the 2008 Thermi Award, the 2009 InterPACK Excellence Award, the 2014 ITHERM Achievement Award, the 2014 NSBE Golden Torch Award and the 2019 ASME Heat Transfer Memorial Award. In 2020, he was the recipient of Howard University Charter Day Award for Distinguished Postgraduate Achievement: research engineer. Also in 2020, he received the SemiTherm Lifetime Achievement Award in “Recognition of Significant Contributions to the Field of Electronics Thermal Management.” He is a fellow of the National Academy of Inventors, the American Association for the Advancement of Science and the American Society of Mechanical Engineers. In 2019, he was elected to the National Academy of Engineering. According to Dean Crouch, “the first current faculty member elected to the Academy.”  Professor Agonafer is married to his wife Carolyn and they have two children; a son, Dr. Damena Agonafer who is Professor of Mechanical Engineering & Materials Science at Washington University in St. Louis, and a daughter, Dr. Senayet Agonafer, a Radiologist, who works at Lennox Hill Radiology in New York City.

Prof. Agonafer’s website:  http://emnspc.uta.edu

Watch Lecture

Watch Panel

Panel topic and participants