Just how energy efficient is NVIDIA’s new DGX SaturnV? At about 9.5 gigaflops/watt, the Pascal P100 powered system is about 50% more energy efficient than the nearest non-NVIDIA powered system. But unless you follow the Top500 list of world’s fastest supercomputers, it probably is hard for you to visualize if 9.5 gigaflop/watt is a good or not. But many have taken note of NVIDIA’s 50% jump over competitors in efficiency. Think of buying a Tesla Model X that can go 300 miles on a single charge and then hearing a competitor had released a car capable of 450 miles. You would take note.
But back to DGX SaturnV’s intended use supporting NVIDIA’s AI software development. Does energy efficiency matter there? At a recent GTC, Baidu’s Andrew Ng stated that it cost about $100 in electricity alone to train their Deep Speech 2 Deep Neural Network (DNN) for English and Mandarin speech recognition. It is not uncommon for a data scientist to train a DNN 50 or 100 times before getting good results and Baidu has 100’s if not 1000’s of data scientists. The energy savings of a Pascal P100 powered system can add up quickly. All of us involved in the rapidly growing field of AI need to take energy efficiency into account when thinking about how to power our research.
DGX SaturnV is built out of 124 DGX-1 deep learning supercomputers each in turn with 8 Pascal P100 GPUs. But you don’t need to build the world’s 28th fastest supercomputer to take advantage of the Pascal P100’s energy efficiency. Last week, NVIDIA and Microsoft announced record breaking results for Microsoft’s Cognitive Toolkit running on DGX-1, an amazing 170 times faster than a standard two-socket x86 server. And if your budget can’t afford a DGX-1, you can get started with Microsoft’s Cognitive Toolkit using Azure’s new N-series GPU virtual machines for as little as 56 cents/hour. And of course in-between you have other options including the Titan X Pascal, the most powerful GPU available for PC developers.