Sustaining vs Disruptive Innovation in HPC

Amazon's James Hamilton (no relation) recent blog on microservers didn’t use the term disruptive innovation but he certainly described it well in talking about AMD’s recent ARM announcement. All innovation is good, and at HP you can certainly find many examples of both sustaining and disruptive innovation in HPC.

A good example of sustaining innovation is how HP supports up to 3 GPUs or co-processors in a 1 rack unit size server with the ProLiant SL250s. Most servers of this size support a maximum of 2 GPUs. Furthermore, the SL250s provides full PCIeGen3 support to compatible GPUs versus other vendors who’s designs are still based on PCIeGen2.

But what about microservers for HPC? Sure, one year ago at SC11 the Mont-Blanc project announced their plans to build a hybrid ARM-GPU supercomputer, but most HPC customers today consider microservers as not powerful enough for HPC. While being underpowered is not a sufficient condition to create disruptive innovation, it is a hallmark of nearly every disruptive innovation the HPC industry has ever seen.

Ten years ago, there were only two x86 based supercomputers in the top 10 of the November 2002 Top500 list, both from long gone white box vendors. The rest of the top 10 was rounded out with 4 HP Alpha systems (including #2 and #3 on the list), NEC’s Earth Simulator at #1 with an NEC processor, and 3 IBM Power systems. If you wanted a mainstream HPC system in 2002, x86 was not really in your shopping list. When the November 2012 Top500 list is announced next week, the vast majority of systems will include x86 processors (albeit a growing number will no doubt also include GPUs and co-processors). The advent of x86 in the HPC market was clearly a disruptive innovation.

One can’t predict what processors will be used in the 2022 Top500 list but without discussing HPC specifically, James Hamilton’s analysis makes a strong case for ARM and microserver architectures. Three of James’ points are:

1) Not [all workloads are] CPU Bound
2) Price/Performance
3) Power/Performance (or performance/watt)

Lets take a look at how these apply to the HPC space.

1) Not [all workloads are] CPU Bound
While certain HPC kernels like HPL have been optimized for years on the x86 architecture and are almost 100% CPU bound, the average HPC workload is often memory or network bound. And especially as new technologies like OpenACC gain developer support and make it easier to program GPUs as well as other many-core processors, your typical HPC server is likely to get more memory and network bound and less CPU bound because memory and network performance is simply not growing as fast as processor performance.

2) Price/Performance
Even large countries have limits on how much they can spend on national HPC systems and the scale of HPC means price/performance will always be relevant. Price/performance is why x86 saw adoption in the HPC world over the last decade and now dominates the space. But as James Hamilton points out, the volume economics have changed, there were 7.6M server units sold in 2010. Assume an average two x86 processors per server unit and compare that to what ARM reports as 6.1B ARM processors shipped last year. That’s a “M” for x86 and a “B” for ARM. While ARM is not necessarily optimized for HPC workloads today, with 100’s of ARM licensees including the likes of AMD and Nvidia, how long before that happens? The pending adoption of the 64 bit ARM architecture, with multiple vendors planning to sample 64 bit ARM silicon in 2013, will only accelerate the price/performance of ARM.

3) Power/Performance (or performance/watt).
With its roots in the mobile space, this is where ARM has a significant advantage today, even on certain types of HPC workloads. While the x86 vendors aren’t standing still, ARM appears to have a clear lead in Power/Performance, especially when coupled with GPUs or other co-processor cores.

Of course microservers don’t equate with ARM only. In fact, HP’s Project Moonshot, has already announced plans to use Intel’s Atom processor in addition to ARM and other processors. But what are some of the longer term reasons that HPC servers will look more like microservers than today’s two-socket x86 processors?

Microservers are typically single socket servers, i.e. they only use 1 processor. As Moore’s law continues to provide more and more transistors in a single chip, it is inevitable that by the end of the decade, if not much sooner, most HPC servers will move to single socket. The reason is simple. The cost, in terms of power and latency, of moving data between two processors, coupled with Moore’s law, will make it more efficient to build single socket systems.

Another trend that seems almost inevitable is that the network interface card or chip (NIC) will move onto the processor die. It doesn’t take a genius to look at some of Intel’s recent investments in networking to figure this out. In fact, it is already happening in today’s microservers. HP’s Redstone server development platform uses the Calxeda ARM processor, which is really a SOC (system on chip) and includes four ethernet NICs built into the processor. AMD also recently announced they plan to build their Sea-Micro fabric technology into their ARM processors.

However, true to the nature of disruptive technologies, you are not likely to see any ARM or other microservers on the Top500 list next week. But HP’s Project Moonshot certainly lays the groundwork for future disruptive HPC solutions based on ARM and microservers. While the first Moonshot customers are planning to use the product for lighter weight web applications, as the underlying technologies such as ARM processors and SOCs develop, the Moonshot platform is well positioned for other uses. We will start talking about some of these to our customer’s this evening at the HP-CAST event in Salt Lake City, and I’m really looking forward to the November 2022 Top500 list, not to mention the one that comes out next week and all the one’s in between.


About Marc Hamilton

Marc Hamilton – Vice President, Solutions Architecture and Engineering, NVIDIA. At NVIDIA, the Visual Computing Company, Marc leads the worldwide Solutions Architecture and Engineering team, responsible for working with NVIDIA’s customers and partners to deliver the world’s best end to end solutions for professional visualization and design, high performance computing, and big data analytics. Prior to NVIDIA, Marc worked in the Hyperscale Business Unit within HP’s Enterprise Group where he led the HPC team for the Americas region. Marc spent 16 years at Sun Microsystems in HPC and other sales and marketing executive management roles. Marc also worked at TRW developing HPC applications for the US aerospace and defense industry. He has published a number of technical articles and is the author of the book, “Software Development, Building Reliable Systems”. Marc holds a BS degree in Math and Computer Science from UCLA, an MS degree in Electrical Engineering from USC, and is a graduate of the UCLA Executive Management program.
This entry was posted in Uncategorized. Bookmark the permalink.