The last time I visited China several years ago, most of the HPC work was centered in the realm of scientific data processing. In November 2010, the Tianhe-1A system, with 7168 NVIDIA GPUs, had just been named the fastest supercomputer in the world on the Top500 list and outside of a few university research projects, convolutional neural networks were virtually unknown. What a difference a few years makes. While systems like Tianhe-1A are still being used extensively for scientific data processing, much of the growth of China’s HPC industry is centered around new commercial uses of HPC and NVIDIA GPUs, especially in the fast growing machine learning space.
Today, companies across China like Internet giant Baidu, are using NVIDIA GPUs and high performance computing to drive their deep learning projects as discussed last month at the GPU Technology Conference by Ren Wu, a distinguished scientist at Baidu in his GTC talk S4651 – Deep Learning Meets Heterogeneous Computing. Any software developer in the world can study up on machine learning using online courses like Stanford professor’s Andrew Ng online Coursera course. Across China, Internet companies seem to be just as active at hiring machine learning experts as US giants Facebook and Google.
Speaking with the Chinese leader of NVIDIA’s acclaimed “DevTech” team of ninja CUDA programmers reinforces the message. With almost rock star like status at his alma mater Tsinghua University, Julien has no problem getting master’s degree and PhD graduates to apply for new DevTech positions. But Julien admitted to me that of the last half dozen job offers he made to CUDA experts, over half ended up taking positions instead at Chinese Internet companies working on machine learning. I did remind him this was still a win-win for NVIDIA. So if you are top CUDA programmer in China and interested in working for NVIDIA, let me know and I will connect you with Julien. But only CUDA experts need apply, you might very well end up helping NVIDIA work with Baidu or another Chinese Internet giant, so you need to be best of the best.
Like the US, Japan, and Europe, China still has plans to build giant HPC systems like Tianhe. However, increasingly these systems are being looked at to support commercial HPC workloads like machine vision in a cloud environment in addition to just scientific data processing. As one large HPC customer told me, “there are a lot of processors that we could use for future scientific data processing, but NVIDIA is unique in being able to address our entire spectrum of commercial HPC and big data workloads”. After all, what other processor is equally good at computing discrete spatial derivatives in 3D by doing 1D convolutions for an oil company’s seismic processing reverse time migration algorithm in the morning and then running a convolution to simulate depth of field in a video game streamed from the same supercomputer in the evening. Not to mention running some convolution neural networks in between.
One evening out with the local NVIDIA solution architecture team, the entire dinner conversation was captivated by speculation on how Baidu might be improving their visual search application. Complete with many examples of the Baidu app translating for me pictures of delicious food of which I previously had no idea what I was eating. I love Chinese food although sometimes it might be best not to know what I was eating. Not the case however with these Chinese river shells, both beautiful and delicious.
Later this week, in part 2 of my China HPC series, I’ll talk about how some of the Chinese hardware companies are doing to address the growing HPC market.