2 posts tagged with "ml"

We were delighted to see so many Huskies in attendance at the Fortieth International Conference on Machine Learning, which took place at the end of July. The researchers using Hyak are doing incredible work, and we wanted to say congratulations to those who had their papers accepted:

• Ian Connick Covert, Wei Qiu, Mingyu Lu, Na Yoon Kim, Nathan J White, Su-In Lee: Learning to Maximize Mutual Information for Dynamic Feature Selection

• Tim Dettmers, Luke Zettlemoyer: The case for 4-bit precision: k-bit Inference Scaling Laws

• Runlong Zhou, Zhang Zihan, Simon Shaolei Du: Sharp Variance-Dependent Bounds in Reinforcement Learning: Best of Both Worlds in Stochastic and Deterministic Environments

• Haotian Ye, Xiaoyu Chen, Liwei Wang, Simon Shaolei Du: On the Power of Pre-training for Generalization in RL: Provable Benefits and Hardness

• Jikai Jin, Zhiyuan Li, Kaifeng Lyu, Simon Shaolei Du, Jason D. Lee: Understanding Incremental Learning of Gradient Descent: A Fine-grained Analysis of Matrix Sensing

• Runlong Zhou, Ruosong Wang, Simon Shaolei Du: Horizon-Free and Variance-Dependent Reinforcement Learning for Latent Markov Decision Processes

• Yiping Wang, Yifang Chen, Kevin Jamieson, Simon Shaolei Du: Improved Active Multi-Task Representation Learning via Lasso

Also, special congratulations for those with accepted shadow papers:

The single biggest research use for GPUs on Hyak is for machine learning and artificial intelligence and the community has been clammoring for CUDA11 support for some time. Unfortunately, it's not easy to separate the GPU driver from the node images so it had to wait until the next maintenance window and some testing for non-ML GPU workflows on Hyak like our gromacs users in the molecular dynamics community.

tl;dr your existing Pytorch codes should work but if you wanted to use the new features in Pytorch that required CUDA11 you can upgrade Pytorch and it will work.

Installing Pytorch with CUDA11#

Since this is now the latest and greatest on Hyak I've taken the opportunity to update the Python documentation on how to install Pytorch with CUDA11 support within a miniconda3 environment, check out the step-by-step here.

Reverse compatibility with CUDA10#

Before the January 12, 2021 cluster maintenance every GPU on Hyak had a driver with CUDA10 and all of your codes were previously compiled against it. To test that the GPU driver update to CUDA11 wouldn't impact the most popular machine learning libraries we are compiling Pytorch against our pre-maintenance CUDA10 and testing it against a GPU with the newer CUDA11 installed.

Activate your new pytorch-cuda10 environment:

The Pytorch website [www] has a nice getting started matrix that generates the requisite install commands against CUDA10.

The command shown above to copy-and-paste below:

Now we can load the Python interpreter and confirm Pytorch is installed and the CUDA10 compiled library recognizes this GPU with CUDA11 [www].

Success!

Previously compiled libraries against CUDA10 from pre-January 12, 2021 maintenance times should still work on the GPUs now with CUDA11. However, if you want to use the full features of libraries that take advantage of newer capabilities in CUDA11 then you should definitely upgrade your libraries.