How to Scale Up Your Deep Learning at GTC

Whether you are using GPU clusters in the cloud or using your own data center to train deep neural networks, leading deep learning frameworks rely on NVIDIA’s Deep Learning SDK libraries to accelerate the massive amount of computation required to achieve high accuracy with deep learning.

NVIDIA NCCL uses automatic topology detection and intelligent communication ring formation to move data across GPUs to perform reductions. Attend Sylvain Jeaugey’s talk (S7155) at GTC to learn more about inter-node communication using sockets or infiniband verbs with NCCL

To learn more about the cuDNN and NCCL libraries that deliver high-performance training of neural networks across many multi-GPU systems check out the following talks, instructor-led labs and experts sessions at the GPU Technology Conference (GTC) next week about strategies you can employ to train models at scale to converge to your solution faster.

Talks

 S7155 – OPTIMIZED INTER-GPU COLLECTIVE OPERATIONS WITH NCCL

S7601 – CAFFE2: A NEW LIGHTWEIGHT, MODULAR, AND SCALABLE DEEP LEARNING FRAMEWORK

S7554 – DEEP LEARNING APPLICATION DEVELOPMENT ON MULTI-GPU/ MULTI-NODE ENVIRONMENT

S7815 – BUILDING SCALE-OUT DEEP LEARNING INFRASTRUCTURE: LESSONS LEARNED FROM FACEBOOK A.I. RESEARCH

S7600 – CHAINERMN: SCALABLE DISTRIBUTED DEEP LEARNING WITH CHAINER

S7803 – DISTRIBUTED TENSORFLOW

S7569 – HIGH-PERFORMANCE DATA LOADING AND AUGMENTATION FOR DEEP NEURAL NETWORK TRAINING

Instructor-led labs

 L7128 – DIY DEEP LEARNING: A HANDS-ON LAB WITH CAFFE2

L7104 – DEEP LEARNING USING MICROSOFT COGNITIVE TOOLKIT

Connect with the expert sessions:

 H7131 – NCCL

H7122 – FRAMEWORKS FOR TRAINING DEEP NEURAL NETWORK

H7125 – CONNECT WITH THE EXPERTS: ADVANCED DEEP LEARNING

Recordings for all the talks will be available online after the conference.

About Brad Nemire

Brad Nemire
Brad Nemire is on the Developer Marketing team and loves reading about all of the fascinating research being done by developers using NVIDIA GPUs. Reach out to Brad on Twitter @BradNemire and let him know how you’re using GPUs to accelerate your research. Brad graduated from San Diego State University and currently resides in San Jose, CA. Follow @BradNemire on Twitter