Google Cloud today announced the general availability of the NVIDIA T4 GPU, making Google Cloud the first provider to offer the GPUs globally. The NVIDIA T4 GPUs are ideal for machine learning training and inference, high performance computing, data analytics, and graphics applications.
“NVIDIA’s Turing architecture brings the second generation of Tensor Cores to the T4 GPU,” said Chris Kleban, Product Manager at Google Cloud. “This is great for large training workloads, especially as you scale up more resources to train faster, or to train larger models.”
Besides training, the NVIDIA T4 GPUs are also ideal for accelerating inference on machine learning models, reducing latency and delivering high throughput.
“When Tensor Cores are enabled with mixed precision, T4 GPUs on GCP can accelerate inference on ResNet-50 over 10X faster with TensorRT when compared to running only in FP32,” Kleban said.
The NVIDIA T4 GPUs have 16 GB of memory each, offering a range of precision support including FP32, FP16, INT8, and INT4.
Companies and organizations like Snap and Princeton are turning to the NVIDIA T4 GPUs for inference, to accelerate their services and research.
“We are excited to partner with Google Cloud on a landmark achievement for neuroscience: reconstructing the connectome of a cubic millimeter of neocortex. It’s thrilling to wield thousands of T4 GPUs powered by Kubernetes Engine. These computational resources are allowing us to trace 5 km of neuronal wiring, and identify a billion synapses inside the tiny volume,” said Sebastian Seung, a computer science and neuroscience professor Princeton University.
“Snap’s monetization algorithms have the single biggest impact to our advertisers and shareholders. NVIDIA T4-powered GPUs for inference on GCP will enable us to increase advertising efficacy while at the same time lower costs when compared to a CPU-only implementation,” said Nima Khajehnouri, Sr. Director, Monetization, Snap Inc.
The NVIDIA T4 GPUs are also great for running virtual workstations. With the NVIDIA Quadro Virtual Workstations system in GCP, developers can run their applications on the NVIDIA RTX platform which enables real-time ray tracing, video and image processing, and AI-enhanced graphics.
Here’s what software developer manger at Autodesk said about the virtual workstations powered by the NVIDIA T4 GPUs.
“Access to NVIDIA Quadro Virtual Workstation on the Google Cloud Platform will empower many of our customers to deploy and start using Autodesk software quickly, from anywhere. For certain workflows, customers leveraging NVIDIA T4 and RTX technology will see a big difference when it comes to rendering scenes and creating realistic 3D models and simulations. We’re excited to continue to collaborate with NVIDIA and Google to bring increased efficiency and speed to artist workflows., said Eric Bourque, a Senior Software Development Manager at Autodesk.
Right now, the NVIDIA T4 instances are priced at $0.29 per hour per GPU on preemptible VM instances, the Google team said. On-demand instances start at $0.95 per hour per GPU.
