Data science workflows are inherently complex. They scale across clusters of servers running software from different parts of the workflow and they are often compute-intensive. All this results in slow machine learning model development and deployment cycles.
To help accelerate end-to-end data science training, NVIDIA developed RAPIDS, an open-source data analytics and machine learning acceleration platform designed exclusively for GPUs. As a step towards wider integration, the Kubeflow team announced the availability of the NVIDIA RAPIDS GPU-accelerated libraries as an image on the Kubeflow Pipelines.
Kubeflow is a cloud native platform for machine learning built on top of Kubernetes. The platform reduces machine learning production by providing end-to-end workflows in an environment that can be easily scaled to production.
Kubeflow also integrates a collection of Google developed frameworks that allow data scientists and ML developers to build end-to-end pipelines.
“The integration of RAPIDS with Kubeflow Pipelines streamlines the model development workflow and drastically decreases end-to-end model iterations times by automating the deployment of open GPU-accelerated data science tools,” the Kubeflow team wrote in a blog post. “Combining the simple orchestration of machine learning pipelines with RAPIDS, a collection of CUDA-accelerated libraries, data scientists can train and deploy machine learning pipelines significantly faster to solve business problems.”