Artificial Intelligence Generates Christmas Song From Holiday Image

Researchers from University of Toronto developed an AI system that creates and then sings a Christmas song based by analyzing the visual components of an uploaded image.

With the help of CUDA, Tesla K40 GPUs and cuDNN to train their deep learning models, the researchers trained their neural network on 100 hours of online music. Once trained, the program can take a musical scale and melodic profile and produce a simple 120-beats-per-minute melody — it then adds chords and drums.

The next step was to train their ‘Neural karaoke’ program on a collection of pictures and their captions to learn how specific works can be linked to visual patterns and objects – once fed an image, the program can compile relevant lyrics and sing them.

Neural Story Singing Christmas from Hang Chu on Vimeo.

“We are used to thinking about AI for robotics and things like that. The question now is what can AI do for us?” said Raquel Urtasun, an associate professor in machine learning and computer vision at Toronto’s computer science lab. “You can imagine having an AI channel on Pandora or Spotify that generates music, or takes people’s pictures and sings about them,” adds her colleague, Sanja Fidler. “It’s about what can deep learning do these days to make life more fun?”

Read more >

About Brad Nemire

Brad Nemire
Brad Nemire is on the Developer Marketing team and loves reading about all of the fascinating research being done by developers using NVIDIA GPUs. Reach out to Brad on Twitter @BradNemire and let him know how you’re using GPUs to accelerate your research. Brad graduated from San Diego State University and currently resides in San Jose, CA. Follow @BradNemire on Twitter