Agustinus (Gus) Nalwan was awarded the Jetson Project of the Month for his interactive AI bot, Qrio. This bot, running on the NVIDIA Jetson Nano, can ask for a toy, identify and state its name and play videos related to it. Gus was inspired by the curiosity of his toddler and fondly named the bot, Qrio (a clever combination of the words: ‘question’ and ‘curiosity’).
The functionality of the bot is divided among four modules: Sight (for toy/object detection), Visual Presence (for the avatar of the bot), Speech (for interactive prompts and responses), Video Search and Play (for playing back relevant videos) and Coordinator (for managing all the interactions among the modules).
Gus chose the SSDLite MobileNet-V2 architecture for the object detection model. This model, used in the Sight module, was trained using an AWS EC2 Deep Learning AMI running on NVIDIA V100 GPU and deployed on Jetson Nano using TensorFlow.
Furthermore, Gus found Jetson Nano’s GPU acceleration support useful to render the bot’s avatar in real-time. He has integrated several other technologies such as Amazon Polly for text-to-speech support to interact with the user.
Qrio in action – calling out, identifying and playing videos related to the toy in sight
Gus has identified areas to improve Qrio, such as adding support for more toys and using Jetson Xavier NX for better performance. For now though, the adorable Qrio has won over Gus’s toddler and us.
For code and details of the project, visit the Jetson Community Projects page.