This week, a high schooler from Seattle, Washington released an iPhone app that uses deep learning to identify what you’re looking at and then displays matching or similar products.
The SmartLens mobile app can identify over 17,000 objects such as fruits, plants, household items, office products, animals, tools, and foods. When the app detects an item, it delivers product suggestions or facts about the item.
Michael Royzen, the app’s creator, is only 18 years old, and SmartLens is his eighth app on Apple’s App Store.
“SmartLens turns your iPhone’s camera into a search box,” Royzen wrote in a product description. “Have you ever seen a cool product that you wanted to buy instantly? Have you wondered about the breed of the dog you just saw? SmartLens, a search engine for the visual world, will give you an answer quickly and seamlessly.”
Using NVIDIA Tesla GPUs on the Amazon Web Services cloud and the cuDNN-accelerated TensorFlow deep learning framework, the teen trained his system on 13 million images of everyday products and objects. Royzen says he’s now using Tesla V100 GPUs for additional training.
In addition to identifying everyday products, the app can also identify businesses and landmarks.
For inference, the algorithm first runs on the phone. If the app determines that a product or business is being recognized, additional cloud-based models, using the same GPUs used during training, are performed on the image.
“SmartLens will give you useful information about nearly anything you point it at,” Royzen said. If SmartLens recognizes a product, it will enable you to buy it in one tap. If it identifies an animal, landmark, food, or painting, it will show you a Wikipedia description. If it recognizes a business, it will show you reviews.”
It’s worth pointing out that SmartLens is not perfect. It often takes the app several tries to correctly identify an item, and it struggles to recommend specific products for branding-ambiguous tech products. However, on packaged products and books, the app detects the exact product almost every time. It’s remarkable a teen, who developed the app in his spare time after school, has gotten the app to where it is today.
The app is available for free on the App Store via a 30-day trial. Once the trial period ends, users can pay $1.00 per month or $9.99 per year.
The teen says he’s continuing to improve his algorithm every day and is excited to see where the app goes from here.
Read more >
