Researchers from ATR Computational Neuroscience Laboratories and Kyoto University in Japan developed a deep learning-based algorithm that can generate images from brain activity.
“The reconstruction algorithm starts from a random image and iteratively optimize the pixel values so that the DNN (deep neural network) features of the input image become similar to those decoded from brain activity across multiple DNN layers,” mentioned the researchers in their paper ‘Deep image reconstruction from human brain activity’. “The resulting optimized image is taken as the reconstruction from the brain activity.”
Using a GTX 1080Ti and Tesla K80 GPU with the cuDNN-accelerated Caffe deep learning framework, the researchers trained their decoders on fMRI data measured while human subjects looked at images over a ten-month period. The images included geometric shapes, natural phenomena and letters of the alphabet.
Below are the results of the algorithm.
“Our brain processes visual information by hierarchically extracting different levels of features or components of different complexities,” Kyoto University Graduate School of Informatics Professor Yukiyasu Kamitani shared about the impact of their work, “These neural networks or AI models can be used as a proxy for the hierarchical structure of the human brain.”
Read more >
Related resources
- DLI course: Building a Brain in 10 Minutes
- GTC session: Fireside Chat with Yoav Shoham and Vartika Singh: Going Beyond LLMs, the Future of AI That Understands
- GTC session: Empower Large-Scale AI Workloads With Google Cloud AI Hypercomputer Supercomputing Architecture (Presented by Google Cloud)
- GTC session: Human-Like AI Voices: Exploring the Evolution of Voice Technology
- NGC Containers: MATLAB
- Webinar: Accelerate AI Model Inference at Scale for Financial Services
About the Authors
