Cape Town - 2026 ISMRM-ISMRT Annual Meeting and Exhibition
9 May 2026 – 14 May 2026 · Cape Town, South Africa
271-01-025 / 501-03-002 ISMRM Abstract

Cortex-Inspired Hierarchical Reconstruction of Visual Stimuli from fMRI Signals

Primary: Analysis Methods - Multi-Modal Learning with LLMs/VLMs
Secondary: Brain Function and fMRI - Task/Intervention Based fMRI
271-01-025 · ISMRM AMPC Selected Posters · Sunday, 10 May, 7:00 AM–2:00 PM · Traditional Posters
501-03-002 · fMRI: Advanced Analysis · Wednesday, 13 May, 1:40 PM–3:30 PM · Hall 1A
Keywords: AI/ML Image Reconstruction Generative diffusion model Computational Neuroscience Multimodal Learning Contrastive Learning
Accepted
Shiyi Zhang 1,2, liu ming1, Qihui Ye3, Yanjie Zhu3, Haifeng Wang3, Dong Liang1, Hairong Zheng3, Yihang Zhou1
1Research Center for Medical AI, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
2Department of Computer Science, North Carolina State University, Raleigh, United States of America
3Paul C. Lauterbur Research Center for Biomedical Imaging, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
Presenting Author: Shiyi Zhang

Synopsis

Motivation:
Goals:
Approach:
Results:
Full abstract & presentation

The full text, figures, and any recorded presentation for this abstract are not shown here. Log in if you are a member or registered attendee with access.

Full abstracts, figures, and presentations for Cape Town - 2026 ISMRM-ISMRT Annual Meeting and Exhibition are available to registered attendees. This content becomes freely available to the public roughly two years after the meeting.

To request or purchase access, contact the ISMRM Central Office at info@ismrm.org.

Log in

References

1. Vinje WE, Gallant JL. Sparse coding and decorrelation in primary visual cortex during natural vision. Science. 2000;287(5456):1273-6. DOI:10.1126/science.287.5456.1273 [doi]
2. Ionta S. Visual neuropsychology in development: Anatomo-functional brain mechanisms of action/perception binding in health and disease. Front Hum Neurosci. 2021;15:689912. https://doi.org/10.3389/fnhum.2021.689912 [doi]
3. Xu X, Wang Z, Zhang G, et al. Versatile diffusion: text, images and variations all in one diffusion model. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023. p. 7754–65. https://doi.org/10.48550/arXiv.2211.08332 [doi]
4. Scotti P, Banerjee A, Goode J, Shabalin S, Nguyen A, Dempster A, et al. Reconstructing the mind's eye: fmri-to-image with contrastive learning and diffusion priors. Adv Neural Inf Process Syst. 2023;36:24705-28. https://doi.org/10.48550/arXiv.2305.18274 [doi]
5. Wang S, Liu S, Tan Z, Wang X. Mindbridge: A cross-subject brain decoding framework. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition; 2024. p. 11333-42. https://doi.org/10.48550/arXiv.2404.07850 [doi]
6. Allen EJ, St-Yves G, Wu Y, Breedlove JL, Prince JS, Dowdle LT, et al. A massive 7T fMRI dataset to bridge cognitive neuroscience and artificial intelligence. Nat Neurosci. 2022;25(1):116-26. https://doi.org/10.1038/s41593-021-00962-x [doi]
7. Krizhevsky A, Sutskever I, Hinton GE. Imagenet classification with deep convolutional neural networks. Adv Neural Inf Process Syst. 2012;25. https://doi.org/10.1145/3065386 [doi]
8. Tian Z, Quan R, Ma F, Zhan K, Yang Y. BrainGuard: Privacy-Preserving Multisubject Image Reconstructions from Brain Activities. In: Proceedings of the AAAI Conference on Artificial Intelligence; 2025. p. 14414-22. https://doi.org/10.1609/aaai.v39i13.33579 [doi]
9. Huo J, Wang Y, Wang Y, Qian X, Li C, Fu Y, et al. Neuropictor: Refining fmri-to-image reconstruction via multi-individual pretraining and multi-level modulation. In: European Conference on Computer Vision; 2024. p. 56-73. https://doi.org/10.1007/978-3-031-72983-6_4 [doi]
10. Lin S, Sprague T, Singh AK. Mind reader: Reconstructing complex images from brain activities. Adv Neural Inf Process Syst. 2022;35:29624–36. doi:10.48550/arXiv.2210.01769 [doi]
11. Gu Z, Jamison K, Kuceyeski A, Sabuncu M. Decoding natural image stimuli from fMRI data with a surface-based convolutional network. arXiv preprint arXiv:2212.02409. 2022. doi:10.48550/arXiv.2212.02409 [doi]
12. Seeliger K, Güçlü U, Ambrogioni L, Güçlütürk Y, Van Gerven MAJ. Generative adversarial networks for reconstructing natural images from brain activity. NeuroImage. 2018;181:775–85. doi:10.1109/SMC.2018.00187 [doi]
13. Mozafari M, Reddy L, VanRullen R. Reconstructing natural scenes from fMRI patterns using BigBiGAN. Proc 2020 Int Joint Conf Neural Netw (IJCNN). 2020:1–8. doi:10.1109/IJCNN48605.2020.9206960 [doi]
14. Fang T, Qi Y, Pan G. Reconstructing perceptive images from brain activity by shape-semantic GAN. Adv Neural Inf Process Syst. 2020;33:13038–48. doi:10.48550/arXiv.2101.12083 [doi]
15. Ren Z, Li J, Xue X, Li X, Yang F, Jiao Z, Gao X. Reconstructing seen image from brain activity by visually-guided cognitive representation and adversarial learning. NeuroImage. 2021;228:117602. doi:10.48550/arXiv.1906.12181 [doi]
16. Takagi Y, Nishimoto S. High-resolution image reconstruction with latent diffusion models from human brain activity. Proc IEEE/CVF Conf Comput Vis Pattern Recognit (CVPR). 2023:14453–63. doi:10.1109/CVPR52729.2023.01389 [doi]
17. Ozcelik F, VanRullen R. Natural scene reconstruction from fMRI signals using generative latent diffusion. Sci Rep. 2023;13(1):15666. doi:10.1038/s41598-023-42891-8 [doi]
18. Mai W, Zhang Z. Unibrain: Unify image reconstruction and captioning all in one diffusion model from human brain activity. arXiv preprint arXiv:2308.07428. 2023. doi:10.48550/arXiv.2308.07428 [doi]
19. Chang N, Pyles JA, Marcus A, et al. BOLD5000: a public fMRI dataset while viewing 5000 visual images. Sci Data. 2019;6:49. doi:10.1038/s41597-019-0052-3 [doi]

Cite this abstract

http://echo.ismrm.org/p/ISMRM2026/271-01-025