A new neuroimaging foundation model called NeuroVFM has demonstrated that self-supervised learning on uncurated hospital data, without radiologist reports, disease labels, or manual curation, can outperform language-supervised models on 156 diagnostic tasks, including prospective triage. The work is published in Nature Medicine, with code and model weights released on GitHub.
NeuroVFM is built on Vol-JEPA, a self-supervised algorithm that extends Meta’s JEPA (Joint-Embedding Predictive Architecture) from 2D images to 3D medical volumes. Instead of reconstructing individual voxels, the 3D equivalent of pixels, Vol-JEPA learns by predicting missing latent representations. The model is trained on masked portions of each volume, using visible context patches to predict the content of larger masked target regions. This approach requires no labels, no radiology text, and no voxel decoder.
The training dataset, called UM-NeuroImages, comprises 5.24 million clinical MRI and CT volumes from 566,915 studies collected over more than 20 years at Michigan Medicine. The data is uncurated, it includes routine clinical scans of varying quality, from multiple scanner manufacturers and protocols, making it representative of real-world hospital data rather than a cleaned research corpus.
The results are notable for both performance and efficiency. On the primary endpoint of macro-averaged AUROC across 156 diagnostic tasks, 74 MRI and 82 CT, NeuroVFM achieved 92.49 on MRI and 92.68 on CT. It outperformed models trained with language supervision (HLIP, PRIMA), voxel reconstruction (NeuroMAE), and 2D self-supervised approaches (DINOv3, BiomedCLIP) by margins of 1 to 4 points. Training required fewer than 1,000 GPU hours, more than seven times faster than a comparable 3D self-supervised baseline.
The frozen visual representations from NeuroVFM can be reused for multiple downstream tasks without fine-tuning. A diagnostic head predicts 156 conditions from the embeddings. A vision-language variant called NeuroVFM-LLaVA, pairing the frozen encoder with a Qwen3-14B language model, generates structured radiology-style findings.
A prospective silent study over one week in a health system, involving 1,155 studies, evaluated NeuroVFM-LLaVA for clinical triage. It achieved 92.6% balanced triage accuracy, substantially higher than GPT-5 at 71.2%, and missed 21 out of 155 critical findings, a miss rate of 13.5% compared to GPT-5’s 50.3%. The authors frame the system as decision support rather than autonomous screening.
The model also demonstrates cross-modal generalisation: a diagnostic probe trained only on CT volumes and evaluated on MRI suffered less than a 5-point AUROC drop, suggesting the model learns modality-invariant representations of neuroanatomy and pathology.
Sources: Meet NeuroVFM: Neuroimaging Foundation Model with Vol-JEPA (MarkTechPost, Jul 12, 2026); NeuroVFM on GitHub