This paper introduce a new visual-omics foundation model called OmiCLIP, designed to integrate histopathology images and spatial transcriptomics data for computational biology. OmiCLIP was trained on a curated dataset, ST-bank, which consists of over two million paired tissue images and transcriptomic data across 32 organs, by converting gene expression into "sentences" for language model processing. Building on OmiCLIP, the researchers developed the Loki platform, an infrastructure that offers five key functions: tissue alignment, annotation using bulk RNA sequencing or marker genes, cell-type decomposition, image–transcriptomics retrieval, and spatial gene expression prediction. The platform consistently demonstrated superior accuracy and robustness compared to numerous existing state-of-the-art models across various simulation and experimental datasets. By bridging tissue morphology with genomic insights, this framework aims to streamline workflows and reduce the cost and complexity of high-resolution tissue studies, particularly in 3D tissue analysis.
References:
