Skip to content

Reproducing the analysis

We provide features and predictions extracted from each of the models investigated in this study to allow reproduction of our analyses as well as to allow further exploration of the feature space generated by the FM.

Downloading the model outputs

All the model outputs are stored on hugging face at https://huggingface.co/surajpaib/FMCIB/tree/main

To download these locally, you can run the following 

pip install -U "huggingface_hub[cli]"

huggingface-cli download surajpaib/fmcib outputs outputs # Run in the root of the repo

This will download all the outputs to a local folder outputs.

The contents of the outputs folder are as below,

  • The features folder contains features extracted from all the feature-based models (our pre-trained FM + SOTA baselines) on each of the three tasks. In addition to the baselines, features for pre-training comparisons performed on task 1 are also available.
  • The predictions folder contains predictions made on each of the three tasks for our foundation model adaptation approaches, supervised baselines and pre-trained SOTA baselines. Predictions are also available for different data percentages used while fine-tuning for task 1 and task 2. These are indicated by a _x_percent suffix to the csv.
  • The stability folder is specific to the analysis of input and test-retest stability and contains features and predictions from the pre-trained FM and our supervised finetuned baseline.

Analysis

To support complete transparency of our work and allow you to have a better understanding of our analysis pipelines, the provided notebooks, with detailed walk-throughs, will help explore our analysis as well as reproduce the figures that are included in our research paper.

The analysis notebooks are organized as follows,

analysis
└── task1.ipynb
└── task2.ipynb
└── task3.ipynb
└── stability.ipynb

Detailed walk-throughs are provided in the notebooks. These notebooks reproduce Figures 2, 3, 4 and several Extended Data Figures. Please ensure you download additional dependenices as described in the data section and that you have downloaded all the outputs from our pipelines.