Data

All the data are kindly provided by the Cardiff University Brain Research Imaging Centre (CUBRIC).

Cubric_logoCardiff_logo

Training Data

The training data will be provided by the organisers. Participants will not be restricted to use solely such data, as they will be free to exploit other data at their hand for the training. However, all teams will be asked to perform at least one submission using only the training data provided by the organisers, in order to ensure one fair comparison among all algorithms. 

The training data provided will consist of multi-vendor diffusion-weighted (DW) scans from 10 healthy subjects, in the form of DWI magnitude images that underwent minimal preprocessing. For each brain data set, 4 sets of DW images will be provided, as listed below.

A) Standard (ST) Protocol, scanner 1: 3T Siemens Prisma, isotropic resolution of 2.4 mm, TE = 89 ms, TR = 7.2 s,  30 directions at b = 1200, 3000 s/mm2
B) Standard (ST) Protocol, scanner 2: 3T Siemens Connectom, isotropic resolution of 2.4 mm, TE = 89 ms, TR = 7.2 s,  60 directions at b = 1200, 3000 s/mm2
C) State-of-the-art (SA) protocol, scanner 1: 3T Siemens Prisma, isotropic resolution of 1.5 mm, TE = 80 ms, TR = 7.2 s, 60 directions at b = 1200, 3000 s/mm2 (b = 0 images with same TE and TR as B are also provided); 
D) State-of-the-art (SA) protocol, scanner 2: Siemens Connetcom, isotropic resolution of 1.2 mm, TE = 68 ms, TR = 5.4 s, 60 directions at b = 1200, 3000 s/mm2 (b = 0 images with same TE and TR as C are also provided).

In addition, routine structural scans will be acquired with each MRI scanner for anatomical depiction. The data is available from May 14, 2018.

Test Data

The test data will consist of scans from 5 additional healthy volunteers who were not included within the training set. Training and test data will be allocated at random from the overall set of DW scans available to the organisers from 15 healthy controls. 

The test data will consist of DW data sets acquired with scanner 1 (3T Siemens Prisma) according to the standard protocol (protocol A listed above), alongside the anatomical scans acquired with the same scanner. 

Preprocessing

The provided data has been processed using the following procedures:

Siemens Prisma: the b0 volumes were corrected for EPI distortions by applying FSL TOPUP on reversed phase-encoding pairs (Andersson et al.,NeuroImage 20(2):870-888,2003). The rest of the data was corrected for eddy current distortions, subject motion, and EPI distortions with FSL TOPUP/eddy. The corrected Prisma SA FA was affinely registered to the Prisma ST FA (derived from the respective dwi.nii) with FSL FLIRT and appropriate b-matrix rotation.

Siemens Connectom: the b0 volumes were corrected for EPI distortions by applying FSL TOPUP on reversed phase-encoding pairs (Andersson et al.,NeuroImage 20(2):870-888,2003). The data was corrected for eddy current distortions, subject motion, EPI distortions, and gradient-nonlinearity distortions (Glasser et al.,Neuroimage 80:105–124,2013) with FSL TOPUP/eddy and in-house software kindly provided by MGH. The corrected Prisma SA FA was affinely registered to the Prisma ST FA (derived from the respective dwi.nii) with FSL FLIRT and appropriate b-matrix rotation.

Tasks for teams

From the data set A (the only protocol that will be given for the test data), teams submitting to the challenge will predict data sets (i.e. raw DW images) B, C, D and E. Specifically, teams will face two tasks:

  1. (Harmonisation) predict data acquired with the same protocol: predict protocols B given protocol A;
  2. (Enhancement) enhance spatial and angular resolution: predict protocols C and D given protocol A.

Important information regarding submissions:

  • teams will submit the predicted DW images, given the diffusion encoding protocol, within a given brain mask that excludes the cerebellum;
  • the release of the training data includes a pre-processing step that warps scans to a subject-specific reference space. However, participants can apply their own spatial normalisation step (For example using Elastix, FSL FLIRT, or MRtrix mrregister), and the data prior to spatial normalisation is available upon request (e-mail: cdmri18@cs.ucl.ac.uk);
  • submissions will be in NIFTI format (the same format will be used for all training and test data);
  • teams can enter the challenge with various submissions; however, at least one submission should be obtained using soley the data provided by the organisers for traning.

The data from protocols B, C and D of the test set will be available to the organisers to evaluate quantitatively the goodness of the submissions.

Reference Standard

The reference that will be used for the challenge will consist of the scans comprising data sets B, C and D, mentioned above. Such scans will not be released to the public within the test data and will act as a gold standard. 

The goal of the challenge will be the prediction of data B, C and D from the standard protocol (A), given the set of examples (i.e. the training data). Teams will predict the DW data within a brain mask provided by the organisers, which does not include the cerebellum.

Evaluation

The metrics for the evaulation will assess accuracy and precision of the predictions, and will be evaluated within 3D sliding windows and within different brain regions-of-interest. The evaluation will NOT include the edges of the brain and will NOT include the cerebellum.

Practically, accuracy and precisions will be evaluated looking at the residuals of clinically relevant metrics such as Fractional Anisotropy (FA) and Mean Diffusivity (MD) from Diffusion Tensor Imaging (Basser PJ et al., Biophysical Journal 1994), the Rotation Invariant Spherical Harmonic (RISH) features (Mirzaalian H et al., NeuroImage 2016) of the raw DW signal residuals will also be calcualted, the Mean Kurtosis (MK) measure (Jensen JH et al., Magn. Reson. Med. 2005), and the Return-To-Origin Probability (RTOP) measure (Özarsalan E et al., NeuroImage 2014). FA, MD, RISH, MK and RTOP will be calculated by the organisers from the DW images submitted by the teams.

Submissions will be ranked according to the the quality-of-the-prediction for metrics such as FA, MD, RISH, MK and RTOP features in turn.

Results

The challenge is over. A preliminary result of the challenge was presented at ISMRM 2019, Montréal, Canada. The abstract is avaliable at https://index.mirasmart.com/ISMRM2019/PDFfiles/0771.html. Moreover, a book chapter of the challenge was published in the proceddings of CDMRI. Some sample code used for evaluations and the results can be downloaded via the link.

mushac_evaluation.zip5.51 MB