MRM Highlights
All Posts
If you believe that spending time on reproducibility makes SENSE, we encourage you to join the ISMRM reproducible research study group’s 2019 challenge! As the challenge unfolds, so will sub-Nyquist aliasing artifacts. Have we dropped enough hints about the MRM paper selected to be reproduced?
Yes, you probably guessed right. The paper selected for the 2019 reproducibility challenge is:
Klaas P. Pruessmann, Markus Weiger, Peter Börnert, Peter Boesiger. Advances in sensitivity encoding with arbitrary k-space trajectories. Magn Reson Med. 2001 Oct;46(4):638-51.
Now, let’s cut to the chase and take a look at the algorithm that participants will be asked to implement:
Below is the challenge description, as circulated to the reproducible research study group mailing list:
The goal of this initiative is to select seminal papers from our field, and ask the community to reproduce the core findings/algorithms/implementations from these papers. The main motivation behind this initiative is to:
- Over time, create a library of standard reference implementations that can be used for comparison when publishing new methods.
- The ISMRM is currently creating educational content by curating educational lectures from past annual meetings and assembling them into online courses (https://www.ismrm.org/online-education-program/). An accompanying repository of reference implementations would be valuable additional content that the ISMRM research community could provide.
- Compare the individual submissions in terms of consistency of results, computation time and hardware/programming language requirements. Note: This is not to be intended to be a “ranking” of the submissions as would be done in a competition, but as an assessment of their variability.
- This initiative can be a great opportunity for students and trainees to gain additional visibility, especially if they come from smaller labs and countries where they do not have the opportunities to go to every ISMRM meeting and workshop to present their work to the research community.
First round
As proposed after the ISMRM 2018 meeting, the paper we selected for the first round is: [1] Klaas P. Pruessmann, Markus Weiger, Peter Börnert, Peter Boesiger. Advances in sensitivity encoding with arbitrary k-space trajectories. Magn Reson Med. 2001 Oct;46(4):638-51. The study group has reached out to Dr. Pruessmann, he has supported this initiative.
Provided data and material
We will provide two example datasets, brain (12 receive channels, 96 radial projections) and cardiac (34 receive channels, 55 radial projections), from a radial trajectory acquired with multi-channel coils. The data is provided in the h5 format, and we are following the conventions of the BART toolbox regarding array dimensions of the raw data [1, Readout, Spokes, Channels] and the trajectory [3, Readout, Spokes] where the first dimension encodes the k-space coordinate (for 2D acquisitions the third coordinate is always zero) and the unit of measurement is 1 / FOV. Figure 1 shows plots of the raw k-space data from one coil and a gridding sum-of-squares example reconstruction of the data.
Figure 1: Raw k-space data from one coil and a gridding sum of squares example reconstruction of the provided brain (left) and cardiac (right) data
Brain data: rawdata_brain_radial_96proj_12ch.h5: rawdata: [1, 512, 96, 12], trajectory: [1, 512, 96]
Cardiac data: rawdata_heart_radial_55proj_34ch.h5: rawdata: [1, 320, 55, 34], trajectory: [1, 320, 55]
File-IO code in Matlab and Python is provided as well. The original approach in [1] used a body coil reference image to estimate coil sensitivities. For our data, no body coil reference image is available. Participants are encouraged to use their personal favorite coil sensitivity estimation approach. As the default, we recommend dividing the individual channel images by the sum-of-squares combination.
Download locations
http://wwwuser.gwdg.de/~muecker1/rrsg_challenge.zip https://cai2r.net/sites/default/files/software/rrsg_challenge.zip
Guidelines for submissions
The goal of every submission is to produce the following items:
- An implementation of the algorithm in Figure 1 of [1]. Any programming language, as well as third-party software (e.g. implementations of the Non-Uniform Fourier transform or the Conjugate Gradient Algorithm) is fine, as long as the source code can be shared and the computational results can be reproduced.
- Subsample the provided brain data by factors 2,3 and 4 (48, 32 and 24 projections), do a reconstruction, calculate the error to the 96 projections reconstruction and plot the error and the number of iterations according to the results in Figure 4 in [1].
- For the same data, show reconstruction results for different iteration numbers and a version from a single receive coil according to the results in Figure 5.
- Reconstructions of the provided cardiac data using the first 55, 33, 22 and 11 projections corresponding to the cardiac results in Figure 6.
We prefer that participants create an online code repository (e.g. GitHub) where the submission is uploaded. However, submissions via email attachment, via a shared link to a cloud drive etc. are also acceptable.
Participants should submit their reconstruction code, scripts to perform the experiments as well as the plots described above to:
rrsg.challenge@gmail.com
What happens after the submission?
This is not a competition. We will not be “scoring” or “ranking” any of the submissions at the annual meeting. The goal will be to discuss the experiences the community had reproducing the results from the paper. For example, were there any particular roadblocks and challenges, how consistent are the submissions, etc.
- Submission repositories will be highlighted on a special section for the reproducibility challenge on the ISMRM code webpage, MR-Hub (https://www.ismrm.org/MR-Hub/).
- Participants will get a “Supporting Reproducibility” ribbon at the 2019 annual ISMRM 2019.
- Participants will be given the opportunity for a 3-min time slot at the study group meeting at the 2019 ISMRM annual meeting to present their results and share their experiences (optional).
Timeline
Material sent out and submission window opens: March 28th 2019
Submission deadline: May 1st 2019
Discussion of submissions at ISMRM annual meeting 2019: May 15th 2019
