MRM Highlights
All Posts
By Mathieu Boudreau
The September 2020 MRM Highlights Reproducible Research Insights interview is with Henric Rydén and Stefan Skare, researchers at the Karolinska University Hospital and Karolinska Institutet in Stockholm, Sweden, and authors of a paper entitled “RARE two‐point Dixon with dual bandwidths”. This paper was chosen as the MRM Highlights pick of the month because it reports good reproducible research practices. In particular, in addition to sharing code used in their work, the authors also shared some nice interactive versions of their figures. To learn more about Henric and Stefan, and their research, check out our recent interview with them.
*NEW: Please follow our LinkedIn page to stay up to date with the latest MRM Highlights content: https://www.linkedin.com/company/mrm-highlights
General questions
1. Why did you choose to share your code/data?
A data and reconstruction toolbox for the fat/water field has been available since 2012, and is often referred to in papers. In this same spirit, we decided to make our data publicly available so that other researchers can try out our code with their own reconstruction, without having to re-implement the entire pulse sequence. We also shared interactive plots to help the reader understand some of the more complicated figures and results. The paper itself explains everything you need to know in order to reproduce the results. However, having access to the full sequence plots and being able to visualize the gradient waveforms and the associated encoding efficiencies makes it a lot easier to understand and interpret.
2. Do you think you’ll share code/data again in future publications?
Absolutely, it’s really easy to do and it forces you to clean up your code a little.
3. At what stage did you decide to share your code/data? Is there anything you wish you had known or done sooner?
We developed our interactive plots as a way of verifying the equations and making sure that the sequence was properly set up. We came back to these plots when interpreting the results internally, and it seemed natural to share them with readers.
4. How do you think we might encourage researchers in the MRI community to contribute more open source code along with their research papers?
Open source code can be of great value with some papers, but perhaps not all. It’s up to the authors (and perhaps peer reviewers?) to think about what would help the reader with reproducing and understanding the paper. To encourage this in the ISMRM community, we could perhaps ask reviewers to grade the reproducibility efforts of the group instead of simply giving the current yes/no response. Added to this, authors should be required to provide some reasoning on why they have chosen to share data/code/plots and how this will benefit the reader.
Questions about the specific reproducible research habit
1. What advice do you have for people who would like to create interactive figures?
Have a go and play around with examples at first. Choose whatever programming language you are comfortable with. MATLAB and Python are two of the popular ones used in MRI. Then you can spend time on Stack Overflow until everything works the way you want it to!
When you get to the final stage and feel ready to publish the code, make sure that the default settings are chosen so that a first-time reader doesn’t get presented with strange corner cases initially.
An interactive figure shared by the authors along with their paper, created using Bokeh. It is a Monte Carlo simulation of the equally weighted two-point fat water inverse problem (axes corresponding to dephasing times). To read up on this simulation, visit their website here. To explore more interactive figures that the authors created for this paper, visit this website.
2. Can you share some resources to help readers get started with creating interactive figures with Bokeh?
The Bokeh documentation is well made and requires only basic Python experience. A lot of the features are shown in the gallery (https://docs.bokeh.org/en/latest/docs/gallery.html), and that’s how I (Henric) got started. We’ve also published the source code for generating our interactive Bokeh plots, and maybe that could be useful. Alternatively, if anyone wants to send me an email, I’ll do my best to help.
Here is one short example of interactive matrix plotting https://docs.bokeh.org/en/latest/docs/gallery/image.html.
3. Did you encounter any challenges or hurdles in making the figures from your paper interactive?
Not specifically in making the figures interactive. But when I started using Bokeh, it was still under early development and so I had to rewrite parts every now and then. Now, though, I would say that Bokeh is very stable.
4. Are there any other reproducible research habits that you didn’t use for this paper but might be interested in trying in the future?
I’ve played around with Jupyter notebooks a little bit and I can see those being valuable in some projects. Hopefully, in the future, I’ll do some work that will allow me to use them a little bit more.
5. Where do you stand on reproducibility in the publishing process?
There’s a philosophical question that we’ve been asking ourselves a lot. Basically, should links to interactive plots and code be considered as permanent as the papers that are being published? If so, then one needs to make sure that the links direct users to servers that are not going to be taken down after a short period of time, for example a personal virtual machine. Ideally, these links could be hosted on servers operated by the journal’s publisher.
