MRM Highlights
All Posts
By Mathieu Boudreau
This MRM Highlights Pick interview is with Christopher Rowley and Christine Tardif, researchers at the McConnell Brain Imaging Centre of the Montreal Neurological Institute (MNI, aka The Neuro) in Montreal, Canada. Their paper is entitled: “A model-based framework for correcting B1 inhomogeneity effects in magnetization transfer saturation and inhomogeneous magnetization transfer saturation maps”. Their paper was chosen this month because the authors shared their code with the paper and also provided very detailed documentation inside and outside their code. To learn more about the work done by these two researchers, check out our recent interview with them.
To discuss this blog post, please visit our Discourse forum.
General questions
1. Why did you choose to share your code/data?
We wanted to share our code and data to encourage and help the MRI community to correct their MTsat (Magnetization Transfer Saturation) data for B1 inhomogeneity, by reducing the barriers to doing so. With the growing popularity of ihMT, we thought this would be a much-needed tool to improve data quality and the reproducibility of results.
2. What is your lab or institutional policy on code/data sharing?
Generally, our lab is open to sharing the code we have developed once the corresponding work has been published. This also allows the project to have gone through a peer-review process, ensuring that we have generated code that produces meaningful results. This and other open science practices are encouraged by The Neuro’s Tanenbaum Open Science Institute.
3. At what stage did you decide to share your code/data? Is there anything you wish you had known or done sooner?
When we first started on this project, we knew that we would want to share the resulting code for others to use. As the process grew more complex through different iterations, the code was less organized and documented. While I (CR) wish I had done a better job organizing and documenting the code from the start, I have to say that going back through everything after the fact gave me the chance to proofread it all and make sure we had got everything right.
4. How do you think we might encourage researchers in the MRI community to contribute more open-source code along with their research papers?
I think researchers will come to realize that code that is made open source gets used more and thus gets more citations, and this should encourage them to be open with their work. This is even truer for novel methods that are very complex, where the increased complexity constitutes a significant barrier to entry. Even if you produce an improved methodology, if it is difficult for others to implement in their own work, they will just stick to the previous methods.
Another thing I believe might be holding researchers back from sharing their code is that they do not want the prospect of having to spend time in the future providing support to others who are trying to implement the code in their work. I think if the MRI research community as a whole were more supportive and active on forums and GitHub pages, then support could come from the community and not solely from the developers (as you might see with the larger packages such as Freesurfer and SPM). So, in short, I think this barrier could largely be overcome by thorough code documentation (more on that below), and an active, supportive MRI community.
Questions about the specific reproducible research habit
1. What advice do you have for people who would like to improve their code documentation habits?
Personally, I like to write a small comment for almost every line of code. This is helpful for when you hand your code on to others, and they need to debug it, to work out why it is not working with their data. Having extra comments is also handy for when you take a break from a coding project; otherwise, when you come back to it, you might not remember what certain sections of code are doing and why. The extra words might look ugly, but later on you will thank yourself for having added them.
I would also recommend you let someone else try to use your code, once you have it in a state that you are happy with. This process will help you realize whether there is a certain aspect of the code that is not user friendly, and inform you if something needs to be adjusted, or if better documentation is required.
2. What questions did you ask yourselves while you were developing the code that would eventually be shared?
Since I was intending to share the code, I tried to ask myself, “What might other people do with this?” After all, we all have our own niche research area or application, and while the code will likely work great for that, if we can build more features or some redundancy into it, then it will be useful for others as well. To this end, I added some additional saturation parameters to the simulations that were not used for my study, but I know are used by others in the field.
I also ask myself where things could go wrong, and try to add in some debugging options that people can use to check. For instance, you might load a brain mask, but since it was made in a different program from that in which other data was saved, it gets imported in a different orientation. So, after stages where I load an image or segmentation, I include a line of code that people can use to check that it is loaded properly.
3. What advice do you have for people who would like to create good documentation on GitHub (i.e., README, Wiki, etc.)?
I think it is important to make a well-structured README file that is easy to follow, and use different headings to make it easier for people to find what they are looking for.
As for putting it all together, I would say get the documentation to a point where you are happy with it. Then take a few weeks off, if you can, so that you can forget it, and then come back to it. Try to run your code based solely on your own documentation (using a clean build if you can). This helped me realize that there were a few steps that I was jumping to simply because I knew how things worked, and was running the code every day. Once I came back to it after a break, I realized that the steps needed to be better broken down and written out to make it all easier to follow for someone who has never seen the code before.
4. Are there any other reproducible research habits that you haven’t used for this paper but might be interested in trying in the future?
I would be interested in implementing a Jupyter Notebook script in my future work to recreate my figures and analysis. I am intrigued by the idea of producing interactive plots. I think they could be beneficial for helping people see the data the way you (the researcher) see it after looking at it for so long.
