Share it all: challenges in reproducibility from data acquisition to management and analysis
Under the requests of many journals and funding bodies, our science is becoming more and more open and reproducible. With the increased adoption of open scientific practices, new challenges emerge, from cultural and ethical issues to practical issues in dealing with those steps of our scientific journey that are still not open or not completely reproducible. Making our science open can be time consuming and requires skills that might not be part of our academic curricula yet - and that adds to the challenges of junior and early career researchers. Data availability and accessibility are fundamental principles to reproducible science but they clash against national and international legislation on data protection and the equally fundamental principle of individuals’ protection. Closed-source or vendor-specific methods to create that data make it hard to obtain complete reproducibility of the scientific process, and advanced analyses that can take the form of non-transparent, black, boxes, lower the chances of reproducing results. These are the topics that we will address in this symposium, organised by the Open Science SIG. Specifically, we will address the four following questions:
- How do we survive academia as (junior and early career researcher) open scientists?
- How do we make (fMRI) data acquisition open and reproducible?
- How do we share data in a way that keeps privacy but allows access?
- How do we maximize reusability and reproducibility of our analyses (especially in AI)?
Featured Talks:
Breaking Barriers: A Practical Guide to Thriving as Open Scientists in Neuroimaging
Speaker: Yu-fang Yang
Navigating the complexities of academia as open scientists demands more than mastering reproducible workflows and open tools; it requires resilience in the face of systemic barriers and a steadfast commitment to fostering transparency
This talk offers a pragmatic guide tailored for Early Career Researchers, addressing challenges in neuroimaging research (fMRI and EEG) and the adoption of open science practices.
From curating datasets to publishing reproducible findings, we share lessons learned and practical strategies for aligning innovation with the principles of Findable, Accessible, Interoperable, and Reusable (FAIR) data.
The talk concludes with actionable steps for integrating open standards and tools while fostering a research environment that prioritizes transparency, reliability, and collaborative progress.
Harmonizing functional MRI with open-source data acquisition and reconstruction
Speaker: Scott Peltier
In principle, pooling functional MRI data across different sites and points in time should allow for increased statistical power, provided that the imaging experiment can be conducted in a known and reproducible way. While post-processing attempts to correct for ‘site’ variance have been somewhat successful, it would be better to control for potential differences (e.g., vendor, scanner model, scanner software) during the acquisition itself. Historically this has been a challenge, as the details of the vendor-provided fMRI acquisition and reconstruction software are generally not known.
The researcher can in principle implement the imaging protocol themselves using each vendor’s pulse sequence programming environment, but this is a technically difficult and time-consuming task. Existing multi-site studies have therefore invested significant resources into making the fMRI protocols on different systems as similar as possible and maintaining them across scanner software upgrades. However, even then the protocols can often only be ‘harmonized’ with respect to high-level sequence parameters such as FOV, matrix size, and net acceleration factor; subtle but potentially important differences in, e.g., details of sequence timing or RF and gradient waveform shapes, are often not known to the researchers or are beyond their control.
In this talk we will discuss open-source fMRI approaches, and describe a fully vendor-agnostic and portable fMRI protocol that ensures identical sequence execution and image reconstruction across different scanners, and across scanner software upgrades. The acquisition sequence is defined in simple human-readable text files, and sequence execution on real hardware is made possible by openly available sequence-agnostic interpreters. We believe this framework will enable reproducible, truly harmonized multi-site and longitudinal functional studies.
BRIDGE
Speaker: Kimberly Ray
AI for open neuroimaging: opportunities and challenges from an open science perspective
Speaker: Helen Zhou
Recent advancements in large-scale brain imaging, combined with the rapid progress in deep learning and generative AI, are transforming our understanding of the human brain.
This convergence offers a unique opportunity to enhance cognitive abilities throughout the lifespan and to develop personalized, precise treatments for neurological disorders.
In this talk, I will explore how open science approaches—such as open datasets, open-source software, and collaborative tools—are driving the development and refinement of AI methods for brain imaging.
These include applications in brain lesion segmentation, vision decoding, and the creation of brain foundation models.
I will discuss how open science fosters the reusability and reproducibility of AI-driven brain imaging analyses, while addressing key challenges, such as adaptation across diverse cohorts, database harmonization, cross-validation, hyperparameter optimization, and benchmark evaluation.
Open science is crucial for advancing AI applications in neuroscience and medicine, and it plays an essential role in overcoming current barriers to progress.