INTRODUCTION

Selma Lugtmeijer, Nils Muhlert

The Organisation for Human Brain Mapping (OHBM) Open Science Room (OSR) is one of two principal events coordinated by the Open Science Special Interest Group (OSSIG). While the OHBM Brainhack1–3 occurs before the main OHBM conference, the OSR runs concurrently with the main program, offering a varied schedule parallel to other sessions. The OSR focuses on the opportunities, advancements, and challenges presented by open science to the neuroscience community. Sessions in the OSR emphasize interactive discussions aimed at advancing neuroscience research through open science practices.

The Open Science Room (OSR) was initiated in 2016 with Dr. Cameron Craddock (Computational Neuroimaging Lab, The University of Texas at Austin Dell Medical School) and Dr. Daniel Margulies (Integrative Neuroscience & Cognition Center, Université de Paris) as co-organizers. It emerged from the success of the OHBM hackathon and encouragement from Dr. Jean-Baptiste Poline (Department of Neurology and Neurosurgery, McGill University), who was Education Chair, to create an open science SIG. The first educational course offered was Brainhack 101. Over the years, the OSR has evolved from a space for spontaneous, informal talks and idea sharing into a structured program that runs in parallel with the main conference. Throughout its development, central themes such as reproducible science and open-source tools have remained at its core.

Today’s OSR features three types of sessions: Panel Discussions, Table Talks, and Emergent Sessions. Panel Discussions feature four experts who engage in dialogue on a preset topic, guided by a moderator, with audience participation encouraged through a question-and-answer format. The OSR (co-)chairs select Panel Discussion topics considering current developments in open science, the interests of researchers at various career stages, and the specific relevance of topics to the local scientific community where the conference is held. Each Panel Discussion is followed by a corresponding Table Talk, which serves as an extension of the discussion. In the more informal, non-hierarchical setting of the Table Talk, participants can engage in dialogue, propose new ideas, and challenge existing perspectives.

Emergent Sessions focus on current issues in open neuroscience and are open for submission from anyone in the community. These sessions are selected shortly before the conference to ensure the inclusion of recent developments and facilitate discussions on emerging trends.

The OSR welcomes both established members and newcomers into the OHBM open science community and provides a starting point for researchers at the early or intermediate stages of their open science journey. Through careful selection of panelists, self-nominations, and open submissions for Emergent Sessions, we aim for diversity in panelists and speakers. This approach creates opportunities for underrepresented groups. Furthermore, we strive for a balance between established and early career researchers to enhance the depth of discussions.

This report summarizes the events that took place in the OSR during the OHBM conference in June 2024, in Seoul, Korea.

OHBM OSR 2024

Selma Lugtmeijer, Ju-Chi Yu, Xiangzhen Kong

The OSR had a comprehensive four-day schedule during the OHBM conference in Seoul. A wide range of topics related to open science in neuroscience research were discussed; early career researchers became involved in the OSR, established researchers shared their views on changes and challenges, and new tools were showcased. Panel Discussions and corresponding Table Talks covered varied topics: 1) the costs of open science, 2) getting started in open science, 3) the evolution of open science over the years, 4) open science in Asia/Korea, and 5) crowdsourcing for brain mapping. Latest developments and challenges presented and discussed in Emergent Sessions included: 1) independently developed simulated datasets of the interplay between brain development and behaviour, 2) carbon emissions of fMRI research computing, 3) obstacles to open science when working with industry or government, 4) the Generalist Repository Ecosystem Initiative (GREI),4 and 5) updates on Collaborative Informatics and Neuroimaging Suite Toolkit for Anonymous Computation (COINSTAC).5

Each OSR 2024 session was attended by approximately 20 – 60 physical participants. To enhance inclusivity, all sessions were live-streamed, recorded, and made publicly accessible on YouTube (https://www.youtube.com/playlist?list=PLVso6Qs8PLCjWjg8DBN4w6J5c56zu3g4i).

The following sections provide summaries of all sessions, highlighting their relevance to the broader neuroscience community and addressing current challenges and developments.

PANEL DISCUSSIONS

Open science: Who pays the bill?

Selma Lugtmeijer, Ju-Chi Yu

With the increasing interest in open science, questions emerge regarding who is paying for what and what ‘inclusive’ truly means. If open access publication entails high fees, one might question whether this shift in payment responsibility—from the reader to the author—excludes more individuals than it includes. Additionally, the relationship between open access and impact factors raises concerns about how these factors affect researchers from diverse backgrounds and at various stages of their careers.

We had four panelists, including Dr. Cyril Pernet (Neurobiology Research Unit, Rigshospitalet), Dr. Franco Pestilli (Department of Psychology, Department of Neuroscience, Center For Perceptual Systems, Center for Learning and Memory, Center on Aging and Population studies, The University of Texas at Austin), and Dr. Jean-Baptiste Poline (Department of Neurology and Neurosurgery, McGill University), with Dr. Pierre Lune Bellec (Département de psychologie, University of Montreal) participating virtually. Strong opinions and innovative ideas were shared regarding who is currently paying for and should pay for open science.

This discussion focused on the relationship between open access and publication fees. Different models of publishing exist with journals that are fully open access, journals for which readers need a subscription, and combinations of both. The fees for open access vary greatly with a correlation between the costs of publishing and the impact factor of the journal. As publishing houses are companies with a goal of economic growth, they benefit from high fees. There is a push back to these high open access publication fees from the community and by journals like Imaging Neuroscience and Aperture Neuro that base publishing fees on actual costs. There are also a few completely free journals (e.g., Machine Learning for Biomedical Imaging, Neurons, Behavior, Data Analysis and Theory), but they rely heavily on volunteers, which is also a hidden cost. A challenge is that journals with lower fees for open access publishing are often more specialized and newer so they have a lower impact factor as it will take time for these journals to gain reputation. This is exacerbated by publishing companies owning the name of journals, which means that journals who want to change towards more affordable open access can’t take their names to another publisher. Whether researchers should still aim to publish in journals like Nature or Science, which are prestigious to the general field of science, but have open access fees that are not affordable to all, was debated. On one hand, it marks a paper important as it can compete with submissions from different fields; on the other hand, a paper of such kind of publication might not be necessary to make a career.

Several suggestions on how to deal with the high costs of open access and the power of publishing houses were made during the session:

  1. Form cross-university and cross-society unions to put pressure on publishers and/or nudge journals to change their income model. At the university level, librarians who lobby for subscription deals with publishers could play an important role.

  2. Rethink researchers’ overhead costs at universities. At the moment, a substantial part of a researcher’s grant money goes to library costs. At the same time, open access publication fees are paid with grant money. A change we can make as a community is incorporating open access fees into library costs.

  3. Putting responsibility for publishing on universities or societies. Aperture Neuro is an example of a journal rooted in the OHBM society. This means they are not tied to a specific publisher.

  4. Peer review platforms that are based on preprints and updates thereof. This would completely cut out publishers. An example of a journal this moves in that direction is eLife.

  5. A controversial idea, proposed by Dr. Pestilli, is to take an example from the incentive structure of acting, where actors receive royalties for movies for the rest of their lives. In research, this would mean a system in which the author (and maybe co-authors and reviewers) receive a small compensation every time a paper is cited. Currently, only the publisher benefits.

A second topic that was shortly discussed was the cost of open software and its maintenance. Distributing the responsibility of development and maintenance of software over multiple labs might help in covering the costs and guarantee continuation. If the problem the software solves is important, more people will want to contribute, which will also improve the quality. Still, most software belongs to one lab and relies on a small number of people. Here, institutional support for governance might be a solution. This could come from the overhead paid to libraries at the university level. It was pointed out that the incentive structure for software development and maintenance is important. Recognition might come from publications. Developing a software library should give the same credit as publishing a research paper.

Getting started in open science

Subapriya Suppiah, Nur Shahidatul, Nabila Ibrahim

The scientific community is undergoing a rapid transformation, driven by the open science movement. The panelists in this session critically discussed, examined and defined key components of open science, including open access, open data, materials, and code. They also addressed the promotion of reproducible analyses, preregistration, and registered reports, such as those found in preprints, and the ease of replicating research methods. Additionally, the discussion among the panelists explored integrating open science practices into educational frameworks.

This open forum aimed to stimulate critical dialogue regarding the optimal structure of open science, culminating in recommendations for best practices. The panelists also shared their personal experiences in practicing open science.

We had four panelists in this session, including Dr. Elizabeth DuPre (Department of Psychology, Stanford University), Dr. Imbrahim Faye (Department of Fundamental and Applied Sciences, Universiti Teknologi Petronas), Dr. Shella Keilholz (Biomedical Engineering, Emory University and Georgia Tech), and Dr. Hyeong Hun Lee (METLiT Inc.). Dr. Keilholz and Dr. DuPre attended the conference in-person, while Dr. Faye and Dr. Lee attended virtually.

The focus of the discussion was on the role of open access as a platform for accessing and reusing data and code. Dr. Keilholz shared her academic experience in making her lab’s data and code readily available for others to use. She highlighted challenges in open science such as the lack of personnel to manage the data and deal with bugs that arise in the system. Dr. Faye said that, for him personally, the primary definition of open science is the right to inclusivity, whereby the science is not reserved for a certain group of people. This is followed by transparency of the whole research process in order to make it reproducible. Dr. Lee, who represented the industry’s point of view, suggested that academia and industry could work together to develop new technologies through the validation of applications by utilizing mutual datasets. Dr. DuPre introduced an important gap in the discussions about open science, which is the need to train more people to ensure a seamless workflow and make open science more sustainable.

Most of the panelists shared that their experience with open science usually began with a need to train a postgraduate student or a postdoc, as well as to fully utilize data gathered from previous research.

Several interesting ways to get started in open science included participating in journal clubs to discuss open access publications and other open resources (Dr. Faye), developing interinstitutional collaborations to gain access to larger datasets and perform validation of neuroscience tools (Dr. Lee), keeping detailed records of research processes to enable others to reuse protocols and results, while also being conscious of open-source ecosystems, with a focus on improving the reproducibility of analytic workflows (Dr. Keilholz), and publishing code with helpful links to repository sites (Dr. DuPre).

The recommendations made by the panelists in this session include:

  1. The need to lower publishing barriers for researchers.

  2. Requirements for more current open education resources that are easily accessible, which can help train and retrain personnel as the field of open science evolves.

  3. Collaboration between academia and industry to share insights in neuroscience and to develop clinically relevant solutions by harnessing new technology.

  4. Develop a local institutional data repository to begin archiving curated datasets.

  5. Encourage multi-centre studies that investigate the comparability of protocols and data across sites.

  6. Develop a system for reproducibility buddies, especially for code development in computational methods.

  7. Ensure that journal submissions contain data and code availability statements.

  8. Explore the role of video publications and link-rich publications (with e.g., reference to the GitHub site) to share research protocols and code, such as publishing with the Journal of Open Source Software.

  9. Introduce postgraduate students to the benefits of open science and develop systematic training on how to practice it.

We also had an interactive open dialogue with attendees who posed questions to the panelists, and some attendees also shared their own experience getting started in open science. The panelists closed the session with some insights regarding best practices in open science, which include overcoming the fear of getting their research scooped by others, balancing the data they share with being selective about curated data that has been tested, being aware of the net benefit of gaining from others’ work that can enhance the careers of early career researchers, and also exploring ‘pay to play’ or allowing advertising methods that are ‘open access’ to provide some ways to fund and maintain websites and give recognition to ecosystems that support software.

The changing face of open science

Nils Muhlert, Selma Lugtmeijer

There has been a dramatic shift in the uptake of open science practices in the past thirty years. This has led to considerable change in how we collect, analyze and distribute data, how we publish and disseminate findings, and how we recruit staff and seek employment. Many OHBM members have been at the forefront of these initiatives. Here, past members of the OSSIG gave their views on how open science has changed in neuroimaging, and where its path may lie.

Panelists for this session were Dr. Janine Bijsterbosch (Computational Imaging Research Centre, Washington University), Dr. Oscar Esteban (Ambizione FNS fellow in Radiology at the University of Lausanne and co-lead on fMRIPrep6), Dr. Daniel Margulies (Director of the Cognitive Neuroanatomy lab at the Integrative Neuroscience & Cognition Centre, University of Paris, and a founder member of the OHBM OSSIG), and Dr. Pradeep Reddy Raamana (Centre for AI Innovation in Medical Imaging, University of Pittsburgh).

Dr. Margulies started by discussing OSSIG’s roots from the Neurobureau7 - creating a community to share ideas, code and resources. It was a way to allow access to resources for all. This resonated with Dr. Esteban who noted his early challenges in science, which was revitalized through new opportunities from the Human Connectome Project8 and other open science initiatives.

Dr. Bijsterbosch highlighted the joy of working with open science communities. Over years, this has expanded from small niche groups to widely accepted practices throughout academia. This hasn’t been an overnight success story. Yet, the process of convincing mentors and developing effective training has progressed steadily. One of the hurdles that the adoption of open science in academia has faced has been the current incentive structures. Dr. Bijsterbosch pointed out that careful strategic planning is needed to influence funders and educational programmes. The panel discussed some great examples of this influence, for example with grant funders requiring open sharing, with open access publishing, and with registered reports. Indeed, Dr. Margulies noted that those entering the field now will have a completely different experience of science to those starting in the early 2000s.

Dr. Raamana noted the remaining challenges - that academia has problems in appropriate recognition for software development, influencing both the sustainability of the software and the careers of developers. There is still an emphasis on novelty, problems with maintaining the ever-expanding range of open datasets, and challenges in publishing software. The currency of science has been changing from acquiring (and keeping) datasets, to analytical approaches. But not everyone is signed up yet, with regional differences in incentives for open science approaches. Large collaborative projects are, however, being increasingly recognized by grant funders and institutions around the world.

An interesting question from the community was whether there is a form of ‘green washing’ occurring in open science. Are changes by publishers and funders having real impact or just cynical attempts to game metrics? Standardising how open science is measured and pushing for higher minimum standards were Dr. Esteban’s ways to make more concrete improvements. Dr. Bijsterbosch noted that anything in a positive direction is a useful step, towards an ‘open science utopia’. As a counterpoint, Dr. Margulies pointed out that whilst we are continually aiming for best practices, we have to carefully balance this against, for instance, the European Union’s General Data Protection Regulation (GDPR) laws or the risks of over-constraining scientists.

Finally, there was discussion about how we pursue these changes, through either demonstrating better practices ourselves, or by actively lobbying funders, institutions and consortiums to push for higher minimum standards. More on this at the OSR next year! Overall, the panel concluded that whilst treading the paths towards open science utopia has involved a slow testing of each step, those steps are continually forward-moving and indelibly forged.

Open science in Asia/Korea

Ju-Chi Yu, Xiangzhen Kong

Although open science has been well-recognized by the neuroimaging field as a crucial step towards better scientific practices that facilitate reproducibility and reliability, the progress has mainly taken place in the Western world. Asia, or Korea specifically, is relatively new to the practice and the idea and has encountered unique challenges. With the Annual Meeting of OHBM being held in an Asian, non-English speaking country, this meeting provides a unique opportunity for local researchers who are interested in open science to share their experiences and the challenges they face in their own countries. Specific topics of discussion include “What is stopping people from practicing open science?,” “What is stopping institutions and governments from supporting open science?,” and “How do you think the community could help?” This discussion will foster the development of open science through international collaboration and advance the open science practice to become more inclusive globally.

This panel featured four speakers who are involved in promoting open science in China (Dr. Chuan-Peng Hu, School of Psychology, Nanjing Normal University), Taiwan (Dr. Chun-Chia Kung, Department of Psychology, National Cheng Kung University), Japan (Dr. Hiromasa Takemura, Department of System Neuroscience, National Institute for Physiological Sciences, Center for Communication Networks, National Institute for Physiological Sciences), and South Korea (Dr. Won Mok Shim, Department of Biomedical Engineering, Sungkyunkwan University). They share unique experiences in advancing open science in their countries, including specific and common challenges they face.

In this session, the panelists first shared their successes in open science. All panelists are active in open science practices and projects or are members of their local open science communities. They addressed how open science is seen among the local research community, which has led to successes but also challenges. In general, the research community is still hesitant to practice open science given limited institutional and policy support. Some members of the research community are supportive and have been promoting the ideas, but these efforts are carried out more on the individual level. Institutional support has recently begun but is still in its initial stages. Regarding support from institutions and funding agencies, one main concern of open science is the privacy laws and rules for sharing human participants’ data. This has remained a challenge as most efforts to promote open science practices come from the community, but privacy laws need to be changed in a top-down manner.

Another general concern is the increasing costs of publishing open access papers, which is a core part of open science practices. While some countries made transformative agreements with major publishers, others did not, making open access publication a luxurious option.

The panelists also find it difficult to participate or to encourage their trainees or colleagues to participate in international open science community/events (e.g., BrainHack, OSSIG, etc.). Many of these activities rely on networking skills and exchanges that follow the Western culture. For colleagues and trainees from East Asian cultures, it is culturally difficult for them to speak up and initiate conversations. They also found that many of these activities rely on communication on social media platforms that are more popular in the Western world. As a result, being involved in the international community often requires extra effort aside from connecting with local communities. In general, the open science community is Western-based, and cultural and language barriers exist for the Asian community to participate.

In this session, the panelists and the audience shared their concerns and challenges in practicing open science in Asian countries. These challenges involve language barriers, cultural differences, limited resources, and the top-down bureaucratic culture in Asia. Given limited financial resources from research funding, it is crucial to show the incentives for practicing open science. To alleviate cultural challenges, building a pan-Asian open science research community or special interest group could be a possible solution. With Asian countries that have similar cultures, it is worth sharing experiences and communal efforts to promote open science practices. Additionally, this approach could also foster potential international collaborations.

Many a little makes a mickle: Crowdsourcing for brain mapping

Xiangzhen Kong, Wei Zhang

With increasing reliance on collaborative research in neuroscience, crowdsourcing has emerged as a critical strategy for advancing brain mapping. Crowdsourcing—whether through data sharing, software development, or research collaboration—raises questions about how it can be better integrated into academic and research frameworks. This includes ensuring recognition for contributors, addressing barriers to participation, and fostering more sustainable and effective practices. These were the core issues discussed during this Panel Discussion.

The panel featured four speakers, including Dr. Paul Thompson (Imaging Genetics Center, Stevens Neuroimaging & Informatics Institute, University of Southern California), Dr. Seok-Jun Hong (Center for Neuroscience Imaging Research, Institute for Basic Science, Department of Biomedical Engineering, Sungkyunkwan University), and Dr. Chao-Gan Yan (Department of Psychological and Cognitive Sciences, Tsinghua University), with Dr. Francesco Santini (Basel Muscle MRI, Department of Biomedical Engineering, University of Basel, Department of Radiology, University Hospital of Basel) participating online. Each panelist brought a wealth of experience in neuroscience, collaborative research, and/or the development of tools and methodologies that facilitate crowdsourcing. Their insights spanned the practical, theoretical, and systemic aspects of crowdsourcing in the context of brain research.

Crowdsourcing can take several forms, including data sharing, open software development, and multi-author collaborative projects. The discussion began with an exploration of what crowdsourcing means in the context of neuroscience.

While commonly associated with open science, the panelists highlighted unique aspects that differentiate it. For example, crowdsourcing involves pooling resources—data, software, and expertise—from diverse contributors across multiple institutions. This approach is exemplified by the ENIGMA Consortium,9 co-founded by Dr. Thompson, which aggregates brain imaging data from around the world to improve the reproducibility and generalizability of findings. Beyond data aggregation, the intellectual diversity inherent in crowdsourcing was noted as a key benefit. Multidisciplinary teams bring varied perspectives, enhancing the creativity and comprehensiveness of research. Crowdsourcing also democratizes access to resources, as seen in platforms like UK Biobank,10 which has over a thousand registered users globally analysing neuroimaging data.

While the panel emphasized the scientific and practical advantages of crowdsourcing, it also raises critical challenges, such as ensuring equity in participation and acknowledgment of contributions. A major theme of the discussion was the need for better recognition and incentives for contributors to crowdsourcing initiatives. While papers remain the dominant currency in academia, the development of open-source software or sharing of curated datasets often lacks comparable acknowledgment. The panel highlighted that software, though integral to modern research, does not yet receive the same academic credit as traditional publications.

Several suggestions were made to address this gap:

  1. Improved citation practices: Incorporating automated citation systems within software tools could ensure that all contributors are properly credited.

  2. Incentive mechanisms: Funding agencies and promotion committees could recognize software development and data sharing as metrics of academic success, similar to publications.

  3. Global standards for recognition: Establishing international guidelines to reward contributions to open science initiatives could create more uniformity and fairness.

  4. The panelists also highlighted regional disparities, noting that in countries like China and Korea, evaluation systems heavily prioritize impact factors, making it harder for open science contributions to gain recognition.

The panel also discussed how early career researchers can engage with crowdsourcing and initiate collaborations. Attending conferences, participating in hackathons, and leveraging mentorship opportunities were identified as key strategies. Dr. Thompson shared that many of his collaborations stemmed from chance meetings at conferences, underscoring the importance of networking. Mentors play a critical role in supporting early career researchers, from providing funding for conference attendance to facilitating introductions to potential collaborators. Smaller, focused meetings were recommended as more accessible entry points for junior researchers compared to large conferences.

Balancing collaborations with other academic responsibilities remains a challenge. It was noted that while collaborations can be intellectually rewarding, they can also become overwhelming without proper boundaries. The panelists emphasized the importance of delegating responsibilities and strategic decision-making. For instance, involving students in managing specific aspects of collaborations can lighten the load for principal investigators while providing valuable training opportunities. In addition, prioritizing collaborations that align with personal and institutional goals can prevent overcommitment.

The panel concluded with reflections on the transformative potential of AI in crowdsourcing and brain mapping. From improving data sharing platforms to enabling personalized research tools, AI is poised to further democratize and accelerate neuroscience. However, the speakers stressed that systemic changes - such as better recognition systems and sustainable funding models - are needed to fully realize the benefits of crowdsourcing.

The consensus among panelists was that crowdsourcing holds enormous potential to enhance research impact. The session left attendees with a sense of optimism and a call to action: while challenges remain, the collective effort of the neuroscience community can overcome them, paving the way for more collaborative, inclusive, and impactful research.

EMERGENT SESSIONS

The interplay between brain, behaviour, and cognition from childhood to adulthood: Panel Discussion with independent research groups on simulated datasets.

Neda Sadeghi, Isabelle van der Velpen, Tonya White

Neuroimaging has contributed considerably to our understanding of brain development and its relationship to cognition and behaviour. Despite these advancements, replicability remains a critical issue in the field, and no gold standard exists for evaluating the neuroanatomical correlates of cognition, behaviour, and their interactions. Researchers from the National Institute of Mental Health, Georgia Institute of Technology, Beijing Normal University, the Radboud University, Donders Institute for Brain, Cognition, and Behaviour, Forschungszentrum Juelich, and University of Oslo have created simulated datasets of the interplay between brain development and behaviour. Each group has worked independently, was unaware of the approaches and assumptions made by the other groups, and was provided the same set of variables. They were tasked with creating three datasets with each embedding how they envision the interplay between brain development, behaviour, and cognition emerges throughout development. These simulated datasets, which include data from 10,000 participants across 7 longitudinal waves (ages 7 to 20), are being released to challenge and engage the research community in identifying the underlying patterns and assumptions used in their creation. The simulated data along with a design preprint paper is available on OSF: https://osf.io/yjt9p/. The goal of the session was to announce the release of these simulated data and answer any questions the community might have.

Dr. White started the session by providing an overview of the project, while Dr. Sadeghi moderated the discussion. During the panel, Drs. Calhoun, Plachti, Sprooten, van der Velpen, and Zi-Xuan, representatives from each group that developed simulated datasets, shared insights on the challenges and opportunities of modelling brain development, as well as addressed questions about the simulated datasets from the research community.

A key challenge discussed was determining the “ground truth” and generating simulation data, which involves questions about data distributions and how to validate them against existing literature. One panelist highlighted how we often take certain assumptions for granted when analyzing data, but creating simulated data requires a deeper understanding. For instance, questions were raised about the actual distribution of brain volume in the paediatric age range and how these measures interact with other variables. Even with fewer than 20 variables, the number of possible interactions and covariances can quickly become overwhelming, with changes to one variable potentially triggering cascading effects.

The panel also debated whether the brain predicts behaviour or vice versa. While traditional models have focused on the brain predicting behaviour, recent evidence suggests that behaviour may also influence brain development.

Looking ahead, there was a shared excitement about the next stage, when the research community begins analyzing these datasets. The panel also emphasized the importance of continually updating simulations based on new evidence and methods, while identifying areas of consensus and divergence within the research community. Analyzing these simulated datasets will help assess how closely the results align with the assumptions made during their creation, offering insights into how real data is interpreted. This process may lead to surprising findings that challenge existing theories in the field.

Five groups created simulated datasets, and the list of group members is provided below:

  1. Neda Sadeghi, Isabelle van der Velpen, Dustin Moraczewski, Philip Shaw, Audrey Thurm, Adam Thomas, Tonya White (National Institute of Mental Health, NIH).

  2. Zi-Xuan Zhou, Xi-Nian Zuo (Beijing Normal University).

  3. Anna Plachti, Øystein Sørensen, Sarah Genon (Forschungszentrum Juelich and University of Oslo).

  4. Vince D. Calhoun, Masoud Seraji, Ishaan Batta, Rogers Ferreira Da Silva, Najme Soleimani, Bradley T Baker, Kyle Joseph Cahill (Tri-institutional Center for Translational Research in Neuroimaging and Data Science [TReNDS], Georgia State, Georgia Tech, Emory).

  5. Rogier Kievit, Léa Michel, Ethan McCormick, Emma Sprooten (Radboud University Medical Centre, Donders Institute for Brain, Cognition, and Behaviour).

Measuring and reducing the carbon emissions of fMRI research computing

Nick Souter, Niall Duncan, Nikhil Bhagwat

Representatives from OHBM Sustainability and Environmental Action Special Interest Group (SEA-SIG) hosted a session focused on measuring and reducing the compute power cost, and therefore the carbon emissions, of preprocessing and statistical analysis of fMRI data. Speaker Dr. Souter joined the session virtually, while Dr. Duncan and Dr. Bhagwat provided live presentations and demonstrations in person in the OSR. Dr. Kalc assisted in organizing the session.

The overall goals of this session were for attendees to:

  1. Learn about the source of carbon emissions that arise from neuroimaging computing and data storage.

  2. Learn how to measure and reduce carbon emissions for one’s own neuroimaging data processing.

  3. Reflect on best practice for environmentally sustainable and open neuroimaging research.

During the first presentation, Dr. Souter provided some context on the scale of the impact of computing on the climate crisis, and outlined how these emissions arise. Dr. Souter also discussed findings from recent empirical work, including an investigation of how the carbon footprint of fMRI preprocessing tool fMRIPrep8 can be reduced, and a comparison of the carbon footprint of FSL,11 SPM,12 and fMRIPrep.6 Focus was also given to the use of carbon tracking tools and real-time carbon intensity task schedulers.

Dr. Duncan then conducted a live demonstration of the use of these green computing tools to attendees. This included an online carbon tracker (Green Algorithms, https://www.green-algorithms.org/), a server-side carbon tracking tool (Calc_Carbon, https://ohbm-environment.org/carbon-tracker-toolboxes/) and the Climate Aware Task Scheduler (CATS, https://github.com/GreenScheduler/cats).

Finally, Dr. Bhagwat reflected on the relevance of open science practices to sustainability goals. Green computing is an increasingly important aspect of socially responsible science, and intersects with a number of open science practices including data standardization, provenance tracking of compute-heavy processing pipelines, development of clear data management plans, and reflection on how and where to share (processed) data publicly.

The session ended with an interactive discussion with the audience concerning the practical uses and limitations of green computing tools, as well as on how to best balance any tension between open science and environmental sustainability (e.g., public sharing of large amounts of data).

As open as possible and as closed as necessary - Revisited

Gorana Pobric, Peter Fox, David N. Kennedy

The title of this Emergent Session reflected the stated aim of Horizon Europe (the European Union’s key funding program for research and innovation): it reflects the need for balancing commercial and personal data protection against the push for open data and software sharing. The three speakers presented their experiences of working with constraints on data sharing from the perspective of government agency-funded research, from clinical studies and from commercial enterprises.

Dr. Pobric (Neuromodulation director at the University of Manchester, UK) started off by discussing her recently completed research, funded by government defense agencies. Ownership of data, papers and conference presentations was controlled and regulated by the government agency. Security classification policies had to be negotiated. This influenced decisions on what can be presented and how.

Dr. Kennedy (Professor of Psychiatry and Radiology, University of Massachusetts Chan Medical School) then discussed his open science experiences, working with clinical child development studies with NITRC13 (a data and software sharing resource) and with ReproNim14 (which aims to improve the levels of reproducibility in science). Dr. Kennedy noted that ‘as closed as necessary’ in his case usually meant ways to prevent access from causing harm - limiting disclosures of people and about people. People should be protected from revealing medical histories or their risk of disease. We want participants to take part in research, and to feel that they can help others by doing so; concern about disclosure harms or profiteering could affect decisions to enrol. On the other hand, commercial restrictions can however slow the process of moving interventions or software to market, which is often what we’re aiming for.

Finally, Dr. Fox (Neurologist & Neuroscientist in Texas, and Director of the UT Health Sciences Centre at Austin) offered an alternative view on why he pulled back from allowing commercial use. He noted that data-sharing policy and commercialization is driven by the Bayh-Dole act (1980) which put universities in charge of patents and copyrights from data. True open-sharing without commercial licence would prevent marketisation of that data or software. The NIH Resource Dissemination Policy (1999) then changed this policy to still protect copyright and patents, but to allow greater sharing. Dr. Fox discussed his early experiences in PET research, working in St Louis in 1983 - commercializing and licensing that research allowed PET scanning to take off throughout the world.

Dr. Fox noted that tabular data (e.g. published in journals) is free for reuse. This allowed the creation of the BrainMAP15 software. But datasets themselves can be copyrighted. Metadata is critical for this. Currently BrainMAP15 is free for use, but not for commercial use. Tools, data and developer tools are available to create node or connectomic models, which can be validated in primary data, then rolled out for presurgical planning, disease modelling or other uses.

The discussion following focused on how we train neuroscientists about licensing. Creative Commons has moved the field forward but end users don’t always know what this allows. Discussion in lab meetings of what is being commercialized and what the limitations of this route are can be helpful. Another question was about ethical approval - how can we ‘future-proof’ our ethics protocols? Dr. Pobric responded that she had gone back for amendments, but that this takes time. Dr. Fox said that at the centre level you can have repository consent - a separate IRB that can be held and renewed. This is offered to all participants for all studies. All consented data can then be added to a local XNAT offering a huge data resource with minimal trouble for investigators. Dr. Fox highlighted that there is an ‘Open Brain Consent’ which aggregates consent issues (GDPR, commercialization etc.). The panel concluded by saying that ways to promote this as best practice between institutions could be helpful for many researchers.

The spirit of open science is to reduce obstacles to knowledge, data and tools in an attempt to speed up scientific discovery and promote scientific accountability and collaboration. Whilst we all subscribe and aspire to adhere to these worthy goals, the reality is that not all science is, or should be made open. Increasingly, neuroscience research is funded by industry, commercial companies and governmental bodies. The issues around “openness” arise when Intellectual Property Rights (IPR) are involved or when the research study and its findings involve potentially classified data. In this session, we will discuss some obstacles to open science when working with industry and governmental agencies, from different cultural perspectives. We will share our experiences, propose various solutions (e.g. hybrid open science-intellectual property models), and open a discussion about safe research practices in neuroscience for future adoption of open science in a commercial world.

What can generalist repositories do for you? A community feedback gathering session from the NIH Generalist Repository Ecosystem Initiative program

Ana Van Gulick

The neuroimaging community has been a leader in open science and data sharing for many years and neuroimaging researchers are frequent users of both discipline-specific and generalist data repositories as part of their open science workflows. This session was conducted in the OSR as part of the NIH Generalist Repository Ecosystem Initiative4 (GREI), to learn from the neuroimaging community about how they use generalist repositories (GRs) for sharing data and other research materials and to gather feedback on how GREI could prioritize its work to enhance GR functionality and resources to better serve the needs of this research community.

In February 2022, the US National Institutes of Health (NIH) Office of Data Science Strategy (ODSS) launched the Generalist Repository Ecosystem Initiative (GREI) Program, which brings together seven generalist repositories (Dataverse,16 Dryad,17 Figshare,18 Mendeley Data,19 Open Science Framework,20 Vivli,21 and Zenodo22) to work collaboratively to enhance support for data sharing and discovery in GRs. A generalist repository is one that accepts data and other research outputs regardless of research area, methodology, or file type. (See the Generalist Repository Comparison Chart for more information on these repositories). Through GREI, NIH recognizes that GRs play a key role in the data sharing landscape for the FAIR sharing of data in trusted repositories, offering broad flexibility to publish any research output alongside discipline- and method-specific data repositories when they are available, especially for researchers seeking to comply with global data sharing mandates and to practice open science. Together the GREI repositories are working to enhance common metadata, persistent identifiers, and standard metrics to support cross repository search and lower the barriers for data sharing and reuse.

While there are many high quality, discipline-specific data repositories for neuroimaging data of specific types (e.g. OpenNeuro,23 ADNI,24 CNDA,25 CONP,26 NITRC,13 NeuroVault,27 NIMH Data Archive28), the volume and diversity of research outputs generated by neuroimaging research necessitates the use of GRs for some outputs. Neuroscience is a top research category for all of the GREI repositories, often in part because these repositories are used to publish materials beyond data including software and code, images and media files, workflows, posters and presentations, and other supplementary files. GRs are also often used in conjunction with discipline-specific repositories and data standards.

A key objective of GREI is community outreach, and this Emergent Session provided an opportunity for us to engage with the neuroimaging community to share the work of GREI and resources offered by GRs, as well as to learn about the practices and needs of the neuroimaging research community for broad data sharing and reuse. As frequent users of repositories and keen practitioners of open science with a wide variety of data types and research outputs to share, the neuroimaging community is an especially valuable group for GREI to engage with to inform our work. Through this interactive session, GREI was able to learn from OSR participants about their data sharing and repository experiences and gather feedback about what GRs could do to better support them through repository functionality or resources.

The session began with an introduction to GRs as well as an overview of GREI goals and activities to enhance GR support for data sharing and discovery (see all GREI outputs at: https://zenodo.org/communities/grei). We then facilitated an interactive audience poll activity and audience discussion to learn about neuroimaging use cases for GRs including for sharing non-data materials and to uncover gaps in GR functionality and needs for resources or other support. In addition to the OSR session, an online survey was also circulated via social media and OSR channels to gather more detailed information about repository uses and feedback.

Key takeaways from this session include:

Use of Generalist Repositories in neuroimaging:

  1. Despite the availability of many discipline-specific repositories for neuroimaging research, generalist repositories are often used for sharing data, software, and other materials.

  2. BIDS29 standard and OpenNeuro23 cannot be used for all types of data or all study types, so the flexibility of GRs is needed in those cases.

  3. Researchers share many different types of research outputs beyond datasets including code and software, images, training materials, and other supplementary files and presentations.

  4. Datasets produced in neuroimaging may be quite large - many TBs or even PBs for large scale studies.

  5. Researchers sometimes share the same data or materials from the same project in multiple repositories and/or multiple formats.

  6. OSF,20 Zenodo,22 and Figshare18 anecdotally seem most commonly used

    • OSF for sharing all materials and collaborating

    • Zenodo for publishing software and tools

    • Figshare for publishing data as well as posters and other materials

  7. GRs are perceived as flexible, easy to use, and quick to use, making them low barrier ways to share research outputs.

  8. Researcher knowledge of repository infrastructure and best practices may be limited even among those participating in open science practices regularly.

Feedback on features of Generalist Repositories:

  1. Flexibility, ease of use, DOIs, and integrations (e.g. GitHub) offered by GRs are their most appealing functionality.

  2. Metadata is important, especially linking data or materials to related publications.

  3. Researchers would like more support from the repository user interface in creating high quality metadata including automating metadata creation and automatically verifying metadata quality.

  4. Researchers would like it to be easier to upload large and complex data and to generate metadata in an automated fashion.

  5. Usability is key including ease of use, integrations, and automated methods (e.g. API30) especially functionality to automate workflows and add repositories into existing processing workflows.

  6. Searching for data in repositories is not a priority use case for this community; researchers are more likely to find open data based on a URL or DOI in a publication/preprint.

  7. Licensing, human subject data considerations, and GDPR requirements are concerns for repository use.

  8. Researchers are willing pay reasonable costs to share data in repositories for additional storage space for large datasets and for curation support.

GREI is grateful for the engagement of OSR participants in this session and the associated survey as it provided an important perspective on GR use and needs from a specific research community with advanced data sharing and open science practices. We will incorporate the learnings and feedback from this session into our work adapting repository functionality to meet researcher needs and facilitate an interoperable repository ecosystem for FAIR data sharing. In the coming year, GREI plans to publish a collection of real-world use cases of generalist repositories and hopes to capture uses from the neuroimaging community in these as well. GREI repositories will continue to develop our repository features including metadata, file capabilities, and automation as a result of feedback from this session and other community engagement efforts. We hope that GRs will continue to be valuable resources as part of the larger open neuroimaging landscape and look forward to continued engagement with the OHBM and OSR communities.

Enabling federated analysis using NVIDIA FLARE powered COINSTAC architecture and showcase new algorithms

Sandeep Panta

COINSTAC5 (Collaborative Informatics and Neuroimaging Suite Toolkit for Anonymous Computation) promotes collaborative research by removing large barriers to traditional data-centric approaches. It allows groups of users to run common analyses on their machines over their own datasets and other’s datasets with ease. The source data never leaves the user’s machine and the results of these analyses are synchronized to the cloud and undergo aggregate analysis processes using all contributor data. It also offers data anonymity through differentially private algorithms, so members do not need to fear protected health information (PHI) traceback. Federated (decentralized) pipelines enable distributed, iterative, and feature-rich analyses, opening up new possibilities for global collaborative research.

The goal of this discussion was to briefly introduce COINSTAC and its new features as outlined below:

  1. Statistical analysis using Singularity containers.

  2. Command line pre-processing tools to support data analysis on clusters with job schedulers.

  3. New curated accessible datasets via COINSTAC Vaults.

  4. New algorithms including but not limited to Decentralized Source Based Morphometry.

  5. New COINSTAC federated analysis architecture powered by NVIDIA FLARE architecture.

We would like to hear feedback about our software such as how to improve the experience for researchers. We welcome anyone who wants to contribute to this open source and open data project with their datasets, algorithms, and code. We would also like to work with other organizations to pursue grants together, including small business grants. Collaborating with other organizations is the best way for us to answer interesting neuroscience-related questions that would not have been possible without COINSTAC and COINSTAC Vaults.

During the discussion, the audience expressed that COINSTAC would be a valuable tool for neuroscience research and that the software needs to be promoted more in the community via research groups, workshops etc.

CONCLUSION AND FUTURE DIRECTIONS

Selma Lugtmeijer, Ju-Chi Yu

The OSR offered a space for sharing ideas and fostering discussions on how open science can advance neuroscience research. Novel ideas were pitched, and various challenges were highlighted. The discussions revealed both common ground on certain issues and differing opinions on others. The active participation of many individuals during the Table Talks underscored the relevance of the topics and indicated that a safe space had been created for voicing diverse perspectives.

The dominant overarching theme that emerged across sessions was the question of incentives - how the current academic reward systems sometimes support but often hinder or pose challenges to the adoption of open science practices. Panellists and participants repeatedly emphasized that while the ideals of open science are widely shared, researchers often face misaligned incentives: contributions such as software development, dataset curation, or community engagement are rarely recognized in hiring, promotion, or funding decisions. As the need for education on and promotion of tools related to open science remains, recurring challenges and opportunities also surfaced from discussions across sessions. For the general research community, financial, institutional, and cultural barriers to open science still exist and disproportionately affect early career researchers and those from non-Western regions. For open tools developers, the sustainability of open infrastructures, from software maintenance to repository governance, was another cross-cutting concern. For users and developers of open datasets, ethical considerations were raised around privacy, data protection, and commercial partnerships highlighted the conflict between openness and responsibility, calling for a more complementary approach in the open science framework with necessary constrained access. Finally, several sessions touched on the environmental and technical infrastructure that underpins open science, including the need for greener computing practices. Together, these themes point to a maturing open science movement, one that focus not only on increasing accessibility and reproducibility, but also on developing the structural, cultural, and institutional foundations needed to sustain and support these practices into the future.

Despite these valuable insights, there is also room for improvement. The OSR could benefit from reflecting on which topics are best suited to specific formats. Maybe not every topic needs to be addressed as both a Panel Discussion and a Table Talk. For instance, a more applied topic like “Getting started in open science” is well-suited for a Panel Discussion, where panelists who have navigated that journey can field questions. Conversely, a topic focused on change or calls for action, such as crowdsourcing, may be more effectively addressed in a Table Talk format, allowing for diverse voices to be heard and fostering collaboration. This is something to keep in mind for the following editions.

A second point pertains to visibility within the OHBM community. Feedback from participants in the OSR indicates that the space for open discussions around the broad topic of open science is highly valued. However, many OHBM attendees are unaware of the events taking place in the OSR. Future iterations of the OSR will benefit from improved integration with the main OHBM program.

Finally, in the future, we wish to bring back more Emergent Sessions and a free working space where discussions happen more organically. Future OSR will feature fewer planned sessions with more space for spontaneous discussions.

In summary, the OSR offers a unique space at the OHBM conference for discussion and networking. We hope it will continue to do so for many years to come.

OSR 2024 Funding Sources

We thank the Wellcome Centre for Integrative Neuroscience and OpenNeuro for their sponsorship to make the events in the OSR 2024 in Seoul possible.

Author Funding Sources

Selma Lugtmeijer: SL was partly funded by a Guarantors of Brain travel grant and partly by the Canadian Institutes of Health Research (PJT 180591) grant awarded to Karen L. Campbell; Ju-Chi Yu: Grant support from the Discovery Fund of the Centre for Addiction and Mental Health (CAMH); Xiangzhen Kong: National Natural Science Foundation of China (32171031, 32400882), STI 2030-Major Projects (2021ZD0200409), and the Information Technology Center, Zhejiang University; Janine Diane Bijsterbosch: R01 MH128286 & R01 MH132962; Oscar Esteban: Swiss NSF; Seok-Jun Hong: The National Research Foundation of Korea (NRF) grant funded by the Korea Government (MSIT) (NRF-2022R1C1C1007095, RS-2023-00217361, RS-2024-00398768); Shella Keilholz: NIH 1R01EB029857;Chun-Chia Kung: This study was funded in part by Higher Education Sprout Project, Ministry of Education to the Headquarters of University Advancement at NCKU, NCKU-D113-B2980; Daniel Margulies: European Research Council Grant 866533-CORTIGRAD; Cyril Pernet: Novo Nordisk fonden (NNF20OC0063277); Franco Pestilli: NIH NIBIB R01EB029272, R01EB030896NSF, R01EB030896, NSF BCS 1734853 and 1636893, NSF ACI 1916518, NSF IIS 1912270, Wellcome Trust 226486/Z/22/Z, a gift from the Kavli Foundation and a Microsoft Investigator Fellowship to Franco Pestilli. NIMH UM1NS132207 to K. Ugurbil; Jean-Baptise Poline: JBP was partially funded by the National Institutes of Health (NIH) NIH-NIBIB P41 EB019936 (ReproNim) The Canadian Institutes of Health Research (CIHR PJT-185948), the This work was partially funded by the National Institutes of Health (NIH) NIH-NIBIB P41 EB019936 (ReproNim) The Canadian Institutes of Health Research (CIHR PJT-185948), The Natural Sciences and Engineering Research Council of Canada; Won Mok Shim: IBS-R015-D1 from the Institute for Basic Science funded by the Korean government; Chao-Gan Yan: National Natural Science Foundation of China (grant numbers: 82122035); Niall Duncan: Taiwan National Science and Technology Council (110-2628-H-038-001-MY4); Nikhil Bhagwat: Lab travel funds; Peter Fox: NIH MH074457; Ana Van Gulick: NIH OT2DB000006; David Kennedy: NIH grant award NIH-NIBIB P41 EB019936; Neda Sadeghi: National Institutes of Health Intramural Research Program; Sandeep Panta: https://reporter.nih.gov/project-details/10058463; Isabelle van der Velpen: National Institute of Mental Health, National Institutes of Health; Tonya White: Intramural Research Program of the National Institute of Mental Health; Qing Wang: Shanghai Mental Health Center key project: 2023zd01; Povilas Karvelis: CIHR; Yu-Fang Yang: OHBM; Hong Ji Kim: IBS-R015- D1 (Institute for Basic Science); Wei Zhang: McDonnell Neuroscience Postdoctoral Travel Award, McDonnell Center, Washington University in St. Louis; Jess Haigh: Medical research council doctoral training partnership; Hurshitha Vasudevan: OHBM travel award; Yuping Yang: UoM PhD research budget; Yi-Ju Lee: Academia Sinica Scientific Research Fund - Overseas Program for Elite Scholars in Humanities and Technology [grant number: NBA11353017-44]

Acknowledgements

A special thanks to Dr. Hiromasa Takemura, panelist in session 2.4 Open science in Asia/Korea. The authors also thank colleagues, supervisors, and collaborators: Dr. Chun-houh Chen and Dr. Hsin-Chou Yang of Smart Health Project, Academia Sinica, collaborators of the NIH Generalist Repository Ecosystem Initiative (GREI), Dr. Vince Calhoun, Dr. Sergey Plis, Dr. Jessica Turner, Dr. Anand Sarwate, Dr. Catie Chang, Kimberly Rogge-Obando, Dr. Matthew A. J. Apps, Dr. Karen L. Campbell.

Conflicts of Interest

Ana Van Gulick is employed by Figshare, a part of Digital Science, and a repository that receives funding from the NIH Generalist Repository Ecosystem Initiative. Hyeong Hun Lee is employed by METLiT Inc.