(Press-News.org) Researchers from more than 50 international institutions have launched Open Problems (https://openproblems.bio), a collaborative open-source platform to benchmark, improve, and run competitions for computational methods in single-cell genomics. Co-led by Helmholtz Munich and Yale University, the initiative aims to standardize evaluations, foster reproducibility, and accelerate progress towards open challenges in this fast-moving field.
A Common Language for a Complex Field
Single-cell genomics allows scientists to analyze individual cells at unprecedented resolution, revealing how they function, interact, and contribute to health and disease. But as the field has grown, so has the number of computational tools – now numbering in the thousands – designed to process and interpret this complex data. This rapid growth presents a major challenge: how can researchers identify the most suitable tool, or determine the best combination of processing steps to achieve a specific analytical goal? Many tools are specialized, and evaluating their performance is challenging due to the limited availability of datasets with known, accurate outcomes (so-called ground truth). As a result, researchers often turn to large-scale benchmarking studies. However, these studies can be inconsistent, quickly become outdated, and often make comparisons difficult to interpret – making it challenging to identify the best method for a given task.
“We need a common language to measure what works – and what doesn’t – that can stand the test of time,” says Prof. Fabian Theis, Director of the Computational Health Center at Helmholtz Munich and Professor at the Technical University of Munich. “With Open Problems, we’re introducing a reproducible, living, and transparent framework to guide tool development and evaluation – one that the community can actively shape and use.”
Transparent, Reproducible, and Community-Driven
Open Problems currently includes 81 public datasets and tests 171 methods across 12 core tasks in single-cell analysis. Each method is evaluated using a suite of metrics – quantitative measures that show how well a method performs on a specific task. These metrics include accuracy, scalability, and robustness, among others, and are chosen based on the goals of each task. In total, 37 different metrics are used across the platform, with each task using the most relevant ones.
All evaluations run automatically in the cloud and follow standardized procedures to ensure the results are fully reproducible. Researchers can see how each method performs, explore the underlying code, and suggest improvements. To remain relevant and impactful over the long term, the platform is designed to be open to contributions: scientists can propose new tasks, add their own methods, join regular community calls, and take part in collaborative hackathons to help shape the future of the project.
Real-World Benefits
By comparing tools side by side, Open Problems helps researchers identify the most effective methods for their specific scientific questions and often challenges established assumptions in the process. As Dr. Smita Krishnaswamy, Associate Professor of Genetics and of Computer Science at the Yale School of Medicine, explains: “We found that looking at overall patterns of gene activity gives more accurate results than focusing on individual genes when studying how cells communicate. And for some tasks, like identifying cell types across different datasets, a simple statistical model can actually outperform complex AI methods, making the analysis both faster and more efficient for many researchers.”
The platform also powers major machine learning competitions, including the NeurIPS multimodal integration challenges. These global contests bring together experts in biology and artificial intelligence to solve real-world problems using common datasets and evaluation standards.
“Open Problems lowers the barrier for AI researchers outside biology to contribute to genomics,” says Dr. Malte Lücken, who co-led the project. “It’s a blueprint for interdisciplinary innovation.”
All code and results are openly available under a CC-BY licence at github.com/openproblems-bio/openproblems.
Original Publication
Lücken et al., 2025: Defining and benchmarking open problems in single-cell analysis. Nature Biotechnology. DOI: 10.1038/s41587-025-02694-w
About the Researchers
Prof. Fabian Theis, Head of the Computational Health Center and director of the Institute of Computational Biology at Helmholtz Munich; Professor of Mathematical Modelling of Biological Systems at the Technical University Munich (TUM)
Dr. Smita Krishnaswamy, Associate Professor of Genetics and of Computer Science at the Yale School of Medicine
Dr. Malte Lücken, Group leader at the Institute of Computational Biology and the Institute of Lung Health & Immunity at Helmholtz Munich
About Helmholtz Munich
Helmholtz Munich is a leading biomedical research center. Its mission is to develop breakthrough solutions for better health in a rapidly changing world. Interdisciplinary research teams focus on environmentally triggered diseases, especially the therapy and prevention of diabetes, obesity, allergies, and chronic lung diseases. With the power of artificial intelligence and bioengineering, researchers accelerate the translation to patients. Helmholtz Munich has around 2,500 employees and is headquartered in Munich/Neuherberg. It is a member of the Helmholtz Association, with more than 43,000 employees and 18 research centers the largest scientific organization in Germany. More about Helmholtz Munich (Helmholtz Zentrum München Deutsches Forschungszentrum für Gesundheit und Umwelt GmbH): www.helmholtz-munich.de/en
END
Since their first detection, powerful bursts of X-rays from distant galaxies, known as fast X-ray transients (FXTs), have mystified astronomers. FXTs have historically been elusive events, occurring at vast distances away from Earth and only lasting seconds to hours. Einstein Probe (EP), launched in 2024, is dedicated to observing transient events in the X-ray and is changing the game for astronomers looking to understand the origin of these exotic events.
In January 2025 EP alerted astronomers to the nearest FXT known at the time, named EP 250108a. Its proximity to Earth (2.8 billion light-years away) ...
During the coronavirus pandemic, german universities had to act quickly: Lectures and seminars had to be offered online via Zoom. After the pandemic-related lockdowns, many lecturers introduced synchronous hybrid teaching/learning settings. These are courses in which students can take part either on site in the seminar room or online at the same time.
Students are therefore faced with the decision every session as to whether they want to take part in the course online or on site. Three researchers from the Professorship of Adult Education/Continuing Education at the Institute of Education at Julius-Maximilians-Universität Würzburg (JMU) have investigated the factors ...
High-resolution flow field data are essential for accurately evaluating the aerodynamic performance of aircraft. However, acquiring such data via large-scale numerical simulations or wind tunnel experiments is highly resource-intensive. Flow field super-resolution techniques aim to reconstruct high-resolution information from low-resolution data, significantly improving data acquisition efficiency. With the rapid advancement of artificial intelligence, especially deep learning, neural network-based super-resolution methods have been widely adopted for flow field reconstruction. Nonetheless, these ...
The conceptual design stage is a key step in aircraft development, laying the foundation for performance, efficiency and innovation. Traditionally, this process relies heavily on experienced designers to iterate designs based on design theory, design experience, and engineering calculations, which places high demands on designers' design experience and professional skills. With the continuous improvement of aircraft design requirements and the continuous expansion of application scenarios, it is necessary to explore new conceptual design methods to free designers from a large amount of low-value, highly repetitive, and experience-dependent ...
Scientists have been working on the artificial production of blood for several decades. Now, researchers from the University of Konstanz and Queen Mary University of London have taken an important step closer to that goal with a new discovery.
Roughly 15,000 units of blood are needed daily in Germany, most of which currently come from donations. Research into developing alternative sources, such as large-scale artificial blood production, has been ongoing for decades but is still far from reaching its widespread utility. The main challenge ...
The Vortex Particle Method (VPM), a meshless vortex flow simulation approach, is gaining traction for its efficient simulation of unsteady vortex wakes evolution that is shed by aircrafts, rotors and wind turbines. It outperforms traditional grid-based CFD methods with faster computation, lower dissipation, and easier satisfaction of the CFL stability condition. However, traditional VPM has huge challenge on accurately simulating these complex flows, due to its poor numerical stability, which is compromised by factors such ...
Nitrogen fixation is a critical ecological process that converts atmospheric nitrogen into bioavailable forms, essential for plant growth and carbon sequestration. This study, published in Forest Ecosystems, focused on two primary forms of biological nitrogen fixation (BNF): symbiotic nitrogen fixation (SNF), which occurs within the root nodules of nitrogen-fixing plants, and asymbiotic nitrogen fixation (ANF), which is carried out by free-living microorganisms in soil and litter. Understanding the environmental controls on these processes is crucial for predicting ...
Immune checkpoint pathways regulate T cell function and play pivotal roles in the treatment of both cancer and autoimmune diseases. One key component of these pathways is Lymphocyte Activation Gene 3 (LAG-3)—a classical immunosuppressive receptor that has long posed unresolved questions regarding its biological mechanisms.
In a new study published on June 30 in Cell, researchers from the Institute of Biophysics of the Chinese Academy of Sciences, the Grossman School of Medicine at New York University, and the School of Medicine at Zhejiang University have ...
A pairing of two experimental drugs inhibits tumor growth and blocks drug-induced resistance in ovarian cancer, according to a preclinical study led by Weill Cornell Medicine investigators. The research reveals a promising strategy against this hard-to-treat malignancy, and more generally demonstrates a powerful new approach for the identification of effective regimens to treat genetically diverse cancers.
Ovarian cancer is genetically diverse in the sense that it can be driven by mutations in many different genes. This complicates the standard strategy of developing drugs to target common driver mutations. In the study, published July 7 in Cell Reports Medicine, the researchers applied a ...
“Space ice” contains tiny crystals and is not, as previously assumed, a completely disordered material like liquid water, according to a new study by scientists at UCL (University College London) and the University of Cambridge.
Ice in space is different to the crystalline (highly ordered) form of ice on Earth. For decades, scientists have assumed it is amorphous (without a structure), with colder temperatures meaning it does not have enough energy to form crystals when it freezes.
In the new study, ...
