CZI Jamboree Launches Working Group for Single-Cell Benchmarking

CZI is driving forward the development of a robust benchmarking infrastructure for biological modeling, with a core focus on ensuring these benchmarks are both biologically relevant and broadly useful to the community.

Recently, CZI convened a specialized jamboree, bringing together machine learning researchers and computational biologists specializing in single-cell transcriptomics. This gathering addressed the complex challenge of benchmarking models trained on single-cell transcriptomic data. Through focused working sessions, participants explored the unique considerations involved in benchmarking generative, cross-species, and multi-modal methods. A primary goal of the event was to define biologically relevant tasks, aiming to accelerate the field and deliver tangible benefits to biologists. This collaborative effort resulted in the definition of key benchmarking assets, paired with a comprehensive implementation plan.

Building on the momentum from the jamboree, a dedicated working group has been formed, comprising both jamboree attendees and members of the wider community. This group is tasked with developing standardized benchmarking assets for evaluating generative, cross-species, and multimodal models trained on single-cell transcriptomic data. By prioritizing biological relevance, the group aims to advance the state of benchmarking in this critical domain. Ultimately, the working group's output will be available on CZI's benchmarking platform, ensuring robust and consistent evaluation methods are available to the community. If you are interested in joining, please contact Liz Fahsbender at efahsbender@chanzuckerberg.com.

Working Group for Single-Cell Benchmarking Members

Alma Andersson, Genentech, gCS, BRAID

Ediem Al-jibury, Bioptimus

Lisa Chen, Yale University

Silvia Domcke, University of Zurich

Valentina Giunchiglia, Imperial College London, Harvard University

Tobias Heinen, Bioptimus

Yepeng Huang, Harvard Medical School

Ana-Maria Istrate, Chan Zuckerberg Initiative

Xiang Lin, Harvard Medical School

Tianyu Liu, Yale University

Malte Luecken, Integrative Genomics Lab, Helmholtz Munich

Giovanni Palla, Chan Zuckerberg Initiative

Irene Papatheodorou, Earlham Institute

Thomas Peeters, Bioptimus

Anna Schaar, Bioptimus

Yuyao Song, EMBL-EBI, University of Cambridge

Philipp Sven Lars Schäfer, Heidelberg University

Dasha Valtar, Bioptimus

Ramon Viñas, EPFL - Swiss Federal Technology Institute of Lausanne

Chloe Wang, University of Toronto

Geifei Wang, Yale University

Jia Zhao, Yale University

Marinka Zitnik, Harvard Medical School, Kempner Institute for the Study of Natural and Artificial Intelligence, the Broad Institute of MIT