A Framework for Identifying, Assessing and Mitigating Biological Bias for AI in Biology

A compilation of resources that offer a framework for identifying and addressing biases in AI models for biology.

Katrina Kalantar | December 9, 2024

Introduction

Artificial intelligence (AI) has become a powerful tool in biology, helping us solve complex problems with genomic data, protein folding, and more. Powering these advances are the use of novel deep learning approaches, integration of massive datasets, and increasingly interdisciplinary teams. The merging of these aspects presents many opportunities, capturing ambitions of biologists and ML researchers in creating virtual cells (1). As biologists and AI experts dive deeper into these collaborations, critical challenges around bias in biological data have surfaced. We're excited to introduce a framework we've been working on—the Biological Bias Assessment Guide. It's designed to provide a structured framework with a unified vocabulary to help AI developers and biologists identify and address bias at key points in the development process, ensuring the models we build are robust, reliable, and truly useful across biological applications.

Let's walk through the motivation behind this guide, the unique challenges we face when developing AI for biology, and how this guide is structured to tackle these issues.

The Challenge of Bias and Pitfalls of Applying AI to Biology

Bias in machine learning refers to the phenomenon that occurs when an algorithm produces results that are systemically prejudiced due to erroneous assumptions in the ML process (2).

For AI models in biology to be truly effective in advancing the ambitious goals of curing, managing, and preventing all disease, they must capture underlying biological phenomena with reliable accuracy and performance across all humans, regardless of factors like ancestry, sex, or age. Likewise, models of molecular processes must account for and generalize across the full spectrum of biological diversity, including variations in cell types, tissues, and molecular interactions. However, bias in these models can severely undermine this goal, leading to inconsistent results, distorted biological interpretations, and reduced utility across different domains and human and molecular populations.

We’re certainly not the first to point out these problems. In fact, there’s a lot of research on the common pitfalls of using machine learning in genomics and other biological fields. Issues like distributional differences (where training and testing data are drawn from different populations), confounding variables, and data leakage are well documented (2). These challenges are often amplified in biology due to the complexity of biological systems and the assays that are used to interrogate them.

Models are developed based on historical data, which can harbor a range of biases with the potential to propagate into the learned systems at multiple stages, necessitating careful consideration throughout the course of model development. While there are guidelines for mitigating bias in healthcare data, biology-specific resources have been lacking. That’s where this guide comes in.

The Biological Bias Assessment Guide: Bridging the ML-Biology Gap

The interdisciplinary nature of AI for biology presents exciting opportunities, but also presents unique challenges. One of the biggest barriers to creating reliable AI tools in biology is the communication gap between machine learning (ML) scientists and biologists. These two groups often use different languages, and they’re used to different workflows. As a result, ML scientists might not understand the biological nuances that introduce bias, while biologists might struggle with the intricacies of how AI models handle data.

The Biological Bias Assessment Guide aims to bridge this gap. It’s structured to help both biologists and ML developers work together to identify sources of bias that could compromise their models. Specifically, it guides teams through four critical phases of model development and deployment:

  1. Data Considerations – Are there inherent biases in the type of biological data being collected? For instance, is your dataset over-representing certain populations or species, or are there technical artifacts that may influence the underlying distributions?

  2. Model Development – How might these biases impact your model during training? It’s crucial to ask whether your model is learning meaningful biological signals or overfitting to quirks in the data.

  3. Model Evaluation – Do the metrics and datasets used for evaluating your model reflect real-world conditions? Do they provide sufficient resolution to quantify robustness against sources of bias? For example, testing a model on a dataset similar to the training data can inflate performance metrics while high-level metrics may obfuscate critical variability in performance across classes.

  4. Post-Deployment – Bias doesn’t stop when the model is deployed. How does your model perform in new contexts? Regular checks and updates are essential to ensure that your AI tool remains reliable as new data is processed.

At each of these stages, the guide provides a series of prompts and reflection questions designed to help teams identify, assess, and mitigate sources of bias.

The Biological Bias Assessment Guide in Action

Here is an example workflow for how the Biological Bias Assessment Guide might be integrated into your work:

  1. Become aware of trends and best practices: Start by using available resources to understand common sources of bias in biological data and explore methods for identifying and mitigating these biases ahead of model development (2-4).
  2. Reflect on your research question: Early in project planning, use the guide to systematically reflect on potential sources of variability in your data, such as tissue types or experimental conditions, and consider how these might impact your model.
  3. Perform targeted experiments and analysis: Design and execute experiments to directly test and quantify the sources of bias you’ve identified, ensuring your model is as robust and fair as possible.
  4. Documentation is key! As you develop your model, make documentation a priority. Tools like Model Cards and Data Cards can help ensure transparency and track your findings throughout the project.

At CZI, we’re piloting these steps towards integrating the Biological Bias Assessment Guide to identify potential sources of bias early in model development. The experiment is ongoing!

An Ecosystem of Resources

The Biological Bias Assessment Guide was developed to extend existing resources aimed at supporting careful consideration throughout model development, with a particular focus on pitfalls associated with molecular biological data. But this is just one tool within a broader ecosystem of pre-existing work. A few key resources to support similar aims generalized across ML domains are:

  • REFORMS guidelines – presents a consensus-based checklist for improving the transparency, reproducibility, and validity of machine learning (ML)-based science. It addresses common pitfalls in ML research, including issues with performance evaluation, reproducibility, and generalizability across disciplines.
  • Datasheets for Datasets – proposes a standardized documentation method for machine learning datasets, similar to datasheets used in the electronics industry. These datasheets aim to document the motivation, composition, collection process, and recommended uses of datasets to promote transparency, reproducibility, and accountability. By providing clear documentation, the goal is to mitigate societal biases in machine learning models and help researchers and practitioners select more appropriate datasets for their tasks.

Your Input Matters

We’ve designed the Biological Bias Assessment Guide as an evolving tool, and we’re eager to hear feedback from the community. We believe this guide will enhance the reliability and transparency of AI tools in biology, but we also know there’s room for improvement. Whether you’re a biologist, a computational scientist, or somewhere in between, we want your input! Tell us how you’re using the guide, and let us know what works and what doesn’t.

We encourage you to try it out on your next AI project. Together, we can make AI work for biology—not the other way around.

Relevant Resources:

  1. Bunne, Roohani, Rosen, et al. How to Build the Virtual Cell with Artificial Intelligence: Priorities and Opportunities. arXiv. 2024
  2. Whalen S, et al. Navigating the pitfalls of applying machine learning in genomics. Nat Rev Genet. 2022.
  3. REFORMS Checklist for ML-based Science
  4. Gebru et al., Datasheets for Datasets. arXiv. 2021