From Genomics to AI Biology (Series 3)

 Why AI Biology Needs Better Evaluation

By Li Lei on July 13, 2026

In AI biology, we love numbers.

Training loss.
Validation loss.
Perplexity.
AUROC.
AUPRC.
Correlation.
Benchmark rankings.

A new foundation model improves one of these numbers, and we often call it progress.

Sometimes it is.

But increasingly, I find myself asking a harder question:

What does better model performance actually mean for biology?

Imagine that a DNA foundation model is trained on billions or trillions of nucleotides.

Its training loss falls.

Its validation loss falls.

Its sequence likelihood improves.

The optimization worked.

But what exactly have we learned?

Does the model understand gene regulation better?

Can it predict the effect of a variant it has never encountered?

Does its representation transfer to another species?

Can it predict what happens after a genetic perturbation?

Can it identify a biologically meaningful candidate that a simpler model would miss?

And ultimately:

Would its prediction change what experiment a scientist chooses to do next?

Those are very different questions.

Recent evaluations of DNA and single-cell foundation models make this distinction increasingly important. Large pretrained models do not automatically outperform specialized models or even simple baselines across every biological task. Performance can depend strongly on training objective, architecture, pretraining diversity, biological context, task definition, and how evaluation data are separated from training data [1–8].

At the same time, models such as AlphaGenome and Evo 2 demonstrate that large biological models can achieve remarkable capabilities when model design and evaluation are tightly connected to meaningful biological questions [9,10].

So the lesson is not:

Foundation models do not work.

Nor is it:

Bigger models will solve biology.

The more interesting lesson is:

There is no single metric that tells us whether a foundation model understands biology.

In the first essay of this series, I argued that biologists need to move beyond simply using AI tools and learn to think in AI systems.

In the second, I argued that biology is not just data — biological meaning depends on context.

Now comes the next question:

How do we know when an AI result deserves to influence science?

A lower loss is not biological understanding.

A benchmark score is not biological truth.

A prediction is not a mechanism.

And a prediction is certainly not yet a discovery.

---

1. When does a prediction become a discovery?

I find it useful to think about AI biology evaluation as a six-level stack.

Figure 1. When does an AI prediction become a discovery?
AI biology evaluation can move from foundation-model health through task performance, generalization, biological validity, experimental validation, and finally scientific decision value.

At the bottom, we ask a very technical question:

Did the model learn its training objective?

At the top, we ask a much more scientific question:

Did the model help us make a better decision or discover something new?

Those are not the same thing.

A model can perform beautifully at Level 0 and disappoint at Level 1.

It can perform well on a benchmark and collapse when confronted with a new species or experimental condition.

It can generalize statistically while learning a shortcut rather than a meaningful biological relationship.

And it can generate a biologically plausible hypothesis that fails when tested experimentally.

That is why I think we need to stop talking about AI evaluation as if it were one number.

It is a chain of evidence.

---

2. What does training loss actually tell us?

Before asking whether a foundation model understands biology, we should ask:

Did training work?

During pretraining, researchers may monitor:

• training loss

• validation loss

• cross-entropy

• token or sequence likelihood

• masked-token reconstruction

• calibration

• convergence

• scaling behavior

These metrics are essential.

If training loss does not decrease, optimization may be failing.

If training loss continues improving while validation loss deteriorates, the model may be overfitting.

But there is a crucial distinction.

Figure 2. Lower loss does not automatically mean better biology.
Training metrics tell us whether a model is learning its optimization objective. They do not by themselves establish downstream biological capability.

A lower validation loss tells us:

the model has become better at its mathematical objective.

It does not automatically tell us:

the model understands enhancers better;

predicts gene expression better;

generalizes across species better;

predicts perturbations better;

or makes better scientific decisions.

This distinction is particularly important for foundation models.

A masked DNA language model, an autoregressive nucleotide model, and a model using different tokenization schemes may optimize very different objectives. Their losses may not even be directly comparable.

Recent benchmarking supports this caution.

Feng and colleagues evaluated several DNA foundation models across genomic and genetic applications and found substantial task-dependent differences. General-purpose models were competitive for some problems, such as pathogenic-variant identification, while performing less strongly on others, including gene-expression prediction and causal-QTL identification [1].

Tang and colleagues similarly found that pretrained genomic-language-model representations did not consistently outperform conventional sequence representations or strong supervised models on biologically aligned regulatory-genomics tasks [2].

The important lesson is:

A foundation model can become a better model of its training data without necessarily becoming a better model of the biology we care about.

---

3. Foundation-model evaluation is a portfolio, not one score

Instead of searching for one universal metric, I think biological foundation models need several complementary types of evaluation.

Figure 3. Foundation-model evaluation is a portfolio, not one score.
Intrinsic, downstream, generalization, and biological evaluation answer fundamentally different questions.

Intrinsic evaluation

Did the model learn its pretraining objective?

Loss, likelihood, masked-token recovery and convergence belong here.

Downstream evaluation

Can the learned representation solve biologically meaningful tasks?

For DNA models, this might include:

variant-effect prediction,

enhancer classification,

gene-expression prediction,

splice prediction,

or regulatory activity.

For single-cell models:

cell-state representation,

perturbation response,

cell-type prediction,

or gene-expression reconstruction.

Generalization evaluation

Does the model still work when it encounters biology that is meaningfully different from what it saw during training?

New species?

New populations?

New tissues?

New sequence families?

New perturbations?

Biological evaluation

Do the predictions survive independent biological evidence?

Genetics?

Evolution?

Functional assays?

Perturbation?

Prospective validation?

These categories answer different questions.

That is why statements such as:

“Model A has lower perplexity, therefore it is better for biology”

are too strong.

Scale can create new capability.

But evaluation tells us what capability was actually created.

---

4. Zero-shot performance is useful — but it needs careful interpretation

One particularly interesting promise of foundation models is zero-shot prediction.

Imagine introducing a nucleotide variant into a sequence.

A genomic language model can compare the probability or likelihood assigned to the reference and alternative sequence.

If the variant disrupts a strongly learned sequence pattern, the model may assign the altered sequence a lower probability.

That creates a variant-effect score without training directly on pathogenicity labels.

Conceptually, this is powerful.

But recent work shows why evaluation matters.

Alfisi and colleagues systematically compared DNA foundation models for zero-shot variant-effect prediction and found that performance depended on more than model size. Architecture, training objective, sequence context, and diversity of pretraining data all mattered [3].

One particularly interesting observation was the value of multi-species training.

Evolution has already performed billions of years of natural experiments.

Training across diverse species may therefore expose models to patterns of biological constraint that are difficult to learn from one genome alone.

This suggests something deeper:

the composition of the training data may matter as much as the number of model parameters.

And it reinforces why foundation-model evaluation cannot be reduced to a scaling curve.

---

5. Strong baselines keep us scientifically honest

One of the most useful lessons from recent AI biology studies is remarkably simple:

Always compare a sophisticated model against a strong simple baseline.

This has become especially clear in single-cell biology.

Kedzierska and colleagues evaluated Geneformer and scGPT in zero-shot settings and found that their representations did not consistently outperform simpler established approaches [5].

Ahlmann-Eltze and colleagues examined prediction of transcriptional responses to genetic perturbation and found that several deep-learning approaches did not consistently outperform deliberately simple linear baselines [6].

This does not mean that single-cell foundation models have failed.

It means that a complicated model should earn its complexity.

If a very large model cannot outperform a linear model or strong task-specific baseline on the capability being claimed, that tells us something important.

Perhaps the objective needs improvement.

Perhaps the benchmark is too easy.

Perhaps the training data do not contain enough causal information.

Perhaps the representation captures cell identity well but not intervention response.

Emerging bioRxiv work is pushing this idea further. The scFME framework, for example, evaluates single-cell foundation models specifically for in-silico perturbation, comparing predicted changes against control, zero, and random perturbation baselines rather than simply asking whether embeddings appear structured [7].

That distinction matters.

A beautiful embedding is not necessarily a useful perturbation model.

A beautiful UMAP is not a discovery.

---

6. Is the model generalizing — or recognizing something familiar?

This is one of the most important problems in biological machine learning.

Biological observations are rarely independent.

DNA sequences share homology.

Proteins belong to families.

Individuals share ancestry.

Cells may come from the same donor.

Species share evolutionary history.

Experiments share protocols and batch effects.

A random train-test split can therefore create the illusion of generalization.

Figure 4. Unseen rows do not necessarily mean unseen biology.
A random split may place highly homologous biological examples in both training and test sets. Biology-aware splits provide a stronger test of genuine generalization.

Suppose Gene A1 and Gene A2 are in training.

Gene A3 is placed in the test set.

Technically, A3 is unseen.

But if A3 is highly homologous to A1 and A2, has the model really encountered new biology?

Not necessarily.

Recent bioRxiv work by Rafi and colleagues examined this problem explicitly, demonstrating how homology can create leakage between training and evaluation partitions in genome-trained sequence models [4].

This leads to one of the most important principles in biological AI evaluation:

A test example is not truly “unseen” simply because its row was absent from the training set.

Biological independence matters.

Depending on the scientific question, stronger evaluation may require holding out:

an entire chromosome,

a gene family,

a homology cluster,

a donor,

a population,

an environmental condition,

or an entire species.

These evaluations are harder.

But discovery happens precisely when models encounter something they have not already seen.

---

7. Did the model learn biology — or a shortcut?

Even if a model generalizes statistically, we still need to ask:

What did it actually learn?

Suppose an AI system predicts genes associated with drought tolerance.

Perhaps the top candidates are:

expressed in the relevant tissue,

responsive under drought,

near genetic associations,

supported by chromatin accessibility,

connected to stress-response pathways,

or evolutionarily conserved.

Our confidence increases.

But perhaps the model instead learned:

GC content,

gene length,

annotation density,

batch identity,

phylogenetic similarity,

or another feature correlated with the label.

Both models might produce high AUROC.

Only one may teach us useful biology.

This is where interpretability becomes valuable.

Not because every neural network must produce a simple human-readable explanation, but because probing model representations can reveal whether recognizable biological features have emerged.

Evo 2, for example, reported internal features associated with exon-intron boundaries, transcription-factor-binding sites, protein structural properties, and prophage-related sequence [10].

That is encouraging.

But interpretability is still not the final test.

A model feature that resembles biology is evidence.

It is not yet mechanism.

---

8. AlphaGenome: evaluation closer to biological measurement

AlphaGenome offers an interesting example of evaluation that is more closely aligned with biological observables.

Rather than producing only a generic embedding, AlphaGenome predicts thousands of genomic tracks from long DNA sequences, including:

gene expression,

transcription initiation,

chromatin accessibility,

histone modifications,

transcription-factor binding,

chromatin contacts,

and splicing [9].

Its variant-effect evaluations then ask how sequence changes alter these measurable molecular outputs.

That is scientifically attractive because the evaluation moves closer to quantities biologists actually measure.

The principle is broader than AlphaGenome:

The closer an AI evaluation is to a meaningful biological measurement or intervention, the stronger the scientific claim we can make.

Predicting a benchmark label is useful.

Predicting an experimentally measurable consequence is stronger.

Predicting a new experimental outcome prospectively is stronger still.

---

9. Can nature prove the model wrong?

For me, this is where AI biology becomes most exciting.

A good scientific prediction should eventually confront reality.

Imagine an AI system predicts that a regulatory sequence controls a drought-responsive gene.

What would test that hypothesis?

CRISPR perturbation?

Reporter assay?

Perturb-seq?

Expression profiling?

Chromatin accessibility?

Protein binding?

A field experiment?

The exact experiment depends on the biological question.

But the principle is universal:

A scientific AI system should ideally generate hypotheses that can be falsified.

We should be able to ask:

What result would prove this prediction wrong?

Loss cannot answer that.

AUROC cannot answer that.

A benchmark cannot completely answer that.

Eventually, nature has to answer.

That is why prospective validation is so powerful.

Make the prediction first.

Then collect new evidence.

Blind prediction challenges such as CASP became influential in structural biology because predictions were evaluated against experimentally determined structures that participants did not know when making their predictions.

The answer existed outside the model.

AI biology needs more tests with this character:

prospective prediction,

blind validation,

independent experiments,

orthogonal assays,

and perturbational testing.

Instead of asking only:

Can the model explain biology we already measured?

we should increasingly ask:

Can it correctly predict biology we have not measured yet?

That is much closer to discovery.

---

10. Does AI change the next scientific decision?

There is one final level of evaluation that I think receives too little attention.

Does the AI system improve a decision?

Scientists rarely build models because they simply want more predictions.

Eventually, we need to decide:

Which gene should we perturb?

Which variant should we prioritize?

Which molecule should we synthesize?

Which experiment should we run?

Which hypothesis deserves another six months of work?

Imagine two models ranking 10,000 genes.

Model A:

AUROC = 0.92

Model B:

AUROC = 0.89

Model A appears better.

But the laboratory can test only ten genes.

Among those ten predictions:

Model A contains two validated candidates.

Model B contains six.

Which model is more scientifically useful?

Probably Model B.

This suggests that AI biology may need to pay more attention to metrics such as:

top-k enrichment,

experimental hit rate,

number needed to test,

cost per validated candidate,

search-space reduction,

and information gained from the next experiment.

These metrics move us from prediction performance toward decision value.

And ultimately, decision value may be one of the most important measures of AI's contribution to science.

---

11. What about scientific AI agents?

Evaluation becomes even more difficult when we move from models to agents.

A model might perform one task:

predict a variant effect.

An AI agent might:

interpret a scientific question,

search literature,

retrieve data,

write analysis code,

choose statistical methods,

run tools,

interpret results,

generate hypotheses,

and recommend an experiment.

How do we evaluate such a system?

Only the final answer?

The retrieved evidence?

The analysis code?

The statistical choices?

The reproducibility?

The proposed experiment?

One subtle error early in the workflow can propagate through everything downstream.

ScienceAgentBench was developed specifically to test data-driven scientific agents using 102 tasks derived from 44 peer-reviewed publications [12].

Even advanced agent configurations solved only a fraction of the tasks reliably.

That does not mean scientific agents are unpromising.

It means we should be cautious about moving from impressive demonstrations to claims of autonomous scientific discovery.

The more autonomous an AI system becomes,

the more rigorous its evaluation must become.

Not less.

---

Human agreement is not the final benchmark

There is another complication.

Human expertise is essential.

But human agreement cannot always be the ultimate standard.

AlphaGo's famous Move 37 is a useful reminder.

The move initially appeared highly unusual to expert players.

Its value became clear later.

Science may eventually encounter its own versions of Move 37.

An AI system may propose:

an unexpected gene,

a surprising regulatory mechanism,

an unusual protein sequence,

a non-obvious molecular interaction,

or an experiment an experienced scientist would never have prioritized.

If our evaluation asks only:

Does the expert agree with the AI?

we may reject precisely the discoveries AI could help us make.

So human judgment matters.

But the stronger question is:

Does reality agree?

Does the molecule function?

Does the sequence regulate expression?

Does the protein fold?

Does the perturbation change the phenotype?

Does the result replicate?

The ultimate judge of scientific AI should not be AI itself.

And it should not always be human intuition.

It should be evidence.

---

From prediction to discovery

We are entering an extraordinary period in biology.

Models such as AlphaGenome and Evo 2 demonstrate capabilities that would have seemed extraordinarily ambitious only a few years ago [9,10].

Foundation models are learning increasingly rich representations of DNA, proteins, cells, and molecules.

Scientific AI agents are beginning to interact with multi-step research workflows.

The opportunity is real.

But greater capability creates a greater need for scientific discipline.

As models become more powerful, generating plausible predictions becomes easier.

Determining which predictions deserve our trust becomes harder.

So perhaps the most important question in AI biology is no longer simply:

Can AI make a prediction?

We should ask:

Did the model actually learn?

Then:

Does it work on the biological task?

Then:

Does it generalize to genuinely new biology?

Then:

Did it learn meaningful biology rather than a shortcut?

Then:

Can we test its prediction?

Then:

Did it improve the next scientific decision?

And eventually:

Did it help us discover something we did not know before?

That is the journey from prediction to discovery.

Training loss tells us whether optimization is working.

Benchmarks tell us whether a model performs on defined tasks.

Strong baselines tell us whether complexity adds value.

Generalization tests tell us whether the capability survives unfamiliar biology.

Biological validation tells us whether the prediction survives context.

Experiment tells us whether nature agrees.

Decision value tells us whether AI actually changes science.

None of these alone is enough.

Together, they form a much stronger definition of progress.

The future of AI biology will therefore not be defined only by who builds the largest model,

trains on the most data,

achieves the lowest loss,

or reaches the top of a leaderboard.

It will be defined by our ability to distinguish between:

a well-trained model,

a strong benchmark result,

a biologically meaningful prediction,

a plausible hypothesis,

reliable evidence,

and a real discovery.

Those are not the same thing.

Knowing the difference is scientific judgment.

And as AI becomes more powerful, I believe that judgment will become more valuable — not less.

---

References and Further Reading

[1] Feng, H., Wu, L., Zhao, B. et al. Benchmarking DNA foundation models for genomic and genetic tasks. Nature Communications 16, 10780 (2025). doi:10.1038/s41467-025-65823-8.

[2] Tang, Z., Somia, N., Yu, Y. & Koo, P. K. Evaluating the representational power of pre-trained DNA language models for regulatory genomics. Genome Biology 26, 203 (2025). doi:10.1186/s13059-025-03674-8.

[3] Alfisi, I., Ciapi, F., Baragli, M. & Magi, A. Benchmarking DNA foundation models for zero-shot variant effect prediction shows the importance of context, training, and architecture. Genome Biology (2026). doi:10.1186/s13059-026-04238-0.

[4] Rafi, A. M., Kiyota, B., Yachie, N. & de Boer, C. Detecting and avoiding homology-based data leakage in genome-trained sequence models. bioRxiv (2025). doi:10.1101/2025.01.22.634321. Preprint.

[5] Kedzierska, K. Z., Crawford, L., Amini, A. P. & Lu, A. X. Zero-shot evaluation reveals limitations of single-cell foundation models. Genome Biology 26, 101 (2025). doi:10.1186/s13059-025-03574-x.

[6] Ahlmann-Eltze, C., Huber, W. & Anders, S. Deep-learning-based gene perturbation effect prediction does not yet outperform simple linear baselines. Nature Methods 22, 1657–1661 (2025). doi:10.1038/s41592-025-02772-6.

[7] Boylan, J., Solovyeva, E., Bouiller, T. et al. Single Cell Foundation Models Evaluation (scFME) for In-Silico Perturbation. bioRxiv (2025). doi:10.1101/2025.09.22.677811. Preprint.

[8] Weidener, L. S., Brkić, M., Jovanović, M., Ulgac, E. & Meduri, A. VCBench: A Multi-Dimensional Benchmark for Single-Cell Foundation Models. bioRxiv (2026). doi:10.64898/2026.06.18.733146. Preprint.

[9] Avsec, Ž., Latysheva, N., Cheng, J. et al. Advancing regulatory variant effect prediction with AlphaGenome. Nature 649, 1206–1218 (2026). doi:10.1038/s41586-025-10014-0.

[10] Brixi, G., Durrant, M. G., Ku, J. et al. Genome modelling and design across all domains of life with Evo 2. Nature 652, 1349–1361 (2026). doi:10.1038/s41586-026-10176-5.

[11] Kapoor, S., Cantrell, E. M., Peng, K. et al. REFORMS: Consensus-based Recommendations for Machine-learning-based Science. Science Advances 10, eadk3452 (2024). doi:10.1126/sciadv.adk3452.

[12] Chen, Z., Chen, S., Ning, Y. et al. ScienceAgentBench: Toward Rigorous Assessment of Language Agents for Data-Driven Scientific Discovery. International Conference on Learning Representations (ICLR) (2025).

Note on preprints

Several references above are bioRxiv preprints and have not yet undergone peer review. I include them because biological foundation-model evaluation is developing rapidly and some important methodological work appears first as preprints. Their findings should therefore be interpreted as emerging evidence rather than established consensus.

From Genomics to AI Biology (Series 2)

Biology Is Not Just Data
Why Context Is the Bottleneck for AI Biology

By Li Lei on July 13, 2026

When I first worked with large-scale genomics data, I remember staring at long tables of significant genes, variants, and enriched pathways.

The results looked impressive.

But the real question was harder:

Which signals were biologically real?

Which were artifacts?

Which were statistically significant but biologically weak?

Which were worth following up on in an experiment?

That experience still shapes how I think about AI biology.

At first glance, genomics looks like a perfect field for artificial intelligence. Biology is producing enormous amounts of data: genome sequences, variants, RNA-seq, ATAC-seq, single-cell profiles, methylomes, proteomics, metabolomics, phenotypes, images, environmental records, and clinical measurements. Over the past decade, deep learning has become increasingly important for genome analysis and interpretation, including regulatory genomics, variant effect prediction, and functional genomics [1,2].

It is tempting to think that biology is simply waiting for better models.

More data.

Bigger models.

More compute.

Better prediction.

But the longer I work at the intersection of genomics, computational biology, and AI, the more I believe this view is incomplete.

Biology is not just a data problem.

Biology is a context problem.

A gene is not just a gene.

A variant is not just a variant.

A cell is not just a cell.

A phenotype is not just a label.

A model prediction is not automatically a discovery.

Biological meaning depends on context: tissue, cell type, developmental stage, genotype, environment, population history, evolutionary constraint, experimental design, assay technology, and validation evidence.

This is why I believe one of the biggest bottlenecks in AI biology is not only model architecture or compute.

It is context.

A prediction is not yet a discovery

Imagine an AI system ranks a gene as a top candidate for drought tolerance in barley or sorghum.

At first, this sounds exciting. The model has scanned a large biological space and found something worth attention. Maybe the gene is near a trait-associated variant. Maybe it is differentially expressed under drought. Maybe it appears in a regulatory network. Maybe the literature suggests a connection to stress response.

But that ranking is not yet a discovery.

It is the beginning of a biological investigation.

A biologist will immediately ask:

Is this gene expressed in the relevant tissue — root, leaf, seed, or reproductive organ?

Is it active at the relevant developmental stage?

Is the response specific to drought, or is it a general stress response?

Is the association driven by population structure or linkage disequilibrium?

Is there chromatin or regulatory evidence?

Is the gene conserved across grasses?

Does perturbing the gene actually change the phenotype?

Would improving drought tolerance create a tradeoff with growth, yield, or reproduction?

These questions are not secondary details.

They are the biology.

This is where AI biology becomes difficult. A model can learn patterns from data, but biological meaning depends on where, when, how, and under what conditions those patterns appear.

In biology, signals are often real, but their meaning is conditional.

That conditionality is exactly what makes biological interpretation hard. It is also what makes AI biology hard.

The same signal can mean different things

A gene highly expressed in leaves under drought stress may be important for water response. But the same gene expressed in roots, seeds, or reproductive tissues may tell a very different story.

A transcription factor may regulate stress response in one developmental stage and growth in another.

A variant may look associated with a trait, but the signal may be driven by population structure, linkage disequilibrium, or environmental stratification.

A chromatin region may be accessible, but that does not automatically mean it is an active enhancer.

A gene may be differentially expressed, but that does not automatically mean it is causal.

A pathway may be enriched, but that does not tell us which mechanism is driving the phenotype.

This is why context is not a detail added after analysis.

Context is part of the biological signal.

If the data does not carry enough context, an AI model may learn shortcuts instead of biology. It may learn species-specific bias, batch effects, tissue composition, sequencing depth, publication bias, or experimental artifacts. It may perform well on a benchmark but fail when applied to a new genotype, tissue, environment, disease cohort, or crop field.

In biology, a prediction is useful only when we understand its boundary.

Biology has many layers of context

When we say “context,” it can sound vague. But in biology, context is not abstract. It has real layers.

I think of biological context as at least seven connected layers.

1. Molecular context

At the molecular level, we ask: what is happening in the genome, epigenome, transcriptome, proteome, or metabolome?

A DNA variant may change a protein sequence. Or it may affect a regulatory element. Or it may do nothing obvious by itself but matter in combination with other variants. A gene expression signal may reflect transcriptional regulation, RNA stability, cell composition, or technical noise.

A sequence is never just a string of letters. It exists in chromatin, in a genome, in a regulatory architecture, and in an evolutionary history.

This is why models such as Enformer are exciting. They show that sequence-based models can improve gene expression prediction by incorporating long-range regulatory information, but they also remind us that regulatory interpretation depends on genomic and cellular context [3].

2. Cellular context

The same gene can behave differently across cell types. This is one of the major lessons from single-cell and single-nucleus biology.

A signal that disappears in bulk RNA-seq may become clear at the cell-type level. A gene may be active only in a rare cell population. A regulatory program may be specific to a transient developmental state. A disease or stress response may not affect all cells equally.

Large efforts such as the Human Cell Atlas were built around this idea: to understand human biology, we need to define cell types by molecular profiles and connect them with location, morphology, and function [4].

AI models that ignore cell type and cell state may average away the biology we actually care about.

3. Tissue context

Biology is spatial and tissue-specific.

A drought response in roots is not the same as a drought response in leaves. A regulatory element active in one tissue may be silent in another. A variant affecting seed development may have little relevance to vegetative growth. A disease marker in blood may not reflect what is happening in the target tissue.

This matters deeply for both plant biology and human biology.

When context is missing, a model may connect the right entities in the wrong place.

4. Organismal context

Genes act inside organisms, not in isolation.

The same variant may have different effects depending on genetic background. The same pathway may behave differently across accessions, cultivars, strains, populations, or species. A phenotype may result from many small effects across the genome rather than one obvious causal gene.

This is something population genetics teaches us very well: variation is structured. History matters. Background matters. Selection matters. What appears as a clean association may be entangled with demography, ancestry, and environment.

AI systems that ignore organismal and population context may confuse correlation with cause.

5. Environmental context

Biology is responsive.

Temperature, drought, light, nutrients, pathogens, microbiome, management practice, diet, treatment, and climate can all change biological meaning. A gene that matters under stress may not matter under control conditions. A genotype that performs well in one environment may fail in another.

In agriculture, this is obvious. A trait is rarely just genetic. It is genotype-by-environment interaction.

In medicine, it is also true. Disease risk, treatment response, and immune state are shaped by environment, history, and exposure.

AI biology must learn not only biological entities, but biological conditions.

6. Evolutionary context

Biology has history.

Some genes are deeply conserved. Others are lineage-specific. Some regulatory elements evolve quickly. Others remain constrained across millions of years. A pathway discovered in Arabidopsis may or may not translate directly to maize, sorghum, rice, or Brachypodium. A signal from mouse may or may not translate to human.

Evolutionary context helps us ask whether a pattern is likely to be functional, conserved, diverged, or species-specific.

Without evolutionary thinking, AI may treat all biological observations as flat examples. But biology is not flat. It is shaped by descent, divergence, constraint, and adaptation.

7. Experimental context

Finally, every biological dataset comes from an experiment.

How was the sample collected?

What protocol was used?

What sequencing platform?

What batch?

What controls?

What replicates?

What normalization?

What quality filters?

What reference genome?

What annotation version?

These details are not technical decoration. They determine what conclusions can be trusted.

This is also why the FAIR principles — making data findable, accessible, interoperable, and reusable — are so important for data-intensive science. They emphasize that data, metadata, tools, and workflows should be reusable by both humans and machines [5].

A beautiful machine learning model trained on poorly annotated, biased, or inconsistent data may produce confident predictions with weak biological value.

In AI biology, metadata is not paperwork.

Metadata is part of intelligence.

Why bigger models are not enough

I am not against bigger models.

Foundation models have already shown remarkable potential in protein biology, genomics, chemistry, and medicine. AlphaFold demonstrated the power of deep learning for protein structure prediction and changed how many scientists think about structural biology [6]. Models such as Enformer show how deep learning can extract regulatory information from DNA sequences at scale [3].

These are real breakthroughs.

But bigger models alone do not solve the context problem.

A model trained only on sequence can learn sequence patterns, but unless tissue-specific activity is represented in the training data or system design, it may not distinguish where a regulatory element is active.

A model trained on expression can learn cell states, but unless treatment, genotype, and sampling conditions are represented, it may not understand why those states appear.

A model trained on literature can learn published knowledge, but it may also inherit publication bias, outdated assumptions, and unsupported claims.

The issue is not whether models are powerful.

They are.

The issue is whether the system around the model carries enough biological context to make the output meaningful.

This is where many AI biology efforts become fragile. They focus heavily on model architecture but underinvest in data provenance, metadata, knowledge representation, benchmark design, biological validation, and feedback loops.

In other words, they build impressive engines but give them incomplete maps.

From prediction to meaning

AI is very good at prediction. But science is not only prediction.

Science also asks:

Why does this happen?

Under what conditions does it happen?

Is the relationship causal?

Can it generalize?

Can we intervene?

What evidence would change our mind?

A prediction becomes biologically valuable only when it helps us move toward understanding, decision, or action.

This is why AI biology needs scientists who understand both models and mechanisms. The model can point us toward a pattern. But the scientist must decide whether the pattern is meaningful, testable, and worth pursuing.

For the drought-tolerance gene example, the model’s ranking may be the first signal. But the biological work begins afterward.

We need to ask whether the gene is expressed in the right tissue, active under the right condition, supported by regulatory evidence, robust across genotypes, conserved across species, and testable through perturbation or validation.

That is how a prediction becomes a hypothesis.

And only through evidence does a hypothesis become discovery.

The danger of context-free AI

Context-free AI can be fast, fluent, and impressive.

It can summarize papers.

It can rank genes.

It can generate hypotheses.

It can produce beautiful explanations.

But if it does not represent biological context, it can also mislead.

It may overgeneralize from one species to another.

It may treat correlation as causation.

It may ignore tissue specificity.

It may miss population structure.

It may combine evidence from incompatible assays.

It may present a weak hypothesis with strong language.

It may sound confident because language models are good at fluency, not because the biology is correct.

This is especially risky because biological AI outputs often look plausible.

A wrong gene ranking may not look obviously wrong.

A weak mechanistic explanation may sound elegant.

A false literature connection may appear reasonable.

A biased model may perform well on an internal benchmark.

The danger is not only that AI can be wrong.

The danger is that AI can be wrong in a biologically plausible way.

That is why context-aware evaluation is essential.

We should not only ask whether a model performs well on average. We should ask where it performs well, where it fails, and why.

Does it generalize across species?

Across tissues?

Across environments?

Across populations?

Across experimental platforms?

Across time?

A model that performs well only in familiar contexts may still be useful. But we must know its boundary.

In biology, knowing the boundary of a model may be as important as knowing its prediction.

Context engineering: making biological meaning computable

If prompt engineering was one of the first popular skills of the AI era, context engineering may become one of the most important skills in AI biology.

By context engineering, I mean the work of making biological context computable.

This is where knowledge graphs, ontologies, metadata standards, multimodal integration, retrieval systems, and validation loops become important.

A knowledge graph can connect genes, variants, traits, tissues, pathways, publications, experiments, species, environmental conditions, and evidence types. Biomedical knowledge graphs are often used to represent biological and clinical concepts as nodes and relationships, making them useful for data integration, machine learning, and reasoning [7].

Graph representation learning extends this idea by learning from network structure and node relationships. In biomedicine, such approaches have been applied across molecular networks, disease relationships, patient data, drug discovery, and other connected biological systems [8].

This does not make the system perfect. But it gives the model a richer map.

Instead of asking AI to reason from isolated data points, we can ask it to reason over structured biological relationships.

This is especially important for AI agents in science.

If an agent is expected to answer biological questions, design analyses, prioritize candidates, or propose experiments, it needs more than a language model. It needs access to reliable data, structured knowledge, metadata, literature, tools, evaluation criteria, and human feedback.

An AI agent without biological context is like a brilliant student with no lab notebook, no experimental design, and no memory of what was actually measured.

It may be clever, but it is not yet a scientist.

What context-aware AI biology might look like

A context-aware AI biology system would not simply take a gene list and return a polished interpretation.

It would ask better questions.

What species is this from?

Which tissue?

Which developmental stage?

Which treatment?

Which genotype?

Which assay?

Which controls?

Which reference annotation?

Which evidence is direct and which is inferred?

Which claims come from literature and which come from data?

Which results are robust across datasets?

Which hypotheses are testable?

It would not only produce an answer.

It would show the evidence path.

It would connect a prediction to the data, the metadata, the prior knowledge, the uncertainty, and the next experiment.

This is the kind of AI biology I find exciting.

Not AI that simply gives us more answers.

AI that helps us ask better questions.

What biologists should contribute

In the AI era, biologists should not underestimate their role.

It is easy to feel that machine learning experts are driving the future and biologists are only data providers. But that would be a mistake.

Biologists understand what the question means.

Biologists understand whether the data is appropriate.

Biologists understand what the model might be missing.

Biologists understand what evidence is convincing.

Biologists understand whether a prediction is experimentally actionable.

This expertise is not secondary.

It is central.

The future of AI biology needs biologists who can define meaningful problems, identify relevant context, design better evaluation, and interpret model outputs with scientific discipline.

This does not mean every biologist must become a machine learning engineer. But more biologists should learn enough AI to participate in system design, not only tool usage.

They should be able to say:

This dataset is not enough.

This benchmark is misleading.

This prediction is interesting but not actionable.

This context is missing.

This result needs validation.

This hypothesis is worth testing.

That is scientific leadership in the AI era.

My own lesson from genomics

Genomics taught me humility.

When I first learned to analyze large biological datasets, I was fascinated by the power of scale. Suddenly, we could look across the genome, across populations, across tissues, across conditions. We could move from one gene at a time to thousands of genes at once.

But scale also made the problem harder.

More data did not automatically create more understanding. Sometimes it created more ambiguity. More correlations. More candidates. More possible explanations.

The real work was to connect signals back to biology.

Which signal is robust?

Which one is confounded?

Which one generalizes?

Which one has mechanism?

Which one can be validated?

Which one matters?

AI biology is entering the same stage.

We are surrounded by new models, new tools, new predictions, and new promises. The opportunity is real. But the central question remains deeply scientific:

What does it mean?

Biology is data with meaning

I believe AI will transform biology. It will help us read genomes, design proteins, interpret cells, mine literature, integrate knowledge, generate hypotheses, and accelerate discovery.

But the most important progress will not come from treating biology as a giant spreadsheet.

It will come from building systems that understand relationships, constraints, mechanisms, uncertainty, and context.

Biology is not just data.

Biology is data with history.

Data with structure.

Data with environment.

Data with measurement noise.

Data with evolutionary memory.

Data with experimental limits.

Data with meaning.

The future of AI biology will depend on how well we teach machines to work with that meaning.

And that work cannot be done by AI alone.

It requires biologists who understand data, computational scientists who understand models, engineers who understand systems, and teams that respect the complexity of life.

In the first essay of this series, I wrote that AI will not replace biologists, but biologists who understand AI systems will replace those who only use AI tools.

In this second essay, I want to go one step deeper:

Biologists who understand context will be essential to the future of AI biology.

Because in biology, context is not background.

Context is the bottleneck.

And perhaps, context is also the key.

If we want AI to become truly useful for biology, the question is not only how large the model is.

The question is:

What context does the system need in order to make a biological answer trustworthy?

That is the conversation I hope more biologists will join.

References and further reading

[1] Zou, J., Huss, M., Abid, A., Mohammadi, P., Torkamani, A. & Telenti, A. A primer on deep learning in genomics. Nature Genetics 51, 12–18 (2019). doi:10.1038/s41588-018-0295-5.

[2] Eraslan, G., Avsec, Ž., Gagneur, J. & Theis, F. J. Deep learning: new computational modelling techniques for genomics. Nature Reviews Genetics 20, 389–403 (2019). doi:10.1038/s41576-019-0122-6.

[3] Avsec, Ž. et al. Effective gene expression prediction from sequence by integrating long-range interactions. Nature Methods 18, 1196–1203 (2021). doi:10.1038/s41592-021-01252-x.

[4] Regev, A. et al. The Human Cell Atlas. eLife 6, e27041 (2017). doi:10.7554/eLife.27041.

[5] Wilkinson, M. D. et al. The FAIR Guiding Principles for scientific data management and stewardship. Scientific Data 3, 160018 (2016). doi:10.1038/sdata.2016.18.

[6] Jumper, J. et al. Highly accurate protein structure prediction with AlphaFold. Nature 596, 583–589 (2021). doi:10.1038/s41586-021-03819-2.

[7] Nicholson, D. N. & Greene, C. S. Constructing knowledge graphs and their biomedical applications. Computational and Structural Biotechnology Journal 18, 1414–1428 (2020). doi:10.1016/j.csbj.2020.05.017.

[8] Li, M. M., Huang, K. & Zitnik, M. Graph representation learning in biomedicine and healthcare. Nature Biomedical Engineering 6, 1353–1369 (2022). doi:10.1038/s41551-022-00942-x.

From Genomics to AI Biology (Series 1)

 AI Will Not Replace Biologists. But Biologists Who Understand AI Systems Will Replace Those Who Only Use AI Tools

By Li Lei on June 21, 2026

When I first moved from genomics and computational biology into AI biology, I thought the biggest challenge would be learning new models.

Large language models. Graph neural networks. Foundation models. AI agents. Embedding spaces. Retrieval-augmented generation. Model evaluation. MLOps.

All of these were important. But over time, I realized something deeper:

The real challenge is not simply learning how to use AI tools.

The real challenge is learning how to think in AI systems.

This distinction matters because biology is entering a new era. AI is no longer just a convenient assistant for writing code, summarizing papers, or generating figures. It is beginning to shape how we organize knowledge, design experiments, prioritize hypotheses, analyze multimodal data, and make decisions in discovery pipelines.

We have already seen powerful examples. AlphaFold changed how many scientists think about protein structure prediction [1]. Protein language models showed that large-scale learning from sequences can capture biological structure and function [2]. Deep learning models such as Enformer demonstrated that sequence-based models can improve gene expression prediction by modeling long-range regulatory information [3]. More broadly, deep learning has become an important part of modern genomics and genome interpretation [4,5].

But these advances also reveal something important: the future of AI biology is not only about bigger models. It is about building better scientific systems around those models.

In this new era, the biologists who thrive will not necessarily be the ones who use the most AI tools. They will be the ones who understand how AI systems work, where they fail, how to evaluate them, and how to connect them to real biological questions.

That is why I believe:

AI will not replace biologists. But biologists who understand AI systems will replace those who only use AI tools.

From using tools to understanding systems

Many scientists today are already using AI. They ask ChatGPT to polish manuscripts, write Python scripts, explain error messages, summarize literature, or draft emails. These uses are helpful. They save time. They reduce friction. They make technical work more accessible.

But using AI as a tool is only the first layer.

A tool user asks:

“Can AI help me finish this task faster?”

A systems thinker asks:

“What is the input? What is the output? What knowledge does the model have? What knowledge is missing? How do I evaluate whether the answer is correct? How does this fit into a larger scientific workflow?”

This difference may sound small, but it changes everything.

For example, if I ask an AI model to summarize papers about drought tolerance in barley, I may receive a fluent answer. But a fluent answer is not necessarily a reliable answer. A biologist who only uses the tool may accept the summary too quickly. A biologist who understands AI systems will ask: Which papers were retrieved? Are they current? Are the claims supported by experimental evidence? Are the genes discussed in the right tissue, developmental stage, and environmental context? Are we mixing evidence from Arabidopsis, rice, sorghum, and barley as if they were interchangeable?

In biology, context is not decoration. Context is the science.

A gene is not simply “important.” It is important in a genotype, tissue, cell type, developmental stage, environment, and evolutionary history. A variant is not simply “associated.” It has allele frequency, linkage disequilibrium, population structure, effect size, uncertainty, and biological plausibility. A regulatory element is not simply “predicted.” It has chromatin accessibility, transcription factor binding, conservation, activity, target gene ambiguity, and experimental validation limits.

This is why biological AI requires more than prompting. It requires system-level thinking.

Biology is not just data

One common misunderstanding in AI biology is the idea that biology is simply a data problem.

More data, bigger model, better prediction.

Sometimes that is true. Often, it is incomplete.

Biological data is noisy, biased, incomplete, heterogeneous, and deeply contextual. Different data types capture different layers of life: genome sequence, chromatin accessibility, gene expression, methylation, protein structure, metabolites, phenotypes, environmental variables, clinical outcomes, and evolutionary constraints. Each layer has its own measurement errors, assumptions, and missingness.

A machine learning model can find patterns. But not every pattern is meaningful. Not every correlation is causal. Not every prediction is actionable.

This is where trained biologists remain essential.

Biologists understand experimental design. They understand confounding. They know that a beautiful heatmap can hide a batch effect. They know that a significant association can be driven by population structure. They know that a gene expression signal may reflect cell type composition rather than regulation. They know that a model trained on one species, tissue, or condition may not generalize to another.

AI can accelerate discovery, but it does not automatically understand what makes a biological conclusion trustworthy.

That judgment still comes from scientists.

The future belongs to scientists who can combine biological judgment with AI system design.

The next skill is not just coding

For the past decade, many biologists were told: “Learn to code.”

That advice was useful. Coding opened the door to bioinformatics, genomics, data analysis, and reproducible research. It allowed biologists to work directly with large datasets rather than relying entirely on others.

But in the AI era, coding alone is no longer enough.

The next skill is understanding how biological knowledge becomes computable.

This includes questions such as:

How do we represent biological entities and relationships?

How do we connect genes, variants, traits, pathways, tissues, environments, publications, and experimental evidence?

How do we integrate structured databases with unstructured literature?

How do we build workflows where AI agents can retrieve, reason, analyze, and report?

How do we evaluate whether an AI-generated hypothesis is biologically meaningful?

How do we prevent models from producing confident but unsupported conclusions?

These are not just computer science questions. They are scientific questions.

A good AI system for biology is not just a model. It is a carefully designed connection between data, knowledge, algorithms, evaluation, and human decision-making.

That is why I believe knowledge representation will become one of the most important skills in AI biology.

Biomedical knowledge graphs already show why representation matters. They provide a way to connect entities such as genes, proteins, diseases, drugs, phenotypes, pathways, and publications into structured relationships that both humans and machines can query and reason over [6]. Graph representation learning further extends this idea by learning from the topology and semantics of biological and biomedical networks [7].

In genomics, we often start with sequences. But discovery rarely ends with sequence alone. We need to connect sequence variation to gene regulation, gene regulation to cellular function, cellular function to phenotype, and phenotype to environment or disease. This chain is complex. It is full of uncertainty. But it is also where the real biological meaning lives.

AI systems that ignore this complexity may generate answers. AI systems that model this complexity may generate insight.

The danger of becoming only an AI consumer

There is a risk in the current AI wave: scientists may become passive consumers of AI outputs.

The model suggests a candidate gene.

The model ranks a variant.

The model proposes a pathway.

The model writes the interpretation.

If we are not careful, scientists may slowly lose the habit of questioning the reasoning behind the output.

That would be dangerous.

Science advances through skepticism. We ask why. We ask how. We ask what evidence supports the claim. We ask whether there is another explanation. We ask what experiment could prove us wrong.

AI should not weaken this habit. It should make it stronger.

A biologist who understands AI systems does not blindly trust the model. But she/he/they also does not reject it out of fear. Instead, she treats AI as a powerful but imperfect collaborator.

she/he/they asks:

What data was this model trained on?

What assumptions are built into the system?

What is the failure mode?

What kind of uncertainty is being hidden?

What evidence would increase my confidence?

What experiment should come next?

This is the mindset we need.

Not AI worship.

Not AI fear.

AI literacy with scientific discipline.

What should biologists learn now?

Not all biologists need to become machine learning engineers. But I do think more biologists need to understand the architecture of AI-enabled discovery.

At minimum, future-ready biologists should understand five things.

First, they should understand data. Not only how to download it, but how it was generated, normalized, biased, and limited.

Second, they should understand representation. In biology, how we represent a problem often determines what the model can learn. A sequence, a graph, a table, an image, a time series, and a knowledge graph all expose different aspects of the same biological system.

Third, they should understand models. They do not need to derive every equation, but they should know what different models are good at, what they assume, and when they are likely to fail.

Fourth, they should understand evaluation. In AI biology, a high benchmark score is not the same as biological usefulness. We need to evaluate models based on generalization, interpretability, robustness, experimental relevance, and decision value. Recent discussions of large language models in scientific discovery also emphasize that these systems should be integrated into scientific workflows with clear human goals and clear evaluation metrics [8].

Fifth, they should understand workflows. AI is most powerful when embedded into real scientific workflows: literature mining, data integration, hypothesis generation, prioritization, experiment design, and feedback from validation.

This is the shift from using AI tools to building AI-assisted scientific systems.

A personal transition

My own path into AI biology did not start from computer science. It started from population genetics, evolutionary biology, and genomics.

Population genetics trained me to think about variation, structure, uncertainty, history, and selection. Genomics trained me to work with large-scale biological data. Bioinformatics trained me to build pipelines and extract signals from complexity. AI is now teaching me to think about representation, reasoning, automation, and decision systems.

Each stage did not replace the previous one. It expanded it.

This is why I do not see AI as a departure from biology. I see it as a new language for asking biological questions.

But learning this language requires humility.

We need to admit that many AI methods are unfamiliar. We need to learn new concepts. We need to collaborate with engineers, data scientists, and machine learning experts. But we also need to remember that biological insight is not outdated. It is more important than ever.

The scientist of the future will not be defined by one discipline. She/he/they will be able to move between biology, computation, data infrastructure, AI models, and real-world decisions.

She/he/they will not simply ask, “What can this tool do?”

She/he/they will ask, “What kind of scientific system are we building?”

The future biologist

The future biologist will still care about genes, cells, organisms, evolution, disease, crops, ecosystems, and patients.

But she/he/they will also understand embeddings, knowledge graphs, agents, multimodal data, model evaluation, and feedback loops.

She/he/they will know how to ask good biological questions and how to design AI systems that make those questions computable.

She/he/they will be skeptical but not afraid.

Technical but not narrow.

Biological but not limited by traditional boundaries.

Curious enough to learn new tools, and wise enough not to be ruled by them.

AI will change biology. There is no doubt about that.

But the deepest transformation will not come from replacing scientists. It will come from changing what scientists are capable of doing.

The most valuable biologists in the AI era will not be those who simply use AI to work faster.

They will be those who understand enough biology to ask meaningful questions, enough AI to build powerful systems, and enough scientific judgment to know when the answer is real.

References and further reading

[1] Jumper, J. et al. Highly accurate protein structure prediction with AlphaFold. Nature 596, 583–589 (2021). doi:10.1038/s41586-021-03819-2.

[2] Rives, A. et al. Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences. Proceedings of the National Academy of Sciences 118, e2016239118 (2021). doi:10.1073/pnas.2016239118.

[3] Avsec, Ž. et al. Effective gene expression prediction from sequence by integrating long-range interactions. Nature Methods 18, 1196–1203 (2021). doi:10.1038/s41592-021-01252-x.

[4] Zou, J., Huss, M., Abid, A., Mohammadi, P., Torkamani, A. & Telenti, A. A primer on deep learning in genomics. Nature Genetics 51, 12–18 (2019). doi:10.1038/s41588-018-0295-5.

[5] Eraslan, G., Avsec, Ž., Gagneur, J. & Theis, F. J. Deep learning: new computational modeling techniques for genomics. Nature Reviews Genetics 20, 389–403 (2019). doi:10.1038/s41576-019-0122-6.

[6] Nicholson, D. N. & Greene, C. S. Constructing knowledge graphs and their biomedical applications. Computational and Structural Biotechnology Journal 18, 1414–1428 (2020). doi:10.1016/j.csbj.2020.05.017.

[7] Li, M. M., Huang, K. & Zitnik, M. Graph representation learning in biomedicine and healthcare. Nature Biomedical Engineering 6, 1353–1369 (2022). doi:10.1038/s41551-022-00942-x.

[8] Zhang, Y. et al. Exploring the role of large language models in the scientific method: from hypothesis to discovery. npj Artificial Intelligence 1, Article 14 (2025). doi:10.1038/s44387-025-00019-5.

[9] Bommasani, R. et al. On the opportunities and risks of foundation models. arXiv:2108.07258 (2021).