CharlieKey takeaways from your favorite podcasts
← 15 October 2026Latent Space

Why Models Trained on Public Data Learn the Chemist, Not the Chemistry — Ian Quigley, Leash Bio

Ian Quigley argues that small-molecule ML fails when trained on public data that encodes chemist behavior, and that Leash's pooled, physically grounded screening data, engineered mutations and language-model agents are a better route.

Watch on YouTube ↗

Charlie's summary

  1. Pooled DNA-encoded libraries are the scalable way to measure small-molecule binding, and the combinatorial structure helps ML

    Ian explains that a DEL puts millions of barcoded molecules in one tube, pulls out binders with a protein on a bead, and sequences the barcodes. Because molecules share building blocks, you can see which part drives binding. Ian's team also uses 32 or 48 barcodes per molecule to catch DNA-binding false positives, and repeated screens return “the same hits in the same order”.

    ▶ 3:00
  2. Leash's Kaggle contest showed models do no better than chance on truly novel chemistry

    Leash released about four billion interactions, versus under three million in BindingDB, and held out three evals of rising difficulty. Across 2,000 teams, there were gains with shared building blocks. On the fully new library, no one beat chance, and three of the top five were the tutorial notebook. Ian read this as validation: the problem is hard, so “pour gas on this”. Simple SMILES strings beat fancier representations.

    ▶ 27:23
  3. Split by building-block cluster and always run XGBoost, or you only measure memorization

    Ian describes Hermes, a deliberately simple model: ESM protein embeddings plus a chemistry embedding joined by cross-attention. To test it, Leash held out entire clusters of similar building blocks. XGBoost fell steadily as the splits got farther from the training data, while Hermes stayed steady. Cheap baselines and harsh splits show whether a model generalizes or just remembers.

    ▶ 43:41
  4. Models trained on public data learn the chemist, not the chemistry

    Ian's team trained a classifier on ChEMBL to predict which of 1,815 chemists made a molecule, reaching 60% top-five accuracy. A vector of creator probabilities alone predicted activity as well as chemical structure. Ian thinks the models learn sociology: what chemists chose to make and test, and what they skipped. His summary: When you train models on the exhaust, you might be in trouble. He compares it to Clever Hans.

    ▶ 46:44

Give your email to unlock the rest of the key points — for every episode.

Do more with Charlie

Free — sign in with your email, no password.