Author Archives: Lorenzo Tarricone

Atom14 and Atom37: how protein design models write down atoms

AKA why a protein does not naturally fit neatly into a tensor, and how RFdiffusion3, BoltzGen and La-Proteina cope when the amino-acid sequence is not known yet

The problem: proteins are not rectangles

Neural networks like fixed-shape arrays. Proteins are not fixed-shape: the 20 standard amino acids contain between 4 and 14 heavy atoms (hydrogens are usually ignored). Atom14 and Atom37 are two ways of squeezing that variety into a rectangle. For a protein with L residues, both give a coordinate tensor of shape [L, A, 3]: one row per residue, A atom slots, and three Cartesian coordinates per slot. Atom14 uses A = 14 and Atom37 uses A = 37. These are maximum widths, not claims that every residue has that many atoms, so each tensor travels with a mask that marks which slots hold a real atom. [1, 2]

This post first explains the two layouts, then asks the question that matters for generative models: what happens in de novo design, where the sequence, and therefore the list of atoms, is not known in advance? We look at three recent answers: RFdiffusion3, BoltzGen and La-Proteina.

Same backbone, different side chains

Every amino acid shares a backbone of four heavy atoms: N, CA (the alpha carbon), C and O. The side chain starts at CB, the beta carbon, and is what makes each residue type different. Glycine has no side chain at all, serine and cysteine add two atoms, arginine adds seven, and tryptophan adds ten. Any representation must hold the shared backbone and a variable-length side chain at once. [1]

Figure 1. Five residues drawn atom by atom, using the AlphaFold atom names. Backbone atoms have a thick outline. Bonds are schematic and hydrogens are omitted.
Continue reading →

A timeline of sampling methods of diffusion models

When approaching the methods used in de-novo protein design, one is quickly confronted with a plethora of overlapping formulations of what looks superficially like “the same thing”. One paper trains an ϵ\boldsymbol{\epsilon}-prediction network with a simple MSE loss; another trains a score network with a stochastic-differential-equation justification; a third trains a clean-data predictor under yet another schedule. Each formulation carries its own notation, its own variance schedule, and its own sampler. Qualitatively, this zoo of formulations is doing the same thing: it starts from some unstructured noise and iteratively refines it to eventually produce a protein structure similar (but different!) to other proteins we have experimentally determined in the past. What is not immediately obvious to a newcomer is that all of these formulations are historical descendants of a small number of foundational ideas, and that essentially every architectural and algorithmic decision in a modern protein-design diffusion model has a specific paper of origin and a specific motivation for being there.

This post is my attempt to put these formulations onto a single timeline. I trace the trajectory of the field through four foundational works: DDPM (Ho et al., 2020), DDIM (Song et al., 2021a), the score-based SDE unification (Song et al., 2021b), and EDM (Karras et al., 2022), explaining at each step what specific problem with the previous formulation the next paper was attacking and how the new formulation generalises or simplifies the old one. The goal is coherent motivation rather than exhaustive coverage; the reader interested in implementation details is referred to the original papers and the references at the end.

Continue reading →