Category Archives: Deep Learning

Is the race for AGI a scam? Are the tech bros con men or visionaries?

To find out, we formally introduce the Stinkometer

In research, it can be tough even for experts to tell fact from fiction. This is especially true for more speculative fields like AI. It can feel like every model is “State-of-the-Art” and that AGI (whatever that means) will be “6 months away” for at least the next 6 months. How do we know we can trust these people?

What is a visionary? What is a con man?

Visionaries sell a fringe idea of the future, usually with strong self-belief and charisma. Con men are visionaries who don’t believe what they’re saying. It’s hard to tell the difference because we can’t read minds and can’t see the future.

I solved this by adapting the insightful “New Political Compass” from Harper O’Connor, an American political YouTuber. Harper found a similar problem. He’s active in local politics and has to decide who he should build alliances with to tackle certain issues. Interestingly, the “left-right” axis wasn’t a helpful guide. Most people he canvassed were reasonable but didn’t follow politics closely enough to have a robust ideology. He realised that someone’s psychology can be more important than their stated ideology.

Therefore, it was more useful to ask: Am I talking to a reasonable person? Here, I expand his framework into the Belief Compass (or the “Stinkometer”). We can answer our question by answering three simpler ones:

Continue reading →

Predicting ADME Properties with Machine Learning: 82% of the Performance From Two Descriptors

A Nature survey of ~1500 scientists reported that more than 70% had failed to reproduce another scientist’s experiments, and 50% had failed to replicate their own (Baker, 2016). This study wasn’t specific to machine learning, but the crisis has its own flavour in computational drug discovery.

One aspect of this is that available datasets have known quality issues, and results built on them can be fragile. MoleculeNet and the Therapeutic Data Commons (TDC) opened drug discovery to a wider ML community, but they’re no longer sufficient for driving further advances (Wognum et al., 2024).

Amongst a number of issues, 71% of molecules in one MoleculeNet dataset (BACE) contained at least one undefined stereocenter (a point where the same atoms can sit in two different 3D arrangements), making it unclear what chemical entity is actually being modelled (Li et al., 2026).

The stakes are high. In 2017, a research group found their cancer-target inhibitor inactive from one vendor and highly active from another. Eventually they traced this to vendors selling different mixtures of the compound’s two 3D forms where only one was active on the target and mechanism they were investigating (Baker, 2017).

Most approved pharmaceuticals are relatively small chemical molecules, typically weighing under 900 g/mol, with most of the rest being biologics. Whilst in Oxford on the UNIQ+ summer internship programme, I focused on ADME, an aspect of early stage drug discovery, where assays are measured in vitro as a stand-in to predict how a compound will behave in vivo (in the human body) before clinical trials. Absorption (does it enter the body), Distribution (where does it go), Metabolism (how quickly it’s transformed) and Excretion (how quickly it’s eliminated) are processes critical to whether a candidate succeeds. 

Continue reading →

Beyond the Vaccine: AI’s Expanding Role in LNP mediated Drug Delivery

Lipid nanoparticles (LNPs) have evolved from a specialised drug-delivery technology into a cornerstone of modern medicine. Their success became evident during the COVID-19 pandemic, enabling the delivery of the mRNA used in the Pfizer-BioNTech and Moderna vaccines. However, their applications extend far beyond vaccination, as LNPs can also deliver siRNA, plasmid DNA and gene-editing machinery, creating opportunities to treat genetic diseases, cancer and many other conditions.

Yet a fundamental challenge remains: how do we design an LNP that delivers the right payload, to the right cells, in the right place?

Continue reading →

Peering Inside the Black Box: A Beginner’s Introduction to Mechanistic Interpretability

Over the last few years, large language models (LLMs) have gone from being curiosities tucked away in research labs to something most of us interact with on a daily basis; whether for drafting emails, debugging code, or simply pondering the meaning of life at 2am. And yet, for all our reliance on these systems, a rather inconvenient truth lingers in the background: nobody, not even the people who built them, can fully explain what is going on inside.

This is where mechanistic interpretability comes in.

In essence, mechanistic interpretability is the approach of explaining complex machine learning systems through the behaviour of their functional units (Kästner and Crook, 2024) by reverse-engineering them into their more elementary computations (Rai et al., 2025). The aim is not simply to know that a model gives the right answer, but to pull apart the underlying machinery and uncover the causal relationships between input and output. Think of it as neuroscience for neural networks, except we can read every neuron at any moment, rewind, replay, and intervene mid-thought.

Continue reading →

Spin Lattices and Proteins – How state-based discretisations have enabled modern protein modelling

I got into protein modelling not long before AlphaFold2 first released. At that time some of the prevailing methods for protein structure prediction came from highly interpretable energy functionals that arose from a particularly beautiful intersection of statistical mechanics and biology. These “Potts” models are going to be the centre of a larger discussion in this blog on state-based discretisations of proteins, how they’ve shaped modern deep learning methods and whether there is still more to learn from them.

In the age of black box deep learning, does the Potts model still have a place?

The Potts/Ising Model

The Ising model is a well established popular theoretical physics model of ferromagnetism. Simply put, given a lattice of atoms each capable of adopting 1 of 2 spins (up and down) ferromagnetism arises when their spins align and their associated magnetic moments point in the same direction. The Ising model tries to parameterise the local and non-local relationships between atoms and their spin states such that we can learn the Hamiltonian of the system and its different configurations under the magnetic field. The Hamiltonian takes the following form for a system of N atoms


$$
E = -\sum_{i}^Nh_ix_i – \sum_{i<j}^N J_{ij}x_i x_j,
$$

where J is the “coupling energy” between any two atoms x_i and x_j, and h represents the magnetic field, or more appropriately for our purposes it can be framed as a single-site field dictating how an individual atom independently acts within the model. You might recognise the form this binary spin model takes as it arises naturally across the sciences including in Hopfield networks and graphical models.

Everything is an Ising-like model if you’re brave enough

Continue reading →

Will TurboQuant save us from the RAM apocalypse?

The LLM boom is causing a global shortage of the very same computer memory it needs to sustain itself. Reports suggest OpenAI’s Stargate project alone could consume up to 40% of global DRAM output. Frontier labs like Google DeepMind need to make their models more memory-efficient.


One such technique is TurboQuant, released by Google. TurboQuant is an example of an online “quantisation” method. LLMs represent information using large tensors of numerical values, where each number typically uses 64 or 32 bits. However, many values do not require full numerical precision, so we can “round” them using fewer bits and less memory. We can see this in the example below:

The rounded value now requires 4x less memory. Source

Some quantisation methods are applied offline before inference begins. TurboQuant is ‘online’ because it compresses the KV cache dynamically during inference.

Continue reading →

Three Resources I Keep Coming Back to for Learning Deep Learning

There is no shortage of AI content online, but over time I have found myself returning to the same handful of resources again, and I wanted to share the three that have helped me the most.

AI Summer

This one I would recommend to anyone who is earlier in their journey. AI Summer at theaisummer.com is a free platform run by Sergios Karagiannakos and Nikolas Adaloglou, and it covers everything from the basics of neural networks through to building and deploying real ML systems. The tone is friendly and practical, and there are proper code examples throughout. It is one of those rare resources that manages to be beginner-friendly without feeling watered down.

Continue reading →

What Molecular ML Can Learn from the Vision Community’s Representation Revolution

Something remarkable happened in computer vision in 2025: the fields of generative modeling and representation learning, which had developed largely independently, suddenly converged. Diffusion models started leveraging pretrained vision encoders like DINOv2 to dramatically accelerate training. Researchers discovered that aligning generative models to pretrained representations doesn’t just speed things up—it often produces better results.

As someone who works on generative models for (among other things) molecules and proteins, I’ve been watching this unfold with great interest. Could we do the same thing for molecular ML? We now have foundation models like MACE that learn powerful atomic representations. Could aligning molecular generative models to these representations provide similar benefits?

In this post, I’ll summarize what happened in vision (organized into four “phases”), and then discuss what I think are the key lessons for molecular machine learning. The punchline: many of these ideas are already starting to appear in our field, but we’re still in the early stages compared to vision.

For a more detailed treatment of the vision developments with full references and figures, see the extended blog post on my website.

Continue reading →

Chemical Languages in Machine Learning

For more than a century, chemists have been trying to squeeze the beautifully messy, quantum-smeared reality of molecules into tidy digital boxes, “formats” such as line notations, connection tables, coordinate files, or even the vaguely hieroglyphic Wiswesser Line Notation. These formats weren’t designed for machine learning; some weren’t even designed for computers. And yet, they’ve become the wedged into the backbones of modern drug discovery, materials design and computational chemistry.

The emergent use of large language models and natural language processing in chemistry posits the immediate question: What does it mean for a molecule to have a “language,” and how should machines speak it?

if molecules are akin to words and sentences, what alphabet and grammatical rules should they follow?

What follows is a tour through the evolving world of chemical languages, why we use them, why our old representations keep breaking our shiny new models, and what might replace them.

Continue reading →

An Introduction to the Basics of Reinforcement Learning

Reinforcement learning (RL) is pretty simple in theory – “take actions, get rewards, increase likelihood of high reward actions”. However, we can quickly runs into subtle problems that don’t show up in standard supervised learning. The aim of this post is to give a gentle, concrete introduction to what RL actually is, why we might want to use it instead of (or alongside) supervised learning, and some of the headaches (figure 1) that come with it: sparse rewards, credit assignment, and reward shaping.

Figure 1: I’d like to help take you from confusion/headache 🙁 (left) to having a least some clarity 🙂 (right) with regard to what reinforcement learning is and where its useful

Rather than starting with Atari or robot arms, we’ll work through a small toy environment: a paddle catching falling balls. It’s simple enough to understand visually, but rich enough to show how different reward designs can lead to completely different behaviours, even when the underlying environment and objective are the same. Along the way, we’ll connect the code to the standard RL formalism (MDPs, returns, policy gradients), so you can see how the equations map onto something you can actually run.

Continue reading →