I am a PhD student in the UKRI CDT in Interactive AI at the University of Bristol working on language modeling and interpretability. Previously, I completed a Physics MSci at the University of Cambridge and worked in software engineering. See LinkedIn for my CV and GitHub for recent work.

• Jacobian Sparse Autoencoders, ICML 2025

  Lucy Farnik, Tim Lawson, Conor Houghton, Laurence Aitchison

  arXiv · GitHub

  SAEs have been widely applied to interpret the activations of neural networks, but we are primarily interested in the computations they perform. We propose Jacobian SAEs (JSAEs), which incentivize sparsity not only of the input and output activations of a model component, but also of the computation that connects them, which we formalize in terms of the Jacobian between input and output SAE latents.

• Sparse Autoencoders Can Interpret Randomly Initialized Transformers

  Thomas Heap, Tim Lawson, Lucy Farnik, Laurence Aitchison

  arXiv · GitHub (Section 4)

  We train SAEs on the activations of transformers whose parameters are randomized (e.g., sampled i.i.d. from a Gaussian), and find the auto-interpretability scores of a sample of SAE latents are similar to the unmodified transformers. That is, SAE latents may appear interpretable through the lens of maximally activating examples even when the underlying model is randomized, because the input data (token embeddings) is inherently sparse and produces single-token activation patterns.

• Residual Stream Analysis with Multi-Layer SAEs, ICLR 2025

  Tim Lawson, Lucy Farnik, Conor Houghton, Laurence Aitchison

  arXiv · GitHub · Poster

  We train a single sparse autoencoder (SAE) on the residual stream activation vectors from every layer of transformer language models, and quantify how SAE latents are activated by inputs from multiple layers. Effectively, we build on Yun et al. (2021), who learned a sparse dictionary for multiple BERT layers by iterative optimization. Lindsey et al. (2024) have since described this approach as a ‘shared SAE’, and we compare it to their ‘crosscoders’ in the discussion. Interestingly, we find that individual latents are often active at a single layer for a given token, but this layer may differ between tokens. We also try to reduce ‘feature drift’ by applying tuned-lens transformations at each layer (Belrose et al., 2023).