Hi there stranger, welcome to my page!

I’m a PhD student in machine learning at the University of Edinburgh supervised by Siddharth Narayanaswamy and Ivan Titov. I focus on designing architectures that incorporate explicit hierarchies, using the inductive bias to power efficient, effective representation learning.

I also work with MSR, mentored by Paul Smolensky and Roland Fernandez. Here I’ve looked at what kind of structured reasoning transformer models are capable of representing implicitly. Understanding their successes and shortcomings, and using that to drive architecture design.

Earlier, I did my undergrad in Philosophy at UCL, which got me interested in mathematicizing language and cognition. I was also teaching myself coding and ML in my free time, which let me take the MSc in Speech and Language Processing, and from there start my PhD.

Outside of research I have also held some more applied part time roles in industry, mainly focussing on information extraction and RAG, you can find more details and a full list of my publications in my CV..

News:

(Last Update 06.2025)

• (06.2025) I’m back at MSR as an intern over the summer 🌞 Where I’ll be working on developing architectures to make transformers better at generalisation and reasoning..
• (06.2025) Threshold Relative Attention is accepted to TMLR. We will also be presenting it at the MOSS workshop at ICML. Come check it out if you are interested in length generalisation and long context language models!
• (05.2025) Banyan has been accepted to ICML 2025! Here we develop a recursive neural network that learns strong embeddings for different levels of hierarchy (words, sentences, paragraphs etc.) super efficiently, ideal if you are working with low resource languages or specialised domains.
• (12.2024) S-DTM accepted as a spotlight to NeurIPS 2024! This model improves upon the existing DTM architecture by making it applicable to many more types of a problems while retaining its ability to demonstrate unique kinds of generalisation. Big contratulations to Paul Soulos, the first author, was a really fun paper to be a part of!
• (07.2024) Our paper stress testing our Self-StrAE architecture in a real competition is accepted to SemEval 2024. This self-structuring (recursive) autoencoder is small but mighty, placing competitively against million/billion parameter LLMs with only 14 non-embedding parameters 😎
• (06.2024 - 02.2025) My first stint as an intern and then part-time at MSR, working with Paul Smolensky and Roland Fernandez on the implicit mechanisms of symbolic reasoning and compositional generalisation in transformer LLMs.
• (12.2023) Two papers at EMNLP! One (StrAE) in the main conference where we look at what an inductive bias for implicit composition buys you (very efficient representation learning), and introduce two new architectures: the Structured AutoEncoder and itself unsupervised Self-Structuring twin. We also have a paper at the CoNLL BabyLM challenge looking at whether curriculum learning can make pre-training more effective 🤓