Large language models are fluent, but fluency is not reliability. My group works on language models and agents that reason reliably, where their outputs can be constrained to what we specify, or the expected requirements, which are then checked against what is true, and trusted by the people who use them. We draw on symbolic AI, cognitive science and human-computer interaction to get there. Three directions anchor the work.
Many tasks come with rules that a model's output must obey, say a grammar, a planning domain, a schema, or a logical theory. Rather than hoping a model learns these rules from data, we enforce them during generation. We combine LLMs with answer-set programming, context-sensitive grammars and search, so that even small models produce outputs that are valid by construction. We also learn the constraints themselves from data, quantify uncertainty with conformal prediction, and extend these ideas to embodied agents acting in the world.
With Mohammad Albinhassan, Lachlan McPheat, Navdeep Kaur, and collaborators Alessandra Russo (Imperial) and Anthony Cohn (Leeds).
A model that is right most of the time is hard to use if we cannot tell when it is wrong. We study when models should and should not answer, especially in retrieval-augmented generation. We build evaluations that look past headline scores to where and why models fail. And we also study the human side, how people, including expert researchers, check the outputs of LLMs, and how systems could make that verification easier. This line also covers LLMs as tools for building and judging knowledge bases.
With Syed Huq, Monica Visani Scozzi, and Alex Clay.
Charts and dashboards are how much of the world's data reaches people, and they are a demanding test of multimodal reasoning. We build benchmarks of real-world charts and analytical narratives, study how vision-language models read visualisations and where their attention goes, and develop agents that check claims against interactive visualisations. This work grows out of my earlier research on grounding language in vision.
With Maeve Hutchinson, and collaborators Radu Jianu, Aidan Slingsby and Jo Wood (City St George's).
Across these directions, I am interested in what human cognition can teach us about learning. Constraints that look like limitations, such as bounded working memory, can act as useful inductive biases when data is scarce, and Bayesian models of belief help explain why people fall for misinformation.
We apply these methods where reliability matters most, such as detecting and decoding the coded language of online extremism, and using LLMs for computer networks and cybersecurity (with India-Jane Barry).
Before LLMs, my work focused on multimodal machine learning: visually grounded machine translation, image and video captioning, and evaluation of vision-language generation. Earlier still, I worked on structured prediction with low-rank tensor models for dependency parsing.