Large Language Models -- A Physicist's Perspective

A seminar by Tobias Osborne on orchestrating a swarm of coding agents to work on physics problems

In December I started experimenting with using Claude Code for physics calculations, and was playing with different methods for verifying their output. The buzz around Claude Code was huge, but I didn’t see many physicists using it, until I stumbled upon Tobias Osborne’s GitHub. I immediately contacted him, and have learned a lot from our discussions these past months. He recently visited us at UCL and gave a talk to the Physics department. We also held a few skill-sharing sessions. His talk is available on our YouTube channel, which we plan to relaunch in September (along with a new initiative at the interface of quantum information theory and gravity).

We’re in a funny time, where AI evangelists seem to think models are far better at physics and maths than they actually are. At the same time, many researchers completely underrate the power of frontier models — they tried ChatGPT once, found it unreliable and over-confident and haven’t engaged with more recent coding agents.

The truth lies in between. These models are neither geniuses nor useless, and you need to know the physics well to understand where they fail. I completely agree with Tobias when he describes coding agents as hyperactive Masters students. They’re fast and relentless, but very unreliable. So finding ways to verify their output is important. But with a bit of scaffolding around these models and domain expertise, you can make them more reliable. I highly recommend Tobias’s talk, especially if you’re new to using coding agents for tasks outside of software engineering.

You can find the slides of Tobias’s talk on his GitHub. During the talk he mentions a story about me getting Claude to reveal its system prompt (the hidden instructions that tell the model how to behave, before you even type anything). That was a wild conversation, so I’ve posted it here. The demo he gives during his talk can be found on his GitHub too. We’ve confirmed with some experts that Claude Code did make a reasonable start on a very hard problem.

As always, we are entering both an exciting and terrifying period. If you want to see what some of the fuss is about, watch Tobias’s talk, and find details of Tobias’s and other upcoming seminars on our seminar page.