Researchers Need to Learn How AI Makes Decisions, Says Goodfire’s Tom McGrath

To understand the decision-making of AI, developers often look at the text that models produce as they reason through problems. But these “chains of thought” are increasingly hard to read, which is prompting AI developers to consider other approaches. One alternative, known as “mechanistic interpretability,” seeks to explain the inner workings of AI models, rather than just relying on their outward behavior.

Leading AI companies have formed mechanistic interpretability teams in recent years, and the field has spawned several startups, the most prominent of which is Goodfire AI, whose chief scientist Tom McGrath joined the latest episode of AI Deep Dive. (The full episode is available here.)