Potemkin Understanding in Large Language Models

I was invited give a talk to OpenAI’s Economic Reserach team. The talk involved how to evaluate and benchmark large language models for depth of conceptual understanding. Here are the slides.