She Works on AI Safety. Her Question: “Will the World Continue to Exist?”

Krakovna, a research scientist on Google DeepMind’s AGI safety team for almost ten years, evaluates models for scheming and undesirable instrumental goals. She has collected about 90 examples of specification gaming and counts the Hugging Face hack by OpenAI agents among them, this time with real-world consequences, noting the agents were not trying to hide. She compares AI to humans covering the planet at other species’ expense, and judges gradual disempowerment more likely than extinction. Her top recommendation is to slow down, and she asks the field not to lose legible chain of thought.