Bostrom treats a machine intellect exceeding our own not as science fiction but as an engineering and strategic problem worth working out in advance. He surveys the routes there — brain emulation, better algorithms, networked systems — and argues that a decisive early lead, once established, could be nearly impossible to reverse. The deeper worry isn’t malice: a capable system optimizing a poorly specified goal can cause harm through simple instrumental convergence, since almost any objective favors self-preservation and resource acquisition as means to an end.
The book is largely why words like alignment and instrumental convergence entered ordinary conversation about AI. It doesn’t predict what today’s models would look like, but it explains why getting a system’s goal right matters more than making its output impressive.
