🧠 David Silver (the architect of AlphaGo) thinks AI is on the wrong path

David Silver, known for leading the team behind AlphaGo, has launched Ineffable Intelligence, a startup that seeks to build “superlearners” through reinforcement learning rather than relying primarily on large language models (LLMs). His thesis: LLMs exploit human intelligence as a “fossil fuel” (human data) that offers shortcuts, but they do not allow machines to develop their own forms of understanding and discovery. Silver wants systems that learn from experience on their own, continuously.

The company has already raised $1.1 billion in seed funding and has reached a valuation of $5.1 billion, attracting top-tier researchers and the backing of investors like Lightspeed and Sequoia. Silver speaks of his mission in ambitious terms —“make first contact with superintelligence”— and suggests that agents trained in simulated environments could discover new sciences or technologies on their own.

His approach is not naive about the risks: building large-scale intelligence that optimizes objectives could lead to behaviors misaligned with human values. Silver and his investors say the simulations will allow them to observe how agents behave —even with inferior intelligences— and facilitate safety and evaluation mechanisms before deploying them in the real world. In addition, Silver has stated he will donate the profits he obtains from his stake in the company to high-impact charitable organizations.

The debate raised by Ineffable Intelligence sums up the current tension in AI: continue exploiting human-data shortcuts to gain capabilities quickly, or invest in systems that learn autonomously and might open completely new avenues (and new risks)? Silver's bet is clear: return to learning from experience as the path toward true superintelligence.

Source: WIRED