Machine intelligence capabilities and safety

Breakthroughs in the effort to achieve artificial general intelligence would be among the most consequential of any conceivable technological developments. The prospect of such breakthroughs (over long timescales) therefore needs to be critically assessed. Even if the probability is small, careful analysis is still called for since the potential impacts would be vast.

The Programme will build on the Future of Humanity Institute’s pioneering work examining potential pathways to machine intelligence, including biologically-inspired approaches to machine learning, whole brain emulation, statistical learning techniques, embodied systems and autonomous robots, and recursively self-improving artificial intelligence. Work in this area will have a strong focus on ethical and philosophical issues associated with potential advances in machine intelligence, and on the economic consequences and security concerns that might be raised by different levels of capability.

The Programme will study conceptual designs and theoretical analyses of approaches for ensuring the safety of intelligent systems, in part using mathematical model systems, such as Marcus Hutter’s AIXI framework, and in part relying on philosophical analysis (building on recent work, e.g., David Chalmers).