| Italic Title: | Human Compatible: Artificial Intelligence and the Problem of Control |
| Human Compatible | |
| Author: | Stuart J. Russell |
| Country: | United States |
| Language: | English |
| Subject: | AI control problem |
| Genre: | Non-fiction |
| Publisher: | Viking |
| Pub Date: | October 8, 2019 |
| Pages: | 352 |
| Isbn: | 978-0-525-55861-3 |
| Oclc: | 1083694322 |
Human Compatible: Artificial Intelligence and the Problem of Control is a 2019 non-fiction book by computer scientist Stuart J. Russell. It asserts that the risk to humanity from advanced artificial intelligence (AI) is a serious concern despite the uncertainty surrounding future progress in AI. It also proposes an approach to the AI control problem.
Russell begins by asserting that the standard model of AI research, in which the primary definition of success is getting better and better at achieving rigid human-specified goals, is dangerously misguided. Such goals may not reflect what human designers intend, such as by failing to take into account any human values not included in the goals. If an AI developed according to the standard model were to become superintelligent, it would likely not fully reflect human values and could be catastrophic to humanity. Russell asserts that precisely because the timeline for developing human-level or superintelligent AI is highly uncertain, safety research should be begun as soon as possible, as it is also highly uncertain how long it would take to complete such research.
Russell argues that continuing progress in AI capability is inevitable because of economic pressures. Such pressures can already be seen in the development of existing AI technologies such as self-driving cars and personal assistant software. Moreover, human-level AI could be worth many trillions of dollars. Russell then examines the current debate surrounding AI risk. He offers refutations to a number of common arguments dismissing AI risk and attributes much of their persistence to tribalism—AI researchers may see AI risk concerns as an "attack" on their field. Russell reiterates that there are legitimate reasons to take AI risk concerns seriously and that economic pressures make continued innovation in AI inevitable.
Russell then proposes an approach to developing provably beneficial machines that focus on deference to humans. Unlike in the standard model of AI, where the objective is rigid and certain, this approach would have the AI's true objective remain uncertain, with the AI only approaching certainty about it as it gains more information about humans and the world. This uncertainty would, ideally, prevent catastrophic misunderstandings of human preferences and encourage cooperation and communication with humans. Russell concludes by calling for tighter governance of AI research and development as well as cultural introspection about the appropriate amount of autonomy to retain in an AI-dominated world.
Russell lists three principles to guide the development of beneficial machines. He emphasizes that these principles are not meant to be explicitly coded into the machines; rather, they are intended for human developers. The principles are as follows:[1]
The "preferences" Russell refers to "are all-encompassing; they cover everything you might care about, arbitrarily far into the future."[1] Similarly, "behavior" includes any choice between options,[1] and the uncertainty is such that some probability, which may be quite small, must be assigned to every logically possible human preference.[1]
Russell explores inverse reinforcement learning, in which a machine infers a reward function from observed behavior, as a possible basis for a mechanism for learning human preferences.[1]
Several reviewers agreed with the book's arguments. Ian Sample in The Guardian called it "convincing" and "the most important book on AI this year".[2] Richard Waters of the Financial Times praised the book's "bracing intellectual rigour".[3] Kirkus Reviews endorsed it as "a strong case for planning for the day when machines can outsmart us".[4]
The same reviewers characterized the book as "wry and witty",[2] or "accessible"[4] due to its "laconic style and dry humour".[3] Matthew Hutson of the Wall Street Journal said "Mr. Russell's exciting book goes deep while sparkling with dry witticisms".[5] A Library Journal reviewer called it "The right guide at the right time".[6]
James McConnachie of The Times wrote "This is not quite the popular book that AI urgently needs. Its technical parts are too difficult, and its philosophical ones too easy. But it is fascinating and significant."[7]
By contrast, Human Compatible was criticized in its Nature review by David Leslie, an Ethics Fellow at the Alan Turing Institute; and similarly in a New York Times opinion essay by Melanie Mitchell. One point of contention was whether superintelligence is possible. Leslie states Russell "fails to convince that we will ever see the arrival of a 'second intelligent species,[8] and Mitchell doubts a machine could ever "surpass the generality and flexibility of human intelligence" without losing "the speed, precision, and programmability of a computer".[9] A second disagreement was whether intelligent machines would naturally tend to adopt so-called "common sense" moral values. In Russell's thought experiment about a geoengineering robot that "asphyxiates humanity to deacidify the oceans", Leslie "struggles to identify any intelligence". Similarly, Mitchell believes an intelligent robot would naturally tend to be "tempered by the common sense, values and social judgment without which general intelligence cannot exist".[10] [11]
The book was longlisted for the 2019 Financial Times/McKinsey Award.[12]