Communications of the ACM
Designing AI Systems that Obey Our Laws and Values
Credit: Willyam Bradberry
Operational AI systems (for example, self-driving cars) need to obey both the law of the land and our values. We propose AI oversight systems ("AI Guardians") as an approach to addressing this challenge, and to respond to the potential risks associated with increasingly autonomous AI systems.a These AI oversight systems serve to verify that operational systems did not stray unduly from the guidelines of their programmers and to bring them back in compliance if they do stray. The introduction of such second-order, oversight systems is not meant to suggest strict, powerful, or rigid (from here on 'strong') controls. Operations systems need a great degree of latitude in order to follow the lessons of their learning from additional data mining and experience and to be able to render at least semi-autonomous decisions (more about this later). However, all operational systems need some boundaries, both in order to not violate the law and to adhere to ethical norms. Developing such oversight systems, AI Guardians, is a major new mission for the AI community.
All societies throughout history have had oversight systems. Workers have supervisors; businesses have accountants; schoolteachers have principals. That is, all these systems have hierarchies in the sense that the first line operators are subject to oversight by a second layer and are expected to respond to corrective signals from the overseers. (These, in turn, are expected to take into account suggestions or even demands by the first line to change their modes of oversight). John Perry Barlow, in his famous "Declaration of the Independence of Cyberspace" in 1996, described the burgeoning online world as one that would be governed by a social contract formed among its users.b
Comments
David Parnas
What would you need to change in this article if you replaced "AI" with "Computer System"?
Oren Etzioni
David's question/comment is a fair one, and the answer is that it's a continuum. Computer systems raise the same issues, but their gradually increasing autonomy, adaptability, and unpredictability when they are AI systems--makes this issue particularly pressing in this context.
Displaying all 2 comments