Explainable artificial intelligence
Explainable AI (XAI) is a neologism that has recently reached the parlance of artificial intelligence. Its purpose is to provide accountability when addressing technological innovations ascribed to dynamic and non-linearly programmed systems, e.g. artificial neural networks, deep learning, and genetic algorithms.
It is about asking the question of how algorithms arrive at their decisions. In a sense, it is a technical discipline providing operational tools that might be useful for explaining systems, such as in implementing a right to explanation.[1]
AI-related algorithmic (supervised and unsupervised) practices work on a model of success that orientates towards some form of correct state, with singular focus placed on an expected output. E.g., an image recognition algorithm's level of success will be based on the algorithm's ability to recognize certain objects, and failure to do so will indicate that the algorithm requires further tuning. As the tuning level is dynamic, closely correlated to function refinement and training data-set, granular understanding of the underlying operational vectors is rarely introspected.
XAI aims to address this black-box approach and allow introspection of these dynamic systems tractable, allowing humans to understand how computational machines develop their own models for solving tasks.
Definition
A universal definition of this term has yet to have been fully established; however, the DARPA XAI program defines its aims as the following:
- Produce more explainable models, while maintaining a high level of learning performance (prediction accuracy)
- Enable human users to understand, appropriately trust, and effectively manage the emerging generation of artificially intelligent partners[2]
History
While the term "Explainable AI" is new, the field of understanding the knowledge embedded in machine learning systems itself has a long history. Researchers have long been interested in whether it is possible to extract rules from trained neural networks[3], and researchers in clinical expert systems creating neural network-powered decision support for clinicians have sought to develop dynamic explanations that allow these technologies to be more trusted and trustworthy in practice.[1]
Newer however is the focus on explaining machine learning and AI to those whom the decisions concern, rather than the designers or direct users of decision systems. Since DARPA's introduction of its program in 2016, a number of new initiatives seek to address the issue of algorithmic accountability and provide transparency concerning how technologies within this domain function.
- 25 April 2017: Nvidia published the paper "Explaining How a Deep Neural Network Trained with End-to-End Learning Steers a Car"[4]
- 13 July 2017: Accenture recommended "Responsible AI: Why we need Explainable AI"[5]
Accountability
A cross-section of industrial sectors will be affected by these requirements, as accountability is delegated to a greater or lesser extent from humans to machines.
Examples of these effects have already been seen in the following sectors:
- Neural Network Tank imaging[6]
- Antenna design (evolved antenna)[7]
- Algorithmic trading (high-frequency trading)[8]
- Medical diagnoses[9]
- Autonomous vehicles[10][11]
Recent developments
As regulators, official bodies and general users dependency on AI-based dynamic systems, clearer accountability will be required for decision making processes to ensure trust and transparency. Evidence of this requirement gaining more momentum can be seen with the launch of the first global conference exclusively dedicated to this emerging discipline, the International Joint Conference on Artificial Intelligence: Workshop on Explainable Artificial Intelligence (XAI).[12]
References
- ^ a b Edwards, Lilian; Veale, Michael (2017). "Slave to the Algorithm? Why a 'Right to an Explanation' Is Probably Not the Remedy You Are Looking For". Duke Law and Technology Review.
- ^ "Explainable Artificial Intelligence (XAI)". DARPA. DARPA. Retrieved 17 July 2017.
- ^ Tickle, A. B.; Andrews, R.; Golea, M.; Diederich, J. (November 1998). "The truth will come to light: directions and challenges in extracting the knowledge embedded within trained artificial neural networks". IEEE Transactions on Neural Networks. 9 (6): 1057–1068. doi:10.1109/72.728352. ISSN 1045-9227.
- ^ "Explaining How a Deep Neural Network Trained with End-to-End Learning Steers a Car" (PDF). Arxiv. Arxiv. Retrieved 17 July 2017.
- ^ "Responsible AI: Why we need Explainable AI". YouTube. Accenture. Retrieved 17 July 2017.
- ^ "Neual Network Tank image". Neil Fraser. Neil Fraser. Retrieved 17 July 2017.
- ^ "NASA 'Evolutionary' software automatically designs antenna". NASA. NASA. Retrieved 17 July 2017.
- ^ "The Flash Crash: The Impact of High Frequency Trading on an Electronic Market" (PDF). CFTC. CFTC. Retrieved 17 July 2017.
- ^ "Can machine-learning improve cardiovascular risk prediction using routine clinical data?". PLOS One. PLOS One. Retrieved 17 July 2017.
- ^ "Tesla says it has 'no way of knowing' if autopilot was used in fatal Chinese crash". Guardian. Guardian. Retrieved 17 July 2017.
- ^ "Joshua Brown, Who Died in Self-Driving Accident, Tested Limits of His Tesla". New York Times. New York Times. Retrieved 17 July 2017.
- ^ "IJCAI 2017 Workshop on Explainable Artificial Intelligence (XAI)". Earthlink. IJCAI. Retrieved 17 July 2017.
External links
- ‘Explainable Artificial Intelligence’: Cracking open the black box of AI
- Attentive Explanations: Justifying Decisions and Pointing to the Evidence
- Explainable AI - International Research Group
- End-to-End Deep Learning for Self-Driving Cars
- Explaining How End-to-End Deep Learning Steers a Self-Driving Car
- Accenture's Responsible AI Imperative
- PWC - Responsible AI