Progress in artificial intelligence
Artificial intelligence applications have been used in a wide range of fields including medical diagnosis, stock trading, robot control, law, scientific discovery and toys. However, many AI applications are not perceived as AI: "A lot of cutting edge AI has filtered into general applications, often without being called AI because once something becomes useful enough and common enough it's not labeled AI anymore." "Many thousands of AI applications are deeply embedded in the infrastructure of every industry." In the late 1990s and early 21st century, AI technology became widely used as elements of larger systems, but the field is rarely credited for these successes.
To allow comparison with human performance, artificial intelligence can be evaluated on constrained and well-defined problems. Such tests have been termed subject matter expert Turing tests. Also, smaller problems provide more achievable goals and there are an ever-increasing number of positive results.
In his famous Turing test, Alan Turing picked language, the defining feature of human beings, for its basis. But there are many other useful abilities that can be described as showing some form of intelligence. This gives better insight into the comparative success of artificial intelligence in different areas.
In what has been called the Feigenbaum test, the inventor of expert systems argued for subject specific expert tests. A paper by Jim Gray of Microsoft in 2003 suggested extending the Turing test to speech understanding, speaking and recognizing objects and behavior.
Broad classes of outcome for an AI test may be given as:
- optimal: it is not possible to perform better
- super-human: performs better than all humans
- high-human: performs better than most humans
- par-human: performs similarly to most humans
- sub-human: performs worse than most humans
- Connect Four
- Rubik's Cube
- Heads-up limit hold'em poker: statistically optimal in the sense that "a human lifetime of play is not sufficient to establish with statistical significance that the strategy is not an exact solution".
- Chess: top human can compete only with handicap in their favor
- Go: beat a top human 4–1 in a five-game match in 2016
- Jigsaw puzzles: 
- Reversi: 
- Scrabble: 
- Bridge: world class
- Backgammon: probably world class
- Arimaa: "Beat 3 selected players...Currently the best Arimaa players are humans."
- Quiz show: question answering although the machine did not use speech recognition
- Texas hold 'em; 
- Optical character recognition for ISO 1073-1:1976 and similar special characters.
- Classification of images
- Crosswords: Solves 80% of New York Times clues.
- Handwriting recognition 
- Optical character recognition for printed text (nearing par-human for Latin-script typewritten text)
- Object recognition
- Driving a car: "Between September 2014 and November 2015, Google’s autonomous vehicles in California experienced 272 failures and would have crashed at least 13 times if their human test drivers had not intervened."
- Speech recognition
- Word-sense disambiguation
- Natural language processing
- Applications of artificial intelligence
- List of artificial intelligence projects
- List of emerging technologies
- AI set to exceed human brain power CNN.com (July 26, 2006)
- Kurtzweil 2005, p. 264
- National Research Council (1999), "Developments in Artificial Intelligence", Funding a Revolution: Government Support for Computing Research, National Academy Press, ISBN 0-309-06278-0, OCLC 246584055 under "Artificial Intelligence in the 90s"
- Turing, Alan (October 1950), "Computing Machinery and Intelligence", Mind, LIX (236): 433–460, ISSN 0026-4423, doi:10.1093/mind/LIX.236.433, retrieved 2008-08-18
- Feigenbaum, Edward A. (2003). "Some challenges and grand challenges for computational intelligence". Journal of the ACM. 50 (1): 32–40. doi:10.1145/602382.602400.
- Gray, Jim (2003). "What Next? A Dozen Information-Technology Research Goals". Journal of the ACM. 50 (1): 41–57. Bibcode:1999cs.......11005G. arXiv: [cs.GL].
- Schaeffer, J.; Burch, N.; Bjornsson, Y.; Kishimoto, A.; Muller, M.; Lake, R.; Lu, P.; Sutphen, S. (2007). "Checkers is solved". Science. 317 (5844): 1518–1522. Bibcode:2007Sci...317.1518S. CiteSeerX . PMID 17641166. doi:10.1126/science.1144079.
- "God's Number is 20".
- Bowling, M.; Burch, N.; Johanson, M.; Tammelin, O. (2015). "Heads-up limit hold'em poker is solved". Science. 347 (6218): 145–9. Bibcode:2015Sci...347..145B. PMID 25574016. doi:10.1126/science.1259433.
- see for example: https://www.chess.com/news/komodo-beats-nakamura-in-final-battle-1331
- AlphaGo versus Lee Sedol
- "Computer software sets new record for solving jigsaw puzzle".
- Reversi#Computer opponents
- Sheppard, B. (2002). "World-championship-caliber Scrabble". Artificial Intelligence. 134: 241–275. doi:10.1016/S0004-3702(01)00166-7.
- Computer bridge#Computers versus humans
- Tesauro, Gerald (March 1995). "Temporal difference learning and TD-Gammon". Communications of the ACM. 38 (3): 58–68. doi:10.1145/203330.203343.
- "The Arimaa Challenge". 2015. Retrieved Jan 12, 2017.
- Watson beats Jeopardy grand-champions. http://www.nytimes.com/2011/02/17/science/17jeopardy-watson.html
- Jackson, Joab. "IBM Watson Vanquishes Human Jeopardy Foes". PC World. IDG News. Retrieved 2011-02-17.
- "Microsoft researchers say their newest deep learning system beats humans -- and Google - VentureBeat - Big Data - by Jordan Novet". VentureBeat.
- "Proverb, the Crossword-Solving Computer Program". American Crossword Puzzle tournament. Retrieved Dec 18, 2016.
- Proverb: The probabilistic cruciverbalist. By Greg A. Keim, Noam Shazeer, Michael L. Littman, Sushant Agarwal, Catherine M. Cheves, Joseph Fitzgerald, Jason Grosland, Fan Jiang, Shannon Pollard, and Karl Weinmeister. 1999. In Proceedings of the Sixteenth National Conference on Artificial Intelligence, 710-717. Menlo Park, Calif.: AAAI Press.
- Wernick, Adam (24 Sep 2014). "'Dr. Fill' vies for crossword solving supremacy, but still comes up short". Public Radio International. Retrieved Dec 18, 2016.
- "One-shot Learning with Memory-Augmented Neural Networks; Page 5: Table 1". 19 May 2016. Retrieved 2017-06-04.
4.2. Omniglot Classification: "The network exhibited high classification accuracy on just the second presentation of a sample from a class within an episode (82.8%), reaching up to 94.9% accuracy by the fifth instance and 98.1% accuracy by the tenth."
- There are several ways of evaluating machine translation systems. People competent in a second language frequently outperform machine translation systems but the average person is often less capable. Some machine translation systems are capable of a large number of languages, like google translate, and as a result have a broader competence than most humans. For example, very few humans can translate from Arabic to Polish and French to Swahili and Armenian to Vietnamese. When comparing over several languages machine translation systems will tend to outperform humans.
- Harris, Mark (12 Jan 2016). "Google reports self-driving car mistakes: 272 failures and 13 near misses". The Guardian. Retrieved Dec 18, 2016.