Jump to content

Geoffrey Hinton

From Wikipedia, the free encyclopedia

This is an old revision of this page, as edited by Ptrnext (talk | contribs) at 03:28, 3 May 2023 (→‎Views: ce). The present address (URL) is a permanent link to this revision, which may differ significantly from the current revision.

Geoffrey Hinton
CC FRS FRSC
Hinton in 2013
Born
Geoffrey Everest Hinton

(1947-12-06) 6 December 1947 (age 76)[11]
Wimbledon, London, England
Education
Known for
Awards
Scientific career
Fields
Institutions
ThesisRelaxation and its role in vision (1977)
Doctoral advisorChristopher Longuet-Higgins[2][3][4]
Doctoral students
Other notable students
Websitewww.cs.toronto.edu/~hinton/ Edit this at Wikidata

Geoffrey Everest Hinton CC FRS FRSC[12] (born 6 December 1947) is a British-Canadian cognitive psychologist and computer scientist, most noted for his work on artificial neural networks. From 2013 to 2023, he divided his time working for Google (Google Brain) and the University of Toronto, before publicly announcing his departure from Google in May 2023 citing concerns about the risks of artificial intelligence (AI) technology.[13] After leaving, he has commended Google for acting "very responsibly" while developing their AI[14] but changed once Microsoft started incorporating a chatbot into its Bing search engine, and the company began becoming concerned about the risk to its search business. In 2017, he co-founded and became the chief scientific advisor of the Vector Institute in Toronto.[15][16]

With David Rumelhart and Ronald J. Williams, Hinton was co-author of a highly cited paper published in 1986 that popularised the backpropagation algorithm for training multi-layer neural networks,[17] although they were not the first to propose the approach.[18] Hinton is viewed as a leading figure in the deep learning community.[19][20][21][22][23] The dramatic image-recognition milestone of the AlexNet designed in collaboration with his students Alex Krizhevsky[24] and Ilya Sutskever for the ImageNet challenge 2012[25] was a breakthrough in the field of computer vision.[26]

Hinton received the 2018 Turing Award, together with Yoshua Bengio and Yann LeCun, for their work on deep learning.[27] They are sometimes referred to as the "Godfathers of AI" and "Godfathers of Deep Learning",[28][29] and have continued to give public talks together.[30][31]

In May 2023, Hinton announced his resignation from Google in order to be able to "freely speak out about the risks of A.I."[32] He has voiced concerns about deliberate misuse by malicious actors, technological unemployment, and existential risk from artificial general intelligence.[33]

Education

Hinton was educated at King's College, Cambridge, graduating in 1970 with a bachelor of arts in experimental psychology.[11] He continued his study at the University of Edinburgh where he was awarded a PhD in artificial intelligence in 1978 for research supervised by Christopher Longuet-Higgins.[2][34]

Career and research

After his PhD, Hinton worked at the University of Sussex and (after difficulty finding funding in Britain),[35] the University of California, San Diego and Carnegie Mellon University.[11] He was the founding director of the Gatsby Charitable Foundation Computational Neuroscience Unit at University College London[11] and is currently[36] a professor in the computer science department at the University of Toronto. He holds a Canada Research Chair in Machine Learning and is currently[when?] an advisor for the Learning in Machines & Brains program at the Canadian Institute for Advanced Research. Hinton taught a free online course on Neural Networks on the education platform Coursera in 2012.[37] He joined Google in March 2013 when his company, DNNresearch Inc., was acquired, and was at that time planning to "divide his time between his university research and his work at Google".[38]

Hinton's research concerns ways of using neural networks for machine learning, memory, perception, and symbol processing. He has written or co-written more than 200 peer reviewed publications.[1][39] At the Conference on Neural Information Processing Systems (NeurIPS) he introduced a new learning algorithm for neural networks that he calls the "Forward-Forward" algorithm. The idea of the new algorithm is to replace the traditional forward-backward passes of backpropagation with two forward passes, one with positive (i.e. real) data and the other with negative data that could be generated solely by the network.[40]

While Hinton was a postdoc at UC San Diego, David E. Rumelhart and Hinton and Ronald J. Williams applied the backpropagation algorithm to multi-layer neural networks. Their experiments showed that such networks can learn useful internal representations of data.[17] In an interview of 2018,[41] Hinton said that "David E. Rumelhart came up with the basic idea of backpropagation, so it's his invention". Although this work was important in popularising backpropagation, it was not the first to suggest the approach.[18] Reverse-mode automatic differentiation, of which backpropagation is a special case, was proposed by Seppo Linnainmaa in 1970, and Paul Werbos proposed to use it to train neural networks in 1974.[18]

During the same period, Hinton co-invented Boltzmann machines with David Ackley and Terry Sejnowski.[42] His other contributions to neural network research include distributed representations, time delay neural network, mixtures of experts, Helmholtz machines and Product of Experts. In 2007, Hinton coauthored an unsupervised learning paper titled Unsupervised learning of image transformations.[43] An accessible introduction to Geoffrey Hinton's research can be found in his articles in Scientific American in September 1992 and October 1993.[44]

In October and November 2017 respectively, Hinton published two open access research papers on the theme of capsule neural networks,[45][46] which according to Hinton, are "finally something that works well".[47]

In May 2023, Hinton publicly announced his resignation from Google. He explained his decision by saying that he wanted to "freely speak out about the risks of A.I." and added that a part of him now regrets his life's work.[13][32]

Notable former PhD students and postdoctoral researchers from his group include Peter Dayan,[48] Sam Roweis,[48] Max Welling,[48] Richard Zemel,[2][5] Brendan Frey,[6] Radford M. Neal,[7] Yee Whye Teh,[8] Ruslan Salakhutdinov,[9] Ilya Sutskever,[10] Yann LeCun,[49] Alex Graves,[48] and Zoubin Ghahramani.

Honours and awards

In 2016, from left to right,
Russ Salakhutdinov, Richard S. Sutton, Geoffrey Hinton, Yoshua Bengio, and Steve Jurvetson

Hinton was elected a Fellow of the Royal Society (FRS) in 1998.[12] He was the first winner of the Rumelhart Prize in 2001.[50] His certificate of election for the Royal Society reads:

Geoffrey E. Hinton is internationally distinguished for his work on artificial neural nets, especially how they can be designed to learn without the aid of a human teacher. This may well be the start of autonomous intelligent brain-like machines. He has compared effects of brain damage with effects of losses in such a net, and found striking similarities with human impairment, such as for recognition of names and losses of categorisation. His work includes studies of mental imagery, and inventing puzzles for testing originality and creative intelligence. It is conceptual, mathematically sophisticated, and experimental. He brings these skills together with striking effect to produce important work of great interest.[51]

In 2001, Hinton was awarded an honorary doctorate from the University of Edinburgh.[52] He was the 2005 recipient of the IJCAI Award for Research Excellence lifetime-achievement award.[53] He has also been awarded the 2011 Herzberg Canada Gold Medal for Science and Engineering.[54] In 2013, Hinton was awarded an honorary doctorate from the Université de Sherbrooke.[55]

In 2016, he was elected a foreign member of National Academy of Engineering "for contributions to the theory and practice of artificial neural networks and their application to speech recognition and computer vision".[56] He also received the 2016 IEEE/RSE Wolfson James Clerk Maxwell Award.[57]

He has won the BBVA Foundation Frontiers of Knowledge Award (2016) in the Information and Communication Technologies category "for his pioneering and highly influential work" to endow machines with the ability to learn.[58]

Together with Yann LeCun, and Yoshua Bengio, Hinton won the 2018 Turing Award for conceptual and engineering breakthroughs that have made deep neural networks a critical component of computing.[59][60][61]

In 2018, he became a Companion of the Order of Canada.[62] In 2022, he received the Princess of Asturias Award in the Scientific Research category, along with Yann LeCun, Yoshua Bengio, and Demis Hassabis.[63]

Views

Risks of artificial intelligence

See also: AI safety

In 2023, Hinton expressed concerns about the rapid progress of rapid A.I.[33][32] Hinton previously believed that artificial general intelligence (AGI) was "30 to 50 years or even longer away."[32] However, in a March 2023 interview with CBS, he stated that "general-purpose AI" may be fewer than 20 years away and could bring about changes "comparable in scale with the Industrial Revolution or electricity."[33]

In an interview with The New York Times published on 1 May 2023,[32] Hinton announced his resignation from Google so he could "talk about the dangers of AI without considering how this impacts Google."[64] He noted that "a part of him now regrets his life’s work" due to his concerns and he expressed fears about a race between Google and Microsoft.[32]

On early May 2023, Hinton revealed in an interview with BBC that AI might soon surpass the information capacity of the human brain. He described some of the risks posed by these chatbots as "quite scary". Hinton explained that chatbots have the ability to learn independently and share knowledge. This means that whenever one copy acquires new information, it is automatically disseminated to the entire group. This allows AI chatbots to have the capability to accumulate knowledge far beyond the capacity of any individual.[65]

Existential risk from AGI

Hinton expressed concerns about AI takeover, stating that "it's not inconceivable" that AI could "wipe out humanity."[33] Hinton states that AI systems capable of intelligent agency will be useful for military or economic purposes.[66] He worries that generally intelligent AI systems could "create sub-goals" that are unaligned with their programmers' interests.[67] He states that AI systems may become power-seeking or prevent themselves from being shut off, not because programmers intended them to, but because those sub-goals are useful for achieving later goals.[68] In particular, Hinton says "we have to think hard about how to control" AI systems capable of self-improvement.[69]

Catastrophic misuse

Hinton worries about deliberate misuse of AI by malicious actors, stating that "it is hard to see how you can prevent the bad actors from using [AI] for bad things."[32] In 2017, Hinton called for an international ban on lethal autonomous weapons.[70]

Economic impacts

Hinton was previously optimistic about the economic effects of AI, noting in 2018 that: "The phrase ‘artificial general intelligence’ carries with it the implication that this sort of single robot is suddenly going to be smarter than you. I don’t think it’s going to be that. I think more and more of the routine things we do are going to be replaced by AI systems."[71] Hinton also previously argued that AGI won't make humans redundant: "[AI in the future is] going to know a lot about what you’re probably going to want to do... But it’s not going to replace you."[72]

In 2023, however, Hinton became "worried that AI technologies will in time upend the job market" and take away more than just "drudge work."[32]

Politics

Hinton moved from the U.S. to Canada in part due to disillusionment with Ronald Reagan-era politics and disapproval of military funding of artificial intelligence.[35]

Personal life

Hinton is the great-great-grandson of the mathematician and educator Mary Everest Boole and her husband, the logician George Boole,[73] whose work eventually became one of the foundations of modern computer science. Another great-great-grandfather of his was the surgeon and author James Hinton,[74] who was the father of the mathematician Charles Howard Hinton. Hinton's father was the entomologist Howard Hinton.[11][75] His middle name comes from another relative, George Everest, the Surveyor General of India after whom the mountain is named.[35] He is the nephew of the economist Colin Clark.[76] His second wife died of ovarian cancer in 1994.[77]

References

  1. ^ a b Geoffrey Hinton publications indexed by Google Scholar Edit this at Wikidata
  2. ^ a b c Geoffrey Hinton at the Mathematics Genealogy Project
  3. ^ Geoffrey E. Hinton's Academic Genealogy
  4. ^ Gregory, R. L.; Murrell, J. N. (2006). "Hugh Christopher Longuet-Higgins. 11 April 1923 -- 27 March 2004: Elected FRS 1958". Biographical Memoirs of Fellows of the Royal Society. 52: 149–166. doi:10.1098/rsbm.2006.0012.
  5. ^ a b Zemel, Richard Stanley (1994). A minimum description length framework for unsupervised learning (PhD thesis). University of Toronto. OCLC 222081343. ProQuest 304161918.
  6. ^ a b Frey, Brendan John (1998). Bayesian networks for pattern classification, data compression, and channel coding (PhD thesis). University of Toronto. OCLC 46557340. ProQuest 304396112.
  7. ^ a b Neal, Radford (1995). Bayesian learning for neural networks (PhD thesis). University of Toronto. OCLC 46499792. ProQuest 304260778.
  8. ^ a b Whye Teh, Yee (2003). Bethe free energy and contrastive divergence approximations for undirected graphical models. utoronto.ca (PhD thesis). University of Toronto. hdl:1807/122253. OCLC 56683361. ProQuest 305242430.
  9. ^ a b Salakhutdinov, Ruslan (2009). Learning deep generative models (PhD thesis). University of Toronto. ISBN 9780494610800. OCLC 785764071. ProQuest 577365583.
  10. ^ a b Sutskever, Ilya (2013). Training Recurrent Neural Networks. utoronto.ca (PhD thesis). University of Toronto. hdl:1807/36012. OCLC 889910425. ProQuest 1501655550.
  11. ^ a b c d e Anon (2015) Hinton. "Hinton, Prof. Geoffrey Everest". Who's Who (online Oxford University Press ed.). A & C Black. {{cite encyclopedia}}: Unknown parameter |othernames= ignored (help) (Subscription or UK public library membership required.) doi:10.1093/ww/9780199540884.013.20261
  12. ^ a b Anon (1998). "Professor Geoffrey Hinton FRS". London: Royal Society. Archived from the original on 3 November 2015. One or more of the preceding sentences incorporates text from the royalsociety.org website where:

    "All text published under the heading 'Biography' on Fellow profile pages is available under Creative Commons Attribution 4.0 International License." --"Royal Society Terms, conditions and policies". Archived from the original on 11 November 2016. Retrieved 9 March 2016.{{cite web}}: CS1 maint: bot: original URL status unknown (link)
  13. ^ a b "Deep learning pioneer Geoffrey Hinton quits Google". MIT Technology Review. Retrieved 1 May 2023.
  14. ^ "Geoffrey Hinton on Twitter". Twitter. 1 May 2023. Retrieved 2 May 2023.
  15. ^ Hernandez, Daniela (7 May 2013). "The Man Behind the Google Brain: Andrew Ng and the Quest for the New AI". Wired. Retrieved 10 May 2013.
  16. ^ "Geoffrey E. Hinton – Google AI". Google AI.
  17. ^ a b Rumelhart, David E.; Hinton, Geoffrey E.; Williams, Ronald J. (9 October 1986). "Learning representations by back-propagating errors". Nature. 323 (6088): 533–536. Bibcode:1986Natur.323..533R. doi:10.1038/323533a0. ISSN 1476-4687. S2CID 205001834.
  18. ^ a b c Schmidhuber, Jürgen (1 January 2015). "Deep learning in neural networks: An overview". Neural Networks. 61: 85–117. arXiv:1404.7828. doi:10.1016/j.neunet.2014.09.003. PMID 25462637. S2CID 11715509.
  19. ^ "Geoffrey Hinton was briefly a Google intern in 2012 because of bureaucracy – TechCrunch". techcrunch.com. 14 September 2017. Retrieved 28 March 2018.
  20. ^ Somers, James. "Progress in AI seems like it's accelerating, but here's why it could be plateauing". MIT Technology Review. Retrieved 28 March 2018.
  21. ^ "How U of T's 'godfather' of deep learning is reimagining AI". University of Toronto News. Retrieved 28 March 2018.
  22. ^ "'Godfather' of deep learning is reimagining AI". Retrieved 28 March 2018.
  23. ^ "Geoffrey Hinton, the 'godfather' of deep learning, on AlphaGo". Maclean's. 18 March 2016. Retrieved 28 March 2018.
  24. ^ Gershgorn, Dave (18 June 2018). "The inside story of how AI got good enough to dominate Silicon Valley". Quartz. Retrieved 5 October 2018.
  25. ^ Krizhevsky, Alex; Sutskever, Ilya; Hinton, Geoffrey E. (3 December 2012). "ImageNet classification with deep convolutional neural networks". Nips'12. Curran Associates Inc.: 1097–1105. {{cite journal}}: Cite journal requires |journal= (help)
  26. ^ "How a Toronto professor's research revolutionized artificial intelligence | Toronto Star". Toronto Star. Retrieved 13 March 2018.
  27. ^ 27 Mar, Emily Chung · CBC News · Posted; March 27, 2019 6:00 am ET | Last Updated. "Canadian researchers who taught AI to learn like humans win $1M award | CBC News". Canadian Broadcasting Corporation. Retrieved 27 March 2019.{{cite web}}: CS1 maint: numeric names: authors list (link)
  28. ^ Ranosa, Ted (29 March 2019). "Godfathers Of AI Win This Year's Turing Award And $1 Million". Tech Times. Retrieved 5 November 2020.
  29. ^ Shead, Sam. "The 3 'Godfathers' Of AI Have Won The Prestigious $1M Turing Prize". Forbes. Retrieved 5 November 2020.
  30. ^ Ray, Tiernan. "Nvidia's GTC will feature deep learning cabal of LeCun, Hinton, and Bengio". ZDNet. Retrieved 7 April 2021.
  31. ^ "50 Years at CMU: The Inaugural Raj Reddy Artificial Intelligence Lecture". cs.cmu.edu. Retrieved 2 March 2022.
  32. ^ a b c d e f g h Metz, Cade (1 May 2023). "'The Godfather of A.I.' Leaves Google and Warns of Danger Ahead". The New York Times. ISSN 0362-4331. Retrieved 1 May 2023.
  33. ^ a b c d ""Godfather of artificial intelligence" talks impact and potential of new AI". CBS News. CBS. 25 March 2023. Retrieved 28 March 2023.
  34. ^ Hinton, Geoffrey Everest (1977). Relaxation and its role in vision. ed.ac.uk (PhD thesis). University of Edinburgh. hdl:1842/8121. OCLC 18656113. EThOS uk.bl.ethos.482889. Free access icon
  35. ^ a b c Smith, Craig S. (23 June 2017). "The Man Who Helped Turn Toronto into a High-Tech Hotbed". The New York Times. Retrieved 27 June 2017.
  36. ^ https://www.cs.toronto.edu/~hinton/fullcv.pdf [bare URL PDF]
  37. ^ "Archived copy". Archived from the original on 31 December 2016. Retrieved 30 December 2016.{{cite web}}: CS1 maint: archived copy as title (link)
  38. ^ "U of T neural networks start-up acquired by Google" (Press release). Toronto, ON. 12 March 2013. Retrieved 13 March 2013.
  39. ^ Geoffrey Hinton publications indexed by the Scopus bibliographic database. (subscription required)
  40. ^ Hinton, Geoffrey (2022). "The Forward-Forward Algorithm: Some Preliminary Investigations". arXiv:2212.13345 [cs.LG].
  41. ^ Ford, Martin (2018). Architects of Intelligence: The truth about AI from the people building it. Packt Publishing. ISBN 978-1-78913-151-2.
  42. ^ Ackley, David H; Hinton Geoffrey E; Sejnowski, Terrence J (1985), "A learning algorithm for Boltzmann machines", Cognitive science, Elsevier, 9 (1): 147–169
  43. ^ Hinton, Geoffrey E. "Geoffrey E. Hinton's Publications in Reverse Chronological Order".
  44. ^ "Stories by Geoffrey E. Hinton in Scientific American". Scientific American.
  45. ^ Sabour, Sara; Frosst, Nicholas; Hinton, Geoffrey. October 2017. "Dynamic Routing Between Capsules"
  46. ^ "Matrix capsules with EM routing" 3 November 2017. OpenReview.net
  47. ^ Geib, Claudia. 2 November 2017. "We’ve Finally Created an AI Network That’s Been Decades in the Making" Futurism.com
  48. ^ a b c d "Geoffrey Hinton's postdocs". Geoffrey Hinton.
  49. ^ "Yann LeCun's Research and Contributions". yann.lecun.com. Retrieved 13 March 2018.
  50. ^ "Current and Previous Recipients". David E. Rumelhart Prize. Archived from the original on 2 March 2017.
  51. ^ Anon (1998). "Certificate of election EC/1998/21: Geoffrey Everest Hinton". London: Royal Society. Archived from the original on 5 May 2017.
  52. ^ "Distinguished Edinburgh graduate receives ACM A.M. Turing Award". The University of Edinburgh. Retrieved 9 April 2019.
  53. ^ "IJCAI Awards | IJCAI". www.ijcai.org. Retrieved 5 August 2021.
  54. ^ "Artificial intelligence scientist gets M prize". CBC News. 14 February 2011.
  55. ^ "Geoffrey Hinton, keystone researcher in artificial intelligence, visits the Université de Sherbrooke". Université de Sherbrooke. 19 February 2014.
  56. ^ "National Academy of Engineering Elects 80 Members and 22 Foreign Members". NAE. 8 February 2016.
  57. ^ "2016 IEEE Medals and Recognitions Recipients and Citations" (PDF). IEEE. Retrieved 7 July 2016.
  58. ^ "The BBVA Foundation bestows its award on the architect of the first machines capable of learning the way people do". BBVA Foundation. 17 January 2017.
  59. ^ "Vector Institutes Chief Scientific Advisor Dr.Geoffrey Hinton Receives ACM A.M. Turing Award Alongside Dr.Yoshua Bengio and Dr.Yann Lecun". NAE. 27 March 2019.
  60. ^ "Three Pioneers in Artificial Intelligence Win Turing Award". The New York Times. 27 March 2019. Retrieved 27 March 2019.
  61. ^ "Fathers of the Deep Learning Revolution Receive ACM A.M. Turing Award - Bengio, Hinton and LeCun Ushered in Major Breakthroughs in Artificial Intelligence". Association for Computing Machinery. 27 March 2019. Retrieved 27 March 2019.
  62. ^ "Governor General Announces 103 New Appointments to the Order of Canada, December 2018". 21 December 2018. Archived from the original on 19 November 2019. Retrieved 7 June 2020.
  63. ^ Princess of Asturias Awards 2022
  64. ^ "In the NYT today, Cade Metz implies that I left Google so that I could criticize Google. Actually, I left so that I could talk about the dangers of AI without considering how this impacts Google. Google has acted very responsibly". Twitter. Retrieved 1 May 2023.
  65. ^ "AI 'godfather' Geoffrey Hinton warns of dangers as he quits Google". BBC.
  66. ^ Full interview: "Godfather of artificial intelligence" talks impact and potential of AI, retrieved 1 May 2023; video at 31:45
  67. ^ Full interview: "Godfather of artificial intelligence" talks impact and potential of AI, retrieved 1 May 2023; video at 32:00
  68. ^ "AI 'godfather' Geoffrey Hinton warns of dangers as he quits Google". BBC News. 2 May 2023. Retrieved 2 May 2023.
  69. ^ Full interview: "Godfather of artificial intelligence" talks impact and potential of AI, retrieved 1 May 2023; video at 36:00
  70. ^ "Call for an International Ban on the Weaponization of Artificial Intelligence". CDTS / CLTS uOttawa. Retrieved 1 May 2023.
  71. ^ "Geoffrey Hinton and Demis Hassabis: AGI is nowhere close to being a reality". venturebeat. 17 December 2018.
  72. ^ "Geoffrey Hinton and Demis Hassabis: AGI is nowhere close to being a reality". venturebeat. 17 December 2018.
  73. ^ "Geoffrey Hinton: The story of the British 'Godfather of AI' - who's not sat down since 2005". Sky News. Retrieved 7 April 2021.
  74. ^ The Isaac Newton of logic
  75. ^ Salt, George (1978). "Howard Everest Hinton. 24 August 1912-2 August 1977". Biographical Memoirs of Fellows of the Royal Society. 24: 150–182. doi:10.1098/rsbm.1978.0006. ISSN 0080-4606.
  76. ^ Shute, Joe (26 August 2017). "The 'Godfather of AI' on making machines clever and whether robots really will learn to kill us all?". The Daily Telegraph. Retrieved 20 December 2017.
  77. ^ Shute, Joe (26 August 2017). "The 'Godfather of AI' on making machines clever and whether robots really will learn to kill us all?". The Telegraph. Retrieved 30 January 2018.
Wikiquote has quotations related to Geoffrey Hinton.
Scholia has an author profile for Geoffrey Hinton.
Retrieved from "https://en.wikipedia.org/w/index.php?title=Geoffrey_Hinton&oldid=1152922460"