Jump to content

Open-source artificial intelligence: Difference between revisions

From Wikipedia, the free encyclopedia
Browse history interactively
← Previous editNext edit →
Content deleted Content added
VisualWikitext
Added a table of some open source base large language models. These models have not been fine tuned.
Citation bot (talk | contribs)
Bots
4,954,351 edits
Alter: title, template type, url. URLs might have been anonymized. Add: class, eprint, archive-date, archive-url, isbn, series, magazine, authors 1-13. Removed proxy/dead URL that duplicated identifier. Removed access-date with no URL. Removed parameters. Some additions/deletions were parameter name changes. | Use this bot. Report bugs. | #UCB_CommandLine
Line 1: Line 1:
'''Open-source artificial intelligence''' is the application of [[open source]] practices to the development of [[artificial intelligence]] resources.
'''Open-source artificial intelligence''' is the application of [[open source]] practices to the development of [[artificial intelligence]] resources.


Many open-source artificial intelligence products are variations of other existing tools and technology which major companies have shared as open-source software.<ref name="Heaven 2023">{{cite web |last1=Heaven |first1=Will Douglas |title=The open-source AI boom is built on Big Tech’s handouts. How long will it last? |url=https://www.technologyreview.com/2023/05/12/1072950/open-source-ai-google-openai-eleuther-meta/ |website=MIT Technology Review |language=en |date=May 12, 2023}}</ref>
Many open-source artificial intelligence products are variations of other existing tools and technology which major companies have shared as open-source software.<ref name="Heaven 2023">{{cite web |last1=Heaven |first1=Will Douglas |title=The open-source AI boom is built on Big Tech's handouts. How long will it last? |url=https://www.technologyreview.com/2023/05/12/1072950/open-source-ai-google-openai-eleuther-meta/ |website=MIT Technology Review |language=en |date=May 12, 2023}}</ref>


Companies often developed closed products in an attempt to keep a competitive advantage in the marketplace.<ref name="Solaiman 2023">{{cite web |last1=Solaiman |first1=Irene |title=Generative AI Systems Aren't Just Open or Closed Source |url=https://www.wired.com/story/generative-ai-systems-arent-just-open-or-closed-source/ |website=Wired |date=May 24, 2023}}</ref> A journalist for [[Wired (magazine)|Wired]] explored the idea that open-source AI tools have a development advantage over closed products, and could overtake them in the marketplace.<ref name="Solaiman 2023"/>
Companies often developed closed products in an attempt to keep a competitive advantage in the marketplace.<ref name="Solaiman 2023">{{cite magazine |last1=Solaiman |first1=Irene |title=Generative AI Systems Aren't Just Open or Closed Source |url=https://www.wired.com/story/generative-ai-systems-arent-just-open-or-closed-source/ |magazine=Wired |date=May 24, 2023}}</ref> A journalist for [[Wired (magazine)|Wired]] explored the idea that open-source AI tools have a development advantage over closed products, and could overtake them in the marketplace.<ref name="Solaiman 2023"/>


Popular open-source artificial intelligence project categories include [[large language models]], [[Machine translation]] tools, and [[chatbots]].<ref name="Castelvecchi 2023">{{cite journal |last1=Castelvecchi |first1=Davide |title=Open-source AI chatbots are booming — what does this mean for researchers? |journal=Nature |date=29 June 2023 |volume=618 |issue=7967 |pages=891–892 |doi=10.1038/d41586-023-01970-6}}</ref>
Popular open-source artificial intelligence project categories include [[large language models]], [[Machine translation]] tools, and [[chatbots]].<ref name="Castelvecchi 2023">{{cite journal |last1=Castelvecchi |first1=Davide |title=Open-source AI chatbots are booming — what does this mean for researchers? |journal=Nature |date=29 June 2023 |volume=618 |issue=7967 |pages=891–892 |doi=10.1038/d41586-023-01970-6}}</ref>


For [[software developers]] to produce open-source artificial intelligence resources, they must [[Trust (social science)|trust]] the various other open-source software components they use in its development.<ref name="Thummadi 2021">{{cite journal |last1=Thummadi |first1=Babu Veeresh |title=Artificial Intelligence (AI) Capabilities, Trust and Open Source Software Team Performance |journal=Responsible AI and Analytics for an Ethical and Inclusive Digitized Society |date=2021 |volume=12896 |pages=629–640 |doi=10.1007/978-3-030-85447-8_52}}</ref>
For [[software developers]] to produce open-source artificial intelligence resources, they must [[Trust (social science)|trust]] the various other open-source software components they use in its development.<ref name="Thummadi 2021">{{cite book |last1=Thummadi |first1=Babu Veeresh |title=Artificial Intelligence (AI) Capabilities, Trust and Open Source Software Team Performance |journal=Responsible AI and Analytics for an Ethical and Inclusive Digitized Society |series=Lecture Notes in Computer Science |date=2021 |volume=12896 |pages=629–640 |doi=10.1007/978-3-030-85447-8_52|isbn=978-3-030-85446-1 }}</ref>


== Large Language Models ==
== Large Language Models ==


=== LLaMA ===
=== LLaMA ===
'''LLaMA''' ('''Large Language Model Meta AI''') is a family of [[Large language model|large language models]] (LLMs), released by [[Meta AI]] starting in February 2023. <ref name=":0">{{Cite web |date=2023-09-11 |title=Introducing LLaMA: A foundational, 65-billion-parameter language model |url=https://web.archive.org/web/20230911095237/https://ai.meta.com/blog/large-language-model-llama-meta-ai/ |access-date=2023-10-03 |website=web.archive.org}}</ref>
'''LLaMA''' ('''Large Language Model Meta AI''') is a family of [[Large language model|large language models]] (LLMs), released by [[Meta AI]] starting in February 2023. <ref name=":0">{{Cite web |date=2023-09-11 |title=Introducing LLaMA: A foundational, 65-billion-parameter language model |url=https://ai.meta.com/blog/large-language-model-llama-meta-ai/ |access-date=2023-10-03 |archive-url=https://web.archive.org/web/20230911095237/https://ai.meta.com/blog/large-language-model-llama-meta-ai/ |archive-date=2023-09-11 }}</ref>
{| class="wikitable"
{| class="wikitable"
|+Open Source Large Language Foundation Models Comparison
|+Open Source Large Language Foundation Models Comparison
Line 45: Line 45:
|Apache 2.0
|Apache 2.0
|-
|-
|Pythia<ref>{{Cite web |date=2023-10-03 |title=[2304.01373] Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling |url=https://web.archive.org/web/20231003152523/https://arxiv.org/abs/2304.01373?trk=public_post_comment-text |access-date=2023-10-03 |website=web.archive.org}}</ref>
|Pythia<ref>{{Cite arXiv |date=2023-10-03 |title=[2304.01373] Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling |eprint=2304.01373 |last1=Biderman |first1=Stella |last2=Schoelkopf |first2=Hailey |last3=Anthony |first3=Quentin |last4=Bradley |first4=Herbie |last5=O'Brien |first5=Kyle |last6=Hallahan |first6=Eric |author7=Mohammad Aflah Khan |last8=Purohit |first8=Shivanshu |author9=USVSN Sai Prashanth |last10=Raff |first10=Edward |last11=Skowron |first11=Aviya |last12=Sutawika |first12=Lintang |author13=Oskar van der Wal |class=cs.CL }}</ref>
|EluetherAI
|EluetherAI
|70 million - 12 billion
|70 million - 12 billion

Revision as of 12:35, 25 October 2023

Open-source artificial intelligence is the application of open source practices to the development of artificial intelligence resources.

Many open-source artificial intelligence products are variations of other existing tools and technology which major companies have shared as open-source software.[1]

Companies often developed closed products in an attempt to keep a competitive advantage in the marketplace.[2] A journalist for Wired explored the idea that open-source AI tools have a development advantage over closed products, and could overtake them in the marketplace.[2]

Popular open-source artificial intelligence project categories include large language models, Machine translation tools, and chatbots.[3]

For software developers to produce open-source artificial intelligence resources, they must trust the various other open-source software components they use in its development.[4]

Large Language Models

LLaMA

LLaMA (Large Language Model Meta AI) is a family of large language models (LLMs), released by Meta AI starting in February 2023. [5]

Open Source Large Language Foundation Models Comparison
Model Developer Parameter Count Context Window Licensing
LLaMA[5] Meta AI 7B, 13B, 33B, 65B 2048 ——
LLaMA 2[6][7] Meta AI 7B, 13B, 70B 4k Custom Meta License
Mistral 7B[8] Mistral AI 7 billion 8k[9] Apache 2.0
GPT-J[10] EleutherAI 6 billion 2048 Apache 2.0
Pythia[11] EluetherAI 70 million - 12 billion —— Apache 2.0 (Pythia-6.9B only)[12]

References

  1. ^ Heaven, Will Douglas (May 12, 2023). "The open-source AI boom is built on Big Tech's handouts. How long will it last?". MIT Technology Review.
  2. ^ a b Solaiman, Irene (May 24, 2023). "Generative AI Systems Aren't Just Open or Closed Source". Wired.
  3. ^ Castelvecchi, Davide (29 June 2023). "Open-source AI chatbots are booming — what does this mean for researchers?". Nature. 618 (7967): 891–892. doi:10.1038/d41586-023-01970-6.
  4. ^ Thummadi, Babu Veeresh (2021). Artificial Intelligence (AI) Capabilities, Trust and Open Source Software Team Performance. Lecture Notes in Computer Science. Vol. 12896. pp. 629–640. doi:10.1007/978-3-030-85447-8_52. ISBN 978-3-030-85446-1. {{cite book}}: |journal= ignored (help)
  5. ^ a b "Introducing LLaMA: A foundational, 65-billion-parameter language model". 2023-09-11. Archived from the original on 2023-09-11. Retrieved 2023-10-03.
  6. ^ "meta-llama/Llama-2-70b-chat-hf · Hugging Face". huggingface.co. Retrieved 2023-10-03.
  7. ^ "Llama 2 - Meta AI". ai.meta.com. Retrieved 2023-10-03.
  8. ^ "mistralai/Mistral-7B-v0.1 · Hugging Face". huggingface.co. Retrieved 2023-10-03.
  9. ^ AI, Mistral (2023-09-27). "Mistral 7B". mistral.ai. Retrieved 2023-10-03.
  10. ^ "EleutherAI/gpt-j-6b · Hugging Face". huggingface.co. 2023-05-03. Retrieved 2023-10-03.
  11. ^ Biderman, Stella; Schoelkopf, Hailey; Anthony, Quentin; Bradley, Herbie; O'Brien, Kyle; Hallahan, Eric; Mohammad Aflah Khan; Purohit, Shivanshu; USVSN Sai Prashanth; Raff, Edward; Skowron, Aviya; Sutawika, Lintang; Oskar van der Wal (2023-10-03). "[2304.01373] Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling". arXiv:2304.01373 [cs.CL].
  12. ^ "EleutherAI/pythia-6.9b · Hugging Face". huggingface.co. 2023-05-03. Retrieved 2023-10-03.
Retrieved from "https://en.wikipedia.org/w/index.php?title=Open-source_artificial_intelligence&oldid=1181822885"