BLOOM (language model)
 | This article has multiple issues. Please help improve it or discuss these issues on the talk page. (Learn how and when to remove these template messages)
|
BigScience Large Open-science Open-access Multilingual Language Model (BLOOM[1]) is a transformer-based language model. It was created by over 1000 AI researchers to provide a free large language model for everyone who wants to try. Trained on around 176 billion parameters over March through July 2022, it is considered an alternative to OpenAI's GPT-3 trained on 176 billion parameters. BLOOM uses a decoder-only transformer model architecture modified from Megatron-LM GPT-2.
The BLOOM project[2] was started by a co-founder of Hugging Face. Six main groups of people were involved, including HuggingFace's BigScience team, the Microsoft DeepSpeed team, the NVIDIA Megatro-LM team, the IDRIS/GENCI team, the PyTorch team, and the volunteers in the BigScience Engineering workgroup.
References
- ^ "BigScience Large Open-science Open-access Multilingual Language Model". Retrieved 1 October 2022.
- ^ "The Technology Behind BLOOM Training". Retrieved 1 October 2022.
 | This computational linguistics-related article is a stub. You can help Wikipedia by expanding it. |