Large language models – ETO Research Almanac

ETO Research Almanac

Between 2018 and 2023, researchers published roughly

35,000

English-language articles on large language models

The global large language models literature grew

1,143%

between 2018 and 2023

Articles on large language models made up

of all articles on artificial intelligence

Authors from private companies contributed to

17%

of large language models articles

What counts as "large language models" research?•How precise are these numbers?•What are the other caveats?

Explore relevant research in the Map of Science

Data last updated January 6, 2025

Vital signs

35,730
large language models articles were released between 2018 and 2023 (estd.)
1,143%
growth in research between 2018 and 2023
2%
of all artificial intelligence articles focused on large language models

Citation impact

51
citations for the average large language models article
16
citations for the average artificial intelligence article

Countries

25%
of large language models articles have American authors
25%
have Chinese authors
12%
have European authors

Country	Articles (2018-2023)	Growth (2018-2023)	Global share	Citations per article
United States	8,969	596%	25%	109.7
China	8,944	1,193%	25%	29.8
United Kingdom	2,159	819%	6%	68.3

Top-cited articles

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
2018: arXiv. 96,902 citations
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
2020: arXiv. 35,209 citations
Language Models are Few-Shot Learners
2020: arXiv. 34,811 citations
RoBERTa: A Robustly Optimized BERT Pretraining Approach
2019: arXiv. 23,742 citations
Learning Transferable Visual Models From Natural Language Supervision
2021: arXiv. 22,081 citations
Language Models are Unsupervised Multitask Learners
2019. 18,882 citations
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
2019: arXiv. 17,846 citations
Deep contextualized word representations
2018: arXiv. 12,819 citations
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
2019: arXiv. 11,681 citations
BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension
2019: arXiv. 10,234 citations

Research organizations

Top producers of large language models research (by article count)

#1 Microsoft (United States)
#2 Google (United States)
#3 Tsinghua University (China)

Organization	Articles (2018-2023)	Citations per article	Country
Microsoft	1,055	114.6	United States
Google	952	401	United States
Tsinghua University	683	59.2	China

Patents and industry

7%
of large language models articles are cited in patents
5%
of artificial intelligence articles are cited in patents
17%
of large language models articles had authors from private companies (vs 6% of artificial intelligence articles in general)

Top industry organizations by publication count

Company	Articles (2018-2023)	Citations per article	Country
Microsoft	1,055	114.6	United States
Google	952	401	United States
Tencent	472	40.4	China

The Research Almanac runs on bibliometric data from Clarivate Web of Science and other sources. Research subject classifications are based on work supported in part by the Alfred P. Sloan Foundation under Grant No. G-2023-22358. For details on our sourcing, methodology, and limitations, see the tool documentation.