Natural language processing – ETO Research Almanac

ETO Research Almanac

Between 2017 and 2022, researchers published roughly

95,000

English-language articles on natural language processing

The global natural language processing literature grew

105%

between 2017 and 2022

Articles on natural language processing made up

10%

of all articles on artificial intelligence

Authors from private companies contributed to

10%

of natural language processing articles

What counts as "natural language processing" research?•How precise are these numbers?•What are the other caveats?

Data last updated November 20, 2023

Vital signs

94,407

natural language processing articles were released between 2017 and 2022 (estd.)

105%

growth in research between 2017 and 2022

10%

of all artificial intelligence articles focused on natural language processing

Citation impact

citations for the average natural language processing article

citations for the average artificial intelligence article

Countries

21%

of natural language processing articles have American authors

21%

have Chinese authors

16%

have European authors

Country	Articles (2017-2022)	Growth (2017-2022)	Global share	Citations per article
United States	19,899	15%	21%	46.3
China	19,517	133%	21%	17
United Kingdom	6,331	-2%	7%	38.6

Top-cited articles

Attention Is All You Need
2017: arXiv: Computation and Language. 75,827 citations
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
2018: arXiv: Computation and Language. 64,121 citations
RoBERTa: A Robustly Optimized BERT Pretraining Approach
2019: arXiv: Computation and Language. 15,211 citations
Language Models are Unsupervised Multitask Learners
2019: Unknown source. 12,106 citations
Deep contextualized word representations
2018: arXiv: Computation and Language. 10,815 citations
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
2019: arXiv: Learning. 10,166 citations
XLNet: Generalized Autoregressive Pretraining for Language Understanding
2019: arXiv: Computation and Language. 6,510 citations
Improving Language Understanding by Generative Pre-Training
2018: Unknown source. 6,454 citations
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
2019: arXiv: Computation and Language. 6,021 citations
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding
2018: arXiv: Computation and Language. 4,760 citations

Research organizations

Top producers of natural language processing research (by article count)

#1 Microsoft (United States) (United States)
#2 Microsoft Research (United Kingdom) (United Kingdom)
#3 Chinese Academy of Sciences (China)

Organization	Articles (2017-2022)	Citations per article	Country
Microsoft (United States)	1,349	61.8	United States
Microsoft Research (United Kingdom)	1,325	61.6	United Kingdom
Chinese Academy of Sciences	1,307	29.3	China

Patents and industry

of natural language processing articles are cited in patents

of artificial intelligence articles are cited in patents

10%

of natural language processing articles had authors from private companies (vs 6% of artificial intelligence articles in general)