Natural language processing – ETO Research Almanac

ETO Research Almanac

Between 2016 and 2021, researchers published roughly

200,000

English-language articles on natural language processing

The global natural language processing literature grew

37%

between 2016 and 2021

Articles on natural language processing made up

13%

of all articles on artificial intelligence

Authors from private companies contributed to

of natural language processing articles

What counts as "natural language processing" research?•How precise are these numbers?•What are the other caveats?

Data last updated May 17, 2023

Vital signs

199,361

natural language processing articles were released between 2016 and 2021 (estd.)

37%

growth in research between 2016 and 2021

13%

of all artificial intelligence articles focused on natural language processing

Citation impact

citations for the average natural language processing article

citations for the average artificial intelligence article

Countries

20%

of natural language processing articles have American authors

17%

have Chinese authors

23%

have European authors

Country	Articles (2016-2021)	Growth (2016-2021)	Global share	Citations per article
United States	39,076	58%	20%	24.4
China	34,440	100%	17%	13.1
United Kingdom	12,573	25%	6%	19.3

Top-cited articles

Attention is All you Need
2017: arXiv: Computation and Language. 34275 citations
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
2018: arXiv: Computation and Language. 26397 citations
Deep contextualized word representations
2018: arXiv: Computation and Language. 9234 citations
Enriching Word Vectors with Subword Information
2016: arXiv: Computation and Language. 6790 citations
RoBERTa: A Robustly Optimized BERT Pretraining Approach
2019: arXiv: Computation and Language. 5087 citations
Neural Machine Translation of Rare Words with Subword Units
2016: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 4971 citations
Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
2016: arXiv: Computation and Language. 3872 citations
SQuAD: 100,000+ Questions for Machine Comprehension of Text
2016: arXiv: Computation and Language. 3834 citations
Hierarchical Attention Networks for Document Classification
2016: Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 3649 citations
Neural Architectures for Named Entity Recognition
2016: arXiv: Computation and Language. 3589 citations

Research organizations

Top producers of natural language processing research (by article count)

#1 Microsoft (United States)
#2 Chinese Academy of Sciences (China)
#3 Carnegie Mellon University (United States)

Organization	Articles (2016-2021)	Citations per article	Country
Microsoft	1,790	54.8	United States
Chinese Academy of Sciences	1,729	26	China
Carnegie Mellon University	1,559	96	United States

Patents and industry

of natural language processing articles are cited in patents

of artificial intelligence articles are cited in patents

of natural language processing articles had authors from private companies (vs 4% of artificial intelligence articles in general)