Get started
Keyword

tokenization

Description: Tokenization is the process of dividing text or other data into discrete units for computational analysis or processing.
OpenAlex ID: keywords/tokenization
Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet
2021 · Li Yuan, Yunpeng Chen, et al. · IEEE/CVF International Conference on Computer Vision (ICCV) · 2,411
PDF
A Call for Clarity in Reporting BLEU Scores
2018 · Matt Post · 2,107
PDF
Mayo clinical Text Analysis and Knowledge Extraction System (cTAKES): architecture, component evaluation and applications
2010 · Guergana K Savova, James J Masanz, et al. · Journal of the American Medical Informatics Association · 1,966
PDF
Stanza: A Python Natural Language Processing Toolkit for Many Human Languages
2020 · Peng Qi, Yuhao Zhang, et al. · Annual Meeting of the Association for Computational Linguistics: System Demonstrations (ACL Demo) · 1,475
PDF
quanteda: An R package for the quantitative analysis of textual data
2018 · Kenneth Benoit, Kohei Watanabe, et al. · The Journal of Open Source Software · 1,342
PDF
Blockchain for the future of sustainable supply chain management in Industry 4.0
2020 · Behzad Esmaeilian, Joe Sarkis, et al. · Resources Conservation and Recycling · 858
OM Forum—Distributed Ledgers and Operations: What Operations Management Researchers Should Know About Blockchain Technology
2019 · Volodymyr Babich, Gilles Hilary · Manufacturing & Service Operations Management · 788
Explorations in Automatic Thesaurus Discovery
1994 · Gregory Grefenstette · Springer eBooks · 749
Finite-State Morphology: Inflections and Derivations in a Single Framework Using Dictionaries and Rules
1997 · David Clemenceau · The MIT Press eBooks · 690
Tokenizing, POS Tagging, Lemmatizing and Parsing UD 2.0 with UDPipe
2017 · Milan Straka, Jana Straková · 567
PDF
Multimodal Fusion Transformer for Remote Sensing Image Classification
2023 · Swalpa Kumar Roy, Ankur Deria, et al. · IEEE Transactions on Geoscience and Remote Sensing · 530
PDF
“Found in Translation”: predicting outcomes of complex organic chemistry reactions using neural sequence-to-sequence models
2018 · Philippe Schwaller, Théophile Gaudin, et al. · Chemical Science · 443
PDF
Tokenization as the initial phase in NLP
1992 · Jonathan J. Webster, Chunyu Kit · International Conference on Computational Linguistics · 379
PDF
UDPipe: Trainable Pipeline for Processing CoNLL-U Files Performing Tokenization, Morphological Analysis, POS Tagging and Parsing
2016 · Milan Straka, Jan Hajič, et al. · Language Resources and Evaluation Conference (LREC) · 369
PDF
The Core Language Engine . Ed. by Hiyan Alshawi. (A Bradford book.) Cambridge, MA: The MIT Press, 1992. Pp. xviii, 322. $37.50.
1994 · Michael A. Covington · Language · 328
Extracting relations with integrated information using kernel methods
2005 · Shubin Zhao, Ralph Grishman · Annual Meeting of the Association for Computational Linguistics (ACL) · 314
PDF
VOLO: Vision Outlooker for Visual Recognition
2022 · Li Yuan, Qibin Hou, et al. · IEEE Transactions on Pattern Analysis and Machine Intelligence · 304
PDF
Fake News Detection Using Machine Learning Approaches
2021 · Z Khanam, B N Alwasel, et al. · IOP Conference Series Materials Science and Engineering · 294
PDF
Type/Token Ratios: what do they really tell us?
1987 · Brian Richards · Journal of Child Language · 292
Using AI to write scholarly publications
2023 · Mohammad Hosseini, Lisa M. Rasmussen, et al. · Accountability in Research · 284
PDF
Construction accident narrative classification: An evaluation of text mining techniques
2017 · Yang Miang Goh, C.U. Ubeynarayana · Accident Analysis & Prevention · 277
Universal Dependency Parsing from Scratch
2018 · Peng Qi, Timothy Dozat, et al. · Proceedings of the · 265
PDF
FreeLing: An Open-Source Suite of Language Analyzers
2004 · Xavier Carreras, Isaac Chao, et al. · Language Resources and Evaluation Conference (LREC) · 262
PDF
BERT applications in natural language processing: a review
2025 · Nadia Mushtaq Gardazi, Ali Daud, et al. · Artificial Intelligence Review · 256
Arabic preprocessing schemes for statistical machine translation
2006 · Nizar Habash, Fatiha Sadat · 253
PDF
Works count: 9,304
Citation count: 54,170
Year
Topic
Type