Product
Help
Pricing
About
Jobs
Log in
Get started
Keyword
Add to collection
TinyBERT
Description:
TinyBERT is a compact transformer language model derived from BERT through knowledge distillation for efficient natural language processing.
Primary topic:
Topic Modeling (21%)
Related topics:
Topic Modeling (21%),
Advanced Neural Network Applications (11%),
Sentiment Analysis and Opinion Mining (10%),
Spam and Phishing Detection (8%)
OpenAlex ID:
keywords/tinybert
TinyBERT: Distilling BERT for Natural Language Understanding
2020
·
Xiaoqi Jiao
, Yichun Yin
, et al.
·
Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP)
·
3,170
PDF
TAG: Gradient Attack on Transformer-based Language Models
2021
·
Jieren Deng
, Yijue Wang
, et al.
·
Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP)
·
130
PDF
Accelerating Framework of Transformer by Hardware Design and Model Compression Co-Optimization
2021
·
Panjie Qi
, Edwin Hsing-Mean Sha
, et al.
·
International Conference on Computer Aided Design
·
82
PDF
CATBERT: Context-Aware Tiny BERT for Detecting Social Engineering Emails
2020
·
Lee, Younghoo
, Saxe, Joshua
, et al.
·
arXiv (Cornell University)
·
64
PDF
Comparative analysis of model compression techniques for achieving carbon efficient AI
2025
·
Eileen Paula
, Jayesh Soni
, et al.
·
Scientific Reports
·
54
PDF
Knowledge Distillation of LLM for Automatic Scoring of Science Education Assessments
2023
·
Latif, Ehsan
, Fang, Luyang
, et al.
·
arXiv (Cornell University)
·
50
PDF
Simplified TinyBERT: Knowledge Distillation for Document Retrieval
2021
·
Xuanang Chen
, Ben He
, et al.
·
Lecture notes in computer science
·
47
PDF
AutoTinyBERT: Automatic Hyper-parameter Optimization for Efficient Pre-trained Language Models
2021
·
Yichun Yin
, Cheng Chen
, et al.
·
42
PDF
XTC: Extreme Compression for Pre-Trained Transformers Made Simple and Efficient
2022
·
Xiaoxia Wu
, Zhewei Yao
, et al.
·
neural information processing systems
·
35
A Method for Detecting Phishing Websites Based on Tiny-Bert Stacking
2023
·
Daojing He
, Xin Lv
, et al.
·
IEEE Internet of Things Journal
·
34
Processing Natural Language on Embedded Devices: How Well Do Transformer Models Perform?
2024
·
Souvika Sarkar
, Mohammad Fakhruddin Babar
, et al.
·
ACM/SPEC International Conference on Performance Engineering (ICPE)
·
30
PDF
Unified Mandarin TTS Front-end Based on Distilled BERT Model
2020
·
Zhang, Yang
, Deng, Liqun
, et al.
·
arXiv (Cornell University)
·
30
PDF
Knowledge Distillation of LLMs for Automatic Scoring of Science Assessments
2024
·
Ehsan Latif
, Luyang Fang
, et al.
·
Communications in computer and information science
·
28
An intent classification method for questions in "Treatise on Febrile diseases" based on TinyBERT-CNN fusion model
2023
·
Helong Yu
, Chunliu Liu
, et al.
·
Computers in Biology and Medicine
·
28
PDF
Distilled BERT Model in Natural Language Processing
2024
·
Yazdan Zandiye Vakili
, Avisa Fallah
, et al.
·
International Conference on Computer and Knowledge Engineering
·
25
MicroBERT: Distilling MoE-Based Knowledge from BERT into a Lighter Model
2024
·
Dashun Zheng
, Jiaxuan Li
, et al.
·
Applied Sciences
·
24
PDF
Improving question answering performance using knowledge distillation and active learning
2023
·
Yasaman Boreshban
, Seyed Morteza Mirbostani
, et al.
·
Engineering Applications of Artificial Intelligence
·
23
PDF
CogTaskonomy: Cognitively Inspired Task Taxonomy Is Beneficial to Transfer Learning in NLP
2022
·
Yifei Luo
, Minghui Xu
, et al.
·
Proceedings of the conference - Association for Computational Linguistics. Meeting
·
22
PDF
Distilling Wisdom: A Review on Optimizing Learning From Massive Language Models
2025
·
Dingzong Zhang
, Devi Listiyani
, et al.
·
IEEE Access
·
20
Differential Impacts of Environmental, Social, and Governance News Sentiment on Corporate Financial Performance in the Global Market: An Analysis of Dynamic Industries Using Advanced Natural Language Processing Models
2024
·
Minjoong Kim
, Jinseong Kang
, et al.
·
Electronics
·
20
PDF
Leveraging BERT, DistilBERT, and TinyBERT for Rumor Detection
2025
·
Aijazahamed Qazi
, R. H. Goudar
, et al.
·
IEEE Access
·
17
A 28-nm 28.8-TOPS/W Attention-Based NN Processor With Correlative CIM Ring Architecture and Dataflow-Reshaped Digital-Assisted CIM Array
2024
·
Ruiqi Guo
, Zhiheng Yue
, et al.
·
IEEE Journal of Solid-State Circuits
·
17
OptimCLM: Optimizing clinical language models for predicting patient outcomes via knowledge distillation, pruning and quantization
2024
·
Mohammad Junayed Hasan
, Fuad Rahman
, et al.
·
International Journal of Medical Informatics
·
16
Alternative non-BERT model choices for the textual classification in low-resource languages and environments
2022
·
Syed Mustavi Maheen
, Moshiur Rahman Faisal
, et al.
·
Workshop on Deep Learning Approaches for Low-Resource NLP (DeepLo)
·
16
PDF
MoL: Mixture of Layers in Cross-Tokenizer embedding model distillation
2026
·
Hai An Vu
, Minh-Phuc Truong
, et al.
·
Knowledge-Based Systems
·
15
Show more works
View as search
Works count:
222
Citation count:
4,399
Year
Topic
Type