Product
Help
Pricing
About
Jobs
Log in
Get started
Keyword
Add to collection
SwiGLU
Description:
A gated neural-network activation function combining a swish-based gate with a linear transformation, used in machine-learning models.
Primary topic:
Topic Modeling (21%)
Related topics:
Topic Modeling (21%),
Advanced Neural Network Applications (20%),
Parallel Computing and Optimization Techniques (9%)
OpenAlex ID:
keywords/swiglu
Enhancing Skin Cancer Diagnosis Using Swin Transformer with Hybrid Shifted Window-Based Multi-head Self-attention and SwiGLU-Based MLP
2024
·
Ishak Pacal
, Melek Alaftekin
, et al.
·
Journal of Imaging Informatics in Medicine
·
127
ReLU$^2$ Wins: Discovering Efficient Activation Functions for Sparse LLMs
2024
·
Zhang, Zhengyan
, Song, Yixin
, et al.
·
arXiv (Cornell University)
·
98
PDF
Turbo Sparse: Achieving LLM SOTA Performance with Minimal Activated Parameters
2024
·
Song, Yixin
, Xie, Haotong
, et al.
·
arXiv (Cornell University)
·
57
PDF
Efficient LLM Inference using Dynamic Input Pruning and Cache-Aware Masking
2024
·
Federici, Marco
, Belli, Davide
, et al.
·
arXiv (Cornell University)
·
14
PDF
Deriving Activation Functions Using Integration
2024
·
Huang, Allen Hao
, Schlag, Imanol
·
arXiv (Cornell University)
·
11
PDF
LoLA-SpecViT: Local attention SwiGLU vision transformer with LoRA for hyperspectral imaging
2025
·
Fadi Abdeladhim Zidi
, Djamel Eddine Boukhari
, et al.
·
International Journal of Applied Earth Observation and Geoinformation
·
10
DeformNeXt-Swin: A hybrid CNN-transformer framework for breast lesion classification in ultrasound and mammography
2026
·
Ishak Pacal
, Merve Ersoy
, et al.
·
Chemometrics and Intelligent Laboratory Systems
·
9
MPSANT: A novel multi-projection self-attending neural Tucker factorization model for high-dimensional and incomplete data representation learning
2026
·
Peng Tang
, Yikai Hou.swu.edu.cn
, et al.
·
Information Fusion
·
6
AudioMAE++: Learning Better Masked Audio Representations with Swiglu FFNS
2025
·
Sarthak Yadav
, Sergios Theodoridis
, et al.
·
Machine learning for signal processing ...
·
6
PDF
TokenFLEX: Unified VLM Training for Flexible Visual Tokens Inference
2025
·
Hu, Junshan
, Mao, Jialiang
, et al.
·
arXiv (Cornell University)
·
6
PDF
Medical Document Embedding Enhancement with Heterogeneous Mixture-of-Experts
2024
·
Xiangyang Liu
, Yi Zhu
, et al.
·
IEEE International Conference on Bioinformatics and Biomedicine (BIBM)
·
6
Can Video World Models Track Unobserved World States?
2026
·
Shin, Joonghyuk
, Hong, Yicong
, et al.
·
arXiv (Cornell University)
·
5
PDF
PowLU: An Activation Function for Stable Pre-Training of LLMs
2026
·
Jiang, Peijie
, Feng, Yuqi
, et al.
·
arXiv (Cornell University)
·
5
PDF
Dissecting Outlier Dynamics in LLM NVFP4 Pretraining
2026
·
Dong, Peijie
, Fan, Ruibo
, et al.
·
arXiv (Cornell University)
·
5
PDF
Synthetic aperture radar image change detection based on multi-scale deep adaptive convolution and spatial-frequency dual-domain feature extraction
2025
·
Lu Wang
, Jiahui E․
, et al.
·
Expert Systems with Applications
·
5
Kuramoto Attention: Synchronizing Self-Attention on the Torus
2026
·
Nunley, Joshua
·
arXiv (Cornell University)
·
3
PDF
More Expressive Feedforward Layers: Part I. Token-Adaptive Mixing of Activations
2026
·
Wang, Mingze
, Wang, Jinbo
, et al.
·
arXiv (Cornell University)
·
3
PDF
A Two-Parameter Weibull Framework for Diagnosing Transformer Weight Distributions
2026
·
Ding, Tiexin
·
arXiv (Cornell University)
·
3
PDF
VectraYX-Nano: A 42M-Parameter Spanish Cybersecurity Language Model with Curriculum Learning and Native Tool Use
2026
·
Santillana, Juan S.
·
arXiv (Cornell University)
·
3
PDF
Beyond N-gram: Data-Aware X-GRAM Extraction for Efficient Embedding Parameter Scaling
2026
·
Chen, Yilong
, Xie, Yanxi
, et al.
·
arXiv (Cornell University)
·
2
PDF
GLUScope: A Tool for Analyzing GLU Neurons in Transformer Language Models
2026
·
Gerstner, Sebastian
, Veloso, Leonor
, et al.
·
arXiv (Cornell University)
·
2
PDF
Speed is Confidence
2026
·
Dillon, Joshua V.
·
arXiv (Cornell University)
·
2
PDF
Chronicals: A High-Performance Framework for LLM Fine-Tuning with 3.51x Speedup over Unsloth
2026
·
Nair, Arjun S.
·
arXiv (Cornell University)
·
2
PDF
From Black-Box to White-Box: Control-Theoretic Neural Network Interpretability
2025
·
Moon, Jihoon
·
arXiv (Cornell University)
·
2
PDF
Masked Gated Linear Unit
2025
·
Yukito Tajima
, Nakamasa Inoue
, et al.
·
neural information processing systems
·
2
PDF
Show more works
View as search
Works count:
112
Citation count:
412
Year
Topic
Type