Tensor Core Matrix Multiplication on the NVIDIA T4: A WMMA FP16 GEMM with Double Buffering, Compared Against cuBLAS
Year: 2026
Type: article
Abstract: Matrix multiplication (GEMM) is the main cost in deep-learning workloads, and Tensor Cores are the hardware NVIDIA GPUs provide to speed it up. This report describes a from-scratch CUDA GEMM that uses the WMMA API with FP16 inputs and FP32 accumulation, in two forms: a baseline with one shared-memory buffer and a double-buffered version that overlaps loading the next tile with computing the current one. Both were verified against cuBLAS and benchmarked on a Tesla T4 (Turing, sm_75) in Google Colab at three matrix sizes. The kernels reach about 63-67% of cuBLAS throughput. On this GPU, double buffering gave no measurable gain over the single-buffer baseline. The report explains the design, states that the T4 has no hardware asynchronous copy (cp.async), gives a likely reason for the flat result, and lists what would be tried next. Source code: https://github.com/19YearsOlds/61.-Tensor-Core-GEMM-WMMA-FP16-on-NVIDIA-T4
Author: EJAZ, SYED TARTEEL
Language: English
Subfield: Hardware and Architecture
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7235877320
