Work

The Free Transformer

Year: 2025
Type: preprint
Abstract: We propose an extension of the decoder Transformer that conditions its generative process on random latent variables which are learned without supervision thanks to a variational procedure. Experimental evaluations show that allowing such a conditioning translates into substantial improvements on downstream tasks.
Open Access status: green
OpenAlex ID: W4415964986
Cites:
Cited by:
Related to: