Work

Feed-Forward Networks with Attention Can Solve Some Long-Term Memory Problems

Year: 2015
Type: preprint
Abstract: We propose a simplified model of attention which is applicable to feed-forward neural networks and demonstrate that the resulting model can solve the synthetic "addition" and "multiplication" long-term memory problems for sequence lengths which are both longer and more widely varying than the best published results for these tasks.
Institution: Columbia University
Language: English
Open Access status: green
OpenAlex ID: W2201092681
Cites: 32
Cited by: 428
Related to: 10