Transformers are Multi-State RNNs

Keyphrases

• Transformer 100%
• Recurrent Neural Network 100%
• Token 33%
• Hidden States 33%
• Key-value Cache 33%
• High-throughput 16%
• Cache Size 16%
• Full Model 16%
• State Size 16%
• Encoder-Decoder Transformer 16%
• Free Compression 16%
• Training-free 16%
• Transformer Neural Network 16%

Computer Science

• Recurrent Neural Network 100%
• Large Language Model 33%
• High Throughput 16%
• Computational Bottleneck 16%