Self-Attention

1 article tagged with this topic

Transformer: 7 Years, 120K Citations—Key to the LLM Race

Google's 2017 Transformer is the LLM bedrock, replacing RNNs with parallel attention. Grasping it reveals who takes shortcuts in the LLM race.