Per-Layer Embeddings

Per-Layer Embeddings: How Gemma 4's Small Models Work

Gemma 4's E2B and E4B models use per-layer embeddings, not MoE, enabling new inference performance tradeoffs.