What Happened

Google's Gemma 4 31B has posted strong results on the EuroEval multilingual benchmark, placing in the top 3 across five European languages and top 5 in all eight languages tested. The model ranks 1st in Finnish, 2nd in Danish, 2nd in French, 2nd in Italian, 3rd in Dutch, 3rd in English, 3rd in Swedish, and 5th in German. These results come from the EuroEval leaderboard at euroeval.com/leaderboards, which specifically targets European language performance rather than English-centric benchmarks like MMLU or HellaSwag.

What makes these numbers notable is the model's parameter count. At 31 billion parameters, Gemma 4 31B is competing against and beating models that are significantly larger in several language categories. The benchmark community on r/LocalLLaMA flagged this as a meaningful signal for users who need capable multilingual models that can still run on consumer or prosumer hardware — a 31B model fits in 24GB VRAM at 4-bit quantization, or across two consumer GPUs.

EuroEval is a relatively specialized evaluation suite focused on Nordic and broader European languages, making it more relevant than general English benchmarks for teams building products targeting European markets. The source post notes curiosity about whether real-world performance matches the benchmark scores, which is a fair caveat given benchmark-to-production gaps common in LLM evaluations.

Technical Deep Dive

Gemma 4 is Google DeepMind's fourth generation of the Gemma open-weight model family. The 31B variant uses a decoder-only transformer architecture with improvements in tokenizer coverage for non-English scripts and vocabulary expansion compared to Gemma 2. Google has not published the full technical report at time of writing, but the multilingual gains likely stem from a higher proportion of European-language data in the pretraining corpus and potentially improved tokenization efficiency for morphologically complex languages like Finnish.

Finnish ranking first is particularly telling. Finnish is an agglutinative language with complex morphology — words are built from many suffixes — which causes tokenizers trained primarily on English to fragment Finnish text into many subword tokens, reducing effective context and increasing inference cost. A model that ranks 1st in Finnish likely has a tokenizer with better Finnish vocabulary coverage, meaning fewer tokens per sentence and more efficient use of the context window.

EuroEval evaluates models across tasks including reading comprehension, named entity recognition, sentiment analysis, and linguistic acceptability, all in the target language. This is distinct from translation benchmarks — the model must reason in the target language, not translate to English first.

For comparison, models like Mistral 7B and Llama 3.1 8B score noticeably lower on Nordic languages on EuroEval, while larger models like Llama 3.1 70B or Qwen 2.5 72B tend to dominate the upper ranks. Gemma 4 31B sitting above many 70B+ models in Danish and French suggests a favorable efficiency-to-performance ratio for European deployments.

Running Gemma 4 31B locally via Ollama:

ollama pull gemma4:31b ollama run gemma4:31b

Or via llama.cpp with a GGUF quantized version from Hugging Face, targeting Q4_K_M for the best quality-size tradeoff at approximately 19GB model size.

Who Should Care

This benchmark result is directly relevant to three groups. First, developers building customer-facing applications in France, Italy, Denmark, Sweden, or Finland — chatbots, document summarization, search assistants — who need a model that understands and generates fluent text in those languages without routing to a proprietary API. Second, teams with data privacy constraints (GDPR compliance, on-premises requirements) who cannot send European customer data to US-based cloud APIs and need a self-hosted model that performs well. Third, researchers and fine-tuners working on low-resource European language tasks who want a strong multilingual base model to fine-tune from.

The 31B size is practical for inference on a single A100 80GB, two A6000 48GB GPUs, or quantized on a single RTX 4090 24GB. This makes it deployable without expensive multi-GPU server clusters, which matters for smaller European startups and academic institutions.

What To Do This Week

1. Check the EuroEval leaderboard directly at euroeval.com/leaderboards to compare Gemma 4 31B against your target language and task type.

2. Pull the model via Ollama or download the GGUF from Hugging Face (search bartowski/gemma-4-31b-GGUF for community quantizations).

3. Run a quick qualitative test in your target language with 10-20 representative prompts from your actual use case — benchmark ranks don't always translate directly to production tasks.

4. If you're already running a larger model (70B+) for European language tasks, benchmark Gemma 4 31B against it on your internal eval set. The latency and cost difference at 31B vs 70B is roughly 2x, and if quality is comparable, the switch is straightforward.