What Happened

A developer testing local language models for grammar-checking tasks found that multiple models — including Google's gemma-4-E4B-it ( quantized at Q5_K_S), OpenAI's gpt-oss-20b, and Alib aba's qwen3-next-80b-a3b-instruct — incorrectly flagged words in a grammat ically correct sentence as spelling errors, according to a post on r/LocalLLaMA.

The test prompt asked each model to grammar-check a single , error-free sentence: "Although the Western Roman Empire collapsed in 476 CE, its sociopolitical and legal legacy continues to exert a profound influence on the institutional frameworks of the contemporary world." All three models returned false positives, claiming words such as "Although" and "contemporary" required spelling corrections — while leaving the words themselves unchanged in the output.

Why It Matters

Grammar checking is one of the most common production use cases developers reach for when integrating LLMs into writing tools , CMS pipelines, and document editors. A model that hallucinates errors in clean text is not merely unh elpful — it actively degrades output quality and erodes user trust in downstream applications .

The failure pattern here is particularly problematic: the models confidently ass erted corrections were made while producing output identical to the input . This class of hallucination — fabricated task completion — is harder to catch than a factually wrong answer because the final text looks superficially correct.

  • Tooling risk: Developers using these models as grammar linting backends in CI/CD pipelines or editorial tools could ship false error reports silently.
  • Model size correlation: The fact that this reproduces across a 4B parameter model and an 80B parameter model suggests parameter count alone does not resolve this failure mode.
  • Quantization as variable: The Gemma instance tested was a Q5_K_S GGUF quant ization. Whether the failure persists in the full-precision model is not established by this report.
The Technical Detail

The failure mode appears to be a form of task-completion hallucination: models trained to produce structured correction outputs may generate the expected response scaffold — correction labels, bolded terms, a "Corrections Made" section — even when no corrections are warranted. The reinfor cement signal during instruction fine-tuning likely rewards producing a formatted correction list, inadvertently incentivizing the model to manufacture corrections on clean input.

This is distinct from a simple factual hallucination. The models are not confusing "contemporary" with another word — they are producing a false-positive detection event, then labeling the word-to-itself substitution as a valid spelling fix. The output structure mimics a correct grammar -check response exactly, making automated validation against expected output format unreliable as a quality gate.

Prompt : grammar check: [correct sentence] Model output: " The sentence has two spelling errors." Correction listed: 'contemporary ' → 'contemporary'

Affected models as reported by the original poster:

  • gemma-4-E4B-it- Q5_K_S.gguf (Google, 4B parameters, quant ized)
  • openai/gpt-oss-20b (OpenAI, 20B parameters)
  • qwen3-next-80b-a3b-inst ruct (Alibaba, 80B active parameters, MoE architecture)

No benchmark data, reproduction rate, or systematic prompt variation results are provided in the source report . This is a single-prompt observation from one developer, not a controlled evaluation.

What To Watch

If you are shipping grammar-checking features on top of local or API-hosted models in the next 30 days, consider the following mitigations based on this failure pattern:

  • Add a null-hypothesis prompt path: Before calling the grammar model, run a secondary check asking whether any errors exist at all. If the model returns " no errors," skip the correction pass.
  • Diff-based validation : If the model's corrected output is identical to the input string, discard the result as a false positive regardless of the explanation text.
  • Instruction framing: Test prompt variants that explicitly include the instruction "If the text contains no errors, respond with only: No corrections needed." Structured output constraints via JSON schema may also reduce this failure mode.
  • Community follow -up: The r/LocalLLaMA thread may surface reproduction data across additional models and quantization levels. Monitor for whether full-precision Gemma 4B and non-quantized Qwen3-80B exhibit the same behavior.