Gemma

Gemma is Google DeepMind's family of open-weight language models, first released in February 2024 and now on its fourth major generation. Unlike the closed Gemini API, Gemma weights are fully downloadable and self-hostable: no subscription, no usage cap, and since Gemma 4 (released April 2, 2026), no restrictive custom license. The name comes from "gemstone," the idea being small, precise, and valuable. The family sits between hobbyist experiments and production-grade local inference, serving developers, researchers, and enterprises that want Gemini-class intelligence without cloud dependency.

Gemma 4 ships in four sizes: Effective 2B (E2B, 2.3B active parameters), Effective 4B (E4B, 4.5B), a 26B Mixture-of-Experts variant activating 4B parameters per forward pass, and a 31B dense flagship. All models handle image and video input natively; the E2B and E4B additionally support audio input via a USM conformer encoder, making them the only sub-5B open models with native audio understanding. Context windows reach 256K tokens on the larger models. The family is multilingual by design, trained on 140+ languages, and built on the same research foundation as Gemini 3. Apache 2.0 licensing means no MAU restrictions, no attribution requirements, and no barriers to commercial redistribution or fine-tuning for competing products.

Gemma at a glance, April 2026

The current Gemma 4 lineup covers the full deployment spectrum, from on-device mobile inference to single-GPU workstations to multi-GPU server racks.

• Gemma 4 E2B: 2.3B effective parameters. Context: 128K. Multimodal: image, video, audio. Targets mobile and edge via MediaPipe/LiteRT. Achieves ~60 tok/s on an RTX 3070.

• Gemma 4 E4B: 4.5B effective parameters. Context: 128K. Same multimodal stack as E2B. The recommended "best small model" for local desktop inference.

• Gemma 4 26B-A4B: Mixture-of-Experts, 26B total, 4B active per token. Context: 256K. Runs at ~11 tok/s on RTX 4090 Q4. Reaches 1441 on LM Arena. 97% of 31B dense quality at roughly 8x less compute per inference step.

• Gemma 4 31B Dense: The flagship. Context: 256K. Benchmarks: MMLU Pro 85.2%, AIME 2026 89.2%, GPQA Diamond 84.3%, LiveCodeBench 80.0%. Ranked #3 globally on the open-model LM Arena text leaderboard as of April 2026.

The architecture introduces Per-Layer Embeddings (PLE): each token receives dedicated per-layer vectors combining token identity with context-aware components rather than a single upfront embedding, enabling layer-specific specialization. A shared KV cache (last N layers reuse K/V tensors from earlier layers) reduces memory pressure during long-context inference. Attention alternates between local sliding-window (512-1024 tokens) and global full-context layers, cutting compute on long inputs without sacrificing coherence.

Previous generations remain available. Gemma 3 (March 12, 2025) in 1B, 4B, 12B, and 27B sizes is still widely used for fine-tuning, and Gemma 2 (2B, 9B, 27B) remains supported on Hugging Face. All generations are available through Ollama, llama.cpp, MLX (Apple Silicon), and Hugging Face Transformers.

What Gemma is actually good at

Gemma 4's strongest areas, backed by benchmarks and user testing, are math reasoning, code generation, and multimodal understanding. On AIME 2026, the 31B scores 89.2%, higher than either Llama 4 Scout or Qwen 3.5 in the same parameter class. On Codeforces, it achieves ELO 2150, leading the 27-31B tier. The 26B MoE produces code that users describe as qualitatively indistinguishable from paid API models at a fraction of the operating cost.

"The kind of good that makes you re-read the request to check whether you accidentally routed it through Claude Sonnet somehow." - PIXIPACE, Medium, April 2026

The multimodal story is real and differentiated. All Gemma 4 models handle variable-resolution images with configurable token budgets (70 to 1120 vision tokens), native bounding box output for object detection, HTML reconstruction from screenshots, and chart analysis. The audio capability on E2B and E4B is unique in the sub-5B class. For teams building on-device pipelines that process spoken input alongside images, there is currently no competing open-weight model at that size with the same coverage.

"I'm impressed, extremely intelligent models, frontier-level intelligence without being resource hog like the 70+billion models.. This equals the playing field and lets everyone who has a halfway semi decent system have access to AI." - DorkMckork1, Hugging Face, April 2026

The Apache 2.0 license change that came with Gemma 4 matters practically: legal teams at large companies have Apache 2.0 pre-approved. The prior Gemma Terms of Use, while reasonable in spirit, had vague carve-outs that made procurement nervous. That friction is gone.

Where Gemma breaks: the failure modes users keep hitting

The most persistent complaint is KV cache memory pressure at longer contexts. Users running the 26B or 31B at 40K+ tokens on 24GB VRAM report out-of-memory crashes and quality degradation as the cache grows. The shared KV cache architecture is efficient in theory but unforgiving when a large quantized model fills available memory. A workaround (KV cache quantization to Q4) helps but adds configuration overhead.

The MoE routing overhead is a practical surprise. Despite activating only 4B parameters per forward pass, the 26B-A4B MoE runs at approximately 11 tokens/sec on an RTX 4090 Q4, slower than Qwen 3.5's equivalent at around 35 tok/s. Users expecting "4B speed" from a 26B MoE are often disappointed. For latency-sensitive applications, the 31B dense or a smaller model is a better choice.

Inference ecosystem fragility at launch is a recurring Gemma pattern. Gemma 4's release on April 2, 2026 was accompanied by: gibberish output after ~230 tokens with -nkvo enabled in llama.cpp (GitHub issue #21726), broken tool calling in Ollama v0.20.0, Flash Attention hangs on Apple Silicon prompts beyond 500 tokens, and crashes with "unknown model architecture: 'gemma4'" on unpatched containers. The model had to be re-uploaded to Hugging Face four times as fixes landed. The community settled on a clear message: wait at least a week before judging a new Gemma release's quality through an unstable inference stack.

Vision OCR on dense text is inconsistent. A Hugging Face discussion thread on the Gemma 4 31B-it model flagged that the model "can't recognize the entire text on image" reliably. Chart analysis is stronger than document OCR. Basic factual recall precision was documented as weak in Gemma 3 (SimpleQA score of 10.0); Gemma 4 improved substantially but the family does not excel at rote lookup-style queries without retrieval-augmented generation.

One critical voice on Hugging Face from launch week captured a real frustration:

"Somewhat disappointing release, in my opinion.. I just wish larger tech companies would take a page out of OpenAI's book and release actually competitive OSS instead of putting out generic models.." - urroxyz, Hugging Face, April 2026

That critique overstates the case. The benchmarks are not generic. But it reflects a segment of the local AI community that finds Gemma less differentiated than they'd like for specific agentic and tool-calling workflows.

Gemma vs. Llama 4 vs. Qwen 3.5

Llama 4 (Meta) uses a MoE-only architecture at a different scale: Scout has 109B total parameters with 17B active per token; Maverick has 400B total with 17B active. The Scout's 10M-token context window is its headline differentiator (Gemma 4 tops at 256K). However, Llama 4's custom community license imposes a 700M MAU cap, requires "Built with Llama" branding, and prohibits using Llama outputs to train competing models. For enterprise teams shipping commercial products, this license requires dedicated legal review. Gemma 4's Apache 2.0 has no such conditions. On benchmarks, Gemma 4 31B leads Llama 4 Scout on GPQA Diamond (84.3% vs 74.3%), AIME 2026, and coding. Llama 4 has no native audio support; Gemma's E2B/E4B do. For raw context needs above 256K, Llama 4 Scout is the only option in the open-weight tier.

Qwen 3.5 (Alibaba) also ships under Apache 2.0 and offers the widest model spectrum (0.8B to 397B MoE), with the 35B-A3B activating just 3B parameters per token, the most aggressive sparsity ratio in the tier. Qwen 3.5 runs approximately 3x faster than Gemma 4 31B dense on equivalent hardware (35 tok/s vs 25 tok/s on RTX 4090 Q4). MMLU Pro slightly favors Qwen (86.1% vs 85.2%). Key Gemma advantages: audio input on small models, per-layer embedding architecture for better long-context coherence, tighter Gemini ecosystem integration (AI Studio, Vertex AI), and a broader range of deployment targets including MediaPipe/LiteRT for on-device. Both are genuinely strong; the practical choice often comes down to whether you need audio input or faster MoE throughput.

The March 12, 2025 Gemma 3 launch was the inflection point. The 27B outperformed closed Gemini 1.5 Pro across benchmarks, and the 4B beat Gemma 2's entire 27B model. LMSys placed Gemma 3 27B in the global top 10. That established the family's credibility and set the trajectory for Gemma 4's more aggressive positioning.

Is the paid tier worth it?

There is no paid subscription for Gemma. The weights are free under Apache 2.0. The cost question is purely about infrastructure choice. Self-hosting is free beyond hardware (the 26B MoE fits in 16GB VRAM at Q4; the 31B needs 24GB). Google AI Studio provides free rate-limited API access. OpenRouter routes the 31B at $0/M tokens on its free tier, $0.13/$0.38 per million input/output on standard. Vertex AI runs $0.15/$0.60 per million for the 26B MoE and is the right choice for teams that need managed SLAs and GCP billing consolidation. For most developers, Google AI Studio or self-hosting covers everything until production scale demands managed infrastructure.

Best use cases (and when to skip it)

Best fits:

• Local inference on consumer hardware without cloud API costs. The 4B or 26B MoE covers most coding and assistant tasks.

• On-device mobile and edge applications requiring multimodal input (images, audio, video) with no cloud dependency. Only Gemma offers sub-5B models with native audio input.

• Fine-tuning for specialized domains. The Unsloth and TRL integrations support multimodal fine-tuning with CARLA-style agentic examples. A Hugging Face commenter noted it "exactly fits our kid education domain" and immediately put it into production.

• Enterprise teams that need a permissive, legally pre-approved license (Apache 2.0) and want Google-quality intelligence without Google's cloud pricing.

• Multilingual assistant applications across 140+ languages where training data quality at language-specific tokenization matters.

Skip it when:

• You need context beyond 256K tokens. Llama 4 Scout handles up to 10M.

• Raw inference speed is the top priority. Qwen 3.5 runs faster on equivalent hardware and offers smaller starting model sizes (0.8B).

• You need to deploy within the first week of a major Gemma release and cannot tolerate inference ecosystem instability. The pattern of launch-week bugs is consistent across generations.

• Your use case is primarily document OCR on dense text. Vision capabilities are stronger on charts and GUI understanding than on text-dense page recognition.

Getting started with Gemma

The fastest local path is Ollama: ollama run gemma4 pulls the default size automatically. For Apple Silicon, MLX outperforms Ollama for throughput. For quantization control, use llama.cpp with ggml-org GGUF builds from Hugging Face. Google AI Studio (ai.google.dev) gives free API access without any infrastructure. OpenRouter routes to the 31B model at zero cost on its free tier. Fine-tuning is covered by Unsloth Studio, TRL, and Vertex AI managed containers. The model's native function-calling tokens work with JSON schemas or typed Python functions directly.