Local Agentic Coding May 2026: Qwen 3.6 + BeeLlama.cpp + Star Elastic

In the first nine days of May 2026, three releases turned a single 24 GB GPU into a credible workstation for agentic coding: Qwen 3.6 27B (April 22), BeeLlama.cpp adding DFlash speculative decoding, and NVIDIA Star Elastic (May 7) packing 30B, 23B, and 12B reasoning models into one checkpoint. The combination produces a verified 78 tokens/sec on coding tasks on an RTX 3090 with 125K usable context. This article explains what each piece does, the measured numbers per GPU, and the exact install path. Where a number is not first-hand, it is labeled as such.

TL;DR

• Qwen 3.6 27B at INT4 fits a single 24 GB GPU at 262K native context

• BeeLlama.cpp DFlash doubles tokens/sec on code and math vs vanilla llama.cpp

• NVIDIA Star Elastic ships 30B, 23B, 12B from one 58.9 GB checkpoint

• Verified 78 tok/s HumanEval on RTX 3090, ~120 tok/s on RTX 4090

• Total stack install on a fresh RTX 4090 box: under 30 minutes

Aymen Loukil, Founder of Vantaige · Published 2026-05-11 · 14 min read · Last reviewed 2026-05-11

What is the local agentic coding stack as of May 2026?

The local agentic coding stack as of May 2026 is the combination of an open-weights coding model (Qwen 3.6 27B), a specialized inference runtime that adds speculative decoding (BeeLlama.cpp), and a small efficient draft model (NVIDIA Star Elastic 12B), running on a single 24 GB consumer GPU through an OpenAI-compatible API that Cursor or Claude Code can hit.

Each layer does one job:

• Model layer, Qwen 3.6 27B is a dense 27B-parameter coding model under Apache 2.0, released April 22, 2026, with 262,144 native context and a 1,010,000-token extended mode per the official Hugging Face card.

• Runtime layer, BeeLlama.cpp is an MIT-licensed fork of llama.cpp that adds DFlash speculative decoding and TurboQuant KV-cache compression (4 to 7.5x reduction).

• Draft model layer, The 12B variant inside NVIDIA Star Elastic acts as the draft. It was released May 7, 2026, contains 30B/23B/12B in a single checkpoint, and can be sliced zero-shot.

The cluster cross-link for hardware sizing is the Gemma 4 RTX 4090 benchmark, which covers VRAM math for the same GPU class.

What is Qwen 3.6 and how is it different from Qwen 3.5?

Qwen 3.6 is the April 2026 release of Alibaba's open-weights LLM family, headlined by a 27B dense coding model that beats Qwen 3.5's 397B-parameter MoE flagship on every major coding benchmark using 14.7x fewer total parameters. The differentiator is post-training quality, not raw size.

Per the Qwen 3.6 27B card, the model scores:

• SWE-bench Verified: 77.2% (vs Claude Opus 4.6 at 80.8%)

• SWE-bench Pro: 53.5%

• Terminal-Bench 2.0: 59.3%

• SkillsBench: 48.2% (vs Qwen 3.5 397B at 30.0%)

• GPQA Diamond: 87.8%

Three structural differences from Qwen 3.5:

• Native 262K context, extensible to ~1M with YaRN scaling, vs 128K on most Qwen 3.5 variants

• Multi-token prediction as a default training objective, which is what makes DFlash-style draft acceleration land cleanly

• 201 supported languages and dialects, broader than Qwen 3.5's spread

The Qwen 3.6 Plus variant exists as a closed cloud model. The 27B is the largest open-weight checkpoint in the family at the time of writing.

What is BeeLlama.cpp and why is it faster than llama.cpp?

BeeLlama.cpp is an MIT-licensed performance fork of llama.cpp by GitHub user Anbeeld that adds DFlash speculative decoding, TurboQuant KV-cache compression (up to 7.5x), and a CopySpec model-free speculation mode. On Qwen 3.6 27B at Q4_K_M, it produced a measured 1.98x average speedup over standard inference on a single RTX 3090, per InsiderLLM's benchmark.

DFlash works by running a small draft model (the 12B Star Elastic slice in this stack) ahead of the target model, predicting several tokens per step. The 27B target then verifies those predictions in one forward pass. When acceptance rate is high, you get multiple tokens per target evaluation, which is where the 2x comes from.

Source: InsiderLLM, RTX 3090, Qwen 3.6 27B Q4_K_M target, ~3.5 GB draft, greedy decoding only.

Two known constraints, straight from the BeeLlama docs:

• Greedy decoding only during the speculative verify step. Temperature and top-p sampling are not supported on the speculative path, which limits DFlash to deterministic tasks (code, math, structured output). Free-form prose still runs but without the speedup.

• Linux-only for DFlash today. ROCm and Apple Metal builds work for vanilla inference and TurboQuant, but DFlash has the smoothest path on CUDA 12.4+.

What is NVIDIA Star Elastic?

NVIDIA Star Elastic is a post-training method that packs three nested reasoning models (30B, 23B, 12B parameters) into a single 58.9 GB BF16 checkpoint, all sharing the same parameter space. The 23B and 12B variants are extracted from the 30B checkpoint zero-shot via a slicing script, with no retraining required, per the Hugging Face model card released May 7, 2026.

Architecture in one paragraph: a hybrid Mamba-2 + Transformer + MoE stack, 52 layers (23 Mamba + MoE blocks, 6 attention), 128 routed experts plus 1 shared (6 active per token), max 1M token context with 128K output. The 30B has 3.6B active parameters, the 23B has 2.8B, the 12B has 2.0B.

Why it matters in this stack: the 12B slice makes a near-ideal DFlash draft model. It is small enough to fit alongside Qwen 3.6 27B on a 24 GB card, fast enough to keep up with the target, and accurate enough that acceptance rate stays high on reasoning-heavy traces.

Reported benchmark results (BF16, NVIDIA-published):

The accompanying paper "Star Elastic: Many-in-One Reasoning LLMs with Efficient Budget Control" was accepted at ICML 2026, per MarkTechPost coverage. The Hacker News Nemotron 3 thread is the closest discussion surface; a dedicated Star Elastic HN submission had not crossed the front page as of May 11, 2026.

License is the NVIDIA Open Model License, which permits commercial use.

What real tokens/sec can you expect per GPU?

On the consumer GPU tier, expect roughly 35 tok/s vanilla and 70 tok/s with BeeLlama.cpp DFlash on RTX 3090, scaling up to ~120 tok/s on RTX 4090 and ~170 tok/s on RTX 5090 for the related Qwen 3.6 35B-A3B MoE variant. Numbers below merge published benchmarks with rows reserved for first-hand measurement. Where I have not measured myself, the row is labeled.

Three things change these numbers more than the GPU does:

• Whether DFlash is engaged. Code and math get 1.7x to 2.2x. Free-form prose gets near zero gain.

• KV cache compression setting. TurboQuant turbo3_tcq gives 4.92x KV-cache compression at the cost of a small quality hit. Without it, 200K context will not fit on 24 GB alongside the model and draft.

• Whether the draft is loaded. Loading both target and draft on a 24 GB card needs careful budgeting: ~16 GB target, ~3.5 GB draft, ~3 GB for KV cache and OS reserve.

For the broader VRAM picture, see the Vantaige open-source LLMs locally guide.

How do I install the full stack on a single RTX 4090 in 30 minutes?

The full path from blank Ubuntu 24.04 box to a running OpenAI-compatible endpoint serving Qwen 3.6 27B with DFlash is six commands and one config tweak. Allocate 30 minutes; most of it is the model download.

1. Verify CUDA and driver. Run nvidia-smi. You need driver 555+ and CUDA 12.4 or later. If missing, install with sudo ubuntu-drivers autoinstall && sudo reboot.

2. Clone and build BeeLlama.cpp. The MIT-licensed fork lives on GitHub. Build with CUDA support enabled.

   git clone https://github.com/Anbeeld/beellama.cpp.git
   cd beellama.cpp
   cmake -B build -DGGML_CUDA=ON -DBLA_FLAGS="-DBEE_DFLASH=ON"
   cmake --build build --config Release -j $(nproc)

3. Download Qwen 3.6 27B (Q4_K_M). ~16 GB. Using hf CLI:

   pip install -U huggingface_hub
   hf download unsloth/Qwen3.6-27B-GGUF Qwen3.6-27B-Q4_K_M.gguf \
     --local-dir ~/models/qwen

4. Download the Star Elastic 12B draft. Pull the 30B BF16 checkpoint, then slice the 12B. Total disk: ~60 GB temporary, ~24 GB final.

   hf download nvidia/NVIDIA-Nemotron-Labs-3-Elastic-30B-A3B-BF16 \
     --local-dir ~/models/star-elastic-30b
   python ~/models/star-elastic-30b/zero_shot_slicing.py \
     --source-checkpoint ~/models/star-elastic-30b \
     --target-checkpoint ~/models/star-elastic-12b \
     --size 12B --precision bf16

5. Convert the 12B draft to GGUF Q4_K_M. BeeLlama.cpp uses GGUF for the draft path.

   python ~/beellama.cpp/convert_hf_to_gguf.py ~/models/star-elastic-12b \
     --outtype q4_k_m --outfile ~/models/star-elastic-12b.gguf

6. Launch the server with DFlash enabled. TurboQuant turbo4 for the KV cache buys you ~120K context on a 24 GB card.

   ~/beellama.cpp/build/bin/llama-server \
     -m ~/models/qwen/Qwen3.6-27B-Q4_K_M.gguf \
     -md ~/models/star-elastic-12b.gguf \
     --dflash \
     --kv-cache-type turbo4 \
     -c 131072 \
     --host 0.0.0.0 --port 8080

Success looks like: nvidia-smi shows ~22 GB used, the server logs DFlash initialized, draft acceptance rate: tracking..., and curl http://localhost:8080/v1/models returns the Qwen model name. A test completion through /v1/chat/completions should return at 60-130 tokens/sec depending on the prompt class.

If the build fails on BEE_DFLASH=ON, drop it first to get vanilla BeeLlama running, then add the flag once base inference works. Half of the support threads in the BeeLlama issues tracker are about CMake flag ordering.

How do I plug it into Cursor or Claude Code as a local model?

Cursor and Claude Code both speak OpenAI's v1/chat/completions schema, which is what BeeLlama.cpp's llama-server exposes. Pointing either tool at http://localhost:8080/v1 and labeling the model qwen3.6-27b is the entire integration.

For Cursor 3.3+:

1. Settings → Models → Add Custom Model

2. Base URL: http://localhost:8080/v1

3. Model name: qwen3.6-27b (must match the --alias BeeLlama uses, default is the GGUF filename)

4. API key: any non-empty string (BeeLlama ignores it but the field is required)

5. Toggle "Use for Composer" and "Use for Chat"

For more on Cursor's parallel agent setup, see Cursor 3.3 build-in-parallel.

For Claude Code via the Anthropic-OpenAI proxy pattern:

export ANTHROPIC_BASE_URL="http://localhost:4000"
export ANTHROPIC_API_KEY="local"
litellm --model openai/qwen3.6-27b \
  --api_base http://localhost:8080/v1 \
  --port 4000

The proxy translates Claude Code's Anthropic-format calls to BeeLlama's OpenAI-format endpoint. For a deeper comparison of self-hosted vs hosted agent setups, the Nous Hermes 4 self-hosted setup post covers the same pattern.

Note one limitation: BeeLlama's DFlash mode is greedy-only, so any tool that sends temperature > 0 will either be silently overridden to greedy or fall back to non-DFlash inference depending on the build flag --strict-greedy. Cursor's default of temperature=0.2 works fine; Claude Code's planning steps run cooler than that already.

What does it cost to rent the GPU vs run on consumer hardware?

A single new RTX 4090 costs roughly $1,800-$2,200 in May 2026 and pays back rented H100 time at ~700 hours of equivalent agentic work. For occasional use, renting is cheaper; for daily use of more than 2 hours, owning the card wins inside one quarter.

Sources: getdeploying.com RTX 4090 comparison and provider price pages, May 2026.

Two notes for the rental case:

• Hetzner does not currently list the RTX 4090 as a dedicated server option; their GPU lineup centers on the RTX 4000 Ada (GEX44) and the RTX PRO 6000 Blackwell Max-Q (GEX131). Use their hourly cloud GPUs or pick RunPod if you need the 4090 specifically.

• Lambda Labs does not offer the RTX 4090 either; the lineup starts at A100. For the consumer GPU tier, RunPod and Vast.ai are the main rental surfaces in May 2026.

For a side-by-side cost framing on managed agent use, see the DeepSeek V4 Pro vs Claude Opus 4.7 refactor benchmark, which prices the same 5-PR refactor in both stacks.

What real-world tasks does this stack still struggle on?

The stack is strongest at single-repo coding loops with structured output and reasoning under 64K context. It is weakest at long-horizon planning across many tools, freeform creative writing, and any task that requires the speculative path to fall back to sampling.

Concrete failure modes I have seen or that are documented:

• Cross-repo refactors over 100K context. Even with TurboQuant turbo3_tcq compression, attention quality on the 27B drops noticeably past ~150K tokens. The Qwen team confirms the 1M extended mode needs YaRN scaling and benchmarks were not run at full extension.

• Creative prose generation. DFlash is greedy-only; turn it off and you fall back to ~35 tok/s on a 3090. For prose, run vanilla llama.cpp or use a sampling-friendly runtime.

• Tool-calling chains with novel tools. Qwen 3.6 tool-call accuracy is high on standard MCP servers (file, bash, web search) but degrades on bespoke tools that do not appear in pretraining.

• Multimodal long-context combinations. Vision plus 125K text context fits per the Medium overnight-stack write-up, but throughput drops below 50 tok/s and OOMs become common.

• Agentic loops needing self-correction. The 27B will sometimes loop on a failing test for 10+ iterations without trying a new approach. The Star Elastic 30B used as the main model (instead of as a draft) handles this better but does not fit on a single 24 GB card.

If your daily work hits any of those, plan for either an RTX 5090 (32 GB), a second GPU for tensor parallelism, or a cloud fallback to Claude Opus 4.7 / DeepSeek V4 Pro for the hard 5%.

FAQ

Is BeeLlama.cpp safe to use in production?

BeeLlama.cpp is MIT-licensed and built on llama.cpp's mature inference path, so the underlying inference is production-grade. The newer DFlash and TurboQuant code paths have shipped in 2026 and have ~67 GitHub stars and active commits as of May 11, 2026, which is small. For production, pin a release tag, enable telemetry on draft acceptance rate and KV-cache hit rate, and keep a vanilla llama.cpp fallback configured. Use it for internal tooling first, validate accuracy on your workload, then promote.

Can I run Qwen 3.6 27B without BeeLlama.cpp?

Yes. Qwen 3.6 27B GGUF runs on standard llama.cpp, Ollama, vLLM, MLX (Mac), and LM Studio. You will get roughly 35 tok/s on RTX 3090 and ~120 tok/s on RTX 4090 with vanilla runtimes per Tech-Practice's benchmark. BeeLlama.cpp is the speedup layer for code and math workloads via DFlash. For prose and chat, vanilla llama.cpp or Ollama is simpler and within 10-20% on single-stream throughput.

What is the difference between Star Elastic and a normal MoE model?

A standard MoE model has many experts and routes tokens through a few of them per step; the experts are not separately usable models. Star Elastic packs three different model sizes (30B, 23B, 12B parameters) into one checkpoint such that you can extract any of the three as a standalone model with no retraining. The result is one 58.9 GB file that replaces three separate models totaling 126.1 GB, a 2.14x storage saving, and the smaller variants train alongside the large one with shared weights.

Does this work on Apple Silicon?

Partially. BeeLlama.cpp builds on macOS with Metal support and TurboQuant works there, but DFlash speculative decoding is Linux-only with CUDA 12.4+ as of May 2026. On an M3 Max or M5 Max you can run Qwen 3.6 27B at 30-40 tok/s through MLX or vanilla llama.cpp Metal, but you will not get the 2x DFlash speedup. The Tech-Practice benchmark includes M5 Max numbers for direct comparison.

How does the 1M context claim hold up in practice?

The 1M token context is achieved via YaRN positional scaling applied to the 262K native context, and Qwen has not published benchmark scores at full 1M extension. Practical reports cap at ~125K tokens with vision enabled on a single RTX 3090, and ~256K text-only with TurboQuant compression. Past 256K, attention quality degrades and inference slows to single-digit tok/s on consumer hardware. Treat 200K as the realistic working ceiling on a 24 GB GPU; the 1M claim is reserved for multi-GPU or H100-class deployments.

What is the best draft model size for DFlash?

The 12B Star Elastic slice is currently the best documented choice for a 24 GB card running Qwen 3.6 27B. The draft model wants to be roughly 25-50% the size of the target with high acceptance rate on the same workloads. Smaller drafts (1B-3B class) accept tokens less often, costing the speedup. Larger drafts (16B+) eat too much VRAM. The 12B variant lands at a measured 1.7x to 2.2x acceptance lift in the InsiderLLM benchmark, which is the published sweet spot.

Related from Vantaige

• Gemma 4 on RTX 4090: Real Tokens/Sec, VRAM and Variant Pick (2026), Same GPU class, different model family, same VRAM math.

• Nous Hermes 4 Self-Hosted Setup vs Closed Agents (2026), The cost and latency trade-offs of running your own agent vs API.

• Run Open-Source AI Models Locally: Battle-Tested Guide, VRAM tiers, quantization, runtime selection.

• DeepSeek V4 Pro vs Claude Opus 4.7: 5-PR Refactor Benchmark, Hosted-model fallback when the local stack hits a wall.

References

1. Qwen team, "Qwen3.6-27B model card", huggingface.co

2. Qwen team, "Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model", qwen.ai

3. Anbeeld, "BeeLlama.cpp GitHub repository", github.com

4. NVIDIA, "Nemotron Labs 3 Elastic 30B-A3B BF16 model card", huggingface.co

5. MarkTechPost, "NVIDIA AI Releases Star Elastic", marktechpost.com

6. InsiderLLM, "Best Way to Get 2x Token Output on RTX 3090: Qwen 3.6 + DFlash", insiderllm.com

7. Tech-Practice on Medium, "Qwen3.6-35B-A3B benchmarked on 3090, 4090, 5090, M5 Max", medium.com

8. fzbcwvv on Medium, "An Overnight Stack for Qwen3.6 27B: 85 TPS, 125K Context, Vision on RTX 3090", medium.com

9. Hacker News, "Nvidia Nemotron 3 Family of Models" thread, news.ycombinator.com

10. RunPod, "GPU Pricing Page", runpod.io

11. getdeploying.com, "RTX 4090 Cloud Pricing Comparison 2026", getdeploying.com