Nous Hermes 4: Why Self-Hosters Are Picking the Hermes Stack Over Closed-Model Agents (2026)

Nous Research's Hermes line is a family of open-weight, hybrid-reasoning fine-tunes built for people who want to run their own agent without renting tokens from Anthropic or OpenAI. The current public family covers four sizes from 14B to 405B parameters, with Hermes 4.3 36B (released December 2025 on the Psyche distributed training network) the most interesting variant for self-hosters because it nearly matches the 70B at half the VRAM cost. This guide is the model-layer companion to our OpenClaw setup guide: Hermes is the brain, OpenClaw (or any other framework) is the arms and legs.

TL;DR

• Hermes 4 is a fine-tuned model family, not an agent. It powers agents.

• Sizes: 14B (Qwen3 base), 36B (Seed-OSS), 70B and 405B (Llama 3.1)

• 36B and 14B are Apache 2.0; 70B and 405B inherit the Llama 3 license

• Built-in <tool_call> tags + JSON schema mode = honest function calling

• Run locally via Ollama or vLLM; self-host on RunPod for ~$1.49/hr

What is Nous Hermes 4?

Nous Hermes 4 is a family of open-weight large language models from Nous Research, post-trained for reasoning, function calling, structured output, and reduced refusal rates. The current Hugging Face collection lists four sizes: a 14B based on Qwen 3, a 36B (Hermes 4.3) based on ByteDance's Seed-OSS-36B, a 70B based on Llama 3.1, and a 405B based on Llama 3.1. The 14B and 36B ship under Apache 2.0; the 70B and 405B inherit the Llama 3 community license. The original Hermes 4 family was published August 25, 2025 alongside the Hermes 4 Technical Report, and Hermes 4.3 36B followed in December 2025.

Each variant is a hybrid-reasoning model: it can answer fast, or it can wrap a long chain-of-thought inside <think>...</think> tags before producing the final response. The reasoning mode is system-prompt-toggleable, which matters when you're building an agent and want to force "no thinking" on cheap classification calls and "deep thinking" on planning steps.

What changed versus Hermes 3, per the model card on Hugging Face:

• Post-training corpus expanded from ~1M samples / 1.2B tokens to ~5M samples / ~60B tokens

• Native hybrid <think> reasoning blocks

• Improved JSON schema adherence and structured output

• "Steerability" emphasis: the model refuses less for legitimate-but-edgy prompts

You can try the hosted version at chat.nousresearch.com or download weights from the Hermes 4 Hugging Face collection.

Why is Hermes 4 powerful for agents specifically?

Hermes 4 was trained with agents in mind: tool calls are emitted inside dedicated <tool_call>{...}</tool_call> tokens that vLLM and SGLang parse natively, JSON outputs follow the schema you hand it, and the hybrid reasoning mode means the same model can plan a multi-step task and then act on it. That combination is rare in the open-weight space. Most fine-tunes are good at chat or at function calling, not both.

Three practical reasons it's getting picked up by self-hosters:

1. Tool-call tokens, not regex. The model emits <tool_call> and </tool_call> as added vocabulary tokens, not as best-effort markdown. Both vLLM and SGLang ship a hermes tool parser, so you don't have to write a regex extractor that breaks every time the model decides to escape a brace.

2. Schema repair. When you pass it a JSON schema, the model is trained to produce valid JSON or repair its own malformed objects mid-generation. That removes the "retry until it parses" loop that haunts most open-weight agents.

3. Lower refusal rate. Nous reports 74.60% on their RefusalBench non-reasoning track for Hermes 4.3 36B versus 59.5% for Hermes 4 70B, per the model card. For agentic work touching legitimate-but-flagged content (security research, red-teaming your own infra, adult creative writing), that's the difference between a usable tool and a model that lectures you.

Hermes is neutrally aligned, not aligned-to-Nous. That phrasing is intentional. As Nous Research's karan4d clarified in the Hermes 4 launch thread on Hacker News: "this isn't our default sysprompt, just showcasing how the model adapts to a variety of different prompts."

Hermes 4 vs Claude Opus 4.7, GPT-5, DeepSeek V4 Pro

Hermes 4 wins on license, deployability, and refusal rate. It loses on raw frontier benchmarks against Claude Opus 4.7 and GPT-5, and trades within margin against DeepSeek V4 Pro. The right framing is "best open-weight model you can self-host" rather than "best model overall."

A few honest caveats on this table:

• The 87.7 MMLU figure for Hermes 4.3 36B is the score Nous publishes on their own model card. Independent re-evals on lmsys-style harnesses may shift +/- 2 points.

• "RefusalBench" is a Nous-built benchmark. Treat it as a directional signal for steerability, not as an apples-to-apples academic eval.

• Frontier-model scores (Opus 4.7, GPT-5) are not directly comparable here; they're listed as the ceiling against which open weights are judged.

• DeepSeek V4 Pro figures are from the model's release notes; we benchmarked V4 Pro against Opus 4.7 in our DeepSeek refactor benchmark and the spread depends heavily on task type.

If you only care about benchmark numbers, Opus 4.7 wins. If you care about running an agent under your own roof, on your own GPU, with weights you actually possess, Hermes is the only entry on this list under 100B parameters that satisfies all three.

How does Hermes 4 compare to OpenClaw?

It doesn't, because they live in different layers of the stack. OpenClaw is the agent runtime: it handles tools, channels, memory, and skills. Hermes 4 is a model: it handles token generation. You can run OpenClaw with Hermes 4 as the underlying model, the same way you'd run it with Claude or GPT.

OpenClaw, originally built by Peter Steinberger, is an open-source personal-assistant agent that connects messaging platforms (WhatsApp, Telegram, Signal, iMessage) to a chosen LLM with persistent memory and a skills registry. It is not a model; it brings its own model via configuration. From the official install docs you can point it at any OpenAI-compatible endpoint, which is exactly what vllm serve NousResearch/Hermes-4.3-36B exposes on localhost:8000.

The practical pairing looks like this:

[ messaging / browser / CLI ]
        |
[ OpenClaw / Cline / OpenCode / Aider ]   <-- agent layer (orchestration, tools, memory)
        |
[ vLLM or Ollama on your GPU ]            <-- runtime
        |
[ Hermes 4 weights ]                      <-- model layer

Picking Hermes inside OpenClaw rather than Claude Sonnet inside OpenClaw gives you three things: zero per-token cost after the GPU is paid for, no provider can shut you off mid-task, and the conversation never leaves your network. The trade is that you're now running ops for an inference server, and you'll feel that the first time vLLM OOMs at 3am.

For the OpenClaw side of the setup (gateway config, skills, channel wiring) see our OpenClaw setup guide. The rest of this article is the model-layer half.

How to set up Hermes 4 yourself

There are three honest deployment paths depending on how much hardware you have and how serious the workload is. Pick one and skip the other two.

A. Quick local on Mac or PC (Ollama)

The fastest way to try Hermes 4 is the GGUF quants on Ollama or LM Studio, running on consumer hardware. The official Hermes 4.3 36B GGUF repo ships 4-bit through 8-bit quantizations. The 4-bit Q4_K_M of the 14B fits in ~10GB of RAM and runs on an M2 Mac at usable speeds; the 36B Q4_K_M needs ~22GB and is realistic only on a 32GB+ Mac or a 24GB GPU.

# 14B - works on 16GB Mac or 12GB+ GPU
ollama pull hermes-4:14b-q4_K_M
ollama run hermes-4:14b-q4_K_M

# 36B - needs 32GB+ unified memory (Mac) or 24GB+ VRAM
ollama pull hermes-4.3:36b-q4_K_M

If a tag isn't available in Ollama's registry yet, download the GGUF from Hugging Face and create a Modelfile pointing at the local file. Success looks like a >>> prompt and a response under 10 seconds for a one-paragraph query.

B. Self-hosted on a GPU VPS

For real agent workloads you want full-precision (BF16 or FP8) on a real GPU, served by vLLM. The 36B fits on a single H100 80GB with room for a long context window; the 70B FP8 needs a single H100 or H200; the 405B realistically needs 8x H100s and you should think hard about whether you actually need it.

(Tokens-per-second numbers are community-reported on Llama 3.1 of the same parameter count, which is what Hermes 70B and 405B fine-tune from. Treat as directional, not a guarantee. Verify against your own workload.)

Standard launch (vLLM, Hermes 4.3 36B):

pip install vllm
vllm serve NousResearch/Hermes-4.3-36B \
  --tool-call-parser hermes \
  --enable-auto-tool-choice \
  --max-model-len 65536

That exposes an OpenAI-compatible endpoint on :8000 that any agent (OpenClaw, Cline, Aider, your own code) can call. The --tool-call-parser hermes flag is the one most people forget; without it, function calls come back as raw tags and your agent code has to parse them by hand.

C. CPU-only on a budget VPS

You can run Hermes 4 14B Q4 on a high-RAM CPU VPS (Hostinger KVM 8 with 32GB RAM, Hetzner CCX33, Contabo Cloud VPS XL) but performance is pedestrian: expect 3-7 tokens/sec on a modern AMD EPYC core, which is usable for chat but painful for agentic loops that emit 2-3K tokens per turn. The 36B on CPU drops below 2 tok/s and is not recommended.

This path makes sense for: (1) personal assistants where latency doesn't matter, (2) weekend tinkering, (3) air-gapped deployments where GPUs aren't available. For anything production, pay for the GPU.

Best hosting options for self-hosted Hermes

The right host depends on which Hermes variant you're running and whether the workload is bursty (rent by the hour) or steady (rent by the month). Below is the matrix we'd hand a friend asking which provider to start with.

Some links are affiliate and buying from the providers might help us get a commission.

A few honest notes:

• The DigitalOcean $200 credit covers ~111 hours on their cheapest GPU droplet: enough for a long evaluation, not enough for production.

• RunPod's Community Cloud is the cheapest hourly rate, but instances can be reclaimed; use Secure Cloud for anything that can't be interrupted.

• Hetzner's GPU offering is limited; their CPU instances are cheap enough that running a 14B Q4 there as a personal-assistant brain is genuinely viable.

What people are saying: Twitter/X reactions

Hermes 4 launched in late August 2025 and Hermes 4.3 in December. Most of the Twitter discussion is from the Nous team and a handful of named open-weight watchers; here are real, datable references rather than fabricated quotes.

• @NousResearch announced the Hermes Agent API control ("you can now control a full Hermes Agent programmatically via API"). The official account is the canonical source for Nous product updates.

• @Teknium1 (Ryan Teknium) is the lead author of the Hermes 4 Technical Report (Aug 25, 2025) and posts model-card-level technical detail on his account.

• @rogershijin (Roger Jin) co-authored the technical report; his X account is the second-most-cited handle for Hermes-internal context.

If you want raw reactions rather than founder-team posts, the higher-signal venue is the Hacker News thread on Hermes 4, which has named user comments rather than algorithmically-promoted X chatter. Quoted directly:

• whymauri: the model trades "a few points on fundamental benchmarks" to become "more steerable (less refusals)" for downstream tuning applications.

• jug: the model "holds its own surprisingly well in benchmarks" despite using the "battle tested Llama 3.1 base."

• BoorishBears: a more critical take. "Beating refusals doesn't take a wild amount of post-training" questions whether the marketing leaned harder than the modeling.

We deliberately did not invent X quotes. If you find a high-signal Hermes 4.3 thread from a named account after publication, drop the URL in our comments and we'll add it on the next review.

What people are saying: Reddit r/LocalLLaMA

r/LocalLLaMA's Hermes coverage is uneven. As of the May 11, 2026 review window, our search-indexed retrievals did not surface a high-upvote r/LocalLLaMA Hermes 4.3 thread we could quote directly without paraphrasing. Most of the visible community write-ups were on Medium, on independent blogs, or on Hugging Face's discussion tabs. Rather than fabricate Reddit quotes, we'd rather flag the gap.

What you can find on r/LocalLLaMA right now (browse directly via reddit.com/r/LocalLLaMA):

• Quant-comparison threads testing Hermes 4 70B GGUF variants on dual 3090 / dual 4090 setups

• Tokenizer-mismatch reports on early Ollama imports of the 14B (Qwen 3 chat template vs Llama 3 chat template confusion)

• Function-calling format comparisons (Hermes <tool_call> tags vs Llama 3.1 default tool format vs Mistral)

• Long discussions in the Hermes 4 70B Hugging Face discussion tab about FP8 vs BF16 quality drop

If you publish your own Hermes 4 r/LocalLLaMA write-up with real benchmark numbers from your own GPU, that's currently a high-citation gap in the corpus.

Common pitfalls and how to avoid them

These are the issues that come up repeatedly across the Hugging Face discussion tabs, Ollama issues, and the Hermes 4 70B ollama issue #12119.

1. Wrong tokenizer for the variant. The 14B uses ChatML (Qwen 3 base); the 36B and 70B use Llama 3 chat tags. Mixing them up gives garbage output that looks almost-right. Fix: always start from the model card's apply_chat_template example for that exact variant, never copy-paste a template from a different size.

2. Forgetting --tool-call-parser hermes in vLLM. Without it, vLLM returns the raw <tool_call> text inside the assistant message and your agent framework can't see a tool call. Fix: add --tool-call-parser hermes --enable-auto-tool-choice to your vllm serve line.

3. Quant choice too aggressive for agentic work. Q4_K_M is fine for chat, but for multi-step agent loops (where each step depends on the previous one being correct) the cumulative quality drop bites. Fix: use Q6_K or Q8 for agent work; reserve Q4 for one-shot chat.

4. Reasoning mode left on for cheap calls. Hybrid <think> mode is great for planning and terrible for "classify this string." Leaving it on burns tokens and latency. Fix: set the system prompt explicitly to non-thinking for cheap routes; only enable <think> mode on planning steps.

5. No explicit --max-model-len. vLLM defaults to the model's max, which can OOM your VRAM at startup. Fix: cap --max-model-len to the longest context your agent actually uses (32K-65K is plenty for most agent workloads).

FAQ

What is Nous Hermes 4 used for?

Nous Hermes 4 is used as the model layer inside self-hosted AI agents and chat applications. The most common use cases are: powering a coding agent (OpenCode, Cline, or Aider) on a private GPU, running a personal-assistant agent like OpenClaw without sending conversations to a cloud provider, building structured-output pipelines that need reliable JSON, and red-team / security research where commercial models refuse legitimate prompts. It is not a coding-only model, but its function-calling tokens make it a strong fit for agent workloads.

Is Hermes 4 better than Claude Opus 4.7?

No, on raw frontier benchmarks Claude Opus 4.7 still leads. Hermes 4 wins on different axes: open weights, Apache 2.0 (for the 14B and 36B), no per-token cost after you own the GPU, and the ability to run fully air-gapped. If your agent workload is bounded and you have GPU capacity, Hermes is cheaper and more controllable. If you need the absolute best reasoning per query and can pay $15/$75 per million tokens, Opus 4.7 wins on quality. We compared the two paradigms in our DeepSeek V4 Pro vs Opus refactor benchmark.

What hardware do I need for Hermes 4?

For the 14B Q4: a 16GB Mac or a 12GB+ GPU (RTX 3060 and up). For the 36B Q4: 22GB+ of RAM/VRAM, realistically a 32GB+ Mac or an RTX 4090. For the 36B BF16: a single A100 80GB or H100. For the 70B FP8: a single H100 80GB. For the 405B: 8x H100s and a real reason to need it. CPU-only is possible on the 14B but caps around 5-7 tokens/sec on modern EPYC cores, which is fine for chat and slow for agents.

Is Hermes 4 free to use commercially?

The Hermes 4.3 36B and Hermes 4 14B are released under Apache 2.0, which permits commercial use without restriction. The Hermes 4 70B and 405B inherit the Llama 3 community license, which is permissive for most commercial use but has clauses around naming attribution and a 700M monthly active user threshold. Read the relevant license text before shipping a paid product on top of the 70B or 405B; for most teams it's fine, but for very large deployments check with counsel.

How does Hermes 4 do function calling?

Hermes 4 emits tool calls inside dedicated <tool_call>{...}</tool_call> tokens that are part of the model's added vocabulary, not improvised tags. vLLM and SGLang ship native parsers for the format (--tool-call-parser hermes in vLLM). The model is also trained to repair its own malformed JSON mid-generation, which removes the retry loop most open-weight agents need around tool calls. The format is documented inline on the Hermes 4.3 36B model card.

Can I run Hermes 4 with Ollama?

Yes for the GGUF quants, with caveats. The 14B Q4 and 36B Q4 GGUFs run on Ollama if you import them via Modelfile (or use a community-pushed tag if one exists). Watch for tokenizer-template confusion: the 14B is Qwen 3 / ChatML, while the 36B and 70B are Llama-3-chat. Use the template from the matching model card's apply_chat_template snippet, not a template borrowed from a different size. For production agent workloads, vLLM is the better runtime.

What is the difference between Hermes 4 and "Hermes Agent"?

Hermes 4 is a model. "Hermes Agent" appears in some 2026 search results as a separate framework around Nous's models, but the canonical Nous Research output is the model family (Hermes 1, 2, 3, 4, 4.3) plus the Hermes Function Calling repo. When you see content marketing the phrase "Hermes Agent," verify it's pointing at official Nous releases (the @NousResearch X account is the canonical source) and not a third-party repackage.

References

1. Nous Research, "Introducing Hermes 4.3: Local Intelligence Globally Trained" nousresearch.com/introducing-hermes-4-3

2. Hermes 4.3 36B model card (Hugging Face) huggingface.co/NousResearch/Hermes-4.3-36B

3. Hermes 4 70B model card (Hugging Face) huggingface.co/NousResearch/Hermes-4-70B

4. Hermes 4 405B model card (Hugging Face) huggingface.co/NousResearch/Hermes-4-405B

5. Hermes 4 14B model card (Hugging Face) huggingface.co/NousResearch/Hermes-4-14B

6. Hermes 4 Technical Report (arXiv 2508.18255) arxiv.org/abs/2508.18255

7. Hermes 4 Hugging Face collection huggingface.co/collections/NousResearch/hermes-4-collection

8. Hermes Function Calling repo (GitHub) github.com/NousResearch/Hermes-Function-Calling

9. Hermes 4 Hacker News discussion news.ycombinator.com/item?id=45037064

10. RunPod GPU pricing runpod.io/pricing

11. Hermes 4 70B Ollama issue #12119 (GitHub) github.com/ollama/ollama/issues/12119

12. Nous Research X account x.com/NousResearch

13. OpenClaw install docs openclaws.io/install

14. Vantaige, "DeepSeek V4 Pro vs Claude Opus 4.7: Real-Repo Refactor Benchmark (2026)" vantaige.io/blog/deepseek-v4-pro-vs-claude-opus-4-7-refactor-benchmark-2026

15. Vantaige, "OpenClaw Setup Guide: Secure Config & Automation Hacks" vantaige.io/blog/openclaw-setup-guide-secure-config-automation-hacks

Related from Vantaige

• OpenClaw Setup Guide: Secure Config & Automation Hacks

• DeepSeek V4 Pro vs Claude Opus 4.7: Real-Repo Refactor Benchmark (2026)

• Run Open-Source AI Models Locally: A Battle-Tested Guide (2026)

• Gemma 4 on RTX 4090: Tokens-per-Second & VRAM Benchmark (2026)