Hugging Face Model Card Guide
How to Read Any Hugging Face Model Card & Config
Learn how to inspect an open LLM repository like a machine learning engineer to calculate real-world VRAM requirements before downloading gigabytes of weights.
Interactive Hugging Face config.json Inspector
Click any line in the code block to see what it means, how it impacts your GPU, and the traps to avoid.
{
"model_type": "\"llama\"",weights "max_position_embeddings": 131072,context "num_hidden_layers": 32,attention "num_attention_heads": 32,attention "num_key_value_heads": 8,attention "head_dim": 128,attention "sliding_window": 4096,context "num_local_experts": 8,moe "num_experts_per_tok": 2,moe "torch_dtype": "\"bfloat16\"",weights}"num_key_value_heads": 8
Key-Value Heads (GQA Indicator)
The number of KV heads stored in the KV cache. When this value is smaller than num_attention_heads (here: 8 vs 32), the model uses Grouped-Query Attention (GQA).
In this example (8 KV heads vs 32 query heads), KV cache VRAM is slashed by 75%.
If num_key_value_heads equals num_attention_heads, the model uses legacy MHA and will consume 4x to 8x more VRAM for context!
1. Decoding config.json: The Single Source of Truth
Do not rely on the repository README for hardware sizing. Always check the config.json file in the 'Files and versions' tab. Key attributes include num_hidden_layers (depth), hidden_size (width), num_attention_heads (queries), and num_key_value_heads (KV cache architecture).
2. Finding the Real Context Limit (max_position_embeddings & RoPE)
The parameter max_position_embeddings defines the model's native training context length. If the repository specifies a rope_scaling block, the model uses frequency scaling (such as YaRN or linear scaling) to extend context beyond native limits.
3. The MoE Parameter Trap: Active vs Total Parameters
In Mixture of Experts (MoE) architectures like Mixtral 8x7B, Qwen 2.5 57B A14B, or DeepSeek V3 (671B total, 37B active), only a fraction of parameters are activated per token during compute. However, ALL weights must reside in VRAM. You cannot run a 671B MoE on a 48GB GPU even though active compute is modest.
4. Understanding GGUF Quantization Suffixes (Q4_K_M, IQ4_XS, FP8)
In GGUF repositories (like TheBloke, Bartowski, or Unsloth), suffixes indicate quantization method. Q4_K_M denotes 4-bit medium K-quant (the sweet spot for quality vs memory). IQ suffixes (e.g. IQ3_M, IQ4_XS) use importance matrix quantization for superior perplexity at low bit rates.
Frequently Asked Questions
Why does my downloaded model use more VRAM than the file size on disk?
What is the difference between GGUF, AWQ, EXL2, and GPTQ?
How do I know if an open weights model supports commercial use?
What does num_key_value_heads mean in config.json?
Where can I find the model architecture class?
Verify any Hugging Face model in our calculator
Enter model parameters or choose from our verified catalog to calculate real-world VRAM requirements instantly.
Knowledge & Deep Dives
Local LLM & VRAM Knowledge Hub
In-depth guides, architectural breakdowns, and hardware sizing manuals to help you build, run, and scale offline AI models with zero guesswork.