Hugging Face Model Card Guide
How to Read Any Hugging Face Model Card & Config
Learn how to inspect an open LLM repository like a machine learning engineer to calculate real-world VRAM requirements before downloading gigabytes of weights.
Interactive Hugging Face config.json Inspector
Click any line in the code block to see what it means, how it impacts your GPU, and the traps to avoid.
{
"model_type": "\"llama\"",weights "max_position_embeddings": 131072,context "num_hidden_layers": 32,attention "num_attention_heads": 32,attention "num_key_value_heads": 8,attention "head_dim": 128,attention "sliding_window": 4096,context "num_local_experts": 8,moe "num_experts_per_tok": 2,moe "torch_dtype": "\"bfloat16\"",weights}"num_key_value_heads": 8
Key-Value Heads (GQA Indicator)
The number of KV heads stored in the KV cache. When this value is smaller than num_attention_heads (here: 8 vs 32), the model uses Grouped-Query Attention (GQA).
In this example (8 KV heads vs 32 query heads), KV cache VRAM is slashed by 75%.
If num_key_value_heads equals num_attention_heads, the model uses legacy MHA and will consume 4x to 8x more VRAM for context!
1. Decoding config.json: The Single Source of Truth
Do not rely on the repository README for hardware sizing. Always check the config.json file in the 'Files and versions' tab. Key attributes include num_hidden_layers (depth), hidden_size (width), num_attention_heads (queries), and num_key_value_heads (KV cache architecture).
2. Finding the Real Context Limit (max_position_embeddings & RoPE)
The parameter max_position_embeddings defines the model's native training context length. If the repository specifies a rope_scaling block, the model uses frequency scaling (such as YaRN or linear scaling) to extend context beyond native limits.
3. The MoE Parameter Trap: Active vs Total Parameters
In Mixture of Experts (MoE) architectures like Mixtral 8x7B, Qwen 2.5 57B A14B, or DeepSeek V3 (671B total, 37B active), only a fraction of parameters are activated per token during compute. However, ALL weights must reside in VRAM. You cannot run a 671B MoE on a 48GB GPU even though active compute is modest.
4. Understanding GGUF Quantization Suffixes (Q4_K_M, IQ4_XS, FP8)
In GGUF repositories (like TheBloke, Bartowski, or Unsloth), suffixes indicate quantization method. Q4_K_M denotes 4-bit medium K-quant (the sweet spot for quality vs memory). IQ suffixes (e.g. IQ3_M, IQ4_XS) use importance matrix quantization for superior perplexity at low bit rates.
Frequently Asked Questions
File size represents static model weights only. Inference requires additional memory for CUDA context runtime overhead (~500MB to 1GB), activation buffers, and the expanding KV cache as conversation length increases.
GGUF is designed for llama.cpp and Ollama, supporting both CPU offloading and GPU inference. AWQ and GPTQ are optimized for pure GPU inference with vLLM and TensorRT-LLM. EXL2 is tailored for ExLlamaV2 on Nvidia GPUs with variable bit precision.
Check the license tag in the repository header and the LICENSE file. MIT, Apache 2.0, and BSD permits unrestricted commercial use. Custom licenses like Llama 3 Community License impose user caps and output restrictions.
It indicates the attention mechanism. If num_key_value_heads equals num_attention_heads, the model uses MHA. If it is smaller (e.g. 8 vs 32), the model uses GQA, which drastically reduces KV cache VRAM footprint.
Inspect the architectures array in config.json (e.g. LlamaForCausalLM, Qwen2ForCausalLM, DeepseekV3ForCausalLM). This identifies the exact transformer variant and framework compatibility.
Verify any Hugging Face model in our calculator
Enter model parameters or choose from our verified catalog to calculate real-world VRAM requirements instantly.
Knowledge & Deep Dives
Local LLM & VRAM Knowledge Hub
In-depth guides, architectural breakdowns, and hardware sizing manuals to help you build, run, and scale offline AI models with zero guesswork.