MLX

MLX is an open-source array framework for machine learning, released by Apple ML Research in December 2023 under the MIT license. It is purpose-built for Apple Silicon chips (M1, M2, M3, M4, and M5) and built around a core architectural advantage: Apple's unified memory, where the CPU and GPU share the same physical pool of RAM. That design eliminates the data-copy bottleneck that slows frameworks like PyTorch when moving tensors between processor types. The result is that MLX can run large language models faster on a MacBook than most alternatives, without any paid cloud subscription or external API.

The framework exposes Python, C++, C, and Swift APIs. The Python interface mirrors NumPy conventions; the higher-level neural-network and optimizer layers follow PyTorch idioms, so researchers familiar with either can pick it up quickly. MLX-LM, the companion package for LLM inference and fine-tuning, handles model downloads from Hugging Face, quantization (4-bit, 8-bit, bf16), prompt caching, rotating key-value cache for memory efficiency, LoRA and QLoRA fine-tuning, and distributed inference across multiple Macs. Installation is a single pip command. The MLX Community on Hugging Face hosts thousands of pre-quantized models ready to run. As of April 2026, MLX sits at version 0.31.2 with 73 releases and active maintenance by Apple engineers.

What MLX actually does in May 2026

MLX is the inference and training engine for local ML workloads on Mac. On the inference side, MLX-LM lets you pull any Hugging Face model, quantize it on the fly, and run a chat interface directly from the terminal. There is no server to configure, no Docker container, no GPU driver stack. On an M3 Max with 96GB unified memory, you can load a 4-bit Llama 70B and generate 10-15 tokens per second. On an M4 Max running a 35B Mixture-of-Experts model, real benchmarks show roughly 130 tokens per second. The M5 chip pushed time-to-first-token below 10 seconds for dense 14B architectures, and under 3 seconds for 30B MoE models, with Apple's new GPU Neural Accelerators providing dedicated matrix-multiplication circuits that MLX addresses directly via Metal 4.

On the training side, MLX supports LoRA and QLoRA fine-tuning via a single command. You specify your model, your dataset in JSONL format, and your training duration. Training Mistral-7B on 5,000 examples takes roughly 45-90 minutes on an M2 Max, peak memory around 7GB at full precision. Training on a 4-bit quantized model cuts memory roughly 3.5x with minimal accuracy loss. Adapters can then be uploaded to Hugging Face and shared. Distributed training is available via mx.distributed for teams with multiple Apple Silicon machines.

Lazy computation means arrays only evaluate when results are needed. Dynamic computation graphs rebuild when input shapes change, avoiding slow recompilation during research iteration. The Swift API is stable enough that iOS and macOS app developers are embedding models directly into applications without cloud dependency. At WWDC 2025, Apple devoted three dedicated sessions to MLX, positioning it as the preferred framework for LLM inference on Apple Silicon. The March 2026 announcement that Ollama was switching to MLX as its inference engine, delivering 2x faster response generation in its beta, validated the framework as production-ready infrastructure.

Where MLX sits versus llama.cpp and PyTorch MPS

MLX vs. llama.cpp: llama.cpp uses GGUF format and runs on CPU, Metal, Vulkan, CUDA, and other backends. It is genuinely cross-platform: Windows, Linux, macOS, Android. MLX runs only on macOS. On raw throughput for models under 14B parameters, MLX wins clearly: M2 Ultra with Qwen-2.5 7B achieves around 230 tok/s in MLX versus 150 tok/s in llama.cpp, a 53% advantage. On M4 Max with a 0.6B model, MLX clocks 525 tok/s against llama.cpp's 281 tok/s. At 27B+ parameters where memory bandwidth is the bottleneck, both frameworks converge and performance becomes approximately equal.

The meaningful architectural difference is memory handling. MLX uses true zero-copy unified memory through lazy evaluation. llama.cpp uses CPU+GPU layer splitting via the -ngl flag, which means you can load a model larger than your GPU memory by overflowing layers to CPU RAM. MLX has no equivalent capability: if the model does not fit in unified memory, it will not run. llama.cpp also supports over 10 quantization formats (Q2_K, Q4_0, Q8_0, IQ variants) versus MLX's four. And llama.cpp is inference-only; MLX includes on-device LoRA fine-tuning natively. For users on Windows or Linux, or who need to load models close to or above their memory ceiling, llama.cpp is the pragmatic choice. For Mac-first inference where throughput matters and models fit comfortably in memory, MLX wins.

MLX vs. PyTorch MPS backend: PyTorch's Metal Performance Shaders backend lets Mac users run existing PyTorch code on the Apple GPU. For language model token generation, the gap is large: MLX achieves around 230 tok/s on M2 Ultra for LLM inference; PyTorch MPS manages 7-9 tok/s. That is not a typo. The difference comes from unified memory architecture: PyTorch still copies tensors between CPU and GPU in the traditional way, incurring overhead that MLX avoids entirely. One benchmark on an M3 Pro found PyTorch MPS 5.5x faster than MLX for raw matrix multiplication (128x128, 10K iterations), which shows MLX's advantage is specifically optimized for end-to-end LLM workloads, not every primitive operation. If you have a large existing PyTorch training pipeline and want to run it on Mac without rewriting it, PyTorch MPS is the compatibility path. If you are starting fresh and want the best Mac LLM performance, MLX is the right foundation.

"This is extremely impressive, the amount of technical lift here totally changes how I'm thinking about my next app." - Nathan Tarbert, developer, dev.to WWDC 2025 session thread, June 2025

"When I first ran this code and saw text appearing on my screen, generated entirely on my laptop without any API calls, it felt like a small revolution. My MacBook was suddenly a self-contained AI powerhouse." - developer testimonial, willitrunai.com, 2025

What the daily development reality looks like

Getting started is genuinely fast. Install MLX-LM with pip into a virtual environment, then run one command to launch a chat session pointing at any Hugging Face model slug. The model downloads and quantizes automatically. You can swap from Mistral-7B to Llama-3-70B by changing a flag. Prompt caching means repeated long-context sessions do not re-process the full context on each turn, which is important for coding agents that maintain long system prompts.

Fine-tuning follows a similar pattern. Prepare your data as JSONL, run mlx_lm.lora with the model, data path, and iteration count. No CUDA driver installation, no environment variable configuration, no compatibility matrix to navigate. One developer described fine-tuning Mistral-7B to generate API documentation in a specific format, the whole run taking 45 minutes on a MacBook Pro with peak memory around 6.8GB. Training on a 4-bit quantized base model cuts memory roughly 3.5x, making it feasible to fine-tune 13B models on 24GB systems.

Integration with other tools continues to improve. LM Studio now supports MLX as a backend, giving users a graphical interface over MLX inference. Ollama announced in March 2026 that it had switched to MLX as its inference engine on Apple Silicon, giving Ollama users 2x faster generation without any workflow change. The Hugging Face MLX Community hosts thousands of pre-quantized models in safetensors format, covering most architectures that would otherwise only appear first in GGUF. Users who want to run models in a GUI alongside web search or document workflows often pair MLX inference with Ollama's REST API layer, accepting a modest throughput overhead for the convenience of standardized endpoints.

The December 2023 public launch was notable for Apple. The company released MLX on GitHub without a press conference on December 5, 2023, the same week Google launched Gemini. Tech press coverage went almost entirely to Gemini. Within days, the ML community had found MLX and initial benchmarks showed 30% faster performance than PyTorch for some workloads on Apple hardware. The MIT license was read as a strong signal: Apple was not trying to control the ecosystem the way CUDA tied the GPU world to NVIDIA. That framing accelerated adoption among researchers who had been skeptical of Apple's AI seriousness.

Who MLX is built for

MLX is the right choice for Mac-based ML researchers and developers who want to run or fine-tune large language models without cloud costs or sending data off-device. Data scientists frustrated by PyTorch MPS inference speeds will find MLX a substantial upgrade. Privacy-focused professionals who cannot send data to external APIs have a clear reason to adopt it: all inference and training stays on the local machine. Swift developers building iOS or macOS applications with embedded models can use MLX's Swift API without a Python interpreter layer. Teams building on the Apple ecosystem long-term should treat MLX as a strategic dependency, given Apple's intentional M5 hardware optimization for MLX workloads.

Users who want to explore a wide range of open-weight models across platforms should also check out llama.cpp and Llama directly, since llama.cpp's GGUF ecosystem offers broader model availability for newly-released architectures. The two tools complement each other depending on the task.

What MLX is not

MLX is not a cross-platform framework. If your team includes Windows or Linux developers, or if you deploy models on Linux servers, MLX cannot follow you there. llama.cpp runs on all three platforms with the same model files. MLX's safetensors models will not port.

MLX does not handle models that exceed unified memory. There is no -ngl-style layer overflow to CPU. If your Mac has 24GB unified memory and you want to load a 32B model at 8-bit precision, MLX will refuse. llama.cpp can split that model across CPU and GPU. This matters for users with M1 or M2 base models (8-16GB) who want to run anything larger than a 7B at decent quality.

Prefill performance is a genuine weakness. For short chat exchanges, MLX's generation advantage can be outweighed by its slower prompt processing. A 650-token prompt on an M1 Max reported 94% of time spent on prefill, collapsing effective throughput below llama.cpp. M1 and M2 chips lack native bf16 support, compounding the problem on older hardware.

MLX is not a production LLM serving platform. There is no built-in request batching, rate limiting, or multi-user session management. The MLX-based server ecosystem (mlx-lm server, Rapid-MLX, vLLM-MLX, oMLX) is still fragmented. For a single developer on their own machine, that is fine. For teams deploying a shared internal endpoint, the tooling is not yet mature.

Finally, MLX is not a beginner tool. There is no graphical interface from Apple directly. The primary entry point is a Python terminal. Users who want a polished GUI experience should start with LM Studio or Ollama with a GUI frontend, which now use MLX under the hood anyway on Apple Silicon without requiring you to interact with the framework directly.