Modal

Modal is a serverless AI infrastructure platform that lets developers run Python code on cloud GPUs without managing containers, Kubernetes clusters, or CUDA drivers. Founded in 2021 by Erik Bernhardsson, who previously led large-scale machine learning systems at Spotify and served as CTO at Better.com, Modal targets a specific frustration: the gap between writing Python locally and running it at scale on GPUs in the cloud. The platform hit unicorn status in September 2025 after a $87M Series B led by Lux Capital at a $1.1 billion valuation, with enterprise customers including Meta, Scale AI, and Ramp running production workloads on the platform.

Modal's core mechanism is Python decorators. Annotate a function with @app.function(gpu="H100") and the platform builds the container, provisions the GPU, handles autoscaling from zero to hundreds of instances, and tears everything down when idle. Beyond inference, Modal handles training runs, fine-tuning pipelines, batch processing jobs, secure sandboxed code execution (for AI agents running untrusted code), and collaborative notebooks. Billing runs per second of actual compute used, with no idle charges. A free Starter tier includes $30/month in compute credits with no credit card required.

What Modal actually does in April 2026

Modal exposes four core products. Inference lets you deploy LLMs, image generation models, and audio pipelines behind auto-scaled web endpoints. Training handles single-node and multi-node GPU clusters for fine-tuning runs. Sandboxes provide programmatically spawnable, gVisor-isolated environments for AI agents that need to execute untrusted code at scale, with network restriction and CIDR allowlists built in. Batch scales to thousands of containers for data processing and offline inference workloads.

Container cold starts range from 2-4 seconds for most workloads. In July 2025, Modal shipped GPU memory snapshots in alpha: the feature checkpoints full NVIDIA GPU state, including device memory, CUDA kernels, and execution contexts, allowing subsequent restores to skip model weight loading entirely. A vLLM deployment of Qwen2.5 that previously took 45 seconds to cold-start dropped to 5 seconds with snapshots enabled. The Parakeet audio transcription model went from 20 seconds to under 2 seconds.

The platform runs on a multi-cloud capacity pool, primarily Oracle Cloud Infrastructure, with intelligent scheduling across regions. GPU availability spans T4 ($0.59/hr) through H100 ($3.95/hr) and H200 ($4.54/hr) up to the B200 ($6.25/hr). CPU and memory bill separately at per-second granularity alongside GPU time.

"Modal has the fastest cold-start I've seen for 10GB+ models." - AndresSRG, Hacker News, June 2025

Where Modal sits versus Replicate and Lambda Labs

Modal vs. Replicate: Replicate is a model registry with a REST API frontend. It hosts thousands of pre-containerized open-source models (Stable Diffusion, SDXL, Flux, LLaMA variants, Whisper) that any developer can call with a few lines of code and no deployment work. Replicate's Cog tool handles custom model containerization, but the primary experience is pointing at hosted models. Modal requires you to write and deploy your own Python code, which means total flexibility (any dependencies, custom logic, multi-step pipelines) but you own the serving layer. Replicate's cold starts can exceed 60 seconds for less-popular models; Modal holds 2-4 seconds consistently. The practical split: use Replicate when you want to call a known open-source model immediately with zero setup. Use Modal when you need custom code, non-standard dependencies, or are combining models into a pipeline.

Modal vs. Lambda Labs: Lambda deprecated serverless GPU in September 2025 and now offers dedicated on-demand GPU VMs with per-hour billing. You SSH in, configure the environment, and pay for reserved time regardless of whether the GPU is doing work. Lambda's approach gives maximum control and the cheapest possible per-GPU-hour rate for sustained 24/7 training jobs because there are no platform multipliers. Modal's per-second billing with scale-to-zero is more economical for bursty or infrequent workloads. The fundamental difference: Lambda gives you a bare metal node; Modal gives you a function-as-a-service interface. Lambda targets researchers running multi-day training runs who want to minimize cost per GPU-hour and don't mind SSH-based infrastructure management. Modal targets application developers who need fast iteration and automatic scaling without an ops team.

"Whenever a team asks about compute, we always tell them to use Modal." - Aakash Sabharwal, VP of Engineering at Scale AI, September 2025

What daily Modal use actually looks like

A typical Modal workflow starts with the CLI. You install the package, authenticate, and write a Python function. The decorator syntax bundles image construction (pip packages, system dependencies, secrets) with the function definition. Running modal deploy pushes the container spec and registers the endpoint. Subsequent deploys rebuild only changed layers.

For inference, the deployment is typically a class with an @enter method (runs once at container startup to load model weights) and a @method decorated inference function. The container stays warm for a configurable idle timeout, then spins down. Cold starts on wake-up are where the latency hit occurs, mitigated by the keep_warm parameter that holds a specified number of warm replicas at the cost of ongoing compute charges.

Secrets management, volume mounts for persistent storage, cron scheduling, and web endpoint routing are all handled through Modal's SDK rather than external services. Teams at Ramp used Modal to run LLM fine-tuning pipelines, reporting 34% reduction in manual intervention and 79% cost savings versus major LLM providers. Scale AI runs Modal Sandboxes for reinforcement learning experiments, spawning thousands of concurrent isolated environments on-demand.

Who Modal is built for

Modal fits ML engineers and backend developers who know Python and want GPU access without Kubernetes. The platform rewards teams with bursty, unpredictable workload patterns: inference APIs that see traffic spikes, batch jobs that run nightly, agent sandboxes that scale to thousands of concurrent environments and then return to zero. The per-second billing model directly benefits workloads that are active less than 24/7.

Startups building AI-native products get the most leverage: inference, training, fine-tuning, and sandboxes under one platform means fewer infrastructure dependencies and faster iteration. The $30/month free tier makes prototyping accessible. Enterprise teams on the Team or Enterprise plan get HIPAA compatibility, audit logs, Okta SSO, and a static IP proxy for egress control.

What Modal is not

Modal is not a model marketplace. There is no library of pre-hosted models to call. If you want to run Flux or Stable Diffusion with zero deployment code, Replicate does that. Modal requires you to write the serving code yourself.

Modal is not cost-optimal for sustained 24/7 workloads. Production multipliers apply to base GPU rates: non-preemptible plus regional multipliers compound to as much as 3.75x the listed rate for US workloads. If you are running a continuous training job around the clock, renting a dedicated Lambda or CoreWeave GPU node by the hour will be cheaper. The serverless model is efficient for bursty usage patterns, not sustained saturation.

Modal does not support private networking, custom VPCs, or service-to-service authentication at the network level. Teams with strict network isolation requirements, or regulated environments needing VPC-locked data flows, will hit this wall.

Finally, Modal is not a full-stack platform. It handles compute but not databases, message queues, or frontend hosting. Teams building complete products need Modal alongside other infrastructure services. The SDK also creates meaningful vendor lock-in: functions decorated with Modal-specific primitives require rewriting to migrate off the platform.