MLflow

MLflow is an open-source platform for managing the complete machine learning lifecycle, from experiment tracking through model deployment and production monitoring. Built originally by Databricks engineers and released in June 2018, it was donated to the Linux Foundation in June 2020 and now operates as a neutral open-source project under the Apache 2.0 license. With over 30 million monthly downloads and 20,000 GitHub stars contributed to by more than 900 developers, it is the most widely adopted MLOps platform in enterprise environments, used by Fortune 500 companies and thousands of organizations building production ML systems.

The platform covers four core areas: experiment tracking (logging parameters, metrics, and artifacts across training runs), a model registry (versioning, staging, and approval workflows for model promotion), model deployment (packaging to Docker, REST APIs, Kubernetes, and cloud platforms like AWS SageMaker and Azure ML), and, as of MLflow 3, production-grade GenAI observability. MLflow 3 added OpenTelemetry-based tracing for LLM applications, LLM-as-a-judge evaluation, a Prompt Registry with version diffing, streaming support via the ResponsesAgent class, and auto-tracing integrations for PydanticAI, smolagents, and 20+ GenAI frameworks. It supports Python, TypeScript, JavaScript, Java, and R, and works with every major ML library including PyTorch, TensorFlow, scikit-learn, XGBoost, and LightGBM. You can pair it with Databricks Mosaic AI for fully managed deployment or run it entirely self-hosted on your own infrastructure.

What MLflow actually does in May 2026

MLflow 3, launched June 11, 2025, marked a turning point from pure experiment ledger to a full AI engineering platform. The core architecture shifted from a run-centric model to a LoggedModel-centric model, where models are first-class entities connected to the runs, traces, prompts, and evaluation metrics that shaped them. This makes it significantly easier to compare model variants across experiments, track lineage from training data through evaluation to production deployment, and manage the version history of GenAI agents with the same rigor previously reserved for classical ML models.

On the classical ML side, the workflow is well-established. A data scientist installs MLflow with a single pip install mlflow, calls mlflow.sklearn.autolog() before training, and every hyperparameter, metric, and artifact from that run is automatically captured. The tracking UI then lets teams compare hundreds of runs side by side, filter by metric thresholds, and register the best model to the Model Registry with a single click. From there, models move through staging and production stages with approval gates, then deploy as REST endpoints, Docker containers, or batch inference jobs.

The GenAI capabilities introduced in MLflow 3 address a problem the original architecture never anticipated: debugging non-deterministic agent behavior at scale. The tracing system, built on OpenTelemetry, captures every prompt, tool call, retrieval step, and LLM response in a structured trace. When an agent hallucinates or fails, engineers replay the full trace in the UI to pinpoint the exact step where context went wrong. The LLM judge system provides research-backed evaluators for measuring free-form text quality systematically, including multi-turn conversation evaluation added in MLflow 3.10 (February 2026) and continuous monitoring with judge-in-the-loop alerting added in 3.9 (January 2026).

"MLflow 3.0's tracing has been essential to scaling our AI-powered security platform. It gives us end-to-end visibility into every model decision, helping us debug faster, monitor performance, and ensure our defenses evolve as threats do." - Sam Chou, Principal Engineer at Barracuda Networks, Databricks blog, June 2025

"MLflow 3.0 gave us the visibility we needed to debug and improve our Q&A agents with confidence. What used to take hours of guesswork can now be diagnosed in minutes." - Daisuke Hashimoto, Tech Lead at Woven by Toyota, Databricks blog, June 2025

Where MLflow sits versus Weights and Biases and Comet ML

Weights and Biases is the most direct competitor for experiment tracking, but the two tools are architecturally opposite on the hosting axis. W&B is cloud-first SaaS: all experiment data syncs to W&B's servers in real time, requiring no infrastructure work. MLflow is self-hosted by default, meaning teams own their data entirely but also own the database, artifact storage, authentication setup, and upgrade path. W&B's visualization layer is also substantially more capable: it offers parallel coordinates plots, embedding projectors, confusion matrices, and shareable research-style reports for non-technical stakeholders. MLflow's UI offers basic line plots and histograms. For hyperparameter optimization, W&B includes Sweeps, a built-in Bayesian optimization system; MLflow has no native equivalent and relies on external libraries like Optuna or Ray Tune. The cost tradeoff is direct: W&B's Team plan runs $50 per user per month, while MLflow is free to run but requires the infrastructure budget and DevOps hours. Teams with a Databricks investment often pair MLflow with Anyscale for distributed training workloads, keeping experiment data in-house while scaling compute externally.

Comet ML occupies a middle ground. Like W&B, it is cloud-first with real-time interactive dashboards, parallel coordinate plots, and collaboration tools that MLflow lacks out of the box. Where Comet differentiates mechanically is in artifact lineage: Comet's artifact layer explicitly tracks which experiments produced and consumed each dataset or model artifact, maintaining provenance at the artifact level rather than the run level. MLflow's lineage is run-level; to get artifact-level lineage in MLflow you need the Databricks Unity Catalog integration, which requires the managed tier. Comet also offers REST API-based model serving as a lightweight deployment path; MLflow's deployment model is richer (Docker, Kubernetes, SageMaker, MLServer) but more infrastructure-heavy. For teams exploring model serving without full orchestration, BentoML offers a complementary deployment layer that works well alongside MLflow's registry. Teams evaluating fine-tuning workflows often use MLflow alongside Predibase, which handles LoRA-based fine-tuning and integrates experiment tracking back into MLflow-compatible registries.

What the MLflow workflow reality looks like

For a team starting fresh, the experiment tracking setup is genuinely fast: install, configure a tracking URI pointing to a PostgreSQL database and S3 bucket, and the first runs start appearing in the UI within minutes. The pain begins at scale and at the edges of the feature set.

Self-hosting complexity is the most consistent friction point. Beyond the local file-based tracker, a production-grade MLflow setup requires a relational database backend (PostgreSQL or MySQL), object storage for artifacts, network configuration for multi-user access, and some form of authentication. The open-source version has no built-in RBAC; multi-user team access is either open by default or requires a custom authentication proxy. Teams without dedicated infrastructure engineers frequently find themselves spending days on what they expected to take hours.

The UI is another reality check. It is functional and sufficient for individual experiment comparison, but it does not support team commenting, shared reports, or the kind of narrative documentation that W&B's Reports feature enables. A data science team presenting model evaluation to non-technical stakeholders will need to export results elsewhere. The query API, meanwhile, has been a recurring complaint: experiments stored in a SQL backend cannot be queried with SQL directly, which frustrates engineers who expect database-native access to their own data.

Once past those setup hurdles, MLflow's depth as a lifecycle platform is hard to match in open-source. The Model Registry's staging and production transitions with approval workflows, combined with the MLflow 3.11.1 (April 2026) addition of automatic issue identification for detecting quality regressions in deployed agents, gives enterprise teams audit-ready model governance without vendor subscription costs. AWS SageMaker, Azure ML, and Google Vertex AI all offer native MLflow integration for managed tracking, which eliminates most of the self-hosting burden for teams already on cloud ML platforms.

Who MLflow is built for

MLflow fits teams where data ownership and infrastructure control are non-negotiable: regulated industries (healthcare, finance, defense) that cannot send training data to third-party SaaS platforms, and enterprises that have already built the DevOps capacity to run their own services. It is the default choice for any team running on Databricks, where Managed MLflow is included and Unity Catalog provides model governance without additional tooling. Teams building both classical ML pipelines and GenAI applications benefit from MLflow 3's unified platform: the same tool that tracks a gradient-boosted model hyperparameter search can now trace a multi-step LLM agent and evaluate its output quality with judge metrics. Researchers and ML engineers at companies with Databricks investments will also find it integrates naturally with Databricks Mosaic AI for foundation model fine-tuning and serving workflows.

The adoption numbers tell a coherent story. Thirty million monthly downloads is not an accident; it reflects how deeply MLflow has embedded itself in the Python ML toolchain as the path of least resistance for experiment logging. Universities teaching MLOps courses, cloud providers building managed ML platforms, and enterprise data teams standardizing on Databricks all default to MLflow as their shared registry and tracking layer. That gravitational pull makes it a reasonable default even for teams that might prefer W&B's UI, because organizational alignment and existing integrations often outweigh product preference. Teams pairing MLflow with a fine-tuning platform like Predibase get a complete lifecycle story from LoRA-based fine-tuning through production registry without leaving the open-source ecosystem. Those optimizing LLM serving efficiency alongside experiment governance can combine MLflow's registry with BentoML for a self-hosted deployment stack that remains fully under their control.

What MLflow is not

MLflow is not a turnkey solution for small teams without infrastructure resources. If your team wants experiment tracking to work in ten minutes without configuring databases and object storage, Weights and Biases or Comet ML will serve you better. It is not a hyperparameter optimization tool: there is no built-in equivalent to W&B Sweeps, and integration with external optimizers requires additional setup. It is not a pipeline orchestrator: MLflow does not schedule training jobs, manage data pipelines, or coordinate multi-step workflows the way Airflow or Prefect do. Teams expecting a fully featured collaboration layer with shareable reports and team commenting will be disappointed by the open-source UI. And despite MLflow 3's GenAI additions, teams building primarily LLM applications without any classical ML workloads may find Langfuse or similar purpose-built LLM observability tools to be more focused fits. The December 2024 JFrog security disclosure (CVE-2024-27132, CVSS 7.2) also highlighted that self-hosted MLflow instances running untrusted recipes can be exposed to client-side remote code execution via XSS, a concern for teams accepting external model submissions or running shared notebook environments.