Databricks Mosaic AI

Databricks Mosaic AI is the AI product suite embedded inside the Databricks Data Intelligence Platform, targeted at enterprise data and ML engineering teams that need end-to-end control over how AI models are built, tuned, deployed, and governed. It is not a standalone AI product: it is a deeply integrated layer on top of Databricks Lakehouse, designed for organizations already running data infrastructure on Databricks and wanting to extend that investment into production AI. The Mosaic AI brand took its name from MosaicML, the generative AI startup Databricks acquired in July 2023 for $1.3 billion, whose team brought GPU-efficient LLM training and the MPT model family into the Databricks ecosystem.

The platform covers six major capability areas: Mosaic AI Agent Framework for building RAG and agentic applications; Mosaic AI Agent Evaluation with automated AI judges for quality measurement; Mosaic AI Vector Search, a managed vector database that auto-syncs from Delta tables; Mosaic AI Model Serving for deploying agents and LLMs at scale; Mosaic AI Model Training for fine-tuning or pretraining open-source models on proprietary data; and Managed MLflow for experiment tracking and MLOps. In June 2025, Databricks added Unity AI Gateway (a centralized governance layer for all LLMs and MCP servers) and Agent Bricks (an automated agent-building tool) at the Data + AI Summit. The platform supports third-party models including GPT-4, Meta Llama, Anthropic Claude, and Mistral alongside Databricks' own DBRX.

What Databricks Mosaic AI actually does in May 2026

Mosaic AI is organized around the full lifecycle of enterprise AI development, from raw data to production agents. Starting from a Delta Lake table, a data team can create a vector index in Mosaic AI Vector Search that stays synchronized automatically as the source data changes. That vector index feeds a RAG pipeline built using the Agent Framework SDK, which handles retrieval orchestration, prompt construction, and tool-use logic. The completed agent is deployed through Model Serving, which as of mid-2025 supports over 250,000 queries per second and includes a proprietary inference engine for open-source LLMs that Databricks claims runs up to 1.5x faster than a standard vLLM configuration.

Agent Evaluation gives teams AI judges that score responses on correctness, groundedness (hallucination detection), retrieval relevance, answer relevance, and safety. A human review interface lets subject-matter experts provide additional feedback without needing ML expertise. These scores feed directly into MLflow for experiment comparison and deployment gating. MLflow 3.0, released June 2025, was redesigned from the ground up for generative AI and now tracks agents deployed outside Databricks, including on AWS, GCP, and on-premise systems.

For teams that need custom models rather than hosted foundation model inference, Model Training handles fine-tuning on managed GPU clusters with no infrastructure management required. The June 2025 Storage-Optimized Vector Search rewrite reduced cost by a claimed 7x while scaling to billions of vectors by separating compute and storage layers. AI Gateway, also generally available since June 2025, provides automatic provider failover, PII and safety guardrails, rate limiting, and usage logging across the enterprise, routable to any LLM or MCP server.

Agent Bricks, still in beta as of mid-2025, takes a higher-level approach: a user describes a task (structured extraction, knowledge assistance, text transformation, or multi-agent coordination), connects enterprise data, and the system generates evaluations and tunes the agent automatically. Supported agent architectures include single-step retrieval, multi-hop reasoning chains, and tool-calling agents. FactSet reported a 44% improvement in accuracy after deploying an agent via this stack; Block reported $10M in productivity gains; Intercontinental Exchange reported 96% response accuracy.

"Mosaic AI Agent Framework has allowed us to rapidly experiment with augmented LLMs, safe in the knowledge any private data remains within our control. The seamless integration with MLflow and Model Serving ensures our ML Engineering team can scale from POC to production with minimal complexity." - Analytics Director, Burberry, Databricks blog, July 2024

"Mosaic AI Agent Framework has been a game-changer for us because it allowed us to evaluate the results of our GenAI applications and demonstrate the accuracy of our outputs while maintaining complete control over our data sources." - VP of Data and AI, Lippert, Databricks blog, July 2024

Where Mosaic AI sits versus AWS Bedrock and Snowflake Cortex

The three dominant enterprise AI platforms as of May 2026 are Databricks Mosaic AI, AWS Bedrock, and Snowflake Cortex, and they differ architecturally in ways that matter more than surface feature lists.

AWS Bedrock is a managed inference gateway on Amazon infrastructure. It offers access to Claude, Llama, Titan, Mistral, and other third-party models via a single API, and Bedrock Knowledge Bases handles RAG backed by Amazon S3 and OpenSearch Serverless. What Bedrock does not do is give teams control over training: fine-tuning on Bedrock is limited to specific models with constrained hyperparameter access, and the storage layer is S3, which lacks the ACID transaction guarantees and schema enforcement of Delta Lake. Bedrock's operational model is simpler: teams do not need Spark expertise, and billing is per API token. The tradeoff is less flexibility for teams that need to combine fine-tuning with RAG on the same data pipeline, or who want full training runs on proprietary model architectures. Mosaic AI supports all four major AI patterns (prompt engineering, RAG, fine-tuning, pretraining) in a unified platform; Bedrock is strongest on the managed inference and RAG ends.

Snowflake Cortex takes a fundamentally different approach: AI as SQL functions. Cortex exposes LLMs including Llama, Mistral, and Snowflake's own Arctic through standard SQL calls via Snowpark, allowing analysts without Python or ML experience to run inference inside warehouse queries. Cortex does not expose model training, fine-tuning, or custom agent orchestration: the model layer is entirely Snowflake-managed. Databricks itself claims AI Functions in SQL run up to 3x faster and 4x lower cost than "other vendors on large-scale workloads" as of June 2025, a benchmark that appears to reference Cortex. Cortex's advantage is accessibility: a SQL analyst can call an LLM without writing a single line of Python. Mosaic AI's advantage is depth: it handles the entire stack for teams that need to own the model, not just call it.

For teams comparing Mosaic AI with Vertex AI, the structural difference is the data layer: Vertex AI is tightly coupled to BigQuery and Vertex AI Feature Store, while Mosaic AI runs on Delta Lake and Unity Catalog. Both platforms support open-source model fine-tuning and managed serving, but Mosaic AI covers a wider range of training scenarios including full pretraining.

What the Mosaic AI workflow reality looks like

In practice, building production agents on Mosaic AI requires a team with overlapping expertise: Python or Scala for data engineering, SQL for Delta table management, ML concepts for evaluation and fine-tuning, and Spark tuning for cost control. The platform does not hide this complexity; it manages it. For a team that already has these skills, the integration is real: updating a Delta table automatically propagates to a live vector index without touching the deployed agent, which is a genuine operational advantage over stacks where data pipelines and AI serving are disconnected systems.

Onboarding begins with Unity Catalog governance configuration, which Databricks' own documentation acknowledges "can easily take months of careful planning." Setting up data permissions, cluster policies, and model serving endpoints before writing any AI logic is a real time cost. Teams that skip this step often find themselves retroactively retrofitting governance when compliance or legal requirements emerge, which is more expensive than building it in from the start.

MLflow integration is a genuine strength. The 3.0 release made agent tracing and prompt versioning first-class features, and the ability to track agents deployed anywhere (not just on Databricks) makes it useful as an observability layer across a hybrid stack. Teams that also use Weights and Biases for experiment tracking sometimes run both systems in parallel during fine-tuning, which adds complexity but preserves the richer W&B visualizations that MLflow does not yet replicate for training runs.

Who Mosaic AI is built for

Mosaic AI fits organizations that are already invested in Databricks Lakehouse and need to extend that platform into production AI without stitching together a separate set of tools. The data-native integration (Delta Lake auto-sync to vector indexes, Unity Catalog governance across AI and data assets, MLflow tracking from training through deployment) eliminates the connector tax that teams pay when managing separate systems for data, ML, and serving.

Industries where the platform lands consistently include financial services (governance and data residency requirements make the Unity Catalog layer valuable), healthcare analytics, and large-scale e-commerce and manufacturing (Burberry, Lippert, FordDirect, ICE are all documented production users). Teams that need to combine fine-tuning with RAG in a single pipeline, rather than choosing between them, benefit most from the unified approach.

What Mosaic AI is not

Mosaic AI is not suitable for small or mid-market teams that do not have dedicated ML engineers and Databricks-certified Spark expertise on staff. The learning curve is steep enough that the eesel.ai platform review describes it as "a cliff" for teams without deep technical experience across all three required domains simultaneously.

It is not a rapid-prototyping platform. Teams report that implementation takes months before a production system is ready, and the compliance and governance setup alone can consume weeks of engineering time before any AI logic is written. If the goal is deploying a customer-facing chatbot in two weeks, Bedrock or Cortex will get there faster.

The cost model is also not suitable for teams with limited or unpredictable AI budgets. Databricks charges DBU fees on top of cloud infrastructure costs, and users consistently report that improperly tuned Spark jobs produce billing surprises. One widely cited Reddit analysis found DBU charges representing a 600% markup over raw cloud server costs. Production AI agent workflows requiring dedicated GPU clusters can run $5,000 to $15,000 per month in base compute before any token-based API charges.

Finally, Mosaic AI carries active legal risk: the ongoing copyright lawsuit (In re Mosaic LLM Litigation) alleging training data drawn from approximately 196,000 pirated book titles via the RedPajama/Books3 dataset remains live as of April 2026, after Judge Charles Breyer denied Databricks' motion to dismiss. This is not a factor in day-to-day platform use, but procurement teams at regulated organizations should be aware of it.