Spring AI Providers

Model providers are the organizations and services that host large language models and embedding models. They expose APIs that accept prompts and return completions, or accept text and return vector embeddings. In the Spring AI ecosystem, providers are not the application; they are the engines behind the abstractions. Your code talks to ChatModel and EmbeddingModel, and the provider adapter translates those calls into provider-specific requests.

Understanding the provider landscape is a critical architectural skill. The provider you choose—or the multi-provider strategy you design—determines your system’s latency, cost, model capabilities, data residency, and operational reliability. This section of the handbook gives you a structured tour of the major providers supported by Spring AI, explains how provider abstraction works, and equips you with the criteria needed to make informed decisions.

Use this page as your entry point. It introduces the concepts, outlines a learning path, and connects provider knowledge to the rest of the Spring AI architecture.

Why Providers Matter

A large language model is not a commodity. Providers differ significantly in the models they offer, how those models perform, and the constraints under which they operate.

• Model capability differences – Some models excel at reasoning, others at code generation, and others at multilingual tasks. Provider choice directly affects the quality of your AI features.
• Cost and latency trade-offs – Cloud providers charge per token; local providers cost only hardware. Latency varies from sub-100ms for small local models to several seconds for large cloud models.
• Regional availability – Not all providers are available in all geographic regions. Data residency requirements may mandate using a provider with infrastructure in a specific jurisdiction.
• Enterprise compliance needs – Industries like finance and healthcare require contractual commitments around data handling, model training, and auditability. Some providers offer enterprise agreements; open-source local models give full control.
• Vendor lock-in concerns – Writing provider-specific code creates lock-in. Spring AI’s abstraction layer mitigates this, but provider-specific features (like structured output modes or custom tool formats) can still create soft lock-in if relied upon heavily.
• Production reliability and operational fit – Managed providers handle scaling and uptime for you. Self-hosted models require your own infrastructure but eliminate external dependencies.

A well-architected Spring AI application treats the provider as a configuration choice rather than a code choice. This section teaches you how to achieve that.

Providers in Spring AI Architecture

Provider adapters are the bottom layer of the Spring AI stack. They translate framework abstractions into HTTP calls and wire formats.

Application – Your business logic. It depends on ChatClient and the framework layer, not on any provider.

Spring AI Framework – The core abstractions: ChatClient, Prompt, Advisors, Memory. These are provider-agnostic.

ChatModel / EmbeddingModel – The portable interfaces that define what it means to chat or embed. Provider adapters implement these.

Provider Adapter – The Spring-managed bean that implements ChatModel (or EmbeddingModel) for a specific provider. It handles authentication, request serialization, response deserialization, and error mapping.

Provider API – The actual REST or gRPC endpoint exposed by the provider. Spring AI never exposes this directly to application code.

LLM / Embedding Service – The model itself, running in a cloud data center or on local hardware.

This layering means you can test your ChatClient integration with a mock ChatModel, swap spring-ai-openai for spring-ai-azure-openai by changing a starter dependency, and even route requests to different providers based on task requirements.

Core Provider Concepts

The following table lists the provider-specific guides available in this section. Start with the Provider Abstraction article if you need a refresher on how the abstraction layer works.

Concept	Responsibility	Related Guide
OpenAI	Cloud-based models including GPT-4o, GPT-4, and text-embedding-ada-002	openai
Azure OpenAI	Enterprise-managed OpenAI models hosted on Azure with compliance and security features	azure-openai
Anthropic Claude	Frontier models with a focus on safety, long context, and tool use	anthropic
Google Gemini	Google’s multimodal models, accessible via Vertex AI or AI Studio	gemini
Ollama	Local, open-source model serving for development and air-gapped deployments	ollama
DeepSeek	Cost-efficient models with strong reasoning capabilities	deepseek
DashScope	Alibaba Cloud’s AI model platform, including Qwen models	dashscope
Provider Comparison	A structured side-by-side evaluation of the supported providers	comparison

Each guide covers auto-configuration, application.properties settings, and code examples specific to that provider.

Recommended Learning Order

The provider section is best approached by understanding the shared abstraction first, then exploring providers in order of increasing deployment complexity.

1. Provider Abstraction – Revisit the ChatModel and EmbeddingModel interfaces and how auto-configuration wires provider beans. If you have completed the Framework section, this is a quick refresher.
2. OpenAI – The most widely used cloud provider and the default in many Spring AI examples. Start here for a smooth introduction to cloud-based models.
3. Azure OpenAI – Extends OpenAI with enterprise controls, network isolation, and compliance certifications. Essential reading for teams on Azure.
4. Anthropic Claude – Introduces a different model family with distinct prompt engineering patterns and extended context windows.
5. Google Gemini – Multimodal capabilities and integration with Google Cloud’s AI platform.
6. Ollama – The entry point for local, open-source models. No API keys, no network calls, full data control.
7. DeepSeek – A cost-optimized provider gaining rapid adoption for text-based tasks.
8. DashScope – Alibaba’s platform, important for teams operating in Asian markets or already using Alibaba Cloud.
9. Provider Comparison – After surveying the landscape, use this guide to formalize your evaluation.

This order builds from the familiar (cloud APIs) to the specialized (local and regional providers), ensuring you can contextualize each option.

Provider Abstraction Mental Model

The key to mastering Spring AI providers is understanding the separation between what you want to do and which provider executes it.

• Model-agnostic programming – Your service code should depend on ChatClient and ChatModel, not on OpenAiChatModel or any provider-specific class. This is the same principle as depending on List rather than ArrayList.
• Provider adapters – Each Spring AI starter includes a ChatModel implementation (e.g., OpenAiChatModel) that is auto-configured based on properties. You never instantiate it directly; Spring injects it into the ChatClient.
• Request/response mapping – The adapter translates the Spring AI Prompt into the provider’s HTTP request structure (typically an OpenAI-compatible chat completion request) and maps the HTTP response back to a ChatResponse. Differences in API semantics are handled in the adapter layer.
• Configuration differences – Each provider has its own set of configuration properties (API keys, endpoints, model names, timeouts). These are managed through Spring Boot’s type-safe configuration and profiles, not through code changes.
• Deployment environment constraints – Some providers are cloud-only; others run locally. The abstraction allows you to use cloud providers in production and local Ollama in development with the same code, switching via Spring profiles.

This mental model turns providers from a “platform choice” into a “runtime configuration.”

Provider Abstraction vs Provider-Specific APIs

A common tension in framework design is how much to abstract versus how much to expose. Spring AI resolves this by offering a clear boundary:

• Spring AI framework abstractions provide the portable, stable API that your application logic uses. They cover the common 80% of use cases: send a prompt, get a response, handle streaming, manage tools.
• Provider-specific capabilities (such as Azure’s content filtering, OpenAI’s structured output response format, or Claude’s extended thinking) may require configuration that is unique to that provider. Spring AI exposes these through provider-specific configuration properties or, where necessary, through casting to the concrete adapter class.

Best practice: Keep your core business logic against the abstractions. Isolate provider-specific tuning in configuration classes or dedicated advisor implementations. This gives you portability where it matters and control where it’s needed.

Major Provider Options

OpenAI

The most widely adopted cloud AI provider. Spring AI’s OpenAI starter supports all GPT models, embedding models, and features like function calling, JSON mode, and streaming. It is the default choice for many Spring AI tutorials and a solid baseline for cloud-based AI.

Azure OpenAI

Microsoft’s managed OpenAI service, running GPT models within Azure data centers. It adds enterprise features: private networking, customer-managed encryption keys, content filtering, and regional deployment. For organizations already committed to Azure, it is the natural path to AI adoption.

Anthropic Claude

Anthropic’s Claude models emphasize safety, long context windows (up to 200K tokens), and structured tool use. Claude is a strong choice for document analysis, multi-step reasoning, and applications where model behavior must be predictable and well-documented.

Google Gemini

Google’s multimodal models, available through Vertex AI (enterprise) and AI Studio (developer). Gemini supports text, images, audio, and video inputs. Its tight integration with Google Cloud services makes it a compelling option for teams in the GCP ecosystem.

Ollama

Ollama provides a local model serving layer that runs open-source models (Llama, Mistral, Gemma, Qwen) on your own hardware. It is ideal for development, CI/CD pipelines, air-gapped environments, and any scenario where data must not leave the machine. Spring AI treats Ollama as just another provider.

DeepSeek

A provider gaining attention for its cost-efficient models that deliver strong performance on reasoning and code tasks. It offers OpenAI-compatible APIs, making integration straightforward. DeepSeek is an attractive option for cost-sensitive, high-throughput text workloads.

DashScope

Alibaba Cloud’s model platform, home to the Qwen family of models (including Qwen-Max, Qwen-Plus, and Qwen-VL). DashScope is the primary provider for teams building AI applications within the Alibaba Cloud ecosystem or targeting Chinese-language markets. Spring AI Alibaba deepens this integration further.

Provider Selection Criteria

Selecting a provider is an architectural decision. Use these criteria to guide your evaluation:

• Model quality – Performance on your specific tasks, measured through internal benchmarks or evaluation harnesses.
• Latency – End-to-end response time, including network overhead. Critical for real-time applications.
• Cost – Token pricing, volume discounts, and infrastructure costs for self-hosted models.
• Context window – Maximum input tokens. Larger windows reduce the need for aggressive chunking but increase latency and cost.
• Embeddings support – Availability of embedding models and their dimensions, which affect downstream retrieval quality.
• Streaming support – Token-by-token delivery, essential for responsive user interfaces.
• Tool calling support – How well the provider supports function calling and whether it aligns with Spring AI’s @Tool model.
• Regional availability – Data center locations that satisfy data residency and latency requirements.
• Enterprise governance – RBAC, audit logs, private networking, and contractual commitments around data usage.
• Data privacy – Whether data sent to the API can be used for model training, and what retention policies apply.
• Operational reliability – SLAs, uptime history, and support quality for production-critical workloads.

The Provider Comparison guide maps these criteria across the major providers to support data-driven decisions.

Provider Integration and Spring AI Features

Provider choice can influence how you use other Spring AI features:

• ChatClient behavior – The same ChatClient code works across providers, but response style, instruction following, and output formatting vary. Test prompts against each target provider.
• ChatModel behavior – The ChatModel implementation handles provider-specific concerns (authentication, retries). You inject the same interface regardless of provider.
• EmbeddingModel behavior – Embedding dimensions, normalization, and quality differ across providers. Ensure the embedding model used for indexing matches the one used for querying.
• Streaming – All major providers support streaming, but the wire format and error handling during streaming can differ.
• Tool Calling – The @Tool annotation works across providers, but the underlying tool call format varies. Spring AI normalizes these, but complex schemas may behave differently.
• Structured Output – Some providers (OpenAI, Azure) offer native JSON mode. Others rely on prompt engineering to produce valid JSON. Spring AI’s converters handle both, but native modes are generally more reliable.
• RAG – The embedding and chat models used in RAG pipelines come from providers. Using different providers for embeddings and chat is common and fully supported.
• Observability – Provider adapters emit metrics and traces. Knowing which provider served a request is essential for cost attribution and latency analysis.

Spring AI Providers in the Handbook

The Providers section connects to every other part of the handbook:

Section	Relationship to Providers
Getting Started	Configures your first provider and verifies the connection.
Framework	Defines the ChatModel and EmbeddingModel interfaces that providers implement.
RAG	Uses embedding and chat models from providers for ingestion and generation.
Vector Databases	Stores embeddings generated by provider embedding models.
Enterprise AI	Addresses multi-tenancy, cost control, and compliance across providers.
Tutorials	End-to-end projects that demonstrate provider configuration and usage.
Source Code	Explores how provider adapters are implemented and auto-configured.

Where Providers Lead Next

After understanding providers, you are ready to:

• RAG – Build retrieval pipelines that leverage embedding and chat models from your chosen providers.
• Vector Databases – Store and query the embeddings generated by provider embedding models.
• Enterprise AI – Design production architectures that manage multi-provider strategies, failover, and cost optimization.
• Tutorials – Build complete applications that integrate providers into real-world workflows.
• Source Code – Study the internal design of provider adapters and learn how to create custom integrations.

These sections build on your provider knowledge, taking you from configuration to complete, production-ready systems.

Who Should Read This Section?

Java Developers

You need to connect your Spring Boot application to an LLM. This section shows you how to add a starter dependency, configure an API key, and start building AI features without being a cloud infrastructure expert.

Spring Developers

You understand auto-configuration and Spring Boot starters. This section maps that knowledge to the AI domain, showing how spring-ai-openai-spring-boot-starter and its peers bring models into your application context.

AI Engineers

You are familiar with multiple providers from Python experience. This section explains how provider integration works in a Spring-managed lifecycle, including configuration, dependency injection, and testing.

Software Architects

You are designing the AI service layer for your organization. This section provides the comparison criteria and abstraction model that support a flexible, future-proof provider strategy.

Common Mistakes to Avoid

• Choosing providers only by popularity – The most talked-about provider may not be the best fit for your use case, budget, or compliance requirements. Evaluate systematically.
• Ignoring cost and latency – Development with a fast, expensive cloud model may work perfectly; production at scale may require a cost-efficient or local alternative.
• Mixing development and production requirements – Using Ollama locally and expecting identical behavior from a cloud provider without testing is a common source of production surprises.
• Assuming embeddings behave the same across providers – Embedding models differ in dimensions, normalization, and semantic alignment. Changing the embedding provider requires re-indexing your vector store.
• Overlooking compliance and regional restrictions – Sending data to a provider in another jurisdiction may violate GDPR, HIPAA, or internal policies. Verify data handling before selecting a provider.
• Hardcoding provider details into business logic – Avoid direct dependencies on OpenAiChatModel or provider-specific configuration objects. Use the ChatModel interface and Spring configuration to keep your code portable.

Summary

Provider abstraction is the cornerstone of Spring AI’s portability promise. It allows you to build AI applications that adapt to changing model capabilities, cost structures, and deployment constraints without rewriting core logic. The provider you choose is important—but the architecture that isolates that choice is what determines long-term success.

This section equips you with a working knowledge of each major provider, a framework for evaluating them, and a clear understanding of how they plug into the Spring AI stack. Start with the OpenAI guide if you are looking for the quickest path to a working cloud model, or the Ollama guide if you prefer local, private development. When you are ready to formalize your decision, the Provider Comparison will help you weigh the options against your specific requirements.