Skip to main content

Spring AI Handbook

Spring AI is the official Spring ecosystem project that brings the power of large language models (LLMs) and modern AI patterns to Java developers. It transforms how enterprise Java teams build intelligent applications by applying familiar Spring principles—dependency injection, portability, abstraction, and production-ready capabilities—to the rapidly evolving AI landscape.

This handbook is your comprehensive engineering guide to Spring AI. Whether you are building your first chatbot, designing a retrieval-augmented generation (RAG) pipeline for enterprise documents, or dissecting the source code to create custom extensions, you will find structured, architecture-first content that takes you from fundamental concepts to production-grade systems.

Why does Spring AI matter?
The Java ecosystem powers the world’s most critical business systems. For decades, Spring has been the backbone of enterprise Java development. As organizations race to integrate AI into their products, Java teams need a way to adopt LLMs that aligns with their existing tools, security models, monitoring infrastructure, and deployment practices. Spring AI bridges that gap. It offers a unified programming model for:

  • Chat Models – interact with LLMs like OpenAI GPT, Anthropic Claude, Google Gemini, and open-source models via Ollama.
  • Embeddings – convert text into vector representations for semantic search.
  • Vector Search – store and query embeddings in databases such as PGVector, Milvus, Pinecone, and Redis.
  • RAG (Retrieval-Augmented Generation) – ground LLM responses in your proprietary data.
  • Tool Calling – enable models to invoke external APIs and services.
  • AI Agents – orchestrate multi-step reasoning and action.
  • MCP (Model Context Protocol) – standardize context exchange between AI applications and tools.
  • Enterprise AI Systems – address security, multi-tenancy, observability, and performance at scale.

Spring AI doesn’t ask you to leave the Spring universe. You define @Bean configurations for model clients, inject ChatClient into your services, use @Tool annotations to expose functions to models, and rely on Spring Boot auto-configuration for seamless integration. The result: clean, testable, maintainable Java code that runs on the JVM you already trust.

This handbook is designed as your primary reference. It covers everything from step-by-step tutorials to deep-dive architecture explorations and source code analysis. Use the roadmap below to navigate your learning journey, and let the handbook evolve with you as you progress from novice to AI engineering leader.

Why Spring AI​

Enterprise Java is not going anywhere. The Spring ecosystem remains the dominant choice for building robust, scalable backend systems. However, the AI revolution has largely been driven by Python and JavaScript frameworks. For Java teams, this creates a significant friction: adopt a new language stack, or risk being left behind.

Spring AI removes that trade-off. By providing a consistent, opinionated, and production-ready abstraction over multiple AI providers, it brings AI development into the world of Maven, Gradle, Spring Boot, and the JVM. Here’s why that matters:

  • Enterprise Java Ecosystem Integration
    Your Spring AI applications live inside the same container as your existing business logic. They share transactions, security contexts, logging, metrics, and deployment pipelines. There’s no need to maintain a separate AI microservice in Python just to call an LLM.

  • Spring Ecosystem Maturity
    Spring Boot’s auto-configuration, Spring’s dependency injection, and the vast library of Spring projects (Security, Cloud, Data, Batch) are directly available to your AI components. You get battle-tested patterns for resilience, retry, and observability out of the box.

  • Vendor Abstraction
    Spring AI decouples your application code from specific LLM providers. Switching from OpenAI to Azure OpenAI, or from a cloud model to a locally hosted Ollama model, can often be done with configuration changes alone. This portability protects you from vendor lock-in and allows you to choose the right model for each task.

  • Production-Ready Architecture
    Building a demo is easy; running AI reliably in production is hard. Spring AI addresses token usage monitoring, error handling with retry strategies, evaluation of AI responses, structured output parsing, and streaming—all with patterns familiar to experienced Spring developers.

  • Solving Real AI Development Challenges
    AI applications require new patterns: prompt templating, context management, vector store interactions, and agentic workflows. Spring AI embeds these concepts in a cohesive programming model rather than forcing developers to stitch together low-level HTTP clients and JSON parsers.

If you are a Java developer who wants to stay at the forefront of technology without abandoning the ecosystem you’ve invested years in, Spring AI is your gateway.

Spring AI Architecture Overview​

Understanding the layered architecture of Spring AI is key to using it effectively and extending it when necessary. The framework is built on a clean separation of concerns:

Application Layer
Your Spring Boot services, controllers, and business logic. You interact with AI capabilities primarily through the ChatClient interface, which is injected as a Spring bean. This layer has no direct dependency on any specific AI provider.

Spring AI Framework
This is the core of the project. It defines the abstractions (ChatModel, EmbeddingModel, VectorStore, etc.), advisors for cross-cutting concerns, memory interfaces, tool calling support, and evaluation utilities. All provider implementations plug into these abstractions.

ChatClient
The central entry point for conversational AI. It orchestrates prompt construction, advisor processing, and response handling. You build a request with ChatClient by specifying a prompt, optional tools, advisors, and memory, then retrieve a ChatResponse that contains the model’s output.

Prompt / Advisors / Memory / Tool Calling
These are the building blocks that enrich a simple prompt with enterprise requirements:

  • Prompt and PromptTemplate structure the input for the model, including system and user messages.
  • Advisors intercept and modify requests and responses; they are used for logging, content filtering, chat memory management, and retrieval augmentation.
  • Memory stores conversation history across interactions, enabling stateful conversations.
  • Tool Calling lets the model decide to invoke pre-registered Java methods (annotated with @Tool) to fetch data or perform actions.

ChatModel / EmbeddingModel
These are the portability interfaces. ChatModel abstracts language model interactions; EmbeddingModel abstracts embedding generation. Providers implement these interfaces, so your code only depends on the contract.

Provider Layer
Concrete implementations for OpenAI, Azure OpenAI, Anthropic Claude, Google Gemini, Ollama, DeepSeek, DashScope, and others. Each provider translates the Spring AI request into the provider-specific API call and maps the response back.

Why this architecture?
This layering means you can test your AI logic with a mock ChatModel, swap providers without changing business logic, and plug in custom advisors for enterprise features like PII redaction or cost tracking. It aligns perfectly with the classic Spring philosophy of programming to interfaces.

Spring AI Learning Path​

Becoming proficient with Spring AI is a journey. We recommend the following stages, each building on the previous one.

Step 1: Getting Started
Set up a Spring Boot project with the Spring AI starter, configure your first model provider, and run a simple “Hello World” chat. Understand project structure and how auto-configuration wires everything together. This stage builds confidence and demos the basic request-response loop.

Step 2: Framework Fundamentals
Dive into the core APIs: ChatClient, Prompt, PromptTemplate, ChatModel, ChatResponse, EmbeddingModel. Learn about structured output, streaming, token usage, error handling, and retry. At this point, you can build sophisticated conversational applications and understand how Spring AI manages state and context.

Step 3: RAG Development
Master retrieval-augmented generation. Understand document processing, chunking strategies, embedding pipelines, metadata filtering, hybrid search, and re-ranking. Learn to use the VectorStore abstraction to connect to various vector databases. This is where you start building AI applications that speak with your own data.

Step 4: Provider Integrations
Explore the capabilities and configurations of different model providers. Compare OpenAI, Azure OpenAI, Anthropic, Gemini, Ollama, and DeepSeek. Understand the trade-offs in latency, cost, model capabilities, and deployment models. Learn to write provider-agnostic code while still exploiting provider-specific features when necessary.

Step 5: Enterprise AI Architecture
Move beyond prototypes. Learn about security (prompt injection prevention, data isolation), multi-tenancy, observability (metrics, tracing, dashboards), performance tuning, testing strategies, and Kubernetes deployment. This stage prepares you to run Spring AI in mission-critical environments.

Step 6: Source Code Analysis
For senior engineers and architects, deep-dive into the Spring AI source code. Understand how ChatClient is implemented, how the advisor chain works, how tool calling orchestrates model interactions, and how streaming responses are handled. This knowledge enables you to contribute to the framework, debug complex issues, and build custom extensions with confidence.

Handbook Structure​

Use the sections below as your primary navigation. Each section groups related topics and provides a clear entry point for different learning goals.

Getting Started​

Description: Start here if you are new to Spring AI. This section covers installation, creating your first Spring AI application, project structure, and a quick-start guide tailored for Java developers. It ensures you have a running environment and a solid mental model before diving deeper.

Who should read it: All Java developers beginning with Spring AI.

Browser Getting Started →

Spring AI Framework​

Description: The heart of Spring AI. Explores the architecture, ChatClient, Prompt, PromptTemplate, ChatModel, ChatResponse, EmbeddingModel, advisors, memory, structured output, streaming, tool calling, evaluation, observability, retry, and token usage. This is the canonical reference for the core programming model.

Who should read it: Developers building AI-powered features; architects designing AI service layers.

Browser Spring AI Framework Guides →

RAG​

Description: Everything about retrieval-augmented generation. Covers the theory of RAG, document processing pipelines, chunking strategies, embedding generation, metadata filtering, hybrid search, re-ranking, the VectorStore API, and production considerations for RAG systems.

Who should read it: Engineers creating knowledge-base Q&A systems, document search, or any application that requires grounding LLMs in external data.

Browse Spring AI Rag Guides →

Vector Databases​

Link: /spring-ai/vector-database/

Description: Guides on integrating Spring AI with popular vector databases: PGVector, Milvus, Pinecone, Redis Vector, Qdrant, Weaviate, and OpenSearch Vector. Includes a comparison to help you choose the right store for your workload.

Who should read it: Architects evaluating vector storage; developers implementing semantic search.

Browser Vector Databases →

Providers​

Description: Detailed reference for each supported model provider: OpenAI, Azure OpenAI, Anthropic Claude, Google Gemini, Ollama, DeepSeek, and DashScope. Explains provider-specific configurations, capabilities, and how the abstraction layer gives you flexibility.

Why provider abstraction matters: It allows you to develop against a stable API while freely switching between models for cost optimization, latency requirements, or feature needs.

Who should read it: Teams selecting LLM providers; developers needing provider-specific optimization.

Browser Spring AI Providers →

Enterprise AI​

Description: Patterns and practices for deploying Spring AI in production, enterprise environments. Topics include enterprise architecture, building knowledge bases, security, multi-tenancy, observability, performance tuning, error handling, testing, and deployment strategies.

Who should read it: Enterprise architects, SREs, and senior developers responsible for production AI systems.

Browser Enterprise AI →

Tutorials​

Description: Hands-on, end-to-end tutorials that guide you through building real applications: a first chatbot, a complete RAG system, an enterprise knowledge base, an AI assistant, a document Q&A system, an AI agent, an MCP server, and deploying to Kubernetes.

Who should read it: Developers who learn by building; anyone wanting a worked example of a specific pattern.

Browser Spring AI Tutorials →

Comparisons​

Description: Objective comparisons between Spring AI and other AI frameworks: LangChain4j, LangChain (Python), Semantic Kernel, and LlamaIndex. Understand the design philosophies, strengths, and trade-offs to make informed technology choices.

Who should read it: Technical leads evaluating AI frameworks for Java projects.

Browser Spring AI Comparisons →

Source Code Analysis​

Description: Deep technical explorations of the Spring AI source code. Covers the internal design of ChatClient, Prompt, ChatModel, EmbeddingModel, advisors, tool calling, streaming, memory, structured output, VectorStore, RAG, agents, and MCP. This section turns you from a user into a contributor.

Why source code analysis matters: For senior engineers and architects, understanding the internals is essential for performance tuning, custom extensions, debugging production issues, and making informed design decisions that align with the framework’s intended usage.

Who should read it: Senior engineers, architects, and open-source contributors.

Browse Spring AI Source Code Analysis

In addition to the structured handbook, we have a growing collection of practical, in-depth articles that address real-world challenges. These guides complement the handbook and are referenced throughout.

Other recommended articles:

Each guide stands alone but also links back to the relevant handbook sections for deeper theory.

Spring AI Knowledge Map​

The following diagram visualizes the high-level organization of Spring AI knowledge domains. Use it to find your way around the handbook.

This map reflects the structure of the handbook itself. Each node leads to detailed articles and tutorials.

Who Should Read This Handbook​

This handbook is designed for a diverse audience, all united by the goal of building AI-powered applications with Spring and Java.

Java Developers​

If you write Java for a living and want to integrate AI into your applications, this handbook is your starting point. We assume Spring Boot knowledge and guide you through the AI-specific concepts with code examples that feel immediately familiar. You will learn to add intelligence to your services without leaving the JVM ecosystem.

Spring Developers​

You already know the power of Spring Boot, Spring Data, and Spring Security. This handbook shows you how Spring AI fits into that world. You’ll discover how to leverage existing Spring patterns—such as @Service, @Bean, and application.properties—to manage AI components, and how to combine Spring AI with Spring Cloud and Spring Batch for advanced use cases.

AI Engineers​

Coming from a Python background but working in a Java-centric organization? This handbook helps you translate your AI expertise into the Spring model. You’ll understand how Spring AI implements concepts like chains, retrieval, and agents in a statically typed, enterprise-grade framework. It is also a valuable reference for comparing design philosophies.

Software Architects​

Architects need to make informed decisions about system design, technology stacks, and operational requirements. The handbook provides architecture overviews, provider comparisons, enterprise patterns, and source code analysis to support your evaluations. You’ll learn how to structure AI services for scalability, security, and maintainability.

Enterprise Architects​

For those responsible for governance, compliance, and long-term technology strategy, the handbook explains how Spring AI aligns with existing Java infrastructure, security models, and monitoring tools. It addresses multi-tenancy, data isolation, cost management, and deployment on platforms like Kubernetes—essential for large-scale AI adoption.

Spring AI does not exist in isolation. To build complete AI-driven systems, consider these complementary handbooks:

  • Spring AI Alibaba Handbook
    Focuses on the Alibaba Cloud AI ecosystem, including DashScope, Qwen models, and extensions specific to the Alibaba Cloud platform. It shares the same architectural principles as Spring AI and adds provider-specific knowledge for teams using Alibaba Cloud services.

  • LLM Systems Engineering Handbook
    A broader look at engineering large language model applications, covering prompt engineering, model fine-tuning, evaluation frameworks, and deployment strategies that go beyond any single framework. Useful for engineers who want a deeper understanding of the ML side.

  • AI Agent Engineering Handbook
    An advanced guide to designing autonomous AI agents, multi-agent systems, and agentic workflows. It builds on Spring AI’s tool calling and agent support, extending into complex reasoning and planning patterns.

Together, these resources form a complete knowledge ecosystem for the modern AI engineer working in the Java/Spring world.

Summary​

Spring AI is the bridge that connects the mature, reliable Java ecosystem with the transformative potential of large language models and modern AI architectures. It empowers Java developers, Spring experts, and enterprise architects to build intelligent applications using patterns they already trust.

This handbook is designed to be your constant companion: a structured, architecture-first guide that prioritizes understanding over mere recipes. Whether you’re taking your first steps with a simple chatbot or designing a multi-tenant, RAG-powered enterprise knowledge base, the knowledge is here, organized for progressive learning and quick reference.

Mastering Spring AI is not just about using a new library. It’s about positioning yourself at the intersection of two powerful forces: the world’s most popular enterprise development platform and the AI revolution that is reshaping every industry. Start your journey here, and return often as both the handbook and the framework evolve.