Skip to main content

Spring AI Structured Output Source Code Analysis

Large language models produce text. That text is often conversational, free-form, and unpredictable in structure. Enterprise systems, however, operate on typed data—Java records, DTOs, and domain objects with well-defined schemas. The gap between a raw string and a validated instance of CustomerOrder is where Spring AI’s structured output machinery lives. This chapter dissects that machinery: the interfaces, the conversion pipeline, the schema injection mechanism, and the error recovery strategies that transform an LLM’s probabilistic generation into deterministic, type-safe Java objects. Understanding this layer is not optional for architects and senior engineers who must build reliable, production-grade AI features.

What Is Structured Output?​

An unstructured LLM response is a plain text block. It might contain the requested data, but it might also contain extra commentary, varying formats, or even hallucinations disguised as valid information. A structured output, by contrast, is a response that conforms to a predefined schema—typically JSON that maps directly to a Java class.

Why JSON and typed objects matter
JSON is the universal data interchange format, and Java’s type system offers compile-time safety. When an LLM returns a JSON payload that exactly matches a record or class, the application can deserialize it automatically, pass it to services, and store it in databases without fragile string parsing. This eliminates a whole class of runtime errors caused by ad-hoc regex extraction or manual field mapping.

Why schema-driven output improves reliability
By injecting the desired schema directly into the prompt—either as a JSON Schema definition or as a detailed description of the expected fields—the model receives a strong signal about the required output shape. Modern LLMs are remarkably good at following such instructions. This schema-driven approach reduces the variance of responses and makes the parsing layer simpler and more robust.

Why structured output is critical in enterprise Java applications
Enterprise systems are built on contracts. A REST API returns a defined DTO. A message queue expects a specific payload. A database table has typed columns. Integrating an LLM into such an environment without structured output is like calling an external service that sometimes returns XML, sometimes plain text, and occasionally a stack trace. Structured output brings AI-generated data into the same world of type safety, validation, and contract versioning that enterprise developers rely on for all other integrations.

Structured Output in Spring AI Architecture​

Structured output is a cross-cutting concern that touches several layers of the Spring AI stack. It is not a standalone module but a coordinated orchestration of schema generation, prompt enrichment, raw model invocation, and post-processing conversion.

Application Layer
The developer calls chatClient.prompt().user(...).call().entity(MyType.class). The intent is clear: “invoke the model and give me back an instance of MyType.”

ChatClient
The client is the primary entry point for structured output requests. It configures the prompt, applies advisors, and at the end of the chain triggers the conversion. It manages the type information (through generics) and delegates to the appropriate converter.

Prompt Assembly and Schema Injection
Before the prompt reaches the model, Spring AI can inject formatting instructions. These are derived from the target type’s structure—using reflection to build a JSON schema or a textual field list. This enriched prompt guides the model to produce output that matches the expected structure.

ChatModel and LLM Provider
The model remains oblivious to Java types. It receives a prompt that includes formatting instructions and returns a raw response. The model’s job is generation; the framework’s job is interpretation.

Structured Output Parser / Converter
This is the heart of the feature. After the ChatResponse arrives, the converter takes the raw content, validates that it contains valid JSON (or another structured format), deserializes it into the target type, and optionally runs bean validation. If anything fails, the converter triggers retry or fallback logic.

Typed Java Object
The final output is a fully populated, validated Java object that the application can use directly—no casting, no manual mapping, no guesswork.

Core Interfaces and Classes​

The structured output capability is realized through a small set of well-defined interfaces and classes. Understanding their responsibilities is the first step in reading the source code.

Class / InterfaceResponsibility
ChatClientEntry point for building prompts and requesting typed responses via entity()
ChatModelAbstraction for invoking the LLM; returns raw ChatResponse
ChatResponseContainer for the raw model output, including message content and metadata
StructuredOutputConverterCore interface for converting raw text into a typed Java object
BeanOutputConverterImplementation that uses Jackson to deserialize JSON into a target class, with schema generation support
JsonOutputConverterSimpler converter that returns a JsonNode or a raw map without a specific target bean
OutputParserLegacy/alternative parsing abstraction that can be used for custom parsing strategies
PromptThe request sent to the model; may contain system instructions that include the output schema
ResponseEntityInternal representation of a parsed response that carries the typed object along with the raw ChatResponse

StructuredOutputConverter is the key extension point. Its primary methods handle converting a String to a typed object and generating the format instructions that will be appended to the prompt. BeanOutputConverter is the workhorse for most enterprise use cases, leveraging Jackson’s object mapping and optionally generating JSON Schema.

Source Code Structure​

The structured output code resides primarily in the org.springframework.ai.chat.client and org.springframework.ai.converter packages.

  • org.springframework.ai.converter
    Contains StructuredOutputConverter, BeanOutputConverter, and related support classes. This package is deliberately separated from the core chat model to keep conversion logic decoupled from model invocation.

  • org.springframework.ai.chat.client
    The ChatClient implementation integrates the converter. The entity() method and its overloads are defined here, along with the internal logic that merges converter-specific format instructions into the prompt.

  • Schema generation utilities
    Some implementations include helpers that reflect on a Java class and produce a JSON Schema. These are often placed in internal utility classes to avoid leaking details to the public API.

This separation means you can study the conversion pipeline without tracing through the entire chat model stack. It also means you can write custom converters that have no dependency on Spring AI’s internal model implementations—only on the public converter interface.

Structured Output Execution Lifecycle​

The following sequence diagram illustrates the complete flow, from the application’s request to the receipt of a typed object.

Key details:

  1. Schema injection happens before the model is called. The converter is asked for format instructions, which are appended to the system prompt or user prompt.
  2. Model invocation is oblivious to Java types; it only processes the enriched prompt.
  3. Conversion is a post-processing step that operates on the raw text. If the model’s output does not match the expected schema, the converter fails, and the framework may retry (depending on configuration) or propagate the error.

Output Conversion Pipeline​

The conversion pipeline is responsible for taking the raw content from ChatResponse and producing a live Java object.

Response format expectations
The converter typically expects the model to return a JSON block. Some implementations are lenient and attempt to extract JSON from a Markdown code fence (e.g., ```json ... ```), while others require the entire response to be valid JSON.

Parsing raw text into structured data
The first step is locating the JSON substring. This may involve stripping Markdown delimiters, trimming whitespace, or even using regex to find the first JSON object or array. The goal is to isolate a string that can be passed to a JSON parser.

Mapping JSON to Java types
BeanOutputConverter uses Jackson’s ObjectMapper to deserialize the JSON into the target class. This means all standard Jackson annotations (@JsonProperty, @JsonIgnore, etc.) are respected. Nested objects, arrays, and generic types are handled transparently.

Bean conversion
For a target type like record Person(String name, int age), the converter expects a JSON object {"name": "...", "age": ...}. Missing fields that are not optional will cause deserialization failures. Additional fields are typically ignored unless Jackson is configured otherwise.

Collections and generics
The entity() method supports ParameterizedTypeReference, allowing deserialization into generic types like List<Person> or Map<String, List<Order>>. The converter resolves the actual type arguments and passes them to Jackson for correct deserialization.

Optional fields
Jackson treats Optional fields with care. If a field is absent in the JSON, the converter populates it with Optional.empty(). This aligns well with the uncertainty of LLM outputs—fields the model occasionally omits should be modeled as optional.

Schema and Prompt Design​

The quality of structured output depends heavily on how the desired schema is communicated to the model.

How schemas guide the model
When a JSON Schema (or a textual description of the expected fields) is included in the prompt, the model has a clear target. It understands the field names, types, and whether a field is required. This drastically reduces the chance of malformed responses.

Why prompt instructions matter
Even with a schema, the prompt must explicitly instruct the model to return only the JSON and no additional commentary. Without such instruction, a model might wrap the JSON in explanatory text, which complicates parsing. The format instructions generated by BeanOutputConverter typically include a directive like: “Your response must be a valid JSON object matching the following schema. Do not include any other text.”

Schema constraints
The generated schema may include constraints like required fields, types, and sometimes enums. The more precise the schema, the more reliable the output, but there is a trade-off: overly complex schemas increase prompt length and may confuse smaller models.

Type safety expectations
Java’s type system is stricter than JSON. The prompt should clarify type expectations (e.g., “age must be an integer, not a string”). The converter can partially mitigate type mismatches by configuring Jackson to coerce values, but it’s better to guide the model correctly upfront.

Response formatting rules
Formatting instructions often include an example of the expected JSON. This few-shot technique is highly effective for LLMs and is sometimes incorporated into the schema generation logic.

Relationship with ChatClient​

ChatClient is the orchestrator for structured output. Its entity() method is the primary API:

  • entity(Class<T> type) – specifies the target type and returns a ResponseEntity<T>.
  • entity(ParameterizedTypeReference<T> type) – for generic types.
  • convert(StructuredOutputConverter<T> converter) – allows passing a custom converter directly.

Internally, ChatClient holds a default converter registry. When entity() is called, the client:

  1. Resolves the appropriate StructuredOutputConverter (typically BeanOutputConverter).
  2. Calls converter.getFormatInstructions(type) to obtain schema instructions.
  3. Injects those instructions into the prompt (usually as a system message or appended to the user message).
  4. Invokes the model via ChatModel.
  5. Passes the raw response to converter.convert(responseContent).
  6. Wraps the result in a ResponseEntity.

This design keeps the conversion concern outside the core prompt and model logic, aligning with the Single Responsibility Principle.

Relationship with ChatModel​

ChatModel is unaware of structured output. It produces a ChatResponse that contains a Message with a String content. The model interface has no concept of typed return values—it simply returns what the provider sends.

This separation is intentional and architecturally sound:

  • Generation is the model’s responsibility. It takes a prompt and returns a response.
  • Conversion is the framework’s responsibility. It interprets the response.

This means you can use structured output with any ChatModel implementation—OpenAI, Ollama, Azure, or a custom adapter—without the model layer needing to know about Java types. It also means that improvements to the converter pipeline do not require changes to model adapters.

Validation and Error Handling​

LLM outputs are not guaranteed to be valid JSON, let alone to match a specific schema. The structured output pipeline must anticipate and handle failures gracefully.

Parsing failures
If the response cannot be parsed as JSON at all, the converter throws an exception. This can happen if the model ignores format instructions or the response is truncated.

Schema mismatches
The JSON might be valid but lack required fields, contain fields of wrong types, or include extra unexpected fields. Jackson throws a MismatchedInputException or similar, which the converter surfaces.

Invalid JSON
Sometimes the model returns JSON with minor syntax errors—trailing commas, unquoted keys, or extra characters. Basic lenient parsing (enabled via Jackson’s JsonParser.Feature) can handle some of these, but aggressive leniency risks accepting incorrect data.

Missing fields
Fields the model failed to include result in null or default values, depending on the Java type. For records, missing required components cause deserialization to fail entirely. This is often the desired behavior: a partial object is invalid.

Partial responses
If streaming is used, the converter must wait for the complete response. Partial JSON cannot be reliably deserialized.

Fallback strategies
Spring AI’s retry mechanism (via RetryTemplate or custom advisors) can be combined with structured output. If conversion fails, the framework can retry the entire request, possibly with a revised prompt that emphasizes the format requirements more strongly. Custom converters can implement a fallback that uses a more lenient parser or attempts to extract partial data.

Design Patterns Used​

The structured output implementation employs several classic design patterns that make it extensible and maintainable.

  • Converter Pattern – StructuredOutputConverter is a textbook converter: it transforms input (String) into output (T). Each implementation encapsulates a specific conversion strategy.
  • Strategy Pattern – Multiple converters exist (BeanOutputConverter, JsonOutputConverter), and the client selects the appropriate strategy based on the target type and configuration.
  • Template Method – Schema generation and conversion follow a fixed sequence of steps (generate format instructions, invoke model, extract JSON, deserialize, validate), with individual steps customizable in subclasses.
  • Builder Pattern – ChatClient and Prompt use builders for constructing requests with optional structured output settings.
  • Adapter Pattern – The ChatModel interface acts as an adapter between the generic conversion pipeline and specific provider APIs.
  • Dependency Injection – Converters are managed as Spring beans, allowing easy replacement and customization through the application context.

These patterns ensure that the code is open for extension but closed for modification—a hallmark of well-engineered Spring modules.

Extension Points​

Spring AI’s structured output is designed to be customized. Key extension points include:

  • Custom StructuredOutputConverter – Implement the interface to support formats other than JSON (e.g., YAML, XML) or to integrate a custom parsing library.
  • Schema generation customization – Override how format instructions are built. For instance, you might generate an OpenAPI schema instead of JSON Schema, or include domain-specific examples.
  • Parsing strategy – Swap the Jackson ObjectMapper configuration, add custom deserializers, or use an entirely different JSON library.
  • Validation behavior – Integrate Bean Validation (javax.validation) by calling a Validator on the deserialized object before returning it.
  • Error recovery – Implement a converter that, on failure, attempts to extract information using regex or partial parsing, perhaps returning a degraded but useful result.
  • Fallback conversion – If the primary converter fails, a secondary converter could attempt a different prompt or a more lenient parsing approach.

These hooks allow teams to harden the pipeline against the specific failure modes of the models they use.

Integration with Tool Calling​

Structured output and tool calling serve distinct but complementary roles.

Complementary responsibilities
Tool calling allows the model to invoke Java methods to retrieve data or perform actions. Structured output converts the model’s final response into typed objects. They can be used together: a tool might return a String that is itself a JSON serialization of a domain object, and structured output can then convert that string into a typed instance for downstream processing.

When to use structured output instead of tools
If the goal is simply to extract structured data from a natural-language request—for example, parsing a user’s message into a SearchQuery object—structured output is the right choice. It does not require the model to call an external function; it only needs to format its response correctly.

Combined usage patterns
A common pattern is to use tool calling for data retrieval (e.g., searching a database) and then use structured output to format the final answer. The agent invokes tools to gather facts, then produces a final response conforming to a FinalAnswer schema.

Orchestration considerations
Tool calls can interrupt the structured output flow. If a tool call is required, the converter must wait for the final, non-tool response. Spring AI manages this internally: the converter is only invoked on the final assistant message after all tool interactions are resolved.

Enterprise Best Practices​

When using structured output in production, adhere to these guidelines:

  • Strict schema design – Define target records with precise types. Avoid Object or overly generic fields. Use Optional for fields the model might legitimately omit.
  • Defensive parsing – Always assume the model will occasionally produce malformed JSON. Configure the converter to log raw responses before parsing for easier debugging.
  • Validation before persistence – Even after successful deserialization, run business-level validation before storing or acting on the object. A syntactically correct object may still contain nonsensical values.
  • Versioned response contracts – Treat the target class as an API contract. Introduce new optional fields without breaking existing parsers. When making breaking changes, create a new version of the schema and migrate gradually.
  • Observability and logging – Record conversion failures, schema mismatches, and retry attempts as metrics. These will become essential for tuning prompts and models over time.
  • Deterministic output handling – Even with schema instructions, model outputs can vary. Architect your system to tolerate minor variations and to flag completely unexpected outputs for review.

Performance Considerations​

Structured output adds overhead at several levels:

  • Prompt length overhead – Including a detailed JSON Schema or field description increases the prompt token count, which directly impacts latency and cost. Keep schemas concise and only include necessary fields.
  • Parsing cost – JSON deserialization is generally fast, but large object graphs or deeply nested structures can add CPU time. Jackson’s streaming API can be used for very large responses.
  • Object allocation – Each conversion creates new Java objects. In high-throughput systems, this can increase GC pressure. Consider caching converter instances (they are typically stateless) and reusing ObjectMapper.
  • Retry implications – If conversion fails and a retry is triggered, the model is invoked again, doubling latency and cost. Invest in precise format instructions to minimize retries.
  • High-throughput systems – Benchmark the full pipeline end-to-end. If latency is critical, explore using models that natively support JSON mode (like OpenAI’s response_format parameter) which can reduce the need for complex format instructions.
  • Large object graphs – For very complex schemas, consider breaking the output into multiple, simpler calls rather than asking the model to produce a massive JSON document in one shot.

Source Code Reading Guide​

To deeply understand the structured output implementation, follow this reading order:

  1. StructuredOutputConverter – Start with the interface. Understand the contract: getFormatInstructions() and convert().
  2. BeanOutputConverter – The primary implementation. Study how it generates JSON Schema, how it configures Jackson, and how it handles errors.
  3. ChatClient entity() methods – Trace how the converter is integrated into the client flow. See how format instructions are merged into the prompt.
  4. AbstractToolCallSupport and tool call handling – Understand how structured output interacts with tool calling, particularly how the final non-tool message is selected for conversion.
  5. Retry and error handling in ChatClient – Look at how conversion exceptions are handled, whether retries are supported, and how custom advisors can intervene.

Focus on the flow from entity() to the final typed object. Set breakpoints in BeanOutputConverter.convert() and observe real responses during integration tests. This hands-on tracing will solidify your mental model of the pipeline.

Structured output does not stand alone. It is deeply connected to other core components. We recommend studying these related chapters:

Summary​

Structured output transforms Spring AI from a text generator into a component that can be safely wired into the typed heart of an enterprise Java application. The source code reveals a carefully layered architecture: the model generates, the converter interprets, and the ChatClient orchestrates. By injecting schemas into prompts, the framework constrains the LLM’s natural variability. By employing robust parsing and validation, it protects downstream code from the inherent uncertainty of AI outputs.

The key architectural ideas—separation of generation and conversion, the converter pattern, schema injection, and defensive error handling—form a blueprint for building reliable AI integrations. Mastering these internals equips senior engineers to extend the pipeline, harden it for production, and reason about failures when they inevitably occur.

We recommend continuing with the ChatClient Source Code Analysis to understand how the structured output call is initiated and orchestrated, and then exploring the Tool Calling Source Code Analysis to see how the two features complement each other in agent-like workflows.