Spring AI Prompt Engineering: Best Practices and Templates

The landscape of Java development is shifting. With the release of Spring AI, Java developers no longer need to switch to Python to build Generative AI applications. However, the effectiveness of any LLM integration—whether you are using OpenAI, Azure, or Bedrock—hinges entirely on one thing: Prompt Engineering.

In the Spring ecosystem, we don’t just concatenate strings; we use the powerful Prompt and PromptTemplate abstractions. This guide serves as your definitive manual for mastering spring ai prompt engineering, moving from basic text generation to complex, structured data extraction.

The Spring AI Prompt Abstraction

#

Before diving into complex templates, we must understand the core domain objects Spring AI provides. Unlike raw HTTP calls where you send a JSON payload, Spring AI wraps the interaction in a Prompt object.

The Prompt Object

#

The org.springframework.ai.chat.prompt.Prompt class encapsulates a list of Message objects and generic ChatOptions. This follows the standard chat completion API structure used by most model providers.

import org.springframework.ai.chat.messages.UserMessage;
import org.springframework.ai.chat.prompt.Prompt;
import org.springframework.ai.chat.model.ChatModel;

@RestController
public class SimpleChatController {

    private final ChatModel chatModel;

    public SimpleChatController(ChatModel chatModel) {
        this.chatModel = chatModel;
    }

    @GetMapping("/ask")
    public String ask(@RequestParam String query) {
        // Constructing a simple Prompt
        Prompt prompt = new Prompt(new UserMessage(query));
        return chatModel.call(prompt).getResult().getOutput().getContent();
    }
}

While this works for “Hello World,” enterprise applications require dynamic data injection, strict formatting, and context management. That is where PromptTemplate comes in.

Mastering PromptTemplate

#

Hardcoding prompt strings in your Java code is an anti-pattern. It makes code difficult to read, hard to test, and impossible to maintain. Spring AI PromptTemplate allows you to parameterize your prompts, similar to how Liquibase handles SQL or how Thymeleaf handles HTML.

1. Inline Templates

#

For short, simple dynamic prompts, you can define templates inline.

import org.springframework.ai.chat.prompt.PromptTemplate;

public String generateEmail(String name, String topic) {
    String templateText = "Write a professional email to {name} regarding {topic}. Keep it under 100 words.";
    
    PromptTemplate template = new PromptTemplate(templateText);
    
    // Map the variables
    Map<String, Object> model = Map.of(
        "name", name,
        "topic", topic
    );
    
    return template.create(model).getContents(); // Returns the rendered string
}

2. Resource-Based Templates (Best Practice)

#

For complex prompts (especially those involving “System” instructions or few-shot examples), you should store your prompts in src/main/resources/prompts/. Spring AI supports the Resource interface natively.

Create a file src/main/resources/prompts/code-review.st:

You are a Senior Java Developer acting as a code reviewer.
Review the following code snippet for:
1. Potential memory leaks
2. Spring Boot best practices
3. Security vulnerabilities (OWASP Top 10)

Code:
{codeSnippet}

Inject and use it in your Service:

@Service
public class CodeReviewService {

    private final ChatModel chatModel;
    
    @Value("classpath:prompts/code-review.st")
    private Resource codeReviewPrompt;

    public CodeReviewService(ChatModel chatModel) {
        this.chatModel = chatModel;
    }

    public String review(String javaCode) {
        PromptTemplate template = new PromptTemplate(codeReviewPrompt);
        Prompt prompt = template.create(Map.of("codeSnippet", javaCode));
        
        return chatModel.call(prompt).getResult().getOutput().getContent();
    }
}

This separation of concerns allows you to tweak your prompt wording without recompiling your Java code.

Structured Output: Getting Objects, Not Strings

#

One of the biggest challenges in Prompt Engineering is getting the LLM to return valid JSON that maps to your Java POJOs. Spring AI solves this with the BeanOutputConverter.

If you ask an LLM for “a list of movies,” it might give you a numbered list, a paragraph, or markdown. We want a List<Movie>.

The BeanOutputConverter Pattern

#

1. Define your POJO (Record):

   public record MovieRecommendation(String title, int year, String reasoning) {}
   

2. Integrate Converter into the Prompt:

   @GetMapping("/recommend")
   public List<MovieRecommendation> getRecommendations(@RequestParam String genre) {
   
       BeanOutputConverter<List<MovieRecommendation>> converter = 
           new BeanOutputConverter<>(new ParameterizedTypeReference<List<MovieRecommendation>>() {});
   
       String promptText = """
           Recommend 5 movies in the {genre} genre.
           {format}
           """;
   
       PromptTemplate template = new PromptTemplate(promptText);
   
       Map<String, Object> params = Map.of(
           "genre", genre,
           "format", converter.getFormat() // This injects the schema instruction
       );
   
       Prompt prompt = template.create(params);
       String responseContent = chatModel.call(prompt).getResult().getOutput().getContent();
   
       return converter.convert(responseContent);
   }
   

Why this works: The converter.getFormat() method automatically generates a strict instruction string (often containing the JSON schema) and appends it to your prompt. The LLM reads this and outputs exact JSON, which Spring then deserializes.

Advanced Prompting Strategies

#

To get the most out of Spring AI, you need to apply standard Prompt Engineering techniques within the framework’s architecture.

1. System Message (Persona) Pattern

#

Setting the context is crucial. In Spring AI, we distinguish between SystemMessage (instruction) and UserMessage (input).

String systemText = "You are a helpful customer support agent for a bank. " +
                    "You answer only in polite, professional English. " +
                    "If you don't know the answer, say 'I will connect you to a human agent'.";

SystemMessage systemMessage = new SystemMessage(systemText);
UserMessage userMessage = new UserMessage("My account is locked.");

Prompt prompt = new Prompt(List.of(systemMessage, userMessage));

Tip: Always keep the System Message static and secure to prevent Prompt Injection.

2. Few-Shot Prompting

#

Providing examples significantly improves model accuracy.

Template (classification.st):

Classify the sentiment of the following reviews.

Examples:
Input: "The UI is terrible and laggy."
Output: NEGATIVE
Input: "Great service, highly recommended!"
Output: POSITIVE

Review to classify: {review}
Output:

3. Chain-of-Thought (CoT)

#

For complex logic, force the model to explain its steps. This reduces hallucinations.

Template (math-tutor.st):

Solve the following physics problem.
Before giving the final answer, break down the problem into steps:
1. Identify known variables.
2. Select the correct formula.
3. Show the calculation.

Problem: {problem}

Catalog of Spring AI Prompt Templates

#

Here are ready-to-use templates for common backend scenarios. Copy these into your project.

Template 1: SQL Generator

#

Use Case: Natural language to SQL for dashboards (read-only).

You are a PostgreSQL expert. Convert the following natural language request into a valid SQL query.
Table Schema:
- users (id, name, email, signup_date)
- orders (id, user_id, amount, status, created_at)

Rules:
1. Return ONLY the SQL query. No markdown, no explanation.
2. Use standard SQL-92 syntax.
3. Do not assume columns that are not listed.

Request: {request}

Template 2: Unit Test Generator

#

Use Case: Generating boilerplate tests for legacy code.

Generate a JUnit 5 test class using Mockito for the following Java class.
Ensure you cover:
1. The happy path.
2. Edge cases (null inputs).
3. Exception handling.

Class Source:
{javaClassSource}

Template 3: Data Sanitization / PII Redaction

#

Use Case: Cleaning logs or user input before storage.

Analyze the following text and replace all Personally Identifiable Information (PII) with placeholders like [EMAIL], [PHONE], [NAME].

Input: {rawInput}
Redacted Output:

Prompt Engineering Best Practices in Spring

#

When building production-grade AI applications with Spring, adhere to these operational standards.

1. Manage Context Window

#

LLMs have token limits. Sending a 50-page PDF into a PromptTemplate will crash your application or cost a fortune.

• Solution: Use Spring AI’s TokenCounter (if available for your specific model) or generic estimators to truncate input before creating the prompt.
• RAG: Only inject relevant chunks of data (retrieved from a Vector Database) into the prompt context, not the whole database.

2. Parameter Validation

#

Never pass raw user input directly into a template without validation. While PromptTemplate prevents structure breakage, it doesn’t prevent “Jailbreaking” (e.g., “Ignore previous instructions”).

• Defense: Validate the {input} variable length and content type before calling template.create().

3. Temperature Control

#

The Prompt object accepts ChatOptions.

• For Creative Writing (Marketing emails): High temperature (0.7 - 0.9).
• For Data Extraction/Code: Low temperature (0.0 - 0.2).

OpenAiChatOptions options = OpenAiChatOptions.builder()
    .withTemperature(0.1f) // Deterministic
    .build();

Prompt prompt = new Prompt(new UserMessage(text), options);

4. Handling Hallucinations

#

When using Spring AI for factual data, instruct the model explicitly on what to do if it lacks information.

• Bad Prompt: “Who is the CEO of Company X?”
• Good Prompt: “Who is the CEO of Company X? If you do not have this information in your internal knowledge base, reply strictly with ‘UNKNOWN’.”

Conclusion

#

Prompt Engineering in Spring AI is about bringing the discipline of software engineering to the chaotic world of Large Language Models. By utilizing PromptTemplate for maintainability, BeanOutputConverter for type safety, and structured architectural patterns, you can build resilient AI-powered Java applications.

As we continue to explore the Spring AI ecosystem, our next articles will cover RAG (Retrieval Augmented Generation) implementation and Function Calling, where prompts can trigger actual Java methods.

Key Takeaways:

1. Avoid string concatenation; use PromptTemplate.
2. Store prompts as external resources (classpath:prompts/*.st).
3. Use BeanOutputConverter to map AI responses to Java Records.
4. Always define a System Message persona.

Start refactoring your AI calls today—your future self (and your code reviewers) will thank you.

Stay tuned to Spring DevPro for the next installment: “Spring AI RAG: Connecting your Data to LLMs”.