AI assistant with Spring AI (Part I)

In a new translation from the team Spring IO shows the process of integrating AI into the well-known Spring Petclinic application.

In the article, the author shares his step-by-step experience of implementing Spring AI to make the application more interactive.

Introduction

In this two-part article I will talk about the modifications I made to the project Spring Petclinic to integrate an AI assistant that allows users to interact with the application in natural language.

Introduction to Spring Petclinic

Spring Petclinic serves as the primary reference application in the Spring ecosystem. According to GitHub, the repository was created on January 9, 2013. Since then, it has become a model application for writing simple, user-friendly code using Spring Boot. As of this writing, it has received over 7,600 stars and 23,000 forks.

The application implements a veterinary clinic management system for pets. In the application, users can perform several actions:

• Getting a list of pet owners

• Adding a new owner

• Adding a pet to an owner

• Documenting a pet-specific visit

• Obtaining a list of veterinarians in the clinic

• Simulating a server-side error

Although the application is simple and straightforward, it effectively demonstrates the usability of Spring Boot application development.

Additionally, the Spring team continually updates the application to support the latest versions of the Spring Framework and Spring Boot.

Technologies used

Spring Petclinic is developed using Spring Boot version 3.3 at the time of this publication.

Frontend UI

The frontend is built using Thymeleaf. The Thymeleaf template engine makes it easy to integrate server-side API calls directly into your HTML code, making it easy to understand. Below is the code that gets the list of veterinarians:

<table id="vets" class="table table-striped">
  <thead>
  <tr>
    <th>Name</th>
    <th>Specialties</th>
  </tr>
  </thead>
  <tbody>
  <tr th:each="vet : ${listVets}">
    <td th:text="${vet.firstName + ' ' + vet.lastName}"></td>
    <td><span th:each="specialty : ${vet.specialties}"
              th:text="${specialty.name + ' '}"/> <span
      th:if="${vet.nrOfSpecialties == 0}">none</span></td>
  </tr>
  </tbody>
</table>

The key line here is ${listVets}which references a model in the Spring backend containing the data to be populated. Below is the relevant code block from Spring @Controllerwhich fills this model:

private String addPaginationModel(int page, Page<Vet> paginated, Model model) {
	List<Vet> listVets = paginated.getContent();
	model.addAttribute("currentPage", page);
	model.addAttribute("totalPages", paginated.getTotalPages());
	model.addAttribute("totalItems", paginated.getTotalElements());
	model.addAttribute("listVets", listVets);
	return "vets/vetList";
}

Spring Data JPA

Petclinic interacts with the database using Java Persistence API (JPA). The project supports H2, PostgreSQL or MySQL, depending on the selected profile. Communication with the database is carried out through interfaces @Repositorysuch as OwnerRepository. Here is an example of one of the JPA requests inside the interface:

/**
* Returns all the owners from data store
**/
@Query("SELECT owner FROM Owner owner")
@Transactional(readOnly = true)
Page<Owner> findAll(Pageable pageable);

JPA greatly simplifies your code by automatically implementing standard queries for your methods based on naming conventions. It also allows you to specify a JPQL query using an annotation @Querywhen necessary.

Hello Spring AI

Spring AI is one of the most exciting new projects to come out of the Spring ecosystem in a while. It allows you to interact with popular Large Language Models (LLMs) using familiar Spring paradigms and techniques. Similar to how Spring Data provides an abstraction that allows you to write code once by delegating the implementation to the provided dependency spring-boot-starter and property configuration, Spring AI offers a similar approach for LLM. You write your code once in the interface, and @Bean is injected at runtime for your specific implementation.

Spring AI supports all major large language models, including OpenAI, Azure OpenAI, Google Gemini, Amazon Bedrock, and many others.

Considerations for Implementing AI in Spring Petclinic

Spring Petclinic has been around for over 10 years and was not originally designed with AI in mind. This project is a classic candidate for testing AI integration into legacy code. In the process of adding an AI assistant to Spring Petclinic, I had to consider several important factors.

Selecting an API model

The first step was to determine the type of API I would like to implement. Spring AI offers a variety of capabilities including support for chat, image recognition and generation, audio transcription, text-to-speech, and more. For Spring Petclinic, the familiar “chatbot” interface was most suitable. This will allow clinic staff to communicate with the system in natural language, simplifying their interaction instead of navigating through UI tabs and forms. I'll also need embedding capabilities, which will be used for Retrieval-Augmented Generation (RAG) later in the article.

Possible interactions with the AI ​​assistant may include:

• How can you help me?

• Please list the owners who come to our clinic.

• Which veterinarians specialize in radiology?

• Is there a pet owner named Betty?

• Which owners have dogs?

• Add a dog for Betty: her name is Moopsy.

These examples illustrate the range of requests that AI can handle. The advantage of LLMs lies in their ability to understand natural language and provide meaningful answers.

Selecting a Large Language Model Provider

The tech world is currently experiencing a veritable gold rush with large language models (LLMs) coming out every few days, offering enhanced capabilities, larger context windows, and cutting-edge features like improved reasoning logic.

Some of the popular LLMs include:

• OpenAI and its implementation on Azure, Azure OpenAI

• Google Gemini

• Amazon Bedrock, a managed AWS service that can run a variety of LLMs, including Anthropic and Titan

• Llama 3.1, as well as many other open source models available through Hugging Face

For our Petclinic app, I needed a model that handles chat capabilities well, can be tailored to the specific needs of my app, and supports function calls (more on that later!).

One of the big benefits of Spring AI is the ease of performing A/B testing with different LLMs. You simply change the dependency and update a few properties. I tested several models, including Llama 3.1, which I ran locally. Ultimately, I concluded that OpenAI remains a leader in this field because it provides the most natural and fluid interactions while avoiding the common problems faced by other LLMs.

Here's a simple example: when greeting an OpenAI-based model, the response is:

Great. Exactly what I wanted. Simple, concise, professional and user friendly.

Here is the result using Llama 3.1:

You get the point. He's just not at that level yet.

Installing your desired LLM provider is easy – just install its dependency in pom.xml (or build.gradle) and provide the required configuration properties in application.yaml or application.properties:

<dependency>
    <groupId>org.springframework.ai</groupId>
    <artifactId>spring-ai-azure-openai-spring-boot-starter</artifactId>
</dependency>

Here I've chosen Azure's OpenAI implementation, but I could easily switch to Sam Altman's OpenAI by changing the dependency:

<dependency>
		<groupId>org.springframework.ai</groupId>
		<artifactId>spring-ai-openai-spring-boot-starter</artifactId>
</dependency>

Since I'm using a public LLM provider, I need to provide a URL and API key to access the LLM. This can be configured in `application.yaml`:

spring:
  ai:
    #These parameters apply when using the spring-ai-azure-openai-spring-boot-starter dependency:
    azure:
      openai:
        api-key: "the-api-key"
        endpoint: "https://the-url/"
        chat:
          options:
             deployment-name: "gpt-4o"
    #These parameters apply when using the spring-ai-openai-spring-boot-starter dependency:
    openai:
      api-key: ""
      endpoint: ""
      chat:
        options:
           deployment-name: "gpt-4o"

Let's start coding!

Our goal is to create a WhatsApp/iMessage style chat client that integrates with Petclinic's existing Spring UI. The frontend UI will call the backend API endpoint, which takes a string as input and returns a string. The dialogue will be open to any user questions, and if we are unable to assist with a specific request, we will provide an appropriate response.

Creating a ChatClient

Here is the implementation of the chat endpoint in the class PetclinicChatClient:

@PostMapping("/chatclient")
  public String exchange(@RequestBody String query) {
	  //All chatbot messages go through this endpoint and are passed to the LLM
	  return
	  this.chatClient
	  .prompt()
      .user(
          u ->
              u.text(query)
              )
      .call()
      .content();
  }

The API takes a string request and passes it to Spring AI ChatClient as custom text. ChatClient is a Spring Bean provided by Spring AI that manages sending custom text to LLM and returns results to content().

All Spring AI code operates within a specific framework @Profile called openai. Additional class PetclinicDisabledChatClient runs when using the default profile or any other profile. This disabled profile simply returns a message indicating that chat is unavailable.

Our implementation basically delegates responsibility ChatClient. But how do we create the bean itself? ChatClient? There are several customizable options that can affect the user experience. Let's look at them one by one and examine their impact on the final application.

Simple ChatClient

Here is a minimal, unmodified bean definition ChatClient:

public PetclinicChatClient(ChatClient.Builder builder) {
		this.chatClient = builder.build();
}

Here we are simply asking for an instance ChatClient from the builder, based on the currently available Spring AI starter in the dependencies. Although this setup works, our chat client knows nothing about the Petclinic domain or its services:

He is certainly polite, but he lacks understanding of our business domain. He also seems to be suffering from severe amnesia – he can't even remember my name from the previous message!

As I was looking through this article, I realized I wasn't following council my good friend and colleague Josh Long. I must be more polite to our new AI overlords!

You may be used to ChatGPT's excellent memory, which makes it talkative. However, in reality, LLM APIs are completely static and do not store any past messages you send. That's why the bot forgot my name so quickly.

You may be wondering how ChatGPT preserves the context of a conversation. The answer is simple: ChatGPT sends past messages as content along with each new message. Every time you send a new message, it includes previous conversations so the model can reference them. Although this may seem wasteful, this is how the system works. This is also the reason why large token windows are becoming increasingly important – users expect to return to conversations from days past and pick up where they left off.

ChatClient with better memory

Let's implement a similar “chat memory” feature in our application. Luckily, Spring AI provides an Advisor out of the box for this. You can think of advisors as hooks that run before the LLM is called. It's useful to think of them as reminiscent of aspect-oriented programming tips, even if they aren't implemented that way.

Here is our updated code:

 public PetclinicChatClient(ChatClient.Builder builder, ChatMemory chatMemory) {
	// @formatter:off
	this.chatClient = builder
			.defaultAdvisors(
					// Chat memory helps us keep context when using the chatbot for up to 10 previous messages.
					new MessageChatMemoryAdvisor(chatMemory, DEFAULT_CHAT_MEMORY_CONVERSATION_ID, 10), // CHAT MEMORY
					new SimpleLoggerAdvisor()
					)
			.build();
  }

In this updated code we have added MessageChatMemoryAdvisorwhich automatically associates the last 10 messages with any new outgoing message, helping LLMs understand context.

We've also included a ready-made SimpleLoggerAdvisorwhich logs requests and responses to and from the LLM.

Result:

Our new chatbot has a significantly better memory!

However, he still doesn't quite understand what we're doing here:

This answer is not bad for a general LLM with world knowledge. However, our clinic is very domain specific, with specific use cases. Additionally, our chatbot should focus solely on helping us with our clinic.

For example, he should not try to answer a question like:

If we allow our chatbot to answer any questions, users can start using it as a free alternative to services like ChatGPT to access more advanced models like GPT-4. Obviously, we need to train our LLM to “mimic” a specific service provider. Our LLM should focus solely on helping with Spring Petclinic: he should know about veterinarians, owners, pets and visits – and nothing else.

ChatClient associated with a specific domain

Spring AI offers a solution for this too. Most LLMs distinguish between user text (the chat messages we send) and system text, which is general text that instructs the LLM to function in a certain way. Let's add system text to our chat client:

public PetclinicChatClient(ChatClient.Builder builder, ChatMemory chatMemory) {
	// @formatter:off
	this.chatClient = builder
			.defaultSystem("""
You are a friendly AI assistant designed to help with the management of a veterinarian pet clinic called Spring Petclinic.
Your job is to answer questions about the existing veterinarians and to perform actions on the user's behalf, mainly around
veterinarians, pet owners, their pets and their owner's visits.
You are required to answer an a professional manner. If you don't know the answer, politely tell the user
you don't know the answer, then ask the user a followup qusetion to try and clarify the question they are asking.
If you do know the answer, provide the answer but do not provide any additional helpful followup questions.
When dealing with vets, if the user is unsure about the returned results, explain that there may be additional data that was not returned.
Only if the user is asking about the total number of all vets, answer that there are a lot and ask for some additional criteria. For owners, pets or visits - answer the correct data.
			      		""")
			.defaultAdvisors(
					// Chat memory helps us keep context when using the chatbot for up to 10 previous messages.
					new MessageChatMemoryAdvisor(chatMemory, DEFAULT_CHAT_MEMORY_CONVERSATION_ID, 10), // CHAT MEMORY
					new LoggingAdvisor()
					)
			.build();
}

This is quite a large default system prompt! But believe me, it is necessary. In reality, it's probably not enough, and as the system is used more often I'll probably have to add more context. The prompt engineering process involves designing and optimizing input prompts to elicit specific, precise responses for a given use case.

LLMs are quite talkative: they like to respond in natural language. This trend can make it difficult to receive machine-to-machine responses in formats such as JSON. To solve this problem, Spring AI offers a set of features dedicated to structured inference known as Structured Output Converter. The Spring team had to determine best practices for engineering prompts to ensure that the LLM responded without unnecessary chattiness. Here is an example from MapOutputConverter in Spring AI:

@Override
public String getFormat() {
	String raw = """
			Your response should be in JSON format.
			The data structure for the JSON should match this Java class: %s
			Do not include any explanations, only provide a RFC8259 compliant JSON response following this format without deviation.
			Remove the ```json markdown surrounding the output including the trailing "```".
			""";
	return String.format(raw, HashMap.class.getName());
}

Whenever a JSON response is required from the LLM, Spring AI appends this string to the request, urging the LLM to comply.

There have been positive developments in this area recently, especially with OpenAI's Structured Outputs initiative (link from the editors of Spring IO: https://openai.com/index/introducing-structured-outputs-in-the-api/). As often happens with such achievements, Spring AI received them enthusiastically.

Now let's get back to our chatbot – let's see how it works!

This is a significant improvement! We now have a chatbot tuned to our domain, focused on our specific use cases, remembering the last 10 messages, not providing any irrelevant information from the outside, and avoiding hallucinations. Additionally, our logs output the calls we make in LLM, making debugging much easier.

2024-09-21T21:55:08.888+03:00 DEBUG 85824 --- [nio-8080-exec-5] o.s.a.c.c.advisor.SimpleLoggerAdvisor    : request: AdvisedRequest[chatModel=org.springframework.ai.azure.openai.AzureOpenAiChatModel@5cdd90c4, userText="Hi! My name is Oded.", systemText=You are a friendly AI assistant designed to help with the management of a veterinarian pet clinic called Spring Petclinic.
Your job is to answer questions about the existing veterinarians and to perform actions on the user's behalf, mainly around
veterinarians, pet owners, their pets and their owner's visits.
You are required to answer an a professional manner. If you don't know the answer, politely tell the user
you don't know the answer, then ask the user a followup qusetion to try and clarify the question they are asking.
If you do know the answer, provide the answer but do not provide any additional helpful followup questions.
When dealing with vets, if the user is unsure about the returned results, explain that there may be additional data that was not returned.
Only if the user is asking about the total number of all vets, answer that there are a lot and ask for some additional criteria. For owners, pets or visits - answer the correct data.
, chatOptions=org.springframework.ai.azure.openai.AzureOpenAiChatOptions@c4c74d4, media=[], functionNames=[], functionCallbacks=[], messages=[], userParams={}, systemParams={}, advisors=[org.springframework.ai.chat.client.advisor.observation.ObservableRequestResponseAdvisor@1e561f7, org.springframework.ai.chat.client.advisor.observation.ObservableRequestResponseAdvisor@79348b22], advisorParams={}]
2024-09-21T21:55:10.594+03:00 DEBUG 85824 --- [nio-8080-exec-5] o.s.a.c.c.advisor.SimpleLoggerAdvisor    : response: {"result":{"metadata":{"contentFilterMetadata":{"sexual":{"severity":"safe","filtered":false},"violence":{"severity":"safe","filtered":false},"hate":{"severity":"safe","filtered":false},"selfHarm":{"severity":"safe","filtered":false},"profanity":null,"customBlocklists":null,"error":null,"protectedMaterialText":null,"protectedMaterialCode":null},"finishReason":"stop"},"output":{"messageType":"ASSISTANT","metadata":{"finishReason":"stop","choiceIndex":0,"id":"chatcmpl-A9zY6UlOdkTCrFVga9hbzT0LRRDO4","messageType":"ASSISTANT"},"toolCalls":[],"content":"Hello, Oded! How can I assist you today at Spring Petclinic?"}},"metadata":{"id":"chatcmpl-A9zY6UlOdkTCrFVga9hbzT0LRRDO4","model":"gpt-4o-2024-05-13","rateLimit":{"requestsLimit":0,"requestsRemaining":0,"requestsReset":0.0,"tokensRemaining":0,"tokensLimit":0,"tokensReset":0.0},"usage":{"promptTokens":633,"generationTokens":17,"totalTokens":650},"promptMetadata":[{"contentFilterMetadata":{"sexual":null,"violence":null,"hate":null,"selfHarm":null,"profanity":null,"customBlocklists":null,"error":null,"jailbreak":null,"indirectAttack":null},"promptIndex":0}],"empty":false},"results":[{"metadata":{"contentFilterMetadata":{"sexual":{"severity":"safe","filtered":false},"violence":{"severity":"safe","filtered":false},"hate":{"severity":"safe","filtered":false},"selfHarm":{"severity":"safe","filtered":false},"profanity":null,"customBlocklists":null,"error":null,"protectedMaterialText":null,"protectedMaterialCode":null},"finishReason":"stop"},"output":{"messageType":"ASSISTANT","metadata":{"finishReason":"stop","choiceIndex":0,"id":"chatcmpl-A9zY6UlOdkTCrFVga9hbzT0LRRDO4","messageType":"ASSISTANT"},"toolCalls":[],"content":"Hello, Oded! How can I assist you today at Spring Petclinic?"}}]}

Defining Core Functionality

Our chatbot works as planned, but currently lacks knowledge about the data in our application. Let's focus on the main features that Spring Petclinic supports and compare them to the features we can enable using Spring AI:

List of owners

On the Owners tab, we can search for an owner by last name or simply list all owners. We can get detailed information about each owner, including their first and last name, as well as their pets and their types:

Adding an owner

The application allows you to add a new owner by providing the necessary parameters dictated by the system. The owner must have a first name, last name, address and a 10-digit telephone number.

Adding a pet to an existing owner

An owner can have several pets. Pet types are limited to the following: cat, dog, lizard, snake, bird or hamster.

Veterinarians

The Veterinarians tab displays available veterinarians in a paginated view, along with their specialties. There is currently no search option in this tab. While in the main branch main Spring Petclinic has several veterinarians represented, I generated hundreds of dummy veterinarians in the thread spring-aito simulate an application processing a significant amount of data. Later we'll look at how we can use retrieval augmented generation (RAG) to manage such large data sets.

These are the basic operations that we can perform in the system. We have mapped our application to its basic functionality and would like OpenAI to be able to interpret natural language queries corresponding to these operations.

Calling Functions with Spring AI

In the previous section we described four different functions. Now let's compare them to functions that we can use in Spring AI by specifying specific beans java.util.function.Function.

List of owners

Next function java.util.function.Function is responsible for obtaining the list of owners in Spring Petclinic:

@Configuration
@Profile("openai")
class AIFunctionConfiguration {

	// The @Description annotation helps the model understand when to call the function
	@Bean
	@Description("List the owners that the pet clinic has")
	public Function<OwnerRequest, OwnersResponse> listOwners(AIDataProvider petclinicAiProvider) {
		return request -> {
			return petclinicAiProvider.getAllOwners();
		};
	}
}
record OwnerRequest(Owner owner) {
};

record OwnersResponse(List<Owner> owners) {
};

• We create a class @Configuration in profile openaiwhere we register standard Spring @Bean.

• Bean should return java.util.function.Function.

• We use annotation @Description from Spring to explain what this function does. Notably, Spring AI will pass this description to the LLM to help it determine when to call that particular function.

• The function accepts a record OwnerRequestwhich contains an existing entity class Owner from Spring Petclinic. This demonstrates how Spring AI can use components you've already developed in your application without having to do a complete rewrite. Comment from the Spring IO team: This request does not require an Owner. The author gave an example of a function that actually does not need arguments.

• OpenAI will decide when to call the function with the JSON object representing the record OwnerRequest. Spring AI will automatically convert this JSON to an object OwnerRequest and execute the function. After receiving the response, Spring AI converts the received record OwnerResponse (which contains List<Owner>) back to JSON format for OpenAI processing. When OpenAI receives the response, it will generate a natural language response for the user.

• The function calls the bean AIDataProvider with annotation @Servicewhich implements the actual logic. In our simple use case, the function simply requests data using JPA:

  public OwnersResponse getAllOwners() {
	  Pageable pageable = PageRequest.of(0, 100);
	  Page<Owner> ownerPage = ownerRepository.findAll(pageable);
	  return new OwnersResponse(ownerPage.getContent());
  }

• The existing legacy Spring Petclinic code returns paginated data to keep the response size manageable and easy to process for paginated presentation in the UI. In our case, we expect that the total number of owners will be relatively small, and OpenAI will be able to process such traffic in a single request. Therefore we return the first 100 owners in a single JPA request.

  You might think that this approach is suboptimal, and for a real application you would be right. If there was a large amount of data, this method would be ineffective – we would probably have more than 100 owners in the system. For such scenarios, we would need to implement a different pattern, which we will cover in the function listVets. However, for our demo use case, we will assume that our system contains less than 100 owners.

Let's reproduce a real example along with SimpleLoggerAdvisorto see what's going on behind the scenes:

What happened here? Let's look at the output from the log SimpleLoggerAdvisor for research:

request: 
AdvisedRequest[chatModel=org.springframework.ai.azure.openai.AzureOpenAiChatModel@18e69455, 
userText=
"List the owners that are called Betty.", 
systemText=You are a friendly AI assistant designed to help with the management of a veterinarian pet clinic called Spring Petclinic.
Your job...
chatOptions=org.springframework.ai.azure.openai.AzureOpenAiChatOptions@3d6f2674, 
media=[], 
functionNames=[], 
functionCallbacks=[], 
messages=[UserMessage{content=""Hi there!"", 
properties={messageType=USER}, 
messageType=USER}, 
AssistantMessage [messageType=ASSISTANT, toolCalls=[], 
textContent=Hello! How can I assist you today at Spring Petclinic?, 
metadata={choiceIndex=0, finishReason=stop, id=chatcmpl-A99D20Ql0HbrpxYc0LIkWZZLVIAKv, 
messageType=ASSISTANT}]], 
userParams={}, systemParams={}, advisors=[org.springframework.ai.chat.client.advisor.observation.ObservableRequestResponseAdvisor@1d04fb8f, 
org.springframework.ai.chat.client.advisor.observation.ObservableRequestResponseAdvisor@2fab47ce], advisorParams={}]

The request contains data of interest to us about what is being sent to the LLM, including user text, historical messages, an identifier representing the current chat session, a list of advisors to run, and system text.

You may be wondering where the functions are in the query captured above. Functions are not explicitly fixed, they are encapsulated in the content AzureOpenAiChatOptions. Examining an object in debug mode reveals a list of functions available to the model:

OpenAI will process the request, determine that it needs data from the owner list, and return a JSON response to Spring AI requesting additional information from the function listOwners. Spring AI will then call this function using the provided object OwnersRequest from OpenAI, and will send a response back to OpenAI, maintaining a conversation ID to aid in session continuity over a static connection. OpenAI will generate a final answer based on the additional data provided. Let's look at this response as it appears in the log:

response: {
  "result": {
    "metadata": {
      "finishReason": "stop",
      "contentFilterMetadata": {
        "sexual": {
          "severity": "safe",
          "filtered": false
        },
        "violence": {
          "severity": "safe",
          "filtered": false
        },
        "hate": {
          "severity": "safe",
          "filtered": false
        },
        "selfHarm": {
          "severity": "safe",
          "filtered": false
        },
        "profanity": null,
        "customBlocklists": null,
        "error": null,
        "protectedMaterialText": null,
        "protectedMaterialCode": null
      }
    },
    "output": {
      "messageType": "ASSISTANT",
      "metadata": {
        "choiceIndex": 0,
        "finishReason": "stop",
        "id": "chatcmpl-A9oKTs6162OTut1rkSKPH1hE2R08Y",
        "messageType": "ASSISTANT"
      },
      "toolCalls": [],
      "content": "The owner named Betty in our records is:\n\n- **Betty Davis**\n  - **Address:** 638 Cardinal Ave., Sun Prairie\n  - **Telephone:** 608-555-1749\n  - **Pet:** Basil (Hamster), born on 2012-08-06\n\nIf you need any more details or further assistance, please let me know!"
    }
  },
  ...
  ]
}

We see the answer itself in the section content. Much of the returned JSON consists of metadata – such as content filters, the model used, the chat session ID in the response, the number of tokens involved, how the response ended, and more.

This illustrates how the system works from start to finish: it starts in your browser, reaches the Spring backend, and involves B2B type communication between Spring AI and LLM until the response is sent back to the JavaScript that initiated the call.

Now let's look at the remaining three functions.

Adding a pet to the owner

Method addPetToOwner is especially interesting because it demonstrates the power of calling model functions.

When a user wants to add a pet to an owner, they should not be expected to enter a pet type ID. Instead, they will likely say that the pet is a “dog,” rather than simply providing a numeric identifier such as “2.”

To help the LLM determine the correct type of pet, I used the annotation @Descriptionto provide hints about our requirements. Since our veterinary clinic only works with six types of pets, this approach is manageable and effective:

@Bean
@Description("Add a pet with the specified petTypeId, " + "to an owner identified by the ownerId. "
		+ "The allowed Pet types IDs are only: " + "1 - cat" + "2 - dog" + "3 - lizard" + "4 - snake" + "5 - bird"
		+ "6 - hamster")
public Function<AddPetRequest, AddedPetResponse> addPetToOwner(AIDataProvider petclinicAiProvider) {
	return request -> {
		return petclinicAiProvider.addPetToOwner(request);
	};
}

Record AddPetRequest includes the pet type in free text, reflecting how the user would typically provide it, along with the full entity Pet and reference ownerId.

record AddPetRequest(Pet pet, String petType, Integer ownerId) {
};
record AddedPetResponse(Owner owner) {
};

Here's the business implementation: We get the owner by his ID, then add the new pet to his existing pet list.

public AddedPetResponse addPetToOwner(AddPetRequest request) {
	Owner owner = ownerRepository.findById(request.ownerId());
	owner.addPet(request.pet());
	this.ownerRepository.save(owner);
	return new AddedPetResponse(owner);
}

When debugging the application for this case, I noticed an interesting behavior: in some cases the entity Pet in the request has already been pre-populated with the correct pet type ID and name.

I also noticed that I didn't actually use the line petType in your business implementation. Is it possible that Spring AI just “understood” the correct name match PetType the correct identifier yourself?

To test this I removed petType from the request object and simplified @Description:

@Bean
@Description("Add a pet with the specified petTypeId, to an owner identified by the ownerId.")
public Function<AddPetRequest, AddedPetResponse> addPetToOwner(AIDataProvider petclinicAiProvider) {
	return request -> {
		return petclinicAiProvider.addPetToOwner(request);
	};
}

record AddPetRequest(Pet pet, Integer ownerId) {
};
record AddedPetResponse(Owner owner) {
};

I found that in most of the prompts, LLM figured out (surprisingly) how to do the matching. In the end, I kept the original description in the PR because I noticed some edge cases where LLM was having difficulty and couldn't make a correlation.

Still, even for 80% of the use cases it was very impressive. It's things like this that make Spring AI and LLM almost magical. The interaction between Spring AI and OpenAI helped to understand that PetType in the abstract @Entity class Pet needs to match the string “lizard” with the corresponding ID value in the database. This seamless integration approach demonstrates the potential of combining traditional programming with AI capabilities.

// Это оригинанальные insert`ы в data.sql
INSERT INTO types VALUES (default, 'cat'); //1
INSERT INTO types VALUES (default, 'dog'); //2
INSERT INTO types VALUES (default, 'lizard'); //3
INSERT INTO types VALUES (default, 'snake'); //4
INSERT INTO types VALUES (default, 'bird'); //5
INSERT INTO types VALUES (default, 'hamster'); //6

@Entity
@Table(name = "pets")
public class Pet extends NamedEntity {

	private static final long serialVersionUID = 622048308893169889L;

	@Column(name = "birth_date")
	@DateTimeFormat(pattern = "yyyy-MM-dd")
	private LocalDate birthDate;

	@ManyToOne
	@JoinColumn(name = "type_id")
	private PetType type;

	@OneToMany(cascade = CascadeType.ALL, fetch = FetchType.EAGER)
	@JoinColumn(name = "pet_id")
	@OrderBy("visit_date ASC")
	private Set<Visit> visits = new LinkedHashSet<>();

This works even if you make typos in the query. In the example below, LLM identified that I had written “hamstr” instead of “hamster”, corrected the query, and successfully matched it to the correct pet ID:

If you go deeper, you'll find even more impressive things. AddPetRequest transmits only ownerId as a parameter: I provided the owner's name instead of their ID, and LLM was able to independently determine the correct match. This indicates that LLM has decided to call the function listOwners before calling the function addPetToOwner. By adding a few breakpoints we can confirm this behavior. First we hit the breakpoint for getting owners:

Only after the owner's data has been returned and processed do we call the function addPetToOwner:

My takeaway is this: with Spring AI, start simple. Provide the basic data you know will be needed and use short, concise bean descriptions. Spring AI and LLM will probably figure out the rest. Only when problems arise should you start adding more hints to the system.

Adding an owner

Function addOwner relatively simple. It accepts the owner and adds him to the system. However, in this example we can see how to perform validation and ask follow-up questions using our chat assistant:

@Bean
@Description("Add a new pet owner to the pet clinic. "
		+ "The Owner must include first and last name, "
		+ "an address and a 10-digit phone number")
public Function<OwnerRequest, OwnerResponse> addOwnerToPetclinic(AIDataProvider petclinicAiDataProvider) {
	return request -> {
		return petclinicAiDataProvider.addOwnerToPetclinic(request);
	};
}

record OwnerRequest(Owner owner) {
};
record OwnerResponse(Owner owner) {
};

The business implementation is simple:

public OwnerResponse addOwnerToPetclinic(OwnerRequest ownerRequest) {
	ownerRepository.save(ownerRequest.owner());
	return new OwnerResponse(ownerRequest.owner());
}

Here we guide the model to ensure that Owner inside OwnerRequest meets certain validation criteria before it is added. Specifically, the owner must include first name, last name, address, and a 10-digit telephone number. If any of this information is missing, the model will prompt us to provide the necessary details before continuing to add the owner.

The model did not create a new owner before requesting the required additional data, such as address, city, and phone number. However, I do not remember providing the required last name. Will this work?

We identified an extreme case in the model: it does not seem to respect the surname requirement, even though @Description indicates that this is required. How can we solve this problem? Prompt engineering comes to the rescue!

@Bean
@Description("Add a new pet owner to the pet clinic. "
		+ "The Owner must include a first name and a last name as two separate words, "
		+ "plus an address and a 10-digit phone number")
public Function<OwnerRequest, OwnerResponse> addOwnerToPetclinic(AIDataProvider petclinicAiDataProvider) {
	return request -> {
		return petclinicAiDataProvider.addOwnerToPetclinic(request);
	};
}

By adding the “as two separate words” hint to our description, the model gained clarity about our expectations, allowing it to correctly enforce the surname requirement.

Next steps

In the first part of this article, we looked at how to use Spring AI to work with large language models. We created a custom ChatClient, used function calls, and improved the query generation process for our specific needs.

In Part 2, we'll dive deeper into the capabilities of Retrieval-Augmented Generation (RAG) to integrate the model with large, specialized datasets that are too large for a function call approach.

Join the Russian-speaking community of Spring Boot developers in telegram – Spring IOto stay up to date with the latest news from the world of Spring Boot development and everything related to it.

We are waiting for everyone join us