7.2 Spring AI and IoT Integration
7.2.1 Spring AI Overview and Configuration
Spring AI provides abstractions such as ChatModel, ChatClient, Advisors, Chat Memory, and Tool Calling for Java/Spring applications. ChatClient is the unified entry point for business code; underneath it, different providers can supply their own ChatModel implementations — it does not require every model to standardize on the OpenAI Chat Completions protocol.
IoT DC3 currently uses Spring AI 2.0.0 (GA, June 2026) and pulls in the OpenAI, Anthropic, and JDBC Chat Memory starters together:
<dependency>
<groupId>org.springframework.ai</groupId>
<artifactId>spring-ai-starter-model-openai</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.ai</groupId>
<artifactId>spring-ai-starter-model-anthropic</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.ai</groupId>
<artifactId>spring-ai-starter-model-chat-memory-repository-jdbc</artifactId>
</dependency>Model connections are not written only into application.yml. The project keeps the provider type, endpoint, key, default flag, and enabled state in dc3_model_provider, and the concrete models with their capability settings in dc3_model_config. ChatClientFactory resolves the configuration from the model in the request or from the default model: OPENAI_COMPATIBLE builds an OpenAiChatModel, ANTHROPIC builds an AnthropicChatModel, and the resulting ChatClient is cached. Deployment environment variables also provide an OpenAI-compatible fallback, so the platform does not lose its basic conversational entry point when the database configuration is unavailable.
Business code uses one unified ChatClient call shape:
String answer = chatClient.prompt()
.user("What is the boiler's current temperature?")
.call()
.content();A unified interface does not mean providers behave identically. Before switching models you still have to verify the authentication method, available parameters, streaming responses, Tool Calling, context window, and error semantics. A request may select any enabled model and uses the default when none is specified; there is currently no policy engine that routes models automatically by cost, complexity, or sensitivity label.
7.2.2 ChatClient: The Unified Conversation Interface
The best way to understand ChatClient is to start from a piece of code that runs. Assume you have configured the dependencies following the previous section; now open a Spring Boot test class or a @Service.
@Autowired
private ChatClient chatClient;
public String askDeviceStatus() {
String question = "What is the current temperature of boiler No. 3 in zone A? Please give the value and unit.";
String answer = chatClient.prompt()
.user(question)
.call()
.content();
return answer;
}This code shows the first core design decision: the call style. ChatClient decomposes the whole conversation flow into clear chained steps: prompt() builds the message → user() supplies the user input (system() can also be added to set the role) → call() triggers model inference → .content() extracts the plain-text response. The fluent style is common across the post-Java 8 ecosystem, so the onboarding cost for engineering teams is low.
Synchronous calls (sync calls) are the simplest and the easiest to debug. After the request is sent, the current thread blocks on the call() method until the large model returns the complete result. For IoT operations, it is generally used in scenarios that need no real-time streaming display — "query a status once," "parse a command." For example, when an operator says "find me the device ID from the last repair request," synchronous mode is sufficient, and the code logic stays straightforward.
Many IoT scenarios, however, need real-time feedback — when reading a boiler temperature, if the model has to generate an analysis report piece by piece, the operator does not want to wait for the entire report before seeing the first line. This calls for streaming calls, which are also built into ChatClient:
public void streamHealthReport() {
Flux<String> reportStream = chatClient.prompt()
.user("Generate today's health report for boiler No. 3, including temperature trend and anomaly markers")
.stream()
.content();
reportStream.subscribe(chunk -> {
System.out.print(chunk); // or push via WebSocket
});
}stream() returns a Reactor Flux<String>; every time the model generates a new token (a token: the smallest unit of text a large model processes — think of a word or sub-word fragment), the subscribe callback fires once. In a real operations console, the content the user sees refreshes line by line rather than appearing all at once after minutes of waiting. This experience matters especially for long-reply scenarios such as alarm diagnosis and analysis.
The third dimension is function calling. Section 7.2.3 covers it in detail, but one sentence here: the tools() and defaultTools() methods on ChatClient can register @Tool-annotated Spring Beans as tools the large model may call on its own. When the user says "set boiler No. 3's temperature to 85 degrees," the large model does not write code — it calls the setTemperature function you registered, passing deviceId="boiler-03", targetValue=85, and then business code performs the actual operation and returns the result. This mechanism turns ChatClient from a "question-answering machine" into an "operations entry point."
Typical conversation scenarios.
- Device status query. The user: "Show all offline gateways in the plant." The model calls
DeviceTool.listOffline()and renders the result in natural language: "2 gateways are offline: the line-2 PLC (powered off at 10:23) and the warehouse thermostat (network disconnected at 09:15)." - Log analysis. The user: "Any anomalies in boiler No. 3's pressure logs between 2:00 and 3:00 last night?" The model first calls
PointValueTool.queryHistory()to fetch the data, then judges the trend against the normal pressure range in its context. The final output: "Pressure spiked to 1.5 MPa at 2:47 (allowed ceiling 1.2 MPa) and fell back after roughly 4 minutes." - Fault diagnosis. The user: "The alarm keeps sounding — help me look into it." The agent can first call
DeviceToolto query device status, then useDriverToolto confirm the owning driver and the online summary of the devices under it, and finally distinguish a single-device fault from a driver-level fault and give inspection steps. The current provider does not registerEventTool, so the example does not call it.
What all these scenarios share is that ChatClient acts as a translation layer — translating natural language into API calls, then translating the API results back into natural language. No bespoke parsing logic is needed for each device.
A few engineering notes. Synchronous calls are intuitive, but if the model responds slowly (seconds to tens of seconds), prolonged blocking can exhaust the thread pool. ChatClient has no method such as async(): in production the synchronous call is usually placed on an async executor or in a WebFlux context, wrapping the asynchrony yourself with mechanisms such as CompletableFuture; when content must come back incrementally, switch to the stream() streaming call demonstrated above. Streaming calls fit non-blocking architectures naturally, but backpressure still needs to be managed so that pushing too fast does not overflow the front-end buffer. Function calling involves user confirmation and permission checks, so an interception step is usually added before tool execution — for example, IoT DC3's Agentic Center passes tenant and user identity through the ToolContext, and business code consults RBAC to decide whether a write is allowed.
Overall design summary. ChatClient's three call modes map to different IoT operations needs:
| Call mode | Fitting scenario | Data flow | Typical example |
|---|---|---|---|
| Synchronous call (Sync) | Quick Q&A, simple commands | Request → block → complete response | "Check the current room temperature" |
| Streaming call (Stream) | Long analyses, watching progress in real time | Request → push chunk by chunk | "Analyze anomalies in today's trends" |
| Function call (Function) | Executing operations, writing values back | Request → model decision → call business code → return result | "Set the fan speed to 1500 rpm" |
In design terms, ChatClient adds a layer of clever abstraction: it does not care whether you connect GPT-5 or DeepSeek — as long as the model exposes an OpenAI-compatible Chat Completions endpoint, the calling style stays consistent. This gives the IoT platform freedom in "model choice": use GPT today, switch to a privately deployed DeepSeek tomorrow, and the upper-level business code usually needs no changes — the switching cost is mainly a configuration edit. That said, authentication, Tool-Calling behavior, and response semantics must still be re-verified provider by provider (Section 7.4.1 elaborates); an adapter is not a guarantee that "nothing differs after the config change." IoT DC3's Agentic Center is a product built on exactly this design; turning one chat message into a device command relies on combining ChatClient's synchronous or streaming conversation interface with the function-calling mechanism.
With these three call styles in hand, the next question is how function calling is defined and registered in practice — that is the key mechanism through which Spring AI lets a large model "touch" devices.
7.2.3 Function Calling: From Model Request to Controlled Tool Execution
ChatClient can answer "what is the boiler temperature," but operations also need to query live status, create work orders, or issue device writes. Function Calling (also known as tool calling) extends the LLM from plain-text generation to structured capability requests. It solves "how the model selects a capability and fills in its parameters"; it is not responsible for authorization, approval, state recovery, or physical-control safety — those duties belong to Tools, Workflows, and the Agent Runtime.
How the mechanism works
The Function Calling flow is not complicated. The application first registers a set of callable functions with the LLM (name, description, parameter structure); during inference the model judges whether the user's intent matches one of them. On a match it outputs a structured JSON containing the function name and arguments, rather than natural language. The application intercepts that JSON, executes the corresponding backend method, then feeds the execution result (typically success/failure and a return value) back to the model so it can compose the final natural-language reply. There is no magic anywhere in the process — the LLM executes no code; its whole job is "pick the function, fill in the parameters."
Consider an example. The user says: "Set the blower speed of boiler No. 3 in zone A to 1500." The LLM will not turn the fan itself; it only emits a candidate request like { "function": "setDevicePointValue", "arguments": { "deviceId": "boiler-003", "pointId": "fan-speed", "value": 1500 } }. A production system must first validate the target, parameters, permissions, risk level, and operating conditions, then decide whether to reject, wait for confirmation, or enter a deterministic workflow. Only after execution completes and an objective receipt has been read can the system report the result to the user.
The following in-memory smart-light example demonstrates the Function Calling mechanism. It illustrates tool registration and invocation only; it is not a suggestion that an industrial site should skip the governance plane and execute directly.
Tool definition: toggling the light
Defining a tool that an LLM can call is extremely simple in Spring AI — just add the @Tool annotation to a Bean method. Here is the implementation of the toggle-light tool.
import org.springframework.ai.tool.annotation.Tool;
import org.springframework.ai.tool.annotation.ToolParam;
import org.springframework.stereotype.Component;
@Component
public class LightTool {
private boolean lightOn = false;
private String currentLocation = "Zone A";
@Tool(description = "Toggle the smart light in the specified zone and return its current status")
public String toggleLight(
@ToolParam(description = "Zone name, e.g. Zone A, Zone B, Zone C") String location,
@ToolParam(description = "Target status: true turns the light on, false turns it off") boolean turnOn) {
// In the real IoT DC3, this is where the DeviceTool write API would be called
// Illustrative logic only
this.lightOn = turnOn;
this.currentLocation = location;
String status = turnOn ? "switched on" : "switched off";
return String.format("The light in %s is %s", location, status);
}
@Tool(description = "Query whether the light in the specified zone is currently on or off")
public String getLightStatus(
@ToolParam(description = "Zone name") String location) {
String status = lightOn ? "on" : "off";
return String.format("The light in %s is currently %s", location, status);
}
}Two key points. First, the description on the @Tool annotation is the only channel through which the LLM understands the function — the more precise the description, the less likely the model is to call it wrongly. Second, the description on @ToolParam helps the model fill parameters correctly; for instance, if the turnOn parameter used the numbers 1/0 instead of a boolean, the model could still infer the intent from the description.
Tool registration and invocation
Once the tools are defined, they must still be registered with ChatClient explicitly. Merely declaring LightTool as a Spring Bean does not make ChatClient.Builder scan all @Tool methods automatically. You can register default tools for requests built by the same Builder with defaultTools(lightTool), or call tools(lightTool) on a single request.
@Autowired
private LightTool lightTool;
public void demoFunctionCalling() {
ChatClient chatClient = ChatClient.builder(chatModel)
.defaultTools(lightTool)
.build();
String userRequest = "Please turn off the light in Zone A";
String response = chatClient.prompt()
.user(userRequest)
.call()
.content();
// Output: turned off the light in Zone A
System.out.println(response);
}At execution time, ChatClient internally first sends the user message plus the tool descriptions (the two method signatures of LightTool) to the LLM; the model decides that "turn off the light" maps to toggleLight(location="Zone A", turnOn=false) and emits the function-call request. The client executes the function, returns the result to the model, and the model composes the final reply. All of this is transparent to the developer.
If the user asks in sequence — first "what is the status of the zone-A light," then "turn it off" — the two calls pass through the same conversation context. That is the work of the next section, "chat memory": the model remembers the state it looked up in the previous turn.
Engineering risks and controls
When function calls connect to physical devices, there must be a deterministic boundary between the model generating a request and execution being authorized.
Permission checks. Not every user should be able to operate every device. Every @Tool method should obtain the current authenticated user and tenant ID through the ToolContext, then run an RBAC check before executing. IoT DC3's approach: every AI action ultimately goes through the platform's real APIs, the Gateway injects the principal context, and the authorization center performs permission checks and tenant isolation — the model never holds more permission than the underlying account.
Parameter validation and range constraints. Parameters filled in by the LLM can exceed the expected range — setting a speed to 100000, for example. The tool method must validate parameter legality internally; @ToolParam itself has only two attributes, description and required, and cannot declare a value range, so state the unit and range in the description and enforce strict validation on the server side (consistent with Section 7.6.1 CHK-06). For high-risk writes, a "parameter preview + confirmation" step can be designed so the user confirms on the interface before execution.
Recovery and idempotency. Device operations do not always succeed: network interruptions, offline devices, and protocol timeouts can all leave "did it take effect" uncertain. Tools should declare timeout, retry, and idempotency semantics; the Runtime should persist execution state and evidence of side effects. Recovery decisions must not be left to the model, and no one can promise that every physical action can be rolled back.
Avoiding the natural-language trap of "misoperation." A user saying "shut down all the devices" may be joking, yet the model may still issue a batch operation request. Batch and high-risk operations must be rejected by server-side policy or routed into an approval workflow; warnings in tool descriptions and clarifying questions from the model only improve the interaction — they do not constitute a security control.
With tool definition covered, the natural next question is: across multiple turns, how does the model remember the device IDs and parameters it looked up in the previous turn? That calls for the chat-memory mechanism.
7.2.4 Chat Memory: Keeping Context Continuous
In conversational operations, an operator may first query historical data and then request an operation on a certain segment. Without a memory mechanism, the model cannot resolve the reference in the second sentence — there is no explicit link between the "segment" mentioned in the previous turn and the parameters to adjust in the next. This is not a usability problem but a structural tension between stateless APIs and multi-turn interaction: each request to a large language model is handled independently by default, information from the previous turn does not carry over automatically, and the application layer must manage the session history itself.
The engineering cost of stateless design
The Chat Completion API follows a stateless design: each request carries its own complete messages, and the model does no cross-request correlation internally. This simplifies the API itself but hands the entire responsibility for context management to the caller. In IoT operations, one session may run for many turns, spanning device queries, parameter interpretation, command issuance, and result confirmation. If every turn starts from zero, reference resolution necessarily fails, and "multi-turn conversation" degenerates into single-turn Q&A. This is the first layer of cost to weigh when choosing ChatClient: you gain the high-availability scaling of a stateless service, and you must pay back context continuity with extra memory or storage.
Three memory strategies
Spring AI 2.0 converges chat memory into two abstractions: ChatMemory, which organizes messages by conversation and decides the retention policy, and ChatMemoryRepository, which handles reading and writing messages in storage. The current implementation is MessageWindowChatMemory — a sliding window that keeps only the most recent messages; swap the repository for the JDBC implementation introduced in Section 7.2.1, and messages persist to the database. The 0.x-era InMemoryChatMemory, MessageChatMemoryAdvisor, and similar APIs have been superseded by this combination — old examples found online must not be copied as-is. The first two strategies are built into Spring AI, while knowledge-graph memory requires custom development or an optional extension; the three differ markedly in how well they fit IoT scenarios:
| Strategy | Principle | Fit for operations scenarios |
|---|---|---|
| Message history | Appends the full message list (user + assistant) to every request | Short conversations (usually within 10 turns); keeps context with no information loss |
| Summary memory | Compresses history into a single summary to avoid token overflow | Long conversations or tight token budgets, but key operation results must be retained |
| Knowledge-graph memory (custom / optional extension) | Maintains entity relationships and retrieves only relevant entities for context | Complex-reasoning scenarios, such as tracing historical operation chains across multiple devices |
Operations conversations usually revolve around a limited set of devices and points, with a controllable number of turns, so the message-history mode is the most direct. But when conversations stretch long or involve frequent Tool Calling feedback, summary memory with automatic compression is the safer choice. The compression rules deserve special care: operation history must retain execution results and status codes, lest the model re-issue the same command because context was lost.
Key implementation: MessageWindowChatMemory and conversationId
MessageWindowChatMemory automatically pulls the historical messages associated with the current conversationId and injects them into the prompt before each call, then writes the current turn's messages back to the repository after the call ends. The conversationId is the session's unique identifier — giving different sessions different IDs is enough to isolate their contexts. The following code shows typical usage (illustrative; for the exact method signatures and parameter names, refer to the official Spring AI 2.0 documentation):
// maxMessages is the sliding-window size, replacing the old advisor's history-count setting:
// only the most recent 20 messages are injected into the prompt, so a long session cannot blow up the context window
ChatMemory chatMemory = MessageWindowChatMemory.builder()
.chatMemoryRepository(chatMemoryRepository) // JDBC implementation, auto-configured by the Section 7.2.1 starter
.maxMessages(20)
.build();
// Round 1
String response1 = chatClient.prompt()
.user("What was the average temperature of line 3 yesterday?")
.advisors(a -> a.param(ChatMemory.CONVERSATION_ID, "session-line-3"))
.call().content();
// Round 2: the same conversationId links back to the previous turn
String response2 = chatClient.prompt()
.user("For this temperature range, how should the air-cooling parameters be adjusted?")
.advisors(a -> a.param(ChatMemory.CONVERSATION_ID, "session-line-3"))
.call().content();Omitting the conversationId is the most common wiring mistake in multi-turn conversations: the advisor never receives the session identifier, history injection comes up empty, and the model appears to have amnesia — answering off the point or repeatedly asking for information it was already given. When troubleshooting, first check whether the request's advisor parameters carry the session ID, and only then suspect the model itself.
Engineering trade-off: conversation length and the token budget
Injecting the full history carries an obvious token cost. For models with short contexts, keeping several complete turns exhausts the budget quickly, leaving little room for instructions and tool returns. In Spring AI 2.0, this length is no longer governed by a configuration item on the old advisor; it is declared directly as maxMessages when building the MessageWindowChatMemory: messages inside the window are injected in full, messages outside it are dropped, and any information that must be retained long-term has to be compressed into a summary by the application layer and written back to storage ahead of time. In engineering practice, the common compromise is to keep the most recent turns in full while a summarizer produces one structured summary of the earlier history — that summary must include key operation results and timestamps, so the model neither repeats an execution nor misjudges because information is missing.
Session persistence in IoT DC3
IoT DC3's Agentic Center takes exactly the JDBC-repository route: ChatMemory storage is wired to the platform database, and conversation records are written directly into the center database's tables, supporting session replay, audit, and post-incident review. The Chapter 6 deployment topology has no Redis, and no new middleware is introduced here — reusing the platform's existing database settles the persistence, backup, and cross-process sharing of session state along with the database itself, naturally satisfying the audit-replay requirement. From one sentence — "show what we did to line 3 last time" — the system can retrieve the complete history of that session. This traceability not only provides context continuity for multi-turn interaction; it also digitizes every operations action into an auditable record — the infrastructure underpinning operational compliance and incident retrospection.
Chat memory is the precondition for Function Calling to execute correctly across turns — the model must know the previous turn's operation results before it can decide which point to query or which parameter to adjust next. Without it, Tool Calling works only within a single turn, and much of the application value is lost.