7.6 Before Going Live: Practice Checklist and Common Pitfalls
7.6.1 Practice Checklist and Common Pitfalls
Pushing an AIoT agent from concept into production: technology selection and architecture design are only the starting point. The real risks hide in runtime details — context may leak across tenants, tool parameters may go out of bounds, side effects may repeat after a process restart, and approvals and receipts may become impossible to trace. The checklist below examines the system along "model and context — capability contract — security controls — runtime governance — testing and release," rather than checking only whether the model can invoke a Tool.
Table 7-4: AIoT Agent engineering practice checklist
| Check area | ID | Check item | Result | Notes |
|---|---|---|---|---|
| Model selection | CHK-01 | Is the provider protocol one of the currently supported OpenAI-compatible or Anthropic types? | □ Pass □ Fail | ChatClientFactory selects OpenAiChatModel or AnthropicChatModel by provider type; other protocols require a new adapter |
| CHK-02 | Is a fallback model configured and verified in dc3_model_provider and dc3_model_config? | □ Yes □ No | A request can select a model or fall back to the default model; configuring multiple providers does not mean automatic failover already exists | |
| CHK-03 | Is there a planned routing strategy that sends simple queries and complex diagnostics to different models? | □ Yes □ No | This is future strategy design; the current project has no engine that routes automatically by complexity, cost, or sensitivity label | |
| Tool design | CHK-04 | Do the description of each @Tool method and its @ToolParam descriptions explicitly state parameter units, value ranges, and typical examples? | □ Pass □ Fail | The model relies on descriptions to decide whether to call a tool. Vague descriptions cause needed calls to be missed and unneeded calls to fire indiscriminately. Illustration: "the desired speed value (unit: rpm, range 0-3000)" leaves far less room for model guesswork than "the desired value." |
| CHK-05 | Are read-only tools and write tools clearly separated at the tool-design level? | □ Yes □ No | In principle, a read-only tool states "read-only" in its return, and a write tool marks "write operation + risk level" in its description. | |
| CHK-06 | Do write tools perform parameter-range and type validation beyond the method-signature level? | □ Yes □ No | Example: a written temperature value should be constrained to -50 to 150 °C; anything outside the range is rejected outright by a thrown exception. | |
| CHK-07 | Does each tool wrap an existing service-layer method rather than copying business logic? | □ Yes □ No | Logical consistency depends on a single source of definition | |
| Security controls | CHK-08 | Do all tool calls carry and validate tenant and user context? | □ Yes □ No | ToolContext injects principal information; actual authorization is still guaranteed by the business layer behind each Tool call and by interface boundaries |
| CHK-09 | Do operations with side effects have manual confirmation or external approval in place? | □ Yes □ No | What is explicitly implemented today is the point-write Action; batch writes, driver changes, and deletions cannot yet be generalized into a "built-in confirm button" | |
| CHK-10 | Do MCP endpoints enable OAuth 2.1 + a tool whitelist + risk tiering? | □ Yes □ No | When external agents connect, OAuth authorization is mandatory before any tool can be exposed | |
| Runtime governance | CHK-11 | Is a unified run_id used to link model, Tool, Action, command, receipt, and final status? | □ Yes □ No | There is no universal run_id yet; when building the Runtime, a unified execution identifier should be put in place first |
| CHK-12 | Is task state persisted independently of the session, with support for awaiting confirmation, failure, cancellation, and manual takeover? | □ Yes □ No | Session memory cannot replace a long-lived task state machine; this still needs to be implemented | |
| CHK-13 | Do Tools declare timeout, retry, idempotency, side-effect, result-verification, and compensation semantics? | □ Yes □ No | Device commands usually cannot be recalled; never retry blindly when side effects are unknown | |
| Logging and audit | CHK-14 | Does every tool call record tenant ID, operation time, input parameters, return status, and exception stack? | □ Yes □ No | Tenant information is already injected into ToolContext; missing logs make failures impossible to trace |
| CHK-15 | Is there a monitoring dashboard showing task status, Tool success rate, timeout rate, repeated side effects, and manual takeovers? | □ Yes □ No | The unit of observation should be raised from a single model request to a complete task run | |
| Testing and deployment | CHK-16 | Do test doubles or an isolated environment cover the typical Tool Calling scenarios? | □ Yes □ No | There is no universal "simulation mode" switch today; test environments must not connect to real critical devices |
| CHK-17 | Is access opened first to pilot tenants and R0/R1 scenarios, with explicit fallback conditions set? | □ Yes □ No | Autonomy should be opened level by level on evidence, not through a single all-tenant Agent switch | |
| CHK-18 | Are Tool timeouts, process restarts, lost receipts, duplicate events, and manual takeover rehearsed? | □ Yes □ No | Without recovery drills, there is no entering the Workflow Runtime or bounded autonomy | |
| Continuous improvement | CHK-19 | Is the Agent Eval re-run after changes to the model, Prompt, Tool Schema, or policy? | □ Yes □ No | The release gate should cover results, trajectories, safety, recovery, and cost |
| CHK-20 | Are tool visibility under long contexts, evidence contamination, and cross-task memory isolation tested? | □ Yes □ No | Long conversations can weaken Tool descriptions; task memory must also set retention and eviction boundaries |
Common Pitfalls
Pitfall 1: Over-trusting model output. Engineers easily take the model's "earnest fluency" as "absolute correctness." When calling a function, the model may fill in wrong parameters — especially when the parameter type depends on its guesswork. The mitigation is to first validate device, point, and tenant ownership, then check parameters against the metadata, value-range rules, and scenario whitelists that actually exist on the platform; a point write today must also go through Action confirmation. A @ToolParam description cannot replace strong server-side validation. This judgment aligns with the research community: the industry-agent survey (arXiv:2510.17491) likewise flags LLMs' weak long-horizon reliability and insufficient real-time performance, arguing they should not make decisions inside high-frequency control loops.
Pitfall 2: Ignoring failure compensation. "The device command has been issued" comes with no universal recall button. Illustrative scenario: if batch point writes are opened in the future, some may fail on communication timeouts while the rest have already taken effect. Without a compensation plan, the field team must restore devices one by one by hand. The mitigation is to validate first, execute in small batches, confirm results batch by batch, and design reverse commands for the specific devices; the current provider has no batch-execution CommandTool — do not describe the present state through interfaces that do not exist.
Pitfall 3: Imprecise tool parameter descriptions. Spring AI's @ToolParam annotation contains no strong validation logic of its own. Developers must add a second layer of constraints inside the tool method, through Assert.notNull or custom validators. A common problem in practice: the parameter description reads "the desired speed value" without stating the unit (rpm or percent), so the model guesses wrong.
Pitfall 4: Ignoring the context window's effect on tool visibility. As conversation turns accumulate, the model's early tokens are squeezed out, and the early tool descriptions are likely to be forgotten by the attention mechanism. In engineering terms, the complete list of currently available tools must be injected on every turn of the conversation, not just once in the first turn. Spring AI's ToolCallback mechanism by default re-registers tools each turn within the same thread, but developers still need to confirm, under long-conversation stress tests, that tools remain correctly callable.
Pitfall 5: Writing the evolution roadmap as a present-day mode switch. The current implementation is explicitly registered Tools, session memory, and the point-write Action; there is no tenant-level agent_mode and no one-click switch into a full Agent/Copilot product mode. When an orchestrator is added in the future, the device scope, scenario whitelist, confirmation or external-approval nodes, and concrete compensation strategy must be spelled out; never let the model judge on its own and wave risky actions through.
7.6.2 Further Reading
This chapter is knowledge-dense, spanning three threads: model principles, engineering frameworks, and hands-on platform work. The resources below are organized in a "theory → framework → practice" order, for convenient cross-reference when digging deeper.
Official documentation and project repositories
- Spring AI official documentation: covers the configuration and core APIs of
ChatClient, Function Calling, and conversation memory — the first desk reference for integration work. - IoT DC3 project repository (GitHub: pnoker/iot-dc3): for the Tool registration status of the Agentic source, see Section 7.3.1; when reading it, also check the Provider configuration — do not judge the tools visible to the model by class count alone.
- LangChain official documentation: provides reference implementations of RAG and the agent loop, useful to compare against the Spring AI practice.
Protocols and standards
Industry-agent survey (Tang et al., 2025): Empowering Real-World: A Survey on the Technology, Practice, and Evaluation of LLM-driven Industry Agents (arXiv:2510.17491 abstract page) — jointly released by Harbin Institute of Technology (Shenzhen) and Huawei in October 2025; it systematically reviews the memory/planning/tooling pillars, an L1–L5 capability maturity ladder, evaluation methods, and six application domains. Its conclusion that LLM real-time performance is insufficient for high-frequency control loops agrees with this chapter's "models stay out of the real-time loop, deterministic backstop" boundary, and its three evaluation tensions (fidelity vs. reproducibility, cost vs. efficiency, privacy vs. data quality) pair well with 7.5.4.
MCP (Model Context Protocol): defines a standardized interface between models and external resources; it was donated in December 2025 to the Agentic AI Foundation under the Linux Foundation and has become one of the widely adopted de facto standards for agents accessing tools. The IoT DC3 MCP gateway is an engineering realization of this specification; the protocol layering and standard evolution of MCP are covered in Section 9.5 of Chapter 9.
OpenAI Chat Completions and Anthropic Messages API specifications: IoT DC3 currently integrates with them through the OpenAI-compatible and Anthropic providers respectively. Understanding each side's Tool Calling protocol and parameter differences helps troubleshoot tool-invocation problems after a model switch.
Key papers and framework code
- "ReAct: Synergizing Reasoning and Acting in Language Models": the foundational paper of the agent field. The think-act loop in this chapter's Agentic Center architecture derives from this work.
- Spring AI official sample projects: the demonstration projects under
spring-projects/spring-aion GitHub, providing minimal prototypes that run directly.
Self-hosted deployment
- Ollama: the starting point for local model deployment. It loads models such as DeepSeek and Qwen on a single machine and exposes an OpenAI-compatible endpoint, well suited for local validation with sensitive data.
- vLLM: a production-grade inference acceleration solution, providing PagedAttention optimization and continuous batching.
Suggested reading order: read the ReAct paper through first to understand the agent loop; then follow the Spring AI official documentation to write a "query device temperature" ChatClient prototype; finally work through the IoT DC3 Agentic Center source, focusing on how DeviceTool and PointValueTool inject the security context. Every step can be cross-checked against this chapter.
At this point, an agent can read context, call controlled Tools, and generate candidate actions, but "can call" does not mean "should be authorized." Chapter 8 places identity, least privilege, data protection, confirmation, and audit on the same call path, creating an unavoidable deterministic boundary around the probabilistic capabilities developed here.
Mark this chapter’s position with the four words: Reason lands here, carrying its boundary — it only proposes candidates; Act has just received its admission rules, and the full deterministic boundary closes in the next chapter.