7.1 The AIoT Technology Landscape and Evolution

Author: Zhang Hongyuan · © 2016–2026 · All Rights Reserved · Copyright & License

Before this chapter unfolds the agent capabilities, it first picks up the conclusion of Chapter 6: the cloud-native foundation formed by microservices, containerization, and observability does more than make the platform "run" — it is at the same time the operating surface of the agent: model invocations, Tool execution, session state, and audit records all land on this foundation. Without that deployable, scalable, observable, service-oriented platform from Chapter 6, the Agent capabilities discussed in this chapter could survive only in a demo environment.

7.1.1 The Definition and Evolution of AIoT

A red alarm pops up on the large screen — a cooling pump's vibration reading has crossed its limit. The operator manually pulls up the trend chart, leafs through the equipment records, compares maintenance logs, and only after a round of human judgment can the operator distinguish an incidental fluctuation from the first sign of bearing wear. The data is there to see; the decision is left to human guesswork. The Internet of Things (IoT) solved the problem of "connection" — sensors, PLCs (Programmable Logic Controllers), and RFID readers stream data up to the platform without pause. But the end of that connection is still, more often than not, a human operator: data is presented on dashboards, analysis relies on experience, decisions rely on judgment, and execution relies on manual clicks.

AIoT (Artificial Intelligence of Things) breaks this split. It embeds artificial intelligence — above all large language models (LLMs) and multimodal models — into the IoT closed loop of "collect → analyze → decide → execute," so that machines not only see the data but also understand semantics, reason about causes, and act automatically. One line captures it: IoT makes the world perceivable; AI makes perception actionable.

The design of the IoT DC3 platform picks up exactly this thread. Every action the AI proposes eventually enters a real platform API, has principal context injected by the gateway, and then passes role-based access control (RBAC) permission checks and tenant isolation at the authorization center — the model never holds more privilege than the corresponding account. This means AIoT is not a layer of intelligence "stacked" on top of the IoT; it brings the model into an existing controlled operation chain. What is trustworthy is the execution process — authorized, validated, confirmed, and audited — not the model itself.

1. Three Stages of Evolution: Connection, Intelligent Analysis, and Autonomous Decision-Making

AIoT has matured through roughly three stages, each clearly different in technical characteristics and degree of intelligence. The diagram below shows the evolutionary path as the author has organized it.

Figure 7-1 AIoT Evolution Stages

Stage 1: connection and data acquisition. The theme is "get the devices connected." Large IoT platforms focus on device registration, protocol adaptation, data acquisition, and storage. The platform works like a data pipeline: sensor values pass through gateways and stream-processing engines, land in a time-series database, and finally appear on dashboards for people to view. The intelligence is very shallow — mostly a threshold-based alarm rule engine (raise an alarm when, say, temperature crosses a limit). A rule engine is strongly deterministic, but it cannot handle scenarios that are ambiguous, volatile, or semantically rich. Operators must constantly retune thresholds to track changing operating conditions, and false alarms and missed alarms remain persistent pain points. The core deliverable of this stage is readable, queryable data — not executable intelligence.

Stage 2: intelligent analysis and human-machine collaboration. Edge computing and lightweight machine-learning models begin to reach production. Algorithms such as anomaly detection and predictive maintenance are introduced. Models run on separate inference services, and their outputs feed the alarm system or the large screen. In recent years, mainstream LLMs have gained multi-step reasoning and tool calling (Tool-Calling), letting an IoT system — for the first time — understand natural-language tasks with the help of device manuals and live data, while the application invokes platform capabilities from the structured requests the model generates. IoT DC3's Agentic Center was born in exactly this context — it connects OpenAI-API-compatible large models to devices, points, and data capabilities; users ask questions in natural language, and the model selects built-in platform tools as needed to read metadata or query live values; point writes with side effects first create an Action pending confirmation. The core difference from Stage 1: the model is no longer a spectator but a decision-making participant inside the controlled operation chain.

Stage 3: constrained decision-making and bounded autonomy (still taking shape). The agent no longer merely waits for humans to ask; it can be triggered by alarm events or scheduled tasks, proactively gathering evidence, diagnosing root causes, and proposing strategies. When execution actually happens, the Agent Runtime must still bound identity, device scope, time window, tool whitelist, and risk budget, and hand off to deterministic Workflows or human confirmation at critical nodes. Typical features of this stage include scheduled health reports, multi-model routing by task complexity, and external AI agents discovering and calling whitelisted capabilities through authorized MCP endpoints (as of mid-2026). Humans shift from step-by-step operation to supervision, approval, and exception takeover, but they do not exit safety-critical decisions. IoT DC3 already has the foundations — sessions, explicit Tools, tenant context, and point-write Actions — while event triggering, long-running task state machines, recovery, and unified governance still need to be built.

2. The Core Driving Forces: Compute and Models in Symbiosis

The evolution from Stage 2 to Stage 3 is driven by two parallel forces.

The first: compute moving to the edge. The classic IoT pain points are high cloud-inference latency, expensive bandwidth, and privacy risk. The sound engineering division of labor is "train in the cloud, infer at the edge, respond on the device": models are trained in the cloud on full historical data, pushed down to the edge for low-latency inference, while the device side only performs the final, fastest response. Edge computing devices — embedded AI chips being the representative case — can already run lightweight LLMs or vision models under limited power budgets, making edge-side deployment of large language models an engineering reality. The direct benefit of moving compute down is markedly lower inference latency, and sensitive data never has to leave the local network.

The second: the leap in model capability. In recent years, large language models have leapt from "text conversation" to "Tool-Calling." Traditional IoT intelligence relies on rules and classification/regression models, whereas today's LLMs can take a natural-language instruction such as "set the feed valve on Line 2 to a lower opening" and reason out which API to call, what parameters to pass, and even how to perform boundary checks. This capability is a natural match for the IoT's command-intensive character. IoT DC3's approach is pragmatic: through Spring AI, Tool-Calling becomes an ordinary Java method call, letting the model's comprehension plug straight into the platform's existing business logic. The model never needs to perceive underlying protocol differences (Modbus, OPC UA, MQTT), because those differences are already shielded by the platform's device abstraction layer.

These two driving forces point to one conclusion: AIoT has moved from concept to engineering practice. The following subsections take this apart one by one — the concrete role of large models in the IoT (Section 7.1.2), how agents achieve autonomous decision-making (Section 7.1.3), and the key technologies that let models reach out and touch the physical world — RAG, Tool-Calling, and MCP (Sections 7.1.4 and 7.1.5), and how to evaluate a RAG system in layers (Section 7.1.6).

7.1.2 The Role of Large Models in the IoT: From Perception to Cognition

The rule engine has run for years as an important analytical instrument of the traditional IoT: if temperature crosses a threshold, raise an alarm; if a device goes offline, send a notification. Its boundary is clear — it excels at executing predefined deterministic conditions, while complex time-series comparisons and multi-source correlation require additional code. When an operator faces a compound judgment such as "the temperature in pump house 2 is 5 degrees higher than the same time yesterday, yet the load has fallen," a simple threshold rule can output only "temperature over limit." Comparing the same period, load, and maintenance records requires added queries, feature computation, and correlation logic.

Once connected to Tools and retrieval, a large model can organize season, load, historical trends, and maintenance records into an evidence-backed explanation and propose candidate hypotheses such as "check cooling-pump efficiency." The extension here is from a single threshold to organizing multi-source evidence, not causal proof produced from text by a model. A root cause still needs validation through time-series analysis, a mechanistic model, a controlled experiment, or field maintenance. Rule engines continue to handle deterministic events, while large models handle information retrieval, evidence synthesis, hypothesis generation, and human interaction. Their responsibilities are complementary.

1. From Rules to Semantics: Natural-Language Instructions Penetrating the Device Layer

The first visible change is the entry point for device operation. The traditional path — open the device list → find the target device → expand attributes → enter a value → click write — involves many steps and deep nesting. A large model can compress the user's expression into structured candidate actions: "turn off the first-floor corridor light" maps to a controllable switch point, and "set the temperature to 85 degrees" maps to a target value and a device point. After receiving a candidate action, the platform still completes schema, permission, operating-condition, and risk checks; anything with side effects enters a Workflow or human confirmation, and only then does deterministic code call the real device interface.

IoT DC3's Agentic Center is designed along exactly these lines. Through Spring AI's @Tool annotation, it exposes platform capabilities — devices, Driver, thing models, points, and point values — to the large model. When the operator says "read the boiler temperature and the fan speed," the Agentic Center can first locate the device and points, then read the two latest values. Tools reuse platform capabilities through the project's Facades, and tenant and user context enters the Tool with the request, ensuring the model reads current platform data rather than memories from its training set.

One engineering boundary must be made explicit here: natural-language instructions suit scenarios where the operational intent is clear and the safety risk is controllable. IoT DC3's current point-write Tool does not dispatch directly; it creates an Action pending confirmation. Only after the user confirms through the Action interface does the platform submit the write command. This design is not meant to protect the model — it keeps the human inside the decision loop.

2. Multimodal Fusion: More Than Text Conversation

Input in industrial settings is not limited to text and numbers. A camera catches an abnormal indicator light blinking on an equipment panel; an operator photographs it, posts it to a group chat, and asks "what does this mean?" — a traditional platform cannot process such input. Multimodal large models (mainstream examples include OpenAI's GPT-5 and Anthropic's Claude 4.5, as of mid-2026) accept image and text input at the same time: the blinking-light pattern in the photo, the gauge needle's position, the scorched color of a wire — all of it can be brought into the reasoning scope.

But the division of responsibility must be drawn clearly: large models excel at semantic reasoning and are not responsible for millisecond-level real-time control. Responses such as an emergency motor brake or a relay trip remain the duty of hardware controllers and edge real-time systems. The large model's attention sits at the cognitive layer — helping operators understand "why did this abnormality occur" and "what should be done next." The division of labor resembles a fire-protection system: sprinkler heads are triggered instantly by temperature sensors, but the judgment "should the whole building evacuate, and which departments must be notified" is entrusted to a decision-maker who understands the context. The large model plays exactly this decision-support role; its working focus is reducing the human's cognitive load, not replacing hardware control loops.

3. From Description to Reasoning: Generating O&M Strategies Automatically

A rule engine can reliably generate an event saying "temperature exceeded 85 °C." Through controlled Tools, a large model can retrieve the past seven days of trends, comparable-period data, and maintenance logs and form a diagnostic summary such as: "The rate of temperature rise is above the selected baseline. Reduced cooling-pump efficiency is one candidate cause; first verify current, outlet pressure, and sensor quality marks." The baseline, time window, and decision threshold must be calculated in code and returned with the evidence; the model must not invent precise conclusions such as "twice" or "within 30 minutes" from wording alone.

This is the leap from descriptive analytics ("what is the temperature now") to diagnostic analytics ("why is the temperature high"), and on to prescriptive analytics ("what should be done next"). The key foundation under this leap is Tool-Calling — the large model itself holds no permission to read live data; it must obtain devices, Driver, thing models, points, and values through the 8 Tool classes explicitly registered in the Agentic Center's current Provider, and only then synthesize a judgment and output a recommendation. Tools not registered in the Provider do not count as default session capabilities (registration list in Section 7.3.1).

Table 7-1: Comparison of typical application scenarios of large models in the IoT

Scenario	Traditional rule-engine approach	Approach after large models enter
Device control	Manual clicks on the dashboard or pre-set write commands	Natural-language instructions parsed into intent automatically; tools invoked to execute, written after user confirmation
Alarm triggering	Fixed threshold checks, templated notifications after trigger	Organizes contextual evidence and generates root-cause hypotheses and validation steps
Anomaly analysis	Displays out-of-limit data and basic statistics	Sorts out trends, correlates logs, generates natural-language explanations and response strategies
Operations and maintenance (O&M) strategy	Formulated manually from historical data reports	The model synthesizes multiple data sources and proactively offers operation suggestions and reports

The role of large models in the IoT can be summed up this way: they fill in a long-missing cognitive layer. Sensors acquire massive data and the rule engine renders fast verdicts, but "understanding context, generating suggestions, conversing with people" has always been missing. Large models fill exactly this gap, letting the IoT evolve from passive perception to active cognition — without replacing the existing real-time control logic. The next question goes one step further: how to host this cognitive capability in an Agent Runtime, and shape probabilistic decisions into governable industrial execution.

Figure 7-2 The LLM in IoT: from perception to cognition

7.1.3 Agent Runtime: From Model Capability to Governed Execution

Rule engines handle pre-defined judgments, and large models understand vague intent, but "investigate the temperature anomaly on Line 2" is neither a single-step alarm nor one model call. It requires the system to establish task context, query devices and historical data, choose the next capability, handle timeouts and empty results, wait for human confirmation when necessary, verify results after execution, and persist the whole process as an auditable record. Discussing only "whether the model can call tools" cannot cover these engineering responsibilities.

Therefore, this book separates Agent from Agent Runtime:

• An Agent is the decision-making entity that judges the next action within a given context, skilled at understanding intent, marshaling evidence, and planning dynamically.
• An Agent Runtime is the governed execution environment that hosts the agent's run, responsible for context, state, capabilities, permissions, scheduling, recovery, audit, and human takeover.

One model plus a few Tools can carry off a demo, but only the Runtime can answer the questions production systems actually care about: which step the task has reached, who authorized what, whether calls were duplicated, how to recover after failure, when the task must be handed back to a human, and whether the system can prove it never crossed a safety boundary.

1. The Four Planes of the Runtime

An industrial Agent Runtime can be decomposed into four mutually constraining planes.

The decision plane is responsible for understanding the goal and generating candidate next actions — intent recognition, task planning, model routing, and completion judgment. The large model lives on this plane, but it is not the whole runtime. What the model outputs are candidate plans or tool requests, which cannot simply be equated with device commands already approved for execution.

The context plane supplies trustworthy information for every step: the current user and tenant, target devices, real-time state, session history, retrieved evidence, and task memory. Context and memory must be distinguished here: context is the working set visible to the current decision, while memory is information that can be stored, retrieved, and evicted across turns or across tasks. Stuffing every historical conversation back into the prompt unconditionally is not reliable memory — and it is not safe either: it brings data-leakage and context-poisoning risks.

The execution plane turns candidate actions into controlled calls — deterministic Workflows, reusable Skills, atomic Tools, MCP connections, and business APIs. The execution plane does not trust natural-language promises; it accepts only structured requests that have passed schema validation, permission checks, and risk-policy processing.

The governance plane cuts across the other three and is responsible for identity and tenant isolation, risk grading, human confirmation, timeouts, retries, idempotency, compensation, audit, observability, and evaluation. The fundamental difference between an industrial system and an ordinary chat application shows up precisely on the governance plane: an inaccurate answer can be corrected, but one wrong device command may produce irreversible side effects.

Figure 7-3 Four-Plane Industrial AI Agent Runtime Architecture

2. The Relationship Among Tool, MCP, Skill, and Workflow

These concepts are frequently conflated. To avoid drift as frameworks change, this book uses the following engineering definitions.

Concept	Definition in this book	Primary question answered	Owns process state?
Tool	An atomic capability with well-defined input, output, and side-effect semantics	"What can be done?"	Usually not
MCP	A connection protocol for AI applications to discover and call external Tools, Resources, and Prompts	"How are capabilities exposed and connected in a standardized way?"	Does not own business process state
Skill	A reusable capability package for domain-specific tasks, composing prompt templates, knowledge, Tools, and Workflows	"How are domain practices reused?"	Depends on the internal implementation
Workflow	A deterministic orchestration of explicit steps, conditions, timeouts, compensation, and approval nodes	"How does a prescribed process run reliably?"	Yes
Agent	The decision-making entity that dynamically selects the next action from the current Context	"What should be done right now?"	Should not bear persistence alone
Agent Runtime	The runtime environment hosting the Agent's lifecycle, state, capabilities, governance, and execution	"How is a task finished safely and continuously?"	Yes

A Tool is a capability, not a complete task. "Query device status" and "write point value," for example, can be two Tools. MCP can expose them to external agents, but it will not automatically orchestrate them into a reliable maintenance procedure. Skill is this book's name for the unit of domain reuse: a "pump-house offline troubleshooting Skill," for example, can contain troubleshooting prompts, equipment-topology knowledge, three read-only Tools, and one human-confirmation Workflow. Frameworks do not yet agree on how Skills are named and packaged, so in engineering you must make explicit what a Skill contains — a label alone is not enough.

Workflow and Agent are not substitutes for each other either. Workflows suit processes with stable steps, clear responsibilities, and known failure compensation; agents suit tasks whose goal is clear but whose path must be chosen dynamically from on-site information. The combination the industry commonly uses is: the Agent chooses the path, the Workflow guards the critical steps, the Tool performs the atomic action, MCP connects external capabilities, and the Runtime manages the whole lifecycle.

3. Task State Matters More Than the "Thinking Loop"

ReAct (Reasoning + Acting) explains how a model loops among "reason — act — observe," but a production system also needs a task state machine independent of the model. A minimal set of states includes:

RECEIVED → CONTEXT_READY → PLANNING → POLICY_CHECK
                                      │
                         ┌────────────┴────────────┐
                         ▼                         ▼
                WAITING_APPROVAL                RUNNING
                         │                         │
                         └──────────→ VERIFYING ←──┘
                                      │
                         ┌────────────┼────────────┐
                         ▼            ▼            ▼
                    SUCCEEDED       FAILED      CANCELLED

The state machine must be persisted by the Runtime, not left to the model "remembering where it got to." Every task must persist at least run_id, tenant and operator, target resources, current state, deadline, Tools already called, idempotency keys, approval records, and a summary of side effects. Only then, after a model timeout or a process restart, can the system decide whether to retry safely, await the receipt, run compensation, or escalate to a human.

Three classes of failure must also be distinguished here:

1. Decision failure: the plan is incomplete, the evidence insufficient, or the tool choice wrong — return to the context or planning stage.
2. Invocation failure: network timeout, MCP unavailable, or an error from downstream — handle according to the Tool's retry semantics.
3. Uncertain side effect: the command was sent but the receipt was lost — never retry blindly; query device state, use the idempotency key, or escalate to human confirmation.

The third class is the most dangerous, because "no successful response received" does not mean "the device did not execute." This is also why an industrial Agent Runtime must manage state and a ledger of side effects independently.

4. An Illustrative Example: Investigating an Offline Pump House

Taking "pump house 1 is offline — investigate" as an example, the Runtime's chain of responsibility unfolds like this:

1. Accept the task: record the operator, tenant, target pump house, and task deadline.
2. Build context: query the device, the Driver, recent status, and maintenance windows, handing the model only the information the current task needs.
3. Generate a plan: the agent proposes first determining whether it is a single-device failure, a Driver failure, or a network-domain failure.
4. Execute read-only Tools: query device status, Driver status, and the scope of impact; record input, result, and latency for every call.
5. Verify the conclusion: if the Driver is online while a single device is offline, output suggestions for inspecting the field link; if the Driver and its devices are offline at the same time, switch to the Driver-recovery workflow.
6. Enter the deterministic boundary: if the next step is to restart the Driver, the Runtime first checks that the Tool exists, that the caller is authorized, and that the device is inside an allowed maintenance window, then waits for approval per the risk policy.
7. Close out the task: save confirmed facts, unconfirmed hypotheses, execution results, and the follow-up owner. If conditions fall short, end explicitly with FAILED or "escalate to human" — do not let the model paper over failure with natural language.

In this example, the model decides "what to investigate next," while the Runtime guarantees "under what identity each step runs, how far it may go, whether execution is allowed, what happens on failure, and where the evidence is kept." Neither can be missing.

5. Boundaries That Cannot Be Crossed in Industrial Settings

An Agent Runtime can raise diagnostic and O&M efficiency, but it must not disguise probabilistic reasoning as deterministic control. The following responsibilities should remain in PLCs, safety instrumented systems (SISs), edge controllers, or explicit Workflows:

• Millisecond-level real-time control and safety interlocks;
• Fail-safe protections such as e-stop, depressurization, and overload protection;
• Process steps with hard constraints on timing, sequencing, and consistency;
• High-risk physical actions that cannot be reliably compensated.

The Runtime's value is not in letting the model bypass these systems, but in converting human intent into constrained tasks that complete querying, analysis, recommendation, orchestration, and limited execution outside the safety boundary.

Table 7-2: Agent Runtime rollout checklist

Dimension	Questions that must be answered
Context	Are identity, tenant, device scope, and evidence version explicit?
State	Can the task recover after a process restart, and is failure distinguished from uncertain side effects?
Capability	Are the Tool's input, output, side effects, timeout, and idempotency semantics declared?
Orchestration	Where is the boundary between dynamic decision-making and deterministic Workflows?
Governance	Which actions pass automatically, which wait for approval, and which are permanently forbidden?
Recovery	Are retry, compensation, human takeover, and a kill switch available?
Evidence	Are call traces, approvals, receipts, and final states preserved — rather than internal chains of thought?

7.1.4 RAG and Tool-Calling: Extending the Knowledge Boundary

Once large models are plugged into IoT operations, they quickly run into two very real shortcomings. The first is the knowledge boundary: the moment a model finishes training, what it knows is already stale — the variable-frequency drive commissioned last night, the register mapping table updated just now, and the standard operating procedure changed this quarter are all unknown to it. The second is the action boundary: however clever, the model can only output text and cannot put a command on the bus directly. Asked to "restart pump 3," it can only reply, "please log in to the platform, find pump 3 in the device-management screen, and click the restart button." Retrieval-augmented generation (RAG) and Tool-Calling each close exactly one of these gaps: the former lets the model answer questions with up-to-date material in hand; the latter lets the model actually operate equipment.

1. RAG: No More "Answering Out of Thin Air"

The core idea of RAG is straightforward: before generating a reply, the model first retrieves the most relevant fragments of information from an external knowledge base as context, and only then generates. The large language model no longer has to answer from memories sealed in its training parameters — memories that may already be stale, or that never contained your system's proprietary equipment in the first place. In IoT operations, what RAG retrieves typically includes equipment installation manuals, Modbus register mapping tables, historical fault records, standard operating procedures (SOPs), and driver upgrade logs.

A typical retrieval flow: the operator asks in conversation, "this temperature controller reports fault E4 — what should I do?" The system first converts the query into a vector representation, retrieves the most relevant troubleshooting records from the document vector store, and sends them to the large language model together with the original question; the model generates troubleshooting steps and lists the points to check. For IoT DC3, this is an optional intelligent-alarm extension; its current implementation does not yet ship a vector store, a case-ingestion job, or an automated alarm-trigger pipeline. Two levels must be kept apart here: RAG is an extension capability a complete AI-native platform should have, and DC3's current implementation is only part of it — Chapter 14 will spell out the boundary of this capability and the path to adopting it, rather than equating "not yet implemented" with "should not exist."

The engineering difficulty of RAG lies in retrieval quality. If an outdated maintenance record slips into the knowledge base, the model may base its advice on wrong information; if vectorization chunking puts SOP step A and step D into the same chunk, the context the model receives is garbled. Corpora of industrial manuals put chunking strategy to an especially hard test: parameter tables, register mapping tables, and alarm code tables often carry one knowledge point per line, and chunking by a fixed character count slices such a table in half — retrieving half a table is the same as retrieving nothing. The usual engineering approach is structured parsing first: split along the document structure of headings, paragraphs, and tables so a table enters the index as a whole or row by row; at retrieval time this is paired with parent-document retrieval — when a child chunk hits, the section or the entire table it belongs to is returned, ensuring the model receives the complete context. In practice, two engineering measures are typically introduced: document version management and re-ranking of retrieval results. Newly deployed equipment documents must carry a version number, and expired documents are removed from the vector store or down-weighted; retrieved candidate entries are then re-ranked once more by a lightweight ranking model (such as Cohere Rerank or BGE Reranker), ensuring the most relevant documents enter the large language model's context window first.

Here is RAG implemented with LangChain:

from langchain_community.vectorstores import FAISS
from langchain_openai import OpenAIEmbeddings, ChatOpenAI
from langchain.chains.combine_documents import create_stuff_documents_chain
from langchain.chains import create_retrieval_chain
from langchain_core.prompts import ChatPromptTemplate

# Load the O&M knowledge base (equipment documents, SOPs)
embeddings = OpenAIEmbeddings()
# Security note: allow_dangerous_deserialization=True triggers pickle deserialization,
# and loading a tampered index file can lead to arbitrary code execution — only load local indexes you generated yourself and keep properly guarded
vectorstore = FAISS.load_local("iot_knowledge_base", embeddings, allow_dangerous_deserialization=True)
retriever = vectorstore.as_retriever(search_kwargs={"k": 3})

llm = ChatOpenAI(model="gpt-5", temperature=0)
prompt = ChatPromptTemplate.from_template(
    "Answer the question based on the following material:\n\nMaterial:\n{context}\n\nQuestion: {input}"
)
question_answer_chain = create_stuff_documents_chain(llm, prompt)
rag_chain = create_retrieval_chain(retriever, question_answer_chain)

response = rag_chain.invoke({"input": "The No. 2 dust-collector fan keeps triggering high-temperature alarms. What should I do?"})
print(response["answer"])
# Output: retrieved the 2024-08 maintenance record; first step is to check whether the VFD's cooling air duct is blocked.

This code assumes you already have a local vector store holding the equipment's O&M documents and SOPs. In a real production environment, you also need to consider incremental document updates, vector-database performance, and isolation of knowledge bases across tenants.

2. Tool-Calling: From "Saying" to "Doing"

Tool-Calling has the large language model emit a structured function-call request while generating a reply — a function name and parameters, not natural language. The application layer executes the corresponding business logic and returns the result for the model to organize into its reply. The IoT DC3 source contains 10 Tool classes in total; the current Provider explicitly registers 8 of them, and the two unregistered classes do not form default session capabilities (registration list in Section 7.3.1). Nor are tool Beans globally auto-scanned by ChatClient.Builder merely for carrying @Tool — they must be registered through tools(), defaultTools(), or an explicit ToolCallbackProvider.

A typical Tool-Calling sketch (based on Spring AI):

@Tool(description = "Create a new alarm rule")
public String createAlarmRule(
    @ToolParam(description = "Rule name, e.g. 'temperature over limit'") String ruleName,
    @ToolParam(description = "Trigger condition expression, e.g. 'pointValue>100'") String condition,
    @ToolParam(description = "Notification method: sms/email/webhook") String notifyMethod
) {
    return alarmRuleService.create(ruleName, condition, notifyMethod);
}

When the operator says "add an alarm rule for the Line 1 temperature point that sends an SMS above 90 degrees," the large language model parses the intent, automatically calls the createAlarmRule method, and fills in ruleName="Line 01 temperature over limit", condition="line01_temp>90", and notifyMethod="sms". The method returns the rule ID after execution, and the model organizes the result as "rule created." The whole exchange spares the operator the tour through multiple screens to configure it.

The security risk of Tool-Calling deserves particular attention. If the model misreads the intent — reading "pause pump 3" as "shut down pump 3," for instance — a single wrong call can damage equipment. IoT DC3's currently executable write path is PointValueTool.writePointValue: it only creates an Action pending confirmation and does not write the device directly; only after the user confirms does ActionService call PointCommandFacade to submit the command. This flow is implemented by business code and persistent state, not by a @WriteOperation annotation or automatic interception by Spring AI.

3. Using Them Together: Retrieve First, Then Execute

RAG solves the problem of whether the model "knows," and Tool-Calling solves the problem of whether the model "can do." In complex O&M scenarios, the two are often used in series: first retrieve the correct operating steps or parameter template through RAG, then execute the specific operation through Tool-Calling.

A typical dialogue for the combined workflow:

Operator: "The dehumidifier in Workshop 2 keeps tripping; investigate per the standard procedure."
Agent execution:

1. RAG retrieval: the knowledge base hit "DC-DEHUM-02 repeated-trip SOP V2"
2. Step 1: check current status → call PointValueTool to read dehum02/status and dehum02/fault_code
3. Step 2: consult the SOP and determine that fault_code=0xE3 means "compressor over-current"; output a preliminary diagnosis
4. Step 3: recommended actions: perform the on-site checks per the SOP; if a control point must be written, PointValueTool creates an Action pending confirmation
5. Result: return the diagnostic basis and the proposed actions; only after the user confirms and the platform executes successfully may the outcome be phrased as "executed."

Without RAG, the model does not recognize the fault code 0xE3 and has no way to know what the SOP says; without Tool-Calling, the model can only offer text advice like "restart recommended," and the operator still has to hop through several screens to act. With the two combined, the large language model truly turns from "an advisor that can talk" into "a duty operator who can act."

Figure 7-4 RAG + Tool-Calling Workflow

The combination of RAG and Tool-Calling gives the large language model two concrete capabilities in IoT operations at once: the knowledge surface updates as the corpus updates, without waiting for the model to be retrained; and the action surface is converged through the platform's schema, permission, and confirmation checks, so a natural-language promise never turns directly into a device command. The former compresses the time lag of knowledge maintenance; the latter guarantees the determinism of operation semantics. The view now rises from single tool calls to the system-integration level, to see how these capabilities are exposed to external AI agents through a standard protocol.

7.1.5 The MCP Protocol: A Cross-System Interaction Standard

RAG patches the knowledge lag, and Tool-Calling lets the model execute actions. When an IoT platform wants to expose devices, data, and O&M APIs to external AI agents, if every client adapts interface descriptions, authorization, and versions separately, maintenance costs quickly spiral out of control. MCP (Model Context Protocol) provides a uniform way to negotiate, discover, and invoke capabilities.

Tools, Resources, and Prompts Are Not the Same Concept

MCP builds on JSON-RPC 2.0 and divides server-side capabilities into three categories:

• Tools: actions or functions the model can call, with an input-parameter schema, discovered via tools/list and invoked via tools/call.
• Resources: context data the client can read, accessed through methods such as resources/list and resources/read.
• Prompts: enumerable, parameterizable prompt templates, accessed through methods such as prompts/list and prompts/get.

Platform capabilities must therefore not all be labeled Resources, and tools/call must not be described as "calling a Resource." The client negotiates protocol version and capabilities during initialize, and afterwards may call only the capabilities the server actually declares.

Transport and Authorization: stdio and Streamable HTTP

The published 2025-11-25 MCP specification defines two standard transports: stdio and Streamable HTTP. stdio targets local child processes, while Streamable HTTP targets remote HTTP endpoints and replaced the earlier HTTP+SSE transport. The 2026-07-28 document is a release candidate proposing changes such as a stateless lifecycle; readers must distinguish a stable specification, a candidate design, and the project's actual implementation. The IoT DC3 source snapshot exposes a JSON-RPC-handling POST /mcp route in the Gateway, confirming a network-reachable HTTP POST MCP entry. A POST route alone does not prove implementation of every Streamable HTTP GET, SSE, and session semantic. Regardless of the transport subset, the endpoint must enforce authentication, authorization, and access control to Web API standards and must not receive the trust level of a local process.

The specification also defines a Client-declared sampling capability: while processing a request, the Server can ask the Client-side model to generate content. This IoT DC3 MCP endpoint neither declares nor implements the related methods. The source establishes only "not currently implemented," not the product rationale. If it is enabled later, tenant-data boundaries, user consent, model selection, quotas, and the audit surface require separate assessment.

On authorization, MCP's authorization framework is built on the OAuth 2.1 draft, and a client must complete the standard OAuth flow before accessing a protected MCP server — this is where the "OAuth 2.1" in Section 7.6.1 CHK-10 comes from; the mechanism details are in Section 9.5 of Chapter 9 and in Chapter 8.

The Current MCP Boundary in IoT DC3

The IoT DC3 source snapshot 987c96d50 exposes an MCP JSON-RPC entry at the Gateway's POST /mcp, declares protocol revision 2025-06-18 and only the tools capability, implements initialize, ping, tools/list, and tools/call, and accepts notifications/initialized. Resources, Prompts, and Tasks are not declared, nor are their corresponding methods implemented.

Nor is the Tool catalog a "Resource list" generated by scanning Spring AI @Tool methods. The McpOpenApiAggregator in Auth combines the platform catalog in dc3_api and dc3_resource with versioned static OpenAPI snapshots to derive Tool names, descriptions, and input schemas. The Gateway's tools/list then returns the catalog visible to the current caller after OAuth-scope, tenant, permission, and risk-policy filtering. Before every tools/call, the Gateway revalidates the Bearer Token, connection context, Tool visibility, and authorization, then forwards the call to the real REST backend.

{
  "jsonrpc": "2.0",
  "id": 1,
  "method": "tools/call",
  "params": {
    "name": "manager__device__get",
    "arguments": {"id": 1001}
  }
}

This catalog mechanism also answers an attack surface that is easy to underestimate: to the model, a tool description is itself untrusted input. A malicious or tampered MCP server can plant inducement instructions inside tool descriptions (tool poisoning), leading the model to leak data or take privilege-escalating actions in later calls; the tool catalog can also be swapped out quietly — the safe tool listed today may be displaced tomorrow by a malicious implementation under the same name (rug pull); and when an Agent follows one server's guidance to call another server, a confused deputy attack surface forms. DC3's design reduces these risks: Tool definitions come from the Auth-side platform catalog and versioned OpenAPI snapshots rather than arbitrary runtime fetching; the Gateway trims visible Tools by scope, tenant, permission, and risk policy, and every call revalidates the Token and authorization. Static snapshots and local catalog data still require supply-chain verification, change review, and version synchronization; "controlled" must not be read as "inherently trusted." Chapter 8 expands on these emerging attack surfaces.

How MCP Relates to REST and MQTT

REST remains the platform's real business API; MCP sits on top of it with a model-facing Tool catalog and a unified invocation protocol. MQTT and RabbitMQ serve device connections and platform message flows, while MCP serves external AI clients. The three solve different problems: MCP does not replace device protocols, nor does it bypass the existing tenant, permission, and security checks.

An external agent can dynamically discover visible capabilities through tools/list and call several Tools in sequence to complete multi-step tasks such as "find device → find points → read history → generate recommendation." But every step remains an independent controlled call — using MCP grants no higher privilege by default and no automatic execution of high-risk operations.

From MCP to A2A: interoperability among agents (outlook). MCP answers "how does an agent call tools," while A2A (Agent-to-Agent) answers "how do agents discover one another, delegate tasks, and exchange results." A2A uses Agent Cards to describe capabilities and supports task delegation, but adoption should follow real interoperability tests, security models, and ecosystem maturity rather than assuming inevitable scale in a particular year. The MCP specification is also evolving quickly: experimental Tasks appeared in the 2025-11-25 specification, and the 2026-07-28 release candidate proposes further changes such as a stateless lifecycle. None of those standard features may be projected onto IoT DC3 as already implemented. Evolution should be tracked against a named specification revision. For IoT, once a platform exposes Tools through MCP, different agents may still need A2A to coordinate work; a complete solution should assess the MCP Tool layer and A2A orchestration layer separately. IoT DC3 currently implements only an MCP Tool subset, while A2A remains an evolution direction discussed in Chapter 14.

Figure 7-5 MCP Architecture in the IoT Platform

The accurate engineering conclusion: what IoT DC3 currently exposes through MCP is a set of Tools constrained by OAuth and the whitelist, not a complete implementation of every server-side capability in MCP. Protocol knowledge and project implementation must be described separately.

7.1.6 RAG Eval: Layered Evaluation of Retrieval and Generation

A RAG system's ability to return a fluent answer does not mean it has production value. A single answer may have picked the wrong equipment model or document version at the retrieval stage, or retrieved correctly yet added conclusions at the generation stage that exist nowhere in the evidence. To localize problems, evaluation must be split into four layers — dataset, retrieval, generation, and end-to-end task — rather than having a human award one overall score to the final answer.

Fix the Evaluation Set First, Not the Metrics

IoT knowledge has boundaries — tenant, equipment model, firmware version, and validity period. A reproducible evaluation sample contains at least: the question, the expected evidence, the acceptable answer points, whether the system should refuse to answer, the tenant, the equipment model, the document version, and the validity period. The evaluation set should cover six input classes: ordinary answerable questions, questions whose answers do not exist in the knowledge base, new-versus-old version conflicts, expired operating procedures, similar documents across tenants, and questions that require combining multiple pieces of evidence.

Production data must not be randomly split into the index and the evaluation set at the same time — near-duplicate text leakage follows easily. The safer practice is to split by time and document version, and to build a separate adversarial set for high-risk write operations. The evaluation set itself must also be versioned; when equipment is added, firmware upgraded, or manuals replaced, the questions, evidence, and refusal conditions should be updated in step.

Retrieval Layer: Was the Correct Evidence Retrieved?

The retrieval layer does not judge the answer's prose style; it judges the candidate evidence. Common metrics include:

• Recall@k: whether the top k results cover the evidence that ought to be retrieved;
• MRR: whether the first correct result ranks high enough;
• nDCG@k: the ranking quality across multiple relevant pieces of evidence;
• Context Precision/Recall: within the context sent to the model, the proportion of useful content and the coverage of the evidence that should be present;
• Correct-version hit rate: when the answer needs the v4 manual, whether v3 was fetched by mistake;
• Cross-tenant mis-retrieval rate: any content not belonging to the current tenant entering the context counts as a security failure;
• Empty-retrieval rate and P50/P95 latency: used to identify coverage gaps and long-tail cost.

These metrics should be reported for the sparse-retrieval, vector-retrieval, hybrid-retrieval, and hybrid-plus-reranker baselines alike. Showing only the best scheme leaves the reader unable to judge whether the added complexity actually pays off.

Generation Layer: Is the Answer Faithful to the Evidence?

RAGAS research decomposes RAG quality into dimensions such as retrieval relevance, the answer's faithfulness to the retrieved content, and final answer quality. Engineering evaluation should include at least:

• Groundedness/Faithfulness: whether the facts in the answer are supported by the given context;
• Answer Relevance: whether the answer addresses the question rather than reciting the material;
• Citation Precision/Recall: whether citations support the corresponding claims, and whether every key claim carries a citation;
• Unsupported-answer rate: whether the model still fabricates conclusions when no reliable material is retrieved;
• Refusal accuracy: whether both sample classes — those that should be refused and those that can be answered — are handled correctly;
• Operational-step completeness: for equipment maintenance, whether shutdown, confirmation, rollback, or safety conditions are omitted.

Automated scorers carry bias of their own, so high-risk samples should be spot-checked by domain experts, with scoring rationales, evidence locations, and dispute records preserved. Automatic scores suit continuous regression; they must not replace human judgment in publication or production acceptance.

End-to-End Layer: Did It Solve the Real Task?

End-to-end evaluation puts the question, retrieval, generation, and subsequent actions together. Metrics to record include the O&M problem-resolution rate, the expert-review pass rate, the expired-SOP usage rate, the share of refusals escalated to humans, total P50/P95 latency, token consumption, and cost per successful task. For flows involving Tools, also record whether the recommendations in the answer agree with actual device state — but do not mix Tool execution traces into RAG metrics; agent trajectories are evaluated separately in Section 7.5.4.

Question set v3
  → Retrieval config v8 (BM25 + Embedding + Reranker)
  → Retrieval metrics
  → Generation model and prompt v5
  → Faithfulness, relevance, and citation metrics
  → End-to-end tasks, latency, and cost

Failure Classification Is More Actionable Than a Total Score

Every failed sample should be sorted into a repairable category: question misinterpretation, retrieval miss, wrong document version, mutually conflicting context, correct evidence but unfaithful generation, and cases that should have been refused yet returned action recommendations. Different failures lead to different repair entry points: expand the corpus, change chunking, tune filtering, swap the reranker, tighten the prompt, or add refusal strategies. Staring at a single aggregate score usually hides exactly these engineering differences.

Experiment card EXP-7-RAG-01

• Subject: IoT O&M knowledge Q&A;
• Fixed items: corpus snapshot and checksum, chunking parameters, embedding, reranker, generation model, prompt, top-k;
• Baselines: no RAG, BM25, vector, hybrid, hybrid plus re-ranking;
• Metrics: Recall@k, MRR, nDCG, version hit rate, cross-tenant mis-retrieval rate, Groundedness, refusal accuracy, P50/P95, tokens and cost;
• Result requirements: preserve per-sample retrieval results, answers, citations, scoring rationales, and raw logs; mark any item not actually measured as NA rather than substituting illustrative numbers.

The ultimate purpose of RAG evaluation is not to prove one framework more advanced than another, but to build a repeatable chain of evidence: when the corpus, index, model, or prompt changes, the team can tell what improved, what broke, and whether the system still satisfies tenant isolation and the refusal boundary for high-risk tasks.

Figure 7-6 Layered RAG Evaluation

← 6.4 Engineering Summary and Further Reading7.2 Spring AI and IoT Integration →