1.5 The Paradigm Shock of Large AI Models
1.5.1 From Passive Connection to Active Intelligence: The Reasoning Capability of Large Models
The operating model of traditional IoT can be summarized as a fixed "sense-respond" loop: devices collect data, and the platform reacts against pre-set rules. This model works well in scenarios with clear boundaries — warehouse temperature control, environmental monitoring: temperature out of range triggers an alarm; CO₂ concentration above threshold turns on fresh air. But when the number of connected devices grows from a few dozen to tens of thousands, the number of rules swells sharply and maintenance costs climb quickly. Run a rough calculation: with N devices and M states each, requiring the state combinations of any two devices to interlock puts the rule count on the order of O(N²M²) — even with only 50 devices, each with just the two "on/off" states, the combinatorial rules already reach ten thousand, before counting added conditions such as time slots and thresholds. More critically, a rule engine is essentially an "If-Then-Else" branching structure; it cannot handle vague descriptions or composite, context-dependent scenarios. When a user says "it feels stuffy," the rule engine can only wait for a pre-configured measurement to exceed its limit — it cannot understand the word "stuffy"; it only recognizes "CO₂ > 1000 ppm."
Today's mainstream large language models (LLMs) combine language understanding and generation, and some also accept multimodal input such as images and audio. Once connected to an IoT system, the model can "understand" the contextual meaning of the data devices report, rather than merely look up values. For example, when a user says "the room feels a bit stuffy," a traditional rule engine does nothing; a large model, drawing on context such as temperature and humidity, CO₂, and the user's window-opening habits, can infer the best action — turning on fresh air and fine-tuning the blind angle, say, instead of simply firing one pre-set rule. Behind this sit the model's attention mechanism and probabilistic reasoning: it is not matching fixed conditions, but computing "given the current state, what is the most reasonable set of actions."
Figure 1-16 compares the decision chains of the two models. The left side is the traditional rule-driven path: the user enters commands through a fixed control panel or app, the rule engine matches them exactly, and the devices execute directly. The right side is the new path with a large language model in the loop: the user describes the need in natural language, the large model parses the intent, queries the devices' real-time state, generates a decision plan and presents its reasoning, and only after the user's second confirmation is the plan executed.
Embedding reasoning capability into IoT requires solving several engineering problems. Standardized data formats are the precondition: devices report binary point values or JSON messages, which must be converted through prompt templates into structured natural-language descriptions. Latency and cost also need balancing: large-model inference typically takes hundreds of milliseconds to several seconds, and does not suit real-time control that must respond in under a second. The current industry consensus places the large model in the "decision engine" position of the platform layer, while real-time closed loops remain the responsibility of edge rule engines or lightweight models. This is essentially a hybrid decision architecture — tasks are layered by response-time window and complexity. Some open-source IoT platforms are already exploring this route: they integrate a large-model interface on the platform side as a high-level decision layer, while keeping real-time control loops at the edge.
The probabilistic output of large models is no panacea. The same input may yield different results, and "hallucinations" can occur — judgments that look plausible but are in fact wrong. Introducing large models into IoT systems must therefore be paired with "sandbox validation" and "high-risk action confirmation" mechanisms: the model may propose actions, but an operator must confirm a second time before execution. This design couples the model's reasoning strength with the human's final authority of judgment, instead of letting a black-box model directly control physical equipment. From the perspective of engineering evolution, this "propose-confirm" pattern fits the industry's current risk appetite better than fully automated reasoning.
When IoT shifts from rule-driven to reasoning-driven, does the system architecture need redefining? Do device ends need local models? How should the cloud-edge collaboration model be adjusted? The next section uses an illustrative smart-home case to show how large-model control changes everyday interaction, and from there leads to the architectural adjustments it demands.
1.5.2 Case Study: From Rule Engine to Large-Model Control in the Smart Home
Rule engines have long been the core of smart-home automation: if the temperature falls below a pre-set threshold, turn on the air conditioner; if the door/window sensor detects an opening, shut off fresh air. This pre-set logic is predictable and runs stably, but the moment a user's expression falls outside the pre-defined conditions, the system fails completely. Large models open a new route beyond that control boundary. The example below compares the two paths (all device parameters and control temperatures are for demonstration only), making it clear where the change lies.
Example: The user says "I feel a bit cold." A traditional rule engine must map that sentence onto one definite IF branch. Suppose an engineer wrote this rule: "when the indoor temperature is below 20 °C and the time slot is 18:00–22:00, start the air conditioner in heating mode and set it to 26 °C." If the room is slightly above 20 °C when the user says "cold," the rule never fires and the system does nothing at all. The greater challenge: a window in the room is open and cold outside air is pouring in — the rule engine has no idea that "window state" and "cold" are related, because window state is not among that rule's conditions. The result is a fragmented table of control logic: temperature goes through temperature rules, windows go through window rules, and the two never meet.
With a large language model as the control hub, the processing path is completely different. After the user says "I feel a bit cold," the system first performs intent understanding: it recognizes that "cold" is an intent about thermal comfort, not a literal temperature. It then pulls the environmental context: indoor temperature slightly below the comfort band, humidity normal, window open, outside temperature distinctly low and wind rather strong. It then executes multi-step reasoning: the open window is letting heat escape (cause); closing the window reduces the inflow of cold air (action 1); then enable the air conditioner's heating mode to replenish heat (action 2), with the target temperature set to a lower level to avoid overheating from the stacked warming after the window closes (action 3). Throughout, the user expressed only a vague feeling and specified no device parameters at all.
Code implementation comparison
A rule engine needs engineers to pre-write the combination logic entry by entry; every new device or new scenario means adding or editing rules. The pseudocode:
// Rule-engine pseudocode: engineers must pre-write every combination
Rule: "Night_Heating"
WHEN:
time_slot IN ["18:00-22:00"] AND
indoor_temp < 20 AND
window_state IS "CLOSED"
THEN:
set_ac_mode("heat")
set_ac_temp(26)
END_RULEThe large model performs reasoning through an API and needs no pre-set condition branches. The call below, with interface and parameters for demonstration only:
# Large-model dynamic reasoning (code)
user_text = "I feel a bit cold"
env_context = """
Indoor temperature: slightly below the comfort band, humidity normal;
Window is open;
Outdoor temperature distinctly low, wind rather strong.
"""
from openai import OpenAI
client = OpenAI()
resp = client.chat.completions.create(
model="your-model",
messages=[
{"role": "system",
"content": "You are a smart-home hub. Based on the environmental context and the user's intent,"
"generate device-control command JSON. Available devices:"
"[air conditioner (mode, temperature), window (open/close)]."},
{"role": "user",
"content": f"Current state: {env_context}\nUser said: '{user_text}'"}
]
)
# Returned result:
# {"reasoning": "The open window lets cold air in; close it first, then heat.",
# "steps": [
# {"device": "window", "command": "close"},
# {"device": "ac", "command": "set_mode", "value": "heat"},
# {"device": "ac", "command": "set_temp", "value": "lower"}
# ]}The large model plays the role of a "digital butler": it receives vague intent, queries environmental data, reasons out a feasible plan, and dispatches it for execution. This does not mean the rule engine is wholly replaced — in production deployments, the rule engine still handles fast, predictable device-level execution control; what the large model takes over is the understanding and planning work that used to require engineers to write rules and match parameters one by one.
Engineering concerns: Deploying large-model control in a production environment means handling latency, safety boundaries, and cost. The common practice is to keep the rule engine as the fallback while the LLM handles only priority judgment and combined recommendation, with commands still dispatched through the original execution channel. This separated "reasoning layer + execution layer" architecture is the mainstream approach for putting large-model control into production in industry today. The smart-home case shows clearly: when the user's need is a vague feeling rather than a precise numeric command, the large model changes the human-thing interaction at the architectural level — from "imperative" toward "intent-driven."
Figure 1-17 returns to the "stuffy" case from Section 1.5.1 and draws the two chains — rule engine and large model — side by side: the same word "stuffy," yet the two chains give completely different answers.
1.5.3 Paradigm Change: Restructuring at the Architecture Level
When large language models enter IoT, the first wall they run into is not algorithmic accuracy but how computing resources are distributed. A model with billions of parameters needs compute and energy for a single inference far beyond the capability boundary of traditional IoT devices. Forcing an entire LLM into a microcontroller (MCU) is neither realistic nor economical under current technology. This forces a fundamental shift in IoT topology: no longer a plain "device-cloud" data pipe, but a gradual evolution toward a three-layer "edge-device-cloud" collaborative AIoT architecture.
The compute-intensive nature of large-model inference is the first driving force of the architectural restructuring. A single response requires massive floating-point computation and memory bandwidth, several orders of magnitude beyond the lightweight inference on traditional IoT devices (decision-tree classifiers or simple threshold checks, for example). The natural "home" of large models is the cloud data center. But that creates an engineering dilemma: if every intelligent decision on the device side must wait for the cloud model to finish inference and return a result, network latency and bandwidth costs will choke most real-time applications. Take an industrial example: for abnormal-vibration detection on a robotic arm, the time window from sensor capture to e-stop actuation is extremely short — it simply cannot afford an end-to-end round trip to the cloud for inference.
The layered strategy of "the edge blocks the first wave, the cloud handles the hard cases" is the key to resolving this contradiction. Take a factory production line: the cloud-side large model can precisely diagnose dozens of equipment faults, while lightweight models at the edge locally complete identification and alarming for most common anomalies; only the hard cases they cannot judge are uploaded to the cloud for processing. Cloud call frequency and device response latency drop sharply as a result — and this division of labor does not require edge devices to have full large-model capability.
The core pattern of the new architecture: lightweight models on the device + large models in the cloud, collaborating. The device side (MCU/sensors) stays at minimum power, responsible only for data collection and key wake-up events; the edge side (gateways/compute boxes) runs compressed inference models, taking on real-time decisions and local closed-loop control; the cloud side handles the training, fine-tuning, and complex multi-step reasoning of large models, and periodically pushes the updated models down to the edge, forming a continuous optimization loop. How the lightweight models are compressed out of the large model — distillation, quantization, and other concrete techniques — is covered in Section 1.6.2.
The difference between the traditional architecture and the new AIoT architecture is plain at a glance in Figure 1-18.
Take IoT DC3 as an example. Its Agentic Center is built on Spring AI and explicitly registers controlled @Tools for tenants, users, devices, Drivers, profiles, points, point values, and system operations. Command and Event Tool classes that exist in source but are not registered cannot be counted as currently available capabilities. A point write first creates an Action awaiting confirmation before it enters the platform command path, so this is not an LLM acting directly on a device. Meanwhile, the Gateway exposes a separate platform Tool catalog, trimmed by permissions and policy, to external AI agents through MCP (Model Context Protocol). The two entries reuse platform governance but do not share the same catalog source. AI reasoning and action are thus embedded in the existing IoT pipeline and divided from deterministic real-time response at the edge.
The key to adjusting the architecture is to place probabilistic models where they belong: the device handles sensing and execution, the edge carries low-latency rules and lightweight inference, and the cloud handles knowledge-intensive analysis; the exact boundary is still determined by safety, latency, bandwidth, privacy, and cost. A model may generate recommendations or candidate actions, but final execution must pass through permissions, policies, and a feedback loop.