11.3 Ultra-Large-Capacity Architecture Challenges

Author: Zhang Hongyuan · © 2016–2026 · All Rights Reserved · Copyright & License

11.3.1 Architecture Challenges of Million-Scale Device Access

A connected vehicle reports GPS coordinates, speed, acceleration, tire pressure, and battery voltage to the cloud every second — a few dozen data items. Roadside units (RSUs) broadcast traffic-signal phases, traffic flow, and weather information at even higher frequencies. Each smart lamp pole simultaneously handles lighting control, photographic enforcement, and environmental monitoring. Suppose a new district plans a typical deployment of 200,000 lamp poles, 100,000 roadside sensors, and several hundred thousand connected vehicles — these figures are illustrative only, yet they already approach the real boundary a city-scale IoT platform must face.

Morning and evening rush hours, major sporting events, or sudden accidents push device reporting frequencies up in an instant. Unlike industrial IoT, where access volume typically runs from a few thousand to a few tens of thousands of devices, the load profile of city-scale scenarios is clear: individual messages are small (tens to a few hundred bytes), while connection counts and message frequencies are an order of magnitude higher. The platform must not only receive this data but also complete forwarding, storage, and response within milliseconds.

The pressure of concurrent connections first shows up at the protocol layer. TCP long connections require the server to maintain socket handles, send/receive buffers, and heartbeat timeout detection. Take a typical 16-core, 32 GB cloud server: in a pure MQTT long-connection scenario it can realistically sustain roughly tens of thousands to a hundred thousand connections (an experience-based estimate for common configurations; the actual figure depends on application-layer logic, log writes, and memory-allocation policy). Scaling up relieves the pressure only linearly, while scaling out introduces problems of even connection distribution and business consistency, which demand a precise load-balancing strategy. Intermittent device disconnects and reconnects further amplify connection churn.

Another easily underestimated bottleneck is the concurrent shock of device authentication. Suppose a large number of devices come online in the same window — for example, roadside systems running a unified self-check before the morning rush — the platform may receive tens of thousands of login or authentication requests within a few seconds. If every authentication queries a relational database, response time quickly degrades to unacceptable levels. Common practice is to pre-issue tokens or cache authentication results in Redis, cutting average authentication latency from hundreds of milliseconds to the microsecond level.

When device messages actually pour in, the test of data throughput follows. Suppose each vehicle reports 10 messages per second at 200 bytes each, with 100,000 vehicles online simultaneously — the ingress traffic is about 200 MB/s. And that is only from vehicles. Add roadside devices and sensors, and a city-scale IoT platform's input throughput easily reaches the level of a million messages per second. If any single point in the message-processing chain blocks — say a single-threaded consumer, or insufficient database write throughput — the entire pipeline builds backpressure, ultimately appearing as message backlog and timeout retries on the device side, forming an avalanche effect.

Horizontal scalability should be a design goal from the start, not an after-the-fact remedy. For an MQTT broker cluster, horizontal scaling hinges on two points: message routing must not depend on a central node (otherwise that node becomes the bottleneck); and client connections must be evenly distributed across brokers, usually achieved through a load balancer's hashing strategy. For message queues, the number of partitions determines the maximum concurrent consumption capacity — as a rule of thumb, set the partition count to at least twice the number of consumers to reserve processing headroom.

Scalability needs no home-made formula; the systems field already offers a ready theoretical reference. Amdahl's law states that the portion of a system that cannot be parallelized caps the achievable speedup; the Universal Scalability Law (USL) that Neil J. Gunther built on top of it goes one step further: coordination and consistency overhead between nodes grows superlinearly with scale, pushing the scaling curve past its peak and then pulling it back down — keep adding nodes and throughput actually falls. Mapped onto an MQTT broker cluster: with a centralized coordination node, coordination overhead grows roughly with the square of the node count, and horizontal scaling quickly turns uneconomical; with stateless brokers plus external session storage, coordination overhead is pressed down to nearly a constant, and throughput grows nearly linearly with the node count. The empirical conclusion compresses into one sentence: when coordination overhead grows faster than linearly, scaling is already uneconomical — eliminate the coordination bottleneck before talking about expansion.

The following table summarizes key performance indicators and engineering rules of thumb for million-scale access scenarios. All values in the table are ranges based on typical engineering scenarios.

Table 11-7 Performance indicators and engineering rules of thumb for million-scale access

Indicator	Operating environment	Rule of thumb and strategy
Concurrent connections	200,000 lamp poles + 100,000 RSUs + 700,000 in-vehicle terminals (an illustrative scale)	Keep a single MQTT broker's connection count in the tens of thousands; beyond that, scale horizontally, combined with session persistence
Message throughput	In-vehicle terminals reporting every second, roadside devices every few hundred milliseconds	When peak throughput exceeds one million messages/second, introduce a message queue to shave peaks and a stream-processing engine for aggregation
Protocol overhead ratio	MQTT's minimal 2-byte header + payload vs HTTP/1.1's fixed headers of several hundred bytes	Prefer MQTT for long-connection scenarios; consider CoAP for scenarios with sleeping sensors
Authentication shock	Tens of thousands of simultaneous authentications during unified device startup (an illustrative scenario)	Cache tokens in Redis to avoid querying the database on every request
Storage write I/O	Several hundred thousand time-series writes per second	Use a partitioned write strategy with columnar storage or a time-series database (such as TimescaleDB)

The impact of protocol overhead also belongs in the design-phase evaluation. MQTT's packet structure, QoS tiers, and long-connection mechanism have already been taken apart one by one in the protocol comparison of Section 9.1 and the MQTT walkthrough of Section 9.2, so here we only settle the city-scale selection conclusion: massive long-connection device access is led by MQTT; battery-powered nodes that report only occasionally can be evaluated for CoAP, at the cost of accepting its weaknesses in NAT traversal and reliable delivery; the request/response model of the HTTP-family protocols is inefficient for low-power device-side scenarios and is generally reserved for platform-to-platform integration. For a city platform, the bottleneck of access capacity often lies not in packet size but in how efficiently the broker itself multiplexes connections — a dedicated MQTT broker, through optimized message scheduling, can support tens of thousands to a hundred thousand concurrent connections per node under typical configurations (estimated from common cloud-server configurations); beyond that, horizontal scaling is required.

The core tension in server pressure lies in the trade-off between state maintenance and statelessness. Long connections lower handshake costs, but every server must maintain connection state; once a server crashes, all connections it holds are severed, and clients must reconnect and restore their subscriptions. In production deployments, MQTT clusters therefore usually adopt "shared subscription" and "session persistence" strategies: device state goes into external Redis or a database, and broker instances themselves become elastic nodes. This design improves the elastic scaling of nodes but adds the overhead of cross-node state lookups on every message publish.

Engineering checklist for million-scale access (for planning reference)

1. Connection layer: Is the MQTT broker cluster horizontally scalable? Is session affinity configured on the load balancer?
2. Authentication: Are tokens pre-issued or cached, to absorb the authentication peak when devices come online in bulk?
3. Message processing: Is a message queue in place to shave peaks and fill valleys? Are Kafka partitions set to at least twice the number of consumers?
4. Protocol choice: Is MQTT the first choice for long-connection scenarios? Has CoAP been evaluated for battery-powered sensors?
5. Storage design: Does the time-series database use a partitioned write strategy, to avoid a single-point write bottleneck?
6. Disaster recovery: Is session persistence implemented, so that devices can quickly reconnect and restore state after a broker node fails?
7. Load testing: Have tests been run at key connection counts (such as 100,000, 500,000, 1,000,000), with throughput and latency targets verified?

---

Figure 11-6 Million-Device City IoT Access Architecture

Capacity Estimation: Turning "Million-Scale" into Recomputable Parameters

"Million connections" is often written as a marketing figure; a publication-grade chapter should offer a recomputable, parameterized model. Given the number of devices N, the average heartbeat period T_h, the average business period T_b, and the peak multiplier K, an empirical estimate of the peak message rate follows:

QPS_avg  = N × (1/T_h + 1/T_b)
QPS_peak = QPS_avg × K
total_daily_messages = QPS_avg × 86 400
required_broker_shards ≈ QPS_peak / broker_capacity
timeseries_write_throughput ≈ QPS_peak × points_per_message

A worked example:

• N = 1,000,000, T_h = 60s, T_b = 5s, K = 5, giving QPS_avg ≈ 2.17×10⁵ and QPS_peak ≈ 1.09×10⁶;
• a single MQTT broker with a steady-state throughput ceiling of QPS_ceiling = 200 k needs at least 6 shards, and a real deployment should keep 30%–50% redundancy for failure recovery;
• with 8 points per message, the time-series store must sustain roughly 8.7 M points/s, corresponding to 3–5 write nodes; write amplification and index choice need dedicated evaluation.

Table 11-8 Suggested template for capacity-estimation parameters

Parameter	Definition	Suggested source
N	Target number of connected devices	Project SOW / contract
T_h, T_b	Heartbeat and business periods	Device profiles and scenario requirements
K	Peak amplification factor	Scenario load testing or historical data
broker_capacity	Per-node steady-state throughput	Target broker product / self-testing
storage_ratio	Message-to-time-series data ratio	Data contracts and point counts
Redundancy factor	Failure-recovery headroom	Target SLO

The capacity model is not a precise formula but a decision tool: the moment any parameter changes — for example, T_b shrinking from 5s to 1s — every downstream resource must be re-estimated. A marketing claim of "million connections" that cannot be recomputed along this model does not qualify as publication-grade measured data.

Data Governance and Cross-Department Permissions

City AIoT systems often span many departments — traffic, energy, public security, fire protection, health, housing and construction — with data simultaneously belonging to different legal entities and functions. Engineering-wise, the governance contract must be put on the table from day one:

• For each data category, spell out "data subject, controller, processor, and sharing scope," build a data catalog, and bring it under the platform's compliance audit;
• Cross-department sharing is authorized on demand, with explicit data purpose, time limit, de-identification level, and refusal conditions; once revoked, access can be recalled or invalidated in downstream systems;
• Access granted to agents, AI analytics, or third-party developers is audited separately, distinguished from the permissions held by data subjects;
• Data for city dashboards, public portals, and research projects must go through de-identified or synthetic channels — never raw production data;
• When emergencies, disasters, or public safety temporarily require elevated access, use a separate approval process with after-action review — never treat it as routine authorization.

Cross-department governance is not a paper document — it requires capabilities implemented at the platform layer: tenant models, role matrices, approval workflows, audit events, public interfaces. Without platform support, data sharing inevitably degrades into "issue a document first, then have people move data by hand," and AI systems can hardly run automatically in such an environment.

Spatiotemporal Data Contracts and Real-Time Access

City-scale systems place additional requirements on spatiotemporal data; recommendations for a publication-grade implementation:

• Every record carries a timestamp, spatial coordinates (latitude/longitude or WGS84/CGCS2000), coordinate-system version, and precision;
• Time is recorded twice, in UTC and the local time zone, to avoid drift from daylight-saving or time-zone changes;
• Spatial indexing uses standard tiles such as H3, S2, or Geohash; avoid mixing them within one system;
• Once V2X, AI vision, and signal control form event streams, they should also be linked to ground topology through "spatiotemporal joins," rather than reporting data by device ID alone;
• Privacy-sensitive spatial data (such as personal trajectories and home addresses) is treated with anonymization or differential privacy, and must never be exposed directly in raw tables;
• A city data platform should support replay: given a time and space range, it can reproduce the states and alarms of that moment, for after-action review or algorithm validation.

Only by considering capacity, governance, and spatiotemporal contracts on the same layer can a city AIoT system's "scaling up" go beyond "stacking up more dashboards" and become a runnable, auditable, extensible engineering system.

11.3.2 Message Queues and Data Stream Processing

The previous section sketched the engineering outline of million-scale concurrent device access: connected vehicles driving through the city road network, environmental sensors under lamp poles, and RSUs at intersections, all pouring messages into the cloud at hundreds of thousands per second. The mechanism details of the generic pipeline of "message-queue buffering and decoupling, parallel computation on the consumer side" — Kafka's persistence strategy, partitions and consumer groups, fault-tolerance measures — were already laid out in Section 5.2; this section does not repeat the principles but turns the lens on city-scale parameters: what a message rate of hundreds of thousands per second means for partition planning, consumer parallelism, and stream-processing windows. If the backend system terminated these devices' TCP long connections directly, thread blocking and memory exhaustion would be almost inevitable. The thornier problem is that the data is highly heterogeneous — real-time road conditions, pollutant concentrations, traffic flow, violation photos — each with its own processing latency and computation logic. With upstream and downstream tightly coupled, an upgrade or failure on either side ripples through the whole chain, and platform maintainability is out of the question.

The message queue is the standard decoupling solution. It separates senders (producers) from receivers (consumers): devices no longer connect directly to business services but deliver messages to the queue's topics; the backend's real-time stream-computation engines, AI inference services, and storage systems each consume the topics they care about as subscribers. This architecture lets a city IoT platform withstand traffic spikes and tolerate partial failures, while enabling parallel scaling of different processing logic.

Technology choice: Kafka or RocketMQ?

For city-scale IoT message throughput, Apache Kafka and Apache RocketMQ are the two open-source middleware packages most discussed in engineering circles. Both support the publish-subscribe model and horizontal scaling, but they differ markedly in design philosophy and applicable scenarios.

Kafka was originally designed for log aggregation; its core strength is high-throughput sequential writes. Messages are appended to partitioned logs, consumer offsets are managed by the clients themselves, and it can support coordinated consumption across large numbers of producers and consumers. Kafka's horizontal scalability underpins city-scale throughput: adding partitions and broker nodes raises write capacity — a linear-scaling property widely recognized in the industry. For the massive time-series data produced by GPS reporting and traffic-flow detection in city traffic scenarios, this implementation of sequential writes and zero-copy consumption is a near-perfect match.

RocketMQ comes from e-commerce scenarios; it likewise pursues high throughput but emphasizes reliable delivery and flexible transactions. It natively supports transaction check-backs, delayed messages, and message-trace tracking, making it suitable for business scenarios that need exactly-once semantics — for example, smart-parking billing commands or emergency-response dispatch confirmations. RocketMQ guarantees no message loss through a file-based storage structure and synchronous flushing, at the cost of slightly higher write latency than Kafka under extreme pressure.

The typical practice for a city IoT platform is a hybrid deployment: Kafka for data pipelines with heavy writes and light reads, such as mass sensor status reporting and connected-vehicle trajectory collection; RocketMQ for short-message channels that need transactional guarantees, such as command dispatch and payment deduction. The two queues expose a standard topic interface through a unified middleware layer, transparent to upper-layer applications.

Partitioning is the key to throughput

In both Kafka and RocketMQ, a topic is only a logical classification; the real unit of parallelism is the partition. One way to picture it: a topic is a multi-lane highway, and each partition is one lane. Producers are like cars at the entrance, merging into free lanes in parallel; different consumer instances within a consumer group are like toll stations along different segments, each channeling the traffic in its own lane. Both the read side and the write side scale linearly.

Kafka guarantees ordering within a partition and imposes none across partitions. If one sensor's data must be processed in strict time order, all of its messages must be routed to the same partition. The common routing strategy takes the device ID modulo the partition count: data from the same lamp pole or the same vehicle always lands in a fixed partition, so the consumer side can rebuild the event sequence in arrival order, avoiding the performance cost of locking and sorting the whole topic.

The partition count directly determines consumer-side concurrency. Kafka has a basic constraint: a partition can be consumed by only one consumer instance within a consumer group. If partitions are fewer than consumers, the surplus consumers sit idle. Planning partition counts involves a trade-off: more partitions raise read/write parallelism but also increase file-handle counts and metadata-management overhead on the brokers. By industry experience, high-throughput topics (for example, traffic-flow status reporting) typically start with a modest number of partitions and grow gradually with actual consumption pressure, rather than being oversized from the start.

Integrating real-time stream processing

The message queue itself buffers and dispatches; the real computational value emerges on the consumption side of stream-processing engines. Apache Flink and Spark Structured Streaming are the real-time computation frameworks most often paired with message queues, pulling data from the queue and running continuous analysis in different ways.

The Kafka-Flink integration is especially tight. Flink wraps the Kafka consumer as its own Source Operator and builds in exactly-once processing guarantees. When a Flink checkpoint completes successfully, it automatically commits the Kafka consumer offsets, ensuring that recovery after a failure neither re-reads nor skips data. Under this mechanism, a typical real-time stream-processing pipeline for city traffic is shown in Figure 11-7.

Figure 11-7 City IoT Messaging & Stream Processing

Flink jobs run on a cluster, receiving messages from devices such as traffic-flow detectors and signal-status reporters, executing windowed aggregation (for example, counting traffic flow per intersection in tumbling windows), and outputting a refined stream to downstream AI prediction services. The stream-processing engine plays the role of "cleaning and refining": starting from the massive raw data in the message queue, it executes predefined computation logic (filtering dirty data, enriching device metadata, averaging over time windows), then writes the processed results back to another queue or directly into a storage system.

Spark Structured Streaming defaults to a micro-batch model, slicing the real-time stream into small batches at intervals of a few seconds and executing them batch by batch with the batch engine. This approach is simpler for scenarios with less stringent latency requirements (second-level response), such as energy-consumption optimization and statistical analysis. As long as the Spark application connects to the Kafka data source through the readStream interface and reads broker addresses and topic names from a configuration file, the development work focuses mainly on tuning the batch interval and partition mapping.

Combining message queues with stream-processing engines shifts city IoT data processing from "store first, compute later" to "compute as it arrives." Sensor data can be filtered and aggregated at millisecond level without ever touching disk, triggering emergency responses or adaptive signal adjustment. This is the key engineering support for a city platform's "sense–analyze–control" data loop.

The following is a sample Kafka consumer and Flink job configuration, illustrating parameter settings commonly seen in engineering (an example, not a real project configuration):

# Illustrative scenario: a Kafka + Flink configuration snippet for a smart-traffic platform in a new district
kafka:
  bootstrap.servers: "broker1.ny-city-iot:9092,broker2.ny-city-iot:9092"
  consumer.group.id: "traffic-flink-cg-01"
  auto.offset.reset: "earliest"
  enable.auto.commit: false
  session.timeout.ms: 30000
  max.poll.records: 1000

flink:
  job.name: "UrbanTrafficStreamProcessor"
  execution.mode: "STREAMING"
  parallelism.default: 8
  kafka.source.topic: "traffic_raw_msg"
  sink.topic: "traffic_5min_stats"
  window.size.seconds: 300
  checkpoint.interval.ms: 30000

stream.process:
  - type: filter
    condition: "is_valid(sensor_id) && reading_type == 'vehicle_count'"
  - type: enrich
    with: "device_metadata_cache"
  - type: aggregate.windowed
    key: "intersection_id"
    metric: "vehicle_count"
    function: "sum"

In this example, this set of configuration lets the Flink job consume the traffic_raw_msg topic at a given parallelism, aggregate intersection traffic flow over the specified time window, and write the results to a downstream topic. The checkpoint interval must ensure recovery from the most recent checkpoint when a node fails. The consumer disables automatic offset commit, leaving it to Flink's checkpoint mechanism — the standard practice for guaranteeing data consistency in production.

One design decision deserves note: the example above embeds the Kafka connection parameters directly in the Flink job, but in a microservice architecture the more common practice is to externalize connection parameters and topic mappings into a configuration center (such as Consul or Nacos), allowing consumption behavior to change dynamically without restarting the Flink job. City-scale IoT platforms usually involve many collaborating teams, and centralized configuration management improves the resilience of the overall architecture.

Back to the original question: the ability to absorb data floods depends not only on the size of the message-queue cluster but, more importantly, on how the consumer side organizes partitions and how stream-processing jobs set parallelism and windows. As the stable buffering layer, the message queue must withstand million-scale concurrent writes while applying automatic backpressure when consumption-side pressure rebounds, preventing consumer crashes. Kafka's slow consumers adapt by throttling their pull frequency; RocketMQ retries failed consumption until messages reach the dead-letter queue — both provide engineering guarantees that "a data flood cannot crush the system."

11.3.3 Cloud-Edge Collaboration Architecture Design

Message queues solve asynchronous decoupling and peak shaving between backend components, but city IoT faces a more fundamental bottleneck: when hundreds of thousands of devices generate data continuously at short intervals — sensors reporting every 100 milliseconds, cameras outputting dozens of frames per second — funneling all raw data to the cloud for processing makes network bandwidth and transmission latency an insurmountable limit. The layered principle of "the edge handles real-time response, the cloud handles global optimization" was established in Section 5.3; this section does not restate the principles but migrates it to capacity governance for million-scale urban concurrency: which tier an edge node sits on, by what criteria tasks are offloaded, and how the conclusions change once the parameters are scaled up by an order of magnitude. The inherent delay of physical transmission cannot be eliminated by software optimization.

The industry introduced edge computing to address this tension. The core idea is to sink part of the computing and decision-making capability to edge nodes close to the data source, so that data completes initial processing and rapid response locally; only the "roughly processed data" — aggregated, filtered, or preliminarily analyzed — is uploaded to the cloud. This architecture is called cloud-edge collaboration. The edge handles rapid response and initial filtering; the cloud handles global optimization and continuous iteration.

Edge Node Placement

In city IoT scenarios, edge nodes fall into three tiers by deployment location and computing capability, each resolving a different tension between latency and bandwidth:

• Roadside edge nodes (RSUs): closest to end devices, deployed at the roadside and connected to sensors such as traffic signals, cameras, and radar. Real-time requirements are the most stringent and computing resources relatively limited, so embedded platforms are common. Typical applications include local signal-phase switching, forwarding and filtering of V2V safety-warning messages, and local OBU verification. RSUs can also distribute digitized traffic-signal information to connected vehicles, addressing the reliability problem of relying solely on visual detection of traditional signal lights.
• Aggregation edge nodes (base stations / aggregation rooms): covering a block or district, usually deployed as edge gateways or small server racks co-located with 5G base stations. More computing power than an RSU, capable of running lightweight AI inference models; they aggregate data from multiple RSUs and perform preliminary analysis such as short-term traffic-flow prediction.
• Regional edge nodes (district data centers): deployed in district-level data centers with computing resources close to cloud specifications, responsible for data caching, protocol conversion, local model inference, and data synchronization with the cloud. As the intermediate layer between cloud and RSUs, they play the role of data forwarding and model caching.

Task Offloading Strategy

The central engineering decision is: which tasks run at the edge, and which go to the cloud? The decision rests on three dimensions:

1. Latency sensitivity: tasks with extreme latency requirements (typically within 10 milliseconds) — collision warnings, emergency braking — must be offloaded to RSUs; tasks with higher tolerance, such as historical data analysis or secondary video audits, can go to the cloud.
2. Data volume and sustained throughput: performing object detection and event extraction on high-bitrate video streams at the edge (the output being only cropped images and structured messages) saves substantial backhaul bandwidth. Low-throughput environmental sensor data (a few KB per second) imposes acceptable bandwidth pressure when uploaded to the cloud.
3. Computing-resource heterogeneity: edge nodes commonly use embedded GPUs or NPUs. Training and inference placement should follow model size, data governance, bandwidth, energy, and update cadence; small-model incremental training or federated learning can run at the edge, so training is not categorically cloud-only. Model distribution needs signed artifacts, version management, rollback, and a device-management channel. If an AI Agent must invoke an edge data-processing service, an MCP Server can be deployed above the gateway as one governed interface. MCP itself neither distributes models or Tools nor guarantees that an invocation is secure.

In practice, a three-tier decision matrix usually guides task allocation: first judge from the latency requirement whether the task can run at the edge; then assess whether the data volume justifies occupying edge storage; finally check whether the edge computing power matches. If any tier fails, the task flows to the cloud. This decision process needs quantification: if latency tolerance exceeds a threshold (for example 50 milliseconds) and the data volume fits within the edge node's storage capacity, edge processing takes priority.

Example: A Cloud-Edge Collaboration Scheme for a New District

Take an illustrative scenario: in a new district's smart-traffic system, several intersection RSUs and multiple aggregation edge nodes are deployed.

• RSU level: directly handles signal-phase switching, local OBU verification, and forwarding and filtering of V2V safety-warning messages. The RSU keeps only the last few seconds of raw sensor data and periodically sends statistics (such as traffic flow and average speed) to the aggregation edge.
• Aggregation edge nodes: run a traffic-flow prediction model trained in the cloud and pushed down. They receive the periodic traffic-flow statistics from surrounding RSUs, predict road-network congestion over the coming interval in real time, and write the results into a lightweight in-memory database for RSU queries. The aggregation nodes compress the prediction results and raw statistics, and upload them to the cloud in minute-level batches.
• Cloud: runs the global travel-demand prediction model and a reinforcement-learning-based algorithm for coordinated multi-intersection signal scheduling. The cloud retrains the models on domain-wide historical data, then updates them and pushes them down to the aggregation nodes.

This design introduces new engineering considerations: insufficient edge computing power can cause task queues to back up, requiring monitoring and elastic scaling mechanisms to adapt; out-of-sync model updates call for version control and rollback strategies; during network outages, edge nodes must switch to a "degraded operation" mode to keep essential local functions running.

Figure 11-8 Layered Edge-Cloud Collaboration

Comparing Latency and Bandwidth Pressure

When different task types are handled at different tiers, end-to-end latency, network bandwidth consumption, and computing cost differ significantly. The table below compares them; the figures are illustrative values based on typical engineering ranges:

Processing tier	End-to-end latency (estimated)	Backhaul bandwidth saved	Typical tasks	Computing cost
Cloud only	High (hundreds of milliseconds to seconds)	– (baseline)	Global AI training, report analysis	High
Aggregation edge	Medium (tens of milliseconds)	Medium	Traffic-flow prediction, protocol conversion	Medium
Roadside edge	Low (<10 milliseconds)	High	Signal control, collision warning	Low (embedded)

Table 11-9 Latency, bandwidth, and cost comparison across tiers (illustrative data, based on typical engineering ranges)

Overall, the core of cloud-edge collaboration design is: fast local decisions, slow cloud optimization. Edge nodes handle "this moment" and "this place"; the cloud handles "trends" and "the big picture." This layered design is the core engineering means of solving the city-scale IoT challenges of "million-device access, real-time data processing, and cross-system coordination." Section 11.4 discusses further how AI models can be optimized collaboratively between the edge and the cloud.

← 11.2 Urban Governance Scenarios11.4 AI Traffic Prediction and Optimization →