Most OTA pipelines were designed to solve one problem: get a new firmware binary onto a device, safely, at scale. That architecture — A/B partitions, signed images, staged rollout — remains essential, but a growing share of IoT devices now carry a second artifact that needs the same lifecycle discipline: the machine learning model running on their onboard NPU. A model isn't firmware, but it isn't quite application software either, and treating it like either one exactly is where a lot of edge AI deployments run into trouble.
This post looks at why AI models need their own update discipline distinct from firmware, what makes model updates a genuinely harder engineering problem, and how a production OTA pipeline needs to evolve to handle both.
Why This Is Suddenly Everyone's Problem
The majority of the roughly 21 billion IoT devices deployed today still rely on external processing or simple rule-based logic rather than on-device intelligence — but that's changing quickly. New IoT SoCs are increasingly shipping with lightweight NPUs, vector extensions, and DSP-like AI cores specifically to support on-device tasks like anomaly detection, small-model vision, and local audio intelligence, shifting immediate reactions to the edge while the cloud takes on a more strategic role for analytics and long-term storage.
That shift changes what "maintaining a device" means. Each AI model deployed to the edge becomes another component that has to be versioned, validated, and maintained over time — and for products built on a subscription or continuous-improvement model, this isn't optional: AI-enhanced features require reliable delivery of continuous updates, since customers paying recurring fees expect the product to keep improving, and vulnerabilities in a deployed model need to be patchable just like a firmware CVE. Without a robust update mechanism for the model itself, the entire subscription business model built around AI-enhanced features simply doesn't function.
Why a Model Update Isn't Just a Bigger Firmware Update
It's tempting to treat a model file as just another binary that fits into an existing A/B partition and signing pipeline. In practice, model updates introduce failure modes firmware updates don't have.
Model-hardware version dependencies add a whole extra compatibility axis. At the edge, teams are managing a genuinely fragmented mix of GPUs, NPUs, CPUs, and other accelerators — and a model update might require a specific firmware version to actually access new hardware-acceleration primitives on the NPU. If the OTA orchestrator isn't version-aware across every layer of that stack, a model update can silently trigger a fallback to CPU-only inference, introducing latency that breaks the real-time requirements of whatever perception or detection pipeline depends on it — a failure mode that isn't a crash or a bricked device, just a silently degraded one.
Model files are often much larger than firmware images, and edge devices frequently operate over exactly the constrained connectivity — cellular, satellite, low-power WAN — that struggles most with large payloads. Transmitting a multi-gigabyte deep learning model over a 4G link isn't just expensive in raw bandwidth cost; it also means the device spends more time in a vulnerable, non-operational state during the download, and consumes meaningfully more radio power than a typical firmware delta would.
Model quality degrades silently, which firmware doesn't. A buggy firmware update usually fails in an observable way — a crash, a rollback, a clear error. A model, by contrast, can be technically "working" while slowly becoming less accurate as real-world data drifts away from what it was trained on — a model trained on summer imagery degrading through winter, or a factory-inspection model missing defects on a new product batch it was never trained on. The model doesn't break in any way an existing firmware-focused health check would catch; it just quietly gets worse, often unnoticed until the cost of that degradation has already been paid.
Fleet model-version fragmentation is a harder problem to track than firmware fragmentation. With multiple devices in the field, each one might end up running a different model version depending on rollout timing, connectivity gaps, or device-specific compatibility — and unlike a firmware version number, a model version doesn't tell you anything about accuracy or drift on its own, meaning fleet-wide model management needs its own observability layer beyond simple version reporting.
What a Model-Aware OTA Pipeline Actually Looks Like
The emerging pattern in edge AI tooling treats models as versioned, independently deployable artifacts — conceptually similar to how modern containerized application updates work, rather than folding them into a single monolithic firmware image.
Models get packaged and versioned as distinct deployable units. One practical implementation pattern organizes deployment around three concepts: packages (the specific artifact being delivered — a model, a labels file, or similar component), cohorts (which devices receive a given package), and deployments (the actual rollout event). Each version of a package is treated as immutable once uploaded, and importantly, a single model package can be deployed independently to multiple device cohorts, rather than requiring a full firmware rebuild every time a model needs updating on a subset of the fleet.
Modular architecture lets model and firmware update independently. Because models and firmware evolve on genuinely different cadences — a firmware fix might ship rarely, while a model might be retrained and redeployed frequently as new field data comes in — a modular update architecture allows components like inference engines, drivers, or the model itself to be updated separately, rather than bundling every change into a single all-or-nothing firmware release.
Update triggers can be tied to model-specific signals, not just version numbers. Rather than only checking "is there a newer build," an update check can incorporate the last modification date of the model project, measured model performance, or a specific release/version tag — giving teams a path to trigger updates based on detected accuracy drift, not purely on elapsed time since the last release.
Hot-swapping models without full device restart is increasingly the target. More mature implementations aim for updates that take effect within seconds while inference is still actively running, rather than requiring a full reboot cycle — a meaningful reliability improvement for devices that can't tolerate extended downtime, such as a security camera or an industrial inspection system that needs to keep running during the swap.
- The Security Model Doesn't Change — It Extends
The same non-negotiable security practices from firmware OTA architecture — covered in more depth in our post on A/B partitions and rollback — apply directly to model artifacts, with a few model-specific wrinkles worth flagging. Code signing still matters: only models signed by trusted keys should be accepted, exactly as with firmware images, since a compromised model update pipeline is just as capable of introducing malicious behavior as a compromised firmware pipeline. Encrypted transport (TLS/SSL) protects the model in transit the same way it protects firmware. And rollback capability matters arguably more for models than firmware, since a model that silently degrades accuracy in the field is a genuinely harder failure to detect through standard health checks than a firmware crash — meaning the rollback trigger logic may need to incorporate model-performance signals, not just crash detection.
What This Means for Hardware and Firmware Teams
For teams building the next generation of edge AI-capable IoT hardware, a few architectural decisions are worth making deliberately from the start:
Design the OTA pipeline to treat models as a distinct artifact class from day one, with their own packaging, versioning, and cohort-targeting logic — retrofitting this separation after a monolithic firmware-plus-model image architecture is already deployed is significantly harder than building it in from the start.
Build version-awareness across the full hardware/firmware/model stack, not just at the model layer — a model update needs to check compatibility against the current firmware and NPU driver version, not assume it will always find compatible hardware acceleration available.
Plan for constrained-bandwidth model delivery explicitly if the target deployment includes cellular, satellite, or LPWAN connectivity — delta updates, model compression, or staged/chunked delivery matter even more for multi-megabyte model files than they do for typical firmware payloads.
Build model-performance monitoring into the fleet telemetry pipeline, not just crash and rollback reporting — since model drift is a silent failure mode, detecting it requires actively tracking accuracy or confidence signals from the field, not just waiting for an obvious malfunction.
Treat model updates with the same security rigor as firmware — signed artifacts, encrypted delivery, and rollback capability aren't optional extras for AI components; they're the same baseline expectation regulators and standards bodies are already applying to firmware OTA broadly.
Conclusion
Edge AI turns every deployed device into something with two things that need to be maintained over its lifetime, not one — firmware and the model running on top of it — and those two artifacts age, fail, and need updating in genuinely different ways. The OTA architecture that already exists for firmware is a solid foundation, but building a production-grade edge AI product means extending that architecture with model-specific versioning, drift monitoring, and hardware-compatibility awareness, rather than assuming a model can simply ride along inside an existing firmware update pipeline unchanged.
At CoBuild Labs, we treat model lifecycle management as a first-class part of edge AI hardware architecture — designing OTA pipelines that treat firmware and models as related but distinct — see A/B rollback and AI integration.
Shipping on-device models that need field updates? Talk to CoBuild Labs — see AI integration, A/B partition OTA, and LLMs and IoT.