Push a firmware update to ten thousand deployed devices, and even a 99.9% success rate leaves ten of them bricked in the field — unreachable, non-functional, and in most IoT deployments, unrecoverable without a truck roll. The technology that prevents this isn't exotic. It's a well-established architectural pattern — A/B partitioning, code signing, and staged rollout — that has to be designed into a device before the first prototype is built, not bolted on once devices are already in customers' hands.
This post breaks down how that architecture actually works, why it has to be a day-one decision rather than a later addition, and what a production-grade OTA pipeline looks like end to end.
Why "Just Overwrite the Firmware" Doesn't Work
The naive approach to a firmware update — download a new image and write it directly over the existing firmware — has an obvious failure mode: if power is lost, the network drops, or the download is corrupted midway through that write, the device is left with neither a complete new image nor an intact old one. Common causes of bricked devices during updates include unexpected power interruptions during critical write operations, corrupted downloads from network instability, failed cryptographic verification of signed images, and inadequate recovery logic such as missing rollback support.
For a device sitting on an engineer's desk, that's an inconvenience. For a device deployed in a customer's home, a remote industrial site, or a vehicle, it's a field failure with no easy fix.
The A/B Partition: A Fail-Safe by Construction
The core idea behind A/B partitioning is straightforward: rather than one firmware partition that gets overwritten in place, the device's flash storage is divided into two independent slots. The currently running firmware occupies one slot while an update downloads and installs into the other, inactive slot — meaning the device always has a complete, known-good firmware image available no matter what happens during the update process.
Once the new image is fully written to the inactive slot, the bootloader validates it — typically checking a cryptographic signature and running basic integrity checks — before switching over. If validation fails, or if the new firmware fails to boot successfully after switching, the bootloader falls back to the previous slot, which was never touched during the process. This is what makes A/B updates fail-safe by construction rather than by hope: a failed update simply means the device stays on its old firmware, rather than being left in a half-written, non-functional state.
It's worth noting that A/B partitioning isn't strictly mandatory for every product — but production systems need some equivalent fallback path, whether that's a true dual-slot layout, a dedicated recovery image, or external storage for a backup — the underlying requirement is that a failed update can never leave the device with no valid firmware to boot into at all.
The Bootloader Is the Real Foundation
It's tempting to think of OTA capability as a software or cloud problem, layered on top of whatever hardware a team has already chosen. That's backwards. If a device ships with a bootloader that doesn't support A/B partitioning, doesn't verify image signatures, and can't recover from a failed update, none of those capabilities can be added remotely later — the bootloader is the thing responsible for fixing a broken update, so if it's missing key features from the start, there's no OTA path capable of repairing that gap after the fact.
This is exactly why OTA architecture has to be planned before the flash memory map, bootloader, and release process are locked in — production-ready OTA firmware update systems generally require signed firmware images, bootloader-side validation, an A/B or fallback partition strategy, rollback triggers, anti-downgrade protection, release versioning, staged rollout, and telemetry from devices already in the field — a list that touches hardware, firmware, and cloud infrastructure simultaneously, not something any one of those layers can deliver alone.
Code Signing: Trusting the Update, Not Just Delivering It
A/B partitioning protects against corrupted or interrupted updates. Code signing protects against a different threat entirely: a malicious or unauthorized firmware image being installed, whether pushed by an attacker or downgraded to a known-vulnerable older version. Every firmware image is cryptographically signed before release, and the bootloader verifies that signature before accepting the image — so a device rejects any firmware it can't confirm came from a trusted source, regardless of how it arrived.
Rollback protection closes a related gap: without it, an attacker who can't inject malicious firmware directly might instead convince a device to reinstall an older, previously-patched-but-now-vulnerable firmware version, achieving effectively the same compromise through a different path. Enforcing rollback protection at the bootloader level ensures a device can never install a firmware version older than the one it's currently running.
Key management deserves specific attention here, since it's the part of code signing most commonly underestimated. A production system needs answers, planned before launch, to what happens if a signing key is compromised: is there a rotation mechanism, a way to revoke the compromised key, and a path to push new trust anchors to devices already in the field? A signing key that can't be rotated has the potential to freeze an entire deployed fleet — a risk that has to be architected around from the start, not discovered during an incident.
Staged Rollout: Containing the Blast Radius
Even a well-signed, A/B-safe update can still contain a genuine bug — a firmware image that passes every integrity check but behaves badly once running. A staged rollout is the operational safeguard against exactly this: rather than pushing an update to an entire fleet simultaneously, it's released incrementally to defined device cohorts, typically starting with a small percentage and expanding only after monitoring confirms the update is behaving as expected.
This matters because even with perfect security and solid A/B partitioning, a functionally broken update can still reach every device if it's pushed all at once — staged rollout is the practice that keeps a bad release contained to a small, recoverable subset of the fleet rather than the whole deployment.
Making staged rollout genuinely useful requires pairing it with real observability — devices need to report OTA outcomes (success, rollback, or specific failure reason) back to the fleet management system, since without that visibility, a team has no way to know an update is failing until customer complaints start arriving, by which point the rollout has often already progressed too far.
Full Images vs. Delta Updates
One practical architecture decision worth planning for, though not necessarily solving on day one, is whether updates ship as full firmware images or as deltas — binary diffs containing only the changed regions between versions. Full images are simpler and more robust, at the cost of larger downloads and more flash write cycles; delta updates shrink bandwidth requirements substantially but require the device to prove its current version before a patch can be safely applied, add CPU and power overhead during patch application, and need a fallback path to a full image if the device's actual state doesn't match what the delta assumes. A reasonable rule of thumb: start with full images for a first product cycle, and move to delta updates once fleet size or payload size makes the added complexity clearly worth it.
The End-to-End Chain
Put together, a production OTA system for a connected device typically forms a chain that looks something like this:
- Signed firmware build (cloud/CI)
- ↓
- Staged rollout targeting (by cohort, hardware variant, region)
- ↓
- Chunked transfer to device (with resume support)
- ↓
- A/B partition write (inactive slot)
- ↓
- Integrity check + signature verification
- ↓
- Bootloader switch + runtime health check
- ↓
- Mark valid — or automatic rollback to previous slot
- ↓
- Telemetry reporting outcome back to fleet management
Every link in that chain exists to catch a specific, real failure mode — and skipping any one of them doesn't just weaken the system marginally, it reintroduces exactly the bricking risk the rest of the architecture was built to eliminate.
Why This Is No Longer Optional
This architecture has moved from best practice to regulatory expectation. The EU Cyber Resilience Act, the UK PSTI Act, evolving NIST guidance, and the U.S. Cyber Trust Mark are converging on the same baseline expectation: connected products must remain maintainable and stay current throughout their supported life, with secure OTA explicitly required across the product's security-update lifetime under the CRA specifically. Retrofitting this architecture into a product that shipped without it is enormously more expensive than building it in from the start — in some analyses, three to five times more costly — precisely because so much of it depends on decisions made in silicon and bootloader design, not just cloud infrastructure that can be added after the fact.
What This Means for Hardware Teams
For teams designing the next connected product, a few decisions are worth locking in early:
Choose a bootloader and MCU/SoC platform with A/B and signature verification support from the start. This is the one part of the stack that genuinely can't be fixed remotely later — treat it as a hard requirement in component selection, not a nice-to-have.
Plan key management before the first signed image ships, including a rotation and revocation strategy — not after a key incident forces the question.
Build staged rollout and device-side telemetry into the fleet management plan from day one, even for a modest initial fleet size — the cost of adding observability later is far higher than designing for it upfront.
Decide deliberately between full-image and delta updates based on realistic fleet size and payload size projections, rather than defaulting to whichever is easier to implement first.
Treat the OTA pipeline as part of the product, spanning device, bootloader, and cloud backend together — a secure device-side architecture only delivers its full value when paired with an equally robust signing and rollout pipeline on the backend.
Conclusion
A/B partitioning, code signing, and staged rollout aren't three separate features — they're one integrated architecture for turning "bad firmware update" from a fleet-wide incident into a contained, recoverable event. Getting this right depends far more on early hardware and bootloader decisions than on cloud tooling choices made later, which is exactly why OTA architecture belongs in the initial system design conversation, not the post-launch roadmap.
At CoBuild Labs, we treat OTA and firmware update architecture as a core system design requirement from the earliest hardware decisions — choosing bootloaders, flash layouts, and signing via firmware engineering — related: edge model OTA.
Designing field-updatable products? Talk to CoBuild Labs about firmware engineering, automotive OTA, and model OTA at the edge.