Edge AI Is Not Magic — It Is a Power Budget Problem

Author: Srihari Maddula  •  Founder & Technical Lead, Eurth Techtronics Pvt Ltd

Category: Power vs Performance  •  Estimated Reading Time: 18–20 minutes

Published: July 2026

The Demo Ran. The Battery Did Not Survive the Week.

Running YOLOv8n on an ESP32-S3 is genuinely impressive. The demo video looks great. Object detection at the edge, no cloud dependency, 5-second inference on a microcontroller — the kind of result that gets excited reactions at conferences and in investor meetings. Then someone asks the power budget question.

The ESP32-S3 in inference mode with the camera active draws approximately 250 to 350 milliamperes at 3.3V — roughly 900 milliwatts to 1.15 watts. A standard 18650 lithium cell at 3,000 mAh delivers approximately 11 Wh usable energy. At 1 watt of continuous load, that is 11 hours of operation. For a device intended to run for months on a single charge, continuous inference on an ESP32-S3 is not a viable architecture. It is a demo architecture.

Edge AI in real products is not a compute problem — or rather, it is a compute problem that is inseparable from a thermal problem, a power rail problem, and a duty-cycle problem. This blog frames edge inference as a power budget exercise first, and a compute exercise second. Because that is the order of priority in any battery-powered or power-constrained deployment.

The Three Axes of Edge AI Power Design

Axis 1: Joules Per Inference, Not Just Milliseconds Per Inference

Inference latency — how long a single forward pass takes — is the number that benchmarks report. It determines user experience in interactive applications. But for IoT edge AI, the number that determines battery life is joules per inference: how much energy is consumed to produce one inference result.

Joules per inference = average power during inference × inference duration. A processor that runs inference in 50 ms at 500 mW consumes 25 mJ per inference. A processor that runs inference in 200 ms at 80 mW consumes 16 mJ per inference. The second option is four times slower but uses 36% less energy per inference. For a device inferring once per minute, the second option extends battery life by roughly 60% compared to the first.

This is why hardware neural accelerators — dedicated NPUs in chips like the STM32N6, MAX78000, Kendryte K210, or Ambiq Apollo — can be more energy-efficient than running inference on a general-purpose MCU or even a Cortex-A application processor. The NPU runs the same computation in less time at lower power, because it is optimised for the specific pattern of multiply-accumulate operations that neural network inference requires.

Axis 2: Duty Cycle — How Often Do You Actually Need to Infer?

The most powerful tool in edge AI power budgeting is duty cycle reduction. If your application requires a decision once per minute, you do not need to run inference continuously. You need to run inference for the duration of one inference cycle, then sleep everything until the next required decision.

A device that draws 300 mA during a 200 ms inference window, then sleeps at 10 μA for the remaining 59.8 seconds, has an average current of approximately (300 mA × 0.2s + 0.01 mA × 59.8s) / 60s ≈ 1.01 mA. On a 3,000 mAh battery, that is approximately 125 days of operation — not 11 hours. The inference hardware is identical. The duty cycle architecture changes the battery life by a factor of 272.

Duty cycle architecture requires that the inference trigger — the event that wakes the system and causes an inference to run — itself be low-power. A wake-on-motion accelerometer, a passive infrared detector, a low-power microphone with keyword detection, or a simple timer can all be implemented at microampere-level current draw, keeping the system asleep until an inference is genuinely needed.

Axis 3: Model Efficiency — Architecture Before Quantisation

Before quantising your model or pruning weights, choose the right model architecture for the task. YOLOv8n is the nano variant of YOLO — already the smallest in the family. But for many edge applications, it is still too large. A custom-trained MobileNetV3-Small classifier may achieve comparable accuracy for a specific detection task at one-fifth the parameter count and one-eighth the compute.

Quantisation — reducing weights from 32-bit float to 8-bit integer (INT8) or 4-bit integer (INT4) — reduces model size by 4x to 8x and reduces inference compute by a similar factor on hardware with INT8 acceleration. TensorFlow Lite with post-training quantisation or quantisation-aware training produces INT8 models that typically lose less than 2% accuracy compared to the float32 baseline, while halving or quartering the inference time on supported hardware.

The correct model selection process starts with the tightest acceptable accuracy, identifies the smallest model architecture that meets it, then applies quantisation as a further optimisation — not the other way around.

On-Device vs Cloud Inference: The Decision Framework

Not every inference should run on the device. The decision between on-device and cloud inference should be made on four criteria: latency requirement, connectivity reliability, data privacy, and power budget.

On-device inference wins when latency must be below 500 ms, when connectivity is unreliable or absent, when the raw data cannot leave the device for privacy or bandwidth reasons, and when the inference model is small enough that the compute cost is within the power budget.

Cloud inference wins when the model is too large for the device, when inference needs to be updated frequently without OTA firmware updates, when the device is mains-powered with reliable connectivity, and when the accuracy requirement justifies a larger model than can run at the edge.

The hybrid approach — run a small trigger model on-device that decides whether to wake up and send data to the cloud for a larger model — is often the right architecture for constrained devices with intermittent connectivity requirements. The device spends most of its time in low-power mode running a lightweight classifier, and only activates the cloud pipeline when the classifier signals a high-confidence event.

Thermal Management: The Constraint Nobody Models

Sustained compute load generates heat. Heat in an enclosed enclosure raises the temperature of the MCU, battery, and sensors. Above approximately 85 degrees Celsius junction temperature, most MCUs throttle their clock frequency to protect themselves — and inference latency increases precisely when compute demand is highest. Above this temperature for sustained periods, silicon reliability degrades, solder joint fatigue accelerates, and electrolytic capacitor lifetime reduces.

For edge AI devices in outdoor enclosures in Indian field conditions — ambient 45 degrees, solar heating adding 10–15 degrees internal — the thermal budget is tight. A device running continuous inference may be operating at its thermal limit within 30 minutes of operation on a hot afternoon. The duty cycle architecture that solves the power problem also solves the thermal problem: sleeping between inferences allows the thermal mass to dissipate the heat generated during inference before the next cycle.

THE RULE: Frame edge AI as a power budget problem first. Calculate joules per inference.

Design duty cycle before selecting hardware.

Choose model architecture before choosing quantisation.

The compute capability is only useful if the power budget permits it to run in the real world.

© 2026 Eurth Techtronics Pvt Ltd  |  eurthtech.com  |  All rights reserved.