The MobileNetV2-0.5 That Almost Fit

The best model came out of selection, compiled for the STM32H7 — and missed by 100 KB of flash and 70 ms of latency. This video builds the compression-journey calculator with Claude Code: FP32 → INT8 PTQ → structured prune → distillation, each step tracked against flash, SRAM, latency, and accuracy ceilings. The order of operations matters.

Built with Claude Code + Manim. Part of the Embedded AI with Claude series.

Nik Bear Brown · @NikBearBrown nikbearbrown.com

Transcript

A lightly formatted transcript of the video, with timestamps so you can jump to any section on YouTube.

[0:00] Nick Bear Brown, build it with a CLI, then take it apart. The model selection passed. Mobile Net V2 0.5 89.8% accuracy on the defect detection task compiled for the STM32H7. Then the numbers came back. 4 MGB of flash, 1.2 megabytes of RAM, 320 milliseconds per inference. The ceilings were 2 mgabytes, 1 MGAB, 250 milliseconds. It missed on every axis. Three knobs can close that gap. Quantization, pruning, distillation. The question is in what order and whether the combination ships. As for the experiment in the terminal clawed write compression journey.pi mobile net v20.5 on STM32H7 with 2 m by flash 1 m by RAM 250 mms. Steps FP32 to NT8 PTQ. Then add

[0:53] 25% structured prune. Then add distillation track flash latency and accuracy against ceilings. Print a four-state table. Read it before trusting it. Int8 cuts weights from four bytes to one. So flash drops from 4 mabby to 1 me. Pruning removes 25% of channels, shrinking flash further to 0.75 meby and latency to 240 mm. Distillation adds back 0.8% accuracy with no size change. The order matters. Quantise first, then prune the quantise network. Run it. Four states, four metric columns. FP32 blows all three ceilings. Flash in red, in red, latency in red. Int8 drops flash from 4 to 1 megabyte and to 0.8. Latency stays at 320. Quantization doesn't move the

[1:44] Compute bottleneck. Add pruning. Flash falls to 0.75. Latency finally drops to 240 under the ceiling. Add distillation accuracy climbs 1.6 points from 89.0 to 90.6 above where we started. That's the ship state. Now check what happens if you do it in the wrong order. Iterate clawed. Add a fifth column showing quantise before prune. The order matters accuracy penalty. Rerun two bars now. Quantise then prune versus prune then quantise. Quantise before prune accuracy in crimson 88.1% barely above the ceiling. Prune before quantise in ink 90.6% a 2.5 point spread from sequencing alone. Same operations same model different order. So pruning earned the fit. Distillation earned the margin.

[2:38] Quantization made it possible. And doing them out of order compounds error. The ship state is 0.75 m by flash 0.6 6 m byram 240 minutes 90.6% accuracy three knobs applied in the right sequence your move. Take your own models parameter count and run the four state table against your target ceilings. If pruning alone doesn't close the gap add distillation against the float teacher and check the order before you run. 2.5 accuracy points is not a rounding error. Nick Bear Brown. Build it with a CLI, then take it apart at Nick Bear Brown.