On-Device Stable Diffusion Profiling and Export
Pipeline profiling, few-step generation with LCM and SDXL Turbo, ExecuTorch export with INT8 calibration, and Qualcomm QNN
What you'll learn
- ✓Profile a Stable Diffusion pipeline and identify the mobile bottlenecks
- ✓Explain why the U-Net repeats and how few-step models change the arithmetic
- ✓Use LCM-LoRA and SDXL Turbo and measure the quality trade-off
- ✓Export text encoder, VAE and U-Net with ExecuTorch and quantize with calibration
- ✓Understand the Qualcomm QNN workflow for the Hexagon NPU and its limits
About this course
Diffusion is compute-bound: a billion-parameter U-Net run dozens of times per image. Putting it on a phone is the hardest workload in on-device AI, and this course is the model half of that job.
We start with the hardware and the compression theory that governs every later decision. Then we profile a Stable Diffusion pipeline stage by stage — text encoder, U-Net, VAE — find where the time actually goes, and derive from the diffusion equations why the U-Net must repeat at all. Next, few-step generation: LCM-LoRA and SDXL Turbo, what adversarial diffusion distillation changes, and the quality trade-off measured rather than claimed. Then export: ExecuTorch for the text encoder, VAE decoder and U-Net, backend delegation, and INT8 post-training quantization with real calibration data. Finally the Qualcomm QNN path for the Hexagon NPU: the workflow, context binaries, calibration, the tuning checklist, and an honest account of its limits and alternatives.
Every session is a notebook shown on screen with the narration explaining each cell. The follow-up course, *Ship Stable Diffusion in an Android App*, puts these exported components into Kotlin and React Native apps.
**Licensing.** Exporting and quantizing weights creates derivative works, and the upstream licence follows them onto the device — so this course ships a `Resources/LICENSES.md` with every asset checked against its actual licence text. Stable Diffusion 1.5 is CreativeML Open RAIL-M (commercial use allowed, use restrictions must be passed through to your users); **SDXL Turbo is under the Stability AI Community License, free for commercial use only under US $1,000,000 annual revenue**; LCM-LoRA is OpenRAIL++, and the Tiny VAE alternative is MIT.
Curriculum7 sections · 18 lectures · 2h 24m
Section 0. Welcome
Free previewSection 1. On-Device AI and Mobile Hardware
Free preview- ▶section 1 intro0:12
- ▶ondevice ai and mobile soc9:00
- 🔒bandwidth roofline runtimes10:24
Section 2. Model Compression Theory
- 🔒quantization formula and ptq10:00
- 🔒qat ste pruning distillation9:36
- 🔒llm quantization and decision tree9:12
Section 3. Stable Diffusion Pipeline Analysis
- 🔒section 3 intro0:12
- 🔒sd anatomy and stage profiling9:12
- 🔒unet op profiling attention memory prompt cache8:54
- 🔒diffusion math step reduction mobile priorities8:24
Section 4. Fast Generation LCM and Turbo
- 🔒lcm add sdxs theory9:48
- 🔒lcm lora sdxl turbo benchmark selection guide9:06
Section 5. ExecuTorch Export
- 🔒executorch intro install text encoder export9:24
- 🔒vae unet export backend delegation8:48
- 🔒int8 ptq calibration real unet app loading9:48
Section 6. Qualcomm QNN
- 🔒hexagon npu qnn workflow unet context binary11:18
- 🔒shipping tuning checklist ai hub closing part39:24
Requirements
- · A computer with Python 3.10+ (a free Google Colab account is enough for most sessions)
- · Comfort reading Python code; you do not need to be an expert
- · Basic familiarity with the diffusers library
- · A GPU for the diffusion notebooks (Colab works)
- · Willingness to read `Resources/LICENSES.md`: SD 1.5 is OpenRAIL-M and SDXL Turbo is free commercially only under US $1M annual revenue
Who this is for
- · ML engineers deploying diffusion models to mobile hardware
- · Developers who need images generated on the device rather than in the cloud
- · Anyone who wants a measured view of mobile diffusion performance
Read alongside the course
The voice-over in this course is synthesized with a text-to-speech model from scripts written and reviewed by the instructor, and the on-screen material (notebooks, code, slides) is the instructor's own work.