Ship an LLM in an Android App

What you'll learn

• ✓Integrate an on-device LLM in a Kotlin/Compose app with MediaPipe and a foreground service
• ✓Integrate the same model in React Native with llama.rn, including model download and streaming
• ✓Build an on-device RAG pipeline with embeddings and a SQLite vector store
• ✓Use execution-provider fallback, profile with Android Studio Profiler and Perfetto, and verify NPU offload
• ✓Spot and fix the threading, lifecycle and memory mistakes that make on-device AI apps fail

About this course

A model that runs in a notebook is not an app. This course puts a quantized language model inside a real Android application — twice, in two stacks — and is explicit about everything that breaks on a real phone.

Track A is Kotlin: project and Gradle setup, a MediaPipe LLM engine wrapper, a ViewModel, a Jetpack Compose chat screen, a foreground service that keeps generating in the background, model files in assets, and an on-device RAG pipeline with embeddings and a SQLite vector store. Track B is React Native with llama.rn: project setup, Metro configuration for large model files, an LLM service module, a model downloader with progress, and a streaming chat hook and screen. Then hardware acceleration: the execution-provider fallback chain, calling Qualcomm QNN through JNI, profiling with Android Studio Profiler and Perfetto, and verifying from logcat that the NPU is really doing the work. An appendix covers migrating to Gemma 4 and LiteRT-LM with native tool calling.

The code is walked through by intent — threading, lifecycle, memory, cancellation — and the narration says out loud which parts of the sample code must be fixed before you ship, which makes them the real exercises of the course.

**Licensing.** Getting weights onto a user's phone is redistribution, so this course ships a `Resources/LICENSES.md` with every asset checked against its actual licence text. **Gemma 2, used in Track A and Track B, is commercially usable but obliges you to give every user a copy of the Gemma Terms of Use, reproduce its notice string verbatim, and carry its Prohibited Use Policy into your own terms** — all of which are things you build into the app, not paperwork you do afterwards. The Section 4 appendix moves to Gemma 4, which Google released under Apache-2.0 and which drops those obligations entirely.

Curriculum5 sections · 14 lectures · 1h 42m

Read alongside the course