Feature Request: On-Device AI Study Companion for AnkiDroid
1. Problem
Students using AnkiDroid regularly hit a wall that flashcards alone can’t solve:
- A card’s back field answers “what,” but not “why” — when a student misses a card or doesn’t fully understand the answer, there’s no way to ask a follow-up question without leaving the app.
- Before a study session, students often want a quick overview of what a deck actually covers, or a summary of what’s due today — instead they have to open and read cards one by one.
- Existing AI-flashcard tools (ChatGPT-based AnkiWeb add-ons, LLM-powered CLI tools) only work on desktop, and only by sending flashcard content to a cloud API. For students studying sensitive material — medical case notes, personal language journals, proprietary coursework — that’s a real privacy concern, and it doesn’t help at all on the phone, which is where most actual review happens.
- Generic on-device AI chat apps exist and can run small models fully offline, but they have no concept of “deck,” “note,” or “due card” — they’re not integrated with what the student is actually studying.
There is currently no offline, privacy-preserving, AnkiDroid-native way to ask questions about your own cards or get a summary of your own deck.
2. Proposed Solution
Add an opt-in, fully offline “Study Companion” to AnkiDroid:
- A small, fine-tuned Gemma 4 E2B-it model runs entirely on-device — no network calls at inference time, no flashcard content ever leaves the phone.
- The model answers questions about the current card, and can summarize a deck or a set of due cards, by pulling the actual note/card content from AnkiDroid’s own collection through a small internal tool-calling layer (an in-process MCP-style interface), not by guessing from its own training data.
- Every answer visibly shows which note/field it drew from, so answers stay grounded and a student can verify at a glance that it isn’t hallucinating.
- The feature adds nothing to app size, memory, or battery use for anyone who doesn’t enable it — the model is downloaded on demand, not bundled.
This is scoped narrowly on purpose: it does not touch card generation, scheduling/FSRS, or become a general-purpose chatbot. It answers questions about the student’s own material and summarizes it — nothing else, at least for a first version.
3. Goals / Non-Goals
Goals
- Fully offline inference; zero flashcard content leaves the device.
- Grounded, citable answers sourced from actual note/card content via explicit tool calls.
- Strictly opt-in with no cost to users who don’t enable it.
- Fits AnkiDroid’s existing coroutine-based data-access pattern rather than introducing a parallel path to the collection.
Non-goals (v1)
- Not a card-generation tool.
- Does not touch spaced-repetition scheduling.
- Not meant to let a student skip recall during a graded review — framed as “help me understand what’s already in my deck,” not “give me the answer.”
- Not a general-purpose chatbot.
4. Architecture Overview
┌─────────────────────────────────────────────────────────┐
│ AnkiDroid app process │
│ │
│ Reviewer / DeckPicker UI │
│ │ (opens, opt-in) │
│ ▼ │
│ StudyCompanionViewModel ──────────────► Local inference │
│ │ ▲ engine │
│ │ │ tool calls (llama.cpp JNI │
│ │ │ (function-calling) or LiteRT-LM) │
│ ▼ │ running the │
│ In-process MCP server (kotlin-sdk) fine-tuned │
│ tools: list_due_cards, get_note_fields, Gemma 4 E2B-it │
│ search_notes, get_deck_stats GGUF/.task file │
│ │ │
│ ▼ │
│ CollectionManager.withCol { ... } → libanki → rsdroid │
│ (existing, unmodified data-access path) │
└─────────────────────────────────────────────────────────┘
Nothing in this design changes scheduling, libanki, or the Rust backend. It is a new, self-contained module that reads through the existing collection-access pattern and adds a new on-device inference component alongside it.
5. Technical Design
5.1 Model choice
Gemma 4 E2B-it — Google’s current-generation open-weight model family (Apache 2.0 licensed), in its smallest “effective ~2.3B parameter” variant, purpose-built for on-device/mobile deployment. Quantized (4-bit, GGUF Q4_K_M) it runs in roughly 2–3 GB RAM with a ~1.3 GB on-disk footprint — plausible on most Android phones from the last several years, though not the lowest-end devices still common in AnkiDroid’s user base. It natively supports structured function/tool calling, which the tool layer below relies on directly instead of prompting the model into an ad hoc format.
E4B (the next size up, ~4.5B effective) can be offered as an optional higher-quality tier for capable devices, gated by a runtime RAM/storage check — never the default.
5.2 Fine-tuning pipeline
- Use Unsloth for LoRA/QLoRA fine-tuning. Their own guidance for this model family recommends fine-tuning E4B with QLoRA rather than E2B with plain LoRA (the 4-bit accuracy hit on the larger model is minimal, so you get a better base for about the same eventual deployed size) — worth validating this against E2B-only fine-tuning empirically before committing to it.
- QLoRA fine-tuning of E4B fits on a single consumer GPU with about 16 GB VRAM, or a free-tier Colab GPU for lighter runs.
- The actual novel work is dataset design: build (question, grounded-context, answer) triples from real note fields, where the answer only ever uses information present in the given fields — including explicit examples where the correct answer is “this deck doesn’t cover that” — so the model learns to refuse rather than hallucinate.
- After training, merge adapters and export to GGUF for on-device deployment.
5.3 On-device runtime
Two viable paths exist; pick one deliberately rather than building both:
| llama.cpp / GGUF | LiteRT-LM (Google’s stack) | |
|---|---|---|
| License | MIT — clean fit with a GPL-3 app | Apache-2.0, heavier Google-specific dependency graph |
| Maturity for this use case | Very mature; what a Unsloth fine-tune exports to directly | Actively developed; what Google’s own official quantized checkpoints target; the older MediaPipe LLM Inference API is now maintenance-only in favor of this |
| Hardware acceleration | CPU/GPU, less turnkey NPU support | First-class CPU/GPU/NPU delegation |
| Distribution friendliness | Small, fully open-source dependency | Also license-compatible, but a larger toolchain to pull in |
Recommendation: start with GGUF/llama.cpp for the first working version — least friction from a fine-tuned export, smallest dependency footprint. Treat LiteRT-LM as a deliberate v2 migration if hardware-accelerated inference becomes the bottleneck on mid-range devices, not something to build in parallel from day one.
Default to 4-bit quantization (Q4_K_M). Google’s officially released 2-bit quantized checkpoints for this model are worth benchmarking for the lowest-RAM device tier specifically, but shouldn’t be the default given the higher risk of visible quality loss on a reasoning/QA task.
5.4 On-device data access (MCP-style tool layer)
Rather than string-stuffing card text into a prompt ad hoc, expose a small, explicit, read-only set of tools that the model calls before answering:
list_due_cards(limit)— deck name, card id, question/answer text for due cardsget_note_fields(note_id)— all fields of a notesearch_notes(query)— thin wrapper over AnkiDroid’s existing search syntaxget_deck_stats(deck_id)— card counts, due counts, maturity breakdown
Each tool internally just calls into the existing CollectionManager.withCol { } suspend-function pattern already used everywhere else in the app — no new access path to the collection is introduced. The model, given a question, decides which tool(s) to call, receives structured data back, and only then generates an answer.
This can be implemented with the official Kotlin Multiplatform MCP SDK running fully in-process (no network socket, no separate process) — or, if a reviewer prefers less new surface area for a v1, as a plain internal Kotlin interface with the same shape, with MCP framing kept as a future upgrade path rather than a hard requirement. Note: this is distinct from an existing “Android MCP SDK” project that wires the same protocol into Android apps for a different purpose (letting external developer tools inspect a running app over a debug connection) — that project is debug-build-only by design and isn’t applicable here, since this feature needs to run for real users in release builds.
5.5 Module structure and storage
- New, self-contained Gradle module (e.g.
:ai-companion) so users who never enable the feature see no change to app size or build. - Model weights are never bundled in the APK. On opt-in, the quantized model file is downloaded on demand into app-specific storage, with a visible progress indicator, resumable download, and a Wi-Fi-only default.
- Settings: enable/disable the feature; choose model tier (E2B/E4B) based on a plain-language device capability check; delete downloaded model to reclaim storage.
6. Implementation Plan
Each phase is scoped to be its own independently mergeable, revertable pull request:
- Standalone spike — prototype the fine-tuning → GGUF export → on-device inference loop outside the app entirely, using exported deck data, to validate quality and latency before any integration code.
- Module scaffold — empty new module plus a settings toggle (default off), no model logic yet.
- Tool layer over existing data — implement and unit-test
list_due_cards/get_note_fields/search_notes/get_deck_statsagainst the existing collection-access pattern, with no LLM involved yet. - Base model wiring — llama.cpp integration running the unmodified quantized Gemma 4 E2B-it model, downloadable on demand, behind the settings toggle.
- Fine-tuned model swap-in — replace the base model with the fine-tuned checkpoint once evaluation clears a quality bar defined up front.
- UI integration — Reviewer/DeckPicker entry points, chat interface, grounding-citation display, night-mode support.
- Device-tier gating and distribution resolution — RAM/storage checks, E2B/E4B choice, resolve model-download/distribution questions with maintainers before shipping broadly.
- Docs and test coverage — usage documentation, instrumented tests, following the project’s existing testing conventions.
7. Risks and Open Questions
- Device fragmentation is the largest real risk — a meaningful share of AnkiDroid’s actual users are on older or storage-constrained devices; this must stay cleanly opt-in and capability-gated, never default-on.
- Answer quality at this model size for grounded QA is not guaranteed and needs real evaluation against held-out (card, question, answer) examples before claiming it works well.
- Battery and thermal impact of on-device inference needs measurement on real mid/low-tier hardware, not just a flagship device.
- Storage cost (roughly 1–3 GB depending on tier) is a real cost on devices where large media-heavy decks already consume significant space.
- Whether maintainers want this scope of change at all is a bigger open question than any technical detail here — the realistic path is proposing a small proof-of-concept first (e.g., the tool layer alone, with a mocked model) and getting feedback before building the rest.
8. Privacy and Academic-Integrity Notes
- No network calls at inference time — this should be enforced in code, not just claimed.
- The model should be trained and prompted to answer only from tool-returned card content, or say it doesn’t know, rather than generate freely — critical for a study tool where a confidently wrong answer is worse than no answer.
- Framed explicitly as a comprehension aid (“help me understand what’s in my deck”), not an answer key — this should be reflected in the UI copy and default behavior, not just this document.