Wikipedia Companion: A Talking AI Tutor That Runs on Small Local Models

I built a talking Wikipedia Companion that runs entirely on small local models. It uses: Gradio, llama.cpp, Kokoro, faster-whisper, sentence-transformers, NeMo, and Wav2Vec2. Built for the Build Small Hackathon: the one where you're handed at most 32 billion parameters and asked to think small.

Why I built this

Growing up and working in Kampala, Uganda, I learned firsthand that having internet doesn't mean having equal internet. Data is expensive and connections can be slow. While text is cheap to load, rich educational video isn't, because it eats gigabytes of data and buffers endlessly. For students in emerging markets, reading dense Wikipedia articles on small mobile screens can be overwhelming, especially when the language of instruction is a second language.

How do we provide the warm and engaging learning experience of a video tutor without the massive bandwidth tax?

I built a Wikipedia Companion: you ask a question out loud or by typing, it fetches the answer as plain text, and a friendly animated face reads it back to you. The trick is that all the AI runs on-device. The only thing it pulls from the internet is the article's text (a few kilobytes). You get the warmth and engagement of a video tutor at a tiny fraction of the data cost.

Here is the Wikipedia Companion in action:

The Wikipedia Companion introducing itself and demonstrating how it works.

The two design decisions that matter most

1. Live fetching, not offline dumps
Why fetch text over the internet instead of bundling Wikipedia offline? A full local database dump is tens of gigabytes to download, instantly eating up a phone or laptop's storage, and is frozen at download time. By fetching just the specific article's text live, the app guarantees the answers use today's facts while still only costing a fraction of a cent in data.

2. The model never answers from memory
The tiny 3-billion-parameter LLM is never asked to act as an encyclopedia. Its only job is to read the passage it's handed and explain it warmly. This keeps answers grounded in the fetched text rather than the model's training data, which prevents hallucinations and proves that a tiny model is genuinely enough for the task.

How it works

The whole system is a single Gradio app. While the architecture diagram below maps out the 18 technical interactions, the pipeline boils down to 7 core stages where every AI model runs locally:

1. Listen: your voice is transcribed locally with Whisper.
2. Understand the question: the local LLM reads "how big is the moon" and figures out the subject is the Moon. It proposes candidate article titles, which are verified against Wikipedia before use.
3. Fetch: the chosen article's text is pulled live from the Wikipedia API (the only network call!).
4. Retrieve: a tiny embedding model semantically ranks the article's sections to find the passage that actually answers the question.
5. Explain: the local LLM turns that passage into a short, warm, spoken answer, grounded only in what the passage says.
6. Speak: the text is normalized for natural speech (numbers, units, abbreviations expanded via NVIDIA NeMo), then synthesized to audio with Kokoro.
7. Come alive: forced alignment (Wav2Vec2-CTC) produces millisecond word-level timings, which drive *both* the animated face's lip movements and the follow-along transcript.

Bridging the gap: The face and the follow-along text

To make this feel like a tutor rather than a search box, the UI does two things, both driven by local computation, with no extra bandwidth:

A zero-bandwidth video tutor: Instead of streaming a 1080p video of a teacher, the app features an animated SVG companion. Driven by the local forced alignment, the face lip-syncs to the spoken audio. It provides the sense of someone explaining things to you, at the cost of a few kilobytes of text rather than megabytes of video.

A follow-along reading experience: For learners and ESL students, just listening isn't enough. As the companion speaks, the transcript highlights word-by-word on the screen ("karaoke-style"). This trains reading tracking, actively bridging the gap between spoken and written language to improve comprehension and literacy.

What "small" taught me

Working under a 32B parameter ceiling changed how I designed. Instead of reaching for one big cloud model to do everything, I ended up with a handful of tiny specialists (a speech recognizer, a language model, an embedding model, a text normalizer, a speech synthesizer, and an alignment model), each doing one job well, none of them large. The biggest is ~3B; most are under 100M.

The constraint pushed me toward an architecture that's cheaper, private, current, and honest about what each piece is for. Sometimes the right answer isn't an enormous API. Sometimes it's a small model, thoughtful design, and building for the reality of the community you come from.

How to run this companion locally

The whole app is containerized, so you only need Docker installed. Everything else (Python, the models, all dependencies) is handled inside the docker image.

Note: All AI runs locally, but the app fetches Wikipedia article text over the network, so you'll need an internet connection at question time. The models are downloaded on first run and cached, so the first launch takes longer than later ones.

1. Clone the repository

git clone https://github.com/kevinbazira/wikipedia-companion.git
cd wikipedia-companion

2. Build the image

docker build --network=host \
-t wikipedia-companion \
-f Dockerfile.cpu .

3. Run the container

docker run --rm \
--network=host \
-e GRADIO_SERVER_NAME=0.0.0.0 \
-v hf-cache:/root/.cache/huggingface \
wikipedia-companion

The -v hf-cache:... volume caches downloaded models so they aren't re-fetched every time you run the container. The first launch downloads several models and may take a few minutes; subsequent runs start quickly.

4. Open the app

Visit http://localhost:7860/ in your browser, then ask the companion a question by voice or text.

Conclusion (Support & Resources)

Thank you for reading about the Wikipedia Companion. What started as a way to bring the warmth of a video tutor to places where bandwidth is expensive turned into a small proof of something I care about: that thoughtful, useful AI doesn't have to live in a giant cloud model. A handful of tiny local models, each doing one job well, can add up to something genuinely helpful.

If you would like to explore further:

• Try the live demo on Hugging Face Spaces.
• Read or fork the source code on GitHub.
• Run it yourself using the local setup steps above.

If you have any questions, run into a challenge, or just want to share what you built with it, please feel free to ask in the comment section below or contact me here. I will be happy to help.