Data privacy first
Your device. Your RAM. Your GPU.
Data stays on your PC. The page is hosted on Cloudflare; the model downloads once and runs with WebGPU in your browser. Chats live in IndexedDB, not on our servers.
Chrome or Edge · WebGPU · start with Qwen2.5 3B
Privacy is the product
Most chat apps send every prompt to a remote API. Here, the model is on your machine. Drafts, notes, code, and personal questions do not need to leave your desk for the model to answer.
Cloudflare only serves the static app shell. Model weights come from the public WebLLM / Hugging Face CDN into your browser cache. Conversation context is written to IndexedDB on this PC.
- Your RAM holds the loaded weights while you chat.
- Your GPU does the token generation through WebGPU.
- Your disk keeps cached models and chat history in this browser profile.
What actually happens
You open chat, pick a model, and download it once. Weights land in browser cache. Chat runs on your GPU. Closing the tab does not wipe your threads. Next time, a cached model loads much faster.
Your hardware. Inference uses your RAM and GPU. Prompts are not sent to a remote model API for the local path.
Multiple models. Keep several downloaded. Only one is loaded in VRAM at a time.
Familiar workspace. Sidebar threads, streaming replies, stop, copy, and suggestion chips.
Models worth knowing
The library filters by size and marks recommended picks. Start small if you are testing hardware; move up when quality matters.
Qwen2.5 3B
Best everyday chat for most GPUs
~1.8 GB
Llama 3.2 1B
Low VRAM daily driver
~700 MB
Qwen2.5 Coder 3B
Stronger for code help
~1.8 GB
SmolLM2 360M
Fast smoke test / demos
~200 MB
Qwen2.5 7B
Higher quality if VRAM allows
~4.5 GB
More options live in the in-app model library, including Phi, Gemma, and Mistral variants.
Who this is for
People who want local answers
Drafts, brainstorming, and questions you would rather not paste into a cloud chat box.
Developers testing WebLLM
A working ChatGPT-style shell on Cloudflare with IndexedDB context and multi-model downloads.
Low-trust environments
Shared machines, travel laptops, or air-gapped style workflows after models are cached.
Curious hardware owners
See how far your GPU goes with open instruct models without installing a desktop app.
What we do not do
- We do not train on your chats. There is nothing to train on server-side for the local path.
- We do not require an account to open the browser chat and talk to a downloaded model.
- We do not run the chat model on Workers AI for this PoC. The page is edge-hosted; the brain is local.
Under the hood: Next.js on Cloudflare Workers via OpenNext, WebLLM in the client, IndexedDB for threads. The fullstack-next-cloudflare template remains if you later want auth, D1, or R2.
Next.js · Cloudflare Workers · WebLLM · WebGPU · IndexedDB
FAQ
- Does my chat leave this computer?
- No. After the page loads, prompts and replies stay in the browser. Inference runs with WebGPU on your machine. History is stored in IndexedDB on this device.
- Why do I need to download a model?
- WebLLM needs the model weights locally to run. The first download is larger. After that, the browser cache usually means a much faster load.
- Which model should I start with?
- Qwen2.5 3B is the recommended everyday pick. On low VRAM, try Llama 3.2 1B. For coding, try Qwen2.5 Coder 3B. Tiny models are fine for quick demos.
- Can I use more than one model?
- Yes. Download several and switch when you want. Only one model sits in VRAM at a time; the others stay cached on disk.
- What browser do I need?
- Chrome or Edge with WebGPU support on a GPU-capable device. If WebGPU is missing, the chat will tell you before you download.
Ready when your GPU is.
Open the chat, download a recommended model, ask something. Your data stays on this PC.