> ## Documentation Index
> Fetch the complete documentation index at: https://docs.origami.techmitten.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Configure AI Models: WebLLM and Cloud API Settings
> Run AI inference locally with WebGPU and WebLLM, or connect any OpenAI-compatible cloud provider for narration and slide generation.
Origami AI can run AI inference entirely on your device using WebGPU and WebLLM—no internet connection required after the initial model download. If you prefer a cloud provider, or if your machine doesn't support WebGPU, you can route requests to any OpenAI-compatible API endpoint instead. Both modes are available simultaneously, and you can mix them per feature.
***
## Local WebLLM (Default)
Open **⚙️ Settings → WebLLM** to configure on-device inference.
### Enable / Disable Local AI
Use the **Enable local AI** toggle to turn WebLLM on or off globally. When disabled, Origami AI falls back to your configured cloud API for all AI tasks. When enabled, local inference is used by default unless you override it per feature in **Settings → API**.
### Model Selection
Pick the model you want to load from the model list. Models download on first use and are cached in your browser's storage. Only one model is loaded at a time to preserve tab stability.
### Precision Filter
Filter the model list by numerical format:
| Option | Description |
| ------- | ----------------------------------------------------------------------------------------------- |
| **All** | Show all available model variants |
| **f16** | 16-bit floating point — uses roughly half the VRAM of f32; recommended for most GPUs |
| **f32** | 32-bit floating point — higher precision; use if f16 causes driver instability on your hardware |
### Capability Filter
Narrow the model list by what the model can process:
| Option | Description |
| ---------- | -------------------------------------------------------- |
| **All** | Show all models |
| **Vision** | Show only multimodal models that accept images and video |
| **Text** | Show only text-only models |
### Available Models
| Model | Download Size | Approx. VRAM | Capabilities |
| ------------------ | ------------- | ------------ | -------------------------------------------- |
| **Gemma 2 2B** | \~1.4 GB | \~2 GB | Fast, lightweight text & narration |
| **Llama 3.2 1B** | \~800 MB | \~1.5 GB | Ultra-fast, minimal memory footprint |
| **Llama 3.2 3B** | \~1.7 GB | \~2.5 GB | Balanced performance and reasoning |
| **Phi 3.5 Vision** | \~3.9 GB | \~4 GB | Multimodal — text, image, and video analysis |
On machines with limited GPU memory, start with **Gemma 2 2B** or **Llama 3.2 1B**. Both deliver solid narration quality and load quickly, leaving headroom for FFmpeg rendering and TTS synthesis running in parallel.
***
## Using a Cloud API Instead
Open **⚙️ Settings → API** to connect an OpenAI-compatible cloud provider. Cloud API and local WebLLM are not mutually exclusive—you can enable cloud for specific tasks while keeping local inference for others.
### Connection Settings
| Field | Description |
| --------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| **Endpoint base URL** | The base URL of the OpenAI-compatible API |
| **Model name** | The model identifier to send in each request |
| **API key** | Your provider's secret key (stored locally in browser IndexedDB, never sent to Origami's servers) |
| **Disable thinking** | Turn off extended chain-of-thought reasoning tokens for providers that support a thinking mode (e.g. Claude, DeepSeek R1). Disabling thinking reduces token usage and latency. |
**Example endpoint URLs:**
```text theme={null}
# OpenAI
https://api.openai.com/v1
# Google Gemini (OpenAI-compatible path)
https://generativelanguage.googleapis.com/v1beta/openai/
# OpenRouter
https://openrouter.ai/api/v1
# Groq
https://api.groq.com/openai/v1
# Ollama (local)
http://localhost:11434/v1
```
### Supported Providers
Origami AI works with any OpenAI-compatible API, including:
* **OpenAI** (GPT-4o, GPT-4o-mini, etc.)
* **Google Gemini** (gemini-2.0-flash, gemini-1.5-pro, etc.)
* **OpenRouter** (access hundreds of models through a single key)
* **Groq** (fast inference for open-source models)
* **Ollama** (run models locally with an OpenAI-compatible server)
### Per-Feature Overrides
Use the checkboxes in **Settings → API** to enable cloud routing for specific tasks independently:
| Checkbox | What it controls |
| ------------------------- | -------------------------------------------------- |
| **Use for OCR** | Routes PDF text extraction to the cloud API |
| **Use for script fixing** | Sends AI script-repair requests to the cloud API |
| **Use for slide gen** | Routes slide narration generation to the cloud API |
| **Use for Assistant** | Routes AI Assistant chat to the cloud API |
***
## AI Prompt Customization
Open **⚙️ Settings → AI Prompt** to customize how Origami AI generates narration scripts. You can adjust:
* **Narration tone** — formal, conversational, enthusiastic, concise
* **Narration length** — short summaries vs. in-depth explanations
* **Generation behavior** — system prompt and context hints that guide every script the AI writes
Changes here apply globally to all script generation, but you can always edit individual slide scripts in the Slide Editor after generation.
***
Your API key is stored exclusively in your browser's local IndexedDB store. It is never transmitted to Origami AI's servers and is not included in any telemetry.