> ## Documentation Index
> Fetch the complete documentation index at: https://docs.origami.techmitten.com/llms.txt
> Use this file to discover all available pages before exploring further.

# AI Assistant: Local WebLLM Chat with Vision Support

> Chat with local WebLLM models in your browser — zero data uploaded. Attach images and videos for vision analysis, or fall back to any cloud API.

The AI Assistant lets you chat with powerful language models entirely inside your browser. After a one-time model download, every conversation runs locally on your device using WebGPU — no internet connection required, nothing ever leaves your machine. If your device doesn't support WebGPU, you can route the Assistant through any OpenAI-compatible cloud API instead.

## Local vs Cloud Mode

<Tabs>
  <Tab title="Local (WebGPU)">
    When WebGPU is available, Origami AI runs inference directly on your GPU using WebLLM. Every token is generated on your device — your messages are never sent to any server.

    **Requirements:**

    * Chrome or Edge 113+, Firefox Nightly, or Safari 18+ (macOS Sonoma+)
    * Hardware acceleration enabled in your browser settings
    * Sufficient GPU memory for the model you choose (see the table below)

    You can verify WebGPU status in Chrome by navigating to `chrome://gpu` and checking for **WebGPU: Hardware accelerated**.
  </Tab>

  <Tab title="Cloud Fallback">
    If your browser doesn't support WebGPU, or you prefer a cloud model, you can connect any OpenAI-compatible API:

    1. Open **⚙️ Settings → API**.
    2. Enter your **Base URL**, **Model name**, and **API key**.
    3. Toggle **Use for Assistant** to route AI Assistant chat through your provider.

    Compatible providers include OpenAI, Google Gemini (via the OpenAI-compatible endpoint), OpenRouter, Groq, Ollama, and others. Your API key is stored locally in your browser and never sent to Origami AI's servers.
  </Tab>
</Tabs>

## Supported Models (Local)

Each model downloads once and is cached in your browser for offline use. Choose based on your hardware and the kind of tasks you need.

| Model              | Download Size | Approx. VRAM | Best For                                       |
| ------------------ | ------------- | ------------ | ---------------------------------------------- |
| **Gemma 2 2B**     | \~1.4 GB      | \~2 GB       | Fast responses, general text, narration drafts |
| **Llama 3.2 1B**   | \~800 MB      | \~1.5 GB     | Ultra-fast replies on low-memory devices       |
| **Llama 3.2 3B**   | \~1.7 GB      | \~2.5 GB     | Balanced reasoning quality and speed           |
| **Phi 3.5 Vision** | \~3.9 GB      | \~4 GB       | Multimodal — adds image and video analysis     |

You can filter by precision (f16 / f32) or capability (Text / Vision) in **⚙️ Settings → WebLLM** to narrow down the model list to what your hardware can support.

## Attaching Files

Phi 3.5 Vision (local) and vision-capable cloud models such as `gpt-4o` or `gemini-1.5-flash` can analyze images and video frames you attach to a message.

<CardGroup cols={2}>
  <Card title="Images" icon="image">
    **Formats:** JPEG, PNG, WEBP\
    **Max size:** 8 MB per file

    Click the attachment button in the chat input, select your image, and send it with your question. The model will describe, analyze, or answer questions about the visual content.
  </Card>

  <Card title="Video Clips" icon="video">
    **Formats:** WebM, MP4\
    **Max size:** 20 MB per file

    Attach a short clip and ask the model to summarize, identify objects, or describe what's happening on screen.
  </Card>
</CardGroup>

<Note>
  File analysis requires a vision-capable model. Locally, load **Phi 3.5 Vision** (select **Vision** in the **⚙️ Settings → WebLLM** capability filter). With a cloud provider, use a model that supports multimodal input such as `gpt-4o` or `gemini-1.5-flash`.
</Note>

## Chat Sessions

Every conversation is automatically saved to your browser's local storage. You can:

* **Start a new chat** at any time from the session panel.
* **Resume a previous session** — your full message history is preserved across page reloads and browser restarts.
* **Switch between sessions** without losing context in any of them.

No account or cloud sync is required. All session data lives in your browser's IndexedDB.

<Note>
  The first time you select a local model, Origami AI downloads it to your browser cache. Download sizes range from \~800 MB (Llama 3.2 1B) to \~3.9 GB (Phi 3.5 Vision). Subsequent loads are instant — no re-download needed unless you clear your browser's site data.
</Note>

<Warning>
  If you see a **"Device lost"** error during a conversation, your GPU ran out of memory. To recover:

  1. **Close other browser tabs** to free up GPU memory, then reload the page.
  2. **Switch to a smaller model** — for example, Gemma 2 2B or Llama 3.2 1B use significantly less VRAM.
  3. **Use a cloud provider** — go to **⚙️ Settings → API**, configure an OpenAI-compatible endpoint, and enable **Use for Assistant** to bypass local GPU requirements entirely.
</Warning>