Engines and your hardware

The engine is what runs your offline models. The standard one works on any graphics card; optional engines get more out of specific hardware.

Standard engine

Ships with the app. Runs on any graphics card - NVIDIA, AMD, Intel, Apple - via Vulkan (Linux, Windows) or Metal (macOS), and falls back to your processor when needed. If the GPU path misbehaves, the app steps down gracefully and tells you, with a one-click "try again" once things change.

CUDA engine (NVIDIA)

A one-click ~850 MB add-on built for NVIDIA cards, offered on the home page when your machine qualifies and managed under Settings → Engines. It reads long prompts and documents much faster; generation speed varies by card generation. Needs an NVIDIA driver from 2023 or newer.

Your own server

Connect any OpenAI-compatible server you run - another machine with llama.cpp, a vLLM box, a Mac cluster. Settings → Engines → Your own server: paste the URL, the app health-checks it, measures its real speed, and its models join your picker. Your conversations go only to it.

With a server connected, a third auto mode appears - Auto - My Hardware - that treats your connected machines as routing candidates alongside this device, never the online relay.

Utility work stays small

Embeddings and background utility tasks always run on small bundled models on your processor - they never compete with your chat model for graphics memory.