Engines and your hardware
The engine is what runs your offline models. The standard one works on any graphics card; optional engines get more out of specific hardware. Engine changes apply now, not at the next launch: installing or removing an engine reloads the model you have open onto the engine that now applies.
Standard engine
Ships with the app. Runs on any graphics card - NVIDIA, AMD, Intel, Apple - via Vulkan (Linux, Windows) or Metal (macOS), and falls back to your processor when needed. If the GPU path misbehaves, the app steps down gracefully and tells you; Try GPU again reloads the open model onto the graphics card at once. A reboot, or a run that never reached the card, is not counted as a crash.
Speed is one number with one meaning on every engine - completion tokens over the time they streamed - and the tokens panel also shows the reading speed when the engine reports it.
CUDA engine (NVIDIA)
A one-click ~850 MB add-on built for NVIDIA cards, offered on the home page when your machine qualifies and managed under Settings → Engines. It reads long prompts and documents much faster; generation speed varies by card generation. Needs an NVIDIA driver from 2023 or newer.
MLX engine (Apple Silicon, preview)
Optional, per model, chats only, no speed promise. Needs macOS 14 or later. Get a model's MLX version and chats for that model run on MLX - its row says so and keeps separate measured speeds for MLX and the standard engine. Remove the MLX version, or the engine, and the standard engine's fit applies again.
Your own server
Connect any OpenAI-compatible server you run - another machine with llama.cpp, a vLLM box, a Mac cluster. Settings → Engines → Your own server: paste the URL, the app health-checks it, measures its real speed, and its models join your picker. Your conversations go only to it. The card shows the server's reading room, its speed and its models.
A connected server is part of your offline models. There is no separate mode for it: on Offline Only and on Online and Offline alike, the server takes a question when it holds the clearly stronger model for it, and stays out of it otherwise. A turn the server answered says so on the turn. A server that stops answering, on connect or halfway through a reply, is set aside for a minute and picked up again on its own; the reply carries on here. Project work can run there too: Settings › Routing has Project work runs on - Auto, this computer, or your server - see Routing settings.
Fine-tune does not reach a server: its reading room and threads are set on that computer, and the card says so. An AI still set to the old Auto - My Hardware choice behaves as Offline Only.
Several graphics cards
A model runs on the biggest card alone when it fits there, and is pooled across all cards only when it does not; pooling is always slower, since a token passes through every card in turn. Model sizes are graded against the biggest card, not a pooled total, and the system line names each card with its size, biggest first. Fine-tune has a Graphics cards row on such computers; see Fine-tune.
Fine-tune this computer
At the bottom of the Engines page: worker threads, the graphics-cards choice on a computer with several, and where the helper models run. The sliders for how replies are written live in their own Settings section, Reply style for every AI. See Fine-tune.
Utility work stays small
Memory, routing verdicts and document reading run on the optional helper - two small models on their own servers, downloaded only when you ask and removable in Settings → Storage. After your chat model loads, each helper starts on the graphics card when the measured room allows and on your processor when it does not, and gives the card back before a model switch. One choice in Settings › Engines, Automatic or Keep on the processor, decides. On the processor a helper gives its memory back when the computer runs short: under sustained pressure an idle helper is stopped, the activity card says so, and it starts again on demand once there is room.