Local Providers

Three cards in Settings → Models cover local models:

• Ollama (Local) — for Ollama's native API
• HuggingFace (Local) — for running Hugging Face Hub models on the SynapseOrch host via transformers + torch
• Local V1 Compatible — for any locally hosted OpenAI-compatible server (vLLM, LM Studio, llama.cpp server, Jan, Text Generation WebUI, Ollama's /v1 endpoint, etc.)

Ollama

Model names: ollama.<name> — a name with no recognised prefix is also sent to Ollama

Install Ollama from ollama.com and pull the models you want:

ollama pull mistral
ollama pull llama3
ollama pull qwen2.5:14b
ollama pull phi4

SynapseOrch looks for Ollama at http://127.0.0.1:11434. There is nothing to fill in: open Settings → Models, and the Ollama (Local) card lists the models it detected, or says "Ollama is not running". Pulled models appear in the selectors as ollama.mistral, ollama.llama3 and so on. ollama.mistral is the default model of a new install. Until Ollama answers and has the model pulled, Chat's welcome line says Model not ready and why — Ollama is not reachable at http://127.0.0.1:11434., or Ollama has no model named mistral. Pull it with ollama pull mistral, or choose another model.

If Ollama runs somewhere else — another host, or outside the SynapseOrch container — point SynapseOrch at it with the ollama_base_url setting. It is used for Ollama chat, embeddings, the model list and memory embeddings. There is no field for it in the UI; set it through the settings API, from a script signed in as Calling the settings API from a script shows:

curl -b synapse.cookies -X POST http://localhost:3000/api/settings \
  -H "Content-Type: application/json" \
  -d '{"ollama_base_url": "http://ollama-host:11434"}'

While the setting is empty, the OLLAMA_BASE_URL environment variable on the backend is used instead; a change to the variable takes a restart.

Nothing embeds through Ollama unless you choose it. An Ollama embedding model (for example nomic-embed-text, after ollama pull nomic-embed-text) can be the Embedding model in Settings → Models, which code search and each step's semantic cache use. Agent Memory searches by keywords and needs no embedding model.

Hugging Face (Local)

Model names: hf.<huggingface_model_id>

Loads models directly from the Hugging Face Hub and runs inference in the SynapseOrch backend process via transformers + torch. No API call leaves your machine, but the model is downloaded the first time it is used. It runs on a CUDA GPU if there is one, then Apple's MPS, then the CPU.

Prerequisites on the host: torch and transformers installed and importable from the backend's Python environment. A GPU is strongly recommended for anything beyond small models.

In Settings → Models click the HuggingFace (Local) card and fill:

Click Save at the foot of the card, then reference models as hf.<the full repo id>, e.g. hf.Qwen/Qwen2.5-7B-Instruct.

The first call to a model pays a 20–60s load cost; afterwards the model stays in memory for fast subsequent calls. Each call generates up to 1,024 new tokens — the huggingface_max_new_tokens setting, which has no field on the card.

The card's fields are stored as the huggingface_token and huggingface_models settings.

Local OpenAI-Compatible (vLLM, LM Studio, etc.)

Model names: locv1.<model_name>

In Settings → Models click the Local V1 Compatible card and fill:

Click Save at the foot of the card (the key has its own Save beside its field). If the server answers /v1/models, its models are listed automatically. Reference models as locv1.mistral-7b (prefix + the model's name).

The address is reached from the SynapseOrch backend: localhost means the backend's own machine — or its container, under Docker.

Compatible servers

Embedding models

Names in Embedding model names — and models with "embed" in their name that the server lists — are offered in the Embedding model selector under Defaults in Settings → Models. Nothing is used until you pick one there. The embedding model indexes code repositories and matches each step's semantic cache.

The card's fields are stored as local_compatible_base_url, local_compatible_key, local_compatible_models and local_compatible_embed_models.

Hardware requirements

What a model needs depends on its size, its quantisation and the runtime — a 4-bit 7B model runs on hardware that the same model at full precision would not. Check the model card and your runtime's documentation (Ollama, vLLM, LM Studio) for memory requirements. As a rule, a model has to fit in GPU memory to run at interactive speed; CPU-only inference works for small models, slowly.