Models & Providers

Anyy routes ordinary conversations through one main provider/model. You can add ordered text fallbacks, switch a single session independently, and configure separate image, video, and voice routes.

Choose a Configuration Surface

Goal	Recommended path
Configure one provider and main model	anyy setup model
See and edit every model route visually	Dashboard → Models
Add or revise the fallback chain	anyy setup model fallback or Dashboard → Models
Inspect active paths and credential status	anyy config
Switch only the current conversation	Dashboard Chat selector or TUI /model

For most first-time setups, start with the interactive wizard:

anyy setup model

What Makes a Model Route

A usable route has four parts:

1. Provider — the service family Anyy integrates with.
2. Endpoint and protocol — the API contract and base URL used for requests.
3. Authentication — API key, supported subscription login, or no key for a trusted local service.
4. Model — the exact model ID sent to that endpoint.

The installed provider catalog defines valid combinations. A model name alone does not imply that every provider or endpoint supports it.

Local providers

Start Ollama or LM Studio before setup, then select it in the provider list. If discovery fails, verify the local server address and confirm that at least one model is installed.

Compatible endpoints

Choose an OpenAI-compatible or Anthropic-compatible route only when the service implements that protocol. Enter its exact base URL and model ID.

Provider choice is a data-boundary choice

The selected provider can receive prompts, context, attachments, and tool results needed for a request. Review its privacy, retention, account, and billing terms.

Configure the Main Model in Dashboard

Open Dashboard → Models → Main model. The form exposes only choices supported by the selected provider contract:

The Text, Image, Video, and Voice cards show their current provider/model and whether each route is primary, enabled, disabled, or unavailable.

1. Choose a provider type.
2. Choose the service and protocol when more than one is available.
3. Choose an authentication method.
4. Enter an API key or complete subscription login when required.
5. Keep the default base URL, or enter a trusted compatible endpoint.
6. Refresh the model catalog or switch to manual model entry.
7. Choose the model and reasoning defaults.
8. Select Test connection, then Save.

The configuration preview shows the non-secret values that will be written. Saved API keys are never displayed in full.

Test before saving

Test again after changing the provider, endpoint, credential, protocol, or model. A saved model ID does not prove that the provider accepts real requests.

For the terminal workflow, run:

anyy setup model
anyy config
anyy status

anyy config shows the selected provider, model, endpoint, config path, and whether credential references resolve. It does not print stored secret values.

Background Capability Refresh

When Gateway starts, Anyy performs a best-effort background refresh of model capability metadata. A provider catalog is skipped when the configured models already have fresh cached metadata from the last 24 hours; the filtered models.dev metadata cache uses the same 24-hour freshness window.

The refresh may send a catalog request to a configured provider's models endpoint, using its resolved credential when that endpoint requires authentication, and may fetch metadata from models.dev. It does not send conversation content or run model inference. Refresh failure does not stop Gateway: Anyy continues with usable cached or bundled metadata.

Select Refresh models when you need an immediate catalog check instead of waiting for the next Gateway start.

Authentication and Credentials

Depending on the provider, Dashboard can:

• enter, replace, or clear an API key;
• start or repeat an OAuth/subscription login; or
• disconnect a connected subscription profile.

Authentication belongs to the selected provider endpoint. Changing the service or protocol can change the credential realm and available model catalog.

Anyy-managed credentials are stored separately under <PROFILE_HOME>/secrets/ with file mode 0600; config.yaml contains references and non-secret route settings. Keep credentials out of prompts, shell history, screenshots, and issue reports.

Configure Fallback Routing

Add a fallback from Dashboard's Text tab or run:

anyy setup model fallback

Fallbacks are evaluated in their saved order. Each entry can be enabled or disabled and can control:

• whether routing may accept reduced reasoning capability;
• whether it may accept a smaller context window; and
• the maximum attempts for that route.

Remove a fallback from its detail page when it should no longer be eligible.

Fallback routing chooses an alternative when the current target cannot complete under policy. It does not merge answers or ask every configured model.

One logical turn can still produce more than one physical provider attempt. Anyy may retry a transient failure, recover from a context-limit response, refresh authentication, or move to a compatible fallback. A failed route can enter a cooldown, allowing a later turn to start on an eligible fallback instead of repeatedly calling the same unavailable target. These attempts can add latency, token usage, and provider charges; use the Chat Usage view and runtime status when investigating them.

Use credentials and endpoints that are actually independent when resilience matters. Two models behind the same unavailable service do not provide provider-level redundancy.

Change the Current Session

The saved main model is the default for new sessions. An existing session can retain its selected provider, model, and reasoning state until you change it or start a new session.

Use the model/reasoning control in Dashboard Chat. In the TUI:

/model
/provider
/effort

Start /new after a large model or reasoning change when you want clean context. Switching reasoning effort can invalidate prompt-cache reuse and increase token charges.

Configure Image and Video Routes

Dashboard groups routes into Text, Image, Video, and Voice.

The main model can understand image or video input automatically only when its catalog declares that capability. Otherwise, choose a dedicated provider/model. Understanding routes can also configure:

• enabled state;
• auto or summary_first processing;
• a capability prompt;
• token or character limits;
• timeout; and
• thinking behavior.

summary_first requires a dedicated route. Test manually entered model IDs; Anyy does not assume that an unknown model supports image or video input.

Image generation is a separate output route with its own provider/model, default aspect ratio, timeout, and byte limit.

See Media Understanding and Image Generation for usage.

Configure Voice Routes

The Voice tab contains:

• Text to speech (TTS) — provider, model, protocol, voice, output behavior, and audio preview; and
• Speech to text (STT) — provider, model, protocol, default language, and connection test.

Enable only routes with a working backend. A provider that supports text chat does not automatically support speech endpoints.

See Voice & Speech for transcription and reply behavior.

When Changes Take Effect

Dashboard saves through the running Gateway. New sessions use the saved main model. Existing sessions can keep their own selection, and newly enabled capabilities may require a new session before their tools appear.

Troubleshooting

Problem	Fix
Model list is empty	Check authentication and base URL, then select Refresh models
Test connection fails	Verify endpoint, credential realm, model ID, and provider account access
Main model shows unavailable	Open its detail page, test it, and save a valid route
Local connection is refused	Start Ollama or LM Studio and verify its base URL
Fallback never helps	Test each route separately and confirm it is enabled and correctly ordered
Image/video requires a dedicated model	The main model lacks the declared input capability, or summary_first is selected
A saved model is absent in Chat	Switch the current session explicitly or start a new session
Subscription login is stale	Reauthenticate, or disconnect and connect the intended profile
Cost or latency is unexpected	Inspect provider, model, reasoning effort, fallback attempts, and media routes