Service Provider Specific Notes

The following tables show which operations of the LLMClient component each service provider supports, and are available as a quick reference. A No marked with (804) means the component itself refuses the operation with error 804, because the API that the provider implements has no equivalent. Every other entry reflects the provider's own documentation, so consult the documentation (linked below) of the individual service provider for the most up to date information on supported operations, and on which models support them.

OpenAI (API Reference) Supported
Generate Yes
Streaming Yes
Tool calls Yes
CountTokens No (804)
CreateEmbedding Yes
ListModels Yes
Anthropic (Messages API) Supported
Generate Yes
Streaming Yes
Tool calls Yes
CountTokens Yes
CreateEmbedding No (804)
ListModels Yes
Google Gemini (Gemini API) Supported
Generate Yes
Streaming Yes
Tool calls Yes
CountTokens Yes
CreateEmbedding Yes
ListModels Yes
Ollama (OpenAI compatibility) Supported
Generate Yes
Streaming Yes
Tool calls Yes*
CountTokens No (804)
CreateEmbedding Yes
ListModels Yes
LM Studio (OpenAI compatibility) Supported
Generate Yes
Streaming Yes
Tool calls Yes
CountTokens No (804)
CreateEmbedding Yes
ListModels Yes
Groq (API Reference) Supported
Generate Yes
Streaming Yes
Tool calls Yes
CountTokens No (804)
CreateEmbedding No
ListModels Yes
OpenRouter (API Reference) Supported
Generate Yes
Streaming Yes
Tool calls Yes
CountTokens No (804)
CreateEmbedding Yes
ListModels Yes
Together AI (OpenAI compatibility) Supported
Generate Yes
Streaming Yes
Tool calls Yes
CountTokens No (804)
CreateEmbedding Yes**
ListModels Yes
Fireworks AI (OpenAI compatibility) Supported
Generate Yes
Streaming Yes
Tool calls Yes
CountTokens No (804)
CreateEmbedding Yes
ListModels Not documented

* Ollama does not support choosing tools, so the ToolChoice configuration setting must be empty or auto.

** Together AI offers no embedding models through its serverless API, so embeddings require a dedicated deployment.

The custom values of ServiceProvider (spCustomGemini, spCustomAnthropic, and spCustomOpenAI) follow the table of the API they use, except that support for each operation depends on the service: consult its documentation.

Other differences between providers that are worth noting:

  • Ollama does not accept images by http or https URL, so send an image through the Data property of its part or as a base64 data: URL.
  • Groq does not support structured output (the ResponseSchema configuration setting) together with streaming or tools.
  • The EmbeddingDimensions configuration setting is accepted only by models that can shorten their embeddings, such as OpenAI's text-embedding-3 models; other models may reject the request.