LlmComposer.Providers.Bedrock (llm_composer v0.20.2)

Copy Markdown View Source

Provider implementation for Amazon Bedrock.

Handles chat completion requests through Amazon Bedrock Converse API. Any Bedrock compatible model can be used. To specify any model-specific options for the request, you can pass them in the request_params option and they will be merged into the base request that is prepared.

Timeout configuration

The HTTP receive timeout (how long to wait for data from Bedrock before giving up) can be tuned via application config. The lookup order is:

  1. config :llm_composer, :bedrock, receive_timeout: <ms> — Bedrock-specific override
  2. config :llm_composer, :timeout, <ms> — global llm_composer timeout
  3. Default: 30_000 ms (30 s)

Example:

config :llm_composer, :bedrock, receive_timeout: 60_000

The timeout applies to all Mint-based requests (streaming and non-streaming). Finch regular (non-streaming) requests use Finch's own timeout configuration.

Structured outputs

:response_schema requests JSON output matching a schema. :structured_output_strategy picks how that's requested from Bedrock:

  • :native (default) - uses outputConfig.textFormat.json_schema. Only supported by some models (e.g. newer Anthropic Claude models on Bedrock or Nova models).
  • :tool_use - forces the model to call a synthesized structured_response tool whose input schema is :response_schema, then unwraps that call's arguments into the response content. Works on models without native structured-output support (Mistral, DeepSeek, older Qwen/Llama, etc.), since tool calling is supported far more broadly than outputConfig across Bedrock's model vendors. Note that this forces toolChoice: {"any": {}}, so combining it with :functions means the model could call one of those tools instead of structured_response.

Summary

Functions