Search documentation of instructor_lite

# InstructorLite.Adapters.Llamacpp (instructor_lite v1.0.0)

View Source

LLaMA.cpp HTTP Server adapter.

This adapter is implemented using llama-specific completion endpoint.

## Params

`params` argument should be shaped as a completion request body

## Example

    InstructorLite.instruct(%{
        prompt: "John is 25yo",
        temperature: 0.8
      },
      response_model: %{name: :string, age: :integer},
      adapter: InstructorLite.Adapters.Llamacpp,
      adapter_context: [url: "http://localhost:8000/completion"]
    )
    {:ok, %{name: "John", age: 25}}

# Summary

## Functions

initial_prompt(params, opts)

Updates `params` with prompt based on `json_schema` and `notes`.

parse_response(response, opts)

Parse API response.

retry_prompt(params, resp_params, errors, response, opts)

Updates `params` with prompt for retrying a request.

send_request(params, opts)

Make request to llamacpp HTTP server.

# Functions

# initial_prompt(params, opts)

Updates `params` with prompt based on `json_schema` and `notes`.

It uses `json_schema` and `system_prompt` request parameters.

# parse_response(response, opts)

Parse API response.

Can return:

- `{:ok, parsed_json}` on success.
- `{:error, :unexpected_response, response}` if response is of unexpected shape.

# retry_prompt(params, resp_params, errors, response, opts)

Updates `params` with prompt for retrying a request.

# send_request(params, opts)

Make request to llamacpp HTTP server.

## Options

- `:http_client` (`atom/0`) - Any module that follows `Req.post/2` interface The default value is `Req`.

- `:http_options` (`keyword/0`) - Options passed to `http_client.post/2` The default value is `[receive_timeout: 60000]`.

- `:url` (`String.t/0`) - Required. API endpoint to use, for example `http://localhost:8000/completion`