Search documentation of instructor_lite

# Text Classification

View Source

    Mix.install(
      [
        {:instructor_lite, "~> 1.0"},
        {:req, "~> 0.5"}
      ]
    )

## Motivation

Text classification is a common task in NLP and broadly applicable across software. Whether it be spam detection or support ticket categorization, NLP is at the core. Historically, this required training custom, bespoke models that required collecting thousands of pre-labeled examples. With LLMs, a lot of this knowledge is already encoded into the model. With proper instruction and guiding the output to a known set of classifications using GPT, you can be up and running with a text classification model in no time.

Hell, you can even use instructor to help generate the training set to train your own more efficient model. But let's not get ahead of ourselves, there's more on that later in the tutorials.

## Setup

In order to run code in this notebook, you need to add your OpenAI API key as an `OPENAI_KEY` Livebook secret. It will then be accessible through an environmental variable.

    secret_key = System.fetch_env!("LB_OPENAI_KEY")
    :ok

    :ok

## Binary Text Classification

Spam detection is a classic example of binary text classification. It's as simple as returning a true / false of whether an example is in the class. This is pretty trivial to implement in instructor.

    defmodule SpamPrediction do
      use Ecto.Schema
      use InstructorLite.Instruction

      @notes """
      ## Field Descriptions:
      - class: Whether or not the email is spam.
      - reason: A short, less than 10 word rationalization for the classification.
      - score: A confidence score between 0.0 and 1.0 for the classification.
      """
      @primary_key false
      embedded_schema do
        field(:class, Ecto.Enum, values: [:spam, :not_spam])
        field(:reason, :string)
        field(:score, :float)
      end

      @impl true
      def validate_changeset(changeset, _opts) do
        changeset
        |> Ecto.Changeset.validate_number(:score,
          greater_than_or_equal_to: 0.0,
          less_than_or_equal_to: 1.0
        )
      end
    end

    is_spam? = fn text ->
      InstructorLite.instruct(%{
          model: "gpt-4o-mini",
          input: [
            %{
              role: "user",
              content: """
              Your purpose is to classify customer support emails as either spam or not.
              This is for a clothing retail business.
              They sell all types of clothing.

              Classify the following email:
              ```
              #{text}
              ```
              """
            }
          ]
        },
        response_model: SpamPrediction,
        max_retries: 1,
        adapter_context: [api_key: secret_key]
      )
    end

    is_spam?.("Hello I am a Nigerian prince and I would like to send you money")

    {:ok, %SpamPrediction{class: :spam, reason: "Common spam trope", score: 0.95}}

We don't have to stop just at a boolean inclusion, we can also easily extend this idea to multiple categories or classes that we can classify the text into. In this example, let's consider classifying support emails. We want to know whether it's a `general_inquiry`, `billing_issue`, or a `technical_issue` perhaps it rightly fits in multiple classes. This can be useful if we want to cc' specialized support agents when intersecting customer issues occur

We can leverage `Ecto.Enum` to define a schema that restricts the LLM output to be a list of those values. We can also provide a `@notes` description to help guide the LLM with the semantic understanding of what these classifications ought to represent.

    defmodule EmailClassifications do
      use Ecto.Schema
      use InstructorLite.Instruction

      @notes """
      A classification of a customer support email.

      technical_issue - whether the user is having trouble accessing their account
      billing_issue - whether the customer is having trouble managing their billing or credit card
      general_inquiry - all other issues
      """
      @primary_key false
      embedded_schema do
        field(:tags, {:array, Ecto.Enum},
          values: [:general_inquiry, :billing_issue, :technical_issue]
        )
      end
    end

    classify_email = fn text ->
      {:ok, %{tags: result}} =
        InstructorLite.instruct(%{
            input: [
              %{
                role: "user",
                content: "Classify the following text: #{text}"
              }
            ]
          },
          response_model: EmailClassifications,
          adapter_context: [api_key: secret_key]
        )

      result
    end

    classify_email.("My account is locked and I can't access my billing info.")

    [:technical_issue, :billing_issue]

← Previous Page Changelog
Next Page → Computer Vision - Extracting Data from Images