BinClass

An easy-to-use Elixir library for building, training, and deploying binary text classifiers with Axon.

This library provides a simplified interface for training a neural network on text data and using it for predictions, handling tokenization, vectorization, and model training out of the box.

Installation

The package can be installed by adding bin_class to your list of dependencies in mix.exs:

def deps do
  [
    {:bin_class, "~> 0.3.0"}
  ]
end

Quick Start

1. Prepare your data

Data can be an enumerable of maps (%{text: "...", label: 1} or %{"text" => "...", "label" => 1}) or tuples ({"...", label}).

data = [
  %{text: "This is a great product!", label: 1},
  %{text: "I really hated this experience.", label: 0},
  # ... more samples
]

# Inspect dataset balance and text length statistics:
summary = BinClass.summary(data)
IO.puts(BinClass.format_summary(summary))

# Stratified train / validation split:
{train_data, val_data} = BinClass.Dataset.split(data, validation_split: 0.1, seed: 42)

2. Train the model

# Output labels can be customized with a two-element list or map.
classifier = BinClass.train(train_data,
  epochs: 5,
  labels: %{0 => "negative", 1 => "positive"},
  label_smoothing: 0.02
)

By default, training uses the :parallel_cnn architecture with 2-head attentive pooling, a fixed sequence length of 512, NFKC lowercasing normalizer, vocabulary size 8_000, and argmax classification.

3. Evaluate the model

result = BinClass.evaluate(classifier, val_data)
IO.puts(BinClass.format_report(result))

This outputs a comprehensive evaluation report with Accuracy, Balanced Accuracy, Precision, Recall, Specificity, F1 Score, Cross-Entropy Loss, and the full Confusion Matrix.

Soft targets

Training labels may also represent a graded positive-class probability. Enable soft targets explicitly and provide values from 0.0 through 1.0:

data = [
  %{text: "Clearly negative example", label: 0.0},
  %{text: "Weak positive evidence", label: 0.2},
  %{text: "Strong positive evidence", label: 0.9},
  %{text: "Clearly positive example", label: 1.0}
]

classifier = BinClass.Trainer.train(data,
  target_mode: :soft,
  target_threshold: 0.5
)

Each value p is trained as the distribution [1 - p, p]. The target threshold is used only to derive binary classes for balancing and validation metrics. Hard 0/1 targets remain the default. Target settings are retained as training metadata when the classifier is serialized.

Decision policies and input validation

Classification defaults to argmax. Decision behavior is explicit and does not depend on model architecture. A fixed positive threshold can be selected with:

classifier = BinClass.Trainer.train(data,
  decision_policy: {:positive_threshold, 0.7}
)

Threshold optimization is opt-in and uses the validation partition. Supported objectives are :accuracy, :balanced_accuracy, and :f1. An optional false-positive cost is subtracted from the objective score as a multiple of the false-positive rate. Optimization requires at least one eligible validation example from each class:

classifier = BinClass.Trainer.train(data,
  decision_policy:
    {:optimize_threshold,
     [objective: :balanced_accuracy, false_positive_cost: 0.75]}
)

The selected threshold and optimization metadata are persisted with the classifier. To abstain from classifying short inputs, configure input validation separately:

classifier = BinClass.Trainer.train(data,
  minimum_input_tokens: 64
)

An input below the minimum returns label: :insufficient_input and confidence: nil. Its raw class probabilities remain available in the :probabilities map. Every result also reports :active_token_count, defined as the number of truncated token IDs other than the tokenizer's padding ID. Literal [PAD] tokens therefore cannot satisfy the minimum. Ineligible inputs are excluded from threshold optimization and reported validation accuracy.

Evaluation and calibration

BinClass.Evaluation can also evaluate probabilities produced by external or multi-stage inference pipelines. Fixed-threshold evaluation returns confusion counts, accuracy, precision, recall, specificity, false-positive rate, balanced accuracy, F1, Brier score, and the configured objective score:

metrics =
  BinClass.Evaluation.evaluate(rows,
    threshold: 0.7,
    probability: :positive_probability,
    label: :positive?,
    target: :soft_target,
    eligible: :eligible?
  )

The probability, label, soft target, and eligibility selectors may be map keys or one-argument functions. The :target and :eligible selectors are optional.

Use calibrate/2 to inspect every distinct decision boundary represented by the observed probabilities:

metrics =
  BinClass.Evaluation.calibrate(rows,
    objective: :balanced_accuracy,
    false_positive_cost: 0.75
  )

Named probability strategies can be calibrated and compared in one call:

selection =
  BinClass.Evaluation.select(rows,
    strategies: [
      maximum: & &1.probabilities.maximum,
      top_two_mean: & &1.probabilities.top_two_mean
    ],
    label: :label,
    false_positive_cost: 0.75
  )

selection.strategy
#=> :top_two_mean

4. Save and Load

You can save the entire model (including tokenizer, parameters, decision policy, and metadata) to a single file.

BinClass.save(classifier, "my_model.bin")

# Load the model as an Nx.Serving struct (recommended for most apps)
serving = BinClass.load("my_model.bin")

# Or build a serving from an existing classifier without serializing it
serving = BinClass.serving(classifier, batch_size: 32)

5. Optimized Inference

There are two ways to run predictions:

A. Using Nx.Serving (High Throughput)

Recommended for web servers and concurrent applications. It handles automatic batching.

prediction = Nx.Serving.run(serving, "I love this library!")

B. Using a Compiled Predictor (Ultra-Low Latency)

Recommended for CLI tools or scripts where you want the lowest possible latency for single items by bypassing the serving overhead.

classifier = BinClass.load_classifier("my_model.bin")
predict = BinClass.compile_predictor(classifier)

result = predict.("This is ultra fast.")

C. Inspecting a Complete Document

Compile a document predictor when an input may exceed the classifier's fixed vector length. It scans the complete string using overlapping token windows and returns every chunk with its exact byte span and raw prediction:

classifier = BinClass.load_classifier("my_model.bin")

predict_document =
  BinClass.compile_document_predictor(classifier,
    chunk_overlap: 128,
    batch_size: 32
  )

%{status: :ok, chunks: chunks} = predict_document.(large_document)

Enum.each(chunks, fn chunk ->
  IO.inspect({
    chunk.start_byte,
    chunk.end_byte,
    chunk.token_count,
    chunk.active_token_count,
    chunk.prediction.probabilities
  })
end)

The predictor accepts one document at a time. It does not normalize, truncate, aggregate, or apply a separate document-level decision. Byte offsets refer to the exact input string, with an inclusive start and exclusive end. A chunk's :token_count is its encoded size, while :active_token_count excludes padding special tokens and controls minimum-input validation. Individual chunks that fail validation retain their raw probabilities.

Examples

Check out the examples/ directory for scripts demonstrating various use cases:

• simple_inference.exs: Shows how to quickly run predictions with a pre-trained model.
• train_and_save.exs: Demonstrates the full workflow of training a model and saving it to disk.
• production_serving.exs: Illustrates how to integrate BinClass into a supervision tree for production environments.
• configurable_backend.exs: Shows how to use custom compilers and definition options for training and inference.

Public API

The supported API is organized around these modules:

Modules hidden from the generated documentation are implementation details and may change without notice. Use the BinClass entry points for persistence and inference instead of calling serialization, serving, or predictor internals directly.

Features

• Production Ready: Built on Nx.Serving for automatic batching and process isolation.
• Top-Level Convenience: One-line training (BinClass.train/2), dataset inspection (BinClass.summary/1), and comprehensive evaluation (BinClass.evaluate/3).
• Unified Serialization: Save and load the entire classifier state, including tokenizer, model parameters, explicit decision policy, input validation, and training metadata, from a single file.
• Named Architectures: Serialized classifiers identify their architecture explicitly by name.
• Multiple Architectures: Supports Parallel CNN with Attentive Pooling (default), CNN variants, Sep-SE-CNN (Separable Convolutions + Squeeze-and-Excitation), and Transformer Encoder.
• Target Regularization: Supports :label_smoothing to improve calibration and prevent overconfidence on ambiguous boundary labels.
• Explicit Decisions: Defaults to argmax and supports fixed or validation-optimized positive thresholds independently of architecture.
• Input Abstention: Optionally returns :insufficient_input for inputs below a configured token minimum while retaining raw probabilities.
• Complete-Document Inspection: Scans arbitrarily long text into overlapping token windows and returns every chunk's exact input byte span and raw prediction.
• Early Stopping: Automatically halts training when validation loss stops improving.
• Automatic Class Balancing: Handles imbalanced datasets via automated oversampling.
• Automated Tokenization: Automatically builds vocabulary from training data or accepts custom streams.
• Efficient: Uses EXLA as the default compiler for high-performance training and inference, with support for other Nx backends and compilers.

Model Architectures

Classifiers store one of these named architectures:

• :parallel_cnn: parallel CNN with multi-scale kernel convolutions and multi-head attentive pooling (default)
• :dense_dilated_cnn: dense-connected multi-scale dilated CNN with 4-head attentive pooling
• :cnn: original CNN
• :cnn_mixed_pooling: CNN with mixed pooling
• :multi_scale_cnn: multi-scale CNN
• :sep_se_cnn: separable CNN with squeeze-and-excitation
• :transformer: transformer encoder

No architecture implicitly changes label thresholds or rejects short inputs. Those behaviors are configured through :decision_policy and :minimum_input_tokens.

Numeric architecture identifiers are not accepted. Serialized classifiers must contain one of the names above.

Production Notes

For production training, start with argmax unless validation results justify a different operating point:

classifier = BinClass.train(data,
  epochs: 5,
  architecture: :parallel_cnn,
  vector_length: 512,
  decision_policy:
    {:optimize_threshold,
     [objective: :balanced_accuracy, false_positive_cost: 0.5]},
  label_smoothing: 0.02,
  minimum_input_tokens: 64
)

The explicit decision_policy and minimum_input_tokens are saved with the model and reused identically by BinClass.load/2 and BinClass.compile_predictor/2. Raw probabilities are returned for every input, including inputs that fail minimum-length validation.