faber-tweann

View Source

Topology and Weight Evolving Artificial Neural Networks for Erlang

Hex.pm Documentation License Buy Me A Coffee

Evolutionary neural networks that evolve both topology and weights, now with Liquid Time-Constant (LTC) neurons for adaptive temporal processing. Based on DXNN2 by Gene Sher.

Highlights

  • First TWEANN library with LTC neurons in Erlang/OTP
  • CfC closed-form approximation - ~100x faster than ODE-based LTC
  • Rust NIF acceleration - Native implementations of fitness statistics, novelty search, selection and network evaluation (speedup not yet measured; see below)
  • Pure Erlang reference implementation - Selected explicitly, held in agreement with the native path by a conformance test
  • Hybrid networks - Mix standard and LTC neurons in the same network
  • Production ready - Comprehensive logging, error handling, and process safety

Quick Start

%% Add to rebar.config
{deps, [{faber_tweann, "~> 2.0"}]}.

%% Create and evolve a standard network
genotype:init_db(),
SpecieId = genotype:generate_id(specie),
AgentId  = genotype:generate_id(agent),
Constraint = #constraint{morphology = xor_mimic},
genotype:construct_Agent(SpecieId, AgentId, Constraint),
genome_mutator:mutate(AgentId).

%% Use LTC dynamics directly
{NewState, Output} = ltc_dynamics:evaluate_cfc(Input, State, Tau, Bound).

LTC Neurons

Liquid Time-Constant neurons enable adaptive temporal processing with input-dependent time constants:

LTC Architecture

%% CfC evaluation (fast, closed-form)
{State1, _} = ltc_dynamics:evaluate_cfc(1.0, 0.0, 1.0, 1.0),
{State2, _} = ltc_dynamics:evaluate_cfc(1.0, State1, 1.0, 1.0).
%% State persists between evaluations - temporal memory!

Key equations:

  • LTC ODE: dx/dt = -[1/τ + f(x,I,θ)]·x + f(x,I,θ)·A
  • CfC: x(t+Δt) = σ(-f)·x(t) + (1-σ(-f))·h (100x faster)

See the LTC Neurons Guide for details.

Documentation

Features

Neural Network Evolution

  • Topology Evolution: Networks add neurons and connections. Complexification only: there is no remove-neuron or remove-link operator, so a lineage's structure is monotone non-decreasing. Parsimony pressure is available instead, through fitness_postprocessor:size_proportional/2
  • Weight Evolution: Synaptic weights optimized through selection
  • Speciation: Behavioral diversity preservation (NEAT-style)
  • Multi-objective: Pareto dominance optimization
  • Genotype serialization: genotype:to_binary/1, from_binary/1 and genome_id/1. Canonical and lossless, so an evolved topology can be persisted, sent to another node, and content-addressed. Hand-rolled rather than term_to_binary/2, whose deterministic option is not stable across OTP releases and is therefore unfit for an address
  • Reproducible runs: genotype_rand:seed/1 makes construction and mutation a function of the seed, and the genotype layer draws from its own generator so it never advances the caller's. state/0 and set_state/1 let a caller carry the generator across a call
  • Two ways to evaluate an evolved genotype: network_evaluator:from_genotype/1 for a topology that fits a stack of dense layers, and genotype_to_dag:compile/1 for any acyclic topology at all, through the Rust DAG evaluator. Both convert faithfully or refuse with a reason; neither approximates
  • Memory on the DAG path, in three kinds, evaluated with tweann_nif:evaluate_with_state/3 and carrying one float of state per stateful node rather than one per neuron:
    • delay emits what it captured last tick and applies no activation, so it adds no ordering constraint and a feedback path through one is not a cycle
    • leaky moves its state toward its input by one part in time_constant each tick; it reads this tick's inputs, so it is ordered normally
    • cfc is the closed-form continuous-time neuron, computing exactly what tweann_nif:evaluate_cfc/4 computes
  • Evolution introduces them: add_delay and add_leaky splice an organelle into an existing connection, and mutate_time_constant tunes a leaky organelle's tau. Opt in per constraint: organelles run on the DAG path only, and the process phenotype raises on one rather than evaluating it as an ordinary neuron

LTC/CfC Neurons

  • Temporal Memory: Neurons maintain persistent internal state
  • Adaptive Dynamics: Input-dependent time constants
  • CfC Mode: ~100x faster than ODE-based evaluation
  • Hybrid Networks: Mix standard and LTC neurons

Production Quality

  • Process Safety: Timeouts and crash handling
  • Comprehensive Logging: Structured logging throughout
  • Rust NIF (optional): High-performance network evaluation
  • ETS Storage: Fast in-memory genotype storage (see ROADMAP.md for planned Mnesia persistence)

Native Acceleration

Rust NIFs for the numeric hot paths ship with this package and are built from source at compile time. A Rust toolchain is required.

There is one edition. The faber-nn-nifs package was absorbed into faber_tweann in v2.0.0; if you depended on faber_nn_nifs directly, depend on faber_tweann instead.

tweann_nif:impl().       %% faber_nn_nifs | tweann_nif_fallback
tweann_nif:is_loaded().  %% true when the native path is active

The native path is the default. If the library is missing or fails to load, faber_tweann raises on first use rather than quietly falling back. To use the pure Erlang implementation deliberately:

[{faber_tweann, [{nif_impl, fallback}]}].

See the Native Acceleration guide.

On performance claims: earlier versions of this README quoted speedups between 30x and 200x, while the faber-nn-nifs README quoted 10-15x for the same code. Neither figure was backed by a committed measurement, and the two were mutually inconsistent. No speedup figures are published here until a benchmark runs and its output is committed. test/benchmark/bench_nif_vs_erlang.erl exists for this purpose.

Architecture

Module Dependencies

Process-based neural networks with evolutionary operators. See Architecture Guide for details.

Testing

rebar3 eunit          # Unit tests
rebar3 dialyzer       # Static analysis
rebar3 ex_doc         # Generate documentation

Academic References

TWEANN/NEAT

LTC/CfC Neurons

  • Hasani, R., Lechner, M., et al. (2021). Liquid Time-constant Networks. Proceedings of the AAAI Conference on Artificial Intelligence, 35(9), 7657-7666.

    • Introduces adaptive time-constant neurons with continuous-time dynamics.

  • Hasani, R., Lechner, M., et al. (2022). Closed-form Continuous-time Neural Networks. Nature Machine Intelligence, 4, 992-1003.

    • CfC closed-form approximation enabling ~100x speedup over ODE-based LTC.

Weight Initialization

Evolutionary Algorithms

  • Holland, J.H. (1975). Adaptation in Natural and Artificial Systems. MIT Press.

    • Foundational text on genetic algorithms.

  • Yao, X. (1999). Evolving Artificial Neural Networks. Proceedings of the IEEE, 87(9), 1423-1447.

    • Comprehensive survey of neuroevolution approaches.

ONNX Export

  • ONNX Consortium (2017-present). Open Neural Network Exchange.
    • Open standard for neural network interoperability enabling cross-platform inference.

Faber Ecosystem

  • macula - HTTP/3 mesh networking platform with NAT traversal, Pub/Sub, and async RPC. Enables distributed neuroevolution across edge devices.

  • faber_neuroevolution - Population-based evolutionary training engine that orchestrates neural network evolution using this library.

License

Apache License 2.0 - See LICENSE

Credits

Based on DXNN2 by Gene Sher. Adapted with LTC extensions by R.G. Lefever.