lc_reward (faber_neuroevolution v1.2.4)

View Source

Reward Signal Computation for Liquid Conglomerate Silos.

Part of the Liquid Conglomerate v2 architecture. This module computes reward signals for each silo's L0 TWEANN, including cooperative penalties/bonuses to encourage alignment between silos.

Reward Structure

Each silo has local objectives plus cooperation terms:

Resource Silo: reward = 0.35*throughput + 0.25*stability + 0.15*efficiency - 0.15*task_blocked - 0.10*distribution_blocked

Task Silo: reward = 0.40*improvement + 0.20*diversity - 0.15*complexity - 0.15*resource_pressure + 0.10*distribution_diversity

Distribution Silo: reward = 0.30*load_balance + 0.25*migration + 0.20*network_eff - 0.15*resource_pressure + 0.10*task_diversity

Global Health Bonus

All silos receive: +0.1 * global_health_improvement where global_health = weighted sum of all silo performances

Summary

Functions

Add global bonus to a silo's reward based on global health improvement.

Compute reward signal for Distribution Silo L0.

Compute global health metric from all silo rewards.

Compute reward signal for Resource Silo L0.

Compute reward signal for Task Silo L0 TWEANN.

Compute reward breakdown for Task Silo L0 - returns individual component scores.

Compute cumulative reward for an LC TWEANN trial.

Functions

add_global_bonus(SiloReward, GlobalState)

-spec add_global_bonus(float(), map()) -> float().

Add global bonus to a silo's reward based on global health improvement.

Each silo gets +0.1 * (current_global - previous_global) as bonus. This encourages silos to cooperate for overall system improvement.

compute_distribution_reward(Metrics)

-spec compute_distribution_reward(map()) -> float().

Compute reward signal for Distribution Silo L0.

Input metrics map should contain: - load_balance_score: How evenly distributed load is (0-1) - migration_effectiveness: Do migrations improve diversity? (0-1) - network_efficiency: Low overhead, successful transfers (0-1) - resource_pressure_caused: Pressure on Resource Silo (0-1) - task_diversity_contribution: Helping Task Silo diversity (0-1)

compute_global_health(SiloRewards)

-spec compute_global_health(map()) -> float().

Compute global health metric from all silo rewards.

Global health is a weighted average of silo rewards. This provides a holistic view of system performance.

compute_resource_reward(Metrics)

-spec compute_resource_reward(map()) -> float().

Compute reward signal for Resource Silo L0.

Input metrics map should contain: - throughput: Evaluations per second (normalized 0-1) - gc_triggers: Number of GC triggers in period (0 = best) - pauses: Number of evolution pauses (0 = best) - memory_utilization: Current memory usage (0-1) - cpu_utilization: Current CPU usage (0-1) - task_silo_blocked: Was Task Silo blocked? (0-1) - distribution_blocked: Was Distribution blocked? (0-1)

compute_task_reward(Metrics)

-spec compute_task_reward(map()) -> float().

Compute reward signal for Task Silo L0 TWEANN.

Designed to reward LC controllers that: 1. MAXIMIZE fitness improvement velocity (faster learning) 2. MINIMIZE training time to convergence (efficiency) 3. PUNISH premature convergence (getting stuck at low fitness) 4. MINIMIZE resource usage (memory/CPU efficiency)

Input metrics map should contain: - improvement_velocity: Fitness improvement rate per 1000 evals (-1 to 1) - best_fitness: Current best fitness achieved (0-1) - fitness_target: Target fitness to reach (0-1) - evaluations_used: Total evaluations consumed - evaluations_budget: Total evaluation budget - stagnation_severity: How stuck evolution is (0=progressing, 1=stuck) - diversity_index: Population diversity (0-1) - memory_pressure: Current memory usage (0-1) - cpu_pressure: Current CPU usage (0-1)

compute_task_reward_breakdown(Metrics)

-spec compute_task_reward_breakdown(map()) -> map().

Compute reward breakdown for Task Silo L0 - returns individual component scores.

Returns a map containing: - total_reward: The combined weighted reward (-1.0 to 1.0) - velocity_score: Fitness improvement velocity score (40% weight) - efficiency_score: Training time efficiency score (20% weight) - convergence_score: Premature convergence penalty score (20% weight) - resource_score: Memory/CPU efficiency score (20% weight) - velocity_weight, efficiency_weight, etc.: The weights used

This allows visualization of which factors are contributing to the reward.

compute_trial_reward(Metrics)

-spec compute_trial_reward(map()) -> float().

Compute cumulative reward for an LC TWEANN trial.

Called at the end of a trial period to evaluate the LC controller's performance. Uses fitness improvement achieved during the trial as the primary metric.

Input metrics map should contain: - start_fitness: Fitness at trial start - end_fitness: Fitness at trial end - evaluations_used: Evaluations consumed during trial - evaluation_budget: Evaluation budget for trial - peak_memory: Peak memory usage during trial (0-1) - avg_cpu: Average CPU usage during trial (0-1) - stagnation_events: Number of stagnation events during trial - convergence_reached: true if fitness target was reached