lc_reward (faber_neuroevolution v1.2.4)
View SourceReward Signal Computation for Liquid Conglomerate Silos.
Part of the Liquid Conglomerate v2 architecture. This module computes reward signals for each silo's L0 TWEANN, including cooperative penalties/bonuses to encourage alignment between silos.
Reward Structure
Each silo has local objectives plus cooperation terms:
Resource Silo: reward = 0.35*throughput + 0.25*stability + 0.15*efficiency - 0.15*task_blocked - 0.10*distribution_blocked
Task Silo: reward = 0.40*improvement + 0.20*diversity - 0.15*complexity - 0.15*resource_pressure + 0.10*distribution_diversity
Distribution Silo: reward = 0.30*load_balance + 0.25*migration + 0.20*network_eff - 0.15*resource_pressure + 0.10*task_diversity
Global Health Bonus
All silos receive: +0.1 * global_health_improvement where global_health = weighted sum of all silo performances
Summary
Functions
Add global bonus to a silo's reward based on global health improvement.
Compute reward signal for Distribution Silo L0.
Compute global health metric from all silo rewards.
Compute reward signal for Resource Silo L0.
Compute reward signal for Task Silo L0 TWEANN.
Compute reward breakdown for Task Silo L0 - returns individual component scores.
Compute cumulative reward for an LC TWEANN trial.
Functions
Add global bonus to a silo's reward based on global health improvement.
Each silo gets +0.1 * (current_global - previous_global) as bonus. This encourages silos to cooperate for overall system improvement.
Compute reward signal for Distribution Silo L0.
Input metrics map should contain: - load_balance_score: How evenly distributed load is (0-1) - migration_effectiveness: Do migrations improve diversity? (0-1) - network_efficiency: Low overhead, successful transfers (0-1) - resource_pressure_caused: Pressure on Resource Silo (0-1) - task_diversity_contribution: Helping Task Silo diversity (0-1)
Compute global health metric from all silo rewards.
Global health is a weighted average of silo rewards. This provides a holistic view of system performance.
Compute reward signal for Resource Silo L0.
Input metrics map should contain: - throughput: Evaluations per second (normalized 0-1) - gc_triggers: Number of GC triggers in period (0 = best) - pauses: Number of evolution pauses (0 = best) - memory_utilization: Current memory usage (0-1) - cpu_utilization: Current CPU usage (0-1) - task_silo_blocked: Was Task Silo blocked? (0-1) - distribution_blocked: Was Distribution blocked? (0-1)
Compute reward signal for Task Silo L0 TWEANN.
Designed to reward LC controllers that: 1. MAXIMIZE fitness improvement velocity (faster learning) 2. MINIMIZE training time to convergence (efficiency) 3. PUNISH premature convergence (getting stuck at low fitness) 4. MINIMIZE resource usage (memory/CPU efficiency)
Input metrics map should contain: - improvement_velocity: Fitness improvement rate per 1000 evals (-1 to 1) - best_fitness: Current best fitness achieved (0-1) - fitness_target: Target fitness to reach (0-1) - evaluations_used: Total evaluations consumed - evaluations_budget: Total evaluation budget - stagnation_severity: How stuck evolution is (0=progressing, 1=stuck) - diversity_index: Population diversity (0-1) - memory_pressure: Current memory usage (0-1) - cpu_pressure: Current CPU usage (0-1)
Compute reward breakdown for Task Silo L0 - returns individual component scores.
Returns a map containing: - total_reward: The combined weighted reward (-1.0 to 1.0) - velocity_score: Fitness improvement velocity score (40% weight) - efficiency_score: Training time efficiency score (20% weight) - convergence_score: Premature convergence penalty score (20% weight) - resource_score: Memory/CPU efficiency score (20% weight) - velocity_weight, efficiency_weight, etc.: The weights used
This allows visualization of which factors are contributing to the reward.
Compute cumulative reward for an LC TWEANN trial.
Called at the end of a trial period to evaluate the LC controller's performance. Uses fitness improvement achieved during the trial as the primary metric.
Input metrics map should contain: - start_fitness: Fitness at trial start - end_fitness: Fitness at trial end - evaluations_used: Evaluations consumed during trial - evaluation_budget: Evaluation budget for trial - peak_memory: Peak memory usage during trial (0-1) - avg_cpu: Average CPU usage during trial (0-1) - stagnation_events: Number of stagnation events during trial - convergence_reached: true if fitness target was reached