Lesson Learned: Over-Engineering Trading System

ID: LL-OVERENG-001

Date: December 11, 2025 Severity: CRITICAL Category: System Architecture, Trading Strategy Impact: 0% live win rate, negative Sharpe ratios (-7 to -72)

The Mistake

We built a “Don’t Lose Money” machine instead of a “Make Money” machine.

What Happened

Metric Value Problem
Live Win Rate 0% System doesn’t work
Backtest Sharpe -7 to -72 ALL NEGATIVE
Entry Gates 8-10 sequential 90% of signals blocked
Agent Frameworks 5 None validated with real trades
RL Filters 3 None validated with real trades
Circuit Breakers 6+ Too many stops
Lines of Code 50,000+ Massive complexity

Root Cause Analysis

  1. Fear-Driven Development: Every potential loss scenario was “protected” with another gate
  2. Complexity Creep: Each new feature added more filters instead of proving value
  3. No Validation Before Scaling: Added 5 agent frameworks before validating ANY with real trades
  4. Theoretical Over Practical: Built sophisticated ML models without proving basic edge first

The Cascade of Over-Engineering

Fear of Loss → Add Gate → Fewer Trades → Can't Validate → Add More ML → Still No Edge → Add More Gates...

Result: A system so defensive it never trades, therefore can never make money.

The Fix

Created Minimal Viable Trading System:

Feature Complex System Minimal System
Lines of code ~50,000+ ~400
Entry gates 8-10 1
Exit conditions 7+ 3
Agent frameworks 5 0
RL models 3 0
LLM calls Multiple 0
Time to trade Minutes Seconds

What We Kept (Proven Value)

  • Simple SMA(20/50) momentum crossover
  • Fixed 2% position sizing
  • -3% stop-loss, +5% take-profit
  • -2% daily circuit breaker
  • Max 5 positions

What We Removed (Unvalidated Complexity)

  • LangChain agent framework
  • DeepAgents framework
  • ADK framework
  • DiscoRL (bleeding edge, 0 validated trades)
  • SB3 RL framework
  • LLM sentiment analysis
  • Mental toughness coaching
  • Macro context adjustments
  • 10+ complex risk layers

Prevention Rules

Rule 1: Prove Edge Before Complexity

NEVER add a new component until the current system is profitable.

  • Win rate > 50%
  • Sharpe ratio > 0.5
  • 20+ closed trades
  • Positive total P/L

Rule 2: One Change at a Time

When adding complexity, add ONE thing and measure impact.

  • Add feature
  • Run for 30 days
  • Compare metrics
  • Keep if improves, remove if doesn’t

Rule 3: Validation Gate Checklist

Before adding ANY new feature, answer:

[ ] Does current system trade? (If not, simplify first)
[ ] Does current system make money? (If not, fix first)
[ ] Is this feature validated with real trades? (If not, test first)
[ ] What specific metric will this improve? (Must be measurable)
[ ] What is the rollback plan? (Must be documented)

Rule 4: Complexity Budget

Maximum allowed complexity at each stage:

Stage Max Entry Gates Max Frameworks Max Lines
Proof of Concept 1 0 500
Basic Validation 2 0 1,000
Validated Edge 3 1 5,000
Scaled System 5 2 20,000

Rule 5: Weekly Complexity Audit

Every Friday, run:

1. Count entry gates
2. Count active frameworks
3. Count lines of trading logic
4. Compare to complexity budget
5. If over budget, remove least validated component

Verification Tests

Test 1: Trading Frequency

def test_trades_execute():
    """System must execute at least 1 trade per day on average."""
    trades_per_week = count_trades(last_7_days)
    assert trades_per_week >= 5, f"Only {trades_per_week} trades/week - system over-filtered"

Test 2: Gate Pass Rate

def test_gate_pass_rate():
    """At least 20% of signals should pass all gates."""
    signals_generated = count_signals(last_7_days)
    signals_executed = count_executed(last_7_days)
    pass_rate = signals_executed / signals_generated
    assert pass_rate >= 0.2, f"Only {pass_rate*100:.1f}% pass rate - over-filtered"

Test 3: Component Validation

def test_all_components_validated():
    """Every active component must have validation data."""
    for component in get_active_components():
        assert component.trades_validated >= 20, f"{component.name} has no validation"
        assert component.sharpe > 0, f"{component.name} has negative Sharpe"

Test 4: Complexity Check

def test_complexity_within_budget():
    """System must stay within complexity budget."""
    gates = count_entry_gates()
    frameworks = count_active_frameworks()
    lines = count_trading_logic_lines()

    assert gates <= 5, f"{gates} gates exceeds budget of 5"
    assert frameworks <= 2, f"{frameworks} frameworks exceeds budget of 2"
    assert lines <= 20000, f"{lines} lines exceeds budget of 20k"

Key Quotes

“A system that never trades can never make money.”

“Simple rules > complex ML - until ML proves edge.”

“Trade more, learn faster.”

“Prove edge first, add complexity later.”

  • See: rag_knowledge/lessons_learned/fear_driven_development.md (to be created)
  • See: rag_knowledge/lessons_learned/validation_before_scaling.md (to be created)

Tags

#over-engineering #trading-system #complexity #validation #lessons-learned #critical