Judge replay
Two complete runs of LengoAI, read-only, drawn with the same pages as live goals. No login and nothing to set up.
Run 1: the goal is achieved
closedGoal: reduce failed payments caused by gateway timeouts by 20%. The agents plan and build a retry with back-off behind a flag; Core rolls it out 1% → 10% → 50% → 100% as the evidence allows, proves the improvement after 34 simulated minutes, stores the outcome as memory and closes the goal.
11 state changes · 34 statistics decisions · 8 agent handoffs
Open the replay →Run 2: a guardrail breach, a kill, and the lesson applied
killed, then reopenedThe first build retries without back-off. More payments succeed, but the gateway answers with more 5xx errors: the guardrail's always-valid bound crosses its limit and Core switches the flag off at minute 11, opens an incident and reopens the goal. The new plan cites the stored lesson, the second cycle uses back-off on a fresh flag, and it is proven at minute 45.
20 state changes · 46 statistics decisions · 15 agent handoffs
Open the replay →SRM: lost telemetry blocks proof
rollout 50The change really improves payments, but 8% of the treatment arm's events are lost before they reach Core. The sample-ratio-mismatch check sees the skewed split and the change is never called proven on evidence that can't be trusted.
7 state changes · 150 statistics decisions · 6 agent handoffs
Open the replay →What to look for
- The goal chart: the shaded band is the always-valid interval for the change with the flag on. Once it sits entirely below zero (for a decrease), the improvement is real; the dashed line is the goal.
- The guardrail chart: when the band rises above the red kill line, the change is confidently harmful and Core switches the flag off without waiting for anyone.
- What happened: every flow start, agent handoff (accepted or rejected, with citation validity), flag change and approval, in order.
- Run 2's second cycle: the plan after the kill cites the lesson the first cycle stored in memory.