Memory

Outcome M-0007 for goal G-0003: Reduce failed payments caused by gateway timeouts by 20%. Change: Behind the flag, retry a gateway timeout at most once, after an exponential back-off with jitter of at least 200 ms, and send the same idempotency key on both calls. Also enforce a total time budget per payment: compute the remaining budget before the retry, shorten the per-attempt timeout so both attempts fit, and skip the retry if it cannot finish inside the budget. Keep the budget below the 1756 ms p95 threshold that killed M-0006. Do not use immediate retries or more than one retry (M-0002), (

In outcome M-0007 (Behind the flag, retry a gateway timeout at most once, after an exponential back-off with jitter of at least 200 ms, and send the same idempotency key on both calls. Also enforce a total time budget per payment: compute the remaining budget before the retry, shorten the per-attempt timeout so both attempts fit, and skip the retry if it cannot finish inside the budget. Keep the budget below the 1756 ms p95 threshold that killed M-0006. Do not use immediate retries or more than one retry (M-0002),), the goal metric failure_rate changed by -25.9% relative (control 0.10119675403

In outcome M-0007 (Behind the flag, retry a gateway timeout at most once, after an exponential back-off with jitter of at least 200 ms, and send the same idempotency key on both calls. Also enforce a total time budget per payment: compute the remaining budget before the retry, shorten the per-attempt timeout so both attempts fit, and skip the retry if it cannot finish inside the budget. Keep the budget below the 1756 ms p95 threshold that killed M-0006. Do not use immediate retries or more than one retry (M-0002),), guardrail http_5xx_rate was ok: control None, treatment None.

In outcome M-0007 (Behind the flag, retry a gateway timeout at most once, after an exponential back-off with jitter of at least 200 ms, and send the same idempotency key on both calls. Also enforce a total time budget per payment: compute the remaining budget before the retry, shorten the per-attempt timeout so both attempts fit, and skip the retry if it cannot finish inside the budget. Keep the budget below the 1756 ms p95 threshold that killed M-0006. Do not use immediate retries or more than one retry (M-0002),), guardrail p95_latency_ms was ok: control None, treatment None.

Outcome M-0006 for goal G-0003: Reduce failed payments caused by gateway timeouts by 20%. Change: Behind the flag, retry a payment call that timed out at the gateway at most once, after an exponential back-off with jitter, and send the same idempotency key on the original call and the retry so an in-flight attempt cannot be duplicated. Do not use immediate retries or more than one retry, because M-0002 showed that three immediate retries without an idempotency key raised 5xx tenfold, and recovered payments came from the first retry only. Keep the old path unchanged when the flag is off, add (

In outcome M-0006 (Behind the flag, retry a payment call that timed out at the gateway at most once, after an exponential back-off with jitter, and send the same idempotency key on the original call and the retry so an in-flight attempt cannot be duplicated. Do not use immediate retries or more than one retry, because M-0002 showed that three immediate retries without an idempotency key raised 5xx tenfold, and recovered payments came from the first retry only. Keep the old path unchanged when the flag is off, add ), the goal metric failure_rate changed by n/a relative (control None, treatment

In outcome M-0006 (Behind the flag, retry a payment call that timed out at the gateway at most once, after an exponential back-off with jitter, and send the same idempotency key on the original call and the retry so an in-flight attempt cannot be duplicated. Do not use immediate retries or more than one retry, because M-0002 showed that three immediate retries without an idempotency key raised 5xx tenfold, and recovered payments came from the first retry only. Keep the old path unchanged when the flag is off, add ), guardrail http_5xx_rate was ok: control None, treatment None.

In outcome M-0006 (Behind the flag, retry a payment call that timed out at the gateway at most once, after an exponential back-off with jitter, and send the same idempotency key on the original call and the retry so an in-flight attempt cannot be duplicated. Do not use immediate retries or more than one retry, because M-0002 showed that three immediate retries without an idempotency key raised 5xx tenfold, and recovered payments came from the first retry only. Keep the old path unchanged when the flag is off, add ), guardrail p95_latency_ms was breach: control None, treatment None.

Outcome M-0005 for goal G-0005: Lower gateway timeout from 3 s to 1.5 s to fail fast. Change: Client timeout lowered to 1,500 ms (flag goal_0005_fast_timeout). Decision: inconclusive. Primary metric failure_rate relative effect +4.0% (interval -3.0% to +11.0%). Lessons: Sample ratio mismatch: 54.6% of users landed in the flag-on arm at the 50% step because one client stopped refreshing flag state. Results were discarded. Clients must refresh flags every 5 seconds and evaluate with the user ID; see RB-004.

In outcome M-0005 (Client timeout lowered to 1,500 ms), the goal metric failure_rate changed by +4.0% relative (control 0.1, treatment 0.104).

In outcome M-0005 (Client timeout lowered to 1,500 ms), guardrail http_5xx_rate was ok: control 0.003, treatment 0.003.

In outcome M-0005 (Client timeout lowered to 1,500 ms), guardrail p95_latency_ms was ok: control 3000, treatment 1500. p95 improved because slow attempts now fail at 1.5 s

Outcome M-0004 for goal G-0004: Reduce support tickets about fees by showing the fee before confirming. Change: Show the transaction fee on the confirm screen (flag goal_0004_fee_display). Decision: inconclusive. Primary metric failure_rate relative effect -2.0% (interval -9.0% to +5.0%). Lessons: Showing the fee did not move failed payments; the effect, if any, is smaller than 5%. Ticket volume is the better metric for fee confusion; failed payments was the wrong primary metric.

In outcome M-0004 (Show the transaction fee on the confirm screen), the goal metric failure_rate changed by -2.0% relative (control 0.098, treatment 0.096).

In outcome M-0004 (Show the transaction fee on the confirm screen), guardrail http_5xx_rate was ok: control 0.003, treatment 0.003.

Outcome M-0003 for goal G-0003: Cut balance-screen latency (p95) by 25%. Change: Cache balance lookups for 5 seconds per user (flag goal_0003_balance_cache). Decision: proven. Primary metric p95_latency_ms relative effect -31.0% (interval -36.0% to -26.0%). Lessons: A 5-second cache on balances cut p95 latency by about 31% with no change in errors. Cache invalidation on payment was required: without it, balances looked stale right after paying.

In outcome M-0003 (Cache balance lookups for 5 seconds per user), guardrail failure_rate was ok: control 0.1, treatment 0.1.

In outcome M-0003 (Cache balance lookups for 5 seconds per user), guardrail http_5xx_rate was ok: control 0.003, treatment 0.003.

In outcome M-0003 (Cache balance lookups for 5 seconds per user), the goal metric p95_latency_ms changed by -31.0% relative (control 380, treatment 262).

Outcome M-0002 for goal G-0002: Reduce failed payments caused by gateway timeouts by 20%. Change: Retry gateway timeouts immediately, up to three attempts, without an idempotency key (flag goal_0002_timeout_retry). Decision: killed. Primary metric failure_rate relative effect -17.8% (interval -31.0% to -4.0%). Kill cause: Retry storm on gateway timeouts raised 5xx from 0.3% to 3.1%: immediate retries without back-off or idempotency key collided with in-flight attempts (duplicate_in_flight).. Lessons: Retries on payment calls need exponential back-off with jitter and an idempotency key; immedia

In outcome M-0002 (Retry gateway timeouts immediately, up to three attempts, without an idempotency key), the goal metric failure_rate changed by -17.8% relative (control 0.101, treatment 0.083).

In outcome M-0002 (Retry gateway timeouts immediately, up to three attempts, without an idempotency key), guardrail http_5xx_rate was breach: control 0.003, treatment 0.031. duplicate_in_flight errors from retries colliding with attempts still in progress

In outcome M-0002 (Retry gateway timeouts immediately, up to three attempts, without an idempotency key), guardrail p95_latency_ms was ok: control 1500, treatment 1500.

Outcome M-0001 for goal G-0001: Reduce checkout abandonment on the payment form by 10%. Change: Shorter payment form: amount and confirm on one screen (flag goal_0001_short_form). Decision: proven. Primary metric failure_rate relative effect -12.1% (interval -19.0% to -5.0%). Lessons: Removing a step from the payment form reduced abandonment by about 12% without touching error rates. Proof arrived at the 50% step after about 48,000 payments; smaller effects need longer dwell.

In outcome M-0001 (Shorter payment form: amount and confirm on one screen), the goal metric failure_rate changed by -12.1% relative (control 0.124, treatment 0.109).

In outcome M-0001 (Shorter payment form: amount and confirm on one screen), guardrail http_5xx_rate was ok: control 0.003, treatment 0.003.

In outcome M-0001 (Shorter payment form: amount and confirm on one screen), guardrail p95_latency_ms was ok: control 410, treatment 402.