Experiments
What's working
ranks on: reply (reply) · conclusive winners onlyA read-only roll-up of what has actually separated. A variant is only a winner once the engine's deterministic math (frequency-of-success, Wilson interval — never an LLM) crowns a leader: every arm well-sampled AND the leader's pessimistic read clears the runner-up. Running and inconclusive experiments are counted but never shown as winners.
Outreach · reply experiments: 1 experiment · 0 conclusive · 0 still running
| Asset type | Conclusive / experiments | Avg winning lift | Trials |
|---|---|---|---|
| subject_line | 0 / 1 | — | 0 |
Web · conversion experiments: 1 experiment · 0 conclusive · 0 still running
| Asset type | Conclusive / experiments | Avg winning lift | Impressions |
|---|---|---|---|
| subject_line | 0 / 1 | — | 0 |
No attributed intervention outcomes yet.
Correlational, not causal · read-only: correlational and confounded (field volatility, season, list quality); informs the human, never auto-executed
New experiment — generate variants (the model PROPOSES; calc DISPOSES the winner)
Generates a batch of variants for an asset and records them as arms (generation 0). This NEVER sends anything — a human activates & sends. The leader is then picked by deterministic math (frequency-of-success with a Wilson interval), never by an LLM, and only once there's enough data to separate the arms.
A slot only applies to a web asset type (e.g. landing_page_section, gbp_post) — it names where the winning copy renders, like home.hero_headline. The engine stores it NULL for outreach.
subject_line · exp:68245c4d38
ranks on: reply (reply) · status: draftstaging button test
Outreach experiment — ranked on the reply rate.
| Arm | Gen | Rate | 95% CI (Wilson) | Successes / trials | |
|---|---|---|---|---|---|
var:subject_line:g0:b8dd2e1c00 [STUB subject_line variant one — Anthropic not live; fresh angle. Set ANTHROPIC_API_KEY for the live draft.] | 0 · seed | 0.0% | 0.0% – 100.0% | 0 / 0 |