Use cases · Dating
Boost-pricing personalisation.
Lower the boost price, conversion climbs 1 point. Heavy users cost you 42¢ each.
Boost-style add-on purchases in dating apps are price-elastic in different directions for different cohorts. A fixed price treats every user the same. A t-test on the global revenue-per-user metric obscures the cohort-level wins and losses entirely. Per-engagement-decile CATE, keyed on engagement, days-since-last-match, and current online-cohort density, is the canonical bandit problem.
Worked audit
Velvet · Series-B dating app · ~$26M ARR · 318,940 boost-eligible users
VELVET-2026Q2-AUDIT-005
Projected impact
+$1.1M / yr boost revenue
1 · What the team reported
Boost price test: $7.99 (control) vs $4.99 (variant). Boost-conversion rate 6.8% → 7.8%. Revenue per eligible user $0.54 → $0.39.
Team called it "lower price wins on conversion" and shipped $4.99 globally. Boost revenue fell against forecast; the gap was attributed to "low elasticity."
2 · What our re-analysis found
Doubly-robust re-evaluation by engagement decile shows the $4.99 winner is correct for light-engagement users (deciles 1–3), both conversion and revenue improve. It's wrong everywhere else.
Heavy-engagement users (deciles 8–10) at $4.99 cost $0.42 per user of boost revenue versus what they would have paid at $7.99. The conversion lift is illusory at this segment, they would have converted anyway. Across the ~87,000 users in those deciles over 90 days, that's roughly $330K of foregone revenue in one quarter.
Mid-engagement and recently-matched cohorts also lose revenue at $4.99, though by smaller per-user amounts.
3 · Why the t-test missed it
The aggregate revenue-per-user metric is closer to a population mean and was negative (−$0.15/user), but the team prioritised the conversion-rate winner over the revenue signal, a common pattern when the conversion metric is the team's primary OKR.
Even if the team had prioritised revenue, the global metric still hides which cohorts lose how much. Per-decile CATE is the only readout that says 'raise the price for these users; keep it low for those.'
4 · What we'd recommend
Bandit-personalise boost pricing on (engagement decile × days-since-last-match × current online-cohort density). The bandit handles the cross-vertical drift (cohort density shifts daily) that no static segmentation captures.
Estimated +14% boost revenue · +$1.1M / yr with the same boost feature, no copy change.
Doubly-robust readout · $4.99 vs $7.99 · conversion lift + revenue/user
| Cohort | DR conv. lift | DR rev / user | 95% CI (rev) | ESS | Verdict |
|---|---|---|---|---|---|
| All boost-eligible | +13.8% rel. | −$0.15 / user | [−0.21, −0.09] | 0.59 | conv. up, rev. down |
| Light engagement (1–3) | +24.1% rel. | +$0.07 / user | [+0.02, +0.12] | 0.46 | positive on both |
| Mid engagement (4–7) | +12.0% rel. | −$0.18 / user | [−0.25, −0.11] | 0.52 | conv. up, rev. down |
| Heavy engagement (8–10) | +3.2% rel. | −$0.42 / user | [−0.51, −0.33] | 0.48 | money left on the table at $4.99 |
| Recently-matched (last 7d) | +6.1% rel. | −$0.21 / user | [−0.29, −0.13] | 0.41 | conv. up, rev. down |
Read the full audit, then audit your own test.
Same shape we'll send back on your last A/B test — free, in three business days.