Capabilities · Recommendations
Product Recommendations
HealthyIn-context predictionEndpoint product-recommendationsProduction v12Challenger v13
Experiment: Product Recommendations v12 vs v13
Split live traffic between versions and compare them on a business metric, with guardrails.
SimulatedRUNNING · DAY 4 OF 14
Results
Primary metric: Revenue per session
| v12 | v13 | |
|---|---|---|
| Traffic | 80% | 20% |
| Requests | 2.5M | 625K |
| CTR | 8.7% | 9.1% |
| Revenue per session | $1.84 | $1.93 |
| P95 latency | 58ms | 61ms |
Simulated results. Directional only — the platform reports confidence intervals once the minimum sample size is reached. Quality and business rows only count predictions whose outcomes have matured (median delay 1.6 days).
Guardrails
✓P95 latency < 100ms
✓Error rate < 0.1%
✓Catalog coverage > 60%
Revenue per session by day
day 4 of 14
v12v13
Remaining days are not yet observed.