Capabilities · Classification
SaaS Churn
HealthyIn-context predictionEndpoint saas-churnProduction v3Challenger v4
SaaS Churn / Deployments
Simulated
BLUE— Production
Healthyv3
- Traffic
- 100%
- AUC
- .890
- P95
- 31ms
- Cost / 1K
- $0.09
- Engine
- Managed A
- Created
- 3 weeks ago
GREEN— Candidate
Readyv4
- Traffic
- 0%
- AUC
- .902
- P95
- 34ms
- Cost / 1K
- $0.11
- Engine
- Managed A · usage events added to context
- Created
- 2 days ago
Live traffic
BLUE v3100%
GREEN v40%
Deploy v4
Choose how the challenger reaches production
Production
v3
Challenger
v4
Customer responses still come from v3. v4 predictions are evaluated silently on 10% of mirrored requests.
No customer impact. Shadow traffic is billed at challenger cost.
Shadow results — 168 hoursRequests evaluated 127K
Simulated| Metric | v3 | v4 |
|---|---|---|
| AUC | .890 | .902 |
| Precision | 62% | 65% |
| Recall | 58% | 61% |
| P95 latency | 31ms | 34ms |
| Cost / 1K | $0.09 | $0.11 |
✓P95 latency 34ms < 100ms
✓Availability 99.998% > 99.9%
✓Precision 65% > 55%
✓Cost / 1K $0.11 < $0.25
v4 satisfies all configured production constraints.
Latency and cost are measured on live shadow traffic. Quality rows come from replaying both versions on predictions with matured outcomes (Jun 1 – Jun 30, 98% coverage) — outcomes for this shadow window arrive in ~90 days.
Version timeline
Today
- 08:00Weekly scoring batch completed — 18.2K accounts
2 days ago
- 14:10v4 created — product usage events added to context
Last week
- 10:02Feature drift detected on seats_active — severity moderate
3 weeks ago
- 17:45v3 promoted to production