Capabilities · Classification
SaaS Churn
HealthyIn-context predictionEndpoint saas-churnProduction v3Challenger v4
SaaS Churn
v3· Binary classificationIN PRODUCTION
Quality
- ROC AUC
- .890
- Precision
- 62%
- Recall
- 58%
- False positive rate
- 2.3%
- CalibrationECE 0.018
- Healthy
Evaluation
- In-context examplesauto-selected
- 1,024
- Evaluation examplesheld out, last 6 months
- 10K
- Temporal holdout
- ✓
- Customer-specific trainingpretrained models only
- None
Estimated production
- P50 latency
- 12ms
- P95 latency
- 31ms
- Cost / 1K predictions
- $0.09
Candidates
Pretrained foundation models, scored on your held-out examples and selected automatically under your production constraints
| Candidate | AUC | P95 | Cost / 1K | Status | Note |
|---|---|---|---|---|---|
| Managed A | .890 | 31ms | $0.09 | Production | Best quality within constraints |
| Managed B | .893 | 64ms | $0.33 | Candidate | +.003 AUC at 3.7× cost |
| Managed C | .871 | 9ms | $0.04 | Candidate | Cheapest, needs no examples |