Client Success Intelligence
| KPI | Coverage | Numerator | Denominator | Target | vs Target |
|---|---|---|---|---|---|
| Loading… | |||||
| CSM Owner ID | Accounts | Health-check coverage (30d) | Onboarded on time |
|---|---|---|---|
| Loading… | |||
| KPI | Coverage | Numerator | Denominator | Target | vs Target |
|---|---|---|---|---|---|
| Loading… | |||||
| Cohort Quarter | Contracts | Actually Churned | Flagged ≥60d | Early-Warning Rate | Predicted Churn | Actual Churn | Variance (pp) |
|---|---|---|---|---|---|---|---|
| Loading… | |||||||
churn_probability into
ml/churn_snapshots.parquet. For every contract whose
end-date has passed, we look back ~60 days and compare what the
model said to the actual renewal outcome.
Targets: early-warning rate ≥75%, forecast variance ±5pp.
| Account Name | Health | Revenue | Risk Level | Revenue at Risk | Top Drivers | Tenure | |
|---|---|---|---|---|---|---|---|
| Loading… | |||||||
| Capability | Where it lives | How it's built |
|---|---|---|
| Composite Health Score (0–100) 8 components, transparent weights |
Health Score tab + at-risk table column | etl/build_account_health_score.py →
indexes/index_account_health_score.parquet.
Blends churn (M2), usage tier, health-check cadence, onboarding execution,
Insider calls, QBR cadence, pipeline health, and tenure.
|
| CS Operations KPIs 6 KPIs vs Angela's targets |
CS Operations tab | etl/build_cs_activity_index.py →
indexes/index_cs_activity_monthly.parquet +
indexes/index_cs_account_summary.parquet.
Sources activities.parquet (Salesforce custom fields:
Activity_Type__c, Health_Check_Type__c) joined to
ml_contracts.parquet for the StartDate anchor.
|
| CSM Leaderboard Per-owner cadence rollup |
CS Operations tab | Same activity index, grouped by OwnerId. |
| At-Risk drill enriched | At-Risk Accounts tab | Adds Health Score, Score Band, Top Drivers
columns by joining index_account_health_score on the
existing churn-risk filter.
|
| Renewal forecast accuracy Snapshots churn predictions nightly for 60-day backtest |
Forecast Accuracy tab | etl/build_renewal_forecast_accuracy.py appends to
ml/churn_snapshots.parquet (rolling 365d) and
publishes per-quarter cohorts to
indexes/index_renewal_forecast_accuracy.parquet.
|
| Capability | Status | What it needs |
|---|---|---|
| ≥60-day early-warning rate | Snapshotting now | The backtest needs ≥60 days of accumulated
ml/churn_snapshots.parquet history. First eligible
cohort lands ~60 days after the first nightly run that included
build_renewal_forecast_accuracy. The Forecast Accuracy
tab shows the warmup ETA.
|
| Per-account email engagement Open / click / QBR coverage / value-driven email coverage |
Awaiting HubSpot ETL run | Three new extractors are wired into master_etl.run_hubspot_etl:
hubspot_entities (Contact + Company + Engagement) and
hubspot_email_events (CampaignEmailEvent rolling 90-day,
chunked watermark). The downstream rollup
etl/build_account_email_engagement.py is also live in
build_all_indexes. All three skip silently when their
parquets are missing — running the master ETL once on the local
network (where the HubSpot Azure SQL firewall allows) is enough
to light up the Engagement tab.
|
| Capability | Source needed | Plan |
|---|---|---|
| Training Satisfaction Score NPS / CSAT |
Alchemer surveys | New extractor etl/alchemer_surveys.py to pull survey
responses from the Alchemer REST API, joined on Email
back to HubSpot Contact → Company → SF account. Adds a 7th
component (component_csat) to the Health Score and a
CSAT trend column to the at-risk table. ETA: 1 sprint once
Alchemer API credentials are issued.
|
| Direct product usage telemetry Navigator + Insider login activity, per-feature usage |
Product event logs (Mixpanel / app DB) | Today's usage_tier is ML-derived from product
history (procurements, attribution). True usage telemetry would
replace it with measured logins / saved-search count / report
downloads per account, sharpening the component_usage
signal. Net-new ETL etl/product_usage_events.py;
blocked on getting an event-stream consumer in place.
|
| Onboarding training completion within 30d | Alchemer + LMS | The Salesforce activity classifier already detects training
tasks (Activity_Type__c LIKE '%Training%'). The
missing piece is completion confirmation — currently
tracked in Alchemer post-training surveys. Same ETL as the
Training Satisfaction work above.
|
| Health Score account-detail drill Click any account → see all 8 components + history |
None — UI work | Component scores are already in
index_account_health_score.parquet. Build a modal or
side panel triggered by clicking the Health column, rendering
the 8 components as a radar chart with each component's
source links (e.g. "cadence: see last 5 health-checks →
/api/account/<id>/activities?type=health_check").
|
| Score history / trend lines | None — needs nightly snapshot accumulation | The build_account_health_score ETL doesn't yet
snapshot its output. Mirror the
build_renewal_forecast_accuracy pattern: append the
per-account score nightly to
indexes/index_account_health_score_history.parquet,
then render a sparkline next to each at-risk row.
|
| CS playbook automation "Account dropped from Healthy → Watch → ping CSM" |
None — needs notifications wiring | Once score history is logged (above), a daily diff job can fire
Slack / email alerts via the existing
core.notifications service. Similar to the existing
pipeline-stall alerts — just keyed on
score_band transitions.
|
The composite score is intentionally a transparent weighted sum, not a learned model — when CS asks "stop counting QBR cadence so heavily" we flip a number, not retrain. To retune:
- Edit
WEIGHTSat the top ofetl/build_account_health_score.py. Must sum to 100. - Re-run
python -m etl.build_account_health_score— finishes in <5 s on the active customer cohort (~1K accounts). - Refresh this page; the bands and at-risk drivers recompute against the new weights on the next API hit.
To add a 9th component (e.g. survey CSAT once Alchemer lands):
implement a _component_csat() expression that returns a
0–100 polars column, register it in the WEIGHTS dict, and
add the label to _DRIVER_LABELS so the top-3-drivers
column picks it up. Cohort-only components return None
for out-of-cohort accounts so the weighted average normalizes
correctly — see _component_insider for the pattern.