How Chili Piper scores 135,000 accounts
by Morgan Cliburn & Melanie Ma
We have about 135,000 accounts we could sell to. A rep can work maybe 40 or 50 in a week. So we needed to answer one question: which ones should they work first?
For us, that comes down to two things:
- How likely is this account to buy?
- What could it be worth if it does?
Those are different questions. So we use two different models, trained on two different sets of data.
Our first account score failed
We didn't start with these two models.
Our first score was a point system. We chose the things that looked like signs of a good account, gave them points, and added everything up.
We trusted that score for two years and 26 SDRs used it to decide where to spend their time.
Then we paid an expensive data science consultant to check if it actually worked.
He compared the scores we'd given accounts with the accounts that went on to buy. If the score worked, companies at the top of the list should have bought more often than companies at the bottom.
They didn't.
The GINI score checks whether a model puts future buyers near the top of the list. Zero means random. Ours was below 7.
The consultant said it was about as useful as a blindfolded monkey throwing darts.
Zero is random. The old score came in below 7. The rebuilt model scored 37.72 and picks a future winner from a random pair about 69% of the time.
I don't think the worst part was that the model failed. MODELS FAIL. The worst part was that we'd used it for two years without checking.
So we started again. This time we saved and tested every prediction.
Model one: how likely is the account to buy?
The first model gives every account a chance of buying. We call that propensity.
It works a lot like credit scoring. We use what we know about the account now to estimate the chance that it will buy later.
We trained it on 9,000 closed opportunities.
The model is only allowed to use information we had before anyone at Chili Piper contacted the company. Once a sales conversation starts, any new information from that conversation is off limits.
Otherwise, the model could cheat. It would be very good at describing deals already in the pipeline and useless for finding the next ones.
To make the prediction, we use gradient-boosted trees. The name is more complicated than the idea.
Think of hundreds of small checklists. One might ask whether the company is mid-market, sells software, and uses Gong. The model compares those combinations with our past deals and gives the account one probability of buying.
This is not another point system. No single fact gets a fixed number of points. The mix is what matters.
What goes into the prediction?
The model sticks to things we can know before sales gets involved: how old the company is, where it is, how many people work there, what industry it's in, how active it is on Glassdoor, whether we've crossed paths before, and whether it's hiring marketing leaders.
None of those facts tells us much alone. The pattern does.
Model two: what could the customer be worth?
A deal that closes and churns quickly doesn't help us much.
So the second model estimates lifetime value, or LTV. We trained it on 4,000 customers.
It uses a Cox Proportional Hazards model. I know, it sounds more like a warning label. The idea is much less dramatic.
You also find this kind of model in medical research, when a study is still running and researchers don't know the final outcome for every patient yet.
Most of our customers haven't churned, so we don't know their full lifetime yet. The model can still estimate the chance that they'll be with us in each of the next five years. We combine that with the expected contract value, giving later years a little less weight.
What helps us estimate lifetime value?
The CRM a company uses is one of the biggest clues. The model also looks at things like revenue, funding, software reviews, sales tools, and how many people work in sales development and customer success.
In our customer history, companies using Salesforce or Gong, and companies with more software reviews, have often stayed longer. That doesn't mean buying Gong will magically make someone a better customer. It is just a pattern in our data.
We check which facts are driving the estimate so we notice if the model starts paying too much attention to something silly.
Then we multiply the two numbers
The final score is simple: chance of buying multiplied by expected lifetime value.
That gives us one dollar figure for every account.
How we check if the model works
We tested it on deals it had never seen before.
We put the accounts into ten groups, from the lowest scores to the highest. Then we compared the model's prediction with what actually happened.
The top 10% closed at 40%. The average was about 18%.
Across all ten groups, the predictions were within about 1.5 percentage points of the real results. At the top, the model predicted 35% and the actual result was 40%.
The accounts it ranked high really did buy more often.
How sales uses the score
We first split accounts by company size because the same chance of buying means something very different for a 50-person company and a 5,000-person company.
Inside each size group, we compare the two numbers again: chance of buying and lifetime value.
That gives us four groups.
We compare likelihood to buy and lifetime value inside each size group. S1 is the only focus group.
- S1: High on both. This is where we put personalized sales and marketing effort.
- S2: More likely to buy, but worth less if it does. We use automated outreach.
- S3: Worth a lot, but less likely to buy. We nurture these accounts without targeted SDR outreach.
- S4: Low on both. We don't spend outbound time here, but of course we'll help if someone comes inbound.
Inside S1, reps sort by Combined Score and start at the top. They don't have to calculate anything. The group and score are already in Salesforce.
For enterprise S1 accounts, a high score earns more than a few extra emails. We coordinate personalized SDR outreach with content, events, and dinners. We can also afford to be patient because winning one of these accounts may pay back for years.
Two real S1 accounts
The model doesn't give us one neat reason like “uses Gong.” It looks at the whole mix. But we can see why each account clears the two S1 cutoffs.
Drata is a security and compliance automation company. The model rates it as more likely to buy than the typical mid-market account. Its sales team is also large enough to make the possible contract valuable, and the model expects it to stay long enough to put lifetime value above the cutoff. High on both means S1.
Motive is a fleet-management company with a very large sales team. The model rates it as more likely to buy than the typical enterprise account. Its lifetime value is high mainly because that sales team makes the possible contract much bigger. Again, high on both means S1.
- Has raised a Series C round
- Hiring 7 sales roles and growing demand gen
- About 9,611 employees
- Has raised venture funding
- Hiring 28 sales roles and growing demand gen
- About 6,381 employees
Both clear the S1 cutoffs for very different reasons — the mix is what matters, not any single signal.
Enterprise gets its own cut because those accounts behave differently. The companies most likely to buy usually aren't the biggest, and contract size matters more to lifetime value there than how long a customer may stay.
The model is always a work in progress
That's normal. Our product keeps changing, and so does the market around it. An account that looks great now may not look great next year.
So we keep comparing the predictions with what actually closes. We ask reps what they see when they work the accounts, and then we adjust the model.
We learned this the hard way. Changing the model isn't the problem. Trusting it for two years without checking is.
This is a working guide too. We'll update it as we learn from real results, rep feedback, and the ways sales and marketing use the score. That includes a proper example of how the score shapes our ad spend once we have one worth sharing.
For the other ways to score accounts, plus tools and vendors, read the B2B account-scoring guide.