One Size Does Not Fit All – Part 2
In my last post I highlighted the importance of having different predictive models for your inbound and outbound channels. But that is just one example of when having a single model can be a detriment to your business.
We’ve seen hundreds of companies deploying a single model when in fact their business is too complex for just one model to truly be predictive. In reality, if you have a single model, you are taking the risk of being too broad or too narrow. Too broad means that the model can apply to multiple situations, but will have minimal predictive power. Too narrow means that the model can only be applied to one situation, and will not identify prospects that are actually a good fit for the company.
Let’s explore some other cases when we recommend building multiple models.
As long as you have enough data to build a separate model for each geographical locations, you should be creating multiple models. Leads and customers are likely to behave differently in different countries; what you learned from one country or region is not always applicable to a different one.
Another point to remember is that the quality and quantity of available business information differs from country to country, and will affect the nature of the model. It’s always better to have consistent data coverage across all your data when building a single model.
Finally a given location may easily dominate the model and the other locations will be badly described by it. Let’s take a specific case. Imagine that your win rate is 1% in Europe; you probably don’t have enough salespeople to follow up on each lead. In the US the win rate is 10% because you have a much bigger sales team. A model will be considered “successful” if it does really well on the 10% leads in the US, regardless of how poorly it may be doing for the 1% in Europe. Such a model is too broad; great for the US, and largely worthless for Europe. All of the good leads from Europe will have been ignored by the model, because they are a much smaller population in the training data than the positive US leads. Furthermore the US leads are also likely to have different characteristics than the European leads.
This graph illustrates the consequences of having a single model for different geographical locations at a successful technology company. The overall model quality looks great – using back testing techniques – because the customers’ database mostly contains US leads and the model ranks those leads correctly.
However, when only looking at European leads, the model performance significantly degrades. In that case a predictive vendor should have the mechanisms in place to detect and recommend a separate model for the customer in Europe.
Sales Teams Alignment
Another situation requiring multiple models is when your sales team is structured around more than one go-to-market strategy. It’s typical for sales organizations to divide sales team by verticals or customer size (e.g. Enterprise and SMB, Financial and Telco). In that case, we have found that it’s best to have a separate model for each “sales division.” This will give you the ability to fine tune each segment by model , define separate MQL thresholds, and avoid the traps of building models that are too broad.
While I know it sounds easier to just build one model and push it live, companies that do this are unlikely to see the return on investment they are hoping for. Predictive analytics is a detail-rich practice, and those details matter! For teams that really want to improve the efficiency of their marketing and sales teams, multiple models are the only way to go.