Share on facebook
Share on twitter
Share on linkedin
predictive analytics CX

Stop Asking and Start Watching: How Predictive Analytics Can Unlock Insights and Growth

Three percent. 

That’s the proportion of outpatient encounters captured by one patient experience survey at a hospital in the United States. The vast majority—some 96.5%—went unheard and unrecorded. If the three percent were perfectly representative of the wider audience, the low sample size wouldn’t have mattered. Unfortunately, they were not.

“The demographics of the responding population differed significantly from the population as a whole with regards to age, gender, payor status (Medicare and Medicaid), marital status, race, and native language,” wrote Jocelyn Compton, Natalie Glass, and Timothy Fowler, three medical professions who investigated the hospital’s satisfaction surveying methodology.

With such variation between surveyed and non-surveyed patients, the hospital’s patient experience metrics were dogged by biases, specifically selection bias (when the participants of a study systematically differ from the population of interest) and nonresponse bias (when non-responders from a sample differ in a meaningful way to responders).

Survey-based experience and satisfaction surveys are not unique to healthcare, though. According to research from McKinsey, more than 90% of customer experience leaders use a survey-based metric as their primary CX yardstick. However, just 15% say they are “fully satisfied with how their company was measuring CX.” Worse, just 6% expressed “confidence that their measurement system enables both strategic and tactical decision making.”

Why customer experience leaders stick with survey-based metrics is a complex question, but the answer is depressingly simple: Their options are severely limited. Many lack the technical chops to deploy complex solutions. Others simply don’t know where to start. Without a means to proactively track individual people and assess their individual satisfaction at scale, they put the impetus on customers, relying on them to raise their hands and offer insights.

But that is all about to change—and the story bears a striking resemblance to the development of weather forecasting.

From forecasting weather to predicting churn

The world’s first weather forecast arrived six weeks late. It was the early-1990s and English meteorologist Lewis Fry Richardson was pioneering a new way of predicting the weather—numerical weather prediction (NWP). Richardson’s approach used mathematical models of the atmosphere and oceans to predict future weather conditions. The problem was that the calculations were fiendishly complex. In practice, it took his small team six weeks to crunch the numbers. Although he could theoretically have completed the calculations in time to be a real prediction, Richardson estimated that he’d need approximately 60,000 human mathematicians.

Today, we still use the same NWP approach, but with one key difference: technology.

Instead of relying on human observers to record current weather conditions and human mathematicians to solve the equations, we have bus-sized supercomputers capable of performing 14,000 trillion arithmetic operations per second. These enormous machines take minutes, not weeks, to crunch enormous data sets.

Why is weather forecasting important?

The exact same technological narrative is playing out in customer satisfaction.

If you manually follow a customer around all day, reading their social media feedback, listening to their interactions with your support and service agents, and studying their product usage, you can accurately assess their satisfaction without them completing a survey. It’s what Gartner calls ‘contextualizing a large volume of operational data’. For example, if a customer leaves consistently positive feedback and sounds appreciative on the phone to customer service, you can be confident that they’re happy. Conversely, if they spend all day shouting at a customer support agent and all evening trashing your company online, it’s a safe bet that they’re dissatisfied. 

We’ve known manual monitoring works for years. Just like Richardson’s early weather models, it’s been impractical to implement. Employing people to monitor individual customers was prohibitively expensive and borderline ridiculous. But just as technological advances solved weather forecasting, new advances are revolutionizing predictive customer satisfaction, too. There are two key advances to highlight:

  • Automatic Speech Recognition (ASR): Using software to process human speech into a written format.
  • Natural Language Processing (NLP): Using machines to understand and respond to human speech.

These two technologies—which have both developed at lightspeed over the last five to 10 years—allow you to turn every customer interaction into technology-usable data. For example, you can transcribe a prospect’s demo. You can record a customer’s support calls. You can save their interactions on social media. You can build a complete picture of their experience with your company… and then mine it for insights.

There are dozens of ways predictive analytics is being implemented to customer experience. Some of the most impactful applications are:

Predictive churn: Churn is a backward-looking metric, which works by counting those who have already canceled their contract or declined their renewal. Predictive analytics empowers organizations to get in front of retention problems, locating at-risk customers before they’ve made their decision to leave. Predictive models work by analyzing customer behaviors (seat saturation, activity, support requests, and so on) and sentiment (in-the-moment, trend, and account) for substitution trends – indications that someone is likely to churn. Once you know which customers are at risk, you can proactively reach out and work to improve their experience.

Pre-emptive service: Customer needs rarely remain static throughout their life. As consumers experience significant life events (marriage, career change, job loss) and businesses undergo maturation (startup, scaleup, enterprise), they require different things from their vendors. For example, a young, single twenty-something won’t need a health insurance plan that covers dependents. When they’re married with children, they will. Predictive analytics enables businesses to look beyond immediate customer needs, forecasting what they will need in the short, medium, and long term.

Real-time engagement: Today’s businesses have access to such immense cloud computing power that they can transcribe, process, and analyze conversations in real-time. Now, you can highlight any negative interactions during the conversation, escalating them to supervisors or human agents.

The appeal of predictive analytics is clear: It provides a more comprehensive picture of your audience, including their risks and opportunities. It allows customer experience teams to be proactive rather than reactive. It empowers teams to get in front of problems, rather than waiting for them to land and dealing with the fallout. But integrating predictive analytics into an existing customer experience function, which likely relied on survey-based metrics, isn’t always easy. 

Guiding principles for predictive success

Too much data, not enough insight. That’s the challenge facing most data-driven disciplines right now. With the proliferation of cloud technology, the rise of third-party data services, and the explosion in data warehousing services, it’s never been easier to generate and collect customer data—behavioral, demographic, firmographic, interactional, transactional, attitudinal, and so on.

Without a clear battle plan and comprehensive data assets, CX operations teams cannot build useful predictive models like decision trees, regressions, and neural networks. Without those models, they lack the means to reliably predict a customer’s current and future behavior. When considering implementing a predictive analytics function, Suraj Soundararajan, Lead Data Scientist at Freshworks, recommends they discuss, set, and agree on guiding principles for analysis. To provide foundational structure and direction, he offers three universal principles:

Prepare for scalable operations. Before embarking on a predictive analytics transformation, start by auditing your current analytics stack. Most predictive strategies rely on multiple data streams—marketing, sales, customer success, customer service, social media, and so on. How are you consolidating those data sets? Is your data lake equipped to handle huge data volumes? Can your data warehouse support the sort of analysis you’ll need to perform? Most importantly, can your technology stack scale these functions? (Data operations usually grow larger and more complex over time.) If your current analytics stack isn’t fit-for-purpose or is incapable of scaling, does it make sense to overhaul your technology?

Create data guardrails. Like any data-driven tactic, predictive analytics is only as good as its input data. If its input data is faulty, the output will also be faulty. “Teams need to be proactive, designing systems that inform you when something is wrong with input data,” says Soundararajan. One example guardrail is to use a baseline and deviation. First, record every important metric daily. Second, compare today’s output to yesterday’s, last week’s, and last year’s entry. Finally, where anomalies exceed pre-set thresholds, trigger a manual review and alert stakeholders that the data may be temporarily unreliable.

Suraj Soundararajan, Lead Data Scientist at Freshworks

Build a data catalog. Most predictive analytics teams start small. With just a handful of employees, it’s easy for information and knowledge to become siloed or lost. Soundararajan advises companies to build a data catalog—immediately. “A data catalog gives employees access to information without having to query it,” he explains. It democratizes knowledge gained through an admittedly complex process.

Data knows people better than they know themselves

Customer experience teams have relied on survey-based metrics for decades. But we’ve known that they aren’t perfect for much of that time. Responses follow the inverse bell curve, peaking at both extremes of the axis and dipping in the middle. It means our CSAT metrics and NPS scores are biased by those on the extremes: the delighted and angry. While data scientists have designed increasingly complex models to account for an unheard majority, they’re always treating the symptoms, not the root cause.

Predictive analytics offers a remedy to the foundational problem.

By harnessing new technology and increased computing power, you can hear every voice, not just the loudest. You can tap into the unfiltered voice of the customer, granting you unparalleled insights into your customer base. Because, as Charles Duhigg wrote in ‘The Power of Habit’:  “Someday soon, say predictive analytics experts, it will be possible for companies to know our tastes and predict our habits better than we know ourselves.”

Leave a Reply

Your email address will not be published.

Ask an expert

Ask an expert Have a question related to this story? Or a thought you’d like an expert to weigh in on? Write to us.