Raddon Report

You are here

Measuring Customer Experience: Should You Use an Annual Survey or a Transactional Survey?

August 15, 2019
Share this post: 

It is difficult, if not impossible, to serve customers in any industry without understanding how they view your company, products and service. The customer experience (or member experience for the credit union industry) is paramount for ensuring repeat business, and as we know from our Performance Analytics program, institutions that get repeat business are most likely to be high performers.

Asking your customers directly about their experience remains the best way to get the feedback you need. But what’s the best way to ask questions and measure responses? There’s been a great debate: Should you do a large annual survey of your entire customer base or conduct an immediate survey of those who purchase your product or complete a transaction? Both processes can have tremendous value, and both can have pitfalls. As a former financial institution executive, I’ll share my experience.

Annual Surveys

Conducting surveys once a year gives you time to observe your customers for a long time over many cycles and ask questions that seem most relevant. However, in an annual survey, you potentially miss reactions to events that customers have since forgotten.

Annual surveys give you a complete view of your customer base

A good annual survey is weighted so that the picture created reflects all your customers, from the highly engaged to the not-at-all engaged. That weighting matters because if you survey only customers who consistently use your products or services, you’ll start believing that your entire customer base is like them. As a result, you run the risk of blinding yourself to issues that keep others from being as engaged.

Some important metrics do not change much

Some metrics, like share of wallet, loyalty and service quality, tend not to change much from month to month. Over a year’s time, they might see movement, but monthly changes can often be noise and not signal.

Statistical significance matters

We conduct surveys of a sample group of customers to attempt to understand the opinions and behaviors of a larger population. The bigger the sample, the more confident we are that the answers are equivalent to the ones the entire population would give. But even then, answers from the sample and the population could be different. We call the mathematical gap a confidence interval (or margin of error). Basically, if you were to ask a question of a sample, there is a 95% chance that the answer would be the same as what the entire population would have said, plus or minus the confidence interval.

Many transaction surveys have a low sample size, or “n.” That means that the confidence interval is much larger than on an annual survey with a high sample size. Since many metrics may change only a couple of percentage points from survey to survey, having a five-point confidence interval makes it virtually impossible to determine movement in the numbers.

Benchmarking is important

Let’s say you survey your customers and find that 55 percent are very satisfied and 40 percent are satisfied. Is this finding good? Bad? How can you tell? Yes, if you do a second survey, you’ll have a trend, but you will still be in the dark as to where you are and what your goal should be. A good annual survey will have benchmarks with peer institutions, so you can see where you outperform or underperform. This information can be critical in setting your strategic direction.

Immediate Transaction Surveys

Transactional surveys offer the advantage of providing information that enables you to take action in real time. The responses may be positive or negative, and they may not give you the big picture, but they ensure that you can address issues before customers become upset and leave.

Immediate surveys help you respond right away to criticism

If you are launching a new product, staffing a new branch or implementing a significant change to mobile banking, getting immediate feedback can be instrumental in fixing small problems before they become large problems. A typical transactional survey will have triggers so that negative responses will go directly to your customer care team.

Immediate transaction surveys help you understand employee performance

Memories tend to fade, so for customers’ view of front-line staff, an in-the-moment response is more valuable than an annual survey. The challenge here, of course, is to get enough responses per employee to be statistically significant. Although some results might come in immediately, wait until you receive a large enough number of responses before taking action.

Improve Customer Experience Either Way

Both annual and transaction surveys play roles in helping an organization move forward because they measure different things: long-term customer feelings vs. short-term customer reactions. Understanding that difference will help you improve your customers’ experience.