SaaS companies are constantly innovating, and sometimes we deliver exactly what our customers want. Other times? We fall way short. Try as we might to deliver helpful features based on customer feedback, it’s really impossible to tell how a new feature will do in the wild.
The amazing thing about being a SaaS company is that you typically don’t need to spend hundreds of thousands of dollars on market research before introducing a new feature. By contrast, car or laptop manufacturers have to be damn sure they’re releasing a feature that people will like before they spend millions to produce it. In the SaaS world, however, you typically just have to have make an educated guess, build the new feature, release it (to either a small subset of users or to your entire user base), and evaluate the results.
The following tools will help you figure out whether a new feature needs to be scrapped, tweaked, or expanded and promoted immediately.
Product Satisfaction (PSAT) surveys: Product Satisfaction (PSAT) is a metric that tells you how people feel about your product as a whole or about a particular feature you want to test. For example, if Salesforce wanted to evaluate a new task management feature, they could embed an in-app survey using a survey platform like Wootric or Hotjar asking, “How helpful do you find this feature?” Users then rate the feature on a scale from “Very Helpful” (the highest number) to “Not Helpful at All” (the lowest).
PSAT surveys are often presented on a 5-point or 7-point scale, but you can also use a binary “thumbs up” or “thumbs down.” Just be sure to include a followup question asking customers why they gave you the score they did. You’ll start to see patterns in the qualitative data over time, and you can use that data to improve the feature (and your product as a whole).
Customer Effort Score (CES) surveys: Another important thing to measure is how much people struggle to use a new feature. We can all get miopic about our own products since we know them so well. CES surveys tell us whether a product is as intuitive as you think it is. And if it’s not intuitive at all? The survey will tell you how to improve it.
SaaS companies use all kinds of formats to ask CES questions, but they often employ a 5- or 7-point scale, asking customers how easy it was to use the product. Just like with PSAT, you’ll want to follow up with an open-end question asking why they gave you the score they did.
Making a feature easier to use directly contributes to Customer Lifetime Value (CLV) for obvious reasons, but did you know that some experts consider customer effort the most important thing you can track? According to the Harvard Business Review, CES is a better predictor of long term success than Customer Satisfaction. This is probably due to the fact that negative experiences impact us harder than positive ones.
Do not stop innovating and developing new features by any means, but be sure integrate the qualitative data from your CES surveys to improve your products, your messaging, and your Customer Success/Support initiatives.
Usage: This one is pretty straightforward, but we have to mention it. You can hardly consider a new feature successful if the target audience doesn’t adopt it.
Of course, that doesn’t mean the new feature needs to gain widespread adoption across your entire customer base. Often new features are highly specialized to appeal to a specific customer segment. Just be sure to conduct a cohort analysis to study that particular segment and see whether they use it as much as you’d hoped.
Session recordings: One great way to get a sense of how customers use a new feature is to anonymously observe them using it—the way you would in a User Experience (UX) lab. If your app is web-based, you can use Sessions Recordings from a platform like Hotjar that allows you to play back individual user sessions and see how they behave. You can also anonymously tie those sessions to feedback (like PSAT), so if someone notes a point of friction within a feature, you can see exactly how that user behaved.
Obviously, a session recording isn’t a scientific study. Each one is an anecdote, and “anecdote” isn’t the singular of data. That said, watching enough session recordings will give you the ability to empathize with the customer and spark insights that may have never occurred to you.
Sentiment analysis: Sentiment analysis software, like Wootric, uses machine learning to sort through things like customer reviews and give you a sense of how people feel about your company and your product. It scans for positive and negative words so you can get actionable data from an otherwise unwieldy collection of qualitative feedback. And if you have enough feedback on a given feature, you can derive some real insights about how your users feel about it.
A long-term strategy for feature development
Measuring features immediately following their release is essential to agile product development, but let’s not forget about the data you can generate to guide your long term strategy. To this end, consider conducting cohort analyses, isolating the segment that uses each feature and seeing how use correlates to Customer Lifetime Value (CLV), churn, and overall brand satisfaction metrics like Net Promoter Score (NPS).
In the end, building a product that people love is a big picture strategy. It will involve some false starts here and there, but all knowledge is good knowledge. Even if a new feature bombs, you’ll learn something and you can come back stronger with the next one. This is the micro-application of that Silicon Valley maxim that commands us to “fail fast, fail forward,” and if you work with it, you’ll produce some exceptional SaaS products.