Why keyword tagging fails for customer feedback

The standard way to make sense of a pile of customer feedback is to tag it. Define categories, route each piece into one, count the buckets. It's tidy, explainable, and every support tool ships with it.

It also breaks in three predictable ways, and the third one is fatal.

1. Customers don't use your vocabulary

You created a tag called Payments. Your customers wrote:

• "it won't take my card"
• "the security code keeps getting rejected"
• "tried three times, gave up"
• "says invalid but the details are right"

Not one contains the word "payment". A keyword rule misses all four. A human tagger catches them, which is why tagging feels like it works — right up until the volume makes a human tagger impossible, which is exactly the volume at which you needed the analysis.

You can patch this with synonym lists. Then you maintain synonym lists.

2. Taxonomies decay as the product changes

A category set is a snapshot of what mattered when someone designed it. Ship a redesign, and half your tags describe screens that don't exist. Add a feature, and there's nowhere to put its feedback, so it lands in Other.

Other grows. It always grows. On most mature taxonomies it's the largest bucket, which means the single biggest category in your analysis is the one that means "we don't know".

Fixing this means periodically restructuring the taxonomy — and the moment you do, this quarter's numbers stop being comparable with last quarter's. You've traded the maintenance problem for a continuity problem.

3. You can only find what you already named

This is the one that actually matters.

A tag is a hypothesis. To create the tag "checkout CVV failures", someone had to already suspect that CVV failures were a thing. Which means a tagging system is structurally incapable of telling you about a problem nobody has thought of yet.

But that's the entire job. You don't need a tool to tell you about the problems you already know about — you know about them. You need it to surface the one that's been growing for six weeks that nobody has put a name to.

A taxonomy answers "how much of what I expected did I get?" The useful question is "what's here that I didn't expect?"

Grouping by meaning instead

The alternative is to skip the categories and let the groups emerge from the feedback itself. Each piece of feedback becomes a vector — a numeric representation of what it means rather than which words it used — and pieces that mean similar things cluster together.

All four of those payment complaints land in the same cluster despite sharing almost no vocabulary, because they mean the same thing. And the cluster gets its name after it forms, from what's actually in it, rather than before from what someone guessed.

Two consequences follow. You stop maintaining a taxonomy, because there isn't one. And problems you never anticipated can surface on their own, because nothing had to be named in advance.

Where this approach is weaker

It would be dishonest to present clustering as free. It has real tradeoffs:

• Cluster boundaries involve a threshold. Set it loose and distinct problems merge into one vague blob; set it tight and one problem splinters into five near-duplicates.
• Clusters are less predictable than tags. If you need this month's categories to exactly match last month's for a board report, fixed buckets are genuinely easier.
• It needs enough volume. Twenty pieces of feedback don't cluster into anything meaningful — you're better off just reading them.
• A cluster's name is generated, so it can be a slightly odd summary of its contents, especially for small clusters.

The practical answer to most of these is that clustering shouldn't be a black box. You should be able to open a group, see the actual quotes inside it, and correct it when it's wrong — and a system worth using should remember that correction rather than making you make it again.

The test

Ask of whatever you're using now: could it have told me about a problem I hadn't already thought of?

If the answer is no — if it can only count the buckets someone defined in advance — then it isn't finding your problems. It's confirming your assumptions, at volume, with a number attached.