Blog
Read the comments every week, or the score is a trophy
A stable NPS while billing complaints pile up is not stability. How to read, tag, and close the loop on comments without inventing themes.
Monday, 09:07. Dana, who owns CS at a 40-person SaaS, opens NPS results before standup. The headline is +41, same as last Monday, n=86. She does not screenshot it. She filters to detractors from the last seven days. Fourteen comments. Six mention the new invoice line. Three mention CSV export. One Scale customer wrote, at 22:17 on Sunday, "we pause if export stays broken." Slack already delivered that row to #cs-alerts at 22:18.
That is a survey program. The number is an index. The comments are the work.
The weak version, and what it costs in six weeks
The weak week looks busy:
Monday: paste +41 NPS into the exec deck.
Wednesday: dump 400 comments into a sheet and ask someone to group them into themes.
Friday: a slide titled Top themes: Pricing, UX, Other.
Nobody replies to the person who wrote about export.
The week that changes the product looks like this:
Monday 09:00 to 09:20: new comments only, newest first.
Tag as you go: billing-surprise, export-gap, praise.
Same day: a human replies to named detractors who left a way to reach them.
Promote a theme only when it repeats and the slice can carry a number.
The weak version costs you the six weeks between billing-surprise showing up in the feed and the churn report that treats it as a surprise. The score looked stable the whole time. A flat index is not the same as silence in the text.
What a weekly read actually is
Twenty minutes. Not a workshop.
- Sort by newest. Do not start with promoters unless you are hunting a quote for a launch email.
- Read the sentence before the rating. A 9 that says "we will leave if billing stays like this" is not a promoter story.
- Search last week's words this week. If invoice showed up four times last Monday, search invoice again before you declare it handled.
- Keep the score on the same screen. A rising billing-surprise cluster under a flat NPS is the early warning. The older post on why a naked score lies is the rule for the number. This is the rule for the text.
UserVane's results view is built as that sitting: score and distribution, then the verbatim feed, shown as written, with search and filters. The measurement honesty contract is the same one as the score: we do not summarize comments into a claim the respondent did not make.
Tags are a reading aid, not a research product
A tag named billing-surprise on six rows helps Dana find them next Monday. It does not prove billing moved the NPS. Three people using the word export is a cluster you can inspect, not a root-cause certificate.
The weak move is feeding the week's comments to a model and presenting "Customers feel the product is too complex" as if the customers said that sentence. They did not. You invented a spokesperson.
If you tag, tag the words they used: invoice-line, csv-export, too-slow-on-mobile. When the tag no longer matches what you search, the tag is lying. Retire it.
Paid UserVane plans keep those tags on the feed, next to the score, so you are not maintaining a second copy of the truth in a sheet. The practice still works if you export. The point is the sitting, not the software.
Close the loop on a named person, same day
A detractor who left a comment at 22:17 is still in the building on Monday morning, or they are already drafting the cancel. The useful action is a human reply, not a theme meeting.
Closed-loop Slack and webhook alerts exist so #cs-alerts gets the comment when it lands, not when someone remembers the dashboard. If you already work in an agent, UserVane MCP exposes the same feed as list_responses: pull new verbatims, do not invent a summary. The open-source SDKs that collect the optional comment in the first place live at github.com/uservane/uservane-js.
You will not reply to every 7. Reply to people who named a breakable thing and left a way to reach them. Write down that you replied, or the next reader will double-contact on Tuesday.
A comment is one person's words
Free-text is the primary PII sink. People type names, emails, and call me at. Treat the feed like customer data, not like marketing copy. The privacy guide is the operational version of that: export and erase exist because of the comments, not as a checkbox.
That is also the limit, and it belongs here rather than as a closing hedge. Tagging a row does not prove the cause. A weekly read proves you looked. It does not prove the sample represents everyone who stayed silent, and three comments is still three comments. Check the slice on the sample size calculator before you promote a cluster onto a roadmap slide. Plan and role still belong on the respondent, not as extra questions; that is the traits post, and it is how you later read comments from Pro separately from Free without asking again.
Bottom line: put twenty minutes on the calendar. Read new comments next to the score. Tag what they actually wrote. Reply to named breaks the same day. The NPS is an index so you notice movement. The comments are how you know what to do.