Blog
All channels is three samples in one NPS
Monday's product-nps headline is NPS 17 on 47 answers. The legend under it is in-product 16, link 24, email 7.
Monday, 09:30. Chris owns customer experience at Fieldnote, a 19-person field-service product. Survey product-nps, version 2. The results view is on the defaults: source measured by UserVane, channel All channels. The score is NPS 17 on 47 answers. The legend under that score is in-product 16, link 24, email 7.
Those 47 are three collects from the same week. The 16 in-product rows are technicians who scored on the job-complete screen, where the app calls survey("product-nps") after the job is marked done. Thursday, 15:12, job 4821 is one of them. The line that repeats on those rows: the dispatch board loaded before they got back to the truck. The 24 link rows came from a hosted URL the CSM pasted into Wednesday's QBR notes at 11:05. The line that repeats there: we still retype the site notes into the invoice. The 7 email rows are accounts with no login in 60 days, invited Tuesday at 08:10 from Fieldnote's own email tool. Six of the seven scored 6 or below. One of those says the admin who had the login left in August.
09:41. The slide: "NPS 17 (n=47). The dispatch rewrite missed. Reopen the board."
The weak version, and what it costs at 09:30
The read that produced that slide looks like this:
Leave Channel on All channels. One NPS is the program.
n=47 clears the floor of ten, so 17 is the number to quote. The legend is a breakdown for later.
Drop the dispatch comments and the invoice comments into one theme list. Both came from product-nps.
The read that keeps Monday honest looks like this:
The legend is the week. In-product 16, link 24, and email 7 are three samples that happen to share a question.
Set Channel to In-product only before anyone judges the rewrite. Set it to Link only before anyone says the QBR was fine. Leave email on not enough responses yet.
A verbatim labeled link stays with the link score. A verbatim labeled in-product stays with the job-complete score.
The weak version costs you Thursday's board and Wednesday's invoice at the same time. The room reopens a dispatch board whose own channel scored 75, and the retype still has no owner, because that sentence was sitting inside the 17.
All channels recounts every person once
All channels puts every measured rating on version 2 into one count and computes NPS once. Promoters, passives, and detractors from the widget, the hosted link, and the email share a numerator and a denominator. On these rows that count is 21 promoters, 13 passives, and 13 detractors, which is NPS 17. The job-complete screen and the QBR link can print as one mild number.
The segment on that headline stays measured by UserVane. The channel name is added to the segment after you change the filter: channel=in-product, channel=link, or channel=email. Until you do, the split is the legend, and the label on each verbatim, which reads in-product, link, or email.
The widget answers are stored as in-product. The hosted URL is link. An answer that comes in through the open path your email tool already sent is email. The channel chooser is which path to turn on. This post is the Monday after more than one path has already answered the same wording.
Score the filter you mean
In-product only, on these rows, is NPS 75 on n=16: 12 promoters, 4 passives, no detractors. Link only is NPS 8 on n=24: 9 promoters, 8 passives, 7 detractors. Both are still under a hundred, so the results view prints a plus/minus and treats the sample as still stabilizing. Check each slice on the sample size calculator before 75 or 8 becomes the climate. The calculator wants the people who answered that channel, not the pooled 47.
Email only stays suppressed. Seven answers is under the floor of ten, so the headline stays on not enough responses yet for a stable score. Those seven still sit inside the All channels 17. That is how a dormant-account week moves the company number without ever being allowed a headline of its own. The older post on why a naked score lies is that floor, and it applies to each channel, including the one the pool made easy to skip.
Separate scores show that the technicians on the job-complete screen did not score like the people who opened the QBR link. They do not show that the invoice sentence caused the 8, and 16 job-complete answers are not a stable read of every technician. That is the limit. Read the labeled comments in the same sitting, and give the retype an owner because the link rows say it.
Bottom line: when the legend lists more than one channel, the headline is a pool. Score in-product, link, and email as three samples. Leave a channel under ten suppressed. Monday should be a decision about job 4821 or about the invoice retype, not a single 17 that contains both.