Jobs to be done and segments: using AI to code every online mention

Jobs to be done and segments: using AI to code every online mention

Part 5 of 5 in our series on the next generation of R&D.

Every review, listing question and social post about your product can now be coded against jobs and segments, with the quote kept beside each code. Reading the latest 200 and calling it a sample was a labor constraint, and that constraint is mostly gone. The coding frame still decides the quality of everything downstream, and building it is still a job for people who know the category.

The frame: who, in what circumstance, for what job

A job is the progress a buyer is trying to make in a particular situation. Clayton Christensen, Scott Cook and Taddy Hall put the consequence bluntly in "What Customers Want from Your Products": "The job, not the customer, is the fundamental unit of analysis."

For product decisions, a segment is a group of buyers who share a circumstance and a main job. Define each one with three parts: who the buyer is, the situation they are in and the job they most need done. "Frequent flyers packing a carry-on who need lotion that survives the bag" is a segment. "Women 25 to 44" is an audience for media buying, and it won't tell an engineer what to change.

A common mistake is promoting complaint themes to segments: "the leak people", "the price people". A complaint is an outcome under a job. The same leak means something different to a commuter with a gym bag and a parent packing a diaper bag, and the fix that suits one may not suit the other.

Why sampling was the norm

Assume an analyst can read a text, decide which job it concerns, which segment wrote it and whether the product succeeded, at 60 texts an hour. That rate is our assumption for illustration. A product and three competitors with 5,000 texts between them is 5,000 ÷ 60 = 83 hours, more than two working weeks before a second analyst checks the first one's judgment. Then new reviews arrive.

So teams read the most recent few hundred, or the one-star ones, and the segment that writes less often drops out of the analysis.

The source is worth more than that. Artem Timoshenko and John Hauser compared online reviews with conventional interviews in a 2019 Marketing Science paper, working in oral care. They found user-generated content at least as valuable a source of customer needs as the interviews, and found that machine learning made the search more efficient. In their method the machine picked out the informative sentences and professional analysts wrote the needs. Language models have moved that line. The machine can now propose the job, the segment and a verdict for each text. People still own the frame and the audit.

Coding the texts

Start by having the model read a broad sample and propose the jobs buyers describe and the segments that appear. Then put the list in front of people who know the category. Merge duplicates, rename anything written in research jargon, and add the job everyone in the building knows about that buyers rarely write down. Twenty segments is too many. If two groups would get the same product decision, they are one segment.

For each text, record the job it concerns, the segment the writer belongs to if the text reveals it, and a verdict: the product did the job, failed it, or the result was mixed. Keep the quote and the source with every record. Allow "can't tell". A review that says "great product, fast shipping" concerns no job, and forcing it into one adds noise that looks like data.

These four are illustrative, written for the product this series follows, a 90 ml travel pump bottle of body lotion.

MentionSourceSegmentJobVerdict
"Pump got pressed in my carry-on and emptied into my shoes."ReviewFrequent flyers packing a carry-onCarry it in a bag without leaksUnmet
"Does the pump lock? I travel every week."Listing questionFrequent flyers packing a carry-onCarry it in a bag without leaksUnmet, explicit request
"Keep one at my desk, one pump and no greasy keyboard."Social postOffice workers with dry handsMoisturize without greasy handsMet
"Arrived fast, love it."ReviewCan't tellNoneNot judged

Count the share met

For each segment and job, calculate the share of judged texts in which the product did the job. Mention counts measure attention. The share met measures whether the product worked, segment by segment.

Suppose frequent flyers wrote 140 texts that judge "carry it in a bag without leaks" for this product, and 48 say it did the job. That is 48 ÷ 140 = 34% met. For Competitor A, which has a locking pump, 97 of 120 judged texts say it did the job: 97 ÷ 120 = 81%.

A mention count would have put this product ahead, 140 against 120. The share met has it losing the segment by 47 points on the job that segment cares about most. At these sample sizes the sampling margin on each figure is roughly 7 to 8 points, so the gap is not noise. Set a floor all the same. Below a minimum number of judged texts, show the raw counts and no percentage.

The second row of the table carries a detail worth keeping: the buyer asked outright for a lock. That request is the start of a product case, and Part 1 sizes it in finding your next winning product in market evidence.

Audit it before anyone acts

Pull a random set of coded texts for each job and have a person code them blind, then compare. Have a second person code the same set. Two analysts won't agree on every text either, and their agreement rate is the realistic ceiling for the model. Where the model is wrong in a pattern, the definition is usually at fault. Fix it and rerun.

Some weaknesses don't audit away. Models read sarcasm badly and can be confidently wrong about who wrote a text. Many reviews reveal nothing about the buyer's circumstance, so segment coverage is always partial, and the count of texts that couldn't be coded should be published beside the ones that could. Review writers are not a random sample of buyers, and a sampling margin says nothing about that. A verdict is a reading of one person's words, which is weaker evidence than a return or a repeat purchase.

Interviews still have a job. Public text is broad and current, and it misses what buyers assume and never write down. Use the coded mentions to decide whom to interview and what to ask.

How Ontevo codes the texts

Ontevo's Full Diagnostic builds a Product Definition for the product you name and its direct competitors, as part of a 50-point diagnostic delivered in under 24 hours. It reads the reviews, listing questions and social posts collected for each product, proposes the segments in the category and the jobs each one hires the product for, and scores how often each product did the job, by segment and by job. Each cell opens the texts behind it.

The definition is a draft for your team. You can edit, add and re-rank segments and jobs, and thin cells show counts in place of a percentage. The sample report opens with this view for an air purifier, and the page for consumer product teams shows where it leads.

Jobs and segments are the first link in a longer chain. We're building the connections from each need to the features, parts and costs behind it, described in connecting customer needs to design and cost. For reading competitor reviews by hand, our earlier guide to analyzing competitor reviews still applies.

Tell us one product. We'll show you its Full Diagnostic in a 30-minute demo. Book a Demo.

Ontevo Research. Where this post carries figures, they come from Ontevo's own scan corpus or are modeled from scan patterns across the category. No figure is measured from a named customer.

Book a Demo
RELATED PAIN POINTS
AGENTS THAT FIX THIS

See what your competitors are taking. Priced in dollars.

Tell us one product, brand or location. We'll show you its Full Diagnostic in a 30-minute demo. If it's a fit, you get a one-week trial. No charge.