Consumer research: what it measures, and what it misses

Consumer research is the systematic study of how people choose, buy and use things. It is very good at counting. It is unreliable at causes, for a reason that has nothing to do with how well it is run.

The two halves

Quantitative research counts. A representative sample, a fixed questionnaire, and numbers you can project onto a population: how many people buy this, how often, at what price, and how that has moved since last year. Done properly it is the most reliable instrument in the field.

Qualitative research investigates. Depth interviews, groups, ethnography, watching somebody shop instead of asking them about shopping. It produces language, texture and hypotheses, on a base too small to project.

The standard sequence is qualitative first to find out what to ask, then quantitative to find out how many. It works. What it does not fix is the problem underneath both.

Where it breaks

Almost all consumer research is self-report: a person is asked about themselves and the answer is recorded. Self-report fails in three ways, and they are usually addressed in the wrong order.

1. People do not know. The reason for a purchase is assembled after the purchase, out of reasons that are respectable to hold. The respondent is not hiding anything. They believe the answer they gave. This is the largest source of error and the one least often designed for.
2. People edit. On anything with a socially correct answer — money, health, drinking, sustainability, how much television — the answer moves towards the correct one. This is predictable, and the direction is predictable too, which means it can be planned for rather than discovered afterwards.
3. Questions leak. A question can hand the respondent the answer in its own wording. This is the failure everybody checks for, and the smallest of the three.

What market research companies do

A market research company designs the questions, recruits a sample that represents the population, fields the study, weights and analyses the results, and reports them. Panel quality and weighting get the attention because they are auditable. Question design gets less, because a badly framed question still returns tidy data — it just returns tidy data about nothing.

Market research is the broader field: sizing, pricing, distribution, competitor tracking, brand health. Consumer research is the part of it pointed at the person doing the buying. In ordinary use the two terms are interchangeable and nobody minds.

Designs that get past self-report

None of these are exotic. They are standard techniques that are simply used less than they should be.

Ask about other people

A respondent asked what they fear gives you a managed answer. A respondent asked what the people around them fear gives you an unmanaged one, because nothing about it reflects on them. Social circle questioning has been used in public health and election research for decades, and transfers directly to commercial work.

Ask in three tenses

The same question about the past, the present and the expected future. Each answer is unremarkable on its own. The shape of the line across all three exposes beliefs the respondent would not state — in particular, whether they think a thing is getting better or worse, which drives behaviour far more than their stated position on it.

Ask about behaviour and let motive be inferred

What did you buy, what did you put back, what did you nearly buy and why did you not. Recall of action is far more reliable than introspection about cause.

Know in advance which questions will be distorted

Sensitivity is a property of the question, and it can be estimated before fielding. Knowing that an item will be over-reported by a given kind of respondent is worth more than discovering the same thing in the crosstabs.

How to read a research proposal

• Ask what the questionnaire actually says, not what the topics are. The wording is the study.
• Ask which items are expected to be distorted and what has been done about it. "Nothing" is a common and honest answer; "none of them" is not.
• Ask what result would falsify the hypothesis. A study that cannot come back negative is a presentation with a sample attached.
• Ask for the base size of every subgroup in the report, not the total. Most overclaiming happens at the subgroup level.

Questions people ask

What is consumer research?

The systematic study of how people choose, buy and use products, covering both quantitative work that measures how many and qualitative work that investigates why.

How much does consumer research cost?

A nationally representative quantitative study is usually priced on sample size, questionnaire length and how many markets are involved. Qualitative is priced on sessions. The variable that moves the cost of being wrong, rather than the cost of the study, is question design — and it is the cheapest line on the invoice.

How big should the sample be?

Large enough that the subgroups you intend to act on are readable, which for a national study usually means around 1,000. More sample does not fix a biased question. It measures the same distortion more precisely.

Is qualitative or quantitative better?

Wrong question. Qualitative tells you what to ask; quantitative tells you how many. Run without the first, the second is confident about the wrong variables.

Can AI replace consumer research?

It can draft questionnaires, code open ends and summarise findings, and it does all three quickly. It cannot supply the fact of what people in a specific market actually did last week. A model asked to simulate respondents returns the consensus view of a population, which is exactly the thing research exists to check.