Research quality standards: fit the method to the question

Research quality is a set of standards, not a department. Good research starts with planning that challenges assumptions, picks the method for the question rather than the budget, recruits the right participants in sufficient numbers, moderates neutrally, asks questions that neither lead nor demand people predict their own behavior, and invests real time in synthesis. One quote is not an insight. There are two families of method: generative research finds the problem, through observation, interviews, and diary studies; evaluative research tests the answer, through tree testing and usability testing. The order matters: generative first, then define the problem, then build and test.

Why it matters to the business

Bad research is worse than no research, because it manufactures confidence that steers real money. McKinsey found the typical CX survey samples only about 7% of a company's customers, and while 93% of firms measure by survey, just 6% are confident the results inform decisions. Self-report has a say-do gap: a study by the analytics firm 84.51 found 75% of respondents misstated their own purchase behavior against loyalty-card data. The encouraging news is that rigor is affordable. Nielsen Norman Group research shows five users uncover about 85% of usability problems, and three rounds of five-user testing beat one large study.

How to use it

• Start every initiative with generative research before anyone proposes solutions.
• Run iterative usability rounds: five users, fix, retest with five fresh users.
• Recruit real customers per segment, and include participants with disabilities.
• Triangulate: pair what people say in interviews with what behavioral data shows they do.
• Ban leading questions and prediction questions. 'Would you use this?' produces fiction.
• Budget synthesis time to match fieldwork time; patterns need analysis, not headlines.

Where teams get it wrong

Research gets designed to prove the idea: leading questions, cherry-picked quotes, an A/B test launched before anyone understands the problem. Speed and volume get mistaken for rigor, when a thousand survey responses on the wrong question are outweighed by five hours of honest observation. The result reads beautifully in a deck and steers the roadmap wrong.

Ask your team

• What question was this research designed to answer, and did it answer it?
• Did we check what customers said against what they actually did?
• When did we last observe a customer, rather than survey one?

Research designed to prove an idea is not research.