Innovation or Market Research?

By Alexandra Mack, Designer, and Karen Holtzblatt, InContext Co-founder and CEO

Points to consider :

Field data collection is becoming more common in many industries

Field data can be used for design, innovation, marketing, and product placement

Many field methods rely on set questions and do not collect design data

Design data comes from careful observation and partnering with the customer

Download pdf

A recent New York Times article highlights the use of field research for the creation of products. In the high-tech industry methodologies like Contextual Inquiry, our field interviewing process based on ethnographic and anthropological techniques, have been used for the past 15 years to understand customers and design innovative software and hardware products and business systems. Manufacturers of consumer products like rakes, baby monitors, blenders, microwaves and even product packaging, have not traditionally used these techniques to understand their customer's needs and design to them — but they are waking up.

However, the application of field techniques to consumer product design runs the risk of being a focus group in disguise. The New York Times article focused on one company using field data and its methodologies. A close examination of their methods versus field methods within a design process like Contextual Design reveals significant differences. If consumer product companies are going to get the full power out of field research, they need to understand that it is a significant part of design — not a new kind of focus group or market survey.

Is it Design Data?

Contextual Inquiry is a style of ethnographic inquiry applied specifically to obtain design data. At a superficial glance, it may look the same as interviews used for market research, but it is in fact very different.

Market research traditionally does focus groups, surveys, and direct interview to assess whether or not people like a product or to test sales messages and benefits. These studies can characterize market trends, buying patterns, and opinions about what is out there. While these methods all have their applicability, their scope is limited. Market researchers are not interested in fixing a product, or for that matter, creating new products. To identify latent customer needs that can direct product definition, the method of field data collection and use must support getting design data not opinion data.

When market researchers go outside the focus group box, their goals are still the same — too often they continue to collect the same kind of data using a different methodology.

The New York Times article contained the following excerpt from a field study of barbecue-grill customers while they were grilling chicken:

Husband: "I had a flame-up earlier," Eric says grimly of the Ducane. "It's the chicken fat, see."

Wife: "We like lean buffalo burgers best," his wife puts in.

Interviewer: "So, what other issues do you have? Smoke? Carcinogens in overcharred food?"

Interviewer: "What about ignitability? Do you feel that gas is more ignitable than charcoal?"

Husband: "It's more convenient. Briquettes take longer to warm."

Interviewer: "And you think men should make the fire?"

Wife: "Well it's kind of a primordial thing, isn't it? Men make the fire."

Husband: "Yeah. Men make the fire."

Field data gathering for the purpose of design focuses on observing how people use the product your are designing in the context of everyday use. Talk with the user/consumer is about what they are doing with the product. We do not pose a series of questions stemming from hypotheses about potential issues: why is the quoted interviewer asking about ignitability or men and fire? These are simply survey questions in disguise.

During field data collection for design, discussion emerges from what is happening in real life: during product use, the social dynamics surrounding product use and the overall values associated with product choice. The practice observed with the product suggests related products to sell or ways to change the existing product. For example, watching people juggle plates and utensils at the side of the cooker suggests a wider counter surface. Seeing people put the barbeque up against their house for one-step access during the winter, suggests proper ventilation for porches and small footprints. Combining the two observations suggests wrap-around counter models creating out-door cooking environments with no "installation."

And design from field data produces the value proposition and market message: "Have the taste of BBQ all year long." Designing the market message is part of designing the product. Both stem from understanding the value of the designed product in the lives of the people using it.

Field interviews may leave the conference room, but as the New York Times article demonstrates, they do not necessarily lead to any different data being collected.

If You Ask a Leading Question…

You often get the answer you are looking for. But it may not be the right (or real) answer for that user. Leading questions present the answer to the user, and often require a simple yes or no answer. Questions like "Do you like that the menu has a rollover graphic on it?" simply ask the user to agree with you. Without watching how the user really interacts with their artifacts and environment, the researcher doesn't learn anything new.

The myth of men and fire has been around for many years. Validating this myth rather than looking for the male/female dynamic as it unfolds in current times will not lead to new product or market messages. If the field researcher saw the wife acting squeamish about lighting the fire, they might share this observation. If they saw the husband tell the wife to move out of the way, outdoor cooking was his job, they have a reason to talk about attitudes about role relationships. But if husband and wife are both working at the grill in partnership then there is no reason to raise the issue of sex differences. Rather discussion of shared partnership is in order. Any entering hypothesis about role differences is already invalidated in the very behavior of the people living their lives.

By sharing the observation about role differences with the users the interviewers check their interpretation and give the users an opportunity to validate and extend the observation. In Contextual Inquiry, we keep the hypotheses and interpretations of interviewers in check by ensuring that they are tied to the observed data. By sharing the hypothesis, we make sure we take home an understanding that is reflective of the person being interviewed rather than our preconceptions.

Statements prompted by direct questions such as the ones in the interview quoted above do not produce new insights. Articulating the implicit meaning of the actions of people living their lives, on the other hand, do. If interviewers are too focused on their pet questions they end up looking for confirming data. They can find themselves ignoring the real data — and the real product opportunity — that is right before their eyes.

The Focus of the Study Directs What We See

The interviewer can still direct the interview without asking leading questions that are not relevant to the behavior at hand. The direction of the interview should be determined by the focus of the study, which tells the interviewer what to pay attention to and what to ignore. For a barbeque company, a good focus might be to learn how the mechanism is manipulated, what implements and surfaces are needed ready-to-hand, how many people are involved in meal preparation, what is the social dimension of the cooking itself, and the location of the product in the house.

With these design-relevant dimensions of practice in mind, the interviewer will be directed to look at the way the product is used and experienced. Field data is best used for getting at unarticulated value and needs. People know everything about their lives and how they use products, but they do everything habitually. If needs are habitual and unarticulated people can't tell you about them in response to a question-that is what the observation is for. Good field data collection helps people articulate their thoughts, feelings, values, and actions around a product so companies can design to them. Once we know how a product is used, we also know how to improve and market it.

A Field Study Avoids Disruption — the Natural Environment is the Goal

The New York Times article also highlighted a study in which users tested different liquid shower soaps while watched by a camera and "other eager researchers who crowded into her bathroom". The user was described as she "crouches beneath the shower nozzle and blinks into the blazing lights of a one-person film crew…[she] seems a little nervous as she adjusts the faucets." This kind of setup is not anything like the user's natural environment, and for data collection may not be much different than a usability lab or focus room with one-way mirrors and disembodied voices.

Contextual Inquiry is based on entering the environment in a non-disruptive way much as an apprentice would sit at the knee of a master to learn how they work. (View related article: Apprenticing with the Customer: A Collaborative Approach to Requirements Definition). Good field interviewing is a one-on-one relationship that lets the user engage in their natural activities while being watched by an interested person trying to understand user behavior and experience. Cameras, crews, multiple researchers, all disrupt the natural environment and heighten the "theater" aspect of the activity. It is harder to know what is real and what is for show in this altered life context. So it's harder to get reliable design data.

The apprentice model improves data quality because it reminds interviewers that:

The apprenticeship model establishes a trusting relationship, in which the user (master) feels comfortable in the workplace with the interviewer present. It also allows the user to go about his or her work as they would on any other day, allowing the interviewer to collect real design data. In this way, we can see and talk about the real dimensions of life and the way products and systems fit into it and impact it.

What is Design Data?

A good research focus for product design collects design data: aspects of the user's physical environment, the language and artifacts they use, the people they work with, and their work culture. These are the details that are needed in order to design for real support of life practice and product innovation. These elements can only be found in the places people are living their lives with these products, and by an interviewer committed to learning from the user.

Market researchers aren't designers. They can tell their clients about the general market and include interesting anecdotes, but they cannot supply data for design. If the people collecting the field data do not understand how to use it for design, they will not collect the kind of data that can drive design. And if there is no reasonable process for using that data in a follow-on design process, that field data will at best go to waste and at worst misdirect product direction.

Contextual Inquiry is part of the Contextual Design product definition process. (Learn Contextual Design) It has been used to successfully identify directions for innovation, create new products (or rework old ones) so that they actually fit into people's lives.

By learning about the life and work practice, Contextual Inquiry tells you about what your market really is and provides a springboard for real product design.

About the authors

Alexandra joined InContext as a designer in 2001. She has 10 years of experience in qualitative and quantitative research and analysis. She holds a PhD in Anthropology from Arizona State University, a MA from the University of York, and a BA from Harvard University.

Karen has been a leader in the usability and user interface design community for more than a decade and is recognized worldwide as an expert on customer-centered design. She's been at the forefront of the movement to lead the high-tech industry from technology-driven to customer-driven design.

Published 02/14/2002