Good Interview Data — It’s More Than Just the Facts

You’ve made the big step and are doing Contextual Interviews. You’re collecting customer data in the field by observing customers and talking with them while they do their work or live their lives. But, when you come back to the office and interpret the data it sometimes feels like something is missing; you have a nagging sense (or your team mates are nagging you!) that you could be getting more out of your interviews. If this sounds familiar it may be that you’re forgetting that interpretation starts during the interview itself.

What Kind of Data Detective Are You: Sergeant Joe Friday or Sherlock Holmes?

(With apologies to our non-US readers who didn’t grow up with the US’s old television shows or reruns.) Remember that TV police show Dragnet and the character Sgt. Joe Friday? At least once in every episode Sgt. Friday would interview a crime victim or witness and while he (or usually she) was recounting the tale Friday would utter those immortal words — “Just the facts, ma’am.” That’s how many of us think we’re supposed to gather data in our interviews — “stick to the facts” of what happened.

The facts are critical, they lead us to the data that really matters for design — the true meaning of those facts. But they aren’t the end goal. Instead of being a fact detective, picture yourself as a different kind of detective—Sherlock Holmes. Holmes received the facts from Dr. Watson, and then interpreted the true meaning of the facts through inductive reasoning, while delivering his famous line — “it’s elementary, my dear Watson.”

Of course, Holmes being a brilliant, not to mention a fictional character, has a big advantage over the rest of us. He could sit back in 221B Baker Street, removed from where the action took place, mulling over the details and was always able to come up with the right interpretation. We don’t have that luxury, so whenever possible we want to check out our in-the-moment interpretations with our users to find out if we are correct.

So How Do You Interpret During an Interview?

As you are observing customers, watching what they are doing, part of your mind should be making interpretations or hypotheses. You should ask yourself: Why are they doing these things? What is their intent? What was their thought process? You should then check your answers to these questions with the customer. We need to test our interpretation of what the facts mean before we can decide what implications they have for our design. Keeping the interpretation to yourself is pointless; you need to share it with the customer so he or she can confirm, correct, tune, or extend it.

The chain of reasoning you should go through during an interview is this:

You have the fact(s) You offer a hypothesis to the customer The customer reacts to the hypothesis You record their reaction If you think of an implication for the design, you share it with the customer The customer reacts to the implication You record their reaction If you think of a design idea, you share it with the customer The customer reacts to the design idea You capture their reactions to the design idea

Here’s how it might sound in an interview:

Interviewer:	Let me stop you for a moment. I see that you print out the project plan each time you update it and send it in paper to your boss. I’m thinking you do that because you don’t trust that he will look at it if you put the file on the server.
Customer:	Well, not really. It’s more because he has an hour train ride to and from work each day. If he has it in paper, he’ll look at it on the train. When we used to give him the file electronically he didn’t feel comfortable pulling out his computer on the train.
Interviewer:	Hmm, so I’m thinking if we assume that all project plans are viewed only online that won’t work here.
Customer:	That’s right. Electronic doesn’t mean portable as far as we’re concerned.

Let’s step back and analyze this exchange. There are three things to notice.

• The interviewer didn’t ask, “Why do you print the plan?”
• The interviewer wasn’t afraid to offer a hypothesis that might be wrong.
• The customer had no problem telling the interviewer that she got it wrong.

Offer an Interpretation Instead of Asking an Open-ended Question

Whenever possible, make a hypothesis about why the user is doing or thinking something. You’ll get a better, more accurate response when the customer can react to your idea rather than having to stop and self-analyze his or her own actions. Because you interrupted during the work, the customer is having the experience right now and can tell you if your words match their inner experience.

Don’t be afraid or worried about offering a hypothesis. I find that some of us are and don’t need to be.

If you’re thinking “I really don’t know what they are doing and I’ll sound stupid if I try to come up with a hypothesis”

Don’t. You won’t sound stupid. First, you are watching the work; you’ll be able to come up with something within in the realm of reasonableness. Second, as long as your hypothesis has some small amount of sense to it you will not sound stupid. You will sound like someone who is really trying to understand.

If you’re thinking “It’s obvious why they are doing that, I’ll sound stupid if I ask it”

Don’t. Anytime you make an assumption about why someone is doing something, you are at risk of being wrong. If you ask, the user may surprise you or extend the “obvious” answer with a distinction you hadn’t thought about. Worse case, you’ll actually have user confirmation of what you believe. This worry is really about handling your own ego. If you are worried about sounding stupid, you can always tell the user “At times I may ask you what sound like stupid questions. Because I don’t want to make any assumptions about why you do things, I’m going to ask you questions even if the answers seem obvious to you.”

If you are feeling that “I’m leading the user”

Don’t. Sometimes people tell us they worry that they are going to “bias” or “taint” the data by asking “leading” questions. Nothing can be further from the truth; customers want you to get it right. If you are doing a good job, you’ll be showing more interest in their work than perhaps anyone ever has, or ever will again. But, there’s a catch. You have to be scrupulously honest about the customer’s reaction to your interpretations.

Recording Customers’ Reactions—Sometimes “No” Isn’t Spelled N-O

When you listen to the customer’s reaction, you have to listen for the “no” and recognize that “no” can also sound like:

Customer words	What they are really saying
“Huh?”	The user is really saying, “I don’t know what you are talking about.” In other words, you got it wrong.
“Umm… could be.”	You just got yourself a “no.”
“They would like it.”	That’s a “no” from this user. We’re not currently interviewing “them.” If we need to, we’ll interview “them” later.
“Yes” with elaboration	The real answer is the elaboration, not “yes.”

Don’t Wait for the Interpretation Session

As someone who coaches teams in using Contextual Design, I know that learning to interpret during a Contextual Interview can be one of the harder skills to master. It’s tempting to simply postpone all interpretation until you get back to the office and go into an interpretation session. But with some practice, you’ll get better at it and become comfortable. It’s definitely worth the effort. You’ll find that your data will be richer and your post-interview interpretations sessions will be more productive. Plus, it’s my hypothesis that you’ll also enjoy your interviews even more!