Focus Setting: Getting the Right Interview Mix

In my previous design corner, Focus Setting: Building a Strong Foundation for Your Project, I discussed how your project would be at risk before you even start if you don’t explicitly set the project focus. However, sometimes the leader of a design project gets the appropriate people to agree on the project focus, but is then unsure of how to translate that into the next step: determining how many users, and which ones, should be interviewed. Doing this is a three-part process, consisting of:

1. Determining an “interview budget” based on type of project
2. Identifying work roles and allocating interviews among them to determine “role mix”
3. Determining the “organization mix” you want in order to allocate your total number of customer interviews

Even though I’m using the words “customer” and “organization,” this process applies equally to products sold to a market and internal projects.

Determining the interview budget

The project goals are critical choices for a design project because they affect the timeline and cost of the project. Goals can be tactical or strategic. Evaluating the current product to identify the top 10 issues, get user validation, or make isolated usability improvement are all examples of tactical goals. Extending the existing product in a limited way, such as enhancing an existing feature or adding a new, isolated feature, is a greater design effort, but still fairly tactical. Upgrading an existing product to contain significant new functionality, or creating a new product, are examples of strategic goals.

Tactical project goals require fewer user interviews and fewer Contextual Design (CD) activities, resulting in a smaller effort and shorter timeline. Strategic goals require more user interviews and more Contextual Design activities, and have a correspondingly larger effort and longer timeline.

The first step in focus setting is to choose the project goals and then to determine an interview budget based on the type of the project goals. The interview budget is the number of user interviews that will be done during the user investigation part of the project.

Identifying the role mix

The next step is to identify the organizational job titles or roles involved in the work being studied, and the activities that each performs. Many products have different kinds of users. However, a design project needs to address a particular work problem, and the issues and strategies of people involved in that particular area of work. Think about both direct users (let’s call them primary users) that do their main job by using the product, and indirect users (outlier users) who use the product as a source of information or other support to do their main job. To help you plan, you’ll see below a Role Mix table you can use as a model. Start by filling in just the Job Title/Roles and Activities for the user work problem that will be studied and addressed during the project.

Not all roles are equally important for the purpose of the design effort. Primary user roles are the ones that will be given the most attention during the project. More interviews will be done for these roles, and more extensive data will be gathered on the activities and strategies of users performing these roles. The rest of the roles are outlier roles—these roles are part of the work being studied, but are not the main targets of the project. Often primary roles involve creation of a key artifact in the domain of work being studied, while outlier roles are typically ones that supply information to, consume information from, or evaluate the output of the primary roles. Fewer interviews will be done for outlier roles, and the data gathered will be less extensive. Make explicit choices about which roles will be primary ones for the project, and which will be outliers. Identify them in the Role Type column in the Role Mix table. Try to keep the number of primary roles manageable. For a small CD project there may be only one or two primary roles, while for a large CD project there may be several primary roles.

Allocating the roles

At this point the interview budget must be divided between the roles. Each primary role interview is a full two-hour contextual interview, and counts as one interview toward the interview budget. Outlier interviews are often not full contextual interviews: they are an hour (or less), and each one counts as one-half of an interview toward the interview budget. But remember that for any role you never want to do interview only one person; you always need a counterpoint. Decide how many interviews from the budget you want to allocate to each role, and fill in the number of interviews and number of people to interview, keeping in mind that one interview from the budget for an outlier role is actually two user contacts.

To illustrate how to do the role mix, here’s a partially completed sample role mix table (the activities and tasks are only a partial list). We’ve simplified it for illustration purposes, but imagine that the project focus is to create the next version of a product used by project leads creating development schedules. Since we’re developing a tool to be used by them to create and publish the schedule, they are our primary user. We also know that people like developers, development managers, product managers, and documentation writers will consume their schedule, so we’re going to interview those roles as outliers.

Table 1: Role Mix

Job Title/Role	Activities & Tasks	Role Type (Primary or Outlier)	Number of Budgeted Interviews	Number of People to Interview
Project Leads	1. Build Initial Schedule 2. Keep Schedule Updated 3. Communicate Schedule	Primary	10	10
Developers	1. Use published schedule 2. Provide info for schedule	Outlier	1	2
Development Managers	1. Use published schedule 2. Provide info for schedule	Outlier	1	2
Product Managers	1. Use published schedule 2. Provide info for schedule	Outlier	1	2
Documentation Writers	1. Use published schedule 2. Provide info for schedule	Outlier	1	2
Totals			14	18

While it is necessary to choose the important roles and activities during focus setting, expect that the team will adjust this emphasis as they begin to gather data. They might find that they need to concentrate on slightly different activities, or even slightly different roles, after they see the data that is obtained during the initial few interviews.

Determining the organization mix

The next step starts with identifying organizations that are good candidates for interviews. There are a lot of potential factors to consider when trying to figure out which companies or organizational units to visit. Which factors really matter depends entirely on your project focus and your company. Some factors to consider might include:

• Industry, as well as whether the industry is a current money maker or an important area in your company’s plans for future growth
• Size of the organization
• Whether the organization uses best practice processes
• The product in use (your company’s, homegrown, competitor, or none)
• Whether companies using your product have new or mature implementations, or have novice vs. expert users
• Geography, if you think geographical differences might really matter
• Whether your company has a good relationship with the organization
• Whether political issues will be generated if you do not visit a certain organization
• Ease of travel to the customer site

Here we are discussing factors that are appropriate for a business-to-business product. Of course there will be an entirely different set of factors if you are building a consumer product or a product for internal use.

Allocating interviews across organizations

All that’s left now is to allocate the role interviews among the organization candidates. Since the goal is to see the broadest variation of work during the fewest possible interviews, this allocation must be done carefully.

Before starting to allocate interviews between organizations, it is a good idea to discuss and agree upon any financial and travel limitations. These may restrict the organization candidates. Keep in mind that more restrictions make it less likely that you will be able to gather data on all of the work variations that you are interested in seeing. If finances or travel are significant issues, be selective, but try to maintain the diversity of the organization choices. If possible, leverage your personnel who may not be part of the core design project team, but who may be located near to a desirable organization, to perform interviews.

When allocating the role interviews, it’s important to maximize the variation between organizations given all of the factors you are considering. Spread the interviews for a role along each of the organization characteristics you identified. For instance, plan to see users who use your product, your competitor’s product, homegrown system, and no product (manual processes). One interview may satisfy several of the characteristics (both industry and size, for instance) that you are interested in. A rule of thumb is to make sure the final mix includes two organizations of each significant type or with each significant characteristic. If a particular organization is not available to you during the Contextual Inquiry portion of the project, keep them in mind for the paper prototype interviews that happen later in the project.

Here’s what the organization mix might look like for our sample project. The table is showing the interview budget, which does not necessarily equal the number of actual people you will interview. Remember that each of our outlier interviews are only going to be one hour, and count as only ½ an interview toward the interview budget. The totals in this table need to match the totals in your Role Mix table. After you’ve done the allocations, review the spread to make sure that you have planned for the maximum variety in work practices—the most different environments and work situations.Table 2: Organization Mix

Organization	Project Leads Using Our Product	Project Leads Using Home Grown Product	Project Leads Using Competition	Developers	Dev. Managers	Product Managers	Doc. Writers	Total
ABC Company	2			.5	.5			3
XYZ Company		1					.5	1.5
AXX Company		1		.5		.5		2
123 Company			1		.5			1.5
456 Company			1				.5	1.5
AAA Company	2					.5		2.5
XXX Company	2							2
Total	6	2	2	1	1	1	1	14

Ready to roll

Once you have used the project focus to determine an appropriate interview budget, decided on the role mix, and identified the organization mix you are ready to roll. By knowing organizations you need to contact, what type of user role to ask for, and the activities to specify that the person have among their responsibilities you’ll be starting off down the right path, not one filled with problems before you even begin.[/fusion_builder_column][/fusion_builder_row][/fusion_builder_container]