Challenges and Solutions

At CHI 2005, a formal SIG provided a forum for participants to hear how different companies have tackled these problems. After initial presentations by a panel, over 80 people broke into eight topic groups to brainstorm issues they’re facing and possible solutions. The topics were:

Topic 1: Effectively communicating the customer data and keeping stakeholders informed

1. Working with and managing teams
2. Getting to the customer: finding them, working in different languages, dealing with large projects with multiple roles types
3. Staying an iteration ahead when Development is using an Agile method
4. Introducing user-centered techniques into the organization
5. Balancing being rapid with being creative
6. Communicating with distributed teams and distributed customers
7. Maintaining the quality of customer data and interpretation sessions

The groups’ results are displayed below. Also available are the slides from each panel member:

Karen Holtzblatt, InContext

Lisa Baker, LANDesk

Joerg Beringer, SAP

Topic 2: Effectively communicating the customer data and keeping stakeholders informed

Issues

• There is a lot of data, and raw data isn’t suitable for communication
• It sometimes isn’t clear how many data points we should have before we start to communicate
• Consolidated data is what needs to be communicated, but we don’t consolidate until we have all the data
• There’s a difference in communicating requirements versus communicating design specifications
  • We need to keep focused on the user when determining requirements
  • We need to make sure the design is implemented as designed
• We need to communicate not only solutions but also the underlying problem so the team can develop its own solutions
• Some of the Contextual Design models are less useful for communicating with developers
• We need to take into consideration the stakeholders’ needs when communicating to them
  • But sometimes stakeholders aren’t clear about what they need, or don’t know what they need
• Requirements need to be communicated so that they help in the development of test cases

Solutions

• Communicate upfront, early on
• Train some of the developers to understand HCI and user requirements
• Get the developers interested in the things they care about
  • Recognize that this varies from project to project and developer to developer
• Get the developers involved in the requirements phase and design process
• Help the developers understand the purpose and benefit of the Contextual Design models
• Reinforce open communication, don’t just expect it to happen or happen without work
• Treat the design as a blueprint
• Establish the role of “design communicator”

Topic 3: Working with and managing teams

Issues

• Uniformed managers intervene in the process, and impose their own values
• Egos have to be dealt with, people want to impose their ideas
• Roles are unclear: who controls the information, consolidates it, communicates it
• Leadership and continuity is needed when the project is long

Solutions

• Implement online project sites as a repository of user research results
  • Create a central place to find out what’s going on with the project and how ideas evolved over time
  • Take communication out of email
• Use webcams projecting virtually to see what happens during activities like wall walks
• Have physical project rooms with continuity in their location
• Overcome egos and “I think” by teaching a clean (empirical) approach (field data, interpretation, design response)
  • Stay grounded in the data
• Use a round robin process to inform one another about what we can do, what we’re good at
• Involve the entire team in the synthesis of ideas (i.e., interpretation sessions) so there’s a shared understanding

Topic 4: Getting to the customer: Finding them, working in different languages, dealing with large projects with multiple role types

Issues

• Recruiting customers needs to go faster
  • The time needed increases when it’s a new market
  • The time needed increases if the customer data is controlled by a contractor/partner
• There can be issues around the organizational structure for contacting customers, including the account reps’ thinking we are stepping on their turf
• The are cultural differences to approaching contextual interviews
  • When you use translators to capture the data that are not UCD professionals the data quality becomes questionable
  • Some cultures don’t want to be filmed or are uncomfortable with Contextual Inquiry
• The more customers you get to, the more data problems you can have
  • More data means a larger affinity diagram that takes more time to build, and can be so big that there may be information overload, especially if you haven’t allocated enough time
    • Requesting more time or resources has to be justified
  • If you bring helpers in they have to be knowledgeable, not just “warm bodies” — you have to have the right people
• Finding the right users can be a full-time job
  • You need a user profile in order to recruit, but sometimes you don’t know who is right until you’ve done some interviews

Solutions

• Create a “design partner program” with your existing user base
  • Have an up front agreement to participate
  • Partners get perks
• Address the reality of how long it takes to recruit
  • Use a marketing research company
  • Create positions for in-house recruiters (two full-time people)
• Agree in advance with the project manager on the user profile
• Resource appropriately
  • Recognize that recruiting time is always underestimated

Topic 5: Staying an iteration ahead when Development is using an Agile method

Issues

• There is a lack of time and personnel
• Requirements are unclear
• It’s unclear about how tasks are passed from one discipline to another
• Unrealistic deadlines are set by someone else
• Expectations are not set
• There are problems with getting feedback incorporated

Solutions

• Plan multiple sprints ahead and work with the product marketing managers or project leads to get a global view of the goal — and to get team to agree to it
• Co-locate the different disciplines
• Keep a war room with the artifacts like wire frames visible to all
  • Use a Wiki for remote workers
• Constantly re-asses where you’re slipping and re-scope
• Do two versions of project simultaneously:
  • “Fast” version using best practices to design but without user input
  • “Slow” version incorporating feedback
• Plan room for feedback in your sprint

Topic 6: Introducing user-centered design methodologies into the organization

Issues

• Organizational resistance needs to be overcome; it is hard for organizations to change
• Resources are scarce
• Prior bad experiences with trying user-centered design result in not wanting to try again
• People don’t understand how to resolve a perceived overlap in roles
• There isn’t faith in the sample size, and this can cut both ways
  • Either Marketing doesn’t think a small sample is reliable or they rely on too little data
• Time or market pressures are dictated by external events that can’t be changed
• Different teams have different goals, and they don’t match
• Different teams have different needs and different idea of what is valuable to them
• Once you have a successful project, you have to figure out how to make user-centered design a standard practice and not a one-time event

Solutions

• Start with a pilot project
  • Pick a pilot project that you know will be successful; stack the deck in your favor
• Take people from a successful project and make them “ambassadors” to other teams
• Communications with other disciplines should always be based on reinforcing that you are in a partnership and not trying to invade their turf
• Start by explicitly stating the goals, and put them in writing so everyone knows what everyone wants

Topic 7: Balancing being rapid with being creative

Issues

• There’s no time for design
• Iterations are too short
• The design of the existing product is a constraint
• People will self-censor big ideas because there’s a short time frame
• There’s a tendency to divide and conquer instead of taking a holistic approach
• There isn’t time for a rich envisioning of entire design
• The team size may be too small or too narrow to stimulate divergent or creative thinking
  • Team doesn’t include and (or enough) designers/UX members
• There’s too much focus on development versus design
• The focus is on “crash and burn”
• Handling problems presented in the previous release leaves no room for creativity in the next iteration
• The team feels it cannot take risks

Solutions

• Design ahead and be aware of where you are going, with three possible approaches:
  • Do an initial, holistic guidance for the design up front
    • Vision and be creative upfront as guidance for subsequent releases, with freedom for divergent thinking
  • Before each iteration, do the design for the next one out so you stay ahead and have time to be creative
  • Do the entire detailed design up front, and then iterate in the development with the flexibility to adjust the design
• Use somewhat longer iterations, and include a planning phase
• Work on maintaining open, positive relationships
• Be willing to re-factor, don’t get too bound to a single concept

Topic 8: Communicating with distributed teams and distributed customers

Issues

• People work in different time zones
• Budgets for traveling are limited
• Off shoring is a common practice
• Languages are different
• Cultures are different
• Ideas are not in sync; there are different expectations, traditions, departmental structures
• Coordination is difficult
• Data and knowledge sharing is difficult
  • It has to be stored somewhere everyone can access, but with security

Solutions

• To support synchronous communication, be very flexible about meeting times and have standard meeting times that can be planned for
• To support synchronous communication, use a common web space
• Capture and share collaborative experiences
  • Videotape the affinity diagramming process to better share the experience, but then run it in a fast or condensed mode
  • Use digital whiteboards
• Employ a local expert or local consultant to do CI interviews

Topic 9: Maintaining the quality of the customer data and interpretations

Issues

• There are concerns that the right questions are being asked in interviews
• It’s not clear that the right users are being interviewed
• Interviewers lose track of the problem focus
• Data is coming from multiple sources
• The data being collected and interpreted doesn’t address the project’s goals

Solutions

• Review as you go along to see how the data integrates with the project’s goals
• Get a large pool of users to start with, and then focus on those who are more experienced or can fill your data holes
  • Go broad, and then deep on a few
• Use an iterative process
  • Construct a persona → improve the focus → refine the persona → get team and stakeholder feedback
• Compare and check your data with other data sets like industry data, logs, marketing

Acknowledgments

This article reflects the effort of the three panelists who shared their experiences and then acted as discussion group leaders at the CHI 2005 SIG.

Karen Holtzblatt, PhD — InContext Enterprises
Lisa Baker — LANDesk
Joerg Beringer — SAP

Our thanks to everyone who attended the SIG.