Troubleshooting UX: What People Say Vs. What They Really Think

balaganHow to best improve a concept, a service or product is rather obvious: there is now a growing movement to find ways to bring the users/visitors/clients (or whatever you chose to call them) into your process. To learn from and together with the user is what experience design is all about.

The very existence of this conference is a natural consequence of this trend.

But anyone who has been involved in a UX process is familiar with the problem of not knowing how to value the input given by respondents.

For example! There are studies, especially interesting for us at the library, that show that people usually want to recommend books they themselves haven’t read (nor will probably ever read). This tendency might obviously pose a huge problem for bookstores or libraries in a social economy. It’s not possible to design a business campaign in a store, or a special drive at a library, based on such input.

”…products are improved by listening to what users want—and not necessarily what they say they want.”

The quote is from ”Bridging the gap between actual and reported behavior”, an article by Kat Matfield recently published on UX Booth. The article is quite to the point, identifying three major “danger zones” regarding getting useful input from respondents involved in UX processes. I borrow heavily from it below, adding some reflections based on my personal experiences.


Social Desirability and Embarrassment

Life is a stage, and as an actor you are more or less constantly observed by others. Most people are concerned with doing the right thing, looking right, not to be seen as weird or extreme, etc. This poses potential problems for a UX process, since we will get dishonest input. Often when developing something, we are more interested in new images rather than cementing the old.

Never in my line of work have I´ve encountered this problem stronger than when talking about “playing together” with kids aged 9-12 during the Balagan project (where we developed new library services for kids in their tweens).

Observing the kids it was plain and clear that they played all the time, using play and game mechanics for everything from moving through the room to solving problems as a group. But when asked, they all claimed that the didn’t play at all any longer “cause we´re not kids anymore”. This taught our project an important lesson – we would have to use play as the foundation of all our services, but without calling it play, as this conflicted with the kids’ self-image.

Matfield stresses the importance of creating a very relaxed and safe place for the interview. From setting up a scenario where the respondent has good expectations before even entering the place for the interview, to creating a physical and social space that is safe, inviting and professional. (A.k.a. using your experience design skills even to design your input process!) And second, to maybe avoid group sessions – at least for some more sensitive topics – where social conformity may play a big part.


Battling Wishful Thinking

It’s not uncommon for respondents to answer in a way that is more positive and utopian rather than realistic. Just like when someone has bought a new membership for a gym, saying that “yeah, I´ll probably go there two, three times a week”… We do this when our self-image differs from reality – in real life we can’t live up to that image, but it´’ still there influencing our answers.

The problem with troubleshooting wishful thinking is that there is a high possibility that the respondents honestly think they are being truthful.

The advice here is to try to avoid questions that are open for speculation. Instead of “what would you do in situation A or B”, rather ask about previous experiences, like “when you solved a problem like this before, what did you do”. Ask for descriptions, not speculation.

I would like to add the importance of sometimes (often!) asking follow up questions to see if the answers “hold up” to further inspection. Asking kids if they read, most say “yeah!!!”, but it´s important to see how often, what, when, if they liked it or if they even remember what they read a week ago.


Differing Contexts and Mindsets

The context of a UX meeting is crucial to be able to value the results. As stated in the Matfields article, bank costumers that find mobile tech complicated will emphasize “simplicity” when asked for improvements of a bank app, where the more tech savvy will answer “security”. Different people got different views on a topic due to their context.

When designing our UX-processes at the library we’ve learned a lot about how to respect how we need to work differently with different groups in Malmö depending on their everyday life context. When we wanted to get in touch with two very different groups of parents, we quickly found out that we needed two totally different ways of communicating. One group wanted formal, written communication while the other favored personal contact through people they already knew and trusted.

To not understand these contexts would’ve resulted in no UX process at all; we would not even have reached our target groups.

”Mindsets” is something new for me, but when I read about it I got very curious. It goes like this: we got two modes of thinking, two mindsets. One is slow and rational, one is quick and intuitive.

Matfield claims that when we are asked a question to reflect upon, we tend to go into the slower and more rational way of thinking. But, when we are making decisions on the spot, we are more prone to be intuitive and quick of thought.

So basically we might tend to ask the slow and rational brain about how to design a store, when it´s actually the quick and intuitive brain that will shop there later…

This is new for me, and for sure a topic I will examine deeper. Maybe some of you are more familiar with the theories?


Let´s Talk!

For me, troubleshooting UX in many ways boil down to knowing a lot about the people getting involved in the processes before the UX sessions even begin. This is the one experience that I value the most – that people are just “people”. You can’t ever circumvent the fact that every respondent involved is a human being. We have to be curious, and learn more about our target groups.

There are of course lots and lots of other ways to troubleshoot UX – these few examples are simply food for thought. During Alibis 15 we will most certainly learn more, and I’d invite any of you to discuss this further either in the comments or when we meet in October (I’m damn curious on what you’ve learned about this topic)!


Written by

Andreas Ingefjord is a development officer at Malmö City Library. Together with his colleague Linda Johansson he is consulting for Alibis for Interaction 2015.


E-postadressen publiceras inte. Obligatoriska fält är märkta *