I recently attended a conference on conversion optimisation. One of the speakers talked about how they had used focus groups to provide feedback on alternative designs for their new home page. One of the designs received much more favourable feedback than any of the other variants. This gave the client confidence that this design would win over their visitors. However, when they used A/B testing to measure the performance of each variant the preferred design completely bombed. They were relieved that they had decided to test the design before changing their home page.
The outcome of the test did not surprise me. How we articulate what we think about a user interface design is very different from how we are likely to behave in reality. When we browse the internet we are normally seeking to complete a task that takes us closer to achieving a personal goal. This is not a group experience.
Further, neuroscience indicates that most of our decisions are taken intuitively, often with little conscious awareness. We use behavioural short-cuts to minimize cognitive load and our emotions can over-ride our rational thought processes.
How often do you browse the internet with a group of people you have never met whilst being observed through a one way mirror? Website navigation and task completion is more often than not undertaken by an individual without any group interaction. This may be changing to an extent with mobile devices, but in most cases these are still people we know.
Context is critical in how we respond to any product or service. And yet focus groups are often conducted in a completely alien and artificial environment which does not match the reality of website browsing. Viewing facilities exasperate this problem as they create a laboratory atmosphere.
Some times it is suggested that focus groups are used before designs are drawn up to understand user’s wants, needs and likes. I would also caution against this as focus groups are problematic on a number of counts:
As a result I would certainly avoid using focus groups for evaluating new website pages or journeys. All research methods have their limitations but focus groups are especially problematic and inappropriate in the context of website design.
Individual usability interviews can provide much more meaningful insights about how a website or web page works or not. Learning to listen and observe rather than asking direct questions is likely to provide more useful feedback on how to improve your site. There are also other online research tools that can provide relevant sources of insight. These include:
Eye tracking: AttentionWizard.com - Uses algorithms to predict what page elements visitors are likely to look at before the page goes live.
Feedback from usability & conversion experts: ConceptFeedback.com
Thank you for reading my post and good luck with improving your website’s performance.