UX is based on the idea that it is possible to identify (or at least screen for) a majority of the usability problems on any site by testing a standard number of users.
Usually the magic number is considered 8, or so. This came into vogue to dispute the Stats people and the Marketing crowds, who believe that the MORE the better. Maybe they were influenced by the business people too. It stands to reason, if you test THOUSANDS of users, you'll find most of what you are looking for, but how to SCALE it down, so that you can spend a lot of time (1 hr) with the fewest numbers of people. 8. Or fewer.
That was the standard rationale, at least for the past 20 years. But now that we've become more sophisticated, both as designers and users, aren't our problems MORE complicated?
Sadly, yes.
Jared Spool sent out his UX thought for the day and cited this study, "Eight is NOT Enough", stating that among 18 users, every test had a NEW problem discovered. And only 35% of problems were found with the first five users.
Also, the researchers "estimated" over 600 "obstacles to purchase" on the site, and after 18 people found only 247.
This leads to a deeper question: now that digital business has evolved into a complex language of Semiotics, how are we to achieve the DREAM of a Common Language (thanks to the poet Adrienne Rich for her title)?
Not only is the digital world made up of words (in a variety of languages, including all the computer based ones), but it is encoded with signs, symbols and shadings which change the meanings of words.
(A word highlighted in blue might be a hyperlink, or something underlined. Or larger. Or it could only be revealed on Hover Over-or not at all).
We also now have to shift our idea of langauge to include:literary, visual and ALSO layer in the complexities of Time/Interaction. If I click onto one page and take another action, is it possible to navigate back to the first-or am I now in a different mode? (i.e. signing in to a webpage) AND-big question-does the screen REFLECT this change of state, or have the developers/designers introduced an inequality of information? (Cookies, bad UI display, HIDDEN UI state)
It's not that UX Testing can be limited to 8 users, it's still a miracle that it is done at all, and that some companies think they are "done" at a certain point.
In fact, there is an inherent fallacy in any UX Team that gets assigned by project can sign off on anything, as if representational Users can be found who cover all the future users. At best, UX tests find the biggest problems which might not have been found through ANY OTHER PROCESS.
Nothing is perfect.
Usually the magic number is considered 8, or so. This came into vogue to dispute the Stats people and the Marketing crowds, who believe that the MORE the better. Maybe they were influenced by the business people too. It stands to reason, if you test THOUSANDS of users, you'll find most of what you are looking for, but how to SCALE it down, so that you can spend a lot of time (1 hr) with the fewest numbers of people. 8. Or fewer.
That was the standard rationale, at least for the past 20 years. But now that we've become more sophisticated, both as designers and users, aren't our problems MORE complicated?
Sadly, yes.
Jared Spool sent out his UX thought for the day and cited this study, "Eight is NOT Enough", stating that among 18 users, every test had a NEW problem discovered. And only 35% of problems were found with the first five users.
Also, the researchers "estimated" over 600 "obstacles to purchase" on the site, and after 18 people found only 247.
This leads to a deeper question: now that digital business has evolved into a complex language of Semiotics, how are we to achieve the DREAM of a Common Language (thanks to the poet Adrienne Rich for her title)?
Not only is the digital world made up of words (in a variety of languages, including all the computer based ones), but it is encoded with signs, symbols and shadings which change the meanings of words.
(A word highlighted in blue might be a hyperlink, or something underlined. Or larger. Or it could only be revealed on Hover Over-or not at all).
We also now have to shift our idea of langauge to include:literary, visual and ALSO layer in the complexities of Time/Interaction. If I click onto one page and take another action, is it possible to navigate back to the first-or am I now in a different mode? (i.e. signing in to a webpage) AND-big question-does the screen REFLECT this change of state, or have the developers/designers introduced an inequality of information? (Cookies, bad UI display, HIDDEN UI state)
It's not that UX Testing can be limited to 8 users, it's still a miracle that it is done at all, and that some companies think they are "done" at a certain point.
In fact, there is an inherent fallacy in any UX Team that gets assigned by project can sign off on anything, as if representational Users can be found who cover all the future users. At best, UX tests find the biggest problems which might not have been found through ANY OTHER PROCESS.
Nothing is perfect.
No comments:
Post a Comment