Should the sample size n be greater than 30? If you’ve taken any introductory statistics course or an AP statistics class (or helped your child with it), you’ve encountered the n ≥ 30 rule. The “magic number 5” rule we’ve written extensively about applies (with its important caveats) to problem discovery for usability testing. But the
Human research moderators aren’t going away. Despite technological advancements, such as remote unmoderated testing (with and without thinking aloud) and AI moderators, a live researcher asking questions to a live person will always be needed. Technological innovations are less likely to render things obsolete than make them more specialized (like Internet > TV > Radio).
In the movie 2001: A Space Odyssey (made in 1968), a vision of the future included video conferencing. And 1989’s Back to the Future’s vision of 2015 included not only video conferencing but also getting fired over it. While we may scoff at how hoverboards and flying cars still haven’t arrived, it’s now normal for
You invite your friends over for a gathering at your house. As the party gets started, you watch in horror as a friend trips on the edge of your carpet before she even takes the first sip of her Cabernet (now spilled on your white couch). You rush to fix the carpet. But before you
Would having a system for classifying usability problems be helpful to UX researchers and designers? Would it reduce the evaluator effect? Categorization frameworks have been around for decades, but in our experience, they haven’t seen a lot of adoption by commercial development teams. There are probably a few reasons for this: they might not be
Finding and fixing problems is a core activity of much of UX research (similar to identifying and preventing software bugs and product defects). The problems found while users attempt tasks are often broadly referred to as UI problems because the friction points tend to involve issues that blur the lines between bugs, functional deficits, and
Sample size estimation is an important part of study planning. If the sample size is too small, the study will be underpowered, meaning it will be incapable of detecting sufficiently small differences as statistically significant. If the sample size is too large, the study will be inefficient and cost more than necessary. A critical component
If different researchers conduct a study, will they identify different problems? While it was originally controversial, the evaluator effect is now well-established in the field of UX research. The evaluator effect is the finding that, in general, different teams or individuals conducting a usability study will identify and report different problems (Hertzum & Jacobsen, 2003).
One of the most popular UX research methods is Think Aloud (TA) usability testing. Having participants speak their thoughts while working on tasks helps researchers identify usability problems and potential fixes. After a TA session, many UX research teams also collect study-level metrics. Study-level metrics are typically asked only once in a study (unlike task-level
One of the most popular UX research methods is Think Aloud (TA) usability testing. Having participants speak their thoughts while working on tasks helps researchers identify usability problems and potential fixes. But does the added burden of speaking while attempting a task make the experience harder and affect perceptions of the website or app being
