Quantifying the User Experience: Practical Statistics for User Research

seek to minimize systematic bias in your sample but remember that representativeness is more important than randomness.

With any comparison we also want to know the size of the difference (often referred to as the effect size). The p-value we get from conducting the paired t-test tells us only that the difference is significant.

Chapters 6 and 7 contain a thorough discussion of power and computing sample sizes to control Type II errors.

Usability has an international standard definition in ISO 9241 pt. 11 (ISO, 1998), which defined usability as the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use.

rely on a subset of the population to use as a proxy for the population.

We need a way to know how good (precise) our estimates are. To do so, we construct a range of values that we think will have a specified chance of containing the unknown population parameter. These ranges are called confidence intervals.

A narrated visualization of the standard deviation is available online at http://www.usablestats.com/tutorials/StandardDeviation.

after subtracting each value from the mean, square the values then take the average of the squared values. This gets you the average squared difference from the mean, a statistic called the variance.

The primary motive behind sample size estimation, as illustrated in Figure 6.1, is economics.

A confidence level of 95% (the typical value) means that if you were to sample from the same population 100 times, you’d expect the interval to contain the actual mean or proportion 95 times.