Quantifying the User Experience: Practical Statistics for User Research

(assuming the two-week period was representative)

The difference is statistically significant, but is it practically significant? The answer to this question depends on how we interpret the lowest and highest plausible differences.

keep in mind that statistical confidence means confidence in the method of constructing the interval—not confidence in a specific interval

Most tests contain some combination of completion rates, errors, task times, task-level satisfaction, test-level satisfaction, help access, and lists of usability problems (typically including frequency and severity).

consider sampling from different groups if you have reason to believe: • There are potential and important differences among groups on key measures (Dickens, 1987). • There are potential interactions as a function of a group (Aykin and Aykin, 1991). • The variability of key measures differs as a function of a group. • The cost of sampling differs s

after subtracting each value from the mean, square the values then take the average of the squared values. This gets you the average squared difference from the mean, a statistic called the variance.

Comparing the two outcomes of binary variables for two independent groups happens to be one of the most frequently computed procedures in applied statistics. Surprisingly, there is little agreement on the best statistical test for this situation. For large sample sizes, the chi-square test is typically recommended. For small sample sizes, the Fishe

bulk of the values in a population cluster around the “head” of the bell-curve. In general, they’ve found that 68% of values fall within one standard deviation of the mean, 95% fall within two standard deviations, and 99.7% fall within three standard deviations.

Ideally, your sample is also selected randomly from the parent population. In practice this can be very difficult. Unless you force your users to participate in a study you will likely suffer from at least some form of nonrandomness. In usability studies and surveys, people decide to participate and this group can have different characteristics tha