People and Computers XXII
Culture, Creativity, Interaction

Proceedings of HCI 2008
The 22nd British HCI Group Annual Conference
Liverpool John Moores University, UK

1 - 5 September 2008

AUTHORS

Martin Schmettow

ABSTRACT

Current prediction models for usability evaluations are based on stochastic distributions derived from series of Bernoulli processes. The underlying assumption of these models is a homogeneous detection probability despite of it being intuitively unrealistic. This paper contributes a simple statistical test for existence of heterogeneity in the process. The compound beta-binomial model is proposed to incorporate sources of heterogeneity and compared to the binomial model. Analysis of several data sets from the literature illustrates the methods and reveals that heterogeneity occurs in most situations. Finally, it is demonstrated how heterogeneity biases the prediction of evaluation processes. Open research questions are discussed and preliminary advice for practitioners for controlling their processes is given.

PAPER FORMATS

PDF Version of this Paper (595kb)