David W. Biers / Richard T. Barker
Abstract: The purpose of the study was to evaluate the effect of laboratory environment, user self-consciousness, and user experience on the user's subjective evaluation of software usability. The study employed a 2 X 2 X 2 factorial between-subjects design with 2 levels of Laboratory Environment (Cameras and Mirror vs. No Cameras and Mirrors), 2 levels of User Self-Consciousness (Low vs. High), and 2 levels of User Experience (Novice, Experienced). The users were asked to learn, then use, and finally subjectively evaluate a restricted subset of common word processing features over three hours of participation. Day 1 was a training day and Day 2 was a test day. Results indicated that high self-conscious and novice users make more word processing errors. However, they were no more likely to make those errors in the presence of cameras and a mirror. More importantly, the evidence for any effect of the independent variables on subjective evaluation was sparse -- limited to the interaction of self-consciousness and laboratory environment on just three of twelve factors. Moreover, the patter of these interactions indicated that self-consciousness and the laboratory environment did not influence subjective evaluation in any predictable manner. Despite some methodological shortcomings, the conclusion was drawn that these variables do not have a major impact on subjective evaluation of software usability.
Keywords: Evaluation, Empirical studies, Usability testing, Laboratory testing methodology, Self-consciousness
Note: Originally published in Proceedings of the Human Factors and Ergonomics Society 38th Annual Meeting, 1994, pp. 1131-1134, (online access).
Republished: G. Perlman, G. K. Green & M. S. Wogalter (Eds) Human Factors Perspectives on Human-Computer Interaction: Selections from Proceedings of Human Factors and Ergonomics Society Annual Meetings, 1983-1994, Santa Monica, California: HFES, 1995, pp. 369-372.