Unlike business and e-commerce sites, web applications don't necessarily have an end-goal or target for users to achieve. What are some metrics that can be used to measure how 'engaged' users are with the application? I was thinking if there are known ways for linking together user actions (e.g. button clicks, menu clicks, field input, etc.) or if there are some ways to infer this (e.g. time spent per screen). I believe this is better than putting social media links or provide forums for users to find out this type of information, so I would like to try adding this to the web application.
A quick research on the literature in this area includes some frameworks like the UES, HEARTS and UMUX. However, these describe general concepts and processes behind measuring user experience/engagement but does not mention exactly what is physically measured. It would be good to try and understand what is measured and how it might be relevant to user experience.
Answer
Below are the 6 factors used in the O'Brien/Toms User Engagement Scale.
1) Perceived Usability: users' perceived effort, their ability to accomplish tasks, the navigation and organization, and the emotions evoked in user.
2) Aesthetics: screen layout and graphics/images, overall aesthetic impressions of attractiveness, and sensory appeal.
3) Focused Attention: absorption (ability to become absorbed and lose oneself in the experience), awareness (about what was taking place outside of the interaction), and perceptions of time passing.
4) Endurability: the likelihood to remember things that we have enjoyed and a desire to do again an activity that has been fun.
5) Novelty: the curiosity evoked by the task and the participants' interest in the task.
6) Felt Involvement: feeling of being drawn into and involved in the task and the overall assessment of the experience as fun.
As you can see, these are things that require you to get feedback from the user. Measuring a complex (qualitative) construct like engagement purely through a pattern of user actions appears problematic.
References:
O’Brien, H.L. & Toms, E.G. (2010). The development and evaluation of a survey to measure user engagement in e-commerce environments. Journal of the American Society for Information Science & Technology, 61(1), 50-69. DOI: 10.1002/asi.v61:1.
O’Brien, H.L. & Toms, E.G. (2008). What is user engagement? A conceptual framework for defining user engagement with technology. Journal of the American Society for Information Science & Technology, 59(6), 938-955., DOI: 10.1002/asi.v59:6.
No comments:
Post a Comment