Model-Based Evaluations

Introduction

Another expert-based evaluation technique is the model-based evaluation. This approach uses cognitive and design models. These models help to identify project specifications along with design evaluation. In this lesson, we are going to study two models: the GOMS model and the Keystroke-level model.

The GOMS model

GOMS stands for Goals, Operations, Methods, and Selection.

• Goals: This describes user goals, i.e. what the user wants to achieve. For example, “Close a tab in Safari,” is a goal.

• Operations: The set of basic actions that users can perform to interact with the system and achieve a goal. For example, “press key X,” and “mouse click,” are operations.

• Methods: These are the different ways of achieving the same goal. Method X will have a different set of subgoals from method Y. Both methods can be used to achieve the goal. For example, a tab in Safari can be closed either by selecting the “close tab” option from the menu, by clicking the “cross icon” on the tab, or by using the key shortcut “command+W”. There are three methods to achieve this goal.

• Selection: These are the conditions and situations that predict which method will be used. Selection basically depends on three things:

  • Particular users
  • System state
  • Goal details

  For example, an expert user will prefer either the “cross icon” method or the “command+W” shortcut. In contrast, a novice user may not know about the shortcut and chooses to close the tab using the menu option or clicking the “cross icon.”

GOMS example

Here is an example of a task to move the word “cute” before the word “baby” in the following sentence.

“What a baby she is cute!”

The Keystroke-level model

The Keystroke-level model, abbreviated as KLM, is a model that predicts user performance on the basis of a deep understanding of the human motor system. It focuses on the time taken by the user to perform basic unit tasks of simple functions. For example, it can be used to evaluate individual unit tasks performed for functions like changing font size, searching and replacing. Complex functions such as making a graph or a diagram will be divided into subtasks with subgoals just like in the GOMS model and then these subtasks can be evaluated using KLM. KLM assumes that the user has decided and established how to perform a task in the mind before initiating actions. It divides the task execution into seven operators. Out of all of them, one is a mental operator, five are physical operators, and one is a system response operator. The mental operator is a slight recall of the action that the user is going to perform. These operators are:

Example

Consider a task to close a tab in Safari. The user will point at the “cross sign” on the tab and click it to close the tab. This task can be divided into KLM operators as follows:

The operators have some explanations as well in the square brackets. For example, homing the mouse or keyboard, which mouse key is pressed, and which keyboard key is pressed in case of a K operator, etc.

Calculating time

The KLM represents the time taken by each operator as $T_o$ where $o$ is the specific operator. If there are multiple occurrences of an operator, for example, multiple keystrokes, then time $t_K$ taken by each keystroke is summed to calculate $T_K$.
The total time taken by the user in the above example can be calculated as:

$T_{execute} = T_k + T_B + T_P + T_H + T_D + T_M + T_R$

$T_{execute} = 0 + t_B + t_P + t_H + 0 + t_M + 0$

$T_{execute} = t_B + t_P + t_H + t_M$

Have a look at this table. These are the common and widely accepted times for each operator.