Heuristic Based Evaluations

Learn how to conduct heuristic-based evaluations.

We'll cover the following

Introduction

Heuristic-based evaluations are one type of evaluation performed by experts. They use rules of thumb or heuristic guidelines to evaluate a design. Nielsen Molich has described ten heuristics that can help evaluators. These heuristics can either be used as is or can be modified according to the application specifications.

Nielsen’s ten heuristics

Nielson’s ten heuristics are:

  1. Visible system state
    The system should inform the user about its current state through proper instructions and feedback. To the right is an example of an iPhone’s delete media operation. Can’t understand what’s happening here? Only one image is selected but the system shows 142 photos selected. Here is what happened. 142 images were selected and the delete button was pressed. The system should have been updated to “deleting 142 selected photos” but the system updated to the state shown in this image. As you can see, the system does not provide correct feedback. Plus, it does not update its state until all selected media has been deleted.
iPhone's indefinite system state

  1. Mapping between system actions and the real world
    The system should communicate using words and concepts similar to what users use in real-world communication. For example, the command-line interface is a difficult interface to understand and use because it uses more system-oriented conventions rather than real-world terms. That’s why copying a file in the WIMP interface is easier than in the command-line interface.

  2. Offer user control and freedom
    The system should allow users to control their actions. For example, if a user mistakenly selects the wrong actions, the system should provide easy one-step undo and redo options to reverse the error.

  3. Consistency and standards
    The system should have a consistent design. The words and actions being used should mean the same thing in different contexts. The information across multiple screens should be consistent. Also, the system should follow guideline conventions and standards for a more consistent design across platforms. For example, all iOS devices have a similar design but are completely different from all Windows devices. Here is an example of inconsistent information on an ecommerce website. Have a look at these two images. On the search results page, the product is on sale. But when you open the project page, there is no discount.

  1. Prevent errors
    The system should warn and prevent users from making errors. Good error and warning messages should be displayed. A good design provides easy reversal of errors but a better design prevents the errors from happening. For example, always have the system ask the user before closing an unsaved document.
  1. Recognition rather than recall
    The system should provide proper instructions required to understand actions and provide proper feedback after the action has been performed. For example, take a look at the image we saw in the quiz. First, the error message does not clearly state which field produced the error. Secondly, only the field that produced the error should have been updated to an empty field rather than the whole form.
System requires more recall than recognition
System requires more recall than recognition
  1. Flexible and efficient to use
    The system should provide multiple ways to perform actions. This helps both novice and expert users. For example, a novice user can go through a basic and simple way of selecting and copying some text in a text editor. On the other hand, experts can use accelerators, i.e. shortcuts to select and copy the text.
  1. Minimal design
    The system design should be minimal, focusing mainly on the important information. Irrelevant and extra information can distract the user from the main task. Here is an example of Wikipedia’s signup page. The form on the left has fields overloaded with information. Look at the form on the right. Does not it look better, easier to read, and more understandable?

  1. Provide help to recognize and recover from an error
    The system should display errors in human-understandable language, rather than display some error code, to help the user better understand, recognize, and diagnose the error. The system should also provide an easy step-by-step guide to recover from the error.

  2. Provide help through documentation
    The system should provide detailed instructions and guidance in the form of well-explained documentation to help the user perform tasks that cannot be performed without instructions. For example, setups always have a detailed guide with them and require the least possible user input.

How to perform a heuristic evaluation?

These are some steps that experts are expected to perform during heuristic evaluation:

  1. Clearly define what needs to be evaluated. Is it the whole product or a few functions of the product?

  2. Select three to five evaluators that have expertise in usability and relevant fields.

  3. Define heuristics. Depending on system specifications, Nielsen’s heuristics could be used directly, could be modified, or a completely different set of heuristics could be developed.

  4. Define the severity scale or rating. The following factors contribute to define the severity of a usability problem:

    • How often can the problem occur?
    • Will the user easily overcome this problem?
    • Will it be a persistent problem?
    • Will it be perceived easily?

  5. Based on these factors, a scale of severity can be described. The lowest severity is a problem that does not affect usability and the highest severity is a problem that is catastrophic in nature and must be fixed before product release.

  6. In the first walkthrough, evaluators go through each function, using them freely to get familiarized with the system and to identify the elements that need to be analyzed.

  7. In the second walkthrough, evaluators analyze elements critically based on the heuristics and document any issue they encountered.

Sample evaluation process

Let’s take a look at an evaluation process an expert might go through while evaluating an application. One thing to keep in mind is that heuristic evaluation checks the usability of the system and does not focus on detailed design. In the slides below, you can see a screen in each slide and some comments made by the evaluator. The evaluator will write down all the problems/findings in a form under the respective heuristic rule.