Responsible AI Practices: Interpretability
In addition to being fair, AI should be understandable.
We'll cover the following...
Interpretability
AI systems should be understandable.
We touched upon this topic when talking about creating AI solutions that users trust. Interpretability, or explainability, plays a crucial role when it comes to trusting AI systems.
Automated predictions and decision making are becoming integrated into our lives more and more. In some scenarios, it can be acceptable to have AI systems that make decisions we do not fully understand. For example, if a healthcare application classifies an image as cancerous, the patient wouldn’t likely be interested in knowing how the algorithm is able to make those predictions; whether the deep learning model was trained using biomarker levels or skin color of the patient, wouldn’t be their primary concern. However, if there is a loan application that denies legit requests or a surveillance system that shows bias in classifying people, then in order to establish the human-AI trust, we want to have a better understanding of how ...