Demystifying Explainability Themes for Black Box Models

Intrinsic vs. post hoc explainability

Explainability deals with understanding why a model makes the prediction that it does. There are multiple factors that influence the final model decision, and explainability can be applied at different stages in the AI life cycle.

Post hoc and intrinsic explanations are two different approaches to understanding and explaining model decisions.

• Explanation as a built-in feature: Intrinsic explainability refers to the property of a machine learning model itself to provide interpretable and understandable results. In other words, it’s the ability of the model to inherently produce explanations as part of its predictions.

• Explanation generated after model training: Post hoc explainability, on the other hand, involves generating explanations for machine learning models after they have been trained. These explanations are not inherently produced by the model but are generated externally using various techniques. ...