Before defining interpretability, we first need to understand why we are even thinking about studying interpretability of ML models? Well in today’s era, Machine learning models are deployed in every sector including health sector, banking, Decision making at Judicial level as well medicine. The decisions in all these sectors are crucial.
 

• Lets take an example of a situation where the doctor suggests some medicine to a patient on the basis of it’s effectiveness predicted by a Complex DNN model against a disease. But the doctor has no idea about how does the model came up with that certain prediction. That prediction affects the life of a person! which arises the need to comprehend that complex modal.

• Another example is that of a self driving car accidents, why does the DNN model predicted to turn in the direction where the possibility of accident was high? What features of the image collected by the car influenced this wrong prediction? To be able to answer such questions, we need to better understand how exactly does our model works!

• What if the model used for predicting loan approvals is biased? Is it taking into account the colour, caste or the gender of the applicants? How will we even know that?
   

Although there are many models where the explanation is not necessarily needed (eg: movie recommendor systems ) because there are almost no consequences for their incorrect predictions and such models were also studied and validated extensively, so they can trusted but the fields where the predictions are crucial and affects high stake settings, interpretability becomes essential.