This paper reports ongoing research for the definition of a data-driven self-healing system using machine learning (ML) techniques that can perform automatic and timely detection of fault types and locations. Specifically, the proposed method makes use of spectrogram-based CNN modeling of the 3-phase voltage signals. Furthermore, to keep human operators informed about why certain decisions were made, i.e., to facilitate the interpretability of the black-box ML model, we propose a novel explanation approach that highlight regions in the input spectrogram that contributed the most for the prediction task at hand (e.g., fault type or location) - or visual explanation.