There has been a strong intuition in the Machine Learning community that interpretability and causality ought to have a strong connection. However, the community has not arrived at consensus about how to formalize this connection. In this talk, I will raise questions about conceptual and technical ambiguities that I think make this connection hard to specify. The goal of the talk is to raise points for discussion, expressed in causal formalism, rather than to provide answers.