Abstract In food diet communication domain, images convey important information to capture users’ attention beyond the traditional ingredient content, making it crucial to influence user-decision about the relevancy of a given diet. By using a deep learning-based image clustering method, this paper proposes an Explainable Food Recommendation system that uses the visual content of food to justify their recommendations. n the recommendation system. Especially, a new similarity score based on a tendency measure that quantifies the extent to which user community prefers a given food category is introduced and incorporated in the recommendation. Finally, a rule-based explainability is introduced to enhance transparency and interpretability of the recommendation outcome. Our experiments on a crawled dataset showed that the proposed method enhances recommendation quality in terms of precision, recall, F1, and Normalized Discounted Cumulative Gain (NDCG) by 7.35%, 6.70%, 7.32% and 14.38%, respectively, when compared to other existing methodologies for food recommendation. Besides ablation study is performed to demonstrate the technical soundness of the various components of our recommendation system.