A Logic-based Approach to Contrastive Explainability for Neurosymbolic Visual Question Answering
2023 (English)In: IJCAI International Joint Conference on Artificial Intelligence, International Joint Conferences on Artificial Intelligence , 2023, p. 3668-3676Conference paper, Published paper (Refereed)
Abstract [en]
Visual Question Answering (VQA) is a well-known problem for which deep-learning is key. This poses a challenge for explaining answers to questions, the more if advanced notions like contrastive explanations (CEs) should be provided. The latter explain why an answer has been reached in contrast to a different one and are attractive as they focus on reasons necessary to flip a query answer. We present a CE framework for VQA that uses a neurosymbolic VQA architecture which disentangles perception from reasoning. Once the reasoning part is provided as logical theory, we use answer-set programming, in which CE generation can be framed as an abduction problem. We validate our approach on the CLEVR dataset, which we extend by more sophisticated questions to further demonstrate the robustness of the modular architecture. While we achieve top performance compared to related approaches, we can also produce CEs for explanation, model debugging, and validation tasks, showing the versatility of the declarative approach to reasoning.
Place, publisher, year, edition, pages
International Joint Conferences on Artificial Intelligence , 2023. p. 3668-3676
Keywords [en]
Logic programming, Query processing, Answer set programming, Logic-based approach, Logical theories, Modular architectures, Performance, Question Answering, Deep learning
National Category
Computer Sciences
Identifiers
URN: urn:nbn:se:hj:diva-63554DOI: 10.24963/ijcai.2023/408Scopus ID: 2-s2.0-85170397066ISBN: 9781956792034 (print)OAI: oai:DiVA.org:hj-63554DiVA, id: diva2:1838581
Conference
32nd International Joint Conference on Artificial Intelligence, IJCAI 2023, 19 August-25 August 2023
2024-02-162024-02-162024-02-16Bibliographically approved