GAM: Explainable Visual Similarity and Classification via Gradient Activation Maps

Oren Barkan; Omri Armstrong; Amir Hertz; Avi Caciularu; Ori Katz; Itzik Malkiel; Noam Koenigstein

doi:https://doi.org/10.1145/3459637.3482430

GAM: Explainable Visual Similarity and Classification via Gradient Activation Maps

Oren Barkan, Omri Armstrong, Amir Hertz, Avi Caciularu, Ori Katz, Itzik Malkiel, Noam Koenigstein

Bar-Ilan University, Open University of Israel, Tel Aviv University

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

We present Gradient Activation Maps (GAM) - a machinery for explaining predictions made by visual similarity and classification models. By gleaning localized gradient and activation information from multiple network layers, GAM offers improved visual explanations, when compared to existing alternatives. The algorithmic advantages of GAM are explained in detail, and validated empirically, where it is shown that GAM outperforms its alternatives across various tasks and datasets.

Original language	English
Title of host publication	CIKM 2021 - Proceedings of the 30th ACM International Conference on Information and Knowledge Management
Place of Publication	New York, NY, USA
Pages	68-77
Number of pages	10
ISBN (Electronic)	9781450384469
DOIs	https://doi.org/10.1145/3459637.3482430
State	Published - 26 Oct 2021
Event	30th ACM International Conference on Information and Knowledge Management, CIKM 2021 - Virtual, Online, Australia Duration: 1 Nov 2021 → 5 Nov 2021

Publication series

Name	International Conference on Information and Knowledge Management, Proceedings

Conference

Conference	30th ACM International Conference on Information and Knowledge Management, CIKM 2021
Country/Territory	Australia
City	Virtual, Online
Period	1/11/21 → 5/11/21

Keywords

deep learning
explainable & interpretable ai
saliency maps

All Science Journal Classification (ASJC) codes

General Decision Sciences
General Business,Management and Accounting

Access to Document

https://doi.org/10.1145/3459637.3482430

https://doi.org/10.1145/3459637.3482430

Cite this

Barkan, O., Armstrong, O., Hertz, A., Caciularu, A., Katz, O., Malkiel, I., & Koenigstein, N. (2021). GAM: Explainable Visual Similarity and Classification via Gradient Activation Maps. In CIKM 2021 - Proceedings of the 30th ACM International Conference on Information and Knowledge Management (pp. 68-77). (International Conference on Information and Knowledge Management, Proceedings).. https://doi.org/10.1145/3459637.3482430

@inproceedings{1934834aa2734089a9a7b79f902d6ded,

title = "GAM: Explainable Visual Similarity and Classification via Gradient Activation Maps",

abstract = "We present Gradient Activation Maps (GAM) - a machinery for explaining predictions made by visual similarity and classification models. By gleaning localized gradient and activation information from multiple network layers, GAM offers improved visual explanations, when compared to existing alternatives. The algorithmic advantages of GAM are explained in detail, and validated empirically, where it is shown that GAM outperforms its alternatives across various tasks and datasets.",

keywords = "deep learning, explainable & interpretable ai, saliency maps",

author = "Oren Barkan and Omri Armstrong and Amir Hertz and Avi Caciularu and Ori Katz and Itzik Malkiel and Noam Koenigstein",

note = "Publisher Copyright: {\textcopyright} 2021 ACM.; 30th ACM International Conference on Information and Knowledge Management, CIKM 2021 ; Conference date: 01-11-2021 Through 05-11-2021",

year = "2021",

month = oct,

day = "26",

doi = "https://doi.org/10.1145/3459637.3482430",

language = "الإنجليزيّة",

isbn = "9781450384469",

series = "International Conference on Information and Knowledge Management, Proceedings",

pages = "68--77",

booktitle = "CIKM 2021 - Proceedings of the 30th ACM International Conference on Information and Knowledge Management",

}

Barkan, O, Armstrong, O, Hertz, A, Caciularu, A, Katz, O, Malkiel, I & Koenigstein, N 2021, GAM: Explainable Visual Similarity and Classification via Gradient Activation Maps. in CIKM 2021 - Proceedings of the 30th ACM International Conference on Information and Knowledge Management. International Conference on Information and Knowledge Management, Proceedings, New York, NY, USA, pp. 68-77, 30th ACM International Conference on Information and Knowledge Management, CIKM 2021, Virtual, Online, Australia, 1/11/21. https://doi.org/10.1145/3459637.3482430

GAM: Explainable Visual Similarity and Classification via Gradient Activation Maps. / Barkan, Oren; Armstrong, Omri; Hertz, Amir et al.
CIKM 2021 - Proceedings of the 30th ACM International Conference on Information and Knowledge Management. New York, NY, USA, 2021. p. 68-77 (International Conference on Information and Knowledge Management, Proceedings).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - GAM

T2 - 30th ACM International Conference on Information and Knowledge Management, CIKM 2021

AU - Barkan, Oren

AU - Armstrong, Omri

AU - Hertz, Amir

AU - Caciularu, Avi

AU - Katz, Ori

AU - Malkiel, Itzik

AU - Koenigstein, Noam

PY - 2021/10/26

Y1 - 2021/10/26

N2 - We present Gradient Activation Maps (GAM) - a machinery for explaining predictions made by visual similarity and classification models. By gleaning localized gradient and activation information from multiple network layers, GAM offers improved visual explanations, when compared to existing alternatives. The algorithmic advantages of GAM are explained in detail, and validated empirically, where it is shown that GAM outperforms its alternatives across various tasks and datasets.

AB - We present Gradient Activation Maps (GAM) - a machinery for explaining predictions made by visual similarity and classification models. By gleaning localized gradient and activation information from multiple network layers, GAM offers improved visual explanations, when compared to existing alternatives. The algorithmic advantages of GAM are explained in detail, and validated empirically, where it is shown that GAM outperforms its alternatives across various tasks and datasets.

KW - deep learning

KW - explainable & interpretable ai

KW - saliency maps

UR - http://www.scopus.com/inward/record.url?scp=85119175338&partnerID=8YFLogxK

U2 - https://doi.org/10.1145/3459637.3482430

DO - https://doi.org/10.1145/3459637.3482430

M3 - منشور من مؤتمر

SN - 9781450384469

T3 - International Conference on Information and Knowledge Management, Proceedings

SP - 68

EP - 77

BT - CIKM 2021 - Proceedings of the 30th ACM International Conference on Information and Knowledge Management

CY - New York, NY, USA

Y2 - 1 November 2021 through 5 November 2021

ER -

Barkan O, Armstrong O, Hertz A, Caciularu A, Katz O, Malkiel I et al. GAM: Explainable Visual Similarity and Classification via Gradient Activation Maps. In CIKM 2021 - Proceedings of the 30th ACM International Conference on Information and Knowledge Management. New York, NY, USA. 2021. p. 68-77. (International Conference on Information and Knowledge Management, Proceedings). doi: https://doi.org/10.1145/3459637.3482430

GAM: Explainable Visual Similarity and Classification via Gradient Activation Maps

Abstract

Publication series

Conference

Keywords

All Science Journal Classification (ASJC) codes

Access to Document

Other files and links

Fingerprint

Cite this