Learning Document Graphs with Attention for Image Manipulation Detection

Hailey James, Otkrist Gupta, Dan Raviv

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

Detecting manipulations in images is becoming increasingly important for combating misinformation and forgery. While recent advances in computer vision have lead to improved methods for detecting spliced images, most state-of-the-art methods fail when applied to images containing mostly text, such as images of documents. We propose a deep-learning method for detecting manipulations in images of documents which leverages the unique structured nature of these images in comparison with those of natural scenes. Specifically, we re-frame the classic image splice detection problem as a node classification problem, in which Optical Character Recognition (OCR) bounding boxes form nodes and edges are added according to a text-specific distance heuristic. We propose a Variational Autoencoder (VAE)-based embedding algorithm, which when combined with a graph neural network with attention, outperforms both a state-of-the-art image splice detection method and a document-specific method.

Original languageEnglish
Title of host publicationPattern Recognition and Artificial Intelligence - 3rd International Conference, ICPRAI 2022, Proceedings
EditorsMounîm El Yacoubi, Eric Granger, Pong Chi Yuen, Umapada Pal, Nicole Vincent
PublisherSpringer Science and Business Media Deutschland GmbH
Pages263-274
Number of pages12
ISBN (Print)9783031090363
DOIs
StatePublished - 2022
Externally publishedYes
Event3rd International Conference on Pattern Recognition and Artificial Intelligence, ICPRAI 2022 - Paris, France
Duration: 1 Jun 20223 Jun 2022

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume13363 LNCS

Conference

Conference3rd International Conference on Pattern Recognition and Artificial Intelligence, ICPRAI 2022
Country/TerritoryFrance
CityParis
Period1/06/223/06/22

Keywords

  • Graph Neural Networks
  • Manipulation detection
  • Variational auto-encoders

All Science Journal Classification (ASJC) codes

  • Theoretical Computer Science
  • General Computer Science

Fingerprint

Dive into the research topics of 'Learning Document Graphs with Attention for Image Manipulation Detection'. Together they form a unique fingerprint.

Cite this