Latent-KalmanNet: Learned Kalman Filtering for Tracking From High-Dimensional Signals

Itay Buchnik, Guy Revach, Damiano Steger, Ruud J.G. Van Sloun, Tirza Routtenberg, Nir Shlezinger

Research output: Contribution to journalArticlepeer-review


The Kalman filter (KF) is a widely used algorithm for tracking dynamic systems that are captured by state space (SS) models. The need to fully describe an SS model limits its applicability under complex settings, e.g., when tracking based on visual data, and the processing of high-dimensional signals often induces notable latency. These challenges can be treated by mapping the measurements into latent features obeying some postulated closed-form SS model, and applying the KF in the latent space. However, the validity of this approximated SS model may constitute a limiting factor. In this work, we study tracking from high-dimensional measurements under complex settings using a hybrid model-based/data-driven approach. By gradually tackling the challenges in handling the measurement model and the task, we develop Latent-KalmanNet, which implements tracking from high-dimensional measurements by leveraging data to jointly learn the KF along with the latent space mapping. Latent-KalmanNet combines a learned encoder with data-driven tracking in the latent space using the recently proposed-KalmanNet, while identifying the ability of each of these trainable modules to assist its counterpart via providing a suitable prior (by KalmanNet) and by learning a latent representation that facilitates data-aided tracking (by the encoder). Our empirical results demonstrate that the proposed Latent-KalmanNet achieves improved accuracy and run-time performance over both model-based and data-driven techniques by learning a surrogate latent representation that most facilitates tracking, while operating with limited complexity and latency.

Original languageAmerican English
Pages (from-to)352-367
Number of pages16
JournalIEEE Transactions on Signal Processing
StatePublished - 1 Jan 2024


  • Data-aided tracking
  • high-dimensional measurements
  • Kalman filters
  • latent space
  • model-based deep-learning
  • model-based deep-learning
  • nonlinear state-space models

All Science Journal Classification (ASJC) codes

  • Signal Processing
  • Electrical and Electronic Engineering


Dive into the research topics of 'Latent-KalmanNet: Learned Kalman Filtering for Tracking From High-Dimensional Signals'. Together they form a unique fingerprint.

Cite this