Manifold learning for data-driven dynamical system analysis

High-dimensional signals generated by dynamical systems arise in many fields of science. For example, many biomedical signals can be modeled by a few latent physiologically related variables measured indirectly through a large set of noisy sensors. In such applications, a notable challenge in analyzing and processing the observed data is that the generating system is typically unknown. In this chapter, this problem is addressed through geometric analysis by applying manifold learning. Specifically, we show how using manifold learning in a purely data-driven manner, with minimal prior knowledge or model assumptions, we can both discover the hidden state, dynamics, and observation function, and also attain a compact linear description of the full system. The main assumption is that the accessible high- dimensional data (the observations of the system) lie on an underlying nonlinear manifold of lower dimensions. Furthermore, when applying manifold learning to time series, critical information is typically overlooked. Time series are processed as data sets of samples, ignoring their embodied dynamics and temporal order. We address the challenge of incorporating the time dependencies into manifold learning and present a purely data-driven scheme. First, an intrinsic representation is derived without prior knowledge of the system by applying diffusion maps. Second, we show that even for highly nonlinear systems, the dynamics of the constructed representation is approximately linear, and present accordingly two filtering frameworks, one based on a linear observer and another based on the Kalman filter. These filtering methods enable us to directly incorporate the inherent dynamics and time dependencies between consecutive system observations into the diffusion maps coordinates. We show that this approach is analogous to Koopman spectral analysis, since applying diffusion maps to the given measurements generates a parametrization of the state space in a known analytic form of the dynamics with a linear drift. We demonstrate the benefits of the approach on simulated data and on real recordings from various applications.