On the statistical learning ability of evolution strategies

We explore the ability of Evolution Strategies (ESs) to statistically learn the local landscape. Specifically, we consider ESs operating only with isotropic Gaussian mutations near the optimum and investigate the covariance matrix when constructed out of selected individuals by truncation. Unlike previous studies, we do not assume a Derandomization adaptation scheme, nor do we use Information Geometric Optimization in our proofs. We prove that the statistically constructed covariance matrix over such selected decision vectors has the same eigenvectors as the Hessian matrix. We further prove that when the population size is increased, the covariance becomes proportional to the inverse of the Hessian. We also devise and corroborate an analytic approximation of this covariance matrix. In the framework we consider, this confirms the classical hypothesis that learning the landscape is an inherent property of standard ESs, and that this capability stems only from the usage of isotropic Gaussian mutations and rank-based selection.