A similarity-based approach to prediction

Assume we are asked to predict a real-valued variable y_t based on certain characteristics x_t=(x_t¹,...,x _t^d), and on a database consisting of (x_i ¹,...,x_i^d,^yi) for i=1,...,n. Analogical reasoning suggests to combine past observations of x and y with the current values of x to generate an assessment of y by similarity-weighted averaging. Specifically, the predicted value of y, yts, is the weighted average of all previously observed values y_i, where the weight of y _i, for every i=1,...,n, is the similarity between the vector x _t¹,...,x_t^d, associated with y _t, and the previously observed vector, x_i ¹,...,x_i^d. The "empirical similarity" approach suggests estimation of the similarity function from past data. We discuss this approach as a statistical method of prediction, study its relationship to the statistical literature, and extend it to the estimation of probabilities and of density functions.