Abstract: Image set matching attracts increasing attention in the field of pattern recognition. A convenient way of dealing with image sets is to represent them as points on manifolds. We naturally formulate the problem of image set matching as matching points lying on the Riemannian manifold spanned by covariance matrices. We derive a kernel function that explicitly maps the covariance matrix from the Riemannian manifold to a Euclidean space. With the explicit mapping, a kernel version of linear discriminant analysis is applied to classify the image sets. The proposed method is evaluated on set-based object classification tasks. Extensive experimental results show that the proposed method outperforms other state of the art set-based matching methods in the public database.

Key words: manifold, set matching, discriminant analysis, pattern recognition, object recognition