 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
 |
 |
 |
|
|||||||||||||||||
 |
|
|||||||||||||||||||
 |
 |
 |
 |
 |
 |
 |
 |
 |
|
|||||||||||
 |
 |
|
||||||||||||||||||
 |
 |
|
||||||||||||||||||
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
|
||||||||||
 |
 |
 |
|
|||||||||||||||||
 |
 |
|
||||||||||||||||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|||||||||||||||||
|
|
|||||||||||||||||||
|
|
|
|
|
|
|
|
|
|
|||||||||||
|
|
|
||||||||||||||||||
|
|
|
||||||||||||||||||
|
|
|
|
|
|
|
|
|
|
|
||||||||||
|
|
|
|
|||||||||||||||||
|
|
|
||||||||||||||||||
Here we list brief descriptions of experiments we've conducted and what aspects are of interest, particularly with respect to statistical evaluation. In all cases, there is a paper with additional detail referenced.
For a complete listing papers by us relating to Evaluation of Face Recognition Algorithms, see Papers.
Code is available to assist in replicating some of these experiments - see the algorithms page.
A paper developing a common nomenclature for describing performance analysis experiments with face recognition algorithms has been written for the "Empirical Evaluation Methods in Computer Vision" workshop being held in conjunction with CVPR 2001. A draft of this paper, titled "Parametric and Nonparametric Methods for the Statistical Evaluation of Human ID Algorithms" is available. This paper describes both parametric and nonparametric methods for estimating uncertainty in performance metrics. such as recognition rate.
This experiment illustrates how a modern non-parametric technique may be used to directly estimate the probability distribution of a performance metric, in this case recognition rate. In this paper, the PCA algorithm is compared to the PCA+LDA algorithm on imagery for 160 people, four images per person. PCA algorithm variants using L1, L2, Angle and Mahalanobis distance measures were considered. PCA+LDA variants using L1, L2 and Angle were considered. PCA alone using Mahalanobis did best on these tests, but it can be fairly argued that the training provided to the LDA algorithm was inadequate. A draft of this paper, titled "A Nonparametric Statistical Comparison of Principal Component and Linear Discriminant Subspaces for Face Recognition" is available.
This experiment compares PCA to ICA (Independent Component Analysis) and concludes the PCA algorithm performs better. The statistical evaluation in this paper is based upon a parametric model of testing as independent sampling of probe images. Algorithms are scored as either succeeding or failing to recognize the probe image. This model gives rise to a binomially distributed success count and pairwise comparisons between algorithms are made using McNemar's test. A paper, titled, "PCA vs. ICA: A Comparison on the FERET data set" is available. The use of background or McNemar's test is covered in more detail in an earlier paper, "Analyzing PCA-based Face Recognition Algorithms: Eigenvector Selection and Distance Measures" available through our papers page.
Last Revised on