Information loss of the Mahalanobis distance in high dimensions: Matlab implementation 1.0

The Mahalanobis distance between a pattern measurement vector of dimensionality D and the center of the class it belongs to is distributed as a chi^2 with D degrees of freedom, when an infinite training set is used. However, the distribution of Mahalanobis distance becomes either Fisher or Beta depending on whether cross-validation or re-substitution is used for parameter estimation in finite training sets. The total variation between chi^2 and Fisher as well as between chi^2 and Beta allows us to measure the information loss in high dimensions. The information loss is exploited then to set a lower limit for the correct classification rate achieved by the Bayes classifier that is used in subset feature selection.Installation:-------------The 5 functions should be in the current path of Matlab.Usage:------LowCCRLimit = LowCCRLimitInfLoss(D, CCR, NDc, CClasses, ErrorEstMethod)% D: Dimensionality of the vector (2,3,4,5,...)% CCR: The Correct Classification rate in [1/CClasses,1] (e.g. 0.8)% NDc: The number of training samples per class (>D+1)% CClasses: The number of classes in your problem (2,3,4,...)% ErrorEstMethod: "Resub" for resubstitution% "Cross" for cross-validationExample:--------LowCCRLimitInfLoss(5, 0.75, 100, 5, 'Cross')ans = 0.7288References:-----------[1] Dimitrios Ververidis and Constantine Kotropoulos, "Information loss of the Mahalanobis distance in high dimensions: Application to feature selection,"IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 31, no. 12, pp. 2275-2281, 2009.[2] Jeffrey, Knuth, "On the Lambert W Function", Advances in Computational Mathematics, volume 5, 1996, pp. 329-359.Special thanks to Dr. Pascal Getreuer for implementing the lambertw2 function from Jeffrey Knuth publication.

Requirements:	No special requirements
Platforms:	Matlab
Keyword:	Application,  Cross,  Dimitrios,  Kotropoulos,  Lowccrlimitinfloss,  Quotinformation,  References,  Ververidis
Users rating:	0/10