Optimal Hierarchical Structure Design of Decision Tree SVM Using Distance Based Approach

Abstract

In literature multi-class SVM is constructed using One against All, One against One and Decision tree based SVM using Euclidean and Mahalanobis distance. To maintain high generalization ability, the most separable classes should be separated at the upper nodes of decision tree. In this paper, A deterministic quantitative model based on distance based approach (DBA) method has been developed and applied for evaluation, optimal selection SVM model for the first time. DBA recognizes the need for relative importance of criteria for a given application, without which inter-criterion comparison could not be accomplished. It requires a set of model selection criteria like information gain, gini index, chi-squared, chernoff-bound, kullbak divergence and scatter-matrix-based class separability in kernel-induced space, along with a set of SVM Models and their level of criteria for optimal selection, and successfully presents the results in terms of a merit value which is used to rank the SVM models. One real dataset from distinct published papers have been used for demonstration of DBA method. The result of this study will be a selection of SVM Model at the root node of decision tree One Versus One (OvO) SVM based on the Euclidean composite distance of each alternative to the designated optimal SVM Model. It is shown that the Optimal Decision Tree (ODT) SVM requires less computation time in comparison to conventional One against All SVM. Experimental results on UCI repository dataset demonstrates better or equivalent performance of our proposed decision tree scheme in comparison to conventional One versus One (OvO) SVM in terms of classification accuracy for most of the datasets. The proposed scheme outperforms conventional One versus One SVM in terms of computation time for both training and testing phase using DBA approach employed for determining the structure of decision tree.