Paper Review: A Divisive Information-Theoretic Feature Clustering

A Divisive Information-Theoretic Feature Clustering Algorithm for Text Classification
Author: Inderjit Dhillon

Overivew
Text Classification of high-d data using SVMs is challenging. They propose using feature clustering instead. They propose a global criterion for feature clustering and an algorith, for clustering based on this objective function. Their experiments contrast their approach with SVM and Naive Bayes.

Interesting points/concepts

1. Claim - dimensionality of 14,538 can be a severe obstacle for classification algorithms based on SVMs, LDAs, k-nearest neighbors
2. "One can reduce dimensionality by the distributional clustering of words and features. Each word cluster can then be teated as a single feature and dimensionality can be drastically reduced. Feature Clustering is more effective than feature selection. "
1. Feature Clustering? is better than feature selection for reducing dimensionality
   
3. "Extend any classifier to perform hierarchical classification by constructing a (distinct) classifier at each internal node of the tree using all the documents in its child nodes as the training data. Thus the tree is assumed to be “is-a” hierarchy, i.e., the training instances are inherited by the parents."
4. "Hierarchical classification along with feature selection has been shown to achieve better
   classification results than a flat classifier"

Comments

1. 
   

References

1.