在多類別分類的問題中，AdaBoost.M1演算法扮演了相當成功的角色，並在準確度方面有著不錯的提升效果。然而，此演算法假定所有基礎分類器於準確度方面都必須大於1/2，這在多類別分類的問題中是難以達到的。因此，我們提出一個新的演算法- AdaBoost.MK，它只要求在多類別分類問題中，每個分類器的正確率需比隨機亂猜的機率(1/k)還要高即可。
早期支援向量機在處理多類別資料分類問題時，都將它分解成數個兩類別的問題再加以解決，如此一來將需要花費大量的時間以及空間計算。為了減少時間與空間上的花費，我們提出了一個結合數個單類別支援向量機(one-class svm)以及判別函式(discriminant function)的方法來解決多類別資料分類的問題。
在本論文中，我們利用整合數個單類別支援向量機以及判別函式的方法並與AdaBoost.MK做結合來解決多類別資料分類的問題。實驗中，我們將使用來自UCI 和Statlog的真實資料，並與許多多類別分類的演算法做比較，包括利用支援向量群聚(support vector clustering)的方法與AdaBoost.M1結合單類別支援向量機的方法。

AdaBoost.M1 has been successfully applied to improve the accuracy of a learning algorithm for multi-class classification problems. However, it assumes that the performance of each base classifier must be better than 1/2, and this may be hard to achieve in practice for a multi-class problem. A new algorithm called AdaBoost.MK only requiring base classifiers better than a random guessing (1/k) is thus designed.
Early SVM-based multi-class classification algorithms work by splitting the original problem into a set of two-class sub-problems. The time and space required by these algorithms are very demanding. In order to have low time and space complexities, we develop a base classifier that integrates one-class SVMs with discriminant functions.
In this study, a hybrid method that integrates AdaBoost.MK and one-class SVMs with improved discriminant functions as the base classifiers is proposed to solve a multi-class classification problem. Experimental results on data sets from UCI and Statlog show that the proposed approach outperforms many popular multi-class algorithms including support vector clustering and AdaBoost.M1 with one-class SVMs as the base classifiers.

摘要 i
Abstract ii
第一章 簡介 1
第二章 文獻探討 6
2.1 核心函式 6
2.2單類別支援向量機 10
2.3支援向量群聚 15
2.4支援向量機應用於多類別分類 18
第三章 研究方法 - KOCSVM & Adaboost.MK 22
3.1研究動機 22
3.2研究方法 - KOCSVM 27
3.2.1方法流程 27
3.2.2判別函式 31
3.3 Boosting演算法 34
3.3.1 Adaboost.M1演算法 35
3.3.2 Adaboost.MK演算法 38
第四章 實驗結果與分析 43
4.1資料集介紹 43
4.2實驗一 KOCSVM與KSVDD的比較 44
4.3實驗二 加入Adaboost.M1演算法前後之比較 48
4.4實驗三 Adaboost.MK與Adaboost.M1的比較 51
第五章 結論與未來研究方向 54
5.1 結論 54
5.2 未來研究方向 55
參考文獻 56
附錄一 支援向量機簡介 59
附錄二 序列最小優化簡介 61
附錄三 兩類別的Adaboost演算法 63

[1] K. Goh, E.Y. Chang, and B. Li, “Using One-class and Two-Class
Svms for Multiclass Image Annotation”, IEEE Transaction on
Knowledge and. Data Engineering, Vol. 17, No. 10, pp. 1333-1346,
2005.
[2] C.W. Hsu and C.J. Lin, “A comparison of methods for multi-class
support vector machines”, IEEE Transactions on Neural Networks,
Vol. 13, pp. 415-425, 2002.
[3] R. Rifin and A. Klautau, “In Defense of One-Vs-All Classification”,
Journal of Machine Learning Research, Vol. 5, pp. 101-141, 2004.
[4] Y. Freund and R. Schapire, “A decision-theoretic generalization of
on-line learning and an application to boosting”, Journal of
Computer and System Sciences, Vol. 55, No. 1, pp. 119-139, 1997.
[5] A. Ben-Hur, D. Horn, H.T. Siegelmann, and V. Vapnik. “Support
Vector Clustering”, Journal of Machine Learning Research, Vol. 2,
pp. 125-137, 2001.
[6] B.Y. Sun and D.S Huang, “Support vector clustering for multi-class
classification problems”, The Congress on Evolutionary
Computation, Canberra, Australia, pp.1480-1485, 2003.
[7] L. Manevitz and M. Yousef. “One-Class Svms for Document
Classification”, Journal of Machine Learning Research, Vol. 2, pp.
139-154, 2002.
[8] B. Schölkopf, J. Platt, J. Shawe-Taylor, A.J. Smola, and R.C.
Williamson, “Estimating the Support of a High-Dimensional
Distribution”, Neural Computation, Vol. 13, No. 7, pp. 1443-1472,
2001.
[9] Y. Freund and R. Schapire, “A decision-theoretic generalization of
on-line learning and an application to boosting”, Journal of
Computer and System Sciences, Vol. 55, No. 1, pp. 119-139, 1997.
[10] J. Shawe-Taylor and N. Cristianini, “Kernel Methods for Pattern
Analysis”, Cambridge,UK:Cambridge University Press,2004.
[11] K.-R. Muller, S. Mika, G. Ratsch, K. Tsuda, B. Scholkopf, “An
introduction to kernel-based learning algorithms”, IEEE
Transactions on Neural networks, Vol. 12, No. 2, pp. 181-201,
March 2001.
[12] L. Bottou, C. Cortes, J. Denker, H. Drucker, I. Guyon, L. D. Jackel,
Y. LeCun, U. Muller, E. Sackinger, P. Simard, and V. Vapnik,
“Comparison of classifier methods: a case study in handwritten digit
recognition”, In International Conference on Pattern Recognition,
pp. 77-87, 1994.
[13] J. Friedman, “Another Approach to Polychotomous Classification”,
Technical Report, Department of Statistics, Stanford University,
Stanford, CA. 1996.
[14] U. Kreßel, “Pairwise classification and support vector machines, In
B. Scholkopf, B.J. Burges, and A.J. Smola, editors, Advances in
Kernel Methods - Support Vector Learning”, pp. 255-268,
Cambridge, MA, MIT Press, 1999.
[15] J.C. Platt, N. Cristianini, and J. Shawe-Taylor, “Large margin DAGs
for multiclass classification”, In Advances in Neural Information
Processing System, Vol. 12, pp. 547-553, MIT Press, 2000.
[16] A. Sachs, C. Thiel and F. Schwenker, “One-Class Support-Vector
Machines for the Classification of Bioacoustic Time Series”,
International Journal on Artificial Intelligence and Machine
Learning, Vol. 6, No. 4, pp. 29-34, 2006.
[17] R. Schapire, “Using output codes to boost multiclass learning
problems”, In Proc. of the 14th International Conference on
Machine Learning, 1997.
[18] R. Schapire and Y. Singer, “Improved boosting algorithms using
confidence-rated prediction”, Machine Learning, Vol. 37, No. 1, pp.
297-336, 1999.
[19] L. Li, “Multiclass boosting with repartitioning”, In Proc. 23rd
International Conference on Machine Learning, 2006.
[20] G. Eibl, K.-P. Pfeiffer, “Multiclass Boosting for Weak Classifiers”,
Journal of Machine Learning Research, Vol. 6, No. Feb, pp.
189-210, 2005.
[21] J. Zhu, S. Rosset, H. Zhou, and T. Hastie, “Multiclass adaboost”,
Technical Report #430, Department of Statistics, University of
Michigan, 2005.
[22] C.C.Chang and C.J.Lin. LIBSVM: A Library for Support Vector
Machines. April 1, 2007.
[23] J.C.Platt, “Fast Training of Support Vector Machines using
Sequential Minimal Optimization”, Advances in Kernel Methods
-Support Vector Learning, pp. 185-208 , 1999.
[24] J.C.Platt, “Sequential Minimal Optimization: A Fast Algorithm for
Training Support Vector Machines”, Technical Report
MSR-TR-98-14, Microsoft Research, 1998.