在過去的文獻中證明若能有效利用邊界附近的資料為學習樣本，則可增加學習型分類器的分類效能。然而針對類別間發生重疊的型態分類問題，過去找尋決策邊界的方法往往計算繁雜冗長，或是表現的不理想。本論文透過最近鄰居法則，提出一個新方法以重新描述資料點與決策邊界之間的遠近關係。並將此新方法整合至既有的最近鄰居分類器(CNN)與類神經網路分類器(RLS-OI)中，以驗證其可行性。經過實驗證明整合此新方法的分類器可以有效減少神經元或原型用量且擁有較高的分類精度。

It has been shown that focusing the training algorithms to the decision boundary vicinity data can improve the accuracy of several classification methods. However, previous approaches for fining decision boundary vicinity data are either computationally tedious or may perform poorly in handling problems with class overlapping. With the application of the nearest neighbor rule, this work proposes a new criterion to characterize the nearness of the training samples to the decision boundary. To demonstrate the effectiveness of the proposed approach, the proposed method is integrated with a nearest neighbor classifier design method and a neural work training approach. Experimental results show that the proposed method can reduce the size and classification error for both of the tested classifiers.

第一章 緒論 1
1.1 前言 1
1.2 決策邊界附近資料在型態分類問題上的重要性 2
1.3 研究動機 3
1.4 論文架構 3
第二章 過去找尋決策邊界區域的方法－交互鄰居值 5
2.1（最小）交互鄰居值演算法 5
2.2 交互鄰居值在應用上的不足 10
第三章 找尋決策邊界的新方法 12
3.1 重新思考決策邊界附近之資料點特性 12
3.2 以k最近鄰居投票差異值為前處理程序的資料點排序法 14
3.2.1 資料點輸入空間與機率密度空間的關係 14
3.2.2 k最近鄰居投票差異值排序法則 16
3.3 k最近鄰居投票差異值排序法則的修正方案 20
3.3.1 k-NNVD排序法則於未重疊區域的先天弱勢與調整 20
3.3.2 k-NNVD排序法則的隨機性與調整 26
3.4 能有效利用決策邊界附近資料的資料點前處理程序 28
第四章 實驗與結果 30
4.1 待測的類神經網路分類器 30
4.1.1 濃縮式最近鄰居分類器(Condensed-NN Rule) 30
4.1.1.1 最近鄰居分類器的優缺點 30
4.1.1.2 濃縮式最近鄰居分類器 31
4.1.1.3 濃縮式最近鄰居分類器的處理程序 31
4.1.1.4 CNN應用於本論文的修改 33
4.1.2 最佳內插網路(Optimal Interpolative net) 34
4.1.2.1 RLS-OI的優點 34
4.1.2.2 RLS-OI的訓練步驟 35
4.1.2.3 RLS-OI的訓練精神 36
4.1.2.4 RLS-OI應用於本論文的修改 39
4.2 型態分類問題範例 41
4.2.1 人為虛擬的型態分類問題 41
4.2.1.1類別間未重疊的型態分類問題 41
4.2.1.2 類別間重疊的型態分類問題 42
4.2.2 真實世界中的型態分類問題 42
4.3 不同前處理程序之分類器分類效能與分析 46
4.3.1 實驗數據 46
4.3.2 實驗分析 53
第五章 結論 56

參考文獻 58
附錄A 針對多類別分類問題的k-NNVD值求法 61
附錄B 實驗數據列表 62
附錄C 真實世界數據詳細資料 66

[1] Chidananda Gowda and G. Krishna 1978, “Agglomerative clustering using the concept of mutual nearest neighborhood”, Pattern Recognition, vol.10, pp. 105-112.

[2] Chidananda Gowda and G. Krishna 1979, “The condensed nearest neighbor rule using the concept of mutual nearest neighborhood,” IEEE Trans. Inform. Theory, vol. IT-25, pp. 488-490.

[3] D. T. Davis and J.-N. Hwang 1992, “Attentional focus training by boundary region data selection,” Proc. IJCNN, vol. I, pp. 676-681.

[4] G. P. Zhang 2000, “Neural networks for classification: a survey,” IEEE Trans. Syst., Man, Cybern., Part C : Applications and Reviews, vol. 30, pp. 451-462.

[5] J. L. Leva 1992, “A fast normal random number generator,” ACM Trans. Math. Softw., vol. 18, no. 4, pp. 449-453.

[6] J. L. Leva 1992, “Algorithm 712: a normal random number generator,” ACM Trans. Math. Softw., vol. 18, no. 4, pp. 454-455.

[7] J. Sklansky and L. Michelotti 1980, “Locally trained piecewise linear classifiers,” IEEE Trans. PAMI, vol. 2, no. 2, pp. 101-111.

[8] K. Fukunaga and L. D. Hostetler 1973, “Optimization of k-nearest-neighbor density estimates”, IEEE Transactions on Information Theory, IT-19 (3), pp. 320-326.

[9] K. Hara and K. Nakayama 2000, “A training method with small computation for classification,” IEEE Trans. Neural Networks, vol. 3, pp. 543-548.

[10] K. Sin and R. J. P. deFigueiredo 1992, “An evolution-oriented learning algorithm for the optimal interpolative net,” IEEE Trans. Neural Networks, vol. 3, pp. 315-323.
[11] K. Sin and R. J. P. deFigueiredo 1993, “Efficient learning procedures for optimal interpolative nets,” IEEE Trans. Neural Networks, vol. 6, pp. 99-133.

[12] L. Holmstrom, P. Koistinen, J. Laaksonen, and E. Oja, “Neural and statistical classifiers – taxonomy and two case studies,” IEEE Trans. Neural Networks, vol. 8, pp. 5-17.

[13] P. E. Hart 1968, “The condensed nearest neighbor rule,” IEEE Trans. Inform. Theory, vol. IT-14, pp. 515-516.

[14] R. J. P. deFigueiredo 1983, “A generalized fock space framework for nonlinear system and signal analysis,” IEEE Trans. Circ. Syst. CAS-30, pp. 637-647.

[15] R. J. P. deFigueiredo 1990, ”A new nonlinear functional analytic framework for modeling artificial neural networks,” IEEE ISCAS., pp.723-726.

[16] R. J. P. deFigueiredo 1990, “An optimal matching-score net for pattern classification,” IEEE IJCNN., pp.909-916.

[17] R. P. Lippmann 1989, “Pattern classification using neural networks,” IEEE Communications Magazine, vol. 27, pp. 47-50, 59-64.

[18] S. Haykin 1999, “Neural networks: a comprehensive foundation,” 2nd Edition, Prentice Hall, Section 1.1, pp. 2-6.

[19] S. Haykin 1999, “Neural networks: a comprehensive foundation,” 2nd Edition, Prentice Hall, Chapter 4, pp. 156-184.

[20] S. Haykin 1999, “Neural networks: a comprehensive foundation,” 2nd Edition, Prentice Hall, Chapter 7, pp. 351-355.

[21] T. M. Cover and P. E. Hart 1967, “Nearest neighbor pattern classification,” IEEE Trans. Inform. Theory, vol. IT-13, pp. 21-27.

[22] W. Duch and N. Jankowski 2000, “Taxonomy of neural transfer function,” IEEE Trans. Neural Networks, vol. 3, pp. 477-482.

[23] W. H. Press, B. P. Flannery, S. A. Teukolsky, and W. T. Vetterling 1990, “Numerical recipes in C: the art of scientific computing,” Cambridge University Press, Chapter 7, pp. 204-213.

[24] UCI Machine Learning, http://www.ics.uci.edu/~mlearn/MLRepository.html