我們提出了一個從影片中擷取出人體物件的方法，其中人體物件包含了人臉以及身體。在MPEG-4和MPEG-7等應用中，擷取出人體物件通常是最重要的課題。我們提出了利用時間和空間的資訊並結合neuro-fuzzy的方法來擷取人體物件。首先，我們利用提出的fuzzy self-clustering technique將整張video frame分為一個個的區塊，並從中找出的膚色區塊，將其合併產生出可能的候選臉部。對於候選臉部內是否真的存在人臉，我們先尋找可能的臉部器官位置，找出眼睛和嘴巴所形成的三角形配對。再將所找到的配對與事先定義的template作比對。然後，前景與背景中大略屬於人體的區塊藉由多種資訊被找出來。最後，人體物件不確定的邊緣地帶利用訓練完成的fuzzy neural network作判斷，產生更精準的人體物件。其中類神經網路使用SVD-based hybrid learning algorithm作學習。經由實驗和比較結果，我們的方法可以找到準確的人臉的位置，並且擷取出更精準的人體物件。

We propose a novel approach for segmentation of human objects, including face and body, in image sequences. In modern video coding techniques, e.g., MPEG-4 and MPEG-7, human objects are usually the main focus for multimedia applications. We combine temporal and spatial information and employ a neuro-fuzzy mechanism to extract human objects. A fuzzy self-clustering technique is used to divide the video frame into a set of segments. The existence of a face within a candidate face region is ensured by searching for possible constellations of eye-mouth triangles and verifying each eye-mouth combination with the predefined template. Then rough foreground and background are formed based on a combination of multiple criteria. Finally, human objects in the base frame and the remaining frames of the video stream are precisely located by a fuzzy neural network which is trained by a SVD-based hybrid learning algorithm. Through experiments, we compare our system with two other approaches, and the results have shown that our system can detect face locations and extract human objects more accurately.

第一章 簡介 1
第二章　其他方法之介紹 4
2.1 GARCIA'S APPROACH 4
2.2 FAN'S APPROACH 10
第三章　OUR APPROACH 17
3.1 OVERVIEW 17
3.2 ROUGH IMAGE SEGMENTATION 18
3.2.1 FUZZY SELF-CLUSTERING ALGORITHM 18
3.2.2 LABELING AND SMALL SEGMENT MERGING 18
3.3 INITIAL HUMAN OBJECT EXTRACTION 23
3.3.1 SKIN SEGMENT DETECTION 23
3.3.2 CANDIDATE FACE GENERATION 23
3.3.3 CANDIDATE FACE NORMALIZATION 28
3.3.4 旋轉人臉之偵測與特徵點偵測之SURVEY 28
3.3.5 FACIAL FEATURE DETECTION & FACE VERIFICATION 30
3.3.6 HUMAN BODIES DETECTION 37
3.4 HUMAN OBJECT REFINEMENT 39
第四章　實驗及比較結果 43
4.1 臉部偵測之比較結果 43
4.2 人體物件分割之比較結果 48

第五章　結論 58
附錄 - APPENDIX 59
A.1 BEST-FIT ELLIPSE CALCULATION 59
A.2 HAUSDORFF DISTANCE 61
A.3 VALLEY DETECTION FILTER 62
A.4 EYE-MOUTH TRIANGLE以及平均臉部等相關數據之統計 63
參考文獻 65

[1] J. Fan, L. Zhang, and F. Gan, "Spatiotemporal segmentation based on spatiotemporal entropic thresholding," Opt. Eng., vol. 36, pp. 2845-2851, 1997.

[2] J. Fan, D.K.Y. Yau, A.K. Elmagarmid, W.G. Aref, "Automatic image segmentation by integrating color-edge extraction and seeded region growing," IEEE Transactions on Image Processing, Vol. 10, No. 10, pp 1454 -1466, Oct 2001.

[3] J. Fan, X. Zhu, L. Wu, "Automatic model-based semantic object extraction algorithm," IEEE Transactions on Circuits and Systems for Video Technology, Vol. 11, No. 10, pp. 1073 -1084, Oct 2001.

[4] J. Fan, G. Fujita, M. Furuie, T. Onoye, I. Shirakawa, and L. Wu, “Automatic moving object extraction toward compact video representation,” Opt. Eng., vol. 39, no. 2, pp. 438–452, 2000.

[5] R.-L. Hsu, A.-M. M., A.K. Jain, "Face detection in color images," IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 24 No. 5 , pp. 696-706, May 2002.

[6] D. Chai and K. N. Ngan, “Face Segmentation Using Skin-Color Map in Videophone Applications,”IEEE Transactions on Circuits and Systems for Video Technology, Vol. 9, No.4, June 1999.

[7] N. Doulamis, and A. Doulamis, and S. Kollias, “Improving the Performance of MPEG Compatible Encoding at Low Bit Rates Using Adaptive Neural Networks,” in Real-Time Imaging, Vol 6, No.5, 2000.

[8] I. Kompatsiaris, and M. G., Strintzis, “Spatiotemporal Segmentation and Tracking of Objects for Visualization of Videoconference Image Sequence,” in IEEE Transactions on Circuits and Systems for Video Technology, Vol. 10, No. 8, Dec. 2000.


[9] C. Garcia and G. Tziritas, "Face detection using quantized skin color regions merging and wavelet packet analysis " Multimedia, IEEE Transactions on , Volume: 1 Issue: 3 , Sep 1999, Page(s): 264 -277.

[10] H. Wu, Q. Chen, and M. Yachida, "Face detection from color images using a fuzzy pattern matching method," IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.21, No.6 , June 1999.

[11] H. Rowley, S. Baluja, and T. Kanade, “Neural Network-Based FaceDetection,” IEEE Trans. Pattern Analysis and Machine Intelligence, Vol. 20, No. 1, pp. 23-38, Jan. 1998.

[12] K.-K. Sung and T. Poggio, “Example-Based Learning for View-Based Human Face Detection,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 1, pp. 39-51, Jan. 1998.

[13] K.J. Thoresz. Qualitative Representations for Recognition. Master's thesis, MIT, 2002.

[14] C. Kotropoulos and I. Pitas, “Rule-Based Face Detection in Frontal Views,” Proc. Int’l Conf. Acoustics, Speech and Signal Processing, vol. 4, pp. 2537-2540, 1997.

[15] Hadid, A.; Pietikainen, M.; Martinkauppi, B., "Color-based face detection using skin locus model and hierarchical filtering," 16th International Conference on Pattern Recognition. Proceedings., Vol. 4 , 2002, pp. 196 -200.

[16] M.-H. Yang and N. Ahuja, “Detecting Human Faces in Color Images,” Proc. IEEE Int’l Conf. Image Processing, vol. 1, pp. 127-130, 1998.

[17] X. Zhu; J. Fan; A.K. Elmagarmid, "Towards facial feature extraction and verification for omni-face detection in video/images " Image Processing. 2002. Proceedings. 2002 International Conference on , Vol. 2 , 2002, pp. II-113 -II-116 Vol.2.

[18] C. Lin, K.-C. Fan, "Human face detection using geometric triangle relationship " Pattern Recognition, 2000. Proceedings. 15th International Conference on , Volume: 2 , 2000. Page(s): 941 -944 vol.2

[19] R.-S. Wang, Y. Wang, "Facial feature extraction and tracking in video sequences " Multimedia Signal Processing, 1997., IEEE First Workshop on , 23-25 Jun 1997 Page(s): 233 -238.

[20] E. Saber and A.M. Tekalp, “Frontal-View Face Detection and Facial Feature Extraction Using Color, Shape and Symmetry Based Cost Functions,” Pattern Recognition Letters, vol. 17, no. 8, pp. 669-680, 1998.

[21] J. Miao, B. Yin, et al. A Hierarchical Multiscale and Multiangle System for Human Face Detection in a Complex Background Using Gravity-Center Template, Pattern Recognition, 1999, Vol.32, No.5, pp.1237-48.

[22] S. J. Lee, C. S. Ouyang, and S. H. Du, ``A neuro-fuzzy approach for segmentation of human objects in image sequences' IEEE Transactions on Systems, Man and Cybernetics, Part B, Vol 33, No.3, pp. 420--437, June 2003.

[23] H.A. Rowley, S. Baluja, T. Kanade,``Rotation Invariant Neural Network-Based Face Detection' Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 963--963, June 1998.

[24] X.-G. Lv; J. Zhou; C.-S. Zhang, "A novel algorithm for rotated human face detection," IEEE Conference on Computer Vision and Pattern Recognition, 2000. Proceedings. Vol.1, Page(s): 760 -765, June 2000.

[25] G.-C. Feng and P. C. Yuen, "Multi-cues eye detection on gray intensity image", Pattern Recognition, Vol. 34, Issue 5, May 2001, Pages 1033-1046.

[26] K. M. Lam, Y. L. Li, "An efficient approach for facial feature detection" Fourth International Conference on Signal Processing Proceedings, Vol 2, pp. 1100--1103, Oct.1998.

[27] S.-J. Lee and C.-S. Ouyang, “A Neuro-Fuzzy System Modeling with Self-Constructing Rule Generation and Hybrid SVD-Based Learning” IEEE Transactions on Fuzzy Systems, Vol. 11, No.3, pp. 341--353, June 2003.

[28] R. Adams, L. Bischof, "Seeded region growing" IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 16, No. 6 ,pp. 641 -647, June 1994.