影像分割技術在影像處理的過程中扮演相當基礎且重要的一個步驟，從基本的影像分析、影像識別等工作，乃至於比較高階的應用如軍事監控，影片內容搜尋等。都需先將把影像畫面分割成具有意義的物件單位，再把這些物件單位作進一步的處理。在MPEG-4多媒體標準中，就把影像區分成各個物件，作為壓縮的基本單位，因此得以支援不同類別的各種應用。從人類視覺系統的觀點看來，影像切割技術分割出有意義的部分，也較符合人類視覺的感受。因為當人類肉眼看一個畫面時，是看到各個物件組合而成的場景，而不是細微的看到各個畫素。因此，本論文之主要目的將著重於影像暨視訊中物件之分割及其相關應用。我們希望能夠就現有的技術，作出調整並加以改良，切割出影像中符合人類視覺的區域，也就是說，我們會將影片分成兩個部分: 變動的部分稱之為前景，不變的部分則稱為背景。之後讓其他影像處理應用可以有更進一步的發展。

Image/video segmentation is a basic but important step in image processing. In some basic image processing works such as video analysis, video object recognition, etc., or some high level applications such as military surveillance, content-based video retrieval, etc., all the frames have to be segmented into meaningful parts at first. And then those parts can further be processed. MPEG-4 multimedia communication standard enables the content-based functionalities by using the video objects plane as the basic coding element. From the point of view of human vision system, video segmentation segments meaningful parts from the video stream that conform to what human vision feels. Because while seeing a scene by human naked eye, the scene is composed of many objects, not pixel by pixel. In this thesis, we will focus on the image/video segmentation and its applications.

One of our goals in this thesis is to design and implement an image/video segmentation system based on existing methods, which are widely used in image/video segmentation nowadays. We decompose the system into several stages, each of which performs a specific task. Then, based on the output of each stage, we can refine the algorithms in that stage to obtain a better result.

We can retrieve areas from image data which more accurately conform to what human vision system feels. In other words, we retrieve the moving part, say, foreground, from the static background. After obtaining the segmentation results, a compression algorithm such as MPEG-4 can be used to compress these retrieved regions, which is referred to as content-based coding. Besides, other image processing applications can be further developed. For example, remote surveillance and monitoring system can be developed for detecting the moving objects using the segmentation algorithms described in this thesis.

Chapter 1 Introduction 1
1.1 Background and Motivation 1
1.2 Organization of the Thesis 2
Chapter 2 RelatedWorks 3
Chapter 3 Video Format 8
3.1 Raw Format 8
3.2 Y Format 11
3.3 YUV444 Format 12
3.4 YUV420 Format 15
3.5 YUV422 Format 17
Chapter 4 Steps in Video Segmentation Algorithm 19
4.1 Pre-Processing 19
4.2 Frame Difference 21
4.2.1 Noise of the Frame 21
4.2.2 Object Motion 21
4.2.3 Contrast between the Object 21
4.3 Block Filtering 24
4.4 Close-Open Operation 32
4.5 Edge Detection 36
4.5.1 Sobel Algorithm 38
4.5.2 Watershed Algorithm 42
4.5.2.1 Watershed Algorithm by Several Methods 42
4.5.2.2 The Vincent and Soille algorithm 43
4.5.2.3 The Meyer Algorithm 45
4.5.2.4 Experimental Results and Comparison 45
4.5.2.5 A Modified Watershed Algorithm 47
4.6 Buffer Concept 51
4.7 Object Detection 55
4.8 Background Detection 57
4.9 Post-Processing 59
4.10 Performance Tuning 64
Chapter 5 Experimental Results 67
Chapter 6 Conclusion and FutureWorks 75
6.1 Conclusion 75
6.2 Future Works 75

[1] S.-Y. Chien, S.-Y. Ma, and L.-G. Chen, “Efficient moving object segmentation algorithm using background registration technique,” Circuits and Systems for Video Technology and IEEE Transactions, pp. 577–586, 2002.

[2] S.-Y. Chien, Y.-W. Huang, and L.-G. Chen, “Predictive watershed: a fast watershed algorithm for video segmentation,” Circuits and Systems for Video Technology and IEEE Transactions, pp. 453–461, 2003.

[3] D. Hagyard, M. Razaz, and P. Atkin, “Analysis of watershed algorithms for ggreyscale images,” Image Processing, pp. 41–44, 1996.

[4] J. Fan, G. Fujita, J. Yu, K. Miyanohana, T. Onoye, N. Ishiura, L. Wu, and I. Shirakawa,“Hierarchical object-oriented video segmentation and representation algorithm,” Signal Processing Proceedings, pp. 954–958, 1998.

[5] B. Parker and J. Magarey, “Three-dimensional video segmentation using a variational method,” Image Processing and IEEE, pp. 768–768, 2001.

[6] S. Valette, I. Magnin, and R. Prost, “Active mesh for video segmentation and objects tracking,” Image Processing and IEEE, pp. 77–80, 2001.

[7] H. Zhu and Z. Li, “A video segmentation algorithm based on spatial-temporal information,”Communications and Circuits and Systems and West Sino Expositions and IEEE, pp. 566–569, 2002.

[8] A. Mani, “Video segmentation using stabilized iinverse diffusion,” Multimedia and Expo, pp. 687–700, 2003.

[9] W. Si, Z. Yong-dong, and L. Shou-xun, “An automatic segmentation algorithm for moving objects in video sequences under multi-constraints,” Multimedia and Expo and IEEE, pp. 555–558, 2004.

[10] B.-G. Kim and P.-S. Mah, “Non-contrast based edge descriptor for image segmentation,”Pattern Recognition, pp. 572–575, 2004.

[11] P. L. Correia and F. Pereira, “Classification of video segmentation application scenarios,”Circuits and Systems for Video Technology and IEEE, pp. 735–741, 2004.

[12] D. Xu, J. Liu, Z. Liu, and X. Tang, “Indoor shadow detection for video segmentation,”Multimedia and Expo and IEEE, pp. 41–44, 2004.

[13] W.Wei, K. N. Ngan, and N. Habili, “Multiple feature clustering algorithm for automatic video object segmentation,” Acoustics and Speech and Signal Processing and IEEE, pp. 625–628, 2004.

[14] L.-Y. Duan, M. Xu, Q. Tian, and C.-S. Xu, “Mean shift based video segment representation and applications to replay detection,” Acoustics and Speech and Signal Processing and IEEE, pp. 709–712, 2004.

[15] Yuv format introduction. [Online]. Available: http://www.fourcc.org/yuv.php

[16] Rgb/yuv pixel conversion. [Online]. Available: http://www.fourcc.org/fccyvrgb.php

[17] R. Gonzalog and R. Woods, Digital Image Processing. Prential-hall, 2002.

[18] 林灶生, 數位影像處理實務─使用C, 松崗, 1990

[19] 王新成, 多媒體實用技術：影像分冊, 儒林, 1996

[20] 陳建宏, 多媒體導論, 全華, 2003

[21] 楊武智, 影像處理與辨認, 全華, 1994

[22] C.-M. Huang, K.-C. Yang, and J.-S. Wang, “Support fast scan operations with video streaming technology,” Multimedia and Expo, pp. 463–466, 2004.