隨著監視系統普及、設置數量及密度不斷上升，考量空間與顯示器之數量及成本，傳統監視系統多以分割畫面方式，將數個縮小之監視影像於同一螢幕顯示，惟於大範圍監控區域配置數以百計監控攝影機，如校園、城市路口等場合，若沿用傳統之分割畫面，必須同時監控上百個即時影像，並不合理，監視人員難以自如此大數量之監控影像推知其對應之實際場景位置，不能對突發事件快速進行處理，監控效能不佳。
　　為改善上述傳統監視系統之缺失，本論文提出透過全狀況掌控之沉浸式監視系統（Immersive Surveillance for total situational awareness）將多個監控視訊加以融合、分析，並於三維場景內呈現之立體場景監控技術，另搭配多攝影機協同偵測技術，進行入侵者移動路徑偵察、預測，虛擬巡邏路徑之規劃。本論文區分為三階段：1. 三維場景建構，2. 動態立體場景，3. 虛擬巡邏、物件偵測等應用；各階段研究內容概述如下：
　　第一階段：三維場景建構，以監視區域之衛星影像為基礎，結合建物高度、樓層數、各樓面、房間配置等資訊，並貼上靜態材質，視監控需求，建立詳細程度不等之監視區域三維靜態場景。
　　第二階段：動態立體場景，將監視區域中各攝影機擷取之即時二維影像映對至所建構之三維場景，以單一監控環境呈現所有攝影機畫面，建構立體動態監控場景。
　　第三階段：虛擬巡邏、物件偵測等應用，將二維動態影像前景、後景分離，擷取出移動物件並標示於三維場景，利於追蹤，使用者亦可以固定頻率依自行設定之路徑進行巡邏，動態選擇最佳攝影視角以即時偵測移動物件，取得入侵物體數量、位置與時間並同時針對多個目標進行跟蹤，如虛擬實境般置身場景進行巡邏，於共同時間點對不同空間狀態與突發事件進行識別，展現全域場景監控。

Digital surveillance system is indispensable for household protection, community security and traffic monitoring. With the instant awareness of unusual events, timely measures can be taken to efficiently curb the happening of crimes. The surveillance video can be subjected to further analysis to pin down the relevant figures, timing and process of a specific event.

With the growing number of surveillance cameras set up and the expanse of surveillance area, the traditional split-screen display approach cannot provide intuitive correspondence between the images acquired and the areas under surveillance. This might further lead to failure of timely response to an event requiring immediate attention. Therefore, a mapping between the images acquired and the 3D surveillance area is needed to establish intuitively visual correspondence. Also, the traditional monitoring system usually equipped with a plurality of wide-angle lenses with stationary bases or single wide-angle lens travel periodically on a chassis. The cost for setting up a system with multiple surveillance cameras are higher. After displaying the images captured in a split-screen monitor, the images are usually too small to be useful in identifying the contents of a scene. On the other hand, a single-lens configuration cannot continuously monitor the same area of a scene. The periodic scanning pattern unavoidably leaves some portions unattended. Even though wide-angle lenses provide wider coverage of surveillance area, yet the lower resolution of images captured cannot provide effective identification of figures or articles in the scene. The system configuration and functionality of the traditional surveillance system leave much to be desired.

In the Event-Trigged Virtual Surveillance Environment proposed, the satellite picture will be employed to provide 2D surveillance area information. Users then enter relevant site information, such as the number of floors, floor map, camera setup location and types, etc. This information is combined to construct 3D surveillance scene, and the images acquired by surveillance cameras are pasted into the constructed 3D model to provide intuitively visual presentation. The status of different types of surveillance devices, such as infrared, circuit breaker, etc., can also be displayed visually in the 3D scene constructed.

The images captured through a plural number of cameras are integrated. Intrusion path is analyzed and predicted. With the coordination between different lenses, suspected objects can be detected and tracked. In one embodiment, Both wide-angle and telephoto lenses are used to improve the pitfalls of the aforementioned system. A wide-angle lens is responsible for the wide-area surveillance. When an unusual event is detected, one or several telephoto lenses, mounted on movable chassis, are directed to lock and track targets. Images with much higher resolution than those of wide-angle one can be acquired through the telephoto lenses. With the proposed system setup, a wide area of coverage and an improved resolution of target images can be simultaneously satisfied. Face detection and face recognition paradigms can be further applied to the images acquired to determine whether the target contains human faces and recognize the identities of the subjects. Depending on the results of face recognition, no warning is given for persons registered, or instant warning is dispatched for illegal intrusion. Combined with the analysis of target size and pattern of movement, and dynamic background updating, the false alarms due to changes attributable to environmental lighting condition, fallen leaves, water ripples, etc., can be significantly reduced. On detecting of security threat, the surveillance video and warning message are transmitted through internet and wireless channels to pre-selected network terminals or personal mobile devices to facilitate timely process. The settings of message warning can be modified by the network terminals or personal mobile devices.

The focuses of this paper include 3D surveillance scene construction and mapping of surveillance images to the constructed scene, detect objects and virtual patrol. In the first phase, 2D information extracted from satellite pictures and site data entered manually are combined to construct 3D surveillance scene. The images received from cameras are pasted into the corresponding areas in the 3D scene constructed. The mapping between the surveillance images and scene can provide more intuitive presentation in a wide-range area equipped with massive number of cameras. In the next phase, incorporated with the dynamic background updating, the analysis of target size and movement pattern, the false alarm rate can be improved in the face of environmental lighting condition change, fallen leaves, etc. Users can also real-time inspect the surveillance scene or change the setting of the remote surveillance system.

第一章 簡介 1
第二章 相關研究 4
第一節 多攝影機間之協同偵測 4
第二節 動態警戒配置 7
第三節 物件追蹤及行為分析 9
第四節 人臉之偵測與辨識 11
第三章 理論依據 16
第一節 相關技術 16
第二節 影像接合 17
第三節 視訊融合 18
第四節 增強虛擬實境 19
第四章 實際操作方法、步驟及結果 22
第一節 三維場景建造 22
第二節 動態立體場景 30
第三節 虛擬巡邏、物件偵測等應用 34
第五章 未來工作 38

[1] A. Elgammal, D. Harwood, and L. Davis, “Non-parametric Model for Background Subtraction,” Proc. of ICCV '99 FRAME-RATE Workshop, 1999.
[2] C. Stauffer and E. Grimson, “Adaptive Background Mixture Models for Real-time Tracking,” Proc. of ICCV and Pattern Recog., pp. 246-252, , 1999.
[3] K. Kim, T. H. Chalidabhongse, D. Harwood, and L. S. Davis, “Real-Time Foreground-Background Segmentation using Codebook Model,” Real-Time Imaging, pp. 172-185, 2005.
[4] M. Heikkila and M. Pietikainen, “A Texture-Based Method for Modeling the Background and Detecting Moving Objects,” IEEE Trans. Pattern Anal. Mach. Intell., Vol. 28, No. 4, pp. 657-652, 2006.
[5] O. Javed, K. Shafique, and M. Shan, “A Hierarchical Approach to Robust Background Subtraction Using Color and Gradient Information,” Proc. of IEEE Workshop on Motion and Video Computing (MOTION’ 02), 2002.
[6] T. Kanade, R. Collins, A. Lipton, P. Burt and L. Wixson, “Advances in cooperative multi-sensor video surveillance”, Proc. of DARPA Image Understanding Workshop, Vol. 1, pp. 3-24, 1998.
[7] Nam T. Nguyen, Dinh Q. Phung, Svetha Venkatesh, Hung Hai Bui, “Learning and Detecting Activities from Movement Trajectories Using the Hierarchical Hidden Markov Models,” CVPR, pp. 955-960, 2005.
[8] C. W. Chang, J. J. Tang and Z. Lee,“An Adaptive Surveillance System Platform for Cooperative Object Detection and Tracking in Multiple Cameras,” ICOS 2007 International Conference on Open Source, Taipei Taiwan, 2007.
[9] P. L. Rosin, “Thesholding for Change Detection,” Proc. of ICCV, 1998.
[10] I. Haritaoglu, D. Harwood, and L. S. Davis, “W4: Real-time surveillance of people and their activities,” IEEE Trans. Pattern Anal. Mach. Intell., Vol. 22, pp. 809–830, Aug. 2000.
[11] C. R. Wren, A. Azarbayejani, T. Darrell, and A. Pentland, “Pfinder: Real-time Tracking of the Human Body,” IEEE Trans. Pattern Anal. Mach. Intell.Vol.19, No.7, pp. 780-785, 1997.
[12] A.F. Bobick, S.S. Intille, J.W. Davis, F. Baird, L.W. Campbell, Y. Ivanov, C.S. Pinhanez, A. Schütte and A. Wilson, “The KidsRoom: A Perceptually-Based Interactive and Immersive Story Environment,” Presence, vol. 8, no. 4, pp. 368-393, Aug. 1999.
[13] P. Shih, C. Liu, “Face detection using discriminating feature analysis and support vector machine,” Pattern Recog., vol. 39, pp. 260-276, 2006.
[14] N. Rajagopalan, R. Chellappa, N. T. Koterba, “Background Learning for Robust Face Recognition With PCA in the Presence of Cluster,” IEEE Trans. Image Process., vol. 14, No. 6, pp. 832-843, 2005.
[15] H.A. Rowley, S. Baluja, T. Kanade, ”Neural network-based face detection,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 20, pp. 23-38, 1998.
[16] Rein-Lien Hsu, Mohamed Abdel-Mottaleb, and Anil k. Jain, “Face detection in color images,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 24, no. 5, pp. 696-706, 2002.
[17] M. Lades, J.C. Vorbruggen, J. Buhmann, J. Lange, C. von der Malsburg, R.P. Wurtz, and W. Konen, “Distortion Invariant Object Recognition in the Dynamic Link Architecture,” IEEE Trans. Computers, vol.42, No.3, pp.300-311, 1992.
[18] X. Liu and T. Chen, “Video-based face recognition using adaptive hidden markov models,” Proc. of Intl. Conf. on Computer Vision and Pattern Recog., 2003.
[19] Yamaguchi, K. Fukui, and K. Maeda, “Face recognition using temporal image sequence,” Intl. Conf. on Automatic Face and Gesture Recog., Nara, Japan, 1998.
[20] C. Fruh and A. Zakhor, “3D Model Generationfor Cites Using Aerial Photographs and Ground Level Laser Scans”, IEEE Conference on Computer Vision and Pattern Recognition, 2001.
[21] S. C. Lee, S. K. Jung and R. Nevatia,“Automatic Integration of Façade Textures into 3D Building Modelings with Projective Geometry Based Line Clustering”, EUROGRAPHIC’02 , 2002.
[22] P. E. Debevec, C.J.Taylor and J. Malik, “Modeling and Rendering Architecture from Photograph: A Hybrid Geometry-and Image - based Approach”, SIGGRAPH’ 96, pp. 11-20, 1996.
[23] P .E. Debevec, Y. Yu, and G.D. Borshukov, “Efficient View-Dependent Image-Based Rendering with Project ive Texture-Mapping”, 9th Eurographics Workshop on Rendering, pp. 105-116, 1998.
[24] Y. Hsieh, “SiteCity: A semi-automated Site Modeling System”, IEEE Conference on Computer Vision and Pattern Recognition, pp. 499-506, 1996.
[25] D. Liebowitz, A.Criminisi and A. Zisserman, “Creating Architectural Modeling From Images”, EUROGRAPHIC’99, pp. 39-50, 1999.
[26] William Ribarsky, Tony Wasilewski, and Nickolas Faust, “From Urban Terrain Models to Visible Cities”, IEEE Computer Graphics & Applications, Vol. 22, No. 4, 2002.
[27] A Gruen and R. Nevatia (editors), “Special Issue on Automatic Building Extraction from Aerial Images”, Computer Vision and Image Understanding, November 1998.
[28] Eden, A., Uyttendaele, M., and Szeliski, R. “Seamless Image Stitching of Scenes with Large Motions and Exposure Differences. ” Proceedings of CVPR 2006, Vol.3, pp. 2498-2505, 2006.
[29] Ranjan Rahul, Singh Harpreet, Meitzler Thomas, Sohn E. J., “Video image fusion process using fuzzy technique,” Signal Processing, Sensor Fusion, and Target Recognition XV, Vol. 6235, pp. 62350Y, Jun, 2006.
[30] Neumann U., You S., Hu J., Jiang B., and Lee J. “Augmented Virtual Environments (AVE): Dynamic Fusion of Imagery and 3D Models” VR03, March 2003
[31] Ismail Oner Sebe , Jinhui Hu , Suya You , Ulrich Neumann, “3D video surveillance with Augmented Virtual Environments, First ACM SIGMM international workshop on Video surveillance,” Berkeley, California, Nov. 02-08, 2003.
[32] Jing Li, Stan Z.Li, Quan Pan, Tao Yang, “Illumination and Motion-Based Video Enhancement for Night Surveillance,” 2nd Joint IEEE Intl. Workshop on Visual Surveillance and Performance Evaluation of Tracking and Surveillance, Pages 169-175, 15-16 Oct., 2005.