如何有效且真實建立三度空間人臉模型及其各種動態表情，在電腦圖學領域中一直是個困難且有趣的問題，隨著電腦技術進步，人們對於影像呈現要求也越來越高，因此以電腦為基礎之三度空間人臉模型建立和動作模擬相關研究相當受重視。
建立三度空間模型方法有許多種，而目前最常被使用方法包括雷射掃描系統、電腦圖學、機器視覺等方法。雷射掃描系統雖然可建立細緻立體模型，但卻有無法追蹤動態物體之缺點。還有以電腦圖學之方法來達成，但此方法建構人臉模型相當耗時耗力，且對於人臉模型之動態動作必須完全以人工方法完成，因此很難達到逼真的效果。機器視覺的方法則是在人臉上貼上一些反光片或感應器，以此建立三度空間人臉模型及其表情動作，但受限於人臉面積，所能貼上之反光片或感應器相當有限，難以表達人臉細膩之表情。
因此本論文提出以影像為基礎直接建立三度空間人臉動作模型；先利用兩部CCD攝影機，同時取得人臉左右影像，由兩台攝影機間距離，模擬人類兩眼視差而推導深度，對三度空間人臉模型加以建構，再針對其中一部CCD之連續影像，和原本之影像進行相同之特徵點比對，取得到特徵點之對應，進而計算出特徵點位移向量，將位移向量結合先前推導之三度空間人臉模型，建立立體人臉表情序列模型，於標準模型建立後，任何角色之二維平面臉部影像僅須與資料庫中儲存之立體人臉表情序列模型進行特徵點對應，便可以直接轉換成為三度空間臉部連續動作，此一過程中完全由電腦自動完成，不需人工來判斷，不僅可快速建立三度空間人臉立體模型，更可細緻地呈現人臉表情。且由於採用Delaunay三角化將人臉分割更多特徵點對應，可不受限於人臉面積，將人臉擷取出數千點特徵點，在本論文實作中達到5000點，且由於完全以真實攝影機攝取之人臉表情序列為依據，更能細膩且真實的反映人臉的表情動作。
但由於本實驗室之前學長在撰寫相關論文時，沒有考慮到人臉上一些較特殊之部位如眼睛、嘴巴，這些部位在整個影像序列並非只是作單純位移，而且眼睛區域中之眼球、嘴巴區域之牙齒、舌頭有可能只出現在整個影像序列中之某些畫框(frames)中，因此造成了一些對應上之困擾，最後在連續表情序列中便產生了某些嚴重錯誤，因此本論文增加了一些眼部和嘴部的特殊處理，包含眼部、嘴部特徵擷取，眼部、嘴部狀態分析，建立眼睛、牙齒、舌頭模型等。經過這些改進後，所建立的人臉表情動作將更具真實感且更能符合真實世界中的人臉動作。

none

目錄
摘要 2
第一章 簡介 6
第一節 立體成像 10
第二節 2D和3D關係 16
第三節 比對技術 19
第四節 人臉特徵擷取 26
第五節 Delaunay 三角形 30
第二章 相關研究 34
第一節 三度空間模型建立 34
以電腦圖學技術為基礎 35
直接量測技術 38
第二節 特徵擷取與狀態偵測 40
眼部偵測及狀態判定 40
嘴部偵測及狀態判定 46
第三節 人臉連續動作模擬 51
內差法 51
移動偵測 53
直接模擬 55
其他方法 58
第三章 研究方法與步驟 68
第一節 取像環境與相機校正 69
第二節 人臉特徵擷取與狀態判定 73
特徵擷取 73
眼部與嘴部狀態判定 79
第三節 建立三度空間模型 81
第四節 人臉表情連續動作建立 87
第五節 二維對應影像轉換至三維 91
第四章 結論 97
參考資料 97

[1] Wolfgang Niem,”Automatic reconstruction of 3D objects using a mobile camera”, Image And Vision Computing Vol. 17 (2) (1999), p:125-134。
[2] R. Bowden, T.A. Mitchell and M. Sarhadi,”Non-linear statistical models for the 3D reconstruction of human pose and motion from monocular image sequences”, Image And Vision Computing Vol. 18 (9) (2000), p:729-737。
[3] E. Grossmann, and J. Santos-Victor,”Uncertainty analysis of 3D reconstruction from uncalibrated views”, Image And Vision Computin Vol. 18 (9) (2000), p:685-696。
[4] Chia-Yang Huang, Wen-Hsiang Tsai, “Knowledge-Based Tracking and Modeling of Facial Expressions by Stereo Vision Techniques”, The 13th IPPR Conference on Computer Vision, Graphics and Image Processing 2000. p286-298.。
[5] Zicheng Liu, Zhengyou Zhang, Chuck Jacobs, Michael Cohen, “Rapid Modeling of Animated Faces From Video”, Technical Report MSR-TR-2000-11, http://research.microsoft.com/~zhang。
[6] K. Waters and D. Terzopoulos,”Modeling and animating faces using scanned data”, J. of Visualization and Computer Animation, 2(4) (1991), p:123-128。
[7] H. Hugli, C. Schutz, “Geometric Matching of 3D Objects; Assessing the Range of Successful Initial Configurations”, Int.Conf. on Recent Advances in 3-D Digital Imaging and Modeling, pp.101-106, 1997.
[8] C.L. Bajaj, F. Bernardini, and G. Xu. “Automatic reconstruction of surfaces and scalar fields from 3D scans”. In Proceedings of SIGGRAPH ’95 (Los Angels, CA, Aug. 6-11, 1995), pages 109-118. ACM Press, August 1995.
[9] W.T Reeves, ”Simple and complex facial animation, In State of the Art in Facial Animation”, SIGGRAPH’90 ACM(1990), p:88-106。
[10] C.L. Waite Langwidere, ”Hierarchical spline based facial animation system with simulated muscles” (1993)。
[11] Reinhard Klette, Karsten Schluns, Andreas Koschan, “Computer Vision Three-Dimensional Data from Images”, 1996.
[12] Tsai, R.Y.: “An efficient and accurate camera calibration technique for 3D machine vision”. Proc. International Conference on Computer Vision and Pattern Recognition, 1986, pp. 364-374.
[13] Robert M. Haralick, Linda G. Shapiro, “Computer and Robert Vision Volume II”, 1993.
[14] H.P.Moravec, ”Towards automatic visual obstacle avoidance”, in Proc. 5th Int. Ioint Conf. Artificial Intell. (1977), P.584。
[15] D. B. Gennery,”Object detection and measurement using stereo vision”, in Proc . ARPA Image Understanding Workshop, College Park(1980), P:217-253。
[16] M. J. Hannah,”Bootstrap stereo”, in Proc. ARPA Image Understanding Workshop, College Park (1980), p:201-208。
[17] F. Glazer, G. Reynolds, and P. Anandan,”Scene matching by hieratchical correlation”, in Proc.IEEE Conf. Comput. Vision Patt. Recogn(1983)., p:432-441。
[18] H. S. Lim and T. O. Binford, ”Stereo correspondence: A hierarchial approach”, in Proc. Image Understanding Workshop (1987)。
[19] D. Marr and T. Poggio, ”A theory of human stereo vision”, in Proc. R. Soc. London vol B204 (1979), p:301-328。
[20] S. S. Sinha and B. G. Schunck,”Discontinuity preserving surface reconstruction”, in Proc. Conf. Comput. Vision Patt. Recogn(1989), p:229-234.
[21] D. Marr and E. Hildreth, Theory of edge detection”, Proc. Royal Soc.London vol B207 (1980), p:187-217。
[22] M. Peitikainen and D. Harwood,“Depth from three camera stereo”, in Proc. IEEE CS Conf. Pattern Recognition (1986) , p:2-8。
[23] N. Ayache and B. Faverjon,”Efficient registration of stereo images by matching graph descriptions of edge segments”, Int. J. Comput. Vision(1987), p:107-131。
[24] Hannah, M. J., “A System for Digital Stereo Matching,” Phototgrammetric Engineering & Remote Sensing, 1989.
[25] Barnard, S. T., “Stereo Matching by Hierarchical, Microcanonical Annealing,” Proceedings of the Image Understanding Workshop, Los Angels, 1987, pp.792-797.
[26] Ohta, Y., and T. Kanade, “Stereo by Intra- and Inter- Scanline Search Using Dynamic Programming,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. PAMI-7, 1985,pp. 139-154.
[27] Witkin, A., D. Terzopoulos, and M. Kass, “Signal Matching through Scale Space,” Interational Journal on Computer Vision, Vol.1, 1987, pp. 231-258.
[28] Thiessen, A.H., “Precipitation averages for large areas” Monthly Weather Review (1911), 39, 1082-1084.
[29] Nowacki, V.W. “Uber allgemeine eigenschaften von wirkungsbereichen” Zeitscrhift fur Kristallographie, (1976) 143, 360-385.
[30] Evans, I.S. “The properties of patterns of points, measured by apace filling and angular relationships” Geographical Articles (Cambridge) (1967), 8, 63-77.
[31] Frank, F.C. and J.S. Kasper “complex alloy structures regarded as sphere packing. I. Definitions and basic principles” Acta Crystallographica(1958), 11, 184-190.
[32] Brown, G.S. “Point density in stems per acre”, New Zealand Forestry Service Research Notes, (1965) 38, 1-11.
[33] Whitney, E.N. “A real rainfall estimates” Monthly Weather Review, (1929) 57, 462-463.
[34] Popoff, C.C. “Computing reserves of mineral deposits: principles and conventional methods” U.S. Department of the Interior, Bureau of Mines, Information Circular (1966), 8283.
[35] Thiessen, A.H. “Precipitation averages for large areas” Monthly Weather Review(1911), 39, 1082-1084.
[36] Haynes, K.E. and A.S. Fotheringham “Gravity and Spatial Interaction Models, Sage Scientific Geography Series, Volume 2 (1984) Beverly Hills, Calif.: Sage Publications.
[37] J. Bloomenthal and B. Wyvill, ”Interactive techniques for implicit modeling”,SIGGRAPH’93(1993) 。
[38]C.; Jost, T.; Hugli, H. “Virtual Systems and MultiMedia, Free-form 3D object reconstruction from range images Schutz”. VSMM '97. Proceedings., International Conference on , 1997 Page(s): 69 -70
[39] A.A. Ricci.,”A constructive geometry for computer graphics”, The computer Journal (1973)16(2), p:157-160。
[40] R. I. Hartley. “An algorithm for self calibration from several views.” In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 908-912, Seattle, WA, June 1994.
[41] Digital Image Processing, Rafael C. Gonzalez & Richard E. Woods, 1993.
[42] Lisa, G Brown, “A Survey of Image Registration Techniques”, ACM Computing Survey. Vol 24, No. 4, （1992） p.325-376
[43] Abdel-Aziz, Y. I., Karara, H. M.: “Direct linear tranformation into object space coordinates in close-range photogrammetry.” Proc. ASP Symposium on Close-Range Photogrammetry, Urbana, Illinois, USA, 1971 ,pp1-18.
[48] B. Caprile and V. Torre. “Using Vansishing Points for Camera Calibration”. The International Journal of Computer Vision, 4(2): 127-140 Mar. 1990.
[49] G. Stein. “Accurate internal camera calibration using rotation, with analysis of sources of error”. In Proc. Fifth International Conference on Computer Vision, pages 230-236, Cambridge, Massachusetts, June 1995.
[50] Zhengyou Zhang. “A Flexible New Technique for Camera Calibration”. Technical Report MSR-TR-98-71. http://research.microsoft.com/~zhang. Dec. 2,1998.
[51] R. Hartley. “Estimation of relative camera positions for uncalibrated cameras”. In Proc. 2nd European Conf. On Computer Vision, Lecture Notes in Computer Science 588, pages 579-587, 1992.
[52] R. Bartles, J. Beatty, and B. Barsky. “Introduction to Splines for Use in Computer Graphics and Geometric Modeling”. Morgan Kaufmann, Los Altos, CA, 1987.
[53] Q. –T. Luong. And O. Faugeras. “Self-calibration of a moving camera from point correspondences and fundamental matrices”. The International Journal of Computer Vision, 22(3): 261-289, 1997.
[54] R.I. Hartley. “An algorithm for self calibration from several views.” In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, page 908-912, Seattle, WA, June 1994. IEEE.
[55] C.L. Waite Langwidere, ”Hierarchical spline based facial animation system with simulated muscles”, (1993)。
[56] W.T Reeves,”Simple and complex facial animation,In State of the Art in Facial Animation”, SIGGRAPH’90 ACM (1990), p:88-106
[57] D. Terzopoulos, ”The computation of visible-surface representations”, .IEEE Trans . on Pattern Analysis and Machine Intelligence(1988), p:417-438。
[58] J. Kleiser. “A fast, efficient, accurate way to represent the human face”. In State of the Art in Facial Animation, SIGGRAPH ’89 Tutorials, Volume 22, pages 37-40. ACM, New York, 1989.
[59] Xiaoming Liu,Yueting Zhuang, Yunhe Pan,”Video Based Human Animation Technique”, ACM 7th (1999)。
[60] N. Magnenat-Thalmann, H. Minh, M. deAngelis, and D. Thalmann,”Design, transformation and animation of human faces”, The Visual Computer (1988)5, p:32-39。
[61] P. Ekman, W. V. Friesen. “Manual for the Facial Action Coding System” Consulting Psychologists Press, Inc., Palo Alto, CA, 1978.
[62] X. Q. Deng. “A Finite Element Analysis of Surgery of the Human Facial Tissue.” PhD thesis, Columbia University, New York, 1988.
[63] Trucco, E. and Verri, “A. Introductory Techniques for 3-D Computer Vision”, to be published soon by Prentice Hall.
[64] Reg Wilson's "Tsai Camera Calibration Software" at http://www.cs.cmu.edu/afs/cs.cmu.edu/user/rgw/www/TsaiCode.html
[65] Frederic Pighi, Richard Szeliski, David H. Salesin, Resynthesizing “Facial Animation through 3D Model-Based Tracking”, University of Washington, Microsoft Research.
[66] Bregler, C.; Hertzmann, A.; Biermann, H. , “Recovering non-rigid 3D shape from image streams”, Computer Vision and Pattern Recognition, 2000. Proceedings. IEEE Conference on , Volume: 2 , 2000 Page(s): 690 -696 vol.2
[67] Aboul-Ella, H.; Nakajima, M. Information Visualization “Image warping based on elastic body spline transformation: application for facial animations”. Proceedings., 1997 IEEE Conference on , 1997 Page(s): 106 –110
[68] Yuencheng Lee, Demetri Terzopoulos and Keith Walters; “Realistic modeling for facial animation”Proceedings of the 22nd annual ACM conference on Computer graphics, 1995, Pages 55 – 62
[69] Ming-Shing Su; Ming-Tat Ko; Kuo-Young Cheng, “Control of feature-point-driven facial animation using a hypothetical face”, Computer Graphics and Applications, 2000. Proceedings. The Eighth Pacific Conference on , 2000 Page(s): 359 –455
[70] E.; Graf, H.P. “Photo-realistic talking-heads from image samples Cosatto” Multimedia, IEEE Transactions on , Volume: 2 Issue: 3 , Sept. 2000 Page(s): 152 –163
[71] Ying-li Tian, Takeo Kanade, Jeffrey F. Cohn “Multi-State Based Facial Feature Tracking and Detection” Robotics Institute, Carnegie Mellon University, August,1999
[72] Ashish Kapoor, Rosalind W. Picard “Real-Time, Fully Automatic Upper Facial Feature Tracking” MIT Media Laboratory, May, 2002
[73] M. Kass, A. Witkin and D. Terzopoulus “Snakes: Active contour models.” International Journal of Computer Vision, 1(4) : 321-331, 1988
[74] J. Luettin，N. A. Tracker, S. W. Beet “Active Shape Models for Visual Speech Feature Extraction.” Electronic Systems Group Report No. 95/44, University of Sheffield, UK, 1995.
[75] Y. Tian, T. Kanade, and J.F.Chon “Dual-state parametric eye tracking.” In Proceedings of Conference on Computer Vision and Pattern Recognition, 2000.
[76] Y. Tian, T. Kanade, and J.F.Chon “Recognizing upper face action units for facial expression analysis.” In Proceedings of Conference on Computer Vision and Pattern Recognition, June 2000.
[76] I. Essa, S Basu, T. Darrell, and A. Pentland “Modeling, tracking and interactive animation of faces and heads using input from video.” In Proceedings of Computer Animation Conference, 1996.
[77] Shinjiro Kawato and Jun Ohya, “Two-Step Approach for Real-time Eye Tracking with a New Filtering Technique”IEEE Int. Conf. on Systems, Man & Cybernetics, pp 1366-1371, 08-11 Oct. 2000.
[78] Rogerio Schmidt Feris ,Teofilo Emidio de Campos and Roberto Marcondes Cesar Junior, “Detection and Tracking of Facial Features in Video Sequences” Artificial Intelligence, vol. 1793, pp. 197-206, April 2000.
[79] Pengyu Hong, Zhen Wen, Thomas S. Huang and Heung-Yeung Shum, “Real-Time Speech-Driven 3D Face Animation” IEEE 3D Data Processing Visualization and Transmission, 2002. Pages: 713-716.