傳統的3D影像合成需使用多支攝影機同時拍攝多個視角，不僅拍攝成本相當昂貴，且傳輸時需大量頻寬。本篇論文中虛擬視角影像是由一原始影像與一深度圖，使用基於深度影像繪圖法(Depth Image Based Rendering,DIBR)求得，而深度圖的獲得是使用基於動態規劃法之立體匹配演算法(Disparity Estimation Using Dynamic Programming,DP)。在來源影像與深度資訊沒有完全匹配的情況下，使用DIBR的演算法所繪製的虛擬視角物體常常有幾何失真，我們提出一個深度圖前處理前處理的方法，來解決此一情形以提升虛擬影像品質。針對虛擬視角中未填補區域(disocclusion area)，我們結合基於深度資訊的水平外插補洞法與水平背景鏡射法來消除畫面中的破洞。硬體實作方面將深度圖前處理、影像位移與破洞填補管線化的執行，來最大化整體執行效率。最後我們整合基於動態規劃法之立體匹配硬體與DIBR硬體，在處理連續影像序列時以每次執行一條掃描線為單位，來節省硬體成本並提升連續影像序列之資料運算速度。

Standared 3D stereo showing images at different view positions requires images captured from various view angles and thus needs large transmission bandwidth. In this thesis, the vitual images at different viewing locations are generated from an image and a depth map using depth-image-based rendering (DIBR) where the depth map is obtained using stereo matching based on dynamic-programming (DP). To reduce the geometry distortion in DIBR due to the mismatch of source images and depth information, we propose a depth-map preprocessing method by modifying the depth information near the image edges. Regarding disocclusion, horizontal extrapolation and background mirroring are adopted for hole filling in order to improve the quality of synthesized virtual images. Depth-map preprocessing, image warping and hole filling are executed in pipelining in the proposed DIBR hardware. Furthermore, we combine the hardware of DP-based stereo matching for depth map geneneration and the subsequent DIBR for virtual image synthesis by processing the image sequences scanline by scanline so that the area cost of the overall system is reduced with satisfactory performance.

中文論文審定書 i
中文摘要 iii
Abstract iv
目錄 v
圖目錄 vii
方程式目錄 xi
第1章 概論 1
1.1 研究背景 1
1.1.1 立體視覺成因 1
1.1.2 立體顯示技術 3
1.1.3 多視角相關議題 8
1.2 研究動機 9
1.3 本文大綱 10
第2章 相關研究 11
2.1 基於深度影像繪圖法 11
2.2 深度圖處理 14
2.3 影像位移 15
2.4 破洞填補 16
2.5 相關研究論文 19
2.6 立體視差估算 20
2.6.1 彩色影像轉亮度影像（RGB to Intensity） 21
2.6.2 匹配代價計算（Matching Cost Computation） 22
2.6.3 最小累計代價（Minimum Cost Accumulation） 23
2.6.4 視差值最佳化（Disparity Optimization） 24
2.6.5 深度圖轉換（Depth Map Conversion） 26
第3章 演算法架構與設計 27
3.1 演算法架構 27
3.2 基於深度影像繪圖法(DIBR) 29
3.3 深度值轉換(Depth map conversion) 30
3.4 視差圖前處理(Pre-processing of disparity map) 32
3.5 影像位移(Image warping) 38
3.6 破洞填補(Hole filling) 39
3.7 混合影像位移與破洞填補(Hybrid image warping and hole filling) 42
3.8 視差值優化(Disparity Refinement) 45
第4章 硬體設計與實現 47
4.1 DIBR硬體架構 47
4.2 視差圖前處理硬體實現 47
4.3 混合影像位移與破洞填補硬體實現 49
4.4 整合立體視差估算與DIBR硬體 50
第5章 實驗結果分析與比較 53
5.1 週期數與執行時間分析 53
5.2 邏輯合成數據與分析 55
5.3 數據比較 57
5.4 結果呈現 60
第6章 結論 63
6.1 結論 63
參考文獻 (References） 64

[1] A. Smolic, K. Mueller, P. Merkle, C. Fehn, P. Kauff, P. Eisert, et al., "3D Video and Free Viewpoint Video - Technologies, Applications and MPEG Standards," in Multimedia and Expo, 2006 IEEE International Conference on, 2006, pp. 2161-2164.
[2] C. Fehn and R. S. Pastoor, "Interactive 3-DTV-Concepts and Key Technologies," Proceedings of the IEEE, vol. 94, pp. 524-538, 2006.
[3] I. Sexton and P. Surman, "Stereoscopic and autostereoscopic display systems," Signal Processing Magazine, IEEE, vol. 16, pp. 85-99, 1999.
[4] Y.-C. Wu, "Image Synthesis Technology for Multi-view 3D Displays," Master's Thesis, Dept. of Electrical Engineering, National Cheng Kung University, 2008.
[5] N. A. Dodgson, "Autostereoscopic 3D Displays," Computer, vol. 38, pp. 31-36, 2005.
[6] M. Zwicker, A. Vetro, Y. Sehoon, W. Matusik, H. Pfister, and F. Durand, "Resampling, Antialiasing, and Compression in Multiview 3-D Displays," Signal Processing Magazine, IEEE, vol. 24, pp. 88-96, 2007.
[7] C. Fehn, "A 3D-TV system based on video plus depth information," in Signals, Systems and Computers, 2004. Conference Record of the Thirty-Seventh Asilomar Conference on, 2003, pp. 1529-1533 Vol.2.
[8] C. Vázquez, W. J. Tam, and F. Speranza, "Stereoscopic imaging: filling disoccluded areas in depth image-based rendering," in Proc. SPIE, 2006, pp. 63920D-63920D.
[9] L.-M. Po, S. Zhang, X. Xu, and Y. Zhu, "A new multidirectional extrapolation hole-filling method for Depth-Image-Based Rendering," in Image Processing (ICIP), 2011 18th IEEE International Conference on, 2011, pp. 2589-2592.
[10] Y.-R. Horng, Y.-C. Tseng, and T.-S. Chang, "Stereoscopic images generation with directional Gaussian filter," in Circuits and Systems (ISCAS), Proceedings of 2010 IEEE International Symposium on, 2010, pp. 2650-2653.
[11] P.-F. Jin, S.-J. Yao, D.-X. Li, L.-H. Wang, and M. Zhang, "Real-time multi-view rendering based on FPGA," in Systems and Informatics (ICSAI), 2012 International Conference on, 2012, pp. 1981-1984.
[12] L. Wang, J. Lei, H. Zhang, K. Fan, and S. Bu, "A novel virtual view rendering approach based on DIBR," in Computer Science & Education (ICCSE), 2012 7th International Conference on, 2012, pp. 759-762.
[13] W. J. Tam, G. Alain, L. Zhang, T. Martin, and R. Renaud, "Smoothing depth maps for improved steroscopic image quality," pp. 162-172, 2004.
[14] W. J. Tam and L. Zhang, "Nonuniform smoothing of depth maps before image-based rendering," pp. 173-183, 2004.
[15] L. Zhang and W. J. Tam, "Stereoscopic image generation based on depth images for 3D TV," Broadcasting, IEEE Transactions on, vol. 51, pp. 191-199, 2005.
[16] W.-Y. Chen, Y.-L. Chang, S.-F. Lin, L.-F. Ding, and L.-G. Chen, "Efficient Depth Image Based Rendering with Edge Dependent Depth Filter and Interpolation," in Multimedia and Expo, 2005. ICME 2005. IEEE International Conference on, 2005, pp. 1314-1317.
[17] I. Daribo, C. Tillier, and B. Pesquet-Popescu, "Distance Dependent Depth Filtering in 3D Warping for 3DTV," in Multimedia Signal Processing, 2007. MMSP 2007. IEEE 9th Workshop on, 2007, pp. 312-315.
[18] T.-C. Lin, H.-C. Huang, and Y.-M. Huang, "Preserving depth resolution of synthesized images using parallax-map-based dibr for 3D-TV," Consumer Electronics, IEEE Transactions on, vol. 56, pp. 720-727, 2010.
[19] L. Pei-Jun and Effendi, "Nongeometric Distortion Smoothing Approach for Depth Map Preprocessing," Multimedia, IEEE Transactions on, vol. 13, pp. 246-254, 2011.
[20] I. J. Cox, S. L. Hingorani, S. B. Rao, and B. M. Maggs, "A maximum likelihood stereo algorithm " Computer Vision and Image Understanding, vol. 63, pp. 542-567, 1996.
[21] C.-J. Tsai and A. K. Katsaggelos, "Dense disparity estimation with a divide-and-conquer disparity space image technique," Multimedia, IEEE Transactions on, vol. 1, pp. 18-29, 1999.
[22] C.-H. Kim, H.-K. Lee, and Y.-H. Ha, "Disparity space image-based stereo matching using optimal path searching," in Proceedings of SPIE–IS&T Electronic Imaging, 2003, pp. 752-760.
[23] X.-H. Lu, F. Wei, and F.-M. Chen, "Foreground-Object-Protected Depth Map Smoothing for DIBR," in Multimedia and Expo (ICME), 2012 IEEE International Conference on, 2012, pp. 339-343.
[24] S.-F. Hsiao, J.-W. Cheng, W.-L. Wang, and G.-F. Yeh, "Low latency design of Depth-Image-Based Rendering using hybrid warping and hole-filling," in Circuits and Systems (ISCAS), 2012 IEEE International Symposium on, 2012, pp. 608-611.
[25] W.-H. Huang, "Real-time novel rendering architecture for 3D display," in The 23rd IPPR Conf.Comput. Vision, Graphics, Image Process. (CVGIP), 2010, pp. 15-17.
[26] 王文玲, "使用動態規劃法之立體視差估算硬體設計," 碩士, 資訊工程學系, 國立中山大學, 2012.
[27] F.-H. Cheng, Y.-W. Chang, and Y.-S. Huang, "A hardware architecture for real-time stereoscopic image generation from depth map," in Machine Learning and Cybernetics (ICMLC), 2011 International Conference on, 2011, pp. 1622-1627.
[28] The Middlebury Computer Vision. Available: http://vision.middlebury.edu/