近年來，立體視覺技術（Stereo Vision）被廣泛的應用於各種應用領域，而深度圖（Depth Map）是產生立體視覺的重要資訊。一般而言，深度圖可由兩張影像經過立體匹配（Stereo Matching）所產生的視差（Disparity）求得，但是藉由立體匹配產生深度圖的計算複雜度高，因此若僅以軟體來實現，往往無法達到即時性的要求。本論文提出一個可達到即時性產生深度圖的立體視覺硬體架構，加速立體匹配產生影像深度資訊的運算。透過輸入兩張左右眼影像後，利用全域性搜尋的動態規劃演算法（Dynamic Programming, DP）來尋找相對的視差向量之步驟中，硬體複雜度較高的三部分為匹配代價計算（Matching Cost Computation, M.C.C.）、最小累計代價（Minimum Cost Accumulation, M.C.A.）與視差值最佳化（Disparity Optimization, D.O.）。本論文探討在M.C.C.模組與M.C.A.模組，拿取左右眼影像進行運算的順序性對硬體成本的影響。另外D.O.模組使用兩種做法實現，一種為Systolic-Like架構，可使硬體模組化、規則化，另一種為使用記憶體來降低硬體成本。由實驗結果得知，本論文最終提出的架構設計配合管線化（Pipeline）與使用記憶體實現D.O.模組，可節省大量的硬體成本並提升連續影像序列之資料運算速度。

Recently, stereo vision has been widely used in many applications, and depth map is important information in stereo vision. In general, depth map can be generated from the disparity using stereo matching based on two input images of different viewing positions. Due to the large computation complexity, software implementation of stereo matching usually cannot achieve real-time computation speed. In this thesis, we propose hardware implementations of stereo matching to speed up the generation of depth map. The proposed design uses a global optimization method, called dynamic programming, to find the disparity based on two input images: left image and right image. It consists of three main processing steps: matching cost computation (M.C.C.), minimum cost accumulation (M.C.A.), and disparity optimization (D.O.). The thesis examines the impact of different pixel operation orders in M.C.C and M.C.A modules on the cost of hardware. In the design of D.O. module, we use two different approaches. One is a Systolic-Like structure with streaming processing, and the other is memory-based design with low hardware cost. The final architecture with pipelining and memory-based D.O. can save a lot of hardware cost and achieve high throughput rate for processing a sequence of image pairs.

中文論文審定書 i
英文論文審定書 ii
中文摘要 iv
Abstract v
致謝 vi
第1章 概論 1
1.1 研究背景 1
1.1.1 立體視覺成因 1
1.1.2 極線幾何與極線幾何限制 3
1.1.3 立體顯示技術 4
1.2 研究動機 9
1.3 本文大綱 9
第2章 相關研究 11
2.1 匹配代價計算（Matching Cost Computation） 11
2.2 代價函數聚合（Cost Aggregation） 13
2.3 視差計算（Disparity Computation） 15
2.3.1 區域性演算法 15
2.3.2 全域性演算法 16
2.4 視差修正（Disparity Refinement） 18
第3章 研究方法及硬體架構設計與實現 21
3.1 參數定義 21
3.2 系統架構 21
3.2.1 彩色影像轉亮度影像（RGB to Intensity） 22
3.2.2 匹配代價計算（Matching Cost Computation） 23
3.2.3 最小累計代價（Minimum Cost Accumulation） 24
3.2.4 視差值最佳化（Disparity Optimization） 26
3.2.5 深度圖轉換（Depth Map Conversion） 27
3.3 硬體架構設計第一版 28
3.3.1 彩色影像轉亮度影像（RGB to Intensity） 28
3.3.2 匹配代價計算（Matching Cost Computation） 29
3.3.3 最小累計代價（Minimum Cost Accumulation） 31
3.3.4 視差值最佳化（Disparity Optimization） 34
3.4 硬體架構設計第二版（管線化） 37
3.5 硬體架構設計第三版（面積最佳化） 38
3.4.1 匹配代價計算（Matching Cost Computation） 41
3.4.2 最小累計代價（Minimum Cost Accumulation） 43
3.4.3 視差值最佳化（Disparity Optimization） 45
第4章 實驗結果分析與比較 47
4.1 週期數分析 47
4.2 邏輯合成數據與分析 48
4.3 軟硬體驗證 50
4.4 測試影像 54
第5章 結論與未來展望 57
5.1 結論 57
5.2 未來展望 57
參考文獻 （References） 59

[1] D. Scharstein, “View Synthesis Using Stereo Vision”, Dissertation of Cornell University PHD, 1997
[2] A. Smolic, et al., “3-D video and free viewpoint video – Technologies, appli- cations and MPEG standards,” in Proc. IEEE Int. Conf. Multimedia Expo., pp. 2161-2164, 2006.
[3] C. Fehn, R. de la Barre, and S. Pastoor, “Interactive 3-DTV—concepts and k- ey technologies,”in Proc. IEEE, vol. 94, no. 3, pp.524-538, 2006.
[4] I. Becton, et al., “Stereoscopic and Autostereoscopic Display Systems,” IEEE Signal Processing Mag, pp. 85-99, May. 1999.
[5] C. Fehn, “A 3D-TV system based on video plus depth information,” in Proc. of Asilomar Conference on Signals, Systems and Computers, vol. 2, pp. 1529-1533, 2003.
[6] Y.-C. Wu, “Image Synthesis Technology for Multi-view 3D Displays,” Master’s Thesis, Dept. of Electrical Engineering, National Cheng Kung University, 2008.
[7] A.F. Bobick and S.S. Intille, “Large Occlusion Stereo,” Int’l J.Computer Vision, vol. 33, no. 3, pp. 1-20, 1999.
[8] D. Scharstein and R. Szeliski, “A taxonomy and evaluation of dense two-frame stereo correspondence algorithms,” Int. J. Comput. Vision,vol. 47, pp. 7–42, 2002.
[9] M. Gong, R. Yang, W. Liang, and M. Gong. “A performance study on different cost aggregation approaches used in realtime stereo matching,” Int. J. Comput. Vision, pp. 283–296, 2007
[10] N. Y.-C. Chang, T.-H. Tsai, B.-H. Hsu, Y.-C. Chen,and T.-S. Chang, “Algorithm and Architecture of Disparity Estimation With Mini-Census Adaptive Support Weight,”in IEEE Transactions on Circuits and Systems for Video Technology, VOL. 20, NO. 6, JUNE 2010
[11] Y. Ruigang and P. Marc, “Multi-Resolution Real-Time Stereo on Commodity Graphics Hardware”, in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp I-211-217, 2003
[12] L. Jiangbo, G. Lafruit, and F. Catthoor, "Fast Variable Center-Biased Windowing for High-Speed Stereo on Programmable Graphics Hardware," in Proceedings of IEEE International Conference on Image Processing, pp. VI - 568-VI – 571, 2007.
[13] X. Chang, Z. Zhou, L. Wang, Y. Shi, and Q. Zhao, “Real-Time Accurate Stereo Matching Using Modified Two-Pass Aggregation and Winner-Take-All Guided Dynamic Programming,” in 3DIMPVT, 2011.
[14] K. J. Yoon and I. S. Kweon, “Locally Adaptive support-Weight Approach for Visual Correspondence Search,” in Proc. IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 924-931, 2005.
[15] L. Wang, M. Liao, M. Gong, R. Yang and D. Nister, "High-Quality Real-Time Stereo Using Adaptive Cost Aggregation and Dynamic Programming," Proc. of IEEE Int’l Symposium on 3D Data Processing, Visualization, and Transmission, Chapel Hill, NC, pp.798-805, June 2006.
[16] K. Zhang, J. Lu, and G. Lafruit, “Cross-based local stereo matching using orthogonal integral images,” IEEE Trans. Circuits Syst. Video Technol., vol. 19, pp. 1073-1079, Sept. 2009.
[17] Y. Yang and S. Du, "A stereo algorithm using edge-based orthogonal dynamic programming," in Proceedings of the 2nd International Conference on Interaction Sciences Information Technology Culture and Human, 2009
[18] I. J. Cox, S. L. Hingorani, S. B. Rao, and B. M. Maggs, "A maximum likelihood stereo algorithm," Computer Vision and Image Understanding, vol.63, no.3, pp.542-567, 1996.
[19] Y. Ohta and T. Kanade, “Stereo by intra- and inter- scanline search using dynamic programming”, IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 7, no. 2, pp. 139–154, 1985.
[20] A. Delong and Y. Boykov, “A scalable graph-cut algorithm for N-D grids,” in Proc. IEEE Conf. on Comput. Vision Pattern Recognition, Jun. 2008.
[21] N.Y.-C. Chang, T.-H. Tsai, B.-H. Hsu, Y.-C. Chen, and T.-S. Chang, “Algorithm and Architecture of Disparity Estimation With Mini-Census Adaptive Support Weight,” IEEE Trans. on CSVT., vol. 20, no. 6, pp. 792-805, June 2010.
[22] J. Sun, N.N. Zheng, and H.Y. Shum, “Stereo matching using belief propagation,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 25, no. 7, pp. 787–800, 2003.
[23] P. Fua, “A Parallel Stereo Algorithm that Produces Dense Depth Maps and Preserves Image Features,” Machine Vision and Applications, vol. 6, pp. 35-49, 1993.
[24] C.-J. Tsai and A. K. Katsaggelos, Fellow, IEEE “Dense Disparity Estimation with a Divide-and-Conquer DisparitySpace Image Technique” IEEE Trans. on Multimedia, vol. 1, NO. 1, MARCH 1999.
[25] C.-H. Kim*, H.-K. Lee, and Y.-H. Ha, “Disparity space image based stereo matching using optimal path searching,” Proceedings of SPIE–IS&T Electronic Imaging, vol.. 5022 ,2003.
[26] A.L. Yuille and T. Poggio, “A Generalized Ordering Constraint for Stereo Correspondence,” A.I. Laboratory Memo 777, MIT, Cambridge, Mass., 1984
[27] W.J. MacLean, S. Sabihuddin, and J. Islam, “Leveraging Cost Matrix Structure for Hardware Implementation of Stereo Disparity Computation Using Dynamic Programming,” Computer Vision and Image Understanding, vol. 114, no. 11, pp.
1126-1138, 2010.
[28] S. Sabihuddin, “Dense stereo reconstruction in a field programmable gate array,” Master’s thesis, University of Toronto, 2008.