近年來，立體視覺技術(Stereo Vision)被廣泛的推廣利用在各應用領域，包括遊戲，無人自動駕駛、物件辨識等，而深度資訊是立體視覺的重要資訊之一。一般而言，深度影像是利用兩張影像進行立體匹配所產生的視差，純粹使用軟體來實現，要達到即時性的要求，往往比較難以實現。本論文實作了利用動態規劃演算法(Dynamic Programming, DP)的即時深度計算軟硬體整合系統，並且實際接上兩台攝影機，透過攝影機輸入左視角與右視角之影像資訊，即時的處理每一份進來的資訊，並實際透過FPGA實現，這過程中有不同的實現方式，展現了軟硬體分工的加速成果，實作中也考慮影像的前處理(影像鏡頭校正)，最後透過螢幕呈現演算法運算的結果。論文中探討以不同的軟硬體協同設計方式，並分析其不同方式的優缺點和改進方法

Stereo vision is widely used in many computer vision applications including games, autonomous driving, object recognition, etc. Depth is the key information in stereo vision. In general, depth map is generated by stereo matching computation of two input images captured by cameras at different view angles. In this thesis, we use FPGA SoC platforms to realize a real-time dynamic programming-based stereo matching algorithm where the left and right input images are captured real-time and the computed depth maps are shown on screen. Image rectifications are also considered during the implementations. We study and analyze various hardware-software co-design options and improve the performance using different hardware platform environments.

中文摘要 i
Abstract ii
第1章 概論 1
1.1 研究背景 1
1.1.1 立體成像技術 1
1.1.2 深度獲取 4
1.2 研究動機 5
1.3 本文大鋼 5
第2章 相關研究 6
2.1 立體影像匹配 6
2.2 Dynamic Programming Algorithm 8
2.2.1 彩色影像轉亮度影像(RGB to Intensity) 9
2.2.2 匹配代價計算(Matching Cost Computation) 10
2.2.3 最小累計代價(Minimum Cost Accumulation) 11
2.2.4 視差值最佳化(Disparity Optimization) 12
2.2.5 深度圖轉換(Depth Map Conversion) 13
2.3 實現平台 14
2.3.1 編譯用軟體 15
第3章 實現流程與架構 18
3.1 Standalone 版本 18
3.1.1 硬體動作流程 18
3.2 Linux version 20
3.2.1 在Zedboard上安裝Linux 作業系統 21
3.2.2 環境安裝 22
3.2.2.1 攝影機校正 22
3.2.3 系統架構圖 25
3.3 Hardware only version 28
3.3.1 環境安裝 28
3.3.1.1 攝影機選擇 28
3.3.1.2 影像輸入與連接 29
3.3.1.3 IIC 溝通與頻率同步問題 29
3.3.1.4 IIC Command 30
3.3.1.5 動作流程圖 31
3.4 版本比較 32
第4章 硬體實現結果與數據 33
4.1 Standalone 版本 33
4.2 Linux版本 35
4.2.1 鏡頭校正 35
4.2.2 DP深度演算法 37
4.2.2.1 完成深度圖 37
4.2.2.2 演算法軟硬體分工與時間 39
4.3 Hardware only 版本 42
4.3.1 完成深度圖 42
4.4 三版本數據比較 44
第5章 問題分析與討論 45
5.1 Standalone 版本 45
5.2 Linux 版本 45
5.2.1 DP已經無法完整寫入Zedboard 46
5.3 Hardware only 版本 47
第6章 結論與未來展望 48
6.1 結論 48
6.2 未來展望 48
參考文獻 49

[1] W.-L. Wang, ”Hardware Design for Disparity Estimation Using Dynamic Programming,” Master thesis, Department of Computer Science, National Sun Yat-Sun University, 2012.
[2] W.-H. Zhou, et al., “Real-time Implementation of Panoramic MosaicCamera based on FPGA,” IEEE International Conference on Real-time Computing and Robotics, June 6-9, 2016.
[3] S. Jin, et al., ”FPGA Design and Implementation of a Real-Time Stereo Vision System,” IEEE Trans. Circuits and Systems for Video Technology, , vol. 20, no. 1, Jan., 2010.
[4] I. J. Cox, et al.,“A maximum likelihood stereo algorithm,” Computer Vision and Image Understanding, vol. 63, no.3, pp.542-567, 1996.
[5] C.-J. Tsai and A.K. Katsaggelos, “Dense Disparity Estimation with a Divide-and-Conquer DisparitySpace Image Technique,” IEEE Trans. Multimedia, vol. 1, no. 1, Mar.1999.
[6] D.-W Yang, et al., “Depth-Reliability-Based Stereo-Matching Algorithm and Its VLSI Architecture Design,” IEEE Trans. on Circuits and Systems for Video Technology, , vol. 25, no. 6, June 2015.
[7] C. Ttofis and T. Theocharides, “High-Quality Real-Time Hardware Stereo Matching Based on Guided Image Filtering,” Design, Automation & Test in Europe Conference & Exhibition, pp. 1-6, 2014.
[8] Y.-W Lee, “Vision based Parking Assistance Systems,” Master Thesis, Department of Automation Technology, National Taipei University, 2009.
[9] D. Scharstein and R. Szeliski, “A taxonomy and evaluation of dense two-framestereo correspondence algorithms,” Int. J. Comput. Vision, vol. 47, pp. 7–42, 2002.
[10] Y. Ohta and T. Kanade, “Stereo by intra- and inter- scanline search using dynamicprogramming,” IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 7,no. 2, pp. 139–154, 1985.
[11] Zedboard Hardware User’s Guide. [Online]. Available: http://zedboard.org/sites/default/files/documentations/ZedBoard_HW_UG_v2_2.pdf
[12] OV7670 datasheet. [Online]. Available: https://www.voti.nl/docs/OV7670.pdf
[13] J. D , et al., “Real-Time System for High-Image Resolution Disparity Estimation,” IEEE Trans. Image Processing, , vol.16, no. 1, Jan., 2007.

[14] K. Jawed, et al., “Real Time Rectification for Stereo Correspondence,” International Conference on Computational Science and Engineering, 2009.
[15] H.-S Seong, C.-E Rhee, H.-J Lee, “A Novel Hardware Architecture of the Lucas-Kanade Optical Flow for Reduced Frame Memory Access,” IEEE Trans. Circuits and Systems for Video Technology, , vol. 26, no.6, June 2016.
[16] Y. Ebisawa, “Head-Free, Remote Eye-Gaze Detection System Based on Pupil-Corneal Reflection Method With Easy Calibration Using Two Stereo-Calibarated Video Cameras,” IEEE Trans. Biomedical Engineering, , vol. 60, no. 10, Oct. 2013.
[17] T.-H Tsai, et al., “Algorithm and Architecture of Disparity Estimation With Mini-Census Adaptive Support Weight,” IEEE Trans. Circuits and Systems for Video Technology, vol. 20, no. 6, June 2010.
[18] C.-K Liang, “Hardware-Efficient Belief Propagation,” IEEE Trans. Circuits and Systems for Video Technology, vol. 21, no. 5, May 2011.
[19] Y.-C Tseg, et al., “Architecture Design of Belief Propagation for Real-Time Disparity Estimation,” IEEE Trans. Circuits and Systems for Video Technology, , vol. 20, no. 11, Nov. 2010.
[20] Y.-C. Wu, “Image Synthesis Technology for Multi-view 3D Displays,” Master Thesis, Dept. of Electrical Engineering, National Cheng Kung University, 2008.