手勢控制已經成為人機互動的方式之一，特別是在鍵盤滑鼠或觸控螢幕輸入有困難的情境下，手勢輸入似乎成為一個重要的選項，例如：車輛內部控制、機器人控制。本論文以個人電腦為研究平台，借助於Leap Motion Controller高精度的3維手勢追蹤能力，實作一個能以手勢輸入的資訊交換系統。從Leap Motion Controller取得手部於三維立體空間的座標後，經由類神經網路以機器學習的方式進行手勢分類辨識，然後以手勢指揮控制檔案上傳至雲端空間進行分享，其中結合2種動態手勢，與4種靜態手勢，選擇檔案分享的對象，與對電腦下達指令與操控。
在本系統中，透過即時的手勢辨識與操控，我們可以快速選定目標檔案與分享對象，略過應用程式或網頁操作的步驟來達成上傳或分享的目的，以本地端主導檔案分享，以實現無鍵盤滑鼠操控機器的應用。最後，論文評估一些系統效能指標與複雜度。

Hand gesture control has become an important human-machine interaction interface, especially when in a situation keyboard/mouse or touch panel are not available. In our study, with the help of Leap Motion Controller’s high precision of 3D gesture tracking capabilities, we developed a desktop application for information exchange. After obtaining the coordinates of hands in three-dimensional space from Leap Motion Controller, we design a neural network algorithm to recognize hand gesture, and then to share the selected files to target receivers. We combine two dynamic and four static hand gestures to map to commands to select files and send to destinations to share with others.
The key contribution of our thesis is the learning algorithm of hand gesture recognition. We are using neural network method to design the algorithm. Through the machine learning concept, we input dataset from Leap Motion controller to perform the learning phase, and then the learning results are used in the decision phase. We illustrate some performance results to show the recognition rate. The rate must be more than 90% to be usefully applied in practical application. In addition, the time must be less one second. We apply this algorithm in the information exchange and get acceptable performance. Thus, hand gesture recognition is a workable solution for human-machine communication.

論文審定書 i
誌謝 ii
摘要 iii
Abstract iv
目錄 v
圖次 viii
表次 ix
第一章 序論 1
1.1 研究動機與目的 1
1.2 論文架構 3
第二章 背景知識 4
2.1 手勢辨識相關研究 4
2.1.1 手勢辨識之方法 4
2.1.2 手勢辨識之類型 7
2.2 檔案分享 10
2.3 Leap Motion Controller 11
2.3.1 Leap Motion Controller簡介 11
2.3.2 3D追蹤資料 13
2.4 雲端儲存空間 15
2.4.1 基本介紹 15
2.4.2 Google Drive 16
2.4.3 OAuth 2.0 17
2.5 人工類神經網路 19
第三章 系統架構與功能介紹 22
3.1系統架構 22
3.2系統流程 23
3.3 系統環境 24
3.3.1 系統開發環境 24
3.3.2 系統測試環境 25
3.4 系統功能介紹 26
3.4.1 登入認證 26
3.4.2 滑鼠模擬 27
3.4.3 虛擬鍵盤 29
3.4.4 辨識手勢與確認目標 30
3.4.5 上傳與分享檔案 34
3.5 系統需求 35
3.5.1 軟體需求 35
3.5.2 硬體需求 35
第四章 系統實作 36
4.1 類神經網路 36
4.1.1 訓練程式 36
4.2 帳號系統 40
4.3 手勢追蹤與辨識 41
4.3.1 滑鼠模擬 41
4.3.2 手勢辨識 42
4.4 檔案上傳 44
4.4.1 Google Drive檔案構成 44
4.4.2 創建資料夾 45
4.4.3 獲取檔案資訊 46
4.4.4 對應Google Drive MIME Type 47
4.4.5 上傳或分享至Google Drive 47
4.4.6 上傳至自有雲端儲存空間 48
第五章 實驗數據與系統成果分析 49
5.1 實驗方法 49
5.2 實驗數據 49
第六章 結論與未來展望 52
參考文獻 53
附錄 58

[1] V. I. Pavlovic, R. Sharma, and T. S. Huang, "Visual interpretation of hand gestures for human-computer interaction: a review," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 19, pp. 677-695, 1997.
[2] J. Ga, x, ka, M. M, x, sior, et al., "Inertial Motion Sensing Glove for Sign Language Gesture Acquisition and Recognition," IEEE Sensors Journal, vol. 16, pp. 6310-6316, 2016.
[3] L. A. F. F. Vitor F. Pamplona, Jo˜ao L. Prauchner ,Luciana P. Nedel,Manuel M. Oliveira, "The Image-Based Data Glove," Proceedings of X Symposium on Virtual Reality, pp. 204-211, 2008.
[4] 許明翔, "具有深度學習精神之人類手勢影像辨識系統," 碩士, 自動化科技研究所, 國立臺北科技大學, 台北市, 2015.
[5] T. Ahmed, "A Neural Network based Real Time Hand Gesture Recognition System," International Journal of Computer Applications, vol. 59, pp. 17-22, 2012.
[6] 王俊偉, "高效率基於手勢辨識的行動裝置滑鼠模擬系統," 碩士, 資訊工程系所, 國立臺北科技大學, 台北市, 2015.
[7] H.-H. H. Trong-Nguyen Nguyen, "Static Hand Gesture Recognition Using Artificial Neural Network," Journal of Image and Graphics, vol. 1, pp. 34-38, 2013.
[8] R. Agrawal and N. Gupta, "Real Time Hand Gesture Recognition for Human Computer Interaction," in 2016 IEEE 6th International Conference on Advanced Computing (IACC), 2016, pp. 470-475.
[9] Y. Lai, C. Wang, Y. Li, S. S. Ge, and D. Huang, "3D pointing gesture recognition for human-robot interaction," in 2016 Chinese Control and Decision Conference (CCDC), 2016, pp. 4959-4964.
[10] L. Yuhai, L. Jing, and J. Zhaojie, "Data fusion-based real-time hand gesture recognition with Kinect V2," in 2016 9th International Conference on Human System Interactions (HSI), 2016, pp. 307-310.
[11] C. H. Chuan, E. Regina, and C. Guardino, "American Sign Language Recognition Using Leap Motion Sensor," in Machine Learning and Applications (ICMLA), 2014 13th International Conference on, 2014, pp. 541-544.
[12] K. Y. Fok, N. Ganganath, C. T. Cheng, and C. K. Tse, "A Real-Time ASL Recognition System Using Leap Motion Sensors," in Cyber-Enabled Distributed Computing and Knowledge Discovery (CyberC), 2015 International Conference on, 2015, pp. 411-414.
[13] L. E. Potter, J. Araullo, and L. Carter, "The Leap Motion controller: a view on sign language," in Proceedings of the 25th Australian Computer-Human Interaction Conference: Augmentation, Application, Innovation, Collaboration, Adelaide, Australia, 2013, pp. 175-178.
[14] J. K. Sharma, R. Gupta, and V. K. Pathak, "Numeral Gesture Recognition Using Leap Motion Sensor," in 2015 International Conference on Computational Intelligence and Communication Networks (CICN), 2015, pp. 411-414.
[15] 黃雅琦, "利用攝影機二維影像做三維手勢追蹤," 碩士, 資訊科學與工程研究所, 國立交通大學, 新竹市, 2015.
[16] M. H. Hsu, T. K. Shih, and J. S. Chiang, "Real-Time Finger Tracking for Virtual Instruments," in International Conference on Ubi-Media Computing and Workshops (UMEDIA), 2014, pp. 133-138.
[17] J. Y. R. McCartney, and H.-P. Bischof, "Gesture Recognition with the Leap Motion Controller," Conference on Image Processing, Computer Vision, & Pattern Recognition, vol. 12, pp. 3-9, 2015.
[18] R. B. Mapari and G. Kharat, "Real time human pose recognition using leap motion sensor," in IEEE International Conference on Research in Computational Intelligence and Communication Networks (ICRCICN), 2015, pp. 323-328.
[19] H. H. Chang, C. H. Huang, T. K. Lin, T. K. Shih, and S. Wu, "Real-time virtual instruments based on neural network system," in International Conference on Ubi-Media Computing (UMEDIA), 2015, pp. 163-167.
[20] G. Marin, F. Dominio, and P. Zanuttigh, "Hand gesture recognition with leap motion and kinect devices," in IEEE International Conference on Image Processing (ICIP), 2014, pp. 1565-1569.
[21] G. Chanhan and P. Chandhari, "Gestures based wireless robotic control using image processing," in 2015 5th Nirma University International Conference on Engineering (NUiCONE), 2015, pp. 1-7.
[22] H. Cheng, L. Yang, and Z. Liu, "A Survey on 3D Hand Gesture Recognition," IEEE Transactions on Circuits and Systems for Video Technology, vol. PP, pp. 1-1, 2015.
[23] H. S. Hasan and S. A. Kareem, "Human Computer Interaction for Vision Based Hand Gesture Recognition: A Survey," in Advanced Computer Science Applications and Technologies (ACSAT), 2012 International Conference on, 2012, pp. 55-60.
[24] 林宗勳, Support Vector Machines 簡介. Available: http://www.cmlab.csie.ntu.edu.tw/~cyy/learning/tutorials/SVM2.pdf, (2016-08-21)
[25] L. Breiman and A. Cutler, Random Forests. Available: http://www.stat.berkeley.edu/~breiman/RandomForests/cc_home.htm#intro, (2016-08-21)
[26] M. Elmezain, A. Al-Hamadi, J. Appenrodt, and B. Michaelis, "A Hidden Markov Model-based continuous gesture recognition system for hand motion trajectory," in Pattern Recognition, 2008. ICPR 2008. 19th International Conference on, 2008, pp. 1-4.
[27] Hidden Markov Model. Available: http://www.csie.ntnu.edu.tw/~u91029/HiddenMarkovModel.html, (2016-08-21)
[28] C. Hong, D. Zhongjun, and L. Zicheng, "Image-to-Class Dynamic Time Warping for 3D hand gesture recognition," in 2013 IEEE International Conference on Multimedia and Expo (ICME), 2013, pp. 1-6.
[29] Wikipedia, Dynamic time warping. Available: https://en.wikipedia.org/wiki/Dynamic_time_warping, (2016-08-21)
[30] L.-W. Chen, Y.-F. Ho, W.-T. Kuo, and M.-F. Tsai, "Intelligent file transfer for smart handheld devices based on mobile cloud computing," International Journal of Communication Systems, 2015.
[31] F. Weichert, D. Bachmann, B. Rudak, and D. Fisseler, "Analysis of the Accuracy and Robustness of the Leap Motion Controller," Sensors, vol. 13, p. 6380, 2013.
[32] J. S. Artal-Sevil and J. L. Monta, "Development of a robotic arm and implementation of a control strategy for gesture recognition through Leap Motion device," in 2016 Technologies Applied to Electronics Teaching (TAEE), 2016, pp. 1-9.
[33] Leap Motion, Leap Motion C# SDK Documentation. Available: https://developer.leapmotion.com/documentation/index.html?proglang=current, (2015-10-12)
[34] M. Anicas, An Introduction to OAuth 2. Available: https://www.digitalocean.com/community/tutorials/an-introduction-to-oauth-2, (2014-07-21)
[35] DTREG, A Brief History of Neural Networks. Available: https://www.dtreg.com/solution/view/21, (2016-08-21)
[36] M. Mazur, A Step by Step Backpropagation Example. Available: https://mattmazur.com/2015/03/17/a-step-by-step-backpropagation-example/, (2015-03-17)
[37] G. Theodoropoulos, Accessing Google Services Using the OAuth 2.0 Protocol. Available: http://code.tutsplus.com/tutorials/accessing-google-services-using-the-oauth-20-protocol--mobile-18394, (2016-08-21)
[38] Microsoft, Windows app development. Available: https://msdn.microsoft.com/library/windows/desktop/ff657751.aspx, (2016-08-21)
[39] Google, Google Developers – Drive Rest API. Available: https://developers.google.com/drive/v3/web/about-sdk (2016-08-21)
[40] L. Lawton, Damito – Google API C#. Available: http://www.daimto.com/google-drive-api-c/, (2013-08-13)