近年來以人型機器人與人類互動為主題的研究逐漸流行，隨著人型機器人處理事情的複雜度越來越高，透過人類與人型機器人互動的控制方式將會是一種趨勢。由於人型機器人於機構設計上近似人體架構，因此機器人可以藉由模仿示範者的姿態來實現高複雜度的任務。本論文以人型機器人NAO實現對示範者姿態的模仿，其內容分成兩大部分：對人類動作中的姿態進行擷取，以及在機器人重現姿態的過程中，透過學習的方式重建其平衡感。首先，利用體感控制器來捕捉人類的動作，並將其骨架中的關節資訊組成骨架姿態。在人類連續的動作中，包含著許多高關聯性的姿態。本論文將動作中相對重要的姿態擷取出來，並以分群的方法合併相似的關鍵姿態。當機器人重現姿態的時候，往往會因不平穩而跌倒，利用加強式學習使機器人透過分析穩定情況來學習姿態平衡。最後，達到完整重現示範者姿態的目的。

In the recent years, the research of the interaction between human and humanoid robot becomes a popular topic. As the level of difficulty of the task that robots dealing with increases, the controlling method by the interaction between human and robots is going to be a tendency. Because of the design of humanoid robots, it allows robots represent a difficult task by imitating the posture of a demonstrator. This thesis uses the robot NAO to accomplish the imitation of the demonstrator’s posture. It includes two parts in this thesis: the extraction of posture in human motion and rebuilding the sense of balance of robot via learning method. First, we use the somatosensory devices to capture the motion of human and transfer the information of body joint into the skeleton posture. In the continuous motion, there are a lots of highly relative postures. This thesis extracts the important postures, called key postures, and clusters the similar key postures via clustering method. While the robot represents the human posture, it often falls down since the unbalance. Consequently, we use Reinforcement Learning to make robots learn the sense of balance by analyzing the stable situation. Finally, the robots complete the purpose of representing the posture of demonstrator.

摘要 i
ABSTRACT ii
TABLE OF CONTENTS iii
LIST OF FIGURES v
LIST OF TABLES ix
LIST OF ALGORITHMS x
LIST OF SYMBOLS xi
I. INTRODUCTION 1
1.1 Motivation 1
1.2 Thesis Organization 2
II. BACKGROUND 3
2.1 Motion Capture 3
2.1.1 Microsoft Kinect 3
2.1.2 Natural User Interface 4
2.2 Humanoid Robot 5
2.2.1 Aldebaran Robot NAO 5
2.2.2 NAOqi 6
2.3 Key Posture Identification Algorithm 7
2.4 Online Clustering Method 8
2.5 Reinforcement Learning 10
III. PROPOSED METHOD 13
3.1 Posture Data Acquisition 13
3.1.1 Skeleton Posture 13
3.1.2 Joint Angle Computing 14
3.2 Key Posture Extraction 16
3.3 Posture Clustering 18
3.4 Balance Learning 19
3.4.1 State Space 19
3.4.2 Action Space 22
3.4.3 Reward Function 24
3.4.4 Learning Process 25
IV. EXPERIMENT RESULTS 28
4.1 Implementation: Posture Data Acquisition 28
4.2 Implementation: Key Posture Extraction 34
4.3 Experiments: Balance Learning 39
4.3.1 Single Support Posture 39
4.3.2 Double Support Posture 43
V. CONCLUSIONS AND FUTURE WORK 48
REFERENCE 49

[1] D. Lee, C. Ott, Y. Nakamure, and G. Hirzinger, “Physical Human Robot Interaction in Imitation Learning,” IEEE International Conference on Robotics and Automation, 2011
[2] V. V. Nguyen, and J. H. Lee, “Full-Body Imitation of Human Motions with Kinect and Heterogeneous Kinematic Structure of Humanoid Robot,” IEEE/SICE International Symposium on System Integration, pp. 93-98, 2012
[3] Y. Chen, Q. Wu, X. He, C. Du, and J. Yang, “Extracting Key Postures in A Human Action Video,” IEEE 10th Workshop on Multimedia Signal, pp. 569-573, 2008
[4] D. Y. Chen, H. Y. M. Liao, H. R. Tyan, and C. W. Lin, “Automatic Key Posture Selection for Human Behavior Analysis,” IEEE 7th Workshop on Multimedia Signal, pp. 1-4, 2005
[5] M. Nakada, B. Allen, S. Morishima, and D. Terzopoulos, “Learning Arm Motion Strategies for Balance Recovery of Humanoid Robots,” International Conference on Emerging Security Technologies, 2010
[6] K. S. Hwang, and F. W. Kuo, “Memorization and Replay of Humanoid Robot Actions by Emulation,” master thesis, National Sun Yat-sen University, 2014
[7] K. Y. Yu, and Y. H. Sun, “Static Stable Regions According to Center of Press to Quantify Humanoid Walking Stability,” IEEE International Symposium on Robotic and Sensors Environments, pp. 172-177, 2011
[8] K.D. Nguyen, I.M. Chen, S.-H. Yeo, and B.-L. Duh, “Motion Control of a Robotic Puppet through a Hybrid Motion Capture Device,” IEEE International Conference on Automation Science and Engineering, pp. 753-758, 2007
[9] L. Cheng, Q. Sun, H. Su, Y. Cong, and S. Zhao, “Design and Implementation of Human-Robot Interactive Demonstration System Based on Kinect,” Chinese Control and Decision Conference, pp. 971-975, 2012
[10] H. I. Lin, and C. C. Chou, “Humanoid Robot Imitation Using Kinect,” International Conference on Advanced Robotics and Intelligent Systems, pp. 1-4, 2015
[11] Kinect for windows, “Developer technologies,” only available online: https://dev.windows.com/en-us/kinect
[12] D. Gouaillier and P. Blazevic, “A Mechatronic Platform, the Aldebaran Robotics Humanoid Robot,” IEEE Annual Conference on Industrial Electronics, pp. 4049-4053, 2006
[13] Aldebaran Robotics, “NAO Documentation,” 2012, only available online: www.aldebaran-robotics.com/documentation
[14] W. C. Jiang, “The Applications of Model Learning and Knowledge Sharing of Multi-Agents for Dyna-Q Architecture,” Ph.D dissertation, National Chung Cheng University, 2013
[15] K. S. Hwang, W. C. Jiang, and Y. J. Chen, “Model Learning and Knowledge Sharing for a Multiagent System with Dyna-Q Learning,” IEEE Transactions on cybernetics, Vol. 45, No. 5, pp. 978-990, 2015
[16] R. S. Sutton, and A. G. Barto, Reinforcement Learning: An Introduction, MIT Press, Cambridge, 1998
[17] Leslie Park Kaebling, Michael L. Littman, and Andrew W. Moore, “Reinforcement Learning: A survey,” Journal of Artificial Intelligence Research 4, pp. 237-285, 1996
[18] Watkins, C. J. C. H., Learning from Delayed Rewards. Ph.D. thesis, Cambrideg University, 1989
[19] Watkins, C. J. C. H., and P. Dayan, “Technical note: Q-Learning,” Machine Learning, 8(3-4): pp. 279-292, 1992
[20] S.F. Huang and W.-J. Wang, “Human’s Postures Recognition by Using Kinect,” National Symposium on System Science and Engineering, pp.187-192, 2011
[21] A. Mukerjee, P. Guha, and A. Kar, “Skeletal Tracking using Microsoft Kinect,” Methodology, 2010
[22] N. Kofinas, “Forward and Inverse Kinematics for the NAO Humanoid Robot,” diploma thesis, Technical University of Crete, Greece, 2012
[23] K. S. Hwang, J. L. Lin, T. C. Huang, and H. J. Hsu, “Humanoid Robot Gait Imitation,” IEEE SICE Annual Conference, pp. 2124 – 2128, 2014
[24] C. L. Hwang, B. L. Chen, H. H. Huang, and H. T. Syu, “Hybrid Learning Model and MMSVM Classification for On-Line Visual Imitation of a Human with 3-D Motions,” Journal of Robotics and Autonomous Systems, Vol. 71, pp. 150-165, 2015