近年來,隨著機器人工業的發展,機器人開始出現在各式不同的任務中,尤
其是和人有密切的接觸的任務,這些任務需要因應不同的情境與社會規範,給予
使用者不同的回饋,例如接待型機器人、娛樂型機器人或是寵物機器人等,這類
的機器人除了外表愈來愈加擬人化外,也加入了許多心智狀態的設計,例如前幾
年 Sony 出的寵物狗'AIBO',除了可愛的外表與動作外,還放入了情緒的機制,讓
使用者在與其互動上感到更為親切有趣。因此本研究的目標是透過生物機制的計
算化,將情緒系統做的更加擬人,期望可以和使用者達成親密的互動。本研究最
後透過了兩種情境實驗測試,成功的學習出刺激與獎勵的對應,並且產生出對應
的情緒。

Robots began to appear in a variety of different tasks with the development of robot
industrial in recent years, in particular the tasks have close contact with people. These
tasks require in response to different situations and social norms to give users different
feedback, such as reception robots, entertainment robots or pet robots. Some of the tasks
requires responses to people in different situations, in particular the task which has close
relationship with people. These robots in addition to having more anthropomorphic
appearance, but also added a lot of the mental state of the design. For example, Sony
had developed the pet dog 'AIBO' a few years ago. In addition to the lovely appearance
and actions, it also has the emotion mechanisms which allow users feel interesting to
interact. Therefore, the goal of this study is using computational biological mechanisms
to build an more anthropomorphic emotion system. Finally, this study has two situations
of experiment. The system has success in learning the relation between stimulations and
rewards, and generating the corresponding emotion.

論文審定書 ....................................................................................................................... i
中文摘要 .......................................................................................................................... ii
英文摘要 ......................................................................................................................... iii
第一章 序論 .................................................................................................................... 1
1.1 研究背景 ........................................................................................................... 1
1.2 研究動機與目的 ............................................................................................... 2
第二章 文獻探討 ............................................................................................................ 3
2.1 情緒 ................................................................................................................... 3
2.2 神經調節系統 ................................................................................................. 10
2.3 制約學習 .......................................................................................................... 13
2.4 強化式學習 ...................................................................................................... 14
第三章 研究方法 .......................................................................................................... 16
3.1 系統架構 ......................................................................................................... 16
3.2 系統參數 ......................................................................................................... 28
第四章 實驗與結果分析 .............................................................................................. 31
4.1 實驗設計 .......................................................................................................... 31
4.2 實驗情境 .......................................................................................................... 31
4.3 實驗結果與評估 .............................................................................................. 32
第五章 結論 .................................................................................................................. 36
5.1 研究結果 .......................................................................................................... 36
5.2 未來研究 .......................................................................................................... 36
參考文獻 ........................................................................................................................ 38

1. Dang, T.-H.-H., G. Hutzler, and P. Hoppenot, How to Gain Emotional Rewards during Human-Robot Interaction Using Music? Formulation and Propositions, L. Rutkowski, et al., Editors. 2010, Springer Berlin / Heidelberg. p. 247-255.
2. Cattinelli, I., M. Goldwurm, and N. Borghese, Interacting with an artificial partner: modeling the role of emotional aspects. Biological Cybernetics, 2008. 99(6): p. 473-489.
3. Robi. Available from: http://robi.tw/.
4. Samani, H.A. and A.D. Cheok. Probability of love between robots and humans. in Intelligent Robots and Systems (IROS), 2010 IEEE/RSJ International Conference on. 2010.
5. Ziemke, T. and R. Lowe, On the Role of Emotion in Embodied Cognitive Architectures: From Organisms to Robots. Cognitive Computation, 2009. 1(1): p. 104-117.
6. Matsumoto, D. and P. Ekman, Japanese and Caucasian facial expressions of emotion and neutral faces1988: els autors.
7. Becker-Asano, C. and I. Wachsmuth, Affective computing with primary and secondary emotions in a virtual human. Autonomous Agents and Multi-Agent Systems, 2010. 20(1): p. 32-49.
8. Arbib, M.A. and J.-M. Fellous, Emotions: from brain to robot. Trends in Cognitive Sciences, 2004. 8(12): p. 554-561.
9. Jianwei, L., Y. Haifeng, and W. Xufa. A hormone-modulated emotional model. in Computer Engineering and Technology (ICCET), 2010 2nd International Conference on. 2010.
10. Nishida, T., Modeling Machine Emotions for Realizing Intelligence – An Introduction –, in Modeling Machine Emotions for Realizing Intelligence Foundations and Applications, T. Nishida, L.C. Jain, and C. Faucher, Editors. 2010, Springer Berlin Heidelberg. p. 1-15.
11. Seung-Ik, L., P. Gunn-Yong, and K. Joong-Bae. Natural Emotion Expression of a Robot Based on Reinforcer Intensity and Contingency. in Robotics and Automation, 2007 IEEE International Conference on. 2007.
12. Carter, R., 大腦的祕密檔案2011.
13. Ortony, A., G.L. Clore, and A. Collins, The cognitive structure of emotions1990.
14. Carter, R., Mapping the Mind1999.
15. Silber, K., The Physiological Basis of Behaviour: Neural and Hormonal Processes1999.
16. Pessoa, L., On the relationship between emotion and cognition. Nat Rev Neurosci, 2008. 9(2): p. 148-158.
17. Qingji, G., W. Kai, and L. Haijuan, A Robot Emotion Generation Mechanism Based on PAD Emotion Space, Z. Shi, E. Mercier-Laurent, and D. Leake, Editors. 2008, Springer Boston. p. 138-147.
18. Steunebrink, B.R., M. Dastani, and J.J.C. Meyer. The OCC model revisited. 2009.
19. 高木繁治, 大腦構造地圖2011.
20. Doya, K., Metalearning, neuromodulation, and emotion. Affective minds, 2000. 1: p. 101-104.
21. 葉重新, 心理學(簡明版)2006.
22. 馮觀富, 情緒心理學2005.
23. Arbib, M. USC Brain Project. Available from: http://neuroinformatics.usc.edu/.
24. Khamassi, M., et al., Robot cognitive control with a neurophysiologically inspired reinforcement learning model. Frontiers in neurorobotics, 2011. 5.
25. Mirolli, M., F. Mannella, and G. Baldassarre, The roles of the amygdala in the affective regulation of body, brain, and behaviour. Connection Science, 2010. 22(3): p. 215-245.
26. Niv, Y., Reinforcement learning in the brain. Journal of Mathematical Psychology, 2009. 53(3): p. 139-154.
27. Schweighofer, N. and K. Doya, Meta-learning in reinforcement learning. Neural Networks, 2003. 16(1): p. 5-9.
28. Akiguchi, S. and Y. Maeda. Emotion Behavior Learning System Based on Meta-Parameter Control of Q-Learning with plural Q-values. in Proceedings of the 3rd International Symposium on Autonomous Minirobots for Research and Edutainment (AMiRE 2005). 2006. Springer.
29. Shi, X., Z. Wang, and Q. Zhang, Artificial Emotion Model Based on Neuromodulators and Q-learning, in Future Control and Automation2012, Springer. p. 293-299.