本研究旨在調查在觀看三種類型的漢字：象形字、會意字以及形聲字時，信息處理模式的差異性。本研究採實驗設計。研究對象是24位來自南部兩所大學的來台交換的碩博士生以及對外漢語學生，他們都是漢語初學者。當所有受試者在觀看30張由多媒體演示的投影片時，眼動儀同時收集他們的眼動追踪數據。這30張投影片包含10張象形文字，內容包含有插圖以及文字說明，另外10張形聲字和10張會意字，僅有文字說明呈現在電腦屏幕上。眼動儀Tobii的T60分別將注視時間與眼球凝視蹤跡都記錄下來。實驗結果發現：
1. 使用多媒體演示來識別三種不同漢字時，受試者會因為多媒體不同的配置，在識別會意字時的認知負荷明顯高於識別象形文字時的認知負荷。
2. 受試者在三種類別的漢字之總注視時間上，會意字的總注視時間明顯高於其他兩種字型的時間。

The study aimed to investigate the difference between the information processing patterns when viewing three types of Chinese characters: pictograms, associative compounds, and pictophonetic characters with multimedia In addition, participants’ cognitive activities, including cognitive load and information processing when viewing the presentations was also collected to determine the effectiveness of multimedia presentations. In the experiment, twenty-four participants who are exchange postgraduate students and Chinese as Foreign Language students in two universities of southern Taiwan were recruited and their eye-tracking data were collected when viewing multimedia presentations. The participants’ total fixation duration, and fixation count were recorded by a Tobii T60 eye-tracker. Results demonstrated that information processing onscreen text needed greater mental effort than processing on visual information. A significant difference was found in total fixation duration among the three categories of characters (F=24.868, p<0.001). The study found viewers’ total fixation duration on associative compounds was apparently less than that on other two categories of characters.

論文審定書 ....................................................................................................................... i
ACKNOWLEDGEMENTS .............................................................................................. ii
摘 要 ............................................................................................................................. iii
Abstract ............................................................................................................................ iv
TABLE OF CONTENT ................................................................................................... vi
LIST OF FIGURES ....................................................................................................... viii
LIST OF TABLES ......................................................................................................... viii
Chapter 1. Introduction ................................................................................................. 1
1.1 General Research Background ............................................................................... 1
1.2 Research Questions
.................................................................................................................................
5
Chapter 2. Literature Review ........................................................................................ 7
2.1 The Construction of Chinese Characters
......................................................................................
7
2.1.1 Chinese Characters .......................................................................................... 7
2.1.2 Shuowen Jiezi (說文解字) and Liushu (六書) ................................................ 8
2.1.3 Chinese Character Recognition (CCR) .......................................................... 10
2.2. Cognitive Theory and Cognitive Load in Multimedia Learning
..................................
11
2.2.1 Cognitive Theory of Multimedia Learning (CTML) ..................................... 11
2.2.2 Cognitive Load .............................................................................................. 13
2.3 Eye Movement and Cognitive Processes
...................................................................................
15
Chapter 3 Method ......................................................................................................... 18
3.1 Participants ........................................................................................................... 18
3.1.1 Materials ........................................................................................................ 18
3.2. Equipment and procedures .................................................................................. 20
3.3 Data Analysis
...........................................................................................................................................
23
Chapter 4 Result ........................................................................................................... 25
4.1 Cognitive Load Rating .......................................................................................... 25
4.2 Total Fixation Duration
......................................................................................................................
26
Chapter 5 Discussion and Conclusion ........................................................................ 32
5.1 Summary of the Findings and Discussion ............................................................ 32
5.2 Implications .......................................................................................................... 34
5.3 Suggestion for the future research
................................................................................................
36
Reference ....................................................................................................................... 37

Brünken, P., Plass, J.L., & Leuther D. (2004). Direct measurement of cognitive load
in multimedia learning. Educational Psychologist, 38(1), 53–61
Chen, B., & Feng, Y. (2000). Determinants of economic growth in China: Private
enterprise, education, and openness. China Economic Review, 11(1), 1-15
Chen, H.Y., & Liu, K.Y. (2008). Web-based synchronized multimedia lecture system
design for teaching/learning Chinese as second language. Computers & Education,
50(3), 693-702. doi: 10.1016/j.compedu.2006.07.010
Cheng, L., & Liu, S. J. and Nakagawa, M. (2004). Online recognition of Chinese
characters- the state-of-the-art. IEEE Trans Pattern Anal Mach Intell, 26(2).
Chuang, H.H., & Liu, H.C. (2011). Effects of different multimedia presentations on
viewers’ information-processing activities measured by eye-tracking
technology. Journal of Science Education and Technology, 21(2), 276-286.
doi: 10.1007/s10956-011-9316-1
Chung, K.H. (2003). Effects of pinyin and first language words in learning of Chinese
characters as a second language. Journal of Behavioral Education,12(3), 207-223.
Dakwar, K., Froud, K., & Gordon, P. (2012). Acquiring diglossia: mutual influences of
formal and colloquial Arabic on children's grammaticality judgments. J Child Lang,
39(1), 61-89. doi: 10.1017/S0305000910000784
Du, J., Zhai, J.F., Hu, J.S., Zhu, B., Wei, S., & Dai, L.R. (2015, August). Writer adaptive
feature extraction based on convolutional neural networks for online handwritten
Chinese character recognition. Paper presented at 13th International Conference
in Tubis.
Feldman, L.B., & Siok, W.W.T. (1997). The role of component of function in visual
recognition of Chinese characters. Journal of Experiment Psychology: Learning,
Memory, and Cognition, 23(3), 776-781.
Furman, N., Goldberg, D., & Lusin, N. (2010). Enrollments in languages other than
English in United States Institutions of Higher Education. The Modern Language
Association of America.
Gerjets, P., & Scheilter, K. (2003). Goal configurations and processing strategies as
moderators between instructional design and cognitive load - evidence from
hypertext-based instruction. Educational Psychologist, 38(1), 33-41.
Hegarty, M. (1992). The mechanics of comprehension and comprehension of
mechanics. Eye movement and visual cognition: Scene perception and reading
(pp. 428–443). New York: Springer Verlag.
Hegarty, M., & Just, M. A. (1993). Constructing mental models of machines from text
and diagrams. Journal of Memory and Language, 32, 717–742.
Jarodzka, H., Scheiter, K., Gerjets, P., & van Gog, T. (2010). In the eyes of the
beholder: How experts and novices interpret dynamic stimuli. Learning and
Instruction, 20(2), 146-154.
Jiao, Y., & Peng, Y. (2011). A Study on Eye Movement of Thai Students Reading
Chinese Texts with or without Marks for Word Boundaries. IEEE Computer
Society, 35, 291-294. doi:10.1109/ialp.
Just, M. A., & Carpenter, P. A. (1976). Eye fixations and cognitive processes.
Cognitive Psychology, 8, 441–480.
Kalyuga, S. (2000). When using sound with a text or picture is not beneficial for l
earning. Australian Journal of Educational Technology, 16(2), 161-172.
Kalyuga, S., Chander, P., & Sweller, J. (2004). When redundant on-screen text in
multimedia technical instruction can interfer with learning. Human Factors, 46(3),
567-581.
Kirschner, P. A. (2002). Cognitive load theory- implications of cognitive load theory on
the design of learning. Learning and Instruction, 12(10), 1-10.
Kou, Z. H., & Tan, C.F. (2009). Globalization of Chinese - an analysis of changes to t
teaching Chinese as a foreign language. Journal of Basic Education, 18(1).
Lee, C. H., & Kalyuga, S. (2011). Effectiveness of different pinyin presentation formats
in learning Chinese characters: a cognitive load perspective. Language Learning,
61(4), 1099-1118. doi: 10.1111/j.1467-9922.2011.00666.x
Lee, C. H., & Kalyuga, S. (2011). Effectiveness of on-screen pinyin in learning
Chinese: An expertise reversal for multimedia redundancy effect. Computers in
Human Behavior, 27(1), 11-15. doi: 10.1016/j.chb.2010.05.005
Liang, M., Kao, A., &. Hooper, S. (2004). The effects of visual and verbal coding
mnemonics on learning Chinese characters in computer-based instruction.
ETR&D, 52(3), 23-38.
Liu, C. L., Jaeger, S., & Nakagawa, M. (2004). Online recognition of Chinese
characters: the state-of-the-art. IEEE Trans Pattern Anal Mach Intell, 26(2), 198-
213. doi:10.1109/TPAMI.2004.1262182
Liu, C.L., Weng, J.Y., Chuang, Y.H., & Tsai, J.L. (2010). An exploration of native
speakers' eye fixations in reading Chinese text. 66-71. doi: 10.1109/taai.2010.22
Liu, H.C., Lai, M.L., & Chuang, H.H. (2011). Using eye-tracking technology to
investigate the redundant effect of multimedia web pages on viewers’ cognitive
processes.Computers in Human Behavior, 27(6), 2410-2417. doi:
10.1016/j.chb.2011.06.012
Li, Q., & Po, Y.X. (2014). Click on the ancient culture of the core concepts of
translation rhetoric - with the "Liu Shu" namely. Contemporary Rhetoric 183(3).
Mayer, R. E., Heiser, J., & Lonn, S. (2001). Cognitive constraints on multimedia
learning: When presenting more material results in less understanding. Journal of
Educational Psychology, 93(1), 187-198. doi: 10.1037/0022-0663.93.1.187
Mayer, R.E., & Moreno, R. (2003). Nine ways to reduce cognitive load in multimedia
learning. Cognitive Load in Multimedia Learning Educational Psychologist, 38(1),
45-52.
Merriënboer, V., Jerocen, J. G., & Sweller, J. (2005). Cognitive load theory and
complex learning: Recent developments and future directions. Educational
Psychology Review, 17(2), 147-177. doi: 10.1007/s10648-005-3951-0
Paas, F., Renkl, A., & Sweller, J. (2003). Cognitive load theory and instructional
design-recent developments. Cognitive Load Theory and Instructional Design:
Recent Developments, Educational Psychologist, 38(1), 1-4.
Peng, D.I. (1995). Role of early presemantic phonological code in Chinese character
Identification. Journal of Experimental Psychology: Learning, Memory, and
Cognition 21(1), 43-54.
Pérez-Milans, M. (2014). Mandarin Chinese in London education- language
aspirations in a working-class secondary school. Working Papers in Urban
Language & Literacies (pp.129).
Plassa, J.L., Chunb, D.M., Mayerb, R.E., & Leutnerc, D. (2003). Cognitive load in
reading a foreign language text with multimedia aids and the influence of verbal
and spatial abilities. Computers in Human Behavior 19, 221-243.
Rayner, K. (1990). Eye Movements and Visual Cognition: Introduction (pp.1-7).
Springer-Verlag New York Inc.
Rayner, K. (1998). Eye movements in reading and information processing. 20 years of
research. Psychological Bulletin, 124(3), 372-422.
Rayner, K., Li, X.S., Juhase, B.J., & Yan, G. (2005). The effect of word predictability on
the eye movements of Chinese readers. Psychonomic Bulletin & Review,12(6),
1089-1090.
Shu, H., Chen, X., Anderson, R.C., Wu, N., & Xuan, Y. (2003). Properties of school
Chinese-implications for learning to read. Child Development, 74(1), 27-47.
Sorden, S. D. (2012). The Cognitive Theory of Multimedia Learning. In B. J. Irby, G.
Brown, & S. Jackson (Eds.), The Handbook of Educational Theories:
Information Age Publishing, Incorporated.
Sweller, J. (1994). Cognitive load theory, learning difficulty, and instructional design.
Laming and Insbuction, 4(199), 293-312.
Tabbers, H.K., Martens, R.L., & Merrienboer, J. J. G. (2004). Multimedia instructions
and cognitive load theory-Effects of modality and cueing. British Journal of
Educational Psychology -The British Psychological Society, 74, 71-81.
Tsukumo, J., & Tanaka, H. (1998). Classification of hand-printed Chinese characters
using non-linear normalization and correlation Methods. IEEE.
Tang, Y.Y., Tu, L.T., Liu, J., Lee, S.W., Lin, W.W., & Shyu, I.S. (1998). Offline
recognition of Chinese handwriting by multifeature and multilevel classification.
IEEE Transactionson Pattern Analysis and Machine Intelligence, 20(20), 556-561.
Underwood G, Radach R (1998) Eye guidance and visual information processing:
reading, visual search, picture perception and driving (pp 1–28). Oxford:
Elsevier Science,
Wang H., & Guan, M. (2010). The research and analysis on junior high school’s
Chinese educational reform based on cognitive load theory 299(3), 17-19.
doi: 10.1109/ICEIT.2010.5608369
Wang, Y.W., Zheng, Y.W., Shen, D.-L., Cui, L., & Yan, G.-L. (2013). Reading another’s
mind and reading the interactive mind from Chinese idioms: evidences
from eye-movements and ERPs. Acta Psychologica Sinica, 44(1), 100-111.
doi: 10.3724/sp.j.1041.2012.00100
Wang, Z.Y. (2014). Chinese character recognition method based on image processing
and hidden markov model. 276-279. doi: 10.1109/isdea.2014.68
Wieger, S.J. (1965) Chinese characters: Their origin, etymology, history, classification
and signification. A Thorough Study From Chinese Documents (pp.10). New York:
Dover Publications, Inc.
Wolfgang, S., & Kürschner, C. (2007). A reconsideration of cognitive load theory.
Educational Psychology Review, 19(4), 469-508. doi: 10.1007/s10648-007-9053-4
Yan, G., Tian, H., Bai, X., & Rayner, K. (2006). The effect of word predictability on the
eye movements of Chinese readers. British Journal of Psychology, The British
Psychological Society, 97, 259-268.
Yan, G., Tian, H., Bai, X., & Rayner, K. (2006). The effect of word and character
frequency on the eye movements of Chinese readers. Br J Psychol, 97(2), 259-
268. doi: 10.1348/000712605X70066
Ye, L.J. (2011). Teaching and learning Chinese as a foreign language in the United
States: To delay or not to delay the character introduction (Doctoral dissertations,
College of Art and Sciences of Georgia State University. Retrieved from
http://scholarworks.gsu.edu/cgi/viewcontent.cgi?article=1021&context=alesl_diss
Yeh, S.L., & Li, J.L. (2002). Role of structure and component in judgments of visual
similarity of Chinese characters. Journal of Experimental Psychology:
Human Perception and Performance, 28(4), 933-947. doi: 10.1037/0096-
1523.28.4.933
Yu, P. (2013). A study on Japanese students (intermediate level) reading Chinese
texts with or without marks for word boundaries. Asian Language Processing
international Conference (pp. 273-277). IEEE doi: 10.1109/ialp.2013.38
Zhen, Z. (2015). Advantages and thinking on design of Chinese characters’ graphics.
Studies in Literature and Language, 11(2), 82-87. doi: 10.3968/7447