World Academy of Science, Engineering and Technology 44 2008

Interactive Chinese Character Learning System

though Pictograph Evolution
J.H. Low, C.O. Wong, E.J. Han, K.R Kim K.C. Jung, and H.K. Yang


Abstract²This paper proposes an Interactive Chinese Character Chinese character is evolved through pictograph evolution
Learning System (ICCLS) based on pictorial evolution as an initiates our idea in this paper. Therefore, a learning mechanism
edutainment concept in computer-based learning of language. The involves interactive Chinese character learning through
advantage of the language origination itself is taken as a learning
pictograph evolution is proposed.
platform due to the complexity in Chinese language as compared to
other types of languages. Users especially children enjoy more by Although not all Chinese characters are created from
utilize this learning system because they are able to memories the pictograph evolution, but our target is towards pre-school
Chinese Character easily and understand more of the origin of the children aged between 4 to 6 years old. Therefore it is sufficient
Chinese character under pleasurable learning environment, compares to use the limited characters that are able to be represented
to traditional approach which children need to rote learning Chinese through pictograph evolution as pre-school children starts
Character under un-pleasurable environment. Skeletonization is used
learning things mostly through a more visualized method using
as the representation of Chinese character and object with an animated
pictograph evolution to facilitate the learning of the language. Shortest pictures or objects around them.
skeleton path matching technique is employed for fast and accurate Taking the advantage of learning through visual and mind
matching in our implementation. User is required to either write a mapping [2], our proposed system is deployed on the similar
word or draw a simple 2D object in the input panel and the matched concept but directly focusing on the learning process of Chinese
word and object will be displayed as well as the pictograph evolution characters. Learning the Chinese character involves visual
to instill learning. The target of computer-based learning system is for
recognition and continuous practice and this creates a problem
pre-school children between 4 to 6 years old to learn Chinese
characters in a flexible and entertaining manner besides utilizing of frustrating the young learners and creating an unpleasant
visual and mind mapping strategy as learning methodology. learning experience. Therefore, our approach is relevant to ease
the learning and to certain extend providing an exciting and fun
Keywords²Computer-based learning, Chinese character, in learning the language.
pictograph evolution, skeletonization. The proposed interactive Chinese character learning system
is designed based on the original creation of the character itself.
I. INTRODUCTION Our mechanism involves matching procedure whereby children

M ASTERING Chinese character is a difficult task because

Chinese character is categorized as sinograph and
alphabet-less type of language. On average, a student with well
are required to either sketch an image or write a word as an input
and the matching will obtain the related image, word or
pictograph evolutionary animation to re-enforce the learning
language aptitude requires 1320 hours of instruction in order to process. In other words, our proposed system uses simple
achieve level 2 language proficiency compared to the matching techniques to match the character or object drawn by
Latin-based languages which the latter requires 450 hours [1]. freehand and relate it with an animated pictorial evolution of the
As for young children to learn and recognize a language, a Chinese character as a visual and mind mapping strategy.
special mechanism towards effective learning is desired to Skeletonization is applied as a representation of the sketched
advance in learning the Chinese language. image and word. Skeletonization is chosen as it is a flexible and
Realizing the complexity in learning and memorizing the efficient shape representation using a compact shape
shape of Chinese characters, our motivation to research on an representation that is able to maintain the original structure,
assistive platform based on the grounding base of how the topology and physical appearance of the original shape.
Shortest skeletonization path similarity Matching technique is
J.H. Low, C.O. Wong and K.R. Kim are postgraduate student of the HCI used based on Xiang et. al [3].
Lab, Soongsil University, Seoul, Korea (e-mail: [email protected], The notable contribution in this paper includes presenting an
[email protected], [email protected]).
E.J. Han is a Post-Doctorial at the School of Media and Graphic
interactive way to learn Chinese character based on their origin
Interdisciplinary Centre , University of British Columbia, Vancouver, Canada of creation, using one algorithm for both sketched image and
(e-mail: [email protected]). word matching instead of two different matching mechanism
K.C. Jung is a professor at the School of Digital Media, Soongsil University, based on shortest path similarity concept. There are several
Seoul, Korea (e-mail: [email protected]).
H.K Yang is a professor at the Department of Computer Information advantages in our proposed method. It is suitable to match both
Engineering, Dongseo University, Busan, Korea (e-mail: singularity appearance object and non-singularity appearance
[email protected]).

299
 World Academy of Science, Engineering and Technology 44 2008

object. In addition, simple statistical method is used for image just allows the user to remember the Character by image rather
and word matching utilizing Hungarian algorithm [16] as the than understand the unique structure of the Chinese character.
matching strategy between query and model. In terms of technology wise, skeletonization approach is
The remaining of this paper is structured as follow. In section implemented to match the Chinese character. Bai et al. have
II, we discuss the related work on learning and matching used skeletonization approach to represent the shape [3]. This
Chinese character. Section III outlines the main concept and line like presentation is taking advantage of preserving the
proposed system. Section 4 shows the proposed method used in originality by minimum deformation representation. For the
our system and system prototype. A conclusion is presented in matching procedure, beside just consider the shortest skeleton
the next section. path between end node for matching, a modification is added by
considering the shortest skeleton path between the junction
II. RELATED WORK node.
Our work is related to interactive Chinese characters learning.
Much of the application had been done base on this direction III. MAIN CONCEPT
with different approaches. Some of the related work on research The concept behind this paper deals with interactive and
and system application is described in this section. effective learning of Chinese characters for pre-school children.
There are several applications that have been developed to Utilizing the concept of pictograph evolution and simple
help in the Chinese language learning. Tang et al. have matching techniques for both words and pictures, a new
proposed a web-based Chinese handwriting system that allows approach in learning Chinese characters is introduced. This
student to learn Chinese handwriting at anytime and anyway [4]. approach is consistent with the concept of pictograph
This system combines several important technologies that are representation that is based on the oldest form of the Chinese
able to provide intelligent and effective learning environment to character classification [9][10].
users. This system also able to reduce the time used to learn Our proposal is intended to solve the problem of difficulty to
Chinese character compare with traditional approach. learn Chinese characters for pre-school children and utilize a
Chen at al. have developed augmented interface for Children simple and fast matching technique that does not require heavy
Chinese learning [5]. Basically, this study tries to create a more computation. The system is targeted to ease learning and create
efficient and interesting learning environment through fun in learning as learning Chinese character is proven to be a
Augmented Reality. In addition, this research describes the difficult task especially involves memorization of each
exertion of AR to construct a leaning system, where rich and character [11][12][13]. Learning the character has no other way
effective interactions in leaning were considered. The prototype except heavy memorization as each of the character differs with
of this AR system enables school children to have a better their meaning and pronunciation.
understanding in pronunciation and memorization of Chinese.
A. Chinese Character Classification Based on Pictograph
Several applications have been developed specifically for
Evolution
memorizing the Chinese character. Flash card is one of the
learning tools that aid in memorizing the Chinese characters. In Chinese is a language with the largest number of users in the
general, flashcards are widely used as a learning drill to aid world, and the Chinese character is created with the longest in
memorization by way of space repetition. [6] and [7] have history of language. It is very hard to determine the time when
created an online Chinese character flashcards application. has the Chinese character emerged and the character that we see
Children might able to remember Chinese characters by using it today are the character on the tortoise shells and animal bones.
for several time but this blindly memorize approach may cause Therefore, Chinese characters are very unique in shape and each
boringness to the users. character has a story behind the stone.
Another approach that has been adapted by the researcher to There are several principles that are used to define and
DLGOHDUQLQJDQGPDVWHULQJWKH&KLQHVHFKDUDFWHUVLV³JUDEDQG explicate the Chinese character and one of the principles is
match it´ as in [8]. In general, children are required to pictographs. Pictographic Chinese character is composed of
recognize and match the relevant character that enables children different symbols that often resembled the objects that it
to remember the Chinese character. This approach is an represents and sometimes are combined in some way to
interactive approach by allowing the user to grab those represent a whole new word or idea. After an evolution of time,
randomly moving Chinese character and match it base on those symbol or diagram are simplified and adjusted as Chinese
understanding. Nevertheless, this idea generally is similar to the writing to be further developed and expanded. So, each
flash card whereby the user still needs to memorize blindly on pictographic Chinese character has a story behind the shape as
the Chinese character before using the application. No prior its appearance. Today, roughly 600 Chinese characters are
knowledge on the character will result much errors or leading to created based on the pictograms principles and among those are
frustration and terminate to use the system further. claimed to be the oldest form of character. Figure 1 shows the
Word and pictures is another example of application that had example of the Chinese character falling in the pictogram
been developed for Chinese characters learning as in [7]. A
category and its evolution. In this system, the advantage of the
diagram comes along with a Character and enable user to learn
pictogram Chinese characters is taken whereby each Chinese
the Chinese character visually. Nevertheless, this application
character is an image vision of character. This will drastically
facilitate the learning of the character besides creating fun in

300
 World Academy of Science, Engineering and Technology 44 2008

learning. drawing. The same matching technique is used to match the

words or pictures and in return display the output as in Figure 4.
B. Interactive Chinese character learning system (ICCLS)
,Q)LJXUHWKHH[DPSOHVKRZVDQLQSXWRIDZRUG³ቅ´DVDQ
In the recent years, computer-based learning has emerged to LQSXW E\ WKH XVHU DQG WKH SLFWXUH RI D ³PRXQWDLQ´ UHWULHYHG
be a new genre of instructional media which utilized the together with the animated pictorial representation displayed to
capabilities of multimedia personal computer to animate enhance the learning in the children. In this way, the user will be
software designed to both educate and entertain young children. able to visualize and relate the word with the object and be able
Learning Chinese character is more relevant to have a fun and to remember the word in future.
proper learning method due to the nature of its complexity.
Therefore, our proposed system consists of the elements of
visual, mind mapping, learning and entertaining.

Fig. 2 System overview

Fig. 1 Chinese character in evolution

Figure 2 shows the interactive Chinese character learning

system overview. The user in our domain is pre-school children
aged between 4 to 6 years who are starting to learn languages,
specifically learning the Chinese characters. The child sits on a
computer terminal and able to input Chinese words or sketch a
simple picture on the input panel and the matching takes place to
display the object or words together with an animation of how Fig. 3 Input panel for writing and drawing
the Chinese character evolved from the pictograph evolution.
The input from the sketch panel representing either a word or
a simple 2D picture will first represented using a skeletonization
approach. Based on the skeletonization representation of the
2D Image
input, the shortest skeleton path pair between end node and
junction node is generated. Therefore, each input and model is
represented by a sequence of shortest skeleton path between the
end node pair and junction node pair. The dissimilarity distance
matrix between the model and query is generated. Next, based
on the dissimilarity matrix, the matching procedure is taken Animation
place by using the Hungarian algorithm and the possible
matched result is generated. If a word is drawn on the input
panel, matching of the word will identify the correct word and at
the same time retrieve the image and animation about the word
out from the database and vice versa.
Our proposed system has an advantage of flexibility to input
as a word or as a sketch image. Both inputs use the same Fig. 4 Output panel
algorithm although they are from two types of representation.
The animated pictograph evolution is a learning strategy that we
propose for children to learn Chinese characters. It also serves IV. OVERALL MACTHING PROCEDURE AND SYSTEM
as an edutainment element in the computer-based learning REPRESENTATION
system thus creating a sense of entertaining while learning a new
Figure 5 shows the overall structure of matching technique that
language that is closely related to visual representation.
is implemented in our system. This system starts with using
Figure 3 shows an example of input panel for our prototype better skeletonization method to generate the skeleton [14].
system. The input panel accepts both forms of input; word and Based on the skeleton, our system computes the quantity of the

301
 World Academy of Science, Engineering and Technology 44 2008

end node and junction node, and the matching procedure is utilize it for Chinese character representation for matching.
conducted within the chosen category whereby the category is
C. Chinese Character Representation
define by the quantity of the end node and junction node. On the
other hand, Chinese character representation for matching is We had modified Bai et al object representation for our Chinese
computed base on the generated skeleton. We use the shortest character representation [3]. This representation uses the shortest
skeleton path of each end node pair to represent an object in 2D. Since
path similarity for matching. Lastly, the matched data will be
Chinese character can be recognized as objects in 2D, we have
displayed to the user in the viewing panel. included junction node shortest skeleton pair to represent the Chinese
character. End node is a node that only has 1 neighbor node and
junction node is a node that has more than 2 neighbor nodes. Figure 6
and 7 shows a representation of the Chinese character ³⬏´ draw) and
³㕞´ goat), and the representation set is listed as Table 1.

)LJ5HSUHVHQWDWLRQRI&KLQHVHFKDUDFWHU³⬏´

Fig. 5 Overview of the matching procedure

A. Input Fig. 7 5HSUHVHQWDWLRQRI&KLQHVHFKDUDFWHU³㕞´

Since we are targeting on low cost and common system, we TABLE I

use the mouse as an input device in CLLS. We have set the SET REPRESENTATION OF CHINESE CHARACTER
brush size to 30 points as a default input. This is due to the Chinese Shortest skeleton path Set in pair.
robustness issue especially for pre-school children to handle Character
mouse input and write a character or draw a simple picture it is ⬏ {{(1,2),(3,4)},{(a,b),(b,c),(d,b),(e,b)}}
much more easier that the font appears to be thicker 㕞 {(1,9),(2,3),(4,8),(5,7),(6,7)}
representation. In general, more noise is generated by using a
mouse to write. Therefore, we use better skeleton method to
generate the skeleton because this skeletonization method is This representation is particularly good to integrate into our
able to reduce the noise which is generated by strokes written learning system because our system is a small scale system.
with a mouse as input. Hungarian algorithm is used as a simple statistical matching
approach to implement this representation. Our argument for
B. Skeletonization using a simple statistical Chinese character matching instead of
Skeletonization is a shape deformation technique that is complex algorithm is due to a more cost effective and time
introduced by H. Blum as the result of Medial transform [15]. A saving approach [3].
very explanatory definition of the skeleton is given by In our representation, each shortest path distance is calculated
prairie-fire analogy which the boundary of an object is set on based on the geodesic distance. It is the actual distance along the
fire and the skeleton is the loci where the fire fronts meet the path. Each pair of shortest skeleton path is represented by a
quench each other. The advantage of skeletonization is ensuring series of pixels that connects between the end node pair or
the topology and structure of the original shape under the junction node pair whereby each pixel is represented by the
deformation situation. Therefore, skeletonization is an approach distance transform, dt, of the original Chinese character shape.
that uses minimum information to represent a shape and we The representation is shown in mathematic equation as follows:

302
 World Academy of Science, Engineering and Technology 44 2008

sampling shortest skeleton path of end node pair and junction

Ens ,ne {dt1 , dt2 ,..., dtn } (1) node pair.

J ns ,ne {dt1 , dt2 ,..., dtn } (2)

n 1

Equation (1) denotes the shortest skeleton path between each ¦ dt

o
n
end node, E and equation (2) denotes as the shortest skeleton dt
path between each junction node, J. No (5)
Next, sampling took place by sampling K nodes along the ( Ens ,ne ( g ))
path for each shortest skeleton end node pair and junction node ( NEns ,ne ( g )) g 1,2,..., K
pair. Let Ens , ne ( g ) denotes the sampling distance transform dt
{dt1 , dt2 ,..., dt3}
pixels with index g and J ns , ne ( g ) denotes the sampling
distance transform pixels with index g as shown in the equation
dt (6)
below: ( J ns ,ne ( g ))
( NJ ns ,ne ( g )) g
Ens ,ne ( Ens ,ne ( g )) g 1,2,..., K
1,2,..., K
dt
{dt( p1 1) (1), dt n (2),..., dt n (K ) {dt1 , dt2 ,..., dt3}
( p2 p1  ) ( p g p g 1  )
K K
(3) dt (7)
J ns ,ne ( J ns , ne ( g )) g 1,2,..., K D. Database
{dt( p1 1) (1), dt n (2),..., dt n (K ) Our database contains 50 pictogram Chinese characters
( p2 p1  ) ( p g p g 1  )
K K which are suitable for pre-school children of aged between 4-6
(4) years old. We had classified those Chinese characters by the
Figure 8 is a visualized explanation of equation (3) and (4). need of our matching system based on the node type of Chinese
Let the highlighted pixels as the shortest skeleton path of end character. Table 2 shows the category of Chinese character and
node pair mark with distance transform value and K = 5. Since some example data of each category in our database. The
the shortest skeleton path is 25, therefore: purpose of classification is to increase the matching speed by
minimize the brute force search
Ens ,ne {i, ii, iii, iv, v}
TABLE II
DIFFERENT CATEGORIES OF CHINESE CHARACTER TYPE
Chinese Character Type Example
Singularity with Looping ³Ⳃ´
Non-singularity with Looping ³᱊´
Singularity without Looping ³໽´
No-singularity without Looping ³ᵫ´
Singularity combine Looping ³⬉´
Non-singularity combine Looping ³⬏´

E. System Prototype
A prototype system is developed to test the efficacy of the
proposed method for Chinese character learning. Figure 9
Fig. 8 Shortest skeleton path between the nodes shows the interface of our prototype with a breakdown example
of input, output and animated pictograph evolution. User can
input a word or a picture of their preference.
Next, we normalize the sampling nodes based on the average The figure shows user input a ZRUG³ቅ´ZLWKWKHPRXVH,W
distance transform of the shape that is able to solve the scale OLWHUDOO\ PHDQV ³PRXQWDLQ´ 7KHUHIRUH LQ WKH 2XWSXW VHFWLRQ
problem. The average of distance transform as in equation (5) is WKHDFFXUDWHZRUGRI³ቅ´LVGLVSOD\HGDORQJZLWKDQLPDJHRI
equal to the sum of all distance transform in Chinese character ³PRXQWDLQ´LVUHWULHYHG7KHDQLPDWHGSLFWRJUDSKHYROXWLRQRI
divide by the size of the Chinese character in pixels, No. Each KRZWKHZRUG³ቅ´LVGHULYHGIURPWKHREMHFW³PRXQWDLQ´EDVHG
shortest skeleton path between end node and junction node will on pictograph evolution theory is also displayed to the user. Fig.
be normalized in respect to the average distance transform of the 10 shows animation sequence of the evolution of Chinese
original Chinese character as in equation (6) and (7). Therefore, Character from the object itself to the word for the word ³ቅ´ and
each Chinese character is represented by a normalized set of the ³∈´.

303
 World Academy of Science, Engineering and Technology 44 2008

Fig. 9 Prototype system overview

V. LEARNING PEDAGOGY Fig. 10 Evolution sequence of Chinese Character ³ቅ´ DQG³∈´ that
The key pedagogy learning of Chinese language is listening, displayed as animation
reading, writing, memorizing and speaking [13]. Our system is
Our approach uses character and object matching with
emphasis on memorizing learning pedagogy because the
skeletonization and shortest path matching algorithm to identify
existing computer-based learning of memorizing is no very
the written input both in words and in sketched diagram in order
helpful enough as discussed in III Main Concept.
to relate it with pictograph evolution of the Chinese character.
Our interactive learning system provides a frame work for
Our strategy is to use the early evolution of Chinese character as
student centered learning focus. Our system acts as an assistant
a learning mechanism involving visual and mind mapping
for children to memories the Chinese Character. Besides that,
concept to instill the learning of the language.
our system further extern by adding the animation display that
Statistical matching technique using Hungarian algorithm is
allow children to understand the origin and evolution of the
Chinese Character, from the shape itself to the word itself as able to solve the problem of single paired end node as well as
shows in Fig. 10. This allows creating an interesting, fancy and junction nodes. This special characteristic in Chinese character
funny element in the process of Chinese Character learning. is able to match using the mentioned algorithm. We also take the
Diagram and animation is utilized in our learning system advantage of using a pictograph evolution animation as a tool
because we taking on the advantage of image are ease to allow for learning the character in a more effective manner. The
the children to remember [2] the appearance of the Chinese animated pictograph evolution also serves as an entertaining
Character since diagram is able to create a profundity memory component in learning the language. This
in their mind. Therefore, the evolution of the Chinese Character
and profundity memory is use as the pedagogy in our interactive ACKNOWLEDGMENT
Chinese Character learning system and this idea is able to This research was supported by the Ministry of Culture,
reduce the burden of the children in memories of complex Sports and Tourism, Korea, under the CTRC (Culture
languages learning especially in Chinese Character learning.
Technology Research Center) support program supervised by
the ITRC (Information Technology Research Center) support
program.
VI. CONCLUSION
We proposed a new approach in edutainment for pre-school REFERENCES
children learning Chinese language. Chinese character is a
[1] :%D[WHU³How hard is Chinese"´ The Defense Language Institution,
complex character in linguistic. Learning the character require Monterey, California. Available:
memorization and practice. Therefore, learning could be http://www-personal.umich.edu/~wbaxter/howhard.html
frustrating especially to pre-school children. [2] A. Hollingworth, C.C. Wiliam and J.M. Henderson, ³To see and
remember: Visually specific information is retained in memory from

304
 World Academy of Science, Engineering and Technology 44 2008

previously attend objects in natural scenes,´ Psychonomic Society

Publication, Phychomic Bulletin & Review.8,4 (Dec 2001), 761-768(8).
[3] X. Bai and L.J. Latecki, ³Path Similarity Skeleton Graph Matching,´
PAMI, IEEE Trans, Jul 2008, Vol.30, No.7, pp 1282-1292.
[4] K.T. Tang, K.K. Li and H.A. Leung, ³Web-based Chinese handwriting
education system with automatic feedback and analysis,´ LNCS
Springer, Oct 2006, Vol 4181/2006, session 3, pp 176-188.
[5] C.H. Chen, C.C. Su, P.Y. Lee and F.G. Wu, ³Augmented Interface for
Children Chinese Leaning,´ ICALT 2007, IEEE, July 2007, pp 268-270.
[6] E. Peterson, ³Chinese charater flashcards,´ 1996-2005 Online Chinese
Tool. Available: http://www.mandarintools.com/flashcard.html
[7] J. Lau, ³Yellow Bridge flash cards,´ Yellow Bridge-Chinese Language
Center, 2003-2008. Available :
http://www.yellowbridge.com/chinese/flashcards.php
[8] J. Winters, ³Language Lab Project: Activity Builder.´ Available :
http://www.csulb.edu/labs/langlabs/builder/
[9] L. Wieger, ³Chinese characters: Their Origin, Etymology, history,
classification and signification a thorough study from Chinese
documents,´ Dover Publication, 1965. Available :
http://books.google.com/books?hl=en&lr=&id=odrkZvbqJQoC&oi=fnd
&pg=PA1&dq=history+of+chinese+character+&ots=qjOysgvFIL&sig=J
cWQKE2zG0YmZdFAfg6OKL2Hy-o#PPA2,M1
[10] Y.X. Jiang, ³Carvings may rewrite history of Chinese characters,´
Xinhua net, May 2007. Available :
http://news.xinhuanet.com/english/2007-05/18/content_6118003.htm
[11] ³How hard is it to learn Chinese?,´ Jan 2006. Available :
http://news.bbc.co.uk/2/hi/uk_news/magazine/4617646.stm
[12] Y.B. Liu, ³7HDFKHU- student talk in Singapore Chinese language
classrooms: a case study of initiation/response/follow-up (IRF),´ Asia
Pacific Journal of Education, Mar 2008, Vol. 28, Issue 1, pp. 87-102.
[13] C. Tan, ³Change and continuity: Chinese Language Policy in Singapore,´
Language Policy, Springer, Mar 2006 Vol. 5, No. 1, pp. 41-62(22).
[14] D.Reniers and A. Telea, ³Tolerance-based Feature Transforms,´ VISAPP,
Springer, Feb 2006, Vol. 4, pp 187-200.
[15] A.K. Robert and M.P. Stephen, ³Untangling the Blum Medial Axis
Transform,´,-&9, 2003, Vol. 55, Issue 2-3, pp. 139-153.
[16] J.G. Wang, P.K. He and W. Coo, ³Study on the Hungarian algorithm for
the maximum likelihood data associated problem,´ Journal of System
Engineering and Electronics, Mar 2007, Vol. 18. Issues 1, pp 27-32.

305