an application of neural network for extracting arabic word roots

An Application of Neural Network for Extracting

Arabic Word Roots

HASAN M. ALSERHAN & ALADDIN S. AYESHCentre for Computational Intelligence (CCI)

School of Computing

Faculty of Computing Sciences and Engineering

De Montfort University, Leicester

UNITED KINGDOM

{alserhan, aayesh}@dmu.ac.uk

Abstract:- A good Arabic stemming algorithm is needed in many applications such as: naturallanguage processing, computerized language translation, compression of data, spells checking andin information retrieval. Arabic language is considered to be one of the world’s most complicatedlanguages due to the complexity of its morphological structure, so it has huge morphologicalvariations and rules. Majority of the existing Arabic stemming algorithms use a large set ofrules and many algorithms also refer to existing pattern and root files. This requires largestorage and access time. In this paper, a novel numerical approach for stemming Arabic wordsis described. The present approach attempts to exploit numerical relations between charactersby using backpropagation neural network. The empirical positive results show that the stemmingproblem can be solved by neural network.

Key-Words:- Backpropagation Neural Networks, Stemming, Natural Language Processing(NLP), Arabic Language.

1 Introduction

Stemming refers to the process of reducing to-kens to root/stem form of words to recognizemorphological variation. This done by remov-ing word affixes (prefixes and suffixes). For ex-ample, the words computing, computer, com-putation, computerize, and computers could bemapped to a common stem, comput-. In textsearch, this permits a search for ”computers”to find documents containing all words with thestem comput-

Affixes in Arabic, unlike in English, are pre-fixes, suffixes and infixes. Prefixes are attachedat the beginning of a word, where suffixes areattached at the end of the word, and infixes arefound in the middle of the word. For exam-

ple, the Arabic word ” �HAÓñÊªÖÏ @” ”alma’lwmat”,

which means ”information”, consists of the fol-lowing parts: ”ÕË @” represent prefix-part , ” �

H@”

represent suffix-part , two ”ð”, which is in themiddle, represents infix-part , and the remain-ing letters ”ÕÎ«” represent the root-part .

Stemming can be used in many applicationssuch as in data compression, in spell checking,and in information retrieval (IR) systems wheremany studies showed that using roots as an in-dex word in IR gives much better results thanusing full words. It also can be used in text gen-eration to generate different part-of-speech of agiven word by attaching a certain affixes to theroot verb. In spite of the rapid research con-ducted in other languages, Arabic language stillsuffers from the shortages of the researchers and

1

Proceedings of the 10th WSEAS International Conference on COMPUTERS, Vouliagmeni, Athens, Greece, July 13-15, 2006 (pp637-641)

development.

2 Related Works

2.1 Arabic Language Overview

Arabic is the language spoken by over 200 millionpeople throughout the Middle East and NorthAfrica. Plus, at least another 35 million speakArabic as a second language. It is one of thesix official languages of the United Nations, thelanguage of Islam and its holy book the Qur’aan,and the language in which some of the world’sgreatest works of literature, science and historyhave been written.

Arabic language is considered to be a mem-ber of a highly sophisticated category of naturallanguages, which has a very rich morphology. Inthis category one root can generate several dif-ferent words that have different meaning.

Like other Semitic languages, Arabic wordsare generally based on a root that uses three con-sonants to define the underlying meaning of theword. Each Arabic word can be reduced to, orstems from, a root. 90% of Arabic roots are 3letters, few are 4 letters and fewer still are 5 let-ters. Various vowels, prefixes and suffixes areused with the root letters to create the desired in-flection of meaning. For example, table 1 showssome words based on the root ”I.

�J»” ”ktb” which

means ”write”, and each word has its own mean-ing.

Arabic Pronunciation MeaningFormI.

�KA¿ kaatib a writer

H. A�JºË@ alketab the book

�éJ.�JºÓ maktab an office

I.�JºK yaktubu he writes

I.�J» @ ’aktaba he dictated

Table 1: Some of derived forms of the root ”I.�J»”

The vowels and the non-root letters (affixes)give the word its specific meaning. The patternof vowels and non-root letters itself carries mean-

ing. For example, the pattern ”mC1C2aC3”(where C is a root letter) means ”place”, so mk-tab ”I.

�JºÓ” means ”place of writing”, mal¨ab

”I. ªÊÓ” (root ”I. ªË” ”l¨b” ”play”) means ”play-

ground”, and matbakh ”qJ.¢Ó”’ (root ”qJ.£””tbkh” ”cook”) means ”kitchen”.

2.2 Stemming Techniques

Stemmers are commonly designed for each spe-cific language. Stemmers design requires somelinguistic expertise in the language itself. ForEnglish language, stemming is well understood,with techniques such as those of Lovin [5] andPorter [6]. Many Stemmers have been imple-mented for many languages. Due to its non-concatenative nature, Arabic is very difficult tostem[4], so regrettably there are only few stem-mers in Arabic.

In literature, stemmers can be classified intotable lookup, linguistic, and combinational ap-proaches.

Table lookup approach [3] utilize huge listthat stores all valid Arabic words along theirmorphological decompositions. This methoddoes not use stemming process. For a givenArabic word, it access the list and retrieve theassociated root/stem. Consequently, the stemsobtained are guaranteed to be highly accurate.However, the availability of such a table thatshould include all the language words is prac-tically impossible.

Linguistic approach[1], attempts to simulatethe behavior of a linguist by considering Arabicmorphological system and thoroughly analyzingArabic words according to their morphologicalcomponents. In such approach, prefix and suffixof a given word are removed by comparing lead-ing and trailing letters with a given list of affixes.In literature, most of the published works weremainly linguistic-based.

Finally, in Combinational approach[1], agiven word is used to generate all combina-tions of letters. These combinations are com-pared against predefined lists of Arabic roots. If

2


matched, stem and patterns are extracted.

2.3 Backpropagation Neural Network(BPNN)

Backpropagation Neural Network (BPNN) is oneof the most common neural network architec-tures, which has been used in a wide range ofmachine learning applications, such as characterrecognition[2].

Typically, BPNN architecture consists ofthree layers[2]: input, hidden, and output lay-ers. Figure 1 shows BPNN architecture in whichthe input layer receives input data from an exter-nal source, the output layer transmit the resultof the neural network processing, and the hid-den layer serves to provide the internal relationsbetween input and output layers.

Figure 1: Backpropagation Neural Network

3 Methodology

The purpose of this study is to show numericalrelations between characters can be extracted forthe purpose of fast accurate stemming. This isinspired by linguistic studies that identify thelikely redundant characters in words. A neu-ral network approach for extracting Arabic rootswithout referring to root or pattern files is pro-posed here and proven to work.

Backpropagation neural network architec-ture is used. The model was trained to accept anencoded Arabic word as an input and to generatethe correct root of that word as an output.

3.1 Data

Most Arabic words are derived from verb formswhich extend or modify the meaning of the rootform of the verb, giving many shades of mean-ing. The simple or root form of the verb iscalled the ”stripped” verb, while the derivedform is said to be the ”increased” verb. Sev-eral hundreds of Arabic words can be derivedfrom a single root by adding affixes. Arabic lin-guists have identified the commonly used lettersin these affixes and presented them in the fol-lowing set: {�,

@,È, �H,Ð,ð, à,ø

,�ë, @}. The Arabic

linguistics experts classified these letters accord-ing to their frequency in appearing as affix letters[7]. The highest frequently used are { @,ð,ø

}, then

{ @,Ð, à, �H}, and the lowest are {�,�ë,È}. Depend-

ing on this, we classify all Arabic letters into fourclasses and assign a numeric value to each classas shown in table 2. These values are used in thenetwork as inputs.

Class Decimal Code Binary code@,ð,ø

4 100

@,Ð, à, �H 3 011�,�ë,È 2 010

¬ðQmÌ'@ ú

�¯AK. 1 001

Table 2: Arabic Letter Classification

Any letter in the input word is encoded asthree binary digits, as shown in table 2. Limit-ing an input word to length of four letters, givesus an input matrix of 4×3, where each row rep-resented contains three binary digits code for theletter.

The output matrix will also 4×3. If the inputletter in any given word is in the root word, wepresented it by three ones ”111”, other wise itpresented by three zeros ”000”.

3


3.2 Network Architecture

The architecture of the model is summarized inFigure 2. The network has an input layer of12 neurons (4 letters and each letter representedby 3 bits), followed by one hidden layer consistsof 12 tansig neurons. The hidden layer is fullyconnected to the input layer, output layer. Thenumber of hidden neurons was estimated experi-mently. The output layer has 12 purelin neurons.

Figure 2: The Proposed Model Architecture

The network was trained by trainlm back-propagation learning algorithm with learningrate η = 0.5. Training was performed until anacceptable convergence was found for 515 epochswith mean squared error less than 0.01.

The procedures and functions of the softwarepackage ”MATLAB” were used to design, train,and test the proposed neural network.

4 Experiments and Results

The data for this study were taken from a gener-ated lexicon. A shell program was implementedby the first author. The program generated a setof words from a given set of roots. The lexiconcontains about 9000 Arabic words and its cor-rect word roots. A set of 1000 randomly chosenwords was selected from this lexicon, restrictedto words of length four letters. The correct out-

put Arabic root for each word was available tothe model during training.

The trained model has been tested using aset of 250 randomly chosen words from the gen-erated lexicon. The word length was four letters.Comparing the output from the trained networkwith the actual roots, it shows that the networkproduces 209 correct roots from the given testset. The proposed technique has proved a pos-itive behavior with accuracy rate 84%. Table 3

Word Input Matrix Root Output MatrixI.

�KA¿ 001100011001 I.

�J» 111000111111

I. ªÊÓ 011010001001 I. ªË 000111111111

I. «B 010100001001 I. ªË 111000111111

YJª� 010001100001 Yª� 111111000111�éJ.ªË 010001001100 I. ªË 111111111000

�PYK 100001001010 �PX 000111111111

Table 3: A Sample of Tests Results

shows a sample of tests results. While, figure 3shows how the network extracts the Arabic root”Yª�” from the input Arabic word ”YJª�”, as anexample.

Figure 3: Extraction of Root ”Yª�” from the word

”YJª�”

5 Conclusion

In this paper, we presented a backpropagationneural network approach to the problem of stem-

4


ming the Arabic words.A neural network can be used to solve a vari-

ety of problems. However, the successful imple-mentation depends on the nature of the problem.Stemming Arabic words is not a straight forwardkind of problem, which appears at first time tobe not quite suitable for neural networks. How-ever, the positive results that we have obtainedthrough our experiments, show that a neural net-work can be used for stemming Arabic words.

Currently, the process is limited to triliteralroots and four letters Arabic words length. Fu-ture work will concentrate on extending our ap-proach to cover all length of Arabic words androots.

References

[1] Ibrahim A. Al-Kharashi and Imad A. Al-Sughaiyer. Rule merging in a rule-based arabicstemmer. In COLING, 2002.

[2] Laurene V. Fausett. Fundamentals of Neural Net-works. Prentice Hall, 1994.

[3] Jerey Coombs Kazem Taghva, Rania Elkhoury.Arabic stemming without a root dictionary. In-ternational Conference on Information Technol-ogy: Coding and Computing (ITCC’05), 1:152–157, 2005.

[4] Larkey, Leah S., Lisa Ballesteros, Connell, andMargaret E. Improving stemming for arabicinformation retrieval: light stemming and co-occurrence analysis. In Proceedings of the 25thAnnual International ACM SIGIR Conference onResearch and Development in Information Re-trieval, Arabic Information Retrieval, pages 275–282, 2002.

[5] J. B. Lovins. Development of a stemming al-gorithm. Mechanical Translation and Computa-tional Linguistics, 11(1-2):22–31, 1968.

[6] Martin F. Porter. An algorithm for suffix strip-ping. Program, 14(3):130–137, 1980.

[7] Fagher Aldeen Qabaweh. Tasreef Al Asma’ wa AlAfa’l. Al Maerf Library, 1988.

5


an application of neural network for extracting arabic word roots

Documents