■ We have developed a sales-support system fordoor-to-door sales of cosmetics based on a systemcalled Skin-Expert, a skin-image grading servicethat includes analysis and diagnosis. Skin-Expertanalyzes a customer’s current skin quality from apicture of the skin. Several parameters are extract-ed by image processing, and the skin grading isdone by rules generated by data mining from abaseline of grades given by human skin-care ex-perts. Communication with the Skin-Expert isthrough a cellular telephone with a camera, usinge-mail software and a Web browser. Salespeoplephotograph the customer’s skin using the camerain a standard cellular telephone and then send ane-mail message that includes the picture as an at-tachment to our analysis system. Other parametersassociated with the customer (for example, ageand gender) are included in the body of the mes-sage. The picture is analyzed by our skin-gradingsystem, and the results are made available as a pagein HTML format on a customer-accessible Website. An e-mail is sent when the results are avail-able, usually within minutes. Salespeople checkthe results by using a Web browser on their cellulartelephones. The output not only provides a grad-ing result but also gives recommendations for thecare and cosmetics that are most suitable for thecustomer. Our system integrates cellular commu-nication, Web technology, computer analysis, datamining, and an expert system. Though salespeopleuse only a cellular telephone with very little com-puting power as the front end, they can take ad-vantage of intelligent services such as computergrading and data mining. The salespeople do notneed to think about what is running in the back-ground, and there is no requirement that end usershave any special hardware.

Door-to-door sales is one of the mostpopular sales strategies in Japan. Sales-people visit a customer’s home, pro-

mote new products, and help the customer se-lect suitable products through face-to-facecommunication. We can regard this strategy asone of customer-relationship management (CRM)(Goldenberg 2002). CRM is not the product-oriented concept of “good products can besold well” but the customer-oriented conceptof “only products that a customer desires canbe sold.” The most important point of CRM isto maintain good customers, and the periodiccommunication door-to-door salespeople havewith customers is an effective way for a compa-ny to do this. However, door-to-door sales vol-ume is decreasing because of the many onlineshops from which one can get detailed productinformation without talking with salespeople.

The target users for our system are associatedwith cosmetics companies. Basically, cosmeticscompanies have failed to keep up with infor-mation technology (IT), so we designed a door-to-door sales-support system that helps sales-people by employing IT and AI technologies.This sales-support system is a CRM tool fordoor-to-door sales of cosmetics based on Skin-Expert, a skin-image grading system that pro-vides analysis, diagnosis, and recommenda-tions. Skin-Expert analyzes the current grade(quality) of a customer’s skin from a picture ofthe skin. Several parameters are extracted byimage processing, and the skin grading is doneby rules generated through data mining from a

Articles

FALL 2004 17

A Cellular Telephone-Based Application for

Skin-Grading to Support Cosmetic Sales

Hironori Hiraishi and Fumio Mizoguchi

Copyright © 2004, American Association for Artificial Intelligence. All rights reserved. 0738-4602-2004 / $2.00

AI Magazine Volume 25 Number 3 (2004) (© AAAI)

baseline of grades given by humanskin-care experts.

Communication with the Skin-Ex-pert is through a cellular telephonewith a camera, using e-mail softwareand a Web browser. This type of a cel-lular telephone is quite popular in Ja-pan. Thus our system does not requirethat end users have any special hard-ware. Salespeople take a picture of thecustomer’s skin using the camera inthe telephone and send the picture bye-mail to our analysis system (figure1). The skin-grading system analyzesthe picture. The results are made avail-able as a page in HTML format on acustomer-accessible Web site. An e-mail is sent when the results are avail-able, usually within minutes. Sales-people check the results by using aWeb browser on their cellular tele-phones. The output provides not only

Articles

18 AI MAGAZINE

Figure 1. Skin-Expert.A. Cellular telephone equipped with a camera. B. Taking a skin image with a cellular tele-phone. C. Skin image. D. Sending the skin image by e-mail. E. Accessing the URL in the re-turn message. F. Checking skin grade. G. Recommendations.

A B

C D

F G

E

Close-upmode 15 cm.

a grading result but also recommendations forthe care and cosmetics that are most suitablefor the customer.

Our system integrates Web technology, com-puter analysis, data mining, and an expert sys-tem. Although salespeople use only a cellulartelephone with very little computing power asthe front end, they can take advantage of intel-ligent services such as computer grading and da-ta mining. The salespeople do not need to thinkabout what is running in the background.

This service is provided through Wisdom-Tex, Inc. (www.wisdomtex.com), a venturecompany from Tokyo University of Sciencethat provides data-mining services. At the timeof this writing, Skin-Expert has been deployedwithin seven cosmetics companies. In our busi-ness model, we intend to charge a minimumrate to make the service affordable for part-time salespeople; we charge only ten dollars amonth. This business model makes sense be-cause the largest cosmetics company has morethan one hundred thousand door-to-doorsalespeople, and overall there are about onemillion such salespeople in Japan.

We have organized this article as follows: thenext section provides an outline of our Skin-Expert; the third section explains the skin-im-age processing of our Skin-Expert; the fourthsection describes the data mining used to cre-ate the skin grading; the fifth section intro-duces the expert system that constructs adviceabout cosmetics suitable for a customer; thesixth section shows the performance of ourSkin-Expert; and the final section contains ourconclusions.

Skin-ExpertSkin grading from skin images is common inmost cosmetics companies in Japan. They haveused a special camera to take close-up imagesof the skin. These images are then judged byhuman experts who determine the skin gradefrom the image and who then provide individ-ualized product recommendations.

Our Skin-Expert automatically diagnosesskin images using rules extracted by data min-ing. Moreover, we can use a cellular telephonewith a camera to obtain and transmit the im-ages (figure 1). About 50 percent of Japanesepeople now have a cellular telephone, andmost cellular-telephone manufacturers nowproduce a cellular telephone with a camera, sosending pictures and movies by e-mail has re-cently become common in Japan.

Cellular-telephone technology has been pro-gressing rapidly. Some cellular telephones nowhave close-up lenses (with up to seven times

magnification), zooming functions (up to tentimes), and autofocus. For example, the CasioA5403CA has a 2-megapixel charge-coupleddevice (CCD), autofocus, and ten times zoom.When we first started deploying our system, wehad to provide a loupe (magnifying glass) overthe standard cellular-telephone camera lens fortaking close-up images of the skin (Hiraishiand Mizoguchi 2003). Now, however, we cantake close-up images for analysis by Skin-Ex-pert using only the standard camera on currentcellular telephones.

Using the close-up mode of the cellular tele-phone camera (figure 1a), we take a skin imagefrom about 15 centimeters, as depicted in fig-ures 1b and 1c. We can take the skin image eas-ily by fixing the camera on the customer’sshoulder. (It was a little hard to take a clear skinimage by using a loupe because the focallength was very short. Now it is easier to takethe skin image.) The skin image is sent to oursystem by using e-mail software on the cellulartelephone (figure 1d). An e-mail is sent backwhen the results are available, usually withinminutes. The return message contains the URLof the results page (figure 1e). When we clickthe URL, the browser on the cellular telephonestarts, and we can check the grading result andrecommendations as illustrated in figures 1fand 1g.

The most significant feature of our system isthat it does not require that end users havecomputers and special camera devices to per-form automatic skin grading. Almost all sales-people in cosmetics companies work part time,and almost all salespeople are very inexperi-enced with computers. For these reasons, ourapproach in designing the Skin-Expert sys-tem—not forcing salespeople to buy any spe-cial hardware but using the cellular telephonethey ordinarily use—is well suited for this ap-plication.

Figure 2 shows homepages of our Skin-Ex-pert as the user interface on the cellular tele-phone. The upper-left page is the member’spage: there are links for checking the results,referring to past results, requesting skin grad-ing, inputting user information (user name, e-mail address, and so on), and modifying thesystem configuration (user password, time-out,and so on).

The upper-right page is a member’s resultspage. Seven factors represent the current statusof the skin: the number of pores, the depth ofthe pores (this is reflected in the degree of dark-ness of skin), the thickness of the wrinkles (wecan regard this as the degree of tension of theskin), the depth of the wrinkles (we can regardthis as the degree of dryness of the skin), the

Articles

FALL 2004 19

with the cellular telephone to an e-mail messageand add information such as the customer’sname, age, and normal sleeping hours in themail body. This is processed by our server.

We are using a GRID computer (Foster andKesselman 2003) to cope with many requests.In the current configuration, it takes about 1second of processor time to analyze one skinimage. If one hundred thousand requests ar-rived at the same time (this is a little hard toconsider, even though one company has morethan one hundred thousand salespeople) andthey were processed sequentially, it would takemore than twenty-seven hours to finish all therequests. So, we need a lot of accessible com-puting power for door-to-door sales support inorder not to make customers wait.

The server software running on the masterhost of the GRID computer checks the mailboxperiodically. When a mail request arrives in themailbox, the server software reads the attachedimage and information in the mail body. Theskin-grading software is the main part of ourSkin-Expert system. The software analyzes skinimages by image processing and grades theskin using the rules generated by data mining;it also generates some recommendations. Wepresent the details of these functions in the fol-lowing sections.

The skin-grading task is distributed by theGrid Engine (Sun Microsystems 2002) and isexecuted by the skin-grading software runningon an execution host. After finishing the skingrading, the skin-grading software puts the re-sults on the Web server as an HTML file, andthe skin-grading software sends an e-mail mes-sage to the cellular telephone to report thecompletion of the skin grading. The URL to theresults page is included in the e-mail message,so customers can check the results and the rec-ommended cosmetics using the cellular tele-phone’s Web browser.

Skin-Image ProcessingIn our Skin-Expert, the original color image(240 x 320 pixels) is converted to a gray image,an edge image, a noise-reduced image, and aline image like the ones shown in figure 4. Sev-eral parameters are extracted during the imageprocessing.

The extracted parameters are (1) the numberof intersections; (2) the image depth at the in-tersections; (3) the line thickness; (4) the imagedepth of lines; and (4) the strength of line di-rection.

The number of the intersections is the num-ber of the intersections between each line onthe line image. We can interpret intersections

clearness of a wrinkle, the clearness of the skintexture, and the cumulative skin grade basedon these parameters. The evaluation is alsomoderated by parameters such as the person’sgender and age.

The lower-left page of figure 2 is the skin-grade page. The skin grade is represented as anumber of stars. Five stars represent the bestskin status and one star the worst. We can checkthe balance of seven factors from this page, rep-resented in a radar chart (figure 1f). We can seethe recommended cosmetics that are linkedfrom each results page on the lower-right page.

Architecture of Skin-ExpertFigure 3 shows the architecture of Skin-Expert.We attach a skin image (in JPEG format) taken

Articles

20 AI MAGAZINE

Figure 2. Homepages of Skin-Expert.

as pores. In the case of figure 4, the number ofintersections is 84.

The image depth at the intersections is theaverage color depth (0–255) of the intersec-tions in the line image. It can be interpreted asthe normalized pore depth. The coordinates ofthe intersections can also be obtained from theline image. The intersections are located in thegray image, and the color depth at that pointof the image is closely correlated with the poredepth. The parameter is 126 for figure 4.

The line thickness indicates the thickness ofwrinkles. The black part of the noise-reducedimage of figure 4 can be defined as the wrinklepart. The line thickness is calculated by divid-ing the sum of the black pixels in the noise-re-duced image by the sum of the black pixels inthe line image. In the case of figure 4, it is 4.04.

The image depth of lines correlates with thedepth of wrinkles. It is computed in the sameway as the image depth of the intersection. Itis represented as the color depth related to thegray image by using the coordinates of the linein the line image. It is 123 in figure 4.

Skin in which we can see the wrinkle direc-tion is not in good condition. In the case of fig-ure 4, we can see the wrinkle direction fromupper right to lower left. We define thestrength of line direction for skin grading.

First, the standard deviation of the pixel val-ue on the 36 lines (every 5 degrees) from thecenter is calculated on the gray image (figure5a). Here, the pixel value on the line is the av-erage of pixels on the perpendicular line (figure5b). Then the strength of the line direction isrepresented as the standard deviation of thestandard deviation of each line. If the directionof the wrinkle is clear, the standard deviationof the line across the wrinkles is big (figure 5c).In contrast, if the direction of the wrinkles isnot clear, the standard deviation of the linealong the wrinkles is small (figure 5d), so thewhole standard deviation becomes small. Thestrength of figure 4 is 11.8.

Data Mining for Skin-ExpertWe have about ten thousand sample skin im-ages that have been gathered through collabo-ration with cosmetics companies and that havebeen graded by human experts. The parame-ters previously described have been extractedfrom these baseline cases, and we used datamining (as we will describe) to extract rules forskin grading to try to come close to the humanperformance. There are three other parametersof skin grade: (1) clearness of wrinkle, (2) clear-ness of texture, and (3) total evaluation.

The clearness of wrinkle of the skin is evalu-

ated at one of three levels (three stars, twostars, and one star). Skin that has a clear textureis good skin. Clearness of texture is evaluatedat three levels (three stars, two stars, and onestar). Considered with the preceding twogrades, the total evaluation of the skin is deter-mined as being at one of five levels (five stars,four stars, three stars, two stars, and one star)by the experts, and also by our program.

We used inductive-logic programming (ILP)(Muggleton 1995) for doing our machinelearning. ILP is a machine-learning techniquebased on first-order logic. It can extract rela-tional rules among several attributes. Figure 6is an example rule for skin grading extracted byour constraint version of ILP (Mizoguchi et al.1995).

The grade in line 1 of figure 6 indicates thatthis rule is for the three-stars grade. This rulemeans that “If the sex is female (line 2), the ageis 40s (line 3), the number of the intersectionsNI is between 80 and 90 (line 4), the imagedepth of the intersections DI is between 120and 130 (line 5), the line thickness LT is be-

Articles

FALL 2004 21

Skin-gradingSoftware

Skin-gradingSoftware

ServerSoftware

Grid Computer

MailServer

CellularTelephone

WebServer

1: E-mail5: E-mail including URL

6: http

4: HTML file

Skin-gradingSoftware

3: Distributed by Grid Engine

2: pop3

Figure 3. The Skin-Expert Architecture.

tribute (favor, age, sex, grade) and the productattribute (maker, component, effect). This rulemeans that “The manufacturer of the cosmet-ics is maker_a (line 2); the ingredient is olive oil(line 3); the cosmetics are effective for dry skin(line 4) and good for the customer whose fa-vorite manufacturer is maker_a (line 6), whosesex is female (line 7), and whose age is 40s (line8); and the grade is three stars.”

This type of rule is generated by using the in-formation about the recommendations thatwere given by the experts and that were accept-ed by the cosmetics customers. The rules takeinto account customer and cosmetics informa-tion and the actual effectiveness of the rule(shown in terms of the good_for clause in line5). This is typical of rules used by the expertsystem to provide advice to customers.

Generating RecommendationsAfter skin grading, our Skin-Expert can giverecommendations about care and cosmeticssuitable for a customer. We use the expert sys-tem to make recommendations. The expertsystem can propose the most suitable productfor a customer through question-and-answerdialog. If there are some similar products, thesystem can show them in the order of the “cer-tainty factor” (Shortliffe 1976, Randall, p. 982),which one can regard as the strength of therecommendation.

Figure 8 shows homepages that advise aboutrecommended cosmetics. A customer answersquestions shown in the left page. The recom-mended cosmetics are shown on the rightpage. The cosmetic with the highest certaintyfactor is put on the top. Other cosmetics areput in the order of the certainty factor. The pa-rameters from image processing and skin grad-ing and the user profile’s age and sex informa-tion, which the Skin-Expert system alreadyhas, are input automatically without askingquestions.

The following processes are repeated in theexpert system:

Step 1: QuestionQuestions about the customer are asked. Theserelate to age, sex, sleep time, lifestyle, cosmet-ics the customer usually uses, their effective-ness, and so on. A question is described in thefollowing form:

<question name=cosmeticsinitial=no type=single>

<prompt>Which cosmetics are you using?

</prompt><condition>

<fact qname=maker item=maker_a active=true />

tween 3.1 and 5.1 (line 6), the image depth ofthe lines LD is between 20 and 130 (line 7), andthe strength of the line direction LS is between9.1 and 12.5 (line 8), then the grade is three-stars skin.”

Table 1 shows the comparison between thegrade judged by human expert and the gradeevaluated by computer. Perfect accuracy isachieved 38 percent of the time, and a mistakeof just one grade happens 42 percent of thetime. A two-grade mistake occurs 13 percent ofthe time, a three-grade mistake occurs 7 per-cent of the time, and there were no four-grademistakes. Cosmetics companies have indicatedthat a one-grade mistake is acceptable. Thus weachieve sufficient accuracy 80 percent of thetime.

In addition to rules for the skin grading, therules related to cosmetics are also extracted byILP. The rule in figure six represents which cos-metics are suitable for a customer.

The rule in figure 7 contains the customer at-

Articles

22 AI MAGAZINE

Figure 4. Image Processing of Skin-Expert.A. Gray image. B. Edge image. C. Line image. D. Noise reduced image.

6 mm.

8 m

m.

A B

C D

Articles

FALL 2004 23

Strong wrinkle

A. Check the 36 directions B. Average of pixels on the perpendicular line

D. Standard deviation is small

C. Standard deviation is big

Figure 5. Strength of Line Direction.

1. grade(A, ‘3 stars’):-2. sex(A, ‘female’),3. age(A, ‘40’),4. int_num(A, NI), 80<NI<90,5. int_depth(A, DI), 120<DI<130,6. line_thick(A, LT), 3.1<LT<5.1,7. line_depth(A, LD), 20<LD<130,8. line_strength(A, LS), 9.1<LS<12.5.

Figure 6. Example Rule for Skin Grading.

1. cosmetics(A):-2. maker(A, ‘maker_a’),3. component(A, ‘olive oil’),4. effect(A, ‘dry’),5. good_for(A, B),6. favor(B, ‘maker_a’),7. sex(B, ‘female’),8. age(B, ‘40’),9. grade(B, ‘3 stars’).

Figure 7. Which Cosmetics Are Suitable for a Customer.

is other, a product has already been selectedwhen this question is made. The type parame-ter indicates the question type. If the type val-ue is single, the customer can select only oneitem. In contrast, if the type value is multiple,the customer can select several items.

The prompt tag represents a query for a cus-tomer. The condition tag indicates the condi-tion that causes this question to be made. Inthis case, the question is asked if the item’smaker_a is selected (active=true) in the ques-tion about the cosmetics that a customer usu-ally uses. Finally, the candidate items are de-scribed in the candidate tag. The item tagcontains the description of the item. In thiscase, it is the name of the cosmetics, which isdisplayed in figure 8 on the left.

Step 2: Apply the RuleAfter one question and one answer, all of therules that match the condition are applied. Therule is in the following form:

<rule name=rule><if>

<and><fact qname=grade item=3stars

active=true /><fact qname=age item=40

active=true /><fact qname=sex item=female

active=true /><fact qname=maker item=maker_a

active=true /></if><then>

<hypo name=premium value=70 /></then>

</rule>

This means that “if the grade of the cus-tomer’s skin is three stars, the customer’s age is40s, the customer is female, and the customerusually uses maker_a, then the certainty factorof the premium is incremented by 70 percent.”

Step 3: Apply the Hypothesis The hypothesis is necessary to provide similarcosmetics, and it is used to describe how to dis-play a recommendation in our system.

<hypothesis name=premium super=white><item name=html>

<a href=premium.html>5 stars premium</a><item name=img>

<img src=premium.jpg></item>

</hypothesis>

The exact meaning of this hypothesis is thatthe value white is the superior to the value pre-mium. Our system understands that the premi-um is similar to the white. So, if the certaintyfactor of the premium is changed, the certaintyfactor of the white is changed as well. Further-more, the hypothesis tag contains several itemsto display this recommendation. For example,

</condition><candidate>

<item name=premium>5 stars premium

</item><item name=aqua>

5 stars aqua</item><item name=white>

5 stars white</item><item name=other>

other</item>

</candidate></question>This represents a question about the cosmet-

ics that a customer usually uses. The initial pa-rameter in the question tag indicates the initialselection. The initial value of no indicates thatthere is no initial selection. If the initial value

Articles

24 AI MAGAZINE

Figure 8. Recommendations of Skin-Expert.

Human expert5 4 3 2 1

5 6 5 1 2 04 2 9 5 4 23 4 6 6 5 12 3 2 9 14 4

ComputerGrading

1 0 0 1 6 3

Table 1. Comparison between the ComputerGrading and Human Expert (%).

the item named html contains the HTML de-scription, and the item named image containsthe image of the cosmetics product. The rec-ommended cosmetics are thus displayed onthe page as in figure 8 on the right.

The expert system of our Skin-Expert repeatsthe processes of asking a question, applyingrules, and changing certainty factors. When allquestions have been answered, some recom-mendations are displayed in the order of cer-tainty factor as indicated by the description inthe hypothesis tag.

Here, the grade of the customer’s skin as de-termined through our Skin-Expert and the rela-tion between the customer’s grade and cosmet-ics, generated by ILP, are used in the rules ofexpert system. So, this expert system containsthe computer grading and data mining.

Performance of Skin-ExpertOur Skin-Expert consists of a mail server to re-ceive skin-grading requests, a Web server to ac-cess the results, and a Grid Engine to distributethe skin-grading task (see figure 3).

We tested the performance of our Skin-Ex-pert in our GRID computing environment (fig-ure 9) in the office of WisdomTex Inc., whichis a 1DK apartment. We constructed a 100megabyte per second optic fiber network (KD-DI) in the office. The Grid Engine (Sun Grid En-gine 5.3) runs on the Sun Fire V880 system(Sun UltraSPARCIII 750 MHz CPU x 4, 8 giga-bytes of memory, Solaris8), and there are six-teen Dell OptiPlex SX270 systems (Intel Pen-tium 4 3.2 GHz CPU, 2 gigabytes of memory,RedHat 8.0) to cope with the skin-grading task.

These machines are on the bookshelf as in fig-ure 9 on the right. One mail server (Sendmail8.12) is running on Sun Enterprise 450 (Ultra-SPARCII 450 MHz CPU, 1 gigabyte of memory,Solaris8), and the Web server (Sun One Appli-cation Server 7.0) is running on the Sun FireV880. All machines are connected through the100 megabyte per second local area network.

Table 2 shows the performance of each partof our Skin-Expert. We started with a situationin which ten thousand request messages hadalready arrived at the mail server. We recordedthe process time from reading the request mes-sage to sending the reply message to notify theuser of the finished skin grading. Our GRIDcomputer could process sixteen requests persecond. Ten to fifteen accesses were availableper second for the Web server. This is not justaccess to the homepage, but access to the JavaServerPages (JSP) program. The mail servercould receive only two request messages persecond. From these results, only two requestscould be processed per second, since the mailserver was the bottleneck in our environment.So we have set up a mail server on each ma-chine now. Our Skin-Expert can process sixteenrequests per second. Even if one hundred thou-sand requests arrive at the same time, all re-quests can be finished in about one and one-half hours.

Articles

FALL 2004 25

Table 2. Performance of Each Part of Skin-Expert.

Figure 9. GRID in Small Office/Home Office (SOHO).A. Sun-Fire V880 (left) and Sun Enterprise 450 (right). B. Bookshelf Grid (Dell Optiplex SX270 x 16).

GRID Computer 16 requests per second

Web Server 10–15 accesses per second

Mail Server 2 messages per second

A B

tion for Skin-Grading to Support CosmeticSales. In Proceedings of the Fifteenth Innova-tive Applications of Artificial Intelligence,19–24. Menlo Park, Calif.: AAAI Press.

Mizoguchi, Fumio; and Ohwada, Hayato1995. Constrained Relative Least GeneralGeneralization for Inducing ConstraintLogic Programs. New Generation Computing13 (3–4): 335–368.

Mizoguchi, Fumio; Ohwada, Hayato;Daidoji, Makiko; and Shirato, Shiroteru1997. Learning Rules that Classify OcularFundus Images for Glaucoma Diagnosis. InProceedings of the Sixth International Work-shop on Inductive Logic Programming, LectureNotes in Artificial Intelligence, No. 1314,146–159. Berlin: Springer-Verlag.

Muggleton, Stephen 1995. Inverse Entail-ment and Progol. New Generation Comput-ing 13(3–4): 245–286.

Shortliffe, E. H. 1976. Computer-Based MedicalConsultations: MYCIN. New York: Elsevier.

Sun Microsystems 2002. Sun Grid EngineEnterprise Edition 5.3, Administration andUser’s Guide. Mountain View, Calif: Sun Mi-crosystems

Hironori Hiraishi re-ceived his B.S degree in1993, his M.S degree in1996, and a Ph.D. in1999 in informationtechnology from TokyoUniversity of Science. Heis a research associate ofInformation Media Cen-

ter in Tokyo University of Science and alsochief technical officer of WisdomTex, Inc.His recent research interests have focusedon real-world applications that integrate AItechniques and GRID computing. His e-mail address is hiraishi@wisdomtex.com.

Fumio Mizoguchi is aprofessor in the Depart-ment of Industrial Ad-ministration and a direc-tor of Information MediaCenter at Science Univer-sity of Tokyo. He ob-tained his Ph.D. fromTokyo University in 1978.

He is also a senior research associate at theCenter for the Study on Language and Infor-mation and a member of the editorialboards of Artificial Intelligence Journal, NewGeneration Computing Journal, and the Jour-nal of Logic Programming. He has publishedmore than 150 papers and thirty books incomputer science and applied AI. He start-ed up his venture company in 2001, and heis the chief executive officer of WisdomTex,Inc. His e-mail address is mizo@wisdom-tex.com.

will have to expand it so as not to de-crease the service level as the request-ed services increase. We have set upseveral types of GRID computing envi-ronment. We can use open grid servicearchitecture (OGSA) (Foster andKesselman 2003) to integrate these en-vironments for more computationalpower and back up.

We are in contact with more thansixty-eight cosmetics companies. Ofthese, 40 percent are considering adop-tion of our Skin-Expert. Avon ProductsCompany, Ltd., has adopted our ser-vice, and about five hundred of thecompany’s salespeople began using ourservice in March 2004. We customizedand coordinated our service for them.We are constructing a database of com-ponents for cosmetics products to pro-vide knowledge for custom-producingcosmetics for a customer, and we havedeveloped a skin-aging simulation. Weare also gathering skin-cancer data anddiagnoses to extract an expert skin-can-cer-diagnosis system. The analysis ofthe data will enable us to judge the skincancer from just the spot of the skin inskin images taken by the camera of acellular telephone.

AcknowledgmentsWe would like to thank WolfgangGentzsch of Sun Microsystems, Inc.His presentation at ApGrid 2001 heldin Tokyo helped us construct ourGRID computer in a short time.Tsuyoshi Kaburaki of Avon ProductsCompany, Ltd., gave us many com-ments from the end-user viewpoint.His comments helped us to make oursystem very much more useful. Final-ly, we have to thank Daniel G. Bobrowof the Palo Alto Research Center(PARC). He has given us many pre-cious comments about our researchprojects, and they have helped us verymuch in achieving our success.

ReferencesDavis, Randall, and Lenat, Douglas B. 1982.Knowledge-Based Systems in Artificial Intelli-gence. New York: McGraw-Hill.

Foster, Ian; and Kesselman, Carl 2003. TheGRID 2. San Francisco: Morgan KaufmannPublishers.

Goldenberg, Barton J. 2002. CRM Automa-tion. Upper Saddle River, N.J.: Prentice Hall.

Hiraishi, Hironori; and Mizoguchi, Fumio2003. A Cellular Telephone–Based Applica-

ConclusionsIn this article, we introduced our Skin-Expert, which is an integration of Webtechnology, computer grading, datamining, and an expert system. OurSkin-Expert analyzes the customer’scurrent skin grade from a picture ofthe skin. Several parameters are ex-tracted by the image processing, andthe skin grading is done by rules gen-erated by data mining. Our systemprovides not only the skin grade butalso gives advice about cosmetics andskin care to customers. Our systemshows the expert knowledge of theskin graders and the experience ofhigh-achieving salespeople. It allowsbeginners to promote cosmetics as ex-perts and to emulate these highachievers. It yields high-level door-to-door sales for cosmetics companies.

In our system, salespeople use a cel-lular telephone just as the front end;they need not be concerned aboutcomputers running in the back-ground. The intelligence is hidden inour system, providing an example ofnext-generation Web service. Theloupe was necessary for our previousservice (Hiraishi and Mizoguchi 2003),but the progress of cellular telephoneshas made it much easier to use our ser-vice. Our service thus parallels theprogress of small devices such as cellu-lar telephones and the progress ofcomputing power like the GRID com-puter. In particular, we set up ourGRID computer on the bookshelf inour SOHO. The GRID computer is nolonger special but can be used foreveryday things.

This model can also be applied toother areas such as health care andmedical expert systems. We can regardthe cellular telephone of our model asa sensor for monitoring the status ofthe human body. Small devices moni-tor the current status of our body, anda supercomputer gathers data andevaluates our health status. We can al-so integrate some genome and druginformation for evaluating health. Wedeveloped an expert system for evalu-ating glaucoma more than twentyyears ago (Mizoguchi et al. 1997). Weare now expanding the system to in-corporate our model.

Our current GRID computer isenough for our service. However, we

Articles

26 AI MAGAZINE