Keki B. Irani, University of Michigan Jie Cheng, Ford Motor Company Usama M. Fayyad, Jet Propulsion Lab, California Institute of Technology Zhaogang @an, Elettronic Data Services

T H E PROMISE OF AUTOMATED manufacturing has not yet been fulfilled. partly because the intelligent systems need- ed to perform complex tasks rely on knowl- edge acquired from human experts, who often find it difficult to explain their ac- tions in concise, correct, situation-action rules. This knowledge acquisition bottle- neck is a well-known problem in building expert systems.' In some cases, even the experts who perform the tasks do not un- derstand them well: They make decisions based on their familiarity with the data and an intuitive knowledge ofthe domain. How do you build an expert system in such a situation ?

There is often a lot of experiential data available in manufacturing domains, but no models explaining how controlling vari- ables affect output variables. However, the machine learning approach takes advan- tage of this data and avoids the knowledge acquisition bottleneck by extracting clas- sification rules directly from data. Rather than asking an expert for domain knowl- edge, a machine learning algorithm ob- serves expert tasks and induces rules emu- lating expert decisions. The examples are typically easier to get, and many experts maintain a data log of their actions that can

FEBRUARY 1993

ENGINEERS CANNOT ALWAYS EXPLAIN IOW THEY CONTROL COMPLEX MANUFACTURING PROCESSES, SO THE GID?

i~CHIiVE-LEARNING ALGORITHM DERIVES KNOWLEDGE DIRECTLY FROM DATA, AN IMPORTANT STEP IN FULFZLLhVG

THE PROMISE OF AUTOMATED MANCTFACTURING.

be drawn upon. Machine learning tech- niques can also help experts look up cases in a large database or diagnose faults in a complex system by determining the condi- tions relevant to the search.

Our Generalized ID3 (GID3) algorithm takes a training set of experimental data and produces a decision tree that predicts the outcome of future experiments under various, more general conditions. The tree can then be translated into a set of rules for an expert system. We have also developed two systems that work with GID3 to deal with incomplete, inaccurate, or noisy data. These systems have been used succesfully in various semiconductor manufacturing applications in both diagnosis and process modeling.

The machine learning approach

A training example consists of a situa- tion and an expert 's action in that situation. The situation is described by a set of at- tributes, which are either continuous (nu- merical) or discrete (nominal). For exam- ple, Shape is a nominal attribute, with the possible values [Square, Triangle, Circle] ; pressure and temperature are numerical attributes. The expert's action is one of a fixed set of allowed actions (typically de- termined during normal task execution), andit determines theexample'sclass: Raise- Temperature, Decrease-Pressure, Accept- Batch, and so on.

In other words. a training example is a

41

_ _ _ _ _ _ _ _ _ _ _ ~ ~ ~ ~ -

Table 1. A simple training set.

to derive rules, expressed in terms of the

EXAMPLE

rules covering a training set is NP-hard.

A ~ T E CLASS SELECTIVITY A LINE WIDTH

Normal Normal

High High Low Low

Normal Power-Is-High High Power-Is-Low High Power-Is-Low Low Power-Is-High

Normal Flow-Rate-Is-Low High Flow-Rate-Is-Low

~~

applied recursively to each child node until the examples at that node are either of one class or have the same values for all at- tributes. Figure l a shows an ID3 decision tree for the training set in Table I . Each leaf is a classification rule. The path from the root to the leaf determines the rule's conditions, and the class at the leaf repre- sents the rule's action.

An algorithm's criterion for choosing an attribute at each node determines whether the algorithm generates a good or bad tree. ID3 chooses an attribute by applying an informa- tion entropy measure to the examples at a n ~ d e . ~ - j The measure favors attributes that partition the data into subsets with low class entropy (indicating that most of the examples in the subset belong to the same class). The algorithm basically chooses the attribute that provides the locally maximum degree of dis- crimination between

The information entropy criterion typi- cally leads to good attribute choices, but the fact that ID3 branches on every value of the attribute can cause problems. Consider an attributeA with possible values ( a i , U ? ,

..., a r } , where a , and a2 are the only ones relevant to the classification task. ID3 would still partition the data along r branches, even though only the first two have predic- tive value. The extra branches are harmful in three ways:

(1 ) They result in overspecialized rules. The leaf nodes that descend from the nodescreated by the extraneous branches will be conditioned on irrelevant at- tribute values. Since each leaf node corresponds to a classification rule, the irrelevant conditions will appear in the preconditions ofthe corresponding rules.

- - _~ ~ ~ -~ _- __ - - ~ _

,771 L[ HI:-.,, (High, lUaW

I ine widtn I ine wioih Low flowrate Low flowrate

I Normal Hiah Low Hiah I High Normal Low

Low d [ f i power High power High q power

I

Figure 1. Decision trees generated for the data in Table 1 by ID3 algorithm (a) and 6103 (b).

41 ~

(2) They unnecessarily partition the data, thus reducing the number of examples at each child node. The subsequent attribute choices made at these child nodes will be based on an unjustifiably reduced subset of data. The quality of such choices is thus unnecessarily reduced.

(3) They increase the likelihood of the “missing branches” problem, which occurs when the examples don’t con-

Tests

1 : ............ k...-.- ....... 1 1 1 I 1 1 1 I

I I Test and prune irrelevant conditions I I I Test and orune rules I I Test and Drune rules with I I I of low confidence I 1 IOW prediction accuracy I

-~

tain every possible combination of at- tribute values Final set of rules

To understand the third problem, con- sider two examples to be classified by the tree in Figure l a

(Selectivity = Low) & (A-Line-Width = LOW) (Selectivity = Normal) & (A-Line-Width = Low)

L h u r e 2- Doto flow diagram for

Both examples combine attribute values that did not appear in the training set. The tree readily classifies the first example, since the branching indicates that all exam- ples with low selectivity have a low flow rate. But the tree cannot classify the second example because there i s no branch for a low D line width under the normal-selec- tivity branch.

The main problem with the tree i s that the normal- and high-selectivity branches should not be separated: Low selectivity is the only value relevant to a problem’s oc- currence. If the learning algorithm didn’t branch on attribute values that are not indi- vidually relevant, the three problems listed above would not occur as often.

ID3’s overbranching problem could be handled by formulating Boolean combina- tions of attribute-pairs or by more sophis- ticated node tests. Breiman et al. suggest partitioning an attribute’s values into two subsets and letting each branch represent a subset of values.’ However, this is compu- tationally expensive: An attribute with Y

values has 2’possible binary partitions. An algorithm cannot explore each one. (The same authors show that, for the special case when only two classes exist, we need only consider a linear subset of the many partitions.)

GID3

To address some of ID3’s problems, we generalized the algorithm so that it does not necessarily branch on each value of the

chosen attribute. Our CID3 algorithm can branch on arbitrary individual values of an attribute and lump the rest into a single default branch. Unlike the other branches of the tree. which represent a single value, the default branch represents a subset of values of an attribute, reducing the unnec- essary subdivision ofdata. Figure I b shows a GID3 tree for the training set in Table 1 . We can classify both ofthe previous exam- ples with this tree: the missing-branchprob- lem does not occur.

A user-specified parameter, TL (“toler- ance level”). controls GID3’s tendency to branch on sonie or all of the values of a given attribute. With TL = 0. the algorithm will branch on every value, resulting in the same beha\ ior as for ID3. A setting of TL RIST. Noisy data can result from human = 1 results i n the most conservative beha\- error, imperfect sensor repeatability, or ior. with GID3 branching only on those defects in process equipment or sensors. values with exactly the minimum possible Empirical learning algorithms are typical- entropy; this setting often results in branch- ’ ly sensitive to noise because they rely sole- ing on only one value. There is always a TL ly on data to discoverrules; they usually do setting that allows CID3 to generate better not have access to special doinain knowl- trees than ID3, with fewer leaves and a edge to guide their decisions. Since noise lower error rate when classifying new ex- 1 in a training data set can result in irrelevant amples.’.’ (We recently developed GID3*, , rule conditions, we designed a software

~ which does not depend on TL. GID3” package called RIST (for Rule Induction produces better trees than ID3 and CID3 and Statistical Testing) that works with by avoiding the problem of irrelevant CID3 to combat noise. branches, dramatically reducing the tree’s Figure 2 illustrates how RIST works. size and improving its quality.’) After CID3 generates a decision tree, RIST

Depending on the domain, GID3 and applies Fisher’s “exact test.” which re- GID3* perform from two and five times moves rule preconditions deemed statisti- better than ID3.’.8 We measure improve- cally irrelevant. resulting in more general, ment by measuring a decision tree’s ex- “pruned”rules.”he test measures the prob- pected accuracy in classifying new exam- ability that a hypothesis (in this case. ples. by measuring the t ree’s s ize a rule’s condition) is irrelevant to the

syntactically (number ofleaves, number of nodes. average depth, and so on), or both. The number of leaves is the most critical measure; i t can be probabilistically related to the expected error rate.’,’ In addition, experts found the rules produced by CID3 more understandable.

Extensions to GID3

Noisy data and the limited availability of training data are significant problems in developing industrial applications. Two extensions to GID3 - RIST and KARSM - deal with these problems.

FEBRUARY 1993 43 ~

Rules for conditions leading to i suboptimalourme ~

Rules for adjusting parameters to achieve optimal operation Random

samples

Figure 3. Data flow diagram for KARSM.

Table 2. Partial list of data logs for a reactive ion etchinq process. ~ ~ ________________ I POWER1 POWER2 POWER3 TIME^ TIME2 TIME3 W A F E R TOPOLOGY ... O U T C O M E

Nn

1117 895 833 835 859 867 847 776 771 847

806 844 860 867

~

I 851

I 878 I 848

806 881 772

842

1134 1139 854 81 8 835 886 871 833 813 825 843 896 822 809 825 81 9 81 6 778 795 868

826

490 492 442 491 490 466 473 500 490 491 490 493 493 490 452 454 455 484 467 490

494

8.98 10.26

7.4 9.7 9.9 9.9 9.8 8.6 8.7

9.20 17.2 10.5 9.16 9.8 9.7

24.1 321 .O 22.5 22.7 8.7

17.0

7 20 8 2 6 0

7 76 7 92 7 9 7 8 6 9 7 0

7 36 4 3

8 40 7 33 7 84 7 76 14 4 16 8 9 0 13 6 7 0

13 6

5 0 5 0 5 0 5 0 5 0 5 0 5 0 5 0 5 0 5 0 5 0 5 0 5 0 5 0 5 0 5 0 5

5 0 5 0 5 0

5 0

18 V 24 V 1 V 5 V 2 V 7 V 8 V 8 V 4 V 13 V 6 V 2 V 14 V 2 V 2 V 1 T 9 T 8 T 1 T 7 V

1 V

...

...

...

...

...

...

...

...

...

...

Erosion Erosion Sleeves Normal Normal Sleeves Sleeves Normal Normal Erosion Normal Normal Erosion Normal Sleeves Normal Normal Sleeves Normal Normal

Norma I

THEN etching at stage 1 may not be sufficient; Increase etch time for stage 1 to > 8 15 minutes

Rule 4, IF sleeves are observed A N D stage 3 power E [451,487] T H E N Increase overetch percentage for stage 2

Figure 4. Rules for diagnosing faults of an RIE process.

44 I -

outcome. If this probability exceeds a small value, we discard the condition.

RIST runs CID3 many times, each time inducing a tree from a random portion of the data set, applying the statistical test, and keeping only the good rules. Finally, a set of statistical criteria are used to select the best rules from each tree, which are combined to form a single robust rule base.

KARSM. ID3, GID3, and GID3” all rely on large training samples to detect meaningful, reliable patterns or correla- tions. But obtaining training examples can sometimes be expensive, so that only a limited training set is available. The KARSM software package (see Figure 3) provides a way to derive a larger training set from a smaller one. (KARSM stands for Knowledge Acquisition from Response Surface Methodology.)

Response surface methodology (RSM) is a standard method in process optimiza- tion that uses a polynomial surface to ap- proximate the given data.I0 Each point on the surface corresponds to a specification of the input and output variables. Optimi- ration techniques are then used to find a point that optimizes the ouput variables under the given constraints. So, RSM tells us what set of input parameters we need to optimize outputs.

At this point, we can randomly choose a combination of input values, and the sur- face will give us the associated discretized output. Since we know the target condi- tions that the output variable must satisfy, we can quantize the output value as {Good, Bad) or at a finer level as {Very Low, Low, Good, High, Very High]. I n other words, we can extract an arbitrarily large training sample from an original small training set.

Of course. the usefulness of the large set depends on the correctness of the surface, and here a problem arises. RSM is a static method: It finds a fixed optimal point on the surface. If hidden variables, unconsid- ered in the design process, later influence the process, the surface can “drift” so that the output point is no longer optimal. Ob- viously. we need a set of rules telling us which input parameters should be changed, and i n which direction, if the output is not optimal under the current constraints. This requires a dynamic, rather than static, solu- tion to the optimization problem.

KARSM solves this problem by feeding the generated data set into RIST (or GID3)

IEEE EXPERT

Table 3. Experimental data far process optimization. "SCCM" is unit flow rate in cubic centimeters. "CD" is critical dimension, the actual width of an etched line.

diagnostic rules. We applied RIST to get a

to extract qualitative rules that describe the data's behavior. By generating rules from the surface we essentially extract a qualita- tive description of process behavior in terms of desirable and undesirable regions of the output space.

W e could have used the response sur- face alone to describe the process, but it 's not easily understood. Also, qualitative rules are more general: The response sur- face no longer applies when the optimal operating point shifts, but qualitative rules can capture this drift by intitially capturing the surface's useful features rather than its exact topology.

patterns. The rules derived by GID3 were

Applications in semiconductor manufacturing

We have used GID3 in several areas of semiconductor manufacturing, mostly in- volving reactive ion etching (RIE), a wafer- etching process that promises increased pre- cision and higher device density and that has been targeted for automation by the Semi- conductor Research Corporation, a consor- tium of US semiconductor manufacturers. To automate a process, you have to deter- mine parameter settings based on given out- put constraints, but the RIE process is not well understood and there are no satisfacto- ry methods for determining proper control settings. We applied GID3 to RIE process diagnosis and optimization, and to knowl- edge acquisition for an expert system.

Process diagnosis. Diagnosing faults is an important task for process engineers, but abstracting the diagnosis process into rules - especially rules that will transfer across processes and guide further reason- ing to find physical laws governing the observed phenomena - has proved diffi- cult. The goal of our first project was to derive diagnosis rules from a production log containing fault inspection results.

Each data log has about 60 entries, in- cluding machine type, device specifica- tion, material and resist thickness, plasma time, power, DC bias, chamber pressure, gas flow, temperature, valve position, and number of wafers. RIE is a three-stage process, and each stage has its own set of measurements. Three types of inspection results are common: Normal, PR Erosion, and Sleeves (see Table 2).

FEBRUARY 1993

EXPERIMENT PRESSURE FLOW 70 FLOW A CO CD OXIDE LOSS OXIDE No. (MTORR) (SCCM) (YO) (MM) UNIFORMITY ("A) UNIFORMITY

(MM) ("A)

1 300 150 25 0.05 0.05 368 216.0 2 500 150 25 0.25 0.05 316 43.5 3 300 300 25 0.18 0.40 407 162.0

Table 4. Classified random samples.

EXPERIMENT No. PRESSURE TOTAL FLOW CL^ FLOW (YO) CLASS

1 377 192 49 Abnormal, Low, Low, Low 2 342 297 36 High, Low, Low, High 3 491 241 34 Abnormal, High, Low, High

499 452 21 1 37 Medium, High, High, Low 500 303 185 42 . Very Low, High, Low, Low

... ... ... ... ...

Our second diagnosis project was to iden- tify relationships between RIE process prob- lems (such as a reduction in yield) and process parameters (including the flow rate of each gas component and the chamber pressure for different etching steps). W e derived the data set using regression anal- ysis, which statistically identifies the geo-

abies, the values can be discretized as either low or high (based on the constraints). For the output (optimization) variables, the values are discretized as different levels such as very high, high, medium, low, very low. W e then fed the data into GID3 to get a decision tree, where a leaf represents a class of outputs.

~~

45

A leaf can represent an optimal or a faulty class. If an engineer wants to mini- mize the line-width change when selectiv- ity is higher than a certain value and when uniformity is lower than certain values, then an optimal class would be (Very Low, High. Low, Low) for “very low line-width change, high selectivity, and low CD and oxide uniformity,” while a faulty class would be (High, Low, Low, High). To derive the required rules, a faulty leaf‘s condition - the path from root to leaf - i s compared to that of an optimal leaf. The differences represent the changes needed to transform a faulty operation into an optimal one. For example, if the condition of the optimal leaf is “pressure below 300 mTorr. total flow above 180 sccm, and percent flow above 40 percent,” and the condition of afaulty leaf i s “pressure above 312 mTorr. total flow from 215 to 300 sccm. and percent flow below 37.5 per- cent,” then we can derive a rule:

If selectivity is lower than normal. oxide uniformity is higher than normal. and line- width change is high. then to minimize line- width change, pressure should be decreased and percent f l ow should be increased.

Knowledge acquisition. We used CID.? to help acquire knowledge for an expert system for “emitter piloting,” in which an operator tunes integrated circuits printed in wafers to meet device specifications.”

In emitter piloting, the operator relies on experience to set an initial cycle time, and then adjusts it based on two device param- eters. If the parameter values fall within their desired ranges. the operator accepts the cycle time for batch tuning and calls i t

the “shooting time.” Otherwise, the opera- tor increases or decreases the cycle to bring the parameter values into their desirable ranges. The more experienced the opera- tor, the fewer steps this takes. However. this experience is very difficult to encode in rules.

We collected the raw data for all se- quences of trials fromexperiment data logs. each consisting of sequences of cycle-time adjustments targeting a shooting time. The logs recorded the cycle time and two pa- rameter value? in every trial in each se- quence. We used GID3 to learn rules for jumping to a shooting time from an arbi- trary cycle time by letting each data point be the condition under which certain ad- justments can be made to achieve a certain

shooting time. The difference between the current cycle time and the actual shooting time for each example was taken to be the predetermined class. An expert evaluated the rules induced by CID3 and found their performance satisfactory.

2. L. Breiman et al., Classifcarion and Re- ‘ Brooks, gression Trees, Monterey, Calif., 1984.

3. J.R. Qninlan. “InductionofDecisionTrees,” MLichineLetirninR,Vol. 1,No. I., 1986,pp. 8 1 - 1 06.

4 J Cheng et a1 , “Improved Decision Trees A Generalized Version of ID3,” Proc Fiffh W E USED GID3 IN SEVERAL

domains to derive knowledge from data. In each case. the engineer who supervised the experiments and provided the data said the rules were consistent with the data and conformed with their exuectations. A de-

Int’l Conf Machine Learning, Morgan Kaufmann, San Mateo, Calif, 1988, pp 100- 108

1

5 . U.M. Fayyad, On the Induction ofDecision riL ed model or discovered ,,attern that is

pectation i s of great value in two ways:

TI-eesfo;kul t iple Concept Learning, doc-

puter Science Dept., Univ. of Michigan, Ann Arbor, I99 1 .

mitl , a process engineer,s ex- toraldissertation,ElectricalEng. andCom- ~

I ( 1 ) I t provides a previously unavailable

mechanicalmeansforclassifyingevents or relating faults to parameters.

,

6, U,M, Fayyadand K.B. Iran,, . ‘ ~ h ~ A ~ ~ ~ i b ~ ~ ~ . Selection Problem in Decision-Tree Gen-

~

( 2 ) It gives process engineers insight into

i t that were previously implicit as engi-

eration.” Proc. 10th Nat’l Conf oil Artifi-

Cambridge, Mass., 1992. pp. 104-1 10. the process by making patterns explic. C ~ U I Intef[igence (AAA1 ‘921, MIT Press,

neering intuition about the process. ~ 7. U.M. FayyadandK.B. Irani, “What Should

be Minimized in a Decision Tree?,, Proe, ~ j ~ h r h Not’/ conf Arrificia] Intelljgerzce

However. the same properties that make the decision tree approach efficient also have disadvantages. The fact that the size of the data set at any node drops exponen- tially with the node’s depth drastically re- duces the reliability of the attribute selec- tion decisions at the lower nodes (apruning method. as i n R E T . can compensate for this problem). Another limitation derives from the restricted attribute-value pair lan- guage that does not allow for relational tests involving more than one attribute. The burden rests on the user to ensure that all interesting attributes are provided ahead of time.

Acknowledgments

The University of Michigan SRC Research Pro- gram supported this work under contract number 89-MC-085. We would also like to thank Hughes Microelectronics Center for supporting Usaina M. Fayyad ~ i t h an unrestricted research grant.

References

(AAA1 ‘90). MITPress, Cambridge, Mass., 1990, pp. 749.754.

8. U.M. Fayyad and K.B. Irani, “Machine Learning Algorithm (GID3*) for Automat- ed Knowledge Acquisition: Improvements and Extensions,” General Motors Research Report CS-634, GM Research Labs, War- ren, Mich., 1991.

9. J .R. Quinlan. “Generating Production Rules From Decision Trees,” Proc. 10th Int ‘ I Joint Corifererzce on Arfificial Infelligence (IJ- CAI ’87), Morgan Kaufmann. San Mateo. Calif., 1987, pp. 304-307

10. A.S. Bergendahl, S.F. Bergeron, and D.L. Harmon, “Optimization of Plasma Process- ing for Silicone-Gate FET Manufacturing Applications,” IBMJ. Research and Deivl- opnenr, Vol. 26, No. 5. 1982.

1 I . C.R. Friedhoff et al., “Analysis of Intralev- el Isolation Test StructureData by Multiple Regression to Facilitate Rule Identification for Diagnostic Expert Systems,” Proc. Int’l Conj: Microelectroriic Test Structure F , 1989.

1 . E.A. Fiegenbaum. ”Expert Systems in the 1980\.” in Srare (~ - f / i e -Ar . t Report on M a - , chinr Irirrlligence, A. Bond, ed.. Perga- inon-Infotech. Maidenhead.England. 1981.

12. Y.K. Yang. “EPAS: An Emitter Piloting Advisory Expert System for IC Emitter Disposition.” Proc. Seriiicon West , 1989.

- ~~ ~~~

IEEE EXPERT

Keki B. lrani is a pro- fessor of electrical engi- neering and computer science at the Universi- ty of Michigan in Ann Arbor. His research in- terests include comput- er architecture, distrib- uted computer systems, databases, and artificial

Zhaogang Qian works at Electronic Data Services' Artificial Intelligence Services, where he applies statis- tics and AI to vehicle exhaust emission measurements at GM Research Labs. His research interests include induc- tive and deductive learning, nonmonotonic reasoning, and qualitative reasoning. Qian received PhD and MSE degrees in computer, information, and control engineer- ing from the University of Michigan. He is a member of the IEEE Computer Society and AAAI.

intelligence, specifical- ly in planning, heuristic problem solving, and machine learning. Irani received his PhD in electrical engineering from the University of Micbiganin 1953. Heis afellow oftheIEEEand a member of ACM, AAAI, Sigma Xi, and Tau Beta Pi.

Readers can reach the authors in care of Usama M. Fayyad, AI Group M/S 525- 3660, Jet Propulsion Lab, California Institute of Technology, Pasadena, CA 91 109; e-mail fayyad@aig. jpl.nasa.gov

Usama M. Fayyad is technical group leader in artificial intelligence 1 at the Jet Propulsion Lab, California Institute of Technology in Pasade- ~

na, California. He leads ' projects on applications of machine learning to NASA and JPL prob- lems, especially appli-

cations that process large scientific and image databases. He also works in model-based rea- soning and automated diagnosis. Fayyad re- ceived a PhD in computer science and engineer- ing, an MSc in mathematics, MSE and BSE degrees in computer engineering, and a BSE in electrical engineering, all from the University of Michigan at Ann Arbor. He is a member of IEEE, AAAI, ACM, AAAS, Tau Beta Pi, Eta Kappa Nu, and Sigma Xi.

lie Cheng is a senior ' research engineer in the Engine Research De- partment at Ford Re- i \earth Laboratory in 1 Dearborn, Michigan He 1 works on intelligent in- tegrated software sys- tems for combustion engine design and sim- ulation. His research in-

terests are in knowledge-based design and sim- ulation, blackboard techniques for software integration and process scheduling, and knowl- edge acquisition. Cheng received a PhD in com- puter science from the University of Michigan, an MSE from the University of Michigan, and a BSE from the Shanghai University of Science and Technology.

FEBRUARY 1993

Canad!! Research Scientist

Artificial Intelligence $37,036 to $68,888

Canadian Space Agency Space Technology Branch

Ottawa (Shirley's Bay), Ontario (to be relocated to SteHubert, Quebec 0 fall 1993)

You will be responsible for initiating, leading and conductingresearch and development in Artificial Intelligence (AI) for application on future spacecraft missions, and for assisting in the planning and control of AI research and technology development projects with industry, universities and government agencies. You will also be involved in devising new and novel approaches to organizing, representing and distributing knowledge and the mechanisms that operate on that knowledge for applications such as system fault diagnosis, anomaly analysis and planning.

You must possess a doctoral degree from a recognized university in a field of natural sciences that is relevant to the duties of the position or a lesser degree with evidence of research experience and productivity equivalent to that of a doctoral degree. You must possess experience in independently carryingout scientific studies or research projects in the area of AI systems. Proficiency in English is essential.

Experience in spaceor astronautics disciplines gained from employment or specialized university courses would be a definite asset.

Preference in appointment will be given to Canadian citizens.

Please send your application and/or resume before March 5, 1993. quoting reference number S-92-316761-47JG-(G99), to:

Joan Girling Public Service Commission of Canada

66 Slater Street Ottawa, Ontario, Canada K l A OM7

We are committed to employment equity.

Public Service Commission Commission de la fonction publique B M ofCanada du Canada