a cost-sensitive machine learning method for the approach and recognize task

Robotics and Autonomous Systems 8 (1991) 31-45 31 North-Holland

A cost-sensitive machine learning method for the approach and recognize task

Ming Tan and Jeffrey C. Schlimmer School of Computer Science, Carnegie Mellon University, Pittsburgh, PA 15213, USA

Abstract

Tan, M. and Schlimmer, J.C., A cost-sensitive machine learning method for the approach and recognize task, Robotics and Autonomous Systems, 8 (1991) 31-45.

The long-term goal of this research is to build a robot which can survive in an unanticipated environment for a long time. Such a robot must learn which sensors to use, where to use them, and how to generate inexpensive and reliable effector procedures to accomplish its tasks. Current machine learning methods are inadequate because they ignore execution costs and are ill-prepared to handle prediction failures. This paper describes the CSL learning and control system that attempts to address this goal. CSL has been implemented in a real-world robot for the approach and recognize task.

Keywords: Cost-sensitive decision tree; Robot learning; Induction; Classification; Planning; Active perception; Grasping; Control.

1. Introduction

In the fu ture , r o b o t s a re m o r e a n d m o r e l ikely

to be o p e r a t i n g in e n v i r o n m e n t s tha t the i r des ign-

ers h a v e n o t ful ly an t i c ipa t ed . Th i s i n h e r e n t unce r -

t a in ty u n d e r m i n e s theore t i ca l o r m o d e l ba sed ap-

interests lie m the areas learning methods.

Ming Tan is a doctoral candidate in the School of Computer Science at Carnegie Mellon University. He received his B.S. in computer science from Shanghai Jiao Tong University and his two M.S. degrees in computer science from the University of Pitts- burgh and Carnegie Mellon University respectively. He is a member of AAAI and IEEE. His Ph.D. dissertation addresses the theory of cost-sensitive learning and its applications in robotics.

Jeffrey C. Schlimmer is a project sci- entist in the School of Computer Sci- ence at Carnegie Mellon University. He received the Ph.D. from the Uni- versity of California at lrvine in 1987. He was an associate program chair for AAAI-88, has served as a reviewer for a number of AI and machine learning conferences, and is the author of several papers in the area of inductive machine learning. Dr. Schlimmer is a member of AAAI and the Cognitive Science Society. His primary research of self-modeling databases and hybrid

p r o a c h e s to g o v e r n i n g behav io r . W e p r e f e r an

a p p r o a c h tha t a l lows the r o b o t to e m p i r i c a l l y a d a p t

to the p a r t i c u l a r wor ld it f inds i t se l f in. Th i s

a l lows the w o r l d to be its o w n s imu la t i on , a n d it

avo ids the c o m p l e x i t y o f f o r m a l m e t h o d s a n d thei r

d e p e n d e n c e on p a r a m e t e r s t ha t m u s t be p rec i se ly

eva lua ted .

C o n s i d e r a spec i f ic r o b o t task: a p p r o a c h a n d

recognize . A m o b i l e r o b o t is s u p p o s e d to i d e n t i f y

ob jec t s in its w o r k e n v i r o n m e n t as q u i c k l y as

possible . A t its d i sposa l a re sens ing p r o c e d u r e s , a

p a t h p l anne r , and a l ib ra ry o f c o d e d g r a s p i n g

p rocedures . A n a l y t i c a l l y de r iv ing an e f f i c ien t

m a p p i n g f r o m sens ing to g r a s p i n g p r o c e d u r e s is

b e y o n d cu r r en t t e chn iques for all b u t the s imples t

objects . I n d u c t i v e m a c h i n e l e a r n i n g m e t h o d s are

t e chno log ica l l y c loser to b e i n g ab le to c o n s t r u c t

the m a p p i n g , b u t they fall shor t in two i m p o r t a n t ways :

- Cos t : Sensing, mov ing , a n d g r a s p i n g ac t ions

have s ign i f i can t e x e c u t i o n costs . C u r r e n t l ea rn-

ing m e t h o d s use t h e m i n d i s c r i m i n a t e l y [1,2,5].

- N o i s e : Sensing , mov ing , a n d g r a s p i n g ac t ions

are i nhe ren t l y noisy. C u r r e n t l e a rn ing m e t h o d s

to le ra te no i se b u t do n o t c o m p e n s a t e for it [6,9].

0921-8830/91/$03.50 © 1991 - Elsevier Science Publishers B.V. All fights reserved

32 M. Tan, J.C. Sch[tmmer

"Fable 1 The approach and recognize task

Given : • Unknown objects. • Sensing and motion commands . Efficiently recognize each object 's grasping procedure.

In this paper we present an integrated learning and control system named CSL that addresses the approach and recognize task [10]. CSL's learning method is sensitive to costs; it attempts to trade off the quality and the expense of various sensing features. The learning method is a two-stage, partially incremental approach based on ID3 [5]. CSL's control method compensates for noise; it invokes a number of recovery strategies when the unexpected happens.

CSL has been implemented in a laboratory robot, and initial experiments demonstrate that the learned control procedures are robust and efficient. In the remainder of this chapter, we first discuss the approach and recognize task in more detail. Then, we describe the learning and control components of CSL, followed by a simple example of CSL in operation. Finally, we present some empirical results of the system's overall effective- ness.

2. Approach and recognize

In the approach and recognize task, a mobile agent must identify objects in its environment. At its disposal are a number of sensors. Table 1 lists the available inputs and the required outputs. This task differs slightly from a basic recognition task because the agent is mobile and can move to various locations to facilitate recognition.

This task exhibits some interesting interactions between the agent's ability to move and the de- pendence of sensors upon relative locations. For instance, if the agent uses a sonar sweep to determine the angular width of an object, then moving toward the object should increase the magni- tude and accuracy of the sonar reading, i One implication of this is that the agent may wish to move closer to interesting objects to improve the

1 This is not true for all sensors, but it is for many interesting and common ones.

accuracy (and ease) of recognizing them. Of course, the approach and recognize task may involve iterating between approaching an object and recognizing more about it.

A further qualification on the approach and recognize task is an implicit one: the agent must perform quickly. If efficiency were no concern, then the agent could simply approach each object and solve the recognition task from a close per- spective. If there are an unbounded number of objects, or if there is a dwindling resource (like limited time or decreasing energy), approach must be done sparingly and recognition must be done quickly.

We have been exploring an instance of the approach and recognize task in the context of a factory workshop robot. Consider the setting of a modern factory. Fabrication and assembly takes place in flexible manufacturing cells, each with an automated assembly device or robot. To facilitate tracking, transportation, and manipulation, parts are placed in special purpose trays. Because each cell's machinery knows where each part is in a tray, this also reduces the need for each cell to have perception abilities. This type of layout can be highly efficient, but it also has problems. One problem of interest here is that parts may fall out of trays, either during transit from one manufacturing cell to another, or because of accidents during manufacturing operations. Recovering fal- len parts from the factory floor is an instance of an approach and recognize task. Sensing procedures are used to determine the identity of various parts, and when recognized, the recovery agent knows how to grasp the part and where to return it.

Our specific testbed for these ideas has been a retrieval robot operating in a computer laboratory. This robot has the task of retrieving discarded cups and placing them in a receptacle. The robot agent is a Hea th /Zen i th H E R O 2000, a mobile platform with a five degree of freedom arm (torso, arm, elbow, pitch, and roll) and a two finger hand. It has a wrist-mounted sonar that has a range of 127 inches and an accuracy of +0.5 inches. A finger-mounted infrared sensor detects objects between the fingers. A ceiling-mounted camera locates objects in the laboratory.

As sensors for the retrieval robot, we have defined seven sonar sensing procedures by com- posing arm movements with intermittent sonar

A cost-sensitive machine learning method 33

D-SCAN H-SCAN H-SWEEP V-SWEEP

Fig. l. Basic sensing procedure types.

Table 2 Sensing procedures

Procedure Range Speed Error Features

H-Sweep-1 ° 6-127 in. 100 sec. + 3 ° H-Sweep-5 o 6-127 in. 45 sec. + 5 ° V-Sweep-1 ° 6-127 in. 76 sec. + 3 ° V-Sweep-5 o 6-127 in. 42 sec. + 5 o S-Sweep-1 ° 6-6 in. 70 sec. + 3 o H-Scan-0.3 6-6 in. 105 sec. + 0.5 in. D-Scan-0.3 6-6 in. 84 sec. + 0.5 in.

Width-Degree- l ° Width-Degree-5 ° Height-Degree-1 ° Height-Degree-5 o Depth-Degree-1 o Width, Front-Hole? Height, Depth, Top-Hole?

sensing. Each sensing procedure yields one or more sensing features corresponding to properties of its sensing field. Table 2 lists properties of the sensing procedures, and Fig. 1 depicts them di- agrammatically. Consider one sensing procedure in detail. H-Sweep-1 °, for example, can be applied up to 127 inches away from the object in question, it takes 100 seconds to execute, and it produces a measurable feature (Width-Degree-1 °) corresponding to the object's angular width accurate within + 3 o. Some sensing procedures produce more than one sensing feature (e.g., D-Scan-0.3) for a fixed cost. These sensing procedures can be especially attractive since they yield additional information without additional expense.

To approach an object, the robot utilizes a vision subsystem. It consists of a ceiling-mounted camera which is able to view the entire room through a fish-eye lens, a MATROX frame- grabber, and a generalized image library. By ex- amining the overhead visual image, the subsystem can locate regions that correspond to objects, the robot, and other obstacles. The image is thresholded, and regions are extracted by the Phoenix program [7]. Each region is represented as a polygon along with its pixels, location, and a

unique orientation. 2 Since the resolution of an overhead image is fairly coarse (approximately 0.8 inch per pixel), and the image is 2D, the subsystem can locate object candidates but not identify them. A path planner is used to plan a collision- free path from a current robot position to a target position relative to an object. As the robot follows a path, the visual subsystem keeps tracking of robot location and orientation to make sure that the robot is on the right course. At the end of a path, the robot switches to the wrist sonar to center itself more precisely for sensing or grasping operations. Depending on the amount of vision processing and the number of sonar sweeps in- volved, moving can cost from 0 to over 500 seconds and is estimated by a heuristic function.

For the sake of repeatable experimentation, we have settled on a standard set of objects, listed in Table 3, and a corresponding set of grasping procedures described in Table 4 and depicted in Fig. 2. Note that some objects have the grasping procedure No-Grab. This includes objects like the big

2 The orientation of a region is determined by its major axis. Since a major axis has two directionis, it is defined as the one associated with the half having a larger number of pixels.

34 M. Tan, J. ('. Schl immer

Table 3 Set of laboratory objects for approach and recognize task

Training objects Grasping procedure(s)

8 oz. Styrofoam coffee cup 12 oz. soda can (upright) 12 oz. soda can (lying down) 16 oz. short plastic tumbler (upright) 16 oz. short plastic tumbler (lying down) 32 oz. plastic tumbler (upright) 32 oz. plastic tumbler (lying down) tennis ball can (upright) tennis ball can (lying down) 9 × 4 x 5 in. open cardboard box

(upright, facing sideways) (lying down, facing up) (lying down, facing sideways) (lying down, facing down)

11 x 9 × 12 in. closed cardboard box

Front-Wrap Front-Wrap Top-Vert-Grip, Top-Hor-Grip Top-Pinch Front-Pinch Top-Pinch Front-Pinch Front-Wrap Top-Vert-Grip, Top-Hor-Grip

Front-Pinch Top-Pinch Front-Pinch No-Grab No-Grab

Testing objects (any position)

2 x 2 x 2 in. closed cardboard box 16 oz. soda bottle 20 oz. plastic tumbler 9 x 4 × 7 in. cardboard magazine box 10 x 10 x 10 in. closed cardboard box

box that the robot cannot and should not try to retrieve.

Using these sensors, CSL is able to use the retrieval robot to effectively retrieve discarded cups in the laboratory. Instead of programming the robot by hand, CSL learns a mapping from sensing procedures to the appropriate grasping procedures. It builds an efficient procedure and does so with a minimal of human intervention. In the next section, we describe the overall architecture of CSL.

3. Architecture of CSL

CSL is an integrated method for addressing approach and recognize tasks. It is composed of

Table 4 Grasping procedures

Procedure Range Speed

Front -Pinch 6 in. 45 sec. Front-Wrap 6 in. 60 sec. Top-P inch 6 in. 63 sec. Top-Vert-Grip 6 in. 80 sec. Top-Hot-Grip 6 in. 80 sec. No-Grab 6-999 in. 0 sec.

an inductive learning method and a control method. The learning method generates a robot control procedure, and the control method executes it. Generally speaking, robot control procedures specify decision trees. For CSL's tasks, the nodes of a decision tree correspond to testing the outcome of a sensor, the links indicate actions for each outcome, and leaf nodes correspond to appropriate grasping classes. Fig. 3 depicts such a decision tree.

The typical actions in a control procedure involve navigation. Although the onboard sensing procedures can be applied close to an object and from different positions, their values rely on where they are applied. Before executing a sensing or grasping procedure, the robot must move within range (guided by the overhead vision). In the robot domain, a position has two values: a relative distance and a relative orientation between a

WRAP PINCH GRIP

Fig. 2. Basic grasping procedure types.

A cost-sensitive machine learning method 3 5

move-to-(16,90) (95s) width-deg-5 (16,90) (45s)

[15, 18] height-deg-1 (16,*) (76s) [12, 15] move-to-(6,*) (250s)

Front-wrap(6,*) (60s) [19,24] move-to-(6,*) (250s)

height(6,*) (84s) [5.5, 7] Top-pinch(6,*) (63s) [9, 9] Front-wrap(6,*) (60s)

[30, 30] move-to-(6,0) (550s) front-hole? (6,0) (105s)

[0, 0] move-to-(6,181) (650s) Front-pinch(6,181) (45s)

[1, 1] Front-pinch(6,0) (45s) [25,25] move-to-(6,*) (250s) ; if this feature is 25,

height(6,*) (84s) ; then . . .

[3, 3] Top-vert-gnp(6,90) (80s) [4, 4.5] Front-pinch(6,0) (45s) [9, 9] move-to-(6,0) (550s)

front-hole? (6,0) (105s) [0, 0] move-to-(6,181) (650s)

Front-pinch(6,181) (45s) [1, 1 ] Front-pinch(6,0) (45s)

[36, 50] height-deg-5 (16,*) (42s)

; move in a position. ; apply sensing and measure ; this feature.

; if this feature is between ; [15,18] then ...

[8, 8] move-to-(6,*) (250s) Top-vert-grip(6,90) (80s)

[15, 16] height-deg-1 (16,*) (76s) [15, 15] move-to-(6,90) (250s)


Front-pinch(6,272) (45s) [1, 1] Front-pinch(6,90) (45s)

[22, 22] move-to-(6,*) (250s) top-hole? (6,*) (84s)

[0, 0] No-grab [I, 1] Top-pinch(6,*) (63s)

[26, 30] hcight-dcg-1 (16,*) ('/6s) [30, 30] move-to-(6,0) (550s)


Front-pinch(6,181) (45s) [1, 1] Front-pinch(6,0) (45s)

[35, 35] No-grab

; if this feature is between ; [36,50], then ...

Fig. 3. A cost-sensitive decision tree given initial robot position (25in., 90°). Costs are given in parentheses in seconds. The symbol * in a position means "don't care".

sensor and an object. If the robot starts at initial posi t ion (25 in., 90 o), according to the decision tree in Fig. 3, it will move to the posi t ion (16 in., 90 ° ) and measure Width-Degree-5 °. I f the value is 16 o, the robot will cont inue to measure Height-De- gree-1 °. If its value is 13 °, the robot will then

move to the posi t ion (6 in., 90 o) and execute the grasping procedure Front-Wrap. The total execu- t ion cost is es t imated at 526 seconds.

The next subsect ion will describe how to generate a cost-effective decision tree, and the following subsect ion will descr ibe how to execute a decision

36 M. Tan, J. C. Schlimmer

tree in order to grasp an unknown object in an noisy environment.

3.1. Learning in CSL

The learning task is defined in Table 5. A robot agent initially has only basic knowledge about its sensing procedures and grasping procedures (see Table 2 and 4), namely, (1) costs, (2) necessary preconditions, (3) expected errors, and (4) features. The grasping procedures of training objects are assumed to be given. (They could be determined by the robot by conducting grasping experiments.) CSL solves the learning task heuristically in two stages: (1) Training stage: Given a set of training objects

only labeled by their appropriate grasping procedures, direct the robot to interact with objects and build up a library of object descriptions.

(2) Execution stage: Take an initial world state and learned object descriptions as input, build an efficient and reliable cost-sensitive decision tree to grasp an object.

The communication between the two stages are object descriptions. An object description contains an object identification, the class of its grasping procedure, and its feature values (initially empty). The format of an object description is {object-id, grasping-procedure(position), [(feature l(positionl), value 1), (feature 2 (position z), value2) ..... (feature, (position,), value,)]). For example, (Width-De- gree-l°(26,0), 16) means that 16 degrees is the value of feature Width-Degree-l° at the position in which an object is 26 inches away and its orientation relative to the sensor is 0 degrees.

The training stage examines each training object and stores useful feature values in an object description. The execution stage compares the feature values of a new object with known object descriptions. The grasping class of a similar object description is predicted as the one for the new object. The actual learning method is not this simple because of cost considerations. The training stage can not afford to measure every feature for each training object, and neither can the execution stage. Both stages must be selective. For simplicity, we describe the training stage and the execution stage in a reverse order.

3.1.1. Execution stage The execution stage starts with a set of object

descriptions (presumably generated by the training stage) and an initial world state. It builds a cost-effective decision tree (e.g., Fig. 3) to determine the grasping class of a new object. Like ID3, CSL constructs a decision tree using a top- down, divide-and-conquer approach: select a feature, divide the object descriptions into subsets characterized by the possible values of the feature, and follow the same procedure recursively with each subset until no subset contains object descriptions from different classes. These single-class subsets correspond to leaves of the decision tree and can be labeled with that class. To handle bounded sensing noise, knowledge of expected errors is used to smooth the values of features used for subset division. If two values are within the expected error, then they are merged into the same subset, and the branch is labeled with their range. To classify a new object, CSL repeatedly tests the feature at the current subtree root and

Table 5 The learning task

Given: • A set of training objects labeled by appropriate grasping procedures, • Knowledge of sensing, moving, and grasping costs, • Knowledge of sensing and grasping preconditions, • Knowledge of expected sensing and moving errors, and • Knowledge of which sensing and grasping procedures yield which features; Construct a robot control procedure that: • Given:

- An object and - An initial world state,

• Efficiently executes sensing, moving, and grasping actions to appropriately grasp the object.


follows the matching branch. When it reaches a leaf, it predicts the class associated with the leaf. After CSL executes the corresponding grasping procedure, CSL saves the object description of the new object for future tree building. If there is a prediction error, CSL also adds additional features into the new object description (similar to the discrimination process in the training stage described later).

This divide-and-conquer approach depends on making a good choice of feature to test at each stage. ID3 applies an information gain measure to estimate the correlation between example features and classes. Specifically, ID3 selects the feature that maximizes the equation:

I(F) = [iE~(c } -P(Ci) l°g2P(Ci)}

-[ Y'~ p(F= Vi) ~_, -p(CjlF= V~) t iEF j ~ { C }

×log2p(Cj I F = V/)],

where F is a feature, V, is the i th value of F, and (C } is the set of classes. Intuitively, the greater the information gain, the greater the separation between the classes among the partitions of that feature, and therefore the greater the ability to classify over that feature according to the relevant partitioning.

In the robot domain, object descriptions are often incomplete. Some object descriptions may not have a value for each selected feature. CSL evenly distributes the object descriptions with unknown values of a feature across the subsets before calculating its information gain. 3 Since each subset likely includes more classes by the distribu- tion, the information gain of the feature is re- duced. If a feature is selected, the object descriptions with unknown values of the feature are in- cluded in all subsets. However, when CSL tries to predict a class, they are ignored.

To make decision trees cost-effective, CSL modifies ID3's evaluation function - information gain. Instead, it uses the marginal information gain - I(F)2/C where C is the cost of measuring feature F. The run-time cost of a feature at a

3 The subsets includes the unknown subset itself if the existing subsets do not completely cover the range of the feature.

position is the sum of the cost of sensing the feature and the cost of moving to its position. If the information gains and sensing costs of two features are the same, then CSL will prefer the nearby feature. If all features have the same run- time cost, then CSL behaviors just like ID3. Other researchers have suggested different marginal information gains [3,4]. For the robot domain, they seem to generate similar cost-effective decision trees.

Traditionally, ID3 generates a decision tree from existing examples and uses it for all subse- quent classifications. However, a different robot initial position often corresponds to a different decision tree. It is difficult to build a complete decision tree that can characterize the discrete position regions corresponding to the relevant decision trees. In addition, in a dynamic environment, the information gain and cost of a feature is difficult to predict before run time. Sensing features are not always available and their costs frequently change. For these reasons, CSL builds a cost-effective decision tree at run time. More precisely, only the part of the tree relevant to an object classification is actually generated. CSL trades a small computation time for a greater flexibility.

3.1.2. Training stage Before training, each object is labeled only with

its appropriate grasping procedure - - its position-dependent features must be collected via sensing. The goal of the training stage is to collect enough sonar-based features to allow efficient and accurate classification of objects with respect to grasping procedures in the execution stage. Since four of the seven sensing procedures (H-Sweep-l°, H-Sweep-5°, V-Sweep-l°, V-Sweep-5°) may be applied at any distance and at any orientation relative to the object, applying these sensors at every possible position is out of the question. In order to reduce the cost of building a library of annotated object descriptions, the training stage must find useful positions at which to apply these sensing procedures and choose among them when necessary. A two-phase approach does this.

In the first phase of the training stage, CSL chooses a small set of training positions for which the sensing are likely to be useful. This consists of determining orientations and distances (relative to the object) useful for sensing. As we mentioned

38

Table 6 CSL's training stage

M. Tan, J.C. Schlirnmer

1. For each object and its grasping procedure do: • Familiarize: Use the current tree to predict the object grasping procedure. If the selected feature is unknown for this object,

apply the appropriate sensing procedure and record the sensing feature's value in the object's description. • If the tree predicts the wrong grasping procedure for this object,

(a) Discriminate: Apply the next cheapest sensing procedure to the object, record the sensing feature's value in the object's description.

(b) Reorganize: Rebuild the tree using all object descriptions. 2. For each remaining incorrectly classified object, request another presentation of the object, and go to 1. 3. Stop if all objects are correctly recognized or all sensing procedures have been applied at this training position, and output the

object descriptions.

earlier in Section 2, each training object is perceived by the ceiling vision subsystem as a polygon in a region map. Since the sonar works best when applied perpendicular to the face of an object, orientations are chosen which correspond to the edges of the polygon perceived by the vision subsystem. All the perceived face orientations are then clustered into a smaller set of possible orientations. For example, given the set of objects listed in Table 3, CSL comes up with the four orientations, 0 °, 90 °, 181 °, and 272 °. If given just cups which are upright, CSL comes up with just 0 °. Although this 2-D solution does not completely solve the problem of accounting for differences in object perception with respect to orientation, it has proved sufficient for this task.

Once orientations are chosen for the training positions, CSL selects training distances. To do this, CSL relies on the knowledge that two dissimi- lar objects appear more similar with increasing distance. Therefore, CSL searches for distances at which individual sensing features can reliably discriminate between objects with different grasping classes. Specifically, given a training orientation, CSL starts at the closest distance (6 inches) and applies H-Scan-0.3 to measure the width of the training objects. These values plus simple knowledge of geometry are used to form crude estimates of the expected feature values of Width-Degree-1 o for H-Sweep-1 ° (and Width-Degree-5° for H-Sweep-5 ° ) at different distances. A training distance is then defined as the greatest distance at which a rehable sensor distinction (i.e., > expected sensing error × 2) can be made between objects of similar width whose grasping classes are different. For example, using the training objects

listed in Table 3, CSL predicts that for orientation 90 °, H-Sweep-1 ° can distinguish between a cup and a box at 24 inches or closer. CSL repeats this process using D-Scan-0.3 to measure height, and predicts the maximal discriminating distances for V-Sweep-l° and V-Sweep-5 o. 4 Because each sensing procedure may give rise to a number of training distances, and because there could be a large number of sensing procedures, CSL clusters the possible distances into a small set. The final training distances are 6, 16 and 24 inches.

In the second phase of the training stage, CSL uses an iterative, discriminative learning method to build a cost-sensitive decision tree for each of the training positions (see Table 6). The tree itself is not important, but it has the important side-effect that it ensures that enough sensing procedures have been applied to the training objects to discriminate between their different grasping classes.

In building a tree from the currently known descriptions of training objects, the best feature is selected by the same evaluation function used in the execution stage. During classifying each training object, CSL repeatedly executes the corresponding sensing procedure by following a matching branch and adds the feature values into the object description of each training object. If a classification error occurs at a leaf, CSL continues to measure the next cheapest feature of a training object until its object description is differentiated from other conflicting object description(s). To

4 CSL could do some type of binary-search to find useful sensing distances by experiments, but using domain/geometry knowledge is faster.


reflect this new information in future classification, CSL rebuilds a tree from scratch. This organization process is triggered by classification errors and provides an opportunity for all features to compete for top positions in a decision tree, and it possibly eliminates potential misclassifica- tions. If a tree can not classify all training objects correctly, CSL repeats the above training process on remaining misclassified training objects. Ide- ally, only a few and cheap features are sufficient to build a complete discriminative decision tree at each training position. One interpretation of this preference is that CSL's learning method is biased toward the use of inexpensive sensing features.

Finally, CSL combines object descriptions at different training positions and outputs a set of global object descriptions that cover all training positions. During the process of merging position-dependent object descriptions, CSL also tries to detect the similarity of the values of the same features from different sensing positions. If two values of the same feature from different sensing positions are within its expected error, that position information is generalized. For example, if Height(6,0) = Height(6,90) = Height(6, 180) = Height(6, 270) then feature Height is orien- tat ion independent for this object. The four feature-value pairs of feature Height will be replaced by a single pair (Height(6, * ), average-value). (The symbol * means 'don't care.') This kind of generalization on sensing positions is extremely effective for symmetrical objects. Normally, generalized features (having *s in their position param- eters) are cheaper (less moving costs) and more reliable (less moving errors) because their preconditions are easily met. Of course, the price to pay is the possibility of overgeneralization. In practice, whether the position of a feature can be generalized or not depends on the nature of the corresponding sensing procedure and objects. CSL also extends this concept to grasping positions and represents grasping procedures in the form of grasping-procedure (distance, orientation).

In summary, CSL's learning cost includes both the training cost and execution cost. In the training stage, selecting training positions, collecting feature values, and generalizing the positions of features are all object-sensitive and need-driven. Together, they significantly reduce the training cost. Similarly, in the execution stage, using decision trees avoids measuring every feature, and

making decision trees cost-sensitive reduces the execution cost.

3.2. Control in CSL

The control method in CSL cooperates with the learning method in the execution stage. Under ideal circumstances, CSL can classify an object by following the decision tree generated by the learning method (see Section 3.1.1) and then executing the grasping procedure corresponding to a predicted grasping class. In reality, sensing data is noisy, and grasping actions can fail. In the robot domain, the known causes of noise include improper arm calibration, inaccurate orientation estimation, wrong sonar reading, communication problem, low battery, and human interference. Unfortunately, even small changes in the feature values of an object being classified may result in sudden and inappropriate changes to the assigned grasping class in a decision tree. A control strategy is needed to cope with unexpected events to con- tinue classification.

As mentioned earlier, CSL takes advantage of knowledge about expected errors and labels each branch of a tree by a range. As a result, small changes in feature values within a range do not affect classification outcomes. However, if a feature value lies between two disjoint ranges, a decision must be made about which branch to follow. Instead of committing itself to an unrelia- ble decision, CSL decides to choose an alternative feature - the next best one, hoping that it has a reliable value to match. This reselection is guided by the learning method and can be repeated if necessary.

Sometimes, due to severe sensing noise, no (or few) features at a particular selection stage yield reliable values. This happens often when previous feature values are incorrectly matched (i.e., they took wrong branches). If CSL has either failed to select a reliable feature at a node or has already tried alternative features N times (N is a parameter and usually set to 3), it invalidates the feature value of the parent node, and selects a different feature for the parent node. This backtracking process can also be repeated if necessary. If all these reselection strategies can not lead to a successful leaf, CSL resorts to the nearest neighbor match strategy to select a branch.

40 M. Tan, J. ('. Schhmmer

Eventually, CSL will reach a leaf of a decis ion tree. At this point , there are two foreseeable failures before grasping succeeds: (1) Class i f icat ion failure: The leaf is ambiguous in

terms of grasping classes. (2) G r a s p failure: The robot fails to grasp an

object af ter execut ing an unambiguous grasping procedure .

CSL must resolve both failures. In the case of a c lass i f icat ion failure, the sensory

in format ion is not sufficient to d i scr imina te between the remain ing grasping classes. For tuna te ly , CSL treats grasping classes jus t as b ina ry (success or failure) features. It will select a grasping procedure. If it succeeds, then the task is achieved. Otherwise, CSL will follow the ' fa i lu re ' b ranch to select o ther grasping procedures to de te rmine a correct class. In fact, if grasping procedures were inexpensive, then they could be selected at the top pos i t ions of a decis ion tree. However , since they cause side effects (expla ined later), grasping procedure costs are usual ly higher than their ac tual

execut ion times. Af te r executing a grasping procedure , a grasp

fai lure can be de tec ted by a f inger -mounted in- f rared sensor. If all of the grasping classes at a ( ambiguous or unambiguous) leaf fail, CSL mus t look for the next best grasping p rocedure some- where else. There are several op t ions in CSL [12]. One s t rategy is that af ter a grasp failure, CSL removes the object descr ip t ions labeled by the fai led class and rebui lds a new decis ion tree s tar ted

at the c u r r e n t posi t ion. CSL uses the new tree to reclassify an object.

Both recovery strategies assume that a failed grasp act ion does not affect an object . This is not always true in the robot domain . The grasping

procedures do use some in ternal sensors to protect the robot arm. However , they are not sensit ive enough to prevent an object being pushed away or changed to a different object (e.g., knock ing a cup over). In the former case, an objec t remains the same and only needs pos i t ion updat ing . The la t ter case is d i f ferent because of side effects. Al l previous sensing da ta are now invalid, and CSL must s tart all over again! Current ly , CSL chooses to use vision to d i f ferent ia te the two cases. It keeps the number of pixels of the image of an object to be grasped in a memory. Af te r each fai led grasp, CSL takes a p ic ture and compares the numbers of the pixels before and after the grasp act ion. If the difference is large, it assumes that the object has changed, otherwise, upda tes the pos i t ion of the object p repa red for the next act ion. In the future, we p lan to use noncon tac t sensors m o u n t e d at the tips of its fingers to detect an abor t ed grasp ing procedure and, therefore, prevent side effects.

The control me thod al lows CSL able to to lera te sensing noise and recover f rom grasp failure. It controls the execut ion of a decis ion tree. It also extends c lass i f icat ion to p r o b l e m solving. More impor tan t ly , the control me thod makes incremental learning in the execut ion stage become reality. Wi thou t successful grasps, CSL would not be able

Table 7 CSL's execution stage

1. If there is only one grasping procedure left in the remaining object descriptions, execute it, and go to 3. 2. Otherwise,

(a) If no reliable feature can be selected, backtrack, and repeat. (b) Select a new feature from the object descriptions, partition them according to the feature values, and create the corresponding

subtree structures. Move the robot to the position of the selected feature, and apply the feature's sensing (grasping) procedure. (c) If the feature is derived from a sensing procedure,

• Follow the branch which matches the feature's value for this object, and go to 1. • Otherwise, go to (a).

(d) If the feature is a grasping procedure, go to 3. 3. If the grasping procedure succeeds, discriminate and save the new object description for future classification and stop. 4. Otherwise,

• If the procedure has no side effects, - If the procedure is selected as a feature, follow the 'failure' branch, and go to 1. - Otherwise, delete the current tree, remove the object descriptions having the applied grasping procedures, update the current

world state, and go to 1. • Otherwise, restore all object descriptions, start from scratch, and go to 1.


front-wrap

Depth

4.5~.5 front-pinch Width

top-pinch ~ 2 . 5

front-wrap

Depth front-wrap X~. 5

top-pinch

Depth

4.5~2.5 front-pinch Height

4.5//front-wrap

top-pinch

Depth front-wrap/~

front-pinch top-pinch

(c)

Depth

4.5~.5 front-pinch Height

top-pinch front-wrap

(a) (0)

Fig. 4. Growth of a cost-sensitive decision tree during training stage.

to learn the correct grasping concept. The pseudo code of the execution stage including both learning and control is given in Table 7.

4. Detailed example

Assume that there are two objects: a cylin- drical, closed tennis ball can (8 in. tall and 2.5 in. in diameter) and a styrofoam coffee cup (4.5 in. tall and 2.5 in. in diameter). Their grasping procedures are Front-Wrap (50 sec. to execute) and Top-Pinch (70 sec. to execute) if they are upright, respectively, and Front-Pinch (70 sec. to execute) if the cup is lying down. 5 Further assume that there are four sensing procedures, each of which yields one sensing feature: Depth (20 sec. to apply), Width (25 sec. to apply), Height (30 sec. to apply), and Height-Degree-5 ° (10 sec. to apply). The first three must be used at a distance of 6 inches, and the last only at distances greater than this. The expected errors of the first three are + 0.5 in., and for the

The tennis ball can was not presented lying down for simplicity of presentation. The experiments in Section 5 do not make any such simplifications.

l a s t / i t is + 3 °. Assume that the robot takes 2 seconds to move 1 inch in any direction.

To simplify the discussion, assume that CSL selects only one training orientation, 0 °, and two training distances, 6 and 32 inches. 6 Thus, there are two training positions: (6 in., 0 o) and (32 in., 0 o). Fig. 4 depicts the iterative steps at the first training position.

At the first training position, CSL correctly classifies the first object - - an upright tennis ball can - - by default (cf. Fig. 4a). However, the empty tree fails to classify the next object - - an upright coffee cup. This misclassification forces the discrimination process to execute the cheapest sensing procedure Depth which yields a value of 2.5 inches. The tree is expanded to incorporate this new distinction (cf. Fig. 4b). When the styrofoam cup is seen lying down, the familiarization process selects Depth and obtains 4.5 inches and creates a branch in the tree for this value (cf. Fig. 4c). Upon reprocessing the upright tennis ball can (in Step 2, Table 6), the familiarization process applies Depth and obtains 2.5 inches. There- fore, the tree classifies the can as requiring

6 For these objects, CSL actually selects four orientations and the same two distances.

42 M. Tan, J.C. Schlimmer

Table 8 Annotated object descriptions generated by example operation of the training stage

Object ID Grasping procedure sensing feature values

Tennis ball can Front-Wrap Depth(6,0) = 2.5 Width(6,0) = 2.5 Height(6,0) = 8.0 Height-Degree-5 ° (32,0) = ] 5

Top-Pinch Depth(6,0) = 2.5 Width(6,0) = 2.5 Height(6,0) = 4.5 Height-Degree-5 ° (32,0) = 7

Front-Pinch Depth(6,0) = 4.5 Height-Degree-5 ° (32,0) = 6

Styrofoam coffee cup (upright)

Styrofoam coffee cup (lying down)

Top-Pinch, which is incorrect. This misclassification is resolved by introducing the next cheapest sensor Width (cf. Fig. 4d). After applying Width to the upright cup, and realizing that it too is unable to discriminate between the upright cup and can, the discrimination process again tries the next cheapest feature Height. The following reorganization process rebuilds the tree and removes nondis- criminatory feature Width (cf. Fig. 4e). Finally, the tree is completed by the familiarization process as it applies Height to the can during reprocessing (cf. Fig. 4f).

Operation of the training stage at the second training position is similar but simplified. Because only the Height-Degree-5 ° sensing procedure is

available, there is no iteration between discrimination, familiarization, and reorganization.

Table 8 lists the resulting object descriptions after the completed training stage. Note that in one case, the lying-down styrofoam coffee cup, not all sensing procedures were applied. Tradi- tional machine learning approaches frequently assume that all features are available for all in- stances during learning. Instead of implementing this brute force strategy, CSL heuristically anno- tates object descriptions for use in learning during the execution stage.

Consider the operation of the execution stage given the initial position of being 32 inches away at a 15° relative orientation with respect to the unknown object. The task is to recognize the object as quickly as possible, approaching if necessary, and then carry out the appropriate grasping procedure. To select the first sensing feature to test (and where to test it), CSL calculates cumula- tive (moving plus sensing) costs for every sensing feature of the two positions. Using the evaluation func t i on 1 2 / C , Height-Degree-5 ° (32,0) has the value 0.592/(2 × 7 + 10) and Depth(6,0) has the value 0.592/(2 x 27 + 20). (Others are computed similarly.) In this case, Height-Degree-5 ° (32,0) has the highest value and is selected as the root (cf. Fig. 5a). If applying Height-Degree-5 ° at 32 inches and 0 ° yields a value of 15 ° , CSL will execute Front-Wrap and stop. If Height-Degree-5° yields 7 o, CSL will select another feature, in this case Depth(6,0) (cf. Fig. 5b). As discussed in Sec- tion 3.1.1, if the sensing procedures yield unexpected values, or if the grasping procedures do not succeed as expected, then reselection or recovery strategies are applied.

move-to-(32,0) (14s) Height(32,0) (10s) I [6, 7] f ' ~ , ~ 5 , 15]

front-pinch(6,0 )(?Os) I l move-to-(6,0) (52s) top-pinch(6,*) (70s) front-wrp(6,*) (.50s)

move-to-(32,0) (14s) He ght(32,0) (105) I

[6, 7] f ~ " ~ 5 , 15]

move-to-(6,0) (52s) move-to-(6,0) (52s) Depth(6,0) (20s) front-wrp(6,*) (50s)

[4.5, 4 . 5 ] / / ~ " ~ 5 , 2.5]

I move-to-(6,0) (0s) front-plnch(6,0) (70s) l move-to-(6,0) (0s) top-pinch(&*) (70s) ]

(a) (b)

Fig. 5. Growth of a cost-sensitive decision tree during execution stage.


5. Empirical results

We have conducted several e.xperiments using CSL to control the Hero robot. Using the training objects listed in Table 3, CSL identified four orientations 0 °, 90 °, 181 °, and 272 °, and three training distances 6, 16, and 24 inches for each orientation.

After training, we first tested CSL's performance on the training objects. Note that if this were a simulation using the same sensing data then CSL should succeed automatically. However, there was no guarantee of success in the real world because sensing values could be noisy. It turned out that for lying-down cups, if the robot ap- proached them from 0 ° or 181 ° orientations, the Depth values sometimes were unusual small due to the combination of small errors in orientations and the surface curvature of cups. Fortunately, when that happened, CSL would ignore the value of Depth(6,0 °) or Depth(6,181 °) and select other more reliable features, e.g., Depth(6,90°). Occa- sionally, the Width(6,90 o ) of the lying-down Coke can could also be mismeasured, resulting in an inappropriate grasping procedure Front-Wrap (6,90 o ). It failed because the lying-down Coke can was too wide for Front-Wrap(6,90 o). In addition, the Coke can may also roll away as the result of applying Front-Wrap(6,90 o ). CSL was able to track down the Coke can, rebuild a decision tree from the current position, and pick it up by executing the next most likely grasping procedure either Top-Vert-Grip or Top-Hot-Grip depending on its current approach orientation. With the help of the control method, CSL was able to grasp the training objects successfully most of the time.

To find out the generalization ability of CSL's decision trees, we also tested CSL's performance on unseen test objects (listed in Table 3) with arbitrary positions. It succeeded to grasp the majority of them. The overall success rate of CSL was over 80%, which we believe to be relatively high given the low quality sonar sensor and the relative inaccuracies of the robot's drive motors. Failures occur when CSL encountered objects whose grasping classes could not be inferred from training objects. One of such failures is the upside down 20 oz. plastic tumbler. During the classification, CSL only measured the shape of a cup-like object, and determined the upside down cup was similar enough to an upright cup. As a result, CSL

output grasping class Top-Pinch and only found Hero's fingers jammed by the bottom of the cup. Eventually, by failure recovery, it found out that the correct grasp class was Top-Vert-Grip. To prevent the same failure in the future, CSL disam- biguated the upside down cup from other upright cups by measuring additional features. In this case, CSL realized that the upside down cup had no hole on the top and recorded its Top-Hole? value in the new object description. Next time around, CSL would use feature Top-Hole? to distinguish upside down cups from upright cups. Indeed, it succeeded in the following experiment.

In addition to be able to accomplish the object grasping task, the robot control procedures formed by CSL's execution stage also appear to be highly efficient, averaging 12 minutes per object (including 20 seconds computation time). If CSL measured every feature in existing object descriptions as ID3 would do for both classification and learning, it would take over one and half hours to classify each object. Even just for classification, the average cost-sensitive decision trees are at least 30% cheaper than the standard decision trees. Our'initial experiments with CSL demonstrate that as moving costs increase, the control procedure tends to favor sensing at a distance, and inversely, when moving costs decline, the control procedure favors using sensors at close distances. Similarly, if sensing costs increase, the control procedure begins to use grasping procedures as degenerate, succeed-fail sensing procedures.

CSL represents one point in the tradeoff between training and execution. Specifically, it could be possible to perform exhaustive training by applying all sensing procedures at all (or a very large number of) training positions. This should yield a better library of objects from which to build control procedures during the execution stage. The approach taken by Nunez [4] can be interpreted as doing this. The cost for the potential improvement in quality of execution stage output is prohibitive in this domain. A reasonably thorough, non-heuristic training stage would take over 5 years instead of the current 7 hours. But would this great expense yield an equally worthwhile gain in the quality of the resulting control procedures? To test this hypothesis, we computed the output from a (hypothetical) complete training stage. Using these objects, CLS's execution stage produced control procedures that

44 M. Tan, J. C Schlimmer

were on average 9% faster per object. This savings is overwhelmed by the exorbitant cost of training for all but the most stable domains.

It is difficult to give accurate estimates of the performance improvement due to the reselection and recovery strategies. This is partially because there were so many different test conditions, and we could not know when and how often sensing noise would occur. However, the performance improvement by these strategies was obvious, roughly a 15% increase. Also as the number of object descriptions increases, the information gain measure becomes more reliable. Noisy features are be filtered out, and reliable features are chosen. This also reduces the failure rate.

6. Observations and conclusions

CSL is an integrated learning and control system to approach, recognize, and retrieve objects. It assumes that objects are apart and stationary on the floor. Its working environment is benign. In fact, CSL can not handle the situations like the disappearance of an object during sensing, low battery, or moving obstacles. However, CSL can be integrated with other robot systems [8] to remove these restrictions. We have been exploring this possibility.

Excluding the usage of the domain-related heuristics to determine sensing positions, CSL is independent of application domains. Given a different robot, CSL will be functional after modify- ing its initial knowledge (i.e., costs, preconditions, expected errors, and features). If the robot were equipped with side-effect-free grasping procedures, CSL could learn by itself without a teacher. If the robot could determine sensing positions through an onboard sensor, then CSL could eliminate the ceiling camera, and the robot could retrieve objects outside a laboratory. The quality of the sensors is directly related to the complexity of objects to be grasped. If a sensor can not detect the opening of a small cup, then blindly putting a finger into a cup-like object is a dangerous action. Given better sensors, CSL should be able to handle more complicated objects.

The learning method of CSL interacts with an environment. It not only learns the grasping concept from object descriptions but also searches for discriminative features. It adopts an open-world

policy such that new features can be accepted any time. Although the requirement of cost sensitivity forces CSL to be more focused and selective, it is still able to explore the feature space when necessary. It is not necessary that CSL's learning must base on ID3. Other learning-from-examples methods can also be modified for CSL [11].

The control method of CSL is a conservative approach aiming at handling noise. If sensing noise is small, a more aggressive approach (e.g., using the nearest neighbor match fight away) will be more efficient. In fact, CSL has a parameter which controls how soon to switch from reselection to the nearest neighbor match. Other ways to tradeoff between reliability and efficiency include artificially changing expected errors or the estimation of moving costs.

There are several important issues not addre- ssed by CSL. For object recognition, how to determine sensing orientations in general is an open question. The tradeoff between reducing execution cost and handling noise is not explored. CSL also neglects learning failure recovery costs for grasping procedures, which makes it difficult to estimate their actual costs. Nevertheless, we believe CSL's approach can be extended to other robot recognition tasks.

Acknowledgements

We would like to thank Tom Mitchell and Long-Ji Lin for the robotics hardware used in this research, and Rich Caruana, Klaus Gross, and Tom Mitchell for their comments on a draft of this paper. Thanks also to the 'gripe' group for providing a consistent and reliable computing environment. This research is supported by the Na- tional Science Foundation under grant IRI- 8740522 and by a grant from Digital Equipment Corporation.

References

[1] Michalski, R.S. and R.L. Chilausky, Learning by being told and learning from examples: an experimental com- parison of the two methods of knowledge acquisition in the context of developing an expert system for soybean disease diagnosis, International Journal of Policy Analysis and Information Systems 4 (2) (1980) 125-161.


[2] Mitchell, T.M., Version spaces: a candidate elimination approach to rule learning. Proceedings of the Fifth IJCAI, Cambridge, MA. (1977) 305-310.

[3] Norton, S.W., Generating better decision trees, Proceed- ings of the Eleventh IJCAI (Morgan Kaufmann, Los Al- tos, CA, 1989) 800-806.

[4] Nunez, M., Economic induction: a case study, Proceed- ings of the Third European Working Session on Learning (Glasgow, Pitman, 1988) 139-145.

[5] Quinlan, J.R., Induction of decision trees, Machine Learn- ing 1 (1986) 81-106.

[6] Quinlan, J.R., The effect of noise on concept learning, Machine Learning: An Artificial Intelligence Approach (Morgan Kaufmann, Los Altos, CA, 1986) 149-166.

[7] Shafer, S.A. and T. Kanade, Recursive Region Segmenta- tion by Analysis of Histograms. Proc. Intl. Conf. on Acoustics, Speech, and Signal Processing (IEEE, 1982) 1166-1171.

[8] Simmons, R., L,J. Lin and C. Fedor, Autonomous task control for mobile robots, Proc. of the 5th International Symposium on Intelligent Control Philadelphia, PA (1990).

[9] Tan. M. and L. Eshelman, The impact of noise on learning, Technical Report CMU-CS-88-144, School of Com- puter Science, Carnegie Mellon University (1988).

[10] Tan, M., CSL: a cost-sensitive learning system for sensing and grasping objects, Proc. of the 1990 IEEE International Conference on Robotics and Automation (IEEE Computer Society Press, 1990) 858-863.

[11] Tan, M. and J.C. Schlimmer, Two case studies in cost-sensitive concept acquisition, Proc. of the Eighth National Conference on Artificial Intelligence (AAAI Press/The MIT Press, 1990) 854-860.

[12] Tan, M., Cost-sensitive robot learning, Ph.D. thesis, School of Computer Science, Carnegie Mellon University (CMU- CS-91-134).

a cost-sensitive machine learning method for the approach and recognize task

Documents