[studies in computational intelligence] soft computing techniques in vision science volume 395 ||...

S. Patnaik & Y.-M. Yang (Eds.): Soft Computing Techniques in Vision Sci., SCI 395, pp. 171–182. springerlink.com © Springer-Verlag Berlin Heidelberg 2012

Immunised Navigational Controller for Mobile Robot Navigation*

Dayal R. Parhi, B.B.V.L. Deepak, Jagan Mohana, Rao Ruppa, and Meera Nayak

Abstract. Over the last few years, the interest in studying the Artificial Immune System (AIS) is increasing because of its properties such as uniqueness, recognition of foreigners, anomaly detection, distributed detection, noise tolerance, reinforcement learning and memory. Previous research work has proved that AIS model can apply to behavior-based robotics, but implementation of idiotypic selection in these fields are very few.The present research aims to implement a simple system architecture for a mobile robot navigation problem working with artificial immune system based on the idiotypic effects among the antibodies and the antigens. In this architecture environmental conditions are modeled as antigens and the set of action strategies by the mobile robot are treated as antibodies. These antibodies are selected on the basis of providing the robot with the ability to move in a number of different directions by avoiding obstacles in its environment. Simulation results showed that the robot is capable to reach goal effectively by avoiding obstacles and escape traps in its maze environment.

Keywords: Artificial Immune System, Idiotypic effect, Immune Network, Robot Navigation.

1 Introduction

Artificial immune system (AIS) has been developed from the natural immune system to solve engineering problems efficiently. Based on these issues previous

Dayal R. Parhi . B.B.V.L. Deepak . Jagan Mohana . Rao Ruppa Department of Mechanical Engineering, National Institute of Technology- Rourkela e-mail: [email protected], [email protected] Meera Nayak Lecturer, G.I.E.T., Bhubaneswar e-mail: [email protected]

172 D.R. Parhi et al.

researchers have been developed idiotypic-network theory which involves immune responces.

Previous researchers applied AIS to real-world problem such as the recognition of promoters in DNA sequences [1], traveling salesman problem [2], reduce the dimensionality of the image space [10] and generates network structures[10] useful for navigation and localization etc. There are some similarities [3] in between AIS and artificial neural network (ANN) in a computational viewpoint because these two are biologically inspired techniques. Applications of AIS in robotics field are very few [4-11]. Two reactive immune networks [5-7] modeled for robot motion planning. To get global feedback to the system, potential field based immune network and Velocity Obstacle methods have used to guide the robot by avoiding collision with moving obstacles. In order to detect vulnerable areas in its free environment and adapts to the required speed for a mobile robot, an adaptive learning mechanism [8] has been introduced based on the natural immune system. Patrícia et al. proposed a non-parametric hybrid system [9] combining the strengths of learning classifier systems, evolutionary algorithms, and an immune network model to solve autonomous navigation problems effectively. Most of the previous work dealt with AIS which includes stimulation and suppression effects. Because of this the global feedback is not observed by the system.

In this paper immunised system architecture has been developed for mobile robot navigation problem. Later on RL algorithm is integrated to sytem architecture in order to get environmental feedback to the system.

2 Background

Human immune system protects bodies with the help of antibodies which are proteins produced by B cells, when foreign substances called antigens entered into the bloodstream. The response of immune system is in two ways: primary and secondary.

2.1 The Primary and Secondary Response of the Immune System

When the immune system encounters the antigen for the first time the primary response will activate and reacts against it. The secondary response occurs when the same antigen is encountered again. When the same antigen enters into blood stream a large amount of antibody will produce rapidly because of primary response memory.

2.2 Antibodies

Antibodies bind to infectious agents of antigens with its paratope and then react with them. They are actually three-dimensional Y shaped molecules as shown in Fig.1.

Immunised Navigational Con

Fig. 1

2.3 Antibody-Antige

Each antibody consists antibody to identify other can attach are called epitperforming a pattern matcof the antigen. The streng

Jerne [9] formulated aantibodies with respect todeath rate for respective awith p number of antibod[ag1, ag2, ag3 . . . , aconcentration C of antibod

The equation (1) expresseto all presenting antigens.between antibodies and collision probability whic functions between one anthat produces both the nuconcentration when a coprinciple that antibody levthe antigen. In addition, vanished from the system

ntroller for Mobile Robot Navigation 17

Antibody Paratope and Idiotope Regions

en Binding

of two paratopes which are specific portions of thelements. The regions on the elements that the paratope

topes. Antibodies recognize the antigen and can bind bch between the paratopes of the antibody and the epitopeth of the bind is nothing but how closely the two match.

a differential equation that changes the concentration oo the stimulatory and suppressive effects and the naturantibodies (Fig.2).This model supposed that, for a systemdies [ab1, ab2, ab3 . . . , abp] and q number of antigenagq], the differential equation for rate of change idy abi is given by (1).

… … … . 1es the stimulation effect between antibody abi with respe. Here, indicates a stimulatory matching functio

antigens, and the term represents thch depends on their respective concentrations. Simlar

are the stimulatory and suppressive matchintibody with other antibodies. Variable b is a rate consta

umber of collisions per unit time and the rate of antibodollision occurs. Equation (1) is developed based on thvels are dependent upon binding between the antibody an

those with levels below a minimum affinity would band replaced with new ones.

73

he es by es . of ral m ns in

1

ect on he ly ng ant dy he nd be

174

Fig. 2

3 Artificial Immun

Nowadays, one of the navigation problem, wherwithout any external guid

A pair of desirable coThe antibody concentratigiven antigen. Such arranof the robot. In the Distribhave to get knowledge thother robots etc., and deci

Table 1 shows the relimprove the adaptabilityproposed a modified set ofunctions of the stimulatio

Table 1 C

EnStrat

AIn

4 System Architect

4.1 Immune Networ

The matching functions stimulation and suppressiantibodies. Active comm

D.R. Parhi et a

Inter Antibody Suppression and Stimulation

ne System for Robots

most difficult tasks in robotics is the autonomoure a robot, or a set of robots, has to perform certain taskance.

ondition/action rules was assigned to the each antibodion level allowed the selection of the antibodies for ngement made the fittest antibodies for the correct actiobuted Autonomous Robotic Systems (DARS), the robo

he objective of the system, environment, the behaviors oide their own action autonomously. lationship between DARS and the immune system. Ty of their system, previous research work has beeof immune network equations that took into account thon and suppression effects from the immune responses.

omparison of DARS and the Immune System

DARS Immune System nvironment Antigen tegy of Action Antibody Robot B-cell Adequate Stimulus nadequate Suppression

ture

rk Formulation

and in (1) is to determine the levels oion effects by comparing each antibody with all oth

munication is of most interest since all environment

al.

us ks

y. a

on ots of

To en he

of her tal

Immunised Navigational Controller for Mobile Robot Navigation 175

situations are considered here as antigens. It is possible to get a simpler system by removing the background communication i.e., each antibody is required to be compared with the antigenic antibody only. The communication is generated by comparing the paratope of with the idiotopes of the other activating antibodies and vice versa. By considering only the stimulation matching function between the antibody and antigen, then (1) is converted into the equation (2).

… …. . 2

In order to detect the antigenic antibody , the first sum in the square brackets should be evaluated. So equation (2) must be separated into several parts as shown in equations (3)–(5). … … … … … … 3 . … … … … . 4 … … … … … … . 5

Equation (4) is nothing but (2) in terms of , and it deduce the concentration rate of antibody with respect to time. A difference equation (4) is used to compute antibody concentrations (5) discretely.

4.2 Selection of Antibodies and Antigens

Environment conditions can be modeled as antigens and the set of antigens are tabulated in Table 2 according to its priority. The set of action strategies by the mobile robot listed in Table 3. These antibodies are selected on the basis of providing the robot with the ability to move in a number of different directions by avoiding obstacles in its environment.

Table 2 Antigen Conditions with its Priority

No Antigen Priority 1 Object front 2 2 Object left 2 3 Object right 2 4 Goal known 0 5 Object not present 1 6 Robot stalled behind 3 7 Robot stalled in front 3


Table 3 Antibodies and its action strategies

No Antibody Robot direction 1 Continue in its path Robot moves towards the goal 2 Turn towards the goal Robot rotates towards the goal 3 Move up Robot moves in 900 4 Reverse 0 Robot reverse by 00 5 Reverse 180 Robot reverse 6 Reverse 90 Robot reverse by 900 7 Move right 0 Robot moves in 00 right side 8 Move right 45 Robot moves in 450 right side 9 Move left 0 Robot moves in 00 left side 10 Move left 45 Robot moves in 450 left side

4.3 Paratope and Idiotope Matrices

Furthermore, two matrices namely paratope matrix Pa and idiotope matrix Id of dimension pxq are used in order to calculate suppression and stimulation affects. Equation (6) represents the initial hand coded reinforcement learning scores of paratope matrix Pa elements, which reflects the degree of match between each antibody to each antigen. The element values Pa [ , ] are lie in between 0 and 1. These values are then altered by scoring a small number RL(positive or negative). When the program starts, the elements of Pa are updated once in each iteration using RL algorithm (discussed in later section). However, these values are not allowed to fall below 0.00 or rise above 1.00.

0.2 0.2 0.20.1 0.25 0.250.9 0.43 0.42 0.6 0.8 0.1 0.10.6 0.7 0.15 0.150.4 0.2 0.3 0.30.7 0.2 0.60.8 0.3 0.30.80.60.050.60.050.20.20.70.80.1

0.20.80.10.20.70.20 0.1 0.6 0.80.2 0.2 0.1 0.80.30.250.450.30.45

0.10.10.50.10.40.30.30.60.30.6

10.350.10.350.1... (6)

0.2 0 00.2 0 00 0 0 0 0 0.4 0.40 0 0.4 0.40.1 0.6 0.3 00 0.1 00 0 0000.200.200.5000.6

000.60.400.3 0.5 0.1 00 0.5 0.5 00.30000

0.70.500.6000000

000.200.2 … (7)


Only one fixed idiotope matrix Id is used. It means that the idiotope matrix is not permitted to change throughout the period of program execution. This matrix (Id) is hand-coded according to disallowance of antibody–antigen relations as shown in (7). The values of these elements are in the range of 0.00–1.00 and the sum of elements for each antibody (across all antigens) is set to 1.00. This is useful for any antibody not becoming over stimulated or over suppressed.

The initial paratope matrix Pa and fixed idiotope matrix Id are both imported from files, and the robot get knowledge from the environmental situations in a continuous loop. Multiple antigens may present themselves simultaneously, but one of them is dominant according to their priorities. An antigen array AG is formed having the elements equal to number of antigens, according to the environmental situation. The values of each element, has a value of ‘0’ for non-presenting antigens, a value of ‘2’ for a dominant antigen with P [ , ] > 0 and a value of ‘0.25’ for all other presenting antigens. So that the dominant antigen receives a greater scores for all antibodies with positive Pa[ , ].

An antibody is detected to have its action execution in response to the presenting antigens. The selection of fittest antibody is of a three stage process is nothing but the selection of the antigenic antibody by computing using (3), where the matching function is defined by (8) as follows. Here, Pa is the paratope matrix, and AG is the antigen array. , . … . 8

5 Results and Discussion

It is possible to produce the antigenic array from the environmental situation. By considering the initial paratope matrix (Pa) and idiotope matrix (Id) as mentioned in (9) and (10) we can obtain the stimulation effects among the antigens and antibodies. The sum S1 is used to detect the antigenic antibody and this term is no sense of AIS. Once the antigenic antibody obtained it will perform its action.

Table.4 illustrates the numerical results of , antigen-antibody stimulation effect by considering the idiotope matrix from (7) and the example of initial paratope matrix given in (6).

It is observed from the Table5 that for various environmental conditions different fittest antibodies has been selected. The bold terms in the rows S1 (antigen-antibody stimulation effect).


Table 4 Stimulation calculations with suitable examples

Antigens Present Antigenic Array S1(Ag-Ab Stimulation)

Object front; object right; goal known

0.25, 0, 2, 0.25, 0, 0, 0

0.6 0.67 1.16

1.4250.85

0.675 1.81

0.325 0.625 1.525


Object left; goal known; object not present (in front)

0, 2, 0, 0.25, 2, 0, 0

2.15 2.05 1.36 0.65 1.05

0.675 0.663 2.51

1.875 1.113


Object front; object left; goal known

0.25, 0, 2, 0.25, 0, 0, 0

0.6 0.675 1.1850.62 0.85

0.675 0.61

1.525 1.825 0.32

Table 5 Selection of fittest antibody actions for various environmental situations

S.N Antigens Present Antigenic

Array Antibody

Action

1 object not present;

goal known 0, 0, 0, 0.25,

2, 0, 0 1

Continue in its path

2 Object front; goal

known 2, 0, 0, 0.25,

0, 0, 0 3

900 Robot motion

3 Object front &

left; robot stalled behind & in front

0.25, 0.25, 0, 0, 0, 2, 2

4 Robot reverse

by 00

4 Robot stalled in

front; goal known 0, 0, 0, 0.25,

0, 0, 2 6

Robot reverse by 900


Table 5 (continued)

5 Object front;

object right; goal known

0.25, 0, 2, 0.25, 0, 0, 0

7 00 Robot

motion(right)

6 Object left; goal

known; object not present (in front)

0, 2, 0, 0.25, 2, 0, 0

8 450 Robot

motion(right)

7 Object front;

object left; goal known

0.25, 2, 0, 0.25, 0, 0, 0

9 00 Robot

motion(left)

8 Object right; goal known; object not present (in front)

0, 0, 2, 0.25, 2, 0, 0

10 450 Robot

motion(left)

Once the fittest antibody has been obtained, it performs its suitable action. From Table.5 we can notice the various environmental situations, selection of fittest antibody and its action for the current situation.

5.1 Reinforcement Learning

Since the system is working by only global strength in order to select the fittest antibody, it has no global feedback from the network. These values for the other antibodies are not used to adjust the paratope elements. So the new system has been introduced to represents a true AIS, because feedback from the network is global through change in all antibody concentrations using (5).Fig.3 shows the flow chart for the robot navigation using full AIS technique combined with reinforcement learning algorithm.

RL happens when knowledge is indirectly coded in a scalar reward or penalty function. Here, this method is introduced for dynamic approximation of the degree of match between antibodies (actions) and antigens (environmental situations).

Note that the absolute values are somewhat arbitrary and Table.6 represents the some of the RL scores according to the change in environmental situations. Initially RL value is set to zero when the antibody is starting its action. However, if , becomes negative as a result of this, it is set to zero. The algorithm is summarized by , , 0 , 1


Fig. 3 Flow chart for mobile robot navigation using full AIS-RL technique

After reinforcement learning applied to the previous system the final paratope matrix is shown in (9) and how the robot reached to its target from its source position by avoiding obstacles we can observe in Fig.4. various colours indicates various antibody activations.

Yes No

Define environmental conditions as antigens and robot actions as antibodies

Obtain Antigenic Array based on the environmental conditions

Generate random Paratope Matrix and Fixed Idiotope Matrix

Deduce Antigenic Antibody

Calculate Concentration Rate of all antibodies

Calculate New Concentration of all antibodies

Perform fittest antibody action from the obtained antibody concentrations

If Goal

Stop

Apply Reinforcement

Learning to paratope elements

Perform fittest antibody action from the obtained antibody concentrations

Immunised Navigational Controller for Mobile Robot Navigation 181 0.2 0.2 0.20.1 0.25 0.250.43 . 0.6 . 0.1 0.10.6 0.15 0.150.4 0.2 0.3 0.30.7 0.2 0.60.8 0.3 0.30.80.60.050.60.05

0.20.20.70.80.10.20.10.20.7

0.20 0.1 0.6 0.80.2 0.2 0.1 0.80.30.250.450.30.450.1.0.50.10.4

0.30.30.60.30.610.350.10.350.1

… 9

Table 6 Reinforcement values for various situations

Old antigen

New antigen RL Score Description

1-3

1-3 0.02(reward) Object present to object present

situation

4,5 -0.02(penalty) Object present to object not

present situation

6,7 0.03(reward) Object present to robot trapping

situation

4,5

1-3 0.03(reward) Object not present to object

present situation

4,5 -0.01(penalty) Object not present to object not present situation

6,7 0.04(reward) Object not present to robot

trapping situation

6,7

1-3 0.01(reward) robot trapping to object present

situation

4,5 -0.03(penalty) robot trapping to object not

present situation

6,7 0.04(reward) robot trapping to robot trapping

situation

Fig. 4 Robot motion from its source position to goal position


6 Conclusion

An immune based computational method is developed to solve a mobile-robot navigational problem. The system architecture enabled a simulated mobile robot to navigate in its environment. The main goal was to observe the communication and cooperation between robot and its environmental situations for a common target using immune system. Simulation results showed that the mobile robot is capable of avoiding stationary obstacles, escaping traps, and reaching the goal efficiently and effectively. In future, this system architecture can be applied to a real robot.

References

[1] Hunt, J.E., Cooke, D.E.: Learning Using an Artificial Immune System. Journal of Network and Computer Applications, 189–212 (1996)

[2] Zhu, Y., Tang, Z., Dai, H., Gao, S.: Cooperation Artificial Immune System with Application to Traveling Salesman Problem. ICIC Express Letters 2, 143–148 (2008)

[3] Dasgupta, D.: Artificial neural networks and artificial immune systems: similarities and differences. In: IEEE International Conference on Systems, Man, and Cybernetics, Orlando, pp. 873–878 (1997)

[4] Mamady, D., Tan, G., Toure, M.L.: An Artificial Immune System Based Multi- Agent Model and its Application to Robot Cooperation Problem. In: Proceedings of the 7th World Congress on Intelligent Control and Automation, pp. 3033–3039 (2008)

[5] Luh, G.-C., Liu, W.-W.: Dynamic Mobile Robot Navigation Using Potential Field Based Immune Network. Systemics, Cybernetics and Informatics 5, 43–50 (2007)

[6] Luh, G.-C., Liu, W.-W.: An Immunological Approach to Mobile Robot Navigation. Mobile Robots Motion Planning & New Challenges, 291–318 (2008)

[7] Luh, G.-C., Liu, W.-W.: Reactive Immune Network Based Mobile Robot Navigation. In: Nicosia, G., Cutello, V., Bentley, P.J., Timmis, J. (eds.) ICARIS 2004. LNCS, vol. 3239, pp. 119–132. Springer, Heidelberg (2004)

[8] Singh, C.T., Nair, S.B.: An Artificial Immune System for a Multi Agent Robotics System. World Academy of Science, Engineering and Technology, 6–9 (2005)

[9] Vargas, P.A., de Castro, L.N., Michelan, R., Von Zuben, F.J.: An Immune Learning Classifier Network for Autonomous Navigation. In: Timmis, J., Bentley, P.J., Hart, E. (eds.) ICARIS 2003. LNCS, vol. 2787, pp. 69–80. Springer, Heidelberg (2003)

[10] Neal, M., Labrosse, F.: Rotation-invariant appearance based maps for robot navigation using an artificial immune network algorithm. Congress on Evolutionary Computation, 863–870 (2004)

[studies in computational intelligence] soft computing techniques in vision science volume 395 ||...

Documents