an adaptive fuzzy expert system to evaluate human visual performance

Fuzzy Sets and Systems 142 (2004) 321–334www.elsevier.com/locate/fss

An adaptive fuzzy expert system to evaluate humanvisual performance

Jose A. Padilla-Medina, Francisco J. Sanchez-Marin∗

Centro de Investigaciones en Optica, A.C., Apdo. Postal 1-948, Le�on, Gto. 37000, Mexico

Received 19 September 2002; received in revised form 28 January 2003; accepted 18 March 2003

Abstract

In this work, we present a fuzzy expert system to e/ciently evaluate the performance of human observersin a visual detection task. With this system, based on fuzzy logic and the receiver operating characteristictheory, we reduced the duration of visual perception psychophysical experiments, which normally takes toolong, to approximately one fourth. We used as input variables the partial performance indexes obtained alongeach experiment together with the corresponding increments of them. The output variable is the energy of thesignal to be detected, which is adjusted adaptively by our system so that the observer reports approximately80% of correct responses. With shorter adaptive experiments, we eliminate the e5ects of fatigue and loss ofattention that limit considerably the applicability of the related theories and techniques. We also show thatour results are equivalent to those obtained through long-lasting traditional psychophysical experiments.c© 2003 Elsevier B.V. All rights reserved.

Keywords: Fuzzy expert system; Fuzzy control; ROC theory; Performance index; Adaptive experiments

1. Introduction

For more than 30 years, the Receiver operating characteristic (ROC) method [10] has been ap-plied to evaluate the performance of human observers in visual detection and discrimination tasks[25,26,29]. It has also been useful to quantitatively assess image quality [1,5] and to evaluate imagingsystems [18].

In the ROC theory, two kinds of experiments are used: the yes–no procedure and the ratingprocedure. The rating procedure is preferred because it is signiAcantly more e/cient [8]. Withthis procedure, the experimenter is able to obtain several points of the ROC curve with a single

∗ Corresponding author. Centro de Investigaciones en Optica, Lomas del Campestre, Loma del Bosque 115, Apdo. Postal1-948, LCeon, Guanajuato 37150, Mexico. Tel.: +52-477-73-10-17; fax: +52-477-17-50-00.

E-mail addresses: [email protected] (J.A. Padilla-Medina), [email protected] (F.J. Sanchez-Marin).

0165-0114/$ - see front matter c© 2003 Elsevier B.V. All rights reserved.doi:10.1016/S0165-0114(03)00004-6

mailto:[email protected]

mailto:[email protected]

322 J.A. Padilla-Medina, F.J. Sanchez-Marin / Fuzzy Sets and Systems 142 (2004) 321–334

experiment, while with the yes–no procedure each experiment yields a single point of the ROC curve;at least three experimental points are required. However, for this type of experiment a large numberof presentations are required. For non-experienced observers, and even for experienced ones, it isnot easy to properly use the rating categories during long periods of time, as implied in the ratingprocedure. These two inconveniences considerably increase the duration of this type of experimentsin such a way that fatigue and loss of attention a5ect the observer’s performance and, consequently,the obtained results. This, in turn, limits the use of the rating procedure. To solve these problems,some adaptive solutions have been suggested [11,16,23,27,32], none of which have made use offuzzy logic. In this work we present a fuzzy expert system (FES), whose architecture is that of afuzzy logic controller (FLC) [9,30]. This system was developed to overcome these inconveniences.With the FES proposed in this work, we reduced the duration of visual detection experiments toapproximately one fourth of the duration of a typical experiment of this kind. In addition, the obtainedresults are equivalent to those obtained through long and fatiguing experiments. The inputs of ourFES are indexes of performance obtained along the experiments and the corresponding incrementsof such indexes. The output of our FES is the “controlled” value of the signal energy, which isadaptively changed in terms of the performance of the observer in turn.

2. Theory

2.1. ROC theory

ROC theory has been described very well by di5erent authors [4,7,10,18,25]. In this section, webrieMy review some of its main concepts.

ROC theory has been widely used to determine indexes of detectability and discrimination ofsignals to measure the performance of human observers [6,3,29,31], to determinate image quality[1,5], to evaluate diagnostic systems [26], and to investigate sources of human inconsistency [6].

The term ROC is also used to denote a curve whose overall location corresponds to a particulardegree of detection (or performance) that is independent of the variations of the observers decisioncriteria. The ROC curve is uncontaminated by processes such as expectation and motivation that a5ectthe responses of human observers [26,29]. With this theory, the performance of human observersis evaluated quantitatively through the index d′. To calculate such performance (or detectability)index, estimations of the probabilities of hits and false alarms, which are obtained experimentally,are required. The equation to calculate the values of the performance index for each empirical pointis

Pc = 1 −∫ ∞

−∞�(x)�(x − d)G(x) dx; (1)

where Pc is the percent of correct responses (i.e. the estimation of the probability of hits), d isequal to d′ divided by the square root of 2, and G(x) is the probability distribution function for thedecision variable to take the x value:

G(x) =1√2�

exp(−x2

2

)(2)

J.A. Padilla-Medina, F.J. Sanchez-Marin / Fuzzy Sets and Systems 142 (2004) 321–334 323

and

�(x) =∫ x

−∞G(x) dx: (3)

To obtain the experimental values of d′ we numerically evaluated Eq. (1).Perception experiments can be done under the yes–no paradigm or using the rating procedure. In

a yes–no experiment, observers use a single-decision criterion during an entire experimental session.The observers task is simply to indicate whether a signal is present in each trial of the experiment.As mentioned above, each yes–no experiment yields a single point of the ROC curve. In a typicalstudy, at least three experimental points are required. On the other hand, in a rating experiment,observers must use at least three di5erent decision criteria during a single session. In this way, atleast two experimental points are obtained from each experiment. We did rating experiments thatimplied four rating categories: (1) signal deAnitely not present, (2) signal probably not present,(3) signal probably present, and (4) signal deAnitely present. This way we obtain three experimentalpoints from each experiment. Taking into account that a yes–no experiment takes about the sameamount of time as a rating experiment, it is easy to see that the rating procedure is much moree/cient than the yes–no procedure. However, most people And it di/cult to alter their decisioncriteria during a long experiment, so that researchers prefer to do their perception experiments withexperienced observers. This, as can be seen, considerably reduces the applicability of the ratingprocedure.

2.2. Fuzzy controllers

Fuzzy controllers (FC) use fuzzy rules of thought to manage the particular uncertainties andvagueness of each application [12,14,24]. The linguistic rules expressed by a human controller canthen be easily merged into a rule base. Finally, a fuzzy rule-based representation of the globalcontrol strategy is obtained, integrating the behavior and the ability of a human expert [9,21]. Inother words, the synthesis of a fuzzy controller may be viewed as an e/cient way to convert amathematical relationship into a knowledge-based model.

Gradually, FC have substituted conventional engineering systems; especially control and surveysystems [12,13,15,20]. This is basically due to the ease with which this kind of system is imple-mented and to the satisfactory results that have been obtained consistently. The manipulation of the“membership functions” constitutes the foundation of every FC. Fuzzy sets, which are very useful todesign FC, were developed by Lofti Zadeh [33], as a simple new way to represent mathematicallythe vagueness associated to di5erent situations of our daily life. They are also useful to representthe vagueness of a good number of research areas [2,13,33]. Fuzzy logic, using fuzzy data, repre-sents imprecise information with computational models that have the ability to recognize, represent,manipulate, and interpret fuzzy and statistical uncertainty [2,13,33].

The FC performs a mapping from U ⊂Rn to R. We assume that U =U1x : : : xUn, where Ui ⊂R;i=1; 2; : : : ; n. The fuzzy rule base consists of a collection of fuzzy IF–THEN rules [9]:

R(l): IF x1 is Fl1 and · · · xn is Fln; THEN y is Gl; (4)

where x=(x1; : : : ; xn)T ∈U and y∈R are the input and output of the FC, respectively, Fli and Gl

are labels of fuzzy sets in Ui and R, respectively, and l=1; 2; : : : ; M . Each fuzzy IF–THEN rule


Fig. 1. Our adaptive fuzzy system and its sets of fuzzyAnd rules.

of Eq. (4) deAnes a fuzzy implication Fl1x · · · xFln → Gl, which is a fuzzy set deAned in a productspace UxR. Based on generalizations of implications in multivalue logic, many fuzzy implicationrules have been proposed in the fuzzy logic literature, but used more commonly in the design ofFC is the Min-operation rule of fuzzy implication:

�Fl1x···xFln→Gl(x; y) = min��Fl1x···xFln(x); �Gl(y): (5)

In the last step of the design process of an FC, the designer must select a suitable defuzziAcationmethod. The purpose of defuzziAcation is to convert each conclusion obtained by the inferenceengine, that is expressed in terms of a fuzzy set, to a single real number. The resulting number,which deAnes the action taken by the fuzzy controller, is not arbitrary. This defuzziAcation methodis deAned as

F(x) =

∫ ∞−∞ yb(y) dy∫ ∞−∞ b(y) dy

=

∑mj=1 wjaj(x)Vjcj∑mj=1 wjaj(x)Vj

: (6)

We can see in Eq. (6) that the output of the system is a function of the volumes Vj (or areas), thecentroids cj of the fuzzy sets, and the relative rule weights wj.

In a previous publication, the authors of this work used fuzzy logic to facilitate the evaluation ofhuman observers in a visual detection task [19]. However, as far as we know, prior to this work, noone has developed an adaptive fuzzy system to evaluate directly the performance of human observes.

In this work, we adaptively combined two input variables (the performance index d′ and itsincrement) in a fuzzy system (Fig. 1) to obtain the new value of the corresponding signal energy.To calculate the input variable Qd′, we subtracted the current value of d′ from its previous value.

The fuzzy IF–THEN rules of our system incorporate the knowledge of expert experimenters whohave intensively applied ROC theory in visual perception experiments. These rules were formulatedby taking into account d′ and Qd′, which are important indicators of the observers performanceduring the experiments. Also, we took into account that changes in signal energy should not beso abrupt as to confuse the observers and if the observers performance index had increased or


decreased. This way, if after 20 consecutive presentations, an observer obtains a very low (VL)performance index (d′) and a very high (VH) negative increment (Qd′), which indicates that hisperformance level has signiAcantly decreased with respect to the result of the previous short exper-iment (i.e. the previous 20 presentations), it is necessary to increase the signal energy (E). Thatis: If d′ =VL and Qd′ =NH then E=VH. On the other hand, if the observer shows a very lowperformance (VL), but Qd′ is high and positive (PH) (which indicates that his performance levelhas signiAcantly increased with respect to the previous short experiment), it is necessary to increase“slightly” the signal energy (E). That is: If d′ =VL and Qd′ =PH then E=H .

3. Methods and procedures

We tested our fuzzy system with experimental data derived from Ave human observers whoparticipated in visual perception experiments using the ROC rating procedure with four categories[7,10,17]. This procedure yielded three experimental points of the ROC curve for each observer.Prior to testing our FES, we had to tune it. This required only three perception experiments. Thevariables that we adjusted during the tuning process were the starting signal energy and the rangeof variation of the partial outputs of our FES (i.e. the range of the signal energy, through the rulesof our fuzzy system).

3.1. The test images

The test images were 8-bit gray level of 384× 384 pixels. Noise-only and signal-plus-noise testimages used in our experiments were computer generated. The noise-only images contained Gaussiannoise with a mean gray level of 128 and a standard deviation of 32 gray levels. To minimize learninge5ects, the Gaussian noise of each image was unique. The signal-plus-noise images contained, inaddition to Gaussian noise, a target which was a circle of 64 pixels in diameter and whose energywas varied adaptively according to the performance of the observer in turn. This target was addedto the noise at the center of each image.

To determine an appropriate signal level for detection, we completed 12 yes–no and six ratingdetection experiments with di5erent signal energies. Only experienced observers took part in thesepreliminary experiments.

3.2. Experimental design

We started with a signal energy equivalent to 4% of the energy of a totally white circle (i.e.with a gray level of 255). The energy of the target was, increased or decreased automatically,in accordance with the observers performance until the experiments yielded approximately 80% ofcorrect responses.

The computer program that displayed the images and recorded the responses of the observerswas designed in so that, for each 20 presentations, a partial performance index (d′) was calcu-lated, as well as the corresponding increment (or decrement) of such partial index. The programstops the experiments when 200 presentations have taken place or when three consecutive valuesof the performance index are in the range of 1.3–1.8 (i.e. from 74% to 82% of correct responses).The sequence presented in a session consisted of 200 images that, in turn, consisted of signal plus


noise and noise alone. These images were presented randomly interspersed with the same probabilityof presentation.

The experiments were done in a completely darkened room with black walls. The observers werelocated at a distance of 1m form a standard computer monitor that displayed the test images so thatthe target subtended an angle of 1:146◦. The duration of a typical experiment was about 15 min.

The observers responses were recorded and controlled by the same computer that was used todisplay the test images. At the beginning of each presentation, only the test image appeared onthe screen of the computer monitor. When the observers were ready to give a response, with aclick of the mouse, a menu bar appeared on the screen showing the four possible alternatives:(1) signal deAnitively present, (2) signal probably present, (3) signal probably not present, and(4) signal deAnitively not present. The observers were instructed to choose the option that bestdescribed what they perceived. When a correct response was given, the computer emitted a tone toreward the observer. Subjects were given an unlimited time to respond.

Short familiarization sessions were done prior to each experiment, during which we presentednoise-only images and signal-plus-noise images with di5erent signal-to-noise ratios so that, in someimages, the observers could clearly see the signal. At the end of the familiarization session, asequence of 60 test images, similar to the ones used in the deAnitive experiments, was presented toeach observer to simulate a short experiment.

With regard to why 80% correct responses are important for an adaptive method, as Hall [11]mentioned, in psychophysics the experimenter has to present stimuli at levels (i.e. intensities) of“interest” that contribute to an estimation of a threshold. That is, if the signal energy is too highthe observer responds correctly in all trials, and if it is too low, the observer responds at random.In these extremes, the experimenter cannot learn the subjects detection (or discrimination) thresholdand the associated decision process.

3.3. The human observers

Most of the observers that participated in our experiments were male and female graduate studentswith a visual acuity of 20/20, natural or corrected, as tested with a Snellen chart. This was the onlyrestriction for participating in the experiments. Except for one of the authors, all participants werenaRSve to the hypothesis of this work.

3.4. Input and output variables

As mentioned above, we used as input to our system the values of the performance indexd′i (i=1; 2; 3; : : : ; 9) for each 20 presentations, and the corresponding increment of this index. These

two variables contain su/cient information about the perception capabilities of the observers. Withthis information, our FES can control the signal energy at levels that are appropriate for each ob-server. Obviously, our system, does not yield predetermined values of signal energy, but it determinesthe appropriate signal energy levels in terms of what the observer has done during the experimentto obtain approximately 80% of correct responses. With our system (see Fig. 1) we combined theindexes of detectability and their corresponding increments with IF–THEN fuzzyAnd rules to de-termine more e/ciently the performance of the observers. The output of our system is the signalenergy values that will yield eventually the observer in turn to obtain approximately 80% of correct


responses. Our Anal output is a set of values of d′ together with the corresponding values of signalenergy that gave place to them. These sets of values indicate how well each observer has performed.

After identifying relevant input and output variables, as well as the ranges of their values, itwas necessary to select meaningful linguistic states for each variable and express them throughappropriate fuzzy sets. We only needed four linguistic states for the input variables and Ave for theoutput variables, to properly describe them. The following states were selected for each variable:(Input)

Qd′: NH NL PL PH

(NH = negative high, NL = negative low, PL = positive low, PH = positive high),

d′: VL L M H

(VL=very low, L= low, M=medium, H=high).(Output)

E: VL L M H VH

(VL=very low, L= low, M=medium, H=high, VH=very high).The fuzzy quantization of all variables was carried out by symmetric triangular-shaped fuzzy

numbers.

3.5. The inference engine

The knowledge included in our system was formulated in terms of a set of fuzzyAnd inferencerules of the type IF–THEN. To better deAne the knowledge used in our system, the responsesof three observers were analyzed. This analysis helped us to optimally deAne (a) the set of rules,(b) The fuzzy sets of the variables, and (c) the proper range of each input and output variable. Anexample of a fuzzy rule used in our system is: if Qd′ is NH and d′ is M then E is L. Each fuzzyAndrule was evaluated using the minimum. The purpose of the inference engine was to combine the inputvariables with fuzzy information rules to deduce the signal energy changes every 20 presentations insuch a way that the magnitudes of those changes were in terms of the performance of the observer,through Qd′, during the previous 20 presentations.Finally, the fuzzy system transformed each conclusion of the inference engine to non-fuzzy entities

using the centroid method.

4. Results

For training our FES three of our observers did perception experiments. When we analyzed theresults of these experiments we noticed that our FES-generated (as partial outputs) signal energylevels were below the detection threshold of the three observes. Just after the occurrence of such lowsignal energy levels the observers lost control of their decision criteria during the remaining of theexperiment. After analyzing the results of these experiments, we changed the range of possible valuesfor the signal energy (i.e. the range of the output of our FES). We also designed the knowledge


Fig. 2. Fuzzy sets for the input variables Qd′ and d′.

base, using IF–THEN FuzzyAnd rules, with an evaluation criterion that avoided the inclusion ofsignal energies below the detection threshold.

After the training stage, our FES included 16 FuzzyAnd rules, formulated by the authors, withtwo input variables (d′ and Qd′) and a single output variable (E), with fuzzy membership functionsas shown in Figs. 2 and 3.In Fig. 4, it can be seen how the output of our system changes as a function of the two input

variables. As mentioned above, the goal of our system was to generate changes of signal energy insuch a way that the observer could obtain from 74% to 82% of correct responses. Fig. 5 shows thesignal energy variations, and the variations of the performance index (d′), for two of the participantobservers. In this Agure, it is important to note that the variations of the signal energy during the Arstpresentations were signiAcantly larger than the variations that occurred by the end of the experiments.In addition, in the same Agure, one is able see that our FES searches the appropriate signal energylevel for the observer to obtain from 80% to 90% of correct responses (i.e. a d′ value from 1.68


Fig. 3. Fuzzy sets for the output variable E.

Fig. 4. Input–output relationship of the FES.

to 2.55). Observer 1 (very experienced and consistent) obtained a d′ greater than 2.55. This meansthat, within the permitted range of signal energy values, it is not possible to make this observer toobtain consistently from 80% to 90% of correct responses. This will be discussed below.

In order to investigate the appropriateness of the results obtained with our FES, in Fig. 6 wecompare the behavior of one of the participant observers along a traditional visual signal detectionexperiment, with a given constant signal energy level, with his behavior in an experiment controlledby our FES. We analyzed the data of the traditional experiment every 20 trials. As can be seen in thisAgure, the behavior of the observer in both experiments is similar. In both cases, the performanceoscillations of this observer, around a given performance index, occur. The variations of d’ inthe experiments with Axed signal energy were expected to be much smaller than those in the adaptiveexperiments, but it was not the case. In Fig. 7, it is shown how d′ varied in a long traditionalexperiment, with 600 presentations in a single session with an experienced observer. The e5ectsof fatigue and lost of attention are notorious by the middle of the experiment. It is true that large


Fig. 5. The index of performance d′ versus the respective signal energy E for two observers.

variations occur in both directions (i.e. as increments and decrements) in such a way that their e5ectsmight cancel in the Anal result. However, there is no need to waste time and e5ort when this kindof experiments can be done in a considerably more e/cient way.


Fig. 6. Comparison of de the partial performance indexes d′i of a traditional experiments (with Axed signal energy) with

those of an adaptive experiment (with variable signal energy).

Fig. 7. Variation of the performance index (d′) every 20 trials in a long experiment with Axed signal energy and 600presentations.


Table 1Wilcoxon test for d′ obtained as usual and using our fuzzy system

Observer Data S+ S� (�=0:049) H0 : �= �0

1 �=3:049, �0 = 2:802 32 47 Accepted2 �=2:616, �0 = 1:914 43 47 Accepted3 �=2:390, �0 = 2:136 22 47 Accepted4 �=3:390, �0 = 3:202 28 47 Accepted

S+ is the test statistics obtained from data, and S� is the signed rank statistic for �=0:49.

For the adaptive approach, using our FES, we propose as the Anal index of performance, theaverage of the performance indexes (d′

i ; i=1; : : : ; 9) as is supported below.To further compare the results obtained using our FES with those obtained with traditional ex-

periments, we applied the Wilcoxon, non-parametric test [28] to our data. Our null hypothesis wasH0: �= �0, where � was the average of the performance indexes d′

i (i=1; : : : ; 9) along the adap-tive experiment and �0 is the performance index obtained by the same observer in a traditionalexperiment of 600 trials with a Axed signal energy close to the detection threshold. The alternativehypothesis was Ha: � �= �0. According to this test, for all of our observers (see Table 1), the averageindex of performance obtained in short adaptive experiments is very similar to that obtained withlong traditional experiments. However, in every case, the length of the adaptive experiments wasapproximately one fourth that of the traditional experiments of 600 presentations.

5. Discussion

In psychophysics the duration of an experiment is an important issue. In order to reduce thevariance in perception experiments, it has been seen as the only way to design experiments thatinclude a number of trials as large as possible [3]. However, researchers know that fatigue and lossof attention a5ect the results of experiments of long duration in such a way that limit the applicabilityof the related theories and techniques. The typical duration of a signal detection experiment, suchas those we did with 600 presentations, is about 1 h. With our adaptive approach, the experimentslasted around 15 min and, according to the non-parametric Wilcoxon test, the results of both typesof experiments are equivalent.

Traditionally, the oscillations of performance that occur, even those with Axed signal energies ineach experiment, are not taken into account. However, such oscillations can contain useful infor-mation about the observer’s perception capabilities. With our approach, it is easier to access thatinformation, as we did with the performance index that we proposed (i.e. the average of the perfor-mance indexes calculated every 20 presentations along the experiments). That is, with our approach,using fuzzy logic, it is possible to obtain better evaluations, implying the performance developed bythe observers all along the experiments.

Our FES was developed to be used with experienced and non-experienced observers. However,in general, non-experienced observers require signal energy levels higher than those required byexperienced observers. When we had to deAne the range of “possible” signal energies produced byour FES, we decided to use a relatively high minimum energy. This is the reason why, when an


experienced observer was tested (Fig. 5), our Fuzzy System could not infer a signal energy thatwould make this observer to obtain about 80% of correct responses. However, we could evaluatethis observer properly with a short adaptive experiment.

Control systems are normally applied to situations that involve physical variables that are a5ectedby a limited number of factors. In the case of human observers in a visual detection task, many factorscan play an important role in their performance (fatigue, thirst, hunger, health status, emotions, etc.).In spite of this, our adaptive fuzzy system was able to control our experiments in such a way that,with short adaptive experiments, we could obtain evaluations that can be better than those obtainedwith long traditional experiments.

Finally, one of the authors of this work had the opportunity to use the adaptive method proposedby Xue et al. [32], in the same laboratory where it was developed. This author had to use simpleforced choice experiments [22], because Xue’s method yielded contradictory results when applied todi5erent observers. On the contrary, our conclusions were always consistent when we compared theresults of our FES with those obtained with traditional experiments of long duration.

Acknowledgements

This work was supported, in part, by the Consejo de Ciencia y TecnologCSa del Estado deGuanajuato (CONCYTEG).

References

[1] H.H. Barrett, Objective assessment of image quality: e5ects of quantum noise and object variability, J. Opt. Soc.Amer. 7 (1990) 1266–1277.

[2] J. Bezdek, Fuzzy models–what are they and why? IEEE Trans. Fuzzy Systems 1 (1) (1993) 1–5.[3] A. Burgess, On sampling statistics in observer performance studies, science and engineering of medical imaging,

SPIE 1137 (1989) 190–197.[4] A. Burgess, Comparison of receiver operating characteristic and forced choice observer performance measurement

methods, Med. Phys. 22 (1995) 643–655.[5] A. Burgess, Image quality, the ideal observer, and human performance of radiologic decision tasks, Acad. Radiol. 2

(1995) 522–526.[6] A. Burgess, B. Colborne, Visual signal detection. IV. Observer inconsistency, J. Opt. Soc. Amer. A 5 (1988)

617–627.[7] J.P. Egan, Signal Detection Theory and ROC Analysis, Academic Press, New York, 1975.[8] J.P. Egan, A.I. Schulman, G.Z. Greenberg, Operating characteristics by binary decisions and by ratings, J. Acoust.

Soc. Amer. 31 (1959) 768–773.[9] S. Galichet, L. Fulloy, Fuzzy controllers: synthesis and equivalences, IEEE Trans. Fuzzy Systems 3 (1995)

140–148.[10] D.M. Green, J.A. Swets, Signal Detection Theory and Psychophysics, Wiley, New York, 1966.[11] J.L. Hall, Hybrid adaptive procedure for estimation of psychometric functions, J. Opt. Soc. Amer. 69 (1981)

1763–1769.[12] N. Infelise, A clear vision of fuzzy logic, Control Eng. 38 (9) (1993) 28–30.[13] A. Kandel, Fuzzy Expert Systems, CRC Press, Tampa Florida, 1991.[14] B. Kosko, Neural Networks and Fuzzy Systems, Prentice-Hall, Englewood Cli5s, NJ, USA, 1992.[15] B. Kosko, Fuzzy Engineering, Prentice-Hall, Englewood Cli5s, NJ, USA, 1997.[16] R. Madigan, D. Williams, Maximum likelihood psychometric procedures in two-alternative forced choice: evaluation

and recommendations, Percept. Psychophys. 42 (1987) 240–249.


[17] J. Markowitz, J.A. Swets, Factors a5ecting the slope of empirical ROC curves: comparison of binary and ratingresponses, Percept. Psychophys. 2 (1967) 91–100.

[18] C.E. Metz, ROC methodology in radiological imaging, Investigation Radiol. 21 (1986) 720–733.[19] J.A. Padilla-Medina, F.J. Sanchez-Marin, A fuzzy expert system for evaluating human observers in a visual detection

task, Fuzzy Sets and Systems 132 (2002) 389–400.[20] S. Russell, P. Norving, Inteligencia ArtiAcial: un Enfoque Moderno, Prentice Hall, MCexico, DF, 1996.[21] J.P. SCanchez, Y. Beltran, Los Sistemas Expertos: Una MetodologCSa de ProgramaciCon, Macrobit Corporation, MCexico,

1990.[22] F.J. Sanchez-Marin, D.L. Wilson, Y. Srinivas, K.N. Jabri, Quantitative image quality analysis of a non-linear

spatio-temporal Alter, IEEE Trans. Image Process. 10 (2) (2001) 288–295.[23] W.A. Simpson, The step method: a new adaptive psychophysical procedure, Percept. Psychophys. 45 (1989)

572–576.[24] M. Sugeno, A fuzzy logic based approach to qualitive modeling, IEEE Trans. Fuzzy Systems 1 (1) (1993) 7–31.[25] J.A. Swets, Signal Detection and Recognition by Human Observers, Wiley, New York, 1964.[26] J.A. Swets, Signal Detection Theory and ROC Analysis in Psychology and Diagnostics, Lawrence Erlbaum, NJ,

1996.[27] A.B. Watson, D.G. Pelli, QUEST: a Bayesian adaptive psychometric method, Percept. Psychophys. 33 (1983)

113–120.[28] G. Weibenrg, J. Schumaker, D. Oltman, Statistic: an Intuitive Approach, Wadsworth, Belmont, CA, USA, 1981.[29] D.J. Weintraub, H.W. Hake, Visual discrimination, an interpretation in terms of detectability theory, J. Opt. Soc.

Amer. 52 (1962) 1179–1184.[30] K.A. Wen, C.Y. Lu, M.C. Tsai, The transform image codec based on fuzzy control and human visual system, IEEE

Trans. Fuzzy Systems 3 (1995) 253–259.[31] C.D. Wickens, Engineering Psychology and Human Performance, Prentice-Hall, Englewood Cli5s, NJ, 2000.[32] P. Xue, C.W. Thomas, G.C. Gilmore, D.L. Wilson, An adaptive reference/test paradigm: application to pulsed

Muoroscopy perception, Behavior Res. Methods Instrum. Comput. 2 (1998) 332–348.[33] L. Zadeh, Fuzzy Sets, Information and Control, 3, vol. 8, Academic Press, New York, 1965, pp. 338–353.

an adaptive fuzzy expert system to evaluate human visual performance

Documents