review of methods and techniques on mind reading computer machine

STUDENT DETAILS Guide Details D.Madhavi Latha Mrs.M.Jyothi 13341A1210 Asst.Prof Information Technology Dept of IT (GMRIT)

REVIEW OF METHODS AND TECHNIQUES ON MIND READING COMPUTER MACHINE

ABSTRACT

Mind reading is a way to detect or infer other’s mental states. The simplest way for mind reading can be done by simply seeing and understanding the facial expression. For example, a smile can give us an expression of happiness. But now it may be possible that a computer might understand the mental states of a person. The key to this scheme is the electroencephalograph, a device used by medical researchers to pick up electrical currents from various parts of the brain. If we could learn to identify brain waves generated by specific thoughts or commands, we might be able to teach the same skill to a computer.

INTRODUCTION

oPeople express their mental states, including emotions, thoughts, and desires, all the time through facial expressions.

oOur mental states shape the decisions that we make, govern how we communicate with others, and affect our performance.

oTheory of mind or mind reading: The ability to attribute mental states to others from their behavior and to use that knowledge to guide our own actions and predict those of others.

oThe understanding of a human’s thoughts is one of the most complex tasks.

LITERATURE REVIEW

oFacial expression analysis includes both measurement of facial motion and recognition of expression. The automatic facial expression recognition problem is a very challenging problem.(1)

oThe proposed automatic facial expression recognition system can automatically detect human faces, extract facial features, and recognize facial expressions. Facial expressions are the facial changes in response to a person’s internal emotional states, intentions, or communications.(2)

oDuring tracking detailed parametric description of facial features are extracted. The face can express emotion sooner than people verbalize or even realize their feelings.(3)

WHAT IS MIND READING?

oThe team member of University of Cambridge had been working on a model of mind reading in a computer laboratory and has developed mind reading machines that implement a computational model of mind reading to infer mental states of human being from their facial expressions and head gestures.

oThe machine works by using digital video cameras which analyzes a person’s facial expression in real time and infer person’s underlying mental state such as thinking, confused, interested, bored, agreeing, disagreeing, happy, sad and angry.

WHY MIND READING?

oImagine a future, where we are surrounded with mobile phones, cars and online services that can read our minds and react to our moods.

oJust imagine how would that change our use of technology and our lives?

oThe mind-reading computer system may also be used to monitor and suggest improvements in human-human interaction.

oTo detect driver mental states such as drowsiness, distraction and anger.

HOW DOES IT WORK?

oFuturistic Head Band:

Different sensors are fixed all around the futuristic head band which measures the blood pressure ,impulses, temperature of brain, volume and oxygen level of the blood around the subjects of brain using technology called Functional Near-Infrared Spectroscopy (FNIRS) and sends this information to the computer which helps to find whether the person is in happy mood or in aggressive mood.

HOW DOES IT WORK? Continued…

It sends light in that spectrum into the tissues of the head, where it is absorbed by active, blood-filled tissues.

The headband measures how much light was not absorbed, letting the computer gauge the metabolic demands that the brain is making.

For example, a combination of a head nod, with a smile and eyebrows raised might mean interest.

oFacial Expressions analysis:The face is one of the most powerful channel of non-verbal

communication.

HOW DOES IT WORK? Continued…

Facial expression provides clues about emotions, intention, alertness, pain, regulates interpersonal behavior and communication.

Facial expression analysis includes both measurement of facial motion and recognition of expression.

Next step is to extract and represent the facial changes.In facial feature extraction, there are two ways: geometric

feature based and appearance based methods.The geometric feature presents the shape and location to

form a feature vector for facial components.

HOW DOES IT WORK? Continued…

In appearance- based method, image filters are applied either to whole face or specific regions in a face image to extract a feature vector.

Basic structure of facial expression analysis includes three things. They are Face Acquisition, Facial Data Extraction & Representation, Facial Expression Recognition.

HOW DOES IT WORK? Continued…

Facial expression recognition is based on technology of Emotient. Emotient is an image recognition technology. It allows faces, emotions, and other features to be recognized and identified in photos and images.

TECHNIQUES

oEmotional Classification:Statistical Method:

After reading the image, the image must be analysed for duplication.

So that correlation of the matrix will be find. Since the correlation matrix for each image is square,

we can calculate Eigen vector and Eigen value for each matrix.

These are very important so that it gives useful information about the data.

TECHNIQUES continued…

Eigen Vectors and Eigen Value :In Eigen vectors any vector change in magnitude but

not in direction. In Eigen values, the magnitude that the vector is

changed is called an Eigen value

Where A is n x n matrix. x is the length of n column vector. λ is a scalar it’s an Eigen value and X is the Eigen vector.

TECHNIQUES continued…

“The Eigen values for angry image 1:” 0.1369, 0.1371, 0.1372, 0.1373, 0.1371, 0.1368, 0.1366, 0.1375, 0.1382, 0.1285, 0.1394, 0.1402, 0.1406, 0.1408, 0.1412, 0.1417, 0.143”.

 “The Eigen values for angry image2:” 0.1368, 0.1371, 0.1374, and 0.1372.

Figure 1. Angry expression for different situations

TECHNIQUES continued…

Figure 2. Disgust expression in different situations

Figure 3. Happy expression in different situations

TECHNIQUES continued…

“The Eigen values for angry image3:” 0.1367, 0.1371, 0.1375 and 0.1371.

It is important to notice that these eigen vectors are both unit eigen vectors that is their lengths are both 1.

After getting the Eigen values we calculate the mean values for all the images, in order to get highest Eigen values.

The highest Eigen value must contain important feature about the data.

So we select ten highest Eigen values from all the images.

RESULTS CAN BE EVALUATED IN THE FOLLOWING WAY

oThe results are often compared to a Magnetic Resonance Imaging (MRI), but can be gathered with light weight, non invasive equipment.

oWearing the FNIRS sensor, experimental subjects were asked to count the number of squares on a rotating on screen cube and to perform other tasks.

RESULTS CAN BE EVALUATED IN THE FOLLOWING WAY continued…

o  Measuring mental workload, frustration and distraction is typically limited to qualitatively observing computer users.

oPreliminary results are shown using button sized sensors.o"Biological signals arise when reading or speaking to

oneself with or without actual lip or facial movement”.

ADVANTAGES & USES

oCan read minds.oHelp paralytic patients.oHelp Handicapped people. oHelp Comma patients. oHelp people who can’t speak oCan be used for mind gaming.o Eliminate the capability to lie.oTransmit visual images to the mind of a blind person,

allowing them to see.oIt helps injured astronauts to control their machines or

aid disabled people.

APPLICATIONS

oMind-controlled wheel chair:It is developed from the University of Electro-

Communications in Japan.

It is little different from the Brain-Computer Typing machine, this thing works by mapping brain waves when you think about moving left, right, forward or back, and then assigns that to a wheel chair.

This device allows people who can’t use other wheel chairs to get around easier & will be able to move their wheelchair with very less effort.

oBrain-Computer Typing machineBy thinking about left and right hand movement, user can

control the virtual keyboard.

CONCLUSION

Tufts University researchers have begun a three-year research project which, if successful, will allow computers to respond to the brain activity of the computer's user.

These computers can perform telepathy , by translating brain activity into word.

The mental state is recognized by comparing the present real time video with the preinstalled videos which contain different expressions for different mental state.

It represents one of the most complicated and interesting cognitive activities in which we routinely engage.

REFERENCES

oCohn J. F., Tian, Kanade, T., (2001)”Recognizing action units for facial expression analysis”, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 23, No. 2.

oJ. Lien, A. Zlochower, J.Cohn and T. Kanade. Automated facial expression recognition. In Proceedings of IEEE International Conference on Automatic Face and Gesture Recognition, 1998

oY.-L. Tian, T. Kanade, and J.Cohn. Recognizing action units for facial expression analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(2):97–115, February 2001.

i