eye, communicate, thoughts

8/8/2019 Eye, Communicate, Thoughts

1/12

1Eye, Communicate, Thoughts

Tarun Roy, Praxis Business School

Tarun Roy

Eye, Communicate, Thoughts

IntegratingSixthSense[1], prosthetics and

thought control.

The advent of the wireless communication era has brought about a change in themanner we perceive and access information. A wealth of information is available at our

fingertips; however this knowledge is still confined to the digital medium we access. The

SixthSense concept is the first step towards mitigating this digital confine and path breaking

innovations in bionics & robotics pave the way to the future, not only bridging the digital divide

but also in the backward integration of information access into the very essence of a human

form a sensory organ. Information at the blink of an eye.
http://www.pranavmistry.com/projects/sixthsense/http://www.pranavmistry.com/projects/sixthsense/http://www.pranavmistry.com/projects/sixthsense/http://www.pranavmistry.com/projects/sixthsense/


2/12



Praxis Business School

Eye, Communicate, ThoughtsIntegrating SixthSense

[1], prosthetics and thought control.

A research paper

Submitted to

Dr. Prithwis Mukerjee

In partial fulfilment of the requirements of the course

Business Information Systems

On 7th November 2010

By

Tarun Kanti Roy

B10037 (PGP 1)
http://www.pranavmistry.com/projects/sixthsense/http://www.pranavmistry.com/projects/sixthsense/http://www.pranavmistry.com/projects/sixthsense/


3/12



Contents

Integrating SixthSense, prosthetics and thought control.

The Elements

1. The Platform: SixthSense

2. Present: Sight Restoration and Prosthetics

3. Future: Thought Control

4. Further Still: Integrating SixthSense, Prosthetics and Thought Control

5. Beyond Fiction?: The Learning Curve

References

Image:Kts | Dreamstime.comhttp: //www.f uturetimeline.net/22ndcentury/ images/m ind-uploading.jpg
http://www.dreamstime.com/Kts_infohttp://www.dreamstime.com/http://www.dreamstime.com/http://www.dreamstime.com/Kts_info


4/12



THE ELEMENTS

Since the dawn of our time, human beings as aspecies have sought many things. The most

important of these desires however has beenprimarily the quest for knowledge. It is this driverwhich is responsible for the progress of the

civilizations and the advancement of science.

The human brain is a splendid tool, capable of an

enormous array of functions. One of these is theability to analyze problems and situations. However,its a known fact that the brain is an inadequate

container and more often than not, we find bits andpieces of data missing. And it is this absence ofdata that hinders. Can this problem be cured? And

if so, then is there really a need to remember data?

Can the brain be tuned to perform its analytics

provided that the availability of information isnot a constraint?

Information to the human body is fundamentally afunction of the receptivity of the five sensory organs,at the failure of which, the link or the pathway to this

receptivity is blocked. It seems though that it is onlyhuman for something so trivial to lose its capacity.For seamless accessibility to raw data which

essentially is electrochemical signals which are theninterpreted by the brain, we must build inredundancies into the human form which will be

capable of replacing the sensory organ andrestoring its primary function.

Can the prosthetic device be utilized to domore than just convert and relay data?

The magnitude of raw data available on the internet

for use to our benefit is only shadowed by theavailability of a node or an access point throughwhich the wealth of information may be accessed.

There has been a proliferation of portable devicesand mobile gadgets in the recent few decades. Thishowever does not help negate the divide that exists

between the digital world and the physical world.

Can the digital divide be mitigated for seamless

accessibility of desired information and data?

In an effort to answer these questions, at least in

theory if not in practice, let us look into the existingdomain of technology and medicine in order to figureout what is doable and what the future might hold.

We will need to assimilate three distinctive branchesof science in order to proceed. These are:

1. The Platform: SixthSense devices, aconcept developed by Pranav Mistry, MIT.

2. Present: Developments in prostheticsaiding in the restoration of sight.

3. Future: Thought control (outbound

transfer).

Eye, Communicate, Thoughts

Integrating SixthSense, prosthetics and thought control.

Image courtesy: Baudrillard and the Meaning of Meaning

http://my.opera.com/jessheartsben/blog/baudrillard-and-the-meaning-of -meaning


5/12



UNDERSTANDING THE TECNOLOGY'SixthSense' is a wearable gestural interface that augments the physical worldaround us with digital information and lets us use natural hand gestures tointeract with that information.

When we encounter something, someone or some place, we use our five naturalsenses to perceive information about it; that information helps us makedecisions and chose the right actions to take. But arguably the most usefulinformation that can help us make the right decision is not naturally perceivablewith our five senses, namely the data, information and knowledge that mankindhas accumulated about everything and which is increasingly all availab le online.Although the miniaturization of computing devices allows us to carry computersin our pockets, keeping us continually connected to the digital world, there is nolink between our digital devices and our interactions with the physical world.Information is confined traditionally on paper or digitally on a screen. SixthSensebridges this gap bringing intangible, digital information out into the tangibleworld, and allowing us to interact with this information via natural hand gestures.SixthSense frees information from its confines by seamlessly integrating it withreality, and thus making the entire world your computer.

The SixthSense prototype is comprised of a pocket projector, a mirror and acamera. The hardware components are coupled in a pendant like mobilewearable device. Both the projector and the camera are connected to the mobilecomputing device in the users pocket. The projector projects visual information

enabling surfaces, walls and physical objects around us to be used asinterfaces; while the camera recognizes and tracks user's hand gestures andphysical objects using computer-vision based techniques. The software programprocesses the video stream data captured by the camera and tracks thelocations of the colored markers (visual tracking fiducials) at the tip of the usersfingers using simple computer-vision techniques. The movements andarrangements of these fiducials are interpreted into gestures that act asinteraction instructions for the projected application interfaces. The maximumnumber of tracked fingers is only constrained by the number of unique fiducials,thus SixthSense also supports multi-touch and multi-user interaction.

The SixthSense prototype implements several applications that demonstrate theusefulness, viability and flexibility of the system. The map application lets theuser navigate a map displayed on a nearby surface using hand gestures, similar

to gestures supported by Multi-Touch based systems, letting the user zoom in,zoom out or pan using intuitive hand movements. The drawing application letsthe user draw on any surface by tracking the fingertip movements of the users index finger. SixthSense also recognizes users freehand gestures (postures).For example, the SixthSense system implements a gestural camera that takesphotos of the scene the user is looking at by detecting the framing gesture. Theuser can stop by any surface or wall and flick through the photos he/she hastaken. SixthSense also lets the user draw icons or symbols in the air using themovement of the index finger and recognizes those symbols as interactioninstructions. For example, drawing a magnifying glass symbol takes the user tothe map application or drawing an @ symbol lets the user check his mail. TheSixthSense system also augments physical objects the user is interacting withby projecting more information about these objects projected on them. Forexample, a newspaper can show live video news or dynamic information can beprovided on a regular piece of paper. The gesture of drawing a circle on the

users wrist projects an analog watch. Source: Official Website(http://www.pranavmistry.com/projects/sixthsense/)

1The Platform: SixthSenseA Wearable Gestural Interface: Integrating Information with the Real World.Pranav MistryFluid Interfaces Group, MIT Media Labwww.pranavmistry.com/projects/sixthsense/


6/12



2Present:Sight Restoration and Prostheticshttp://science.nasa.gov/science-news/science-at-nasa/2002/03jan_bioniceyes/

www.wired.com/gadgetlab/2010/04/australian-bionic-eye/

THE GIFT AND THE BLIND

Vision is a result of the incident light beingconverted into electrical impulses which are then

transmitted through the optic nerve to the brainwhere the image is formed. There are millions ofbiological solar cells which convert light into

electrical impulses without the presence of which,

we will be blind.

When these cells malfunction, it can cause

disorders like retinitis pigmentosa and maculardegeneration. Retinitis pigmentosa tends to behereditary and may strike at an early age, while

macular degeneration mostly affects the elderly.Together, these diseases afflict millions. Both occurgradually and can result in total blindness.

Scientists at the Space Vacuum Epitaxy Center(SVEC) in Houston are experimenting with thin,photosensitive ceramic films that respond to light

much as rods and cones do. Arrays of such films,they believe, could be implanted in human eyes torestore lost vision.

"There are some diseases where the sensors in the eye,the rods and cones, have deteriorated but all the wiring isstill in place, in such cases, thin-film ceramic sensorscould serve as substitutes for bad rods and cones. The

result would be a bionic eye. - Dr. Alex Ignatiev,Director, SVEC, Houston, Texas.

BIONIC EYE ATTEMPTS TO RESTORE

VISION

A bionic eye prototype developed by researchers inAustralia aims to implant an array of electrodes inthe eye that can deliver electrical impulses directly

to neurons in the retina.

The group called Bionic Vision Australia, hasdeveloped a device called the wide-view

neurostimulator for patients suffering from

degenerative vision loss.

Bionic Vision Australia uses an external camera

with resolution of up to 5 megapixels mounted on apair of glasses. An electrode array is implanted inthe eye and that connects to the central part of the

retina where the greatest number of retinal neuronsare present. An external unit has vision-processingsoftware to help generate the electrical impulses.

The communication between the electrode implant

and the external unit is wireless.

Image courtesy: Bionic Vision prototype/ BVAhttp://www.wired.com/images_blogs/gadgetlab/2010/04/bionic-eye-australia3-660x430.jpg

Image courtesy: NASA Science Newshttp://sc ience.nasa.gov/m edia/medialibrary /2002/01/14/03jan_bion

icey es_resources /retina_med.gif
http://science.nasa.gov/science-news/science-at-nasa/2002/03jan_bioniceyes/http://science.nasa.gov/science-news/science-at-nasa/2002/03jan_bioniceyes/


7/12



It is really designed to give people back their mobility

so they can move around their environment and avoid

obstacles. We are also working on a second-generation

product that will help people recognize faces and read

largeprint.- Anthony Burkitt, Research Director,

Bionic Vision Australia

The camera in itself doesnt need to be verypowerful because the quality of the image isnt the

crucial component but whats important is the

vision-processing software that picks up the

image and transforms it into electrical impulses.

Present:Sight Restoration and Prostheticswww.wired.com/gadgetlab/2010/04/australian-bionic-eye/

The resultant vision is not the same as the

images that a sighted person sees. Instead its a

pixelated version with a relatively small number

of dots: about 100 in early versions. The team is

also working on the next version of the bionic eye

that will include 1,000 electrodes, delivering 10

times the resolution. It will be made of platinum,

instead of the polycrystalline diamond used for

the first one, so more electrodes can be packed

in and better images generated. The company

hopes to do the first human implant in 2013.

Image courtesy: Bionic Vision prototype/ BVA

http: //www.wired.com/ images_blogs/gadg etlab/2010/04/bionic-ey e-australia.jpg


8/12



TELESCOPIC BIONIC EYE IMPLANT,

APPROVED BY FDA

The Food and Drug Administration has approved

a telescopic eye implant that can help elderly

people (over the age of 75) with macular

degeneration, a condition that results in a loss of

vision in the center of the visual field because ofretinal damage.

The procedure involves removing the lens of the

eye completely and replacing it with the now

FDA-approved implant, which is capable of

magnifying things by 2.2-2.7 times.

The device could help some patients suffering

from end-stage age-related macular

degeneration, a disease related to aging that is a

leading cause of vision loss for people over 60.

The implant telescope, created by VisionCare

Ophthalmic Technologies, projects images in the

field of view onto healthy areas of your central

retina outside of the degenerated macula. The

image is enlarged, which reduces the effect the

blind spot has on central vision.

Present:Sight Restoration and Prostheticswww.cbc.ca/health/story/2010/07/07/eye-telescope.html?ref=rsswww.tomsguide.com/us/FDA-Medicine-Medical-Technology-Vision-Care-cyborg,news-7401.html

The implant will be the natural movement of the

eye, however peripheral vision on that side will

be affected. It's for this reason that a patient can't

have the telescope implanted into both eyes.The

FDA also requires patients to undergo rehab to

get used to using their new bionic eye in tandem

with their untreated eye. There's also the

possibility that because of the size of the device,

patients may need a corneal transplant.

Clinical trials involving 219 patients resulted invision improvement (from severe or profound

impairment to moderate impairment) for 75

percent of the participants.

Image courtesy: VisionCare Opthalmic Technologies

http://www.cbc.ca/gfx/images/news/photos/2010/07/07/visioncare-telescope-croppe.jpg

Image courtesy: VisionCare Opthalmic Technologies

http://media.bestofmicro.com/telescopic-eye-implant,9-8-254348-13.jpg
http://www.cbc.ca/health/story/2010/07/07/eye-telescope.html?ref=rsshttp://www.cbc.ca/health/story/2010/07/07/eye-telescope.html?ref=rss


9/12



Future:Thought Control (Outbound Transfer)www.prnewswire.co.uk/cgi/news/release?id=34747

www.interaxon.ca3WHAT IS THOUGHT CONTROL?

Thought control is the process of harnessing ortapping the electrochemical signal which isproduced in the brain whenever there is any

brain activity thoughts with the help of animplant or external neuro-receptors and

transmitting this captured signal as an input to anelectro-mechanical device. For example, think ofswitching on the lights and the lights switch on.

The technology is in its latent stage and its

applications are very limited at the moment.

RECENT DEVELOPMENTS IN

THOUGHT CONTROL TECHNOLOGY

Scientists at the Emory University in Atlanta,

Georgia, US have built a "thought translation"machine that makes it possible to write words

onto a computer screen with the help of brainimplants that allowed paralyzed patients to movea computer screen cursor with the power ofthought.

Scientists from the University of Tubingen,Germany, and the University of Alabama in

Birmingham, Alabama, US, have pushed backthe frontier further by creating a system thatenables whole sentences to be written on acomputer by thoughts alone using electrodes

placed on top of the head rather than brainsurgical implants. Two contact lens-sized discs

recorded voltage-type signals called slowcortical potentials from the brain's motor cortex.

The researchers worked with three patients with

amyotrophic lateral sclerosis, aneurodegenerative disease that results in totalparalysis.

Using the system, i t took the patients an averageof 80 seconds to choose each character andthey were able to write short sentences in abouthalf an hour, the scientists reported in thejournal Experimental BrainResearch.

InteraXon, a Toronto-based company has

developed a system called thought-control

computing which is capable of using the brain

waves to control the environment around us, like

the lights in the home, controlling our household

products, dialing our mobile phones or even

using the toaster and it's exploring a range ofcommercial opportunities that include screens on

airplanes and video games.

The technology involves a headset embedded

with electrodes that read brain waves. The brain

waves are then processed on a computer.

Ariel Garten, CEO of InteraXon predicts the

headset will eventually become as small as a

wireless Bluetooth device and the technology will

be available in big stores in the US within the

next two years.

Image courtesy: InteraXon Logo

http://www.interaxon.ca/
http://www.prnewswire.co.uk/cgi/news/release?id=34747http://www.prnewswire.co.uk/cgi/news/release?id=34747


10/12



Further Still:Integrating SixthSense, Prosthetics andThought Control4

THE SUPREME EYE

Now that we have seen how the SixthSensetechnology interfaces the physical world around uswith the digital world and how bionic eye implants

restore sight to the visually impaired, the way

forward in my opinion would be to couple the twoconcepts. Further advancements in technology

would make it plausible to implant a miniaturecamera directly in the eye along with a chip whichwould have the vision processing software which

would be responsible for converting the images intoelectrical impulses to be understood by the opticnerve. This chip would also be wirelessly connected

to the internet.

The result would be, in t heory, not only natural sightfor the user and also artificial sensory input from the

digital world which would directly be seen andunderstood by the brain all form inside the eye.The user would never have to look at a computer

screen or any other physical interface in order to

access data.

Image courtesy: Digital Eye

http://cdn.leemunroe.com/wp-content/uploads/2008/02/f rog.jpg

While there might not be any problem in theaccessibility of information, there would be a

definitive hurdle in what data to access. Since theobjective is to eliminate any physical or electronichardware interface like a screen or a keyboard, the

means to an input as to what data to search on the

internet seems to be missing. This is where thethought control technology finds its application.

INFORMATION AT THE BLINK OF AN EYE

If we want to eliminate the object of data input in the

process of data access, the input has to come fromsomeplace else. As words and sentences canalready be articulated through control and with

subsequent developments in the technology, datainput for information access on the web through thebionic eye does not seem as farfetched as fiction.

The End Result:

The physical world is visually perceived as is ifnot better, even by those who have lost sight

Access to information will never be limited to a

computer, hand held device or any form ofphysical interface


11/12



Beyond Fiction:The Learning Curve5

A STEP BACK

Now that we have established, in theory, that theconceptualization of the idea of information accessthrough the bionic eye is only but a matter of time,

let us take a step back to ponder upon thestatement The mind is an inadequate container.

It is a researched fact that as we grow, from infancy

through childhood to adolescence, there arethousands of initially made neuro connectionswhich are broken while others are strengthened.

The ones that are strengthened are those that arefrequently used and the ones that are broken arethose that are rarely used. Hence, the highly

developed neuro network during the final phases ofinfancy and childhood loses a chunk of its potentialas age advances.

Can this pruning of unused neuro connections beavoided by keeping the connections active throughartificial stimuli?

THE LEARNING CURVE

If all the neuro connections can be kept alive during

the strengthening process, then the resultantformed neuro network would be significantly higherin potency of function and capacity.

Can this very basic shift of volume initiate a changein the human genome over the generations?

If this is possible, the proliferation of technology will,

for a change make us smarter and not lazier!

LOVE IT, HATE IT?

[email protected]

Image: Engaging the Digital Generation

http: //steve-wheeler.blogspot.com/2010/03/engaging-digital-generation.html

Image: Nerve Prolif eration and Pruning

http://groups.apu.edu/practicaltheo/LECTURE%20NOTES/YMIN/Y MIN%20201/Sp%2008/N erve%20Proliferat ion%20&%20Pruning.JPG


12/12


References:

THANK YOU!

1. The Road to pSingularity, Dr. Prithwis Mukerjee

2. SixthSense, Pranav Mistry,www.pranavmistry.com/projects/sixthsense/

3. http://science.nasa.gov/science-news/science-at-nasa/2002/03jan_bioniceyes/

4. http://www.wired.com/gadgetlab/2010/04/australian-bionic-eye/

5. http://www.bionicvision.org.au/

6. http://www.futuretimeline.net

7. http://www.cbc.ca/health/story/2010/07/07/eye-

telescope.html?ref=rss

8. http://www.tomsguide.com/us/FDA-Medicine-Medical-Technology-Vision-Care-cyborg,news-

7401.html

9. http://www.prnewswire.co.uk/cgi/news/release?i

d=34747
http://science.nasa.gov/science-news/science-at-nasa/2002/03jan_bioniceyes/http://science.nasa.gov/science-news/science-at-nasa/2002/03jan_bioniceyes/http://www.wired.com/gadgetlab/2010/04/australian-bionic-eye/http://www.wired.com/gadgetlab/2010/04/australian-bionic-eye/http://www.bionicvision.org.au/http://www.futuretimeline.net/http://www.cbc.ca/health/story/2010/07/07/eye-telescope.html?ref=rsshttp://www.cbc.ca/health/story/2010/07/07/eye-telescope.html?ref=rsshttp://www.tomsguide.com/us/FDA-Medicine-Medical-Technology-Vision-Care-cyborg,news-7401.htmlhttp://www.tomsguide.com/us/FDA-Medicine-Medical-Technology-Vision-Care-cyborg,news-7401.htmlhttp://www.tomsguide.com/us/FDA-Medicine-Medical-Technology-Vision-Care-cyborg,news-7401.htmlhttp://www.prnewswire.co.uk/cgi/news/release?id=34747http://www.prnewswire.co.uk/cgi/news/release?id=34747http://www.prnewswire.co.uk/cgi/news/release?id=34747http://www.prnewswire.co.uk/cgi/news/release?id=34747http://www.tomsguide.com/us/FDA-Medicine-Medical-Technology-Vision-Care-cyborg,news-7401.htmlhttp://www.tomsguide.com/us/FDA-Medicine-Medical-Technology-Vision-Care-cyborg,news-7401.htmlhttp://www.tomsguide.com/us/FDA-Medicine-Medical-Technology-Vision-Care-cyborg,news-7401.htmlhttp://www.cbc.ca/health/story/2010/07/07/eye-telescope.html?ref=rsshttp://www.cbc.ca/health/story/2010/07/07/eye-telescope.html?ref=rsshttp://www.futuretimeline.net/http://www.bionicvision.org.au/http://www.wired.com/gadgetlab/2010/04/australian-bionic-eye/http://www.wired.com/gadgetlab/2010/04/australian-bionic-eye/http://science.nasa.gov/science-news/science-at-nasa/2002/03jan_bioniceyes/http://science.nasa.gov/science-news/science-at-nasa/2002/03jan_bioniceyes/

eye, communicate, thoughts

Documents