Design of Smart Cane for Blind

Dr. Bhaskar Thakkar Parmar NipunHead & Professor, Student of M.E. EMBEDDED SYSTEMSE & C. Engg. Department, G.H.Patel College of Engineering and Technology,G.H.Patel College of Engineering and Technology, Vallabh Vidhyanagar, IndiaVallabh Vidhyanagar, India nipunparmar2@gmail.combhaskarthakkar@gcet.ac.in

Abstract— In the past fifty years there has been path-breaking inventions in the medical arena to treat many diseases and in the field of medical/assistive technologies assisting the healthy and not so healthy in keeping track of vital body parameters like blood pressure, sugar etc. But there has been a dearth of cost effective assistive technologies that bring the visually impaired people at par with sighted persons and make them self-dependent. What if a blind person no longer requires a sighted person to assist him navigate the city, board the right bus to his destination, cross the road without any assistance from others, recognize a known acquaintance in a crowd and call out his name pointing in his direction and then move towards him and to say hello.

We intend to provide the visually impaired with a smart cane that comes as complete package solution to the person to navigate, travel and socialize without any assistance from others. It will be achieved by a camera to recognize and identify person as well as objects and ultrasonic sensor to identify obstacles and help them to avoid them. Face recognition through OpenCV when in community place like college, social gatherings, internet of things to assist in using public transport and other applications. The Beagle bone Black is employed to handle complex image processing algorithm to detect lane markings, detect and recognize faces in real time using OpenCV libraries running on Ubuntu.

Keywords— Beagle bone black, face detection, face recognization

I. INTRODUCTION

The blind long-cane remains the most widely used travel aid among the visually impaired people, whose number in the world is 285 million, and it is increasing by 7 million each year according to a World Health Organization (WHO) report [1]. The number of people visually impaired from infectious diseases has

reduced in the last 20 years according to global estimates work. Blind persons’ lives and activities are greatly restricted by loss of eyesight, and can only walk in fixed paths that are necessary to their lives. Currently, blind navigation systems are divided into indoor and outdoor navigation technologies. Most of indoor navigation assistances can be classified into basic obstacle avoidance systems and virtual reality technology. The obstacle avoidance systems use ultrasonic-based sensor, robotic assistance, or infrared to detect surrounding obstacles [2]-[4], and translate this information into acoustical sound to inform the user if a particular direction is blocked [9]. However, these systems depend on special provided infrastructure such as Radio Frequency Identification (RFID) tags. On the other hand, some virtual reality technology proposed to assist the blind walking indoor uses a digital map, which was created by recording the complete desired indoor space using a camera and processing it on a PC, to simulate the reality scene. When the blind moves within the space, the position and obstacle information are obtained using a Wireless Local Area Network (WLAN) communications standard and a stereophonic sound is produced to alert the blind. Several outdoor navigation systems have been proposed [5]-[7]. The main technologies and techniques include Geographic Information System (GIS), Global Positioning Satellite system (GPS), radar, ultrasonic, speech and RFID. In [5] and [6], an integrated GIS and GPS navigation system was proposed for positioning and tracking. Also, in [7] and [8], a study presented research on area blind assistance system using RFID tags and readers. C-5 Laser cane [10]: It was introduced in 1973 by Benjamin et al. It is based on optical triangulation by three laser diodes and three photodiodes acting as receivers. These photodiodes are silicon photodiodes [11]. The cane is capable of detecting obstacles at head level; ground level as well as in-front of the user. The device can detect obstacles in between a range of 1.5-3.5 m ahead of the user [12] [13]. The use of laser cane can be harmful if proper precautions are not taken and can affect the eyes of an individual without any proper eye wear. The photodiodes used at the receiving ends are most likely to respond to various ambient

sources, the sun light etc. Moreover, in hot and smoky areas the efficiency of the cane droops drastically [14].

Drawbacks of these devices are: Requirement of continuous scanning by some devices results in a lot of time consumption [15]. Expenses involved were high for some devices as they required conspicuous installation and maintenance cost [16]. The amount of information is not sufficient for the user. Reduction of noise signals and training were the basic necessity need to be focused upon [17]. The devices developed were many steps to lower the difference between the mobility of a sighted person and a visually impaired person. But still they could not develop the confidence among the deprived and hence the devices could not achieve success in the market.

The past developments have provided considerably better technological means for blind walking assistance, but because of technical and economic barriers, such as the required infrastructure and high cost, in addition to the psychological and ergonomic aspects of the blind community, the long cane remains the most widely used and accepted travel aid of the sight impaired. This can be seen from the widespread utilization of traditional long-canes where it gained symbolic importance within the blind community.

II. RELATED WORK

A smart cane was designed which consists of an ultrasonic sensor to measure the distance from the object, an infrared sensor to determine uneven surfaces, a tri-axial accelerometer to measure the angle between the floor and the white cane, and a compass to identify direction [18]. But the major drawback of this system was, how to swing electronic white cane, swing speed and influences from surrounding environment should be considered.

The Binaural Sonic Aid (Sonic-guide) is a device similar in appearance to a pair of spectacle frames. Its principle of distance measurement is based on the frequency shift and hence the receivers have an intermural amplitude difference which thus helps user in the determination of the result in the form of distance and direction of the obstacle [13].

From related work concluded that this embedded system required to be mounted on the cane is aimed to have a good and easy to learn interactive interface with the blind people that can assist them in the most accurate and

relevant manner. Moreover the system must be light in weight, low in cost and must be rigid.

III. SYSTEM DESIGN

The system needs to have an ultrasonic sensor in order to identify the obstacles in front of the blind people. A camera needs to be implemented on the cane to recognize the person with whom the blind person is interacting. Now there can’t be any visual interface to be provided in the system, so an audio interaction is made with the blind person using a Bluetooth headset connected to the system. The system is run by ARM Cortex A8 CPU which acquires the analog signals from the ultrasonic sensors and calculates the distance of the obstacle and notifies the person carrying the cane through the audio headset. The images captured by the camera module is compared with the images stored into the database of the processor and helps to identify and recognize the person standing in front.

Main components used in this system are:

• Beagle Bone Black: AM335x 1GHz ARM® Cortex-A8 procssor

• RadiumBoards HD Camera Cape:• Ultrasonic Ranging Module HC – SR04

• BBConsole Bluetooth module

Following figure shows the block diagram of the proposed system:

Figure 1 Block diagram of proposed system

Following Image processing algorithms will be used in implementing this system:

• Histogram Of Oriented Gradients:The histogram of oriented gradients (HOG) is a feature descriptor used in computer vision and image processing for the purpose of object detection. The technique counts occurrences of gradient orientation in localized portions of an image.

• Speeded up robust features:SURF descriptors can be used to locate and recognize objects, people or faces, to make 3D scenes, to track objects and to extract points of interest.

• Sobel Edge Detection:Can be used particularly within edge detection algorithms where it creates an image emphasizing edges.

IV. IMPLEMENTATION AND EXPERIMENTAL RESULTS

First of all we have connected beagle bone black to the laptop. We can access the terminal of the Linux OS of Beagle Bone Black using Secured Shell. After that connected beagle bone black to the internet. By entering “192.168.7.2” into browser we have checked whether Beagle Bone Black is connected or not.

Figure 2 Beagle bone black connected to internet

After this, Installed Linux (Debian) on beagle bone black and connected to TV with HDMI cable to check whether Debian OS is installed in it or not.

Connection to Ultrasonic sensor and camera cape and output of them are shown in the following figures.

Figure 3 Ultrasonic sensor interfaced with beagle bone black

Figure 4 Accurate signals obtained using ultrasonic sensor

Figure 5 Camera cape with beagle bone black

Now in order to implement the camera cape we need to install OpenCV to our system and thereafter we will have to install the repository for capturing the image.

Figure 6 Small OpenCV test to check whether it is working properly or not

By this type of hardware and software implementation and adjusting focus and lighting conditions of camera cape, we got perfect image which is shown below.

Figure 7 Final captured image

Also converted into gray to occupy less memory.

Text to speech conversion is also implemented using following steps:

First of all we need to install cylon.js, a java script framework for robotics and physical computing. This module provides an adapter/driver for text to speech based on eSpeak[19].

Install the module via NPM:

$ npm install cylon cylon-speech

After the module is installed, but before you run any scripts using it, ensure the espeak utility is installed on your computer.

$ sudo apt-get install espeak

Once the package has finished installing, try this command to verify everything is working as expected:

$ espeak "This is awesome, Linux speaking"

V. CONCLUSION & FUTURE EXPANSION

This project creates a new product for the visually impaired that helps them identify and recognize the person by using the camera and ultrasonic sensor for avoiding obstacles. This project is a novel implementation of Internet of Things concept for a social cause. It utilizes the vast capabilities that an active internet connection provides a discrete device by implementing complex image processing algorithms, voice based commands and text to speech conversion that run on powerful internet servers thus keeping the cost of device low as no such powerful system is required on board the cane. The prototype is very easy to use and has a flat learning curve and interfaces with user using audio commands. This prototype augments all the crucial needs of a visually impaired person at a very low cost.Till now, Ultrasonic sensor is implemented, image and video using HD camera cape are captured and text to speech conversion is also done. Image processing algorithms will be implemented next using OpenCV and Bluetooth module will be used for sending audio signals to the headphones.The future work in our project is to design a complete ready to use Smart Cane with GPS system included

over the cane, so that blind person can be located and can navigate thyself. There must be placed an emergency button over the cane, so that in case of any need by the blind it can directly send his/her location the respective warden looking after them or in case of accident the call can be made directly to the nearest hospital. The user input can be made audio using speech to text conversion method. A proper battery charger system and online support through a server so as to provide guidance and directions over the phone to the user in case of any malfunction of the cane or is the user is out of internet coverage. We also intend to deploy speech conversion libraries that adds Hindi and other local languages for receiving and sending voice based commands so that are product has a wider addressable market. Another valuable addition to our product will be implementing multiple target detection.

REFERENCES

[1] The WHO Global Action Plan, 2014-2019 report.

[2] P. Blenkhorn, and D. G. Evans, “A system for enabling blind people to identify landmarks: the sound buoy,” IEEE Transactions on Rehabilitation Engineering, vol. 5, pp. 276-278, September 1997.

[3] S. Shoval, J. Borenstein, and Y. Koren, “Mobile robot obstacle avoidance in a computerized travel aid for the blind,” in Proc. IEEE Int. Conference on Robotics and Automation, San Diago, CA, USA, 1994, pp. 2023-2028.

[4] R. Golding, and N Lesh, “Indoor navigation using a diverse set of cheap, wearable sensors,” in Proc. 3rd Int Symposium on Wearable computer, San Francisco, CA, USA, 1999, pp. 29- 36.

[5] R. Golledge, J. Loomis, and R. Klatzky, “Designing a personal guidance system to aid navigation without sight: progress on the GIS component,” International Journal of Geographical Information Systems, vol. 5, pp. 373- 395, October 1991.

[6] L Ran, A. Helal, and S. Moore, “Drishti: an integrated indoor/outdoor blind navigation system and service,” in Proc. 2nd IEEE Conference on Pervasive Computing and Communications, Orlando, Florida, USA, 2004, pp. 23-30.

[7] S. Willis, and S. Helal, “RFID information grid for blind navigation and way finding,” in Proc. 9th IEEE Int. Symposium on Wearable Computers, Osaka, Japan, 2005, pp. 34-37.

[8] A. Smailagic, R. Martin, “Metronaut: a wearable computer with sensing and global communication capabilities,” in Proc. 1st Int. Symposium on Wearable Computer, Boston, MA, USA, 1997, pp. 116-122.

[9] A Wireless Ranging System for the Blind Long-Cane Utilizing a Smart-Phone, A.A.Taht.

[10] Takizawa, H., et al. Kinect cane: An assistive system for the visually impaired based on three-dimensional object recognition. in System Integration (SII), 2012 IEEE/SICE International Symposium on. 2012. IEEE.

[11] Ye, C. Navigating a Portable Robotic Device by a 3D imaging sensor. in Sensors, 2010 IEEE. 2010. IEEE.

[12] Bouhamed, S.A., et al. New electronic cane for visually impaired people for obstacle detection and recognition. in Vehicular Electronics and Safety (ICVES), 2012 IEEE International Conference on. 2012. IEEE.

[13] Development of walk safe cane for the rehabilitation of blind people, Krishna Kumar, NIT, Rourkela. 2014.

[14] Jubril, A., et al., Obstacle detection system for visually impaired persons: Initial design and usability testing. Technology and Disability, 2013. 25(3): p. 199-205.

[15] Adebiyi, A., et al. Evaluation of feedback mechanisms for wearable visual aids. in Multimedia and Expo Workshops (ICMEW), 2013 IEEE International Conference on. 2013. IEEE.

[16] Mustapha, B., A. Zayegh, and R. Begg. Multiple sensors based obstacle detection system. in Intelligent and Advanced Systems (ICIAS), 2012 4th International Conference on. 2012. IEEE.

[17] Ercoli, I., P. Marchionni, and L. Scalise. A wearable multipoint ultrasonic travel aids for visually impaired. in Journal of Physics: Conference Series. 2013. IOP Publishing.

[18] Detection and Notification of dangerous obstacles and places for visually impaired persons using smart cane, Yoshirhiro Niitsu, Toshiki Taniguchi and Kouishiro Kawashima, 2014 seventh international conference on mobile computing and ubiquitous networking.

[19] https://www.npmjs.com/package/cylon-speech