big data fuels autonomous cars

Big Data Fuels Autonomous Cars

Jumana BayoumyAssociate System Engineer AnalystDell EMC [email protected]

Moustafa HishamSystem Engineer AnalystDell EMC [email protected]

Knowledge Sharing Article © 2017 Dell Inc. or its subsidiaries.

2017 Dell EMC Proven Professional Knowledge Sharing 2

Table of Contents

Introduction ................................................................................................................................ 3

The Growth of Big Data .............................................................................................................. 4

Defining Big Data ................................................................................................................... 4

How does Big Data work? ...................................................................................................... 6

The Birth of Autonomous Cars ................................................................................................... 8

Big Data and Autonomous Cars Collision ..................................................................................10

Autonomous cars in Action........................................................................................................11

Google self-driving car ...........................................................................................................12

Big Data Benefiting Self-driving Cars .....................................................................................14

Conclusion ................................................................................................................................16

Bibliography ..............................................................................................................................17

Disclaimer: The views, processes or methodologies published in this article are those of the

authors. They do not necessarily reflect Dell EMC’s views, processes or methodologies.


Introduction

In our time, occurrence of road accidents has become mainstream. Reckless drivers and bad

traffic circumstances are two among the many reasons why these accidents occur in the first

place. Unfortunately, these accidents don't only tragically influence the passengers of the

vehicle, but also their families. In addition, some families battle to afford the aftermath of such

horrible accidents. This is where innovative technology emerges to save peoples’ lives. Modern

technology is revolutionizing car driving through self-driving cars.

The idea of autonomous cars will restructure every industry that depends on the vehicle, from

taxis to car manufacturing, buses, and trucking. By removing humans from the equation of

driving, self-driving cars will be capable of saving millions of lives. Moreover, self-driving cars

can be capable of diminishing traffic by lessening the issues that lead to traffic such as bad

signal timing, traffic instances, and bottlenecks. Consequently, this reduction in traffic jamming

can save time and money for drivers. For instance, as the U.S. has started moving towards use

of the self-driving car, it was able to save approximately $2.4M a year which was originally paid

out for traffic signs.

Have you ever thought what really controls self-driving cars? Well, it is Big Data. The

conjunction of vehicles and big data has actually developed to a point where it can be used in

self-driving cars. The use of big data has become a vital fragment of self-driving cars

technology, where all real time data is being sensed, recognized, and stored instantly; thus,

making driverless cars basically a part of a massive data-collection machine. Autonomous

vehicles include GPS receivers, implanted computers, sensors, cameras, and possible access

to the Internet for cars to interact with each other. All of these are primarily directed by Big Data.

In fact, it reveals the foundation of self-driving cars mechanism; how do they realize their

destinations? How can they know the best route? How can they detect distance and

surrounding objects? How can they react to traffic lights?

In this Knowledge Sharing article, we will define big data, show how it works, and reveal how it

is essential to self-driving cars. Moreover, we will drive readers through the world of

autonomous cars and explain how they operate.


The Growth of Big Data

In our world where technology has firmly invaded our lives, the volume of data has been

massively growing. 2016 was a historic year for big data with many organizations storing,

managing, and extracting value from data of different types and volumes. Systems and

organizations that maintain huge volumes of structured and unstructured data will remain to

grow in 2017. The market will require platforms that assist data keepers in protecting and

controlling big data whilst enabling customers to analyze this data. These organizations will

develop to function well inside the world of IT businesses and standards.

Big Data has attained extensive use between not only organizations, but also researchers,

technologists, politicians, and the media. The process of analyzing huge sets of this data will

become a fundamental root of competition, reinforcing new rays of innovation and productivity.

This is according to research done by MGI and McKinsey's Business Technology Office. The

Internet of Things (IoT), expansion of social media, and the ever growing volume and details of

data gathered by enterprises will drive tremendous development in data for the near future. One

should first genuinely understand the meaning of big data in order to discover its role and

importance. The term “big data” has evolved to express a collection of commercial and

technological developments of tremendous amounts of individual data, largely created by social

mobile devices and social networks.

Figure 1 illustrates the continuous growth of big data from 2008 until 2020.

Figure 1: Big Data Growth (Oracle, 2012)


Defining Big Data

In the context of big data, there are three main common dimensions that define it and are known

as the 3 Vs – Volume, Velocity, and Variety. However, for the sake of expanding the value of

business, enterprises such as Microsoft introduced additional attributes to define big data;

hence, the 3Vs have now become 6Vs.

The 6Vs of big data can be described as follows;

Volume

Big data means massive volumes of data. Previously, data was primarily generated by

employees. However, now it is generated by networks, machines, and interaction of individuals

on systems like social media, making the analysis of big data volume an enormous challenge.

Variety

Variety implies several sources and categories of data both unstructured and structured. People

used to store data from limited sources such as databases and spreadsheets. Now, data comes

in various types; for instance, it can be photos, videos, audio, emails, PDFs, etc. The great

variety of this unstructured data creates difficulties for data analysis, storage, and mining.

Velocity

The velocity of big data refers to the speed of data movement from sources like machines,

networks, and individual interaction with things like mobile devices, social media, etc. The

massive and continuous flow of data helps researchers and enterprises make effective

decisions which offer planned and modest advantages. In fact, the process of sampling this data

can assist in dealing with volume and velocity problems.

Variability

Variability is relevant to performing effective analyses. The term “Variability” is defined as the

change in the word’s meaning; for example, one word can have several different meanings.

Therefore, algorithms are needed for proper analyses in order to comprehend the context and

interpret the correct word’s meaning within that context.

Validity

The big data validity means always having accurate and valid data when needed for use;

obtaining correct data is crucial for making the right decisions.

https://datafloq.com/read/12-algorithms-every-data-scientist-should-know/2024?utm=internal


Value

The true value in data comes from the analyses made on that data and how the data is being

transformed to meaningful information and then to sharable knowledge. The value is in how

enterprises can rely on insights retrieved from analyses of data and use them for making

decisions.

It is now obvious that big data deals with challenges other than the traditional 3Vs. Figure 2

illustrates how the attributes of big data have evolved over time.

Figure 2: The 6Vs of Big Data (Jani Puroranta, 2014)

How does Big Data work?

This is a million dollar question, although it seems an easy one. The enormous buildup and the

mystifying range of big data technology suppliers make discovering the correct answer quite

challenging. The main aim is to plan and develop a simple and cost-effective big data

environment. You can imagine big data as an engine. Thus, gathering the right components in a

steady and maintainable way is all that is needed to boost performance. Those components

contain;

Data Sources: related to functional and operative systems, such as sensors and

machine logs, social media, web services, and many other sources.

Data Platforms, Warehouses, and Discovery Platforms: enables data collection,

capture, and management. Later on, this data is converted to user’s insights and, finally,

action.


Big Data Analytics Tools and Apps: the “front end” used by directors, managers,

analysts, and others to retrieve user’s insights.

Reaching this level, it’s all about controlling and developing the power of big data in order to

create the value of business. For all components to work together, accurate big data

architecture and a thoughtful big data plan are required not only for examination of recent

streams of data and repositories but also for long-term development in the market and certain

business objectives. In the end, there is no specific guide for big data to work.

After we have defined big data, it is important to point out its perceptions. Big data should have

the capability to generate useful perceptiveness through the collection of new kinds and

volumes of data in order to avoid saving repetitive or excessive data. The idea of big data can

be outlined by one of the following three perceptions.

The first perception arises to respond to the challenges of technology related to securing,

storing, and analyzing the organization’s gathered volumes of growing data. This includes a

variety of technical inventions like recent forms of cloud storage and databases, which allow

cost-effective types of analysis to exist.

The second perception concentrates on the viable worth that can be augmented by

organizations through more efficient insights that are being produced from this data. This is due

to both enhanced technology and more readiness by customers to share their personal

information on social media and web services.

The third perception reflects the broader societal influences of big data, specifically the

consequences for a person’s privacy. In our case, we are more likely tackling the first perception

as it mainly addresses the technical inventions.

Big data relates to innovative technologies that contain diverse and rapid-changing data and the

bond gets stronger every day. This is because new technologies are capable of realizing value

from big data. Self-driving cars are meant to be one of those modern technologies. In the

following sections, we will explore the world of autonomous cars and explain how big data is a

vital part of such technology.


The Birth of Autonomous Cars

Several IT enterprises and important car industries have announced industrial production of

automated vehicles in early 2017. Many suppose that various models will be present in the

market by 2030. However, it is still uncertain to what degree these automated vehicles will be

fully qualified for self-driving in all situations and conditions.

As mentioned earlier, a fundamental benefit of self-driving vehicles is a reduction in accidents

and enhanced road safety, as human error is involved in most crashes. The test for self-driving

cars will be mainly based on how thoroughly they can imitate crash-free functioning of an

individual driver. Results from initial samples are encouraging, but the chance for new types of

crashes is still there; for instance, crashes might occur during control hand-over.

Self-driving cars may include various technological configurations. Some depend on better

integration between cars and infrastructure. These require investment in new kinds of

infrastructure and improvements in the popular communication protocol. Others depend more

on sensors of vehicles and entail slight investment in infrastructure.

Autonomous driving technology can be achieved through two development directions. The first

includes regularly enhancing automation process in traditional vehicles, where the driving task

done by humans is much less and becomes more automated. This direction induces

challenging concerns of interaction between humans and machines because the individual

driver is obliged to recommence active control whenever provoked to do so.

The second direction is more complex as it introduces cars without human interaction, except in

limited situations only – for example, specific paths and operations that include low speed – and

then slowly magnifying the use. Conventional car manufacturers usually follow the first direction,

and the second is most likely followed by new candidates.

The human-machine interaction concerns occurring in the first development direction will not

occur with fully automated vehicles although their consumption will be limited to areas where the

vehicle can securely grip the complete driving difficulty range.

Now after we have illustrated the emergence of self-driving cars, it is important to know how

using them is going to impact big data. Figure 3 depicts such influence. As per the figure, in

2020, an average user can generate 1.5 gigabytes of data daily, while an average self-driving

car can generate 4,000 gigabytes which means one autonomous car equals 2666 internet

users.


Figure 3: Autonomous Car and Internet Users Equation (Intel,2016)

Adding to the simple equation illustrated in Figure 3 regarding the consumption of data from

users and autonomous cars, various applications will be used and will consume data as well;

such as weather information and real-time traffic and a large number of cameras on the streets.

How can all this data be handled? Will people’s privacy be affected? Is it mandatory for vehicles

to record and share data of everything they analyze while traveling? We will discuss this in the

following sections.


Big Data and Autonomous Cars Collide

The main question is how to enhance transport using big data without jeopardizing privacy?

The volume and rapidity of data being produced, analyzed, and stored will necessarily amend

the transport segment. Location-sensing technologies such as Global Positioning System

(GPS), in which they are exceptionally prominent, are capable of tracking and locating

individuals accurately; measurements are accurate up to a few centimeters precision.

The process of data gathering can be implemented through purposeful recordings and

generation of information from the crowd, exposing new knowledge about transportation

movements and activities. In addition, useful patterns in our daily lives can be revealed from

these data.

Although the use of location-sensing data particularly may invade privacy and cause

mistreatment, we can still positively benefit from these data – for example, working sensors in

vehicles are expected to amend road safety.

Location and route data are integrally personal, but it is difficult to generalize them effectively.

Routes and paths data are as distinctive as fingerprints where only a few data checkpoints can

be sufficient to identify a specific person from an unrecognized cell phone data; such as sending

a text message, using a cash machine, posting on social media, and connecting to a Wi-Fi

network. Identification analysis may be accurate up to 95%.

Data security regulations lag new approaches of data gathering and use, and this is valid for

location data in particular. Regulators are torn between two approaches: 1) Sending users

notification for data collection and waiting for permission confirmation. 2) Enabling data

collection without user permission. To prevent unintentional outcomes, efficient location data

protection should be developed into algorithms, technologies, and processes.

A “Privacy by Design” approach might require, for example, that users would have the authority

to manage and allocate rights associated with their data. Failing to warrant resilient privacy

protection may negatively affect processing and gathering of location data, in which it could

impede the innovation path.


Autonomous cars in Action

As stated earlier, autonomous cars are the future of the automotive industry. Cars will no longer

need the drivers’ full and undivided attention to the road. An autonomous car is capable of

driving and navigating its way from the start to the end point with minimal human interaction.

Google, BMW, Mercedes-Benz, and Tesla are among the companies heavily investing in this

technology. Self-driven cars are equipped with additional mechanisms and technologies which

enable them to operate in real live traffic. So how does it work? A number of systems that

gather real-time data have to work synchronously together in order to analyze and process this

data to the vehicle.

Figure 4: Self-driving Car Main Components (HANNAH AUGUR, 2015)


There are several monitors and sensors currently installed in autonomous cars. To begin with,

multiple cameras are installed in self-driving cars to gather information from the nearby

environment starting from the surrounding cars, traffic and road signals, pedestrians, and

obstacles in the road. In addition to video cameras, autonomous cars are equipped with radar

sensors around the vehicle. These sensors are an essential segment that enables the car to

know the exact distance and position of surrounding objects which mainly creates a road map of

the adjacent vehicles. This is done through a wave emitted from the vehicle and then bounced

back to know the distance. Both video and radar sensors make the readings more accurate and

precise. Moreover, Lidar sensors are added to vehicles to detect lane edges and sides of the

road. These sensors illuminate the surroundings of the car to feed the vehicle with an accurate

road map.

All the big data that is gathered from various sensors and controllers in the car are just raw data

if they are not processed and analyzed efficiently. In the case of autonomous cars, this data

needs to be evaluated instantaneously as the reaction of the vehicle and its sensors need to be

in real-time and adjustable to any sudden changes in the surroundings. Moreover, a centralized

computer is equipped in those vehicles that are able to sync all this data together and regulate

the appropriate feedback from the car through deploying the corresponding response. The

continuous flow and analysis of big data are the controllers that actually drive an autonomous

car. Finally, the combination of all these measurements and computations in a vehicle creates

what we call an autonomous car where the car is fully automated and human interactions and

errors are reduced to the bare minimum.

Google self-driving car

A real-life demo that we will be illustrating is Google self-driving car which is one of the major

players in the autonomous cars industry. Google has implemented and tested the self-driving

technology using a Toyota Prius that has been adjusted to function as an autonomous vehicle.

The car has traveled across the USA logging more than 300,000 Kilometers. The car was

prepared to deal with all travel and road conditions starting from usual city traffic to crowded

highways and travel roads.


Figure 5: Google Self-driving Car (Erico Guizzo,2011)

Google autonomous car uses a laser, anchored on top of the car, as a range finder. The device,

aVelodyne 64-beam laser, produces a full 3D map of the surroundings. After that, the car

combines high-resolution world maps with measurements produced from the laser which

generates several types of data allowing the car to be self-driven and at the same time

respecting traffic regulations and avoiding obstacles. Additionally, the car contains other

sensors; a camera, placed next to the rear-view which is capable of detecting traffic lights; four

radars positioned on the front and back bumpers enabling the vehicle to view far distances in

order to deal with rapid traffic on freeways; and a GPS, wheel encoder, and inertial

measurement unit which reveal the location of the vehicle and keep track of its travel.

Figure 6 depicts a real image of what Google’s self-driving Toyota Prius analyzes in its on-board

computer.


Figure 6: Google on-board computer (Erico Guizzo,2011)

The on-board computer is able to detect the surrounding environment that includes cars, traffic

lights, and pedestrians and accordingly, reacts in an efficient and fast manner. Besides that, the

car is able to respond to any sudden incidents or changes in the surrounding traffic environment

in the most effective way. According to Sebastian Thrun, a Stanford University professor who

guides Google’s Autonomous Cars project mentioned self-driving cars will drastically reduce

road accidents, traffic congestion, and fuel consumption.

Big Data Benefiting Self-driving Cars

We have shown how Google’s autonomous car was driven in the right direction with big data.

However, the whole automotive industry should benefit from the advantages and offerings of big

data, not only Google. The automotive industry deals with lots of moving components which can

be enhanced through sensors for collection of data. While we have already discussed how big

data is essential for a vehicle to drive through using sensors, there are several other benefits of

sensors. For example, big data can reduce time to market for new cars through sensors. The

vehicle’s onboard sensors provide lots of valuable information to car manufacturers about the

behavior of car owners while using their cars; for instance, the average speed the car is driven

or how forcefully the brakes are hit and how cars react to such behaviors.


The previous can be monitored easily helping automotive industries identify areas needing

adjustments and enhancements.

Moreover, big data will have the power to stop car theft. The data gathered from car sensors will

assist in tracking the time and place of stolen cars. You can even know the true car owner from

his behavior while driving; there are algorithms implemented to realize the driving pattern of the

owner. If there are any changes or unexpected behaviors, police will be notified.

Big data can also be very beneficial for enhancing levels of inventory and reducing costs. Car

industries can predict the type and place of the cars required by collecting public data from

databases such as CRM. On top of that, using sensors in cars can help manufacturers predict if

any car parts need to be changed or are about to break down soon. This will lead to great

optimization in the world of supply chain for both dealers and manufacturers.

The examples that we have mentioned above are nothing compared with many other possible

opportunities that come with big data.


Conclusion

With the rapid increase in technologies and innovations, big data has become the key engine

that helps industries proceed with their inventions. Technologies delivered by big data have

promised to enhance the industry of autonomous cars in many ways; such as improving driving

behavior, building cars, reducing accidents, and offering comfort to people. Cars – just like

humans – will be able to see and talk to each other. Smart cars will be proactive instead of

reactive, capable of predicting accidents, traffic, and even suggesting alternative routes to your

destination. Autonomous cars are now heading to the roads of information-driven companies

where they will be used for many other purposes than driving-related tasks. This is not too far

from happening as the future is now the present. Get ready for an era that is full of automation.


Bibliography

[1]https://datafloq.com/read/self-driving-cars-create-2-petabytes-data-annually/172

[2]https://datafloq.com/read/3vs-sufficient-describe-big-data/166

[3]http://spectrum.ieee.org/automaton/robotics/artificial-intelligence/how-google-self-driving-

car-works

[4]http://www.slideshare.net/skissa/bilot-practical-big-data-101

[5]https://www.linkedin.com/pulse/20140911180123-143958496-the-recommendation- engine-

using-big-data-to-improve-mobile-banking-applications

[6] https://www.selfdrivingtimes.com/posts/autonomous-cars-are-the-big-data-providers-of-the

[7] http://dataconomy.com/2015/12/how-data-science-is-driving-the-driverless-car/

[8]https://www.mongodb.com/big-data-explained

[9]www.auvsi.org/driverlesscars

[10]https://www.researchgate.net/profile/Dan_Nunan/publication/270126631_Market_research_a

nd_the_ethics_of_big_data/links/55e69b1c08aecb1a7ccd6ec2.pdf

[11]http://www.tableau.com/asset/top-10-big-data-trends-2017

[12]http://www.mckinsey.com/business-functions/digital-mckinsey/our-insights/big-data-the-

next-frontier-for-innovation

[13]http://bigdata.teradata.com/US/Big-Data-Quick-Start/How-Big-Data-Works/

[14]http://www.itf-oecd.org/sites/default/files/docs/2015-05-28-big-data.pdf

https://datafloq.com/read/self-driving-cars-create-2-petabytes-data-annually/172

https://datafloq.com/read/3vs-sufficient-describe-big-data/166

http://spectrum.ieee.org/automaton/robotics/artificial-intelligence/how-google-self-driving-car-works

http://spectrum.ieee.org/automaton/robotics/artificial-intelligence/how-google-self-driving-car-works

http://www.slideshare.net/skissa/bilot-practical-big-data-101

https://www.linkedin.com/pulse/20140911180123-143958496-the-recommendation-%20%20engine-using-big-data-to-improve-mobile-banking-applications

https://www.linkedin.com/pulse/20140911180123-143958496-the-recommendation-%20%20engine-using-big-data-to-improve-mobile-banking-applications

https://www.selfdrivingtimes.com/posts/autonomous-cars-are-the-big-data-providers-of-the

http://dataconomy.com/2015/12/how-data-science-is-driving-the-driverless-car/

https://www.mongodb.com/big-data-explained

http://www.auvsi.org/driverlesscars

https://www.researchgate.net/profile/Dan_Nunan/publication/270126631_Market_research_and_the_ethics_of_big_data/links/55e69b1c08aecb1a7ccd6ec2.pdf

https://www.researchgate.net/profile/Dan_Nunan/publication/270126631_Market_research_and_the_ethics_of_big_data/links/55e69b1c08aecb1a7ccd6ec2.pdf

http://www.tableau.com/asset/top-10-big-data-trends-2017

http://www.mckinsey.com/business-functions/digital-mckinsey/our-insights/big-data-the-next-frontier-for-innovation

http://www.mckinsey.com/business-functions/digital-mckinsey/our-insights/big-data-the-next-frontier-for-innovation

http://bigdata.teradata.com/US/Big-Data-Quick-Start/How-Big-Data-Works/

http://www.itf-oecd.org/sites/default/files/docs/2015-05-28-big-data.pdf


Dell EMC believes the information in this publication is accurate as of its publication date. The

information is subject to change without notice.

THE INFORMATION IN THIS PUBLICATION IS PROVIDED “AS IS.” DELL EMC MAKES NO

RESPRESENTATIONS OR WARRANTIES OF ANY KIND WITH RESPECT TO THE

INFORMATION IN THIS PUBLICATION, AND SPECIFICALLY DISCLAIMS IMPLIED

WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

Use, copying and distribution of any Dell EMC software described in this publication requires an

applicable software license.

Dell, EMC and other trademarks are trademarks of Dell Inc. or its subsidiaries.

big data fuels autonomous cars

Documents