small data big impact

cochrane.org.ukPeter CochraneBIG

S m a l l D a t a

I M P A C T

Ahmed Elmagarmid 

Mourad Ouzzani

Stochastic complexity on a massive scaleexhibiting fractal behaviours from a local

to a national, international and global scale

ManAgement StruggleGenera l l y confounded by complex i ty

Instinct and experience increasingly dangerousKnee jerk management is always dangerous

Dynamic connectivityand multi-species

non-linearity

L i m i t e d : - A s s u m p t i o n s - M e a s u r e s - T h i n k i n g - M o d e l s - D a t a

E v o l v e d f o r a n s l o w c h a n g i n g a n a l o g u e w o r l d - l i n e a r i t y r u l e d & o u r m a t h w o r k e d

- b u t i t a l l c r a s h e s a b o v e o r d e r ( 5 ) - c o m p l e x i t y n o w r u l e s w i t h 1 0 s t o

1 0 0 0 s o f v a r i a b l e s - a n d p e r h a p s w o r s e

w e c o p e w i t h < 5 - 7 s i m u l t a n e o u s i d e a s

“There are lots of simple solutions to complex problems

and they are all wrong”

tyranny of THE SIMPLEWe are l inear th inkers but the wor ld i s complex

p o l i t i c s M a n a g e r s ECO NOM I CS

“People love simple - it is soobvious and doesn’t involve thinking”

Technology

Ecology

dependency J IGSAW

Economics

Society

Industry

Care

Markets

Trade

Education

Policing

Defence

Commerce

Security

Logistics

Finance

Government

Companies

W e h a v e n o m o d e l o r u n d e r s t a n d i n g

“ S o m e t i m e s W E m i g h t o b s e r v e o r i n f e r s o m e l i m i t e d re l a t i o n s h i p s a l o n g w i t h t y p i c a l

c h a r a c t e r i s t i c s - h o w e v e r, A I c a n p ro d u c e a m o re c o m p re h e n s i v e a n a l y s i s a n d a

m u c h f u l l e r p i c t u re”

The BIG DATA DOMAIN links everything and records relationships and patterns of behaviour - in doing so it creates vast amounts of data vital to the modelling of the real world…

Law & Order

Defence

Education

Health

Institutions

AX I O MSV e r y o b v i o u s ?

AX I O MSV e r y o b v i o u s ?

I n c r e a s e d e f f i c i e n c y l e a d s t o i n c r e a s e d b r i t t l e n e s s a n d m o re f a i l u r e s … .

… m o re d a t a i s a m u s t f o r a c c u ra t e b u s i n e s s m o d e l s a n d wa r g a m e s …

… . B I G a n d s m a l l D ATA a re ke y t o a m o re p re d i c t a b l e t h e f u t u re

S t o p O ve r O p t i m i s i n g i f y o u wa n t yo u r

c o m p a n y t o s u r v i v e a n d p ro s p e r

Avo i d e r ro n e o u s K P I s a n d b o n u s d r i v e n

c u l t u r e s

E m b ra c e a d a t a a n d b u s i n e s s m o d e l d r i v e n

a p p ro a c h

B I G D A T A

M A Y N O T B E E N O U G H

- P o l i s h i n g w h a t w e h a v e w i l l n o t r e a l i s e s u s t a i n a b i l i t y - N e w m a t e r i a l s & m a n u f a c t u r i n g m e t h o d s a r e r e q u i r e d - N e w d i s t r i b u t i o n , s u p p o r t a n d m a i n t e n a n c e r e g i m e s - E n e r g y a n d m a t e r i a l u s a g e h a s t o b e m i n i m i s e d - R e u s i n g , R e p u r p o s i n g , R e c y c l i n g , h a s t o b e r e a l - E c o n o m i c s y s t e m s h a v e t o b e f i t f o r p u r p o s e

AX I O MSNot so obv ious ?

“ To b u i l d a n e r v o u s s y s t e m f o r t h e p l a n e t w i t h m o s t

p e o p l e a n d t h i n g s c o n n e c t i n g ( a t w i l l ) w i t h

d i s t r i b u t e d s t o r a g e o f d a t a a n d i n f o r m a t i o n ”

MISSIONor evo lut ion ?

“ To s t o p p r o d u c i n g m o r e a n d m o r e f o r t h e f e w a n d

s t a r t p r o v i d i n g s u f f i c i e n t f o r t h e m a n y ”

DATA Big & SMALL

Small Data Focussed Detail

Big Data Macro-View Relationships

Limited C o n t a i n e d C o n s t r a i n e d

Expanding Te n d i n g t o

T h e I n f i n i t e

Both impor tan t & dynami c

THE AUDIT TRAILRaw mater ia l s to ro l l out to user

Raw mater ia l extract ion, re f inement Raw mater ia l recover from scrap Shipping log ist ics & supply cha in Process ing : forming, machin ing Test ing and Q A assessment Assembly machine & people Software loading & test Assembly test ing Packaging Shipping Supply Sa le

Sel ler test ing Sa le process /prep

Ownership Registrat ion User habits & maintenance

User feedback to des ign team Onward sa l e ( repea t a l l above )

Disposa l - canniba l i sat ion for parts Recyc l ing/recovery of raw mater ia ls

HEALTH PrognosisHospitalPrime Centre of Expertise & Capability

Medical CentrePrime Point of Local Capability

Home BasedMeasurementsMonitoringMedication

Technology Push - Patient Pull

M i g r a t i n g c e n t r e o f g r a v i t y

Dev i c e to d ev i c eMonitoring the in:out & effect give a complete picture

Self contained home net with no internet connection - or an aggregated and sporadic net connect

My toothbrushWil l not want to connect to the in ternet

….nor w i l l i t use 5G ! !

The co f fee mach ineANECDOTE

PurchasedFa i led in month 12Repa ired under warrantyReturned wi th a user repor t

8000 cof fees consumed in one year

8200 hours in one year ~ 1 cof fee an hour

Hea l th impl i ca t ionsProduct impl i ca t ions

The co f fee mach ineANECDOTE

PurchasedFa i led in month 12Repa ired under warrantyReturned wi th a user repor t

8000 cof fees consumed in one year

8200 hours in one year ~ 1 cof fee an hour

Hea l th impl i ca t ionsProduct impl i ca t ions

What will our applianceshave to tell us ??

FridgeFreezer

MicrowaveDishwasher

Washing MachineTumbler dryer

Oven/HobSecurityAir-ConShower

+++Des igner Feedback

Forward looking radar and car-to-car comms

Rearward looking radar and car-to-car comms

3,4,5G Network Connection

WiFi and WiMax Net Connection to Gas Station, Repair Shop for OS, APP, Info upgrades, failure reporting and real time remote diagnostics

Self contained data storage and analysis Small Data DomainVEHICLE to vehicle Comms

VEHICLE CommsNetworks without infrastructure

BIG DATA = Vehicle type, location, speed, owner, insurance, taxation…

small DATA = Production and repair history, performance data, driver habits,

apps,…

3,4,5G netscannot supportthe demands of

future traffic andlogistic networks…

but vehicle to vehiclesystems can…by providing

the most efficient and relevantlinking of users reporting their status

and sharing local/pertinent information

distributionA u t o m a t e d & i n t e l l i g e n t

BIG DATA Where has the package come from and where is it going ? Where is it now and what does it contain ?

small DATA The deta i led product history of how it was created; where it has been s t o re d a n d d e t a i l e d handling history including humidity, temperature, G-force, damage +++

Item

Box

Pallet

Warehouse

Container

Supplier

Customer

Ship toshore

Containerto Ship

H I D D E N V A L U EI n eve ry conta iner opera t ion g loba l l y

B IG DATA = Where i s my conta iner,s chedu le and content s ?

Sma l l DATA = What i s the cond i t i on o fthe conta iner content s ?

No single solutioncan deal with all forms of attack....

Security

N o o n e s o l u t i o n i ss u f f i c i e n t : t h e r e a r en o g o l d e n b u l l e t s …

I o T a n d D a t a

B I G & s m a l l D ATA r e q u i r e dt o m o d e l & s y n t h e s i s e r e a c t i v ea n t i d o t e s a n d a n t i c i p a t o r y p r o t e c t i o n

… a n a u t o i m m u n e s y s t e mi s r e q u i r e d t o c o p e w i t ht h e I o T a n d s t o c h a s t i cc o m p l e x i t y / a t t a c k s !

N E E D T O S H A R E

T R U M P S N E E D T O K N O W

We m i g h t s u p p o s e t h a t t h i s w o u l d t a k e p l a c e i n s e c o n d s o r m i n u t e s a n d n o t h o u r s a n d d a y s …

How Many Devices ?I m p l a n t e d , w o r n , c a r r i e d b y p e o p l e

To d a y I w e a r a n d c a r r y 6 s m a r t c o m m u n i c a t i n g t h i n g s … b u t i n t h e f u t u r e i t i s l i k e l y t o b e > > 1 0 ! !

H o w m a n y h a v e y o u g o t … . ? ?

T h e y a r e a l l a p a r t o f t h e I o T - a n d m a y / c a n / d o c r e a t e B I G a n d s m a l l d a t a …

Forbes Projection Based on an Industry Average

Forbes Projection Based on an Industry Average

A N I o T O F 5 0 B n T H I N G S

I S C O N S E R V A T I V E !

Stored information growth

1986

I can’t draw a

dot small

enough!!

.1996

0.001ZB

2006

0.13ZB

2016

13ZB

IoT will accelerate this trend to promote ever more on device storage

ZetaByte = 1021 ExaB = 1018 PetaB = 1015 TeraB = 1012

Prognosis: IoT small data distributedstorage may exceed the

internet

2014 2016 2018 2020 2022 2024

2

6

10

14

16

18

20

22

% Data Stored on Things

Year

Projection

Actual

Data extracted from numerous repor t s , papers & presentations by IDC, EMC, ICS,

Stored ON THINGS

Thank You

cochrane.org.uk c a - g l o b a l . o r g