The Future of Big Data

THE FUTURE OF BIG DATA The once and future past.

DAVID WELLMAN

Hello.

I did it wrong. 😔 I’m sorry.

Please tweet all complaints to @dwellman. - Management

It all started in back in 2012 when I made this presentation. 😢

I did my research. 🤓

I thought up a way to visualize it. 😎

I posted it on the internet. 😰

Then, The Internet Ruined My Life... 🤕

How, you ask?

Let’s start with the good parts of my presentation! 😇

I used an analogy to answer to the question... 🙂

If one byte is one grain of rice.

Then one kilobyte would be one cup of rice.

And one megabyte would take eight bags of rice.

One gigabyte of rice would fill three semi trucks.

One terabyte would need two container ships for all that rice.

One petabyte of rice would blanket Manhattan.

And an exabyte of rice would cover the west coast.

It would take just one zettabyte of rice to fill the pacific ocean.

And a yottabyte (not Yoda-bite) of rice would be the size of the earth.

I added historical context for perspective. ⌛

People liked it! 👍

A lot of people liked it! 😍

75,000 Slideshare views.

40,000 Facebook shares and likes.

20,000 Tweets and re-tweets.

12,000 LinkedIn shares.

It showed up in

unexpected places! 😋

In somebody’s homework... 😂

At an international data conference. .

Thanks for the tip of the hat (HT), but what’s are lorries?

http://mybroadband.co.za/news/broadband/147239-this-simple-infographic-explains-how-broadband-speeds-compare.html,

In a marketing infographics. /

PBS even put it in a kids show! 0

It's Okay To Be Smart -- Is Big Data Getting Too Big? https://www.youtube.com/watch?v=NTMkc0bLRlI

Then more viral it went,

the bigger the problem became. 😈

What is the problem, you ask? 👿


Everyone liked this...


Unfortunately, they all missed this, the most important part. 😕

People missed it because I didn’t define “value”.

☹

Yep, it was my fault.


I did really bad job of defining value... 😔


Dear NSA, please don’t get mad at me... 😳


Nothing to see here. Please move along. 😒


I dislike this... 😕


And this, what was I thinking? 🙄


I hope you passed chemistry. ☹


Please. Stop. 😣


Seriously! Stop It. 😩


Will it ever end?! 😫


Will it never end?!... 😖



Um. Yeah.

I’m really sorry about that.

Let’s try it again... 😢

Q: What is the value of BIG DATA?

This is a simple question,

but it’s a hard question to answer.

Here’s an example of why...

A: “The three defining properties or dimensions of BIG DATA are volume, variety and velocity.

Volume refers to the amount of data, variety refers to the number of types of data and velocity refers

to the speed of data processing.”

-  The Internet .

Q: What is the value of Big Data?

Are you sure? 🙃

Because, I don’t think that’s right...


Argument For: The Internet 😴 Argument Against: David Wellman 🙃

Volume

The quantity of generated and stored data. The size of the data determines the value and potential insight - and whether it can actually be considered big data or not.

The volume of a dataset when viewed by one company may seem big while a different company would consider it comparatively small.

Variety

The type and nature of the data. This helps people who analyze it to effectively use the resulting insight.

Data variety in one industry may be structured while another company finds unstructured data useful.

Velocity

In this context, the speed at which the data is generated and processed to meet the demands and challenges that lie in the path of growth and development.

Data velocity, whether it is fast or slow, is describing the rate of data collection, and it can be as different as collecting thousands of tweets per second compared to a single video feed.



Volume



Variety



Velocity





Volume



Variety



Velocity





Volume



Variety



Velocity




“The metrics of volume, velocity, and variety are relative and that makes them poor measures of success.”

-  David Wellman

It’s my presentation and I can quote myself if I want!



Our perspective shapes how we see Big Data And it shapes how we see the future of Big Data.

But, our perspective of Big Data is wrong.


Sinek Law

“People don't buy what you do; they buy why you do it. And what you do

simply proves what you believe” -  ― Simon Sinek

What vs. Why – The wrong questions

WHAT

HOW

WHY

“WHAT is big data?”

“HOW” big, is BIG?

“WHY” should I care?

What vs. Why – The wrong questions

WHAT

HOW

WHY

“WHAT is big data?”

“HOW” big, is BIG?

“WHY” should I care?

What vs. Why - The Right Questions

WHAT

HOW

WHY

3.  What does this mean to me?

2.  How does it change things?

1.  Why does Big Data matter?

20 MINUTE MBA

Let’s play a game of Market Simulation!

Business Value Simulator Governance State

Market Mechanics (The Rules of the Game)

1.  You’re selling Information. 2.  People buy what we believe is valuable. 3.  Why > What > How

Conway’s law

“Organizations which design systems are constrained to produce designs

which are copies of the communication structures of these organizations.”

- M. Conway


Inn

ova

tion

Time

Point A The Starting Line

Point B The Finish Line

How we BELIEVE the world should work.


Inn

ova

tion

Time

But, there are competing theories on the market’s direction.


Inn

ova

tion

Time

Some theories are more right than others.


Inn

ova

tion

Time

Moore’s Law sets the lower boundary for a markets performance.


Innovators Early

Adopters

Basic Business Cycle – Adoption Curve

Early Majority

Late Majority Lagers


Innovators

Early Adopters

Basic Business Cycle – Adoption S-Curve

Early Majority

Late Majority

Lagers

80%

100%

60%

20%

0%

% A

do

ptio

n -

Sta

te

10%


Inn

ova

tion

Time

Normal business cycles create s-curves towards the target.


Inn

ova

tion

Time

🎯

Some believe this is the best the market offers.

Some believe in a better way.


Inn

ova

tion

Time

🎯

🎯


Inn

ova

tion

Time

over

Under

🎯

🎯

Amara’s Law creates conflict between the two.

Amara’s Law

“We tend to overestimate the effect of technology in the short run

and underestimate the effect in the long run.” - Roy Charles Amara


Inn

ova

tion

Time

over

Under

🎯

🎯

1. The Standard Way


Inn

ova

tion

Time

over

Under

🎯

🎯

2. Mind The Gap


Inn

ova

tion

Time

over

Under

🎯

🎯

3. The Hard Struggle


Inn

ova

tion

Time

over

Under

🎯

🎯

4. The Long Road


Inn

ova

tion

Time

over

Under

🎯

🎯

5. The New Normal


Inn

ova

tion

Time

over

Under

🎯

🎯

Achievement Earned – New Game, New Rules


Inn

ova

tion

Time

🎯

🎯

Level UP




Producers:

Consumers:

Connections:

0 0 0


Producers:

Consumers:

Connections:

0 0 0 $0

Market Value


Producers:

Consumers:

Connections:

0 0 0

Training Level 1

How to win.

$0 Market Value


Producers:

Consumers:

Connections:

0 0 0 $0

Market Value

Players


Producers:

Consumers:

Connections:

0 0 0 $0

Market Value

Players

We want really BIG NUMBERS!


Producers:

Consumers:

Connections:

0 0 0 $0

Market Value

Players

We want really BIG NUMBERS!

AND, reach the TOP of the MARKET!!!

🎯 🎯

$0 Market Value


Producers:

Consumers:

Connections:

0 0 0

Level 2

Supply Side Economics

Sarnoff’s Law

“The value of a network is proportional to the number of viewers.”

- David Sarnoff

V(n)

A Potential Customer Sarnoff’s Law: V(n) Innovators

Producers:

Consumers:

Connections:

0 0 0

Consumer

$0 Market Value

Customer Demand Sarnoff’s Law: V(n) Innovators

Producers:

Consumers:

Connections:

0 0 0

Consumer

$0 Market Value

A Potential Producer Sarnoff’s Law: V(n) Innovators

Producers:

Consumers:

Connections:

0 0 0

Producer

$0 Market Value

Entrepreneur Sarnoff’s Law: V(n) Innovators

Producers:

Consumers:

Connections:

0 0 0

Producer

$0 Market Value

Product Market Gap Sarnoff’s Law: V(n) Innovators

Producers:

Consumers:

Connections:

0 0 0

THE SCARY “PRODUCT MARKET GAP”!

$0 Market Value

Product Market Fit Sarnoff’s Law: V(n) Innovators

Producers:

Consumers:

Connections:

1 1 1 $1

Market Value

Excess Consumer Demand Sarnoff’s Law: V(n) Innovators

This is market potential!!!

Don’t Stop!

$1 Market Value Producers:

Consumers:

Connections:

1 3 1

Market Reach Sarnoff’s Law: V(n) Early Adopters

Producers:

Consumers:

Connections:

1 3 3 $3

Market Value

Total Addressable Market Sarnoff’s Law: V(n) Early Adopters


Consumers:

Connections:

1 3 11

A small market means

small numbers...

Market Opportunity Sarnoff’s Law: V(n) Early Adopters

Producers:

Consumers:

Connections:

1 11 3 $3

Market Value

BIG NUMBERS! Opportunity: $11 Landed : $3

Missed: $8

Captured Market Sarnoff’s Law: V(n) Early Majority


Consumers:

Connections:

1 11 11

AWESOME! Now it’s time to get serious.

Monopoly Sarnoff’s Law: V(n) Late Majority

Producers:

Consumers:

Connections:

1 12 12 $12

Market Value

Demand Side Competition Sarnoff’s Law: V(n) Laggards

Producers:

Consumers:

Connections:

0 0 0 $12

Market Value Bargaining Power Of Suppliers

Bargaining Power Of Buyers

Thre

at

of

Ne

w E

ntra

nts Th

rea

t of

Ne

w Sub

stitutes

$0 Market Value


Producers:

Consumers:

Connections:

0 0 0

Level 3

Demand Side Economics New game. New rules.

Metcalfe’s law

“The value of a network is proportional to the square of the number of connected users of the

system.” - Robert Metcalfe

V(n2)

$0 Market Value

Business Value Simulator Metcalfe’s law: V(n2) Innovators

Producers:

Consumers:

Connections:

1 1 1

$1 Market Value


Producers:

Consumers:

Connections:

1 1 1

0 1

$2 Market Value


Producers:

Consumers:

Connections:

1 1 2

0 1

2

$4 Market Value


Producers:

Consumers:

Exchange:

1 1 2

1 0

3 2

$4 Market Value


Producers:

Consumers:

Exchange:

1 1 2

1 0

3 2 4

$4 Market Value


Producers:

Consumers:

Exchange:

1 1 2

V(n2)

V(22) = 4

$3 Market Value


Producers:

Consumers:

Connections:

1 3 3

0

1

2

3

$6 Market Value


1

2

3

4 5

6

Exchange: 4

$9 Market Value


7

8

9

Exchange: 4

$9 Market Value


10 12

11

Exchange: 4

$16 Market Value


13 15

14 Exchange: 4

$16 Market Value


16

Exchange: 4

$16 Market Value


Exchange: 4

V(n2)

V(42) = 16

$16 Market Value


Exchange: 4

$36 Market Value


Exchange: 6

$64 Market Value


Exchange: 8

$144 Market Value


Exchange: 12

$0 Market Value


Producers:

Consumers:

Connections:

0 0 0

Level 4

Global Trade Economics New game. New rules.

Reed’s Law

“The value of large networks scale exponentially with the size of the network.”

- David P. Reed

$?? Market Value

Business Value Simulator Reed’s law: V(2n) Innovators

Exchange: 2

$?? Market Value


Exchange:

Exchange: a(2) b(2)

$?? Market Value


Exchange:

Exchange: a(2) b(2)

$4 Market Value


Markets: 2

$4 Market Value


Markets: 2

$8 Market Value


Markets: 3

$16 Market Value


Markets: 4

$4,096 Market Value


Markets: 12


Inn

ova

tion

Time

The S-Curve -> The smart Ones.

$12 Market Value


Inn

ova

tion

Time

The S-Curve -> The SMARTER Ones.

$144 Market Value


Inn

ova

tion

Time

The S-Curve -> The SMARTEST Ones. $4,096

Market Value

REBOOT!

DataSize Type Network ValueLaw NetworkType

Hobbies Byte Proprietary V(n) Restricted Kilobyte SarnoffLaw

Desktop Megabyte Homogenous V(n2) SimpleSocial Gigabyte Metcalfe'sLaw

Internet Terabyte HeterogeneousV(2n) Market Petabyte Reed'sLaw &PlaDorms

BigData Exabyte Proprietary V(n) Restricted SarnoffLaw

Tomorrow Ze=abyte ??? Homogenous V(n2) Social Metcalfe'sLaw

TheFuture Yo=abyte ??? HeterogeneousV(2n) Market Reed'sLaw &PlaDorms

THE ONCE AND FUTURE PAST

Yesterday Tom

orrow

TODAY

The Future - Data

¤  Open Source Machine Learning

¤  Open Data Standards

¤  More Data Collected – Less Data Moving Location.

¤  Data Platforms – Data Brokers & Algorithm Exchange

¤  Network Effects around the USE OF DATA.

¤  More and more companies sharing RESULTS.

The Future - Management

¤  New management theories & styles are needed

¤  From command & control è community & interchange

¤  From internally è external,

¤  social and community needs come first.

The Future - Governance

¤  + Crowdsourced Compliance & Open Governance

¤  + “Signals of trustworthiness.”

¤  + “Drain out the ego.”

¤  + Trust & Safety

¤  - Unfettered access can destroy value

¤  - Un-vetted Memberships

The Future - Metrics

¤  Interaction Failures

¤  Engagement

¤  Match Quality

¤  Positive & Negative Network Effect

CASE STUDY - EVIDENCE

One Dataset, Many Analysts

One Dataset, Many Analysts

A Social Research Experiment:

29 research teams from around the world where asked to answer the same question,

all using the same dataset.

The Question

Are football (soccer) referees more likely To give red cards (flags) to players with dark skin

Than to players with light skin?

CASE STUDY – THE RESULTS

“The experiment convinced us that

bringing together many teams of skilled researchers can balance discussions, validate scientific findings

and better inform policymakers.”

http://www.nature.com/news/crowdsourced-research-many-hands-make-tight-work-1.18508

BE SMART

Inn

ova

tion

Time

HOW How do we use it?

What What is big data?

Why Why do we care?

WHAT HOW WHY

over

Under

THE END.

“When you put tools before people the people become the tools,

and nobody wants to be a tool.” -  David Wellman

There I go again, quoting myself like I know anything.


Data & Analytics

The Future of Big Data