Upload
denodo
View
28
Download
1
Embed Size (px)
Citation preview
O C T O B E R 1 8 , 2 0 1 6 S A N F R A N C I S C O B A Y A R E A , C A
#DenodoDataFest
RAPID, AGILE DATA STRATEGIESFor Accelerating Analytics, Cloud, and Big Data Initiatives.
Big Data and IoTAnna Yong
Director of Product Marketing: Data-in-Motion, Hortonworks
4ZBDATA
44ZBDATA
TOMORROW
INTERNETOF
ANYTHING
The New World of Business
Pre-transaction insights
Create new sources of revenue
New customer engagement models
Gain actionable intelligence fromall of the data
Business has access to more data
Accelerate Time to Value
5 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
EMBRACE AN OPEN APPROACH
MASTER THE VALUE OF DATA
EVERY BUSINESS IS A DATA BUSINESS
MachineLearning
C L O U D
Edge Analytics
The Connected Data Platforms Reference Architecture
Deep HistoricalAnalysis
D ATA C E N T E R
Stream Analytics
Data in Motion
Data in Motion
Data atRest
Data atRest
EdgeData
EdgeData
Data is Transforming the Enterprise…Now!
Billions of connected devices
Consumers access everything instantly
Data has become a valuable corporate asset
Adoption is Accelerating
FinancialServices
Retail
Telco
Automotive
55% of theUS F100
75% of theUS F100
8 of the top 9 in
North America
5 of the top 10
Automakers
Adoption
Imagine this Future
All data
Link customer, product, and supply chain
Real time analytics at the edge
Data is everyone’s product
Data is your future.
10 © Hortonworks Inc. 2011 – 2016. All Rights Reserved10 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Data Drives the Connected Car
Insurance Premiums
Warranties
Recalls
Pricing Models
DesignInnovation
Autonomous Driving
Connected City Infotainment
Sensors
Scheduled Maintenance
Predictive Maintenance
RouteOptimization
INSURANCECOMPANIES
GOVERNMENTAGENCIES
INFOTAINMENTPROVIDERS
SOFTWARECOMPANIES
AUTOMAKERS
11 © Hortonworks Inc. 2011 – 2016. All Rights Reserved11 © Hortonworks Inc. 2011 – 2016. All Rights Reserved11 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Big data from the Internet of Things incorporates both data in motion and data at rest.
The Connected Car is just one example of IoT
DATA IN MOTION DATA AT REST
12 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
HDPHORTONWORKS
DATA PLATFORMPowered by Apache Hadoop
The Internet of Things
IoT
13 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
…….Becomes the Internet of Anything
14 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Constrained High-latency Localized context
Hybrid – cloud / on-premises Low-latency Global context
CoreInfrastructure
Big Data from the Internet of Things
RegionalInfrastructureSources
15 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
IoT Drives New Requirements for Big Data
Perishable Insights
ConnectivityPrioritization
Security
AdaptabilityExtensibility
Scalability
Provenance
Real-Time
16 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Hortonworks DeliversConnected Data Platforms
DATA AT RESTDATA IN MOTION
Modern Data Applications
Hortonworks DataFlow
Hortonworks Data Platform
INTERNETOF
ANYTHING
PERISHABLE INSIGHTS
HISTORICAL INSIGHTS
ACTIONABLEINTELLIGENCE
17 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Secure
Real-time
Streaming
Integrated
Hortonworks DataFlow for Data in MotionPowered by Apache NiFi, Kafka, and Storm
18 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Hortonworks Data Platform for Data at RestPowered by Open Enterprise Hadoop
Open
Interoperable
Ready
Central
19 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
HDPHORTONWORKS
DATA PLATFORMPowered by Apache Hadoop
HDF Enables IoT Applications
HDPHORTONWORKS
DATA PLATFORMHDPHORTONWORKS
DATA PLATFORMPowered by Apache Hadoop
Before HDF After HDF
20 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
From the Device into the Core for Real-Time Analysis and Insights
21 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
HDF IoT Use CasesConnected Car
https://community.hortonworks.com/content/kbentry/32605/running-nifi-on-raspberry-pi-best-practices.html
22 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Schlumberger Dataflow
Slideshare: http://www.slideshare.net/HadoopSummit/from-zero-to-data-flow-in-hours-with-apache-nifi-64032731LinkedIn: www.linkedin.com/hp/update/6171029401820479488Twitter: https://twitter.com/aldrinpiri/status/747865822228422656
23 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Open Energi Uses HDF for Electricity Demand Response
As a result of its investment in Hortonworks Dataflow, Open Energi is Already:
• Reducing costs thanks to 10-15% less data being transmitted across a mobile network
• Creating a full transparent trail for data provenance that Open Energi can sharewith customers
• Enabling line of business teams to contribute to building dataflow rules and processes
• Standardizing the output of data across various end point devices
Open Energi: hortonworks.com/blog/data-fuel-open-energi-virtual-power-station-hortonworks-dataflow/
The Journey to Safer Global Travel
25 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Prescient’s Journey
Prescient Needed to Rapidly Ingest Security Data from Tens of Thousands of Sources and Analyze it Quickly to Protect the Safety of Global Travelers
Prescient is a risk management company with its roots in the US federal sector
As it developed commercial products for the civilian market, Prescient chose Apache NiFi to ingest data sources that indicate traveler risk down to the street level
Prescient is currently ingesting data from 49,500 unstructured sources like RSS feeds, news feeds, social media, forums and blogs, using humans to identify false positives
Initially, the geospatial analysts could produce one assessment every 3-4 days
Prescient needed a cost-effective way to store the raw data for subsequent post-collection analytics that would further strengthen their product portfolio
26 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Prescient’s Journey
Provenance Metadata
Threat-Proximity Mobile Alerts
Global Threat Map
Streaming Threat Ingest
Threat Assessments
Rapid Ingest & Machine Learning Deliver Accurate, Local Traveler Warnings
HDF ingests 49,500 data sources
Rich data ingest, real-time alerts and deep historical analysis increase analyst output by 700%
HDF connects Prescient’s on-prem/cloud hybrid architecture
HDP stores 5 petabytes of data interconnected to EMC tech
Spark machine-learning algorithms promise additional efficiencies
Innovate
Renovate Safe Global Travel
PREDICTIVEANALYTICS
E T LO N B O A R D
D A T AD I S C O V E R Y
E T LO N B O A R D
A C T I V EA R C H I V E
S I N G L EV I E W
“We know that when we define a high-threat area in a given area of the world, that it is underpinned by very specific data sources. It’s data-driven, and we can point to those sources—if ever asked—and say, ‘Here’s why.’”
Mike Bishop, Chief Systems Architect, Prescient
Security Threat Archive
D A T AE N R I C H M E N T
27 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Prescient Traveler’s Assessment of Santa Clara Yesterday
28 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
29 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Virtualization, Big Data and IoT
30 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Thank You