21
Ein Unternehmen der Daimler AG Lecture @DHBW: Data Warehouse 00 Course intro Andreas Buckenhofer

Lecture @DHBW: Data Warehousebuckenhofer/20201DWH/... · Data Mining Exploration gn Simulation ETL Batch g Profiling Cleansing Map Reduce ELT RAM SSD Harddisk Video Sensors Geospatial

  • Upload
    others

  • View
    5

  • Download
    1

Embed Size (px)

Citation preview

Page 1: Lecture @DHBW: Data Warehousebuckenhofer/20201DWH/... · Data Mining Exploration gn Simulation ETL Batch g Profiling Cleansing Map Reduce ELT RAM SSD Harddisk Video Sensors Geospatial

Ein Unternehmen der Daimler AG

Lecture @DHBW: Data Warehouse

00 Course intro

Andreas Buckenhofer

Page 2: Lecture @DHBW: Data Warehousebuckenhofer/20201DWH/... · Data Mining Exploration gn Simulation ETL Batch g Profiling Cleansing Map Reduce ELT RAM SSD Harddisk Video Sensors Geospatial

Daimler TSS GmbH

Wilhelm-Runge-Straße 11, 89081 Ulm / Telefon +49 731 505-06 / Fax +49 731 505-65 99

[email protected] / Internet: www.daimler-tss.com

Sitz und Registergericht: Ulm / HRB-Nr.: 3844 / Geschäftsführung: Martin Haselbach (Vorsitzender), Steffen Bäuerle

© Daimler TSS I Template Revision

Andreas BuckenhoferSenior DB Professional

Since 2009 at Daimler TSS

Department: Machine Learning Solutions

Business Unit: AnalyticsDHBWDOAG

xing

Contact/Connect

vcard

• Oracle ACE Associate

• DOAG responsible for InMemory DB

• Lecturer at DHBW

• Certified Data Vault Practitioner 2.0

• Certified Oracle Professional

• Certified IBM Big Data Architect

• Over 20 years experience with

database technologies

• Over 20 years experience with Data

Warehousing

• International project experience

Page 3: Lecture @DHBW: Data Warehousebuckenhofer/20201DWH/... · Data Mining Exploration gn Simulation ETL Batch g Profiling Cleansing Map Reduce ELT RAM SSD Harddisk Video Sensors Geospatial

We make Daimler themost innovative mobility provider.We don‘t build cars, but we design innovative, holistic IT solutions forDaimler.

Page 4: Lecture @DHBW: Data Warehousebuckenhofer/20201DWH/... · Data Mining Exploration gn Simulation ETL Batch g Profiling Cleansing Map Reduce ELT RAM SSD Harddisk Video Sensors Geospatial

Daimler TSS 4

Sales & CareEnabling brilliant customer journeys

through customer interaction and

measurable customer proximity

Strategic InitiativesDeveloping new topics and ideas as well as

emerging technologies for Daimler

Cyber SecurityCreating customized, holistic

solutions for IT security

MobilityStaying agile with new

mobility concepts

Digital ProductionCreating optimum production processes

with continuous data analysis

Digital VehicleDesigning reliable software

development for a sophisticated

autonomous driving experience

Page 5: Lecture @DHBW: Data Warehousebuckenhofer/20201DWH/... · Data Mining Exploration gn Simulation ETL Batch g Profiling Cleansing Map Reduce ELT RAM SSD Harddisk Video Sensors Geospatial

Integrated, innovative and always close to the issue.

Daimler TSS

Methodological expertise and solutions coupled with technological excellence and a maximum degree of security are our specialties.

Acutely aware of this special position of trust, we bear responsibility through market-leading IT solutions for the success and future of a globally active group.

Driven by innovative ideas, we always venture across borders to find the best solutions and generate impressive results.

Page 6: Lecture @DHBW: Data Warehousebuckenhofer/20201DWH/... · Data Mining Exploration gn Simulation ETL Batch g Profiling Cleansing Map Reduce ELT RAM SSD Harddisk Video Sensors Geospatial

Daimler TSS 6

Our Locations & Project Hubs

D Ulm (Headquarters), Stuttgart, Berlin, Karlsruhe

KL Project hub Kuala Lumpur

CH Project hub China

*As of January 2019

4 locations / 1203 employees*:

Page 7: Lecture @DHBW: Data Warehousebuckenhofer/20201DWH/... · Data Mining Exploration gn Simulation ETL Batch g Profiling Cleansing Map Reduce ELT RAM SSD Harddisk Video Sensors Geospatial

We attach great importance to both human and digital networking.Passion for IT and a still very intimate start-up culture unite us in these endeavors, ensuring a sense of community and creative freedom in everyday working life.

Page 8: Lecture @DHBW: Data Warehousebuckenhofer/20201DWH/... · Data Mining Exploration gn Simulation ETL Batch g Profiling Cleansing Map Reduce ELT RAM SSD Harddisk Video Sensors Geospatial

Daimler TSS Data Warehouse / DHBW 8

Who are you?

Knowledge or experience:

- Databases

- SQL

- DWH

- Big Data

Your expectations?

About you

Page 9: Lecture @DHBW: Data Warehousebuckenhofer/20201DWH/... · Data Mining Exploration gn Simulation ETL Batch g Profiling Cleansing Map Reduce ELT RAM SSD Harddisk Video Sensors Geospatial

Daimler TSS Data Warehouse / DHBW 9

DWH Lecture – topics with learning targets

• Understand the importance of a data-driven culture

• Know tools relevant for DWH and Big Data

• Name DB technologies and techniques that are well-suited for DWHs

• Describe different DWH and Big Data architectures in detail

• Explain DWH data modeling (physical models)

• Explain Data integration/engineering (ETL) processes

• Understand Streaming technologies

• Specify visualization & metadata & security & PM requirements

• Name current DWH and Big Data trends

Page 10: Lecture @DHBW: Data Warehousebuckenhofer/20201DWH/... · Data Mining Exploration gn Simulation ETL Batch g Profiling Cleansing Map Reduce ELT RAM SSD Harddisk Video Sensors Geospatial

Daimler TSS Data Warehouse / DHBW 10

Daimler TSS

Data

Source

Data

Usage

Data Infrastructure

Data Management

Data Processing

Data Flow

Data Security

Data Governance

Data Culture

LogsOLTP

File Formats

Social Media

Images

Do

cum

en

ts

Machines

Star Schema

ODSData Vault

3NF

(SQL) Engines

Appliance

Virtualization

Reporting

Dashboarding

Search

Data Mining

Exploration

Simulation

Info

rmati

on

Desi

gn

ETL Batch

Str

eam

ing

CleansingProfiling

Map Reduce

ELT

RAM

SSD

Harddisk

Video

Sensors

Geospatial

Staging

Data Mart

Data Lake

Monitoring Scheduling Logging

Ethics Law GDPR

Data Catalog Metadata Domain Model

#dataops agile Project Management

Execution

Integration

Error Handling

Authentication Authorization Anonymization

Clic

kst

ream

Archive

On premises

In-MemoryMPP

RDBMS NoSQL Hadoop

MDM

Cube Core Warehouse Layer

KappaLambda

Real time

Micro Batch

Graph Models

Automation

Collection

Self Service

Planning

An

aly

tics

Audio

Glossary Data Quality

Cloud

Legacy

Text

Mobile

Data Hub

Page 11: Lecture @DHBW: Data Warehousebuckenhofer/20201DWH/... · Data Mining Exploration gn Simulation ETL Batch g Profiling Cleansing Map Reduce ELT RAM SSD Harddisk Video Sensors Geospatial

Daimler TSS Data Warehouse / DHBW 11

Many employment opportunities

DWH department in every (bigger) end user company, also in many medium-sized or small-sized companies

DWH department in every (bigger) consulting company

DWH-only specialized consulting companies

DWH tool vendors

Page 12: Lecture @DHBW: Data Warehousebuckenhofer/20201DWH/... · Data Mining Exploration gn Simulation ETL Batch g Profiling Cleansing Map Reduce ELT RAM SSD Harddisk Video Sensors Geospatial

Daimler TSS Data Warehouse / DHBW 12

Many employment opportunities – challenging JOB requirements

DWHs are complex, much more complex compared to most OLTP systems

Challenging job profiles with comprehensive requirements

• Data Architecture

• Data Integration / ETL

• Data Modeling (not only 3NF)

• Data Visualization

• Data Quality

• Data Security

• Requirements Engineering

• Project Management

Page 13: Lecture @DHBW: Data Warehousebuckenhofer/20201DWH/... · Data Mining Exploration gn Simulation ETL Batch g Profiling Cleansing Map Reduce ELT RAM SSD Harddisk Video Sensors Geospatial

Daimler TSS Data Warehouse / DHBW 13

Job description examples

Page 14: Lecture @DHBW: Data Warehousebuckenhofer/20201DWH/... · Data Mining Exploration gn Simulation ETL Batch g Profiling Cleansing Map Reduce ELT RAM SSD Harddisk Video Sensors Geospatial

Daimler TSS Data Warehouse / DHBW 14

Job description examples

Page 15: Lecture @DHBW: Data Warehousebuckenhofer/20201DWH/... · Data Mining Exploration gn Simulation ETL Batch g Profiling Cleansing Map Reduce ELT RAM SSD Harddisk Video Sensors Geospatial

Daimler TSS Data Warehouse / DHBW 15

About the lecture

• 36 semester periods per week (Semesterwochenstunden)

• Structure of the lecture

• Review of the preceding lecture

• Presentation of content with (group) tasks

• Group exercise

• Exam

• 1h

• Questions in German

Page 16: Lecture @DHBW: Data Warehousebuckenhofer/20201DWH/... · Data Mining Exploration gn Simulation ETL Batch g Profiling Cleansing Map Reduce ELT RAM SSD Harddisk Video Sensors Geospatial

Daimler TSS Data Warehouse / DHBW 16

Course material

• Download slides from

http://wwwlehre.dhbw-stuttgart.de/~buckenhofer/

Page 17: Lecture @DHBW: Data Warehousebuckenhofer/20201DWH/... · Data Mining Exploration gn Simulation ETL Batch g Profiling Cleansing Map Reduce ELT RAM SSD Harddisk Video Sensors Geospatial

Daimler TSS Data Warehouse / DHBW 17

Course contact

• Who is the class representative?

• Do you have a class email address?

• Please send me an email so that I have your contact data

Page 18: Lecture @DHBW: Data Warehousebuckenhofer/20201DWH/... · Data Mining Exploration gn Simulation ETL Batch g Profiling Cleansing Map Reduce ELT RAM SSD Harddisk Video Sensors Geospatial

Daimler TSS Data Warehouse / DHBW 18

Literature

Page 19: Lecture @DHBW: Data Warehousebuckenhofer/20201DWH/... · Data Mining Exploration gn Simulation ETL Batch g Profiling Cleansing Map Reduce ELT RAM SSD Harddisk Video Sensors Geospatial

Daimler TSS Data Warehouse / DHBW 19

Literature

http://btw2017.informatik.uni-stuttgart.de/slidesandpapers/Tutorial/SDM_slides.pdf

Page 20: Lecture @DHBW: Data Warehousebuckenhofer/20201DWH/... · Data Mining Exploration gn Simulation ETL Batch g Profiling Cleansing Map Reduce ELT RAM SSD Harddisk Video Sensors Geospatial

Daimler TSS Data Warehouse / DHBW 20

Finally

Questions?

Any time

Feedback

After lecture or by

email

Page 21: Lecture @DHBW: Data Warehousebuckenhofer/20201DWH/... · Data Mining Exploration gn Simulation ETL Batch g Profiling Cleansing Map Reduce ELT RAM SSD Harddisk Video Sensors Geospatial

Daimler TSS GmbH

Wilhelm-Runge-Straße 11, 89081 Ulm / Telefon +49 731 505-06 / Fax +49 731 505-65 99

[email protected] / Internet: www.daimler-tss.com

Sitz und Registergericht: Ulm / HRB-Nr.: 3844 / Geschäftsführung: Martin Haselbach (Vorsitzender), Steffen Bäuerle

© Daimler TSS I Template Revision