Splunklive! Stockholm 2015 - IKEA

Copyright © 2015 Splunk Inc.

Magnus Johansson Splunk Ninja @ IKEA

IKEA’s journey to end-‐to-‐end visibility: From e-‐commerce to security

2

Personal introducLon

!   Magnus Johansson !   Splunk Ninja @ IKEA !   Worked with security for 8 years !   Linux geek since way back !   Live in the capital of IKEA country, Älmhult

2

3

Agenda !   Why did we exchange the current SIEM !   Access control in a mulLtenancy environment !   Splunk part of our technical standard !   How to handle unknown syslog feeds !   Security posture and business value !   Key benefits

3

4

IKEA Journey

4

Legacy SIEM

New SIEM Requirements

eCommerce

IT Ops

Enterprise-‐wide Security

More than a SIEM


Why did we exchange the old SIEM

6

Why Did We Change SIEM Legacy SIEM New Requirements Splunk

Scalability Expensive @ 200GB/day Difficult to grow

needed +10 TB/day ✔

User # Limited user support (sec team) 1,000s ✔

Role-‐based Access

Single view/control of data Full Role-‐based access control ✔

Data supported Problem imporLng desired data Ability to import all types of data ✔

PlaCorm/cost Appliance gets old, unable to scale in cost effecLve manner

Sogware to adjust compuLng infrastructure easily ✔

Security and other use case

Security only Security, IT, eCommerce, Business ✔

6

7

Big Win on the way to SIEM Replacement

7

Legacy SIEM


eCommerce + Business AnalyLcs

IT Ops


More than a SIEM

Let’s bring eCommerce data in first…

8

The response from eCommerce team !   Went from reacLve troubleshooLng

–  Customer sent an e-‐mail and complained, the SSH and GREP session started, could take days to weeks

–  Only one data source per Lme

!   To proacLve troubleshooLng –  MulLple data sources and correlaLons –  Dashboard that shows environment status, including business impact –  CPU, memory uLlizaLon, capacity planning –  Could troubleshoot in minutes

8

9 9

10

Wow, this is great, we need more! !   AddiLonal 1TB license ager 3 month !   AddiLonal teams as well as eCommerce wanted to add data !   ExisLng environment was expanded !   Business analyLcs

–  Real Lme sales compared to last week for the major regions –  Payment provider availability –  Performance of Akamai –  Business process tracing (orders that takes longer than 10 seconds to process)

10

11

New insight and replacements using Splunk !   NEW -‐ Monitor applicaLon and business processes !   NEW -‐ Get insight in black boxes !   NEW -‐ Replaced other monitoring soluLons !   NEW -‐ Splunk can handle our complex environment !   Broken link app to each area

11

12

ImplemenLng Splunk as SIEM

12

Legacy SIEM




More than a SIEM

More data, more users New SIEM ImplementaLon


Access control in a mulL-‐tenancy environment

13

14

How to provide granular access control !   SeparaLon of data !   Possibility to share data !   Reports without access to raw data !   Each area has its own index

14

15

Access to mixed indexes !   ApplicaLon teams need informaLon various indexes

15

Oracle Linux

Business service Subset of data Subset of data

16

Search filter restricLons !   Blacklist approach:

–  “NOT (index=indexname AND (blacklis(tem1 OR blacklis(tem2 OR …..))”

!   Whitelist approach: –  “NOT (index=indexname NOT (whitelis(tem1 OR whitelis(tem2 OR …..))”

16

17

Combine whitelist and blacklist !   Really granular control to specific data ! srchFilter = NOT (index=linux NOT (host=lx4351* OR host=lx4352*)) NOT (index=linux AND (sourcetype=linux_secure OR sourcetype=pii_data))

17


Splunk part of our technical standard

18

19

How to get massive amount of data in !   How to install Splunk forwarder in 400 locaLons

–  1000 AIX servers –  3500 Linux servers –  5500 Windows servers –  100000 Windows clients

!   Syslog –  Only one load balancer with one ip and port –  Network switches, firewalls, appliances, you name it

19

20

Step by step approach !   Started with Linux !   Part of Standard OperaLng Environment !   Bundle IKEA specific configuraLon in a RPM !   Generic bootstrap principle reused

20

21

Bootstrap RPM !   AutomaLc domain specific configuraLon

–  Closest deployment server –  Closest index cluster

!   DistribuLon of IKEA cerLficates !   Hardening (bind to localhost) !   Everything else, deploy it in an app! !   Take control of splunk.secret file!

21


Unknown syslog feed

22

23

Syslog feed from various devices !   Can’t control syslog devices !   Unable to specify different ports per type !   Single load balancer !   New unknown feed to syslog index

23

24

Labor intensive manual work !   Manual creaLon of inputs.conf !   Many different types of source types !   Different customers, different desLnaLon indexes !   Good admins are lazy

24

25

Challenge !   Template based configuraLon !   Create new and update templates !   VerificaLon before deployment of new code !   Possibility to publish to a GIT hub

25

26

SoluLon TA generator !   Workflow acLon to feed generator !   Simple PHP and Mysql driven webpage

26

27

SoluLon TA generator !   Select log type and go!

27

28

Enterprise Wide Security Using Splunk

28

Legacy SIEM




More than a SIEM… New SIEM ImplementaLon


Increased security posture in organisaLon

29

30

Security awareness was increasing !   Teams increased their collaboraLon with Splunk as a enabler !   Teams started to look in the “background noise” !   New risk areas was detected

–  “Hey – I think we are hacked!” –  Awempts to bypass security mechanisms (slow-‐rate and brute force awacks) –  Google search bot from Ukraine? –  Fraud awempts

!   Start small, do you always need Splunk ES?

30

31

Helpdesk support dashboards !   Access to dashboards without raw events

31

32

Get clarity and overview

32


Key benefits

33

34

Key benefits !   Real-‐Lme reacLon instead of weeks later !   Before it was hard to get access to data – Now we have a queue… !   Splunk is a collaboraLon enabler – teams works together in a new ways !   Security put the ball in play, business is now our driver

34

35

How to engage the data owners !   EducaLon, educaLon, educaLon…

–  Help with geyng the data in –  How to create basic searches –  How to create dashboards

!   Appoint local Splunk champions for each area !   Internal Splunk Newslewers !   CompeLLons !   Splunk T-‐Shirts!

35

36

Security is not the bad guys anymore

Please take my data!!!

36


Key takeaways

37

38

Key takeaways !   EducaLon

–  Make sure you educate yourself and the organizaLon

!   Use Splunk PS !   Think big – act small

–  Make sure your plan and architecture allows for expansion –  Don’t try to do all use-‐cases/data sources at once

!   The more people using the data the cheaper it becomes!

38

Technology

Splunklive! Stockholm 2015 - IKEA