Slide Cong Nghe IDS-IPS Bai5(2012951615)

8/10/2019 Slide Cong Nghe IDS-IPS Bai5(2012951615)

1/84

Outline

1. Introduction Intrusion & current Threat scenario

2. Introduction to IDS

3. Introduction to IPS

4. String matching algorithms

Q&A

9/5/2012 2


2/84

1. What is Intrusion?

A set of actions aimed at compromising thesecurity goals (confidentiality, integrity, availability of acomputing/networking resource)

9/5/2012 3


3/84

Why need to protect?

Current Threat Scenario

Blended Threats

USER

External Internal

Phishing

Malware

Botnets

Malicious intent

Identity theft

Data Corruption

Information Leak

9/5/2012 4


4/84

Why need to protect? (Cont..)

There are two types of threats External threats Internal threats

9/5/2012 5


5/84


External Threats (Targeting the Individuals)Who are the attackers?

It is no longer individuals

Attacks executed as joint ventures among professionalprogrammers with access to greater pooled resources Consortiums dedicated to the creation and distribution of

malicious software intended to steal money fromindividuals

9/5/2012 6


6/84


What are the motives? To gain attention Financial theft (main driver of malware authors)

Identity theftWho are the victims?

Small corporations Key Individuals Basically any one

9/5/2012 7


7/84


Internal Threats Insiders acting as initiators themselves or as conduits for

other attacks User Ignorance

Malicious Intent - Intentional security breaches Disgruntled employees

9/5/2012 8


8/84


Why such Insider threats can lead to moredamage? Employees carry valid authorization and privacy of

the organizations information Dishonest insiders can exploit an organizations

vulnerabilitiesTo commit identity fraud and expose confidential information

For personal gain or organized crime Insider attacks can be more difficult to detect than

external penetration attempts

9/5/2012 9


9/84

How to protect?

There are two ways of protection mechanisms

Intrusion detection (IDS)

Intrusion prevention (IPS)

9/5/2012 10


10/84

Definitions

Intrusion A set of actions aimed to compromise the security

goals, namely Integrity, confidentiality, or availability, of a computing and

networking resource Intrusion detection The process of identifying and responding to

intrusion activities

Intrusion prevention Extension of ID with exercises of access control to

protect computers from exploitation

9/5/2012 11


11/84

2. Introduction to IDS

Intrusion detection system (IDS) A system that automatically identifying and

responding to intrusion activities

9/5/2012 12


12/84

What's Intrusion Detection good for?

Intrusion Detection Systems help to: Recognise damage and affected systems

Evaluating incidents Trace back intrusions Forensic analysis

It doesn't compensate for bad security!

9/5/2012 13


13/84

IDS Principle

Main assumption: intruder behavior differs fromlegitimate user behavior expect overlaps as shown

problems false positives:authorized useridentified asintruder

false negativesintruder notidentified asintruder

9/5/2012 14


14/84

IDS Requirements

run continually with minimal human supervision be fault tolerant resist subversion

minimal overhead on system scalable configured according to system security policies

allow dynamic reconfiguration

9/5/2012 15


15/84

Elements of Intrusion Detection

Primary assumptions: System activities are observable Normal and intrusive activities have distinct evidence

Components of intrusion detection systems: From an algorithmic perspective: Features - capture intrusion evidences Models - piece evidences together

From a system architecture perspective: Various components: audit data processor, knowledge

base, decision engine, alarm generation and responses

9/5/2012 16


16/84

Components of IntrusionDetection System

9/5/2012 17

Audit DataPreprocessor

Audit Records

Activity Data

DetectionModels

Detection Engine

Alarms

DecisionTable

Decision EngineAction/Report

system activities areobservable


17/84

Intrusion Detection Approaches

Modeling Features: evidences extracted from audit data Analysis approach: piecing the evidences together

Misuse detection (a.k.a. signature-based) Anomaly detection (a.k.a. statistical-based)

Deployment: Network-based or Host-based Network based: monitor network traffic Host based: monitor computer processes

9/5/2012 18


18/84

IDS Technologies

9/5/2012 19


19/84

Architecture of Network IDS

9/5/2012 20


20/84

Network based Technologies

Traffic analyser (e.g. Snort) Pre-processors for:

Detecting portscans

Reassembling TCP-streams Decoding RPC, HTTP, ... Detecting viruses (ClamAV plugin)

Signature based pattern matching engine: Detecting traffic pattern Detecting protocol violations (x-mas scan)

9/5/2012 21


21/84

Network based Technologies (2)

Traffic Accounting (e. g. NetFlow) NetFlow is a standardised protocol

Invented for accounting purposes Implementation:

Flow-probes and flow-collectors Implemented in routers and switches

Implementation: fprobe, flow-tools

Value for IDS: Detection of anomalies in network utilisation

9/5/2012 22


22/84

NetFlow Components

9/5/2012 23


23/84

Virtual honeypots/-nets

Honeypot = dedicated system with traps No production purpose: access to a honeypot is alwayssuspect! real honeypots costly to deploy -> virtual honeypots (e.g. Honeyd)

Emulates whole network topology (routers,switches)

Emulates hosts with identity of choice (nmap based) Scriptable fake -services Supports forwarding to real services

Supplement to qualify IDS events 9/5/2012 24


24/84

Host based Technologies (1)

Syslog Centralised logging facility for almost everything Analyzing log files tells you about:

Failed / successful logins

Access to services such as web- or mail servers Firewall (accepted / blocked packets) Creation of new users Hardware events Mounts ...

Hard to wipe out logs if logged to external system Tools for analysis: logcheck

9/5/2012 25


25/84


File-Fingerprinting Calculates and checks cryptographic hashes of

files Detect changed files Additional features (e.g. by Samhain):

Detect changed file access rights and time Creation of new files owner/group changes Deletion of files / log files

Detect kernel rootkits on Linux and FreeBSD Value for IDS: Detect manipulation of files,Remember: Everything is a file

9/5/2012 26


26/84


System integrity checks Chkrootkit

Looks for traces of known root kits Tiger

Listening processes Package database checks

Unknown files

Vulnerability checks Historical performance data

Look for anomalies

9/5/2012 27


27/84


Systrace Security layer for syscalls Can be enabled for selected processes Requested syscall has to match policy Policy manager processes syscall requests Denied syscalls will be logged Implementations

Natively included in OpenBSD and NetBSD Kernel patches for Linux and FreeBSD

RBAC (Role based access control) grsec, rsbac

9/5/2012 28


28/84

IDS ARCHITECTURE

9/5/2012 29


29/84

IDS

9/5/2012 30


30/84

Current Problems

IDS implementations not designed to co-operate Different storage formats for IDS events

Snort: MySQL, flat-files, binary files... NetFlow: sending UDP packets to collector Syslog: flat files or syslog server Samhain: MySQL, Yule, Flat-File Honeyd: flat file

Distributed data storage No common / comprehensive analysis tools (one todo it all)

9/5/2012 31


31/84

Requirements for the Ideal System

Standardised storage format

Centralised data storage

Common analysis tool

9/5/2012 32


32/84

The Intrusion Detection MessageExchange Format (IDMEF)

Problem: Sensors provide different data NIDS: IP-addresses, TCP-flags, payload HIDS: file-names, access-rights

How to store this in a general format? IDMEF is an object oriented format Reference implementation in XML

Yet another file format?

No! IDMEF is an IETF Internet Draft Undergoes evaluation to become RFCone format to store 'em all!

9/5/2012 33


33/84

IDMEF Example

9/5/2012 34


34/84

3. Introduction to IPS

Intrusion prevention system (IPS) A system that has an ambition to both detect

intrusions and manage responsive actions

9/5/2012 35


35/84

Introduction to IPS (Cont..)

Technically, an IPS contains an IDS andcombines it with preventive measures

IPS use IDS algorithms to monitor and drop/allowtraffic based on expert analysis

The firewall part of an IPS can prevent malicioustraffic from entering/exiting the network

9/5/2012 36


36/84

Basic assumptions for IPS

Basic assumptions: System activities are observable Normal and intrusive activities have distinct evidence

The goal of an IPS is to detect the difference

9/5/2012 37


37/84

How IPS Works?

The IPS monitors the network much like the IDSbut when an event occurs, it takes action basedon prescribed rules

Security administrator can define such rules sothe systems respond in the way they would

9/5/2012 38


38/84

How IPS Works? (Cont..)

9/5/2012 39


39/84

IPS ARCHITECTURE

9/5/2012 40


40/84

How IPS Works? (Cont..)

IPS can be achieved through three mainapproaches Building systems with no vulnerability

Taking perfect remediation steps to uncovervulnerabilities and patch them

Detecting the exploit attempts and blocking thembefore serious damage is done

9/5/2012 41


41/84

How IPS Protects?

IPS technologies can respond to a detectedthreat by attempting to prevent it fromsucceeding. They use several response

techniques, which can be divided into thefollowing groups The IPS stops the attack itself The IPS changes the security environment The IPS changes the attacks content

9/5/2012 42


42/84

How IPS Protects?(Cont..)

The IPS stops the attack itself Terminate the network connection or user session

that is being used for the attack

Block access to the target from the offending useraccount, IP address, or other attacker attribute. Blockall access to the targeted host, service, application, orother resource

9/5/2012 43


43/84


The IPS changes the security environment The IPS could change the configuration of other

security controls to disrupt an attack

Common examples are reconfiguring a networkdevice such as firewall, router, and switch to blockaccess from the attacker

9/5/2012 44


44/84


The IPS changes the attacks content IPS technologies can remove or replace malicious

portions of an attack to make it benign

An example is an IPS that acts as a proxy andnormalize s incoming requests and permitting thecleaned data to reach its recipient

9/5/2012 45


45/84


46/84

How IPS detects? (Cont...)

Signature-Based IPS It is the commonly used by many IPS solutions Signatures are added to the devices that identify a

pattern that the most common attacks present Thats why it is also known as pattern matching These signatures can be added, tuned, and updated

to deal with the new attacks

9/5/2012 47


47/84


Policy-based IPS It is more concerned with enforcing the security

policy of the organization

Alarms are triggered if activities are detected thatviolate the security policy coded by the organization With this type approaches security policy is written

into the IPS device

9/5/2012 48


48/84


Anomaly-Based approach IPS It is also called as profile-based It attempts to discover activity that deviates from

what an engineer defines as normal activity Anomaly-based approach can be statistical anomaly

detection and non-statistical anomaly detection The statistical approach is about the traffic patterns

on the network itself, and the non-statistical methodis about information coded by the solution vendor

9/5/2012 49


49/84

How IPS detects?(Cont...)

Protocol-analysis-based IPS It is similar to signature based approach Most signatures examines common settings, but the

protocol-analysis-based approach can do muchdeeper packet inspection and is more flexible infinding some types of attacks

9/5/2012 50


50/84

IPS Detection Techniques

Stateless Most of the network-based IDS currently available

are stateless. They typically monitor and analyze alltraffic in real-time on a packet-by-packet basis against

a database of known patters for a match

State full

A State-full IDS can be defined as a packet filteringand analysis mechanism which makes decision oncurrent packet AND information from previouspackets

9/5/2012 51


51/84

IPS Detection Techniques (Cont..)

Deep Packet Inspection Deep Packet Inspection mostly used in NIDS to look

within the application payload of a packet or trafficstream and make decisions on the significance of thatdata based on the content of that data (analyze thepacket header fields

DPI technology can be effective against bufferoverflow attacks, denial of service (DoS) attacks,sophisticated intrusions, and a small percentage ofworms that fit within a single packet

9/5/2012 52


52/84

Main Types of IPS

Scope based IPS protection (or by location) Host-Based Intrusion Prevention System (HIPS)

Network-Based Intrusion Prevention System (NIPS)

9/5/2012 53


53/84

Host Based IPS

Host-based IPS is a software program that resides onindividual systems such as servers, workstations ornotebooks

Traffic flowing into or out of that particular system isinspected and the behaviour of the applications andoperating system may be examined for indications ofan attack

These host system-specific programs or agents mayprotect just the operating system, or applicationsrunning on the host as well as web servers

9/5/2012 54


54/84

Host Based IPS (Cont..)

When an attack is detected, the Host IPS software eitherblocks the attack at the network Interface level, or issuescommands to the application or operating system to stopthe behaviour initiated by the attack

It binds closely with the operating system kernel andservices, monitoring and intercepting system calls to thekernel or APIs in order to prevent attacks as well as logthem

One potential disadvantage with this approach is that,given the necessarily tight integration with the hostoperating system, future operating system upgrades couldcause problems

9/5/2012 55


55/84

Benefits of Host IPS

Protects mobile systems from attack when attachedoutside the protected network

Prevents internal attack or misuse on deviceslocated on the same network segment, Network IPSonly provides protection for data moving betweendifferent segments

Protects against encrypted attacks where theencrypted data stream terminates at the systembeing protected

9/5/2012 56


56/84


57/84

Network Based IPS (Cont..)

NIPS has at least two network interfaces,one designated as internal and one as external

As packets appear at the either interface they are

passed to the detection engine, at which pointthe IPS device functions much as any IDS wouldin determining whether or not the packet being

examined poses a threat

9/5/2012 58


58/84


59/84

Intrusion Prevention System

IPS with two NICs configured as follows: One NIC has an IP address and handles traffic

management

Second NIC has no IP address and performs detectingattacks only

9/5/2012 60


60/84

IPS with two NICs

9/5/2012 61


61/84

IPS with inline NIDS

9/5/2012 62


62/84

IPS with scrubber

9/5/2012 63

f f k


63/84

Benefits of Network IPS

Easy deployment as a single sensor can protecthundreds on systems

A single control point for traffic can protect

thousands of systems located down stream ofthe device (no matter what the operating systemor application)

Protects against network DoS, DDos attacks andSYN flood etc

9/5/2012 64



64/84


String matching algorithms Boyer-Moore Aho-Corasic

Bloom Filter Approximated Searching Approximated Searching Based on Bloom Filters

9/5/2012 65

i i S


65/84

Intrusion Detection Systems

Three important tasks String matching : searching suspicious strings in packetpayloads

Traceback: to detect intruder who uses forged source address Detect onset of new worm without prior knowledge

The problems of current IDSs Very slow Have a high false-positive rate false positive : answering membership query positively when

member is not in the set

9/5/2012 66

S R l E l


66/84

Snort Rule Example

Snort: one of lightweight detection system, open source www.snort.org

Snort rule example:Alert tcp $BAD 80 -> $GOOD 90 \(content: perl.exe; msg: detected perl.exe;) Looking for string perl.exe contained in TCP packet from IP: $BAD, Port: 80 to IP:

$GOOD, Port: 90

Upon detection, generating alert with detected perl.exe Question: a packet coming, how to check it? Question: how about multiple rules? String matching is bottleneck

9/5/2012 67

S i S hi b f


67/84

String Searching: brute force

Arbitrary string can be anywhere in the packet Naive approach

Input: String size: m; packet size: n (assuming n >m)For i:=0 to n-m do

For j:=0 to m-1 doCompare string[j] with packet[i+j]If not equal exit the inner loop

Complexity:

worst case O(m*n) Best case O(n)

Can we do better?

9/5/2012 68

B M l


68/84

Boyer-Moore: example

9/5/2012 69

B A R N E

Improving by skipping over a larger number of character and by comparinglast character first

How to build the skip table?

B M ki bl


69/84

Boyer Moore: skip table

How far to skip when the last character does not match. For example pattern: CAB Skip: 1 * 2 3 3 Last A B C D E

Care is needed with repeated letters For example

pattern: ABBA Skip: * 1 4 4 4 Last: A B C D E

Skip[c] = distance of last occurrence of c from end inpattern

9/5/2012 70

B M l i h


70/84

Boyer Moore: algorithm

Input: pattern with size m; packet with size ni: =0While i


71/84

Aho-Corasic

9/5/2012 72

Failure pointer- Prevent restarting at top of trie when failure occurring- New attempt made by shifting

How about multiple strings?

M lti l St i T i C t ti


72/84

Multiple String Trie Construction

9/5/2012 73

Ah C i k S hi


73/84

Aho-Corasick: Searching

9/5/2012 74

Ah C i k


74/84

Aho-Corasick: summary

Pros: Computation complexity: worst case O(n) Can scan once and output all matches

Cons: Constructing a finite state machine

Failure pointers needed Too big to be on chip

Each node has maximum 256 pointers

9/5/2012 75

H hi g


75/84

Hashing

One efficient set membership query mechanism Programming trivial Query complexity: O(n) best case (n: size of packet) Query accuracy: possible false positive

However, to handle collision Each hash entry containing a list of IDs of all elements share

the hash value Storage minimal requirement: O(n*w) n: number of

elements, w: minimal width of each element

Question: can we trade accuracy for storage requirementusing hashing idea?

9/5/2012 76

Bloom Filter


76/84

Bloom Filter

Data structured proposed by Burton Bloom Randomized data structure Strings stored using multiple hash functions

(programming)

Check strings presence based on multiple bits (querying) Membership queries result in false positives Powerful tools for

Content networks

Route trace back Network measurements Intrusion Detection

9/5/2012 77

Bloom Filter Programming


77/84

Bloom Filter Programming

9/5/2012 78

Bloom Filter Querying


78/84

Bloom Filter Querying

9/5/2012 79

Bloom Filter: false positive rate


79/84

Bloom Filter: false positive rate

n: number of strings to be stored k: number of hash functions m: the size of bit array

The false positive probability f = (1/2) k

Optimal value hash functions k

K = ln2 * m/n = 0.693*m/n False positive rate decreases exponentially with

number of hash functions & memory

9/5/2012 80

Counting Bloom Filters


80/84

Counting Bloom Filters

Member deletion Deletion of a member requiring clearing all the related

bits A bit once set in the bit vector can not be deleted easily

the bit can be set by multiple members

Solution Assuming member deletion rare case

Counting bloom filter Updating counter when element added or deleted Bit reset in m-bit vector when counter value is 0

9/5/2012 81

Approximate String Searching


81/84


9/5/2012 82



82/84


9/5/2012 83

Summary


83/84

Summary

9/5/2012 84

Questions?


84/84

Questions?

Documents

Slide Cong Nghe IDS-IPS Bai5(2012951615)