2012 02-14-digitale fabriek v - Een praktische aanpak voor implementatie van nieuwe services - Johan Van Noten, FMTC

Digitale Fabriek (deel 5)

A practical approach for the implementation of new services

2012-02-14

© FMTC vzw 2012 • STRICTLY CONFIDENTIAL • p2

Background

+ Flanders’ Mechatronics Technology Centre Partners: machine builders like Atlas Copco, Bekaert, CNH, Dana… Concrete joint research projects Field of Mechatronics

Mechanics Electronics Software

We do not develop commercial products+ Cooperation with KULeuven – Sirris+ Johan Van Noten

Software background a.o. project concerning log analysis


Problem Statement

+ GoalOffer new services based on machine / product usage info

+ New service kinds Reporting Diagnostics / error alerts Maintenance scheduling Prognostics Error analysis Usage advice towards customer Market analysis …

+ All you need is data…


Challenge 1: Is my data available? (1/2)

+ Usage data Logs are produced on the machine by the machine software. Is this exactly what I need? Could happen, but probably not.

+ Can I ask my developer to produce the exact data? Too late:

machine’s software is already in the field. Wrong focus:

data collection and processing is not the developer’s task Too complex:

mixing functionality and data processing complexifies software Impossible:

software could be 3rd party, you can’t control its features

Don’t count on “readily available processed data”.


Challenge 1: Is my data available? (2/2)

+ So, you’re stuck with low-level log files Created by the developer based on passed experience Partially for debugging, partially for reporting Reflects execution flow of the machine Potentially multiple sets of data (corresponding to machine parts)

+ Does this match my desired data? Nope…

Way too detailedlog data

Desired data(statistics or other)

?Derivable?


Originaldata

Analyseddata

Originaldata

Analyseddata

OfficeMachine

Challenge 2: Where to interpret my data?

Originaldata

Analyseddata

Originaldata

Analyseddata

Originaldata Analysed.

data.

Reducedtransfer needs


Challenge 3: How to interpret my data?

+ Difficult? Depends on your needs…+ Easy steps

Filtering Statistics Historical data querying Traditional techniques apply

+ Typically more challenging Time based relationships:

“Did this error occur within two minutes after system startup?” Sliding window queries:

“Did the user push this button when the machine’s rpm exceeded 600 for at least 2 minutes?”

ESP & CEP can be the answer


ESP & CEP: Intro

+ ESP = Event Stream Processing+ CEP = Complex Event Processing+ Background

Financial fraud detection Network intrusion detection Monitoring of complex application servers

+ Strengths Can do the usual filtering and statistics Strong in time relationships

“A followed by B within 10s” “All occurances of A between B and C where C occurs < 5’ after B”

Works on ethernal streams “Mean value of A during the last 5 seconds”


ESP & CEP: How does it work? (1/2)

+ First some definitions Event

Happens at a given time Carries some information about what happened

Pattern Potentially complex relationship between events

Pattern matching Looking for occurrences of given patterns in a data set

+ Working principle You define a set of patterns that express your needs. Data flows through a set of existing queries

(≠ SQL: query works on set of existing data)


ESP & CEP: How does it work? (2/2)

+ Trivial data filtering Events can be filtered on their contents

“select id, pressure from [every ValveEvent(pressure > 5) ]”Matches each time such a ValveEvent appears on the stream.

Considers events one by one (no relationships) Good for data reduction, selection of interesting events

+ Complex Event Processing (CEP)When a CEP pattern matches, this match can be considered as the occurence of a “higher order event”. Relates multiple events by time, causality, origin… “every a=ValveEvent -> b=ValveEvent(pressure > 2*a.pressure)

where timer.within(3s)]” Each match is considered as

a new event of newly created stream “SuddenPressureIncrease”.


Pattern 1

ESP & CEP: Raising the abstraction

+ Raising abstraction + detect errors Initial log file contains low-level events Based on CEP patterns, higher level events can be derived These higher level events can then be re-used in other CEP patterns.

Pattern 2


ESP & CEP: Example

+ Example: Robot with moving arm Initial log stream contains robot arm movements:

ArmDockedInBase, ArmMovesUp, ArmMovesDown, ArmTurns Pattern 1: Raising abstraction

new derived event kind RobotAction =“ArmDockedInBase followed by any number of ArmMovesUp, ArmMovesDown, ArmTurns followed by ArmDockedInBase”

Each RobotAction contains: total execution time , total length traveled. Pattern 2: Detecting error

Selects some of the newly generated RobotAction events:e.g. select * from RobotAction where totalLength/totalTime > 2m/s

The selected events indicate a usage of the robot over specification.

Second query would be more complex without the first one.


Overview

+ Problem Statement+ Challenges

1. Is my data available? You only have low-level logs2. Where to analyze my data? Locally on machine or at office3. How to analyze my data? ESP & CEP make it easier

LogAn tool + concrete usage examples


LogAn tool

+ FMTC created two products LogAn Studio

Definition of patterns Off-line matching of the patterns Graphical analysis of the results

LogAn Matcher Online matching of the patterns

+ Based on ESP & CEP implementation: Esper By EsperTech (http://www.espertech.com)

+ Note: next examples mention LogAn, but principle is valid for all ESP & CEP.

http://www.espertech.com/�


Concrete usage examples

+ Usage 1: Helpdesk improvement Problem: Log files not optimally used

Content too detailed for helpdesk agents Escalation to developers needed disturbing

Solution Developer analyses occurred issue through LogAn Developer adds expert knowledge as a pattern to LogAnLogAn knows patterns for all kinds of issues

Helpdesk agent offers new log file to LogAn LogAn immediately shows occurrences of previously identified patterns



+ Usage 2: Log restructuring / documentation Problem: Logs difficult to understand

Log is flat list of events Too detailed / not clearly structured Even developer does not easily navigate

Solution Developer once adds patterns that clarify structureHuge production log gets structured in production day, batch, job, task…

New log can now easily navigate through detected structure



+ Usage 3: Online monitoring Problem: Associate alarms with complex events

Alarms trigger on complex events Difficult to program in traditional ways

Solution On-line LogAn Matcher post-processes the machine’s output Same patterns as off-line Can be configured to produce the desired info

without touching the machine’s core code. Can fire locally (alerts, local optimization…)



+ Usage 4: Online filtering Problem: What data should be sent to backoffice?

Huge log-file data is too big to be sent It can not be determined up-front what needs to be sent

Solution Dynamically define which (processed) data needs to be sent to back-office~ Technically similar to previous example.

+ Usage 5: Pre-process data for analysis Problem: Log data is too detailed for traditional analysis methods

Data mining, statistics etc require desired data objects Logs only contain them in a derivable way Deriving them with classic techniques is difficult

Solution: Use ESP & CEP to pre-process the data to higher level data objects


Overview

+ Problem Statement+ Challenges

1. Is my data available? You only have low-level logs2. Where to analyze my data? Locally on machine or at office3. How to analyze my data? ESP & CEP make it easier

+ LogAn tool+ Concrete usage examples

Limitations & Future abilities


Limitations

+ ESP & CEP Defining patterns quickly becomes complex.

Knowing the semantics is important. Not suitable for sporadic use

Highly scalable & computationally intensive.Also used for server applications. Not suitable for embedded environments.Requires PC alike environment

+ Esper core is GPLv2 Gnu Public License v2 will be inherited by your software. If required, make sure you have a commercial license.


Future abilities of LogAn

+ Integration of pattern mining Today, patterns are defined manually.

Expert “adds” his knowledge to the tool by means of patterns. Sometimes no-one knows which pattern to look for. Algorithms exist to do pattern mining:

Reveal certain patterns that seem to occur in your data. LogAn will assist you in preparing your data & mining patterns.

+ Assisted definition of patterns FMTC wants to reduce the complexity of the pattern definition This reduces the abilities as well Full capabilities will be kept open but easy scenarios will be offered.


Conclusion

+ Machine does NOT produce the desired information Too detailed for direct use Too big to send to back-office

+ ESP & CEP techniques allow you to Powerfully filter your data Define complex patterns that relate events

+ Typical uses Increase the abstraction level Easier next level patterns Match on complex error patterns Powerful alerting Pre-process data to reduce amount Send relevant info only Pre-process data for analysis Determine analysable data


Contact

Johan Van NotenProject Leader

[email protected]+32 498 91 94 03

Flanders’ Mechatronics Technology Centre vzwCelestijnenlaan 300D

3001 Heverlee

mailto:[email protected]�