SCAPM Technical Series Analytics

2013 IBM Corporation

SmartCloud Application Performance Management and Analytics

SmartCloud Analytics Log Analysis & Predictive Insights

2013 IBM Corporation 2

Please note

IBMs statements regarding its plans, direc3ons, and intent are subject to change or withdrawal without no3ce at IBMs sole discre3on. Informa3on regarding poten3al future products is intended to outline our general product direc3on and it should not be relied on in making a purchasing decision. The informa3on men3oned regarding poten3al future products is not a commitment, promise, or legal obliga3on to deliver any material, code or func3onality. Informa3on about poten3al future products may not be incorporated into any contract. The development, release, and 3ming of any future features or func3onality described for our products remains at our sole discre3on.

Performance is based on measurements and projec3ons using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considera3ons such as the amount of mul3programming in the users job stream, the I/O congura3on, the storage congura3on, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here.


Autonomic Operations

Developer Productivity

Deep Compression

pureXML

pureScale

Pervasive Content

Stream Computing

Content Analytics

Advanced Case Management

Workload Optimized Systems

Social Analytics/Consumer Insight

2013

2005

Decision Management

More than $16B in Acquisitions Since 2005

More than 10,000 Technical Professionals

More than 7,500 Dedicated Consultants

Largest Math Department in Private Industry

More than 27,000 Business Partner Certifications

IBM Investment in Analytics


Shifting market for IT Operations

APM Digest survey* of Senior IT Ops @ Fortune 500 50% growing dissatisfaction with traditional performance

management solutions for Production IT 75% of them are dissatisfied with their Business

management solutions Inability to adapt to rapidly changing applications &

workloads (Systems of Interaction) 30% of them believe that they do not have a way to

proactively detect problems Looking to operate on raw data and gain actionable

insights

IT Analytics solutions can predict, detect and help solve problems by churning through piles of data and translating this to understandable, relevant information, and actionable insights.

* Source: APMDigest: hEp://apmdigest.com/it-analy3cs-emerging-as-dissa3sfac3on-grows-with-apm-and-bsm-tools

Operational Visibility

IT Overwhelmed by data


Business Value to IT Analytics Adoption

Predic've Outage Avoidance

Ensure availability of applica3ons and services

Use learning tools to augment custom best prac3ces Leverage sta3s3cal methods to maximize predic3ve warning Use past maintenance to predict part failures

Predict

Faster Problem Resolu'on

Find & correct problems faster with tools that determine ac3ons

required to resolve issues

Iden'fy problems quicker with insight to large unstructured repositories

Isolate problems quicker by bringing relevant unstructured data into problem inves3ga3ons

Repair problems quicker with the right details quickly to hand.

Search

Op'mized Performance

Track, Op3mize, and Predict capacity and performance needs

over 3me

Track capacity and performance of applica3ons and services in classic and cloud environments Op3mize resource deployment with what-if and best t planning tools Increase u3liza3on of exis3ng assets

Perform

Improved Insight Enhance visibility into systems resource rela3onships while

increasing customer sa3sfac3on

Determine what resources are interdependent to assess impact of failures Gain insight into what is important to your customer

Decrease customer churn and acquisi3on costs while increasing customer reten3on and sa3sfac3on

Know

Lower IT Administra'on Costs with Automated Analy'cs

Escalate performance and capacity issues automa3cally, reducing manual analysis eorts Reduce manual customiza3on using learning tools that automa3cally adjust to new normals Detect and present problems with a proposed resolu3on, to be able to do more with less


IT Data Requirements - Metrics, Events and Logs

Logs

Events

Metrics

When we need to resolve problems with workloads we typically look at three types of data. Metrics - Structured performance data Events - Discrete alerts Logs - Unstructured/semi-structured

data

Metrics and events can tell you what is happening.

To answer why oSen need to look at the logs.

IT professionals need Metrics, Events and Logs to resolve IT issues.


Operations / Performance Data is Exploding

A typical enterprise with 5000 servers, running 125 applications across 2 to 3 data centers generates in excess of 1.3 TB of data per day

Only 3% of the data generated is opera3ons oriented metric data.

97% is made up of unstructured/semi structured data Workloads are running on heterogeneous plaXorms.

3%

97%

Data Ratio Metric Data Unstructured Data


Network

Systems

Security

Applications

Voice

Mainframe

Storage

Wireless

Workloads Assets

InfoSphere Streams

InfoSphere BigInsights InfoSphere Informa'on Server

more . . . IBM Watson

IT Opera'onal Insight Pack Smarter Infrastructure Insight Pack

BP / Customer driven Ecosystem

Flexible Consump9on Models SaaS Embed On Premise Digital Download

Faster problem detec'on and resolu'on Plan and op'mize Insight & Care

Failure Risk Es'ma'on and Avoidance

End Customer Client value

Search Predict Op'mize

SmartCloud Analytics Marketecture Overview

2013 IBM Corporation 9 9 9

SmartCloud Analytics

Monitoring

Solution

SmartCloud Analytics delivers end-to-end problem resolution

Add and search event data in Log Analytics

Show events in the context of Log search

Show log searches in the context of events

Alerts generated from scheduled searches

Detect and alert on anomalies based on trends observed in logs

Search logs in the context of an anomaly event

Link metrics in the context of Log search results.

Add log data into

APM and IT Dashboards

Search metric data

Integration with various types of solutions to accelerate end-to-end problem resolution and increase visibility into the IT systems*

Metrics Events

Logs, Support docs / Social data

Problem / Anomaly Detection Solutions

Event Management

Solutions Cong / Topology

Discovery and APM solutions

Refine / scope search in logs and docs using topology and configuration context

Service Desk Solution

Tickets

Search and analyze service tickets

Search events, logs, docs with ticket context

*Planned roadmap items


Capacity Trending Server, Process, Middleware and DB Trending to automatically highlight risk while there is time available to avoid outages or slow-downs Extend with SPSS Correlation for maximum confidence

Dynamic Thresholds automatically recommend and set thresholds based on attribute performance.

Event Thresholds Manage flows to highlight important alerts: -- throttle floods -- escalate threshold events that are important

Examples: Analytics for Operational Environments

Dynamic Thresholds and Trending are built-in to Tivoli Monitoring for immediate value, and significantly reduce Set-up and Administration of Monitoring Environments


Example: Capacity Analytics for Cloud & Virtualized Infrastructure Provides visibility of how Resources are Allocated to Applications and Services

Cloud Consumers conservative in estimating their system needs (Over / Under Estimate) Understanding Historical behavior helps optimize Capital Management

Know what Resources are Available and Predict how they will be Used Maintain awareness of total and available capacity Predict physical and virtual resource capacity bottlenecks Gain business agility by determining room for expansion via what-if analysis

Optimize Resource Allocation Right-size virtual machines Policy-driven workload placement for

performance and security optimization

Visibility into the cloud infrastructure See and Manage all major Hypervisor

environments from one place Leverage Perspec3ve from the Past to the

Future to Ensure Resource availability


IBM SmartCloud Analytics Log Analysis

Search, and Index unstructured data to provide consolidated view

Built on IBMs Big Data platform Integrate structured and

unstructured data for better problem identification and resolution

Extensible, with IBM and partner expertise built-in Get the last critical piece of data for identifying, isolating,

and correcting problems faster

Faster Problem Resolu'on

Find & correct problems faster with tools that determine ac3ons

required to resolve issues

Iden'fy problems quicker with insight to large unstructured repositories

Isolate problems quicker by bringing relevant unstructured data into problem inves3ga3ons

Repair problems quicker with the right details quickly to hand.

Search

Delivers Problem Isolation and Faster Problem Resolution


Collects large volumes of obscure unstructured data and transforms it through analytics into actionable intelligence.

IBM SmartCloud Analytics Log Analysis

GBs of

Obscure Log Files

Single Actionable Dashboard

Intelligent Support Docs

Integration through

Advanced Text Analytics

Insight Packs


IBM SmartCloud Analy'cs Log Analysis helps IT Generalists and Applica'on Specialists accelerate problem resolu'on through rapid analysis of unstructured data

IBM SmartCloud Analytics - Log Analysis Client Value

Faster Problem Repair -By linking expert knowledge to log error/warning messages

Improved Service Availability and Maintainability of Custom Apps

- Provide users with advanced insights into custom applica'ons quickly

Collec'on and Annota'on of data

Generic Logs Support Federa'on of Data

Faster Problem Identification and Isola'on

Quickly search structured and unstructured data. Perform cross domain analysis on this data.

Value Highlights

Advanced Text Analy'cs Downloadable insight packs on the ISM Library star'ng with WebSphere and DB2

Tools to create custom insight packs for your own applica'ons


SmartCloud Analytics Roadmap Overview

SmartCloud Analy'cs

Log Analysis v1.1

Workgroup Edi'on

Release Key Dates Key Capabili'es

Fast to install and download

Data collec3on, annota3on and indexing

Search UI WAS & DB2 insight

packs Generic log support Insight pack tooling

Enterprise scalability* Integra3on with Tivoli Monitoring and Event Management Solu3ons Addi3onal Content and Tooling Logstash support for data collec3on

June 2013

Q4 2013

SmartClould Analy'cs

Log Analysis V.Next

Enterprise Edi'on

SmartCloud Analy'cs

Log Analysis V.Next

Workgroup Edi'on

* Enterprise scalability only on Enterprise Edi3on


Linking Information

Linking of search results with structured data Supports linking indexed data with federated sources Plan to provide out of the box linkages with key Tivoli/IBM products


Expert Guidance

Provides Expert advice by searching support docs E.g. when there is an error message found in a log file, search in support documentation for relevant

information on further explanation and/or fix.


A Healthcare Provider reduces 3me to diagnose system problems by providing a holis3c view of all relevant data

Need

Have too many tools across structured and unstructured datasets making problem resolu3on dicult and 3me consuming

Desired a solu3on to 3me-correlate a view into many sources of data to perform problem detec3on, isola3on and repair

Benefits Reduced time to determine root cause of

problems by leveraging performance, event and log data

Skills required to diagnose problems were easily saved and repeated to reduce overall costs


90-Day Free Trial Available

hVp://www-01.ibm.com/soZware/'voli/products/log-analysis

20

Few companies are genuinely proactive or preventative Most organization react to service outages in progress

Diagnosis can be complicated by organizational silos, disparate tools, complexity and the sheer volume of data.

Outages and degradation can cost millions of dollars,

impact brand, customer churn & retention

CxOs are challenging their management teams to prevent outages rather than just reacting after failure.

Proactive, Predictive and Preventative Management

Why arent operations teams preventative today?

- Too much data to analyze manually - Existing analytic techniques, such as standard thresholds, are not up to the task - They cannot detect problems while they are emerging (before business impact) - Set threshold too high, insufficient warning before total failure. - Set threshold too low, too much noise, everything is ignored

21

SmartCloud Analytics Predictive Insights

Proactive and self-learning Performance intelligence

Real-time analytics for detecting and avoiding service disruption

Uses advanced Watson research algorithms

Correlates metrics across multiple domains and heterogeneous data sources.

Leverages IBM Big Data technology

Embeds InfoSphere Streams, IBMs unique streaming analytic engine

Enables ultra-high scalability commodity server computing clusters and large algorithm sizes to maximize machine intelligence value

Leverages InfoSphere Datastage, IBMs market leading mediation solution

Quickly integrate to any monitoring source using a large library of out-of-the-box connectors

Leverages your Tivoli and non-Tivoli environments

22

Predictive Anomaly Detection using Behavior Learning Automated problem detection Learn the environment through statistical analysis & correlation Predict problems with high confidence based on changes in metric behavior

Augment manually applied thresholds by noticing when metrics behave deviates from normal behavior Watch related groups of metrics for additional insight and maximum predictive warning

Discover and Group Resources based on behavior

Augments service and application modeling

Powered by IBM BigData and Data Mediation for rapid delivery, performance & scalability

23

Example Scenario: Internet Banking Application

Goal: Automatically learn normal mathematical relationships between metrics Web Response Time

WRT Bad

WRT Good

User Requests

Time

Web Response Time

Anomaly Event Business Impacted

Early Warning

Learns Web Response Time has a normal causal relationship with User Requests - WRT gets slower as user load gets higher.

If this healthy historical relationship breaks down, say due to a memory leak, an anomaly is raised immediately

The problem is detected even while WRT service is good

Emerging problems can be detected even while service levels are good in absolute terms

Core Banking Application

z/OS

ESB

AIX

Java / WAS

RHEL

Oracle

Windows

Application

Internet Banking

24

Correlation of Multiple Metrics Statistical models can discover mathematical relationships between metrics

The extent this can be achieved depends on a number of factors, such as: range and type of data, availability of data, and stability of environment. Analytics falls back to a single metric if metrics are unrelated.

Core Banking Application

z/OS

ESB

AIX

Java / WAS

RHEL

Oracle

Windows

Application

Internet Banking

G

I

B

D

C

E

F

H

A

Internet Banking

25

Multiple Metrics Analysis - Value of this approach

Learns normal operational behaviour across the infrastructure, including how metrics behave together.

Maximize Advance Warning: Identifies metric relationship changes that signal a problem long before traditional thresholds

Identifies problems before you know to look for them

Detects service impacts that are not identifiable by fixed thresholds alone.

Assists with root cause analysis by indicating the most offending metrics.

Reduces expensive and time consuming false alerts.

Provides a more intelligent real-time assessment of data, able to detect

problems as they are emerging

26

Large retail bank increased online banking application availability through predictive analytics

Need Ensure critical retail banking applications

were online 24x7 for high customer satisfaction

Proactive anomaly detection was needed to ensure adequate time to resolve major incidents before they became service impacting

Benefits Alerted 10 major incidents in a 4 week period

in advance of customer detection

Simultaneously monitors over 80 servers and 40K metrics

Estimated savings with outage avoidance analytics was $600K for this 4 week period

26

27

Example: Field Trial at Large Retail Bank

Results: 15 Major Incidents reported during the 4 week trial period 10 major incident were detected or predicted by SCA-Predictive Inisights 5 missed incidents were application code problems and not manifest in health metrics 100% of detectable problems detected Prediction & Detection Intervals: Report included a Problem Start Time, a Problem Detection Time and Problem Resolution Time 6 out of the 10 detected incidents were predicted before the customers Problem Start Time All 10 out of 10 detected problem were detected before or around the customers Problem Detection Time interval Results for this Customer

Using industry average outage costs, potential outage avoidance savings for 4 weeks: $600k Event reduction savings for 4 weeks: $53k

Retail Bank experiencing severe problems with their online banking applica3on Trial Scope: Online banking service with back end application ITM AIX, Linux, Windows, ITCAM for WAS, ITCAM for WRT

~80 servers ~40k metrics


Backup

29

Solution Architecture - Mediation

Mediation IBM InfoSphere Datastage

Analytic Engine IBM InfoSphere Streams

Analytic Application

Post-Processing Rules Uses OMNIbus Rule Engine

Anomaly Consolidation

User Interface & Management Tivoli Integrated Portal

Predictive Insights Market leading mediation - provided as option

Proven rapid integration to new data sources.

Productivity tooling & collaboration included

High performance and scalability.

Large framework of connectors.

Fast integration to common monitoring data formats.

Windows based development environment

30

Mediation Rapid Common Extraction

Predictive Insights provides a quick setup Common Extractor feature that allows fast extraction from the most common interface types such as:

- CSV

- Databases and database connectors, e.g. JDBC

Monitoring Suites Interface Implemented in trials

HP Sitescope JDBC Yes

Quest Foglight Script dump to CSV Yes

CA Wily Introscope JDBC Yes

IBM ITM TDW DB2 Yes

IBM TDW Proxy Agent (low lat) CSV Yes

IBM ITCAM TDW DB2 Yes

Compuware VAM Script dump to CSV Yes

HP Mercury BAC JDBC Yes

IBM Performance Manager CSV Yes

Brix CSV Yes

IBM Service Quality Manager CSV Yes

Other extractions can be quickly built from a large library of Datastage connectors

31

Media3on Connector Library InfoSphere DataStage

RDBMS!DB2 (on Z, I, P or X series)"Oracle"Informix (IDS and XPS)"Ingres"Netezza"Progress"RDB"RedBrick"SQL/DS"SQL Server"Sybase (ASE & IQ)"Teradata"Universe"UniData"NonStop SQL"InfoSphere Federation Server"InfoSphere Classic Federation"And more.."

General Access "Sequential File"Complex Flat File"File Set"Data Set"Named Pipe"iWay"FTP"SFTP "Compressed / Encoded Data"External Command Call"Parallel/wrapped 3rd party apps"EMC InfoMover"Web logs"Email"

Enterprise Applications!JDE/PeopleSoft OneWorld "Oracle Applications"PeopleSoft"SAS"SAP BW"SAP R/3"Siebel"Ariba"Manugistics"I2"Etc"

Standards & Real Time !WebSphere MQ"Java Messaging Services (JMS)"Java"XML & XSL-T"EBXML"Web Services (SOAP)"Enterprise Java Beans (EJB)"EDI"FIX"SWIFT"HIPAA"

CDC!DB2 (on Z, I, P, X series)"Oracle"SQL Server"Sybase"Informix"IMS"VSAM"ADABAS"IDMS"Datacom""

Legacy!Allbase/SQL"C-ISAM"D-ISAM"Datacom/DB"DS Mumps"Enscribe"Essbase"FOCUS"IDMS/SQL"ImageSQL"Infoman"KSAM"M204"MS Analysis"Nomad"Nucleus"RMS S2000"Supra"TOTAL"TurboImage"Unify"And many more."

32

Solution Architecture Analytic Engine







Predictive Insights

Real-time streaming analytic engine, provided as a component

High volume and low latency.

Supports server clustering and redundancy (next rel)

Enables large algorithm capacity 80,000 metrics in a single algorithm instance (a typical banking application produces ~30,000 - 60,000 metrics)

Allows multiple algorithm instances spread across commodity server computing clusters, making maximum advantage of multi-core parallelism (next rel)

33

Solution Architecture Analytics






User Interface & Management Tivoli Integrated Portal Automated anomaly detection and prediction on

time-series performance metrics Behavioural learning to model not only one metric at a time, but the relationships between them for anomaly detection...

Single metric evaluation replacing many manual thresholds for any time series data

Multiple metric correlation enabling earlier detection than traditional thresholds with higher confidence

Predictive Insights

34

Solution Architecture Anomaly Consolidation






User Interface & Management Tivoli Integrated Portal Targeted for next release, the alarm consolidation

framework reduces the events that are presented externally allowing for efficient processing and accurate alerts.

Different techniques will be selectable depending on the richness of the data processed.

It reduces the volume of external alarms forwarded to event consoles or application/domain administrators, without removing any information that could be useful in prediction, detection or RCA.

Internal Events External Events

UV: Node A: Metric 1 EXT: Node A, B, C, Metric 1, 2, 3

UV: Node B: Metric 2 EXT: Node M, Metric 47

U V: Node C: Metric 3

MV: Node B, C: Metric 2, 3

MV Node A, B, C: Metric 1, 3, 2

UV: Node M: Metric 47

Predictive Insights

35

Solution Architecture Anomaly Post Processing







The post-processing engine allows anomaly events to modified, customized, or enriched

It can an be optionally used to put some business/domain context around the domain agnostic analytic anomaly events. Typically this will be used to re-prioritize anomaly severity to major if it is service impacting.

For example, if Metric 2 represents Online

Banking Web Response Time, then the anomaly severity can be changed to Major.

This reuses OMNIbus Probe rules libraries, but is dependent on having a northbound OMNIbus object server to receive the anomaly events.

Internal Events External Events

UV: Node A: Metric 1 EXT: Node A, B, C, Metric 1, 2, 3

UV: Node B: Metric 2 EXT: Node M, Metric 47

U V: Node C: Metric 3

MV: Node B, C: Metric 2, 3

MV Node A, B, C: Metric 1, 3, 2

UV: Node M: Metric 47

Predictive Insights

36

Solution Architecture User Interface & Management







TIP based anomaly visualization

Allow all anomalous metric to be visualized together

Normalizes metric scales, and allows, pan/zoon etc, so that anomalous conditions are more readily apparent.

In-context linking between OMNIbus, TBSM, ITMM AEL and anomaly charts

Inherits all TIP features for unified user management and permissions.

Predictive Insights


IBM SmartCloud Analytics - Differentiators

Expert Advice: Out of the box IBM exper3se based on advanced text analy3cs

Easily Extendable: For customers, OEM or value-added reseller to add their own Insight Packs

Integra3on: Integra3on with Tivoli products

Licensing Model: Based on average data consump3on not your worst day

Data Federa3on: Easily link structured and unstructured data

Mul3 Source: Ingests metrics, cong, events, logs, traces, and topology to perform RCA not just logs

PlaXorm: Core technology based on IBMs Big Data PlaXorm poten3al to use a common Big Data plaXorm for the en3re business

Text Analy3cs: Only product in industry with advanced text analy3cs engine able to extract insights from unstructured sources like service 3ckets & support documents


IBM SmartCloud Analytics - Differentiators

End to end Solu3on: End to end solu3on to predict, analyze and resolve problems, not just a point solu3on that analyzes a narrow spectrum of data

Anomaly Detec3on: Anomaly detec3on and predic3on on logs and metrics - helps users know what to search for and what is trending


User Scenarios

39


C

Applica'on owner/Support Engineer

Applica'on developer

Opera'ons Teams

Targeted Users

Analy'c Content Creator

Develops and tests large distributed mul3-component applica3ons on a middleware stack. Debugs applica3on during development.

Domain experts build content and leverage available support

documenta9on

IT Opera'ons

Con3nuously monitors for anomalies across the en3re infrastructure. When an anomaly is detected, this user routes the problem to the right team.

Supports a business applica3on that is built on middleware. Needs to solve problems quickly to avoid any business impact. Needs to capture best prac3ces in a tool so that diagnosis is less dependent on skills availability.


A developer validates a complex distributed application by running tests overnight. He wants to know which points in his application are causing exceptions through detailed analysis of the stack traces

Example 1

He uses text analy3cs of complex logs such as stack traces to nd paEerns Searches for frequent problems and paEerns to decide which por3on of the code requires aEen3on.


Users of healthcare application are facing problems. Its taking too much time to view patient data and at times they see errors on their browser. They complain to customer support

Example 2

The customer support engineer creates a 3cket. The applica3on support engineer now needs to nd the root cause of the problem.

The applica3on is distributed over mul3ple nodes and involves various middleware and legacy technologies. It generates dierent types and large volumes of metric and log data.

The support engineer searches the applica3on logs to locate a period when transac3ons were slow for a specic set of users.

Using problem context, he searches expert advice for solu3ons


Product Capabilities

43


Key Product Capabilities to Meet Client Needs

Mul3ple op3ons to upload log data

Search a massive amount of log les

Link structured data with unstructured data

Expert guidance

Analy3cs for trends and anomaly detec3on

Ability to build insight packs for domain/applica3on


Multiple Options to Upload Log Data

Business Users

Applica'on/system

Applica3on Components

Log Analy'cs Server

Push logs (Log File Agent, REST interface)

Pull logs using remote monitoring (agent less op3on)

App Developer/ IT Ops Engineer


Searching Information

Log le

[10/9/12 5:51:38:295 GMT+05:30] 0000006a servlet E com.ibm.ws.webcontainer.servlet.ServletWrapper service SRVE0068E: Uncaught exception created in one of the service methods of the servlet TradeAppServlet in application DayTrader2-EE5. Exception created : javax.servlet.ServletException: TradeServletAction.doSell(...) exception selling holding 3111 for user =uid:43 at org.apache.geronimo.samples.daytrader.web.TradeServletAction.doSell(TradeServletAction.java:708)

Log Analy'cs Server


Text Analytics on Logs

Leverages Market leading Text Analytics solution

Need to extract what are errors and class names from WebSphere logs

Informa3on extrac3on may involve complex context sensi3ve grammar that need processing beyond simple regular expression parsing.

Follows immediately

Within a single log record

Developer: Which are the top 10 Java classes that have most errors ?

[7/25/12 8:27:09:391 CEST] 00000028 E com.ibm.bpe.u3l.Assert.asser3on


Sample App Dashboard


Example: Metadata Extraction from Tech Note


Ability to Build Insight Packs

[07/25/12 02:38:25:295 GMT+05:30] 00000010 TraceResponse E DSRA1120E: Application did not explicitly close all handles to this Connection. Connection cannot be pooled.

[07/25/12 02:38:25:295 GMT+05:30] 00000010 TraceResponse E DSRA1120E: Application did not explicitly close all handles to this Connection. Connection cannot be pooled.

Provides ability to build / deliver client, industry or scenario specic use cases. Insight Pack contains

informa3on such as

How to interpret log data ?

How to link log data with metric data and applica3on data

What to search in log les?

What are the sources of expert advice?

2013 IBM Corporation 51 2013 IBM Corporation 51

Documents

SCAPM Technical Series Analytics