Trends in

in Business Intelligence

Studie en Advies Johan Blomme

Data Consulting Services www.the-new-bi.be

Transformational changes that take place in the digital world definitely

change the nature of business intelligence and represent a new normal.

The Internet is the societal operating system of the 21st century and its

underlying infrastructure – the cloud computing model – represents a

« disruptive » change. A networked infrastructure, big data from disparate

sources and social media among other trends as predictive analytics, the

self-service model and collaboration are changing the way BI systems are

deployed and used.

2

Trends in

BI

Introduction

3

• In today’s marketplace, change is a constant.

• Products are increasingly commoditised, development cycles have shortened and expectations

of consumers are rising. To achieve a sustainable competitive position, companies must

react in an agile way to changing market conditions.

• The current business environment evolves from a transition towards globalization and a

restructuration of the economic order. The pace of technological changes that allow instant

connectivity and the current era of ubiquitous computing that resulted from it, represent

« the new normal in business intelligence».

4

• As an industry, business intelligence has to adapt to environmental changes.

• The evolution of the Internet as a new societal operating system, reshapes the future of

business intelligence.

• The Internet evolves as a platform for the use of interoperable resources (storage,

computing, applications and services) and drives the development of information intensive

services in the 21st century. Increasingly, the cloud becomes the vehicle for the Internet of

Services.

• The business ecosystem generates a huge amount of data in terms of volume, variety and

velocity, and requires businesses to take on a data-driven approach to differentiate. It’s about

gaining actionable insights faster than the competition by reducing the data-to-decision gap.

• This highlights the integration of structured and unstructured data (esp. social media content)

to derive actionable insights from « big data » and the leverage of predictive analytics for

agile decision-making.

5

• The exponential growth of data and the increased reliance on insights derived from data for

decision-making, causes a shift in the focus of business intelligence. BI is more than an IT-

function and is about people and business decisions.

• Therefore, the emphasis of next-generation BI should be on designing solutions that focus on

answering business questions of the end user. In the field of BI the finished product is not a

dashboard displaying metrics but actionable intelligence answering the business question at

hand. Users want seamless access to information to support decision-making in their day-to-

day activities.

• The future direction of BI will thereby be shaped by the new age of computing. In both their

personal and professional lives, Web-savvy users have adopted the principles of interactive

computing and have come to demand customizable BI-tools with high responsiveness.

Business intelligence, and the insights it delivers, evolves towards an enterprise service that

follows the lines of a self-service model with business users producing their own reports in an

interactive way and performing analytics on demand.

6

• Furthermore, Web 2.0 and social networks function as catalysts for highly intuitive user

interfaces and the collaborative features of computing allow users to share insights, which

transforms BI from a solitary to a collaborative activity.

• Companies are exploring the connection between analytical activity and knowledge sharing.

Combined with collaborative technologies that « crowdsource » intelligence from various

partners of the extended enterprise, this approach provides the context for better and faster

decision-making.

7

The factors that constitute the new normal in BI can be summarised as follows :

The Future Internet

Big Data

Cloud Computing

Embedded BI

User Empowerment / Self-Service BI

Predictive Analytics

Social Media Analytics

Collaborative BI

8

Trends in

BI

1. The Future Internet

9

Trends in

BI

• The main objective of enterprise computing is to be adaptive to change.

• The new generation of enterprise computing must enable pervasive BI deployments :

– spreading BI to more users and more devices :

• consumerization of IT : enterprise computing aligns with consumer-class technologies ;

• BI-tools are more and more organized around the user’s experience to interactively discover hidden relationships, trends and patterns and to create new information and relate it with external data sources ;

– using multiple data sources : the use of structured as well as semi- and unstructured data sources (e.g. social media content) extends the playing field of BI.

10

• The new generation of enterprise computing needs to be developed within the perspective of

the future Internet :

– the Internet as data source :

• BI applications no longer limit their analysis to data inside the company and increasingly source their

data from the Internet to provide richer insights into the dynamics of today’s business ;

– the Internet as software platform :

• BI applications are moving from company-internal systems to service-based platforms on the

Internet.

11

INTERNET-ENABLED

IT-INFRASTRUCTURENEXT-GENERATION ENTERPRISE COMPUTING

Web-based technologies enable

the implementation of user-configurable

BI applications connecting to a wide

arrangement of data

BI-applications are delivered

as a service on the Web or

hosted in the cloud

12

• The Internet of the future gives rise to a new

business model that allows enterprises to form

business networks :

– in the knowledge economy economic activity is

based on highly networked interactions ;

– the amount of digital collaboration is increasing

among people, things and their interactions

(through the Internet of People and the Internet

of Things, networking is expanding not only in

person-to-person interactions, but also in

person-to-machine and machine-to-machine

interactions).

The

Future

Internet

Business

Networks

Inte

rnet o

f

Serv

ices

Big

Dat

a

13

Cloud ComputingBandwidth

& Connectivity

The Web as Expanding

Ecosystem for

Business Exchanges

Wo

rkfo

rce

Dem

og

rap

hic

Sh

ifts

Drivers of

NETWORKED

INFRASTRUCTURE

Dev

ice-In

dep

en

den

t

Info

rmatio

n A

ccess

Hyper Adoption of

Social Networking Technology

Collaborative

Technologies

The Consumerizationof IT

Globalization

14

• Business networks take on a data-driven approach to differentiate and apply fact-based decision-making enabled by advanced analytics:

– economic interactions are based on the principle of scarcity and in the knowledge economy the concept of scarcity applies to information ;

– information in itself does not create competitive advantage (access to lots of information has already become ubiquitous) ; competitive advantage is defined as access to information, the decisions based on that information and the actions taken on these decisions ;

– business networks manage data in real-time, support anywhere, anytime and any device connectivity and provide the appropriate information to users across and beyond the enterprise (business users, partners, suppliers, customers).

The

Future

Internet

Business

Networks

Inte

rnet o

f

Serv

ices

Big

Dat

a

15

• The Internet serves as a platform for a service-oriented approach that changes the way of enterprise computing. With BI-applications moving to the web, the Internet emerges as a global SOA that is referred to as an Internet of Services. The IoS serves as the basis for business networks.

• The new BI requires technologies that integrate multiple data sources, address business needs in a dynamic way and have a short time to deployment.

• Contrary to large scale application development of traditional BI, the new BI moves towards smaller and flexible applications that can adopt quickly and are supported by a service-oriented architecture.

The

Future

Internet

Business

Networks

Inte

rnet o

f

Serv

ices

Big

Dat

a

16

• SOA is an architecture whereby business applications use a set of loosely coupled and reusable

services that can be accessed on a network.

• Often implemented by Web services, a SOA is a building block for flexible access to multiple

data sources and the very nature of services that can be reused and integrated with each other

allows business processes to be adopted in an agile way to adjust to changing market conditions

and to meet customer demands.

• With cloud computing, this service model is delivered on demand. The delivery model is no

longer installed software but services.

17

User empowerment / Self-service

Users expect to have access to

business information in the same way

as they use the Internet and search

the Web. Self-service BI is the

implementation of this service-

orientation at the end-user level.

Embedded BI

BI moves into the context of business

processes and transforms from a

reactive to a proactive decision-

making tool by monitoring

performance and the prediction of

future events. This change in the use

and delivery of software is guided by

the adoption of a service-oriented

approach.

Cloud computing

Cloud computing emerges as a new

deployment model of BI by the

adoption of a service-oriented

architecture and drives a

transformation in application

architectures through using “the Web

as a platform” for interoperable

applications and services.

Internet of Services and BI

18

2. Big Data

19

Trends in

BI

20

VOLUME

VARIETY

VELOCITY

21

Major sources of « big data »

22

producer generated content user generated content. system generated content

Static WebDesktop/PC era

Semantic Web

Social Web

Internet of People Internet of People and Things

The Internet

The Web

The Cloud

time

Data

3V

The evolution of the Internet and the proliferation of data

23

• As connectivity reaches more and more devices, the volume, variety and velocity of data from

clickstreams, social networks and the Internet of Things (through which the physical world itself

becomes an information system) creates a new economy of data.

• Traditionally, BI applications allow users to acquire knowledge from company-internal data

through various technologies (data warehousing, OLAP, data mining). However, the typical

pattern of cleaning and normalizing proprietary information through an ETL process into a data

warehouse is challenged by the transition to big data that is marked by greater accessibility,

interoperability and 3rd party leverage of online data.

• For businesses to become responsive to market conditions, it is necessary to look at the whole

ecosystem by connecting internal business data with external information systems. BI-

applications must access data from disparate sources inside and outside the firewall, consider

qualitative and quantitative data and include structured as well as semi-structured and

unstructured data.

24

• Data from the Web is feeding BI applications :

– BI applications no longer limit their analysis to data inside the company, but also source data from the

outside, especially data from the Web. The Web is a data repository.

– An important challenge is the extraction, integration and analysis from hererogeneous data sources.

• BI applications move to the Web :

– BI applications are increasingly accessible over the Web : BI is consumed as a service from the cloud.

– The challenge here is the development of Web-based applications that access and analyze both historical

enterprise data and real-time data, especially from the world wide market and making the information

available on a variety of devices.

25

Variety

Velocity

Volume Services

Database Technology

Analytics

The 3 V’s represent the common

dimensions of big data, but the real

challenge lies in extracting actionable

insights from it.

The increasing volume and complexity of data

has forced organizations to look at new data

management and analytic tools to optimize

performance, improve service delivery and

discover new opportunities.

26

• Heterogenous datasets are no longer manageable by a traditional relational database approach.

• Requirements for next-generation BI-tools include :

– connect directly to the underlying data sources to capture distributed data ;

– schema-free : relationships between data are discovered dynamically ;

– anytime, anywhere access with multiple devices ;

– real-time visibility of what is happening now is needed and analytics must be used in the stream of

business operations.

27

• New approaches such as in-database analytics, massive parallel processing, columnar databases

and « No SQL » will increasingly be used for the analysis of structured as well as unstructured

data.

28

• Traditional RDBMS and SQL-based access languages are unfit to the new world of unstructured

information types.

• NoSQL (« Not only SQL ») is a database management system that is more versatile than

traditional database systems.

– Map Reduce and Hadoop, for example, are currently the most widely known NoSQL approaches.

– Data is stored without a pre-defined schema and big data sets are analyzed in parallel by assigning them

to different servers.

– Results are then collected and aggregated and can be further used in conjunction with relational database

systems.

29

• BI has evolved from historical reporting to the pervasive analysis of (real-time) data from multiple data sources. Transactional data is analyzed in combination with new data types from social, machine to machine and mobile sources (e.g. sentiment, RFID, geolocation data).

30

• Organizations that embrace a « socialization of data »-approach by incorporating and converging disparate data sources into their BI-platforms, acquire a holistic view that provides them with the opportunity to derive actionable insights, e.g.

– analytics of real-time customer sentiment and behaviour yield indicators of product or service issues ;

– geospacial information of customers can be combined with transactional data to make targeted product or service offerings ;

– combining internally generated data with publicly available information can reveal previously unknown correlations.

• In its focus on the user experience, BI embraces Web 2.0-technology that focusses on intuitive user interfaces. Organizations must master visualization tools that let business users interactively manipulate data to find tailored insights that can be shared with other stakeholders (customers, partners, suppliers).

31

3. Cloud Computing

32

Trends in

BI

networking

office automation

data warehousing

virtualized connectedenvironment

Internet-based data access & exchange

« as a service »-paradigm

desktop computing

eCommerce

service-orientedarchitecture

Web 2

1970s 1980s 1990s 2000s 2010 & beyond

# a

pps

/ #

use

rs

centralizedautomation

CLIENT-SERVER

INTERNET/DOTCOM AGE

CLOUD COMPUTING

PC

MAINFRAME/MINI

33

• As the competitiveness of businesses increasingly depends on adapting to changing market

conditions, companies outsource tasks and processes to external providers.

• This trend can be linked to the creation of business ecosystems in The Future Internet with

vendors offering their services.

• Software-as-a-Service (Saas), for example, is a type of cloud offering for software delivery.

Applications are hosted by a provider and made available on demand.

• Cloud computing is the backbone for the Internet of Services and provides resources for on

demand, networked access to services.

Infrastructure as a service

Platform as a service

Software as a service

Data as a service

Analytics as a serviceERP

34

“Cloud computing is enabling the consumption of IT as a service. Couple this with the “big

data” phenomenon, and organizations increasingly will be motivated to consume IT as an

external service versus internal infrastructure investments”.

The 2011 Digital Universe Study : Extracting Value from Chaos, IDC, June 2011

35

• Cloud computing alters the way computing, storage and

networking resources are allocated. Through virtualization,

the traditional server-centric architecture model in which

applications are tied to the underlying hardware is altered to

a service-centered cloud architecture. Applications are

decoupled from the physical resource which implies that

services (computing resources, e.g. processing power,

memory, storage, network bandwidth) in a cloud computing

environment are dynamically allocated to on demand

requests.

• In addition to a better utlization of IT resources, hardware

cost reduction and greener computing, cloud computing

provides an agile infrastructure to respond to business needs

in a flexible way.

36

The commoditization of analytics The trend towards the hosting of services, leads to the commoditization of analytics.

As a result, the creation of a competitive advantage depends on 2 factors.

The management of large

data volumes (data integration,

data quality). As data fuels

analytic processes, big data

becomes increasingly important..

Analytics in itself don’t

guarantee a competitive

advantage. The insights,

communications and decisions

that follow analysis become

more important. This stresses the

role of self-service and

collaboration.

37

In the pre-cloud world, the implementation of data warehouses

needed serious upfront costs and designing database schemas was

time consuming. Moreover, database schemas have their

limitations because some data types (e.g. unstructured) don’t fit

the schema. Combined with the need to manage big data volumes

new database technologies (e.g. NoSQL) are used. For example, in

the case of a Hadoop cluster that runs in parallel on smaller data

sets, multiple servers are needed. Making use of cloud computing

services in a pay-for-use formula is appealing. Furthermore, a

service-oriented cloud architecture is ideally suited to integrate

data from various sources (e.g. « mash up » enterprise data with

public data). Cloud computing

and big data

38

Cloud computing gives a new meaning to the consumerization of

IT. The convergence of cloud computing and connectivity is

changing the way technology is delivered and information is

consumed. Cloud applications are available on demand and

developed to meet the immediate needs of users. Cloud

computing is an important catalyst for self-service BI. Users do

not need to be concerned with the technical details of software

and hardware when using services. User-friendly interfaces and

visualization capabilities make the generation, sharing and acting

on information in real-time easier. This permits faster and better

decision-making as well as greater collaboration internally and

outside the firewall.

Cloud computing

and self-service BI

39

4. Embedded BI

40

Trends in

BI

EMBEDDED BI

Process Orientation

The Need for Agile BI

As the market changes faster and faster, BI has to adopt to support decisions in day-to-day

operations. The role of BI has changed beyond its original purpose of supporting ad hoc

queries and analysis of historical information. With changing market dynamics there is a

growing need to monitor performance using the latest data available and to predict

future events.

The new BI delivers information to users within the context of operational activities.

Rather than reporting on the business, BI moves into the context of business processes.

Data is analyzed in the flow of transactions to produce real-time metrics, alerts,

recommendations and predictions for action. BI transforms from a reactive to a

proactive decision-making tool.

Operational BI is related to the subject of real-time processing. Through the Internet

of people (e.g. social media) and the Internet of Things (e.g. RFID and other sensored

data), information becomes available that helps enterprises to improve business

processes.

41

• The consumerization of IT and the need of business decisions to be made on relevant

information are drivers for placing reporting and analytics in the hands of more decision-makers

and to apply analytics in real-time to production data.

• A broader user adoption of BI results from : – faster and easier executive access to information ;

– self-service access to data sources ;

– right-time data for users’ roles in operations ;

– more frequently updated information for all users.

• The business benefits are : – improved customer sales, service and support ;

– more efficiency and coordination in operations and business processes ;

– faster deployment of analytical applications and services ;

– customer self-service benefits.

42

Next-generation business applications will be more people- and process-oriented and have the computing power to

proactively generate information that supports operational decisions.

PROCESS PEOPLE

TECHNOLOGY

Next-generation applications are

not static but interactive,

allowing users to couple the right

actions based on the insights that

are delivered.

For example :

- analytics on browser-based BI

applications allow the mobile

workforce to take actions ;

- in an inventory application, proactive

decision-making is supported through

real-time information about which

items are running low in inventory.

Self-directed analytics give users the

ability to navigate through and

visualize business data, allowing

them to generate views and reports

relevant to their job function.

Business

Analytics

New approaches such as in-memory processing, in-database analytics, CEP,

etc. contribute to the broader adoption of BI.

43

BI delivery framework (adapted from Eckerson, 2011) 44

to

service-oriented architecture

from

monolithic applications

45

changes in the nature of BI : from

stand-alone applications to

embedded applications 1 2 3

changes in the function of

applications : from dedicated

applications to composite

applications

changes in the way data is

accessed : from data as an isolated

resource to data as a service

1

2

3

Source : SINGH KHALSA, R.H., REASON, A., BIERE, M., MEYERS, C., GREGGO, A & DEVINE, M., 2009.

46

Web Services SOA’s are based on the principle that

applications can be created as a

composition of loosely coupled and

reusable services. Open standards and

the implementation of SOA’s through

Internet-based technologies as Web

services represent a new way of

computing.

SOA Companies move away from large-scale monolithic

application development and turn to service-

oriented architectures that represent the

technological foundation of the Internet of

Services.

47

Web services are an important tool for

data integration from multiple sources

and provide access to real-time

information that can be fed into

operational applications.

Web services are user-centric because

information is provided in the context

of day-to-day activities.

Open access makes BI-functionality

accessible across and beyond the

enterprise.

The Internet of Services allows for the personalisation of services, tailored to the user’s needs.

Example : mashups (combining data from different sources into an integrated application)

48

An obvious implementation area for enterprise mashups applies to customer service.

CRM implies multiple processes (customer contact, sales, billing, support). Very often

the delivery of a process like that of customer service relies on end-users accessing

multiple applications. A major drawback is that customer-facing personnel (e.g. call

center agents, sales representatives) lack a unified customer view which causes a poor

quality of the customer experience. On the other hand, applications require a high

involvement of IT in the lifecycle of each application.

Therefore, enterprise mashups can provide a solution by the integration of disparate

data sources into a composite application. End users can use and reuse application

building blocks as “mashable” components to construct user-centric solutions. This not

only reduces the cost and time to build and maintain applications, but also allows

business users to create applications that are mapped with processes. Customer service

processes are optimized because employees are able to service customers more

efficiently.

Mashups and customer service

49

Social media is empowering customers to reveal their thoughts and preferences through

the Internet. This also enables businesses to look for competitive advantage by

monitoring and managing the many conversations that take place in the social media

world. Social media content can be tagged to look for pieces of information that can be

further structured to provide aggregate customer data revealing customer service issues,

consumer attitudes and brand-related topics. Furthermore, sentiment analysis that

extracts the semantics of user-generated content allows for the creation of mashups that

identify trends in unstructured data.

For example, dashboards can use sentiment measures as key performance indicators to

monitor product performance. Consumer sentiment can serve as an indicator of the

performance of a new product that is introduced in the market. Sentiment measures can

reveal the importance of product features and key customer needs. Retailers can

estimate demand for products based on expressed satisfaction of discontent with

products.

Mashups and social media analytics

50

• Another implementation area of mashups is data visualization that integrates location

intelligence in a composite application.

• Data streams within the enterprise can be joined with virtually any data source that can be

accessed from the Web. Web-based visualizations spacially represent the inherent relationships

between the underlying data.

• An example is Visual Fusion, data visualization software of IDV Solutions

(www.idvsolutions.com) that unites data sources in a web-based, visual context for better

insight and understanding. Commercial applications include the monitoring of inventory

through RFID systems, field service management, sales and marketing analysis, supply chain

management, and more.

http://www.idvsolutions.com 51

To view all suppliers for several auto assembly plants, a manufacturer developed an application

that visualizes suppliers on a map. Supply lines show which suppliers support which plants and

can be color-coded based on key information such as deliveries in progress and KPI data. Views

can be analyzed, sorted, filtered and collaborated upon to show how a selected supplier performs

compared to others via KPI-based charts and graphs.

http://www.idvsolutions.com/Products/VisualFusion/Gallery.aspx?view=8

52

reach the long tail of

the application spectrum

incorporate social & collaborative

computing features agility

cloud adoption

user-driven

real-time data view

53

5. User-Empowerment / Self-Service

54

Trends in

BI

• A confluence of factors (including ubiquitous broadband, a growing technology-native workforce, the adoption of social networking tools tools, mobile apps) is driving a trend called the consumerization of IT.

• Enterprise application development is driven by the need for interactive access to disparate data, self-service capabilities that offer a flexibility for personalization and end-user customization. BI shifts towards the self-service delivery model that accomodates knowledge workers to search, access and analyze data from a variety of sources and available on a range of devices.

• Empowerment of users is an important trend in BI. Business users generate their own reports and analysis and are no longer dependent on IT to deliver them. The ownership of BI shifts from IT to the business.

• By incorporating collaborative features, BI environments are getting social. These enhancements facilitate the creation of user-generated content that can be shared with stakeholders across and beyond corporate boundaries, enabling the networked enterprise and optimized decision-making.

55

client server, closed,

proprietary

structured data (data gathering

depends on data warehousing methodology)

analytics and presentation

are separated ; data-centric

based on open standards and loosely

coupled services that can be

reconfigured easily

data of any source is used

(structured, semi- and unstructured

data)

no separation between analytics and

presentation ; decision-centric

Traditional BI The New BI

architecture

analytics

data

56

create data models, control

of data and applications

focused on standard reports ;

predefinied reports to answer predefined questions

on premise, desktop and

server

deliver relevant data, ensure

security and scalability, enable

self-service

focused on interactive analysis

by end-users ; used to derive new

insights (“business discovery”)

on premise and on demand

(cloud, SaaS)

Traditional BI The New BI

IT role

deployment type

BI-delivery

57

STRUCTURED DATA (RDBMS)

data warehousing infrastructure

IT-driven

monolithic applications

close coupled enterprise

architecture

traditional report-centric approach

STRUCTURED & SEMI-/UNSTRUCTURED DATA

Web-based (cloud-)infrastructure

user-driven

intuitive applications

loose coupled services

« app-ification »

data discovery approach

58

Consumerization

of IT

Traditional IT

self-directed analytics

business discovery

long tail solutions

reusability

infrastructure

data governance

security

Adapted from Hinchcliffe, 2011.

technological innovations

are user-driven and increasingly

outside central IT-control

59

Drivers of the consumerization of IT

UBIQ

UIT

OU

S C

ON

NEC

TIV

ITY

CoIT

60

UBIQ

UIT

OU

S C

ON

NEC

TIV

ITY

CoIT

User-generated content Power shift from expert-generated to

user-generated content. Because

markets are more volatile, businesses

seek greater agility to respond faster

to market requirements. The

democratizaton of BI is driven bottom-

up and top-down. Users want

customized tools, while the ability to

mine data is critical for business

competitiveness, which causes

informed decision-making to be

extended across more roles.

Big data. The googlization of BI.

Crowdsourcing. Architecture of participation.

Data and desktop

virtualization Accessing data and applications

from any location, on any

device, at any time.

BI as a service The cloud as a delivery

mechanism for self-service BI.

61

in-memory

data management (processing large amounts

of data)

interactive data

visualization (business discovery)

Web-based

delivery (delivery to a variety

of devices)

self-service, fact-based decisions, agile BI

62

intuitive user

interfaces, easy to use,

work from browser,

real-time,

zero wait,

app-driven, multiple devices

user-driven analysis,

open standards,

loosely coupled services

culture of sharing

and collaboration

The BI-landscape is reshaped by the model of the consumer Web.

63

Business users are empowered to

gain insights into data (through exploration, visualization)

Value created from data can be

shared internally within the

company and externally with customers and partners

Collaboration is more than

distributing and sharing of

documents ; it implies bringing

context to analytics : different

people track the relevancy of

analytics and the decisions that will be based on it

The result is faster and better decision-making

64

6. Collaborative BI

65

Trends in

BI

• The idea of collaborative BI is to extend the processes of data

organization, analysis and decision-making beyond company

borders.

• While Web 2.0-technologies are migrating into the enterprise,

consumer-oriented social media tools do not provide the

necessary components for collaborative BI. Collaborative BI

requires the principle of information sharing to be incorporated

into day-to-day workflows.

• A difference also exists between analyzing social media on the

one hand and collaborative BI on the other hand.

• Social media provide a new source of data that complements

traditional data analysis to help organizations capture market

trends, better understand customer attitudes and behaviour

and uncover product sentiments.

• Collaborative BI uses web-based standards to connect people

(enterprise users, partners, suppliers, customers) to build

dynamic networks that share information and analysis results

to enable timely decisions that drive actions.

66

• Collaborative BI correlates with the analysis of big data and

self-service BI.

• Big data involves the analysis of ever-increasing volumes of

structured and semi- or unstructured data. In the context of

always changing business requirements, organizations need to

act quickly and decisively on business and consumer trends

derived from petabytes of data.

• Closely related to the expectations of users to access

applications anaywhere, at any time on any device are self-

service features that allow them to interact with data in a

flexible way. Accordingly, technologies as advanced data

visualization, embedded BI and in-memory analysis rank high in

preference lists.

• The pervasive use of BI that is stimulated through these

technologies is a necessity to enable analytic agility and

responsiveness.

67

Contrary to the traditional linear nature of data processing, collaborative BI

incorporates various feedback loops at different places in the analysis cycle.

Applied to BI, collaboration frameworks can be built that enable teams

to interact and socialize on data analysis-related topics.

68

« The world is rapidly turning into a network society. … The need to quickly adapt to

this changing environment is evident. The new paradigm in innovation is joining

forces in an online environment and activily working together. If we collaborate, we

can co-create and grow our ideas together, which ultimately leads to better, faster

and higher value Innovation ». www.innovationfactory.eu/vision

A McKinsey study gives evidence that the application of Web 2.0-

technologies to increase collaboration fosters the creation of networked

organizations. Enterprises that connect employees to forge close networks

with customers, business partners and suppliers become more competitive

and show improved performance in the areas of market share gains,

market leadership and margins. Through the use of collaborative tools,

information flows become less hierarchical and access to expert

knowledge is facilitated. Operational costs and time to market for new

products/services are reduced.

The rise of the networked enterprise : Web 2.0 finds its payday, McKinsey Quarterly, spring 2011.

69

• The business value of Web 2.0 for collaborative BI can be situated from the eight core patterns

of Web 2.0.

70

Web 2.0-features focus on the user experience. The

customer-centric focus of Web 2.0 has created a demand

for applications that move from the traditional

transaction platform to a model that is more accessible

and personal for the user.

Web 2.0-applications represent an opportunity for BI to

build Web-based collaboration. Reports can be published

in blogs and wikis, which help construct a knowledge base

to share interpretations. Users will learn to use

information more dynamically which allows the

generation of « crowd-sourced wisdom ». Besides

reporting and analysis, decisions are part of the BI

delivery mechanism.

Gaining insights from data to drive better decisions is no

longer constrained by the limits of internal data. The

open access to information in the Web 2.0-space allows

users to combine existing information with consumer-

generated content from the social networking spectrum

like blogs and wikis.

Social media analytics presents a unique opportunity to

threat the market as a « conversation » between

consumers and businesses. Companies that harness the

knowledge of social networks compile enterprise data

with streams of real-time data from Web 2.0-sources to

better access marketplace trends and customer needs.

The adoption of Web 2.0-technologies and applications

can help businesses to expand the reach of BI and improve

its effectiveness.

71

7. Social Media Analytics

72

Trends in

BI

• An important BI trend is the incorporation of the growing

streams of data generated by social media networks in BI-

applications.

• Social BI is a type of intelligence that focuses on data that is

generated in real-time through Internet-powered connections

between businesses and the public.

• Social media analytics give companies insights into the

mindset of their (prospective) customers, help them improve

media campaigns and offerings and accelerate responses to

shifts in the marketplace.

73

Drivers for social media analytics

74

The spectrum of available data has been enlarged with new soures, esp. social media

data streams.

75

The explosion of social media drives the need to analyze and

get insights from customer conversations.

76

interaction

data

attitudinal

data

behavioral

data

descriptive

data

The mobile and social media explosion empowers customers

and through the rapid growth of digital channels, the customer

experience takes on a new meaning. The objective of social

media analytics is to analyze social media data in context and

generate unique customer experiences across channels.

77

• Baynote (www.baynote.com) provides

recommendation services for websites.

Websites using Baynote recommendations

deliver relevant products and personalized

content that create an intuitive user

experience.

• Baynote applies « interest mining ». It

attempts to cluster consumers to provide

product or content recommendations that

are based on a broader understanding of

consumer behaviour. Baynote goes beyond

the clickstream by examining the words

associated with the clicks the user makes.

Combining the clickstream and the semantic

stream reveals the communality of cluster

members above a pure statistical or

demographic cluster approach. The resulting

« integrest graph » is used to personalize

product and content recommendations that

lead to maximum engagement, conversion

and lifetime value.

• Wise Window (www.wisewindow.com) distills

social media content automatically and in real-

time into industry-specific taxonomies. The

approach that Wise Window calls « Mass

Opinion Business Intelligence » (MOBI) does not

focus on individual behavior but the type of

syndicated research that Wise Window

performs is aimed at giving a broader

understanding of consumer sentiments and

behavior in the market at large.

• MOBI discovers leading indicators with data

derived from social media to make

organizations more agile and responsive.

Application fields include simple mindshare

analysis, discovering new products and niches,

spotting fast movers, performing constituent

analysis and predicting demand.

Examples of the use of social media analytics in day-to-day operations :

78

8. Predictive Analytics

79

Trends in

BI

Reporting (What happened ?)

HISTORY

FUTURE

Predictive Analytics (What might happen ?)

Monitoring (What is happening now ?)

Analysis (Why did it happen ?)

Traditionally, BI systems provided a retrospective view of the

business by querying data warehouses containing historical data.

Contrary to this, contemporary BI-systems analyze real-time event

streams in memory.

In today’s rapidly changing business environment, organizational

agility not only depends on operational monitoring of how the

business is performing but also on the prediction of future

outcomes which is critical for a sustainable competitive position.

Predictive analytics leverages actionable intelligence that can be

integrated in operational processes.

PRESENT

80

-30 -15 0 15 30 45

potential growth

com

mit

ment

advanced analytics (e.g. mining, predictive)

advanced data visualization

predictive analytics

real- time reports or dashboards

text mining

in- memory database

visual discovery

data marts for analytics

enterprise data warehouse (EDW)

statistical analysis

analytic database

outside the EDW

DBMS for transaction processing

hand- coded SQL

OLAP tools

analytics processed

within EDW

data mining

scoring

in- database analytics

DBMS for data warehousing

data warehouse appliance

sandboxes for analyticscolumn oriented storage engine

private c loud

extreme SQL

mixed workloads in a DW

in- line analytics

public c loud

MapReduce, Hadoop, Complex Event Processing

Software as a Service

closed- loop processing

accelerator (hardware or software based)

Potential growth vs. commitment for analytics options

Graphic based on survey results reported in Big Data Analytics, TDW Best Practices Report, Q4 2011, pp. 23.

Potential growth is an indicator for the growth or decline of usage for big data analytics over the next three years.

Commitment is a cumulative measure representing the percentage of respondens (N= 325) who selected using today and/or using in three years.

81

Current trends affecting predictive analytics :

82

• A systematic approach to guide the data mining process

has been developed by a consortium of vendor and users

of data mining, known as Cross Industry Standard for Data

Mining (CRISP-DM).

• In the CRISP-DM model, data mining is described as an

interative process that is depicted in several phases

(business and data understanding, data preparation,

modeling, evaluation and deployment) and their

respective tasks. Leading vendors of analytical software

offer workbenches that make the CRISP-DM process

explicit.

Standards for data mining and model deployment : CRISP-DM

83

• To deliver a measurable ROI, predictive analytics requires a focus on

decision optimization to achieve business objectives. A key element to

make predictive analytics pervasive is the integration with commercial lines

operations. Without disrupting these operations, business users should be

able to take advantage of the guidance of predictive models.

• For example, in operational environments with frequent customer

interactions, high-speed scoring of real-time data is needed to refine

recommendations in agent-customer interactions that address specific goals,

e.g. improve retention offers. A model deployed for these goals acts as a

decision engine by routing the results of predictive analytics to users in the

form of recommendations or action messages.

• A major development for the integration of predictive models in business

applications is the PMML-standard (Predictive Model Markup Language) that

separates the results of data mining from the tools that are used for

knowledge discovery.

Standards for data mining and model deployment : PMML

84

85

PMML represents an open standard for interoperability of

predictive models. Most development environments can

export models in PMML. As analytics increasingly drive

business decisions, open standards like PMML facilitate

the integration of predictive models into operational

systems. The deployment of predictive models in an

existing IT-infrastructure no longer depends on custom

code or the processing of a proprietary language.

Besides the flexible integration of predictive models into business

applications, continuous analysis is key to enable business process

optimization. The broad acceptance of the PMML-standard further

stimulates the exchange of predictive models. Open standards like

PMML contribute to the wider adoption of predictive analytics and

stimulate collaboration between stakeholders of a business

process. In a similar vein, the increased use of open-source

software can profit from PMML. Open-source environments can

visualize and further refine predictive models that were produced

in a different environment.

86

• The field of advanced analytics is moving towards providing a number of solutions for the

handling of big data. Characteristic for the new marketing data is its text-formatted

content in unstructured data sources which covers « the consumer’s sphere of influence » :

analytics must be able to capture and analyze consumer-initiated communication.

• By analyzing growing streams of social media content and sifting through sentiment and

behavioral data that emanates from online communities, it is possible to acquire powerful

insights into consumer attitudes and behaviour. Social media content gives an instant view

of what is taking place in the ecosystem of the organization. Enterprises can leverage

insights from social media content to adapt marketing, sales and product strategies in an

agile way.

• The convergence between social media feeds and analytics also goes beyond the aggregate

level. Social network analytics enhance the value of predictive modeling tools and

business processes will benefit from new inputs that are deployed. For example, the

accuracy and effectiveness of predictive churn analytics can be increased by adding social

network information that identifies influential users and the effects of their actions on

other group members.

Structured and unstructured data types

87

Text mining

pattern detection in unstructured data

Data-as-a-service

making multiple data sources available for analysis

Self-service

business discovery in an interactive way

Collaboration

adding context to decision making

Predictive modeling

Real-time dashboards

monitor KPI’s

Advanced visualization

multidimensional view of data

Social media analytics

analyze customer sentiment

88

• As companies gather larger volumes of data, the need for the execution of predictive models becomes more

prevalent.

• A known practice is to build and test predictive models in a development environment that consists of

operational data and warehousing data. In many cases analysts work with a subset of data through sampling.

Once developed, a model is copied to a runtime environment where it can be deployed with PMML. A user of an

operational application can invoke a stored predictive model by including user defined functions in SQL-

statements. This causes the RDBMS to mine the data iself without transferring the data into a separate file.

The criteria expressed in a predictive model can be used to score, segment, rank or classify records.

• An emerging practice to work with all data and directly deploy predictive models is in-database analytics. For

example, Zementis (www.zementis.com) and Greenplum (www.greenplum.com) have joined forces to score

huge amounts of data in-parallel. The Universal PMLL Plug-in developed by Zementis is an in-database scoring

engine that fully supports the PMML-standard to execute predictive models from commerial and open source

data mining tools within the database.

Advances in database technology : big data and predictive analytics

89

Data is partitioned across multiple

segment servers and each segment

manages a distinct portion of the

overall data.

The Universal PMML Plug-in enables

predictive analytics directly within

the Greenplum Database for high-

performance scoring in a massively

parallel environment.

90

• While vendors implement predictive analytics capabilities into their databases, a similar development is taking

place in the cloud. This has an impact on how the cloud can assist businesses to manage business processes

more efficiently and effectively. Of particular importance is how cloud computing and SaaS provide an

infrastructure for the rapid development of predictive models in combination with open standards. The PMML

standard has yet received considerable adoption and combined with a service-oriented archirtecture for the

design of loosely coupled systems, the cloud computing/SaaS model offers a cost-effective way to implement

predictive models.

• As an illustration of how predictive models can be hosted in the cloud, we refer to the ADAPA scoring engine

(Adaptive Decision and Predictive Analytics, www.zementis.com). ADAPA is an on demand predictive analytics

solution that combines open standarfds and deployment capabilities. The data infrastructure to launch ADAPA

in the cloud is provided by Amazon Web Services (www.amazonwebservices.com). Models developed with

PMML-compliant software tools (e.g. SAS, Knime, R, ..) can be easily uploaded in the ADAPA environment.

Predictive analytics in the cloud

91

Since models are developed outside the ADAPA environment, a first

step of model deployment consists of a verification step to ensure

that both the scoring engine and the model development environment

produce the same results. Once verified, models are executed either

in batch or in real-tile. Batch processing implies that records are run

against a loaded model. After processing, a file with the input and

predicted values is available for download. Real-time execution of

models in enterprise systems is performed through Web services

that are the base for interoperability. As new events occur, a request

is submitted to the ADAPA engine for processing and the results of

predictive modeling are available almost simultaneously.

92

• The on-demand paradigm allows businesses to use sophisticated software applications over the Internet,

resulting in a faster time to production with a reduction of total cost of ownership.

• Moving predictive analytics into the cloud also accelerates the trend towards self-service BI. The so-called

democratization of data implies that data access and analytics should be available across the enterprise. The

fact that data volumes are increasing as well as the need for insights from data, reinforce the trend for self-

guided analysis. The focus on the latter also stems from the often long development backlogs that users

experience in the enterprise context. Contrary to this, cloud computing and Saas enable organizations to make

use of solutions that are tailored to specific business problems and complement existing systems.

93

• PMML represents a common standard for the representation of predictive models.

• PMML eliminates the barriers between model development and model deployment.

• Through PMML predictive models can be embedded directly in a database.

• PMML-models can score data on a massive scale through parallel processing or in the cloud.

94

BI has evolved from performance reporting on historical data to the

pervasive use of real-time data from disparate sources.

To respond faster to market conditions, a much broader user base

needs data access to interactively explore and visualize information

sources and share insights to make faster and better

informed decisions.

In the era of big data, a Web-based platform enables business

discovery and data as well as analytics are consumed as services

in the cloud.

95

References

BOHRINGER, M., GLUCHOWSKI, P., KURZE, Chr. & SCHIEDER, Cgr., A business intelligence perspective on the future Internet,

AMCIS 2010 Proceedings, Paper 267.

COUTURIER, H., NEIDECKER-LUTZ, B., SCHMIDT, V.A. & WOODS, D., Understanding the future Internet, Evolved Technologist

Press, New York, 2011.

ECKERSON, W., BI delivery framework 2020, Beye NETWORK, march 2011.

GUAZZELLI, A., STATHATOS, K., ZELLER, M., Efficient deployment of predictive analytics through open standards and

Cloud computing, SIGKDD Explorations, 11, issue 1, pp. 32-38.

HINCHCLIFFE, D., Next-generation ecosystems and its key success factors, Dachis Group, 2011.

MICU, A.C., DEDEKER, K., LEWIS, I., MORAN, R., NETZER, O., PLUMMER, J. & RUBINSON, J., The shape of marketing research in

2021, Journal of Advertising Research, 51, march 2011, pp. 213-221.

RUSSOM, Ph., Big data analytics, TDWI Best Practices Report, Q4 2011.

SINGH KHALSA, R.H., REASON, A., BIERE, M., MEYERS, C., GREGGO, A & DEVINE, M., A convergence in application architectures

and new paradigms in computing. SOA, composite applications and cloud computing, IBM, january 2009.

SINGH KHALSA, R.H., REASON, A. & BIERE, M., The new era of collaborative business intelligence, IBM, march 2010.

96