How we did it: BSI: Teradata Case of the Tainted Lasagna

BSI TERADATA EPISODE 11:HOW WE DID IT THE CASE OF THE TAINTED LASAGNA

WATCH THE EPISODE AT HTTP://BIT.LY/14P5RMO

2 © Teradata BSI Studios 2013

We’re Getting A Lot of Questions …

Hi Everybody,

We’re the brains behind the scenes and wanted to answer your questions about “how we solved that lasagna case so fast.”

This little write-up will give you an idea of our client’s architecture and some details about how we did the investigation.

Take a look, and if you still have questions, shoot them to us!

Yours truly,

Mike Rinaldi and Jodice Blinco

BSI Teradata

DIRECTOR

JODICEBLINCO

BSI Teradata

Level 2

MIKERINALDI


Story Synopsis Case of the Tainted Lasagna

Situation

Huge worldwide consumer goods food producer, faced with 3-4 major and 5-6 minor recalls per year. Increased government oversight and food safety regulations.

Problem

Current approach – too slow, incomplete, because data is not integrated across the entire food chain. No advanced analytics. Impacts both sales and brand. High risk.

Solution

Used Teradata, Aster, Teradata Applications, and Tableau to re-engineer their Risk and Recall management system, built on top of their current ERP system. Uses big data for tracking and tracing.

Impacts

• Complex problem identification is faster – from 2 weeks to 3 days

• Big data improves root cause isolation and hypothesis testing

• Notification and remedy: recent recalls were 85% faster and at 99% coverage

• Improvement from 75% to 86% bad units verifiably destroyed

• Easy to satisfy regulators / prove issues were resolved

• Lowers risk for the company from bad PR and lawsuits

Great Brands:•Chief Risk Officer: Wiley W. Harvey•VP Supply Chain Management: June Davis

BSI:•Jodice Blinco•Mike Rinaldi

CAST OF CHARACTERS


• Decided to “keep her feet wet” by working on this case• Very interested in “emergency” uses of data• Had food poisoning recently, so personally engaged!

Jodice Blinco – Head of BSI

BSI Teradata

DIRECTOR

JODICEBLINCO


• Tech expert in Teradata, Aster, Teradata Apps, and Tableau• Focuses on architecture improvements, uses of big data

Mike Rinaldi – Principal Investigator

BSI Teradata

Level 2

MIKERINALDI


• Wiley W. Harvey - Risk Officer – very worried about new Food Safety government regulations, ability of Great Brands to comply. Ongoing issues with recalls, negative PR and associated costs

• June Davis - Supply Chain VP – knows her group is on the hook to resolve this problem. Focus is of course on better prevention, but things do slip through, requiring recalls. They need to be faster and more precise.

Great Brands

At Great Brand Corporate HQ

Problems: discussion of the problems, risks, unwieldy current architecture and processes. Commissions BSI to help

SCENE 1


The Problem – Yet Another Recall


Wiley and June from Great Brands have brought in Jodice and Mike from BSI to get their help on a revamp of Great Brands Risk/Recall system. They need to do better track and trace.

•Historical problems at Great Brands:> Reaction time – situational analysis takes too long, especially when

issues are cross-company with upstream suppliers> Root cause identification, scoping for recalls also takes too much time> Execution of the recall, compliance proof

•Impacts:> Number of incidents, complexity and cost of resolving - costly> New issues:– Government regulations – Food and Safety Administration rules, plus more

international rules coming

•Goals:> New Track and Trace system> Fresh ideas, incorporating the latest technologies

Scene 1: ProblemCase of the Tainted Lasagna


Food and Drug AdministrationShuts Down Peanut Factory in New Mexico

News:Huffington Post:http://www.huffingtonpost.com/2012/11/28/sunland-fda-peanut-butter_n_2206353.html FDA Statement: http://www.fda.gov/food/foodsafety/corenetwork/ucm320413.htm


Governments are getting aggressive about food safety Monitoring more closely, demanding compliance …

Read More At: http://www.fda.gov/newsevents/newsroom/pressannouncements/ucm334156.htm


New Headlines Around the World

Sources:http://www.independent.co.uk/life-style/health-and-families/health-news/spain-takes-on-germany-after-cucumber-scare-cripples-farm-exports-2292005.html http://www.nbcnews.com/id/38741401/ns/health-food_safety/

The Spanish cucumbers were not the problem.


More News HeadlinesLabeling Issues – Horsemeat in Europe

Source:http://www.thedailybeast.com/articles/2013/02/27/horsemeat-for-lunch-christopher-dickey-n-paris-s-horse-boucheries.html


The Food Industry Structure is ComplexSimplified Picture


The Real Picture from a CDC Talk

Source: CDC report at www.cdc.gov/about/grand-rounds/archives/2009/.../GR-121709.pdf


• Food manufacturers must increase collaboration with their trading partners (on both ends of the value chain – suppliers and retailers/customers)

• Invest in standardizing recall procedures and tools

• Implement effective technology that improves visibility across the value chain

BSI Teradata’s Recipe for Success

Such improvements can serve to both manage the risk of recall occurrence and reduce negative impact to your company and brand should the improbable happen.


Checkpoint – How Long Does It Take Today for Great Brands to Run A Recall?

Answer: it depends, but usually at least 30 days.


Sales Impact of Recalls Can Be Huge $ Losses


Recalls Also Impact GBRA Stock Price3 Examples

Back at BSI offices, Jodice and Mike tap into the data using new technology to see what changes to recommend

BSI Analytics for Track and Trace Goals – use tools to identify root causes fast (using big data across the supply chain) and create/execute recalls faster

SCENE 2


Simplified Process View


The “Real” Process at the CDChttp://www.cdc.gov/outbreaknet/investigations/figure_outbreak_process.html

When the CDC contacts us we have to do a betterjob on these steps:


Government Epidemiological Chart – Case Outbreaks Drive Interviews Which Lead to Suspect Foods

The process begins with a notification from the Government when they find from interviews of sick people point that they all ate Great Brands Lasagna.


Once notified by the Government, Great Brands steps:

1.Rapid data capture – load data with varying structures, formats, sizes, and velocities more quickly into a Discovery Platform – then find the needles in the haystack

2.Analysis and root cause –use fishbone analytics and temporal sequencing (nPath) to “see” the flow of raw product to consumers; isolation to bad product lots and possible victims

3.Managing recall resolutions – use B2B and B2C campaigns with workflows to guarantee recall notices go out to all downstream participants, and monitor responses

(B2B – Business to Business; B2C – Business to Consumer)

Side benefit – Government audits – build a dashboard portal so everyone - including regulators - can tap into the database, for easier auditing/risk monitoring

What Activities Occur at Great Brands?


• Process > Great Brands needs end-to-end visibility, traceability to close its

part of the loop quickly and thoroughly.

• Data > Great Brands has many data sources: CDC, Public Health inputs,

all manufacturing and transport data, access to supplier data, social media, Retail Store reports, loyalty data – how much depends on whether they can get access to both upstream and downstream data sets, in addition to their own data

• Detailed Analytical Steps In this Episode> Backtracking from sick people to manufacturing lots> Backtracking analytics / traceback from manufacturing lots to

raw product providers > Isolation to transportation introduction of pathogens > Analytics on cooking temperature sensor data

Why Do This? Overview


Internal to Great Brands> Manufacturing process data, by lot, by worker, by equipment> Sample testing reports, plant equipment maintenance inspections

Upstream – Supplier Information > QA reports from Farms, raw goods suppliers> QA reports from Transporters

•Downstream – Consumer information could be acquired much earlier, potentially – from retailers or directly from consumers

Sourcing Data for Great Brands – Details


The BSI investigators go through this case, using analytic tools•They started with this picture that goes from all external suppliers of inputs for the lasagna on the left through Great Brands to customers on the right. This is called a “fishbone” diagram. There can be thousands of potential problems.

The Investigation/Backtracking


• We start with the consumers on the right who were known to be sick, then look at which retailers sold them product, and backtrack.

• FINDING: the product all came from one plant and only selected product lots. That’s good news because the contamination is not widespread. We can next backtrack all the way back to farms.

The Investigation/BacktrackingFrom Sick Consumers to Suspect Manufacturing Product Lots


• Then we accessed farm data (with their permission, via their portals) and drilled into the QA reports and data from those farms for the time period when we sourced product. We started with broccoli (inspected at the farms) – but didn’t find any issues with the farms.

• For a while, we were stumped. Looked at other ingredients but came up empty, there, too.

Investigation/Backtracking to Farms


• On a hunch, we realized we skipped one step in ourflow diagram - the transportation of product from the farms to our plants> So we added the extra transportation step from farm to plant as a new

column and pulled data from transport companies> We pulled in data from the trucking companies used by Great Brands

• We found that all the sick people ate broccoli that was transported by one truck, from one transport company – Jimmy Changa Transport.

• Upon investigation (not shown in the episode) and some testing of swabs from the truck – they were the guilty party! > Sometimes the trucks are used to transport other products, and are

supposed to be decontaminated between loads – but apparently was not on this day

Investigation/Backtracking, Adding Transport


Added the Transportation Step


Drilling into Details: Broccoli from Farms - Transportation - Manufacturing Plant

Jimmy Changa

Jimmy Changa TransportTruck: 12 Date: Dec 29, 2012Truck Pickup Schedule

8:45 McDonnell Farm Corp 9:35 Shaw’s Broccoli & Spinach Farms10:20 Gib’s Healthy Green Coop


How did the bacteria pass through the Kill Step in the manufacturing process? That step should have killed the bacteria at 167 degrees Fahrenheit (or 75 degrees Celsius).

•They load temperature records for the implicated lots and find that it was the first run of the day – so maybe equipment is faulty, taking too long to heat up. •They investigate, by pulling into the Discovery Platform all the cooker sensor data. •They found that on January 2nd (right after a vacation day) the first 3 lots for Plant 21 Unit 1 - never hit the kill temp.•Upon further investigation (not shown) compared to other cookers, this unit is old and they recommended immediate replacement.

Jodice Asked a Good Question


Why Didn’t the Kill Step Solve the Problem?


Going Back to the Big PictureWe discovered the contamination sources, now on to the Recall!


• Using the sick people input and the Fishbone and pathing, we discovered the probable cause. A few more questions:> Any other pickups from that truck that day? (Answer: no – otherwise

that would widen the number of product units that need analysis)> We’ve only heard about those who GOT sick. Who else MIGHT get sick

because they bought the suspect product? > Is there unsold product in the warehouses or at stores that needs to be

pulled immediately?

• Great Brands must work next with the Retailers> Some stores let us (under recall situations only) access the store loyalty

data. If people paid using a loyalty card, or with a credit/debit card – we can create the list of people to notify immediately. This is fast.

> In other cases, we have to call the store’s BI people do the list runs for us. This is slower.

> And some Retailers are happy for us to contact their customers; some want to do the calls themselves. It can get complicated!

• Almost done. We also know from our own ERP system which lots went to which distribution centers and which retail stores.

Isolation for the Recall – Who To Contact?


We create the Product Recall Notices There will be variations for different audiences


Consumers - Who bought the product?> Known? – loyalty or purchase card data (from Retailers)> Unknown? – paid cash. This is where we have to go public.

•If all are known we can contact them all directly, avoid media.

Retailers > Unsold product still on shelves> In Retailer Distribution Warehouses – not yet on shelves

Great Brands > Distribution Centers – not yet shipped from our warehouses> Transportation Companies – if product is enroute from either:– Manufacturing location to Great Brands distribution centers, or – GB Distribution centers enroute to Retailer Distribution Warehouses

Targets for Recall


• For each target, we create the sequencing of events we want to monitor:> We communicate to them (telephone calls, emails, faxes)> We want to monitor that they received the communication> Then we want to monitor (with timeouts and follow-ups if

needed) whether they responded appropriately

• Responses will range, but can include:> Consumers: they consumed the product and got sick/reported (data

could be enroute to the government through public health channels)> Consumers: consumed product, did not get sick> Consumers: did not consume product, will destroy – we need to

reimburse> Retailers: pulled product from shelves> Retailers: pulled product from warehouses and destroyed/return> Great Brands internal: this is far easier.

• Goal is 100% communication and recall coverage.

Recall Communications and Monitoring


Designed and kicked off the Recall Workflows

BSI Readout back at Great Brands HQ

SCENE 3


• Our BSI investigators Jodice and Mike give the summary of changes they’d recommend for a better Track/Trace/Recall system to Wiley and June

Scene 3: Readout at Great Brands HQ


Three Key Technology Requirements


1 - Data Capture and Discovery Platform: Teradata UDA> Any data, any type, any source, any volume> Right toolkit for analytics – Teradata - core enterprise-wide data warehouse– Aster - Discovery Platform – Hadoop – optional data storage layer – Unified Data Architecture™ ties everything together with connectors, adaptors

2 - Recall Platform: Teradata Application - Aprimo > Quickly create recall targets> Quickly launch the recall notices with various workflows> Capture the results

3 - KPI Reporting / Risk Monitoring:> Tableau > Portal for executives > Could also be used for Government compliance reporting

Key Technology and Architecture Points

46 © Teradata BSI Studios 2013FARM DATA TRANSPORT MANUFACTURING CONSUMER SURVEY WEB & SOCIAL

COOKING SENSOR

WAREHOUSEPRODUCTPLANNING

RETAILERS

DISCOVER

Y PLATFOR

M

CAPTURE | STORE | REFINE

INTEGRATED DATA

WAREHOUSE

VIEWPOINT SUPPORT

Data Capture and DiscoveryTERADATA UNIFIED DATA ARCHITECTURE™

LANGUAGES MATH & STATS DATA MINING BUSINESS INTELLIGENCE APPLICATIONS

Engineers

Data Scientists Business Analysts Risk/Recall Managers Marketing/Sales

Operational SystemsCustomers / Partners Executives


Data Capture and Discovery : Teradata UDA

> Teradata is the core repository of enterprise data – historical context, any structured data, e.g., information from ERP systems, product data, production data, sales data, retailer data

> Aster - fast hypothesis testing for multi-structured data, e.g., fast pathing analysis, backtracking analytics, isolation insights. In this case, fast load and hypothesis testing on cooking sensor data, upstream farm raw product data (various formats, semi-structured including government reports from farm inspections), electronic data exchange (EDI) with upstream and downstream suppliers. Hypothesis testing using SQL Map/Reduce®.

> Hadoop as an optional component for fast, cheap ingest, e.g., Twitter feed of social comments, distillation/aggregation for feeding into Aster

Unified Data Architecture™ ties all the platforms together. Experimental results and data from discoveries in Aster or Hadoop flow into Teradata.

Key Technology Points


Data Flows


• There are 4 aspects for doing hypothesis testing: Data Acquisition, Data Preparation, Data Analytics, and Data Visualization. > Acquisition can happen within the UDA through three platforms

Teradata (Structured), Aster Discovery Platform (Structured and Unstructured), and if need be Hadoop (for more historical data)

> Data preparation can happen through proprietary technology (SQL-MapReduce functionality though we do not have to mention this technical detail) within the Discovery Platform

> Analytics such as the ones shown in the video can be done with proprietary technology within the Discovery Platform

> Visualization can be done in unique ways through a Tableau-like front end or some complementary visualization techniques in Aster (see next slides)

• Analytical insights from Aster can then be operationalized in the Teradata EDW from which we can trigger actions via Teradata Campaign Interaction Manager and other marketing automation tools

Summary of Discovery Process


Goal: Faster Hypothesis Testing Aster Discovery Platform

New business insights from all kinds of data with all types of analytics for all types of enterprise users with rapid exploration. Iterative hypothesis testing.

Large Volumes Interaction Data Structured Unstructured Multi-structured Hadoop

1

Relational/SQL MapReduce Graph Statistics, R Pathing

2

Business Users Analysts Data Scientists

3

Fast Iterative Investigative Easy

4


Teradata Aster Discovery Platform New Capabilities in 5.10 release

Industry’s First Visual SQL-MapReduce ® Functions

AFFINITY VISUALIZERVisualize clusters & groups

HIERARCHY VISUALIZERVisualize hierarchical relationships

FLOW VISUALIZERVisualize paths & patterns

Complementary Value•BI: Batch Visualizations Outside the Database, General & Generic•Aster: Rapid Visualizations, in-Database, for Specialized Analytics


Other New Visualizations for Big Data Flow, Affinity, Hierarchy Visualizers

Low Affinity between certain

departments

Low Affinity between certain

departments

Home & Garden,

Bedding and Bath & Fair Trade have high affinity

Home & Garden,

Bedding and Bath & Fair Trade have high affinity


Sample Analytics Modules in Aster

Fastest path to big data analytics

PATHING ANALYSISDiscover Patterns in Rows of Sequential Data

TEXT ANALYSISDerive Patterns and Extract Features in Textual Data

STATISTICAL ANALYSISHigh-Performance Processing of Common Statistical Calculations

GRAPH ANALYSISDiscover Natural Relationships of Entities

SQL ANALYSISReport & Analyze Relational Data

MAPREDUCE ANALYTICSCustom-built, domain-specific analysis


Aster Connectors / Adaptors

HADOOP ACCESSAcquire unstructured data for analysisSQL-H, Hadoop connectors

TERADATA ACCESSAcquire structured data for analysisAster-Teradata connector

RDBMS ACCESSAcquire structured data for analysisDB connectors

DATA ADAPTERSInterpret Data for AnalysisWeblogs, XML, PST, Machine Logs, JSON

DATA TRANSFORMATIONPrepare Data for AnalysisSessionization, Pivot, Unpivot, Pack, Unpack


Architecture: Teradata Aster Discovery PlatformFastest path to big data apps and new business insights

• Integrated hardware and software appliance

• Software only and cloud options

• Relational-data architecture can be extended for non-relational types

• SQL-MapReduce framework

• Analyze both non-relational + relational data

Growing the Development Bucket•70+ pre-built functions for data acquisition, preparation, analysis & visualization

•Richest Add-On Capabilities: Attensity, Zementis, SAS, R

•Visual IDE & VM-based dev environment: develop apps in minutes

Data ScientistsAnalysts Customers Business

Interactive & Visual Big Data Analytic Apps

Store

Process

Develop

Row Store Column Store

Data Acquisition

Module

SQL-H

Teradata

RDBMS

Data Preparation

Module

Unpack

Pivot

Apache Log Parser

AnalyticsModule

Pathing

Graph

Statistical

Viz Module

Flow Viz

HierarchyViz

Affinity Viz

Partner & Add-On Modules

Attensity

Zementis

SAS, R


Tying Everything Together - Recall Campaign Data, Discovery, Insights, Context, and Communications via Workflows

Big Data AnalyticsDiscovery

Customer Reports

RecallIn Progress

Recall CostsPrevious Recalls

ProductQA HistoryIntegrated

Marketing Management

CookieID UserID Attribution_Path

Real-time Interactions

MarketingSpend

Digital Marketing

Campaign Management

Consumer and Retailer Data


Teradata Application: Aprimo Components Used To Create and Run Recall Workflows

57


Recall Workflows


• The Teradata Campaign Interaction Manager collects information about workflow responses and creates summary tables • These can be visualized with reporting tools• BSI Teradata investigators used Tableau in this episode to mock up dashboards> Impacts on sales compared to other recalls> Waterfall diagrams showing the recall

precision> Effectiveness and efficiency reports for both B2C and B2B recall campaigns

• These results link to key performance indicators (KPIs) for Great Brands• Wiley’s next step is to come up with mobile executive dashboards so they can self-monitor on tablets how recalls are going

Recall Monitoring and Reporting

Photo: Tableau


Reporting: Tableau Recall Dashboard

A year later, Jodice and Wiley link up at a coffee shop

What was the impact of building the new Track and Trace system?

SCENE 4


• 1 year later – system has been used for more recall cases and we take a look at the impacts. Some key KPIs are:

> Speed and Accuracy of Exploration, Root Cause Analysis > Speed, Precision, and Accuracy of Recalls

• Jodice asks Wiley for an example recall they did with the new system and he shows her the results for a pepper-crusted salami product. The problem was bad spices that were imported from overseas. International tracing can also be included in the system.

Overall result: much more in control and reduced risk!!!

Scene 4: Impact


Salami Recall Process Improvement Compared to Previous System

Faster isolation and recall design steps: from 13 to only 3 days

Time

1 day to DesignRECALL

2 Days toGET DATA

8 days – Run RECALL: responses

2 days toISOLATE


Tableau – Waterfall Isolation of Targets


• Great Brands is also now using social media analytics to track all mentions of Great Brands on Twitter> Subscription service to the

Twitter Firehose

• Keywords include “Great Brands” “sick” “ill” “food poisoning”> Can also use this to track

comments about competitors

• Provides very early warning about potential problems> In this case, Great Brands

spotted the salami problem before the government health officials knew there was a problem

Social Listening Platform

WRAPUP


For more information – UDA

• Teradata UDA> http://www.teradata.com/products-and-services/unified-data-

architecture/


For more information - Aster

Teradata Aster: www.asterdata.com


For more information – Teradata Applicationswww.aprimo.com – Component used is “Teradata Customer Management Interaction Manager”


• www.teradata.com

For more information: Teradata


• www.tableausoftware.com

For more Information: our partner Tableau


For more informationConsumer Goods Manufacturing Industry …


• This episode appears at: http://bit.ly/13M0SMl

• You can see all our episodes at www.bsi-teradata.com on Facebook: links to 11 Videos and “How We Did It” Powerpoints

Thanks for watching!


Other BSI: Teradata Episodes

1.CASE OF THE DEFECTING TELCO CUSTOMERSA Telco uses analytics to see why they have a big customer retention problem

2.CASE OF THE MIS-CONNECTING PASSENGERS An Airline improves its customer rebooking engine using analytics

3.CASE OF THE RETAIL TWEETERSA Fashion Retailer uses social media tweets to get insights on hot and cold products and to find the FashionFluencers!

4.CASE OF THE CREDIT CARD BREACHA Bank and a Retailer collaborate to solve a stolen Credit Card case

5.CASE OF THE FRAGRANT SLEEPER HIT A Consumer Goods Manufacturer uses Social Media to recalibrate Manufacturing and Marketing plans

6.CASE OF DROPPED MOBILE CALLSA major Telco digs into real-time dropped call data to understand high-value and high-influence customers, where to place new towers. Create 5 campaigns to retain their most valuable and influential customers .

7, 8, 9: The SAD CASE OF STAGNOBANKCustomer service is lousy, most marketing offers are rejected by customers, and the bank has lost its appeal to younger households. BSI is engaged to work on new ideas for Better Marketing, Better Customer Service, and New Mobile Apps.

10. CASE OF THE RETAIL TURNAROUNDA Big-Box Retailer learns how to use web path purchase and bailout analytics to create ways of driving shoppers into stores.