An Excel-based Data Mining Tool

Chapter 4

4.1 The iData Analyzer

Figure 4.1 The iDA system architecture

Data

PreProcessor

Interface

HeuristicAgent

NeuralNetworks

LargeDataset

ESX

MiningTechnique

GenerateRules

RulesRuleMaker

ReportGenerator

ExcelSheets

Explaination

Yes

No

No

Yes

Yes

No

Figure 4.2 A successful installation

4.2 ESX: A Multipurpose Tool for Data Mining

Root

CnC1 C2

I11 I1jI12

Root Level

Instance Level

Concept Level

. . .

. . .

I21 I2kI22

. . . In1 InlIn2

. . .

Figure 4.3 An ESX concept hierarchy

4.3 iDAV Format for Data Mining

Table 4.1 • Credit Card Promotion Database: iDAV Format

Income Magazine Watch Life Insurance Credit CardRange Promotion Promotion Promotion Insurance Sex Age

C C C C C C RI I I I I I I

40–50K Yes No No No Male 4530–40K Yes Yes Yes No Female 4040–50K No No No No Male 4230–40K Yes Yes Yes Yes Male 4350–60K Yes No Yes No Female 3820–30K No No No No Female 5530–40K Yes No Yes Yes Male 3520–30K No Yes No No Male 2730–40K Yes No No No Male 4330–40K Yes Yes Yes No Female 4140–50K No Yes Yes No Female 4320–30K No Yes Yes No Male 2950–60K Yes Yes Yes No Female 3940–50K No Yes No No Male 5520–30K No No Yes Yes Female 19

Table 4.2 • Values for Attribute Usage

Character Usage

I The attribute is used as an input attribute.

U The attribute is not used. D The attribute is not used for classification or clustering, but

attribute value summary information is displayed in all output reports.

O The attribute is used as an output attribute. For supervised learning with ESX, exactly one categorical attribute is selected as the output attribute.

4.4 A Five-step Approach for Unsupervised Clustering

Step 1: Enter the Data to be Mined

Step 2: Perform a Data Mining Session

Step 3: Read and Interpret Summary Results

Step 4: Read and Interpret Individual Class Results

Step 5: Visualize Individual Class Rules

Step 1: Enter The Data To Be Mined

Figure 4.4 The Credit Card Promotion Database

Step 2: Perform A Data Mining Session

Figure 4.5 Unsupervised settings for ESX

Figure 4.6 RuleMaker options

Step 3: Read and Interpret Summary Results

• Class Resemblance Scores

• Domain Resemblance Score

• Domain Predictability

Figure 4.8 Summery statistics for the Acme credit card promotion database

Figure 4.9 Statistics for numerical attributes and common categorical attri

bute values

Step 4: Read and Interpret Individual Class Results

• Class Predictability is a within-class measure.

• Class Predictiveness is a between-class measure.

Figure 4.10 Class 3 summary results

Figure 4.11 Necessary and sufficient attribute values for Class 3

Step 5: Visualize Individual Class Rules

Figure 4.7 Rules for the credit card promotion database

4.5 A Six-Step Approach for Supervised Learning

Step 1: Choose an Output Attribute

Step 2: Perform the Mining Session

Step 3: Read and Interpret Summary Results

Step 4: Read and Interpret Test Set Results

Step 5: Read and Interpret Class Results

Step 6: Visualize and Interpret Class Rules

Figure 4.12 Test set instance classification

Read and Interpret Test Set Results

4.6 Techniques for Generating Rules

Simple Procedure for Creating Best Set of Covering Rules

1.Choose an attribute that best differentiate all domains.2.Use the attribute to further subdivide instances into classes.3.For each subclass created in step 2 3.1 If the instances in the subclass satisfy a predefined criteria Then generate a defining rule for the subclass. 3.2 If the subclass does not satisfy the predefined criteria Then repeat step 1

4.6 Techniques for Generating Rules (RuleMaker)

1. Define the scope of the rules.

2. Choose the instances.

3. Set the minimum rule correctness.

4. Define the minimum rule coverage.

5. Choose an attribute significance value.

4.7 Instance Typicality

• The average similarity of instance to all other instances within its class.

• Identify prototypical and outlier instances.

• Select a best set of training instances.

• Used to compute individual instance classification confidence scores.

Figure 4.13 Instance typicality

4.8 Special Considerations and Features

• Avoid Mining Delays – at some point copy the original data into another Excel sheet

• The Quick Mine Feature – recommended when the dataset contains more than 2000 instances

• Erroneous and Missing Data – blank lines, beyond the last column, invalid characters