DATA VISUALIZATION USINGMULTIDIMENSIONAL SCALING

ProjectReportSubmittedinPartialFulfilmentoftheRequirements

FortheAwardoftheDegreeofIntegratedDualDegree

inComputerScienceandEngineering

UndertheGuidanceofByProf.A.K.AgarwalSSujana

RollNo10400EN005

DEPARTMENTOFCOMPUTERENGINEERINGINDIANINSTITUTEOFTECHNOLOGY(BANARASHINDUUNIVERSITY)

VARANASI221005INDIA

1

CERTIFICATE

This is to certify that the project entitled, Data Visualization Using Multidimensional Scaling submitted by S Sujana in partial fulfilment of the requirements for the award of Integrated Dual Degree in Computer Science & Engineering at the Indian Institute Of Technology, Banaras Hindu University, is an authentic work carried out by her under my supervision and guidance.To the best of my knowledge, the matter embodied in the project report has not been submitted to any other University/Institute for the award of any degree or diploma.

Supervisor

Prof. A. K. AgrawalDepartment Of Computer Engineering

IIT(BHU), Varanasi

2

ACKNOWLEDGEMENTS

I would like to convey my deepest gratitude to Professor A. K. Agarwal, who guided me through this project. His keen interest, continuous motivation, suggestions and support helped me immensely in successfully completing this project.

I would also like to thank Prof. R. B. Mishra, Head of the Department, Department of Computer Engineering, for allowing me to avail all the facilities of the Department necessary for this project.

S SujanaRoll No. 10400EN005

3

ABSTRACT

Datavisualizationisapowerfultechniqueofdataexplorationthatharnessesthepowerofthehumanmindtodetectstructuresandpatternsinvisualdata.Thisprojectdiscussesdatavisualizationusingmultidimensionalscaling,atechniqueforanalyzingsimilarity/dissimilaritydataonasetofobjects.

Multidimensionalscalingtakesthedissimilaritydataasinput,andreturnsasetofcoordinatesoftheobjectsinalowdimensionalEuclideanspace,suchthatthedistancesbetweenthesepointsarepreservedasmuchaspossible.Here,thishasbeencarriedoutbyimplementinganalgorithmSMACOF(ScalingbyMajorizingACOmplicatedFunction),whichoffersfasterconvergenceontheClassicalmultidimensionalscalingalgorithm.

4

CONTENTS

Abstract....4

1.Introduction.61.1ExploratoryDataAnalysis...61.2DataVisualization.6

2.MultidimensionalScaling....82.1Need82.2Formulation.82.3Paradigm....92.4Classification......92.5Applications...112.6ComparisonofTechniques.12

3.SMACOF.143.1MathematicalBackground..143.2Algorithm...15

4.Implementation184.1PreparatoryWork.184.2Specification..194.3Pseudocode...20

5.UserGuide...225.1InputFormat...225.2Output.....22

6.TestCases...23

7.Conclusion...25

8.FurtherScope..268.1TunnelingMethod......26

9.References...27

5

INTRODUCTION

Exploratory Data AnalysisExploratory data analysis is an approach to analyzing data to summarize its main characteristics, often with visual methods. The purpose of this analysis is to see structure in thedata.

Exploratory data analysis can be described as datadriven hypothesis generation. The data is examined, in search of structures that may indicate deeper relationships between cases or variables. This process stands in contrast to hypothesis testing, which begins with a proposed model or hypothesis and undertakes statistical manipulations to determine the likelihoodthatthedataarosefromsuchamodel.

The distinction here is that it is the patterns in the data which give rise to the hypothesis in contrast to situations in which the hypothesis is generated from theoretical arguments about underlying mechanisms. This increases our confidence in the resulting hypotheses, as these results are less likely to be influenced by external factors, like the context in which the data wasfound,theanalystspersonalbiases,traditionalmethodsusedandsoon.

Multidimensional scaling, principal components analysis, scatter plots, etc are some of the typicalgraphicaltechniquesusedinexploratorydataanalysis.

Data VisualizationData visualization is the presentation of data in a pictorial or graphical format. It is basically an abstraction of data. Its main goal is to communicate information clearly and effectively usingvisualmeans.

For centuries, people have depended on visual representations such as charts and maps to understand information more easily and quickly. As more and more data is collected and analyzed, decision makers at all levels welcome data visualization software that enables them to see analytical results presented visually, find relevance among the millions of variables,communicateconceptsandhypothesestoothers,andevenpredictthefuture.

Visualization methods are used to display data in ways that harness the particular strengths of human pattern processing abilities. Visual methods are important in data mining as they are ideal for sifting through data to find unexpected relationships. They enable the analyst to grasp the structure and hence, meaning of the data, faster. These methods, however, may notbeabletocommunicateclearlyforextremelylargedatasets.

Visualization also enables representing data in a universal and accessible manner. It makes itsimpletoshareideaswithothersandencouragesexplorativedataanalysis.

6

Somecommonvisualizationtechniquesare: Standard2D/3Ddisplays,suchasbarchartsandxyplots Geometricallytransformeddisplays,suchaslandscapesandparallelcoordinates Iconbaseddisplays,suchasneedleiconsandstaricons Densepixeldisplays,suchastherecursivepatternandcirclesegments Stackeddisplays,suchastreemapsanddimensionalstacking

Multidimensionalscalingisavisualizationtechniqueofthefirstkind.

7

MULTIDIMENSIONAL SCALING

Multidimensionalscalingreferstoaclassofalgorithmsforexploratorydataanalysiswhichvisualizeproximityrelationsofobjects,bydistancesbetweenpointsinalowdimensionalEuclideanspace.Proximityvaluesarerepresentedasdissimilarityvalues.

Given a set of data objects, multidimensional scaling aims to place each object in an Ndimensional space such that the interobject distances are preserved as much as possible. For the case when N=2, the problem can be stated as plotting a scatterplot of the givendataset.

NeedOne of the most important goals in visualizing data is to get a sense of how near or far points are from each other. Often, this can be done with a scatter plot. However, for some analyses, the data that we have might not be in the form of points at all, but rather in the form of pairwise similarities or dissimilarities between cases, observations, or subjects. There are no pointstoplot.

Even if the data are in the form of points rather than pairwise distances, a scatter plot of these data might not be useful. For some kinds of data, the relevant way to measure how near two points are might not be their Euclidean distance. While scatter plots of the raw data make it easy to compare Euclidean distances, they are not always useful when comparing other kinds of interpoint distances, city block distance for example, or even more general dissimilarities. Also, with a large number of variables, it is very difficult to visualize distances unless the data can be represented in a small number of dimensions. Some sort of dimensionreductionisusuallynecessary.

Multidimensional scaling (MDS) is a set of methods that address all these problems. MDS allows you to visualize how near points are to each other for many kinds of distance or dissimilarity metrics and can produce a representation of your data in a small number of dimensions.

FormulationFormally,multidimensionalscalingcanbepresentedasfollows

The data to be analyzed is a collection of I objects (colors, faces, stocks, . . .) on which a distancefunctionisdefined,i,j:=distancebetweenithandjthobjects.

Thesedistancesaretheentriesofthedissimilaritymatrix.

8

ThegoalofMDSis,given,tofind vectors suchthat

forall ,

where isavectornorm.

Paradigm1. Multidimensional scaling is typically used as a visualization technique for proximity

data.2. When the dissimilarities are distances between highdimensional objects, it acts as a

(often nonlinear) dimensionreduction technique. Hence, it makes complex and coupleddatamoreunderstable.

3. When the dissimilarities are shortestpath distances in a graph, it functions as a graph layout technique. It is useful in visualizing weighted graphs, both planar and non planar.

ClassificationHere,wediscuss3classificationsofmultidimensionalscalingalgorithms

ClassicalMDS MetricMDS NonMetricMDS

Classical MDS

ClassicalmultidimensionalscalingisalsoknownasPrincipalCoordinatesAnalysis.Ittakesaninputmatrixgivingdissimilaritiesbetweenpairsofitemsandoutputsacoordinatematrixwhoseconfigurationminimizesalossfunctioncalledstrain.

This method tries to find the main axes through a matrix. It is a kind of eigenanalysis (sometimes referred as "singular value decomposition") and calculates a series of eigenvalues and eigenvectors. Each eigenvalue has an eigenvector, and there are as many eigenvectorsandeigenvaluesastherearerowsintheinitialmatrix.

9

Eigenvalues are usually ranked from the greatest to the least. The first eigenvalue is often called the "dominant" or "leading" eigenvalue. Using the eigenvectors we can visualize the main axes through the initial distance matrix. Eigenvalues are also often called "latent values".

The result is a rotation of the data matrix: it does not change the positions of points relative to each other but it just changes the coordinate systems. Thus, we can visualize individual and/orgroupdifferences.Individualdifferencescanbeusedtoshowoutliers.

Metric MDS

Metric multidimensional scaling is a superset of classical MDS that generalizes the optimization procedure to a variety of loss functions and input matrices of known distances with weights and so on. It also offers a choice of different criteria to construct the configuration,andallowsmissingdataandweights.

The distances between objects are required to be proportional to the dissimilarities, or to someexplicitfunctionofthedissimilarities.

A useful loss function in this context is called stress, represented as (X). The function is a cost or loss function that measures the squared differences between ideal (mdimensional) distancesandactualdistancesinrdimensionalspace.Itisdefinedas:

where is a weight for the measurement between a pair of points ,

is the euclidean distance between and and is the ideal distance between the points (theirseparation)inthe dimensionaldataspace.

can be used to specify a degree of confidence in the similarity between points (e.g. 0 canbespecifiedifthereisnoinformationforaparticularpair).

The problem of multidimensional scaling is thus reduced to minimizing this stress function (X). A common approach to this is termed as Least Squares Scaling, which involves . There exist other algorithms which do not rely on gradient descents. One of these methods, aimed at minimizing a stress function of the Sammon type, is known by the acronym SMACOF (Scaling by MAjorizing A COmplicated Function). It is based on an iterative majorizationalgorithmthatintroducesideasfromconvexanalysis.

10

Non Metric MDS

In metric multidimensional scaling, the distances between objects are required to be proportional to the dissimilarities, or to some explicit function of the dissimilarities. In nonmetric multidimensional scaling, this condition is relaxed to require only that the distances between the objects increase in the same order as the dissimilarities between the objects.

In contrast to metric multidimensional scaling, nonmetric multidimensional scaling finds both a nonparametric monotonic relationship between the dissimilarities in the itemitem matrix and the Euclidean distances between items, and the location of each item in the lowdimensionalspace.Therelationshipistypicallyfoundusingisotonicregression.

Nonmetric multidimensional scaling includes an additional optimization step to smooth the data. This is usually done by carrying out isotonic regression. The implementation for nonmetric multidimensional scaling usually smoothly fits into the implementation of metric multidimensionalscalingbyincludingtheoptimizationstep.

ApplicationsMultidimensionalscalingisextremelyusefulforscientificvisualizationanddatamining.Here,someofitscommonapplicationsarementioned.

In marketing, multidimensional scaling is a statistical technique for taking the preferences and perceptionsofrespondentsandrepresentingthemonavisualgrid,calledperceptualmaps.Potential customers are asked to compare pairs of products and make judgments about their similarity. Whereas other techniques (such as factor analysis, discriminant analysis, and conjoint analysis) obtain underlying dimensions from responses to product attributes identified by the researcher, multidimensional scaling obtains the underlying dimensions from respondents judgments about the similarity of products. This is an important advantage. It does not depend on researchers judgments. It does not require a list of attributes to be shown to the respondents. The underlying dimensions come from respondents judgments about pairs of products. Because of these advantages, multidimensional scaling is the most commontechniqueusedinperceptualmapping.

In cognitive sciences, multidimensional scaling is often used to study confusion data. Confusion data represents the possibility of different items being mistaken for each other. Thus, it is a form of a similarity matrix. Multidimensional scaling presents this data in a visual form such that the items which are grouped together in the representation are more likely to be similar, and thus confused for each other. Multidimensional scaling is often demonstrated withRothkopfsMorsecodedataset,asanexampleofaconfusiondataset.

In the social sciences, proximity data take the form of similarity ratings for pairs of stimuli suchastastes,colors,sounds,people,nations,etc.

11

In archaeology, similarity of two digging sites can be quantified based on the frequency of sharedfeaturesinartifactsfoundinthesites.

In classification problems: In classification with large numbers of classes, pairwise misclassification rates produce confusion matrices that can be analyzed as similarity data. Anexamplewouldbeconfusionratesofphonemesinspeechrecognition.

Another early use of MDS was for dimension reduction: Given highdimensional data y1 , ..., yN IRK (K large), compute a matrix of pairwise distances dist(yi , yj ) = Di,j , and use distance scaling to find lowerdimensional x1 , ..., xN IRk (k

common idea to use the Sammon stress function as relative supervisor to train a nonlinear mapping.Wewilllookatoneofthesemethodsattheendofthisreport.

13

SMACOF

SMACOFstandsforScalingbyMajorizingACOmplicatedFunction.Itisastrategytosolvetheproblemofmultidimensionalscalingbyusingmajorizationtominimizeastressfunction.

Mathematical BackgroundMajorization

Before describing details about SMACOF, we give a brief overview on the general concept of majorization which optimizes a particular objective function in our application referred to as stress. More details about the particular stress functions and their surrogates for various SMACOFextensionswillbeelaboratedbelow.

In a strict sense, majorization is not an algorithm but rather a prescription for constructing optimization algorithms. The principle of majorization is to construct a surrogate function which majorizes a particular objective function. For MDS, majorization was introduced by De Leeuw (1977a) and further elaborated in De Leeuw and Heiser (1977) and De Leeuw and Heiser(1980).

From a formal point of view, majorization requires the following definitions. Let us assume we have a function f(x) to be minimized. Finding an analytical solution for complicated f(x) can be rather cumbersome. Thus, the majorization principle suggests to find a simpler, more manageablesurrogatefunctiong(x,y)whichmajorizesf(x),i.e.forallx

g(xy)>=f(x)

where y is some fixed value called the supporting point. The surrogate function should touch the surface at y, i.e., f(y) = g(y, y), which, at the minimizer x* of g(x, y) over x, leads to the inequalitychain.

f(x*)

objects.

AlgorithmMultidimensionalscalinginputdataaretypicallyanxnmatrixofdissimilaritiesbasedonobserveddata.

issymmetric,nonnegative,andhollow(i.e.haszerodiagonal).Theproblemwesolveistolocatei,j=1,.,npointsinlowdimensionalEuclideanspaceinsuchawaythatthedistancesbetweenthepointsapproximatethegivendissimilaritiesij.ThuswewanttofindannxpmatrixXsuchthatdij(X)ij,where

dij(X)=s=1p(xisxjs)2

The index s = 1, , p denotes the number of dimensions in the Euclidean space. The elements of X are called configurations of the objects. Thus, each object is scaled in a pdimensional space such that the distances between the points in the space match as well as possible the observed dissimilarities. By representing the results graphically, the configurationsrepresentthecoordinatesintheconfigurationplot.

Nowwemaketheoptimizationproblemmoreprecisebydenotingstress(X)by

Here,Wisaknownnnmatrixofweightswij,alsoassumedtobesymmetric,nonnegative,andhollow.Weassume,withoutlossofgenerality,that

wijij2=n*(n1)/2i=0.FollowingDeLeeuw(1977a),stress,asgivenin(4),canbedecomposedas

=2+2(X)2(X)

Fromrestriction(5)itfollowsthatthefirstcomponent2=n(n1)/2.Thesecondcomponent2(X)isaweightedsumofthesquareddistancesdij2(X),andthusaconvexquadratic.Thethirdone,i.e.2(X),isthenegativeofaweightedsumofthedij(X),andis

15

consequentlyconcave.

Thethirdcomponentisthecrucialtermformajorization.Letusdefinethematrix

Aij=(eiej)(eiej)

whoseelementsequal1ataii=ajj=1,1ataij=aji,and0elsewhere.

Furthermore,wedefine

astheweightedsumofrowandcolumncenteredmatricesAij.Hence,wecanrewrite

Forasimilarrepresentationof(X)wedefinethematrix

where

UsingB(X)wecanrewrite(X)as

and,consequently,thestressdecompositionbecomes

At this point it is straightforward to find the majorizing function of (X). Let us denote the supporting point by Y which, in the case of MDS, is a n x p matrix of configurations. Similar to (8)wedefine

with

16

TheCauchySchwartzinequalityimpliesthatforallpairsofconfigurationsXandY,wehave

Thusweminorizetheconvexfunction(X)withalinearfunction.Thisgivesusamajorizationofstress

Obviously,(X,Y)isa(simple)quadraticfunctioninXwhichmajorizesstress.Findingitsminimumanalyticallyinvolves

TosolvethisequationsystemweusetheMoorePenroseinverse

whichleadsto

ThisisknownastheGuttmantransform(Guttman1968)ofaconfiguration.Notethatifwij=1forallij,wehave

andtheGuttmantransformsimplybecomes

17

IMPLEMENTATION

Preparatory WorkPython Programming Language

Python is a widely used generalpurpose, highlevel programming language. Its design philosophy emphasizes code readability, and its syntax allows programmers to express concepts in fewer lines of code than would be possible in languages such as C. The language provides constructs intended to enable clear programs on both a small and large scale.

Python supports multiple programming paradigms, including objectoriented, imperative and functional programming or procedural styles. It features a dynamic type system and automaticmemorymanagementandhasalargeandcomprehensivestandardlibrary.

Like other dynamic languages, Python is often used as a scripting language, but is also used in a wide range of nonscripting contexts. Using thirdparty tools, Python code can be packaged into standalone executable programs (such as Py2exe, or Pyinstaller). Python interpretersareavailableformanyoperatingsystems.

CPython, the reference implementation of Python, is free and open source software and has a communitybased development model, as do nearly all of its alternative implementations. CPythonismanagedbythenonprofitPythonSoftwareFoundation.

Numpy library

NumPy is an extension to the Python programming language, adding support for large, multidimensional arrays and matrices, along with a large library of highlevel mathematical functions to operate on these arrays. The ancestor of NumPy, Numeric, was originally created by Jim Hugunin with contributions from several other developers. In 2005, Travis Oliphant created NumPy by incorporating features of Numarray into Numeric with extensive modifications.NumPyisopensourceandhasmanycontributors.

NumPyisthefundamentalpackageforscientificcomputingwithPython.Itcontainsamongotherthings:

apowerfulNdimensionalarrayobject sophisticated(broadcasting)functions toolsforintegratingC/C++andFortrancode usefullinearalgebra,Fouriertransform,andrandomnumbercapabilities

18

Besides its obvious scientific uses, NumPy can also be used as an efficient multidimensional container of generic data. Arbitrary datatypes can be defined. This allows NumPy to seamlessly and speedily integrate with a wide variety of databases. Numpy is licensedundertheBSDlicense,enablingreusewithfewrestrictions.

PyQT

PyQtisaPythonbindingofthecrossplatformGUItoolkitQt.ItisoneofPython'soptionsforGUIprogramming.PopularalternativesarePySide(theQtbindingwithofficialsupportandamoreliberallicence),PyGTK,wxPython,andTkinter(whichisbundledwithPython).LikeQt,PyQtisfreesoftware.PyQtisimplementedasaPythonplugin.

PyQtimplementsaround440classesandover6,000functionsandmethodsincluding: asubstantialsetofGUIwidgets classesforaccessingSQLdatabases(ODBC,MySQL,PostgreSQL,Oracle) QScintilla,Scintillabasedrichtexteditorwidget dataawarewidgetsthatareautomaticallypopulatedfromadatabase anXMLparser SVGsupport classesforembeddingActiveXcontrolsonWindows(onlyincommercialversion)

Matplotlib library

matplotlib is a plotting library for the Python programming language and its NumPy numerical mathematics extension. It provides an objectoriented API for embedding plots into applications using generalpurpose GUI toolkits like wxPython, Qt, or GTK. There is also a procedural "pylab" interface based on a state machine (like OpenGL), designed to closely resemblethatofMATLAB.

matplotlib was originally written by John Hunter, has an active development community, and isdistributedunderaBSDstylelicense.

SpecificationDescription

The multidimensional scaling software allows its user to visualize data points on a 2dimensional scatterplot for given proximity data and analyze its performance with respect tovariousconfigurableparameters.

Product Perspective

19

The software must, upon getting input from user in required format, process it to initialize itself and the run the multidimensional scaling algorithm on the information and provide the outputtotheuserintherequiredformat.

Product Scope

Thesoftwareshallhavethefollowingscope Users can get the multidimensional scaling output for various configuration of cores

foranalysis. It may be specialized for particular types of data by providing wrappers that generate

input in required format and that read the output generated and display or process it forfurtheranalysis.

User Interface

TheGraphicalUserInterfacehasbeendevelopedusingPyQt,aPythonwrapperfortheQt4Framework.

Pseudocode1.Generateaninitialconfigurationofthepointsrandomly.

defcheck_random_state(seed):"""Turnseedintoanp.random.RandomStateinstanceIfseedisNone,returntheRandomStatesingletonusedbynp.random.Ifseedisanint,returnanewRandomStateinstanceseededwithseed.IfseedisalreadyaRandomStateinstance,returnit.OtherwiseraiseValueError."""ifseedisNoneorseedisnp.random:returnnp.random.mtrand._randifisinstance(seed,(numbers.Integral,np.integer)):returnnp.random.RandomState(seed)ifisinstance(seed,np.random.RandomState):returnseedraiseValueError('%rcannotbeusedtoseedanumpy.random.RandomState''instance'%seed)

#Randomlychooseinitialconfigurationrandom_state=check_random_state()X=random_state.rand(n_samples*n_components)X=X.reshape((n_samples,n_components))

20

2.Computethestress

#Computedistancesdis=euclidean_distances(X)

disparities=similarities

#Computestressstress=((dis.ravel()disparities.ravel())**2).sum()/2

3.ComputeGuttmantransform

#UpdateXusingtheGuttmantransformdis[dis==0]=1e5ratio=disparities/disB=ratioB[np.arange(len(B)),np.arange(len(B))]+=ratio.sum(axis=1)X=1./n_samples*np.dot(B,X)

4.Iterate2and3untilconvergence

dis=np.sqrt((X**2).sum(axis=1)).sum()ifold_stressisnotNone:if(old_stressstress/dis)

USER GUIDE

Input FormatTheinputshallcontainthefollowing

Thefirstlineshallcontainalistoflabelsforthedataitemsoflengthn.Thisisfollowedbythedissimilaritymatrixofdimensionnxn.Thisisfollowedbyalinecontainingthenumberofrunsofmultidimensionalscalingwithdifferentinitialconfigurations.Thenextlinecontainsthenumberofrunstobeexecutedinparallel.Thenextlinecontainsthemaximumnumberofiterationsinasinglerun.Thenextlinecontainstherelativetolerancewithrespecttostresstoachieveconvergence.

OutputThe output shows the plot of the points obtained from multidimensional scaling. In the case wherelabelsarenotprovidedintheinput,thepointsaremarkedbytheirindices.

The matplotlib toolbar can be used to pan/zoom into the plot. The navigation buttons can be used to return or move forward to a previous or next view. The plot can be saved in a file for furtheranalysis.

Thestressshownisthefinalstressobtainedinthemultidimensionalscalingprocess.

22

TEST CASES

Input:

23

Output:

24

CONCLUSION

Multidimensional scaling has been implemented using majorization technique, by the SMACOFalgorithm.

For a given dissimilarity matrix, the algorithm successfully computes the positions of the data intwodimensionalspace.Variousparameterscanbeusedtotunethealgorithm.

Thus, multidimensional scaling using SMACOF algorithm has been successfully implemented.

25

FURTHER SCOPE

As noted earlier, one of the drawbacks of the majorization approach to multidimensional scaling is the issue of the algorithm getting stuck in a local minima while minimizing the stress. Only a few global minimization strategies have been developed for MDS, the most prominent algorithm being the tunneling method. This deterministic scheme allows the algorithm to escape local minima by tunneling to new configurations with the same stress, possiblyprovidingastartingpointforfurtherstressreduction.

Tunneling MethodThe tunneling method is an approach to multidimensional scaling. It alternates a local search step, in which a local minimum is sought, with a tunneling step, in which a different configuration is sought with the same STRESS as the previous local minimum. In this manner successively better local minima are obtained, and the last one is often a global minimum.

It consists of an iterative twostep procedure: in the first step, a local minimum is sought, and in the second step, another configuration is determined with exactly the same STRESS. It canbedescribedbythefollowinganalogy.

Suppose we wish to find the lowest spot in a selected area in the Alps. First, we pour some water and see where it stops: the local search. From this point, a global search is performed by digging tunnels horizontally until we come out of the mountain. There we pour water again, find out where it stops, and dig tunnels again. If we stay underground for a long time while digging the tunnel, we simply conclude that the last spot was in fact the lowest place in the area,thecandidateglobalminimum.

An important and attractive feature of the tunneling algorithm is that successive local minima always have lower or equal function values. The tunneling step is the crux of the tunneling method.Itisperformedbyminimizationofaparticularfunction,calledthetunnelingfunction.Theeffectivenessofthetunnelingmethodisdeterminedbythesuccessofthetunnelingstep.

26

REFERENCES

1. D.J.Hand,HeikkiMannila,PadhraicSmytPrinciplesofDataMining

2. http://www.mathworks.in/help/stats/multidimensionalscaling.htmlIntroductiontoMultidimensionalScaling

3. WSTorgersonMultidimensionalscalingI.Theoryandmethod

4. I.Borg,P.J.F.GroenenModernMultidimensionalScaling:TheoryandApplications

5. AndreasBuja,DeborahF.Swayne,MichaelL.Littman,NathanielDean,HeikeHofmann,andLishaChenDataVisualizationwithMultidimensionalScaling

6. PiotrPawliczek,WitoldDzwinelInteractiveDataMiningbyUsingMultidimensionalScaling

7. J.D.LeeuwApplicationsofconvexanalysistomultidimensionalscaling

8. JandeLeeuwConvergenceoftheMajorizationMethodforMultidimensionalScaling

9. J.D.Leeuw,P.MairMultidimensionalscalingusingmajorization:SMACOF

10. PatrickJ.F.GroenenANDWillemJ.HeiserTheTunnelingMethodforGlobalOptimizationinMultidimensionalScaling

11. HansjrgKlockJoachimM.BuhmannDatavisualizationbymultidimensionalscaling:adeterministicannealingapproach

27