Context Interchange for Dynamic Services- Adaptability, extensibility, scalability analysis

Hongwei (Harry) ZhuStuart Madnick

MIT Sloan School of Management{mrzhu, smadnick}@mit.edu

http://interchange.mit.edu/coin

WEB ‘04, December 11, 2004, Washington, D.C.

2

Characteristics of services

Large number of sources– Online travel services– Comparison shopping services

Diverse user needs – Increasing usability with personalization

Cannot establish a single data standardMust get semantics right

– Adaptability, extensibility, scalability

3

Motivating example

Semantic aspect Number of distinctions

Currency 10 different currencies

Scale factor 3 different scale factors: 1, 1K, 1M

Price definition 3 different definitions: base, base+tax, base+tax+SH

Date format 3 different formats, mm/dd/yyyy, dd-mm-yyyy, yyyy-mm-dd

Online comparison shopping– 400 vendor sources in different countries; 270 potential contexts– Different semantic assumptions in data – Compare prices in the context of any source chosen by the user

Need many conversions - 159,600 of them!

4

Desired properties Adaptability

– Capability of accommodating changes in sources

Extensibility– Easy to add/remove sources

Scalability– Effort of enabling interoperation wrt the number of

sources and the size of ontology – Performance wrt number of sources and the size of

each source (query optimization issue)

Flexibility = Adaptability + Extensibility

5

Interoperate: hard-wired approaches(a) BFS approachBrute-force between pair-wise sources

(b) BFC approachBrute-force between contexts

1 2

6

5 4

3

1 2

6

5 4

3Internal

standard

(c) Internal standard approachAdopting a standard

1 2

65 43

context_bcurrency: ‘TRL’; scaleFactor:1e6kind:base+tax; format: dd-mm-yyyy

context_acurrency: ‘KRW’; scaleFactor:1000kind: base; format: yyyy-mm-dd

context_ccurrency: ‘USD’; scaleFactor:1kind:base+tax+SH; format: mm/dd/yyyy

6

Interoperate: COIN Approach

ContextMediator

Source Receiver

ReceiverContext

ConversionLibraries

SourceContext

SharedOntologies

ContextTransformation

Context ManagementAdministrator

Concept: Length

Meters Feet f()meters feet

17

part length

Select partlengthFrom catalogWhere partno=“12AY”

55.79Auto-composition

of conversions

Select partlength/.3048From catalogWhere partno=“12AY”

7

Ontology and conversion function

monetaryValue

price

temporalEntity basic

kind

currency

is_a relationship

attributemodifier

Legend

context_acurrency: ‘KRW’; scaleFactor:1000kind: base; format: yyyy.mm.dd

context_bcurrency: ‘TRL’; scaleFactor:1e6kind:base+tax; format: dd-mm-yyyy

context_ccurrency: ‘USD’; scaleFactor:1kind:base+tax+SH; format: mm/dd/yyyy

context_d is_a context_b scaleFactor:1e3

context_e is_a context_dFormat: yyyy-mm-dd

context_f is_a context_cKind: base+tax

format

scaleFactor

organization

taxRate

Example source: src_turkey(Poduct, Vendor, QuoteDate, Price)

.*])2([),,,_(

][])2([])1([],1@)2,([

|:

222

ruvrCvalueRDTBCACDRBAolsen

TtempAttrxCCcurrencyxCCcurrencyxvuCCcurrencycvtx

luemonetaryVax

CC

t

C

f

tf

8

Demo – same context

No semantic differences

Meaningful data returned

9

(a) Select Vendor, Price From src_turkey Where Product=“Samsung SyncMaster 173P”;

(b) Select Vendor, QuoteDate, Price From src_turkey Where Product=“Samsung SyncMaster 173P”;

Conversion for scale factor

Conversion for date formatConversion for scale factor

Compose only relevant conversions (b e)

10

Introduced because of context difference in auxiliary source

Auto-reconciliation for auxiliary source(b f)

11

Detection and explication (ba)

12

Date format for receiver

Price definition – remove taxScale factor

Date format for auxiliary source olsenCurrency

Mediated query (b a)

13

Flexibility and Scalability

Approach General case In the example BFS N(N-1), N:= number of sources and

receivers 159,600

BFC n(n-1), n:= number of unique contexts 72,630 ETL/GS 2N, N:= number of sources and receivers 800 COIN 1) Worst case:

m

iii nn

1)1( , ni:= number of

unique values of ith modifier, m := number of modifiers in ontology

2)

m

iin

1)1( , when equational relationships

exist 3) m, if all conversions can be

parameterized

1) worst: 108 2) actual number: 5 (3

general conversions plus 2 for price)

Need to update/add many conversion programs

Why other approaches cannot fully benefit from general purpose conversion?– the decision whether to invoke the conversion is in the

conversion program

Update the declarative knowledge base.

flexible

Flexible

Not

14

How COIN scales

Component conversions are defined for each modifier

Overall conversions are automatically composed by abductive reasoning engine

Composition via symbolic equation solver and a shortest path algorithm

Inheritance enabled

15

Conclusion

Semantic differences cannot be standardized away

Must be flexible and scalableCOIN is a good solution

– Modularization, declarativeness– Automatic composition of necessary

conversions