Advance Database S2-2011 Week-5 Dr.Kwanchai Eurviriyanukul [email protected]

Advance Database S2-2011Week-5

Dr.Kwanchai [email protected]

Contents Week-5

• XML Basics 5: 7 ธค.– DOM– XML DTD– XML Schema

• XML Query– Xpath 6: 14 ธค– Xquery7: 21 ธค– XSL 8: 24 ธค

• WebService 9: 4 มค– TH-E-GIF

• DataIntegration 10: 11 มค– Schema matching and

mapping

• Parallel Database 11:18 มค

• GIS 12:

Contents Week-5

• XML Basics 5: 7 ธค.– DOM– XML DTD – XML Schema : 14 ธค

• XML Query– Xpath 6: 21 ธค– Xquery7: 24 ธค– XSL 8: 24 ธค

• WebService 9: 4 มค– TH-E-GIF

• DataIntegration 10: 11 มค– Schema matching and

mapping

• Parallel Database 11:18 มค

• GIS 12:

XML Basics: Content of Week 5

• DOM (Reviews)• XML DTD• XML Schema• Lab: DTD and Schema construction

Week5-Lab

• Construct and Validate your XML document using DTD and php.



XML Document example

8

From Stanford:Well-Formed and Valid XML

• Well-Formed XML allows you to invent your own tags.

• Valid XML conforms to a certain DTD.

•What is “Well-Formed XML ”?

9

From Stanford:Well-Formed and Valid XML


• Valid XML conforms to a certain DTD.

• What is “Well-Formed XML ”?

10

What are XML syntax rules?


• Valid XML conforms to a certain DTD.• What is “Well-Formed XML ”?

11

What are XML syntax rules?1.?

2.?3.?

4.?5.?

12

What are XML syntax rules?

13

From Stanford: Well-Formed XML

• Start the document with a declaration, surrounded by <?xml … ?> .

• Normal declaration is:<?xml version = ”1.0” standalone = ”yes” ?>– “standalone” = “no DTD provided.”

• Balance of document is a root tag surrounding nested tags.

XML DTD

XML DTD

XML DTD

DTD Declaration

DTD Declaration

GUESS??? Internal or External???

DTD Declaration

GUEST??? Internal or External???

DTD Declaration

GUEST??? Internal or External???

DTD Declaration

External Declaration

note.dtd

Internal DTD Declaration

24

RefreshFrom Stanford: Well-Formed XML

• Start the document with a declaration, surrounded by <?xml … ?> .

• Normal declaration is:<?xml version = ”1.0” standalone = ”yes” ?>– “standalone” = “no DTD provided.”

• Balance of document is a root tag surrounding nested tags.

25

From Stanford: Example: (a)<?xml version = “1.0” standalone = “no” ?><!DOCTYPE BARS [

<!ELEMENT BARS (BAR*)><!ELEMENT BAR (NAME, BEER+)><!ELEMENT NAME (#PCDATA)><!ELEMENT BEER (NAME, PRICE)><!ELEMENT PRICE (#PCDATA)>

]><BARS>

<BAR><NAME>Joe’s Bar</NAME><BEER><NAME>Bud</NAME> <PRICE>2.50</PRICE></BEER><BEER><NAME>Miller</NAME> <PRICE>3.00</PRICE></BEER>

</BAR> <BAR> …

</BARS>

The DTD

The document

1.?

2.?

26



]><BARS>


</BAR> <BAR> …

</BARS>

The DTD

The document

2.?

27

From Wiki: Example1.? 2.?

3.?

4.?

28

From Wiki: Example2.?

3.?

4.?

29

From Wiki: Example

3.?

4.?

30

From Wiki: Example

4.?

31

From Wiki: Example

DTD Declaration Exercise


note.dtd

33

From Stanford: Example: (b)

• Assume the BARS DTD is in file bar.dtd.<?xml version = “1.0” standalone = “no” ?><!DOCTYPE BARS SYSTEM ”bar.dtd”><BARS>

<BAR><NAME>Joe’s Bar</NAME><BEER><NAME>Bud</NAME>

<PRICE>2.50</PRICE></BEER><BEER><NAME>Miller</NAME>

<PRICE>3.00</PRICE></BEER></BAR><BAR> …

</BARS>

Get the DTDfrom the filebar.dtd

1.?


34

From Stanford: Example: (b)

• Assume the BARS DTD is in file bar.dtd.<?xml version = “1.0” standalone = “no” ?><!DOCTYPE BARS SYSTEM ”bar.dtd”><BARS>

<BAR><NAME>Joe’s Bar</NAME><BEER><NAME>Bud</NAME>

<PRICE>2.50</PRICE></BEER><BEER><NAME>Miller</NAME>

<PRICE>3.00</PRICE></BEER></BAR><BAR> …

</BARS>

Get the DTDfrom the filebar.dtd


35

From Wiki: Example1.?

2.? 3.?


36

From Wiki: Example

2.? 3.?


37

From Wiki: Example

3.?


38

From Wiki: Example

Why using DTD

Why using DTD

DTD - XML Building Blocks

DTD - Elements

DTD - Attributes

DTD - Entities

DTD - PCDATA

DTD - CDATA


DTD - Elements

Webpage for more details

DTD - Elements

50

From Stanford: Element Descriptions

• Subtags must appear in order shown.• A tag may be followed by a symbol to

indicate its multiplicity.– * = zero or more.– + = one or more.– ? = zero or one.

• Symbol | can connect alternative sequences of tags.

DTD – Elements-Quiz

http://zvon.org/comp/r/tut-DTD.html#Pages~Elements_in_given_order_%28_%2C_%29



tutorial.dtd

Valid XML document????



tutorial.dtd

Valid XML document???? => YES



tutorial.dtd




tutorial.dtd

Valid XML document???? => NO



tutorial.dtd

Valid XML document???? => NO



tutorial.dtd




tutorial.dtd

Valid XML document???? => YES



tutorial.dtd




tutorial.dtd

Valid XML document???? => No



tutorial.dtd

Valid XML document???? => No



tutorial.dtd




tutorial.dtd

Valid XML document????=>NO



tutorial.dtd

Valid XML document????=>NO

65



]><BARS>


</BAR> <BAR> …

</BARS>

The DTD

The document

1.?

2.?

66



]><BARS>


</BAR> <BAR> …

</BARS>

The DTD

The document

1.?


68

From Standford:DTD Structure

<!DOCTYPE <root tag> [<!ELEMENT <name>(<components>)>. . . more elements . . .

]>



Advance Database S2-2011Week-5

Dr.Kwanchai [email protected]

Week-1-Homework

• Marking criteria.1. You have successfully created your database. (2.5

marks)2. You have successfully populated your data into

your big table. (2.5 marks)3. You have successfully populated your data into

your normalized tables (2.5 marks)4. Your have demonstrated the enforcement of

foreign key constraints for your database. (2.5 marks)

Homework-Week-2

1. Data Population (Normalization from last week) 1

2. HTML processing 43. Venn Diagram Creation 4

Lab for Week-31. Marking Week-2 Lab2. Hand-Draw an XML tree structure for the given

example in the class. (2.5)3. Program-Draw an XML tree structure for an html

data (from line 393 - 557)1. http://

elearning.rmutl.ac.th/main/file.php/542/week-2/thailand.htm (2.5)

4. Change the Xpath expression of your previous code to … and observe what is different (5)

http://elearning.rmutl.ac.th/main/file.php/542/week-2/thailand.htm



Contents Week-4

• Create table using MySQL Workbench– Collation => Sorting => UTF-8_general/unicode– Storage Engine => MyISAM, InnoDB

• Populate Data: Select Data into table• Join Optimization• XML Basics– DOM

Lab for Week-41. Write “SQL statements” to populate data into the

following schema

2. Create indexes to speed up the join of these tables.3. Create a program to retrieve data from the web and

populated into your tables– Province coords– Amphur coords

Lab for Week-4-Cont.

Week-4 Homework Explain

ID Structure

• province-district_name-sub_district_name-moo

• 50-02-09-13• จั�งหวั�ด-อำ��เภอำ-ตำ��บล- หม�� => รห�สหม��บ��น• เชี�ยงใหม�-จัอำมทอำง-แม�สอำย-หม��13 => หม��บ��นห�วัยพั�ฒน�

• 50-03-01-02• เชี�ยงใหม�-แม�แจั�ม-ชี��งเค !ง-หม��2 => หม��บ��นตำ�อำเร"อำ

How about ID like ‘000001’? In INT format? => ‘1’ => We will loose

‘00000’ So char(6) will be a solution.

From: refman-5.5-en.html-chapter/data-types.html#numeric-types

MySQL supports the SQL standard integer types INTEGER (or INT) and SMALLINT. As an extension to the standard, MySQL also supports the integer types TINYINT, MEDIUMINT, and BIGINT.

Let’s analyze the possible ID for regions

SQL Here to get =>

Villagedata


Villagedata

How about?

select DISTINCT left(village_id,1), DISTINCT region_namefrom villagedata;

How about?

• select DISTINCT left(village_id,1), DISTINCT region_name

• from villagedata;

• Error

Contents Week-4

• Create table – Collation => Sorting => UTF-8_general/unicode– Storage Engine => MyISAM, InnoDB


Now we need to construct region table

Select name, collation, engine



• What is collation???


What is collation?


What is collation?



Now we need to construct region tableSelect name, collation, engine

Collation

http://dev.mysql.com/doc/refman/5.6/en/charset-unicode-sets.html

utf8, a UTF-8 encoding of the Unicode character set using one to three bytes per character

http://dictionary.cambridge.org/dictionary/american-english/collate?q=collate

=>to collect and arrange in correct order the sheets of a document

9.1.1. Character Sets and Collations in General

Let’s have a look• http://www.collation-charts.org/mysql60/mysql604.utf8_unicode_ci.south_east_asian.html

What are these?

http://www.collation-charts.org/mysql60/mysql604.utf8_unicode_ci.south_east_asian.html








Collationhttp://dev.mysql.com/doc/refman/5.6/en/charset-unicode-sets.html

What collation should we use?

Collationhttp://dev.mysql.com/doc/refman/5.6/en/charset-unicode-sets.html

Collation => ก�รเร�ยงล��ด�บตำ�วัอำ�กษรhttp://dev.mysql.com/doc/refman/5.6/en/charset-unicode-sets.html

We use utf8_unicode_ci because it provides more correctComparison for foreign characters, e.g. B => ‘ss’ in stead of ‘s’.Note it will be slower than utf8_general_ci for comparison.


What is collation?



Database Engine Selection

EB = 10E+18http://dev.mysql.com/doc/refman/5.6/en/storage-engines.html

Database Engine Selection

So we will use InnoDB because we want Transaction and Foreign key support.

EB = 10E+18http://dev.mysql.com/doc/refman/5.6/en/storage-engines.html

Create table region

Create table region

Synchronize with DB

• CREATE TABLE IF NOT EXISTS `sml_test`.`region` (

• ìdregion` CHAR(1) NOT NULL ,

• `nameregion` VARCHAR(45) NULL DEFAULT NULL ,

• PRIMARY KEY (ìdregion`) )

• ENGINE = InnoDB

• DEFAULT CHARACTER SET = utf8

• COLLATE = utf8_unicode_ci;

Synchronize with DB1. CREATE TABLE IF NOT EXISTS `sml_test`.`region` (

2. ìdregion` CHAR(1) NOT NULL ,

3. `nameregion` VARCHAR(45) NOT NULL ,

4. PRIMARY KEY (ìdregion`) )

5. ENGINE = InnoDB

6. DEFAULT CHARACTER SET = utf8

7. COLLATE = utf8_unicode_ci;

8. DROP TABLE IF EXISTS `sml_test`.`tb_region` ;9. DROP TABLE IF EXISTS `sml_test`.`tb_province` ;

Synchronize with DB1. CREATE TABLE IF NOT EXISTS `sml_test`.`region` (

2. ìdregion` CHAR(1) NOT NULL ,

3. `nameregion` VARCHAR(45) NOT NULL ,

4. PRIMARY KEY (ìdregion`) )

5. ENGINE = InnoDB

6. DEFAULT CHARACTER SET = utf8

7. COLLATE = utf8_unicode_ci;

8. DROP TABLE IF EXISTS `sml_test`.`tb_region` ;9. DROP TABLE IF EXISTS `sml_test`.`tb_province` ;

Contents Week-3

• Constraints• Create table using MySQL Workbench– Collation => Sorting => UTF-8_general/unicode– Storage Engine => MyISAM, InnoDB

• Select Data into table• XML Basics– XML Quizs– http://www.w3schools.com/xml/

http://www.w3schools.com/xml/

http://www.w3schools.com/xml/


What is wrong?

Drop primary key of region

• Now please write down an SQL statement to insert into the region table.

Insert into region

• insert into • select distinct left(village_id,1) as idregion,• region_name as nameregion• from villagedata;

Insert into region• insert into • select distinct left(village_id,1) as idregion,• region_name as nameregion• from villagedata;

Insert into region• insert into region• select distinct left(village_id,1) as idregion,• region_name as nameregion• from villagedata;

Insert into region• insert into region• select distinct left(village_id,1) as idregion,• region_name as nameregion• from villagedata;

What should be primary key?• insert into region• select distinct left(village_id,1) as idregion,• region_name as nameregion• from villagedata;

Single column is duplicated


How about two columns for PK.





What should be primary key?


Synchronize database

• Alter table sml_test.region• Add primary key(idregion, nameregion);

Data still exists

Contents Week-4



Join Optimization

80,000 8,000 800 80 8

Join Optimization

80,000 8,000 800 80 8

How about join them back together?

Join Optimization

80,000 8,000 800 80 8

How about join them back together?

Response Time-1

-- Second queryselect region_name, prov_name ,amp_namefrom region r, province p, amphur awhere r.region_id = p.region_id andp.prov_id = a.prov_id;

Response Time-2

select region_name, prov_name ,amp_name, tam_namefrom region r, province p, amphur a, tambon twhere r.region_id = p.region_id andp.prov_id = a.prov_id anda.amp_id = t.amp_id;

Response Time-3select region_name, prov_name ,amp_name, tam_name, vill_namefrom region r, province p, amphur a, tambon t, village vwhere r.region_id = p.region_id andp.prov_id = a.prov_id anda.amp_id = t.amp_id andt.tam_id = v.tam_id;

Join OptimizationHow about join them back together?

80,000 8,000 800 80 8

phpMyAdmin

How to speed up a join?


• Create Index

http://dev.mysql.com/doc/refman/5.6/en/mysql-indexes.html


• Create Index



• Create Index



• Create Index• Which table should be considered?• Which column should be considered?


80,000 8,000 800 80 8

Create Index

Village Table

Response Time-3select region_name, prov_name ,amp_name, tam_name, vill_namefrom region r, province p, amphur a, tambon t, village vwhere r.region_id = p.region_id andp.prov_id = a.prov_id anda.amp_id = t.amp_id andt.tam_id = v.tam_id;

What if We remove an index on village.tam_id?

select region_name, prov_name ,amp_name, tam_name, vill_namefrom region r, province p, amphur a, tambon t, village vwhere r.region_id = p.region_id andp.prov_id = a.prov_id anda.amp_id = t.amp_id andt.tam_id = v.tam_id;

What if We remove an index on village.tam_id?

select region_name, prov_name ,amp_name, tam_name, vill_namefrom region r, province p, amphur a, tambon t, village vwhere r.region_id = p.region_id andp.prov_id = a.prov_id anda.amp_id = t.amp_id andt.tam_id = v.tam_id;

remove an index

Has an index

Where else that we can add indexes?


• Can we create index on every column?


• Can we create index on every column?

http://en.wikipedia.org/wiki/B-tree

Contents Week-3



DOM

XML => Book

XML => Book

DOM Nodes

Node Tree

Node Type Property

Node Type Property

Node List

Node List

Node List

DOM

• XPath::query• Returns a DOMNodeList containing all nodes

matching the given XPath expression• http://th.php.net/manual/en/domxpath.query.php

DOM

$my_xpath_query = "/";

DOM

$my_xpath_query = "//area/@coords";

DOM

$my_xpath_query = "//area[(./@coords != 0) and (./@href != '#')]";

DOM$my_xpath_query = "//area[(./@coords != 0) and (./@href != '#')]";

Week-4-Home-Work

End of Week-4 Lecture

Documents

Advance Database S2-2011 Week-5 Dr.Kwanchai Eurviriyanukul [email protected]