Applications of Micro Geo Data for Urban Monitoring

Yuki Akiyama（[email protected]）

Assistant professor Earth Observation Integration and Fusion Research Institute (EDITORIA) The University of Tokyo

Applications of Micro Geo Data for Urban Monitoring

August 26, 2014 @Coex, Seoul , Korea

Smart Geospatial EXPO 2014 ICGIS2014

2

Recently, various geo data are available.

3

Building scale

Building data：Zmap-TOWNⅡ ©Zenrin CO. Ltd.

Detailed geo data are available. Building scale → household scale → person scale…

4

Building scale

Household scale

Micro Population Census ©Yuki Akiyama


5 People flow data ©CSIS

Person scale

Building scale

Household scale


6

Person scale

People flow data ©CSIS

Big data with location and time

＝Micro Geo Data（MGD）

Building scale

Household scale


7

Japanese government is looking to the utilization of MGD.

NHK News Web（2014/06/09）

http://www3.nhk.or.jp/news/html/20140609/k10015066701000.html

Japanese government begins to introduce legislation toward promotion of utilization of big data.

Ministry of Internal Affairs and Communications of Japan authorized a budget of approx 75million USD for utilization and application of geo big data. http://www.soumu.go.jp/main_content/000264428.pdf

8

Recently, personal computers and mobile phones are becoming widespread and internet environment are being developed rapidly. Anyone can access detailed digital map.

The widespread use of detailed digital map creates demands for more detailed and reliable spatial data then before. Researches to utilize such spatial data attract rising attention.

Spatially and temporally high resolution big data which can cover whole country area

＝Micro Geo data (MGD) It is gathering attention

how to develop, apply and popularize MGD now.

Coming of age to utilize MGD

9

Today’s Keyword

Micro Geo Data (MGD) =Spatial and temporal big data that spatial and

temporal resolutions are very fine

①Available Japanese MGD for urban monitoring ②Examples of urban monitoring using MGD ③MGD Forum

10

Census data aggregated by administrative unit (e.g. Population census)

Digitalization Map data

Integration

You can monitor actual states of residents (populations) in each cities, towns or villages.

©Softnat

11

Road length data (Digital national land information, 500m grid)

Grid statistics（e.g. Commercial statistics）

Commercial statistics aggregated into 1km, 500m or 250m square grids

12

Many previous studies used relatively macro scale GIS data.

In the case to collect micro scale data (i.e. each building, shop or pedestrian), many studies relayed on field survey. > it is difficult to collect micro scale data in broad area.

Data volume of micro scale data in broad area are very large. > Previously, it is difficult to handle them because of constraint of hardware’s performance.

Recently, it is possible to handle large data thanks to improvement of hardware’s performance. In addition, it has almost come to realize to accumulate micro scale data with the popularization of large volume hard discs.


History of commercial-release PC specs

13

0

1

2

3

4

5

6

7

8

9

0

500

1000

1500

2000

2500

3000

3500

19

81

19

82

19

83

19

84

19

85

19

86

19

87

19

88

19

89

19

90

19

91

19

92

19

93

19

94

19

95

19

96

19

97

19

98

19

99

20

00

20

01

20

02

20

03

20

04

20

05

20

06

20

07

20

08

20

09

20

10

20

11

20

12

CPUクロック数[MHz] メインメモリ[GB]

History of CPU clocks and main memory volumes by NEC home PC

[MHz] [GB]

Specs of commercial-release PC are improving rapidly. In addition, SaaS and cloud computing are spreading now.

Windows95 Released

Windows XP Released

Google Founded

Google Map Released

PC8001(1985) 1,750USD

PC-9821Ct20(1997) 4,150USD

VN370(2012) 975USD

CPU clocks [MHz] Main memory[GB]

History of commercial-release HDD specs

14

Volumes of HDD are also improved rapidly. Their volumes are increasing 1.6~2 times / yrs.

※http://www.infonet.co.jp/ueyama/ip/hardware/harddisk2.html

Commercial-release HDD

HDD confirmed in research level

Hundredfold in 10 yrs. （100MB→10GB）

Hundredfold in 10 yrs. （10GB→1TB）

Now, we can develop environment to use MGD easlly!

15

Many previous studies used relatively macro scale GIS data.

In the case to collect micro scale data (i.e. each building, shop or pedestrian), many studies relayed on field survey. > it is difficult to collect micro scale data in broad area.

Data volume of micro scale data in broad area are very large. > Previously, it is difficult to handle them because of constraint of hardware’s performance.

Recently, it is possible to handle large data thanks to improvement of hardware’s performance. In addition, it has almost come to realize to accumulate micro scale data with the popularization of large volume hard discs.

Next interesting stage is… How to utilize MGD?


1 year

２ months

1 hour

5 years

Existing data and census MGD New MGD developed in our study

Open data Commercial data（Some of them is free in case of academic research） 16

Available Japanese MGD for urban monitoring

Some census (grid) Population census Commercial statistics Business census etc.

Temporal Resolution

Real time

Prefectures/ Cities

Each building/ person

Grid scale (1km / 500m grid)

Spatial Resolution

Spatial data with high spatial and temporal

resolution

Collected by field survey or questionnaire survey by researchers

→Fine reliability Large amount of labor

国勢調査商業統計事業所・企業統計

Previously, these kinds of data are not used (or can not used) adequately.

If you can use raw data… ・Development and handling of them are difficult. ・Development of Sharing environment of them is difficult.

1 year

２ months

1 hour

5 years

Various mobile census

People flow dataset

Commercial Accumulation Statistics

Micro Population Census





Temporal Resolution

Real time

Prefectures/ Cities



Spatial Resolution

Web data （Estate, restaurants, SNS etc.)

Residential map

Telephone directory

Micro Geo Data (MGD) High spatial and temporal resolution ・They are very helpful for detailed urban monitoring. ・New MGD can be developed using them. ・Data holders are seeking how to utilize MGD.

1 year

２ months

1 hour

5 years


People flow dataset







Temporal Resolution

Real time

Prefectures/ Cities



Spatial Resolution


Residential map

Telephone directory


Residential map（Zmap-TOWNⅡ Zenrin Co., Ltd.）

You can monitor detailed building information ・Buildings or residents names ・Number of stories ・Usage of building ・Tenants or residents information in all floors or rooms →It covers almost all areas of Japan.

19

20

Digital telephone directory （Zenrin Co., Ltd. , NTT Business Information Service, Inc. etc.）

Telephone directory (text data) Address

Point data with attributes of

telephone directory

We can develop point data by the address geocoding (address → longitude, latitude)

・Name ・Business

・Phone number

・Address etc

of each shops and offices

Yuki Akiyama

四角形

Yuki Akiyama

四角形

21

Yellow page (old)

Digital house map (old) Digital house map (new)

Yellow page (new)

Spatial integration with digital house map

Spatio-temporal integration (Based on location and name similarity)

Time-series digital map

Monitoring of time-series changes on buildings and tenants

Akiyama, Y., Shibasaki, R., 2012,"A Method for Identifying Japanese Shop and Company Names by Spatiotemporal Cleaning of

Eccentrically Located Frequently Appearing Words", Advances in Artificial Intelligence, Vol2012, Article ID 562604.

Integration of time-series data!

Tenant names (in 2008)

22

Monitoring of time-series changes on buildings and tenants

Continuation

Change

Emergence

Demise

Legend Time-series changes between 2003 and 2008

Time-series changes of all floors can be monitored

23

Monitoring of time-series changes on buildings and tenants; 3D mapping

Continuation

Change

Emergence

Demise


渋谷駅 Shibuya Sta.



24

Continuation

Change

Emergence

Demise


Monitoring of time-series changes on buildings and tenants; Utilization of attribute data




Locations of McDonalds (Based on tenant names in 2008)

Locations of new clothing shop opened between 2003-2008 (based on business categories and time-series changes)

25

Monitoring of time-series changes on buildings and tenants; Utilization of attribute data





山手線 Yamanote

line


山手線 Yamanote

line

渋谷 Shibuya

原宿駅 Harajuku Sta.

代々木公園 Yoyogi park

表参道 Omote sando

原宿駅 Harajuku Sta. 代々木公園

Yoyogi park

渋谷 Shibuya

明治神宮 Meiji Shrine



Yoyogi park

渋谷 Shibuya





渋谷 Shibuya



Yoyogi park

渋谷 Shibuya


新宿御苑 Shinjuku Gyoen park


Yoyogi park

渋谷 Shibuya




Yoyogi park

新宿 Shinjuku





新宿 Shinjuku 新宿御苑

Shinjuku Gyoen park



新宿 Shinjuku


新宿 Shinjuku

新宿御苑 Shinjuku gyoen park


早稲田大学 Waseda Univ.

26

Monitoring of time-series changes on buildings and tenants; Data development in broad urban area

Continuation

Change

Emergence

Demise


Extremely detailed for macro scale monitoring… Make a grid map

Tenant changed rate

Percentage of the number of all tenants in 2008 of the number of changed tenants. <Example> Total number: 866 Changed tenants: 379

Changed rate = 379/866 = 0.4376

Evaluation index of briskness of commercial activity

Changed rate[%]

(2003-2008)

27


Extremely detailed for macro scale monitoring…

28

Changed rate[%]

(2003-2008)


29

Changed rate[%]

(2003-2008)


Korea

D.P.R.Korea

Taiwan

China Russia

30

Changed rate[%]

(2003-2008)


1 year

２ months

1 hour

5 years


People flow dataset







Temporal Resolution

Real time

Prefectures/ Cities



Spatial Resolution


Residential map

Telephone directory


This data is developed based on the person trip survey and they are interpolate spatio-temporaly using road and railroad network data.

People flow prohect http://pflow.csis.u-tokyo.ac.jp/index-j.html

32

We can monitor distributions of persons to stay or move (every 1 minute). ・Person attributes (age, sex etc) ・Methods of transport ・Aim of movement →It covers many metropolitan areas of Japan

People flow dataset

http://pflow.csis.u-tokyo.ac.jp/index-j.html





1 year

２ months

1 hour

5 years


People flow dataset







Temporal Resolution

Real time

Prefectures/ Cities



Spatial Resolution


Residential map

Telephone directory


34

Konzatsu Toukei (Mobile census data; Zenrin data com Co., Ltd. ）

We can monitor estimate populations of every hours by 250m square grids (updated everyday). ・Real time data based on GPS logs by mobile phones ・It covers throughout Japan.

1 year

２ months

1 hour

5 years


People flow dataset







Temporal Resolution

Real time

Prefectures/ Cities



Spatial Resolution


Residential map

Telephone directory


36

Example of Web data; Shop information (Hotpepper API)

We can collect detailed and fresh shop information.

37

Example of Web data; Usage of the twitter and flickr

Monitoring of urban activity by the Twitter and the Flickr (Blue dots: twitter, orange: flickr, white: both )

Tokyo, Japan

Eric Fischer, “Eric Fischer’s photostream”, http://www.flickr.com/photos/walkingsf/

New York, USA

http://www.flickr.com/photos/walkingsf/

Collection of business hour of each shops by analysis of search results by search engines (City center of Fukuoka, Japan)

Okamoto,Y., Akiyama, Y., Ueyama, S., Shibasaki, R., 2011, "Visualization of Business Hours for Shops and Offices Classified by Business Categories in Shopping Area" The 32nd Asian Conference on Remote Sensing, CD-ROM（TS3-10）

38

Example of Web data; Analysis of search result by search engine

1 year

２ months

1 hour

5 years


People flow dataset







Temporal Resolution

Real time

Prefectures/ Cities



Spatial Resolution


Residential map

Telephone directory


40

Distributions of commercial statistics are not monitored in detail because aggregated unit is administrative unit.

Problems of current Commercial Statistics

Actual distribution of commercial areas can not be monitored because aggregated units are administrative unit or grid.

Actual situations of accumulations by shops and offices are ignored because aggregated unit is grid.

41

In our study…

商業集積地域A 店舗数：201件新設店舗：52件コンビニ：8件

商業集積地域B 店舗数：73件新設店舗：9件コンビニ：3件

商業集積地域C 店舗数：182件新設店舗：38件コンビニ：6件

Aggregated unit is shapes of actual commercial areas. =Commercial Accumulation Statistics

It was realized to process digital telephone directory (= Micro geo data)

42

Development flow Actual shop information by Web pages of

some commercial areas

Commercial Accumulation Statistics ＝Each 1 polygon means 1 commercial areas.

Digital telephone directory （Telepoint Pack! Zenrin Co., Ltd. ）

Integration based on names and locations (address and X,Y)

Data selection based on business categories

Development of buffer polygons from each point and integration (merging)

Polygons which contain less than 10 shops are removed.

This data was developed and commercialized by the joint research with private company (Zenrin Co., Ltd.).

Joint research Adviser

Univ. Researchers

Private companies

Lend/Share (free)

Selling

43

Number of shops

10~20

20~30

30~50

50~100

100~

Akiyama, Y., Sengoku, H.,Hiroyuki, T., Shibasaki, R., 2011, "Development of Commercial Accumulation Polygon Data Throughout Japan Based on the Digital Classified Telephone Directory", CUPUM2011, F-TC-3(1)

Commercial Accumulation Statistics（Kobe city, Japan）

Locations and shapes of commercial accumulations are expressed by polygons.

Total number of shops

Number of shops in each business

categories

You can monitor numbers of shops of each business category in all commercial areas. 44

Number of shops

10~20

20~30

30~50

50~100

100~


Green points are telephone directory. 45

Number of shops

10~20

20~30

30~50

50~100

100~


46

Number of shops

10~20

20~30

30~50

50~100

100~


商業集積統計（世田谷区東部）

Commercial Accumulation Statistics (2009, Throughout Japan)

47

商業集積統計（世田谷区東部）

Commercial areas around main stations (Tokyo)


48

2 データ開発

商業集積地域ポリゴンデータ（日本全土） Road side shops（Asahikawa city）

Hot spring（Kusatsu town）

Isolated island（Nishinoomote city）


49

50 異なる時点の商業集積を重ねることで商業地域の広がりの変化を把握

We can detect declining shopping areas

Monitoring of changes of commercial area’s shapes to overlay multi-year data

Detection of declining shopping areas

Estimated number of visitors in each commercial areas

It is realized to integrate the Commercial Accumulation Statistics with the mobile census data by ZDC in the city center of Tokyo. (365days in 2012) 51

Legend Estimated number of visitors

Akiyama, Y., Ueyama, S., Horanont, T., Sengoku, H., and Shibasaki, R., 2013, "Spatio-temporal Analyses of Visitors in Commercial Accumulations Using Mass People Flow Data in Kanto Region", IGU2013 Kyoto Regional Conference, JS104-3-1.

52

0

2,000

4,000

6,000

8,000

10,000

12,000

14,000

16,000

18,000

20,000

22,000

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23

Avera

ge n

um

ber

of

vis

itors

in

weekd

ay

Times[hour]

浅草歌舞伎町原宿八重洲1丁目下北沢南口 Asakusa Kabuki-cho Harajuku Yaesu-1 Shimo-kitazawa

It is realized to integrate the Commercial Accumulation Statistics with the mobile census data by ZDC in the city center of Tokyo. (365days in 2012)

Estimated time-series number of visitors in some commercial areas

53

Estimated time-series number of visitors in some commercial areas

0

2,000

4,000

6,000

8,000

10,000

12,000

14,000

16,000

18,000

20,000

22,000

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23

Avera

ge n

um

ber

of

vis

itors

in

weekd

ay

Times[hour]

浅草歌舞伎町原宿八重洲1丁目下北沢南口 Asakusa Kabuki-cho Harajuku Yaesu-1 Shimo-kitazawa

It is realized to integrate the Commercial Accumulation Statistics with the mobile census data by ZDC in the city center of Tokyo. (365days in 2012)

1 year

２ months

1 hour

5 years


People flow dataset







Temporal Resolution

Real time

Prefectures/ Cities



Spatial Resolution


Residential map

Telephone directory


Recently, various census data are being digitalized and we can get them easily from websites of national and local governments.

Especially, the population census is used as base data to understand distribution and movement of population by many previous studies.

More detailed and reliable population data are required than before for following urban problems.

Background

Urban planning

Disaster damage estimation and

prevention

Traffic planning

Marketing support

＞We aim to develop new population census which can be accumulated into arbitrary spatial unit. 55

56

Micro population census (Population census + residential map + housing statistics)

It was realized to disaggregate population census. (Disaggregate: data processing to

reallocate data statistically based on other statistics and spatial features.)

東松原駅 LEGEND

Households

1 person

3 persons

4 persons

5 persons

≧6 persons

2 persons

Akiyama, Y., Takada, T. and Shibasaki, R., 2013, "Development of Micropopulation Census through Disaggregation of National Population Census", CUPUM2013 conference papers, 110.

57

It was realized to disaggregate population census. (Disaggregate: data processing to

reallocate data statistically based on other statistics and spatial features.)

東松原駅

Attribute table Building type houseLongitute 139.65633Latitude 35.663664

Area[m2] 105.34

Family type 8Household size 5Householder [age-gender] 45 - 1Spouse [age - gender] 40 - 2Number of child 2Information of children 5-1 | 10 - 2Number of parent 1Information of parent 75-2Number of others 0Information of others None

LEGEND

Households

1 person

3 persons

4 persons

5 persons

≧6 persons

2 persons



58

LEGEND

Households

1 person

3 persons

4 persons

5 persons

≧6 persons

2 persons



59

LEGEND

Households

1 person

3 persons

4 persons

5 persons

≧6 persons

2 persons



LEGEND

Households

1 person

3 persons

4 persons

5 persons

≧6 persons

2 persons



LEGEND

Households

1 person

3 persons

4 persons

5 persons

≧6 persons

2 persons

The Important thing to understand this data This data is estimated data.

Values of each point are necessarily match as actual states.



Aggregation by grid (250m square meter)


63

Aggregation by grid (250m square meter)

LEGEND

Population

0 - 250

501 - 1000

1001 - 2000

2001 -

251 - 500

Aim of the Micropopulation census =Development the new population census which can be aggregated

into arbitrary spatial unit.


64

Aggregation by city blocks

LEGEND

Population

0 - 50

101 - 250

251 - 500

501 -

51 - 100


LEGEND

Aging rate (Over 65 yrs. old)[%]

0 - 10

20 - 30

30 - 50

50 -

10 - 20

65

Monitoring of aging rate (rate of population over 65years old in 250m square grid)

凡例高齢化率 (65歳以上割合)[%]

0 - 10

20 - 30

30 - 50

50 -

10 - 20

66


LEGEND

Aging rate (Over 65 yrs. old)[%]

0 - 10

20 - 30

30 - 50

50 -

10 - 20

67


Application of MGD for urban problems

Many available MGD

Building data Time-series data Person flow data Mobile census

Commercial accumulation Micro population Web information

Internet

Application for urban problems

Integration

Traffic planning

Urban planning

Disaster damage estimation

Marketing support

Japanese government estimates that victims by Tokai-Tonankai earthquakes are over 300 thousands.

Calculation and estimation of micro-scale regional disaster risks thorughout Japan is very important for planning of earthquake disaster prevention in Japan.

Earthquake Disaster damage estimation by MGD

69

Y, Akiyama., Y, Ogawa., H, Sengoku., R, Shibasaki. And T, Kato., “Development of Micro Geo Data for Evaluation of Disaster Risk and Readiness by Large-scale Earthquakes Throughout Japan”, Proceeding of Annual conference on Infrastructure and Management, 2013, 392. (in Japanese) Y, Ogawa., Y, Akiyama. and R, Shibasaki, “The Development of Method to Evaluate the damage of Earthquake Disaster Considering Community-based Emergency Response Throughout Japan”, GI4DM2013, 2013, TS03-1.

Using various MGD and census data…

Estimation of earthquake disaster risk (fire rate, collapse rate etc.) and death rate

of each building

Earthquake Disaster damage estimation by MGD Earthquake Disaster damage estimation by MGD (250m square grid)

Estimated death rate [%]

In case of earthquake which will occur 2% in 50years



Earthquake Disaster damage estimation by MGD (250m square grid)


























This result received a lot of attention in Japan.

Medias (NHK) Magazines (GIS NEXT)

Research funds (Ministry of Land, Infrastructure, Transport and Tourism)

Revolution of GIS using MGD

Applicative studies which were difficult before are being realized. ・Today is data “abundant” and “accumulating” era.

・Unrealizable studies (only ideas) can be realized by MGD.

Anyone can develop an environment to handle MGD. ・We can buy high spec PC and high-capacity HDD at low prices.

Our study can target multi-temporal broad areas with any spatial units. ・Results can be accumulated by any spatial units. ・Results can cover all over the country in multi times.

81

Conclusions

82

Challenges

Development of human resources who can handle and analyze MGD is needed. ・We need to acquire skills to handle and understand MGD. > Programming skills to handle big data > Skills to handle GIS software for visualization > Knowledge of statistics to analyze results

Integration of MGD with field data ・We need to understand MGD is not universal data. ・To integrate MGD with field data, MGD will be used more than now. > Skill and sense to integrate field data with MGD

Exploitation of MGD utilization field ・To popularize MGD researches, we need to work together with researchers and experts in related fields.

Revolution of GIS by MGD

We established Micro Geo Data Forum in 2011. If you want to know about Micro geo data and this forum in detail, please visit our webpage. http://geodata.csis.u-tokyo.ac.jp/

Micro Geo Data Forum

83

http://geodata.csis.u-tokyo.ac.jp/wp/



84

There are many researches and publications using MGD from our forum From 2011…

Peer reviewed papers: 10 (Domestic: 6, International: 4)

Papers: over 50

Awards: 8 (GISA research encourage award, JSPRS award, ACRS innovation award, etc.) Magazines: GIS NEXT(15 times), Regional development etc. Medias: NHK Special “Disaster big data file3.” (Mar. 2, 2014)

Magazines Medias Awards

Mobmap; MGD visualization and analysis software

85

This is the GIS application on Google chrome. A member of our forum is developing now (Mr. Ueyama, Researcher of CSIS, The Univ. of Tokyo).

http://shiba.iis.u-tokyo.ac.jp/member/ueyama/mm/

Thank you for your kind attention

<Contact>

Yuki Akiyama（[email protected]）

Assistant professor Earth Observation Integration and Fusion Research Institute (EDITORIA) The University of Tokyo URL: http://shiba.iis.u-tokyo.ac.jp/member/akiyama/

（You can download some my papers and slides) Web page of Micro Geo Data Forum http://geodata.csis.u-tokyo.ac.jp/wp/

86

Search「秋山祐樹」or” akiyama.yuuki”!

끝까지 경청해 주셔서 감사합니다.

http://shiba.iis.u-tokyo.ac.jp/member/akiyama/






Engineering

Applications of Micro Geo Data for Urban Monitoring