Embed Size (px)
Citation preview
洪一平
台大網媒所 &南藝大動美所
1
2017/05/11
Outline
2
簡介
石窟導覽VR：敦煌飛天
走入古畫VR：故宮秋色
太極拳自學AR：大師的身影
展望
～15B yrs ago
Life Source：～4B yrs ago
埃及猿～30M yrs ago
北京人～1.8M yrs ago
Evolution of Human
Palmer Luckey: Time magazine cover, Aug 17, 2015
New Era of Virtual Reality
Head-Mounted Dispalys Oculus VR -- founded by Palmer Luckey …… in 2012-- announced in Kickstarter on Aug 1, 2012-- within 4 hours, secured its intended amount
of US$250,000 and eventually ending with > 2M-- acquired by Facebook for ~2B on March 25, 2014
Oculus Rift DK1 ~2012 Oculus Rift DK2: ~2014/7
The First HMD in 1965
The Ultimate Display (The Sword of Damocles, 1965)
Ivan Sutherland
VR HMD（Sales in 2016）
SONY PS VRGoogle Cardboard~ 500K in a week
Oculus Rift HTC VIVE
~ 1,500K sets
~ 500K sets~ 600K sets
一切有為法、如夢幻泡影如露亦如電、應作如是觀
--金剛經
什麼是真實的? 什麼是虛假的?
Avatar (阿凡達)：2009
Matrix 駭客任務（黑客帝國）：1999
Augmented Virtuality
擴增虛境
Virtual Environment
Real Environment
Virtual Reality
虛擬實境
Augmented Reality
擴增實境
Mixed Reality 混合實境
AR HMD (MR)
Google Glass（2013～15）
Magic Leap （？）Microsoft HoloLens（2016）
Epson BT-2000 /BT-300
MR
文化資產之種類
• 物質文化資產 （Tangible cultural heritage）
-- 歷史建築、雕像、石窟、壁畫、歷史文物、…… 等
• 非物質文化遺產（Intangible cultural heritage）
-- 慶典儀式、 表演藝術、傳統舞蹈、傳統武術、……等
Outline
15
簡介
石窟導覽VR：敦煌飛天
走入古畫VR：故宮秋色
太極拳自學AR：大師的身影
展望
Flying Spirit in Dunhuang-- Immersive Experience on Virtual Touring of Mogao Caves
- National Taiwan University
- Academia Sinica
- Dunhuang Research Academy
Research Teams
-- Chiang Ching-kuo Foundation for International Scholarly Exchange
-- Ministry of Science and Technology, R.O.C.
Sponsors
Project 1: 2011-2013-- Research on Applying Interactive Technology
to Experiencing Dunhuang Art
Project 2: 2015-2016 -- Research on Applying Wearable Computing Technology
to Experiencing Dunhuang Art
19
Acquire digitized image data from DunHuang Acedemy
-- High resolution and huge amount of image data
More than 64GB in total.
Up to 300 dpi for each image. Cave 61
3D Model and Texture for Cave 61 Build 3D model with corresponding textures from the image data
20
13 m x 14 m x (6 m ~ 10 m)
Digital Content Production：I. Animations for selected murals in Caves 61 and 254.
II. 2D digital restoration of the “Female Donor Portraits” painting in Cave 61.
III. 3D digital restoration of statues in the central altar of Cave 61.
I. Animations for selected murals in Caves 61 and 254.
Example:
Animation for the “pilgrimage route for Mount Wuta" in Cave 61
-- Manjushri Bodhisattva 文殊師利菩薩13 m x 3.6 m
Four major figures from the "Female Donor Portraits"
painting in Cave 61.
II. 2D digital restoration of the "Female Donor Portraits" painting
in Cave 61.
III. 3D digital restoration of statues in the central altar of Cave 61.
Multiple choices
782年 五台山南禪寺 61窟西壁 文殊赴會
1. 2.
3. 4.
Four possible
statue
combinations
on the central
altar in Cave 61
Four versions of the statue combination on the central altar
Version 1
Version 3
Version 2.a
Version 2.b
mural story
animations
2D digital
restorations
3D digital
restorations
敦煌飛天Flying Spirit in Dunhuang
Exhibition: Future Media
─科技藝術論壇X展示Time: 2015/2/7 ~ 2015/2/15
Place: Museum of NTUE
(北師美術館), Taipei, Taiwan
33
Flying Spirit in Dunhuang
敦煌風華再現 ─ 續說石窟故事
時間：2017/04/26～2017/10/1
地點：台中科博館
Outline
35
簡介
石窟導覽VR：敦煌飛天
走入古畫VR：故宮秋色
太極拳自學AR：大師的身影
展望
The Autumn in Que and Hua Mountains-- now preserved in National Palace Museum
• painted by Zhao Mengfu (趙孟頫）in 1295
• Painted for a friend 周密 who has never been to Shandong
• Que Mountain and Hua Mountain, in Jinan, Shandong
沙洲場景的總面數：881,960、35.2 FPS 黃河場景的總面數：617,576、34.7 FPS
鵲山
華山
山東濟南：鵲山與華山地圖Autumn Colors of NPM
Collaborated with Digimax（太極影音）
two scenes are constructed:
－ a scene of the sandbank
－ a scene of the Yellow River
Scale inconsistency between virtual space and real space
Lack of 3D perspectives
Same style with the original painting
CG simulation of Chinese Ink Painting
Overcoming the VR dizziness
Five difficulties
3D Model－plantation
楊樹細長綠葉密
紅葉樹 深綠葉樹葉尖下垂
枯樹無葉
雜樹葉子細長
雜樹葉子黑短
矮枯樹
雜樹葉綠短密
遠景杉樹 馬尾草 蘆葦
3D Model－miscellaneous
Autumn Colors of National Palace Museum
2016/07/16 -
2016/09/30
42
2016/09/15 -
2017/04/09
故宮潮：夢廻鵲華 多感藝境：蓮池VR
National Taiwan Museum of Fine Arts
Taichung Culture & Creative Industries Park
Outline
44
簡介
石窟導覽VR：敦煌飛天
走入古畫VR：故宮秋色
太極拳自學AR：大師的身影
展望
Introduction
Master at the front
In traditional way of learning TCC…
• follow master demonstrate the moves
• lots of students
• different learning progress
• across all ages
A simulation for illustrating the problems in the traditional way of learning Tai-Chi
Introduction 2
Introduction
Master
A simulation for illustrating the problems in the traditional way of learning Tai-Chi
OcclusionProblem
3
Introduction
A simulation for illustrating the problems in the traditional way of learning Tai-Chi
OcclusionProblem
Distance between the student and the master &
Occlusion caused by other students
4
A simulation for illustrating the problems in the traditional way of learning Tai-Chi
DirectionProblem
Introduction
Master
5
A simulation for illustrating the problems in the traditional way of learning Tai-Chi
OcclusionProblem
Introduction
Has to Move around and
face different directions during practicing
6
We use optical see-through
head-mounted display
System Overview
Microsoft Hololens EPSON Moverio BT-300
22
To solve the occlusion and direction problems of coach,
we design a system that provides the users
Augmented Virtual Coachesstanding on a plane in many different directions
23
To provide the visual feedback of user’s posture,
we propose a drone-enhanced approach:
Augmented Mirror( Third-Person View )
System Overview 24
The view of user when wearing HMD.
There is a drone flying behind the user and capturing the user’s images.
System Overview
Optical See-Through
Self-images
Standard Movement
25
System Overview 27太極拳自學AR：大師的身影
First announced in Intel Asia Innovation Summit
Demonstrations
We interview 60 participants after the experience.
2016 Intel Asia Innovation Summit 2016 imLab@NTU Exhibition
28
Outline
57
簡介
石窟導覽VR：敦煌飛天
走入古畫VR：故宮秋色
太極拳自學AR：大師的身影
展望
• Collaborated with Trend Micro （趨勢科技）• A charity project for remote areas: “Magic VR Bus” & “Space Walker”
展望
Magic VR Bus
• 100 schools in remote areas have been visited.
• 3,500 children in remote areas have experienced VR.
• 86.4% of them think it’s a very good experience and 97.3% children
would like to experience it again.
• We also had a fund raising activity in Taichung for 17 days.
• There are about 1,400 people experienced and contributed
money for our charity project.
Some Figures
Space Walker V1 Content Demo
• Magic VR Bus will visit more schools in remote areas.
• Space Walker will have more new contents
• Immersive solar system menu
• Comet 67p adventure
• Space cleaner
• Space Walker will publish to mainstream VR stores
• STEAM
• Oculus store
• Viveport
What’s next?
Thank you for your attention !
