Upload
ahmad-ridwan-fauzi
View
30
Download
1
Embed Size (px)
Citation preview
Reality Comparison for Walking/Running in VR System Using Inertial Sensor
こんにちは!I am Ahmad Ridwan Fauzi
I am from Tanaka Hiroshi Laboratory
I am from Indonesia, lives temporarily in Surabaya to finish my study
My university is Sepuluh Nopember Institute of TechnologyIn Japanese, my university’s name is スラバヤ工科大学
1
Research Background
Background▫ User and VR application integration need is
getting higher;▫ There are several VR devices on market.
Comparison is needed to find which VR device has the best realism;
▫ There are also several type of scenes in order to be used on VR application. Comparison among the scenes is also needed;
▫ Walking case is chosen to be evaluated as this is the most common people activity.
2
Objective▫ Integrate user’s daily walking activity to the
application using inertial sensor;▫ Evaluate which virtual reality device has the
best realistic experience (Oculus Rift, Google Cardboard, CAVE);
▫ Evaluate which scene provides the most realistic experience (Google StreetView, 360-Degree Camera, 3D Animation).
3
Signal Processing of Accelerometer Sensor to Detect User’s Motion
Method #1: Threshold Method
▫ Threshold is calibrated by asking user to walk, to walk quickly, to run, and to run quickly;
▫ 4 datasets related to those 4 actions will be obtained and the change of distance will be calculated by this equation:
▫ is the x axis of accelerometer, is the y axis, and is the z axis. While is the i-th time the equation is used;
▫ Each dataset will be compared by the threshold value;▫ Threshold value will be chosen if:
Number of data that exceeds threshold value on walking dataset < walking quickly < running < running
quickly▫ If not, add threshold by 0.025 and compare again until the
condition is satisified;▫ Speed will be calculated using the following equation:
speed = speed + 0.005 * threshold
4
𝑥
𝑦𝑧
0 50 100 150 200 2500
0.5
1
1.5
2
2.5
3
Walk
1 16 31 46 61 76 91 106 121 136 151 166 181 196 211 2260
0.5
1
1.5
2
2.5
3
Walk Fast
1 17 33 49 65 81 97 113 129 145 161 177 193 209 2250
0.51
1.52
2.53
3.54
4.55
Run
1 17 33 49 65 81 97 113 129 145 161 177 193 209 2250
0.51
1.52
2.53
3.54
4.55
Run Fast
Total Data: 236
Data that exceeds value 0.25:Walk, 83Walk Fast, 111 Run, 112Run Fast, 108
Data that exceeds value 0.3:Walk, 75Walk Fast, 99 Run, 104Run Fast, 107
0.250.3
0.250.3
0.250.3
0.250.3
5The results are bad because the difference is
only just a little
Method #2: Step Detection Method
▫ This method detects the change of cosine value of the accelerometer sensor;
▫ Threshold value will be used to increase the number of step, if the cosine value exceeds the threshold;
▫ According to Tomlein et al. (2012), the cosine value is calculated using this expression:
▫ The last 10 cosine value (to filter noise) is calculated using this expression:
▫ Every detected step will add the speed in the application;▫ This method is selected because it is
efficient and easier to use.
6
Tomlein, Michal, et al. Advanced Pedometer for Smartphone-Based Activity Tracking. International Conference on Health Informatics, page 2, 2012.
1 11 21 31 41 51 61 71 81 91 101 111 121 131 141 151 161 171 181 191 201 211 221 231 241 251 261 271 281 291 301 311 321 331 341 351 361-0.2
0
0.2
0.4
0.6
0.8
1
1.2
Walking
1 10 19 28 37 46 55 64 73 82 91 1001091181271361451541631721811901992082172262352442532622712802892983073163253343433520
0.2
0.4
0.6
0.8
1
1.2
Running
0.96
0.96
7
System Design and ConfigurationComponents and functions used on the
system
Scene Photos
3D Animation Scene
Google StreetView Scene
360-degree Camera Scene 8
Oculus Rift System Configuration
▫ PC: rendering image▫ Smartphone: detecting
user’s motion in terms of cosine value
▫ Oculus Rift: detecting rotation and displaying image
▫ Access Point: connector for sending data of cosine value from Smartphone to PC
Smartphone (Motion Detector)Detect movement using
Accelerometer
Cosine Value of X, Y, Z from Accelerometer
Calculate
PC (Processing Unit)Adds speed for the user to
move in application if cosine exceeds threshold
Rotates Camera Around
Send(via LAN)
Rift (Display)Detect Head Rotation
Send(via Connected USB Cable)
Display Calculated Image from PC
Render Image
9
Google Cardboard System Configuration▫ Display Smartphone:
displaying image and detecting rotation
▫ Motion Detector Smartphone: detecting user’s motion in terms of cosine value
▫ Access Point: connector for sending data of cosine value from Motion Detector Smartphone to Display Smartphone
▫ Cardboard: place display smartphone in order to display VR image
Detect movement using Accelerometer
Cosine Value of X, Y, Z from
Accelerometer
Calculate
Display Phone (Processing Unit and Display)
Adds speed for the user to move in application if
cosine exceeds threshold
Detect Head Rotation and Rotates Camera Around
Send(via LAN)
Phone (Motion Detector)
Render and Display Calculated Image
10
CAVE System Configuration
▫ PC: rendering image▫ Smartphone: detecting
user’s motion in terms of cosine value
▫ Access Point: connector for sending data of cosine value from Smartphone to PC
▫ Polhelmus G4 Sensor: detecting rotation
▫ Projectors: displaying image
Phone (Motion Detector)Detect movement using
Accelerometer
Cosine Value of X, Y, Z from Accelerometer
Calculate
PC (Processing Unit)Adds speed for the user to
move in application if cosine exceeds threshold
Rotates Camera Around
Send(via LAN)
CAVE (Display)Display Calculated Image
from PC
Render ImagePolhelmus G4 (Rotation Sensor)
Get rotation data
Send(via USB)
11
Experiment MethodExplaining the Experiment Purpose and Method
Experiment Purpose▫ Experiment plays the main role in this
research as it aims to compare the most realistic device and scene;
▫ It is mainly to understand which device and scene that has the most realistic feeling of all;
▫ It uses questionnaire from the participants to determine the most realistic device and scene.
12
Experiment Method▫ This experiment involves 11 participants;▫ Explanation of how system works is given to
know the purpose of the experiment;▫ Participants are asked to walk, to run each
for 20 seconds for each scene and device;▫ After each device finished, 1 device 3
scenes, they have to fill in the questionnaire;
▫ The previous step is repeated until all 3 devices finished.
13
Experiment ResultThe result is divided into two categories, one is for the devices, another one is for the scenes
Comparison by Device
Immersion Dizziness0
1
2
3
4
5
6
7
Oculus Rift
Very Poor Poor Fair Good Excellent
▫Oculus Rift▪Immersion’s average, 4.09▪Dizziness level average,
1.91▫Google Cardboard
▪Immersion’s average, 3.09▪Dizziness level average,
2.45▫CAVE
▪Immersion’s average, 4.09▪Dizziness level average,
1.72
Number of People
Rank
Average: 4.09 Average: 1.91
14
Comparison by Device
Immersion Dizziness0
1
2
3
4
5
6
7
8
Google Cardboard
Very Poor Poor Fair Good Excellent
▫Oculus Rift▪Immersion’s average, 4.09▪Dizziness level average,
1.91▫Google Cardboard
▪Immersion’s average, 3.09▪Dizziness level average,
2.45▫CAVE
▪Immersion’s average, 4.09▪Dizziness level average,
1.72
Number of People
Rank
Average: 3.09 Average: 2.45
14
Comparison by Device
Immersion Dizziness0
1
2
3
4
5
6
CAVE
Very Poor Poor Fair Good Excellent
▫Oculus Rift▪Immersion’s average, 4.09▪Dizziness level average,
1.91▫Google Cardboard
▪Immersion’s average, 3.09▪Dizziness level average,
2.45▫CAVE
▪Immersion’s average, 4.09▪Dizziness level average,
1.72
Number of People
Rank
Average: 4.09 Average: 1.72
14
Oculus Rift Google Cardboard CAVE0
1
2
3
4
5
6
Google StreetView Realism Feeling
Very Poor Poor Fair Good Excellent
Realism Feeling Comparison by Scene
▫Google StreetView realism feeling▪Using Oculus Rift, 2.27▪Using Google Cardboard, 1.91▪Using CAVE, 2.1
▫360-degree Camera realism feeling▪Using Oculus Rift, 2.91▪Using Google Cardboard, 2.36▪Using CAVE, 2.91
▫3D Animation realism feeling▪Using Oculus Rift, 4.27▪Using Google Cardboard, 4.27▪Using CAVE, 4.54
Number of People
Rank
Average: 2.27 Average: 1.91 Average: 2.1
15
Realism Feeling Comparison by Scene
▫Google StreetView realism feeling▪Using Oculus Rift, 2.27▪Using Google Cardboard, 1.91▪Using CAVE, 2.1
▫360-degree Camera realism feeling▪Using Oculus Rift, 2.91▪Using Google Cardboard, 2.36▪Using CAVE, 2.91
▫3D Animation realism feeling▪Using Oculus Rift, 4.27▪Using Google Cardboard, 4.27▪Using CAVE, 4.54
Oculus Rift Google Cardboard CAVE0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
360-degree Camera Realism Feeling
Very Poor Poor Fair Good Excellent
Number of People
Rank
Average: 2.91 Average: 2.36 Average: 2.91
15
Realism Feeling Comparison by Scene
▫Google StreetView realism feeling▪Using Oculus Rift, 2.27▪Using Google Cardboard, 1.91▪Using CAVE, 2.1
▫360-degree Camera realism feeling▪Using Oculus Rift, 2.91▪Using Google Cardboard, 2.36▪Using CAVE, 2.91
▫3D Animation realism feeling▪Using Oculus Rift, 4.27▪Using Google Cardboard, 4.27▪Using CAVE, 4.54
Oculus Rift Google Cardboard CAVE0
1
2
3
4
5
6
7
8
3D Animation Realism Feeling
Very Poor Poor Fair Good Excellent
Number of People
Rank
Average: 4.27 Average: 4.27 Average: 4.54
15
Oculus Rift Google Cardboard CAVE0
1
2
3
4
5
6
7
Google StreetView Acceleration Feeling
Very Poor Poor Fair Good Excellent
Acceleration Feeling Comparison by Scene
▫Google StreetView acceleration feeling▪Using Oculus Rift, 2.63▪Using Google Cardboard, 2.45▪Using CAVE, 2.63
▫360-degree Camera acceleration feeling▪Using Oculus Rift, 2.91▪Using Google Cardboard, 2.54▪Using CAVE, 3.63
▫3D Animation acceleration feeling▪Using Oculus Rift, 4.36▪Using Google Cardboard, 4.1▪Using CAVE, 4.36
Number of People
Rank
Average: 2.63 Average: 2.45 Average: 2.63
16
Oculus Rift Google Cardboard CAVE0
1
2
3
4
5
6
7
360-degree Camera Acceleration Feeling
Very Poor Poor Fair Good Excellent
Acceleration Feeling Comparison by Scene
▫Google StreetView acceleration feeling▪Using Oculus Rift, 2.63▪Using Google Cardboard, 2.45▪Using CAVE, 2.63
▫360-degree Camera acceleration feeling▪Using Oculus Rift, 2.91▪Using Google Cardboard, 2.54▪Using CAVE, 3.63
▫3D Animation acceleration feeling▪Using Oculus Rift, 4.36▪Using Google Cardboard, 4.1▪Using CAVE, 4.36
Number of People
Rank
Average: 2.91 Average: 2.45 Average: 3.63
16
Oculus Rift Google Cardboard CAVE0
1
2
3
4
5
6
7
8
3D Animation Acceleration Feeling
Very Poor Poor Fair Good Excellent
Acceleration Feeling Comparison by Scene
▫Google StreetView acceleration feeling▪Using Oculus Rift, 2.63▪Using Google Cardboard, 2.45▪Using CAVE, 2.63
▫360-degree Camera acceleration feeling▪Using Oculus Rift, 2.91▪Using Google Cardboard, 2.54▪Using CAVE, 3.63
▫3D Animation acceleration feeling▪Using Oculus Rift, 4.36▪Using Google Cardboard, 4.1▪Using CAVE, 4.36
Number of People
Average: 4.36 Average: 4.1 Average: 4.36
16
Rank
Oculus Rift Google Cardboard CAVE0
1
2
3
4
5
6
7
Google StreetView
Very Poor Poor Fair Good Excellent
Overall Realism
▫Google StreetView overall realism▪Oculus Rift, 2.64▪Google Cardboard, 2.1▪CAVE, 2.18
▫360-degree Camera overall realism▪Oculus Rift, 3.27▪Google Cardboard, 2.54▪CAVE, 3.27
▫3D Animation overall realism▪Oculus Rift, 4.27▪Google Cardboard, 3.91▪CAVE, 4.54
Number of People
Rank
Average: 2.64 Average: 2.1 Average: 2.18
17
Oculus Rift Google Cardboard CAVE0
1
2
3
4
5
6
7
8
360-degree Camera
Very Poor Poor Fair Good Excellent
Overall Realism
▫Google StreetView overall realism▪Oculus Rift, 2.64▪Google Cardboard, 2.1▪CAVE, 2.18
▫360-degree Camera overall realism▪Oculus Rift, 3.27▪Google Cardboard, 2.54▪CAVE, 3.27
▫3D Animation overall realism▪Oculus Rift, 4.27▪Google Cardboard, 3.91▪CAVE, 4.54
Rank
Average: 3.27 Average: 2.54 Average: 3.27
17
Number of People
Oculus Rift Google Cardboard CAVE0
1
2
3
4
5
6
7
8
3D Animation
Very Poor Poor Fair Good Excellent
Overall Realism
▫Google StreetView overall realism▪Oculus Rift, 2.64▪Google Cardboard, 2.1▪CAVE, 2.18
▫360-degree Camera overall realism▪Oculus Rift, 3.27▪Google Cardboard, 2.54▪CAVE, 3.27
▫3D Animation overall realism▪Oculus Rift, 4.27▪Google Cardboard, 3.91▪CAVE, 4.54
Number of People
Rank
Average: 4.27 Average: 3.91 Average: 4.54
17
“
Conclusion
Summary▫ Integrating user’s walking
motion proved to be successful. But it still needs some improvements;
▫ According to the device comparison result, CAVE is the best device, following Oculus Rift and Google Cardboard;
▫ The scene result shows that 3D animation scene is the clear winner, following 360-degree camera and Google StreetView.
18
どうもありがとうございました。Any questions, advices, and critics are very well
welcomed