Reality Comparison

Reality Comparison for Walking/Running in VR System Using Inertial Sensor

こんにちは！I am Ahmad Ridwan Fauzi

I am from Tanaka Hiroshi Laboratory

I am from Indonesia, lives temporarily in Surabaya to finish my study

My university is Sepuluh Nopember Institute of TechnologyIn Japanese, my university’s name is スラバヤ工科大学

1

Research Background

Background▫ User and VR application integration need is

getting higher;▫ There are several VR devices on market.

Comparison is needed to find which VR device has the best realism;

▫ There are also several type of scenes in order to be used on VR application. Comparison among the scenes is also needed;

▫ Walking case is chosen to be evaluated as this is the most common people activity.

2

Objective▫ Integrate user’s daily walking activity to the

application using inertial sensor;▫ Evaluate which virtual reality device has the

best realistic experience (Oculus Rift, Google Cardboard, CAVE);

▫ Evaluate which scene provides the most realistic experience (Google StreetView, 360-Degree Camera, 3D Animation).

3

Signal Processing of Accelerometer Sensor to Detect User’s Motion

Method #1: Threshold Method

▫ Threshold is calibrated by asking user to walk, to walk quickly, to run, and to run quickly;

▫ 4 datasets related to those 4 actions will be obtained and the change of distance will be calculated by this equation:

▫ is the x axis of accelerometer, is the y axis, and is the z axis. While is the i-th time the equation is used;

▫ Each dataset will be compared by the threshold value;▫ Threshold value will be chosen if:

Number of data that exceeds threshold value on walking dataset < walking quickly < running < running

quickly▫ If not, add threshold by 0.025 and compare again until the

condition is satisified;▫ Speed will be calculated using the following equation:

speed = speed + 0.005 * threshold

4

𝑥

𝑦𝑧

0 50 100 150 200 2500

0.5

1

1.5

2

2.5

3

Walk

1 16 31 46 61 76 91 106 121 136 151 166 181 196 211 2260

0.5

1

1.5

2

2.5

3

Walk Fast

1 17 33 49 65 81 97 113 129 145 161 177 193 209 2250

0.51

1.52

2.53

3.54

4.55

Run

1 17 33 49 65 81 97 113 129 145 161 177 193 209 2250

0.51

1.52

2.53

3.54

4.55

Run Fast

Total Data: 236

Data that exceeds value 0.25:Walk, 83Walk Fast, 111 Run, 112Run Fast, 108

Data that exceeds value 0.3:Walk, 75Walk Fast, 99 Run, 104Run Fast, 107

0.250.3

0.250.3

0.250.3

0.250.3

5The results are bad because the difference is

only just a little

Method #2: Step Detection Method

▫ This method detects the change of cosine value of the accelerometer sensor;

▫ Threshold value will be used to increase the number of step, if the cosine value exceeds the threshold;

▫ According to Tomlein et al. (2012), the cosine value is calculated using this expression:

▫ The last 10 cosine value (to filter noise) is calculated using this expression:

▫ Every detected step will add the speed in the application;▫ This method is selected because it is

efficient and easier to use.

6

Tomlein, Michal, et al. Advanced Pedometer for Smartphone-Based Activity Tracking. International Conference on Health Informatics, page 2, 2012.

1 11 21 31 41 51 61 71 81 91 101 111 121 131 141 151 161 171 181 191 201 211 221 231 241 251 261 271 281 291 301 311 321 331 341 351 361-0.2

0

0.2

0.4

0.6

0.8

1

1.2

Walking

1 10 19 28 37 46 55 64 73 82 91 1001091181271361451541631721811901992082172262352442532622712802892983073163253343433520

0.2

0.4

0.6

0.8

1

1.2

Running

0.96

0.96

7

System Design and ConfigurationComponents and functions used on the

system

Scene Photos

3D Animation Scene

Google StreetView Scene

360-degree Camera Scene 8

Oculus Rift System Configuration

▫ PC: rendering image▫ Smartphone: detecting

user’s motion in terms of cosine value

▫ Oculus Rift: detecting rotation and displaying image

▫ Access Point: connector for sending data of cosine value from Smartphone to PC

Smartphone (Motion Detector)Detect movement using

Accelerometer

Cosine Value of X, Y, Z from Accelerometer

Calculate

PC (Processing Unit)Adds speed for the user to

move in application if cosine exceeds threshold

Rotates Camera Around

Send(via LAN)

Rift (Display)Detect Head Rotation

Send(via Connected USB Cable)

Display Calculated Image from PC

Render Image

9

Google Cardboard System Configuration▫ Display Smartphone:

displaying image and detecting rotation

▫ Motion Detector Smartphone: detecting user’s motion in terms of cosine value

▫ Access Point: connector for sending data of cosine value from Motion Detector Smartphone to Display Smartphone

▫ Cardboard: place display smartphone in order to display VR image

Detect movement using Accelerometer

Cosine Value of X, Y, Z from

Accelerometer

Calculate

Display Phone (Processing Unit and Display)

Adds speed for the user to move in application if

cosine exceeds threshold

Detect Head Rotation and Rotates Camera Around

Send(via LAN)

Phone (Motion Detector)

Render and Display Calculated Image

10

CAVE System Configuration

▫ PC: rendering image▫ Smartphone: detecting

user’s motion in terms of cosine value

▫ Access Point: connector for sending data of cosine value from Smartphone to PC

▫ Polhelmus G4 Sensor: detecting rotation

▫ Projectors: displaying image

Phone (Motion Detector)Detect movement using

Accelerometer

Cosine Value of X, Y, Z from Accelerometer

Calculate

PC (Processing Unit)Adds speed for the user to

move in application if cosine exceeds threshold

Rotates Camera Around

Send(via LAN)

CAVE (Display)Display Calculated Image

from PC

Render ImagePolhelmus G4 (Rotation Sensor)

Get rotation data

Send(via USB)

11

Experiment MethodExplaining the Experiment Purpose and Method

Experiment Purpose▫ Experiment plays the main role in this

research as it aims to compare the most realistic device and scene;

▫ It is mainly to understand which device and scene that has the most realistic feeling of all;

▫ It uses questionnaire from the participants to determine the most realistic device and scene.

12

Experiment Method▫ This experiment involves 11 participants;▫ Explanation of how system works is given to

know the purpose of the experiment;▫ Participants are asked to walk, to run each

for 20 seconds for each scene and device;▫ After each device finished, 1 device 3

scenes, they have to fill in the questionnaire;

▫ The previous step is repeated until all 3 devices finished.

13

Experiment ResultThe result is divided into two categories, one is for the devices, another one is for the scenes

Comparison by Device

Immersion Dizziness0

1

2

3

4

5

6

7

Oculus Rift

Very Poor Poor Fair Good Excellent

▫Oculus Rift▪Immersion’s average, 4.09▪Dizziness level average,

1.91▫Google Cardboard

▪Immersion’s average, 3.09▪Dizziness level average,

2.45▫CAVE

▪Immersion’s average, 4.09▪Dizziness level average,

1.72

Number of People

Rank

Average: 4.09 Average: 1.91

14

Comparison by Device

Immersion Dizziness0

1

2

3

4

5

6

7

8

Google Cardboard

Very Poor Poor Fair Good Excellent

▫Oculus Rift▪Immersion’s average, 4.09▪Dizziness level average,

1.91▫Google Cardboard

▪Immersion’s average, 3.09▪Dizziness level average,

2.45▫CAVE

▪Immersion’s average, 4.09▪Dizziness level average,

1.72

Number of People

Rank

Average: 3.09 Average: 2.45

14

Comparison by Device

Immersion Dizziness0

1

2

3

4

5

6

CAVE

Very Poor Poor Fair Good Excellent

▫Oculus Rift▪Immersion’s average, 4.09▪Dizziness level average,

1.91▫Google Cardboard

▪Immersion’s average, 3.09▪Dizziness level average,

2.45▫CAVE

▪Immersion’s average, 4.09▪Dizziness level average,

1.72

Number of People

Rank

Average: 4.09 Average: 1.72

14

Oculus Rift Google Cardboard CAVE0

1

2

3

4

5

6

Google StreetView Realism Feeling

Very Poor Poor Fair Good Excellent

Realism Feeling Comparison by Scene

▫Google StreetView realism feeling▪Using Oculus Rift, 2.27▪Using Google Cardboard, 1.91▪Using CAVE, 2.1

▫360-degree Camera realism feeling▪Using Oculus Rift, 2.91▪Using Google Cardboard, 2.36▪Using CAVE, 2.91

▫3D Animation realism feeling▪Using Oculus Rift, 4.27▪Using Google Cardboard, 4.27▪Using CAVE, 4.54

Number of People

Rank

Average: 2.27 Average: 1.91 Average: 2.1

15

Realism Feeling Comparison by Scene

▫Google StreetView realism feeling▪Using Oculus Rift, 2.27▪Using Google Cardboard, 1.91▪Using CAVE, 2.1

▫360-degree Camera realism feeling▪Using Oculus Rift, 2.91▪Using Google Cardboard, 2.36▪Using CAVE, 2.91

▫3D Animation realism feeling▪Using Oculus Rift, 4.27▪Using Google Cardboard, 4.27▪Using CAVE, 4.54

Oculus Rift Google Cardboard CAVE0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

360-degree Camera Realism Feeling

Very Poor Poor Fair Good Excellent

Number of People

Rank

Average: 2.91 Average: 2.36 Average: 2.91

15

Realism Feeling Comparison by Scene

▫Google StreetView realism feeling▪Using Oculus Rift, 2.27▪Using Google Cardboard, 1.91▪Using CAVE, 2.1

▫360-degree Camera realism feeling▪Using Oculus Rift, 2.91▪Using Google Cardboard, 2.36▪Using CAVE, 2.91

▫3D Animation realism feeling▪Using Oculus Rift, 4.27▪Using Google Cardboard, 4.27▪Using CAVE, 4.54

Oculus Rift Google Cardboard CAVE0

1

2

3

4

5

6

7

8

3D Animation Realism Feeling

Very Poor Poor Fair Good Excellent

Number of People

Rank

Average: 4.27 Average: 4.27 Average: 4.54

15

Oculus Rift Google Cardboard CAVE0

1

2

3

4

5

6

7

Google StreetView Acceleration Feeling

Very Poor Poor Fair Good Excellent

Acceleration Feeling Comparison by Scene

▫Google StreetView acceleration feeling▪Using Oculus Rift, 2.63▪Using Google Cardboard, 2.45▪Using CAVE, 2.63

▫360-degree Camera acceleration feeling▪Using Oculus Rift, 2.91▪Using Google Cardboard, 2.54▪Using CAVE, 3.63

▫3D Animation acceleration feeling▪Using Oculus Rift, 4.36▪Using Google Cardboard, 4.1▪Using CAVE, 4.36

Number of People

Rank

Average: 2.63 Average: 2.45 Average: 2.63

16

Oculus Rift Google Cardboard CAVE0

1

2

3

4

5

6

7

360-degree Camera Acceleration Feeling

Very Poor Poor Fair Good Excellent

Acceleration Feeling Comparison by Scene

▫Google StreetView acceleration feeling▪Using Oculus Rift, 2.63▪Using Google Cardboard, 2.45▪Using CAVE, 2.63

▫360-degree Camera acceleration feeling▪Using Oculus Rift, 2.91▪Using Google Cardboard, 2.54▪Using CAVE, 3.63

▫3D Animation acceleration feeling▪Using Oculus Rift, 4.36▪Using Google Cardboard, 4.1▪Using CAVE, 4.36

Number of People

Rank

Average: 2.91 Average: 2.45 Average: 3.63

16

Oculus Rift Google Cardboard CAVE0

1

2

3

4

5

6

7

8

3D Animation Acceleration Feeling

Very Poor Poor Fair Good Excellent

Acceleration Feeling Comparison by Scene

▫Google StreetView acceleration feeling▪Using Oculus Rift, 2.63▪Using Google Cardboard, 2.45▪Using CAVE, 2.63

▫360-degree Camera acceleration feeling▪Using Oculus Rift, 2.91▪Using Google Cardboard, 2.54▪Using CAVE, 3.63

▫3D Animation acceleration feeling▪Using Oculus Rift, 4.36▪Using Google Cardboard, 4.1▪Using CAVE, 4.36

Number of People

Average: 4.36 Average: 4.1 Average: 4.36

16

Rank

Oculus Rift Google Cardboard CAVE0

1

2

3

4

5

6

7

Google StreetView

Very Poor Poor Fair Good Excellent

Overall Realism

▫Google StreetView overall realism▪Oculus Rift, 2.64▪Google Cardboard, 2.1▪CAVE, 2.18

▫360-degree Camera overall realism▪Oculus Rift, 3.27▪Google Cardboard, 2.54▪CAVE, 3.27

▫3D Animation overall realism▪Oculus Rift, 4.27▪Google Cardboard, 3.91▪CAVE, 4.54

Number of People

Rank

Average: 2.64 Average: 2.1 Average: 2.18

17

Oculus Rift Google Cardboard CAVE0

1

2

3

4

5

6

7

8

360-degree Camera

Very Poor Poor Fair Good Excellent

Overall Realism

▫Google StreetView overall realism▪Oculus Rift, 2.64▪Google Cardboard, 2.1▪CAVE, 2.18

▫360-degree Camera overall realism▪Oculus Rift, 3.27▪Google Cardboard, 2.54▪CAVE, 3.27

▫3D Animation overall realism▪Oculus Rift, 4.27▪Google Cardboard, 3.91▪CAVE, 4.54

Rank

Average: 3.27 Average: 2.54 Average: 3.27

17

Number of People

Oculus Rift Google Cardboard CAVE0

1

2

3

4

5

6

7

8

3D Animation

Very Poor Poor Fair Good Excellent

Overall Realism

▫Google StreetView overall realism▪Oculus Rift, 2.64▪Google Cardboard, 2.1▪CAVE, 2.18

▫360-degree Camera overall realism▪Oculus Rift, 3.27▪Google Cardboard, 2.54▪CAVE, 3.27

▫3D Animation overall realism▪Oculus Rift, 4.27▪Google Cardboard, 3.91▪CAVE, 4.54

Number of People

Rank

Average: 4.27 Average: 3.91 Average: 4.54

17

“

Conclusion

Summary▫ Integrating user’s walking

motion proved to be successful. But it still needs some improvements;

▫ According to the device comparison result, CAVE is the best device, following Oculus Rift and Google Cardboard;

▫ The scene result shows that 3D animation scene is the clear winner, following 360-degree camera and Google StreetView.

18

どうもありがとうございました。Any questions, advices, and critics are very well

welcomed