Virtual Reality
Lecture1. Introduction to Virtual Reality
고려대학교 그래픽스 연구실
Contents
• What is VR?• Conceptual Model of VR• VR Related Areas• History• State of the Art and R&D Issues• Some Application Examples
Science vs. Engineering
• What is not VR.
• high road - replication of reality low road - 3-D interface / interaction
• VR = studies on reality computational reality
• analogous to AI– lots of hypes– AI as a science / engineering
What is not VR.
• VR as computational reality– To seek for the computational model of reality.– To apply the model to the VR system.
• VR can be understood in the context of modeling efforts.– intelligence– linguistics– emotion– life– reality– compuational “X”
Definition : What is VR?
• Computer generated environment that is …– Immersive (like IMAX,dome projection),– Interactive (like computer game),– Multi-sensory,– Viewer-centered,– 3-D, and– The combination of technologies required to build such enviro
nments.
VR as a computer technology
• Technological trend– powerful– smart– physical
• computer - human interaction
• primary concern --- software other important issues --- hardware, human
factors, social issues,
infrastructure
Interacting with computers
ComputerWorld
New Paradigm for HCI
Conventional Computer
ComputerWorld
VR-based Computer
ComputerWorld
VR as a media
• All medium attempt to create the “virtual presence”.– Theater, poem, fine arts, novel, telephone, movies, TV, ...
• VR vs. existing media– immersive– interactive– 3-D– multimodal– mediated
• Information is not sent back and forth.• Mediated environments are created and then experience.
• effectiveness of communication depends on ... the sense of “being
there”.
• virtual presence
• virtual presence depends on ...– sensory breadth– sensory depth– interactivity
VR Application
Education/Training
DesignEngineering
Medicine
ScientificVisualization
Entertainment
Communication
ProductsEngineering
MuseumArts VR application
VR technologies
Character CRT
Graphic CRT
Virtual console
Virtual Reality
Tele-conference
TV phone
Telephone
Keyboard
Mouse tablet
3D mouse
Computersimulation/visualization
Tele-Existence
Tele-Operation
Computergraphics
RealtimeCG
Video arts
3D CAD
ComputerAided Design
Virtual products
design
All technologies meet together at VR !!
VR in Real World vs. Virtual World
Virtual Reality
Tele-Existence inReal World
Tele-Existence inVirtual World
• Physical World
• Quasi Physical World
• Non Physical World
• Standard Tele-Existence
• Augmented Tele-Existence
• Size• Sensation• Time
컴퓨터
로보트
인간에의 임장감 제시
시각 , 청각 , 촉각 , 미각 ,후각
체성감각 힘감각
인간의 상태추정외부상태(
비구속 ,
실시간)
내부상태(
비침투
실시간)
운동
음성
몸통
다리
팔
눈
머리
뇌파 , 심전 ,
근전 , 맥박 ,
혈압 , 발한 ,
etc
감각정보의
통합
인간에의
적합변환
인간의 행동음지추정
감각정보
감각정보
제어정보
제어정보
다른
로보트
상호작용
상호
작용 가상환경
실환경
가상인간
다른 가상인간
다른 Tele Existens system 으로
다른 Tele Existens system 으로
VR system 의 구성
Conceptual Model of VR
• 3D image Large-scale display, Head Mounted Display• Sound field by DSP • Force beedback mechanism• Tactile display
SimulationSystem
Computer
Display system
Sensing system• Non-contact type
magnetic field supersonic wave infrared light
• Contact type optical fiber strain gauge potentio-meter
Human
VR Related areas
1. Training simulation
2. Tele-operation
3. Computer graphics
4. Artificial intelligence
Training simulation
• Differences– reconfigurable by changing software– may include highly unnatural environment– highly interactive and adaptive– use of a wide varielty of human sensing modalities
and sensorimotor systems– highly immersive– near-field is synthetic; far-field is synthetic.
Tele-operation
• for at least 30 years.
• Tele-operator– directly (manually) controlled tele-operator– tele-robot
• Tele-operation vs. Virtual reality
• Tele-presence vs. Virtual presence
Computer graphics
• Modeling• Motion control (animation)• Rendering• User interface
Artificial intelligence
• Studies on perception and cognition
• Testbed for AI research
History
• 1’st stage: some visionaries– Morton Heilig : ExperienceTheatre(1962)– Ivan Surtherland : Sketchpad(1963), HMD(1966)– Myron Krueger : Artificial Reality(1972)– William Gibson : “Cyberspace” in Neuromancer(1984)
• 2’nd stage: technology development for specific purposes– training simulator : Earlier works– space exploration : NASA for astronaut simulation– tele-operation
History(con’t)
• 3’rd stage: VR as the general-purpose technology– Jaron Lanier : VPL(1987) – Dataglove,EyePhone,VR system– VR industry : Division Ltd. Sense8, WorldDesign(production h
ouse,W-Industry(game)– VR academia : MIT, UNC, UW,Tokyo U.
• Next stage: Toward a scientific discipline– computational reality– a new computing paradigm– a new media– a new art form– representation, creation and operation of virtual worlds
State of the Art & Issues
• Reference– Virtual Reality: Scientific and Technological
Challenges”, pp. 35-66, National Research Council, National Academic Press, 1995.
• Areas of the study– application domains– psychological issues– VR technologies– evaluation of VR systems
1. Application domains
• design, manufacturing & marketing• medicine, health care• hazardous operations• training• entertainment, military• experimental psychology• education• information visualization• tele-communication, tele-travel
2. Psychological topics
• human performance characteristics
• alteration of sensori-motor loops
• developing the cognitive model
• cognitive side-effect
3. VR technologies
• Gap between the current technology the required technology (exception -- entertainment, tele-operation)
(1) human-machine interface(2) computer generation of VE(3) tele-robotics(4) network
Human-machine interface (cont’)
• visual channelvisual channel• auditory channelauditory channel• haptic channelhaptic channel• motion interfacemotion interface• position trackingposition tracking• video camera• microphone• others
Visual channel
• visual display– HMD– OHD (off-head display)
• perceptual effects– mis-registration– sensori-motor alteration– distortion– time-delay– noise
• research issues– ergonomics– improvement of resolution and fov– wireless– integration of visual, auditory, position tracking– sun glass-like– see-through option– exploiting foveal and peripheral vision
Auditory channel
• Current hardware is adequate.
• Research issues– perceptual issues
• similar to the visual channel• use for sensory substitution (for visual, haptic)
– auditory scene analysis– hear-through display
Position tracking and mapping
• tracking = finding a pointmapping = finding a 3D surface
(e.g., environmental mapping)
• tracking mechanisms– mechanical linkage– magnetic– optical– acoustic– intertial
• eye tracking
• research issues– tracking– mapping
Haptic channel
• force, pressure, tactile feedback
• unique characteristics– Haptic interface requires manipulation and sensing
• mechanism– body-based -- glove, exoskeleton– ground-based -- joystick
• Research issues– haptic science = study on the human haptics
(bio-mechanical, psychophysical, cognitive)
– tool-hand system (which takes its metaphor from real tools.)
– creating the haptic illusion– the interaction effects of haptic and vision– texture, temperature devices
Motion interface
• motion– whole-body motion
• passive -- e.g., motion platform• active -- e.g., locomotion
– part-body motion• passive• active
• motion cues– vestibular system -- inertial– motor– visual– auditory– proprioceptive / kinesthetic -- muscle– tactile
• motion interface– inertial system
• moves the body (e.g., treadmill, motion platform)
– non-inertial system• simulates motion
Other types of interfaces
• olfactory (smell)• gustatory (taste)• heat, wind, humidity
• speech
• direct physiological sensing and control
VR technologies
(1) human-machine interface
(2) computer generation of VE
(3) tele-robotics
(4) network
(2) Generation of virtual environments
• 3D image Large-scale display, Head Mounted Display• Sound field by DSP • Force beedback mechanism• Tactile display
SimulationSystem
Computer
Display system
Sensing system• Non-contact type
magnetic field supersonic wave infrared light
• Contact type optical fiber strain gauge potentio-meter
Human
Generation of VE(cont’)
• the core issue
• general-purpose VR system?
• trade-off between realism and interactivity
• requirements– frame rate– response time– scene quality
• hardware• interaction and navigation• VE management
– simulation– rendering
• modeling• autonomous agent• hypermedia interaction• OS
Generation of VE(cont’)
VE management - simulation
• Task : simulating everyday world• Traditional simulation methods do not
work. (requires pre-processing)
• Need : “meta-modeling”
VE management - rendering
• Issue : load balancing– 1. partitioning VE– 2. LOD
• Much work has been done on static scene.
• Research issues– 1. dynamic scene– 2. parallel rendering
OS
• real-time, multi-modal requirements
• very high-resolution time slicing
• atomic, transparent distribution of tasks• large number of light-weighted processors,
communicating by means of shared memory
• support for time-critical computing:– negotiated, graceful degradation– guaranteed frame rate, lag time
VR technologies
(1) human-machine interface
(2) computer generation of VE
(3) tele-robotics
(4) network
(3) Tele-robotics
• tele-robotics and VR
• hardware• time-delay problem• distributed tele-robots
(4) Network
• The future is here!
• applications– distance learning– group entertainment– distributed training– distributed design
• current• future• What is needed
Research Organizations
International Efforts
U.S.A. defense, space, visualization, medicineU.K. education, training, entertainmentGermanyJapan VR as a logical extension of
robotics, automation, HDTV.
Academia
• HIT Lab, University of Washington• University of North Carolina• Media Lab, MIT• Georgia Institute of Technology• Naval Postgraduate School• University of Pennsylvania• University of California at Berkeley• University of Illinois - Chicago• Columbia University• University of Toronto
And, Some VR Example… (Video)