The VR Continuum: From CAVEs to Digital TV, How VR and AR will change everyday life in the next...

TheVRContinuum:FromCAVEstoDigitalTV,HowVRandARwillchangeeveryday

lifeinthenextdecade

MarceloKnörichZuffo

UniversidadedeSãoPauloEscolaPolitécnica

LaboratóriodeSistemasIntegráveis

CampusPartySãoPaulo,26deJaneirode2010

SIACG2009 2

VirtualReality

Immersion

Interaction Imagination

Immersion

Realism

Interaction

VirtualReality

(Burdea,2003) (LopeseZuffo,2006)

3

HistoricalPerspective-Immersion–  RupestrialPaintingincavesduringthePaleolithicage–  flaresandsmokecompletedtheartificialfeelingofimmersion.–  PicturesofimmersiveenvironmentsinthecaveofLascauxinFrance,magdalenianmen15.000B.C.

SIACG2009

4

HistoricalPerspective-Immersion•  PictorialAllegoryoftheCaveofPlato,427BC,TheRepublicBookVII

REALIDADEVIRTUAL

SIACG2009

5

HistoricalPerspective-Immersion•  Gothiccathedrals,XIIcenturytotheendoftheXVcentury.•  Candles,Vitrals,choirs,organs,incenseburningandpriesttalksintegratesafullyimmersive

REALIDADEVIRTUAL

SIACG2009

HistoricalPerspective-Immersion•  TVEvolutioninthePast85Years•  FamilyOriented•  Interactivityislimited

6 SIACG2009

Wherewewillstop?•  108InchesLCDTVSet

7 SIACG2009

Wherewewillstop?•  ProjectionsupontheBanespaBuildinginSãoPauloCity

8 SIACG2009

9

HistoricalPerspective–InteractiveImmersion•  Firstimmersiveandinteractiveelectronicsystems

Ultimate Display (Sutherland MIT, 1967)

Sketchpad (Sutherland MIT,1963)

Sensorama (MortonHeilig,1962)

SIACG2009

10

HistoricalPerspective-Immersion

1970 Desenho Assistido por

Computador

1980 Desenho e Design Assistido

por Computador

1990 Fotorealismo e

Simulação Física (Virtualização)

REALIDADEVIRTUAL

•  EvolutionofComputerGraphicsanditsapplications

SIACG2009

11

HistoricalPerspective–TheCave•  ThefirstCAVE(CaveAutomaticVirtualEnvironment)wasestablishedattheUniversityofIllinois,Chicago,byresearchersfromThomasFanti,DanielSandinandCarolinaCruz-Neirain1991(Cruz-Neira,1993)

•  Motivation–  ScientificVisualization

•  FirstFullyImmersiveVR

PictureofTheFirstCave1991SIACG2009

12

Microelectronicsadvancesinpast30years

MaterialsusedforIntegratedCircuitManufacturing(Source:Intel)

MOSTransistorSize2001-2011(Source:Intel)SIACG2009

13

DigitalLightProcessing•  DLPTechnology(DigitalLightProcessing)wasinventedin1987byDr.LarryHornbeckatTexasInstruments

•  TheDLPprojectionsystemsaredevicesMEMSs(Micro-Electro-MechanicalSystems)calledDMD(DigitalMicromirrorDevice).

ChipDMD DMDSurface DMDinOperationfonte:Texas

SIACG2009

14

AevoluçãodasCPUseGPUs•  GPUGraphicsProcessingUnit•  CPUComputerProcessingUnit

Evoluçãodosmicroprocessadores(CPUs) EvoluçãodasCPUseGPUs

REALIDADEVIRTUAL

Lei de Moore

SIACG2009

15

CPUxGPU:Desempenho–  GPU:Visualização

•  Ex.:PlacaGráficaATIX800XT(R420):–  520MHz*16pipes*4wide*1flop/inst*1inst/cycle–  66.5GFlops(109operaçõesdepontoflutuante/segundo)

–  CPU:Simulação•  Ex.:MicroprocessadorPentiumP4,3GHz–  6GFlops

PlacaDualCPU2CoreDualx4GPUFotodoChipdaGPUNVIDIAG6800

emprocessode65µmSIACG2009

16

CAVERNADigital•  5Screens•  1Mpixelperscreen•  Pixelsize3mmx3mm•  600lumens/projetor•  total3.000lumens•  FrameRate60Hz

GeneralView GeneralView User

UpperView

SIACG2009

17

CAVERNADigitalVRcluster2001VRcluster2003VRcluster2006

Internet2FAPESPKyatera

CentraldeControle

Rastreador

ProjetoresSIACG2009

18

VRClustersEvolutionatUSP(,SIGGRAPH2002)

2003-2aGeração:• 8nós• dualPentiumXeon3.0GHz• 2GBRAM• placadevideoNVidiaQuadroFX3000G-256MB-AGP8x• 2HDs80SCSI• 2saídasdevídeopornó

2006–3aGeração:• 8nós• dualOpterondualCore3.0GHz• 8GBRAM• duasplacasdevideoNVidiaQuadroFX4400(SLI)-PCIExpressx16dual• 2HDsSATA160GB• 4saídasdevídeopornó

2001–1aGeração:• 6nós• dualPentiumXeon1.76GHz• 1GBRAM• placadevideoWildcat7210(256MB),AGPPRO,• 2HDs80GBSCSI• 1saídadevídeopornó

(IEEEVR2003)

SIACG2009

19

VRClustersonaChipCONSIDERAÇÕESFINAIS

• 8–64CPUS• 8-64GPUS• 1GBUnifiedcache• 50-250Bgates• IntegratedEnergyManagement

• IntegratedMemoryManagement

• DualCPU-GPUcoretobeannouncedbyAMDin2009

MultipleGPUsandCPUsIntegratedinthesamedie

SIACG2009

20

VolumeVisualizationforanatomicalanalysis

RealTimeVisibleHumanTManalysisSIACG2009

21

Molecularvisualization

polipeptidicchain

Useratthecave Diffusion Hemolisina-tree

Potassiumchanel

ElectrofisiologylearningFAPESP

Kyatera-TidiaSoftware:Jinx

SIACG2009

22

CulturalHeritage

CollegeSquare(1stSãoPauloCivilizedLocation)

BaruelBuilding

reconstructionofSãoPaulo1911

SIACG2009

TheDigitalTVTransitioninBrazil•  Itstartedin2003

•  Inourmodel,theUniversityhadakeyrolediscussingandproposinginovations.

•  DigitalTV=EmbeddingaComputeronaTV

•  SoTVswillbemuschmoreinteractiveandimmersiveinthefollowngyears.

MarceloKnörichZuffo Depto.deEngenhariadeSistemasEletrônicosEscolaPolitécnicadaUSP

23

AnalogTerrestrialTVinBrazil•  Majorsourceofinformationandentertainment•  Universaldistribution•  2,3TVsets/house•  Near100MillionTVsets•  7hoursaday

Source: IBGE-PNAD/2003

SIACG2009

AnalogScenario

Transmissão

Analógica

Produção

Digital

Emissora de TV Aberta Analógica

Recepção

Analógica

Distribuição

Analógico Digital

Cidadão

SIACG2009

DigitalScenario

Transmissão

Digital

Produção

Digital

Emissora de TV Digital

Distribuição

Digital

Cidadão

Recepção

Digital

SIACG2009

The4901ACT•  GovernmentACT4901,26November2003–  TheBrazilianDigitalTVSystemisestablished

•  Arguments–  WideAccesstopopulation–  Interoperable–  Mobileandportable–  Needtoconsiderregionaldifferences–  UsabilityandAccessibility–  SocialImplications•  Topromotethedigitalinclusionandsocialinclusion

SIACG2009

ResearchNetworkin2004and2005•  Between2004and2005wecreatedthelargestresearchnetworkeverinBrazil:–  22Consortia–  Near100R&DGroups–  Near1.500Researchersinvolved–  Everythingwasdebated,detailedanddemonstratedatUSPCampusin9December2005.

SIACG2009

SomeSnapshotsfromthatperiod

29

HugePressCoverage

Someprototypes

Alotofcollaboration

The5820ACT•  2006wasanelectionyear•  Sothingsmovedfast:-Theconclusionsare: Wewillwriteanstandardconsidering–  IMultiformat:HDTV(High-def),SDTV(Standard-Def)andLDTV(Low-def)–  II Multiprograming- multipleprogramsinthesamechannel–  III - Interactivity(middlewareandconnectivity).

–  Comerciallaunchin3Dezember2007–  Forumcreation:Industry/University/Broadcast/Goverment

30 SIACG2009

TheSBTVDStandard

Figura: Normas que definem o Sistema Brasileiro de TV Digital

31

N01TransmissionSystem

Serviço Modulação Taxamínima(kbps) Taxamáxima(kbps) LDTV(1segmento) QPSK 280,85 595,76 SDTV(3segmentos) 16-QAM 1685,13 3574,56 HDTV(9segmentos) 64-QAM 7583,13 16085,52

  BST-OFDM modulation   Channel of 6 MHz   13 segments of 428.57 kHz for

information + 1 divided into two bands of custody

  3 modes of operation with different spacing between carriers

  Displacement of 1 / 7 MHz for the top carriers

Service Configuration upon a DTV Channel

BST-COFDM Modulation no SBTVD

Multiprogramming•  Severalprogramsinthesamechannel–  Around19Mbits/second/channel–  Wedodatamultiplexing–  70channelsmeans1.4Tbits/second–  Near540Tbytescanbebroadcastonasingleday

SIACG2009

N01–TransmissionSystem

Figura: Esquema de transmissão no SBTVD

Figura: Faixa de freqüências da TV Digital no Brasil

Multiformat•  1080lines(HD)•  720lines(HD)•  480lines(SD)•  240lines(LD)

1080 linhas

HDTV 16:9

720linhas

EDTV 16:9

480 linhas

SDTV 4:3

240 linhas LDTV

4:3

interactivity•  From2009willviewerscaninteractwiththeTVviaremotecontrol:–  Localinteractivity:noconnectivity•  Stationsendsdatathatcanbenavigated

–  Fullinteractivitywithintermittentconnectivity•  Usersendsthebroadcastshortmessagesviamobileorfixed

–  Fullinteractivitywithpermanentconnectivity•  UserconnectedpermanentlytothestationviaabroadbandInternet

DTVReceiver

37

NavigationandVotingbuttons

EthernetAccessRFInputUSBPort

AnalogOutput

HDMI Interface

256Mbytes32BitCPU4USBs1080iOpenGLEthernet

38

FutureperspectivesInteractiveelectronicswillsupport–  Moreimmersiveness–  Moreinteraction–  Moremodularandflexibe–  Morecomplexandintegrated–  Moresimpleandaccessible–  Morecheaper–  Andmoreandmorepopular

•  ConvergingDTVandVR:interactive3DTVandmakethataccessibleforall

•  BasicallyTVSetswillfollownowtheMOORELAW!

immersion

Realism

Interaction

VirtualReality

39

ImmersiveandInteractive3DTV•  More simple and accessible

•  Low cost display and interactive devices •  Adoption of HDTV 1080i on the Brazilian DTV system

3DTVHDTV1080pLDCDisplayPhillips–LenticularSansungActiveStereoMitshubishiActiveStereo

Auto-stereoscopicdisplayUsinglenticularlenses

InteractiveRemoteControl

Visualcluster

40

•  Clusteringprojectorsforultrarealisticimmersivespaces•  “Casualprojectoralocation

SIACG2009

VisualClusterDemos

41

GoogleEarth CisneBranco

RiodeJaneiro ViewPortResizing

3DandPanoramicVideoDisplay

42

Virtual Rio HD3D

Virtual Rio 240o

SIACG2009

Receptor de TVDigital Acessível •  Parceria com a Secretaria do Estado dos Direitos

da Pessoa com Deficiência (SEDPcD)

•  Projeto visa universalizar o SBTVD pela inclusão dos brasileiros com deficiência

Obrigado!

mkzuffo@lsi.usp.br

MarceloKnörichZuffo Depto.deEngenhariadeSistemasEletrônicosEscolaPolitécnicadaUSP

44