Click here to load reader

CS 691B Computational Photography Instructor: Gianfranco Doretto Modeling Light

  • View

  • Download

Embed Size (px)

Text of CS 691B Computational Photography Instructor: Gianfranco Doretto Modeling Light

CS 591I / CS791B Computational Photography

CS 691B Computational PhotographyInstructor: Gianfranco DorettoModeling LightWhat is light?Electromagnetic radiation (EMR) moving along rays in spaceR() is EMR, measured in units of power (watts) is wavelengthLight:Travels farTravels fastTravels in straight linesInteracts with stuffBounces off thingsIs produced in natureHas lots of energy -- Steve Shafer

Figures Stephen E. Palmer, 2002What do we see?3D world2D image

What do we see?3D world2D imagePainted backdropOn Simulating the Visual ExperienceJust feed the eyes the right dataNo one will know the difference!Philosophy:Ancient question: Does the world really exist?Science fiction:Many, many, many books on the subject, e.g. slowglass from Light of Other DaysLatest take: The MatrixPhysics:Slowglass might be possible?Computer Science:Virtual Reality

To simulate we need to know:What does a person see?The Plenoptic FunctionQ: What is the set of all things that we can ever see?A: The Plenoptic Function (Adelson & Bergen)

Lets start with a stationary person and try to parameterize everything that he can see

Figure by Leonard McMillanGrayscale snapshotis intensity of light Seen from a single view pointAt a single timeAveraged over the wavelengths of the visible spectrum(can also do P(x,y), but spherical coordinate are nicer)

P(,)Color snapshotis intensity of light Seen from a single view pointAt a single timeAs a function of wavelength

P(,,)A movieis intensity of light Seen from a single view pointOver timeAs a function of wavelength

P(,,,t)Holographic movieis intensity of light Seen from ANY viewpointOver timeAs a function of wavelengthP(,,,t,VX,VY,VZ)

The Plenoptic FunctionCan reconstruct every possible view, at every moment, from every position, at every wavelengthContains every photograph, every movie, everything that anyone has ever seen! it completely captures our visual reality! Not bad for a functionP(,,,t,VX,VY,VZ)

Sampling Plenoptic Function (top view)Just lookup -- Quicktime VRRayLets not worry about time and color:

5D3D position2D directionP(,,VX,VY,VZ)

SurfaceCamera No Change in


How can we use this?14Ray ReuseInfinite lineAssume light is constant (vacuum)

4D2D direction2D positionnon-dispersive mediumOnly need plenoptic surface

vuSynthesizing novel viewsLumigraph / LightfieldOutside convex space

4DStuffEmptyLumigraph - Organization 2D position2D directionuLumigraph - Organization 2D position2D position

2 plane parameterizationusLumigraph - Organization 2D position2D position

2 plane parameterizationsuvu,vs,ttu,vs,tLumigraph - OrganizationHold u,v constantLet s,t varyAn image

u,vs,tLumigraph / Lightfield

Lumigraph - Capture Idea 1Move camera carefully over u,v planeGantrysee Lightfield paper

u,vs,tLumigraph - Capture Idea 2Move camera anywhereRebinningsee Lumigraph paper

u,vs,tLumigraph - RenderingFor each output pixeldetermine u,v,s,t

eitheruse closest discrete RGBinterpolate near valuesusLumigraph - RenderingNearestclosest uclosest sdraw it

Blend 16 nearestquadrilinear interpolationusStanford multi-camera array640 480 pixels 30 fps 128 cameras

synchronized timingcontinuous streamingflexible arrangement

28Light field photography using a handheld plenoptic camera

Ren Ng, Marc Levoy, Mathieu Brdif,Gene Duval, Mark Horowitz and Pat Hanrahan29Conventional versus light field camera

30Conventional versus light field camera

uv-planest-plane31Prototype camera4000 4000 pixels 292 292 lenses = 14 14 pixels per lens

Contax medium format cameraKodak 16-megapixel sensor

Adaptive Optics microlens array125 square-sided microlenses



Digitally stopping-downstopping down = summing only the central portion of each microlens34

Digital refocusingrefocusing = summing windows extracted from several microlenses

35Example of digital refocusing

36Digitally moving the observermoving the observer = moving the window we extract from the microlenses

37Example of moving the observer


Moving backward and forward


On sale now:

3D LumigraphOne row of s,t planei.e., hold t constants,tu,v3D LumigraphOne row of s,t planei.e., hold t constantthus s,u,va row of imagessu,vby David Dewey

P(x,t)2D: ImageWhat is an image?

All rays through a pointPanorama?ImageImage plane

2DpositionSpherical PanoramaAll light rays through a point form a ponoramaTotally captured in a 2D array -- P(q,f)Where is the geometry???

See also: 2003 New Years Eve Other ways to sample Plenoptic FunctionMoving in time: Spatio-temporal volume: P(q,f,t)Useful to study temporal changesLong an interest of artists:

Claude Monet, Haystacks studiesSpace-time images

Other ways to slice theplenoptic functionxytThe Theatre Workshop Metaphor

desired image(Adelson & Pentland,1996)PainterLighting DesignerSheet-metalworkerPainter (images)

Lighting Designer (environment maps)

Show Naimark SF MOMA video Worker (geometry)

Let surface normals do all the work! working togetherWant to minimize costEach one does whats easiest for himGeometry big thingsImages detailLighting illumination effects

clever Italians Slide CreditsThis set of sides also contains contributionskindly made available by the following authors Alexei Efros Richard Szelisky Michael Cohen Stephen E. Palmer Ren Ng

Search related