opt vady.pdf

7/29/2019 opt vady.pdf

http://slidepdf.com/reader/full/opt-vadypdf 1/30

Acquisition

Dr. Yossi Rubner

Some slides from: Yung-Yu Chuang (DigiVfx) Jan Neumann, Pat Hanrahan, Alexei Efros

Image/Video Acquisition

Digital Camera

Film

Outline

• Pinhole camera

• Lens

• Lens Aberrations

• Exposure

• Sensors

• Sensor Noise

• Camera Pipeline

Pinhole camera

scene film

Add a barrier to block off most of the rays

• It reduces blurring• The pinhole is known as the aperture• The image is inverted

barrier

pinhole camera



Shrinking the aperture

Why not making the aperture as small as possible?

• Less light gets through• Diffraction effect

Shrinking the aperture

High-end commercial pinhole cameras

~$200

Exposure 4 seconds Exposure 96 minutes

Images copyright ©2000 Zero Image Co.

Pinhole Images



Outline

• Pinhole camera

• Lens


• Exposure

• Sensors

• Sensor Noise

• Camera Pipeline

Adding a lens

scene film

Adding a lens

scene filmlens

“circle ofconfusion”

A lens focuses light onto the film

• There is a specific distance at which objects are “in focus”• Other points project to a “circle of confusion” in the image

(Thin) Lens

• Any object point satisfying this equation is in focus

Thin lens equation: f d d i

111

0

=+−



Circle of Confusion

Circle of confusion proportional to the size of the aperture

Aperture controls Depth of Field

• Changing the aperture affects depth of field– Smaller aperture:

• better DOF

• increased exposure

Depth of Field

http://www.cambridgeincolour.com/tutorials/depth-of-field.htm



Thick Lens

• Corrects aberrations

• Change zoom

Field of View (Zoom)

Field of View (Zoom) Simplified Zoom Lens in Operation

From wikipedia



f

FOV depends of Focal Length

Smaller FOV = larger Focal Length

Outline

• Pinhole camera

• Lens


• Exposure

• Sensors

• Sensor Noise

• Camera Pipeline

Radial Distortion

• Radial distortion of the image

– Caused by imperfect lenses

– Deviations are most noticeable for rays that passthrough the edge of the lens

No distortion Pin cushion Barrel

Radial Distortion



Correcting radial distortion

from Helmut Dersch

No Distortion Barrel Distortion Pincushion Distortion

r u

= r d

+ k 1r d 3

• Radial distance from Image Center:

Radial Distortions

r u

= undistorted radius

r d = distorted radius

Before After

http://www.grasshopperonline.com/barrel_distortion_correction_software.html

Correcting Radial Distortions Correcting Radial Distortions - Video

Video courtesy of Vigilant Ltd.



photo by Robert Johnes

Vignetting

B

A

L3 L1L2

• More light passes through lens L3 for scene point A than scene point B

• Results in spatially non-uniform brightness (in the periphery of the image)

Vignetting

Chromatic Aberration

longitudinal chromatic aberration transverse chromatic aberration

(axial) (lateral)

longitudinal chromatic aberration transverse chromatic aberration

(axial) (lateral)





Near Lens Center Near Lens Outer Edge• Rays parallel to the axis do not converge

• Outer portions of the lens yield smaller focal lengths

Spherical aberration

Astigmatism

Different focal length for inclined rays

Coma

point off the axis depicted as comet shaped blob



Reading: http://www.dpreview.com

Lens Glare

• Stray inter-reflections of light within the optical lens system• Happens when very bright sources are present in the scene

Outline

• Pinhole camera

• Lens


• Exposure

• Sensors

• Sensor Noise

• Camera Pipeline

Exposure

• Two main parameters: – Aperture (in f stop)

– Shutter speed (in fraction of a second)

Aperture

• Aperture is the diameter of the lens opening,usually specified by f-stop, f/D, a fraction of thefocal length.

– f/2.0 on a 50mm means that the aperture is 25mm

– f/2.0 on a 100mm means that the aperture is 50mm

• When a change in f-stopoccurs, the light is eitherdoubled or cut in half.

• Lower f-stop, more light(larger lens opening)

• Higher f-stop, less light(smaller lens opening)



Types of Shutters Leaf Shutter

• Advantages– Uniform illumination

– Entire frame illuminated at once

• Disadvantages– Illumination not constant over time

– Limitations on shutter speed

Focal Plane Shutter

• Advantages– Cost effective (one shutter needed

for all lenses

– Can achieve very fast shutter speeds(~1/10000 sec)

• Disadvantages– May cause time distortion

Shutter speed

Note: shutter times usually obey a power series –each “stop” is a factor of 2

• ¼, 1/8, 1/15, 1/30, 1/60, 1/125, 1/250, 1/500, 1/1000 sec

Usually really is:

• ¼, 1/8, 1/16, 1/32, 1/64, 1/128, 1/256, 1/512, 1/1024 sec



Aperture vs. Shutter

From London and Upton

Shutter speed – Rule of Thumb

• Handheld camera: shutter speed = 1 / f

• Stabilized gear: 2-3 shutter speeds slower

– Optical image stabilization• Canon - IS (Image Stabilizer) lenses• Nikon - VR (Vibration Reduction) lenses• Sigma - OS (Optical Stabilizer) lenses

– CCD-shift stabilization• Konica Minolta’s Anti-Shake cameras• Pentax’s Shake-Reduction cameras

High dynamic range image Short exposure

10-6 106

10-6 106

Real world

radiance

Picture

intensity

dynamic range

Pixel value 0 to 255



Long exposure

10-6 106

10-6 106

Real world

radiance

Picture

intensity

dynamic range

Pixel value 0 to 255

Varying shutter speeds

HDR – High Dynamic Range Outline

• Pinhole camera

• Lens


• Exposure

• Sensors

• Sensor Noise

• Camera Pipeline



Film camera

scene filmlens &motor

aperture& shutter

Digital camera

scene sensorarray

lens &motor

aperture& shutter

• A digital camera replaces film with a sensor array

• Each cell in the array is a light-sensitive diode thatconverts photons to electrons

Spatial Sampling

• When a continuous scene is imaged on thesensor, the continuous image is divided intodiscrete elements - picture elements (pixels)

Spatial Sampling



0x

Sampling

• The density of the sampling denotes theseparation capability of the resulting image

• Image resolution defines the finest details thatare still visible by the image

• We use a cyclic pattern to test the separationcapability of an image

lengthunit

cyclesof numberfrequency =

frequency

1

cyclesof number

lengthunitwavelength ==

200 400 600 800 1000 1200 1400 1600 1800 2000

20

40

60

80

100

Sampling Frequency Sampling Frequency



1D Example:0

Nyquist Frequency

• Nyquist Rule: To observe details at frequency f(wavelength d) one must sample at frequency > 2f(sampling intervals < d/2)

•

• The Frequency 2f is the NYQUIST frequency.

• Aliasing: If the pattern wavelength is less than 2derroneous patterns may be produced.

Aliasing - Moiré Patterns

Quantization Digitizers (Quantization)



Image Sensors

• Considerations

• Speed

• Resolution

• Signal / Noise Ratio

• Cost

MOS (Metal Oxide Semiconductor)

• Photosensitive element

• Charge acquired depends on the number ofphotons which reach the element

• CCD devices are arrays of this basic element

Photoelectric Effect

p h o t o n p

h o t o n

Hole Electron

I n c r e a s i n g

e n e r g y

Valence Band

Conduction Band

1.26eV

• Thermally generated electrons are indistinguishable

from photo-generated electrons

‘Dark Current’.• 1.26eV corresponds to the energy of light with awavelength of 1mm.

CCD (Charge Coupled Device)

• Boyle and Smith, 1969, Bell Labs

• Converts light into electrical signal (pixels)



CCD Readout

•Charge Amplifier

Bucket Brigade

• Integration

• Charge Shift andRead-out

CMOS (Complementary Metal-Oxide Semiconductor)

• Each pixel owns its own charge-voltage conversion

• No need for external shutter (electronic shutter)

• The chip outputs digital bits

• Much faster than CCD devices

CCD vs. CMOS

• Mature technology

• Specific technology

• High production cost

• High power consumption

• Higher fill factor

• Blooming

• Sequential readout

• Recent technology

• Standard IC technology

• Cheap

• Low power

• Less sensitive

• Per pixel amplification

• Random pixel access

• Smart pixels

• On chip integrationwith other components

Spectral Response of a Typical CCD



Sensitivity

∆∆∆∆V

Reset

Output

TintLight intensity (lx)

V o l t a g e D r o p

( V / s )

Slope = sensitivity( V / lx-s)

• Sensitivity

– is the ratio of voltage response to the photo energy illuminated.

Sensor Size Comparison

Sensor Parameters

• Fill factor– The area in the sensor that is truly sensitive to light

– Shift registers and others can reduce it up to a 30%

• Well capacity– The quantity of charge that can be stored in each pixel

– Close relation with pixel dimensions

• Integration time:– Exposure time that is required to excite the CCD elements

– IDepends on the scene brightness

• Acquisition time:– Time needed to transfer the information gathered by the CCD

– Depends on the number of pixels in the sensor

Fill Factor

• The ratio between the light sensitive pixel areaand the total pixel area.

Total pixel area:5µµµµm x 5µµµµm

Fill factor ≈ 40%

PhotoSensing

Area



Outline

• Pinhole camera

• Lens


• Exposure

• Sensors

• Sensor Noise

• Camera Pipeline

Sensor noise

• Photon shot noise

• Blooming

• Diffusion

• Dark current

• Amplifier readout noise

Photon Shot Noise

• Light is quantum in nature

• Noise due to statistics of the detected photonsthemselves

• The probability distribution for N photons to becounted in an observation time T is Poisson

• F = fixed average flux (photons/sec)

( )( )

!,|

N

eFT T F N P

FT N −

=

0 2 0 4 0 6 0 8 0 1 0 00

0 . 0 5

0 .1

0 . 1 5

0 .2

Poisson Distribution, FT = 5

N



0 2 0 4 0 6 0 8 0 1 0 00

0 . 0 2

0 . 0 4

0 . 0 6

0 . 0 8

0 .1

0 . 1 2

0 . 1 4


N

0 2 0 4 0 6 0 8 0 1 0 00

0 . 0 2

0 . 0 4

0 . 0 6

0 . 0 8

0 .1


N


0 2 0 4 0 6 0 8 0 1 0 00

0 . 0 1

0 . 0 2

0 . 0 3

0 . 0 4

0 . 0 5

0 . 0 6

N

• As FT grows, Poisson distribution approachesGaussian distribution

• Shot noise is i.i.d additive Gaussian:

photonsshot N FT ==2

σ

Dark Current Noise

• Electron emission when no light

• Dark current noise is high for long exposures

• To remove (some) of it– Calibrate the camera (make response linear)

– Capture the image of the scene as usual

– Cover the lens with the lens cap and take anotherpicture

– Subtract the second image from the first image



Original image + Dar k Curr ent Noise Image with lens ca p on

Result of subtraction

Copyright Timo Autiokari,

Dark Current Noise Quantum Efficiency

• Not every photon hitting a pixel creates a free electron

• Quantum Efficiency (QE) =– electrons collected / photons hitting the pixel

• QE heavily depends on thewavelength

• QE < 100% degrades the SNR of a camera

• Typical max QE values : 25% (CMOS) … 60% (CCD)

pe SNRQE SNR =

QE [%]

lambda [nm]

blue green red

Outline

• Pinhole camera

• Lens


• Exposure

• Sensors

• Sensor Noise

• Camera Pipeline

Camera pipeline



Sensor Response

• The response of a sensor is proportional to theradiance and the throughput

Measurement Equation

• Scene Radiance L(x,ω,t,λ)

• Optics (x’,ω’) = T(x,ω,λ)

• Pixel Response P(x,λ)

• Shutter S(x,ω,t)

SLR (Single-Lens Reflex)

• Reflex (R in SLR) means that we see throughthe same lens used to take the image.

• Not the case for compact cameras

SLR view finder

lens

Mirror

(when viewing)

Mirror(flipped for exposure)

Film/sensor

PrismYour eye

Light from scene



Camera calibration

• Geometric– How pixel coordinates relate to directions in the

world

• Photometric– How pixel values relate to radiance amounts in the

world

•• GeometricGeometric–– How pixelHow pixel coordinatescoordinates relate torelate to directionsdirections in thein the

worldworld

•• PhotometricPhotometric–– How pixelHow pixel valuesvalues relate torelate to radianceradiance amounts in theamounts in the

worldworld

• Important preprocessing step for many vision and graphicsalgorithms

• Use a color chart with precisely known reflectance

• Use more camera exposures to fill up the curve

• Method assumes constant lighting on all patches and works bestwhen source is far away

• Unique inverse exists because g is monotonic and smooth

3.1%9.0%19.8%36.2%59.1%90%Irradiance = const * Reflectance

P i x e l V a l u e s

0

255

0 1

g

?

?1−

g

Color Chart Calibration

Space of response curves Sensing Color

beam splitter

light

3 CCDBayer pattern

Foveon X3TM



Multi-Chip

wavelengthdependent

Field Sequential

Field Sequential Field Sequential



Embedded color filters

Color filters can be manufactured directly ontothe photodetectors.

Color filter array

Color filter arrays (CFAs)/color filter mosaics

Bayer pattern

Color filter array

Color filter arrays (CFAs)/color filter mosaics

Kodak DCS620x

Bayer’s pattern



Demosaicking CFA’s

bilinear interpolation

original input linear interpolation

Demosaicking CFA’s

Color filter array

red green blue output

X3 technology

red green blue output



Foveon X3 sensor

X3 sensorBayer CFA

Cameras with X3

Sigma SD10, SD9 Polaroid X530 Hanvision HVDUO5M/10M

Color processing

• After color values are recorded, more colorprocessing usually happens:– White balance

– Non-linearity to approximate film response or matchTV monitor gamma

White Balance

automatic white balancewarmer +3



Manual white balance

white balance with

the white book

white balance with

the red book

Camcorder

Interlacing

with interlacingwithout interlacing

Progressive scan

http://www.axis.com/products/video/camera/progressive_scan.htm



Interlace

http://www.axis.com/products/video/camera/progressive_scan.htm

Many Sources for Degradation

• Sampling in space

• Sampling in intensity

• Sampling in color

• Sampling in time

• Sampling in frequency

Exposure, Interlacing

Quantization

Pixels

Lens and pixel PSF (point-spread-function)

Color Filter Array (CFA)

http://www.dmi.unict.it/~battiato/download/DSC1.pdf

Documents

opt vady.pdf