View
214
Download
0
Category
Preview:
Citation preview
9/30/11
1
ENGG1015 Digital Images
1st Semester, 2011
Dr Edmund Lam
Department of Electrical and Electronic Engineering
The content in this lecture is based substan1ally on last year’s from Dr Hayden So, but all errors should be blamed on me…
Back to top-‐level
1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 2
Applications
Systems
Digital Logic
Circuits
Electrical Signals
High Level
Low Level
• Computer & Embedded Systems • Computer Network • Mobile Network
• Image & Video Processing
• Combinational Logic • Boolean Algebra
• Basic Circuit Theory
• Voltage, Current • Power & Energy
This week
Back to top-‐level
1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 3
Applications
Systems
Digital Logic
Circuits
Electrical Signals
High Level
Low Level
• Computer & Embedded Systems • Computer Network • Mobile Network
• Image & Video Processing
• Combinational Logic • Boolean Algebra
• Basic Circuit Theory
• Voltage, Current • Power & Energy
This week
Digital Images
1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 4
Representation
Processing Hardware
Note: The three parts are mostly independent (even with different “language”), but they do intersect.
1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 5
Representing Images
bitmap
R G B
pixel
An image is broken down into small regions called picture elements (pixels)
Digital image (bitmap): A pixel-by-pixel representation of an image. Implications?
1st semester, 2011 6 Digital Images - ENGG1015 - Dr. E. Lam
9/30/11
2
Image Dimensions Image Size
• The number of pixel in X-Y direction • Sometimes quoted using the total number of pixels in a
picture (N megapixels)
Image Resolution • The density of pixels • Measured by pixel-per-inch (PPI) • NOT the number of pixels
15
14
1st semester, 2011 7 Digital Images - ENGG1015 - Dr. E. Lam
Representing Pixels Each pixel is represented by one or more values Black & white images (binary images):
• Each pixel is represented by exactly 1 value (B or W) • 1 bit is enough to represent 2 possible values
Grayscale images: • Each pixel is usually a byte (8 bits), keeping the brightness
or gray levels Color images:
• Each pixel represented a group of color components of that location… often three colors
• Different color systems: RGB, CMYK, YCbCr, etc Hyperspectral images:
• Many values per pixel location, corresponding to different frequencies
1st semester, 2011 8 Digital Images - ENGG1015 - Dr. E. Lam
Binary and Grayscale Images
Binary Image
Each pixel is 1 bit, either 0 or 1
Dithering is used to produce (fake) different intensities
Grayscale Image
Each pixel is usually a byte (8-bit), keeping the brightness or gray levels
1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 9
B&W B&W (w/ dither) Grayscale Color Images
indexed color image # of color support
depends on the # of bits for each pixel • 4 bits 24=16 colors • 8 bits 28=256 colors
Color Look-Up Tables (LUTs)
Color palette
24-bit color image Each pixel is
represented by 3 bytes using a certain color model
Supports 256x256x256 colors • 16 million colors
1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 10
16 colors 256 colors 16M colors
RGB Color Model Additive color model Primary colors: Red, Green,
and Blue Secondary colors obtained by
additive mixing of primary colors: Cyan, Magenta, Yellow
Commission Internationale d'Eclairage (CIE) in 1931 specifies red to be 700nm, green to be 546.1nm and blue to be 435.8nm
Used in media that transmit light (e.g. TV)
1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 11
CMY Color Model Subtractive color model Subtractive primaries:
Cyan, magenta, and yellow
A subtractive primary absorbs a primary color and reflects the other two • E.g. Cyan absorbs red and
reflect blues and green
Used in printing device
1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 12
9/30/11
3
Colors that Can be Reproduced
1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 13
Printing an Image Print Size
• Depends on the mapping between printer’s resolution, image resolution & image size
• A Printer’s printing resolution is usually higher than an image’s resolution because multiple dots of ink are needed to created color of an image pixel
Color Space • On screen display: (additive) • Printing devices: (subtractive)
Color Production • Each pixel may have different color • Each ink drop has only on-off (one bit!)
1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 14
Dithering Create the illusion of new colors and shades by
varying the pattern of dots. • E.g. Newspaper photographs are dithered. If you
look closely, you can see that different shades of gray are produced by varying the patterns of black and white dots. There are no gray dots at all.
1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 15
Dither, Halftone, Grayscale
original dither halftone
1st semester, 2011 16 Digital Images - ENGG1015 - Dr. E. Lam
RGB Color Space
The RGB model describes the formation of color by linearly mixing different portion of “Red”, “Blue” and “Green” light.
Color is represented by a triplet {r,g,b}, which indicates the weighting coefficients
We often normalize the coefficients to be between 0 and 1 (inclusive), or integers between 0 and 255 (8-bit).
1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 17
More Color Models Both RGB and CMY(K) model specify linear
combinations of the “primaries” But they have little resemblance to how human
beings reason about colors E.g. How do you get the RGB values of the
pale orange color on the right?
1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 18
[R G B] = [204 131 42]
[R G B] = [? ? ?]
[248 215 152]
9/30/11
4
HS(B/V), HSL, HSI Color Model The family of HSx models describe colors
similar to how human perceives colors • Also similar to how painters create colors
HSB: Hue Saturation Brightness HSV: Hue Saturation Value HSL: Hue Saturation Lightness HSI: Hue Saturation Intensity Similar, but often comes with confusing (or
even contradicting) definitions
1st semester, 2011 19 Digital Images - ENGG1015 - Dr. E. Lam
Cylindrical-‐Coordination Hue:
• The dominant color • The angle away
from red Saturation
• The amount away from the center
• How “full” the color is
Lightness/Brightness/Value • The amount of
white/black added
1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 20
Luminance-‐chrominance Another common alternative: a luminance-
chrominance representation One value for “luminance” (Y): the
“brightness”, or achromatic image • Y = 0.2126 R + 0.7152 G + 0.0722 B
Need two more numbers for the “chrominance” • YUV and YCbCr
Why? TV broadcast and digital picture compression…
1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 21
More Image Representations? Raster image (bitmap image) - Raster
graphics uses pixel values to describe an image. The file size is independent of the image complexity. For higher resolution, the file size increases dramatically
Vector graphics (draw graphics) - An alternate approach is to use only instructions for drawing lines, circles, ellipses, curves, and other shapes.
1st semester, 2011 22 Digital Images - ENGG1015 - Dr. E. Lam
Vector Graphics Vector-based images are composed of
key points and paths which define shapes, and coloring instructions, such as line and fill colors.
Example:
1st semester, 2011 23 Digital Images - ENGG1015 - Dr. E. Lam
Vector Graphics Advantages Vector graphics can be scaled up and down
easily and quickly while retaining the quality of the picture. Raster images scale poorly and display poorly at resolutions other than that for which the image was originally created.
Vector graphics require less bandwidth and can be accessed and viewed faster than raster graphics.
Vector graphics can be edited and manipulated far easier than raster images.
1st semester, 2011 24 Digital Images - ENGG1015 - Dr. E. Lam
9/30/11
5
(Partial) Summary Many decisions to make
No universally “best” options • Depends on the physical system, e.g. monitor vs
printer • Depends on the requirement, e.g. color vs
grayscale
Intersects with other fields • e.g. Psychology (visual science)
How about our video chats problem?
1st semester, 2011 25 Digital Images - ENGG1015 - Dr. E. Lam 1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 26
Image Processing
Used in digital camera, TV, cell phones…
Used in all kinds of photo editing SW • e.g. Photoshop, GIMP…
1st semester, 2011 27 Digital Images - ENGG1015 - Dr. E. Lam
Image Processing -‐ Examples
Original Grayscale
Blur Edge Detection
1st semester, 2011 28 Digital Images - ENGG1015 - Dr. E. Lam
RGB to Grayscale Conversion Each pixel of a grayscale image has only one
intensity value, V
High V: white, Low V: black
Easiest conversion:
Produce better result if you weight G and R more than B • Human eyes are more sensitive to green and red €
V =R +G + B
3
1st semester, 2011 29 Digital Images - ENGG1015 - Dr. E. Lam
Basic Filtering: Windowing Filters are building blocks of image processing
systems One of the most basic filtering method is by
windowing
€
y[r,c] =1h[i, j]
i, j∑
h[i, j]x[r + i,c + j]j=−1
1
∑i=−1
1
∑
r
c 1st semester, 2011 30 Digital Images - ENGG1015 - Dr. E. Lam
9/30/11
6
Windowing in Action
12 8 27 26 54 48 14 9 16 8 29 9 3 11 10 15 50 60 8 12 34 2 29 52
17 2 44 35 56 72 22 39 43 34 63 77
1 2 1 2 4 2 1 2 1
9 16 8 11 10 15 12 34 2
1 × + 2 × + 1 × 2 × + 4 × + 2 × 1 × + 2 × + 1 ×
+ + = 14
14 19
16 8 29 10 15 50 34 2 29
19
X H Y
34
34 8 29 9 15 50 60 2 29 52
16
1st semester, 2011 31 Digital Images - ENGG1015 - Dr. E. Lam
Gaussian Blur A simple but effective way to blur a picture Each pixel is replaced with a weighted sum of the
values of its surrounding pixels The weighting factors have a Gaussian distribution,
thereby the name Intuitively: each pixel is mixed to certain extent with
its neighbors
1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 32
2 4 5 4 2
4 9 12 9 4
5 12 15 12 5
4 9 12 9 4
2 4 5 4 2
“Generalizing” Windowing So far we are only doing (weighted) average of pixel
values within a window • A linear technique • How about a nonlinear technique, e.g. taking median?
Actually that’s very useful • Which is faster, mean or median?
People often “flip” the filters by 180 degrees:
• We “cheated”, because our filters were symmetric
Why? Because it links us to a signal processing technique called convolution • Extensive body of knowledge, allowing us to know and
compare the effects of these filters for different weights
1st semester, 2011 33 Digital Images - ENGG1015 - Dr. E. Lam
€
y[r,c] =1h[i, j]
i, j∑
h[i, j]x[r − i,c − j]j=−1
1
∑i=−1
1
∑
Edge Detection Useful in understanding an image
• For robot, face recognition, medical imaging etc
In a smooth contour, the pixel values usually do not change rapidly
However, the pixel exhibit sudden jump in values near an edge • E.g. jump from 1 to 130
Sobel edge detection is one of the simplest algorithms that makes use of this observation to find edges • Compares values of the neighbors of pixel
1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 34
Sobel Tilter
A Sobel filter combines the results of the two weight matrices
Each filter kernel estimates gradient in x and y direction from the input image.
Magnitude of the resulting pixel in matrix D is:
1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 35
+1 0 -1
+2 0 -2
+1 0 -1
+1 +2 +1
0 0 0
-1 -2 -1
€
D[r,c] = Dx2[r,c]+ Dy
2[r,c]
From convolving with Hx From convolving with Hy
Hx Hy
Sobel Filter Example – x dir
3 3 3 39 39 39 39 3 3 3 40 40 40 40 3 3 3 41 41 41 41 3 3 3 42 42 42 42 3 3 3 41 41 41 41 3 3 3 40 40 40 40 3 3 3 39 39 39 39
-1 0 +1 -2 0 +2 -1 0 +1
3 3 3 3 3 3 3 3 3
-1 × + 0 × + 1 × -2 × + 0 × + 2 × -1 × + 0 × + 1 ×
+ + = 0
0 152
3 3 40 3 3 41 3 3 42
152
S Gx D
152
152 3 40 40 3 41 41 3 42 42
flipped
0
0 40 40 40 41 41 41 42 42 42
1st semester, 2011 36 Digital Images - ENGG1015 - Dr. E. Lam
9/30/11
7
Sobel Filter Example – x dir
Result in D shows a clear line at the edge Note that Gx is a flipped version of Hx
Some more normalization has to be done in actual implementation
3 3 3 39 39 39 39 3 3 3 40 40 40 40 3 3 3 41 41 41 41 3 3 3 42 42 42 42 3 3 3 41 41 41 41 3 3 3 40 40 40 40 3 3 3 39 39 39 39
-1 0 +1 -2 0 +2 -1 0 +1
0 0 152 152 0 0 0
0 0 152 152 0 0 0
0 0 152 152 0 0 0
0 0 152 152 0 0 0
0 0 152 152 0 0 0
0 0 152 152 0 0 0
0 0 152 152 0 0 0
S Gx D flipped
1st semester, 2011 37 Digital Images - ENGG1015 - Dr. E. Lam
Image Processing Summary Image processing is the task of manipulating
the image by mathematical means to achieve high level requirements
Common operations: filtering Many other operations: E.g. Image forensic, Lithography, medical
imaging, automatic image diagnosis, robot control, etc…
What’s the “(computational) cost” of various image processing algorithms?
What sort of image processing operations do we need in our video chats?
1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 38
1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 39
Digital Cameras Resolution measured in pixels H x V
Image sensing: charge coupled device (CCD) or complementary metal-oxide semiconductor (CMOS)
Megapixels is used to denote the total max pixels in the image • E.g. 5 Megapixel - in the 2520 by 1890 and higher pixel
range. Photo quality 11 x 14 prints from this class of camera.
Comparing film cameras to digital cameras is difficult since resolution is measured differently
1st semester, 2011 40 Digital Images - ENGG1015 - Dr. E. Lam
Taking Pictures 1. Image focused by lens 2. Image captured on CCD 3. CCD generates analog
representation of image 4. Analog signal converts to
digital 5. Digital signal processing
(DSP) adjust quality, etc Step 5
Step 4
Step 3
Step 1
Step 2
1st semester, 2011 41 Digital Images - ENGG1015 - Dr. E. Lam
Marketing Caveats Q: For digital cameras, higher
“megapixel” value always produce better photos?
A: Not really. If you will only look at the photos on websites, or will only print them on 3R papers, you don’t need all the pixels from a 10M pixels camera.
1st semester, 2011 42 Digital Images - ENGG1015 - Dr. E. Lam
9/30/11
8
Area You Ready?
1st semester, 2011 43 Digital Images - ENGG1015 - Dr. E. Lam
Flat Panel TVs and Monitors Pictures displayed as matrix of pixels on screen Two major technologies for generating picture
• Plasma • Liquid Crystal Display (LCD)
Plasma • Neon-Xenon gas trapped between two glasses • When electrically charged, each pixel display red,
blue or green color. LCD
• Liquid crystal between glasses pass/block light depending on electrical signal
• Pass corresponding backlight
1st semester, 2011 44 Digital Images - ENGG1015 - Dr. E. Lam
LED TVs? Misleading term
Proper name: LED-backlight LCD TVs
Use the same LCD display technology as all other “LCD displays”.
Most other standard “LCD displays” use cold cathode fluorescent light (CCFL) for backlight
1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 45
3 Characteristic Dimensions Panel Size • The physical dimension of the panel • A 42” panel has a diagonal measurement of
42” Display Resolution • The number of picture-elements (pixels) along
each X-Y direction Dot Pitch • The distance between two pixel of the screen
Panel Size = Display Resolution * Dot Pitch
1st semester, 2011 46 Digital Images - ENGG1015 - Dr. E. Lam
Standard Display Resolutions
1st semester, 2011 47 Digital Images - ENGG1015 - Dr. E. Lam
Marketing Caveats Q: For flat panel TVs, a bigger screen
always produce better display than a smaller screen?
A: Not really. It depends on the distance you will be watching the TV and the TV source signal.
1st semester, 2011 48 Digital Images - ENGG1015 - Dr. E. Lam
9/30/11
9
More Pixel = Good? Human eye can identify 120 pixels per degree
of visual arc • i.e. if 2 dots are closer than 1/120 degree, then our
eyes cannot tell the difference At a distance of 2m (normal distance to a TV)
our eyes cannot differentiate 2 dots 0.4mm apart.
Closer to TV => easier to differentiate pixels Far away => cannot tell the difference
screen
Minimum: 2 arc minute 1st semester, 2011 49 Digital Images - ENGG1015 - Dr. E. Lam
Image courtesy of www.carltonbale.com 1st semester, 2011 50 Digital Images - ENGG1015 - Dr. E. Lam
Source: http://www.diamond-vision.com/quad_dot_pattern.asp
True LED displays Each pixel is a
LED
Used mostly in outdoor, large- scale displays
1st semester, 2011 51 Digital Images - ENGG1015 - Dr. E. Lam
Dallas Cowboys Stadium Sideline Display 48.64m x 21.76m Pixel Pitch: 20mm Displays World’s Largest High-Definition Video Display
Hong Kong Shatin Racecourse 70.4m x 8m World’s Longest TV screen
1st semester, 2011 52 Digital Images - ENGG1015 - Dr. E. Lam
In Conclusion… Digital signal processing is a very broad
field within EEE The processing of digital image is a
good example of high-level applications that run on digital signal processing systems.
To display and process digital images correctly, you need the right combination of image representation, hardware, and processing power.
1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 53
Homework 1 Homework 1 is out • Due 14 Oct, 2011 • 5pm • Turn in physical copy of your answer • Homework boxes near Room 712, Chow
Yei Ching Building
Individual homework Good way to study for final Zero tolerance on plagiarism
1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 54
Recommended