Design of image

denoising circuit

成功大學 資訊工程系

陳 培 殷

Outline

Embedded System

What CS Students Can Do in Embedded

System Design

Example -Image Denoising

PCB

PC: a general-purpose computing system

PC

Pentium

PCBEmbedded System: a special-purpose computing system

Most embedded systems are designed for

1. special purpose (customizing and non-programmable)

2. real time applications 3. stable applications

4. automatic applications

Embedded System (1/2)

uPUART

MPEG

ROM

RAM

Embedded System (2/2)

Traditional embedded systems use low-level processors only.

ARM PCI

MPEGUSBFLASH

ROM

RAM

DSPAMBA

Advanced embedded systems

Multi-core

Application Everywhere!

Applications

Information Appliances (IA):

1. Smart phone, VOIP

2. Digital TV, set-top box

3. PlayStation

4. PDA, mp3 player

5. Camera, DV

6. Air-conditioner, microwave oven, refrigerator,

vacuum cleaner, sensor network

7. Motorcycles

8. Car (abs, engine firing, air bag) >100 processors

9. … Ubiquitous computing (many computers for everyone)

Requirements

1. Friendly user interface

2. Short time-to-market

3. Real-time (Speed)

4. Cost

5. Power consummation/dissipation

(cooling strategy and battery life)

Synthesis

Applications

傳輸距離： 100 m功率消耗： 2 W功能：影像傳輸、 語音傳輸

螢　幕： 176 x 220 pixel　　　　 65535 色 1.8 吋 TFT

其他

傳輸距離： 100 m功率消耗： 2 W功能：影像傳輸、 語音傳輸

螢　幕： 176 x 220 pixel　　　　 65535 色 1.8 吋 TFT

其他

Specifications

Hardware:CPU,

RAM, I/O…

Hardware:CPU,

RAM, I/O…

Software:C,C++

Software:C,C++

System Design

always @(posedge clk) begin if (sel1) begin out=in1; else out=in2; endend

always @(posedge clk) begin if (sel1) begin out=in1; else out=in2; endend

Component Design

Layout

Placement & RoutingFabrication

Marketing

System Development Flow

Testing

What CS students can do in

Embedded System Design?

My personal Views: 1. Hardware Design - Chip

2. Hardware Design - System

3. Related Tools Design

4. Firmware Design

5. Software Design

6. ESL Design

Design Flow

Specification

System Architecture

Hardware Design Software Design

System Integration

System Verification/Testing

Hardware/software partition is

very difficult !!! (cost time)

IC Industry in Taiwan

晶粒測試及切割

晶圓切割

設計

導線架

測試 封裝 製造 光罩

晶圓

邏輯設計 封 裝

化學品

成品測試光罩設計

長晶

ARM PCI

MPEGUSBFLASH

ROM

RAM

DSPAMBA

Hardware Design -- Chip (1/3)

ASIC

The basic design flow for digital cell-based ASIC

Describe circuits with hardware description language

(HDL 硬體描述語言 ) VHDL and Verilog

Synthesis ( 合成 ) the circuits ….

application specific integrated circuit (ASIC 晶片 ) such as USB, MPEG, ….

Hardware Design -- Chip (2/3)

always @(IN)begin OUT = (IN[0] | IN[1]) &

(IN[2] | IN[3]);end

OUT

IN[0]IN[1]

IN[2]IN[3]

Example:

….

….

Hardware Design -- Chip (3/3)

always @(…) if (a==b) if (c==1) d=f; else d=1; else d=0; a

b

c

f

d

fca

b

d

Translate into Boolean Representation

Optimize + MapHDL Source

Target Technology

Synthesis = Translation+Optimization+Mapping

Process of logic synthesis

Hardware Design -- System

ARM PCI

MPEGUSBFLASH

ROM

RAM

DSPAMBA

ASIC

Input devices: keyboard, touch screen, switch, button, ..

Output devices: monitor, LCD, LED, …

Extended devices: compact flash card (CF), PCMCIA, SD

(for storage, wireless communication, I/O)

Power system:

Transmission Interface: PCI, USB, IEEE 1394, UART, bluetooth…

Bus: AMBA (Advanced Microcontroller Bus Architecture)

Input devices

Output devices

Related Tools Design

晶粒測試及切割

晶圓切割

設計

導線架

測試 封裝 製造 光罩

晶圓

邏輯設計 封 裝

化學品

成品測試光罩設計

長晶

Firmware Design

ARM PCI

MPEGUSBFLASH

ROM

RAM

DSPAMBA

ASIC

Devices drivers for I/O devices, extended devices, transmission interface

Assembly codes and C codes for some dedicated CPUs (ARM, 8086,..)

Architectures and instruction sets of different CPUs, DMA,…

Input devices

Output devices

Software Design

ARM PCI

MPEGUSBFLASH

ROM

RAM

DSPAMBA

ASIC

Input devices

Output devices

Embedded OS: WinCE, Palm OS, uC/OS, Linux, JAVA

Real time OS (time) as small as possible (memory)

Distributed embedded system (+ fault tolerance)

Application Software:

wireless communication, network, multimedia,

health, convenience, Web, ….

Porting a customized embedded system to different

machines is very difficult (need large modification)

FutureChip: tens of millions of transistors or more (.35, .25, .18, .09)

Design shifts from ASIC/board to system

System on a Board(printed circuit board)

System on a chip

uP FPGA

MPEGASIC

ATMROM

ROM

SW SW

SWSWPCB

uP Core SRAM

ROM

ATMMPEG

ROM

FPGA

Glue Logic

A/D Block

PCB

SOCSystem-on-a chip is possible

(the whole system is

built in a single chip)

SOC is industry trend

Example: Mobile Phone

Voice only; 2 processors 4 year product life cycle Short talk time

Yesterday

Voice, data, video, SMS <12 month product life cycle Lower power; longer talk time

Today

• 5~8 Processors

• Memory• Graphics• Bluetooth• GPS• Radio• WLAN

Single Chip

DSP

Radio

FlashMemory

Processor

Source: EI-SONICS

What’s ESL tools ?

Describe a SOC design in a more abstract way.

Provide virtual prototypes for HW and SW implementation.

Help system integration in early design phase.

Help to verify the result after job partition.

Claim that design and verification can be done in minutes.

Electronic System Level (ESL) Tools

Suitable for students in CS

Hardware Software Co-Simulation

SoC Designer

Example: Image Denoising

Noise arises During image acquisition

CCD camera, light levels, sensor temperature During image transmission

Interference in the transmission channel

NoiseDetection

NoiseRemoval

Noise Suppression Process

1,1ˆ

jif jif ,1ˆ

1,1ˆ

jif

1, jif jif , 1, jif

1,1 jif jif ,1 1,1 jif

Hardware Design – Chip !

Drawbacks of conventional algorithms Heavy computation loading Iterations

Large computation loading Additional memory space

Our goals: Cost-effective Good performance Suitable for VLSI implementation Real-time requirement

Motivation

Low-Complexity Algorithms

Embedded System

1. Power consumption

2. Computing time (real time vs. high quality)

VLSI Implementation

1. Circuit cost

2. Circuit speed

3. Power consumption

Low-Complexity algorithms are necessary for

embedded systems and VLSI implementations.

Issues for VLSI Implementation

Circuit Cost fixed point vs. floating point

less complexity, moderate quality vs. high complexity, high quality

+, - vs. *, / 2*x left shift x 5*x 4*x + x

Functional units sharing

Circuit Speed Pipeline architecture

Delay reduction

Power Consumption Sleep

Shut down partially

Cost-Effective Denoising Method (1/4)

1.Find Min & Max in the mask

3.Check Suspected Noise

4.Choose eligible & representativeedge directions

5.Find the minimum difference

6.Reconstruct the corrupted pixel

2.Utilize Min & Max info.to detect noise

Noise Detection Noise Removal

1,1ˆ

jif jif ,1ˆ

1,1ˆ

jif

1, jif jif , 1, jif

1,1 jif jif ,1 1,1 jif

Check suspected noise

Case 1 Case 2 Case 3 Case 4

1,1ˆ

jif jif ,1ˆ

1,1ˆ

jif

1, jif jif , 1, jif

1,1 jif jif ,1 1,1 jif

1,1ˆ

jif jif ,1ˆ

1,1ˆ

jif

1, jif jif , 1, jif

1,1 jif jif ,1 1,1 jif

1,1ˆ

jif jif ,1ˆ

1,1ˆ

jif

1, jif jif , 1, jif

1,1 jif jif ,1 1,1 jif

1,1ˆ

jif jif ,1ˆ

1,1ˆ

jif

1, jif jif , 1, jif

1,1 jif jif ,1 1,1 jif

Cost-Effective Denoising Method (2/4)

Choose eligible & representative edge directions

Case 1 Case 2 Case 3 Case 4

1,1ˆ

jif jif ,1ˆ

1,1ˆ

jif

1, jif jif , 1, jif

1,1 jif jif ,1 1,1 jif

1,1ˆ

jif jif ,1ˆ

1,1ˆ

jif

1, jif jif , 1, jif

1,1 jif jif ,1 1,1 jif

1,1ˆ

jif jif ,1ˆ

1,1ˆ

jif

1, jif jif , 1, jif

1,1 jif jif ,1 1,1 jif

1,1ˆ

jif jif ,1ˆ

1,1ˆ

jif

1, jif jif , 1, jif

1,1 jif jif ,1 1,1 jif

Cost-Effective Denoising Method (3/4)

Find the minimum difference and reconstruct the noisy pixel

Case 1

1,1ˆ

jif jif ,1ˆ

1,1ˆ

jif

1, jif jif , 1, jif

1,1 jif jif ,1 1,1 jif

Cost-Effective Denoising Method (4/4)

Architecture of the Denoising Chip

Line Buffer 1

Line Buffer 2

ShiftRegister

Edge Detection Unit

Min & Max Computation

Unit

Noise Detection

Unit

Image Out

Image In

Output Selection Unit

Shift Register Bank

R0R1R2

R3R4R5

R6R7R8

Line Buffer

Line Buffer1st Raw2nd Raw

1st Raw 1st Raw2nd Raw

1st Raw

3rd Raw

DataflowMax & Min

Computation Unit

Noise Detection Unit

Edge Detection Unit Output Selection Unit

Stage 1

Stage 2

Stage 3

Stage 4

Pipeline Stages

less flexible, long design cycle, larger-scale production to reduce price

more flexible, shorter design cycle, suitable for smaller production

FPGA or CPLD

Real ASIC chipStandard cellStandard cell

PLDPLD

Fab (TSMC, UMC, ..)

Xilinx, Altera

ASIC vs. FPGA

Circuit Realization

Demo (1/3)

Source ImageSending Image

Denoising Circuit

FPGA Board

Receiving Image

Result Image

Software Program

Hardware PlatformPC

Hardware Software Co-Simulation/Verification

USB

PC

Demo (2/3)

Demo (3/3)