By: Navid Einackchi

By: Navid EinackchiBy: Navid Einackchi

Isfahan University of Isfahan University of TechnologyTechnology

Electrical and Computer Department

Spring 2007Spring 2007

Image alignment and stitching using Object Recognition MethodsImage alignment and stitching using Object Recognition Methods

Supervisors: Dr Rasoul Amirfattahi

Dr javad Askari

Advisor: Dr M. Saraee

Master Thesis of Computer Engineering- Artificial Intelligence and Robotic

به نام خدا

By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007

Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods

Presentation ProcessPresentation Process

• IntroductionIntroduction– Definition– Aligning concepts– Stitching Concepts

• Object Recognition ProblemObject Recognition Problem– Salient Feature– What is Descriptor– Matching

• Proposed Method for AligningProposed Method for Aligning– Using Harris Detector– Using SIFT Detector– Stitching

• Concluding and SuggestionsConcluding and Suggestions

Some slides are fro

m other

Some slides are fro

m other

sources

• IntroductionIntroduction– Definition– Aligning Concepts– Stitching Concepts

Image Alignment and StitchingImage Alignment and Stitching

• DefinitionDefinition • ApplicationsApplications

– Mosaic Image

– Panorama

– Virtual Environment

– Robotic

تصوير حاصل

• DefinitionDefinition • ApplicationsApplications

– Mosaic Image

– Panorama

– Robotic

• DefinitionDefinition• ApplicationsApplications

– Mosaic Image

– Panorama

– Robotic

Aligning ConceptsAligning Concepts

• Image TransformationsImage Transformations– Applicable Transformations

• Find transformationsFind transformations– How many parameters?

– Calculating Parameters

Image TransformationsImage Transformations

• TransitionTransition– Number of Parameters: 2

– Number of Points: 1

• EuclideanEuclidean– Number of Parameters: 3

cossin

sincosR

• SimilaritySimilarity– Number of Parameters: 4

cos.sin.

sin.cos.

• AffineAffine– Number of Parameters: 6– Number of Points: 3

• PerspectivePerspective– Number of Parameters: 8– Number of Points: 4

Image Transformation ComputationImage Transformation Computation

• DirectDirect– Mapping one image into another using different parameters– Define an Error Function based on pixel intensity difference

• Using Corresponding pointsUsing Corresponding points– Obtaining Corresponding points– Computing Transformation parameters using corresponding points

Direct MethodDirect Method

Error Function

Using Corresponding pointsUsing Corresponding points

• How many points?How many points?

• How to select?How to select?– Using human

– Automatically

•IntroductionIntroduction–Definition

–Aligning Concepts–Stitching Concepts

•Object Recognition ProblemObject Recognition Problem–Salient Feature

–What is Descriptor–Matching

•Proposed Method for AligningProposed Method for Aligning–Using Harris Detector

–Using SIFT Detector–Stitching

•Concluding and SuggestionsConcluding and Suggestions

StitchingStitching

• StitchingStitching• ChoicesChoices

– Final Plane

– Pixel weighting

Object Recognition MethodsObject Recognition Methods

• Local Feature DetectionLocal Feature Detection– Edge– Corner– Hole

• Interest Regions MatchingInterest Regions Matching– Appearance Matching– Geometric Matching

• Finding Object Relations in ImageFinding Object Relations in Image

Salient FeaturesSalient Features

• Features of Feature!Features of Feature!– Invariants against transformations– Ability to explain the image– Ability to being Matched (Repeatability)

• Selected FeaturesSelected Features– Corners– Holes

Harris DetectorHarris Detector

PlainNo changes in every direction

EdgeHigh changes in Edge

Direction

CornerHigh changes in every

direction

Harris DetectorHarris Detector(Continued)(Continued)

( , ) x x y

x y x y y

I I IM w x y

Slow change

direction

Fast change Direction

(max)-1/2

(min)-1/2

“Corner” 1, 2 both bigboth big

Edge1 >> 2

Edge2 >> 1

•Harris measurementHarris measurement

2det traceR M k M

•Harris measurement responseHarris measurement response

“Corner”

“Edge”

“Plain”

R < 0|R| small

Local Maxim

um bigger than Threshold

Local Maxim

um bigger than Threshold

• FeaturesFeatures– Invariant against rotation

– Invariant against intensity

Threshold

•ExampleExample

• Scale ProblemScale Problem

Scale InvariantScale Invariant

Finding Appropriate Window Size

Scale SelectionScale Selection

• Using Differential of Gaussian FiltersUsing Differential of Gaussian Filters– Laplacian of Gaussian (LoG)– Difference of Gaussian (DoG)

- =Window thst m

aximize fil

Window thst m

aximize fil

Automatic Scale SelectionAutomatic Scale Selection

SIFT Detector SIFT Detector

• Bulb DetectorBulb Detector• Invariant against scaleInvariant against scale• Using DoGUsing DoG

– Bulb detection

– Scale detection

مقیاس

SIFT DetectorSIFT Detector

• Detecting Bulb and Scale SimultaneouslyDetecting Bulb and Scale Simultaneously– Bulb position = maximum in plain

– Scale = maximum in third dimension

Bulb position

Scale position

DescriptorDescriptor

• Vector which describes region of interest pointVector which describes region of interest point– Pixels– Histogram– Differential

• Feature of DescriptorFeature of Descriptor– Invariant against image distortion (view, angle, intensity)– Able to express similarities and differences

SIFT DescriptorSIFT Descriptor

• Based on value and direction of gradientsBased on value and direction of gradients• 128 dimensional Vector128 dimensional Vector• ConstructionConstruction

1. Calculation of value and direction of gradient of each pixel

2. Making direction histogram of gradients

3. Rotating interest region based on dominant direction

4. Dividing the region into 16 region

5. Constructing direction histogram for each region

Construction ExampleConstruction Example

DescriptorDescriptor FeaturesFeatures

• Invariant againstInvariant against– Rotation– Position– Affine changes in intensity (I Ia + b)

MatchingMatching

• Appearance MatchingAppearance Matching– Comparing distance of each two descriptors– Euclidean Distance– K nearest neighbor

• Geometrical MatchingGeometrical Matching– Based on geometrical relations between points– Estimation of image transformation– Rejecting false corresponding points

Example of Geometrical MatchingExample of Geometrical Matching

Automatically image alignmentAutomatically image alignment

• Direct MethodDirect Method

• Based on interest pointsBased on interest points

CompareCompare

Interest pointsInterest points Direct MethodDirect Method

LowLow (Order of number of interest (Order of number of interest

points)points)

HighHigh((order of number of pixelsorder of number of pixels))

Time and Memory Time and Memory orderorder

HighHighLowLow

((Based on pixel intensityBased on pixel intensity))Resisting against Resisting against intensity changesintensity changes

YesYes NoNoAbility to recognize Ability to recognize

corresponding corresponding imageimage

Brown and Lowe AlgorithmBrown and Lowe Algorithm

O(nO(n44 m m44))

Similarity Transform rather than perspectiveSimilarity Transform rather than perspective

• WhyWhy– Detector are unable to handle perspective – Small portion of each images is overlapped– Increasing the tolerance of true corresponding

• So We use Similarity TransformationSo We use Similarity Transformation

First Proposed AlgorithmFirst Proposed Algorithm

1.1. Detecting Interest points using Harris detectorDetecting Interest points using Harris detector

2.2. Defining descriptors for each pointDefining descriptors for each point

3.3. Finding corresponding points using appearanceFinding corresponding points using appearance• Comparing interest points with each other

• K Nearest Neighbor

4.4. Finding relation between 2 imagesFinding relation between 2 images• Using Similarity

• Based on voting

5.5. Rejecting wrong corresponding points based on geometrical relationsRejecting wrong corresponding points based on geometrical relations

First Proposed AlgorithmFirst Proposed Algorithm

1.1. Detecting Interest points using Harris detectorDetecting Interest points using Harris detector

2.2. Defining descriptors for each pointDefining descriptors for each point

3.3. Finding corresponding points using appearanceFinding corresponding points using appearance• Comparing interest points with each other

• K Nearest Neighbor

4.4. Finding relation between 2 imagesFinding relation between 2 images• Using Similarity

• Based on voting

5.5. Rejecting wrong corresponding points based on geometrical relationsRejecting wrong corresponding points based on geometrical relations

Voting MechanismVoting Mechanism

0cossin

0sincos

similarityT

Voting MechanismVoting Mechanism

O(nO(n22 m m22)) از مرتبهاز مرتبه

Applied Rotation

Estimated Rotation

Applied Scale

Estimated Scale

EvaluationEvaluation

ConcludingConcluding

• Very good for rotationVery good for rotation• Good for little scalingGood for little scaling• Weak for big scalingWeak for big scaling

– Due to Harris detector weakness

•Proposed Method for AligningProposed Method for Aligning–Using Harris Detector–Using SIFT Detector

–Stitching

ImprovementImprovement

• Using less time order algorithmUsing less time order algorithm

• Using Scale Invariant Feature TransformUsing Scale Invariant Feature Transform

Second proposed AlgorithmSecond proposed Algorithm

1.1. Detecting Interest pointsDetecting Interest points• SIFT Detector

2.2. Finding Corresponding pointsFinding Corresponding points• Based on appearance• Using k-d tree• Distance ratio threshold

3.3. Finding relation between imagesFinding relation between images• Voting on step 2 based on Appearance corresponding

4.4. Rejecting wrong corresponding pointsRejecting wrong corresponding points• Based on geometric relations between points

Distance Ration ThresholdDistance Ration Threshold

• Find 1NN and its distance = D1Find 1NN and its distance = D1• Find 2NN and its distance = D2Find 2NN and its distance = D2• Find ratio of these distances (D1/D2 = ratio)Find ratio of these distances (D1/D2 = ratio)

– If Lower than threshold -> reject

– Else accept

Distance Ration ThresholdDistance Ration ThresholdMy ImprovementMy Improvement

• Find 1NN and its distance = D1Find 1NN and its distance = D1• Multiply D1 by ratio = rangeMultiply D1 by ratio = range• Look for range in k-d treeLook for range in k-d tree

Results of threshold on ratioResults of threshold on ratio

• Threshold Ratio = 0.5Threshold Ratio = 0.5

Results of threshold on ratioResults of threshold on ratio

• Threshold Ratio = 0.7Threshold Ratio = 0.7

Second proposed AlgorithmSecond proposed Algorithm

1.1. Detecting Interest pointsDetecting Interest points• SIFT Detector

2.2. Finding Corresponding pointsFinding Corresponding points• Based on appearance• Using k-d tree• Distance ratio threshold

3.3. Finding relation between imagesFinding relation between images• Voting on step 2 based on Appearance corresponding

4.4. Rejecting wrong corresponding pointsRejecting wrong corresponding points• Based on geometric relations between points

• Each interest point hasEach interest point has– Dominant Direction

– Scale

• Use this information for votingUse this information for voting

Estimating Geometrical Estimating Geometrical RelationRelation

Estimation HistogramEstimation Histogram

Using threshold methodUsing 3NN method

Direction

O(nm)O(nm)

O(n)O(n)

Maximum Maximum

Percent of True Corresponding

True Corresponding Points

Potential Corresponding points

Method

487% 19 39 Distance Ratio t=0.5

0.06% 49 710 Distance Ratio t=1

0.07% 52 710*2→710 2 Nearest Neighbor

Comparing Knn and Ratio Threshold settingComparing Knn and Ratio Threshold settingBased on Number of true corresponding pointsBased on Number of true corresponding points

Percent of True Corresponding

True Corresponding Points

Potential Corresponding points

Method

0.06% 49 710 Distance Ratio t=1

Ratio Difference 2nd Maximum 1st Maximum Method

8.57 26.5 3.5 30 Distance Ratio t=0.5

7.75 27 4 31 Distance Ratio t=0.6

6.85 30.75 5.25 36 Distance Ratio t=0.7

1.95 46 48 94 2 Nearest Neighbor

Comparing Knn and Ratio Threshold settingComparing Knn and Ratio Threshold setting(In Tolerating False Corresponding)(In Tolerating False Corresponding)

Ratio Difference 2nd Maximum 1st Maximum Method

8.57 26.5 3.5 30 Distance Ratio t=0.5

7.75 27 4 31 Distance Ratio t=0.6

6.85 30.75 5.25 36 Distance Ratio t=0.7

Applied Rotation

Estimated Rotation

Applied Scale

Estimated Scale

EvaluationEvaluation

StitchingStitching

• Pixel Weights based on their distancePixel Weights based on their distance

StitchingStitching

Other examplesOther examples

• Adobe Photoshop could not stitch these imagesAdobe Photoshop could not stitch these images

Other ExamplesOther Examples

ConcludingConcluding

• IdeasIdeas– Using Harris and SIFT as a detector– Using Similarity transform as a general image transform– Using voting based on direction and scale of each interest points– Introducing Evaluation Method

• ResultsResults– Harris is more reliable but SIFT is much better in presence of scale transform– Similarity can handle prospective transform for human eyes– Voting can obtain similarity transform with much lower time order– Threshold ratio has a better behavior than Knn

ThanksThanksQuestions?Questions?

By: Navid Einackchi

Documents

Karaoke Navid

Presentazione standard di PowerPoint di fotografia/Corso di fotografia 2016... · Photo: Navid Baraty . Street Photography La difficoltà di definire la street photography è proprio

Program a de Navid ad para el ciudadan o curioso. · danza sfolclórica sdeIndonesia,coro devoces ... Infantil.Suiza. CinetecaSalaAzcona 16.30 –Concierto: EvaNovotná.República

PROGRAMA 23-11-15 - Tenerife navid… · comparr con los que más queremos estos días de Navidad. Es la primera vez que me dirijo a ustedes como Alcalde para transmirles mis mejores

folleto navidad susaeta 2019 - Susaeta Ediciones 2019/NAVID… · Mi árbol de Navidad Referencia 3057 +4 Ref.: 3308-999 ISBN 978-84-677-4959-5 9 788467 749595 Cartulina plastiﬁ

DAPEM by MAPIekladata.com/gYn3rPNaC2Rur-7I3Ew5CuPeGdU/jeu-peluches-blog.pdf · dapem by mapi dapem by mapi. dapem by mapi dapem by mapi. dapem by mapi dapem by mapi

Inhaltliche Aufbereitung: Brigitte Schwarzlmüller Quiz für Einsteiger Quiz für Profis Buchinfo von Navid Kermani Ayda, Bär und Hase Beenden

VARIACIÓN MORFOMÉTRICA Y GENÉTICA DE Diaphorina citri ... · A mi mejor amiga Navid Pacheco¸ porque aunque estemos lejos, nuestro corazones están unidos. A los que ya no están,

Präsentiert - Zorro Film GmbH€¦ · Besetzung Rafik Karim Saleh Jamal Navid Akhavan Olga Julie Engelbrecht Stab Regie Ali Samadi Ahadi Produktion brave new work Produzenten Mohammad

NAVID AD! LA MAGI A DE LA. 22 de diciembre Sorteo más importante de España Día de la Lotería El primer premio son 3 millones de euros por tira: este es

stedelijk museum amsterdam jaarverslag 2010 · 2019-10-21 · William Leavitt Navid Nuur On Kawara Roman Ondák Willem de Rooij Diana Thater Mario Garcia Torres Lawrence Weiner. 8

REUNIO A LOS JEFES QUE ERAN LOS QUE HEREDARIAN EL REINO, CRECIÓ NAVID, SE HIZO UN HOMBRE FUERTE Y SANO, UN GUERRERO. SE ARREPINTIO

Marija Navid as Bub Anja

€¦ · navid ahmed dwec-141248-06 sonia ajmee dwec-141248-05 pia saha dwec-141248-07 md. imrul hassan dwec-141248-13 abid hassan dwec-141248-03 touhqur rahman dwec-141248-04 rasel

ONCUF žs NAVID DORAS FEL PARA cdpqwvq3o4t2n6.cloudfront.net/.../Resumen-ganadores.pdfONCUF žs NAVID DORAS FEL PARA c RSO EÑAS 2r POSTAL FECAP dc CON DORAS . Title: 9 x 13 cm. (4)

Feliz Navid ad - infonortedigital.com

Navid Kermani Große Liebe...reits in der Raucherecke verbrachte, dachte er nicht im Traum oder genau gesagt ausschließlich unter der Bett - decke daran, sie jemals zu küssen, sie

GUMS-صفحه اصلي دانشگاه علوم پزشکی گیلان list...Navid.Rezvany@gmail.com Assistant Professor Tel: 33368673 Email:Alireza soudmand@yahoo.com Dr, Navid Rezvani

2020.10.01 – 02 | LITEXPO · spalio 2 d. # hr transformation 4.0 – naujos hr kartos beieŠkant! | navid nazemian, vodafone global hr head, executive coach # kaip hr gali prisidĖti

Joyas Literarias Juveniles - 090 - Cuento de Navid