Computer and Robot Vision II

Computer and Robot Vision II

Chapter 17The Consistent Labeling

ProblemPresented by: 傅楸善 & 陳相廷

0935 681 [email protected]指導教授 : 傅楸善博士

mailto:[email protected]

DC & CV Lab.DC & CV Lab.CSIE NTU

17.1 Introduction In this class of problems we are given a set of

objects and a set of possible names or labels for those objects.

We are also given a set of constraints that limits the possible labels for each object.


17.1 Introduction

Unary constraints: Certain features of an object limit the allowable

labels for that object.


17.1 Introduction

Binary constraints: A particular label for one object limits the possible

set of labels for a related object. e.g. N-Queens problem


17.1 Introduction

N-ary constrains: The sets of labels allowed for sets of N mutully co

nstrained objects is limited.


17.1 Introduction N-ary consistent-labeling problem (CLP):

4-tuple ),,,(CLP RTLU

pairs label-unit of set over therelation ary - :relation constraint label-unit :

units of set over therelation ary - :relation constraint-unit :

labels possible ofset :units ofset :,...,1

LUNRR

UNTTL

MMU


17.1 Introduction

. toˆ from ˆ: mapping a is of ,...,,ˆsubset of labeling

.set unit of subsets -size ordered of labeling allowable : ofelement

.,...,, labels ingcorrespond assigned ,...,, units :),(),...,,(),,(

another. oneconstrain mutually ,...,, units :),...,,(

21

N21

212211

2121

LULUfUuuuU

UNR

llluuuRlululu

uuuTuuu

N

NNN

NN


17.1 Introduction

labelings.-consistent all moreor one find to :problem labeling consistent of Goal

.in is ))(,()),...,(,()),(,(then ,in is ),...,,( tuple- theand ˆin are ,...,,whenever

if consistent is units theof ˆsubset a of labelingA

2211

2121

RufuufuufuTuuuNUuuu

Uf

NN

NN


17.2 Examples of Consistent-Labeling Problems

consistent-labeling problems arise in: computer vision artificial intelligence science engineering


17.2.1 The N-Queens Problem N-queens problem:

given chessboard and N queens. queens placed on chessboard:

no queen captures any other queen. no two queens in same row, same column, or same diag

onal of chessboard. N-queens problem:

modeled as consistent-labeling.

NN


17.2.1 The N-Queens Problem unit set

set of rows on chessboard label set

set of columns on chessboard exactly one queen per row:

labeling specifies the column where queen placed unit-constraint relation

set unit-label constraint relation:

:,..,3,2,1 NU

:,..,3,2,1 NL

:T jijiji uuUuuuuT and,|),(

||||,,,,),(|)],(),,[( jijijijijijjii lluuLllllTuululuR


17.2.1 The N-Queens Problem

e.g. pair [(1,1), (2,4)] is in R since two queens do not capture each other. e.g. pair [(1,1), (2,2)] and [(1,1), (3,3)] are not in R

consistent labeling: solve N-queens problem with constraints satisfied

1

2

1 2

34

3 4


17.2.2 The Latin-Square Puzzle The problem is to arrange the objects such that each

row, each column, and each of the two main diagonals of the matrix contains exactly one object of each color and exactly of each shape.


17.2.2 The Latin-Square Puzzle

Latin-square puzzle: n x n matrix, n2 objects arranged on matrix, one

per square e.g. consider 4 x 4 puzzle for ease of

illustration object is one of four colors

C = { red, blue, green, yellow } object has one of four shapes

S = { circle, square, triangle, octagon }


17.2.2 The Latin-Square Puzzle set of units U = { 1, 2,…, 16 }: 16 squares of matrix labels L: objects to be placed on squares, e.g. red

square, blue triangle, … Cartesian product set L = C x S model T as quaternary constraint:

}diagonalor column, row, same in the lie all ,,,|),,,({ 43214321 uuuuuuuuT


17.2.2 The Latin-Square Puzzle The unit-label constraint relation R would then

consist of quadruples of unit-label pairs of the form

. and

, , whereand ,4,...,1for ),( ,in is ) ,,,( where

)]},,(,[)],,(,[)],,(,[)],,(,[{

4321

444333222111

ji

ji

ii

ss

ccjiiLscTuuuu

scuscuscuscu


17.2.3 The Edge-Orientation Problem

If a local edge operator has been applied to an image and has determined, for each pixel, the strength of an edge passing through it in each of eight possible directions.

The output of the edge operator is noisy due to image noise.

Most meaningful edges in the real world are highly continuous with low curvature.



maximum bending angle of any small edge segment: limited to some maximum e.g.

:upward) (direction toiprelationshnortheast in the is ' xx

:upward) (direction toiprelationshnorthwest in the is ' xx

horizon from 90 ,45between :' x

horizon from 135 ,90between : ' x


Return true if l, l’ compatible for x, x’, false otherwise.

U: set of pixels of the image. L: set of possible edge orientations, including special

value none. E(x): set of possible edge orientations of pixel x,

based on the results of the local edge operator. Nbd(x): set of neighboring pixels to pixel x.


:),,,( predicate '' lxlx


When do pairs of neighboring pixels have compatible labels? At least one label is none. Predicate is satisfied.


edge orientation of given pixels: constrained only by neighborhood edge

)',',,( lxlx

)}('|)',{( xNbdxxxT



unit-label constraint relation:

)}',',,()'or (

and ),'(' ),(,)',( | )]','(),,{[(lxlxnonelnonel

xElxElTxxlxlxR


Jooooooooooke!


17.2.4 The Subgraph-Isomorphism Problem



vertices V: set of vertices edges E: nonreflexive, symmetric binary

relation over V graph G: pair (V, E) It is often necessary to determine whether tw

o graphs representing two different entities are identical, except for the labels of the vertices, indicating that the two objects have the same structure.



G = (V, E) is isomorphic to G’=(V’, E’): if there is one-to-one, onto mapping f from V to V’,

satisfying that f: graph isomorphism

')](),([),( EvfufEvu



unit-set U: set of vertices V of G label-set L: set of vertices V’ of G’ unit-constraint relation T: edge set E of G unit-label constraint relation R:

}'' and,')','(,),(|)]',(),',{[(

vuvuEvuEvuvvuuR



Graph-isomorphism problem is a dual consistent-labeling problem f: VV’ and its inverse must be consistent labelin

gs


17.2.5 The Relational-Homomorphism Problem



Subgraph-isomorphism is a special case of relational-homomorphism problem.

Relational-homomorphism is defined on N-ary relations, instead of just binary.

Homomorphism is also a structure-preserving mapping, but not necessarily to be one-to-one.



},...,1,)( with ),...,(|),...,{(:: with ofn compositio

set into set of elements mappingfunction ::set over relation ary :

sets two:,

11 NibafTaaBbbfTfTfT

BABAfANAT

BA

iiNN

N



Relational homomorphism from T to S:

mapping f: AB satisfying Relational homomorphism applied to N-tuple of T:

result is N-tuple of S

SfT

relationary second : NBS N



Finding relational homomorphism: sometimes called relational matching

Relational homomorphism: maps elements of A to B with same relationships

Relational monomorphism: is a relational homomorphism that is one-to-one. A monomorphism indicates a stronger match than

homomorphism.



Relational isomorphism f from N-ary relation T to N-ary relation S: is an one-to-one relational homomorphism from T to S. f-1 is a relational homomorphism from S to T.

Relational isomophism: A, B have same number of elements each primitive in A maps to unique primitive in B each primitive in A mapped to by a primitive of B each tuple in T has corresponding one in S, vice versa A strongest kind of match: symmetric match



Graph isomorphism is a binary-relational isomorphism. Relational-homomorphism fits consistent-labeling model in

much the same way that the graph-isomorphism problem did. A: set of units B: set of labels Unit-constraint relation: relation T of the relational homomorphism

problem. Unit-label constraint:

consistent labeling solution: relational homomorphism from A to B

}),...,( and ),...,(|)],(),...,,{[( 1111 SllTuululuR NNNN


Joooooooke!


17.3 Search Procedures for Consistent Labeling

Given a consistent-labeling problem CLP = (U,L,T,R), then find the set of all consistent labelings.

If there is no consistent labeling, then returns the empty set.

Method: Backtracking tree search Backtracking tree search with speedup:

Forward checking Discrete relaxation


17.3.1 The Backtracking Tree Search

backtracking tree search: begin with first unit of U select second unit of U, begins to construct

children of first node process continues to level |U| of the tree

path from root to any successful nodes at level |U|: consistent labeling


Simple digraph-matching problem][outdegree)]([outdegree],[indegree)]([indegree uufuuf

in out

A 3 2

B 0 2

C 2 2

D 2 2

E 2 3

F 3 1

in out

1 0 2

2 2 1

3 2 1

4 2 2


portion of the tree search for solving the graph-matching problem


17.3.2 Backtracking with Forward Checking

backtracking tree search: has exponential time complexity

forward checking: once a unit-label pair (u,l) is instantiated at a node in the tree, the constraints imposed by the relations cause instantiation of some future unit-label pairs (u’,l’) to become impossible


if 1=A then 2 can either be D or F,…


17.3.2 Backtracking with Forward Checking

FTAB: future-error table FTAB(u’,l’)=1: still possible to instantiate (u’,l’) FTAB(u’,l’)=0: (u’,l’) already been ruled out FTAB(u’,l’)=X: (u’,l’) impossible from previous leve

l of recursion

one future-error table for each level of recursion in tree search


17.3.3 Backtracking with Discrete Relaxation

forward-checking algorithm: prunes search tree of nodes ruled out

discrete relaxation: iterative polynomial complexity procedure greatly reduces search for tightly-constrained

problems constrains search further for not tightly

constrained tree


17.3.4 Ordering the Units

Better to choose the unit that has the fewest labels left as the next unit.

We can modify the algorithms to keep track of how many labels remain for each unit and which unit has the least number of labels left.


17.3.5 Complexity

Consistent-labeling problem: NP-complete problem

Forward checking and look-ahead: Drastically reduce number of nodes searched. But do not change overall complexity.


17.3.6 The Inexact Consistent-Labeling Problem

Extracted line from images: some missing, partially missing, extra, distorted

Inexact consistent-labeling problem: allows some error for real-life problems

recursion of level previous from impossible )','( :)','(

1 of edges 6for ddistributeequally errors :61)','(

penalty no error thus no :0)','(

luXluerror

Gluerror

luerror


17.3.6 The Inexact Consistent-Labeling Problem

tree search: initially called with past_error=0 past_error: never allowed to exceed error thr

eshold ε


part of the inexact forward checking procedure for digraph matching


17.4 Continuous Relaxation

discrete algorithms: calling a label either possible or impossible

continuous procedure: associate probability or certainty that u assigned l


17.5 Vision Applications- Line Labeling with Discrete Relaxation

Assumption: simple blocks world scenes of planar polyhedra no shadows or cracks and trihedral vertices polygonal planar surfaces


Line Labeling with Discrete Relaxation





+: convex interior line segments -: concave interior line segments >, <: boundary line segments with visible

surface on right





each face is a plane, not a curved surface

Zollner Illusion: diagonals are parallelPoggendorf Illusion: two diagonal segments collinearHelmholtz’s Squares: two squares appear rectangularMuller-Lyer Illusion: both lines have the same lengthHering Illusion: two horizontal and parallel straight lines appear bowedWundt Illusion: two horizontal and parallel straight lines appear bowed

mermaid’s tail fin is Poseidon’s moustache

Young Girl/Old Woman: young girl’s chin is old woman’s noseVase/Faces: black region is background or foregroundPop In/Pop Out: spontaneous reversal of perceived concavities and convexitiesSix Cubes/Seven Cubes: reversal of perceived concavities and convexities

Belvedere: seven geometrical inconsistencies not readily apparent

1. cross front to rear, and vice versa

2. ladder’s base inside the building, but its top outside

4. examined by person on bench: geometrically impossible

3. Topmost level at right angles to middle level

Penrose Triangle: if it’s polyhedron, it could not form closed loopPenrose staircase: decends (ascends) all the way around to starting stepEscher Cube: top inconsistent with its baseTwo-pronged trident: three prong terminals with two prong base

consistent line labels necessary but not sufficient for physical realizability

(a) line drawing with no consistent labeling

(b) consistent labeling of line drawing of impossible polyhedron

(c) physically realizable line drawing with unrealizable consistent labeling

interpretation changes when image turned upside down

Implicit assumption: scene is lit from above

right-side up: two lava cones with cratersupside down: craters with mounds



set of units U: set of line segments of the line drawing

set of labels L: label set {+, -, >, <} only lines that meet at a junction constrain one anot

her T={(ui,uj)|ui and uj meet at a junction}

unit-label constraint relation R: defined according to legal labeling


Inexact Matching in 2D Shape Recognition

2D shape matching: recognition task in many machine vision applications

2D shape primitives: simple, near-convex pieces and intrusions into boundary

primitive: allowed to overlap


Inexact Matching in 2D Shape Recognition

intrusion relation Ri: (simple part 1, intrusion, simple part 2)

Ri: two simple parts touch or nearly touch and form boundary of intrusion

protrusion relation Rp: (intrusion 1, simple part, intrusion 2) Rp: simple part protrudes between two intrusion


17.5.1 Image Matching Using Continuous Relaxation

initial strong matches: provide context for matching less well-defined

objects islands of confidence

features: region size, intensity, location, texture, shape

measures relationships among primitives:

adjacency, relative position, distance


17.6 Characterizing Binary-Relation Homomorphism

S HR bbHbaHba

BHHbaB

AHSRH

BAHBBSAAR

3.)'imply )',( and ),((

on valued-single is 2.)),(such that ,bA,a(

on everywhere defined is 1.iff into of smhomomorphi a is

,,


17.6.1 The Winnowing Process)()'( implies )',(,),( bSaHRaaHba


17.6.1 The Winnowing Process)()'( implies ,),(,),'( 1 bSaHHbaRaa


17.6.1 The Winnowing Process


T: check if there is outgoing or incoming arrows

domain rangeR {1, 3, 4} {2, 3, 4, 5}S {a, b, c, d} {b, c, d, e}


from table at left, derive T2


repeat above process using T2

a

aa

aa

a

T

TT

TdcT

aaT

13

12

12

12

11

11

generate toprocess aboverepeat

generate of left table from

of left table generate thus,or beonly can 4,3:

else any thing benot can thus1 from choose:


17.6.2 Binary-Relation Homomorphism Characterization


17.6.3 Depth-First Search for Binary-Relation Homomorphism

similarly derived others above; as 13

1 aa TT

tree determined by depth-first search: range {a, b, c, d, e} allowed to repeat


IPPR Conference on Computer Vision, Graphics, and Image Processing

Title

Abstract

1. Introduction: what is dominant point? Why important 2.

Background: previous research, drawback, advantage Motivation: why your approach? Contribution: what’s new, original, or novel?

3. Approach, theory, method, steps 4. Experiments: setup, assumption, equipment 5.

Results: pictures with dominant points, working prototype on real-world images Performance measure: time, space, probability of detection/misdetection, false

alarm comparison with other work 6. Discussion, Conclusion, Future Work 7. Reference

Documents

Computer and Robot Vision II