Object Segmentation in Video: A Hierarchical Object Segmentation in Video: A Hierarchical Variational Approach for Turning Point Variational Approach for Turning Point

Trajectories into Dense RegionsTrajectories into Dense Regions层次化变分法用于稠密的视频运动分割

Peter Ochs and Thomas BroxUniversity of Freiburg, Germany

ICCV 2011

Thomas Brox, Professor in University of Freiburg, Germany

Experience:

Received PhD from Saarland University in 2005.

2005-2007 Post Doctor in Born University.

2007-2008 Temporary Professor in University of Dresden.

2008-2010 Post Doctor in UC Berkerley with J. Malik.

Main Interests:Optical Flow, Segmentation, Human Motion

Representative Work:Brox Optical Flow(ECCV’04 best paper)

LDOF (PAMI’10)

Segmentation (ECCV’10)

Video SegmentationVideo SegmentationTwo Tasks

◦Shots Segmentation◦Spatial-Temperal Cues Segmentation

Motion Segmentation

Motion SegmentationMotion SegmentationOptical flow based

◦Earlier Methods◦Layers

Feature trajectory based◦Most Popular in the last 10 years◦Utilize 3D Motion◦Related to Subspace Clustering

Hybrid methods using both motion and static cues

Sparse Point Segmentation (1/6)Sparse Point Segmentation (1/6)

Thomas Brox and Jitendra Malik, Object Segmentation by Long Term Analysis of Point Trajectories, ECCV 2010

Optical Flow to obtain long-term point Trajectories

Sparse Point Segmentation (2/6)Sparse Point Segmentation (2/6)

Similarity Definition

Sparse Point Segmentation (3/6)Sparse Point Segmentation (3/6)

Similarity Definition

Sparse Point Segmentation (4/6)Sparse Point Segmentation (4/6)

Standard Spectral Clustering

Sparse Point Segmentation (5/6)Sparse Point Segmentation (5/6)

Spectral Clustering with Spatial Regularity◦Automatically determine cluster number

Sparse Point Segmentation (6/6)Sparse Point Segmentation (6/6)

Main Contribution◦Very Sparse Feature Points 0.01%-> Sparse

points (3%)

MotivationMotivation

Single-Level Variational modelSingle-Level Variational model

+

Solution (1/2)Solution (1/2)

Euler-Lagrange Equation:

Solution (2/2)Solution (2/2)Euler-Lagrange Equation:

Successive over-relaxation:

solve

by

AX=B, where A = D - L - U

Multi-Level Variational modelMulti-Level Variational model

Multi-level Variational ModelMulti-level Variational Model

Why Multi-level continuous model?Why Multi-level continuous model?

Multi-level◦Information can take a shortcut via a coarser

level where this noise has been removed.Continuous

◦Less block artifacts

SolutionSolution

Euler-Lagrange Equation:

k=0

k>0

SolutionSolution

Qualitative ResultsQualitative Results

Qualitative ResultsQualitative Results

Quantitative ResultsQuantitative Results

SummarySummaryCombining Motion Cues and Static CuesPropose a Multi-level Variational Method