1. nvassilieva@hp.com HP Labs Russia 16 2012, Computer Science Center 8:

2. 2 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. ? () , (image classification) , , , (object detection, localization) , (object segmentation) Slide credit: M. Everingham 3. 3 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 4. 4 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. ? , : , , - 5. 5 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. ? Fig. credit: Shi, Malik 6. 6 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. ? Fig. credit: Malik 7. 7 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. ? Tiger Grass Water Sand 8. 8 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. , - ? ? , ? ? 9. 9 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. , , 10. 10 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. - v.s. - Bottom-up v.s. top-down Pixel-based v.s. region-based/area-based Local v.s. global Feature-space based v.s. image-domain based Region-based v.s. edge-based - - ? ? 11. 11 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. - v.s. - Bottom-up v.s. top-down Pixel-based v.s. region-based/area-based Local v.s. global Feature-space based v.s. image-domain based Region-based v.s. edge-based v.s. Automated v.s. user-directed ? ? 12. 12 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 255192, 9603 255192, 8 D. Comaniciu, P. Meer: Robust Analysis of Feature Spaces: Color Image Segmentation, CVPR'97 13. 13 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. - Vincent Levesque, Texture Segmentation Using Gabor Filters http://www.cs.huji.ac.il/~simonp/papers/ip_project.pdf 4 14. 14 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. Thomas Hofmann, Jan Puzicha, Joachim M. Buhmann, Deterministic annealing for unsupervised texture segmentation, EMMCVPR 97, pp. 213-228 15. 15 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. Slide credit: O. Carmichael Stella Yu, PhD thesis, 2003 16. 16 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. , (motion) http://www.svcl.ucsd.edu/projects/motiondytex/demo.htm : , http://vcla.stat.ucla.edu/old/Barbu_Research/Motion_estim/index.html 17. 17 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. (depth) M. Domnguez-Morales, A. Jimnez-Fernndez, R. Paz-Vicente, A. Linares-Barranco and G. Jimnez-Moreno Stereo Matching: From the Basis to Neuromorphic Engineering 18. 18 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. / 19. 19 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. Slide credit: O. Carmichael 2-D 20. 20 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. X1 X2 X1,X2 ( , , ,...) 21. 21 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. Source: K. Grauman 22. 22 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. X1 X2 X3 : ? ( )? ? ? 23. 23 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. k- R G R G : min Slide credit: http://www.cs.washington.edu/education/courses/cse576/08sp/lectures/segment.pdf 24. 24 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. k- (k-means) 1. k 2. k (, ) 3. : 4. , 5. 4, 3 Java : http://home.dei.polimi.it/matteucc/Clustering/tutorial_html/AppletKM.html Slide credit: http://www.cs.washington.edu/education/courses/cse576/08sp/lectures/segment.pdf 25. 25 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 0 1 2 3 4 5 0 1 2 3 4 5 Distance Metric: Euclidean Distance k1 k2 k3 k-: 1 Slide credit: Lihi Zelnik-Manor 26. 26 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 0 1 2 3 4 5 0 1 2 3 4 5 k1 k2 k3 Distance Metric: Euclidean Distance k-: 2 Slide credit: Lihi Zelnik-Manor 27. 27 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 0 1 2 3 4 5 0 1 2 3 4 5 k1 k2 k3 Distance Metric: Euclidean Distance k-: 3 Slide credit: Lihi Zelnik-Manor 28. 28 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 0 1 2 3 4 5 0 1 2 3 4 5 k1 k2 k3 Distance Metric: Euclidean Distance k-: 4 Slide credit: Lihi Zelnik-Manor 29. 29 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 0 1 2 3 4 5 0 1 2 3 4 5 expression in condition 1 expressionincondition2 Distance Metric: Euclidean Distance k-: 5 k1 k2 k3 Slide credit: Lihi Zelnik-Manor 30. 30 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. Image Intensity-based clusters Color-based clusters k- K-means clustering based on intensity or color is essentially vector quantization of the image attributes Clusters dont have to be spatially coherent Slide credit: Lihi Zelnik-Manor 31. 31 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. Source: K. Grauman Distance based on color and position Slide credit: Lihi Zelnik-Manor 32. 32 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. k- Clustering based on (r,g,b,x,y) values enforces more spatial coherence Slide credit: Lihi Zelnik-Manor 33. 33 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. k-Means: Converges to a local minimum of the error function (: K-means++) Memory-intensive Need to pick K Sensitive to initialization Sensitive to outliers Only finds spherical clusters 34. 34 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. Mean-shift for image segmentation Useful to take into account spatial information instead of (R, G, B), run in (R, G, B, x, y) space D. Comaniciu, P. Meer, Mean shift analysis and applications, 7th International Conference on Computer Vision, Kerkyra, Greece, September 1999, 1197-1203. http://www.caip.rutgers.edu/riul/research/papers/pdf/spatmsft.pdf More Examples: http://www.caip.rutgers.edu/~comanici/segm_images.html Slide credit: http://www.cs.washington.edu/education/courses/cse576/08sp/lectures/segment.pdf 35. 35 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. The mean shift algorithm seeks modes or local maxima of density in the feature space Mean shift algorithm image Feature space (L*u*v* color values) 36. 36 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. Find features (color, gradients, texture, etc) Initialize windows at individual feature points Perform mean shift for each window until convergence Merge windows that end up near the same peak or mode Mean shift clustering/segmentation 37. 37 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. Search window Center of mass Mean Shift vector Mean shift Slide by Y. Ukrainitz & B. Sarel 38. 38 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. Search window Center of mass Mean Shift vector Mean shift Slide by Y. Ukrainitz & B. Sarel 39. 39 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. Search window Center of mass Mean Shift vector Mean shift Slide by Y. Ukrainitz & B. Sarel 40. 40 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. Search window Center of mass Mean Shift vector Mean shift Slide by Y. Ukrainitz & B. Sarel 41. 41 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. Search window Center of mass Mean Shift vector Mean shift Slide by Y. Ukrainitz & B. Sarel 42. 42 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. Search window Center of mass Mean Shift vector Mean shift Slide by Y. Ukrainitz & B. Sarel 43. 43 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. Search window Center of mass Mean shift Slide by Y. Ukrainitz & B. Sarel 44. 44 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. Cluster: all data points in the attraction basin of a mode Attraction basin: the region for which all trajectories lead to the same mode Mean shift clustering Slide by Y. Ukrainitz & B. Sarel 45. 45 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. http://www.caip.rutgers.edu/~comanici/MSPAMI/msPamiResults.html Mean shift segmentation results 46. 46 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. More results 47. 47 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. Mean shift: Does not assume spherical clusters Just a single parameter (window size) Finds variable number of modes Robust to outliers Output depends on window size Computationally expensive Does not scale well with dimension of feature space 48. 48 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. Probabilistic clustering Basic questions whats the probability that a point x is in cluster m? whats the shape of each cluster? K-means doesnt answer these questions Basic idea instead of treating the data as a bunch of points, assume that they are all generated by sampling a continuous function This function is called a generative model defined by a vector of parameters Slide credit: http://www.cs.washington.edu/education/courses/cse576/08sp/lectures/segment.pdf 49. 49 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. Expectation maximization (EM) Goal find blob parameters that maximize the likelihood function: Approach: 1. E step: given current guess of blobs, compute ownership of each point 2. M step: given ownership probabilities, update blobs to maximize likelihood function 3. repeat until convergence EM demo: http://lcn.epfl.ch/tutorial/english/gaussian/html/index.html Slide credit: http://www.cs.washington.edu/education/courses/cse576/08sp/lectures/segment.pdf 50. 50 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 51. 51 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. d( , ) >> d( , ) , , / : d(.,.) : , . Slide credit: O. Carmichael 52. 52 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. Slide credit: O. Carmichael 53. 53 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. q Automatic graph cut Fully-connected graph node for every pixel link between every pair of pixels, p,q cost cpq for each link cpq measures similarity o similarity is inversely proportional to difference in color and position p Cpq c Slide credit: http://www.cs.washington.edu/education/courses/cse576/08sp/lectures/segment.pdf 54. 54 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. Segmentation by Graph Cuts Break Graph into Segments Delete links that cross between segments Easiest to break links that have low cost (similarity) similar pixels should be in the same segments dissimilar pixels should be in different segments w A B C Slide credit: http://www.cs.washington.edu/education/courses/cse576/08sp/lectures/segment.pdf 55. 55 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. Min cut Link Cut set of links whose removal makes a graph disconnected cost of a cut: A B Find minimum cut gives you a segmentation Slide credit: http://www.cs.washington.edu/education/courses/cse576/08sp/lectures/segment.pdf 56. 56 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. But min cut is not always the best cut... Ideal Cut Cuts with lesser weight than the ideal cut and it is NP-complete 57. 57 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. A B Normalized Cut a cut penalizes large segments fix by normalizing for size of segments volume(A) = sum of costs of all edges that touch A Normalized Cut Slide credit: http://www.cs.washington.edu/education/courses/cse576/08sp/lectures/segment.pdf 58. 58 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. Slide credit: O. Carmichael 59. 59 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. Slide credit: O. Carmichael 60. 60 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. Slide credit: O. Carmichael 61. 61 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. Slide credit: O. Carmichael 62. 62 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 2-D 2-D (Markov Random Fields, MRF) Slide credit: O. Carmichael 63. 63 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. Slide credit: O. Carmichael 2-D 64. 64 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. Deformable contours/active contours/snakes - , , Lei He, Zhigang Peng, Bryan Everding, Xun Wang, Chia Y. Han, Kenneth L. Weiss, William G. Wee, A comparative study of deformable contour methods on medical image segmentation, Image and Vision Computing 26 (2008) 141163 65. 65 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. : ? ? s (xi(s), yi(s)) 66. 66 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. : , ? : 67. 67 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. : , 68. 68 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. - (, ) 69. 69 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. Einternal , : internal image user 70. 70 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. Eimage / : internal image user 71. 71 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. Euser : / internal image user 72. 72 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 1. , 2. 3. 2 , 73. 73 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. D. Martin, C. Fowlkes, D. Tal, J. Malik. "A Database of Human Segmented Natural Images and its Application to Evaluating Segmentation Algorithms and Measuring Ecological Statistics", ICCV, 2001 Berkeley Segmentation DataSet [BSDS] 74. 74 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. Bottom-up Top-down (active contours)