Upload
ma-salih
View
141
Download
1
Embed Size (px)
Citation preview
1
Computer Vision and Video
Processing Using MATLAB
이웅재 차장,
Senior Application Engineer
2
Agenda
Introduction
Video and Image Processing Blockset
Demo: optical flow
Demo: stereo vision
Questions
3
Example Computer Vision and Video
Processing Tasks
Detect and track objects
Count objects in a scene
Stabilize camera motion
Deinterlace video frames
Create mosaicks and panoramas
Generate depth maps from stereo
image pairs
4
Common Challenges
Accessing and analyzing video data
Exploring algorithms and what-if
scenarios
Understanding the system-wide context
Rebuilding standard algorithms
Visualizing intermediate results
Testing and validating under real-world
conditions
4
5
Video and Image Processing Blockset
5
Design and simulate video and image processing
systems
Multimedia file I/O
Video display
Text and graphic overlays
Pre- and post-processing
Motion-based processing
Object detection and tracking
Feature recognition
Computer vision
6
Demo: Video Processing in MATLAB
6
Use optical flow on a video sequence
7
System Objects
7
Definition
MATLAB objects that represent time-based and data-driven
algorithms, sources, and sinks
Instantiate and configure hReader = video.MultimediaFileReader(‘viptraffic.avi’)
Execute within a loop step(hReader);
Other methods available reset
isDone
close
For related information, see the following webinar:
“Object-Oriented Programming in MATLAB”
88
Batch Processing
8
Load the entire video file and process it all at once
Source
MATLAB
Batch
Video
Algorithm
Memory
9
MATLAB Memory
Stream
Source
Stream Processing
9
Load a video frame and process it before moving on to the
next frame
Stream
Processing
10
Stream Processing in MATLAB is Hard
Need to maintain buffer
Explicit indexing
Explicit state management
myVid = mmreader(‘myvideofile.avi’);
numFrames = myVid.NumberOfFrames;
currentFrame = read(myVid,1);
numIter = 10;
opticalFlowOutput = zeros([size(currentFrame) numFrames]);
i = 1;
while i <= numFrames
prevFrame = currentFrame;
currentFrame = read(myVid,i);
flow = opticalFlow(currentFrame,prevFrame,‘horn-schunck’,...
numIter,‘magitude-squared’);
opticalFlowOutput(:,:,2:end) = opticalFlowOutput(:,:,1:end-1);
opticalFlowOutput(:,:,1) = flow;
i = i+1;
end
implay(opticalFlowOutput)
11
System Objects Make it Easier
Initialize objects
“In-the-loop” code is much simpler
Implicit states, buffering, and indexing
Video player runs in-the-loop
reader = videoMultimediaFileReader
reader.Filename = ‘myvideofile.avi’;
viewer = video.VideoPlayer
optical = video.OpticalFlow
optical.Method = ‘horn-schunck’;
optical.OutputValue = ‘Magitude-squared’;
optical.ReferenceFrameDelay = 3;
optical.MaximumIterationCount = 10;
while ~isDone(reader)
currentFrame = step(reader);
OF = step(optical, currentFrame);
step(viewer, OF);
end
12
Demo: Video Processing in MATLAB
12
Use optical flow to detect and counting moving
vehicles on a road
13
Video and Image Processing System
ObjectsAlphaBlender Deinterlacer IFFT2D MorphologicalErode
Autocorrelator2D DemosaicInterpolator ImageComplementer MorphologicalOpen
Autothresholder DeployableVideoPlayer ImageDataTypeConverter MorphologicalTopHat
BinaryFileReader EdgeDetector ImageFilter MultimediaFileReader
BinaryFileWriter FFT2D ImagePadder MultimediaFileWriter
BlobAnalysis GammaCorrector LocalMaximaFinder OpticalFlow
BlockMatcher GeometricRotator MarkerInserter PSNR
BoundaryTracer GeometricScaler Maximum Pyramid
ChromaResampler GeometricTransformer Mean ShapeInserter
ColorSpaceConverter GeometricTransformEstimator Median StandardDeviation
ConnectedComponentLabeler GeometricTranslator MedianFilter2D TemplateMatcher
ContrastAdjuster Histogram2D MedianFilter2D TextInserter
Convolver2D HistogramEqualizer Minimum Variance
CornerDetector HoughLines MorphologicalBottomHat VideoPlayer
Crosscorrelator2D HoughTransform MorphologicalClose
DCT2D IDCT2D MorphologicalDilate
14
Computer Vision Example 1:
Mosaicking and Stabilization
14
Register neighboring frames to create a large view of
the scene or stabilize camera motion
Applications:
Security
License plate recognition
Aerial surveying
Medical imaging
15
Computer Vision Example 2:
Object Detection and Tracking
15
Detect, classify, and track objects
in a scene
Applications:
Traffic monitoring
Cell counting
Lane departure
warning system
16
Computer Vision Example 3:
Stereo Vision
16
Extract 3D information from a pair of stereo images
Applications:
Obstacle avoidance
Automotive safety
Face recognition
Scene reconstruction
Part pickers
17
Stereo Vision Basics
17
Left Right
18
Stereo Vision Algorithm
18
Stereo rectification
Align images horizontally
Block matching
Find disparity map
Backprojection
Calculate 3D points
19
Video and Image Processing Blockset
Advanced Capabilities
Graphical design in Simulink
Fixed point modeling
Code generation
Targeting and verification
DSPs
FPGAs
For more information, see the following webinar:
“Image and Video Processing with DSPs and FPGAs”
20
Image Acquisition Toolbox
Live video and image
acquisition directly into
MATLAB and Simulink
Device property configuration
Live video previewing
GUI or functional interface
Support for multiple hardware
vendors
21
Why Use MATLAB for Computer Vision
and Video Processing?
System objects simplify coding and reduce errors
Algorithms for computer vision and video processing
Flexible environment enables algorithm exploration
Support for reading and writing many video file formats
Text and graphics annotations on video data
Support for cameras and frame grabbers
Embedded hardware design considerations
22
For More Informationmathworks.com/products/viprocessing/demos.html
Other examples:
People tracking
Video mosaicking
Video stabilization
Lane departure warning system
Abandoned object detection
23
For More Information
Experiment with product by downloading a trial
http://www.mathworks.com/products/viprocessing
Related demos and webinars
More detailed product information
Links to product documentation
User stories
Contact us
Sales representative
24
Questions?