"Air Drum” Project

Elad Gilboa

Zvika Jakobovich

Instructor: Ari Todtfeld

מכון טכנולוגי לישראל - הטכניוןהפקולטה להנדסת חשמל

המעבדה לבקרה ורובוטיקה

Topics

Goals and Objectives System Overview System Architecture

Additional Experiments Improvement Plans

3

Goals and Objectives

Develop a real time monitoring system which

Enables simulation of playing the drums in

a user friendly environment

Is applicable in different environments with

modification of parameters

4

System Overview

Main Window

Drums Settings

Filter Settings

5

Reducing Background Noise

הפרעות הנוצרות מצבעים דומים ברקע

הפרש העוצמות מנורמל

w *= SQR((prevFrame[x][y] - v)/255.0)

6

System Architecture

Rio System to capture video input Add-on modules/functions to

Locate center of massMonitor location of sticks positions to

ensure smooth motionDecide if there was a drum hitCalculate the angle and amplitudeDecide what tone should be played

7

StreamCap.exe

StreamCap.cpp

StreamCapDoc

StreamCapView

Filter

myPoint

drums

DisplayPicture()

RGB2HSV()

updateCM()nextPoint()

doSound()HitDrumL() HitDrumR()

Rio Basic Program

RioRoutines

Utiles

LegendLegendclass

file

8

http://escience.anu.edu.au/lecture/cg/Color/HSV_HLS.en.html

HSV Space RGB Space

RBG2HSV Function

Convert RGB to HSVLess affected by changes in

illumination

9

RBG2HSV Function (Cont’d)

Filter imageSet value of pixel outside the

saturation limits to zeroUse the pix’s Hue to get its new value

from a Look Up Table (LUT)

From every quadrate takeOnly the first value

10

RBG2HSV Function (Cont’d)

Calculate Center of Mass Use the pixels new values to calculate

the center of mass

*i i

cm

i

m xX

m

X

11

Weighted Average

UpdateCM Function Monitor the location of the new Center of Mass point Use a weighted average to smooth motion

.4

RGB2HSV(Current)

New Point

.3.2.1

Previous Point

PredictedPoint

12

newPoint Function

Check whether the new CM point

starts a new motionCalculate linear regression

Three tests for beginning of new

motion

Add point to list

13

newPoint Function (Cont’d)

Calculate linear regression• The basic assumption is that the

motion is linear and smooth

b = =

a = =

14

newPoint Function (Cont’d)

Three Tests• Distance between

3 last points

• Distance of new point to line

• Reversing motion

2 21 0 1 2( ) ( )d x x y y

0 0

2

*

1

y a x bd

a

15

doSound Function

Calculate the Hit’s force – to set volume

Calculate the Hit’s angle

Force =Length

(Number of Points)2

1tan ( )a

16

Additional Experiments

Median filter Greater weight to smooth areas Two calculations of CM Two calculations of CM + motion only Different weight function for Hue window

17

Improvement Plans

Utilization of a positioning sensor Improvements in adaptation to

changes in light Improvements in noise filtering Improvements in the motion control Multithreading Direct Sound

18

Thank You!