Gesture controlling of Robots

April 9, 2023

1

GESTURE CONTROL ROBOTS

Guided By: Presented By:Vibin C Thomas Jibin PouloseAsst.Professor S7 EEEASIET No:-29

Date of Presentaion : 27/08/2014

Contents•Robot and Types•Controlling Methods•Gesture Control•Transmitting

▫Flex Sensor▫Accelerometer▫Robotic Hands▫Image Processing

•Receiving•Comparison of different systems•References

April 9, 2023

3

WHAT IS A ROBOT

ROBOT

Electro-mechanical device

It finds it’s uses in all as-pects of our life.

Performs Various tasks

May be human controlled or automated

April 9, 2023

4

TYPES OF ROBOTS (BASED ON CONTROL)ROBOTS

MANUAL

WIREDWIREL

ESS

SEMI-AUTONOMOUS

AUTONOMOUS

PRE-PROGRAM

MED

SELF LEARNIN

G

[Figure 1]

April 9, 2023

5

CONTROLLING OF ROBOTS

•Gestures

•Bluetooth enable devices

•Voice control

•Text

•Commands

April 9, 2023

6

WHY GESTURE CONTROL?

• Human Machine Interaction (HMI )[3]

• Interact naturally without any mechanical devices.

• Interface to both human and computer.

• Substitute the common interface devices

• Can also be exploited to extend their functionality.

• Avoid to program which required extensive hard work

• minimized the need for text interfaces and GUIs (Graphical

User Interface).

April 9, 2023

7

WHAT IS A GESTURE

• Motion originate from face or hand• An action seen by someone else • Convey some piece of information. • Movement of part of the body, especially a hand or the head, to

express an idea or meaning.

[Figure 2]

April 9, 2023

8

TRANSMITTER SIDE

TransducerSensor

Signal Conditioning

Micro Controller

EncoderTransmitter

Flex Action

Control Signals

[Figure 3]

April 9, 2023

9

SENSORS AND SIGNAL CONDITIONING

• Physical movements cannot be processed and transmitted

• Sensors converts physical movements into electrical signals.

• Converts different gesture movements of hands into corresponding electrical signals.

• Commonly used transducers are Flex sensors, Accelerometer and Gyroscopes.

• They are placed in a glove and used to wear.

April 9, 2023

10

FLEX SENSORS• Analog resistors work as variable analog voltage divider.• Carbon resistive elements with thin flexible substrate.• More carbon means less resistance. • Resistance increase with bending• output a stream of data that varies with degree of bend

[Figure 4]

April 9, 2023

11

VARIATION OF RESISTANCS

Voltage reduces with increase increase in deflection of flex sensor [2]

Resistance increases with increase increase in deflection of flex sensor [2]

[Figure 5]

[Figure 6]

April 9, 2023

12

Vout=5Vx Rsensor Rsensor+Rpullup

Hand glove with flex sensors in each fingers

[Figure 7]

[Figure 8]

April 9, 2023

13

ACCELEROMETER

• Gives an analog data while moving in X,Y,Z direction• Piezoelectric material sandwiched between base and resonant

mass• Motion causes resonant mass to move• Force to move mass is transmitted through piezoelectric

material (F = ma)• Compression/Tension or Shear of piezoelectric material causes

charge buildup proportional to acceleration• Equivalent electric circuit measures the capacitance

April 9, 2023

14

ICP ACCELEROMETER

• To Make Accelerometer Use Simple, Incorporate Electronics Package Into Transducer

• Equivalent Circuit => No Dependence on Cable• Output Sensitivity Constant

[Figure 9]

April 9, 2023

15

ROBOTIC HANDS

• Have five fingers each having three joints

• Each joint requires each sensors

• Readings from each sensor is taken for signal conditioning

[Figure 10] [4]

[Figure 11] [4]

April 9, 2023

16

CONTROL OF ROBOTIC HAND

•I2c communication•SDA ,SCl•Unique ID number•Pwm control of servo

[Figure 12]

April 9, 2023

17

IMAGE PROCESSING

• A Vision Based Gesture Recognition system• Image: array, or a matrix, of square pixels (picture elements)

arranged in columns and rows.• each picture element has an intensity ranges from 0 to 255 (for

8bit)• Common steps : image scanning, storing, enhancing and

interpretation.• image will be converted to digital form using a

scanner/digitizer and then process it. • It is defined as the subjecting numerical representations of

objects to a series of operations

April 9, 2023

18

DIFFERENT TYPES OF VISION BASED GESTURE RECOGINATON [3]

Mono Camera Based System

Multi Camera Based System

Infra red Camera Based System[Figure 14]

[Figure 15

[Figure 13]

April 9, 2023

19

• First Model the motion and posture and infer the gesture [3]

3D Textured volumetric model

3D wireframe volumetric model

3D Skeleton Model

Binary Silhouette

Contour

[Figure 16]

April 9, 2023

20

CONTROLLER AND TRANSMITTER

• Sensors outputs a stream of data that varies with degree of bend

and movement of hand.

• Analog voltage is given to controller.

• Mapped into digital values.

• Compare with the programmed values.

• Generate the corresponding signal of hand movement.

• Signal is transmitted to robot through RF Transmitter.

• Transmitter transmits the signal to robot.

April 9, 2023

21

RECEIVER SIDE

Transducer

Reciver Decoder

Micro Controller

Driver circuitMotor

Signal

Movments

Receiver receives the signal

Converts into electrical signal and give to controller

Controller process the signal and drives the motors corresponding to the signal.

[Figure 17]

COMPARISON

Methodologiess Advantages Disadvantages

Flex Sensor •Simple Programming•Simple Construction•Simple Calibration

Expensive

Accelerometer Simple Construction ExpensiveCalibration is difficult

Image Processing Less CostRequires only a camera

Programming is very difficult

April 9, 2023

23

REFERENCES[1] Ruize Xu, Shengli Zhou, and Wen J. Li, Fellow, IEEE

“MEMS Accelerometer Based Nonspecific-User Hand Gesture Recognition”

[2]Abidhusain Syed, Zamrrud Taj H. Agasbal, Thimmannagouday Melligeri, Bheemesh Gudur

“Flex Sensor Based Robotic Arm Controller Using MicroController”

[3]Miss. Kawade Sonam P, Prof. V. S. Ubale

IOSR Journal of Electronics and Communication Engineering (IOSR-JECE) ISSN: 2278-2834, ISBN: 2278-8735, PP: 19-26

[4] “Gesture Recognition-A Review”

Dr. Shantanu K. Dixit, Mr. Nitin S. Shingi

“Implementation of Flex sensor and Electronic Compass for Hand Gesture Based Wireless

Automation of Material Handling Robot”

[5] Umesh Yadav, Ankur Tripathi , Sonali Dubey ,S.K.Dubey

“Gesture Controll Robot”

Questions??