MOUSE SIMULATION USING TWO COLOURED TAPES

Presented By Neena R Krishna S7 CSE BTECH,SNGIST

Contents
Introduction

Realted Works
Proposed System Hand Recognition and Colour Tape Detection Mouse Cursor Movements Click Events Conclusion

Introduction
Novel approach for Human Computer Interaction

(HCI) . control the cursor movement using a real-time camera . Current method

- involve changing mouse parts

- adding more buttons -changing the position of the tracking ball.

Contd..
Proposed Method

-use a camera and computer vision technology,

-image segmentation and gesture recognition

-to control mouse tasks

1)left and right clicking
2) double-clicking and scrolling

Perform everything as current mouse devices

can.

The software will be developed in JAVA language. Use two colour tapes on finger to perform actions.

Contd..

- One of the tapes will be used for controlling cursor movement while the relative distance between -the two coloured tapes will be used for click events of the mouse. -the system will provide a new experience for users in interacting with the computer.

Contd..
Modern Graphical user interface (GUI)

- current standard interface on personal computers is well- defined - it provides an efficient interface for a user to use various applications on a computer
GUIs combined with mice and track pads

- reducing the richness - variety of human communication down to a single point.

Related Work
Index finger for cursor movement .
Angle between index finger and thumb for clicking

events. Finger tip tracking to control the motion of the mouse. A click of the mouse button was implemented by defining a screen. Such that a click occurred when a users hand passed over the region.

Contd..
Another approach

- only the finger-tips to control the mouse cursor

and click . - based on image density. - required the user to hold the mouse cursor on the desired spot for a short period of time.
Next approach
used the motion of the thumb to mark a clicking event

thumb.

Contd..
A study of the existing systems for on-screen

choice selection reveals that people are still limited to the use of devices Such as mouse, touchpad, joystick, trackball and touch screen. All these devices need contact of hand with them.

Proposed System -System Overview

Hand Recognition and Colour Tape Detection

1.

System is to separate the potential hand pixels from the non-hand pixels.

-done by background subtraction scheme. 2. System startup, a pair of background images is captured -represent the static workspace from camera

view
3. After background subtraction, the process of skin segmentation is done .

Contd..
4.Histogram-based skin classifier assigns each of the RGB pixels in the training set - to either a 3D skin histogram or non-skin histogram.

- the probability is computed that a given RGB

color belongs to the skin or non-skin classes . - skin segmentation process outputs an image which is ready for detection of color tapes in the finger. - algorithm based on HSV color space is used . -HSV stands for Hue,Saturation,Value.

Contd..
5. convert the RGB pixels into the HSV color plane

- so that it is less affected to variations in shades

of similar color. -H runs on 0 to 360 degree -S value is degree of strength or purity and from 0
to 1 - s=1 purest colour.

-V for brightness and from 0 to 1 -0 is black

Contd..
resulting binary image is then run through a

convolution phase to reduce the noise introduced.

Mouse Cursor Movement

Contd..
Using the index finger with yellow colour tape as a

cursor controller to control mouse cursor movement Two different approaches for moving the mouse cursor:
position mapping the index finger on a camera screen to a desktop screen position. 2. weighted speed cursor control - the difference of the finger of the current image and the previous image is found.
1.

Contd..
-And the distance between the two is computed
- the mouse cursor is moves fast if the gap between the two finger images is far. - if the gap is close then the cursor moves slow.
In proposed system First method is used.

-More accurate than second method.

Click Events
The click events for the mouse are mapped with

different hand gestures. processing the distance between the two coloured tapes in the fingers. 1. Left Click
- the system records the distance between the

yellow and red tapes in the index finger and the thumb respectively.

Contd..
-the index and thumb must be apart as much as

possible so as to get maximum distance - This distance is regarded as the threshold distance for the event

- as the thumb moves towards the index finger, the

distance between yellow and red tapes is decreases . - when the thumb is close to the index finger the system records the reduced distance between them.

Contd..
-When the distance between the tapes is reduced or less ,then consider the event as the left click event of the mouse cursor. Thus mathematically, D < D -where D is the distance between the yellow and red tapes in the index finger and the thumb Suppose the distance between the tapes at any time is d then for left click event d D

Contd..
-where D is the reduced distance

-where d is distance between the tapes at any time.

Contd..

Initialisation of threshold distance

Contd..

Reduced distance (D)

Contd..
2.Right Click

- The right click event of the cursor is simulated

using the concept of waiting time. - If the yellow tape on the index finger is waiting for 7 seconds(say) in front of the camera pointing at the same location
- then the event is recognised as the right click event of the mouse cursor

Contd..
-Mathematically,

D < d D Waiting time = 7 sec.

- the distance between the red and yellow tapes
should be between D and D respectively

Contd..

Contd..
3. Double Click

-Similar to right click.

-The only difference is that the finger gesture used . - If both the colour tapes are waiting for the time 7 seconds and the distance between the colour tapes is D then the event is recognised as double click event of the mouse cursor.

Contd..

Contd..
-Mathematically

d D Waiting time = 7 sec.

Conclusion
The proposed system will completely revolutionize

the way people would use the computer system. Presently, the webcam, microphone and mouse are an integral part of the computer system. Product which uses only webcam would completely eliminate the mouse.

Contd..
Also this would lead to a new era of Human

Computer Interaction (HCI) where no physical contact with the device is required.

QUERIES.????