Hand Gesture Recognition to Audio Conversion using flex sensor

SKU: PAN_EMB_065 Categories: , Tag:

Description

Hand Gesture Recognition to Audio Conversion using flex sensor

Abstract:

          Sign language plays a major role in dump people communicating with normal people. It is very difficult for mute people to convey their message to normal people. Since normal people are not trained in hand sign language. In emergency times conveying their message is very difficult. So the solution to this problem is to convert sign language into a human hearing voice. There are two major techniques available to detect hand motion or gesture such as vision and non-vision techniques and convert the detected information into voice through raspberry pi. In vision-based techniques, cameras will be used for gesture detection and non-vision-based technique sensors are used. In this project, a non-vision-based technique will be used. Most of the dumb people are deaf also. So normal people’s voices can be converted into their sign language. In an emergency situation, the message will automatically send to their relation or friends. Hand Gesture Recognition to Audio Conversion using flex sensor


Introduction:

Sign language has become the most common method of communicating with those people who cannot speak. It is a language that uses hand motions to express the alphabet and words. People who are using sign language were recorded. The vision method has become the popular method used for sign recognition in the past decades. It is a system that uses a camera to sense the information that has been obtained through finger motions. It is the most commonly used visual-based method. It has been a tremendous effort and has been gone into the development of vision-based sign recognition systems worldwide. Vision-based gesture recognition systems can be divided into direct and indirect methods.  In earlier days for recognizing hand motion, a vision-based technique is used. But in this method the environmental effect in the recognized image is high.


Hand Gesture Recognition to Audio Conversion using flex sensor

Existing system: 

  • The vision-based technique is used.  
  • It is not accurate in all directions Small variations are not determined
  • Sign language
  • The vision method has become the popular method used for sign recognition in the past decades

Disadvantage  

  • Vision-based technique in this method the environmental effect in the recognized image is high. 
  • They have to show their hands in front of the camera

Proposed system:

  • The dump people’s hand motion or gesture can be detected and then it will be converted into a human hearing voice signal.  
  • Which is a hand glove, is put on by a mute person,  the device would recognize the letters almost accurately
  • Portable Easy implementation, and Small size
  • It is a system that uses a flex sensor to sense the information that has been obtained through finger motions.

Advantage 

  • Reduction in environmental effect

Block Diagram 

Hand Gesture Recognition to Audio Conversion using flex sensor
Hand Gesture Recognition to Audio Conversion using flex sensor

BLOCK DIAGRAM DESCRIPTION

The flex sensor is used to detect the hand motion or gesture and the accelerometer sensor (MEMS sensor) is used to detect the three-dimensional motion of a hand. The controller used here is raspberry pi which does not contain any analog to digital converter. Here both sensors are analog in nature. Flex sensors and memes are connected with raspberry pi in every direction. Flex sensors are connected to GPIO pins of Raspberry pi. This sensor data can be accessed through python programming. The normal people’s voice will be detected and converted the voice into a hand gesture. Raspberry pi does not have an inbuilt sound card and therefore it may not support microphones while using an audio jack.


Hardware tools 

  • Raspberry pi

  • Flex sensor

  • Audio device

  • MEMS


Software tools

  • Program: Python 
  • Platform: Python 3 IDLE
  • Raspberry pi os: Raspian os 
  • Library: OpenCV

REFERENCE 

[1] B.G.Lee, member IEEE and S.M.Lee “smart wearable hand devices for sign language interpretation system with sensor fusion”, volume 18 issue: 3, February 1, 2018, IEEE sensor Journal. 

[2] Mohammed Elmahgiubi, “sign language translator and gesture recognition”, 17 December 2015, IEEE. 

[3] Lih-Jen Kau, member IEEE Bo-xun Zhuo, “a real-time portable sign language translation system”, 26 January 2017, IEEE Journal. 

[4] Merin is Koshi, “a survey on advanced technology communication between deaf/dumb people using eye blink and flex sensor”, 01 February 2018, IEEE Journal. 

[5] Lafayette Ahmed, “electronic speaking system for speech impaired people:  speak up”, 29 Oct 2015, IEEE Journal.

 

Customer Reviews

There are no reviews yet.

Be the first to review “Hand Gesture Recognition to Audio Conversion using flex sensor”

This site uses Akismet to reduce spam. Learn how your comment data is processed.