UCLA bioengineers have designed a glove-like device that can translate American Sign Language into English speech in real time through a smartphone app. Their research is published in the journal Nature Electronics.
The system includes a pair of gloves with thin, stretchable sensors that run the length of each of the five fingers. These sensors, made from electrically conducting yarns, pick up hand motions and finger placements that stand for individual letters, numbers, words and phrases.
Not only will this innovation help clear the language barrier, it will also facilitate the learning of Sign Language for all, promoting inclusivity for all. Moreover, the design is not as expensive as one would think, allowing massive production and sales. This could have a massive positive impact on lives all around the globe.
The device then turns the finger movements into electrical signals, which are sent to a dollarcoin–sized circuit board worn on the wrist. The board transmits those signals wirelessly to a smartphone that translates them into spoken words at the rate of about a one word per second.
The researchers also added adhesive sensors to testers’ faces — in between their eyebrows and on one side of their mouths — to capture facial expressions that are a part of American Sign Language. Its an additional benefit of this device.
When testing the device, four people are engaged who are deaf and use American Sign Language. The wearers repeated each hand gesture 15 times. A custom machine-learning algorithm turned these gestures into the letters, numbers and words they represented. The system recognized 660 signs, including each letter of the alphabet and numbers 0 through 9.
The device developed by the UCLA team is made from lightweight and inexpensive but longlasting, stretchable polymers. The electronic sensors are also very flexible and inexpensive.