What If Sign Language Could Be Heard? This Kenyan Techie Is Making It Happen With His ‘Magic Gloves’

Roy Allela’s six-year-old niece was born with a hearing impairment. Being one of the 34 million children worldwide that are estimated to be battling disabling hearing loss, the poor little girl struggled to communicate with family and friends as attempts at conversation were often punctuated with moments of awkward misinterpretations and a general inability to convey her truest feelings.

Imagine not being able to hear or speak; while most people tend to jump at the idea of sight being the most essential of the five senses (or is it six these days?), I would like to think of hearing and speech ability as a rather close second – or perhaps the margins are even finer than I think and it’s more like a tie.

In any case, that was pretty much it for Uncle Roy’s little niece who would be often be left distraught and throw ‘inaudible’ tantrums whenever she wanted something and couldn’t get it mostly because no one could tell what she wanted. Sad but true for most kids living with the condition, and maybe also living with people that don’t know much about sign language or facial expressions.

It’s not like the rest of the little girl’s family was having a swell time with the whole situation either. Folks at home were often tripping over themselves in attempts to communicate with her or answer her requests. This inability to communicate made it rather difficult for the family to connect with their little girl during the first six years of her life. But all that is beginning to seem like a thing of the past now.

Dejection. Frustration. Exasperation. Pain. Those feelings were mostly the case for most members of the poor girl’s immediate and extended families – well, everybody else except Uncle Roy, it would seem.

Roy Allela had always been fond of his little niece ever since she was an infant. When the unfortunate news of her condition became family knowledge, he cherished her even more. But with time now flying fast, the little girl was beginning to seem distant. Very disturbing, even his for his usually calm and optimistic self.

But he didn’t lose his head, though; which would have been some loss given that he carried a pretty good one on his neck. He began to think up ways to bring back his ‘baby girl’. Now, lavishing her with presents and spending quality time with her would normally seem like a plan, but he may have decided to give her more – something of the ultimate gift for persons living with her condition, as some might say.

Roy Allela opted to ‘gift’ his niece with something that was, to a large extent, neurologically unfeasible for someone with the condition – the ability to communicate by speaking. How he went about the whole thing, though, was nothing short of ingenious. The good thing about Roy was that he was more than just a doting uncle who was big on spoiling kids with chocolatey treats – he was also a highly-skilled engineer (you’ll sure get the idea why I mentioned the head he carries on his neck too).

As necessity is the mother of invention, Roy landed his wonderful creation. Inspired by his niece, the 25-year-old has brought to life a unique invention which is a smart glove that converts sign language gestures into speech.

Essentially, his creation could be thought of as a sign-to-speech device that makes it easier for people living with hearing and speaking disabilities to communicate better with spoken words through audio speech – which is also great for the multitude that doesn’t know the first thing about sign language; a win for both sides, so to speak.

The glove, aptly called Sign-IO, converts sign language into audio speech after identifying several letters signed by sign language users and passing along the data to an Android application through which it is then vocalised. Voila!

Through flex sensors fitted in each finger compartment of the glove, the degree of bend to which a finger is subjected to in the process of signing a letter is characterised and quantified. These signals are then processed and sent via Bluetooth to a mobile application that is also developed by Roy. The app takes up the baton from there and makes out audio speech from the signed letters.

As Roy says; “My niece wears the gloves, pairs them to her phone or mine, then starts signing, and I’m able to understand what she’s saying. Like all sign language users, she’s very good at lip reading, so she doesn’t need me to sign back.”

Roy’s project went through preliminary trials at a special needs school in Migori County in Kenya’s south-western region. It was from the results of those trials that he obtained valuable data which helped him work on one of the most important aspects of the gloves – the speed at which sign language is converted into audio.

Just as it is with spoken language, people speak at different speeds and require varying lengths of time to put together their thoughts into sign language. Hence, it was imperative that this gets incorporated into the mobile application so that just about anyone could use it with ease.

Through the app that goes with the glove, users are allowed to set preferences in terms of gender, language, and voice pitch. Roy also claims that the gloves translation accuracy currently stands at an impressive 93 percent.

True to his fondness for kids, Roy Allela has also designed the gloves in a number of style variants – from ‘Spiderman-themed’ gloves to ‘Disney Princess-esque’ ones – giving kids something to be excited about and putting paid to the stigma and difficulties associated with hearing and speech disabilities.

For his efforts, Roy Allela has been the recipient of numerous awards and nominations. At the 2017 Innovation Showcase (ISHOW) competition organized by the American Society of Mechanical Engineers (ASME), he took home the prestigious Hardware Trailblazer award. The prize money that came with the recognition helped him upgrade on the prototype and develop gloves that give more accurate translations.

Roy Allela is also one of 16 young African innovators drawn out from six countries that were nominated recently for The Royal Academy of Engineering Africa Prize for Inventors. Clinching the grand prize will see him go home with up to EUR 25 K to support his project.

Having made his bones at the University of Nairobi where he won numerous awards before earning a degree in Microprocessor Technology and Instrumentation back in 2016, he also successfully completed a number of Udacity courses in the year that followed, with stints at Microsoft, Soko Store, and Emobilis Academy sandwiched in between.

A 2018 fellow of the Royal Academy of Engineering, Roy Allela currently juggles working at Intel Corporation with tutoring at Oxford University for courses on Data Science and IoT. He also hopes to place at least two pairs of the Sign-IO in every special needs school in Kenya. Well, guess it’s true what they say – superheroes do not always wear capes, some only have gloves on.

 

Feature image courtesy: nairobinewsnation.co.ke