Sign language glove

Transcription

H...E...L...L...O... ...hello. You seem reassured to hear me pronounce these words. I sensed unease among some... ...caused by not understanding what Lina and I were saying. How many here can understand what Lina is saying? Okay wow... not a single person understood. And yet, Lina is fluent in her first language, the Quebec Sign Language. We are in Montreal, Quebec, and yet no one here understands her. I'd like to thank Lina for being with us today and for translating for those who need it. Let us investigate why Lina is not succeeding at being understood here. To identify what isn't working, let's investigate effective communication. In order to speak to you, I first conceive my message... ...I formulate an idea... ...and then I create my message. As this is a francophone conference, I am using French. My words spread through the air in this room and reach your ears. ...which are not as large as these stick figures' ears... ...but which are functional nonetheless. So then, you interpret my message as per your mental map: your culture, communication skills, etc. ...and you understand my message. When the hearing-impaired speak, they use more or less the same process... ...except that instead of using voice, they use an array of signs constituting their language. Contrary to what we may believe, the message does reach us... ...we do see Lina sending her message... ...but we don't understand because we can't decode it. We can't decrypt the information. So for a hearing or speech impaired person to be able to communicate with us... ...they must resort to an interpreter. This intepreter must be present as needed, and paid for their service. You can imagine how this reduces the autonomy of an impaired person. What we present today is a complementary option to the use of an interpreter... ...insofar as technology will allow us to develop it. We are striving to increase the autonomy of people like Lina... ...and enable them to fully embark on the train of progress. My colleague Lucas will now show you how this can be done. What you see on my right hand is the proof of concept we created. It's the first version of a sensorial glove that can translate sign language. In my left hand is the version we're working on. As you can see, it's much sexier... ...and this version may also be wireless. So the wires hanging here would no longer be present. I'd like to show you how such a project might materialize itself... ...and be accessible to the general public. As we observe the signs Lina makes to translate my message... ...we notice she is mostly using her upper body. As she mainly uses her hands to communicate, working with gloves makes good sense. Her gestures are precise and fluid... ...so we must position the hand in space, and also capture the shapes it is making. This requires many sensors, which must be affordable for the product to be accessible. They must also be tiny enough to be woven into fabric... ...in order for the gloves to be comfortable to wear. This naturally leads us to working with optical fibers. Optical fibers are tiny transparent threads that transmit light from one end to the other... ...much like a water hose does with water. If we weave optical fibers throughout the glove... ...while positioning a light sensor on a phalanx and a light detector on the host... ...or a light sensor on a source, pardon me. As we bend one finger, the optical fiber moves away from its light source. This loss of light intensity is perceived by the optical sensor. As this loss of intensity is directly linked to precise movement... ...consequently we can track the hand's shape at all times. Furthermore, to track the full gestures we must be able to position the hand in space. To do this, we use the same chips that are found in smart phones. These small chips change the screen display orientation on your devices. The chip identifies the screen as vertically or horizontally oriented... ...and changes it according to movement. We use a similar system, although more sophisticated. We can both position the hand in space and track its orientation at all times. Consequently, Lina's complete, precise and fluid gestures can be captured... ...and sent to a computer. The computer executes a word for word translation of sign language. Today's demonstration uses only one glove to give you an idea of how the concept would work. This is our proof of concept glove. As you can see, we still have work to do... ...but we can nevertheless realize an efficient translation. I will translate Lina's opening words so that they may now be understood. You are seeing the computer program on the screen. To have... ...idea... ...plenty... ...to have... ...real... ...profitable... ...exchange. What we intended to express was... "I have ideas that are worth sharing." As you can see, the translation is quite different from our intent. This isn't due to a software error. Our software works great, by the way. The problem is that sign language is its own language... ...and its semantics are very different from those of the French language. This example illustrates the challenge of translating sign language. The only way to succeed will be through collaboration. We will have to collaborate with interpreters like Lina... ...with speech or hearing impaired individuals... ...with engineers, computer scientists, linguists... ...in order to bring this technology to maturity. Let's imagine what could arise from such a collaboration. A system that is complete would be able to contextualize content... ...to recognize each sign, and to compose complete sentences... ...which would appear on screen or be read by an artificial voice. Thus technology would successfully lend a voice to the speech or hearing impaired. Furthermore, individuals could teach their device a personalized language. Personalizing their communication by teaching it their own words and signs. And if the technology is really accessible and inexpensive... ...then people all over the world will have access to it... ....and be able to teach their device their own way of speaking. By connecting all systems online, we could create a huge library of sign language. This way, the hearing impaired could understand each other all across the world. Sign language would be enriched by contributions from everywhere. Research can push this even further. We could create translation algorithms for many other languages. We can imagine doing so for all languages on the planet. The hearing impaired would communicate not only among themselves... ...but also with everyone else. If technology can go this far... ...the hearing impaired may not only fill the gap that separates them from other languages... ...they may in fact break all language barriers. What excites us most with this technology is its impact on daily life. Imagine being a hearing or speech impaired person sitting at your desk... ...with no access to an interpreter's service. This is most frequently the case. With access to this technology, you would open your drawer, put on your gloves... ...and make yourself understood. This truly could increase your autonomy. This technology has the potential to change people's lives... ...not only at work, but in their leisure time too. I will leave you with this exercise: next time you get on a plane to go on a trip... ...imagine you are impaired, and sign language being your only language. You will then fully appreciate the potential value of these gloves. Thank you for your attention.