The Lightbulb asks innovators and entrepreneurs how they turned their ideas into reality. This week we talk to Arash Tayebi, co-founder of Kara Technologies which uses AI and digital humans to translate content into sign language.

First of all, give us your elevator pitch for Kara.

Kara translates different materials – books, audio, video – into sign language. We do this by using artificial intelligence (AI) and hyperrealistic avatars to provide accessibility to deaf people. 

What sort of background do you come from?

I’ve just finished my PhD in telecommunications at the University of Auckland, so my background is in engineering, specifically electrical engineering. I got the idea for Kara around two years ago in December 2017.

So what was it that sparked the idea for Kara? What was your lightbulb moment?

I have this disease called Ménière’s disease (an inner ear problem that affects your hearing and balance) and because of that I lost hearing in my left ear. This made me more interested in the [deaf] community so I started talking to deaf education centres. I realised we didn’t have that many deaf students in the engineering department and I wondered why that was. There’s no correlation between being deaf and being smart, so why were there no deaf students at the university? When I talked to the deaf education centres I found out there was a huge gap because we don’t have enough teachers who know sign language in New Zealand. That’s when I thought we could make virtual teachers who knew sign language and were available 24/7. That’s where it all began.

How does Kara translate books/audio/video into sign language?

Sign language is a complete language. It has its own grammar, it has its own vocabulary, and it has its own culture, so providing a seamless translation is very complicated. What we did is we broke it down into three levels. With level one translation we use human plus avatar. With level two translation we use less human and a bit of visual intelligence towards the avatar. And with level three translation we only AI plus avatar. At the moment, we’re at level two.

In what contexts will Kara be used? What use cases do you have in mind?

Recently we received some help from the NZSL (New Zealand Sign Language) board, so in a couple of months from now, we’re going to provide a translation of five books with the help of Kelston and van Asch deaf education centres. The books are going to be narrated with sign language and [you’ll be able to see] how efficient and effective it is to use technology to increase children’s knowledge. 

We want to try and get deaf children to learn by themselves. [It’s not meant] as a replacement for teachers, but as a complementary tool to help make them independent learners. We want to make it available New Zealand-wide for every single deaf child. 

There are already solutions for people with hearing difficulties in terms of subtitles for TV shows and movies, so what’s the benefit of having these materials conveyed via sign language rather than the written word?

Our focus is mainly on children and for children, reading can be difficult. Also, for some deaf people, English will be their second language and sign language will be their main language. So I think these materials could have more effect if they’re presented in sign language. Of course, subtitles are great, but I think a better solution is if we can coincide it with sign language translation since this is the natural language deaf people communicate with. 

What’s been your biggest challenge so far?

The thing that makes us different from animals is language, and language is the most complicated tool that we as humans have developed. So our current challenge is how can we make sure nothing is going to be ‘loose’ if we provide translations in a cost-effective manner? How can we ensure that? That’s one of the challenges we’re trying to solve… we want to make sure we’re really delivering what we’re claiming. 

After research and development, how are you planning to commercialise your product?

We’re a social startup/enterprise so we’re looking to generate some revenue. At the moment we’re figuring out what the best business model for us will look like.