Echonoma

Using Artificial Intelligence and Machine Learning to bridge the accessibility and communication barrier between the Deaf, Blind, and hearing communities.

Veezaviz is an assistive technology social enterprise developing leading-edge technology by leveraging the power of AI to address the communication and accessibility barrier for the Deaf community. We are committed to creating new generation innovative technologies that enable two-way communication between Deaf, blind and hearing through automated sign language translation and production technology. 

We have developed Echonoma - an AI-based application software with the aim of ensuring the inclusion of the deaf in their immediate environments without the need for a sign language interpreter. We are reimagining a world where Deaf patients can naturally communicate using sign language to health practitioners and hearing individuals without the need of them understanding sign language.

Our solution is bringing AI-powered assistive technologies and clinically-focused tools to telehealth and telecommunication in a way that can’t be found anywhere else. Deaf patients won't have to worry about disclosing confidential information to a third party, or rather minimize omitting important details which is common with sign language interpreters during the translation process.

Over 5% of the world’s population, 466 million people, have hearing disabilities. (WHO, 2020) and more than 200,000+ in Kenya alone. It is estimated by 2050, 1 in every 10 people will have hearing disabilities. Individuals from this vulnerable group are unable to express what they want or even their basic human needs. This ultimately creates a communication barrier excluding them from society, deprived of critical information and efficient service delivery.

Over the last 24 months, the pandemic has put enormous pressure on health systems, highlighting the need for innovative healthcare models to assist in connecting health services to individuals, particularly those who face hurdles to access. Although telehealth offers a wide range of applications, the Deaf are left behind.

Due to language barriers, Deaf people don't have access to clear communication which deprives them critical health information and qualified healthcare. Studies show that Deaf patients make more trips to the emergency room and less frequent visits to their primary care provider. Due to this inequity, they are at a higher risk for adverse health outcomes as compared to non-deaf individuals. Although doctors have the best interests of their patients, they are not always appropriately educated to become culturally responsive practitioners which can lead to misdiagnosis because of unfamiliarity with treating Deaf patients.

Our beneficiaries are the Deaf community.  Other stakeholders are the  hearing community, caregivers, families, service stations, corporate clients ,hospitals and other service providers. There are over 150,000 Deaf people in Kenya, according to the 2019 Kenya Population and Housing Census Report. Note: If you ever find census data, you should always view it as a ‘minimum’ estimate in countries where stigma may prevent people from voluntarily reporting their types of disabilities.

Nearly 80% of people with disabling hearing loss live in low- and middle-income countries. The prevalence of hearing loss increases with age, among those older than 60 years, over 25% are affected by disabling hearing loss. Having such a large population unable to be socially participatory is not only a violation of their rights as humans, but a major impediment in the development of countries. The unavailability of sufficient policies that cater for the needs of the deaf risks  excluding them from the overall economic workforce of a country.

A major challenge with accessing service from hospitals is that staff and health practitioners don't understand sign language. Majority of deaf people can't access sign language interpreters. The cost of sign language interpretation is very expensive and most households in Kenya can't afford to pay for the service. 

Veezaviz has been taking steps directly involving end users of our technology, and relevant stakeholders ensuring that they are at the heart of the product development. The user development activities provided us with knowledge that aided us in addressing this communication and accessibility barrier for the Deaf community. The goal for this user-centered design approach, and engaging with persons with disability throughout product development.

Some of the insights we picked from these interactions include:Communication barriers are the number one reason Deaf people have had poorer health compared to hearing people. They are fully aware of this problem. The hearing community does not have assistive technology tools to effectively integrate them into their livesCommon visual communication modes include Kenyan Sign Language, speech reading (lip reading), and gestures. Most participants identified communication barriers as the biggest challenge they faced. Many hearing aids in the market are not as effective as they should be. they also complained about the quality of hearing aids on the market

When we took part in Innovate Now Assistive Tech Accelerator, with the support of Kilimanjaro Blind Trust Africa, we were able to interact with diverse Deaf communities in Nairobi. The feedback and data we gathered from users was extremely valuable and greatly informed the next phases of our product development.

We started creating this platform early 2021 and have grown from idea to a minimum viable product. We collected diverse data, built Kenya Sign Language datasets from the Deaf community, trained several machine learning models, and built a software that accurately interprets the Kenyan sign language alphabets in real-time over a webcam.

We have conducted 4 phases of in person user development and research with the Deaf community in Nairobi between the months of June and September. We conducted multiple tests and feedback sessions engaging 150+ Deaf individuals as we tested our MVP focusing on improving the design of the software without losing the core functionalities that this user group has come to love. We established the testing and market needs of Veezaviz and undertook in person research activities for our innovation which include interviews, focus group discussions and user testing sessions.

In the last few months, we had several product demos and discussions with stakeholders in the disability sector; The National Council for Persons With Disabilities (NCPWD). We talked over potential collaborations with NCPWD and we  are looking forward to partnering with them towards rolling out our product this year.

Echonoma is leveraging the power of Machine Learning, specifically, Computer Vision, and 3D animation enabling two way communication between the deaf and the hearing/blind with no need of the hearing/blind understanding sign language.

Our solution is purely oriented towards the use of deep learning in the augmentation of human receptive skills to software. Our model is trained on YOLO object detection method based on Convolutional Neural Network. (CNN), which is accurate and fast.

Echonoma is powered by AI and neural networks and using your camera feed, it  translates sign language to text or speech in real-time at the comfort of a device.  It is also accompanied with an animation tool for production of sign language and learning purposes. For this case, text is parsed to map text to corresponding sign language visualized using a 3D avatar. We have a powerful Artificial Intelligence communication tool, it has been designed as a complete solution for the worldwide Deaf community. 

Veezaviz is operating in Kenya for now but we look forward to expanding and getting our solution out there for members of the Deaf Community in the African continent, then later scale globally. 

Veezaviz has been taking steps directly involving 150+ end users of our technology, ensuring that they are at the heart of the product development. We established the testing and market needs of Veezaviz and undertook in-person user development activities.  Through this, we ended up creating a community.

We have conducted a user research session for feedback on the application’s impact. Having initiated the development process, we have also engaged them to further test the product to make it more Deaf optimized.
Later in the year, we organized an Innovation boot camp specifically which upskilled 85+ Deaf Youth in changemaking and STEM.

We are iterating version 4 further to accommodate conversations as we approach deployment to market in the next 3 months targeting to benefit 5,000+ members of the Deaf community. 

Our goals are to:

• Make our software available to the deaf and the general public.

• To partner with organizations globally and assist them to become more inclusive to deaf community.

• To promote equitable access to life through engaging  young Deaf people as problem solvers and help them become independent.  

We want the society to adopt our solution, work as partners, collaborate with stakeholders and help unlock accessibility for the deaf community in all sectors.

Currently we are starting with the Kenyan Sign Language, then in the first quarter of 2022 grow our presece in Kenya. We scale to the American Sign Language. This will be easier to do because the structure of the whole model will be the same. Later we will target African sign language dialects then other areas of the world.

Throughout our building processes, we have and still are involving the Deaf people as part of the solution on contract based activities. The Deaf community covers 90% of the people we are directly working with on data collection, data validation and analysis, annotation and compilations of datasets in preparation for the ML model training. And it's the Deaf people who validate and test the performance of the trained computer vision model.

Our measurement initiatives are typically developed by directly involving Deaf users, ie medical practitioners, hospital staff, educational institutions , service providers etc.

In 2021, we have directly involved the Deaf community in Nairobi and its environs where they have been collecting sign language data as they continuously assess the accuracy of our healthcare solution ensuring our measures are meaningful, and the data is accurate. They have cited that this will be an important communication tool for accessing their communication needs which cut across all the basic human needs.The accuracy of our AI interpretation platform and its performance is the outcome.

Here our telecommunication platform and assistive technology enabling solution performance will be measured through the quality and clarity of communication, cost of access ,the level of privacy, efficiency of patient care and outcomes of healthcare consultations, evaluations ,diagnosis, treatment to prescription and even follow-ups with Deaf patients.

We will also continuously track the number Deaf patients who successfully access healthcare, education and services through our platform as we gauge their health knowledge.

We created Echonoma while envisioning breaking significant societal barriers to help accelerate a world with more equality. A world where the Deaf and hard of hearing community can access healthcare services, education material, employment opportunities, the job market, social avenues and be able to be engaged as active participants at their local and national levels.

Huge amount of research and development costs - , especially data collection which is the backbone behind our technology ( There is no publicly available Kenyan sign language dataset ). With the high resource intensive nature of our innovation, training Machine Learning models also requires expensive computer hardware and a lot of data collection on Kenya Sign Language that we have to create a dataset infrastructure. We have adjusted to using fewer data but train longer to get better results.  We plan to leverage on partnerships with different stakeholders in the disability space to gather data cheaply and also partner to fast pace this activity. 

Integrating the trained  Machine Learning model into the application has been a challenge and outsourcing talent is very expensive. We had to learn how to maneuver around this ourselves.

Two years ago, we represented Kenya in the annual First Global Robotics. After we came back we were doing STEM & robotics empowerment training in different regions in Kenya. Here, we had our first repeated interactions with Deaf students in these sessions. The only mode of communication we opted to use as we didn't understand sign language was the pen and paper method. These multiple occurrences exposed this communication barrier between Deaf and hearing communities and it was an “ aha “ moment.

Our motivation comes from directly witnessing the struggles of deaf and hard of hearing (hoh) children, youth and generally people with disabilities struggling to succeed in being able to be part of the healthcare systems, education, labor market and social inclusion

Our team consists of a multi -talented, dynamic, diverse group of technology and accessibility ecosystem thinkers and practitioners.  Over the past 4 years , we have evidenced strong skills in assistive technology , engineering , robotics, software development , health tech and user-centered product design

We Are Family Foundation.

We Are Family Foundation created the Youth To The Front Fund to support and fund under 30-year-old youth-led organizations, and innovations that are at the forefront of fighting systemic racism, inequality, inequity and injustice in the United States and around the world.

Innovate Now

Innovate Now Assistive Technology Accelerator accelerated and assisted Veezaviz to test the initial version of Echonoma with the Deaf community in Nairobi and its environs.

Villgro Africa

Villgro Africa shortlisted Veezaviz for acceleration in the Kenya Healthcare Innovations Programme. Later in 2021, Echonoma was selected “ Top 11 AI Innovations in Africa '' by  Artificial Intelligence 4 Africa - Harnessing the Power of AI for Africa’s Development.

Unicef Kenya, Generation Unlimited

UNICEF is supporting the implementation of Generation Unlimited (GenU) vision of creating better education, skills, employment and entrepreneurial opportunities for young people in Kenya.

Imagine a world where the Deaf community can communicate seamlessly using sign language over video, what impact would this have on telehealth and health based video consultations between Deaf and blind patients with medical practitioners. With the growing adoption of telehealth especially over the pandemic, we believe this is an opportunity to accelerate quality health care and eliminate ineequities in digital healthcare systems.

As far as Assistive Technology and disability inclusion in healthcare is concerned, we are on a mission of accelerating health equity. The impact of our work at Veezaviz addresses critical development issues and challenges facing Africa and the entire worldwide Deaf community.

The Pozen Social Innovation Prize will enable Veezaviz to further develop accurate, more robust sign language interpretation machine learning models, reliable technology and transcend borders with other countries, eliminating the barrier in multiple sign language dialects. The possibilities are endless and we are excited to pioneer the future of assistive technology and healthcare.