Parrot

Picture based Verbal Communication App for Children with Special Needs

The app helps children suffering with autism aiding them in communication through pictures. Children use those pictures to build vernacular sentences and convert those sentences from text to speech to communicate their thoughts as well as learn new things using everyday pictures. 

The app is currently built for Android phones only. It uses the power of inbuilt Google Translate to convert text to speech. For every image in this picture-based app, we are storing the image, the caption for the image in English, and recording the pronunciation of the name of the object. The text is automatically translated to other major Indian languages, namely, Hindi, Bengali, Gujarati, Kannada, Malayalam, Marathi, Tamil, Telugu and Urdu.

About 1 in 100 children in India under age 10 has autism, and nearly 1 in 8 has at least one neurodevelopmental condition. Children with Autism go through developmental delays which often impacts their ability to communicate. Since autism is a spectrum from mild to severe, some children may not speak at all. About 40% children with ASD (autism spectrum disorder) are nonverbal. Lack of ability to communicate leads to a significant impact in their lives. Hence Alternative and Augmentative Communication (AAC) apps are required to facilitate communication for these children. The need for an affordable and efficient AAC app was evident to us and hence we came with Parrot.

Our target population involves children with special needs who have at least one neurodevelopmental condition like attention-deficit/hyperactivity disorder, learning disabilities, intellectual disability, conduct disorders, cerebral palsy, and impairments in vision and hearing. 

The lack of ability to communicate leads to a significant impact in their lives since they aren't able to express themselves to other people without the assistance of another human or an object.

With the help of Parrot, we aim to bridge the gap of communication in their lives by providing a vernacular platform to communicate their thoughts, ideas and feelings to other people as well as go through a phase of holistic development for increasing their skills and learning new things everyday.

Showing the pilot app to multiple psychiatrists and doctors showed that there’s a real need for such an app. We need an indigenous and vernacular app since most children in India communicate with their family using regional language. Many existing apps and literature primarily use English as the mode of communication which becomes a hurdle for the children as they aren't accustomed to the English language in everyday speech. During the development of the app, we went through series of discussions with multiple psychiatrists with prototypes for improving the user experience for our target audience. Further, we went to multiple non-profit organizations for beta testing of the app to ensure that our developed functionalities are actually useful to the children.

We selected the Pilot stage as we have discussed about the app with doctors, teachers and students primarily with specific organizations like Actions for Autism (http://www.autism-india.org/). But we aim to go further to reach out to organizations targeting other neurodevelopmental conditions too to test our pilot app and ensure that its helpful for students of those organizations too. We have two psychiatrists in our advisory panel who constantly provide inputs to our designs & flow of the app and guide us to choose the best practices to follow while working on such issues.

Our solution is primarily a software based product which relies on usage of mobile phones and tablets for providing access to the platform. We have two apps, one for the children, while other one for their parents and instructors to monitor the performance of the child. We utilize cloud-based data management solutions for delivering content on the app and tracking the progress. As a future goal, we plan to leverage the power of machine learning tools to deliver metrics regarding child's growth and develop learning plans accordingly.

Since we launched recently and are still in the pilot phase, we haven't reached out to a large set of audience yet. Our app has onboarded 2 instructors and 10 children so far for as a beta set of users to ensure the functionality of the features of the app. By the next year, we plan to expand to a fairly large set of users involving more than 100 teachers and 1000 children and create a fruitful impact in their lives.

The primary impact goal of the product is to make it more personalized to the children to develop a connection of the child with the app using personalized speech synthesis, interaction-based learning games and game theory. We plan to include audio-visual learning tools as a medium for faster communication and increase the daily retention of the children so that they are able to grasp more information and learn about objects around them. At the same time, we plan to develop deeper metrics for the instructors to track the performance of the children and develop study plans for them accordingly.

We plan to measure the progress towards our impact goals by observing quality education and decent work and economic growth. All actions happening on the platform are measured in real-time and processed to see the potential points of failure in the app and are further used to make predictions for the lifecycle of future functionalities of the app. Using these metrics, we can accurately measure our progress towards the impact goals and make amends accordingly to meet them.

We currently face cultural and technical barriers in reaching out to children from different parts of India since vernacular content is fairly limited in current text and speech translation tools. Over the time, we plan to streamline our target audience to understand our demographic well and later start reaching out to children in different parts of India. We also face a market barrier in convincing organizations to change their existing learning tools and try an alternate form of communication which might be more suited to their needs.

Our team comes from a technical background with the common link of our engineering college, IIIT Delhi, India (https://www.iiitd.ac.in/). The team comprises of 3 undergraduate students and 1 professor of the institute. Our professor has been working with various organizations for the children and using human-computer interaction methods to aid development of the children in the non-government organizations. After observing skills taught at these organizations, we saw the gap of a vernacular picture based tool for the children with special needs and hence decided to build such product. The rest of the team excels in app development and user interface designing and have been a crucial part in the journey of this app.

We currently partner with psychiatrists and doctors for advising us in the development of the app by providing their valuable inputs from their years of experience in teaching children with special needs. We further collaborate with organizations specifically working on providing learning tools to children with Autism and deploy their experience and intellect in development of the application.

NA

NA