Bookbot

Bookbot is an app that uses speech recognition to help children learn how to read and improve their reading skills.

Bookbot’s mission is to disrupt the world of literacy with a learn-to-read App that helps children become proficient readers. Our technology tailors to the needs of users with learning disabilities. We are working with the Indonesian government to overcome the educational barriers created by COVID-19 for Indonesia’s 29.4m primary school children.

Bookbot is compatible with Android/iOS mobiles and tablets and can operate offline after download. The App listens to the reader using speech recognition and highlights the lines in the book as the user reads. Incorrect works are emphasized for further practice and users can hear the words pronounced correctly through a natural speech synthesis.

Our technology is backed up by a scientifically proven, systematic learn-to-read language program made up of specially-written books ranging from beginner through to efficient readers.

This combination of advanced technology with a scientifically designed learning program allows Bookbot to offer a unique reading and speaking environment without the need to have a parent or teacher present, and takes users from the very early stages of reading right through to becoming confident, independent readers and speakers.

In 2021, we were awarded a contract to build Bahasa Indonesia into the Bookbot App in collaboration with the Indonesian government. The ambition is to provide an accessible solution, particularly during COVID-19, to make a significant impact on improved literacy outcomes of Indonesia’s 29.4m primary school children, with a focus on those that have a learning disability (~2.5m).

There are a few root causes for the problems that we are solving. These problems are deeply rooted in Indonesia, were caused by various reasons, and thus require extensive, gradual work and support to be able to solve them. 

Firstly, the pandemic and the subsequent closure of schools hindered over 29.4m school-aged Indonesian children from attending in-person classes. A survey conducted by Statistics Indonesia (BPS) Magelang found that over 50% of students deemed online-based schools to be ineffective, with only less than 6% of students agreeing that they are effective. Obstacles such as lack of infrastructure, stable network connection, and unfamiliarity with technology are only a few of the struggles that students in Indonesia are currently facing, especially those in remote regions.

Secondly, Indonesia is ranked as the 60th most literate nation out of 61 nations, in a study conducted by Central Connecticut State University in 2016. This declining trend is shown to be caused by the limited accessibility to books, as well as unattractive, overly formal, and lecture-like topics found in Indonesian books. Likewise, poor literacy, access to technology, and the infrastructure of libraries are factors affecting the low reading interest in Indonesia.

With over 29.4 million primary aged Indonesian children affected by these difficulties, it is undoubted that the scale of this problem is massive.

The aforementioned factors including the absence of in-person classes, poor network connection, under-developed infrastructure, low reading interests, and poor literacy skills relate strongly to the solution that we present and are currently developing.

The Bookbot application mainly aims to serve and assist school-aged children in Indonesia’s public education system. These children are often hindered from gaining access to books due to varying reasons and hence have one of the world’s lowest literacy skills and reading interests. Furthermore, with the pandemic and the school closures that ensued, this condition became even worse as students do not have access to in-person classes nor school libraries.

To deepen our understanding of their needs, we have taken multiple paths and considered the different perspectives. For instance, we have consulted the help of curriculum and cultural specialists in Indonesia to better understand the needs and problems of Indonesian children. Likewise, developing content in the form of books that are truly interesting for children to read is at the forefront of our endeavor. In practice, we do so by conducting research that dives into understanding the likes and interests of Indonesian children throughout the many regions in Indonesia. 

Moreover, as we develop the Indonesia-specialized Bookbot app, we are constantly testing the app with Indonesian children and gaining feedback from their experience in using the app. We believe that through these methods of evidence-based research, direct user testing, and consultation with experts, we will be able to provide engaging books for children to read at home, whilst gradually building their reading interests and improving their literacy skills.

The Challenge presents us with the current status quo of Indonesian children with limited access to quality education, low proficiency and literacy skills, and school closures due to the pandemic aggravating the situation. 

Bookbot offers a unique, affordable, and accessible solution that provides the closest thing to one-on-one teaching while enabling children to learn independently, with minimal internet connection required. 

The app’s real-time, on-device voice recognition technology listens to the child read out loud and provides feedback through pronunciation modelling. It also produces reports that allow teachers to monitor the progress and improvement of the child’s reading level.

Bookbot has hundreds of specially designed phonics books which accelerate reading skills. Utilising the power of OpenAI, Bookbot will add thousands of books that target a child’s specific interests and reading level.

Within the comfort of their home, Indonesian children will thus be able to learn how to read and enjoy contents that truly cater to the various likes and interests found across the vast Indonesian archipelago. 

At Bookbot, we believe that learning to read is a foundational skill that enables a child to learn independently for the rest of their life. With that motivation in mind, we provide equitable access to various reading materials that are open for all children regardless of their demographic backgrounds.

Pepe Morales

The speech recognition model that we leveraged in the Bookbot app is one of the lightest and fastest open-source models that there is. The app is thus able to give immediate feedback to the reader in real-time as the reader pronounces word-per-word. Likewise, with the book-levelling system that we implemented in the Bookbot library, children would not only be able to learn at a pace that adapts to their current reading ability, but also learn independently in a self-guided manner.

Aside from our personalized library, we are also utilizing the latest machine learning language model, namely the OpenAI GPT-3, to create even more personalized content that our readers would enjoy. By doing so, we are delivering content that specializes uniquely in the various likes and interests of Indonesian children.

We expect that this trend of applying machine learning models for content personalization would become popular over time. As Indonesia's population becomes increasingly diverse in preferences, we believe that the type of content creators have to develop will similarly cater to the varying characteristics of users.

We have tested our solution's approach with a small group of Bookbot app testers in Indonesia. Their children used the app and displayed really positive engagement. With the feedback we have received from this initial testing, we will keep adding the number of testers as our product upgrades over the next couple of months.

We have been using the Microsoft Azure suite of products to develop the Bookbot AI technology including:

1. Indonesian children’s speech recognition acoustic model using the NCv3-series VM to significantly speed up processing and to continue our experiments. We have trained a model of 100 hours of speech and will have 1000 hours of audio recordings and transcripts of children reading books in the next few months.

2. Azure Neural Text to Speech (TTS) – Bookbot uses neural voices in different languages used for pronouncing words and syllables, reading books, and presenting lessons. The Azure TTS is natural and friendly, helping create a connection between readers and the robot characters they interact with.

3. Azure Personalizer – Bookbot will create a personalised book library based on each child’s reading preferences.

Bookbot is generating two Machine Learning models for the children’s reading speech recognition - an acoustic and a language model.

The acoustic model is trained from recordings of Indonesian children in different regions with different accents. The audio includes a variety of readers - from readers that are fluent to new readers who speak with elongated syllables and gaps of silence.

The language model is trained from the text in the library of books. We don’t use a large lexicon to ensure the language model is as efficient and small as possible to work on lower-end devices.

Activities

Children download the application and find books that they like. Children can then have Bookbot read the books to them or they can read the books themselves and have Bookbot listen to them and provide feedback on their reading. As children read more books and improve their reading skills, Bookbot advances them to the next reading level which unlocks new books with more advanced content.

Outputs

Children can learn how words are pronounced correctly by having Bookbot read the books and words to them. Children read a set of books that have been suited to their reading ability and Bookbot provides feedback on their reading, including prompts to practice the words that were pronounced incorrectly. As children read more books and improve their reading skills, Bookbot advances them to the next reading level which unlocks new books with more advanced content.

Short Term Outcomes

The child would start to develop a habit of reading books by practicing through the Bookbot app consistently and start reading even more books available in the Bookbot library. They would gradually move up the book-levelling system as their reading ability improves over time.

Long Term Outcomes

The child would be able to comfortably read any books or texts that may be relevant to their field of study and interests. This would allow them to excel as life-long learners with/without the guidance of a teacher. Essentially, they learn to read before they read to learn.

NA

NA

NA

We already have a market-ready English version of the Bookbot app, and that app has helped various children with visual impairments and/or dyslexia in terms of their reading ability. The Bookbot app gave them the confidence to read and has made a difference in their respective learning journey. For instance, we can see our users progress in terms of their reading ability as they gradually move up the book-leveling system provided within the Bookbot library -- indicating that there are improvements in their reading fluency and accuracy.

Furthermore, there are various methods of gauging the progress that our app is making towards the impact goal that we plan to achieve. Aside from the number of app installations, we also measure the technical capability of our app. For instance, we are constantly making improvements in our speech recognition model, measured by its word error rate (WER). Likewise, the number of high-quality curated, leveling-based books are also ways that we track our progress. We have just recently hired curriculum specialists, authors, producers, and editors to ensure that the educational side of the app is constantly making progress.

Our near-term goal within the next few years would be to release a stable and working Indonesian Bookbot app on a large-scale, across various provinces in Indonesia. This includes an efficient and accurate speech recognition model that runs on various operating systems and low-powered devices, to be included in the app. The distribution of our app would be facilitated through our collaboration with Indonesia's Ministry of Education and Culture, whereby students in national schools would have access to a desktop Windows Bookbot app. In practice, this would mean that students across Indonesia in public schools would thus become active Bookbot users on a regular basis.

On the other hand, our long-term goals would include the setting up of an Indonesian Bookbot subsidiary. By doing so, we would be able to get in closer touch with the local Indonesian community and hence better understand their needs. The English Bookbot app can similarly be leveraged for Indonesian ESL (English as a Second Language) speakers. We are currently already in the process of establishing the subsidiary.

NA

At the moment, our barriers are mainly related to technical problems. Firstly, we are facing barriers in terms of the internet connection bandwidth and storage of our users’ devices. We would want our app to still run offline -- which it is capable of doing -- but our users may have connectivity issues, or even lack an internet connection to make the initial connection to download the core resources.

Furthermore, we are seeking to solve storage issues when it comes to delivering our app’s content to our user, noting that there are assets like images and audio for each book that needs to be stored locally. We think that we can solve this with the help of machine learning algorithms to compress and de-compress assets to smaller storage sizes while still retaining good performance.

When our founder’s son was diagnosed with Dyslexia and was falling further and further behind in school, he decided to put his software creation skills to work and came up with Bookbot. The app is loaded with books written for young children with reading difficulties and allows them to learn how to read independently. 

Adrian's son is living proof that the Bookbot app works, as his son won reading awards in school. Witnessing that success story, Adrian believes that he can bring the same positive impact for children in Indonesia through the expansion of Bookbot in Indonesia.

Founder and CTO, Adrian DeWitts, works full-time, with 1 full-time developer and 4 to 6 part-time employees focused on machine learning, app & content development, and design. Pepe Morales recently joined as CEO and works full-time, supported by non-executive Chairman, David de Campo, and Director of the board, Frank Cooper.

Adrian’s 20+ years of experience in software development, coupled with his knowledge of phonetic-based reading/teaching approaches, provide him with the unique skill required to create and continuously develop Bookbot’s AI-powered speech recognition technology as well as its specially designed phonics books. Pepe’s broad experience managing and investing in businesses across the globe provides him with the expertise and skills needed to successfully grow and expand the business, particularly in emerging markets. In the case of Indonesia, we have a team of local experts and close connections to Indonesia’s Ministry of Education and Culture, uniquely positioning us to distribute our app at scale there.

Our Team Lead is constantly seeking to incorporate the latest technological features into both the Bookbot app and the content creation process that happens behind the scene. For instance, with the recent advancements of machine learning algorithms, our Team Lead ideated the utilization of models for the automation of redundant tasks, as well as as a creative boost. As an example, we leveraged the latest GPT-3 model from OpenAI to help the creation of non-fiction books based on lengthy Wikipedia articles, in a language that is intended for young readers.

Aside from automation through machine learning algorithms, we are also in the process of creating an automated illustration image curation pipeline. We believe that many redundant tasks can be automated and these are only a few examples of how our Team Lead is able to conceptualize new ideas to speed up content creation that is meaningful and attractive.

Key Indonesian government partner: Innovation for Indonesia’s School Children (INOVASI), a program established by the governments of Australia and Indonesia that aims to accelerate improved student learning outcomes for Indonesian children. 

Through INOVASI, we are working with the following government stakeholders: Ministry of Education and Culture that provides guidance with Indonesia’s curriculum, Ministry of Religious Affairs that ensuring accordance with Indonesia’s religious guidelines, and National Development Planning Ministry (Bappenas) that will assist during the rollout phase.

Key Indonesian Disability Partners: Asosiasi Disleksia Indonesia (ADI), Wahana Inklusif Indonesia Foundation (WII), and Circle of Imagine Society (CIS) Timor. 

Key Australian Disability Partners: University of Melbourne / CBM - Nossal Institute Partnership for Disability Inclusive Development, Australian Dyslexia Association (ADA), and Remarkable.

Beyond the funding provided, we expect that a partnership with Octava and MIT solve will help us:

• Consolidate our strategy to establish a sustainable and impact-driven business model for the SE Asian ESL market.

• Leverage the Octava/MIT Solve network to understand and penetrate the SE Asian ESL market as well as to improve our product from a technical perspective, as mentioned on the barriers in previous questions.

• Lay the groundwork to secure further funding and grow our business at scale to accomplish our long term ambitions.

• Consolidate our strategy to establish a sustainable and impact-driven business model for the SE Asian ESL market.

• Leverage the Octava/MIT Solve network to understand and penetrate the SE Asian ESL market as well as to improve our product from a technical perspective, as mentioned on the barriers in previous questions.

• Lay the groundwork to secure further funding and grow our business at scale to accomplish our long term ambitions.