Edugaze AI Assistive tool

The sollution team consists of reasearchers and professors that are a part of the Innovation center of the School of Electrical Engineering, University of Belgrade and Universiy of Singidunum.

"Edugaze AI Assistive tool" is designed to help individuals with reading and learning difficulties (with proven use cases on dyslexia). During the reading/learning process, Edugaze will use image recognition algorithms to estimate eye-tracking data based on web camera recordings and to detect how much attention was placed on which parts of the presented material. Based on this, the sections of the presented material will be divided into categories based on how much time was spent analyzing each part of the text/image. These conclusions will then be used to create automated prompts for a pre-trained large language model, that will then develop questions regarding the parts of the text that are of interest. The process is divided into two steps, testing and learning with personalized feedback. The testing will be based on educational material data that was estimated to be skipped or that took too much time for the child to cover, as well as the data that is estimated to be properly understood by the child. Considering the child’s mental health and emotional response, the solution will offer a difficulty setting that will enable the parent/caregiver/educator to set the ratio between the questions that are posed regarding the materials that were estimated to be difficult/easy for the child based on the eye tracking, to provide a more reassuring or challenging learning environment. The solution will also offer an implementation that does not rely on web camera recordings and eye-tracking for situations where web camera footage is not accessible and will provide assessments at the level of each presented page based on the time spent, scaled by the amount of content on each page. After each question, an answer will be provided based on the original educational material and will create a real-time quiz environment. This will enable a revision of the understanding of the materials that will be suited to each child individually and will create a more personalized learning process.

The "Edugaze AI Assistive tool" is a revolutionary tool specifically designed to support individuals with reading and learning difficulties, particularly those with dyslexia. Using a pre-trained large language model, the tool generates personalized questions related to the areas of interest, creating an interactive learning experience. The process involves two main steps: testing and learning with personalized feedback. After completing the initial step, learners will have the opportunity to customize their learning environment on all devices, adjusting features such as background color, font, and overlay to suit their preferences. This personalized setup, coupled with the feedback provided, will aid in the early diagnosis of learning and reading difficulties, enabling timely intervention and prevention strategies. By addressing these challenges early on, learners are less likely to experience motivational and emotional problems, fostering a more positive and effective learning experience. Edugaze adapts to the learner's needs by adjusting the difficulty setting, ensuring a tailored and supportive learning environment. Additionally, the solution offers an alternative implementation for situations where web camera footage is not available, providing assessments at the level of each presented page.

Regarding its impact on priority Pre-K-8 learners and their educators, Edugaze has the potential to significantly benefit these groups through its AI-enabled assessment capabilities. Priority learners, including Black and Latino students and those from low-income backgrounds, often face significant barriers to academic success. By providing personalized support and feedback tailored to individual learning needs, Edugaze can help bridge the gap and ensure equitable access to quality education. For educators, the tool offers valuable insights into each student's learning process, enabling them to customize instruction and support strategies accordingly. This targeted approach promotes inclusivity and fosters a supportive learning environment where all students can thrive.

Four years ago, our journey began with groundbreaking research into the effects of colored backgrounds and overlays on reading, particularly for children with dyslexia. Our work, which resulted in a PhD thesis at Jozef Stefan International School and collaboration with three additional faculties in multidisciplinary research, demonstrated the potential of colors to improve reading abilities, especially on an individual level. Utilizing biosignal measurements from sensors, we analyzed users' emotional feedback, difficulties, and performance during reading in both regular and color-enhanced environments. Our findings have been presented in prestigious articles and conferences, highlighting the significance of our research. In the past two years, we've developed algorithms for early dyslexia screening using eye-tracking measures. EDUGAZE brings this cutting-edge technology to educators and professors, empowering them to assess students' reading performance, conduct early detection, and provide tailored exercises based on suitable color-text background settings. Our diverse team comprises university professors, Innovation Center employees, and startup co-founders, drawing on extensive experience in psychology, attention, language, neural sciences, and entrepreneurship. EDUGAZE promises to make a significant impact on children, teachers, and parents, addressing various challenges in the learning and reading processes. Endorsed by prominent figures such as Prom Milica Jankovic, Chief of the Department of Biosignals Processing at the School of Electrical Engineering, Prof. Dr. Vanja Kovic, Chief of the Department for Neurocognitive Studies at the Faculty of Philosophy in Belgrade, Dr. Ivan Vajs, Chief at the Innovation Center at the School of Electrical Engineering, and Doc. Dr. Tamara Papic, Assistant Professor at the University Singidunum and co-founder of the health tech startup BabY FM, EDUGAZE stands at the forefront of innovative solutions for dyslexia and reading difficulties [Ref1, Ref2, Ref3, Ref4, Ref5, Ref6]

We've crafted algorithms for early reading difficulty detection and attention tracking. Our research includes various learning-reading environments for K-8 learners, exploring color, background, and overlays' impact on reading performance, assessed through sensor hub data (eye tracking, EDA, EEG, ECG). Previous studies involved over 100 scholars (K-8 age), alongside 10 teachers, 5 speech specialists, and psychologists. Moving forward, we aim to develop the product from prototype to implementation, conducting further tests for insights. This involves training algorithms, testing eye-tracking setups on multiple web cameras, and enabling environment adjustments across devices. Ultimately, our AI assistive tool will dynamically support learners in achieving better performance.

Edugaze leverages cutting-edge technology, including image recognition and AI-driven analysis, to provide personalized assistance tailored to each learner's needs. By combining eye-tracking data with advanced algorithms, Edugaze identifies attention patterns and adapts learning materials accordingly. This approach not only facilitates early detection of reading challenges but also empowers educators to tailor instruction effectively. Moreover, Edugaze's user-friendly interface and accessibility features make it suitable for diverse learners and environments. By catalyzing innovation in learning support, Edugaze has the potential to transform the educational landscape, promoting inclusivity and equitable access to quality education for all learners.

The technology will integrate applied psychology principles and methods with artificial intelligence, web, and mobile platforms, as well as signal processing techniques. Applied psychology insights will guide thorough platform evaluations and facilitate intuitive, user-friendly designs. Cutting-edge tools like Google's mediapipe will enhance video processing capabilities to extract high-quality gaze data under various conditions. Signal processing will refine gaze data to generate robust features with clear quality indicators, ensuring usability even with low-quality recordings. These features will fuel machine learning algorithms for personalized assistance, the pretrained large language models will enable personalized knowledge assessment while the web and mobile app platforms will prioritize accessibility, intuitive presentation of results, and user-friendly text modification options.

The implemented concept are validated through a multitude of scientific papers published by the authors:

ajs, I., Ković, V., Papić, T., Savić, A. M., & Janković, M. M. (2022). Spatiotemporal eye-tracking feature set for improved recognition of dyslexic reading patterns in children. Sensors, 22(13), 4900.

Vajs, I., Papić, T., Ković, V., Savić, A. M., & Janković, M. M. (2023). Accessible dyslexia detection with real-time reading feedback through robust interpretable eye-tracking features. Brain Sciences, 13(3), 405.

Jakovljević, T., Janković, M. M., Savić, A. M., Soldatović, I., Čolić, G., Jakulin, T. J., ... & Ković, V. (2021). The relation between physiological parameters and colour modifications in text background and overlay during reading in children with and without dyslexia. Brain sciences, 11(5), 539.

The implemented algorithms will be based on pre-trained algorithms that consider algorithmic bias such as Google's media pipe, and OpenAI's Chat GPT models. The feature extraction and processing that are developed by the team are based more on signal processing rather then complex datasets and AI algorithms and do not allow for bias or discrimination. 

2 full time

3 part time

1 contractor

We already have proven partnerships with schools and universities in Belgrade which provide a solid foundation for aligning with the standards and needs of K-8 learners in the USA. These partnerships offer invaluable opportunities for testing and refining our solution in diverse educational environments, ensuring its efficacy and relevance across different contexts. 

To ensure that our solution, the "Edugaze AI Assistive tool," is pilot-ready within the next year, we have a comprehensive plan in place. Firstly, we will finalize the development of our AI algorithms for early dyslexia detection and personalized learning support. This involves conducting rigorous testing and validation to ensure accuracy and effectiveness. Additionally, we will refine our user interface to optimize accessibility and ease of use for both learners and educators.

To track our progress toward pilot readiness, we will monitor key milestones such as algorithm development, user interface enhancements, and partnership agreements. Additionally, we will conduct regular evaluations and usability tests with educators and learners to gather feedback and measure the impact of our solution.

We are committed to keeping the cost of our Edugaze AI Assistive tool affordable for schools and educational institutions serving priority learners. This includes offering flexible pricing options, such as subscription models or volume discounts, to accommodate different budgets or nonprofit models with the support of a strategic partner in the USA. Additionally, we will actively seek partnerships with government agencies, non-profit organizations, and philanthropic foundations to secure funding and support for implementing Edugaze in schools serving Black, Latino, and economically disadvantaged students. We will also prioritize user-friendly design and accessibility features to ensure that the solution is easy to use for students with diverse learning needs. Finally, we will leverage technology to scale our solution efficiently, allowing us to reach a larger number of priority learners across the United States. Through these strategies, we aim to make Edugaze widely available and accessible to all priority learners, helping to address educational inequities and improve learning outcomes for underserved communities.

We are eager to contribute to Solve and align with the values of the Bill & Melinda Gates Foundation. Our established scientific research, validated by academia, forms a strong foundation for developing solutions that can positively impact millions of children. However, to realize this vision, we require both technical and financial support. With the assistance of Solve and the Foundation, we aim to make our project accessible to children, ensuring that the reading and learning process is barrier-free. Together, we can empower learners and break down the obstacles to education.