Smart Paper

What is Smart Paper

Smart Paper is an AI-enabled assessment tool that provides adaptive paper-based formative assessments with instant grading and feedback. Our platform will generate contextualized printable worksheets tailored to each student's knowledge level and cultural background, making adaptive assessments equitably accessible regardless of digital access. Our AI will assist educators in grading open-ended responses on paper assessments, giving them deeper insights into the learners.

How Smart Paper Works

Educators can create paper-based assessments on our web application and print and share them in the classroom. When the smart paper sheets are scanned, we use handwriting recognition and LLMs to automatically grade the written responses and provide personalized feedback on every item for every learner. After some assessment data is available for the students, Smart Paper will be able to generate adaptive printable worksheets based on the previous data. Smart Paper worksheets can be completed offline in various settings such as classrooms, homes, or any other scenarios. Once students complete the worksheets, they can be uploaded to the Smart Paper app by anyone for immediate feedback. The work on paper becomes available to all stakeholders, bringing clarity about student learning.

Leveraging the Efficacy of Formative Paper-Based Learning

Research shows that writing on paper enhances math and language learning. Smart Paper leverages this benefit of paper and combines it with adaptive learning. We describe this approach of paper-based adaptive formative learning in our paper "Equitable Access to Intelligent Tutoring Systems through Paper-Digital Integration" (ITS 2022). Handwritten work captured through paper clearly demonstrates students' thinking processes, allowing AI algorithms to deeply understand and accurately predict learner capabilities. Working on paper has proven to be effective and free of distractions, making it an ideal medium for formative assessments.

Technologies Used

Smart Paper utilizes a range of AI and machine learning technologies to connect paper and digital learning. Computer vision digitizes paper worksheets, while large language models enable automated scoring and feedback. Knowledge Component Models are utilized to create student models. These models generate adaptive printables for learners based on their individual needs and context. LLMs contextualize question items when generating adaptive printables.

Adoption in India

Smart Paper has been adopted at a significant scale in India’s Rajasthan state, with 235,000 teachers using it to scan paper worksheets of 4.6 million students. To date, Smart Paper has digitized nearly 100 million paper worksheets in India, collecting over 1 billion learning data points in just two years. The state of Andhra Pradesh in India also piloted Smart Paper in April 2024 in 1200 schools for digitizing paper assessments of over 100,000 students.

Recognition

Smart Paper has been recognized for its potential and impact in various forums. It was the winner in the Assessment track of the 2021 Learning Engineering Tools Competition (sponsored by Schmidt Futures). Recently, UNESCO recognized us for innovating AI for Assessments. The Confederation of Indian Industries also recognized our AI innovation for its impact in India.

Smart Paper Product Demo: https://www.youtube.com/watch?v=wEh05AwTzE4 

Our website: www.SmartPaperApp.com

How Smart Paper will impact the lives of Priority PreK-8 Learners:

Authentic & Predictive Assessments, Immediate Effective Feedback

Paper-based assessments provide more authentic data, as handwriting work clearly shows the learner's thinking process. The misconceptions identified through handwriting are more specific and can help AI language models provide targeted feedback to learners. For teachers, understanding the key misconceptions in their classroom that need remediation becomes easier and more rapid. Free response items in Smart Paper assessments will provide highly detailed insights into each learner, compared to multiple-choice items, enabling more predictive and actionable data for educators.

Leveraging the Benefits of Pencil & Paper

Smart Paper leverages the universal medium of paper, which is highly effective for learning, especially in K-5. Handwriting is essential to building higher academic and cognitive skills, and several studies have shown the effectiveness of reading and writing on paper. Unlike digital devices, paper is distraction-free and scientifically proven to be an effective medium for thinking and problem-solving. The 2023 PISA Global Education Monitoring Report found that mere proximity to a mobile device can distract students and negatively impact learning in 14 countries. Smart Paper delivers the benefits of adaptive formative assessments via paper, ensuring that handwriting-based learning activities target each learner's specific needs.

Ensuring Equitable Access

By utilizing paper, Smart Paper ensures accessibility for all schools, regardless of their location or resources. Digital learning devices and apps are not always available to learners from disadvantaged communities, and rural areas in the United States still face connectivity issues. The type of digital devices accessible to classrooms also varies. Paper is a readily available tool that reaches the last mile, making Smart Paper an equitable solution for delivering the benefits of formative assessments. Our research paper, "Equitable Access to Intelligent Tutoring Systems through Paper-Digital Integration," published at the 2021 Intelligent Tutoring System conference, describes how we can deliver personalized learning on paper for equitable impact.

Transparency for All Stakeholders, Clarity on Learner Progress

The handwriting data collected through Smart Paper assessments offers a transparent view of each learner's progress, misconceptions, and areas for improvement. This clarity enables families to deeply engage in children’s learning at home. The digital portfolio of written work over time clearly shows the journey of a child’s development, enabling educators to make informed decisions about remediation and personalized instruction.

Our Team's Prior Work in EdTech

Smart Paper is led by Derek Lomas and Nirmal Patel, who have a combined 30+ years of experience in cognitive science, product design, learning engineering, and artificial intelligence. Our organization, Playpower Labs, has collaborated on over 50 projects with large education organizations such as Pearson, Carnegie Learning, Harvard, CMU, and the Government of India. Our technology teams regularly work on popular edtech platforms that reach nearly 4 million students in the US. Through our partnerships, we have gained a deep understanding of the challenges faced by educators and learners and how technology can be leveraged to address on-ground needs.

Smart Paper's Proven Impact

We have successfully piloted Smart Paper with over 10 organizations across the US, India, and Japan, demonstrating how paper-digital integration can transform the assessment experience in classrooms. Our solution has been scaled to reach 4.6 million students in India (Read the case study by the Dell Foundation).

Over 100 million paper worksheets have been digitized by Smart Paper in India over the last two years. We are supporting arguably one of the largest assessment data digitization programs in the world, creating one of the biggest longitudinal educational datasets with the least digital infrastructure requirements. This showcases our ability to apply artificial intelligence at scale to solve real problems in assessments.

Our Scientific Rigor

Our team is driven by scientific evidence. We have co-authored over 75 research papers. We have published award-winning papers at venues like ACM CHI, Intelligent Tutoring Systems, and IEEE. We have won several grants and awards for our digital learning research in the US, including a grant from the McArthur Foundation and awards from the White House and the NYC Department of Education. We were one of the winners of the 2019 NAEP Educational Data Mining Challenge, and our educational data visualization work was recognized in the EM:IP DataViz Challenge, highlighting our capability to work with curriculum and assessment data.

Our prior experience and our understanding of the communities we serve position us well to design and deliver a solution that addresses the diverse needs of learners and educators in the US.

Smart Paper is operating in over 65,000 public schools in India's Rajasthan state, with nearly 100 million papers scanned to date. A recent pilot in another Indian state (Andhra Pradesh) with 100,000 students showcased the Smart Paper model’s replicability.

The personalized printable component, tailoring assessments to learners' interests and cultural context, is currently at the prototype stage. We are eager to rapidly develop this feature as part of the Solve challenge and improve the assessment experience for all Pre-K to Grade 8 learners in the US, particularly priority learners.

Focus on Paper-based Assessments

Our solution takes an innovative approach to AI-enabled assessments by focusing on paper-based rather than digital assessments. While most assessment innovations today are happening in the digital space, we recognize the importance and effectiveness of reading and writing on paper, especially for early learners. By leveraging AI to enhance paper-based assessments, we offer a novel, scalable solution that supports handwriting-based formative learning while avoiding the potential harms of digital distractions for young students.

Adaptive Print Technology

Central to our innovation is the concept of Adaptive Print, which delivers the power of personalized learning in offline environments. This approach makes personalized learning accessible to all learners, regardless of their access to high-tech devices or internet connectivity. Our research paper “Equitable Access to Intelligent Tutoring Systems Through Paper-Digital Integration” in the 2021 Intelligent Tutoring Systems conference describes this approach in detail. This innovation is particularly useful for delivering adaptive learning in marginalized communities that often lack digital access.

Scoring Open-Ended Responses on Paper

Most of the paper scoring technologies are focused on multiple choice and other objective assessments. Smart Paper is designed to support multiple types of paper-based assignments, enabling educators to score open-ended responses using the latest large language model technology.

Authentic Assessments

With the availability of GenAI tools, some forms of digital assessments have become less reliable as they don’t always show the true picture of student learning. This becomes trouble for educators. To get around this, many educators are now reverting back to paper-based assessments, which can be administered easily and have much higher reliability in terms of outcomes.

Smart Paper leverages a combination of AI subdomains, including Machine Learning, Deep Learning, Computer Vision, and Natural Language Processing.

Smart Paper uses adaptive learning algorithms that utilize Knowledge Component Models to generate personalized paper worksheets tailored to each learner's context and skill level. For each learner, their needs are identified and formative assessment items are selected for them. These items are optionally contextualized using a Large Language Model to translate them into more engaging and contextual and culturally relevant versions.

Smart Paper's proprietary page processing engine is at the heart of its paper assessment digitization capabilities. It uses classical image processing algorithms for standardization and correction of scanned page photos. Each page component is then processed using the appropriate AI model and processing parameters set in the system.

For handwriting recognition, we have developed models that use Convolutional Neural Networks (CNNs) and Sequence Models (Transformers/LSTMs). CNNs are used for detecting small markers (ticks, crosses, characters), while a combination of CNNs and Transformers enables reading inputs of arbitrary lengths, such as variable length numbers.

Automated scoring and feedback of digitized responses are powered by GPT-4 and Claude 3 models. The AI is provided with the item information, digitized answer, and rubrics for scoring. User ratings of the scoring quality are used to fine-tune open-source LLMs for improved reliability. Our team continuously spot-checks the AI scoring and creates datasets for ongoing fine-tuning.

Smart Paper's web application is accessible on any device via browser, while the native Android app enables faster paper scanning and provides some offline functionality. The AI Engine API, running on a scalable cloud platform, can process up to 25,000 scans per minute.

Smart Paper has achieved significant scale and impact in India, demonstrating the effectiveness of our paper digitization technology in the context of Pre-K-8 learning assessments. Our solution has been scaled to support nearly 5 million learners, from early pilots with thousands of learners, proving its robustness even in challenging conditions with limited digital access. Our AI has achieved 95%+ accuracy in automated grading, showcasing its ability to operate effectively in classrooms across diverse geographies. Read the Dell Foundation case study for key outcomes of our work in India. The assessment program that we support in India also created a world record for the number of assessments completed and digitized within a short timespan, where ~13 million worksheets were scanned in five days.

The importance of paper in educational settings has been examined through various studies, such as:

1. The benefits of using pencil and paper in math (2020). Reboot Foundation.

2. Paper Notebooks vs. Mobile Devices: Brain Activation Differences During Memory Retrieval (2021). Frontiers in Behavioral Neuroscience.

3. The effects of writing on learning in science, social studies, and mathematics: A meta-analysis (2020). Review of Educational Research.

4. Parent-toddler social reciprocity during reading from electronic tablets vs print books (2019). JAMA pediatrics, 173(11), 1076-1083.

5. Comparing memory for handwriting versus typing (2009). In Proceedings of the Human Factors and Ergonomics Society Annual Meeting 53(22), 1744-1747.

These studies underscore the continued relevance and effectiveness of paper-based learning, which our Adaptive Print technology enhances by incorporating personalization based on student data. By combining the proven benefits of paper with the power of AI-driven personalization, Smart Paper is well-positioned to improve the efficiency and utility of Pre-K-8 assessments for learners and educators. Adaptive Print makes personalized learning accessible to all learners, regardless of their access to high-tech devices or internet connectivity. This technology has the potential to transform the landscape of paper-based learning resources, ensuring they remain effective and relevant in an increasingly data-driven world.

Monitoring & Avoiding Bias in Automated Scoring

We are committed to ensuring our AI-powered automated scoring is fair and unbiased. We employ a continuous monitoring process where our team regularly spot-checks the AI scoring results against human scoring. This allows us to identify any potential biases or inconsistencies in the AI's performance across different student demographics.

We also conduct small-scale comparative studies between AI and human scoring to validate the accuracy and fairness of our automated scoring engine. In one of the studies we did for Grade 6 Science, we found GPT-4 had ~90% correlation with Human graders. The insights gained from these QA processes and studies are used to create targeted fine-tuning datasets that help improve the AI models' reliability and mitigate biases.

Ensuring Equitable Access

Smart Paper's use of paper as the primary medium for formative assessments ensures that our solution is accessible to all students, regardless of their access to digital devices or internet connectivity. By digitizing paper-based assessments, we bridge the gap between offline and online learning, enabling schools to collect comprehensive learning data and support all students. 

Our Adaptive Print technology will further enhance equity by generating personalized paper-based formative assessments tailored to each learner's needs and context. This ensures that every student, regardless of their digital access, background, or learning pace, receives targeted support to help them grow. The ability to scale Smart Paper to any geography, even in low-resource settings, makes it a powerful tool for promoting educational equity on a global scale.

Our team actively seeks input and feedback from educators, parents, and students from underserved communities to better understand their unique challenges and incorporate their perspectives into our solution's development.

A team of five people works full-time on the project. One full-time project lead, two AI and ML engineers, and two software developers are creating the Smart Paper AI platform.

Smart Paper is well-positioned to pilot our solution with priority Pre-K-8 learners in the US within the next 12 months. Our technology is already operating at scale, digitizing paper assessments for over 4.6 million students in India. We have a proven track record of deploying our solution in low-resource settings and adapting it to local contexts.

To prepare for a US pilot, we will take the following steps:

1. Engage with US educators, parents, and students from underserved communities to gather input on their specific needs and challenges.
2. Develop the Adaptive Print technology to generate personalized paper assessments tailored to each learner's needs, interests, and cultural background.
3. Conduct small-scale user testing with a diverse group of US students and educators to validate the effectiveness of our personalized printables and gather feedback for refinement.
4. Establish partnerships with school districts serving high proportions of priority learners, particularly Black and Latino students and those experiencing poverty.
5. Train participating educators on using Smart Paper and interpreting the insights generated from digitized assessments.

We aim to launch Smart Paper pilots in multiple US classrooms within 1 year, reaching at least 1,000 priority Pre-K-8 learners. Our experience scaling Smart Paper in India gives us confidence in our ability to meet this goal and positively impact educational equity in the US.

Smart Paper is committed to making our AI-powered adaptive paper assessments available, accessible, and affordable for all priority Pre-K-8 learners in the US public education system, with a focus on Black and Latino students and those experiencing poverty.

For accessibility, the use of paper makes Smart Paper inherently accessible to all learners, regardless of digital access. Personalized printables will be generated in students' preferred languages.

To keep costs low at scale, we plan to price Smart Paper affordably for school districts, with volume-based discounts to support district-wide adoption. We will pursue partnerships with education nonprofits and apply for grants to subsidize costs for under-resourced districts. Our efficient AI technology and focus on leveraging existing school resources (paper, printers) enables us to offer Smart Paper sustainably at a price point accessible to all schools.

We are applying to this challenge because we believe Smart Paper has immense potential to transform assessment and advance educational equity for priority Pre-K-8 learners in the US.

Specifically, we hope Solve and the Gates Foundation can help us:

1. Gain insights into the unique needs and challenges of Black, Latino, and low-income students in the US public education system to inform the customization of Smart Paper.
2. Connect with US education experts, school districts, and nonprofits serving priority learners to establish partnerships for piloting and scaling our solution.
3. Access funding to enhance our Adaptive Print technology, enabling the generation of culturally relevant personalized assessments at scale.
4. Navigate any regulatory or legal considerations around student data privacy and AI use in US classrooms.

The Challenge's focus on leveraging AI to improve assessment for priority learners aligns perfectly with Smart Paper's mission. With the support and expertise of the Solve community and the Gates Foundation, we believe we can meaningfully contribute to advancing educational equity in the US.