RePad

An application that uses a camera to turn any writing surface into a graphics tablet.

Our solution consists of mobile and desktop programs. It uses a smartphone's camera and a piece of paper (or, in the general case, any camera with any surface you can write on) to get an affordable graphics tablets experience, whenever, wherever. 

To use our app, you need to connect your phone to your computer and point your phone's camera on paper. You move your pen on the paper - the app moves the mouse on your screen; it is that easy.

Now you are ready to write! Camera video stream is sent to PC and then processed by our algorithms to capture your handwriting in real-time.

The app takes the best of your computer to operate at an extremely high speed - video is processed faster than your camera can capture it! The program, executing most of the work on video-card, runs smoothly even on old computers. We designed a unique algorithm specific to this problem. Not only does it allow us to make all before mentioned optimization, but it also increases the algorithm's precision.

When the pandemic started, no one expected it to last this long and change education so much.

Due to quarantine, many schools and universities went online,  and education quality has fallen drastically. Over 4 million scholars in Ukraine and more than 1.1 billion students worldwide were impacted by online studying.

Lack of teacher's experience of conducting online classes has led to intolerable results - in Ukraine, for example, 31% of high school graduates have failed the final math exam, and this number seems to grow as the pandemic goes on. 

An additional factor is that most schools cannot afford proper equipment for every teacher, especially students. This issue also applies to many tutors and those who need a tablet for personal use. Some educators find it inconvenient to use ordinary graphics tablets because of the "real paper" or "desk" feeling that tablets don't give, especially those that are more affordable and don’t have a display. In addition, teachers often need students to write and explain their solutions, like near the usual desk.

Some need a graphic tablet for one-time use, such as online signatures, job interviews, and conferences. It is simply an inefficient usage of money to buy graphics tablets for such cases. Sometimes people urgently need a graphics tablet, and they can’t wait for the delivery.

In general, the problem we are solving is the high worldwide demand for affordable, quickly-available, and convenient graphics tablets for education and, in addition, personal usage, online signatures, or one-time purpose.

Our application will mainly serve educators and students. Most of them cannot afford expensive graphics tablets. The majority of countries cannot provide their schools with enough online-studying equipment for everyone. Teachers are showing their paper or desk with cameras, which leads to another problem: the quality of the image gets so bad that it is impossible to read anything. With our application, this will no longer be a problem.

More than 1.1 billion people worldwide were involved in online studying. As a result, the quality of online lessons and students' performance significantly suffer.

Our application will help these people because it will be cheaper than any possible hardware solution. Having default equipment to join online lessons, such as a laptop or smartphone, is enough to substitute graphic tablets.

Teachers can explain topics more clearly, and students can interact and become a part of the learning process - something online lessons currently lack. Using our application, students can explain topics they are trying to understand to other students - it is a well-known way of learning and the most efficient one.

Our team started developing a program when we encountered the problem by ourselves. Each of us has conducted free Physics and Math courses, and we needed a way to transfer our explanations from paper to screen without graphics tablets. Our research began with primary data from ourselves, as those who taught online for quite a long time. 

Nevertheless, we interviewed teachers from our and other schools to understand as many perspectives as possible. We started with Math teachers who heavily need a graphic tablet to explain geometry and ended with Chinese teachers who need to write hieroglyphs. We collaborated very closely with some teachers, and they told us what features would be helpful for them. In addition, we created a survey for teachers and gathered even more feedback about the problem and solution. According to a survey, almost 80% of teachers have trouble conducting online lessons compared to offline. More than 70% responded that students learn offline better than online. One of the reasons for this is poor equipment. Teachers who stream their paper sheet or blackboard with the camera on average, gave a 2.7/5 rating to the quality of the image.

Our program is the first of its kind, so we haven't met anyone working on the same problem. The only existing analog doesn't match teachers' needs. It requires a dark room, a long set-up time, and the work is only seen on a computer screen and not on the paper itself, making it intuitively harder to use.

The Prototype stage best fits our current stage of development. We are in the process of developing and testing our product, fixing any bugs that appear. Our application already handles some simple user cases, but we still need to make the range as wide as possible. We also did primary market research to understand the teachers’ needs.

Firstly, our app uses the open computer vision library to take a camera video stream. Various filters and detectors with tracking techniques are applied to find and track the position of the writing surface in real-time. The program detects a paper, desk, or other writing surfaces and makes it possible to fix the camera at any angle, so that set up would be much quicker. Then every video frame is processed, using acceleration from the GPU, by our algorithm, which we developed for this application. Then the algorithm's result is sent to the CPU, and the program moves the mouse. We also use a technique that accelerates communication between CPU and GPU.

All that results in a moving cursor that precisely repeats lines on the paper in such a short time that it would be impossible for another person to see the delay. Every tact is performed faster than an average camera captures a frame.

Another technology is built video compression algorithms used by many online meeting platforms. The result of our program is a both noiseless and almost monochromatic image on the user's screen. In addition, the only part of the video that changes is the mouse cursor, which is a tiny amount of the screen area. The compression algorithms work much better on these kinds of video, so usage of our app will be very tolerant of users' internet speed. In addition, using google or zoom board with our technology saves internet.

We are in the prototype stage at the moment, and we are in the process of developing and testing our product. In Ukraine, where we want to start to operate first, there are around 400’000 teachers and 4 million students. We want to serve 10’000 customers for the first year. There are 1.1 billion educators and students around the globe, and almost all of them were affected by online studying. We hope to serve 50 million customers around the globe in 5 years. We believe that our solution will not only directly and meaningfully help all of its users, but also will change the lives of many students who will learn from teachers that will use our app.

In other fields, such as online signatures and online one-time meetings, we want to reach at least 500-1000 users per month in the first year.

Our main goal is to make the AltPad experience as close to the graphics tablets one as possible. The mission is to provide it at a much lower price and also keep all paper's advantages. To achieve this, we implement solely software solutions that run on phones and PCs.

1. First of all, we want to make our app available for everyone by improving the algorithm, to reach 98-99% accuracy on all use cases, by the end of this year.

2. We will destroy a barrier for students, who want to be tutors or science bloggers, but can't because of expensive equipment. We plan to have 100-500 of this type of users for the next year.

3. Another goal is to establish partnerships with 500 schools in the next year, so instead of getting one smartboard for the entire school, they'll buy a personal "smartboard" for every student and teacher.

4. Sponsor some Ukrainian competitions for scholars, like “Levenia”, “Kenguru”, and others. By doing so, we share our technology with those who need it and also introduce our program to a bigger audience.

5. Transform our solution to an only-phone approach, making it possible to access our technology even if the user doesn’t have a computer.

Our long-term goal is to completely replace usual graphics tablets in all spheres with our more affordable technology.

We are now at the prototype stage, so we are improving our algorithm's precision and app performance. We are constantly checking how it works in different conditions so that users would need less than 30 seconds to set up the system. Once it will work in almost all user cases, we will measure the time per frame and accuracy.

1. We will also measure the number of new users every month 

2. the number of users who subscribe after a free trial 

3. the lifetime of each account. 

4. percent of users that came after "word of mouth" from others

In this way, we will test our hypotheses regarding pricing, marketing, and our product.

After getting to the pilot stage, we will use users reviews to check the quality of an app, using the net promoter score.

Financial:

Since we are students, our own funding may not be enough for some expenses. 

We need money for marketing, hiring additional experts for future research and development, and scaling to other cities, countries. 

Technical:

A barrier that may limit our impact is the algorithm's precision. Even a small deviation may be enough to ruin handwriting in some cases. We've developed our technology to solve this problem, but it hasn't reached its full potential yet.

In addition, camera quality may affect precision. Some users may experience inconvenience; nevertheless, the average phone's camera should work.

Cultural:

While most languages use Latin-like alphabets, Chinese, Japanese, Arabic, and others have different calligraphies. We need additional testing of how our algorithm performs in such cases.

Legal: 

Most schools are public, meaning that selling subscriptions to schools can become a long bureaucracy. Government structures often require tons of documents and are less willing to sign new contracts.

For online signatures, we need to go through approving our technology to sign documents safely.

Market:

It will be initially challenging to raise awareness of our product, especially among teachers.

We have worked together on different programming and robotics projects for 5 years. All of us are interested in programming, mathematics, and physics, and we have a total programming experience of 19 years. 

We've met the problem with online lessons with the start of the pandemic - online lessons were terrible due to lack of teachers’ experience. Time has passed, but the technologies haven't improved. Also, we volunteered as teachers for younger students and conducted online courses in physics/mathematics. We needed to display our solutions as well as ask students to show their ideas with formulas and diagrams. Sometimes the student, who got the right idea to solve a problem, couldn't explain it due to a lack of equipment. We used different techniques to solve this problem without using side hardware but all of them didn’t work well enough. This is where we got [Name's] idea - and immediately started solving it and exploring how it meets users’ demand. It took us a few months to develop and test different algorithms ideas.

Artur Zhdan is an LPML graduate and a student at Keystone Academy 2023, Beijing, studying programming and business management. He has 6 years of experience in developing for Android, Desktop, IOS, Web, teaching ML, and is a winner of STEM competitions at the international level, including EuPhO 2021.

Vitalii Holopolosov is an LPML graduate, with more than 7 years of programming experience in C/C++, Python, Java, Pascal, glsl. He is a winner of many regional-level programming competitions, international-level physics competitions, and has won a bronze medal in EuPhO in 2021.

Makar Kuznietsov is a physics faculty student at Taras Shevchenko National University of Kyiv. He has 6 years experience in programming. He is a silver medalist of the IPhO and winner of various country level STEM contests.

While we plan to serve mostly educators and students, in Ukraine, most of the teachers are female (almost 85%, while there is a 47%/53% ratio of males to females). Our technology will improve life for teachers since they won’t need to spend almost monthly salary on a graphic tablet, considering that teachers’ salary in Ukraine is especially low (170-350USD), lower than a minimum (an average family in Ukraine spends 350USD per month). Thus, improving the teachers’ life, we will improve the lives of many women in Ukraine, expecting women to be almost 80-85% of our customers. We believe that the problem we are trying to solve is yet unmet because we are the only ones trying to make education cheaper instead of investing money in it, therefore helping educators all around the world, and not only a few we will help personally.

We want to use The Pozen Social Innovation Prize to increase the time of free trial, hire full-time staff for marketing, financial accounting, and additional developers to develop/update and support a web page and the application.