Evidence-Based Writing: Civics in the Classroom

Quill believes that preparing young people for post-secondary success, be it college or career-training, is the surest way to end cycles of poverty and help students access the agency and voice to successfully advocate for themselves and their communities. 

Research indicates that the ability to write well is imperative to future success in college and the workforce; however, data from the National Center for Education Statistics shows that only 27% of seniors at public high schools score as proficient in writing. Furthermore, fewer than 15% of Black, Hispanic, and low income students score as proficient. With 73% of high school students lacking the skills they need to be successful after graduation, the need for improving the education around reading, writing, and critical thinking skills is one of the biggest hurdles low-income students face on their path to life-sustaining employment, post-secondary training, and/or higher education.

Quill.org offers resource strained teachers and schools an incredibly cost-effective solution (about $2.27 per student) that utilizes cutting-edge AI technology designed to improve low-income student's reading, writing, and critical thinking skills while saving valuable time and money. Our solution can be particularly powerful for students facing the incredible disparity in education quality and opportunities available to low-income students as compared to their higher-income peers - a gap that has only widened during the pandemic, especially for students of color. 

Quill.org’s free online tools help low-income students become strong writers and critical thinkers. All of our tools utilize AI to automatically grade and serve feedback on student writing, enabling students to revise their work and quickly improve their skills. 

Decades of research shows that the most effective way to build reading and writing skills is to do so across the curriculum.Our newly launched Quill Reading for Evidence tool is the first in a series of tools Quill is building to enable students to write across the curriculum. With support from the MIT Solve, Quill seeks to expand and strengthen our impact by embedding this tool in high-quality ‘Civics in the Classroom’ content. 

Quill Reading for Evidence is revolutionary in the field of educational technology, using artificial intelligence to provide students with immediate, real-time feedback on the evidence, logic, and syntax in their writing. To develop this tool, Quill had to learn how to develop, incorporate, and scale five different AI technologies developed by Google, Microsoft, and open source researchers. 

This R&D process required more than $4million in funding from philanthropic partners and resulted in more than 100 different algorithms created by Quill that target a range of issues in student writing, including vague or inaccurate evidence, opinionated language, plagiarism, logic issues, spelling mistakes, and grammar errors. Moving forward, Quill’s product engineering team has learned how to quickly evaluate and incorporate new AI technology, enabling our team to rapidly build new AI-powered learning tools.

We conducted an initial usability study with Mathematica (funded by the Bill and Melinda Gates Foundation which resulted in promising outcomes, including: 

• 89% of students were able to complete the activity in under 20 minutes, indicating that teachers will be able to use Quill Reading for Evidence as a supplementary tool. 

• 100% of students independently navigated the Quill Reading for Evidence interface, and students found it to be intuitive and user-friendly.

• 77% of all students' revisions directly aligned with the provided feedback. 

• 79% of students reported that Quill Reading for Evidence is useful. 

Development of ‘Civics in the Classroom’ content is already underway, including activities such as 'Should Food Sold in Schools be Regulated by Law', 'Should Schools Have Strict Dress Codes', and 'Should Schools Have Grade Requirements for Student Athletes'. Quill hopes to expand the current evidence-based writing content to offer a wider variety of activities that specifically addresses school life and policy issues that affect students. The long-term goal of this programming is to create not only strong writers and critical thinkers, but also students who are engaged in policy and can apply their passion to civics both inside and outside the classroom.

Quill’s nonprofit mission overall is to serve underserved students. Since Quill launched in September 2014, 5 million students have written and received feedback on more than 800 million sentences, with 3.35 million students attending low-income, title-1 eligible schools.

For Quill’s intervention, we define success as a student writing and receiving feedback on 100 sentence construction prompts during the course of a school year. By completing 100 practice prompts, students typically master five to ten Common Core writing standards. We recommend that teachers use Quill once or twice per week to practice key writing skills over the course of ten weeks, which takes approximately 150-200 minutes of classroom time. Of the 5 million students who have used Quill.org’s writing tools, 2.25 million students (45% of all students) have completed the full intervention of writing and receiving feedback on 100 sentence prompts.

For teachers to provide feedback on 800 million sentences to their students, they would need to spend at least 6.4 million hours grading writing assignments, which translates to $160 million dollars in time saved on grading (with an average teacher salary of $25 an hour). 

Within the next five years, we aim to enable 10 million low-income students per year to become stronger writers using Quill. Students will write and receive feedback on one billion sentences per year, saving teachers over 8 million hours and $200 million dollars in teacher time saved per year as a result of Quill’s powerful AI-based feedback system.  

As a new tool in the suite of Quill.org's free tools, Quill Reading for Evidence offers an incredibly cost effective and targeted solution to help teachers at under-resourced schools improve writing instruction and support low-income student's development of the reading, writing, and critical thinking skills imperative to future success. With a goal to build critical thinking skills focused on current events, 21st-century issues, STEM curriculum, and ELA topics, Quill is able to help build comprehension about the ideas students are reading about.  

Quill aims to continue the development of AI powered feedback to give students all the tools they need to be successful in their academic, and post-academic careers. Evidence-Based Writing, and more specifically, Civics in the Classroom content, will serve teachers by providing them structured, time efficient content. Additionally, Quill’s solution creates a grammar tool that helps students become analytical readers and writers who are able to succinctly respond to topics they come across, inside and outside of the classroom. 

As a tool designed specifically for low-income students, Quill.org has worked and continues to work closely with educators at low-income schools to co-develop our tools, including Quill Reading for Evidence. This includes a network of lab schools in New York City with a population of low-income students who are predominantly African-American and/or Latino, where we have collaborated with educators to design and test new materials and an advisory network of researchers who help us identify the most effective, research-based strategies for helping students. 

By engaging these schools as our testing partners, and paying these schools to support their costs, we ensure that our software is developed with the specific needs and challenges of low-income middle school students of color, enabling these teachers to be successful with our tools to bolster students' reading, writing, and critical thinking skills.

Additionally, Quill’s curriculum team continues to develop content for users, spanning from ELA, STEM, 21st Century issues, and current events. With a team continually focused on generating new activities, based on community-based research, Quill is able to generate lessons that remain relevant, applicable, and interesting for students.

Quill is positioned to scale, and we would appreciate peer guidance and expert advice as we launch our strategic plan to bring our tools to even more students.

Quill’s growth over the years has primarily been through word of mouth as teachers share our tools with other teachers; we do not have a marketing or brand development team, and our business development team consists of two employees. 

We have relied on the quality of our work - teachers are looking for high-quality, free tools, and when they find Quill, they often share it with others in their network. We measure how likely teachers are to promote Quill through our net promoter score, and our world-class +77 score shows that teachers are enthusiastic about using and sharing Quill.  To accelerate this process, we encourage teachers to share their experiences through blog posts, videos, and social media, and through these posts, we have reached tens of thousands of educators. 

As we continue to develop powerful new learning tools, and distribute those tools for free, a better understanding how to better promote our tools and bolstering our business development strategy would help us to reach even more students and educators. Our ability to scale our solutions - our growth and impact - will benefit immensely from this type of guidance. 

Quill is innovative because it changes the nature in which education technology approaches developing literacy skills and writing instruction. We have been successful at demonstrating how ed tech tools can be integrated into classroom practice to save teachers time, improve instructional practice, and better support students' individual learning needs. By creating content that is specifically focused on school life and policy issues that affect students, Quill will be able to expand our impact by helping students improve critical thinking skills and applying them to civics, both inside and outside the classroom. 

Educational technology frequently uses multiple-choice questions to help students develop literacy skills. With multiple-choice, students do not have the opportunity to construct their own ideas, and this limits engagement, understanding, and comprehension. The majority of AI-powered writing tools focus on algorithmically grading essays, and they do not focus on providing the sentence-level writing strategies that build foundational writing skills. To provide students with robust feedback for students, the Quill team maps out hundreds of potential answers for each item through a combination of human judgment and machine algorithms. By mapping out these responses, Quill can then automatically serve immediate feedback on dozens of strong answers and thousands of potential errors, including logic, grammar, spelling, and punctuation errors. 

Quill.org also uses research-validated writing strategies to develop its learning tools. In 2016, we launched Quill Connect, a tool that builds basic writing skills through sentence combining, a strategy promoted through the Carnegie Corporation’s seminal Writing Next report. The report states, “Sentence combining involves teaching students to construct more complex and sophisticated sentences through exercises in which two or more basic sentences are combined into a single sentence. Teaching adolescents how to write increasingly complex sentences in this way enhances the quality of their writing (Graham & Perin, 2007, p. 18).” Quill is the first organization to build an online tool which provides instant feedback on open-ended sentence combining exercises by using AI to evaluate student results. 

During the 2017-2018 school year, Quill partnered with the University of Nebraska-Lincoln to conduct a large randomized control trial study with 1,141 students from 10 schools across four states. Dr. Hebert, the principal investigator of the research study, is a co-author of the Carnegie Corporation’s seminal Writing To Read the report. Dr. Hebert administered the Test of Written Language as a pre-intervention and post-intervention assessment to measure student writing scores. The study found that students in the treatment group had a statistically significant gain in their writing skills, with students using Quill improving their writing skills at 1.5x the rate of students not using Quill. 

Over the next five years, Quill’s research & development team will build a series of five new tools that enable students to summarize evidence, develop thesis statements, and construct paragraphs. This sequence follows the pathway championed by Peg Tyre, a Quill.org board member, in a 2012 Atlantic piece titled The Writing Revolution. 

Over the next 5 years, Quill will take the highest-quality open source curricula available and will embed that curricula into Quill, supercharging curricula with our AI-powered writing tools so that students can construct their own ideas in writing and receive immediate feedback designed to build their writing and thinking skills. Through an open-source approach of offering robust curricula materials and learning tools for free to all students and teachers, we expect to significantly impact 10 million low-income students per year by 2027, with a series of randomized control trial studies validating that these students are advancing their writing and thinking skills through our tools. 

For teachers to successfully incorporate Quill Reading for Evidence into their classrooms, they need content that is aligned to their core curriculum. To reach Quill’s long-term goal of impacting 10 million low-income students per year, Quill will partner with the top-rated open source curriculum providers on the market. We will embed their free curriculum into Quill and supercharge it with our AI-powered writing tools. By partnering with the best curriculum providers on the market, rather than developing our own competing curriculum, we can reach millions of additional low-income students. 

In addition to Quill’s ELA content, a new focus is being placed on STEM, social studies, and civics. With this all-around approach to reading and writing comprehension, Quill users can have the greatest positive impact on improving skills. The goal of the Civics in the Classroom content is to utilize the developed skills and apply them to policy issues that have an affect on students in their school environment. Through practice and feedback, students will be able to improve their comprehension and writing skills to develop concise and astute feedback that they can use to express their opinions on complex and important issues. Alongside the writing improvements that Quill develops, the skill of thoughtful responses to civics issues is a lifelong tool that users will be equipped with.

• Number of students who have successfully practiced and improved on one writing skill. 
• Number of students who have successfully practiced and improved on one writing skill who attend low-income, Title-1 eligible schools. 
• Number of students who have successfully completed the full intervention by writing and receiving feedback on 100 sentences. 
• Students who have successfully completed the full intervention by writing and receiving feedback on 100 sentences who attend low-income, Title-1 eligible schools. 
• Teachers Net Promoter Score
• Cost Effectiveness of the Intervention 
• Pre- and Post-Intervention skills, including:
  • Whether students improve their writing skills faster than students not using Quill
• % of time that Quill’s feedback is comparable to a senior educator’s feedback 

• Whether students using Quill will more frequently include strong claims, evidence, and reasoning in their writing compared to students not using Quill based on a common rubric for evaluating writing. 

Students develop their writing skills on Quill by engaging in a cycle of feedback and revision. Once a student submits a sentence, the software serves a piece of feedback on a specific part of the sentence so that the student can revise his response. Quill’s feedback is designed to support the student while not revealing the correct answer, requiring the student to think about his work. Once the students revise their work, the system then serves another piece of feedback on another part of the sentence, and the student may revise his sentence up to five times in order to produce the strongest possible sentence. Through this feedback cycle, students progressively develop their skills. 

Quill addresses a number of syntactical issues, such as fragments, run-on sentences, adverb usage, adjective usage, compound sentences, complex sentences, parallel structure, and appositives. When a student is learning complex sentences, for example, the student would be tasked with joining together the following sentences: “The colonists thought the taxes were unfair. The colonists protested the taxes.” An emerging writer could write, “The colonists thought the taxes were unfair and they protested the taxes.” While using the word and is grammatically correct, the program differentiates the words and and since so that the student will specify the causal relationship between the two ideas, “Since the colonists thought the taxes were unfair, they protested them.” By teaching students how to show the causal link between the feeling of unfairness and the protesting, the student gain the ability to clearly express complex ideas in writing. 

As educational software, Quill Connect is an exceptional solution for teaching sentence combining because students receive immediate feedback on their writing. At the moment, traditional sentence combining instruction requires teachers to manually grade student work, a time-intensive process that prevents students from receiving sufficient feedback and quickly understanding their errors. 

Quill is the first organization to build an online tool that provides instant feedback on sentence combining exercises. To provide this feedback, the Quill team maps out hundreds of potential answers for each item through a combination of human judgment and artificial intelligence algorithms. By mapping out these responses, Quill can then automatically serve immediate feedback on dozens of strong answers and thousands of potential errors, including logic errors, grammar errors, spelling errors, and punctuation errors. As a web-based tool, this feedback is served to hundreds of thousands of students, at virtually no per-unit cost for each additional student served. 

Quill.org uses research-validated writing strategies to develop its learning tools. In 2016, we launched Quill Connect, a tool that builds basic writing skills through sentence combining, a strategy promoted through the Carnegie Corporation’s seminal Writing Nextreport. The report states, “Sentence combining involves teaching students to construct more complex and sophisticated sentences through exercises in which two or more basic sentences are combined into a single sentence. Teaching adolescents how to write increasingly complex sentences in this way enhances the quality of their writing (Graham & Perin, 2007, p. 18).” Quill is the first organization to build an online tool which provides instant feedback on open-ended sentence combining exercises by using AI to evaluate student results. 

Quill Reading for Evidence, our new reading and writing tool, is the most complex learning tool that Quill has built to date, and it is a revolutionary tool in the field of educational technology. This tool uses cutting-edge artificial intelligence to provide students with immediate, real-time feedback on the evidence, logic, and syntax in their writing. To develop this tool, Quill had to learn how to develop, incorporate, and scale five different AI technologies developed by Google, Microsoft, and open source researchers. During this learning and evaluation process, we evaluated dozens of different solutions - a time-intensive research and development process that required more than $4 million in funding provided by philanthropic partners. This R&D process resulted in more than 100 different algorithms created by Quill that target a range of issues in student writing, including vague or inaccurate evidence, opinionated language, plagiarism, logic issues, spelling mistakes, and grammar errors. Moving forward, Quill’s product engineering team has learned how to quickly evaluate and incorporate new AI technology, enabling our team to rapidly build new AI-powered learning tools.

As of 2012, there are 27 million US K-12 students experiencing poverty (defined as those attending Title-1 eligible schools) who struggle with basic writing skills, one of the biggest hurdles low-income students face on their path to life-sustaining employment, post-secondary training, and/or higher education. In order to get a job, gain entry into a technical program, or matriculate and succeed at college, students need to write emails, answer questions on digital applications, and compose compelling essays. 

Quill recognizes the incredible disparity in education quality and opportunities that are available to low-income students as compared to their higher-income peers. We also recognize that the African-American and Latinx young people are over-represented in populations with limited access, opportunity, and networks for upward mobility.  The gap for these students only widened during the pandemic. We believe that preparing young people for post-secondary success, be it college or career training, is the surest way to end cycles of poverty. Research indicates that the ability to write well is imperative to future success in college and the workforce.

That being the case, Quill’s mission is to help more than 10 million low-income students become strong writers and critical thinkers. By building free online tools that use AI to automatically evaluate student writing, Quill is able to provide students with immediate feedback and coaching, enabling them to immediately revise their work and quickly build their writing skills. In enabling more than 10 million low-income students to become stronger writers, we will equip more than 33% of all low-income students in the United States with the communication skills they need to succeed in high school, college, and their careers.

Quill’s entire operating budget is covered through earned revenue and all philanthropic capital is currently allocated to new research & development initiatives; Quill does not require philanthropic capital to sustain its operations. Quill generates 66% of its income through earned revenue: 55% through publishers licensing our technology and 11% through schools and districts subscribed to Quill Premium. Quill’s philanthropic revenue is 34% of all revenue, and this funding enables us to develop powerful new tools that help us reach millions of additional students. 

School Revenue: Quill now has 500 schools paying for our premium service and 10,000 schools where teachers are using our free service. Over the coming years, Quill will secure paid partnerships with thousands of additional school districts to provide sustainable, long-term funding as we scale. Quill’s growth is driven by teachers - teachers are searching for high-quality free tools, and, when they find Quill, they often share it with others in their network. Once teachers are successful with our free service, administrators reach out to purchase our Premium services. To increase our school sales revenue, we will continue to invest in our Partnerships Team and Quill Premium offering by thoughtfully examining, experimenting, and iterating our customized instructional work, sales and customer service cycles, and data analytics offerings for schools. 

Publisher Revenue: Quill has been able to quickly scale its earned revenue by licensing our technology to educational publishers. These publishers, including the College Board, are eager to augment their own offerings with our robust content library and powerful technology. While Quill’s publisher revenue provides Quill with immediate capital to cover both operating costs and to invest in R&D, we recognize that there is a “concentration risk” of having a large portion of our revenue coming from a small number of publishers. As we scale, Quill plans to increase our district sales earned revenue from 8% to more than 50% of all of our earned revenue to ensure Quill can sustainability serve school districts for decades to come. 

Foundation Revenue: Quill has successfully attracted and sustained partnerships with major foundations due to the massive number of students that we have impacted as a result of our innovative learning tools and robust partnership engagements. We maintain a core group of philanthropic partners that have provided multiple-year and annual gifts, and who have served as trusted strategic advisors throughout the course of our work together. 

To accelerate our growth, we have built a pipeline of strong prospective partners for cultivation in the next two years of growth. Quill’s primary strategy for attracting philanthropic revenue is to provide our funders with a high social impact “return on investment.” We equip low-income students with better writing skills at a low cost per student impacted, and these costs will go down as we scale. By providing philanthropic partners with an attractive impact ROI and innovative product development, Quill will be able to attract and retain foundation partners dedicated to helping low-income students become stronger writers and critical thinkers.

As highlighted above, Quill does not require philanthropic capital to sustain its operations. Quill generates the majority of our income through earned revenue.

We currently have over 500 schools paying for our premium service and 10,000 schools where teachers are using our free service.We have been able to quickly scale earned revenue by licensing our technology to educational publishers, including a multi-million dollar contract with the College Board. We have also successfully attracted and sustained partnerships with major foundations like the Bill & Melinda Gates Foundation, the Booth Ferris Foundation, the Brothers Brook Foundation, the Chan Zuckerberg Initiative, the Lone Pine Foundation, the Louis Calder Foundation, and, most recently, a six-figure grant from the Overdeck Foundation.