JusticeAI – a Benetech project

Shabnam Mojtahedi  is an experienced human rights lawyer who champions meaningful justice for victims of conflict and state violence. In her role as Senior Program Manager for Benetech’s Human Rights Program, she leads efforts to develop artificial intelligence tools that support civil society organizations promoting justice and accountability in Syria and beyond. Prior to joining Benetech, Shabnam worked at the Syria Justice and Accountability Centre (SJAC) where she developed strategies, reports, and methodologies for SJAC’s documentation, data analysis, and transitional justice work. At SJAC, she led efforts to ethically document cases of sexual and gender-based violence and assessed how new technologies and social media can assist in the documentation and analysis of human rights violations. Previously, Shabnam was an adjunct law professor at Yeditepe University in Istanbul and a senior researcher with the Public International Law & Policy Group. She holds a J.D. from American University’s Washington College of Law.

JusticeAI is tackling an evolving challenge at the intersection of technology and justice. Although platforms like Facebook and YouTube offer the promise of wider access to knowledge about atrocities committed around the world, human rights and international justice institutions are still primarily equipped with traditional tools that cannot handle the volume of data emerging from 21st  century conflicts. Through partnerships with leading Syrian civil society organizations (CSOs), Shabnam has led Benetech’s development of the JusticeAI platform, which enables the processing, analysis, and secure sharing of media files to reduce duplicative work and accelerate investigations. Unfortunately, the challenges in Syria are not unique. With Elevate funds, Benetech will work in other contexts, like Yemen, to implement JusticeAI and build additional analytic capabilities. Ultimately, JusticeAI will be an open source platform that CSOs and justice mechanisms can use to efficiently sift through massive amounts of evidence, resulting in more effective judicial processes.

JusticeAI was developed to tackle the challenges resulting from the widespread use of cell phones and digital cameras by bystanders, citizen journalists, and human rights defenders filming atrocities as they happen. Although greater access to images depicting violations has created opportunities to expose perpetrators and demand accountability, human rights and justice actors must sift through huge troves of data to build evidence for litigation and advocacy. Given that organizations have collected millions of videos from protracted conflicts, like those in Syria and Yemen, the advantages of video are negated by the lack of time, resources, and capacity to review large amounts of data to identify verifiable information and make the necessary linkages among related content. Moreover, staff are subjected to the grueling task of reviewing graphic material. These challenges threaten to overwhelm and delay justice and cause burn out among dedicated professionals. By leveraging machine learning and computer vision, many rote investigatory tasks can be automated, cutting down on thousands of hours of manual labor and making meaningful connections across data collections. With Elevate funds, we will demonstrate the scalability of the JusticeAI platform by applying what we have learned working with Syrian partners to other contexts.

JusticeAI is a digital platform that strategically applies machine learning, computer vision, and metadata analysis to sort, identify, and analyze evidence of potential war crimes.  In partnership with Yemeni CSOs, this project will expand the use of JusticeAI to include the hundreds of thousands of videos that exist documenting the crisis in Yemen.  JusticeAI is open source software that can be installed and used on an organization’s private server or in a secure cloud account. Separately, each organization runs the software through their library of videos at a rate of approximately 2-3 seconds per video, creating an index of unique “fingerprints” representing each media file that cannot be reverse engineered.  Organizations can then share these fingerprints to determine what assets they have in common without the risk of transferring sensitive documentation.  Based on this index, CSOs can deduplicate, triage, and prioritize documentation for review by investigators and analysts as well as identify which videos and photos have not yet been analyzed by any organization. The platform facilitates coordination among civil society actors and enables justice mechanisms like the IIIM (International Impartial and Independent Mechanism) to work more efficiently by providing methods for informed ingestion of structured data from CSOs.

For countries seeking to address the harms posed by repressive regimes and conflict, justice is a long-term endeavor. This project supports the human rights defenders who are on the frontlines of advocating for just peace and an end to impunity. The JusticeAI platform offers a mechanism to process, cluster, and classify their data so they can spend more time doing the important work of complex investigations and reporting. In addition, this project benefits the international justice and inquiry mechanisms that are often the beneficiaries of human rights data and evidence. These mechanisms lack the technologies needed to handle large volumes of digital media, and JusticeAI provides them with a means to deduplicate, categorize and organize the mountains of evidence they receive from today’s conflicts. Ultimately, however, the beneficiaries of faster and more robust investigations are the victims of conflicts. Regardless of the technologies that underpin investigations, the goal of justice is to provide redress to victims and foster an atmosphere of accountability that deters atrocities from ever happening again. JusticeAI seeks to be among the tools in the toolkit of those seeking a just and sustainable peace.

This project supports those seeking justice and accountability by providing emerging technologies that better enable them to leverage digital media to bring greater awareness about atrocities to the public and to decisionmakers. As we stated above, the ultimate goal of a technology project like JusticeAI is to support the process by which victims of conflicts and atrocities are able to access justice and see that perpetrators have been held to account for their crimes.  

The need for JusticeAI was born out of conversations that began in March 2017 between Benetech, Syrian human rights organizations, and the UN’s International, Independent, and Impartial Mechanism (IIIM) to Assist in the Investigation and Prosecution of Persons Responsible for the Most Serious Crimes under Internatioanl Law. At the time, Benetech was asked to provide advice and support to the IIIM regarding strategies, tools, and practices for collecting, managing, and analyzing large-scale data. Meanwhile, Benetech was working closely with three key CSOs, the Syria Justice and Accountability Centre (SJAC), the Syrian Archive, and Bellingcat, in an effort to understand their challenges and lessons-learned from their years of collecting and analyzing data.  Shabnam was working at SJAC at the time and was actively involved in these conversations with Benetech and the IIIM, resulting in the identification of three interrelated needs 1) clustering of duplicate and related media; (2) coordination of analytical tasks to improve eliminate redundant efforts; and (3) streamlining of the data transfer process from CSOs to appropriate accountability institutions like the IIIM. When the project received its initial funding, Shabnam was recruited as the ideal champion to lead the initiative.

Shabnam is passionate about this project because it offers a seemingly simple and elegant solution to a number of obstacles in human rights research and analysis and puts technology that is currently only available to large tech companies into the hands of the human rights community. Human rights and justice actors, especially those working with open source material, are hungry for ways to process data more efficiently and work with coalitions to further investigations and case building efforts. However, security concerns and the inability to cheaply acquire reliable and trustworthy tech, prevent such initiatives from reaching their full potential. We are not creating flashy AI tools that promise to identify perpetrators through questionable facial recognition software or to verify whether a video is a deep fake – we are filling a very practical and broad need that offers the potential to securely connect data siloes and save investigators from looking through thousands of hours of graphic material to identify related content. And what's particularly exciting is that we are committed to helping diverse organizations benefit from this technology, even if they are working in a low-tech environment.

When Shabnam joined the Syria Justice and Accountability Centre in 2014, she did not expect to become immersed in software development and data management decisions. But her work providing guidance on how international criminal law standards applied to the documentation her Syrian colleagues were collecting from social media and on-the-ground sources, resulted in a need to think creatively and adapt. As a small organization, SJAC was flexible enough to adopt tech practices that larger human rights organizations were hesitant to broach. Shabnam has taken that creative mindset and the lessons from her work at SJAC into her leadership role running the JusticeAI initiative.

  Because of Benetech’s twenty-year history of software engineering in the social and public sectors, and Shabnam’s relationships with key CSOs and the IIIM, we are uniquely positioned to successfully develop and implement functional AI software that the human rights community trusts and can actually use in practice. Through Benetech’s project, Martus, we have over 15 years of experience assisting human rights organizations, that have limited tech capacity, adopt secure encrypted technologies to support data collection, management, and analysis. JusticeAI has absorbed the lessons-learned from Martus to fill an existing technological gap in the human rights space. We come with the understanding that it is not enough to build an innovative tool – we must put in the effort to work with partners to help them fit the tools into their workflows and spend time training and mentoring to ensure the tools are fully adopted.

Working with diverse partners in conflict environments comes with many challenges. Shabnam’s focus on building trust and demonstrating ethical and high-quality work products has been key in achieving several successes, such as her efforts setting up referral networks in Turkey to support victims of gross human rights violations. Related to this project specifically, security concerns have been a major barrier to data sharing and collaboration. The JusticeAI tools have been designed to address those security concerns by creating a unique fingerprint for each video and photo that can provide organizations with basic information about the content of the file to allow for data clustering and classifying without needing to share the file itself or revealing sensitive information like the names or faces of those that appear in the video or photo. Even with these security precautions, CSOs have expressed additional concerns about using the platform. Shabnam has spent time understanding their diverse risk models and taken that information back to the engineering team to collaboratively create workable solutions. Through open communication and transparent explanations about how partner feedback has been adopted to improve the tools, Shabnam has overcome much of the hesitation about the platform and increased enthusiasm among partners.

Since joining Benetech, Shabnam has taken on a leadership role, guiding strategic development for the Human Rights Program (HRP). In 2018, Benetech sunset its primary software designed to support human rights documentation, which resulted in a vacuum in the organization's human rights strategy. Through strategic planning with the HRP team and Benetech senior leadership, Shabnam took a lead in writing a new strategic plan and implementing action items, that included furthering thought leadership, monitoring program successes, and developing a human-rights-centered approach and impact assessment to guide Benetech-wide programming. 

Our innovation is in building technologies that are typically only used by large corporate actors and making them functional, secure, and open source for the human rights community. While there have been many initiatives that seek to apply AI in the social good space, most of them fail to deliver impact because they stop short of making the algorithms usable for their beneficiaries. Our approach to technology is full circle, from initial needs assessments to cooperative design and ongoing training and maintenance. We package the tools into an integrated cohesive web app that can be easily deployed in the cloud or on private servers. We are responsive to feedback and understand the barrier that low resourced human rights organizations face when trying to adopt new technologies. As a result, we meet organizations where they are to support their core missions.  In addition, this project is innovative because as global conflicts and human rights violations continue, digital evidence of these atrocities will grow at an alarming rate burying the traditional justice processes that rely too heavily on individuals.  Tools such as JusticeAI and its derivatives will be essential to keeping the international judicial process moving and providing timely and streamlined access to critical evidence that will help ensure that perpetrators of gross human rights violations will be held accountable for their actions.

By working directly with civil society and justice actors to adopt JusticeAI’s innovative and low cost software to automatically process and analyze their human rights media, organizations will have access to cutting edge tools to understand the overall data landscape and securely extract and share media metadata. When organizations better understand the data landscape and can securely share data with one another, they will be empowered to triage investigations and avoid duplicative efforts through collaboration, thus saving time and resources. By triaging investigations and collaborating, they can also more effectively leverage the large volumes of human rights data in their collections and compile comprehensive evidence files for justice efforts. With comprehensive evidence about human rights violations, civil society and justice actors will ultimately increase their capacity to pursue accountability for survivors of gross violations of international law, thereby curtailing impunity and promoting a just peace.

To this end, the outcomes of JusticeAI are:

1) Enable human rights defenders to efficiently and collaboratively analyze large volumes of conflict-related data to compile comprehensive investigations.

-Map the tech tools currently in use by organizations working on Yemen and identify gaps; build additional functionalities to address their needs.

- Provide CSOs access to JusticeAI tools as well as training and tech support so they can effectively adopt the tools to process their media.

2) Increase capacity of justice and inquiry mechanisms to leverage digital media to pursue avenues for truth and accountability in countries experiencing conflict.

-Work with a network of trusted organizations that share media fingerprints to the JusticeAI platform to increase understanding of the wider data landscape. 

-Utilize the JusticeAI platform to enable collaborative investigations and informed, targeted data ingestion by accountability mechanisms.

JusticeAI is currently being used by Syrian civil society organizations to process, deduplicate and share videos of human rights violations and atrocities that are being perpetrated in Syria.  The results of this project will be an index of applicable video evidence that can ultimately be used by the IIIM, justice-oriented international NGOs, and European war crimes offices. The ultimate beneficiaries of these tools will be the countless victims of the conflict.  While we cannot be sure of the timeline for when trials and legal action may take place, we hope that these tools will create pathways for investigators to pursue truth and justice more efficiently. Our goal is to expand this project to other countries facing active conflict, such as Yemen, and in each case, the human rights defenders of those countries will have more tools at their disposal to advocate for an end to impunity and justice for victims. Longer term, in five years or more, we anticipate that investigators working globally on open source investigation will be able to tap into the platform to securely share information they have gathered with trusted networks and gain benefits from what others have collected and analyzed, all while maintaining control, security, and ownership over their data – the number of beneficiaries is unknown at this time but our tools are open source and we have a broad and diverse network within the human rights community, so the impact will likely be widespread and dynamic.

During the next year, we expect to build upon this technology to provide similar services to 1-2 other countries experiencing conflict.  Working with partner organizations who have in-country expertise, we will modify our tools as necessary to benefit those CSOs as they document and organize digital evidence, expanding our ability to support a greater variety of trial processes and technology needs.  In the next 5 years we will continue to expand geographically by deepening our relationships with the UN mechanisms for justice and broadening the tool capabilities to include more languages, file types (audio, text)  to help our CSO partners keep up with the exponential growth of data and the need for tools to facilitate collaboration amongst related organizations.  In addition, we plan to pursue other ethical applications of the JusticeAI technology.  Early conversations indicate the tool could also be useful to investigative journalists who do not currently have tools to efficiently compare videos from multiple sources to determine which are duplicates and which are unique and may provide new information. In addition, the proximity value calculated by the algorithm can indicate which videos are similar but not exactly the same due to the addition of sound or logos or potentially other changes that might require closer investigation.  Still in its infancy, JusticeAI has the potential to revolutionize and streamline the process of analyzing, organizing, sharing, and collaborating on video and other digital assets.

Three primary barriers exist for the JusticeAI project:  1) the hesitancy of the human rights and international justice communities to adopt new technologies in their work due to lack of trust, resources, or time to learn how to integrate new technology into their workflows; 2) longer term, the availability of resources to fund human rights tech to maintain and improve upon the platform for future users; and 3) the limited number of sanctioned and funded mechanisms for pursuing justice for victims of human rights violations. Without a formal mechanism in place, it becomes more challenging for CSOs to seek funds to pursue their justice and accountability efforts and there is no clear path or final destination for the evidence they collect.

The strategy to address these barriers is also threefold: 1) trust building and patience to assist with adoption and implementation, understanding that one conversation or one training is not enough for an organization to feel comfortable adopting new tech as well as making resources available for organizations to acquire necessary hardware or staff to assist with implementation, 2) educating donors on the need for long term assistance and being persistent about pursuing funding resources through partnerships with well-respected human rights organization and our close relationship with a range of human rights-focused donors, and 3) acting not just as a technology service provider, but as a member of the human rights community that advocates for justice mechanisms and works with coalitions to call for an end to impunity for gross human rights violations. On this third point, we will also work with a range of justice actors, whether it be a formal international tribunal, independent U.N. inquiry mechanisms, jurisdictions that have war crimes offices dedicated to investigations through the principle of universal jurisdiction, nonprofits that pursue strategic litigation and case building, or commissions that are tasked with other aspects of transitional justice, such as truth, reparations, or memorialization  – by taking a broad view of justice and associated stakeholders, we anticipate that we can still make an impact even when there is no formal impartial court with jurisdiction over the situation.

JusticeAI would not exist without the collaboration and partnership provided by the Syrian Archive and SJAC.  These two organizations have been with us from the beginning, helping us understand the situation on the ground in Syria as well as the challenges created by the millions of videos documenting atrocities.  In Yemen, we plan to work closely Yemeni human rights organizations and justice-oriented nonprofits that are working on strategic litigation efforts.  Our process for working with each of our partners is similar.  We rely on our partners to help us understand the needs and data landscape, and in turn, we incorporate their feedback into our tools so that they are able to derive the utmost value from what we have developed. In addition to technology, we also aim to mentor partners on tool adoption and implementation as well as provide hardware and resources as needed. Another critical stakeholder on this project is the IIIM.  In support of their mandate which is to “to collect, consolidate, preserve and analyze evidence of violations … in order to facilitate and expedite fair and independent criminal proceedings…,” JusticeAI serves a critical function in helping efficiently process the mountains of evidence for use in later trials. It is imperative that we work closely with their investigators to ensure our tools function for their workflows and enables increased efficiency and ability to strategically ingest data -- not just for their own processes but to also increase JusticeAI’s utility to other similar mechanisms.

JusticeAI is a means for governments to financially and technologically support the justice and accountability mechanisms and local civil society organizations who are doing the day-to-day challenging work of documenting and preserving evidence of violations. State governments are the primary funders of international tribunals and transitional justice mechanisms, and JusticeAI’s function as a tool to efficiently organize and sort evidence and work more strategically with CSOs that have information about violations makes it an appealing product to support and fund. The key beneficiaries are not only the independent justice mechanisms and the human rights CSOs, but ultimately the people at the center of these conflict zones who have a right to access justice and redress for their harms. JusticeAI is currently supported by philanthropic funding sources and has not provided revenue to its partners. 

Benetech sees two potential paths to financial sustainability for JusticeAI in the coming years.  The first is centered around international justice and accountability processes and the establishment of independent mechanisms.  In times of conflict and widespread government repression, the UN is often called upon to observe these situations and sometimes establish mechanisms for truth and accountability.  To be effective in the age of widespread digital media, justice and inquiry mechanisms will need secure and reliable technology.  We anticipate that JusticeAI will be adopted by such mechanisms as a necessary technology to perform investigatory functions and we expect governments that currently support international justice to support the tool financially. A second path to financial sustainability that we are exploring involves working with other constituents, such as major journalistic outlets, who are involved in the quest for truth and justice and would be willing to pay for a tool that makes their jobs easier.  As we discussed in our 5-year goal, we will continue to explore how this technology can be modified to meet the needs of other socially conscious organizations and potentially be integrated with technologies in use today by these entities.  Part of these investigations will include the development of business models around creating customized and privately hosted instances of JusticeAI that contain special features and functionality requested by news organizations and paid for by private foundations.

Benetech’s JusticeAI project in Syria was initially supported by a grant from Microsoft, which has an interest in funding AI for Good projects. Since October of 2018, the project has been supported by a $1.6 million grant from an anonymous funder, which runs for three years until September 2021. 

Benetech is seeking to raise $500,000 over the next two years to support the expansion of JusticeAI into Yemen and possibly another country.  We are pursuing grant funding from government agencies and private foundations that wish to support justice and accountability in countries dealing with conflict and experiencing human rights violations.  The funds will cover project management and engineering costs to define additional functional requirements and the expansion of a dashboard that can be accessed by organizations who have agreed to share the fingerprints of their digital assets.  In addition, we anticipate providing subgrants to Yemeni CSOs to help cover their technology needs and travel to usability workshops where we will be testing the technology.

Expenses for our existing project in Syria are covered by our current funder.  For the expansion of JusticeAI into other conflict contexts, the majority of the budget for 2020 will be focused on an initial assessment by our project and engineering team of the video archive of CSO partners and the systems already in place.  It’s critical that we meet the partners where they are and understand the tech landscape under which they are currently operating. Moving into 2021 and 2022, and keeping in mind the $300,000 parameter provided by the Elevate prize, we are budgeting for the following expenses:

-       Subgrants to our partners (GLAN, Witness and other Yemeni CSOs) - $150,000

-       Benetech Product/Program Management -  $65,000

-       Benetech Engineering - $85,000

We are excited to be applying for the Elevate Prize for a number of reasons.  First, we believe winning the prize will enable us to amplify the profile of our project and provide much needed exposure of the conflict in Yemen which is currently the world’s worst humanitarian crisis. Despite widespread atrocities and ongoing displacement, the crisis has not received substantial media attention or an adequate international response. Secondly, the funding would enable us to expand the functionality of our platform, demonstrate the flexibility and transferable nature of this technology, and provide much needed funds to the Yemeni organizations who are collecting and preserving documentation of the conflict. Third, we would welcome the opportunity to collaborate with other leaders within the Elevate cohort as well as experts at MIT around our use of AI and how we can ethically and effectively continue to optimize the platform for justice and accountability.

Ultimately, we would like to partner with a wide network of institutions and nonprofits with experience in Yemen.  Some of these organizations include 1) Yemeni civil society organizations that have collected hundreds of thousands of videos and other valuable information about the conflict, 2) WITNESS, a nonprofit with an impressive track record of helping human rights defenders film and collect video, including in Yemen; and 3) the Global Legal Action Network, a nonprofit that has been pursuing legal action and advocacy related to Yemen. These organizations will not only be essential to pivoting the current products to another setting and providing feedback and suggestions for additional features, but will also be the users of the product and benefit from the analytics and collaboration features within their existing networks