Machine translation of public health info for low-resource languages

Automatically translating public health information into many languages without large resources, for use in future health emergencies.

Dr Andrew Caines, Principal Investigator

Within “Respond” we address the problem of ‘cutting through the infodemic’ by aiming to provide equitable access to high quality information. We noticed the problem during the COVID-19 pandemic of World Health Organisation webpages (for instance) being only available in six languages (Arabic, Mandarin Chinese, English, French, Russian, Spanish). This means that information from the global public health authority is not accessible to all without access to translation. If translation can be carried out automatically to some extent, then this lowers the barriers to access and enables dissemination of vital information worldwide. In addition there has been a problem of misinformation during the current pandemic, and if people have equal access to reliable sources of information then this helps cut through the infodemic of fake news. It is not clear how many people do not understand one of the six WHO languages (this is one statistic we can endeavour to find out through surveys) but it is clear that those who do not are more likely to be in the economically more disadvantaged areas of the world. Furthermore, almost 1billion people entered the 21st century unable to read or write (UNA-UK) and so speech technology is central to our proposal.

Our target audience is speakers of languages which are “low-resource” in terms of natural language processing technology (NLP). In NLP, English is the best-resourced language: all other languages have lesser pre-trained NLP tools and data resources. But in particular, African languages are under-resourced, as are indigenous American languages and many Asian languages with notable exceptions such as Mandarin Chinese and Japanese. We do think that good quality public health information should be available to all in their mother tongue, but in particular there is a pressing need to provide it to people who do not know one of the languages offered on the WHO website or on other global platforms. This means that they are able to access health information independently of filtering or mistranslation by their own state or media outlets, and to compare different sources of information. We seek to provide support by developing web and mobile apps which offer multilingual health information, and to do so with both text and speech functionality. We need to engage with end users through collaborators at local institutions, and intend to make user feedback central to app design, so that we understand how our prototypes work well and not so well.

Our solution is in itself equally accessible to all, since we intend to develop web and mobile apps for end users. There are of course wider societal factors which mean that in practice this is not the case -- for instance, unequal access to networks, inability to afford devices, and variance in digital literacy -- but these are larger trends than we can address in this project. To mitigate these inequalities, we will deliberately seek out marginalised members of society in the language communities we are targeting, on the basis that such people are less likely to be familiar with the languages served on e.g. the WHO website (Arabic, Mandarin Chinese, English, French, Russian, Spanish), and furthermore are less likely to be fully literate -- hence the eventual aim of incorporating text-to-speech technology in order that our translations may be heard as well as read. Our web and mobile apps will be free to use and will not collect personal data which exploits the end-user in any way. Furthermore we intend to make research findings and data available to others in the research community, in order to seed further research developments.

We are motivated to create impact on the basis of three factors: (1) that there is a ‘digital language divide’ in which content on the internet is strongly biased towards a few global languages*, thereby giving advantage to speakers of those languages, especially native speakers. (2) There is a bias in NLP research towards a few global languages, particularly English, and so the majority of the world’s 1000s of languages are not well-served by current language technologies. (3) There is a need to be better prepared for future health emergencies, as evidenced by the current COVID-19 pandemic in which the highest quality public health information is available in a few languages only, and there has been a global problem of misinformation which has been observed in Africa in particular**. We bring these 3 factors together as motivation for our solution in which we aim to serve populations who have been marginalised in terms of language technology and high quality public health information. We aim to reach these people through local contacts, social media and on-the-ground fieldwork. Collaboration with Internews would be of great benefit in this regard also. 

* http://labs.theguardian.com/di...

** https://africacenter.org/spotl...

We will scale impact over the next year by developing web and mobile applications which are accessible to large speaker populations in the target languages. We will seek out the sweet spot in language technology where the quality of our work is good enough to put in front of end-users (so we do not irretrievably damage our reputation) but also where we get feedback and usage data from end-users as soon as possible in order to inform future development cycles and improvements. Such an approach of ‘getting the outputs in front of people sooner rather than later’ is tried and trusted in ‘Big Tech’. For instance, Google Translate has benefited from a long-term strategy of providing machine translations to users, aware that they will not be perfect, but allowing user corrections of proposed translations which can feed into subsequent model training and improvement. We do not anticipate a user base on the scale of Google Translate of course, but we can still adopt this same approach of learning from our mistakes. In the next three years we plan to reach more users by targeting more languages of both Africa and later Asia, and also generating spoken translations through text-to-speech technology.

We are monitoring our goals by tracking the performance of our machine translation models on ‘held-out’ test data for each target language (texts not seen during model training). We use an automatic measure for measuring translation performance, namely “BLEU” which is the standard measure in the research field. This measures word overlap between the proposed translation and the ‘true’ translation from 0 to 100 where the higher the better. Currently our models achieve BLEU scores of around 10. This is not surprising, since the training data we have access to at present are (a) small and (b) from the religious domain, rather than public health. We plan to address both issues over the next year. For reference, the models achieve BLEU scores >30 when evaluated on held-out Jehovah’s Witness texts (the same domain as seen in training), and scores from 9 to 58 were reported for texts about COVID-19 in several languages as part of the WMT Shared Task 2020. To complement this automatic measure, we will gather human ratings of translation quality, since BLEU relies on the notion of a ‘true’ translation, whereas in reality texts can be translated in many acceptable ways.

The main barrier is lack of suitable translation data from the public health domain in the languages we target, but we intend to address this problem urgently. There is also a general lack of language technology resource for the languages we are targeting, but that is one of the defining characteristics of our work, and a problem we can seek to overcome using established methods in the research field, and also by establishing new methods. In order to improve the situation for others, we will make the data we collect and the models we train available to other researchers.

University of Cambridge

We are applying to The Trinity Challenge in order to scale up our project and overcome problems of data shortage, provide staff time to improve translation models, and reach target populations in order to evaluate the effectiveness of our translation technology. The starting problem is data shortage: this is the primary issue and one which, if we can at least partly solve it, enables all other aspects of our solution to flow onwards. The Trinity Challenge may put us in touch with collaborators at Internews in order to help with sourcing good quality data and evaluating our technology, but also winning a prize would bring us that most valuable commodity in academic research: research personnel and the time to work seriously on a problem.

Internews. In order to engage with diverse populations, deliver information in effective ways, and benefit from a network of local contacts in sub-Saharan Africa. This is a research project which will range from fundamental research on machine translation techniques to applied engagement with end users who might consume the outputs of our translation technology. A recurrent problem in computational linguistics is how to get our technology in front of end users, how to make our research impactful, and how to use insights from user evaluation to continue to improve the technology. Access to the expertise of Internews staff will be invaluable and a way to reach population groups we could not straightforwardly reach in any other way.