EASIER-SignON concertation event “Outcomes and Outlooks of Research on Sign Language in Europe”

On November 29, 2023, representatives from two EC-funded research and innovation projects—EASIER (Intelligent Automatic Sign Language Translation) and SignON (Sign Language Translation Mobile Application and Open Communications Framework)—and the LEAD-ME (COST) Action met in Brussels to discuss the progress and results of the initiatives and sign language research more broadly. This was done as part of the concertation event “Outcomes and Outlooks of Research on Sign Language in Europe,” organized with the support of the European Commission and moderated by the European Union of the Deaf, a partner in both EASIER and SignON projects.  The event brought together researchers and the end user community who discussed the current status and the future of sign language technology in Europe.  In an opening address, Member of the European Parliament, Brando Benifei called for greater accessibility for all citizens. He stressed that we do not only need to do more but that we also need to improve the way we do things. This was echoed in a keynote presentation delivered by the President of the European Union of the Deaf (EUD), Sofia Isari, who noted that we must work together to ensure that the digital world is an accessible world for all. Then the projects that organized the event were introduced by Dimitar Shterionov (Scientific Coordinator for SignON), Eleni Efthimiou (Scientific Coordinator for EASIER), and Krishna Chandramouli (WG3 Leader of the ‘LEAD-ME’ European Cooperation in Science and Technology (COST) Action). Eleni Efthimiou highlighted that minority languages remain disconnected from technology and that this has a major impact on inclusion. After that, a panel discussion moderated by Humberto Insolera (Executive Committee Member of the European Disability Forum) explored the way forward for sign language research in Europe. The panel included Frankie Picron (SignON, EASIER, EUD), Dimitar Shterionov (SignON), Eleni Efthimiou (EASIER), and Krishna Chandramouli (LEAD-ME). Discussing the co-creation process, Dimitar Shterionov highlighted that communication and trust with the community are the most important things to establish. Expanding on this, Eleni Efthimiou discussed the importance of providing features that the community has asked for, including elements of grammar and expressions of emotion. Krishna Chandramouli described the normalization of captioning in media and how we might use this momentum to include synthetically generated sign language. Frankie Picron commented on avatars, stressing that they will not replace humans but rather offer an alternative option—and as he pointed out, “we need more options.” The panelists also discussed different possible use cases for sign language machine translation and reflected on EASIER and SignON as both projects have demonstrated the possibility of sign language machine translation while at the same time highlighting the challenges posed by the lack of data. A lunchtime session of poster presentations and demonstrations showcased a range of perspectives on this area of research and, in the afternoon, a series of technical presentations focused on specific aspects of the projects and shared what the teams have learned throughout the research process. Eleni Efthimiou discussed sign language processing technologies, emphasizing that we need to join forces to ensure sign language technologies keep up with the overall technology landscape.  Dimitar Shterionov explained the sign language machine translation pipeline and how important it is that we standardize the way we collect and analyze data. After the event, expanding on this point, Shterionov explained:  “Since the inception of machine translation for spoken languages, language technology has evolved significantly, reaching unprecedented levels with end-to-end neural models trained on huge amounts of data. However, when it comes to signed languages, we need to turn to a more analytical approach, decomposing the translation into smaller, more focused tasks, i.e., recognition, translation, and synthesis. But even more so, we need to involve the right people. At this stage, we need researchers, linguists, and developers composed of and/or working with deaf communities.” Caro Brosens (SignON) delivered a presentation about the challenges with sign language data, in which he emphasized the importance of investing in high-quality data and better technological environments to process this data so that we can use it. Brosens expanded on this by saying, “The data bottleneck in sign language research is often reduced to ‘there is not enough data,’ but the situation is a lot more nuanced than that. Not only is there not enough data, only a small amount of said data is of high quality and/or computer-readable. Money should therefore not only be put towards creating new (high quality) data, but into developing the necessary technical environment to properly and efficiently process the data as well, otherwise the data remains unusable.” Davy Van Landuyt (SignON, EASIER, EUD) presented user engagement within the research process and shared recommendations for future work. In his insightful presentation, Van Landuyt highlighted three main points,sign language research projects must have deaf professionals at the steering wheel,transparent and trustworthy co-creation and science communication processes, led and carried out by deaf professionals are essential to building a relationship with deaf communities,hearing researchers must be aware of their social responsibility.  Then, Carlos Duarte (LEAD-ME) discussed web accessibility, noting that sign language must keep up with upcoming developments resulting from the strong progress being made on Artificial Intelligence (AI) models. Krishna Chandramouli explored the role of AI in digital accessibility as digital content becomes more pervasive—”AI technology offers new opportunities to be leveraged through content transformation methodologies.” Closing the event, Frankie Picron summarized the presentations and emphasized the importance of diversity and inclusivity in the consortium. The red thread or the central theme of the event was focused on advancing research on sign language technologies in Europe. Key elements included collaboration. Much like in the European Union, where progress is contingent upon the collective efforts of all Member States, research consortia also need to involve all stakeholders to foster a way forward in sign language technology research, with the ultimate goal of enhancing the lives of the deaf communities across Europe.