Spearheading Speech Translation Research

Samsung R&D Institute Poland and Samsung R&D Institute China-Beijing competed with some of the world’s top universities and research labs to win first place in two separate challenges at the International Workshop on Spoken Language Translation (IWSLT), one of the world’s longest-running workshops on automatic language translation. This year, IWSLT joined the Association for Computational Linguistics conference (ACL), a premier conference in the field of computational linguistics, to cover a broad spectrum of research areas that are concerned with computational approaches to natural language.

For the Offline Speech Translation task, which assesses the translation of TED talks from English to German, Samsung R&D Institute Poland won first place for the second time with its own research capabilities in audio to text translation. The conferral of this award marks the fourth consecutive year that teams from Samsung R&D Institute Poland have taken first prize in IWSLT challenges, including previous years’ text translation tasks.

This year’s Offline Speech Translation task allowed participants to submit systems based on either the traditional speech translation pipeline system composed of an automatic speech recognition (ASR) and a machine translation (MT) or an End-to-End (E2E) system. Samsung R&D Institute Poland’s system is based on a single encoder-decoder deep neural network – an E2E system – capable of both English and German texts.

In computational linguistics, E2E systems are harnessed to solve the common problem of error accumulation, wherein, in a traditional pipeline, an error in the speech recognition phase can lead to a nonsensical translation. However, research from over the past three years has shown that traditional systems have constantly been outperforming E2E speech translation systems. The Samsung team’s system not only placed first in the E2E category, but also outscored all traditional pipeline system entrants, a remarkable achievement that puts Samsung R&D Institute Poland at the forefront of speech translation research.

The team from Samsung R&D Institute Poland Team who participated in this year’s IWSLT challenges

Innovative Approaches in the Field of Computational Linguistics AI

Samsung R&D Institute China-Beijing took part in a second challenge, the Open Domain Translation task evaluating Japanese to Chinese translation capability, ultimately taking first place. The main goals of this task were the promotion of research into translation between Asian language, the exploitation of noisy parallel web corpora for machine translation and the thoughtful handling of data provenance.

Samsung R&D Institute China-Beijing submitted a system based on Transformer model architecture and adopted the relative position attention. The team focused on improving the Transform baseline system with elaborate data preprocessing and managed to achieve significant improvements. The team also tried shared and exclusive word embedding and compared different granularity of tokens, approaching the process at a sub-word level, including Byte Pair Encoding (BPE) and Sentence Piece. Large-scale back translation on monolingual corpus was used to improve the Neural Machine Translation (NMT) performance.

Members of the team from Samsung R&D Institute China-Beijing Team who participated in this year’s IWSLT challenges

Achievements in AI Audio Signal Interpretation

In addition to their first-place finish in the IWSLT challenge, Samsung R&D Institute Poland was also recognized as one of the leading teams at the Detection and Classification of Acoustic Scenes and Events (DCASE) 2020 challenge, held by IEEE (Institute of Electrical and Electronics Engineers), which aims to use state-of-the-art AI technology to understand and interpret audio signals.

Engineers from Samsung R&D Institute Poland, who possess previous experience in Acoustic Scene Understanding and Sound Sources Localization tasks (having ranked first place in two tasks in 2019), set their focus on Task 2: Unsupervised Detection of Anomalous Sounds for Machine Condition Monitoring. The goal of this task was to identify whether the sound emitted from a target machine was normal or anomalous. The main challenge was detecting unknown anomalous sounds under a condition within which only normal sound samples have been provided as training data. The engineers scored second place out of 40 teams.

The team from Samsung R&D Institute Poland Team who participated in this year’s DCASE challenge

Envisaging the Future of Computer Vision and Pattern Recognition

In June, Samsung R&D Institute China-Beijing also participated in three challenges hosted by the 2020 Conference on Computer Vision and Pattern Recognition (CVPR 2020): the Embodied AI Challenge, the VizWiz-Captions Challenge and the VATEX Video Captioning Challenge. The team claimed second place in the challenges.

The Embodied AI Challenge aimed to enable robots to understand human commands and perform correct actions within a virtual environment, while the VizWiz-Captions Challenge involved predicting an accurate caption when given an image taken by a visually impaired person and the VATEX Video Captioning Challenge aimed to benchmark progress towards models that can describe videos in various languages including English and Chinese.

Members of the team from Samsung R&D Institute China-Beijing Team who participated in this year’s CVPR challenges

Samsung R&D Institute Poland at IWSLT 2019

2019 marks the third year in a row that Samsung R&D Institute Poland, in partnership with the U.K.’s University of Edinburgh (UEDIN), has received accolades at the International Workshop on Spoken Language Translation (IWSLT), one of the top two global workshops on automatic language translation, along with the Workshop on Machine Translation (WMT). This year, Samsung R&D Institute Poland won first place in two categories, the first being text-to-text translation from English to Czech and the second – an end-to-end system translating English speech into German text.

For the text-to-text translation category, researchers worked to develop a model to translate the transcript of a spoken English-language TED Talk into Czech. Developing their winning model required the Samsung team to develop large, filtered corpora from which to work and generate as much synthetic data as possible. The work done by the Samsung R&D Institute Poland team, together with additional modeling help from UEDIN, was selected as the best in the challenge by human evaluators. This means that the translations produced by Samsung R&D Institute Poland’s system scored the highest both in fluency and adequacy.

Samsung R&D Institute Poland’s participation in their second winning category this year, the end-to-end translation system from English to German, was a first for the team. The task was to produce a German-language transcription of an English-language TED Talk audio recording. This task required the development of a single model that could take an audio file input and subsequently produce a translated transcription. It was made more difficult by the deficiency of the provided audio sources, compared to typical speech recognition task. Samsung R&D Institute Poland proposed several innovative methods for end-to-end speech translation that mitigated this source paucity, obtaining a state-of-the-art result with their final system that won them first place in the challenge.

Samsung Research America at ICCV 2019

This October, researchers from Samsung Research America’s AI Center received first place in the International Conference on Computer Vision (ICCV)’s challenge: Linguistic Meets Image and Video Retrieval (Fashion IQ). ICCV is a premier international computer vision conference that took place in Seoul, Korea, this year.

The challenge Samsung Research America AI Center took part in, sponsored by IBM research, aims to develop conversational shopping assistants that are more natural and real-world applicable. The task given to Samsung Research America AI Center‘s team, the ‘Superraptors’, belonged to the domain of image retrieval. In the task, an input query was specified in the form of a candidate image as well as in two natural language expressions that describe the visual differences of the search target. The goal of this challenge was to gather opinions and experience from researchers on the emerging space of visual content retrieval with a natural language interface.

Samsung Research America AI Center’s submission to the challenge, “Multimodal Ensemble of Diverse Models for Image Retrieval Using Natural Language Feedback”, blended the given data in different modalities with multiple deep learning models. The team’s win marks the first time a Samsung Research team has won a multimodal (language and vision) challenge; previously, Samsung AI Center Moscow, Samsung R&D Institute Poland and Samsung R&D Institute China-Beijing have received awards in single modality challenges.

Samsung Electronics’ artificial intelligence (AI) capabilities are being recognized globally in a competitive field with top researchers all seeking to dominate. Samsung Research, the advanced R&D arm of Samsung Electronics’ device business, has won recent competitions, which will be vital in ultimately rolling out AI in more real-world situations than ever.

In October, Samsung Research’s R&D Institute Poland (SRPOL), in partnership with University of Edinburgh of the U.K., won first place at the International Workshop on Spoken Language Translation (IWSLT), one of the world’s most renowned and longest-running automatic language translation workshops. IWSLT sits alongside the Workshop on Machine Translation (WMT) as the most prestigious competitions in the space. The IWSLT win is the second consecutive time that the Poland research center has topped the workshop.

Every June, IWSLT provides permissible training data sets and allows test run submissions from participants, evaluating the submitted runs with automatic metrics. This year, IWSLT featured two tasks: Low Resource Machine Translation and Speech Translation. SRPOL participated in the first sector, aiming to discover new technologies that can help overcome low-resource situations with scarcity in not only data but also in time and cost that are often required to use AI for multilingual translation.

Samsung’s task was using AI technology to translate text from Basque to English. The SRPOL team was first provided with limited amount of data – about a million Basque-to-English parallel data sets in the form of movie subtitles – to train and develop the algorithm model. Then, the team used this data to translate TED Talks given in Basque into English. Despite the difficulty of proposed translation direction and scarce data, when compared with the reference translations, SRPOL received the highest Bilingual Evaluation Understudy score (BLEU) of 26.21 – showing a high level of correspondence of the machine translation to the reference – and a significantly low error rate.

Samsung Research’s R&D Institute Poland (SRPOL) team

On the bordering country of Ukraine, Samsung’s AI expertise is being applied successfully within the automotive field. Samsung R&D Institute Ukraine (SRK) submitted a solution that it developed to detect pedestrians for the Karlsruhe Institute of Technology and Toyota Technological Institute at Chicago (KITTI) Vision Benchmark Suite, one of the top three benchmarking organizations for automotive driving technology, in August. KITTI sits alongside Caltech Pedestrian and CityScapes as the leading benchmarking institutions.

Among the nine KITTI categories including Stereo, Objects, Tracking, Road and more – SRK submitted its computer vision technology SiRtaKi in the ‘Pedestrian Tracking’ category. Since the submission, the solution has been recognized as the best in the field, retaining its top position on the leaderboard.

To enter the competition in the ‘Pedestrian Tracking’ category, AI algorithms are tested on their capabilities to not only detect objects but trace their movements – an upgrade from the previous technologies where only detection was possible. Likewise, SRK downloaded roughly 3,400 raw images of pedestrian movements from KITTI to run SiRtaKi which employs a tracking-by-detection approach. The team, using SiRtaKi, obtained and analyzed each passerby’s coordination and traveling direction countless times. As a result, SRK’s rigorous work paid off as they received the best score among competitors for a high level of accuracy in the object tracking.

Needless to say, such AI technology is expected to help create new possibilities in the automotive industry – that it can be implemented in autonomous sensors to detect nearby pedestrians and objects’ movements and reduce danger on the roads. Adding on, the new algorithm will also be applied in our daily lives like home appliances and also in the to-be-developed next-generation technologies.

Samsung R&D Institute Ukraine (SRK) team

It doesn’t end here. Last July, Samsung Research placed first in two of the world’s prestigious AI reading competitions, the MAchine Reading COmprehension (MS MARCO) Competition held by Microsoft and TriviaQA hosted by the University of Washington. To date, Samsung is still ranked first in the Web category of TriviaQA. And in March, Samsung R&D Institute China-Beijing (SRC-B) made mark in AI development in computer vision (technologies in analysis of characters and images), winning the International Conference on Document Analysis and Recognition (ICDAR), a global competition in Optical Character Recognition technologies.

“So much innovation took place by Samsung Electronics from all over the world. With multiple achievements in the world’s prestigious AI-related competitions, Samsung is leading the development of artificial intelligence and the related technologies on a global level,” said Seunghwan Cho, Vice President of Samsung Research. “We will continue to contribute to realizing a highly user-centric environment by applying the groundbreaking AI technology to our current products and services.”