This philosophy is reflected even more strongly in One UI 8, launched alongside the latest Galaxy Z series. The update enhances hearing accessibility features, enabling smoother communication even in challenging listening environments.

To learn more about how these improvements came to life, Samsung Newsroom spoke with Soohyun Lee from Experience Planning Group1 of Samsung Electronics’ Mobile eXperience (MX) Business, along with Samsung Supporters Hajung Kim and Wansu Kim, who collaborated with Lee to bring more accessibility features to One UI 8.

▲ (From left) Wansu Kim and Hajung Kim with Soohyun Lee

Real Voices, Real Change: Samsung Supporters Inspire Inclusive Features

The hearing accessibility features in One UI 8 draw on the real-life experiences and feedback of Samsung Supporters. Currently active in Korea and the United Kingdom, the Samsung Supporters program invites users with diverse accessibility needs — including vision, hearing and mobility — to propose accessibility-related ideas directly to Samsung. It played a key role in developing the new hearing accessibility features in One UI 8, with Samsung Supporters Hajung Kim and Wansu Kim, both of whom are hard of hearing, contributing their experiences and ideas.

“We listen closely to the valuable feedback from our Supporters about inconveniences and other challenges that traditional desk research might overlook,” explained Lee, who manages the Samsung Supporters program. “We then incorporate those insights to improve our products and services.”

“We communicate with the Supporters in real time on a daily basis, while also conducting online surveys and in-depth offline interviews to carefully analyze issues and review solutions,” she continued. “Samsung Supporters are part of the entire process — from planning to testing and refinement.”

▲ Soohyun Lee from Experience Planning Group1 of the MX Business

Features directly proposed by Samsung Supporters — such as Interpreter using keyboard and Real-Time Text calls — were brought to life in One UI 8. “After experiencing Galaxy products firsthand as a hard-of-hearing user, I suggested specific improvements regarding certain features,” said Wansu Kim, an international sign language educator. “Knowing that even a small change can make someone’s daily life much more convenient, I focused on ideas that people would truly need and be able to use with ease.”

Content creator and writer Hajung Kim collaborated with Wansu Kim to produce a video introducing One UI 8’s hearing accessibility features. “We wanted the changes we suggested to seamlessly blend into the everyday lives of more people, so I planned the video from the perspective of a real user,” said Hajung Kim with pride.

▲ (From left) Samsung Supporters Hajung Kim and Wansu Kim use the new accessibility features applied to One UI 8.

‘Breaking Language Barriers With a Keyboard’ — Samsung Supporters’ Pick ①: Interpreter Using Keyboard

Starting with the Galaxy S24 series, Galaxy AI’s translation capabilities have already proven to be a reliable companion for overcoming language barriers. However, its voice-based format had limitations in certain situations — for example, when the speaker’s pronunciation was unclear or when the surrounding environment was noisy.

With One UI 8, Samsung enhanced Interpreter¹ in a way that helps users communicate more naturally across languages through text typed on the keyboard. When a user types a sentence in their own language, the device instantly generates a real-time translation² in both text and speech. The screen then displays two sections — one showing the user’s original text and the other the translation for their conversation partner — enabling smooth two-way communication.

▲ Soohyun Lee introduces the Interpreter using keyboard feature.

“Our focus was on creating a seamless, uninterrupted conversation experience,” said Lee. “To reflect real usage patterns and refine the feature, we made typed sentences remain visible even after translation for easy editing and added an option to quickly delete the entire sentence with just one tap when users want to start a new one.”

▲ Wansu Kim demonstrates the Interpreter using keyboard feature.

“I love traveling with my family, but since both my parents and I are hard of hearing, communication has always been a big challenge — especially when traveling abroad,” said Wansu Kim. “On a recent trip, Interpreter using keyboard allowed me to converse freely with people using different languages, making everything so much easier. Now, I can solely focus on the excitement of traveling without concerns about language barriers.”

“Elevators were originally designed for wheelchair users, but today everyone benefits from them,” he added. “In the same way, Interpreter using keyboard can be used by anyone in situations where it’s difficult to talk. Accessibility isn’t only for people with physical limitations — it’s an opportunity for everyone.”

‘Don’t Miss a Word During a Call’ — Samsung Supporters’ Pick ②: Real-Time Call Captions

For users with hearing aids or cochlear implants, phone calls can still cause anxiety. It’s easy to miss parts of the conversation and difficult to jot down important details while talking. To ease this burden, One UI 8 introduces Real-Time Call Captions.³

By tapping the speech bubble icon on the right side of the call screen, the speech of both the caller and receiver are transcribed and displayed in real time. “I used to feel nervous whenever I needed to make a hospital appointment or talk with a bank representative on the phone because it was so important to understand every detail,” said Hajung Kim. “Now, with captions, I can instantly catch any words or sentences I might have missed, which makes me feel much more at ease. It’s not just a tool that converts voice into text — it’s a safety net that helps me follow the conversation and communicate more freely.”

▲ Hajung Kim uses the Real-Time Call Captions feature during a phone call.

The feature assists users during everyday phone calls with friends and family as well. “Before, I often had to ask my grandparents to repeat themselves because I couldn’t clearly understand their pronunciation,” Hajung Kim said. “I often worried that going through this process again and again would tire them out, especially during longer calls. But with Real-Time Call Captions, I can now read my grandmother’s words on the screen instantly, allowing the conversation to flow without interruption.”

“A short phone call can brighten an entire day, making this feature truly invaluable to my family,” she added.

‘Instant Conversations, No Waiting Required’ — Samsung Supporters’ Pick ③: Real-Time Text Calls

The accessibility feature Wansu Kim uses most frequently is Real-Time Text calls,⁴ which allows users to communicate in text during a call, as though they’re exchanging messages. In earlier versions, the text of the other person’s speech appeared only after they finished speaking. This was particularly inconvenient during automated response system (ARS) calls with banks, telecommunications providers or credit card companies, when the input time often expired before users could read the instructions on their screens.

Now with One UI 8, however, Real-Time Text calls display text the moment the other person begins speaking, allowing users to follow along and prepare responses without delay. “Even during personal calls, being able to see what the other person is saying in real time makes me feel much closer to them,” said Wansu Kim.

▲ Wansu Kim uses the Real-Time Text calls feature during a phone call.

“In the past, there were always slight pauses during text calls with the Supporters — it sometimes felt like playing a game of guessing each other’s timing,” Lee said. “Those experiences showed us that timing is the most crucial part of a phone call. By making text appear in real time — without waiting for the other person to finish speaking — we introduced a key improvement. Even a few seconds can make a big difference, and seamless timing helps bring people closer together.”

Making Accessibility a Given for Everyone

What does the future of accessibility envisioned by Samsung Electronics and Samsung Supporters look like?

“Accessibility isn’t a matter of special consideration — it’s something that everyone deserves and should enjoy,” Lee said. “Our goal is to create experiences that more people, across various environments, can access and benefit from without inconvenience at any time.” She added that the journey toward greater accessibility has been a joyful one, made possible thanks to those who understand the need for change and continue to offer their encouragement and support.

For Wansu Kim, accessibility means freedom. “To me, hearing accessibility is freedom,” he said. “Through my work with Samsung Supporters, I want to help ensure the voices of people like me are heard more clearly and contribute to a world where everyone can communicate freely.”

“Accessibility is a bridge that not only connects people physically but also links their hearts,” Hajung Kim added. “Beyond simply reducing inconvenience, I want to help build a world where technology becomes a source of confidence and empowerment for all.”

The accessibility features in One UI 8 — co-created and presented by Samsung Supporters — can be explored in detail in the video below.

¹ Interpreter requires Samsung account login. Certain languages may require language pack download. Service availability may vary by language. Accuracy of results is not guaranteed. Availability and supported features may vary by country, region or carrier. Availability of supported languages may vary. Visit Galaxy AI page at samsung.com for details.
² Speed may vary depending on device and network connection.
³ Feature availability may vary by country, language and device model. Speed may vary depending on network connection. Feature and UI are subject to change without notice.
⁴ Feature availability may vary by country, language and device model. Speed may vary depending on network connection. Feature and UI are subject to change without notice.

Galaxy AI currently supports 16 languages,¹ and by the end of the year that number will go up to 20 with these new additions. This update means even more users will be able to lower language barriers and step into a larger world with the power of Galaxy AI. The new languages and dialects will be available for download as language packs from the Settings app of compatible Galaxy devices.

For more information about Galaxy AI, please visit: Samsung Newsroom, Samsungmobilepress.com or Samsung.com.

¹ Supported languages include Arabic, Chinese (China mainland, Hong Kong), English (Australia, India, United Kingdom, United States), French (Canada, France), German, Hindi, Indonesian, Italian, Japanese, Korean, Polish, Portuguese (Brazil), Russian, Spanish (Mexico, Spain, United States), Thai and Vietnamese.

Whether exploring new places or sightseeing occasionally, AI is opening new opportunities to experience the world like never before while removing common communication barriers. With Samsung’s advanced Interpreter² feature, language barriers are no longer an issue, while Composer³ and Suggested replies⁴ facilitate quick, easy communication on-the-go — making Galaxy AI on the latest Galaxy Z series your perfect travel companion.

Let’s take a little trip to see exactly how this comes to life.

Seamless Live Translation No Matter the Situation

Visiting a museum is a great way to learn about the local history and culture, but language barriers can stop you being fully immersed in a new experience. Interpreter transforms your Galaxy Z series device into a real-time translation tool and maximizes the dual screen capability.

In Listening Mode, you can join a museum tour in another language and see the transcript, so you don’t miss any must-see parts of the exhibition. Plus, if you’re wearing Galaxy Buds3,⁵ you’ll hear an audio translation of the tour so you can go hands-free and really focus on what’s on show while walking around some of the most iconic venues on earth.

While on your travels, ordering food and drink means you interact with locals regularly. Conversation Mode in Interpreter enables easy conversations with someone in another language and thanks to the dual screen functionality of the Galaxy Z series, you can switch your device into FlexMode⁶ so both parties can see the conversation translated in a more natural way.

Interpreter translates every interaction so you can communicate seamlessly in 16 languages — with 20 supported languages expected by the end of this year.

Effortless Messaging Without Language Barriers

Our smartphones are a tool for sharing our thoughts and when traveling we don’t just interact by voice, but via text too. Writing emails in another language can be difficult and when writing something quickly on-the-go, formatting and accuracy can be impacted too. With Chat Assist,⁷ generating emails becomes effortless meaning you just need to input a few simple keywords and Galaxy AI will generate a suitable message in no time for you to review, edit if needed and send while on the move.

Imagine you check out of your hotel but while on your way to the airport, you realize you’ve left something in your room. You need to contact the hotel for assistance and the Composer feature can help. Easily compose an email asking the hotel staff to go and look for the item you left behind and have it shipped back to you with just a few simple inputs. No matter that situation you are in, Composer makes communicating effortless and more efficient.

Chat Assist⁸ enhances sentence structure, refines expressions, corrects grammatical errors and even generates complete sentences to save time while adjusting the tone to match your personal style. Plus, it overcomes language barriers by supporting multiple languages⁹ so you can effortlessly communicate with someone that you might not otherwise be able to understand.

And if you want to post the latest pictures from your travels to keep friends and family up to date, Chat Assist can even draft a social media post based on your previous posts so you can quickly upload and keep going on your travels.

Responses on the Go

When running late to meet a friend for dinner in an unfamiliar neighborhood, you can leverage Suggested replies to make the conversation go smoothly while on the move. The feature creates tailored responses based on an analysis of your latest replies to help arrange a new time for dinner — right from the Galaxy Z Flip6’s FlexWindow without even opening the device.

Our mobile devices go everywhere we go and shouldn’t just be there but be helpful too. The Suggested replies feature for Chat Assist is perfectly suited for the Galaxy Z Flip6, making the most of the FlexWindow so you can reply on-the-go, right from your pocket. You can communicate with friends, family, colleagues or others all without compromising your own tone and style.

Galaxy AI is no longer just potential. It has become a practical, everyday tool that enhances the daily lives of users. Samsung is committed to breaking down barriers and empowering users to make the most of new and existing experiences, including while traveling. No matter where you are or who you are communicating with, you can interact in all new ways thanks to Galaxy AI on the latest Galaxy Z series. A powerful communication tool that fits perfectly in both your palm and pocket — allowing you to explore and engage like never before.

Stay tuned for the next article in the ‘Unfolding Galaxy AI’ series showing how Galaxy AI can uplevel creativity on Samsung’s latest foldables.

¹ Samsung Account login may be required to use certain Samsung AI features. Samsung does not make any promises, assurances or guarantees as to the accuracy, completeness or reliability of the output provided by AI features. Availability of Galaxy AI features may vary depending on the region/country, OS/One UI version, device model and phone carrier. Some function availability may vary by device model. Galaxy AI service may be limited for minors in certain regions with age restrictions over AI usage. Galaxy AI features will be provided for free until the end of 2025 on supported Samsung Galaxy devices. Different terms may apply for AI features provided by third parties.
² Interpreter requires Samsung Account login. Certain languages may require language pack download. Service availability may vary by language. Accuracy of results is not guaranteed. Availability and supported features may vary by country, region or carrier.
³ Composer requires a network connection and Samsung Account login. This feature is activated when a certain number of characters is met. Service availability may vary by language. Accuracy of results is not guaranteed.
⁴ Suggested Replies feature for Chat Assist requires network connection to receive messages. It can suggest replies with or without network connection once the messages are received by the device. Samsung Account login required. Only supported from the FlexWindow of Galaxy Z Flip6. Service availability may vary by language and app. Accuracy of results is not guaranteed.
⁵ Audio interpretation is only available when wearing Buds and connected to a Samsung Galaxy smartphone. Interpreter feature is not directly provided by Buds themselves. If Buds are unavailable, the interpreted content is provided on the screen of your Samsung Galaxy smartphone. Certain languages may require language pack download. Service availability may vary by language, country, region or carrier. Samsung Galaxy devices may require the latest software update to properly support Galaxy AI features.
⁶ Flex Mode supported at angles between 75° and 115°.
⁷ Composer requires a network connection and Samsung Account login. This feature is activated when a certain number of characters is met. Service availability may vary by language. Accuracy of results is not guaranteed.
⁸ Tone suggesting feature in Chat Assist requires a network connection and Samsung Account login. Must meet length requirements to activate feature. Service availability may vary by language. Accuracy of results is not guaranteed.
⁹ Translating feature in Chat Assist may require Samsung Account login. Some chat applications may not support this feature. Certain languages may require language pack download. Service availability may vary by language. This feature is activated when a translatable language is detected. Accuracy of results is not guaranteed.

The process of building language features for Galaxy AI involved much time and effort as each language presents a unique structure and culture. Samsung’s Researchers from around the world — in Brazil, China, India, Indonesia, Japan, Jordan, Poland and Vietnam — shared the challenges and triumphs behind the development of Galaxy AI. Samsung Newsroom compiled a recap of their stories below.

Developing a Translation Model

Galaxy AI features such as Live Translate perform three core processes: automatic speech recognition (ASR), neural machine translation (NMT) and text-to-speech (TTS).

▲ Automatic speech recognition (ASR), neural machine translation (NMT) and text-to-speech (TTS) each require distinct sets of information for training

Samsung R&D Institute Vietnam (SRV) faced obstacles with automatic speech recognition (ASR) models because Vietnamese is a language with six distinct tones. Tonal languages can be difficult for AI to recognize because of the complexity tones add to linguistic nuances. SRV responded to the challenge with a model that differentiates between shorter audio frames of around 20 milliseconds.

Samsung R&D Institute Poland (SRPOL) had the mammoth hurdle of training neural machine translation (NMT) models for a continent as diverse as Europe. Leveraging its rich pool of experience in projects spanning more than 30 languages across four time zones, SRPOL was able to navigate the untranslatability of certain phrases and handle idiomatic expressions that may not have direct equivalents in other languages.

Samsung R&D Institute Jordan (SRJO) adapted Arabic — a language spoken across more than 20 countries in about 30 dialects — for Galaxy AI. Creating a text-to-speech (TTS) model was no small endeavor since diacritics and guides for pronunciation are widely understood by native Arabic speakers but absent in writing. Based on a sophisticated prediction model for missing diacritics, SRJO was able to publish a language model that understands dialects and can answer in standard Arabic.

The Importance of Data

Throughout the process of training Galaxy AI in each language, an overarching theme was the importance of open collaboration with local institutions. The quality of data used directly affects the accuracy of ASR, NMT and TTS. So Samsung worked with various partners to obtain and review data that reflected each region’s jargon, dialects and other variations.

▲ Each language has a distinct set of qualities that pose challenges in creating an AI language model for it. Tones add to the complexity for tonal languages such as Vietnamese.

Samsung R&D Institute India-Bangalore (SRI-B) collaborated with the Vellore Institute of Technology to secure almost a million lines of segmented and curated audio data on conversational speech, words and commands. The students got hands-on experience on a real-life project as well as mentorship from Samsung experts; the rich store of data helped SRI-B train Galaxy AI in Hindi, covering more than 20 regional dialects and their respective tonal inflections, punctuation and colloquialisms

Local linguistic insights were imperative for the Latin American Spanish model because the diversity within the language is mirrored by the diversity of its user base. For example, the word for swimming pool could be alberca (Mexico), piscina (Colombia, Bolivia, Venezuela) or pileta (Argentina, Paraguay, Uruguay) based on which region you’re from. Samsung R&D Institute Brazil (SRBR) worked with science and technology institutes SiDi and Sidia to collect and manage massive amounts of data as well as refine and improve upon audio and text sources for Galaxy AI’s Latin American Spanish model.

Samsung R&D Institute China-Beijing (SRC-B) and Samsung R&D Institute China-Guangzhou (SRC-G) partnered with Chinese companies Baidu and Meitu to leverage their expertise from developing large language models (LLM) such as ERNIE Bot and MiracleVision, respectively. As a result, Galaxy AI supports both main modes of Mandarin Chinese and Cantonese.

In addition to external cooperation, due diligence and internal resources were also essential.

Bahasa Indonesia is a language notorious for its extensive use of contextual and implicit meanings that rely on social and situational cues. Samsung R&D Institute Indonesia (SRIN) researchers went out into the field to record conversations in coffee shops and working environments to capture authentic ambient noises that could distort input. This helped the model learn to recognize the necessary information from verbal input, ultimately improving the accuracy of speech recognition.

There are many homonyms in Japanese as the number of sounds is limited in the language. So many words must be determined based on the context. Samsung R&D Institute Japan (SRJ) used Samsung Gauss, the company’s internal LLM, structure contextual sentences with words or phrases relevant to each scenario to help the AI model differentiate between homonyms.

Samsung’s Global Research Network

The professionals across various Samsung R&D Institutes made full use of Samsung’s global research network.

Before tackling Hindi, SRI-B collaborated with teams around the world to develop AI language models for British, Indian and Australian English as well as Thai, Vietnamese and Indonesian. Engineers from other Samsung research centers visited Bangalore, India, to bring Vietnamese, Thai and Indonesian to Galaxy AI.

▲ Staff and collaborators pose in front of Samsung R&D Institute India-Bangalore (SRI-B)

SRPOL had extensive experience developing ASR, NMT and TTS models for a multitude of languages. A key player in Galaxy AI’s language expansion, SRPOL collaborated across continents to support SRJO with Arabic dialects and SRBR with Brazilian Portuguese and Latin American Spanish.

Samsung developers at each of these locations learned to collaborate across borders and time zones. Developers from SRIN even observed the local fasting customs in India when meeting their SRI-B colleagues. Many reflected on their work with pride and gratitude — realizing the lasting implications this project has on language, culture, heritage and identity.

Ongoing Efforts as the Journey Continues

Samsung recently introduced Galaxy AI to its latest foldables and wearables. Since its release earlier this year, Galaxy AI has already been used on more than 100 million devices. “We’re expecting to reach 200 million devices by the end of 2024,” said Won-joon Choi, EVP and Head of the Mobile R&D Office, Mobile eXperience Business at Samsung Electronics at a recent panel discussion.

Amidst this mission to democratize AI, it is important to look back and celebrate the accomplishments and progress that have led to providing this safe and inclusive technology that will benefit humanity and improve lives. By building up the Galaxy AI ecosystem with even more features, languages and regional variations, Samsung is facilitating cross-cultural exchanges in unprecedented ways to realize its vision of AI for All.

Samsung R&D Institute Japan (SRJ) was established as an R&D center focused on hardware such as home appliances and displays. With the demand for AI innovation ramping up globally, SRJ in Yokohama has also been operating a software development lab to create Galaxy AI’s Live Translate, which automatically translates voice calls in real time, since the end of last year.

“Live Translate is particularly efficient for travel scenarios such as visitors to this year’s Olympic Games in Paris,” says Takayuki Akasako, the Head of Artificial Intelligence at SRJ. “We are currently developing a speech recognition program for people who are both sightseeing and watching the Paris Olympic Games; by training the speech recognition program to learn about the games and locations of stadiums for Paris 2024.”

Understanding Context in Voice Recognition

For those already using the translation features of Galaxy AI, such functionalities may seem very useful. But for developers who have made the features come to life, they know that being able to communicate while traveling abroad isn’t something that can be taken for granted.

One thing the team noted was that there are more homonyms in Japanese than some other languages. For instance, ‘chopsticks’ (Hashi,箸) and ‘bridge’ (Hashi,橋) are relatively easy to distinguish due to the difference in intonation, but words like ‘sightseeing’(Kankō,観光), ‘customs’(Kankō,慣行), ‘public’ (Kōkyō,公共) and ‘prosperity’ (Kōkyō,好況) must be judged based on the context.

“Judgement becomes more difficult when the context is ambiguous, such as names of locale and people, proper nouns, dialects and numbers,” says Akasako. “So in order to improve the accuracy of speech recognition, a lot of data is needed.”

“We always look for ways to fine-tune the AI model for key events and moments in a timely manner,” continues Akasako. “With a lot of new combinations of place names and activities, it’s important that the context is still clear when people are using Galaxy AI.”

Challenges in Collecting Efficient Data

While recognizing the types of data needed is also important, collecting the data in and of itself is a challenge in its own right.

Previously, the SRJ team used human-recorded data to train the speech recognition engine for Live Translate, which didn’t result in sufficient data collection.

Samsung Gauss, the company’s Large Language Model (LLM), uses scripts to structure sentences with words or phrases that are relevant to each scenario. The data collected with Samsung Gauss is not only recorded by humans, but also generated by a speech synthesis text-to-speech (TTS) data, through which human resources do the final check on the quality. Using this method, the team has seen a dramatic improvement in data collection efficiency.

“Every time a problem is identified and solved, the accuracy of speech recognition improves significantly,” says Akasako. “Regardless of where people are, our goal is connecting people with each other, and the tools powered by Galaxy AI will ensure more fun and efficient communication.”

There’s a saying at the Samsung R&D Institute Poland (SRPOL): “A day at SRPOL lasts 96 hours”. It refers to the center’s global role as one of the largest and fastest-growing R&D centers in the region, often working across four different time zones. Sitting at the heart of Europe while covering many European and global markets, SRPOL has worked on automatic speech recognition, neural machine translation and text-to-speech models for more than 30 languages. When it came to bringing 10 languages to Galaxy AI, this expertise meant the team was well suited to seamlessly blend cultural perspectives with Samsung’s global technology.

SRPOL has years of experience in Natural Language Processing. What makes it unique is its adaptability to work on any language thanks to the passionate team and their tools, such as a crowdsourcing platform that enables fast and agile development.

“Collaboration across the continent means relentless data collection, annotation and research, which has become something we really enjoy,” says Kornel Jankowski, Head of Speech Decoding at SRPOL. “We’ve dealt with so many languages that our team developed universal, language-agnostic skills. When we’re asked to support a new language model, everybody’s attitude is: Oh wow, we get to learn another one, that’s going to be fun!”

A European Center for AI Language Development

Language is a cornerstone of culture and communication across Europe regardless of whether it’s incorporated into technology. However, it presents unique challenges for the team at SRPOL, who develop AI models for European languages.

“Each language and the culture that it is part of, comes with hurdles that make us reevaluate how we perceive a specific issue,” explains Adam Ros, Head of Artificial Intelligence at SRPOL. These hurdles include navigating the untranslatability of certain phrases and handling idiomatic expressions that may not have direct equivalents in other languages.

The team saw these challenges as an opportunity to make SRPOL a European center for AI language development. The biggest benefit of this is that it shortens the communication path between different departments and crucially, the decision-making path. Whether it is a matter of automatic speech recognition, neural machine translation or text-to-speech, teams could simply walk over to colleagues in Mobile Quality Assurance and efficiently solve problems together.

While this has helped, it hasn’t overcome all AI challenges. Inevitably, there are limitations in AI models when dealing with multiple European languages, such as translating without context or variations in intonation. However, the team saw these as an opportunity to keep learning and innovating.

“My team never stops at just one example when handling a new word or topic. Some European languages are harder than others,” adds Ros. If you’ve ever been to Spain, you know that Spanish is often spoken at blazing-fast speeds and we need to train AI well to handle that.”

Galaxy AI’s expansion required novel cross-continent collaboration, but the work soon grew beyond European borders. SRPOL supported the Jordan team’s efforts to teach Galaxy AI Arabic’s myriad of dialects, as well as the Brazil team’s work on Latin American languages.

The importance of language and cultural difference subtleties are all on the radar of SRPOL product developers because they can all be noticed by the target — the end users.

“There are subtle differences between European cultures that impact whether something feels natural to the end user. For example, people in some countries expect to read prices with the euro symbol (€), while others are accustomed to seeing it spelled out, e-u-r-o-s,” says Agata Maria Rozycka, Head of Voice Intelligence Research at SRPOL. “If this cultural nuance is not reflected translated text, the interface might seem less intuitive to a user. Implementing these micro-level insights into interface design can make technology feel more natural across diverse cultures.”

“The team has been remotely communicating and collaborating across different countries for many years, building up numerous effective communication channels,” says Marcin Mrugala, Head of Mobile Quality Assurance at SRPOL. “We were ready to do our part in enabling Galaxy AI to lower language barriers around the world.”

Technology for Bridging Cultures

Managing and integrating diverse linguistic and cultural insights is a challenging task, but it is essential for Samsung’s vision for Galaxy AI — lowering the barriers that divide people based on language and culture, and enabling them to create deeper connections.

“We’re not just building technology of the future, we’re building teams of the future too. Our best practices are designed to refine products based on differences across countries, but we fundamentally believe our similarities far outweigh our differences and our technology can unite cultures,” says Mrugala.

“Our goal is to bring people together, to make their lives easier, and to simplify their daily tasks. We’re seeing our families using the Voice Recorder in new ways, and we can now call our friends and different countries and talk with them in their own language. It is magical to see this change in the world and to be part of it. Galaxy AI brought SRPOL people together and now we are bringing together the world,” concludes Rozycka.

Hidden inside the Vellore Institute of Technology in Chennai, India, is a lab filled with futuristic audio equipment. One will find mannequins — known in the industry as head and torso simulators — as well as binaural microphones and hearing devices. They are stored in special chambers treated with an advanced sound absorption system, making this lab the first of its kind in India. Imagine such a facility is used to develop the latest high-end high fidelity (Hi-Fi) equipment.

This is where the Vellore Institute of Technology collaborates with Samsung to produce and develop data and insights that power the latest AI models for Galaxy AI’s language capabilities. The facility was developed as part of Samsung SEED (Students Ecosystem for Engineered Data) Labs — an initiative that enables university staff, students and interns in India to work on projects requested by Samsung since 2021. This is just one of several university programs funded by Samsung in which students have the opportunity to work on projects with technical experts from the company.

“As a student, I love being able to work on multiple projects with a well-known and respected company such as Samsung,” says Yashika Ilanchezhiyan, a Samsung SEED student. “I’m given the confidence to learn new skills in a practical way and feel like I’m making a real difference in current and future products.”

“This kind of collaboration is a win-win situation,” says Giridhar Jakki, Head of Language AI at Samsung R&D Institute India – Bangalore (SRI-B). “Thanks to our projects with universities, we are able to access additional expertise and custom datasets. Partnering universities receive investment, financial incentives and expert mentorship from Samsung as a result.”

Lowering Language Barriers

SRI-B has collaborated with teams around the world to develop AI language models for British, Indian and Australian English as well as Thai, Vietnamese and Indonesian. Recently, core engineers from other Samsung Research centers visited Bangalore, India — where the SRI-B team helped ramp up the technology to bring Vietnamese, Thai and Indonesian to Galaxy AI. SRI-B was therefore ideally positioned to develop the Hindi language for Galaxy AI.

“Every language has its challenges,” says Jakki. “But when you consider the end goal of bringing people the ability to communicate in other languages, it’s worth every ounce of effort. We couldn’t wait to bring Hindi to Galaxy AI.”

Developing the Hindi AI model wasn’t simple. The team had to ensure more than 20 regional dialects, tonal inflections, punctuation and colloquialisms were covered. Additionally, it is common for Hindi speakers to mix English words in their conversations. This required the team to carry out multiple rounds of AI model training with a combination of translated and transliterated data.

“Hindi has a complex phonetic structure that includes retroflex sounds — sounds made by curling the tongue back in the mouth — which are not present in many other languages,” says Jakki. “To build the speech synthesis element of the AI solution, we carefully reviewed data with native linguists to understand all the unique sounds and created a special set of phenomes to support specific dialects of the language.”

Collaborative efforts between Samsung and academic partners were instrumental in developing the AI language model that reflected the cultural nuances of the India’s regions. The Vellore Institute of Technology helped secure almost a million lines of segmented and curated audio data on conversational speech, words and commands. Data was a crucial component for a task as critical as incorporating the fourth most spoken language in the world into Galaxy AI. Working with universities ensured Samsung was using the highest quality data.

Global Connections Deliver Big Impacts

This project perfectly encapsulates Samsung’s philosophy of open collaboration and the company’s belief that sharing expertise and perspectives ensures meaningful innovation. In the case of SRI-B, this not only includes working with academia but also sharing insights and best practices with other Samsung research centers around the world.

“I’m extremely proud of what we’ve achieved with the help of our partners,” says Jakki. “AI innovation through collaboration is a big part of what we do. We will continue to better understand, collect and analyze language data so more people can have access to AI tools in the future.”

A diverse country with more than 203 million people embodying a wide range of cultures and traditions, Brazil uses Brazilian Portuguese as its official language. Meanwhile, 22 neighboring countries use Latin American Spanish.

Although Brazilian Portuguese and Latin American Spanish are widely spoken, intricate variations in both languages presented various challenges when teaching Galaxy AI to discern and distinguish regional differences. That’s why Samsung R&D Institute Brazil (SRBR) collaborated with Samsung experts from Mexico — as well as third-party partners such as the science and technology institutes SiDi and Sidia — to assemble a multidisciplinary and highly skilled team that could tackle the task.

Lower Barriers, Higher Understanding

The team used thousands of sources and a combination of machine learning and language processing tools to improve the AI model’s recognition of speech, written texts and regional variations. But local jargon and names of famous figures — including sports teams, celebrities and bands — vary widely between regions. Also, the same meaning can be expressed in many different words. While language models need localized data to gain a comprehensive understanding of the different languages to be translated, such variations inevitably present obstacles.

For example, swimming pool is “alberca” in Mexico — but it is “pileta” in Argentina, Paraguay and Uruguay. Meanwhile, in Colombia, Bolivia and Venezuela, swimming pool is “piscina”, which is also used in Brazil but with a slight tonal difference. And while Colombians might say “chévere” to refer to something cool, Mexicans instead say “padre.”

These differences represent huge challenges for AI language understanding and learning, but the team overcame them by building larger language models, refining processing tools — and collaborating across borders and time zones.

“We had to consider local slang and different ways of speaking before adapting and testing the model accordingly, which required close collaboration between the SRBR quality assurance (QA) team and development teams,” says Mateus Pedroso, Senior Manager and Head of Software Quality Lab at SRBR. “Since SRBR is located three hours ahead of the QA team in Mexico and 12 hours behind the management team in Korea, we had to create new communication channels and processes to align results and share progress. This multicultural collaboration generated a fiesta of ideas and solutions for Galaxy AI.”

Communicating Success

Samsung’s philosophy of open collaboration came to life during this regional project as it was an iterative process that leveraged evolving technology on a global scale. To overcome linguistic and cultural barriers, the SRBR team needed to collect and manage massive amounts of data — continually refining and improving upon audio and text sources.

The teams carved out key areas of responsibility to ensure everyone could benefit from the collective skill sets across the company’s Latin American offices. The SRBR development team served as the intermediate stakeholder of the project, receiving directions from Samsung’s headquarters and developing new updates to improve the AI model while carrying out tests for numerous use cases.

“The testing phase required extensive communication and collaboration with QA teams to optimize the user experience, and each adjustment needed further testing and review,” says Leandro Flores de Moura, Software Development Manager at SiDi. “The success of Galaxy AI’s language capabilities is built on communication and collaboration as much as it is on technical expertise” adds Nathan Castro, QA Test Developer at SiDi.

A Roadmap for Culture

What makes Galaxy AI particularly interesting for everyone involved is the fact that this wasn’t merely a language project. To them, language is a cultural guide that provides valuable insight into people’s heritage and identity.

“For SiDi’s QA team, this was an endeavor that will change the world by enabling cultures to come together and overcome the difficulty of communicating in different languages,” adds Estefanía Castro Suárez, Test Developer at SiDi. “Knowing we were part of this fills us with pride and motivation.”

“The way the SRBR team collaborated exemplifies what Galaxy AI sets out to achieve — making the world a smaller place through communicating, sharing and interacting with people, even those who speak different languages,” concludes Pedroso. “This capability will only grow as more languages come on board with Galaxy AI.”

Samsung Research combined data and cutting-edge technology. Together with the Mobile eXperience (MX) Business R&D Office, they further honed this technology to develop the translation features powered by on-device AI — which can be used for real-time translation during calls and across various applications. Samsung Newsroom met with Yoonjung Choi and Yonghyun Ryu from Samsung Research’s Global AI Center to learn more about these ambitious features.

▲ (From left) Yoonjung Choi and Yonghyun Ryu of Samsung Research’s Global AI Center

Smooth Communication and Strong Security With On-Device AI

On-device AI is the key differentiator to Galaxy AI’s Interpreter and Live Translate features — introduced to users through the release of the Galaxy S24 series earlier this year. Leveraging the advanced computing resources built into the devices themselves, smartphones with on-device AI can provide services without relying on servers or the cloud. Users can be assured that their data will remain private and secure since information is not shared with external sources.

Samsung Research’s Global AI Center contributed to integrating proprietary technology into its AI translation model to commercialize these features for widespread use.

Envisioning a wide range of applications, the Samsung Research team and MX R&D Office obtained an expansive amount of data. “We collected colloquial data for real-time translation during calls in Live Translate and travel-related data for Interpreter,” explained Yoonjung Choi, who led the project. “To provide the most accurate translation, we studied and incorporated casual language used in chatrooms and HTML tags used in web browsers.”

▲ Use cases for on-device AI translation include ① Live Translate, ② text message and conversation translation, ③ translation in Samsung Notes and ④ web browser translation.

How Samsung Research Trained Its AI Model

The Samsung Research team’s AI translation model is based on deep learning technology that learns from its own data. Yonghyun Ryu, who is in charge of AI research and development, likened this process to raising a child. “Similar to how a child needs excellent educational resources and caregivers to grow and thrive, good language data and talented researchers are required when developing a high-performance AI translation model,” he described.

Samsung Research has both — since 2013, the company has been providing in-house translation services, conducting R&D related to AI translation and accumulating high-quality data.

Samsung Research’s team of deep learning experts played an important role in training the AI model. “If incorrect translations occur during the research and development process, it is necessary to identify the problem and make improvements. However, this can be challenging and time-consuming for researchers without sufficient capabilities and experience,” he explained. “Our researchers used their expertise and know-how to quickly analyze the cause of the issue and come up with a solution to enhance the AI translation model.”

▲ Yonghyun Ryu from Samsung Research’s Global AI Center

To assess the performance of Galaxy AI’s translations, the Samsung Research team used quantitative metrics based on test sets as well as qualitative evaluations by human translators and the MX R&D Office.

In addition, the team gained credibility by competing in global machine translation competitions. “Although participation requires time and effort, good performance in competitions provides momentum for research and development,” Ryu emphasized. “We were able to achieve strong results because our team members could freely discuss new ideas and put them to the test.”

Politeness and Punctuation: Understanding the Quirks of Each Language

Each language has characteristics that are unique to its culture — these can include honorifics, tonal inflections and distinct punctuation symbols. To make translations as accurate as possible, the AI translation model considers all these linguistic idiosyncrasies. For example, in Korean and Japanese, honorifics are translated to maintain a respectful tone.

Samsung collaborated with regional R&D centers to fully understand languages in their cultural context. “By working closely with researchers and linguists in other countries, we were able to offer a more accurate and complete translation,” said Choi.

At the same time, dealing with different languages often involves trial and error. “Vietnamese, for instance, is a tonal language. However, we realized during the research process that Vietnamese users often omit tones in casual conversations when chatting,” she explained. “We needed additional data to help the features translate sentences without tones.” For Thai, Samsung Research developed a special sentence separator because the language does not use punctuation.

Why Lightweight Technology Is the Key to Effective On-Device AI Models

Samsung Research began developing its on-device AI translation model in 2019. “Unlike server-based AI models, on-device AI models must be driven only using users’ devices,” stated Ryu. “Developing lightweight technology that uses minimal resources is key.” To make the model lighter, the team used “knowledge distillation” and “quantization” technology.

Knowledge distillation is a method that extracts knowledge from a large, high-performing teacher model and delivers it to a smaller student model. This is similar to how a teacher summarizes a topic so that a student can digest the information more easily and efficiently.

Quantization simplifies AI algorithms to reduce model size and streamlines the process to increase response speed.

Ryu compared quantization to drawing strawberries. “You need a wide range of colors to paint lifelike strawberries — but strawberries can also be made with just red and green,” he explained. “Quantization is the process of minimizing the number of colors needed to draw strawberries while trying to make it as close as possible to the real thing.”

▲ Yoonjung Choi from Samsung Research’s Global AI Center

In the AI field, knowledge distillation and quantization are well-known approaches to making models lighter. Yet, implementing them on a commercial scale is not easy due to differences in each researcher’s detailed experimental methods and factors. Samsung Research developed proprietary technology by discovering an efficient quantization technique and creating a accelerated algorithm based on it. “Through constant experimentation, we found an optimal way to make the model lighter,” commented Choi.

By combining the high-quality AI translation model with algorithms that make models lighter and speed up response time, lightweight and fast on-device AI features were born.

The Culture Behind the Language: What Would the Perfect AI Translation Model Look Like?

The researchers at Samsung Research’s Global AI Center have bright goals as they lead the field of on-device AI. “My ultimate goal is to help users communicate smoothly and conveniently with people who speak other languages,” said Choi. Ryu revealed his vision to build the perfect translator. “One day, we want to create a translator that truly understands the cultures behind the languages it is translating, equipped with an extensive pool of knowledge,” he expressed. “I want to challenge myself to develop a translator the world has never seen before.”

▲ Samsung Research’s Global AI Center team developed the on-device AI translation feature.

Samsung’s on-device AI translation features allow anyone with an enabled mobile device to communicate freely — without worrying about internet connection or information leakage. Going forward, the Samsung Research team will continue to spearhead innovations in the rapidly evolving field of AI and bring new levels of convenience to users.