As Samsung’s continues to pioneer premium mobile AI experiences, we visit Samsung Research centers around the world to learn how Galaxy AI is enabling more users to maximise their potential. Galaxy AI now supports 16 languages, so more people can expand their language capabilities, even when offline, thanks to on-device translation in features such as Live Translate, Interpreter, Note Assist and Browsing Assist. But what does AI language development involve? This series examines the challenges of working with mobile AI and how we overcame them. First up, we head to Indonesia to learn where one begins teaching AI to speak a new language.

The first step is establishing targets, according to the team at Samsung R&D Institute Indonesia (SRIN). “Great AI begins good quality and relevant data. Each language demands a different way to process this, so we dive deep to understand the linguistic needs and the unique conditions of our country,” says Junaidillah Fadlil, head of AI at SRIN, whose team recently added Bahasa Indonesia (Indonesian language) support to Galaxy AI. “Local language development has to be led by insight and science, so every process for adding languages to Galaxy AI starts with us planning what information we need and can legally and ethically obtain.”

Galaxy AI features such as Live Translate perform three core processes: automatic speech recognition (ASR), neural machine translation (NMT) and text-to-speech (TTS). Each process needs a distinct set of information.

ASR, for instance, needs extensive recordings of speech in numerous environments, each paired with an accurate text transcription. Varying background noise levels help account for different environments. “It’s not enough just to add noises to recordings,” explains Muchlisin Adi Saputra, the team’s ASR lead. “In addition to the language data we obtained from authorised 3^rd party partners, we must go out into coffee shops or working environments to record our own voices. This allows us to authentically capture unique sounds from real life, like people calling out or the clattering of keyboards.”

The ever-changing nature of languages must also be considered. Saputra adds: “We need to keep up to date with the latest slang and how it is used, and mostly we find it on social media!”

Next, NMT requires translation training data. “Translating Bahasa Indonesia is challenging,” says Muhamad Faisal, the team’s NMT lead. “Its extensive use of contextual and implicit meanings relies on social and situational cues, so we need numerous translated texts that the AI could reference for new words, foreign words, proper nouns, and idioms – any information that helps AI understand the context and rules of communication.”

TTS then requires recordings that cover a range of voices and tones, with additional context on how parts of words sound in different circumstances. “Good voice recordings could do half the job and cover all the required phonemes (units of sound in speech) for the AI model,” adds Harits Abdurrohman, TTS lead. “If a voice actor did a great job in the earlier phase, the focus shifts to refining the AI model to clearly pronounce specific words.”

Stronger Together

It takes vast resources to plan for much data, and SRIN worked closely with linguistics experts. “This challenge requires creativity, resourcefulness and expertise in both Bahasa Indonesia and machine learning,” Fadlil reflects. “Samsung’s philosophy of open collaboration played a big part in getting the job done, as did our scale of operations and history of AI development.”

Working with other Samsung Research centers around the world, the SRIN team was able to quickly adopt best practices and overcome the complexities of establishing data targets. Furthermore, collaboration was good for advancing not only technology but also culture. When the SRIN team joined their counterparts in Bangalore, India, they observed the local fasting customs, creating deeper connections and expanding their understanding of different cultures.

For the team, Galaxy AI’s language expansion project took on a new significance. “We are particularly proud of our achievements here as this was our first AI project, and it won’t be our last as we continue to refine our models and improve the quality of output,” Fadlil concludes. “This expansion not only reflects our values of openness but also respects and incorporates our cultural identities through language.”

In the next episode of The Learning Curve, we will head to Samsung R&D Institute Jordan to speak to the team who led Galaxy AI’s Arabic language project. Tune in to learn about the complexities of building and training an AI model for a language with diverse dialects.

Put simply, 3D audio makes listeners feel as if they are really in the middle of the action. Typically only supported at specific venues such as movie theatres or recording studios, 3D audio is making its way into homes to give viewers a new way to experience their favourite content.

Samsung Electronics’ advanced research institute Samsung Research has been striving to popularise 3D audio since 2020. To this end, Samsung worked with Google to jointly develop Immersive Audio Model and Formats (IAMF) — an advanced 3D spatial audio technology that was adopted by the Alliance for Open Media (AOM)^[1] in October 2023. Samsung Newsroom sat down with the team that developed IAMF to hear the story of how the technology came to life.

▲ The Visual Technology Team at Samsung Research led the development of a standard for audio technology. (From left) SungHee Hwang, JeongHoon Park and WooHyun Nam

A New Way To Enjoy 3D Audio at Home

Ambient noise exists everywhere — from the sound of shoes scraping on the sidewalk to the gentle hum of cars driving down the road. While this noise may seem unnecessary in media, it is vital in adding depth and a sense of realness. 3D audio works to seamlessly blend ambient noise with conversations and other sound effects to create a more vibrant and authentic entertainment experience.

“3D audio makes you feel as if you are really in the heart of the action by adjusting the strength, movement and vibrance of the sound,” said WooHyun Nam from the Visual Technology Team at Samsung Research. “In this way, viewers are able to enjoy a more full-bodied sound that captures the 3D aspects of the world around us.”

▲ SungHee Hwang from Samsung Research’s Visual Technology Team

Despite the advantages of 3D audio, it has been difficult to apply the technology to home audio devices due to technological limitations. “3D sound information from content cannot be interpreted properly by home audio systems such as TV speakers or sound bars, resulting in a slightly limited audio experience that lacks detail from the original content,” said SungHee Hwang from the Visual Technology Team at Samsung Research.

To resolve this, Samsung worked diligently with Google to develop an audio solution that would allow viewers to experience content audio as intended. “If the 3D audio data can be read by device manufacturers, they can adjust the sound in audio devices — allowing for immersive audio experiences with standard TV speakers or sound bars at home,” said Nam. “By adjusting the audio to match the home device environment, listeners can experience audio just as the creator intended without any distortions or loss in quality.”

A uniform standard is required to send and receive audio data smoothly between creators and device manufacturers. “Samsung and Google’s respective expertise in devices and content made the companies ideal partners to create IAMF technology,” said JeongHoon Park, Executive Vice President and Head of Visual Technology Team at Samsung Research. “By coming together to develop this unprecedented technology, we are paving the way for consumers to enjoy 3D audio in their homes.”

Three Characteristics of IAMF Technology: Vertical Audio, AI-Based Audio and Customised Audio

IAMF technology offers three distinct features that enhance the audio experience.

▲ WooHyun Nam from Samsung Research’s Visual Technology Team

1. Ability To Express Sound Vertically

Previous open-source audio codecs only supported horizontal sound expression. With IAMF technology, audio can now be expressed vertically so that sound is increasingly multi-directional. “IAMF makes sound more realistic, by allowing listeners to hear audio in front, behind or to either side and also above or below them,” said Nam. “As such, when IAMF technology is applied to home TV speakers and sound bars, it allows listeners to hear sounds such as birds flying over their head on their TVs at home.”

2. AI-Based Scene Analysis and 3D Audio Effects

IAMF utilises AI and deep-learning technology to analyse scenes and emphasise certain aspects of the content — adjusting audio levels for more enhanced sound throughout the viewing experience. “In TV and film, there are certain scenes where the soundtrack or background music is the main focus,” said Nam. “IAMF will balance the sound in these instances. Similarly, the technology will fine-tune audio when there is character dialogue to allow the listener to focus on the conversation.”

In addition, IAMF technology provides optimal sound despite changes in the device environment. “By adjusting the scene analysis audio data according to the device environment, IAMF technology enables listeners to enjoy the content’s original sound quality on standard home TVs,” he added.

3. Highly Customised Audio

Users will be able to freely adjust sound according to their preferences with IAMF technology. Whether viewers are looking to amplify sound effects from an action scene or enhance dialogue, IAMF gives them the flexibility to customise content audio for a more personalised experience.

▲ IAMF analyses data from content and allows viewers to adjust and customise audio settings. When watching a sports game, users can directly choose whether to emphasise the commentator’s voice or the sounds of the game itself.

3D Audio Across the Industry With Open-Source IAMF

Open source is crucial to creating a unified standard across the industry. IAMF is the first open source-based audio technology standard adopted by AOM — meaning both corporate and independent content creators across the industry can access the technology and expand its use.

“In order to allow people to freely create content with 3D audio technology, related technology needs to be open to all,” said Nam. “Providing a complete open-source framework for 3D audio, from creation to delivery and playback, will allow for even more diverse audio content experiences in the future.”

Similarly, Park highlighted how IAMF technology will have a large impact on the audio landscape moving forward. “Because we live in an era dominated by content creation, IAMF will help lead, expand and transform the 3D audio ecosystem,” he said.

Success Accomplished Through Cooperation

Research on IAMF began in 2020 and took nearly four years to complete. Through a combination of perseverance and hard work, the team was able to achieve success.

“The project involved many days of nonstop work, and there were times when we needed to work at night due to the time difference between our office and Google’s,” said Hwang. “Despite these challenges, I was proud in the end. It was a great opportunity to work with developers from other countries.”

The teams developed a sense of camaraderie along the way and built a solid foundation of trust and understanding. “A Google employee mentioned how this partnership with Samsung has been the most enjoyable collaboration he had completed so far,” Park recalled. “We were able to accomplish this project because our teams trusted each other. I am very proud of and thankful for our team members’ efforts and their cooperation.”

▲ JeongHoon Park, Corporate EVP and Head of the Visual Technology Team at Samsung Research

Additionally, Park praised each team member’s unique skills and expertise which helped contribute to the creation of IAMF. “WooHyun Nam was instrumental in proposing the technology development ideas while SungHee Hwang organised and documented our team’s ideas in a clear-cut manner,” he said.

A Vision for ‘Samsung Sound’

Upon its release, the Samsung Research team realised that the standardisation of 3D spatial audio marked a new era of sound technology.

“Thanks to IAMF, we can begin researching technologies that will open up a world of audio possibilities,” said Nam. In line with this, the Samsung Research team is currently developing an advanced version of IAMF technology that can be applied to different sectors such as mobile devices, the metaverse, video games and more.

▲ With their combined efforts and unrelenting persistence, the Visual Technology Team at Samsung Research was able to successfully create a new standard for 3D audio.

Given the success of IAMF, the Samsung Research team is motivated to create better audio technology for consumers. Each of the team members shared their aspirations for the future of 3D audio.

Nam touched on his desire to create audio technology that is more immersive. “I want to create a more advanced 3D audio technology that makes users feel like they are truly in the scene of a movie, TV show or live event,” he said. “I also hope to continue this research until 3D audio is applied to Samsung’s smartphones.”

More broadly, Hwang discussed how he hopes Samsung will be able to create unrivalled audio technology that will place it among the ranks of other leading audio companies. “My goal is to develop a technology that consumers will be able to easily distinguish as ‘Samsung Sound’ when they hear it,” he said. “I am optimistic that the IAMF standard is a stepping stone toward this dream.”

“My hope is that Samsung’s sound technology will enable consumers to enjoy an upgraded audio experience that is on par with that of current visual experiences,” said Park. “I hope that content creators will utilise Samsung’s audio technology to make 3D audio content accessible. Additionally, we aim to create a supportive environment that enables researchers to take on bold and exciting challenges such as expanding Samsung’s audio technology.”

Technological standards such as IAMF will make entertainment more immersive for viewers across the board. The Samsung Research team is paving the way for innovations that will change the future of audio.

^{[1] A non-profit industry consortium formed to develop and share technology for multimedia delivery, allowing creators to utilise the technology without worrying about cost. It is operated by an alliance of 38 companies including Samsung Electronics, Google, Amazon, Apple and Meta.}

Samsung Electronics announced that the company hosts its first-ever Samsung 6G Forum (S6GF) online today.

Under the theme of ‘The Next Hyper-Connected Experience for All.’, the forum is divided into two sessions and broadcasts live online via Samsung’s YouTube channel. Each session consists of talks by experts from academia and industry specializing in next-generation communications technologies, followed by a panel discussion.

“We envision that 6G will provide ultimate experiences for humans and everything through the next level of hyper-connectivity, and this idea serves as the foundation of our 6G vision,” said Sebastian Seung, President and Head of Samsung Research.¹ “We believe that it is the right time to start preparing for 6G. Shaping 6G will require many years, as we have seen with previous generations, and will need lots of discussion and collaborations among players in industry and academia.”

Morning Session — 6G Air Interface