Central to Samsung’s AI innovations is on-device AI. This technology enables AI to operate independently within the device and does not require a server or the cloud. Users benefit from quick AI response times without needing a network connection, reducing concerns over personal data leakage.

On-device AI has now evolved beyond simple functions to handle demands requiring vast amounts of data — such as generative AI — using only the device’s internal resources.

Samsung Newsroom explores how Samsung is leading the world in on-device AI innovations.

No Cloud Required: Transforming the On-Device AI Experience

Citing the need for speed and security, much of the tech industry has focused on running complex generative AI in an on-device environment. With long-term investments that date back more than a decade, Samsung has already customized AI features for many of its devices and positioned itself front and center in providing on-device AI experiences for everyday life.

Samsung’s Galaxy AI, for example, offers on-device capabilities across the company’s latest mobile devices and tablets.¹ Now supporting a total of 20 languages,² Galaxy AI enables communication without language barriers including real-time conversation interpretation as well as message and web page translation.

For TVs, Samsung leverages its expertise as a global leader in the market to train AI models. Since 2020, the company has continuously enhanced its TV-specific processor with a built-in NPU (neural processing unit). The 2024 Neo QLED 8K model’s NQ8 AI Gen3 processor features 512 neural networks for enhanced image and sound quality. By analyzing and modifying pixels, frames and sound sources, the processor delivers superior upscaling, smooth motion and clear dialogue for a seamless viewing experience in all situations.

The key to on-device AI is minimizing model size while maintaining performance. To achieve this, Samsung has focused on efficient implementation while utilizing specialized AI training data tailored to specific tasks.

Smaller and Faster: Samsung’s AI Optimization Research

As AI technology advances and data processing requirements increase, model optimization has become crucial. In other words, high-performance AI models must be able to operate efficiently within the limits of the devices’ resources — including their processors, memories and batteries — while retaining performance levels and reliability for daily use. Key technologies for this include model compression, hardware optimization and data processing acceleration.

Samsung has secured various technologies in the field of model compression to reduce the size of AI models. The company has succeeded in developing light and fast AI models through techniques like quantization, which enhances response speed by simplifying algorithms and optimizing computation processes; pruning, which removes non-essential elements of a larger model; and knowledge distillation, which transfers knowledge from large models to smaller models.

Samsung also continues to innovate in hardware optimization and data processing acceleration to swiftly and efficiently run AI. For example, flash utilization technology partitions large AI models and significantly reduces memory usage. Additionally, the company has developed patented technology that is expected to assist quick inference on low-end devices without an NPU to broaden AI applications across various products. Further research is underway on increasing inference speeds with speculative decoding, a technology that predicts AI model outcomes, and enhancing computing power by running various hardware simultaneously.

End-to-End Expertise: From In-house Development to Implementation

With expertise spanning hardware, software, components and end products, Samsung is spearheading on-device AI innovation from research and development to product implementation.

From a device perspective, Samsung’s market leadership in mobile devices, TVs, home appliances and more represents its excellence in both hardware and software. Samsung integrates AI into its own devices, allowing the company to leverage both in-house development and open collaboration with industry partners to create hardware-optimized AI models.

Samsung’s expertise also includes system software to bridge hardware and application software. For example, the Tizen operating system and the NPUs on Samsung TVs power on-device AI for a smarter, more enhanced viewing experience. The company also provides developers with Vision AI and Language AI SDKs, as well as machine learning APIs to facilitate AI model training and inference.

Samsung Research, the company’s advanced research and development hub, is focused on enhancing competitiveness by utilizing specialized expertise from its global network of labs — from AI model structure development and model compression in the U.K. and U.S. to creating AI acceleration solutions in India and Poland.

From a components perspective, Samsung has endlessly driven progress in NPU performance as a leading manufacturer of semiconductors and plans to continue working with its partners to develop new ways of utilizing on-device AI.

With its on-device AI capabilities expected to expand across more products and services, Samsung is pioneering groundbreaking changes that will see AI technologies transform daily life. In part two of this series, Samsung Newsroom will explore the company’s innovations in personalized AI.

¹ Galaxy AI-supported devices include the Galaxy Z Fold6, Flip6, Fold5, Flip5, Fold4, Flip4, S24 series, S23 series, S22 series, S24 FE, S23 FE, Tab S10 series, Tab S9 series, Tab S8 series, etc.
² Galaxy AI-supported languages include Korean, English, Arabic, Simplified Chinese, Dutch, French, German, Hindi, Indonesian, Italian, Japanese, Polish, Portuguese, Romanian, Russian, Spanish, Swedish, Thai, Turkish and Vietnamese.

Samsung Electronics has transformed people’s lives by popularizing wall-mounted TVs and smart TVs. Long at the forefront of innovation in TV hardware and software, Samsung is now utilizing AI to overcome limitations in the viewing experience and define the next generation of screen experiences.

▲ SW Yong, President and Head of Visual Display Business, Samsung Electronics

How AI Screens are Changing Daily Life

In today’s hyperconnected era, the AI TV is set to become the hub of an AI-powered smart home system — connecting, managing and controlling devices to provide consumers with new levels of convenience for more valuable experiences. Serving as a command center for the AI home, the AI TV allows users to easily monitor and operate connected devices, which can be automatically integrated to the SmartThings ecosystem via Calm Onboarding technology.

To see appliances at a glance, users can simply check the Daily Board on their Samsung AI TVs. The large screen displays 3D Map View to show the status of devices scattered around the home. Meanwhile, energy consumption can be managed by remotely turning off the lights or air conditioner when left on. Cameras and sensors ensure the home is monitored from afar, and users can even play videos to keep their pets entertained.

▲The 8K NQ8 AI Gen3 Processor combines AI sound, AI picture and AI optimization features for a frictionless viewing experience

In addition, AI TVs are offering an ultra-large, high-resolution viewing experience to make users feel as if they are in first-row VIP seats at a stadium, movie theater or concert hall.

The Neo QLED 8K offers 8K AI Upscaling Pro so users can enjoy SD/HD/Full HD high-resolution images at 8K levels. AI Motion Enhancer Pro makes sports content feel even more real and immersive. Meanwhile, Active Voice Amplifier Pro helps users catch every word — from dialogue in a movie amid background music to a sportscaster’s commentary over cheers from a crowd. With these features, the world is entering the era of the AI home cinema and the AI home sports arena.

Furthermore, AI TVs will help make TV more accessible to all. Optical character recognition technology converts on-screen text to audio subtitles in real time for users who are hard of hearing. On the other hand, Relumino Mode configures contents on screen by amplifying contrast, brightness and contours for users with low vision.

AI Screens Lead the Evolution of AI Homes

AI TVs are making new experiences possible — experiences that would have been unimaginable with conventional TVs. To this end, AI homes will evolve with Samsung’s Tizen operating system.

With AI becoming a differentiator in the global home appliance market, operating systems are a key player when incorporating AI into TVs.

Since 2015, Tizen OS has powered hundreds of millions of smart TVs and led the evolution of TVs. Thanks to the platform’s excellent performance, Samsung expanded its capabilities from TV hardware to software.

▲The 8K NQ8 AI Gen3 Processor combines AI sound, AI picture and AI optimization features for a frictionless viewing experience

Equipped with SmartThings and Tizen OS, Samsung AI TVs will increasingly tailor their services to users’ tastes and lifestyles. Boundaries separating products from service platforms will crumble away to open the ‘Era of Device Platform Innovation.’ Continued upgrades to Tizen OS will ensure quality service in the future.

The key to these personalized, customized experiences offered by AI TVs is on-device AI — in other words, AI that is embedded in the TV with no need for a cloud connection. This technology is more secure than cloud-based AI because personal information wouldn’t be sent out to a server.

Above all, ‘Samsung Knox,’ Samsung’s proprietary security solution, will thoroughly protect devices and personal information so users can enjoy a safe AI home.

The Future of AI: What This Means for Consumers

The rapid evolution of AI raises many social, ethical, economic and legal concerns. But ultimately, AI will evolve in a way that provides greater value for consumers.

Samsung strives to create a new way of life with AI screens by understanding, responding and adjusting to users’ needs — for new screen experiences that were not possible before.

Whether breaking down language barriers during communication, sharing content recommendations or controlling home appliances, AI is already impacting all of our lives. Through continued improvements to its AI screens, Samsung will create a more valuable daily life in the AI home.

We are at the beginning of the most exciting and probably one of the most historic moments in technology. The era of mobile AI is here — which for Samsung opened up by the introduction of Galaxy AI features on the Galaxy S24 series, our first AI phone.

Revolutionizing Mobile Experiences With a Hybrid AI Approach

To bring the real-life benefits of generative AI technology to the Galaxy S24 series, we took a hybrid approach to our AI integration. AI offers so many possibilities, and we believe mobile devices are the primary access point for unlocking its potential as users around the globe continue to rely on their phones for most of their day-to-day needs.

The essential role that mobile devices play in everyday life means they are very personal items that people use during many critical and memorable moments. We understand these behaviors and believe smartphones should work harder so that users can truly make the most of important moments in a simpler, more intuitive and more epic way. We also recognize the importance of privacy which is why we give users full control over what they share and what they keep private.

We believe our hybrid approach is the most practical and reliable solution to meet all these needs and puts Samsung ahead of the curve. We are providing users with a balance between the instant responsiveness and extra privacy assurance of on-device AI and the versatility of cloud-based AI through open collaborations with industry-leading partners in offering a variety of functions they need for daily life.

Expanding Possibilities With On-Device AI

We made the Live Translate feature on-device AI based because voice calls are the most basic feature of a smartphone, yet also an intimate and private means of communication. Samsung has committed to empowering users to communicate without language barriers, while at the same time ensuring every communication is safe and private.

To make this feature a reality, our R&D team worked day and night as part of a collaborative process that truly brought out the best in our teams. From determining the right size of AI language models to training and testing in real-life scenarios, our R&D organizations within the MX business challenged themselves and pushed beyond the limits of what we thought was possible to get this feature running entirely on-device.

Samsung’s global R&D networks around the world, which are in charge of growing the best talents and developing core technologies in each region, have also played a pivotal role. Samsung Research Centers around the globe, including Poland, China, India and Vietnam, have dedicated themselves to developing and expanding the languages supported by Galaxy AI. Languages are very cultural, contemporary and localized, so to help people break down language barriers and communicate more naturally, the role of our local R&D offices is even more crucial in opening up new possibilities for users in more regions than ever before.

All these efforts have enabled Samsung to offer something completely new which I’m excited to share. Soon, Samsung is extending the power of Galaxy AI beyond Samsung’s own native calling app by expanding Live Translate to other third-party message apps to support voice calls. So you can stay in contact with friends or colleagues, communicating on your favorite apps in multiple languages.

And since this feature has been integrated into our on-device AI language translation model, users will be able to experience barrier-free communication without worrying about privacy issues like personal data being shared outside their phone when using Live Translate.

I expect that more and more mobile AI features will be incorporated inside mobile devices as the intelligent computing power of the latest chips — particularly NPUs — advances at a blazing fast speed, along with other technologies. This will enable more people to embrace AI and make every day more convenient while offering greater peace of mind.

Growing Galaxy AI for Tomorrow’s Needs, Today

This exciting news is all part of the extension of our commitment to Galaxy AI and entering the next phase of the mobile AI era. The introduction of Galaxy AI on the Galaxy S24 series was only the start. To deliver a completely new and unique AI experience, we will further optimize the Galaxy AI experience for the upcoming foldable devices. Our foldables are the most versatile and flexible form factor in Samsung Galaxy and when combined with Galaxy AI, these two complementary technologies will together unlock all new possibilities.

Samsung is also committed to expanding Galaxy AI experiences across the broader Galaxy ecosystem in a way that only Samsung can. As the mobile AI era moves forward at a rapid pace, Samsung is accelerating mobile AI innovations to meet the needs of not only today, but tomorrow.

Samsung Research combined data and cutting-edge technology. Together with the Mobile eXperience (MX) Business R&D Office, they further honed this technology to develop the translation features powered by on-device AI — which can be used for real-time translation during calls and across various applications. Samsung Newsroom met with Yoonjung Choi and Yonghyun Ryu from Samsung Research’s Global AI Center to learn more about these ambitious features.

▲ (From left) Yoonjung Choi and Yonghyun Ryu of Samsung Research’s Global AI Center

Smooth Communication and Strong Security With On-Device AI

On-device AI is the key differentiator to Galaxy AI’s Interpreter and Live Translate features — introduced to users through the release of the Galaxy S24 series earlier this year. Leveraging the advanced computing resources built into the devices themselves, smartphones with on-device AI can provide services without relying on servers or the cloud. Users can be assured that their data will remain private and secure since information is not shared with external sources.

Samsung Research’s Global AI Center contributed to integrating proprietary technology into its AI translation model to commercialize these features for widespread use.

Envisioning a wide range of applications, the Samsung Research team and MX R&D Office obtained an expansive amount of data. “We collected colloquial data for real-time translation during calls in Live Translate and travel-related data for Interpreter,” explained Yoonjung Choi, who led the project. “To provide the most accurate translation, we studied and incorporated casual language used in chatrooms and HTML tags used in web browsers.”

▲ Use cases for on-device AI translation include ① Live Translate, ② text message and conversation translation, ③ translation in Samsung Notes and ④ web browser translation.

How Samsung Research Trained Its AI Model

The Samsung Research team’s AI translation model is based on deep learning technology that learns from its own data. Yonghyun Ryu, who is in charge of AI research and development, likened this process to raising a child. “Similar to how a child needs excellent educational resources and caregivers to grow and thrive, good language data and talented researchers are required when developing a high-performance AI translation model,” he described.

Samsung Research has both — since 2013, the company has been providing in-house translation services, conducting R&D related to AI translation and accumulating high-quality data.

Samsung Research’s team of deep learning experts played an important role in training the AI model. “If incorrect translations occur during the research and development process, it is necessary to identify the problem and make improvements. However, this can be challenging and time-consuming for researchers without sufficient capabilities and experience,” he explained. “Our researchers used their expertise and know-how to quickly analyze the cause of the issue and come up with a solution to enhance the AI translation model.”

▲ Yonghyun Ryu from Samsung Research’s Global AI Center

To assess the performance of Galaxy AI’s translations, the Samsung Research team used quantitative metrics based on test sets as well as qualitative evaluations by human translators and the MX R&D Office.

In addition, the team gained credibility by competing in global machine translation competitions. “Although participation requires time and effort, good performance in competitions provides momentum for research and development,” Ryu emphasized. “We were able to achieve strong results because our team members could freely discuss new ideas and put them to the test.”

Politeness and Punctuation: Understanding the Quirks of Each Language

Each language has characteristics that are unique to its culture — these can include honorifics, tonal inflections and distinct punctuation symbols. To make translations as accurate as possible, the AI translation model considers all these linguistic idiosyncrasies. For example, in Korean and Japanese, honorifics are translated to maintain a respectful tone.

Samsung collaborated with regional R&D centers to fully understand languages in their cultural context. “By working closely with researchers and linguists in other countries, we were able to offer a more accurate and complete translation,” said Choi.

At the same time, dealing with different languages often involves trial and error. “Vietnamese, for instance, is a tonal language. However, we realized during the research process that Vietnamese users often omit tones in casual conversations when chatting,” she explained. “We needed additional data to help the features translate sentences without tones.” For Thai, Samsung Research developed a special sentence separator because the language does not use punctuation.

Why Lightweight Technology Is the Key to Effective On-Device AI Models

Samsung Research began developing its on-device AI translation model in 2019. “Unlike server-based AI models, on-device AI models must be driven only using users’ devices,” stated Ryu. “Developing lightweight technology that uses minimal resources is key.” To make the model lighter, the team used “knowledge distillation” and “quantization” technology.

Knowledge distillation is a method that extracts knowledge from a large, high-performing teacher model and delivers it to a smaller student model. This is similar to how a teacher summarizes a topic so that a student can digest the information more easily and efficiently.

Quantization simplifies AI algorithms to reduce model size and streamlines the process to increase response speed.

Ryu compared quantization to drawing strawberries. “You need a wide range of colors to paint lifelike strawberries — but strawberries can also be made with just red and green,” he explained. “Quantization is the process of minimizing the number of colors needed to draw strawberries while trying to make it as close as possible to the real thing.”

▲ Yoonjung Choi from Samsung Research’s Global AI Center

In the AI field, knowledge distillation and quantization are well-known approaches to making models lighter. Yet, implementing them on a commercial scale is not easy due to differences in each researcher’s detailed experimental methods and factors. Samsung Research developed proprietary technology by discovering an efficient quantization technique and creating a accelerated algorithm based on it. “Through constant experimentation, we found an optimal way to make the model lighter,” commented Choi.

By combining the high-quality AI translation model with algorithms that make models lighter and speed up response time, lightweight and fast on-device AI features were born.

The Culture Behind the Language: What Would the Perfect AI Translation Model Look Like?

The researchers at Samsung Research’s Global AI Center have bright goals as they lead the field of on-device AI. “My ultimate goal is to help users communicate smoothly and conveniently with people who speak other languages,” said Choi. Ryu revealed his vision to build the perfect translator. “One day, we want to create a translator that truly understands the cultures behind the languages it is translating, equipped with an extensive pool of knowledge,” he expressed. “I want to challenge myself to develop a translator the world has never seen before.”

▲ Samsung Research’s Global AI Center team developed the on-device AI translation feature.

Samsung’s on-device AI translation features allow anyone with an enabled mobile device to communicate freely — without worrying about internet connection or information leakage. Going forward, the Samsung Research team will continue to spearhead innovations in the rapidly evolving field of AI and bring new levels of convenience to users.

The second day of the event, hosted by Samsung Research, the advanced R&D hub of the company that leads the development of future technologies for Samsung Electronics’ Consumer Electronics division and IT & Mobile Communications division, facilitated discussion around how industry experts and academics alike can further research into AI technologies that directly impact and enhance the lives of all people. It was livestreamed on Samsung Electronics’ YouTube channel, providing opportunities for researchers and students in the AI field around the world to interact with world-renowned academics through Live Panel Discussions.

AI Forum Day 2: Exploring ‘AI’ in a Human World

Dr. Sebastian Seung, President and Head of Samsung Research, began his opening speech by emphasizing that “Artificial intelligence is revolutionizing every single R&D area of Samsung Research. AI technology was thought impossible, but is now becoming a reality and makes people’s lives better.”

During his speech, which introduced Samsung Research’s various areas of AI research, Dr. Seung explained how on-device AI technology is enabling smartphone cameras to offer users new ways to express their creativity and manage other devices such as TVs and air conditioners. He also provided insights into other applications for AI technology, which includes enabling robot vacuums to automatically create indoor 3D maps, detect obstacles and clean a space accordingly.

Dr. Seung concluded his speech by describing his excitement to see what Samsung’s researchers would be presenting and inviting participants to follow Samsung Research’s Twitter account to learn more about the innovative research that the R&D hub is conducting.

Seung’s speech was followed by presentations from world-renowned AI experts who are leading research in their respective fields.

First, Professor Leslie Valiant of Harvard University, the recipient of the 2010 Turing Award – often referred to as the Nobel Prize of computing – delivered the day’s keynote lecture, entitled “How to Augment Supervised Learning with Reasoning”. Using robust logic as an example, Professor Valiant emphasized that combining supervised learning and reasoning should be a key focus for the next generation of machine learning technology.

Professor Valiant’s keynote was followed by a presentation from Professor Felix Heide of Princeton University. Professor Heide’s presentation, entitled “The Differentiable Camera”, discussed the camera technology that utilizes deep learning to enhance image quality.

Next, research scientist Been Kim of Google Brain delivered a presentation entitled “Interpretability for Skeptical Minds”, in which she shared the latest advancements in interpretable machine learning and proposed directions that this cutting-edge technology should be heading.

The last session of the day was led by Professor Max Welling, a research chair in machine learning at the University of Amsterdam and a Distinguished Scientist at Microsoft Research. During his presentation, entitled “Understanding Matter With Deep Learning”, Professor Welling shared why he is so excited about the scientific breakthroughs that will come from utilizing deep learning in molecular simulation.

Other highlights of Day 2 included a Lightning Talks session, which saw engineers from Samsung Research’s Global AI Center present some of their latest research, and a live panel discussion that was moderated by Dr. Daniel Lee, Executive Vice President and Head of Samsung Research Global AI Center.

In case you missed it, you can watch a full replay of day two of the Samsung AI Forum 2021 by heading to Samsung Electronics’ YouTube channel.