Unlike server-based systems, on-device environments operate under strict memory and computing constraints. As a result, reducing AI model size and maximizing runtime efficiency are essential. To meet this challenge, Samsung Research AI Center is leading work across core technologies — from model compression and runtime software optimization to new architecture development.

Samsung Newsroom sat down with Dr. MyungJoo Ham, Master at AI Center, Samsung Research, to discuss the future of on-device AI and the optimization technologies that make it possible.

The First Step Toward On-Device AI

At the heart of generative AI — which interprets user language and produces natural responses — are large language models (LLMs). The first step in enabling on-device AI is compressing and optimizing these massive models so they run smoothly on devices such as smartphones.

“Running a highly advanced model that performs billions of computations directly on a smartphone or laptop would quickly drain the battery, increase heat and slow response times — noticeably degrading the user experience,” said Dr. Ham. “Model compression technology emerged to address these issues.”

LLMs perform calculations using extremely complex numerical representations. Model compression simplifies these values into more efficient integer formats through a process called quantization. “It’s like compressing a high-resolution photo so the file size shrinks but the visual quality remains nearly the same,” he explained. “For instance, converting 32-bit floating-point calculations to 8-bit or even 4-bit integers significantly reduces memory use and computational load, speeding up response times.”

A drop in numerical precision during quantization can reduce a model’s overall accuracy. To balance speed and model quality, Samsung Research is developing algorithms and tools that closely measure and calibrate performance after compression.

“The goal of model compression isn’t just to make the model smaller — it’s to keep it fast and accurate,” Dr. Ham said. “Using optimization algorithms, we analyze the model’s loss function during compression and retrain it until its outputs stay close to the original, smoothing out areas with large errors. Because each model weight has a different level of importance, we preserve critical weights with higher precision while compressing less important ones more aggressively. This approach maximizes efficiency without compromising accuracy.”

Beyond developing model compression technology at the prototype stage, Samsung Research adapts and commercializes it for real-world products such as smartphones and home appliances. “Because every device model has its own memory architecture and computing profile, a general approach can’t deliver cloud-level AI performance,” he said. “Through product-driven research, we’re designing our own compression algorithms to enhance AI experiences users can feel directly in their hands.”

The Hidden Engine That Drives AI Performance

Even with a well-compressed model, the user experience ultimately depends on how it runs on the device. Samsung Research is developing an AI runtime engine that optimizes how a device’s memory and computing resources are used during execution.

“The AI runtime is essentially the model’s engine control unit,” Dr. Ham said. “When a model runs across multiple processors — such as the central processing unit (CPU), graphics processing unit (GPU) and neural processing unit (NPU) — the runtime automatically assigns each operation to the optimal chip and minimizes memory access to boost overall AI performance.”

The AI runtime also enables larger and more sophisticated models to run at the same speed on the same device. This not only reduces response latency but also improves overall AI quality — delivering more accurate results, smoother conversations and more refined image processing.

“The biggest bottlenecks in on-device AI are memory bandwidth and storage access speed,” he said. “We’re developing optimization techniques that intelligently balance memory and computation.” For example, loading only the data needed at a given moment, rather than keeping everything in memory, improves efficiency. “Samsung Research now has the capability to run a 30-billion-parameter generative model — typically more than 16 GB in size — on less than 3 GB of memory,” he added.

The Next Generation of AI Model Architectures

Research on AI model architectures — the fundamental blueprints of AI systems — is also well underway.

“Because on-device environments have limited memory and computing resources, we need to redesign model structures so they run efficiently on the hardware,” said Dr. Ham. “Our architecture research focuses on creating models that maximize hardware efficiency.” In short, the goal is to build device-friendly architectures from the ground up to ensure the model and the device’s hardware work in harmony from the start.

Training LLMs requires significant time and cost, and a poorly designed model structure can drive those costs even higher. To minimize inefficiencies, Samsung Research evaluates hardware performance in advance and designs optimized architectures before training begins. “In the era of on-device AI, the key competitive edge is how much efficiency you can extract from the same hardware resources,” he said. “Our goal is to achieve the highest level of intelligence within the smallest possible chip — that’s the technical direction we’re pursuing.”

Today, most LLMs rely on the transformer architecture. Transformers analyze an entire sentence at once to determine relationships between words, a method that excels at understanding context but has a key limitation — computational demands rise sharply as sentences get longer. “We’re exploring a wide range of approaches to overcome these constraints, evaluating each one based on how efficiently it can operate in real device environments,” Dr. Ham explained. “We’re focused not just on improving existing methods but on developing the next generation of architectures built on entirely new methodologies.”

The Road Ahead for On-Device AI

What is the most critical challenge for the future of on-device AI? “Achieving cloud-level performance directly on the device,” Dr. Ham said. To make this possible, model optimization and hardware efficiency work closely together to deliver fast, accurate AI — even without a network connection. “Improving speed, accuracy and power efficiency at the same time will become even more important,” he added.

Advancements in on-device AI are enabling users to enjoy fast, secure and highly personalized AI experiences — anytime, anywhere. “AI will become better at learning in real time on the device and adapting to each user’s environment,” said Dr. Ham. “The future lies in delivering natural, individualized services while safeguarding data privacy.”

Samsung is pushing the boundaries to deliver more advanced experiences powered by optimized on-device AI. Through these efforts, the company aims to provide even more remarkable and seamless user experiences.

Located in the heart of Chelsea, the New York AI Center will be led by Daniel D. Lee, Executive Vice President of Samsung Research and a global authority in AI robotics, who joined Samsung Electronics last June. With cooperation from a leading authority in neuroscience-based AI technologies, H. Sebastian Seung, Executive Vice President of Samsung Research, the New York AI Center will spearhead the advanced AI research in robotics. As the Chief Research Scientist of Samsung Electronics, Seung will also advise Samsung on advanced AI research in developing future business growth opportunities.

“What we need now is to focus on creating new values that make people’s lives easier and more convenient by harnessing the power of AI in Samsung’s products and services,” said Hyun-suk Kim, President and Head of Samsung Research, the advanced R&D arm of Samsung Electronics’ device business. “To do this, our Global AI Centers, including the New York AI Center, must play a pivotal role.”

Samsung has announced plans to expand its advanced AI research capabilities to employ about 1,000 specialists by 2020. This will be Samsung’s sixth AI center around the world, and it will work in partnership with Samsung’s other AI research facilities. The additional AI centers are located in Korea, the U.K., Canada, Russia, and Silicon Valley, the U.S.

Daniel D. Lee, the Executive Vice President of Samsung Research, commemorates the AI Center on September 7, 2018 at the opening ceremony of the facility in New York City.

“We are excited to open a new Samsung AI center in New York, which will specialize in robotics research,” said Daniel D. Lee. “New York is one of the world’s great cities, and with this new facility we will be able to leverage the tremendous talent in the area. We also look forward to collaborating with top universities and academic centers in the region.”

The facility is located at 123 West 18^th St, New York. The facility’s opening ceremony was held on September 7^th in New York City, with about 100 guests in attendance including President Kim; Seunghwan Cho, Executive Vice President of Samsung Research; Geunbae Lee, Head of Samsung AI Center in Seoul; Professor Daniel P. Huttenlocher, the Dean of Cornell Tech; and Professor David Tank, Director of Princeton Neuroscience Institute.”

“The field of AI has made revolutionary progress by finally embracing neural networks,” said H. Sebastian Seung. “This is just the beginning of a new era of innovation in AI, and we at the Samsung AI research center in New York are proud to be part of this exciting quest.”

The global AI Centers will contribute to Samsung’s AI research with their unique regional strengths, and Samsung is expected to expand its AI centers to other technology and talent-rich areas to cement its place as a leader of AI research in the industry.

Attendees at the opening ceremony of Samsung’s AI Center in New York City come together to take a commemorative picture.

For more details on Samsung Research and the global AI Centers, please visit the official Samsung Research webpage at http://research.samsung.com.

At Samsung Research, Drs. Seung and Lee will play a central role in building up fundamental research on AI that will advance human knowledge with the potential for revolutionary business impact. “Samsung is a company with a long history of pursuing innovation, and is committed to tapping the full potential of artificial intelligence,” said Dr. Seung. “I look forward to working at Samsung to help discover what lies ahead in AI.”

DR. Sebastian Seung

Now an eminent computational neuroscientist, Dr. Seung originally studied theoretical physics at Harvard University. Before joining Princeton University in 2014, he worked as a researcher at Bell Labs and a professor at the Massachusetts Institute of Technology (MIT). He serves on the Advisory Committees of the Pan-Canadian Artificial Intelligence Strategy and the Canadian Institute for Advanced Research (CIFAR) program on Learning in Machines and Brains. He is also an External Member of the Max Planck Society, the winner of the 2008 Hoam Prize in Engineering, and the author of Connectome.

DR. Daniel Lee

Likewise, an authority in AI and robotics, Dr. Lee earned his bachelor’s degree in physics from Harvard University and his Ph.D. from MIT. After working as a researcher at Bell Labs, he joined the School of Engineering and Applied Science at the University of Pennsylvania in 2001. Lee is a Fellow of the Association for the Advancement of Artificial Intelligence (AAAI), a Fellow of the Institute of the Electrical and Electronics Engineers (IEEE), and on the Executive Board of the Neural Information Processing Systems (NIPS) Foundation, which runs the premier machine learning conference in the world.

“I am eager to be joining Samsung Research and to help develop next-generation technologies for Samsung Electronics,” said Dr. Lee. “Fundamental research and understanding of machine learning and robotic systems will be key to fulfilling the promise of AI.”

Drawing inspiration from the brain, the two researchers jointly developed algorithms for machine learning by nonnegative matrix factorization. Later on, Dr. Seung devised an electronic circuit modeled on the brain’s cerebral cortex and featured on the cover of the journal Nature, published one of the first walking robots with reinforcement learning, pioneered the application of convolutional networks to image segmentation, and helped found the field of connectomics that reconstructs the brain’s wiring diagrams with AI.

Dr. Lee has developed a number of leading machine learning algorithms in addition to cutting-edge robotic systems throughout his career. He has pioneered innovative algorithms for unsupervised and reinforcement learning which draw inspiration from the brain’s neural circuitry. He has also led research teams to build advanced intelligent robots for a variety of tasks, including self-driving cars, humanoid robots, and collaborative robot teams.

Samsung Research, which reorganized as an advanced R&D hub of Samsung Electronics’ SET Business last year, recently established global AI Centers in five countries including Korea, the U.S., the U.K., Canada and Russia. Leading the latest effort, Samsung Research plans to continuously increase its number of AI Centers and advanced researchers to expand its R&D on AI platform.

The new Center’s main research areas will be computer vision and basic algorithms for AI platform. The Center will also expand the field of AI for key areas such as robotics, intelligent driving assistance, as well as those for future projects by Samsung.

The Samsung AI Center in Russia will be led by Professor Dmitry Vetrov of the Higher School of Economics (HSE). With a Ph.D. degree in Physical and Mathematical Sciences, Vetrov is also the Head of Samsung’s laboratory at the Center for Deep Learning, and Bayesian Methods Research Group at the HSE.

Professor Dmitry Vetrov, the leader of Samsung AI Center Moscow, is delivering an introductory speech at the opening, May 29, 2018.

As the leader of the Center, Professor Vetrov will combine scientific work and administrative activities: interacting with Samsung’s divisions and third-party institutions, organizing the Center’s overall work, managing work groups, as well as controlling and participating in scientific research. Professor Viktor Lempitsky of the Skolkovo Institute of Science Technology will also join the team as the leader of the research group.

“Samsung has always been the first to introduce new products and solutions that change the way people interact. Considering Russia as one of the world’s biggest hubs in the field of technical sciences, it is only natural that we chose the country as one of the sites for our new AI Center,” said Ultack Kim, President of Samsung Electronics Headquarters in CIS countries. “A team of the world’s best scientists and IT specialists at the AI Center will help Samsung bring its robotics and AI technologies to a whole new level.”

While there currently are several joint AI laboratories at Moscow State University, the Higher School of Economics and the PDMI RAS, the new Center will establish additional joint labs with Russia’s leading universities. In addition, Samsung will conduct projects with the local universities and different regions of Russia including Kazan, Samara, Rostov-on-Don, Tomsk, and Novosibirsk. Considerations regarding cooperation with Russian start-ups to solve practical problems are also under discussion. Moving forward, these can be developed into full-fledged services related to AI and machine learning, as well as promising developments in the field of applications and components for the company’s products.

“Samsung plans to introduce AI technologies to all of its connected devices and services by 2020. AI will enhance our customer value by offering information and services under any circumstances,” commented Jin Wook Lee, the Head of Samsung R&D Institute Russia, at the opening of the new Center. “This will be a huge step for the world of technology, and will help simplify the implementation of everyday tasks,” he added.

As the popularity of Internet of things (IoT) devices grows, the area of application of AI-based solutions is expanding rapidly. According to the forecast of Samsung experts, IoT devices with built-in AI will generate enormous amounts of data in the coming years. By processing this information, the devices will be able to provide maximum personalization and full compliance with the users’ needs.

“Currently, AI is one of the most promising branches of technology. The opening of the Samsung AI Center in Russia will allow us to contribute to the development of the industry and to apply the achievements of the Russian mathematical school, which has a high level of practical training of research specialists,” said Professor Vetrov.

Earlier in May, Samsung has opened two new AI Centers in Cambridge (UK) and Toronto (Canada). More information about Samsung AI Center in Russia and global AI Centers is available at Samsung Research website.

Located in Toronto’s downtown core at MaRS Discovery District, the new Samsung AI Centre will contribute to building the connected future by accelerating the adoption of intelligence on multiple devices ranging from household appliances to cars. The Toronto AI Centre is a part of a network of research Centres dedicated to research and development in the field of AI. The Centre is the second Samsung AI Centre to be established in North America, with the other in Mountain View, California. The North America AI Centres are led by senior vice president, Dr. Larry Heck, a renowned expert in machine learning for spoken and text language processing, who also co-leads the expansion of Samsung’s AI Centres around the globe.

“Toronto and the GTA are epi-centres of machine learning and one of the world’s foremost hubs for AI research and development. Home to not only world-class talent, but also some of the most innovative start-ups in the artificial intelligence field,” said Dr. Larry Heck, Co-Head of Global Artificial Intelligence Research. “We are looking forward to contributing to this vibrant AI community and to help push the field forward as we move towards our goal of making all Samsung connected devices intelligent by 2020.”

With a rich history in innovation and discovery, Toronto serves as an ideal place for research and development for speech recognition, where machine-learning technology was applied many years before it was widely applied to other fields. The vision is that the Samsung AI Centre will now serve an important role in the advancement of AI with a focus on language understanding and computer vision technologies that will ultimately reduce the friction between the user and the device/service, whether it be mobile phones, TVs, appliances, or cars.

The Toronto centre will be led by Dr. Sven Dickinson, newly appointed as the head of the Toronto lab, professor on leave and past chair of the Department of Computer Science at the University of Toronto. Dr. Dickinson is an expert in computer vision technologies, especially in the field of object recognition. He will play an integral part in Samsung’s research of core AI technologies that entail language, vision and other multi-modal interactions.

Hyun-suk Kim, President and Head of Samsung Research (front row, third from left), and Dr. Sven Dickinson, Head of Samsung’s new, Toronto-based AI Centre (front row, far right), pose for a photo with guests at the facility’s opening ceremony on May 24.

Samsung will officially celebrate today’s announcement of the Toronto AI Centre with an opening ceremony and presentations at MaRS Discovery District from key Samsung business leaders and special guests including Samsung’s CEO of Consumer Electronics,  Hyun-suk Kim, the Honourable.

François-Philippe Champagne, Minister of International Trade, Dr. Larry Heck, Senior Vice President of Samsung Research America, and key stakeholders and partners from major universities in Canada.

“Canadian ingenuity and innovation are getting noticed and together with our investments in skills and superclusters focused on next generation products like artificial intelligence, the world’s investors are choosing Canada. Congratulations to Toronto and to Samsung for recognizing Canada is the place to be!” said Honourable François-Philippe Champagne, Minister of International Trade.

Key contributing factors for the location selection include the availability of key AI talent, including leading AI researchers at the University of Toronto and the University of Waterloo who have had longstanding relationships with Samsung. Establishing an AI Centre in Southern Ontario also enables Samsung to better collaborate with regional start-ups and expand the current ecosystem. As one of the world’s largest urban innovation hubs, MaRS Discovery district supports promising innovators and ventures tackling key challenges in the sectors of cleantech, finance & commerce, and work & learning. In addition, and importantly, the vast MaRS community fosters cross-disciplinary collaboration which drives breakthrough discoveries and a new solution for global audiences.

“We are proud to welcome Samsung to MaRS,” said Yung Wu, CEO, MaRS Discovery District. “We both recognize the important role Toronto – and Canada – play as a destination for innovation talent, research, and start-ups. We look forward to collaborating and connecting with the Samsung team on a variety of exciting opportunities as this project comes to life.”

Today’s announcement compliments earlier 2018 news of plans to launch additional AI centres in North America. Dr. Darin Graham will spearhead the opening of new labs in Canada as the head of Samsung’s Canadian AI Operations. Until recently, Dr. Graham helped lead the creation and formation of the Vector Institute – the renowned Canadian AI research institute, as a member of the founding team. The opening of AI centres in Canada will allow Samsung to expand its outpost for industry collaboration and talent recruitment in the major AI hubs in North America.

“Investments like this are proof that Canada is truly a world-leader in artificial intelligence. Our talent and innovation ecosystem make us an attractive place for companies to grow and create well-paying jobs for Canadians. Our government is proud to partner with companies like Samsung, as we continue advancing our machine learning knowledge and further establish our place on the world stage as an AI leader,” said the Honourable Navdeep Bains, Minister of Innovation, Science and Economic Development.

To date, Samsung has had great success in leveraging Canada’s unique R&D talents for global impact. The Company’s Vancouver-based R&D centre has contributed to a number of in-market innovations and more than doubled its workforce, since opening with over 100 employees. With the addition of the AI centre in Toronto, the company plans to increase the R&D in Canada from current 100 to 200 in the near future. Additional developments and talent in Canada have been recognized through Samsung Electronics Canada subsidiaries, AdGear Technologies Inc. in Montreal and SigMast Communications Inc. in Halifax, Nova Scotia.

Samsung Research has led the latest effort in expanding the global AI research capabilities of Samsung Electronics and has converged its AI expertise with the creation of the new AI Centers. Samsung Research also oversaw the creation of the new AI Centers in Seoul last November and in Silicon Valley in January. With the addition of the three new AI Centers, Samsung Research will operate five AI Centers around the world.

Also, by securing global AI experts, Samsung Research plans to expand its number of advanced AI researchers to a total of about 1,000 globally by 2020.

“Samsung has a long history of pursuing innovation and we are excited to be bringing that same passion and technology leadership to AI,” said Hyun-suk Kim, President and Head of Samsung Research at the opening ceremony of the new AI Center in Cambridge. With the new AI Centers and recruitment of leading experts in the field, our aim is to be a game changer for the AI industry.”

Hyun-suk Kim, President and Head of Samsung Research at the opening ceremony of the new AI Center in Cambridge, is giving out a congratulatory address at the AI Center Cambridge, U.K. opening on May 22, 2018.

Top Talent from AI Field to Help Lead New Centers

The Samsung AI Center-Cambridge will be led by Andrew Blake, previously Director of Microsoft’s Cambridge Laboratory, and a pioneer in the development of the theory and algorithms that can make it possible for computers to behave as seeing machines. With cooperation from distinguished experts in the field, such as Professor Maja Pantić (Imperial College), Samsung AI Center-Cambridge will play a pivotal role in advanced AI research for Samsung Research.

Samsung AI Center-Cambridge will be led by Andrew Blake, pictured.

“This new Center signifies our commitment to the advancement of AI,” said Andrew Blake. “Our research will help us to better understand human behavior while exploring areas like emotion recognition, and further expand the boundaries of user-centric communication to develop AI technologies that ultimately improve people’s lives.”

Samsung AI Center-Toronto will be led by Dr. Larry Heck, Senior Vice President of Samsung Research America (SRA), and an expert in virtual personal assistant technologies and also the Leader of the SRA AI Center. The AI Center in Toronto will bolster its research efforts in core AI technologies with strategic cooperation from major universities in Canada.

Samsung AI Center-Moscow is expected to capitalize on Russia’s expertise in mathematics, physics, and other fundamental sciences. It plans to include leading AI experts such as Professor Dmitry Vetrov (Higher School of Economics) and Professor Victor Lempitsky (Skolkovo Institute of Science and Technology) to lead its research on AI algorithms.

Key Principles Driving Samsung’s AI Vision

In its vision for AI, Samsung focuses on five core aspects to drive its research. These include AI that is: “user-centric,” realizing personalization through a multi-modal interface; “always learning,” through continuous self-learning from data; “always there,” as an ambient service; “always helpful,” with minimization of user intervention and response to requests; and “always safe,” ensuring user safety and privacy. As they work around these focuses, the new AI Centers will leverage cutting-edge research environments along with the regional strengths of the researchers to carry Samsung’s AI capabilities to new heights.

Through ongoing initiatives including hosting an AI Forum last year, Samsung has been cooperating closely with renowned AI scholars in preparing to set up research bases across areas of expertise. Samsung will continue to expand areas of expertise and technology across the globe, including of particular importance, the eastern area of the U.S.

In the coming years, Internet of Things (IoT) devices embedded with AI will generate a vast array of data that can provide fascinating insights about the lives of users, enabling them to take advantage of intelligent services optimized for their own personal preferences and behaviors. How AI-enabled devices provide consumers with the most optimized options will be critical to the success of AI technology for the near future. To provide a user-centric ecosystem, Samsung aims to build an AI platform under a common architecture that will not only scale quickly but also provide the deepest understanding of usage context and behaviors, making AI more relevant and useful.

For more details on Samsung Research and the global AI Centers, please visit the official Samsung Research webpage at http://research.samsung.com.

Attendees at the AI Center in Cambridge, the U.K opening on May 22, 2018 are smiling for a group photo

We caught up with Geunbae Lee, the Head of the AI Center, to talk about how Samsung plans to lead the field of artificial intelligence.

Q. The integrated AI Center, with Samsung Research, was officially launched last November. What is your vision for the center?

With technology innovation rife within the AI sector, our vision is to prepare industry-leading AI-based solutions in the areas of recognition, thinking and movement to compete with other key market players by 2020. We will also enhance the competitiveness of our existing businesses. Furthermore, the AI Center will contribute to not only new business and product creation but also effective management, as we are applying AI throughout the manufacturing, marketing and data analysis processes.

Q. What projects are the AI Center currently working on?

In 2018, the first year of the AI Center, we plan to focus on establishing AI platforms which will serve as the foundation for Samsung’s AI technology. We will achieve this first by collecting AI research capacity from within Samsung and recruiting global talent. Our mid- to long-term goal is to show leadership in AI technology development along certain major themes including virtual assistants, robotics and data.

Q. What are your thoughts on how AI will progress in the future? How will Samsung lead the conversation within the industry?

Samsung has already demonstrated its leadership in the field of virtual assistants through Bixby. In order to maintain and expand this capability, we are going to not only develop innovative AI-related products and services, but also establish global AI hubs around the world. For example, we can cooperate with the world’s best universities and communicate with external industry talent through regular, open forums. By doing so, we are planning to take the world’s awareness of Samsung’s AI capabilities one step further.

Q. How is Samsung integrating AI into its various products, including home appliances (Chef Collection, Family Hub, Smart TV, etc.)? Any examples of how it is being used?

Bixby, which supported only Korean and English at the beginning, is now provided in Mandarin Chinese, and it will support more languages in the future. It is also innovating the user experience by connecting itself not only to smartphones but to multiple devices and services. Moreover, based on sensor data in appliances like air conditioners, unusual signals are caught and addressed before they develop into severe issues. This shows that through innovation, combined with existing devices and services, smart electronics are evolving into intelligent electronics.

Q. Samsung has AI research labs in Korea, Canada, the UK and Russia. How will you expand upon these labs and what kind of synergistic effect do you expect?

Samsung is conducting AI research based on local insights through 22 overseas research labs in 15 countries, including the US and the UK. Samsung is also planning to establish new hubs in areas with specialized AI capabilities and will focus on finding the best talent to increase competitiveness. By strengthening our global networks, Samsung will continue to build a single robust, flexible and expandable platform to integrate and optimally utilize AI technology. The AI Center will be at the center of this development by working closely with our global network based on each region’s individual competitive advantages and capabilities.