Samsung’s ‘AI for All’ Vision Unveiled at CES 2024

Samsung outlines strategy for using AI technology to make the connected device experience safer, more inclusive, and more energy-efficient

Samsung Electronics Co., Ltd. today shared its vision for how artificial intelligence (AI) technology will enable people to experience their devices more intuitively and conveniently. Presenting these ideas at a press conference at CES® 2024, Samsung teamed up with key partners to outline the technology behind this vision and how new products and services use AI capabilities to make life easier.

Jong-Hee (JH) Han, Vice Chairman, CEO and Head of Samsung’s Device eXperience (DX) Division, opened the press conference by describing how AI will enable connected technologies to improve people’s daily lives. The strategy that he and others outlined focused on the role of AI in enhancing connected experiences that are both simple and useful. Throughout the press conference, Samsung presented a number of products and services that help achieve these aims.

“With the emergence of artificial intelligence, smarter, better experiences will redefine how we live,” Han said. “Samsung’s broad portfolio of powerful devices, along with the pursuit of open collaboration, will help bring AI and hyper-connectivity to all.”

AI-Enabled Visual Display Products and Digital Appliances Bring New Experiences to the Home

During the CES press conference, Jonathan Gabrio, Head of Connected Experience Center at Samsung Electronics America, outlined the ways in which AI technology is transforming how Visual Display products and Digital Appliances are used. For example, Samsung Neo QLED 8K QN900D boasts a built-in AI Processor, NQ8 AI Gen 3, equipped with an 8 times greater AI neural network and a two times faster NPU than its predecessor. Thanks to NQ8 AI Gen3 processor, Samsung Neo QLED 8K automatically upscales low-resolution content to provide users with near 8K quality viewing experience and sharpens fast-moving images via AI Motion Enhancer Pro. Samsung Neo QLED 8K also features Active Voice Amplifier Pro, which analyzes voice and background noise by AI to optimize your TV listening experience so you can enjoy various content as if you are in the front row of a stadium or movie theater. Samsung Neo QLED 8K also features Tizen OS Home, which offers the latest apps, services, personalized content, and service recommendations to upscale your entertainment experience.

As for viewers with different needs, Samsung also provides innovative accessibility features. A sign language feature in Samsung Neo QLED can be easily controlled with gestures for the hearing impaired, and an Audio Subtitle feature turns text subtitles into spoken words in real-time for the visually impaired[1].

Furthermore, Samsung is introducing the new The Premiere 8K, a projector that features not only a 150-inch display, but also the world’s first wireless 8K transmission for a projector. With The Premiere 8K, users will be able to enjoy a full-scale cinema-like viewing experience in the comfort of their own homes.

As for sound, Samsung also introduced Music Frame, a speaker with a customizable cover that integrates seamlessly into users’ homes. Via Q-Symphony feature, the Music Frame syncs with Samsung TVs and soundbars, bringing enriched bass and surround sound of dual built-in woofers to the home.

Samsung has also made major upgrades to Ballie, the rolling AI robot first introduced at CES 2020. Ballie has now evolved into your AI companion capable of interacting with other smart devices to provide customized services such as taking care of bothersome tasks or projecting images and videos on walls so that users can view key information for their daily lives such as weather or other relevant content wherever they are.

To enhance kitchen and food experiences, Samsung is unveiling the Bespoke 4-Door Flex™ Refrigerator with AI Family Hub™+[2] [3]. This refrigerator boasts a 32-inch screen that features the new AI Vision Inside[4], which uses an internal camera to recognize up to 33 different food items that are put into the refrigerator, and suggests recipes using those ingredients. Users can also set “use by dates” on the Bespoke 4-Door Flex™ Refrigerator with AI Family Hub™ screen for food items, which then allows the refrigerator to send a notification when the set date draws near.  These AI features are designed to assist users to reduce food waste, to encourage more environmentally conscious lifestyles. The Anyplace Induction range takes AI connectivity a step further, as recipes saved on Samsung Food can be shared to the cooktop’s 7-inch display so that users can easily follow along while cooking.

Samsung’s AI technology extends to appliances beyond the kitchen, as well. The new Bespoke AI Laundry Combo™ — an All-in-One washer and dryer — features the AI Hub[5], which is a 7-inch LCD display that acts as an intuitive control center for managing laundry, allowing users to enjoy a simple washing and drying experience. It personalizes washing and drying by remembering users’ habits and using machine learning to suggest cycles. Meanwhile, Samsung’s upcoming vacuum and mop robot cleaner, the Bespoke Jet Bot Combo™[6], utilizes AI technology to provide a more convenient, effective cleaning experience. With AI Object Recognition upgraded from the previous robot cleaner lineup, it will be able to distinguish more objects, as well as detect stains and spaces[7]. The Bespoke Jet Bot Combo™ also recognizes the type of floor it encounters — along with carpet length — and adjusts its settings accordingly.

Samsung Reveals Galaxy Book4 as Company’s Most AI-Ready Laptop Series Yet

Samsung has been elevating what a PC can do and has been collaborating with Microsoft to create new, seamless connected experiences. Now, these experiences are being advanced further with new connectivity features that make the Galaxy Book4 series smoother, smarter, and more adaptable than its predecessors, reinforcing Samsung and Microsoft’s commitment to providing the best PC experience possible.

Using intelligent, intuitive AI capabilities, Microsoft Copilot[8] connects the Galaxy Book4 series with Samsung Galaxy smartphone to make them work seamlessly together, so that it feels like just one device. Microsoft Copilot can find, read or summarize text messages from a user’s Galaxy smartphone and even automatically create and send messages on a user’s behalf directly from the PC. Without having to open each individual app, the Galaxy Book4 series can access smartphone functions and information quickly and intelligently.

The Galaxy Book4 series can now turn Samsung Galaxy smartphones’ powerful camera into a PC webcam[9], using it for video calls on virtual conferencing apps. Users can switch freely between front and rear cameras with just a click on their PC, and can also access a variety of features from Galaxy smartphones – including Background Blur and Auto-framing – that help focus on the speaker’s image in various environments.

Through ongoing collaboration between Samsung and Microsoft, more intelligent capabilities will continue to be made possible on the Galaxy Book4 series, with the above connectivity features being available starting from March.

Plus, the Galaxy Book4 series can be paired with other Galaxy devices beyond smartphones. Tab S9 Ultra can be used as an additional PC monitor, and users can enjoy clear sound quality with reduced latency when the PC is connected with Galaxy Buds2 Pro. With all its capabilities, the Galaxy Book4 series is designed to deliver an incredible PC experience.

Better Connecting Users and Devices through Spatial Intelligence

Jaeyeon Jung, Executive Vice President and Head of SmartThings, highlighted the exciting new ways in which AI technology is impacting the relationship between users and their devices, including Samsung’s vision for SmartThings: The more Samsung devices a customer uses, the smarter those devices become and the better they can assist their users.

Spatial AI is the key to enabling this vision, by helping devices understand the living space and routines of the user, thus enabling more a personalized home management experience. SmartThings uses LiDAR[10] on connected devices like robot vacuums to create digital floor plans so users can easily check the status and location of all their devices. And in March, SmartThings will roll out an upgraded 3D Map View for a more detailed experience on Samsung smartphones and TVs. Furthermore, a simple QR code will allow users to add family and friends to their SmartThings ecosystem, so that members of the household can create their own routines.

Also, thanks to the Galaxy SmartTag2, users can track their pets’ whereabouts and digitally store identifying information by attaching the tag to a collar or harness[11].

Another point EVP Jung discussed was how the Bixby voice assistant is allowing smart homes to become more dynamic with its enhanced AI capabilities. Bixby can now automatically route commands to the most appropriate devices based on an understanding of the user’s location and activities.  Key technologies like Multi Device Wakeup combined with a shared communication protocol, enable Bixby to listen to commands with the devices in a room but then only perform the requested action on the most relevant device. So if a user is in the kitchen watching a recipe video on their Family Hub™, and they can call out, “Hey, Bixby! Play music!”  At that point, music will play from the user’s smart speaker, but the recipe video will continue to play on the Family Hub™, uninterrupted.

The TV, with the Samsung Daily+ service, will increasingly become a central hub for much of this smart home functionality, while also granting access to a variety of additional services. This include video calls through Workout Tracker — where workout data can be transferred to Samsung’s smart watches. Samsung also announced the new Daily Board service, which displays useful information about users’ homes, such as the weather or a live security camera feed from a home security system.  Similar to SmartThings features, users can also access this information via voice command.

Bringing the Home and Car Closer Together With Home-to-Car Capabilities

Samsung also announced developments in the automotive industry, starting with a partnership with Hyundai Motor Group.   Through this partnership users will have access to Home-to-Car and Car-to-Home services, courtesy of SmartThings connectivity. Haeyoung Kwon, Head of Infotainment Development Center and SDV[12] Execution Sub-Division at Hyundai Motor Group, delivered remarks regarding the partnership.

“Using Hyundai Motor Group’s software-defined vehicle, SmartThings will now work with Hyundai, Kia, and Genesis vehicles through the app,” Kwon said. “All of us at Hyundai Motor Group appreciate our partnership with Samsung, and we’re excited about the new features we’re bringing to your vehicles through smarter connectivity.”

Through SmartThings, users will be able to utilize voice commands to remotely perform functions in their car, such as heating it up in advance or opening and closing windows. SmartThings also enables  users to control home functions from their vehicle. Users will be able to automatically open and close garage doors based on the location of the car, as well as regulate home temperature. All of these actions can be carried out via voice commands.

Additionally, Samsung continues its longstanding partnership with HARMAN, a wholl- owned subsidiary of Samsung Electronics, to elevate the way users experience driving. Michelle Gattuso, Vice President of Product Management at HARMAN, discussed how the two companies are building new technologies to elevate the driving experience.

“Samsung’s leadership in consumer technology — together with HARMAN’s automotive expertise — has opened the door to game-changing experiences like HARMAN Ready Care, Ready Vision and Ready Display,” Gattuso said. “But now, the work we do with Samsung is getting even more integrated. Samsung and HARMAN are working in lockstep to bring even more impactful experiences to the entire in-cabin ecosystem.”

HARMAN’s Ready Careimplements deep learning neural networking techniques to monitor the driver’s movement and can provide personalized reminders and alerts. Additionally, Ready Care can now identify whether passengers are adults or children, and adjust airbag deployment settings accordingly.

Furthermore, Ready Vision brings an intuitive display directly to windshields — courtesy of augmented reality — in drivers’ line of sight. For example, drivers have access to a map that updates according to routes in real time, as well as recommendations for the fastest route, or where to stop for coffee. Ready Upgrade, allows automakers to frequently update their vehicle’s hardware and software.

Samsung Focuses on Security and Privacy Across Devices

In this future of AI-enabled, deeply interconnected devices, Samsung recognizes that security and privacy are paramount. Shin Baik from the Mobile Experience Security Team discussed the company’s security solutions as a foundation for the hyper-connected era.

One such solution is Samsung Knox Matrix, which now provides end-to-end encryption across many Samsung Galaxy smartphones and smart TVs. The technology will allow devices to monitor each other to identify and isolate security threats. Knox Vault, which helps keep user data secure on some of Samsung’s most popular devices, has been expanded to cover more devices connected by SmartThings, such as Samsung Neo QLED 8K TVs.

Samsung Makes Strides in Recycled Materials, Energy Management, and Accessibility

Meanwhile, Inhee Chung, Vice President of the Samsung Corporate Sustainability Center, highlighted Samsung’s plans for a more circular economy. As part of these plans, Samsung increasingly incorporates recycled materials into its products — including recycled plastic sourced from discarded fishing nets in Galaxy devices[13], recycled plastic in TVs, and recycled aluminum in Bespoke refrigerators[14].

As for the ways Samsung’s technology is designed to contribute to sustainability efforts, the company is utilizing AI to make the use of its products more energy-efficient through features like SmartThings AI Energy Mode, which automatically optimizes energy usage to help users monitor their energy usage, which can save energy

Thanks to a new collaboration with Tesla, SmartThings Energy is becoming even more useful both inside and outside of the home. The collaboration has integrated SmartThings Energy with Tesla products, including electric vehicles (EVs), the Powerwall home battery, Solar Inverter and Wall Connector EV charging solution.  Another benefit is that SmartThings Energy users can take advantage of the Tesla app’s Powerwall “Storm Watch” function, which flags severe weather through SmartThings not only on the Tesla app, but also on users’ connected Samsung TVs and mobile phones.  Users can also activate AI Energy Mode through SmartThings to help extend the usage time of backup energy stored in their Tesla Powerwalls.

Samsung is adding new features to increase device accessibility, such as digital appliances responding to hands-free gestures. Additionally, mobile devices provide auto-generated captions during calls, and TVs and displays offer a new, AI-enabled text-to-speech feature that converts subtitles into spoken audio. To further aid those with low vision, Relumino Together is a new viewing mode on Samsung TVs that enables users  to watch TV together, with or without visual adjustments.

*Official name of the service was updated at launch in March 22, 2024

 —

[1] This feature may not be available at the time of purchase or features/ language may vary by different country or region.

[2] Product name is for North America only. Product is referred to as “Bespoke 4-Door French Door Refrigerator with Family Hub™+” in other regions.

[3] Does not mean all services available on the Bespoke 4-Door Flex™ Refrigerator with AI Family Hub™+ are AI or generate information or outcomes using AI. The services/features that use deep learning models are limited to 1) AI Vision Inside and 2) Bixby features.

[4] As of March 2024, AI Vision Inside can recognize 33 food items like fresh fruits and veggies. If the food is covered by hand and not recognizable, it may be listed as an unknown item. The camera cannot identify or list any food items in the fridge door bins or freezer. Detecting and identifying food items are based on our deep learning model trained using predefined set of data and may yield incomplete or incorrect information. New datasets may be introduced to our learning model from time to time to enhance its accuracy.

[5] Does not mean all services available on the AI Hub are AI or generate information or outcomes using AI. The services/features that use deep learning models are limited to 1) AI Smart Dial 2) AI Opti Wash & Dry and 3) Bixby features. The learning models may be updated time to time and may yield incomplete or incorrect information.

[6] Currently in development and subject to change.

[7] The upgraded AI Object Recognition is expected to be available in the 2^nd half of 2024.

[8] Users must link their phone to Galaxy Book4 through ‘Link to Windows’ on mobile and the Microsoft Phone Link App on PC and follow the set-up prompts. Galaxy smartphone must be OneUI 1.0 or above. Requires the same MS account to be used both on PC and Phone. Timing of feature and availability may vary by market and device.

[9] Users must link their phone to Galaxy Book4 through ‘Link to Windows’ on mobile and the Microsoft Phone Link App on PC and follow the set-up prompts. Galaxy smartphone must be OneUI 1.0 or above. Requires the same MS account, Wi-Fi network connection to be used both on PC and Phone. Availability may vary by device.

[10] Light Detection and Ranging (LiDAR) is an optical technology used for distance sensing.

[11] Contact information and message shown on Lost Mode must be registered beforehand on the SmartThings application, along with user registration by email.  The SmartTag2 finder’s device must support NFC reading and be connected to the internet.

[13] Recycled plastic sourced from discarded fishing nets are incorporated into the Samsung Galaxy S22 series, Galaxy S23 series, Galaxy S23 FE, Galaxy Z Flip4, Galaxy Z Fold4, Galaxy Z Flip5, Galaxy Z Fold5, Galaxy Tab S9 Series, Galaxy Tab S9 FE series, Galaxy Book3 series, Galaxy Book4 series, Galaxy Buds2 Pro, Galaxy Buds FE and more.

[14] Recycled Aluminum is incorporated into BESPOKE refrigerator door panels for models including RQ38B9981APG, RR40B9971APG, RR40B99C5AP/**, RW33B99*****, RW33B99C5TF/**, RZ38B98*****, and RZ38B98C5AP/**