Samsung’s ‘AI for All’ Vision Unveiled at CES 2024

Samsung outlines strategy for using AI technology to make the connected device experience safer, more inclusive, and more energy-efficient

Samsung Electronics Co., Ltd. today shared its vision for how artificial intelligence (AI) technology can help enable people to experience their devices more intuitively and conveniently than ever before. Presenting these ideas at a press conference at CES® 2024, Samsung teamed up with key partners to outline the technology behind this vision and how new products and services use AI capabilities to help make life easier.

Jong-Hee (JH) Han, Vice Chairman, CEO and Head of Samsung’s Device eXperience (DX) Division, opened the press conference by describing how AI will enable connected technologies to improve people’s daily lives while always remaining nonintrusive and “in the background.” The strategy that he and others outlined focused on the role of AI in enhancing connected experiences that are both simple and useful. Throughout the press conference, Samsung presented a number of products and services that help achieve these aims.

“With the emergence of artificial intelligence, smarter, better experiences will redefine how we live,” Han said. “Samsung’s broad portfolio of powerful devices, along with the pursuit of open collaboration, will help bring AI and hyper-connectivity to all.”

AI-Enabled Visual Display Products and Digital Appliances Bring New Experiences to the Home

During the CES press conference, Jonathan Gabrio, Head of Connected Experience Center at Samsung Electronics America, outlined the ways in which AI technology is transforming how Visual Display products and Digital Appliances are used. For example, Samsung Neo QLED 8K QN900D boasts a built-in AI Processor, NQ8 AI Gen 3, equipped with an 8 times greater AI neural network and a two times faster NPU than its predecessor. Thanks to the NQ8 AI Gen3 processor, Samsung Neo QLED 8K can automatically upscale content to provide users with up to 8K quality viewing experience and sharpens fast-moving images via AI Motion Enhancer Pro[1]. The Samsung Neo QLED 8K also features Active Voice Amplifier Pro, which analyses voice and background noise by AI to optimise your TV listening experience so you can enjoy various content as if you are in the front row of a stadium or movie theater. Samsung Neo QLED 8K also features Tizen OS Home, which offers the latest apps, services, personalized content, and service recommendations to heighten your entertainment experience[2].

As for viewers with different needs, Samsung also provides innovative accessibility features. A sign language feature in Samsung Neo QLED can be easily controlled with gestures for the hearing impaired, and an Audio Subtitle feature turns text subtitles into spoken words in real-time for those with low vision.

Furthermore, Samsung is introducing the new The Premiere 8K, a projector that features a 150-inch display, and wireless transmission. With The Premiere 8K, users will be able to enjoy a full-scale viewing experience like that of the cinema in the comfort of their own homes.

As for sound, Samsung also introduced Music Frame, a speaker with a customisable cover (sold separately) that can integrate seamlessly into users’ homes. Via Q-Symphony, the Music Frame syncs with compatible Samsung TVs and soundbars, bringing rich bass and surround sound of dual built-in woofers to the home with minimal effort.

Samsung has also made major upgrades to Ballie, the rolling AI robot first introduced at CES 2020. Ballie has now evolved into your AI companion capable of interacting with other smart devices to provide customised services such as taking care of bothersome tasks or projecting images and videos on walls so that users can view key information for their daily lives such as weather or other relevant content wherever they are[3].

To enhance kitchen and food experiences, Samsung is unveiling the Bespoke 4-Door Flex™ Refrigerator with AI Family Hub™+[4] [5]. This refrigerator boasts a 32-inch screen that features the new AI Vision Inside[6], which uses an internal camera to recognise up to 33 different food items that are put into and taken out of the refrigerator and suggests recipes using those ingredients. Users can also set “use by dates” on the Bespoke 4-Door Flex™ Refrigerator with AI Family Hub™ screen for food items, which then allows the refrigerator to send a notification when the set date draws near[7]. By conveniently helping to save money by reducing food waste. The Anyplace Induction range takes AI connectivity a step further, as recipes saved on Samsung Food can be shared to the cooktop’s 7-inch display so that users can easily follow along while cooking[8].

Samsung’s AI technology extends to appliances beyond the kitchen, as well. The new Bespoke AI Laundry Combo™ — an All-in-One washer and dryer — features the AI Hub[9][10], which is a 7-inch LCD display that acts as an intuitive control center for managing laundry, allowing users to enjoy a simple washing and drying experience. It can personalise washing and drying by remembering users’ habits and using machine learning to suggest cycles. Meanwhile, Samsung’s upcoming vacuum and mop robot cleaner, the Bespoke Jet Bot Combo™[11], utilises AI technology to provide aconvenient, effective cleaning experience. With AI Object Recognition upgraded from Samsung’s previous robot cleaner lineup, it will be able to distinguish more objects, as well as detect stains and spaces[12]. The Bespoke Jet Bot Combo™ can also recognise the type of floor it encounters — along with carpet length — and adjusts its settings accordingly.

Samsung Reveals Galaxy Book4 as Company’s Most AI-Ready Laptop Series Yet

Samsung has been elevating what a PC can do and has been collaborating with Microsoft to create new, seamless connected experiences. Now, these experiences are being advanced further with new connectivity features that make the Galaxy Book4 series smoother, smarter, and more adaptable, reinforcing Samsung and Microsoft’s commitment to providing the best PC experience possible[13].

Using intelligent, intuitive AI capabilities, Microsoft Copilot[14] connects the Galaxy Book4 series with a compatible Samsung Galaxy smartphone to make them work seamlessly together, so that it feels like just one device. Microsoft Copilot can find, read or summarise text messages from a user’s Galaxy smartphone and even automatically create and send messages on a user’s behalf directly from the PC. Without having to turn on the smartphone and open each individual app, the Galaxy Book4 series can access smartphone functions and information more quickly and intelligently.

The Galaxy Book4 series can now turn Samsung Galaxy smartphones’ powerful camera into a PC webcam[15], using it for video calls on virtual conferencing apps. Users can switch freely between front and rear cameras with just a click on their PC, and it also offers a variety of features from Galaxy smartphones – including Background Blur and Auto-framing – that help focus on the speaker’s image in any environment.

Through tight, ongoing collaboration between Samsung and Microsoft, more intelligent capabilities will continue to be made possible on the Galaxy Book4 series, with the above connectivity features being available starting from March.

Plus, the Galaxy Book4 series can be paired with other compatible Galaxy devices beyond smartphones. Tab S9 Ultra can be used as an additional PC monitor, and users can enjoy crystal-clear sound quality with low latency when the PC is connected with Galaxy Buds2 Pro. With all these capabilities and more, the Galaxy Book4 series is designed to deliver the best PC experience possible.

Better Connecting Users and Devices through Spatial Intelligence

Jaeyeon Jung, Executive Vice President and Head of SmartThings, highlighted the exciting new ways in which AI technology is impacting the relationship between users and their devices, including Samsung’s vision for SmartThings: The more Samsung devices a customer uses, the smarter those devices become and the better they can understand and support daily life.

Spatial AI is the key to enabling this vision, by helping devices understand the living space and routines of the user, thus enabling a personalised home management experience. SmartThings uses LiDAR[16] on connected devices like robot vacuums to create digital floor plans so users can easily check the status and location of all their devices.  And in March, SmartThings will roll out the upgraded 3D Map View for a more detailed experience on Samsung smartphones and TVs[17]. Furthermore, a simple QR code will allow users to add family and friends to their SmartThings ecosystem, so that every member of the household can create their own routines.

Using smart sensors and AI, SmartThings will be able to detect unusual circumstances, such as a fall, and send alerts to designated family and caregivers[18].  Also, thanks to the Galaxy SmartTag2, users can track their pets’ activities and digitally store contact information by attaching the tag to a collar or harness[19].

Another point EVP Jung discussed was how the Bixby voice assistant is allowing smart homes to become more dynamic with its enhanced AI capabilities. Bixby can now automatically route commands to the most appropriate devices based on an understanding of the user’s location and activities.  Key technologies like Multi Device Wakeup combined with a shared communication protocol, enable Bixby to listen to commands with all the devices in a room but then only perform the requested action on the most relevant device. So if a user is in the kitchen watching a recipe video on their Family Hub™, and they can call out, “Hey, Bixby! Play music!”  At that point, music will play from the user’s smart speaker, but the recipe video will continue to play on the Family Hub™, uninterrupted.

The TV, with the Samsung Daily+ service, will increasingly become a central hub for much of this smart home functionality, while also granting access to a variety of additional services. These include video calls through ConnecTime[20], video veterinary consultation services[21], and Workout Tracker[22] — through which workout data can be transferred to Samsung’s smart watches. Samsung also announced the new Daily Board service[23], which displays useful information about users’ homes, such as the weather or a live security camera feed from a home security system without the TV even being turned on.  As is the case with SmartThings features, users can also access this information via voice command.

Bringing the Home and Car Closer Together With Home-to-Car Capabilities

Samsung also announced developments in the automotive industry, starting with a partnership with Hyundai Motor Group.  Samsung and Hyundai have signed a memorandum of understanding (MOU), which was announced ahead of CES 2024. Under the agreement, users will have access to Home-to-Car and Car-to-Home services[24], courtesy of SmartThings connectivity. Haeyoung Kwon, Head of Infotainment Development Center and SDV[25] Execution Sub-Division at Hyundai Motor Group, delivered remarks regarding the partnership.

“Using Hyundai Motor Group’s software-defined vehicle, SmartThings will now work with Hyundai, Kia, and Genesis vehicles through the app,” Kwon said. “All of us at Hyundai Motor Group appreciate our partnership with Samsung, and we’re excited about the new features we’re bringing to your vehicles through smarter connectivity.”

Through SmartThings, users will be able to utilise voice commands to remotely perform functions in their car, such as heating it up in advance or opening and closing windows. SmartThings connectivity goes the other way, too, with users able to control home functions from their vehicle. Users will be able to automatically open and close garage doors based on the location of the car, as well as regulate home temperature. All of these actions can be carried out via voice commands.

Additionally, Samsung has further deepened the longstanding partnership with HARMAN to elevate the way users experience driving. Michelle Gattuso, Vice President of Product Management at HARMAN, discussed how the two companies are building new technologies to elevate the driving experience.

“Samsung’s leadership in consumer technology — together with HARMAN’s automotive expertise — has opened the door to game-changing experiences like HARMAN Ready Care, Ready Vision and Ready Display,” Gattuso said. “But now, the work we do with Samsung is getting even more integrated. Samsung and HARMAN are working in lockstep to bring even more impactful experiences to the entire in-cabin ecosystem.”

HARMAN’s Ready Care, which focuses on driver safety, implements deep learning neural networking techniques to monitor the driver’s movement and cognitive alertness, and can provide personalised reminders and alerts. Additionally, Ready Care can now identify whether passengers are adults or children, and adjust airbag deployment settings accordingly.

Furthermore, Ready Vision brings an intuitive display directly to windshields — courtesy of augmented reality — in drivers’ line of sight, right when they need it. For example, drivers have access to a map that updates according to routes in real time, as well as recommendations for the fastest route or where to stop for coffee. Another HARMAN product, Ready Upgrade, allows automakers to frequently update their vehicle’s hardware and software. This keeps vehicles relevant, modern and efficient for longer.

Samsung Focuses on Security and Privacy Across Devices

In this future of AI-enabled, deeply interconnected devices, Samsung recognises that security is paramount. Shin Baik from the Mobile eXperience Security Team discussed the company’s security solutions as a foundation for the hyper-connected era.

One such solution is Samsung Knox Matrix, which will provide end-to-end encryption across many Samsung Galaxy smartphones and smart TVs[26]. The technology will allow devices to monitor each other to identify and isolate security threats. Knox Vault, which helps keep user data safe on some of Samsung’s most popular devices, has been expanded to cover more devices connected by SmartThings, such as Samsung Neo QLED 8K TVs. Additionally, Samsung’s partnerships with major tech companies allow users to experience security in a way that is tailored to their specific needs.

Samsung Makes Strides in Responsible Materials, Energy Management, and Accessibility

Meanwhile, Inhee Chung, Vice President of the Samsung Corporate Sustainability Center, highlighted Samsung’s plans for a more circular economy. Additionally, the company’s Certified ReNewed program — available in the U.S. and select European countries only — offers affordable refurbished phones.

As for the ways Samsung’s technology contributes to sustainability efforts, the company is utilizing AI to help users track the energy usage of their compatible devices through features like SmartThings AI Energy Mode, which can help to automatically optimise energy usage. These features are yet another example of how AI technology can not only make lives easier, but also allow consumers to live in a way that is more responsible.

Thanks to a new collaboration with Tesla, SmartThings Energy is becoming even more useful both inside and outside of the home. The collaboration has integrated SmartThings Energy with Tesla products, including electric vehicles (EVs), the Powerwall home battery, Solar Inverter and Wall Connector EV charging solution.  Another benefit is that SmartThings Energy users can take advantage of the Tesla app’s Powerwall “Storm Watch” function, which flags severe weather through SmartThings not only on the Tesla app, but also on users’ connected Samsung TVs and mobile phones. SmartThings can also help users prepare for power outages by activating AI Energy Mode to extend the usage time of backup energy stored in their Tesla Powerwalls.[27]

Samsung is adding new features to increase device accessibility, such as digital appliances responding to hands-free gestures. Additionally, mobile devices provide auto-generated captions during calls, and TVs and displays offer a new, AI-enabled text-to-speech feature that converts subtitles into spoken audio. To further aid those with low vision, Relumino Together is a new viewing mode on Samsung TVs that enables users of all abilities to watch TV together, with or without visual adjustments. Now, everyone can enjoy home entertainment together at the same time.

[1] Native 8K content is very limited. Picture quality of upscaled content will vary depending on the quality and resolution of source content.

[2] Some apps and services may require additional third party subscriptions.

[3] Ballie is presently under product testing and development and is not available for purchase in Australia.

[4] Product name is for North America only. Product is referred to as “Bespoke 4-Door French Door Refrigerator with Family Hub™+” in other regions.

[5] Does not mean all services available on the Bespoke 4-Door Flex™ Refrigerator with AI Family Hub™+ are AI or generate information or outcomes using AI. The services/features that use deep learning models are limited to 1) AI Vision Inside and 2) Bixby features.

[6] The Australian model will does not include a 32” inch screen. As of March 2024, AI Vision Inside can recognize 33 food items like fresh fruits and veggies. If the food is covered by hand and not recognizable, it may be listed as an unknown item. The camera cannot identify or list any food items in the fridge door bins or freezer. Detecting and identifying food items are based on our deep learning model trained using predefined set of data and may yield incomplete or incorrect information. New datasets may be introduced to our learning model from time to time to enhance its accuracy.

[7] A  Wi-Fi connection and Samsung account are required.

[8] The Anyplace Induction range will be available in other regions but will not be available for purchase in Australia.

[9] Does not mean all services available on the AI Hub are AI or generate information or outcomes using AI. The services/features that use deep learning models are limited to 1) AI Smart Dial 2) AI Opti Wash & Dry and 3) Bixby features. The learning models may be updated time to time and may yield incomplete or incorrect information.

[10] The availability of the SmartThings Energy in Australia is yet to be confirmed

[11] Currently in development and subject to change.

[12] The upgraded AI Object Recognition is expected to be available in the 2^nd half of 2024.

[13] The Galaxy Book4 series will be available in other regions but will not be available for purchase in Australia.

[14] Users must link their phone to Galaxy Book4 through ‘Link to Windows’ on mobile and the Microsoft Phone Link App on PC and follow the set-up promps. Galaxy smartphone must be OneUI 1.0 or above. Requires the same MS account to be used both on PC and Phone. Timing of feature and availability may vary by market and device.

[15] Users must link their phone to Galaxy Book4 through ‘Link to Windows’ on mobile and the Microsoft Phone Link App on PC and follow the set-up prompts. Galaxy smartphone must be OneUI 1.0 or above. Requires the same MS account, Wi-Fi network connection to be used both on PC and Phone. Availability may vary by device.

[16] Light Detection and Ranging (LiDAR) is an optical technology used for distance sensing.

[17] The release date for this feature has not yet been confirmed for the Australian market.

[18] SmartThings functionality currently in development and availability is yet to be confirmed in Australia. SmartThings will be able to inform the designated person(s) of a possible emergency if it detects indications of a fall through partners’ presence sensors with fall detection function or the Galaxy watch. Fall detection service expected to be available within this year.

[19] Contact information and message shown on Lost Mode must be registered beforehand on the SmartThings application, along with user registration by email.  The SmartTag2 finder’s device must support NFC reading and be connected to the internet.

[20] The availability of ConnecTime in Australia is yet to be confirmed.

[21] Samsung Daily+ offers access to 3rd party veterinary consultation services such as Dr. Tail.

[22] Workout Tracker is compatible with Galaxy Watch 4 up, Youtube and TV program content. It will be compatible with various third party devices later in 2024.

[23] The availability of Daily Board service in Australia is yet to be confirmed. Official name of the service was updated at launch in March, 2024.

[24] Home-to-Car and Car-to-Home services are currently in development and their availability is yet to be confirmed in Australia.

[25] Software Defined Vehicle

[26] Availability of Samsung Knox Matrix feature in Australia is yet to be confirmed.

[27] Service is limited to countries where Tesla Powerwall is sold and SmartThings Energy is available. Availability in Australia is yet to be confirmed.