“When Samsung first launched Bixby as a voice assistant, it was part of a vision to create a human-to-machine interface that makes life easier and can advance over time with Samsung Galaxy innovation,” said YoungJip Kim, Executive Vice President and Head of Artificial Intelligence Team in Mobile eXperience Business, Samsung Electronics. “Today, we introduce updates to build a more intelligent interface that is proactive and adaptive, giving people greater control over their mobile experience.”

Giving People Greater Control Over Their Mobile Experience

The new Bixby updates bring several new features and improvements that allow people to customize their user experience further, including the recent availability of Bixby Text Call¹ in English. Now, English speakers can answer calls from anywhere by typing a message, which Bixby converts to audio and communicates to the caller directly on their behalf.

Users can also personalize their Bixby Text Call voice. Using the new Bixby Custom Voice Creator,² users can record different sentences for Bixby to analyze and create an AI generated copy of their voice and tone. Currently available in Korean, this generated voice is planned to be compatible with other Samsung apps beyond phone calls. And for those who want to customize their wake word, Bixby now supports changes to the Custom wake-up³ phrase within Bixby settings as well, adding a new option to create a personalized wake-up phrase.

From Intelligent Assistant to Intelligent Platform

Today, Bixby can provide more seamless continuity and better attune to user needs with the introduction of new scenarios,⁴ including playing music depending on the type of exercise in Samsung Health or saving schedules in Calendar. Now, Bixby can better understand intent and process follow-up requests by understanding context and associating words previously used in interactions. For example, Bixby users can first launch a workout on Samsung Health and then ask Bixby to play music⁵ that best suits that exercise by saying “Play music for this workout.”

Enhanced Language Recognition, Processing and Response

In addition to these great new features, Samsung also expanded Bixby’s on-device AI support.⁶ Users can run key commands entirely offline, such as setting a timer, taking a screenshot or turning on the flashlight. By integrating on-device AI with its native applications,⁷ Samsung was able to further expand its language offering⁸ and capabilities with advanced AI-based voice dictation. These updates to Bixby will be made possible by the powerful innovation found in Samsung Galaxy devices, including the industry-leading performance that enables on-device AI applications.

Availability

Bixby updates will be available for Samsung Galaxy users in February⁹ through a software update. For more information, please visit www.samsung.com/global/galaxy/apps/bixby.

¹ Bixby Text Call is available on selective models including Galaxy S23, S23+, S23 Ultra, Z Fold4 and Z Flip4. It works on phones with One UI 4.1.1 or above for Korean and phones with One UI 5.1 or above for English.

² Bixby Custom Voice Creator is available on selective models including Galaxy S23, S23+ and S23 Ultra. It works on phones with One UI 5.0 or above. Availability of functions may vary depending on the device model. The application is available for download via settings within Bixby or the Samsung Phone application (native call app).

³ The feature is available in Korean as of date and on selective models including Galaxy S23, S23+ and S23 Ultra. Exact timing and availability of the updates may vary by market and are subject to change.

⁴ The eight scenarios are all currently available in Korean. More languages to be supported.

⁵ Supported music applications include Melon, Bugs, Genie and FLO as of date.

⁶ Availability of the on-device AI support may vary by market and model.

⁷ Supported native applications include those apps that require on-device AI support only.

⁸ The on-device mode currently supports English, Spanish, French, Italian, German and Korean.

⁹ Exact timing and availability of the updates may vary by user and market.

As smartphone cameras became the primary go-to camera for many, the technology powering them has become an integral part of the way we share our lives with each other. Samsung has been at the forefront of this change, building a strong heritage of camera technology that has evolved the way people use the camera — from clear night photography, to professional-grade portraits, smooth video stabilization, and selfies.

Samsung Newsroom talked with Joshua Sungdae Cho, VP and Head of Visual SW R&D at Samsung Mobile, to discuss Samsung’s camera philosophy and legacy of imaging technology.

Joshua Sungdae Cho, VP and Head of Visual SW R&D at Samsung Electronics

What is the driving philosophy behind the features and specs for cameras on Galaxy devices?

Our goal is to satisfy two very different needs. First, we help general users by democratizing the experience of professional photography, making it easy for them to create high-quality photos and videos with nothing but their smartphone. Secondly, we aim to bring a more customized shooting experience for pros who want to use their Galaxy smartphone as a professional tool. To do that, we focus on three key areas: the best shooting and viewing experience, ease of use, and controllability.

Our aim with the shooting and viewing experience is to ensure that users get crisp, vivid results every time they capture images and videos. This is why we have focused on high-resolution images, best-in-class image sensors, and other features like High Dynamic Range (HDR) and Optical Image Stabilization (OIS). For ease of use, we focus on creating features that make getting the best results as simple as possible. Single Take, for example, makes it easy to take multiple pictures and videos all at the same time, then recommends the best shots of the series. Finally, controllability is emphasized in our Pro Photo and Pro Video modes, giving users more control over their final results. Underpinning all of these features is AI, which ensures users get the most optimal results.

(Left) Super steady OFF l (Right) Super steady ON

Can you expand on these three pillars? How do you ensure users get the best shooting and viewing experience?

We take a lot of things into consideration for delivering the best shooting and viewing experience for both photos and video. First, we consider portraits our top priority. With warm tones, but a natural brightness and color, our cameras capture your emotions and expressions in vivid detail. We are continuing to make this a key focus area with upgrades that improve details such as skin texture and hair. Improving the resolution of the images from our cameras is one way we accomplish this.

Higher resolution images contain more pixels, and therefore, more information.  We try to find ways to get more pixels into our sensors, as well as improved optics, so that users produce sharp images and videos even when they crop or zoom. Ensuring users get great results no matter the lighting condition is also a key focus for the shooting experience on Galaxy devices.

Our advanced High Dynamic Range (HDR), for example, makes it possible to shoot in mixed lighting and still get great results. This is possible thanks to the AI in our camera, which takes three images at different exposure levels simultaneously and then combines them into one final image. This same approach is used for low-light photography. To deliver more bright, crisp images in low-light conditions, we use nona-binning technology, which uses AI to merge nine pixels into one larger pixel so it absorbs more light and creates clear, vivid images.

Last but not least, we also adopt best-in-class hardware to deliver superior viewing and sharing experiences and improve image quality. Samsung’s cutting-edge sensor technology, such as our 108MP sensors and 50x Space Zoom on the Note20 Ultra, make it possible to capture ultra-high-resolution images and even 8K video. We combine this sensor technology with other strengths of Galaxy devices, such as world-class displays to view the results and fast Wi-Fi connectivity and 5G that make sharing large files easy, to create the best all-around smartphone camera experience.

That sounds like a lot of features to pack into a smartphone camera. How do you make sure they are intuitive and easy to use?

Ease of use is particularly important for our camera philosophy. We aim to create an optimal auto-shooting mode that delivers professional looking images and video. The only thing our users need when shooting with a Galaxy device is a good subject and a memorable moment. We also pack our cameras with features that enable greater creativity while still making it easy to get great results.

Scene optimizer is a great example of how we make it easy to get professional level results. Users just have to point and shoot, and the camera delivers an image optimized for color and brightness based on the scene. We use AI to help deliver these optimizations, ensuring that every cloud, sunset, and smile is captured just as you remember.

Ease of use is also at the heart of features like Single Take. With just a single click, the camera takes multiple pictures and videos with dynamic effects simultaneously, allowing users to capture the moment in a variety of ways and choose their favorite. Features like My Filter also make it easy for users to customize their images. With My Filter, users can create a custom filter, save it, then apply it to all of the photos they shoot.

On the opposite end of this ease of use and auto-shooting, you mentioned that customization and control are an important part of the Galaxy camera experience. How do you implement those control features?

Both Pro Photo and Pro Video mode offer users more control over their final results. In Pro Photo mode, users can tweak all aspects of the photo including the ISO, shutter speed, white balance, and exposure levels. It also gives them the power to save RAW images, which they can later edit on their own. With the enhanced Pro Video mode on the Galaxy Note20 series, we’ve given users more creative control over the fine details in their videos.

In addition to manual adjustment of exposure settings, Pro Video mode introduces features such as Zoom control, allowing users to adjust the speed of their zoom for more dramatic footage. In Pro Video mode, users can also adjust the direction of the microphone to get the best sound. To top it off, we’ve also given users the power to record 8K video at 24fps on the Galaxy Note20 for high-quality results with a cinematic feel.

Many of these features seem to be powered or enhanced by AI. Why is AI such an important component of Galaxy cameras?

AI is one of the core technologies in Samsung’s camera technology because it allows us to do a lot of the heavy lifting for the user. We use AI to improve image resolution, optimize new technology for mobile, and provide better experiences for our customers.

Auto Framing on the Galaxy Z Fold2, for example, automatically shifts the view to stay centered and in focus depending on how many people are in view.  When there is only one person, it can follow their motion and keep them centered – ideal for vloggers and creatives that are using their smartphone to create content.

In the future we will continue to implement these advancements to learn, adapt, and develop new technology that improves the overall camera experience.

In addition to improvements in AI, the trend in smartphone photography has been to add more cameras on flagship devices year over year. Is this a trend you believe will continue?

We keep trying to find a balance between the best AI technology and the optimal number of cameras on our devices. Our AI technology is continuously improving thanks to the image training technology we have developed at Samsung. At the same time, we are also working to deliver the best camera hardware for our devices. If we are able to achieve the same results from multiple cameras with just a single AI-powered camera, then we will merge the technologies. Although we are not at that point yet, we are able to deliver stunning results with features like the Ultra-Wide camera that capture landscapes from a whole different perspective, and 50x zoom on the Galaxy Note20 Ultra.

The Galaxy Note20 series is a prime example of how our philosophy is reflected in our devices. We know that video usage has been steadily increasing, and has led to greater demand for better video features. So, for the Galaxy Note20 series, we improved on Super Steady, delivered 8K recording, and added increased functionality with Pro Video mode to enhance the user experience.

Ultimately, whatever we do is in service of consumer needs, to ensure every feature, experience and design choice offers increased functionality and quality.