As smartphone cameras became the primary go-to camera for many, the technology powering them has become an integral part of the way we share our lives with each other. Samsung has been at the forefront of this change, building a strong heritage of camera technology that has evolved the way people use the camera — from clear night photography, to professional-grade portraits, smooth video stabilization, and selfies.

Samsung Newsroom talked with Joshua Sungdae Cho, VP and Head of Visual SW R&D at Samsung Mobile, to discuss Samsung’s camera philosophy and legacy of imaging technology.

Joshua Sungdae Cho, VP and Head of Visual SW R&D at Samsung Electronics

What is the driving philosophy behind the features and specs for cameras on Galaxy devices?

Our goal is to satisfy two very different needs. First, we help general users by democratizing the experience of professional photography, making it easy for them to create high-quality photos and videos with nothing but their smartphone. Secondly, we aim to bring a more customized shooting experience for pros who want to use their Galaxy smartphone as a professional tool. To do that, we focus on three key areas: the best shooting and viewing experience, ease of use, and controllability.

Our aim with the shooting and viewing experience is to ensure that users get crisp, vivid results every time they capture images and videos. This is why we have focused on high-resolution images, best-in-class image sensors, and other features like High Dynamic Range (HDR) and Optical Image Stabilization (OIS). For ease of use, we focus on creating features that make getting the best results as simple as possible. Single Take, for example, makes it easy to take multiple pictures and videos all at the same time, then recommends the best shots of the series. Finally, controllability is emphasized in our Pro Photo and Pro Video modes, giving users more control over their final results. Underpinning all of these features is AI, which ensures users get the most optimal results.

(Left) Super steady OFF l (Right) Super steady ON

Can you expand on these three pillars? How do you ensure users get the best shooting and viewing experience?

We take a lot of things into consideration for delivering the best shooting and viewing experience for both photos and video. First, we consider portraits our top priority. With warm tones, but a natural brightness and color, our cameras capture your emotions and expressions in vivid detail. We are continuing to make this a key focus area with upgrades that improve details such as skin texture and hair. Improving the resolution of the images from our cameras is one way we accomplish this.

Higher resolution images contain more pixels, and therefore, more information.  We try to find ways to get more pixels into our sensors, as well as improved optics, so that users produce sharp images and videos even when they crop or zoom. Ensuring users get great results no matter the lighting condition is also a key focus for the shooting experience on Galaxy devices.

Our advanced High Dynamic Range (HDR), for example, makes it possible to shoot in mixed lighting and still get great results. This is possible thanks to the AI in our camera, which takes three images at different exposure levels simultaneously and then combines them into one final image. This same approach is used for low-light photography. To deliver more bright, crisp images in low-light conditions, we use nona-binning technology, which uses AI to merge nine pixels into one larger pixel so it absorbs more light and creates clear, vivid images.

Last but not least, we also adopt best-in-class hardware to deliver superior viewing and sharing experiences and improve image quality. Samsung’s cutting-edge sensor technology, such as our 108MP sensors and 50x Space Zoom on the Note20 Ultra, make it possible to capture ultra-high-resolution images and even 8K video. We combine this sensor technology with other strengths of Galaxy devices, such as world-class displays to view the results and fast Wi-Fi connectivity and 5G that make sharing large files easy, to create the best all-around smartphone camera experience.

That sounds like a lot of features to pack into a smartphone camera. How do you make sure they are intuitive and easy to use?

Ease of use is particularly important for our camera philosophy. We aim to create an optimal auto-shooting mode that delivers professional looking images and video. The only thing our users need when shooting with a Galaxy device is a good subject and a memorable moment. We also pack our cameras with features that enable greater creativity while still making it easy to get great results.

Scene optimizer is a great example of how we make it easy to get professional level results. Users just have to point and shoot, and the camera delivers an image optimized for color and brightness based on the scene. We use AI to help deliver these optimizations, ensuring that every cloud, sunset, and smile is captured just as you remember.

Ease of use is also at the heart of features like Single Take. With just a single click, the camera takes multiple pictures and videos with dynamic effects simultaneously, allowing users to capture the moment in a variety of ways and choose their favorite. Features like My Filter also make it easy for users to customize their images. With My Filter, users can create a custom filter, save it, then apply it to all of the photos they shoot.

On the opposite end of this ease of use and auto-shooting, you mentioned that customization and control are an important part of the Galaxy camera experience. How do you implement those control features?

Both Pro Photo and Pro Video mode offer users more control over their final results. In Pro Photo mode, users can tweak all aspects of the photo including the ISO, shutter speed, white balance, and exposure levels. It also gives them the power to save RAW images, which they can later edit on their own. With the enhanced Pro Video mode on the Galaxy Note20 series, we’ve given users more creative control over the fine details in their videos.

In addition to manual adjustment of exposure settings, Pro Video mode introduces features such as Zoom control, allowing users to adjust the speed of their zoom for more dramatic footage. In Pro Video mode, users can also adjust the direction of the microphone to get the best sound. To top it off, we’ve also given users the power to record 8K video at 24fps on the Galaxy Note20 for high-quality results with a cinematic feel.

Many of these features seem to be powered or enhanced by AI. Why is AI such an important component of Galaxy cameras?

AI is one of the core technologies in Samsung’s camera technology because it allows us to do a lot of the heavy lifting for the user. We use AI to improve image resolution, optimize new technology for mobile, and provide better experiences for our customers.

Auto Framing on the Galaxy Z Fold2, for example, automatically shifts the view to stay centered and in focus depending on how many people are in view.  When there is only one person, it can follow their motion and keep them centered – ideal for vloggers and creatives that are using their smartphone to create content.

In the future we will continue to implement these advancements to learn, adapt, and develop new technology that improves the overall camera experience.

In addition to improvements in AI, the trend in smartphone photography has been to add more cameras on flagship devices year over year. Is this a trend you believe will continue?

We keep trying to find a balance between the best AI technology and the optimal number of cameras on our devices. Our AI technology is continuously improving thanks to the image training technology we have developed at Samsung. At the same time, we are also working to deliver the best camera hardware for our devices. If we are able to achieve the same results from multiple cameras with just a single AI-powered camera, then we will merge the technologies. Although we are not at that point yet, we are able to deliver stunning results with features like the Ultra-Wide camera that capture landscapes from a whole different perspective, and 50x zoom on the Galaxy Note20 Ultra.

The Galaxy Note20 series is a prime example of how our philosophy is reflected in our devices. We know that video usage has been steadily increasing, and has led to greater demand for better video features. So, for the Galaxy Note20 series, we improved on Super Steady, delivered 8K recording, and added increased functionality with Pro Video mode to enhance the user experience.

Ultimately, whatever we do is in service of consumer needs, to ensure every feature, experience and design choice offers increased functionality and quality.

Capture with Crystal Clarity: 108MP/64MP High Resolution

More resolution means more details. The Galaxy S20’s camera offers super-high resolution, so you can capture the details every time.  The Galaxy S20 Ultra features a 108MP sensor, while the S20 and S20+ feature a 64MP sensor. Although cropping a shot used to mean losing details, with the high-resolution sensors on the Galaxy S20 you have plenty of resolution to crop and still get crisp, clear images.

It all starts with a fundamental change to the sensor architecture, both in terms of the size of the image sensor and the technology incorporated within. The larger sensors take in more light, so you get rich image quality even in low light situations.

The S20 Ultra takes things a step further with the 108MP sensor which switches between nona-binning and re-mosaic technologies to provide images with more clarity even in very low light or bright environments.

• Nona-binning for superior image quality: Nona-binning technology merges nine pixels into one, so the 108MP sensor provides a 12MP photo. Larger pixels produced by the nona-binning technology can capture more light for improved clarity, even in low light conditions.

• Re-mosaic in bright light conditions: Re-mosaic re-arranges color pixels to bring out more details. The re-mosaic algorithm remaps pixels into a conventional RGB pattern, producing more detailed images in bright light conditions. You can easily switch to the 108MP Mode in the Quick Settings menu: Camera > Photo > Ratio > 108MP.

See Further, See Better: Space Zoom with Telephoto Lens

The Galaxy S20’s Space Zoom technology uses a combination of Hybrid Optic Zoom, and super resolution zoom technology which includes AI multi-frame processing (AI-powered digital zoom) to ensure that even when you are far away, you can zoom in close.

The Galaxy S20’s new Hybrid Optic Zoom combines an advanced high-resolution image sensor with an all new lens, which is designed to provide more value to users than traditional optical zoom does. Thanks to this new technology, the Galaxy S20 and S20+ can achieve 3x Hybrid Optic Zoom, while the Galaxy S20 Ultra can achieve 10x Hybrid Optic Zoom. With AI-powered digital zoom, you can see even further, up to 30x on the S20 and S20+ and up to 100x on the S20 Ultra.

To achieve 100x zoom, the Galaxy S20 Ultra’s uses a folded lens and a high-resolution sensor. The folded lens provides up to 10x Hybrid Optic Zoom, while super zoom resolution technology brings an additional 10x zoom with AI multi-frame processing to reduce hand shaking and quality loss at high zoom levels. Images produced are clear thanks to the combination of the lens optics and a sensor crop on the high-resolution sensor.

• 48MP Sensor: The sensor on the S20 Ultra is a 48MP sensor which switches between tetra-binning and re-mosaic depending on the shooting conditions, such as brightness level or zoom ratio. Tetra-binning combines 4 pixels into 1, resulting in a 12MP photo with larger pixels. When the RGB pixels are re-arranged with the re-mosaic technology, each pixel operates independently to use a sensor crop and provides up to 10x high ratio zoom.

• Folded lens: The Galaxy S20 Ultra leverages a folded zoom lens to squeeze more zoom capability into a small space. The lens lays flat and is positioned at the bottom of the camera on the phone. Light coming in from the back of the phone is redirected by a prism into the lens, refracting to the right at a 90-degree angle through a series of stacked optical elements which increases the focal length, much like the way a periscope works.  Cutting down both the height and width of the camera, we are able to deliver groundbreaking zoom capabilities.