As smartphone cameras became the primary go-to camera for many, the technology powering them has become an integral part of the way we share our lives with each other. Samsung has been at the forefront of this change, building a strong heritage of camera technology that has evolved the way people use the camera — from clear night photography, to professional-grade portraits, smooth video stabilization, and selfies.

Samsung Newsroom talked with Joshua Sungdae Cho, VP and Head of Visual SW R&D at Samsung Mobile, to discuss Samsung’s camera philosophy and legacy of imaging technology.

Joshua Sungdae Cho, VP and Head of Visual SW R&D at Samsung Electronics

What is the driving philosophy behind the features and specs for cameras on Galaxy devices?

Our goal is to satisfy two very different needs. First, we help general users by democratizing the experience of professional photography, making it easy for them to create high-quality photos and videos with nothing but their smartphone. Secondly, we aim to bring a more customized shooting experience for pros who want to use their Galaxy smartphone as a professional tool. To do that, we focus on three key areas: the best shooting and viewing experience, ease of use, and controllability.

Our aim with the shooting and viewing experience is to ensure that users get crisp, vivid results every time they capture images and videos. This is why we have focused on high-resolution images, best-in-class image sensors, and other features like High Dynamic Range (HDR) and Optical Image Stabilization (OIS). For ease of use, we focus on creating features that make getting the best results as simple as possible. Single Take, for example, makes it easy to take multiple pictures and videos all at the same time, then recommends the best shots of the series. Finally, controllability is emphasized in our Pro Photo and Pro Video modes, giving users more control over their final results. Underpinning all of these features is AI, which ensures users get the most optimal results.

(Left) Super steady OFF l (Right) Super steady ON

Can you expand on these three pillars? How do you ensure users get the best shooting and viewing experience?

We take a lot of things into consideration for delivering the best shooting and viewing experience for both photos and video. First, we consider portraits our top priority. With warm tones, but a natural brightness and color, our cameras capture your emotions and expressions in vivid detail. We are continuing to make this a key focus area with upgrades that improve details such as skin texture and hair. Improving the resolution of the images from our cameras is one way we accomplish this.

Higher resolution images contain more pixels, and therefore, more information.  We try to find ways to get more pixels into our sensors, as well as improved optics, so that users produce sharp images and videos even when they crop or zoom. Ensuring users get great results no matter the lighting condition is also a key focus for the shooting experience on Galaxy devices.

Our advanced High Dynamic Range (HDR), for example, makes it possible to shoot in mixed lighting and still get great results. This is possible thanks to the AI in our camera, which takes three images at different exposure levels simultaneously and then combines them into one final image. This same approach is used for low-light photography. To deliver more bright, crisp images in low-light conditions, we use nona-binning technology, which uses AI to merge nine pixels into one larger pixel so it absorbs more light and creates clear, vivid images.

Last but not least, we also adopt best-in-class hardware to deliver superior viewing and sharing experiences and improve image quality. Samsung’s cutting-edge sensor technology, such as our 108MP sensors and 50x Space Zoom on the Note20 Ultra, make it possible to capture ultra-high-resolution images and even 8K video. We combine this sensor technology with other strengths of Galaxy devices, such as world-class displays to view the results and fast Wi-Fi connectivity and 5G that make sharing large files easy, to create the best all-around smartphone camera experience.

That sounds like a lot of features to pack into a smartphone camera. How do you make sure they are intuitive and easy to use?

Ease of use is particularly important for our camera philosophy. We aim to create an optimal auto-shooting mode that delivers professional looking images and video. The only thing our users need when shooting with a Galaxy device is a good subject and a memorable moment. We also pack our cameras with features that enable greater creativity while still making it easy to get great results.

Scene optimizer is a great example of how we make it easy to get professional level results. Users just have to point and shoot, and the camera delivers an image optimized for color and brightness based on the scene. We use AI to help deliver these optimizations, ensuring that every cloud, sunset, and smile is captured just as you remember.

Ease of use is also at the heart of features like Single Take. With just a single click, the camera takes multiple pictures and videos with dynamic effects simultaneously, allowing users to capture the moment in a variety of ways and choose their favorite. Features like My Filter also make it easy for users to customize their images. With My Filter, users can create a custom filter, save it, then apply it to all of the photos they shoot.

On the opposite end of this ease of use and auto-shooting, you mentioned that customization and control are an important part of the Galaxy camera experience. How do you implement those control features?

Both Pro Photo and Pro Video mode offer users more control over their final results. In Pro Photo mode, users can tweak all aspects of the photo including the ISO, shutter speed, white balance, and exposure levels. It also gives them the power to save RAW images, which they can later edit on their own. With the enhanced Pro Video mode on the Galaxy Note20 series, we’ve given users more creative control over the fine details in their videos.

In addition to manual adjustment of exposure settings, Pro Video mode introduces features such as Zoom control, allowing users to adjust the speed of their zoom for more dramatic footage. In Pro Video mode, users can also adjust the direction of the microphone to get the best sound. To top it off, we’ve also given users the power to record 8K video at 24fps on the Galaxy Note20 for high-quality results with a cinematic feel.

Many of these features seem to be powered or enhanced by AI. Why is AI such an important component of Galaxy cameras?

AI is one of the core technologies in Samsung’s camera technology because it allows us to do a lot of the heavy lifting for the user. We use AI to improve image resolution, optimize new technology for mobile, and provide better experiences for our customers.

Auto Framing on the Galaxy Z Fold2, for example, automatically shifts the view to stay centered and in focus depending on how many people are in view.  When there is only one person, it can follow their motion and keep them centered – ideal for vloggers and creatives that are using their smartphone to create content.

In the future we will continue to implement these advancements to learn, adapt, and develop new technology that improves the overall camera experience.

In addition to improvements in AI, the trend in smartphone photography has been to add more cameras on flagship devices year over year. Is this a trend you believe will continue?

We keep trying to find a balance between the best AI technology and the optimal number of cameras on our devices. Our AI technology is continuously improving thanks to the image training technology we have developed at Samsung. At the same time, we are also working to deliver the best camera hardware for our devices. If we are able to achieve the same results from multiple cameras with just a single AI-powered camera, then we will merge the technologies. Although we are not at that point yet, we are able to deliver stunning results with features like the Ultra-Wide camera that capture landscapes from a whole different perspective, and 50x zoom on the Galaxy Note20 Ultra.

The Galaxy Note20 series is a prime example of how our philosophy is reflected in our devices. We know that video usage has been steadily increasing, and has led to greater demand for better video features. So, for the Galaxy Note20 series, we improved on Super Steady, delivered 8K recording, and added increased functionality with Pro Video mode to enhance the user experience.

Ultimately, whatever we do is in service of consumer needs, to ensure every feature, experience and design choice offers increased functionality and quality.

And that would fly in the face of one of the biggest trends for mobile devices: They are now expected to be full-fledged multimedia powerhouses, delivering the quality and performance once the domain of laptops and desktop computers.

Piece of the Action

Meeting and exceeding these high expectations are Samsung’s two newest mobile devices, the Galaxy S6 edge+ and Galaxy Note5. Both have gone big on displays—5.7 inches to be exact—but it’s the way they can be used that really shines. Samsung has brought new features to both phones that enhance the mobile entertainment experience. It’s moving with the times, really, because when it comes to the cameras on mobile device it’s not just about still pictures and selfies anymore. People now increasingly share self-edited videos on their social networks, and watch videos online through streaming services. Samsung’s answer? Offering better video features than ever.

Ultimately, Samsung wants to help people that want to do more with their smartphones. For video, that means not just easier sharing, but also the ability to shoot film like professionals. Take shake-less video recording, for example. The Galaxy S6 edge+ and Galaxy Note5 both have an improved front camera for selfie videos with video digital image stabilization (VDIS). In other words, the camera will detect your face for a smooth selfie video and keep it in the center of the screen with less shake. The rear camera, meanwhile, can record video that is much more stable and has less motion blur because it can combine the power of both VDIS and OIS (optical image stabilization).

While the combination of VDIS and OIS improves the quality of your recordings, the Video Clip mode in the native camera app allows you to add some creative flair. You now have a wide variety of options to soup up your videos: Collage, Slow Motion and Series. And the best thing is, these features works straight out of the box; you don’t need to install any other apps.

With the Collage mode, you can combine two to four different videos into one video clip, depending on the layout you choose. The Slow Motion mode with the rear camera will automatically produce one video and three slow video clips at different speeds. The Series mode lets you combine multiple short clips into one video without a complex editing process.

The native camera app in the Galaxy S6 edge+ and Galaxy Note5 also has the Live Broadcast mode. Sharing video is always at its best when friends and family can enjoy the experience in real time. This special mode is a quick and easy way of making that happen. When you choose Live Broadcast mode, you can invite people to watch from your contacts. Anyone you invite can watch your broadcast on their own devices (mobile, PC, or TV) via YouTube. However, if you want, you can go public and share the broadcast through e-mail or social networks.

Picture Perfect

With the Galaxy S6 edge+ and Galaxy Note5, you will benefit from Samsung’s advanced camera system, including Quick Launch (double click the home button to launch the camera in less than one second), Auto Real-time High Dynamic Range (HDR), and Smart Optical Image Stabilization (OIS). And the 16MP OIS Rear and 5MP Front cameras take better photos thanks to two major upgrades. One of these is the blur-less photo function, which minimizes the blurring of moving objects under low light conditions. The effects of motion blur and hand shakiness are reduced, as the camera takes two to three photos with a short exposure time to produce one clear image.

The other major upgrade is to the filter controls. You can use them to adjust the strengths of a filter and level of vignette effects to create your own style. You can top up your pre-loaded filters with additional ones from Samsung’s Galaxy Apps store and give your pictures a unique look and feel.

Never Miss a Thing

The Galaxy S6 edge+ and Galaxy Note5 aren’t just about creativity and entertainment, they’ll also make you more productive. When you’re working on your computer, you can now see all the notifications that arrive on your phone directly on your computer monitor, so you’re always up to date. No need to check your phone every time you hear a ping or vibration. Both phones offer SideSync with enhanced features for a seamless connection between your smartphone and your computer or tablet. Manually pair the devices once, and after that the new auto connection feature will pair them every time they are within range.

SideSync includes a call pop-up window, which will notify you of a call even when your phone is on mute. When you receive a text message, you can keep your hands on the keyboard and reply directly on your PC monitor.

New for SideSync is two-way file sharing. You can easily move files between your phone and PC by simply dragging and dropping the files. For phone to PC sharing, you can view the mirrored phone screen by selecting the “Phone Screen Sharing” menu on the dashboard, and move files from your smartphone to the desired folder on a PC. When you drag and drop files from your PC to your phone, the files will be stored in the SideSync folder. Installation is easy and free, and SideSync for PC can be downloaded from www.samsung.com.

The Samsung Galaxy S6 edge+ and Galaxy Note5 are two distinct smartphones, but they have one principle in common: they are both built with digital content at heart. And whether you’re enjoying the latest chart hit or want to share your holiday highlights with family, you will be guaranteed an amazing multimedia experience.

What do you care most about when buying a smartphone? For many consumers, camera functions top the list of major concerns. Sharing photos on social networks has become part of our daily life and a quick and easy-to-use camera is now on the must-have list. This demand has resulted in a dramatic advancement of smartphone camera technologies over the last few years. So it begs the question, what will cameras look like in the future? And what kind of technological elements will become essential?

From commemorating special events to sharing everyday experiences, emerging mobile camera technologies to meet rising demand

Cellphone cameras have been advancing rapidly, largely driven by the selfie craze, and now boast high-resolution cameras in both the front and the back, as well as wide-angle lenses and a real-time recording function. Mobile cameras are also expected to perform well in low-light situations and other scenarios where previous mobile cameras could not deliver quality images. The Galaxy S6 and S6 edge are equipped with 16MP main cameras and 5MP front cameras. Both front and main cameras on the Galaxy S6 feature high resolution and bright F1.9 apertures, which makes it possible to generate clear images in dark environments.

During the Imagining Future Camera Technologies forum at the Samsung Institute of Technology, Young-Kwon Yoon, Master of Camera R&D Group, stated that, “Until a few years ago, people used a camera only for special occasions, like when they go on a picnic or celebrate someone’s birthday. People now, however, frequently photograph every little detail of their lives using their smartphone cameras, for example food and drinks, daily fashion, and other small details are now being brought to attention more often. Among those images, only a few are actually printed and the rest are modified, shared, and discarded in a flash.”

Smartphone image trends are shaping the way manufacturers develop cameras for their devices. Samsung is the leader in smartphone camera technology and innovation, constantly trying to push boundaries and create unique features that will improve image quality and user’ overall satisfaction.

The Galaxy S6 is a prime example of a smartphone that has advanced features that set it apart from the rest of the competition, including Optical Image Stabilization (OIS), Phase Detection Autofocus (AF), High-Dynamic Range (HDR) and Selective Focus.

Optical Image Stabilization (OIS)

Photo opportunities may present themselves when the photographer is on the move. What happens when the perfect photo opportunity happens while the photographer is, for example, a passenger in a moving car? The OIS in the Galaxy S6 is prepared for such situations and allows the user to capture crisp images even with unsteady hands and fast-moving environments.

The OIS accurately measures the motion of the camera caused by shaking hands to counteract it. For this to happen, the lens-shift correction technique has been applied. The lens-shift method inserts an actuator into the empty space between the lens and the sensor and makes it move horizontally to neutralize any hand shaking.

Phase Detection Autofocus (AF)

Speed and accuracy define the performance of an autofocus system. Based on how the light reaches the image sensors, the Phase Detection Autofocus determines if an object is front or back focused and by how much and then instructs the lens to adjust its focus rapidly. This technique is often used in DSLR cameras thanks to its agile response. It was first introduced in the Galaxy S5 and came back with more advanced features in the Galaxy S6. The AF allows a user not to miss the perfect photo opportunity by automatically tracking and refocusing on moving subjects. So, when a friend is driving around in a remote-controlled car, the user will be able to capture clear images no matter how sporadic the car moves about.

Phase AF: the camera can determine how much and in what direction the focus of the lens should be shifted.

High Dynamic Range (HDR)

The human eye can easily discern details inside a darkened room and brighter scenes outside the window at the same time. However, registering different lighting situations simultaneously is much more difficult for a camera to achieve. The High Dynamic Range techniques reproduce a greater dynamic range of luminosity than standard digital imaging, and deliver clearer and more vivid results even inside a shady room or against the sunlight.

Most previous HDR technologies post-processed images by combining different, short-exposure and long-exposure images to strike a balance. This lag in image production has been reduced by incorporating real-time HDR, which is built into the Galaxy S6. Enhanced by this new and powerful feature, both the front-facing and main cameras in the S6 can produce clearer and more dynamic images and selfies.

Selective Focus

The Selective Focus creates the DSLR-like, out-of-focus effect. Because DSLRs have a shallow depth of field, controlling focus point blurs the background and accentuates the main subject, giving the picture a dreamy quality. This effect can be duplicated using the Galaxy S6 with the Selective Focus mode. Selective Focus allows users to adjust focal points to achieve the desired look and feel of their images.

Mobile Camera Technologies that Will Change the Future

As smartphone technology advances, so will the expectations for performance and capabilities of their cameras. Here are three major smartphone camera innovations that we can look for in the near feature:

Depth Cameras

Depth Cameras produce a depth image or a depth map that tells the distance between the device and the subject. The method can be either active or passive, depending on the use of infrared (IR) light. Each has its pros and cons. The passive method has relatively lower accuracy at long distances, but consumes less electricity. On the contrary, the active method is less affected by the distance, but uses more power.

No one knows exactly what kind of services will be born out of this emerging area or which depth camera technology will survive in the competition. However, this is an unavoidable trend in the Internet-of-Things-related or video-based services and will find its place in a smartphone device soon.

Dual Cameras

The dual camera technology literally uses two cameras that face the same direction in order to achieve various outcomes. It can be categorized into four types by sensor and optical system. Dual cameras armed with homogeneous sensors whose angle of view is similar to each other are nearing commercialization. Plenty of research is in progress in this field, in order to reap the benefits that are unobtainable from a single camera, such as enhanced zooming, better performance in low light and the slimmer body.

▲ The four kinds of dual camera technologies, depending on sensor type and angle of view

Other Cutting-Edge Image Sensors

• Global Shutter

Currently, a mobile sensor uses the process commonly referred to as a “rolling shutter.” In this process, each pixel is exposed to light one after another as soon as the shutter is clicked. Because of this time gap, the subject is distorted. A global shutter does not have this issue as it controls incoming light to all photo sites simultaneously. Global shutters support only low-resolution images at the current stage; however, it will be further developed before long, powered by strong market demand.

• Curved Sensor

What would it be like if camera lenses resembled the human eye? The curved sensor has a curvilinear surface like the human retina. This artificial retina is expected to boost the optical performance or reduce the camera module’s thickness. But how to apply this technology still remains unclear. It is possible that other industries will find some useful ways of using it and come up with a new service model.

•  Organic Sensor

The organic sensor has an extremely thin, light-sensitive organic layer covering its surface, which dramatically improves its sensitivity. If a sensor is capable of dealing with both strong and weak signals, the image quality can be improved without increasing the sensor size. Or, the hardware can be downsized so that the camera does not ‘pop out’ in the back.

• Binary Image Sensor

Each image pixel on the binary image sensor is very small and can shrink to the size of dozens of nanometers. The resolution and frame rates can be freely configured depending on binary sampling methods. Much attention has been paid to this but the technology still has a long way to go.

Continuing Innovation

“Mobile cameras already produce images whose quality is as good as high-end digital cameras, at least on a display panel. This means that the increasing number of pixels is not going to guarantee the survival in the market. We need to make new breakthroughs,” said Young-kwon Yoon during the forum. “In the short run, the demand for a mobile camera that produces a quick and easy result everywhere is going to grow continuously. Another noticeable, recent phenomenon is the increasing interest in sharing videos online. Besides these ordinary video clips, new video contents, like the one offered by the Samsung Gear VR, will add more colors to user experience. We are going to work hard to come up with killer contents that will transform the mobile camera landscape in the long term.”

Future mobile technologies will do more than just anticipate the changes in consumer lifestyle. They will be the catalysts for these changes in consumer habits around the world. Rich video content created by state-of-the-art mobile cameras will broaden the range of user experiences and the Galaxy S6 and its successors will be the main players of this game.