Capturing stunning photographs at night can be a challenge for everyone. Low light and camera noise can lead to blurry, grainy photos, causing you to miss capturing that memorable moment. But thanks to the AI powering the camera on the Galaxy S21 series, you can now take stunning images in low light conditions with just the click of the shutter.

Intelligent Low Light Performance

Whether you’re trying to snap a photo of a beautiful cityscape at night or a group selfie in a low-lit restaurant, from the moment you click the shutter, the camera AI on the Galaxy S21 series begins a three-phase process. Here’s how it works:

1. During the capture phase, the camera takes multiple pictures of the scene at different exposure levels.
2. After capturing, the camera begins Multi-frame Processing, combining the images into one and enhancing the brightness of the picture.
3. Lastly, the AI image signal processor begins post-processing the image, reducing noise and fine-tuning the details. The results are crystal clear images, with greatly reduced noise and blur.

The Power of Multi-frame Processing

The keys to achieving that clarity and detail are in the multi-frame layering and AI restoration. When the multi-frame processing begins, the Galaxy S21 series intelligently selects the sharpest frame from all the images captured to use as a reference image. After selecting the reference frame, the AI aligns and registers the rest of the images before delivering the finalized photo. When finalized, the image has significantly reduced noise and a superior tone balance between the main subject and its background.

The same multi-frame process allows the Galaxy S21 to support clear, high-resolution images even under poor conditions such as when zooming in on a neon sign to create an image with high-dynamic range. While zooming in on distant subjects, the Galaxy S21 is capable of capturing up to 20 frames at different exposure values (ev), depending on the conditions. Capturing at different exposure values allows the camera to produce a sharper final image that has high-dynamic range with true-to-life exposure of both shadows and highlights. So, when you’re zooming in on that neon sign, you’ll be able to make out every letter.

During multi-frame processing, the AI determines whether it needs to use every frame captured, or just a subset. With more data from multiple frames, the Galaxy S21 series’ camera AI can deliver bright, tack-sharp images even in low light or when zoomed in from extremely far away—whether it’s a photo of a neon sign, or your favorite singer at a concert.

To learn more about the intelligent features behind the Galaxy S21 series’ powerful camera, be sure to check out the rest of the Behind the Snapshot series at Samsung Newsroom.

Sometimes a single picture or video just isn’t enough to truly capture the moment. Single Take, first introduced on the Galaxy S20 series, aimed to solve that problem by using AI processing to capture multiple pictures and videos simultaneously, then automatically apply post-processing edits.

In the second part of our Behind the Snapshot series, we’ll take a closer look at how the AI behind Single Take on the Galaxy S21 series gives you the power to create instantly shareable photos and videos.

Capture More of the Scene

Backed by more powerful AI, the Galaxy S21 ensures you never miss a moment by expanding the capabilities of Single Take with a duration of five to 15 seconds during which it records video while capturing up to ten photos per second. After capturing your scene, the AI takes over and processes the captured content to deliver up to the ten best photos and up to four video clips.

To find the best photos, the AI analyzes your content to determine which enhancements to apply. With photos of people and pets, for example, it may apply Portrait mode to emphasize the subject. For live events or landscapes, it may use Smart Crop to edit around the most interesting part of the scene or apply a variety of AI filters that adjust color and lighting.

For videos, the AI analyzes the footage for action and scenery. By default, videos receive either the Fast Forward (3x) effect or the new Dynamic Slow-mo effect, which applies slow motion when there is fast action in the scene. Other video effects include Boomerang, Reverse, Highlight video and Scene Relighting (a 24 hour time-lapse video). Results that would take normal users, hours of filtering and editing, Single Take can produce in an instant. So just how does it work?

Selecting Your Best Photos in an Instant

In order to deliver results quickly, the Single Take AI uses multiple engines powered by deep learning and computer vision to evaluate and filter images to choose the best moments.

The Aesthetic engine, for example, uses a deep learning engine trained using approximately 300,000 images carefully selected by professionals to evaluate overall clarity, image quality, facial expressions, and more. Through this process, the AI eliminates images where eyes are closed, or the subject’s face is angled down. Once the engine chooses the best moments, the AI then decides which filters and features to apply using the same deep learning technology. The most popular filters are automatically applied to your image.

The Angle Evaluation engine uses the same process, analyzing a database of approximately 100 million photos to determine the best crop angle. Finally, for pets and people, Portrait mode uses AI to segment subjects from the background and applies blur or other backdrop effects to the final photo.

Here’s a quiz to demonstrate how Single Take selects photos: which one do you think will be the best moment?

Creating Instantly Shareable Videos

In addition to photos and filter selection, Single Take uses multiple intelligence engines to do a lot of the heavy lifting of editing and processing your videos like a professional.

Capture intelligence analyzes the scene and leverages high-speed capture to produce the best image based on the available light. Human awareness intelligence, on the other hand, identifies human faces, expressions, body poses, head tilt, and helps to emphasize human subjects by making them stand out from the background. For videos, motion intelligence detects the peak moments of action in a scene. At any given moment while using Single Take, multiple intelligence engines work simultaneously to get the best results.

In the new Highlight video, for example, Single Take uses scene intelligence, capture intelligence, motion intelligence, and human awareness intelligence to pick the best photo and video moments in a scene and stitch them together into an instantly shareable video enhanced with filters, transitions, and background music. In more action-packed scenes, Single Take uses motion intelligence to analyze the scene and applies Dynamic Slow-mo, Boomerang or Reverse effects to emphasize the peak of the action.

In addition to Highlight video, you can also create stunning time-lapses with the Scene Relighting feature, which generates a video of your scene showing the full transition of light from morning to night using a single image. Based on your photo, the AI generates images with lighting adjustments for different times of the day—morning, evening, sunset, blue hour, night, and dawn—using a Generative Adversarial Network (GAN). It then combines these images into a 12-second time-lapse video, smoothly blending the images to show the light transitioning in the scene over a 24-hour period.

The AI is able to analyze the sky, clouds, water, trees, and buildings in your scene, and naturally restore their edges and features just as you captured them. Scene Relighting videos are generated with such realism that you’ll never have to worry about missing a stunning blue sky or precious sunset again—simply capture the scene with Single Take and the AI will recreate it for you.

To learn more about the intelligent features behind the Galaxy S21 series’ powerful camera, be sure to check out the rest of the Behind the Snapshot series at Samsung Newsroom.

Starting with the Galaxy Note9’s Scene Optimizer, which used deep learning to deliver the best photos and editing effects, Samsung has continued to introduce more powerful AI features to its Galaxy cameras.

The Galaxy S21 series is the latest to feature Samsung’s cutting-edge AI-powered cameras, giving mobile photographers of all skill levels pro-grade tools that work behind-the-scenes to ensure every moment you capture is truly epic. In the first of this three-part series, we’ll take a closer look at how AI is improving portraits on the Galaxy S21 series.

Enhanced Selfie Experience

We’ve all had that moment of panic, frantically thumbing for the delete button, after taking a less than adequate selfie. But with the Galaxy S21 series’ selfie camera, you’ll always look your best with AI that delivers a more natural look and improved details. There are two key parts to the enhanced selfie experience:

1. Face information detection: When you snap a photo, the AI-powered selfie camera first identifies faces in the image, then segments them from the rest of the details in the scene, and finally applies natural enhancements to your subjects—all in the blink of an eye.

2.Restoration: The natural enhancements include bringing out the details in your subjects’ hair, eyes, and facial features and adjusting white balance to create more natural looking skin tones in any environment. The results are instantly shareable photos that don’t look overly processed or require further editing.

Samsung also included support for third party apps, so you can take images from your favorite camera or social media app and still get the benefits of AI-powered selfie camera.

Updated Portrait Mode

The Galaxy S21 series goes beyond just improving selfies—it also allows even beginners to capture studio-quality portraits of your friends, family, and even pets. The new advanced Portrait mode, formerly known as Live Focus, uses improved 3D sensing to deliver photos with more depth and dramatic effects.

When using Portrait mode, you can select from a variety of special effects to apply to your final image which include Blur, Studio, High-key Mono, Low-key Mono, Backdrop, and Color Point. As soon as you click the shutter, the onboard AI begins segmenting, matting, and refining the details of your subjects. Here’s how it works:

1. Segmentation Map: First, the AI immediately recognizes the humans or pets in the scene and generates an instant segmentation map to separate each individual subject.

2. Seed Map: Based on this segmentation map, a seed map of the subjects is generated to which your selected effects will apply. In a portrait like in the chart above, a seed map would separate the background from the main subject.

3. Tri-map: Image processing then adds a tri-map on top of the seed map, to identify the subject, the background, and the areas where they blend together.

4. Matte Map: Matting is then applied to the tri-map to bring out the details in the hair and facial features of your subjects.

5.Depth Estimation: Meanwhile, the AI simultaneously measures depth information to refine the background details and depth-of-field.¹

During segmenting, the AI powering Portrait mode is able to discern whether you’re capturing a singular person, a group of people at different depth levels, or a pet cat or dog.

Refining the Details

Regardless of whether your subject is a human or an animal, AI matting plays an important role in ensuring you capture them in as much detail as possible. As noted above, after segmenting your subjects, the seed map generates a tri-map where black represents the background, white represents the subject, and gray represents areas of the image that are uncertain.

During image processing when the background is blurred out in Portrait mode, this map ensures that the blur doesn’t overlap with your subject. A matte map is then generated to refine the details in the gray, unknown areas of the scene and to improve the resolution of the image. At the same time matting takes place, the AI uses the seed map to refine the details of the interior parts of the subject. The maps are then combined and upscaled to produce a final, high-resolution portrait where your subjects really pop off the background. Although it seems like quite an extensive process, it all happens within 3 seconds.²

To learn more about the intelligent features behind the Galaxy S21 series’ powerful camera, be sure to check out the rest of the Behind the Snapshot series at Samsung Newsroom.

¹ Depth-of-field adjustments are only applied when using the Blur effect on portraits of people and animals. If the camera measures an object at the same depth as a person, the effect is not applied, and the image is saved with its natural depth-of-field.

² Portrait mode processing times may vary depending on the shooting conditions.