From presenting information to sharing stories online, multimedia has become the dominant tool of communication on smartphones over the past years. Case in point: the stunning pictures, inventive GIFs, and creative videos produced by Samsung’s new Galaxy S9/S9+’s camera capture emotions that are simply beyond words.

But the Galaxy S9/S9+ is taking it up a notch with an all-new variety of intelligent and personalized emojis, which means you may never want to send plain text messages again.

AR Emoji is designed to bring your unique facial expressions and add more fun to the emoji experience. But how did this AR-based feature come to life? Get a look behind Samsung’s lab doors for a close-up view.

01. The Problem

Regular emojis add more color to the conversation and are a fun way to communicate, but we recognized there are a lot of times when the existing set of emojis just don’t cut it. These lack personality – one smile looks no different from another. So we started looking for ways to bring out more character and personality in emojis.

02. The Goal

We thought in an ideal world, you would be able to teleport yourself and share how you feel with others then and there. So we wanted to come up with a way to replicate that experience but with the simplest tools possible, having seen so many people use messaging apps to communicate even with people right next to them and rely heavily on emojis rather than text. The focus quickly became how to create an image of the user and deliver multiple facial expressions and emotions along with that.

03. The Idea

To accomplish this, we decided to leverage the Galaxy S9/S9+’s camera technology. We thought, if users could take a selfie and transform it into a custom avatar and emoji, it would add something unique to the fun of communicating. In addition, we wanted to find a way to track people’s actual expressions so that they would have at their fingertips emojis that actually felt like their own. It was also important that we give users the best tools to express themselves however they want to look.

04. The Solution

The first step toward this was to use augmented reality to generate an avatar. With facial recognition technology, the Galaxy S9/S9+’s camera starts by producing an initial 3D rendering, using a data-based machine learning algorithm. Next, the camera analyzes the 2D image of the user and pulls information from its deep learning database to create the user’s facial features on the final 3D AR Emoji. To bring people’s unique expressions to life, we decided to include Face Tracking Technology, which tracks over 100 facial features and allows us to capture individual emotions on the spot.

In addition to this, Blend Shape technology helps deliver an even greater range of different facial expressions. Based on 75 basic facial expressions, Blend Shape allows us to reflect a wider set of user emotions by blending multiple expressions together. This is also the technology that has made our Disney emoji offerings possible. Another major step we took was to go through numerous sampling tests. From our labs, we conducted at least 100 tests on a weekly basis to make sure the new feature included enough data to appeal to users regardless of their tastes.

05. The Result

The end product of this is AR Emoji, which among other features offers a realistic or cartoon-like option for creating your own personalized My Emoji. We wanted to offer emojis that would be able to deliver the same emotions across the world and went through a lot of research to understand how certain emotions are categorized and expressed on a fundamental level. In addition to the variety of emotions, AR Emoji also comes with a lot of versatility, allowing you to create the 3D avatars, either as a GIF, video or PNG file.

Samsung has embraced this transformation in Galaxy S9/S9+ with a touch of Disney magic and GIPHY fun. With the roll-out of these all-new AR Emojis – and a range of other new messaging innovations – the Galaxy S9/S9+ makes chatting with friends more fun than ever before. Along with improved hardware, the AR Emojis have turned the camera into a dashboard that makes creating and remixing your shareable moments even easier.

Bring Disney Magic to Life

Have you ever wanted to meet the most famous mouse in the world? Samsung’s latest partnership with Disney is bringing your favorite characters to life with an all-new set of Disney-themed AR Emojis.

Users can now use Mickey Mouse, Minnie Mouse, or Donald Duck AR Emojis by downloading the Disney characters from their camera and hitting record. The new Disney characters add to the variety of AR Emojis already available including your own personal My Emoji avatar and other cartoon characters.

AR Emojis use deep learning and facial recognition technology to map more than 100 facial features to produce a 3D replica, allowing users to create fun, customized messages to send to friends. So, whether you want to make your friends laugh as a silly rabbit, or tell them you love them as Mickey Mouse, there are more options than ever before on the Galaxy S9/S9+. Samsung and Disney plan to continue releasing more classic characters throughout the year, including favorites from The Incredibles, Frozen, and Zootopia.

Share More, Share Easier

Nothing captures a sarcastic quip or reaction like a good emoji or GIF. In fact, they are so good at capturing our emotions we send over 1 billion GIFs every day. With the Galaxy S9/S9+, Samsung has made them more customizable and shareable than ever before. The My Emoji feature automatically generates 18 emotions and saves them as custom stickers that are easily accessible from your Samsung Keyboard, along with other pre-loaded emojis and stickers. For now, there are 18 different custom stickers, but new updates will bring the total to 54. Since they use the AGIF format, you can share them with anyone, no matter what kind of smartphone they have.

Third-party applications also support My Emoji stickers through use of the Samsung Keyboard, so no matter the platform you use, you can send any of your custom emojis to friends and family. Besides the wide variety AR Emoji brings to your chats, Samsung has partnered with GIPHY so that sending your GIFs is even easier. So, if you’re just chatting with friends, or making a sarcastic quip to mom and dad, now you can send GIFs right from your Samsung Keyboard.

The new Samsung Keyboard features 30 categories of GIFs to choose from including happy, fun, thumbs up, and OMG. Whether you’re sassy, silly, or sarcastic, there is a GIF for you.

Choose from More Emojis Than Ever

While the Face with Tears of Joy may have captured the comedic highs and tragic lows of a generation, the time to update our emoji repertoire has come. Samsung has answered that call by adding more than 200 new emojis to the Samsung Keyboard.

In addition to the new emojis, Samsung has also opted for a redesign of the classic icons already available.

Whether you want to cheer up a friend, say “hello” to mom, or reach out to a lost connection, the Galaxy S9/S9+ brings you a plethora of options to express yourself, proving once again that it is built for the way we communicate today.