[Interview] The Galaxy S9’s AR Emoji Designers: Creating New Tools for Visual Communication

“The essence of communication is self-expression, so we are making continual improvements to the AR Emoji feature so that it caters to our consumers' unique qualities.”

As people search for even more ways to express their distinct personalities using their smartphones, Samsung has created the Galaxy S9’s AR Emoji feature as a new tool for visual communication. Generated from the image of an individual user, the customizable My Emojis and their stickers reflect and mimic a user’s facial features and movements, creating a personal 3D avatar that is exclusive to them and personalized stickers they can use in messenger applications.

To learn more about this fun and unique new feature, we sat down with members of the AR Emoji’s design team, Wonhyung Cho and Hye-Bong Kim, from the Global Product Planning Group, and Hoik Hwang and Jiyoun Kim, from the UX Design Group.

Hoik Hwang, Hye-Bong Kim, Wonhyung Cho and Jiyoun Kim (from left)

Q. What was the inspiration behind the Galaxy S9’s AR Emoji feature?

Wonhyung Cho: People are communicating more and more through their smartphones, sending messages to each other throughout the day. Many of us are also looking for new ways to express a range of emotions in the short messages we send each other, and in this context, emoticons and emojis have become one of the most popular ways to express ourselves. As a next step, we wanted to go further and blend the emerging technology of AR with emoticon and emoji tools for a new form of visual communication.

Q. What makes the Galaxy S9’s AR Emoji special?

Cho: We thought for a long time about what the word ‘communication’ means today and it became clear that a large part of communication is ‘self-expression’. As a team, we wanted to make the feature as personal as possible by capturing our unique emotions, using an innovative technology like AR. In creating a My Emoji which captures your own distinct facial features, AR Emojis can provide a truer form of self-expression.

Hye-Bong Kim: The camera uses facial recognition technology to analyze a photo of your face and map more than 100 facial features. Using these data points, a 3D modeling form specialized for the contours of the human face creates a 3D My Emoji and 18 My Emoji stickers which reflect your unique facial features.

Q. How have users reacted to the feature?

Cho: AR Emoji has generated a lot of curiosity – it’s such a fun feature and many people want to see how their My Emoji looks.

Hoik Hwang: For everyone, the question is not ‘does this look like me?’, but rather ‘does this look like I want it to?’ AR Emoji actually creates two types of My Emoji: one that is realistic and one that is more like a cartoon. Some people like the first kind, while others prefer the second.

We want AR Emoji to be the best it can be. A close resemblance to the user isn’t the only thing we considered when developing the feature. Our goal was for creators to be able to use the AR Emoji to express themselves, however they want to look.

We’re currently working to allow users to make their My Emojis better express the mood they want them to.

Q. When it came to UX design, what was your key consideration?

Jiyoun Kim: AR Emoji uses the Galaxy S9’s camera, so we wanted to integrate it with the camera’s UX. From start to finish, the user’s journey while creating their My Emoji and My Emoji stickers has to be simple and engaging.

Then we thought about how people might want to use the feature to express themselves in their daily lives. AR Emoji offers a new way tell our stories and because of that we felt that making it available across many platforms was important.

We created 18 stickers, which each communicate a different emotion. They can be stored in your smartphone Gallery application, can be used freely in your different applications and messaging services and are available as both PNG and GIF files.

Q. How did you go about designing the 18 My Emoji stickers?

Hwang: Our goal was to create emoji stickers that would communicate the same emotions across the world – and would mean the same thing in any country. We read a lot of research as we tried to categorize human emotions and to determine what the visual representations of fundamental human feelings – things like joy, anger, sorrow and pleasure – should be.

In time, the number of stickers will increase to 54 so that even more emotions will be expressed using the emoji stickers.

Q. Did you encounter any difficulties while designing AR Emoji?

Hwang: AR is an innovative technology and the AR field is quite new. We collected data from people around the world, interviewing them to discover their needs and ensure that the feature delivered on those needs.

As a team, we continue to collect this data so that we can make AR Emojis even better. User feedback helps us discover not only what consumers really want, but how we can make adjustments to this new technology.

Q. Can you elaborate on your experiences working with Disney to create the Mickey and Minnie Mouse AR Emojis?

Hwang: Like lots of people, I grew up with Mickey Mouse. I never thought that I would be working on a project like this with Disney when I grew up. When we first started I was really excited and couldn’t stop thinking, “Is this really happening?”

Throughout the project, we engaged in a great deal of knowledge sharing with Disney. For example, there’s a feature known as Blend Shape that allows the feature to recreate 75 basic user facial expressions – for example, if you combine two different facial expressions, you will produce a completely new one – explaining how this feature worked was time-intensive. But by working together, we were able to create these unique Mickey Mouse and Minnie Mouse AR Emojis.

Q. As AR Emoji designers, can you share some tips for creating an AR Emoji?

Jiyoun Kim: First, you need to be in a place with plenty of light – the brighter the better – like when you want to take a selfie. Make sure your face is not hidden by your hair or any other objects. Position your face fully inside the circular frame on the screen and take a photo. Just relax and be natural. After your My Emoji has been created, you can make manual adjustments to skin tone, hair style and color, clothing and other features, to your liking.

Q. Do you have any advice for making the most of the feature?

Hwang: I have seen videos where people create stories by recording their own voice for the dialogue, and use AR Emojis to act out the parts, which are really fun to watch. It’s a new and innovative way to use My Emoji. You can use them to present something your own behalf and remain almost anonymous at the same time.

Hye-Bong Kim: You don’t have to make just one AR Emoji. You can make several characters, as the mood takes you. Give your My Emoji a hairstyle you want try or a funky outfit you would never normally wear. Create AR Emojis of family or friends and share them with each other.

We like to make a game out of changing the hair styles and other features of the My Emojis, then guessing which one belongs to which team member, which is quite amusing.

Q. Can you suggest how users can get the most out characters like Mickey Mouse?

Cho: They are perfect for creating your own cartoon or animation. Since they are designed to mirror human facial expressions, you can make your characters express a range of emotions and make up a whole story using them.