Visualize Your New Room

Samsung’s AR service, Bixby Vision¹, acts as a visual guide, allowing you to see more with your smartphone camera. The home décor function of the Galaxy S10’s Bixby Vision can show you how your home might look with new home appliances and furniture, virtually positioning them wherever you wish to see them. To enable the function, simply open up Bixby Vision, select the home décor function from the apps menu and point your camera to the area where you want to place home appliances or furniture. After choosing an appliance or piece of furniture, the object will appear wherever you tap on the screen. You can choose sofas, rugs, tables, fridges, or any other home product that you want, and virtually place them wherever you like.

The discovery doesn’t end there: Bixby Vision offers even more services that allow you to visualize your new appliances and furniture from the comfort of your own home. The AR service can apply a 3D effect to key Samsung home appliances, allowing you to see how doors open, for example. Collaborating with U.S. online furniture company, Wayfair, Bixby Vision’s home décor function runs automatically whenever you encounter 3D content on the website within the Samsung Internet app, enabling you to see what the product would look like in your room without leaving your house.

“We are planning to increase the types of home appliances and furniture that support the home décor function,” said Martin Lee of Product Strategy team at Samsung Electronics.

Check for the Right Fit

With the Galaxy S10 5G, you can make speedy assessments of an object’s dimensions without frantically searching for the tape measure. Using the 3D Depth Camera on the Galaxy S10 5G, the Quick Measure app shows you the height, width and length of the object captured by your camera. This feature makes it easier for you to buy a mattress that fits your bed frame, or a set of chairs that are the right height for your kitchen table.

¹ Service availability and content provider may vary by country and device.

Galaxy S9 and S9+ users can create AR (augmented reality) avatars of Anna, Elsa, Olaf and Kristoff. To start animating messages as one of your favorite Disney Frozen characters, just open the Galaxy S9/S9+’s AR Emoji mode, tap the plus icon on the right-hand side to access the Galaxy Apps Store and download the Disney Frozen pack.

AR Emojis are a fun and versatile way to personalize communications with friends, harnessing expressions through facial recognition technology, and are shareable across all platforms. The Disney Frozen AR Emoji pack is now available for download on the Galaxy Apps Store. Be sure to stay tuned for more AR Emoji releases from the collaboration coming soon.

With the update, after users snap a selfie to generate their AR Emoji, they’ll be able to easily make detailed adjustments to their AR Emoji’s hair, ears and facial features. This means that, for instance, when a child creates an AR Emoji, they’ll be able to adjust the shape of their face so that it more closely mirrors their own. And if you feel like your AR Emoji’s eyes, nose, lips, cheeks or other facial features are in need of a touch-up, the new My Emoji Editor function offers simple tools to fine-tune these distinguishing characteristics.

The update also enhances AR Emoji’s face-tracking function with a 65-percent increase in ‘landmark detector’ points. This will ensure that features such as your eyes and mouth are tracked more precisely. And thanks to the update’s frame rate boost, your AR Emoji will be able to mimic your movements in a manner that’s smoother and more natural.

^*Availability may vary depending on market.

The smartphones’ camera app update, now available in the ‘My apps’ section of the Galaxy Apps store, has expanded AR Emoji’s library of pre-set expressions from 18 to 36. Samsung also plans to add 18 more expressions in the coming months, which will bring the feature’s total number of emotion-capturing sticker options to 54.

Once users have downloaded the update, they’ll be able to access and share the new AR Emoji stickers directly from the keyboard, or copy and paste them from the Gallery app into their favorite messaging platform.

Get a sense of what the new AR Emoji stickers look like in the sample conversation below.

To learn more about this fun and unique new feature, we sat down with members of the AR Emoji’s design team, Wonhyung Cho and Hye-Bong Kim, from the Global Product Planning Group, and Hoik Hwang and Jiyoun Kim, from the UX Design Group.

Hoik Hwang, Hye-Bong Kim, Wonhyung Cho and Jiyoun Kim (from left)

Q. What was the inspiration behind the Galaxy S9’s AR Emoji feature?

Wonhyung Cho: People are communicating more and more through their smartphones, sending messages to each other throughout the day. Many of us are also looking for new ways to express a range of emotions in the short messages we send each other, and in this context, emoticons and emojis have become one of the most popular ways to express ourselves. As a next step, we wanted to go further and blend the emerging technology of AR with emoticon and emoji tools for a new form of visual communication.

Q. What makes the Galaxy S9’s AR Emoji special?

Cho: We thought for a long time about what the word ‘communication’ means today and it became clear that a large part of communication is ‘self-expression’. As a team, we wanted to make the feature as personal as possible by capturing our unique emotions, using an innovative technology like AR. In creating a My Emoji which captures your own distinct facial features, AR Emojis can provide a truer form of self-expression.

Hye-Bong Kim: The camera uses facial recognition technology to analyze a photo of your face and map more than 100 facial features. Using these data points, a 3D modeling form specialized for the contours of the human face creates a 3D My Emoji and 18 My Emoji stickers which reflect your unique facial features.

Q. How have users reacted to the feature?

Cho: AR Emoji has generated a lot of curiosity – it’s such a fun feature and many people want to see how their My Emoji looks.

Hoik Hwang: For everyone, the question is not ‘does this look like me?’, but rather ‘does this look like I want it to?’ AR Emoji actually creates two types of My Emoji: one that is realistic and one that is more like a cartoon. Some people like the first kind, while others prefer the second.

We want AR Emoji to be the best it can be. A close resemblance to the user isn’t the only thing we considered when developing the feature. Our goal was for creators to be able to use the AR Emoji to express themselves, however they want to look.

We’re currently working to allow users to make their My Emojis better express the mood they want them to.

Q. When it came to UX design, what was your key consideration?

Jiyoun Kim: AR Emoji uses the Galaxy S9’s camera, so we wanted to integrate it with the camera’s UX. From start to finish, the user’s journey while creating their My Emoji and My Emoji stickers has to be simple and engaging.

Then we thought about how people might want to use the feature to express themselves in their daily lives. AR Emoji offers a new way tell our stories and because of that we felt that making it available across many platforms was important.

We created 18 stickers, which each communicate a different emotion. They can be stored in your smartphone Gallery application, can be used freely in your different applications and messaging services and are available as both PNG and GIF files.

Q. How did you go about designing the 18 My Emoji stickers?

Hwang: Our goal was to create emoji stickers that would communicate the same emotions across the world – and would mean the same thing in any country. We read a lot of research as we tried to categorize human emotions and to determine what the visual representations of fundamental human feelings – things like joy, anger, sorrow and pleasure – should be.

In time, the number of stickers will increase to 54 so that even more emotions will be expressed using the emoji stickers.

Q. Did you encounter any difficulties while designing AR Emoji?

Hwang: AR is an innovative technology and the AR field is quite new. We collected data from people around the world, interviewing them to discover their needs and ensure that the feature delivered on those needs.

As a team, we continue to collect this data so that we can make AR Emojis even better. User feedback helps us discover not only what consumers really want, but how we can make adjustments to this new technology.

Q. Can you elaborate on your experiences working with Disney to create the Mickey and Minnie Mouse AR Emojis?

Hwang: Like lots of people, I grew up with Mickey Mouse. I never thought that I would be working on a project like this with Disney when I grew up. When we first started I was really excited and couldn’t stop thinking, “Is this really happening?”

Throughout the project, we engaged in a great deal of knowledge sharing with Disney. For example, there’s a feature known as Blend Shape that allows the feature to recreate 75 basic user facial expressions – for example, if you combine two different facial expressions, you will produce a completely new one – explaining how this feature worked was time-intensive. But by working together, we were able to create these unique Mickey Mouse and Minnie Mouse AR Emojis.

Q. As AR Emoji designers, can you share some tips for creating an AR Emoji?

Jiyoun Kim: First, you need to be in a place with plenty of light – the brighter the better – like when you want to take a selfie. Make sure your face is not hidden by your hair or any other objects. Position your face fully inside the circular frame on the screen and take a photo. Just relax and be natural. After your My Emoji has been created, you can make manual adjustments to skin tone, hair style and color, clothing and other features, to your liking.

Q. Do you have any advice for making the most of the feature?

Hwang: I have seen videos where people create stories by recording their own voice for the dialogue, and use AR Emojis to act out the parts, which are really fun to watch. It’s a new and innovative way to use My Emoji. You can use them to present something your own behalf and remain almost anonymous at the same time.

Hye-Bong Kim: You don’t have to make just one AR Emoji. You can make several characters, as the mood takes you. Give your My Emoji a hairstyle you want try or a funky outfit you would never normally wear. Create AR Emojis of family or friends and share them with each other.

We like to make a game out of changing the hair styles and other features of the My Emojis, then guessing which one belongs to which team member, which is quite amusing.

Q. Can you suggest how users can get the most out characters like Mickey Mouse?

Cho: They are perfect for creating your own cartoon or animation. Since they are designed to mirror human facial expressions, you can make your characters express a range of emotions and make up a whole story using them.

Samsung Newsroom recently sat down with members of Samsung’s Global Product Planning Group – Nayoung Kim, Taihoon Kim, Joohyun Kim and Jonathan Min – who discussed the process of developing a device that’s more powerful, personal and in tune with how we use our smartphones today.

(From left) Taihoon Kim, Jonathan Min, Nayoung Kim and Joohyun Kim, of Samsung’s Global Product Planning Group, helped shape the Galaxy S9’s design.

Q. What inspired you to develop the Galaxy S9’s ‘visual communication’ features?

Nayoung Kim: Current mobile trends are being dictated by Generation Z, but they are very selective. In our research, we took a close look at how Generation Z are communicating, as well as what kind of apps and functionality they most enjoy. We eventually uncovered an increase in the use of visual forms of communication. For example, members of the social media generation will often use emoticons rather than texts, and they enjoy sharing fun images. So, to cater to this preference, we focused on adding new functionalities to the camera.

Q. Could you elaborate on the camera’s development process?

Nayoung Kim: When designing the camera, our team focused on three key points. It needed to be capable of recording dynamic video – the type of content that the younger generation loves to consume; it needed to be able to capture low-light photos, which should be a fundamental feature of any camera; and it needed to be easy to use and offer fun new features, like AR Emoji.

Taihoon Kim: The product development process for the Galaxy S9 was quite different from that of the previous years, as team members with different expertise strived to discover the visual experiences of everyday life through their own eyes, and how that might translate to camera functionality. These diverse insights enabled us to develop a camera catering to wide needs.

Q. How did you refine the Galaxy S9’s user experience?

Taihoon Kim: We focused on enhancing the End-to-End scenario of the user experience. For the camera, that involved ensuring easy-to-use Super Slow-mo video functionality, with immediately accessible videos that could be directly uploaded to social media through one intuitive process.

You see this sort of seamlessness with AR Emoji as well. Once you scan your face, you’re instantly given the option to customize your 3D avatar to your liking, and presented with 18 shareable GIFs encompassing a wide range of expressions.

Nayoung Kim: We tested the device’s usability extensively, and in various contexts, to ensure the processes were intuitive and organic, and could work across all aspects of daily life. As example, I brought a prototype of the phone with me when I went sledding with my children. The first-hand experience of recording a Super Slow-mo video of my children rocketing down the hill gave birth to the idea of an auto motion-detect function able to capture those sorts of fleeting moments.

Q. Which Galaxy S9 feature has received the biggest response from consumers?

Jonathan Min: We’ve found that users are very interested in the AR Emoji feature, which we first started working on in 2016. People are used to expressing themselves with emojis and stickers, but this feature allows them to create and express themselves through emojis that resemble their own faces, in a way that’s much more personal.

When we started it was new, so everything had to be built from the ground up, and it was difficult to predict consumers’ feedback. It was also our development team’s first time working on something like this, so it took a lot of trial and error to bring the function to life.

Nayoung Kim: Our most important goal is to provide users with new experiences by launching the product, enhancing it based on the market’s reactions, and continuing to develop innovative features. AR Emoji will continue to be enhanced in ways that reflect consumers’ needs.

Q. Do you have a favorite feature?

Jonathan Min: I personally use AR Emoji stickers the most, whenever I message my friends and family.

Taihoon Kim: I like the Super Slow-mo function the most. It’s not only great for capturing dynamic actions, but life’s little moments as well, such as the signature facial expression of your loved ones. And the Galaxy S9’s ability to display Super Slow-mo clips on the lock screen creates a touching way to enjoy videos of them.

Joohyun Kim: We redefined the design of the Infinity Display to make it even more beautiful and to minimize any ‘distracting’ elements. Subtle adjustments, such as ensuring that the iris scanner is hidden, and that the bezel and display are the same color, helped us deliver a display that makes it easy for viewers to immerse themselves in their favorite content.

Q. Going forward, how does Samsung plan to ensure that Galaxy smartphones remain among the most innovative in the market?

Taihoon Kim: It will be important to stay a step ahead in innovative features, as well as time them to the market appropriately to grab consumer attention. Consumers can experience visual communications features directly themselves, without an explanation. We plan to continue exploring dynamic visual features, given their unique appeal to the young generation of consumers, as well as integrate new technologies like AR (augmented reality) and AI (artificial intelligence) into our devices.

Nayoung Kim: The basic features of smartphones have been refined to the point that consumers are largely satisfied. What matters is not how diverse the offered features are, but whether the features offered to deliver a high level of satisfaction. At this point, we need to think about the next phase for smartphones, which will involve delivering innovations that really resonate with the new generation of users.

Joohyun Kim: Smartphones have become essential to daily life. As product planners, it’s our duty to constantly look for ways to improve on ‘basic’ features – those that we use every day – while also identifying brand new functions.

1. Faster, more transparent machine learning

Artificial intelligence (AI) will dramatically expand within the next 12 months. It is already changing the way people interact with a number of applications, platforms and services across both consumer and enterprise environments.

In the next couple of years, there will be new approaches on two fronts. Firstly, less data will be required to train an algorithm. This means an image recognition system that currently needs 100,000 images to learn how to operate will only need a small fraction of that number. This will make it easier to quickly implement powerful machine learning systems.

Secondly, the technology will become more transparent. Advances in technology will mean researchers will be able to open the black box of AI and more clearly explain why a particular model made the decision it did. Currently, a lot of academia and start-ups are putting much effort into understanding how a machine makes decisions, how the models are learning from the data and what are the parameters of data that influence the models.

Scott Phoenix, The CEO of Vicarious, makes a presentation about human-level intelligent robots at the Samsung CEO Summit last October in San Francisco. (source: www.vicarious.com)

Samsung plans to build an AI platform under a common architecture that will provide the deepest understanding of usage context and behaviors. This is one of the core strategies to make the user-centric AI ecosystem. Samsung NEXT has also invested in various companies innovating in the field, including Vicarious, a company developing neuroscience-based artificial general intelligence (AGI) for robots for simpler deployment with faster training and Bonsai, which develops an AI platform that empowers enterprises to create, deploy and manage AI models, and FloydHub, a start-up that has developed a cloud service for machine learning.

2. New AR and VR form factors and viewing models

Both augmented reality (AR) and virtual reality (VR) are increasingly being relied upon to create more immersive worlds where technology enables users to get more hands-on with virtual overlays and environments. In the case of AR, devices won’t remove us from our world, but will rather enable us to have objects appear as if they were really there.

2018 will witness more developers embracing AR, starting to make interesting applications moving beyond the world of gaming. One such example is a furniture company planning to make its full catalogue available in AR. Samsung NEXT has invested in companies like 8i, which provides a platform that enables true 3D (fully volumetric) video capture of people, allowing viewers to walk around as real humans in VR and AR.

8i’s Holo augmented reality application enables digital recreation of people and characters to be seen in the real world through a smartphone camera. (source: www.8i.com)

Head Mounted Displays (HMDs) will see foundational technology improvements in the quality of their displays, sensors, and materials. In 2018, there will be a lot of excitement in the industry in the form of M&A and investment activities. “For VR, we will see more standalone devices, falling between existing HMDs powered by mobile phones, and high-end hardware connected to powerful PCs. This will enable more people to experience the technology in new ways,” said Ajay Singh, Samsung NEXT Ventures.

3. Blockchain to look beyond cryptocurrencies

“In 2017, we saw blockchain technology increasingly applied to develop unbanked countries and communities,” said Raymond Liao from Ventures. “With underpinnings in peer-to-peer transactions, blockchain has the power to democratize transactions by removing the middleman and reducing the needless fees that so frequently hamstring those deprived of banking services.”

Cryptocurrency is the dominant killer application for blockchain up to now. However, we will see blockchain entrepreneurs and decentralization idealists, freshly financed by token sales, marching to either empower consumers against the one-sided data monetization paradigm, or break up enterprise data silos in, e.g., supply chain and healthcare industries.

Samsung’s focus on security will be an advantage for the company as far as blockchain is concerned. The elephant in the room around blockchain is that the entire technology is only as secure as the users’ keys. Samsung’s technology enables enterprise customers to be assured of a certain level of security in how their employees interact with their blockchain-based apps. Furthermore, Samsung NEXT includes in its portfolio companies like HYPR that provides enterprise with enhanced security and user experience using blockchain and Filament that secures Internet of Things (IoT) devices with their blockchain protocol.

4. IoT to put power in the hands of healthcare patients

Healthcare is an industry that is ripe for disruption. We will begin to see the power of IoT in healthcare with the emergence of inexpensive, continuous ways to capture and share our data, as well as derive insights that inform and empower patients. Moreover, wearable adoption will create a massive stream of real-time health data beyond the doctor’s office, which will significantly improve diagnosis, compliance and treatment. In short, a person’s trip to the doctor will start to look different – but for the right reasons.

Samsung is using IoT and AI to improve efficiency in healthcare. Samsung NEXT has invested in startups in this area, such as Glooko which helps people with diabetes by uploading the patient’s glucose data to the cloud to make it easier to access and analyse them. Another noteworthy investment in this space from Samsung NEXT is HealthifyMe, an Indian company whose mobile app connects AI-enabled human coaches with people seeking diet and exercise advice.

(source: www.healthifyme.com)

Samsung is uniquely positioned among tech companies in that it already has a significant business in healthcare. The company has solutions in wearables, hospital screens and tablets, and X-ray and MRI machines. By tying all these solutions together and cooperating with other partners, it will enable patients to manage their health from their own devices.

5. IoT breaks free from homes and enters the city

In the next couple of years, one should expect to see IoT transform urban environments thanks to the combination of learnings from smart homes and buildings, and the proliferation of 5G. Transformation will happen in waves, starting with innovation that requires fewer regulations. It is expected to impact the daily life of the community in meaningful ways, such as parking solutions, mapping, and bike share schemes.

Samsung NEXT already has various IoT investments including Stae for data-driven urban planning, and Swiftly that provides enterprise software to help transit agencies and cities improve urban mobility.

The company has its own IoT platform SmartThings—an acquisition that came through the Samsung NEXT team. The platform is connected to ARTIK for enterprises and HARMAN Ignite’s connected car platform, creating a comprehensive IoT ecosystem. Based on its progress on IoT, Samsung showcased its vision for ‘Samsung City 2020’ at this year’s CES, which is on its way to realization.

From the latest in VR/AR, IoT, healthcare and more, Samsung selected the seven new startups for investment based on their business potential and contribution to innovation:

• Hyperity – A VR/AR solution to remotely control smartphones and desktops through virtual screens without a physical monitor.

• Linkface– A unique VR solution that can detect eye and facial muscle movement and let users directly control VR content.

• PIXELRO – A smartphone-based solution to provide a vision correction solution to presbyopia, replacing regular eyeglasses.

• BlueFeel – A personal portable air purifier that doesn’t block the user’s mouth, developed as a response to users who dislike the discomfort of regular dust masks..

• Defind – A smart shoe matching service that carries out a 3D scan of a customer’s foot to enable them to buy shoes that fit perfectly

• Soft Launch – A social media-based recommendation service for restaurants and stores aiming to combat fake reviews, and developed by employees from multiple Samsung affiliates

• 1Drop – An ultra low-cost blood glucose measurement solution using LED and the camera of a smartphone

A Collaborative Approach

The prospective entrepreneurs were provided with intensive training and preparation on key aspects of running a business with the help of experts before launching their startups. They also engaged in various talk sessions with former colleagues with success spinning off businesses to obtain know-how.

“We have provided the support to establish 32 C-Lab alumni startups over the past two years and based on our valuable past experience, we are planning to build up a more profound and actionable program to nurture employees’ ideas and launch new startups,” said Jaiil Lee, Vice President and Head of the Creativity & Innovation Center at Samsung Electronics.

C-Lab alumni startups have performed well in recent years, securing additional global funding, increasing company valuation and opening up unexpected business opportunities. For example, 360-degree camera manufacturer Link Flow started as a business for travellers in their 30’s and 40’s, but after being spotted by the security maintenance industry, underwent further iterations and will be unveiled as an official product at this upcoming CES 2018.

Created in December 2012, the C-Lab is an in-house startup incubation program that nurtures a creative organizational culture and innovative ideas among Samsung employees. The spin-off policy was introduced in 2015 and since its inception C-Lab alumni startups have been striving to open a new startup ecosystem.