Bixby was created to provide a more personalized experience to enhance the way people interact with their phones and the world around them. Bixby is not an ordinary voice assistant, but rather an intuitive and comprehensive interface that leverages contextual awareness to learn users’ habits and respond accordingly.

Bixby offers four main features: Voice, Vision, Reminder and Home.

Voice: A New Way to Interact

Bixby Voice is easily activated. Users simply need to press and hold the dedicated hardware button on the side of the Galaxy S8, say “Bixby” or tap it on the Bixby Home screen to wake it up.

While most traditional smartphone-related tasks require touch activation, Bixby’s multi-modality lets users control their phone using voice and touch controls interchangeably for maximized convenience. Utilizing natural language understanding, Bixby has the ability to adapt to the unique speaking style of the user. But if Bixby doesn’t understand the user’s command, it will ask for more information so it can complete the task, rather than giving up.

Bixby brings a deeper degree of voice integration, but can also easily switch back and forth between each mode of interaction – from voice command to touch, text and back to voice.

Bixby works simultaneously across multiple apps and has the potential to control all settings and features of each service. So, for example, if a user wants to send pictures to a friend, they can select images in their Gallery by tapping them, then tell Bixby to send the selected photos to their designated contact. Meanwhile, Bixby continues to work in the background to continually support the user, without affecting other tasks they might be working on.

Vision: Information in a Snap

With its Vision interface, Bixby can detect what users are looking at and take action according to what they see. Using the Galaxy S8’s Camera app, Bixby makes it easier to search for images and information, shop online and even translate text.

A tourist in Paris, for instance, spots an interesting looking building and snaps a photo of it. Bixby uses the image to determine that the structure is a famous art museum and can then provide related information, such as the museum’s address, phone number and website. At lunch, the user can also leverage Vision to get a translation of the French menu in their own native language or learn more information about the bottle of wine they would like to order.

Shopping is just as easy. Users can take a photo of an item such as a watch or a book and use Vision to locate and purchase it online.* Users can also launch Vision for more in-depth searching of products they see in their phone’s Gallery or while browsing Samsung Internet. Vision can be activated to read QR codes and, in select markets, barcodes that direct users to related information.

Reminder: Friendly Alerts When You Need Them Most

Bixby also helps users remember important events and to-do tasks through its intelligent Reminder feature.

It does so by sending notifications to users reminding them about tasks at specified times or locations, or in certain circumstances. Say a user is out of milk. They can request that Bixby remind them to buy more once they get off work. Or, they can ask Bixby to set their alarm each night before they go to sleep.

When reading an article or watching a video, users can also use Reminder to mark their place should they need to stop and resume where they left off at a later time.

Users can access stored reminders via the Reminder app, Reminder widget, Reminder cards on the Bixby Home screen or the Galaxy S8’s edge screen.

Home: All Important Content in One Convenient Place

Bixby’s Home screen lets users see the content they care most about with just a single swipe on the home screen or a short press of the designated Bixby button.

Home integrates with select native Samsung apps to provide a seamless look at all of the activity across the Galaxy S8 through a customizable card-like interface. For example, Home’s contact card shows frequent and missed call contacts, while music-related cards provide options such as playing paused content. Facebook and other social media cards, meanwhile, display trending content and news cards announce the latest headlines.

But Home doesn’t just relay information; it puts it into context, paying attention to the user’s schedule, routine and personality each time it’s used. In this way, Bixby can actually anticipate what the user needs and respond accordingly.

Say someone often uses Uber to commute to work. Bixby will start to place an Uber card at the top of Home in the morning, with a fare estimate and request button. When it’s time for bed, Home might display a user’s alarm and health summary of daily activity. The more users access Bixby, the more it evolves and adapts.

With its intuitive, comprehensive features and ability to leverage contextual awareness, Bixby provides a more convenient, personalized mobile experience, changing the way people interact with their phones. As the Bixby ecosystem continues to grow, Samsung believes it will evolve from an interface for your smartphone to an interface for your life.

Listen to Injong Rhee discuss Bixby in depth in the interview below.

*At launch, Vision’s shopping service will be available in 11 countries, including the US, UK and Korea, and will expand to others in the future.

Harmonizing Art and Technology

The Galaxy S8 builds on Samsung’s heritage of creating stunning designs with a large Infinity Display – an end-to-end, bezel-less screen that spills over the phone’s sides. Available in 5.8-inch and 6.2-inch models, the seemingly endless Infinity Display defies the borders of conventional framework.

Inspired by nature, the Galaxy S8’s gentle surface screen forms a continuous silhouette with no protrusions or sharp angles, while the phone’s subtle curvature smoothly flows from the front to back with symmetrical balance. The harmonious aesthetic of the display reflects the overall design of the device, complementing its comfortable grip and usability while creating a more organic connection between the device and the user.

Visually obtrusive elements have been minimized to create more tangible screen space, thus enabling enhanced usability and a more immersive viewing experience. A home key embedded underneath the screen functions and responds to pressure just like a physical button would, while the recent and back buttons are now soft keys that can be customized in different colors.

Available in five colors including Midnight Black, the all-new Orchid Gray, Coral Blue, Arctic Silver, and Maple Gold, the Galaxy S8 is enhanced by a rich color palette that envelops the device, intensifying the depth and unity of the design. The phone’s metal frames balance nicely with the natural luster and depth of its glass body, creating a seamless color transition that brings together art and technology to complete the phone’s concept of oneness.

One Continuous User Experience

Designed around the Infinity Display, the UX of the Galaxy S8 was developed with a focus on rhythm to enhance the flow of user interactions and better fit the way people use their phones today. Like the design of the display itself, the UX is seamless, unifying both visual and audio components to establish one continuous experience.

The default wallpaper of the customizable Always-On Display complements the color of the phone, while its aesthetic transitions smoothly to the lock and home screens. Meanwhile, a new vertical swiping interface facilitates navigation between the home screen and app tray – a swipe up or down gives users quick and seamless access to their apps. Or, should they choose to do so, they also have the option of adding an apps screen button to the home screen.

The new UX also adds convenience by making it possible to simultaneously move multiple apps into a folder. Additionally, users can tap and hold an app icon to open a quick options menu that lets them perform tasks such as uninstalling apps or moving them to Secure Folder without having to open the app.

Even the app icons and typography of the Galaxy S8 incorporate the phone’s new design language through a play of continuous curves and lines to create depth.

The Galaxy S8’s ringtones have also been updated to better harmonize with the design of the device, while a new version of Samsung’s brand sound, “Over the Horizon,” has also been released. Arranged and recorded by Grammy Award-winner Jacob Collier, the 2017 version of “Over the Horizon” combines elements of contemporary jazz, pop and dance genres to create a tune that is at once youthful and innovative.

The replay is also available in 360. (Event starts at 1:42:47)

InJong Rhee, Executive Vice President, Head of R&D, Software and Services

Technology is supposed to make life easier, but as the capabilities of machines such as smartphones, PCs, home appliances and IoT devices become more diverse, the interfaces on these devices are becoming too complicated for users to take advantage of many of these functions conveniently. User interface designers have to make tradeoff decisions to cram many functions into a small screen or bury them deeper in layers of menu trees. Ultimately users are at the mercy of the designers with an increasingly steep curve that makes learning a new device difficult. This is the fundamental limitation of the current human-to-machine interface.  Since Samsung makes millions of devices, this problem impacts the core of our business.

Samsung has a conceptually new philosophy to the problem:  instead of humans learning how the machine interacts with the world (a reflection of the abilities of designers), it is the machine that needs to learn and adapt to us.  The interface must be natural and intuitive enough to flatten the learning curve regardless of the number of functions being added. With this new approach, Samsung has employed artificial intelligence, reinforcing deep learning concepts to the core of our user interface designs. Bixby is the ongoing result of this effort.

Bixby will be a new intelligent interface on our devices. Fundamentally different from other voice agents or assistants in the market, Bixby offers a deeper experience thanks to proficiency in these three properties:

1. Completeness

When an application becomes Bixby-enabled, Bixby will be able to support almost every  task that the application is capable of performing using the conventional interface (ie. touch commands).  Most existing agents currently support only a few selected tasks for an application and therefore confuse users about what works or what doesn’t work by voice command. The completeness property of Bixby will simplify user education on the capability of the agent, making the behaviors of the agent much more predictable.

2. Context Awareness

When using a Bixby-enabled application, users will be able to call upon Bixby at any time and it will understand the current context and state of the application and will allow users to carry out the current work-in-progress continuously.  Bixby will allow users to weave various modes of interactions including touch or voice at any context of the application, whichever they feel is most comfortable and intuitive.  Most existing agents completely dictate the interaction modality and, when switching among the modes, may either start the entire task over again, losing all the work in progress, or simply not understand the user’s intention.

3. Cognitive Tolerance

When the number of supported voice commands gets larger, most users are cognitively challenged to remember the exact form of the voice commands. Most agents require users to state the exact commands in a set of fixed forms. Bixby will be smart enough to understand commands with incomplete information and execute the commanded task to the best of its knowledge, and then will prompt users to provide more information and take the execution of the task in piecemeal. This makes the interface much more natural and easier to use.

We know that adopting new ways to interact with your devices will require a change in user behavior. The inconvenience of learning a new interface can cause friction and force users to revert back to old habits (e.g. the touch interface). At the same time we believe the key to success for a new voice interface is to design a scheme that reduces friction and makes the experience significantly more rewarding than the existing interface. So at its core, Bixby will help remove friction. It will simplify user education with new voice interfaces and will make using your phone even more seamless and intuitive.

Another example of removing friction will be the dedicated Bixby button that will be located on the side of our next device. Confusion around activating a voice interface is a barrier we have removed to make it feel easier and more comfortable to give commands. For example, instead of taking multiple steps to make a call – turning on and unlocking the phone, looking for the phone application, clicking on the contact bar to search for the person that you’re trying to call and pressing the phone icon to start dialing – you will be able to do all these steps with one push of the Bixby button and a simple command.

There has been a lot of excitement and speculation about what we will deliver with the launch of the Galaxy S8 later this month, especially due to the advancements in artificial intelligence. We do have a bold vision of revolutionizing the human-to-machine interface, but that vision won’t be realized overnight. Ambition takes time.

Bixby will be our first step on a journey to completely open up new ways of interacting with your phone. At the launch of the Galaxy S8, a subset of preinstalled applications will be Bixby-enabled. This set will continue to expand over time. Our plan is to eventually release a tool (in SDK) to enable third-party developers to make their applications and services Bixby-enabled easily.

Starting with our smartphones, Bixby will be gradually applied to all our appliances.  In the future you would be able to control your air conditioner or TV through Bixby. Since Bixby will be implemented in the cloud, as long as a device has an internet connection and simple circuitry to receive voice inputs, it will be able to connect with Bixby. As the Bixby ecosystem grows, we believe Bixby will evolve from a smartphone interface to an interface for your life.

Bixby is at the heart of our software and services evolution as a company.  We are fundamentally and conceptually changing our attitude toward software and services and working hard on innovation throughout all aspects of our mobile ecosystem. Our investment in engineering resources speaks for itself – we have thousands of software developers supporting this effort. This is something that I’m very excited about. Innovating in software and services enables opportunities for creativity and the ability to build new experiences from the ground up.  With the continued investment from Samsung on artificial intelligence, the possibility of what Bixby can become is endless.