During the event, Samsung highlighted its consumer-centric approach to innovation. In particular, the company showcased how advancements in AI, IoT and security work alongside the breadth of Samsung’s portfolio to create experiences that fit into how people live, ultimately unlocking a new era of connectivity and collaboration.

“We are proud of the pioneering work we are able to do in partnership with the creative developer community,” said DJ Koh, President and CEO of IT & Mobile Communications Division, Samsung Electronics. “As society becomes more closely connected through new tools and devices, we want to make people’s lives easier and smarter – wherever they are.”

Platforms Designed for Connected Experiences

A strong platform of technologies across AI, IoT and security enhances the consumer experience and accelerates the ability of developers to build creative applications. Samsung detailed its plans to strengthen core platform technologies such as Bixby, SmartThings and Samsung Knox.

During the SDC21 keynote, Samsung announced improvements to Bixby, which has become a central part of the user experience for more than 300 million Samsung devices. Bixby now uses on-device AI and deep neural networks to process commands on devices themselves. Thanks to on-device AI, this development makes the service up to 35% faster than before.

For developers, Samsung also announced the Bixby Home Platform, which serves as an intelligent layer between Bixby’s natural language processing and SmartThings. With the platform, voice commands can now be executed more intelligently, taking into account the state and context of various devices.

Samsung made further announcements with regard to SmartThings, including how the platform will be fully integrated with Matter, the newest standard in smart home compatibility. Matter solves developer interoperability challenges and with an expansive list of partners, will be the catalyst that makes smart home connectivity nearly effortless. SmartThings Hub software will also be embedded into the Samsung product portfolio, which includes Samsung Smart TVs and Family Hub refrigerators.

Samsung believes that consumer data and personal privacy is just as important as the experiences they enable. Samsung is committed to providing a seamless consumer experience, and to ensuring that its devices and services are secure from beginning to end. Samsung Knox, the company’s defense-grade security platform, offers built-in protection for more than 100 million Samsung devices. To further secure consumer data, the Samsung Knox Vault comes in a combination of security-specific hardware (a new secure processor and isolated secure memory) and new integrated software that shields the most secure data from the Android OS and applications. During the SDC21 keynote, Samsung highlighted the importance of collaboration and its philosophy of open innovation to improve security and privacy, reiterating its work to continue releasing security analysis tools as open source projects.

Samsung also outlined its Privacy Principles at SDC21, highlighting the three commitments that guide all innovation at the company: Protection, Choice and Transparency. With these principles, consumers, developers and partners are given the information needed to make decisions about how data is managed. The company offers users the option to review and manage their permissions from the privacy setting on their Galaxy devices and Smart TVs and discloses Samsung’s privacy policies through the Privacy Portal. Going forward, Samsung will collaborate with global security experts while continuing to release open source security analysis tools to make its ecosystems more secure.

Tizen Takes Smart TV to New Heights

Based on its ‘Screens Everywhere, Screen for All’ vision, Samsung continues to build on its position as the world leader in TVs by developing new experiences that enhance connectivity, functionality and immersive entertainment. At SDC21, Samsung introduced various ways that Samsung TVs are enabling new experiences, including:

• Video communication based on partnership with Google to allow users to enjoy better video calling on TVs via AI-enabled focus and zoom features.
• Samsung Health on Smart TVs that connects with a webcam to analyze movements and provide real-time feedback during a workout.
• More realistic gaming thanks to HDR10+, with automated settings for low latency and HDR calibration.

For enterprises, Samsung will launch Tizen for Business. This service supports interactive e-boards in schools, kiosks in malls, and even displays in restaurants, airports and subways. With new management portals, businesses can conduct real-time monitoring and control screens remotely, easily applying settings to multiple devices with one simple touch.

With the Tizen TV Platform Licensing, manufacturers who want to introduce a premium TV platform like Tizen can do so quickly with minimal cost. This enables them to use the Tizen TV presence to promote their products in the Smart TV market, and ultimately, help more consumers enjoy Tizen.

Innovating Across the Galaxy Ecosystem

Samsung provides a user experience that meets the needs of all users and devices. And with an expanding range of device types and form factors, developers have a breadth of opportunities to innovate in new ways.

At SDC21, Samsung introduced the new One UI 4, the latest version of its user interface for mobile devices. One UI 4 will provide new levels of customization to enable users to create an experience and a device that is truly theirs. It combines Samsung’s leading privacy and security features with options for users to personalize permissions so that they are always in control of their data when using a Samsung device.

Samsung demonstrated the possibilities of One UI 4 to developers at the event, particularly for foldable devices such as the Galaxy Z Fold3 and Z Flip3. Thanks to One UI 4, users will be able to make use of features such as Flex mode and Multi-Active Window, which enable users to enjoy more intuitive and user-friendly experiences that are unique to the foldable form factor. SDC21 also included a Galaxy Watch session that showcased developer tools to help them build new apps in the new watch app ecosystem.

Samsung invites developers and partners on its journey to bring new and secure experiences to life. For more information about the Samsung Developer Conference 2021, please visit developer.samsung.com.

Voice Recognition With Bixby: More Convenient Than Touch

Artificial intelligence (AI) enhances the performance of all kinds of devices essential to our daily lives, including smartphones, tablets, TVs and more. In particular, AI-powered voice recognition facilitates more seamless interactions between these devices. Principal Engineer Joohwan Kim from the AI ​​Client Development Group, Mobile Communications Business at Samsung Electronics, shares below how the Bixby voice assistant platform is set to enable more convenient experiences – using just your voice.

Q: What does the newly updated Bixby voice assistant platform look like?

Bixby aims to be the best voice interface able to control multiple Samsung devices using only a small amount of data. From this standpoint, we focused on providing a unique user experience that only Galaxy can provide when developing Samsung’s latest devices. For example, Galaxy Z Flip3 provides a Bixby function that allows users to utilize the Cover Screen, and Galaxy Watch4 series includes hands-free functionality.

Bixby has advanced in many ways as an AI platform, and, in terms of the engine, it supports Automatic Speech Recognition (ASR) on-device. Because it processes users’ voices immediately from within the device itself, it is fast and secure in terms of privacy protection. Moreover, the development of Bixby Home Platform further expands the range of supported devices. What used to require multiple voice commands is now possible with only a single command that the device understands and performs accordingly.

Q: What are the areas of possible collaboration with developers around the world for the Bixby Platform?

There are several ways third-party developers can participate in Bixby. They can develop various voice services using Samsung’s AI engine in our smartphones, watches and Family Hub. This service, called a capsule, is provided by the Bixby Developer Center. We have also updated the developer tools to make debugging and natural language learning easier, and to allow convenient use of various UI components, all of which will be presented at SDC21.

The update being announced will make it possible to use for developers to harness multiple contexts when adding devices. Smarter device control has been made possible by Bixby Home Platform, an intelligent layer which sits between Bixby Voice – which provides natural language understanding and issues commands – and SmartThings’ Internet of Things (IoT) layer – which provides connection and operation between devices. Commands that previously had to be made individually for separate devices can now be more efficiently undertaken using the Bixby Home Platform. We are introducing this tool at SDC21 and currently working on providing it to third-party developers in the future.

Enjoy More of the Content You Love: Samsung Smart TV’s Tizen

In the past, TVs were evaluated only on their resolution and design, but given changing lifestyles and user demands, the kind of content a TV is able to provide has become paramount. Thanks to the Tizen platform, Samsung’s Smart TVs are able to provide users with a wide range of content able to suit any taste. Ju-hyun Choi, a Staff Engineer at the Software Development Group, Visual Display Business at Samsung Electronics, describes below how Tizen has evolved to provide partners with the optimal environment in which they can develop with wide compatibility and stability.

Q: What are the new functions of Tizen Web Platform for Samsung Smart TV?

Through a powerful web standard technology called WebAssembly, high-performance web apps can now be developed on Samsung Smart TVs. For SDC21, we have prepared an upgrade plan that allows for the installation of the very latest web features for previously-released Smart TVs.

Q: The competitive edge held by Tizen-based display products has been strengthened thanks to the cooperation of partners from various fields. What are your future collaboration plans?

We are currently working on web-based cloud games, with the goal of allowing users to play the latest games on their Samsung Smart TV equipped with the Tizen platform – all without the latest consoles or expensive graphic cards. In order to achieve this, we are working closely with our current service partners. The Tizen platform, an open source-based operating system (OS), is open to everyone and we are working to provide the best possible development environment to developers that features a wide range of compatibility and development tools.

A Must for Innovative User Experiences: On-Device Personalized Neural Networks

Recently, on-device AI technology has been garnering attention. In the past, a high-performance cloud server capable of performing complex calculations was required in order to run AI-based apps, but when the neural network used in AI apps runs within the device, many advantages for users emerge. Jihoon Lee, an Engineer at the On-Device Lab at Samsung Research, explains below the role of on-device AI and cooperation with developers.

Q: On-device AI is considered a must for powerful and innovative consumer experiences. What kind of benefits will users enjoy from using it?

The biggest advantage offered by on-device AI is that the privacy of the users is maintained since all processing occurs on-device. Developers, too, will be able to provide services that are that much closer to the users, while also free from privacy concerns. For example, a camera app that harnesses facial recognition will not share sensitive data with a server if the AI is run on-device.

Another advantage is that on-device AI models can be used without an Internet connection or any mobile data consumption. For example, if a certain on-device AI model is required to be on constant low-power standby, the necessity of an Internet connection or data consumption could be a big limitation. With on-device AI, user application experiences are improved since there is no need to send data to a server and wait for results – resulting in reduced response times.

Thirdly, one is able to create custom AI directly on their device. Previously, the concept of personalizing the already-deployed AI firsthand was unexplored, as there was no framework to help train a deep-learning model on-device. Moving forward, we expect that NNTrainer, which we are presenting at SDC21, will play a big role in better recognizing a user’s face, their accent and even their pets to automatically create photo albums based on their furry friends – and more.

Q: In the future, what kind of synergies can be expected in this field through cooperation with the developers present at SDC?

Ultimately, we hope to bring users closer to AI through on-device training and personalization. We look forward to developers who attend SDC21’s sessions recognizing and utilizing NNTrainer in order to provide AI that is closer to users. Following such collaboration, NNTrainer, currently in its initial stages will become an even more mature framework. Since it is being developed as an open source, anyone can search for it and contribute directly.

Developing Watch Apps for Wear OS Powered by Samsung

Won-Hee Seo, a Staff Engineer at the Strategic Partner R&D Group, Mobile Communications Business at Samsung Electronics, below introduces the new developer tools and benefits of creating watch apps for the Wear OS Powered by Samsung platform, running on the Galaxy Watch4 series.

Q: It’s now easier to develop a great watch app for the Wear OS ecosystem. What are some new helpful experiences that will be available to users?

With the Galaxy Watch4 series, the most loved and used watch apps and services have been optimized for the new Wear OS Powered by Samsung platform. In addition to Samsung and Google services and apps such as Google Maps and YouTube Music, users will also have access to their favorite watch apps available for download in the Google Play Store. In particular, health-related apps, including Sleep Cycle, are starting to launch and are growing in popularity.

Many developer partners have started to work with the new tools available for the Wear OS platform, which has led to the release of many useful apps.

Moreover, we have launched Watch Face Studio, a new developer tool that makes it easy to design watch faces. Thanks to this tool, designers and developers will be able to create more watch faces, which provides users with a greater variety of watch faces to choose from, helping them to further personalize their watch with their own tastes.

Q: Throughout this session, you discussed the development of new watch apps with other developers. What was at the center of the discussion?

A total of four watch-related sessions were arranged for SDC21: one Highlight Session and three Tech Talk Sessions. They provide all the necessary information for developers, such as the developer guide, in order to help them more easily create diverse watch apps and services. In order to help stimulate the watch face market, a session was also added to SDC21 to provide a detailed introduction to the Watch Face Studio tool that is set to allow designers and developers to easily design and create faces.

Bixby’s introduction in 2017 laid the groundwork for the creation of a connected, open ecosystem of devices and services that allows you to get more done, more comfortably. Now, the platform’s latest update has introduced even more ways for users to simplify their day, including the all-new ability to customize your Galaxy device’s Bixby key in a way that works for you.

The Bixby key allows you to access Bixby anytime, anywhere, with a simple press of a button. When the Bixby was first introduced, a single press would pull up Bixby Home, while a press and hold would allow you to speak to Bixby directly. The new Bixby takes the Bixby key’s convenience to the next level by allowing you to use it to instantly activate an app or one of your Quick commands of your choosing.

To facilitate this handy function, Samsung incorporated the ability to launch Bixby main page with either a single or double press, and customize the remaining option to activate a favorite feature. As Samsung’s Executive Vice President and Head of Software and Artificial Intelligence, Eui-Suk Chung, explained, adding this capability allows users to access frequently used Quick commands or apps faster and more flexibly.

“Our top priority is to offer our consumers the best possible experiences,” said Chung. “So we’re constantly listening to and learning from them. Because they’ve expressed that they’d like more tools to get things done quickly and easily, we decided to offer them the freedom to customize the Bixby key.”

Another useful function of the Bixby key lies in Bixby’s enhanced Dictation feature. Sometimes, it can be very inconvenient to have to manually type out a long text, email or memo on a small keyboard. Bixby’s Dictation function simplifies these situations by allowing you to use the Bixby key to quickly and easily dictate your message directly into your phone. With the keyboard open, simply press and hold the Bixby key to begin your message, and release it once you’ve said what you want to say. It’s as simple as that.

Since first introducing it in 2017, Samsung has consistently improved Bixby based on users’ feedback. One of the ways that the company enhanced the Bixby key specifically was by minimizing instances of accidental Bixby activations. When the Bixby key is pressed while your phone is in your pocket, or when your screen is locked, because it’s unlikely that you intended to do so, the Bixby key has been calibrated not to launch Bixby in these situations.

The next part of the Bixby key’s enhancement process involved enabling the key to launch Bixby with a double press. When using your phone to watch a video or take a picture, for instance, accidentally pressing the Bixby key could interrupt your enjoyment. Adding this capability minimizes this inconvenience, as users are not likely to press the Bixby key twice by mistake.

As Chung explained, going forward, Samsung will continue to enrich the Bixby key with new capabilities that add convenience to users’ lives. “Our guiding rule at Samsung is to evolve our products and technologies in ways that are meaningful to consumers,” said Chung. “Just as Bixby’s evolution is ongoing, so too is the Bixby key’s.”

But how exactly does Bixby work, and how can you ensure that you’re getting the most out of it? Here are some common questions and answers to help you get better acquainted with the intuitive interface.

Q. How did Bixby get its name?

We decided to call our new interface Bixby because we found the name to be playful, smart and gender neutral; perfect for someone who is bold and in the know.

Q. How do I activate Bixby Voice?

You can activate Bixby by saying “Hi, Bixby,” or by either pressing or holding its dedicated button (located under the volume controls of your compatible device) and dictating your request.¹

Q. Does Bixby get smarter the more I use it?

Yes, it does. Thanks to Bixby’s iterative deep learning technology, the more you use the interface, the smarter it gets.

For example, whenever you’re not satisfied with its response, Bixby will work with you to learn how to better serve your needs. Each time Bixby responds to a command, it presents you with the option to “Teach me” – to select a more appropriate response from a list of similar commands. The next time you make that request, Bixby will remember exactly what you wanted it to do.

Q. What can I do to make sure that Bixby Voice hears me clearly?

Bixby is optimized to detect voice commands when users are 15 to 20 centimeters (5.9 to 7.9 inches) away from their device. If you’re too close or too far, it can affect the voice recognition technology’s accuracy. For best results, speak clearly and at a volume comparable to normal conversation levels.

Q. Do I need to be in a quiet area in order for Bixby Voice to hear me?

Bixby works best in environments where your voice may be clearly heard. That being said, Bixby’s deep learning technology allows the interface to distinguish your voice commands from ambient noise, even when you’re listening to music, by carefully analyzing sound frequencies.

When in particularly loud environments, you can activate Bixby Voice using your earphones’ play/pause button and microphone.

 Q. What languages and dialects does Bixby understand?

Bixby Voice is currently available in Korea and the U.S., and supports commands in Korean and U.S. English. Users in the U.S. and Korea can switch between Korean and English by adjusting Bixby’s language settings.

Bixby’s understanding of natural language is what allows it to learn dialects. We’ve taught Bixby various dialects’ distinct intonations, vocabulary and grammatical structures, and we’re currently working on providing support for more.

Q. What are Quick Commands?

Bixby’s Quick Commands allow you to issue multi-step, complicated requests with convenient keywords.

With Quick Commands, users can replace a long request, such as “Track my run using the Samsung Health app,” with a short, simple command, such as “Run.” These shortcuts allow you to interact more seamlessly with your phone. Quick Commands may be accessed under Bixby Home’s “My Bixby” menu.

Q. Will using Bixby eat up my mobile data, and is it possible to use it overseas?

Bixby only utilizes your mobile data when listening to a command, not before or after. As a result, the length of the command ultimately determines the amount of mobile data used.

If Bixby Voice is activated and you’re using a licensed SIM card, it’s possible to use the interface even when you’re overseas.²
Actual range of use may vary depending on the apps you’re using.

 Q. What is Bixby Vision, and how does it work?

Bixby Vision was designed to provide you with a deeper understanding of what you’re looking at, allowing you to use your camera to easily scan and search for relevant information on the objects and places around you.

Bixby Vision offers information on everything from purchasing options for a particular product to details on your current location and even the bottle of wine you’re enjoying. It is also capable of scanning business cards and performing convenient translations.

After scanning an object, users can use Bixby Vision’s image search³
function to locate it online. Once Bixby has identified the object, it will automatically pull up related images on Pinterest.⁴

Bixby Vision works best in bright environments, and when you hold your camera steady. The screen should not be upside down, and if the object in focus is a product, it should be captured from the front to aid identification.

Q. Who are some of Bixby Vision’s current retail partners?

We’ve forged partnerships with popular online shopping platforms in each region to allow consumers to enjoy a much more convenient shopping experience with Bixby. These partners include Amazon in North America, Japan and several major markets across Europe, as well as Taobao in China and Samsung Pay Shopping in Korea.

Q. What languages can Bixby Vision translate?

Bixby Vision’s advanced OCR (object character recognition) technology allows it to scan and recognize a total of 40 languages. These languages can then be interpreted into 104 different dialects via Google’s powerful translation database.⁴

Q. How is Reminder different from my device’s clock or calendar app?

Reminder lets users do much more than simply log an appointment or create an alarm.

The function utilizes Bixby’s voice support and contextual awareness to allow users to set smart reminders for specified times, places or circumstances.

Reminder is also integrated into other apps. You can use Reminder to save webpage URLs in the Samsung Internet app, as well as messages that you’d like to get back to later in the Messages app. You can also pin reminders to your device’s Always On Display to make them visible at all times, and access your collected reminders via your device’s edge panel.

Q. What is Bixby Home, and how does it cater to users’ individual needs?

Bixby Home serves as a gateway to Bixby’s various functions and presents your most contextually relevant apps and services on a single screen, so the content you care about is just a swipe away.

Bixby learns your routine and serves up the information you need when you need it. It looks for patterns in your app usage to predict what you might be looking for, and when you’re most likely to look for it. The more you use your smartphone, the more relevant Bixby’s content suggestions will become.

Bixby Home will automatically populate based on your actions, but you can also customize the experience to best suit your needs. You also have the options to pin your favorite cards to the top of your Home page for easy reference, and hide any cards that you no longer wish to see.

Q. Is my personal data safe with Bixby?

Bixby stores users’ personal data in accordance with stringent security procedures.

Users may choose to delete data that Bixby saves from Bixby’s settings at any time. Bixby only collects data necessary to improve or personalize your experience, and ensures that your personal information is never transmitted.

¹Voice wake-up must be activated in order to use this feature

²Language support is currently limited to Korean and U.S. English, and the user’s device must be Bixby-compatible. App and service compatibility may vary.

³Image searching is currently unavailable in Cuba, Iran, Syria and Sudan

⁴In China, image searches and translation function are both conducted with Baidu