Samsung Electronics Co., Ltd. held its annual Samsung Developer Conference (SDC) today in San Francisco, bringing together developers, creators and designers to explore seamless, connected experiences powered by Samsung.

During the event, Samsung shared more about its commitment to creating simplified, game-changing customer experiences, including the company’s updated vision for SmartThings as it evolves from a connectivity platform to an enabler of smarter lifestyles. From deeper integration with Bixby to seamless connectivity with Matter-compatible devices, SmartThings is creating a richer, more open world that empowers all users to streamline their connections and their daily lives.

“At Samsung, we continuously innovate our devices, platforms and services to be simpler and more convenient. I am proud to share the next generation of our work, like SmartThings, to further our collaboration with partners and developers,” said Jong-Hee (JH) Han, Vice Chairman, CEO and Head of Device eXperience (DX) Division at Samsung Electronics. “As technologies become more complex, we will always search for ways to make life easier, more connected, and more flexible, so our consumers can focus on what matters most.”

Connecting Platforms and Devices with Calm Technology

Samsung’s vision of seamless connectivity is inspired by the philosophy of Calm Technology – where devices instantly work together, so consumers can save time on setup and get right into the experience. To realize this vision, Samsung reimagined SmartThings and its connected services and partnerships, including Samsung’s Hub Everywhere, extending its capabilities to the entire smart home with audio and visual data as well as SmartThings Energy, SmartThings Pet and SmartThings Cooking. Samsung also partnered with Philips Hue to integrate Philips Hue Sync right into Galaxy devices so smart home lighting can match with music.

The seamless experience will extend beyond Samsung’s ecosystem through SmartThings’ integration with Matter and Samsung’s membership in the Home Connectivity Alliance. Google and Samsung have worked together to enable users to find and link their devices across platforms by building multi-admin feature on Matter devices. The collaboration will bring more devices and users into the connected home in the future.

Bixby has been integrated more deeply with SmartThings, a tool for developers to build more intelligent voice user interface experiences. As Samsung’s representative voice assistant platform, Bixby has evolved to become the on-device AI solution able to control individual Galaxy devices, as well as the cross-device experience throughout the Samsung ecosystem. With the new Bixby Home Studio, developers can now build differentiated, customized experiences for the SmartThings platform. In addition, Bixby will be even more widely available to users with support for Latin American Spanish, starting in November.

As the smart home is becoming more advanced, Samsung is introducing a new security paradigm, enabling Samsung devices to protect each other. Samsung Knox Matrix^[1] is a private blockchain based platform that turns eligible Samsung devices into a shield to protect users’ entire device ecosystem – from Galaxy devices to TVs and home appliances. Plus, users can customize personal privacy settings with the new Security and privacy dashboard, which scans for vulnerabilities, recommends security updates, and gives users data management options to keep privacy and security top of mind.

Personalized Services for Better Experiences

In the past year, Samsung TV Plus — Samsung’s free, ad-supported streaming TV and video (FAST) platform available on Samsung Smart TVs^[2] and mobile devices— reached 100% growth in viewership^[3], and projects three billion hours streamed by the end of the year. To build on this momentum, Samsung TV Plus has expanded its offering by adding partnerships with Lionsgate and Vice Media, providing 8K video on demand. Samsung TV Plus has also gone through a redesign, to reflect its extraordinary variety of content with more than 1,600 channels across 24 countries. Samsung TV Plus is the premium choice for a seamless ad experience, where stream stitching makes ad playback simpler than ever.

After 10 years of service, Samsung’s Tizen operating system (OS) continues to offer best-in-class user experiences. The Samsung Gaming Hub brings better, faster and more convenient access to gaming on Samsung Smart TVs, provided by industry-leading partners, such as Xbox, NVIDIA GeForce NOW, Utomik and Amazon Luna. Samsung Gaming Hub bridges expertise in hardware and software to integrate features like AI Upscaling and multitask functions to make gaming on Samsung Smart TVs an immersive, optimized experience.

Tizen OS is also expanding to offer NFTs support with partners like Art Token, laCollection and Nifty Gateway. For B2B customers and developers, Samsung is providing B2B APIs specific to industries and use cases like Syncplay, which allows content to be synchronized and played in multiple signages. Tizen is also introducing SALT, a new content conversion solution to display the highest-quality HDR10+ content on supported TVs.

Your Galaxy, Your Way with One UI 5

By introducing the new One UI 5, Samsung has empowered users to easily customize their device’s look and feel, while enhancing productivity and providing amazing experiences across devices and platforms. One UI 5 brings more personalized experiences with custom-built Modes and Routines, and a Dynamic Lock screen that displays multiple visuals on your phone, your Galaxy Watch and other One UI 5 devices. With the new Bixby Text Call feature, Bixby Voice will answer calls on your behalf and share typed messages with the caller, speaking aloud as if you had answered. One UI 5 also introduces new daily health solutions to help track health and wellness in one place. This includes the Samsung Privileged Health SDK, which enables developers to build apps that leverage the BioActive Sensor on the Galaxy Watch^[4].

Future-Forward Investment and Research

Samsung Research is looking beyond today’s horizon to innovate for a better tomorrow, full of relentless innovation and open collaboration. Samsung Research is making its robot arm manipulation code available on GitHub, enabling academics, researchers and developers to examine new ideas in robotic manipulation.

The team at Samsung Research was also inspired by the new wellness features Samsung has introduced, particularly on the Galaxy Watch5. To find new use-cases leveraging these features, Samsung Research is working with universities and healthcare institutions that will explore areas like heart health, stress, blood pressure, lung health and neuroscience. As a result, Samsung is offering a full stack SDK – the Samsung Health Stack – which will jumpstart research into important health fields and spark new development.

Samsung invites developers, creators, partners and more to join in on its commitment to open collaboration and seamless experiences.

For more information about the Samsung Developer Conference (SDC) 2022, please visit developer.samsung.com.

—

^[1] Starting as early as late 2023, Samsung Knox Matrix will be initially available on selected Samsung devices. Support for other manufacturers’ products will follow later.

^[2] Samsung TV Plus is available for selected Samsung Smart TVs only.

^[3] Comparison between viewership from September 2020 to August 2021 and viewership from September 2021 to August 2022.

^[4] Available on Galaxy Watch4 series and Galaxy Watch5 series.

Samsung Electronics unveiled a series of new tools ahead of the Samsung Developer Conference 2022 (SDC2022) that will aid developers and communities in shaping health, wellness and safety habits for consumers everywhere. These tools include the Samsung Privileged Health SDK program for select partners, the Fall Detection API to support user safety, Samsung’s new end-to-end research solution for educational, clinical and healthcare programmers, as well as ongoing opportunities for partners with Health Connect.

“Samsung’s health foundation is rooted in our advanced hardware and sensor technology and is bolstered by our open ecosystem and collaborative approach,” said TaeJong Jay Yang, Executive Vice President and the Head of Health R&D Team at Mobile eXperience Business, Samsung Electronics. “That is why I am excited to announce expanded developer tools, APIs and partner offerings that allow third-party experts, research centres and universities to advance wearable tracking and insight capabilities for a wider health, wellness and safety.”

Samsung Privileged Health SDK: Advancing Preventative Safety in Driving

Samsung is collaborating with select industry leaders to develop personalized health services. With safety as one of Samsung’s focus areas, these collaborations will deliver all-new preventative tools to make roads safer by identifying early signs of driver fatigue and stress.

Samsung’s work with Tobii, a global leader in eye-tracking and attention computing that develops software to monitor drowsiness, is one example. If users choose to, Samsung’s Privileged Health SDK enables Tobii to sense real-time heart rate captured by Galaxy Watch sensors and process them to determine the user’s level of drowsiness. Also, Harman, a leader in connected car technology, audio innovations, and IoT solutions, has recently introduced Ready Care, a complete automotive in-cabin sensing and tailored solution for driver safety and comfort. In addition to distraction, fatigue and vital sign sensing, users can allow Ready Care to measure cognitive load and stress level and can provide suggestions for alternative routes to lower a driver’s stress.

Health Services: New Fall Detection API for Enhanced Safety Capabilities

Health Services provides APIs that allow developers and third parties to harness the powerful sensors and algorithms on the Galaxy Watch for more accurate and advanced health offerings. Samsung is introducing a new API to the ecosystem that offers adjustable sensitivity for Fall Detection, strengthening preventative safety and setting the stage for even more potential offerings.

Developers can now create services using Galaxy Watch’s Fall Detection algorithms that sense a user’s stumble or fall by combining different sensors including an accelerometer and gyroscope. This API enables development of apps for users who may encounter unexpected accidents. Furthermore, Galaxy users are even able to adjust sensitivity levels – whether standing still, moving, or exercising – via the Galaxy Wearable app. Like sleep tracking that senses restless movements or falls from bed to paint a more complete picture of a user’s sleep, this API opens all new opportunities for developers and users alike.

End-to-End Research Solution: Accelerating Medical Service and Research

Samsung is also introducing an open-source project that provides end-to-end solutions combining SDK, backend system and portal for a variety of health applications. This all-in-one healthcare toolset empowers research and clinical experiences — starting at the developer level — by delivering advanced insights through Galaxy Watches and wearables to support medical research at institutions, hospitals, wellness centres and beyond.

Researchers can use this open-source project to build modules that enable participants to join a study in a frictionless and informed way, streamlining the onboarding process. Flexible survey templates are also in place to satisfy the ever-changing needs of organizations. From there, relevant data and insights are gathered more easily from wearable devices, while participants are guided with actionable insights throughout the process. Encrypted participant data is then transmitted to the backend system where health and medical communities monitor and analyze the collected insights to inform patient treatment plans, upcoming research and much more. More details of the new project will be announced at SDC22 keynote by Sebastian Seung, President and the Head of Samsung Research.

Health Connect: Greater Insights and Data Control

Health Connect, announced in collaboration with Google during the spring of 2022, offers developers a single set of APIs enabling the seamless creation of new fitness and health experiences. Currently available in beta, Health Connect offers users centralized privacy controls that make it easy to give permission to the health and fitness apps they want to share their on-device data with. Now, Samsung Health and Fitbit, along with leading health and fitness apps including Leap Fitness, MyFitnessPal and Withings have adopted Health Connect to enable a more impactful and holistic wellness experience. And with a user’s permission, app developers can leverage certain data shared through Health Connect for use in their own applications in order to provide users with a more complete picture of their health.

Samsung is committed to giving its community – including users, third parties, developers and beyond – enhanced tools and resources to build and benefit from more connected and diverse health services. As part of Samsung’s overarching wellness initiatives, the company will continue to deliver an expansive ecosystem – which the Samsung community will hear more about during the SDC22 that empowers users with more insights and information on health and wellbeing.

The Canadian Samsung community can tune into the keynote livestream on October 12, 2022 at 1:00PM EST by visiting Samsung Canada Newsroom or at www.samsungdeveloperconference.com.

Part one of this two-part series focused on Samsung’s latest home appliances and displays. Here, Junghoe Kim of Samsung Research[1]’s Robot Centre and Hoseong Kang from Samsung’s Mobile eXperience Business offer exclusive insights on the company’s latest robots and the new Galaxy S21 FE.

[Robotics] Reimagining Robots as Life Companions

▲ Junghoe Kim of Samsung Research’s Robot Center stands alongside the Samsung Bot i interactive robot.

Q: What is the theme for Samsung Research’s showcase at CES 2022?

The overall concept for Samsung Research at CES 2022 is “Digital, Meet Physical”. We’ve gone to great lengths to show how the personalized experiences that millennials and zoomers value can be seamlessly connected to the digital world.

To that end, the ‘house’ that we set up at CES was designed to resemble a digital world like a metaverse. Our AI ​​Avatar was created to identify users’ locations in the real world using UWB (Ultra Wide Band) location recognition technology, and interact with them through the nearest smart home product.

At this year’s CES, two types of companion robots will be introduced: the Samsung Bot i, an interactive robot, and the Samsung Bot Handy, a manipulation robot that can assist with household chores. They can help users by arranging a video conference for an urgent meeting, or by setting the table for a meal. Our CES 2022 booth demonstrates how AI and robots can come together to introduce innovative user experiences to the home of the future.

Q: What are some of the Samsung Bot i and Samsung Bot Handy’s most innovative features and technologies?

The Samsung Bot i is an interactive robot that utilizes balance control technology. It moves like a living being, making dynamic and life-like movements. When it’s next to the user, it moves alongside them and offers various forms of support. When it’s away from the user, it manages various tasks for them using its telepresence[2] function.

The Samsung Bot Handy is a robot that helps out with various household chores using manipulation[3]  technology. This enables the robot to recognize objects at home and skillfully pick them up. Whereas at last year’s CES, we showcased a technology that could pick up a single object based on learning, this year, we will introduce a scenario called “tableware handling.” This technology enables the robot to pick items up after recognizing multiple objects and analyzing the surrounding environment.

▲ (From left) The Samsung Bot Handy, an AI Avatar and the Samsung Bot i.

Q: How would you summarize Samsung Research’ R&D philosophy when it comes to robots, and where is the company’s research heading?

Samsung Research’s Robot Centre is conducting research in various areas under the vision, “Samsung Bot, Touching Your Daily Life.” We strive to make our robots companions that help users in their daily lives by closely interacting with them and making their lives happier and more convenient. To achieve this, we’re taking core robot technologies like mobility and manipulation a step further by creating platforms based on our extensive expertise in AI, and we’re developing various robots based on research in those areas.

—

[1] Samsung Electronics’ advanced R&D hub, which leads the development of future technologies for its Device eXperience division

[2] Refers to a virtual video conferencing system that makes participants feel as if they are in the same room.

[3] Refers to the skillful manipulation of objects with hands. This includes picking objects up.

Sven Dickinson, Head of Samsung’s Toronto AI Center

Can you imagine a world where the personal AI assistant on your smartphone is able to understand as much about the world as you do? What about a scenario where communicating with that AI assistant is as natural and easy as interacting with another human? Developing those kinds of capabilities is exactly what the team at Samsung’s AI Center in Toronto are putting their minds to.

Samsung Newsroom sat down with Sven Dickinson, Head of Samsung’s Toronto AI Center to learn more about these exciting topics and what they could mean for the future.

The Vision for Vision

The second Samsung AI center established in North America, Samsung’s Toronto AI Center is led by Dr. Sven Dickinson, an expert in computer vision and former chair of the Department of Computer Science at the University of Toronto.

At the epicenter of AI research and development in Canada, Samsung’s Toronto AI Center is mainly focused on developing the visual understanding capabilities that allow a Samsung device to understand the world in which it’s situated. In addition, the team is working on multi-modal interactions, which are user-machine interactions that encapsulate vision, language and knowledge.

“Allowing Samsung devices to ‘see the world’ through computer vision enables them to ‘visually ground’ their dialog with the user, providing an integrated, multimodal experience that’s far more natural than one that’s solely vision or dialog-based” says Dickinson, whose expertise includes exploring problems surrounding shape perception and object recognition.

Touching on the benefits of multimodal technology, Dickinson claims that, “I should not have to read manuals to figure out which buttons to push on my device and in which order. Rather, I should be able to show my device what I want, and tell it what I want, in natural language that is understandable, and situated in the world that I live in.”

Extrapolating on the interplay between computer vision and multimodal inputs, he goes on to say that, “To achieve this breadth of comprehension, the device has to have a model of my understanding of the world, the capacity to communicate robustly and naturally with me, and the ability to see and understand the same world that I see.”

Remarking on applications for this technology, Dickinson identifies the most compelling as being “a personal assistant that you not only speak to, but that sees the world the same way that you do.” Speaking to the importance of multi-modal device interactions, Dickinson points out how much cancelling out one of the modes of communication (audio, speech, sight etc.) would hamper communication between two people, and says that also applies to personal devices.

A Truly Enhanced User Experience is Key

At the 2019 Consumer Electronics Show (CES), Samsung unveiled its vision for Connected Living, which involves connecting the 500 million devices the company sells every year, and making them intelligent. Dickinson highlights that Samsung’s broad product portfolio will be instrumental in fulfilling this vision, saying that, “What differentiates Samsung is that it makes a multitude of devices in the home, including digital appliances, TVs, and mobile phones. Samsung has a unique opportunity to leverage these devices to yield a multi-device experience which follows the user from one device to another, and one room to another. This will help realize the full potential of each device to effectively communicate, to help the user execute device-specific tasks, and to learn the user’s habits and preferences so that subsequent communication is not intrusive but instead ‘always helpful.’”

Speaking about what his center will need to do to truly realize computer vision and multimodal interaction, Dickinson comments that, “Vision is not about understanding images; vision is about understanding the world. Truly capable AI systems must possess an understanding of our world, of its physics and causality, of its geometry and dynamics. They must also be able to model and understand human behavior.” He extrapolates on this by pointing out that, “If our devices can see the 3D world that we live in the same way as we do, i.e., understand the 3D shapes, positions and identities of objects in our shared environment, then our devices can visually experience the world as we do. Such a shared visual context will be crucial in developing fully realized personal assistants.”

Dickinson says that Samsung is leading the charge when it comes to truly intelligent visual understanding, and identifies ‘visual grounding’ as an essential pre-requisite for well-rounded visual understanding capabilities. “Samsung is leading the way when it comes to developing human-device interaction that closely mimics human-human interaction,” Dickinson says, “We aim to provide visual grounding and knowledge representation scaffolding for dialog-based interaction services. Without these components in place, users become disappointed with services, and quickly tune out.”

 Human-device Interactions Based on Open Information Sharing 

Dickinson goes on to explain that AI also needs to be able to explain itself to the user. He remarks that, after failing to carry out a task or provide an appropriate response, “A device should be able to reflect to the user precisely how and why it came up with that response (or lack thereof). Ideally, it should be able to follow up with the user by asking a question or asking the user to adjust its camera or other input modes so that it can gather more information and formulate an appropriate response.” Dickinson relates that this kind of openness and information sharing will be key to the further sophistication of human-device interactions, noting that “What we call the domain of ‘active dialog and active vision’ is where the system can construct a mental model of what the user understands, and can, in turn, open up its own mental model so that the user can understand the thought processes of the device.”

The Benefits of Being Based in Toronto

Asked about how being based in Toronto affects the AI center, Dickinson remarks that the center enjoys a lot of benefits due to its close proximity to various world-class AI-related institutions, including the University of Toronto, York University and Ryerson University. “Being in Toronto offers us a tremendous regional advantage,” Dickinson comments, “We are across the street from the University of Toronto, home to the Department of Computer Science (DCS), which is one of the top-10 international computer science departments. Over half the members of our AI Center are either active faculty, graduates or current students at DCS.”

On the topic of collaboration between Samsung’s global AI centers, Dickinson relates that, “The seven global Samsung AI centers are working to create industry-leading solutions in their respective areas of focus, while coordinating to achieve the common goal that is realizing Samsung’s ultimate AI vision.” Dickinson touches on the topic of the Toronto AI center collaborating with other AI centers further, saying that, “We are starting to explore possible research collaborations with other global AI centers, and hope to converge on some use cases of value to Samsung and its products and services.”

Located in Toronto’s downtown core at MaRS Discovery District, the new Samsung AI Centre will contribute to building the connected future by accelerating the adoption of intelligence on multiple devices ranging from household appliances to cars. The Toronto AI Centre is a part of a network of research Centres dedicated to research and development in the field of AI. The Centre is the second Samsung AI Centre to be established in North America, with the other in Mountain View, California. The North America AI Centres are led by senior vice president, Dr. Larry Heck, a renowned expert in machine learning for spoken and text language processing, who also co-leads the expansion of Samsung’s AI Centres around the globe.

“Toronto and the GTA are epi-centres of machine learning and one of the world’s foremost hubs for AI research and development. Home to not only world-class talent, but also some of the most innovative start-ups in the artificial intelligence field,” said Dr. Larry Heck, Co-Head of Global Artificial Intelligence Research. “We are looking forward to contributing to this vibrant AI community and to help push the field forward as we move towards our goal of making all Samsung connected devices intelligent by 2020.”

With a rich history in innovation and discovery, Toronto serves as an ideal place for research and development for speech recognition, where machine-learning technology was applied many years before it was widely applied to other fields. The vision is that the Samsung AI Centre will now serve an important role in the advancement of AI with a focus on language understanding and computer vision technologies that will ultimately reduce the friction between the user and the device/service, whether it be mobile phones, TVs, appliances, or cars.

The Toronto centre will be led by Dr. Sven Dickinson, newly appointed as the head of the Toronto lab, professor on leave and past chair of the Department of Computer Science at the University of Toronto. Dr. Dickinson is an expert in computer vision technologies, especially in the field of object recognition. He will play an integral part in Samsung’s research of core AI technologies that entail language, vision and other multi-modal interactions.

Hyun-suk Kim, President and Head of Samsung Research (front row, third from left), and Dr. Sven Dickinson, Head of Samsung’s new, Toronto-based AI Centre (front row, far right), pose for a photo with guests at the facility’s opening ceremony on May 24.

Samsung will officially celebrate today’s announcement of the Toronto AI Centre with an opening ceremony and presentations at MaRS Discovery District from key Samsung business leaders and special guests including Samsung’s CEO of Consumer Electronics, Hyun-suk Kim, the Honourable.

François-Philippe Champagne, Minister of International Trade, Dr. Larry Heck, Senior Vice President of Samsung Research America, and key stakeholders and partners from major universities in Canada.

“Canadian ingenuity and innovation are getting noticed and together with our investments in skills and superclusters focused on next generation products like artificial intelligence, the world’s investors are choosing Canada. Congratulations to Toronto and to Samsung for recognizing Canada is the place to be!” said Honourable François-Philippe Champagne, Minister of International Trade.

Key contributing factors for the location selection include the availability of key AI talent, including leading AI researchers at the University of Toronto and the University of Waterloo who have had longstanding relationships with Samsung. Establishing an AI Centre in Southern Ontario also enables Samsung to better collaborate with regional start-ups and expand the current ecosystem. As one of the world’s largest urban innovation hubs, MaRS Discovery district supports promising innovators and ventures tackling key challenges in the sectors of cleantech, finance & commerce, and work & learning. In addition, and importantly, the vast MaRS community fosters cross-disciplinary collaboration which drives breakthrough discoveries and a new solution for global audiences.

“We are proud to welcome Samsung to MaRS,” said Yung Wu, CEO, MaRS Discovery District. “We both recognize the important role Toronto – and Canada – play as a destination for innovation talent, research, and start-ups. We look forward to collaborating and connecting with the Samsung team on a variety of exciting opportunities as this project comes to life.”

Today’s announcement compliments earlier 2018 news of plans to launch additional AI centres in North America. Dr. Darin Graham will spearhead the opening of new labs in Canada as the head of Samsung’s Canadian AI Operations. Until recently, Dr. Graham helped lead the creation and formation of the Vector Institute – the renowned Canadian AI research institute, as a member of the founding team. The opening of AI centres in Canada will allow Samsung to expand its outpost for industry collaboration and talent recruitment in the major AI hubs in North America.

“Investments like this are proof that Canada is truly a world-leader in artificial intelligence. Our talent and innovation ecosystem make us an attractive place for companies to grow and create well-paying jobs for Canadians. Our government is proud to partner with companies like Samsung, as we continue advancing our machine learning knowledge and further establish our place on the world stage as an AI leader,” said the Honourable Navdeep Bains, Minister of Innovation, Science and Economic Development.

To date, Samsung has had great success in leveraging Canada’s unique R&D talents for global impact. The Company’s Vancouver-based R&D centre has contributed to a number of in-market innovations and more than doubled its workforce, since opening with over 100 employees. With the addition of the AI centre in Toronto, the company plans to increase the R&D in Canada from current 100 to 200 in the near future. Additional developments and talent in Canada have been recognized through Samsung Electronics Canada subsidiaries, AdGear Technologies Inc. in Montreal and SigMast Communications Inc. in Halifax, Nova Scotia.