In this relay series, Samsung Newsroom is introducing tech experts from Samsung’s R&D centers around the globe to hear more about the work they do and the ways in which it is directly improving the lives of consumers.

The sixth and final expert in the series is Brian Harms, a Research Engineer at Samsung Research America (SRA). Following 8 years of exploration into advanced robotics research at SRA, Harms and his team now employ an innovative array of methods in order to work towards changing the way robots are made and perceived. Read on to learn more about the fascinating research Harms and his team are undertaking at SRA.

Q. Can you please briefly introduce the kind of work you undertake at Samsung Research America?

In addition to developing innovative technologies, SRA is conducting research into various fields including AI, 5G/6G communication, Digital Health, AR and Robotics for Samsung’s future innovation.

When I joined SRA, I was drawn to one in particular of my team’s key areas of focus, which was imagining how robots will affect the future of our homes and everyday lives. A lot of my work at SRA focuses on prototyping experiences as rapidly as possible so that we can make decisions about how certain devices or products should or shouldn’t work.

Our projects usually start very organically, and individuals are encouraged to pursue their ideas and then bring them to the team for feedback and creative input. Thanks to our strong relationships with different divisions within Samsung, our team is empowered to think about a really wide variety of ways we can improve people’s lives, and that freedom and support is a really cool aspect of what we do in the Think Tank Team at SRA.

Q. Following the recent accolades you have received for your work in advanced robotics, what are you and your team working on at the moment?

At any one time we may have approximately 10 to 20 projects that are happening simultaneously, but that operate on different time scales and with different resources. In past years our team’s goal was to have the majority of those projects aim to be ‘productizable’ within 3 to 5 years if successful, but in more recent years we have shifted our goal towards 1 to 3 years, as we are striving to make a strong impact on the user-facing market as quickly as possible.

In order to achieve this, we are working on several projects within the umbrella of practical robotics whose scopes are mindfully constrained so that we can work with different teams to transform these prototypes into products. Our goal is to find a balance where we provide a great deal of user value while still constraining the problem space within realistic bounds. We also pride ourselves in being optimistic about finding room for innovation, even in products that have largely remained the same over many years.

Our team is also currently working on many projects that are outside the realm of robotics, including new apps, phone features, connectivity devices and improved appliances with the goal of empowering users and keeping them connected.

Q. Practical robotics is a field that provides innovative and convenient user experiences and is primed to change the way we think about robots. Can you elaborate on this further?

I think that it is important for people to rethink what they consider to be “robots” because the way they are defined tends to vary greatly. Many common definitions might clash with each other or reject some actual robotic devices from the category. Personally, I lean towards an extremely inclusive definition along the lines of: if a machine actuates automatically in response to stimuli, you might as well call it a robot.

The reason I think it is important for people to take a moment to consider what robots really are is that so-called “practical robots” are all around us and affect us every day of our lives in impactful ways. Consider a mattress with sensors that measure sleep quality or temperature, adjust the mattress’ position with actuators and cool the user by pumping fluid through a network of tubes. I think by almost any definition of robot, this is a robot – but perhaps the owner of such a mattress might not actively consider it one.

From automatic doors at grocery stores to cars that measure their distance to other cars and adjust speed accordingly and even to coffee makers that brew a fresh pot of coffee for you in the morning through sensor detection – these are all robots, and if you were to accept this idea of what a robot is, you’ll start seeing them much more frequently in your day-to-day life.

Q. What do you see as the main user benefits brought about by the implementation of novel robot capabilities into consumer-facing technological devices?

The main user benefit brought by the inclusion of robotic technologies in a device will of course vary by device and the problem it solves for the user, but if I had to generalize, I think that the benefits boil down to making an activity or experience faster, easier, safer or more rewarding. Automation is a powerful mechanism in affecting these four criteria, whether it is in an industrial manufacturing plant or someone’s living room.

Q. Your team is made up of a unique range of researchers from a diverse range of backgrounds. Can you give an example of a time when this ability to ideate in an interdisciplinary manner resulted in the development of an innovative new robotics approach or technology?

Occasionally we hold brainstorming sessions where 1 or 2 people have an idea they want to turn into a project. Those people come up with a series of questions or prompts for the participants, and then every person in the room takes a stack of sticky notes and fills as many as they can with ideas and sketches for the new project and puts them all up on the wall. The cool thing about this is that when the prompts are about potential industrial design ideas, for example, we have not only industrial designers, but also programmers, scientists, electrical engineers and more, all responding to the same prompts in different ways.

Through this kind of multidisciplinary collaboration, designers on our team benefit from developing an improved understanding of what is technically possible, and engineers get a better understanding of what constraints good design might add to the project. What this results in is a team made up of designers who speak the language of engineering, and engineers that can speak the language of design. This kind of collaboration was critical for a project like Samsung Bot Chef, where both the aesthetic and engineering elements were highly dependent on one another.

Q. What would you designate as the latest trends in robotics technology right now? How are you incorporating them into your research at SRA?

Automation and robotics are evergreen fields that are growing exponentially at the moment. The main way we approach projects is to first identify a need or possible method of improving some aspect of daily life, and then consider the mechanisms for executing the idea. Fortunately, automation and robotics are effective tools that lend themselves well to addressing and solving some of these problems.

Our future product concept Samsung Bot Chef was one result of us monitoring the latest trends. Our then-team head noticed that there was a huge gap between the kinds of low-cost, low-performance robot arms you might see on crowdfunding platforms and the high-cost, high-performance of industrial robot arms, and had a strong intuition that there was opportunity in the consumer market for a robot offering in between the two. The goal there was to minimize end user cost while maximizing performance and capability, which took us down a long road towards designing our own servo mechanisms from scratch. What we created is one of the best-looking robot arms that I’ve seen on or off the market, that is tailor made for interacting with the same everyday objects that we use at home.

Q. When you envisage a future powered by innovative robotics technologies, what does it look like?

When I picture the future, I try to imagine “what might a typical day look like for me.” I would hope that, in the future, robotics and automation provide opportunities for me to preserve more time for myself to do the activities that I love. Between maintaining relationships, work, hobbies, errands, finding time to rest and unexpected events in life, I constantly feel that I lack the time or energy to engage with each of these activities in balance. I believe that automation might be one mechanism that will help me preserve more of my time so that I can spend it in ways that I choose in order to feel more fulfilled.

Q. What has been your most important achievement at SRA so far, or the one that you are most proud of?

I was really proud of our team achieving our Samsung Bot Chef demonstration in Berlin, Germany at IFA 2019. It truly took a monumental amount of effort for us to design, manufacture and assemble completely new versions of Samsung Bot Chef from the ground up, by hand. We also had to plan a complex demo, program all of the interactions, and test everything repeatedly, not to mention transport our demo robots to Germany, work around the construction of the demo kitchen and collaborate with the host chefs. It was a really challenging but rewarding experience that not only brought our team closer together, but also reminded us that when we are united in pursuit of a single goal we can achieve amazing things.

In this series, Samsung Newsroom has introduced six tech leaders from Samsung R&D Institutes around the world who are actively involved in advanced technology development. Through the consolidation of the research and development capabilities of experts in Samsung’s R&D institutes, just a few of whom have been showcased in this series, Samsung is able to bring next-level technologies and experiences to users in their devices. Samsung Research currently sees collaboration between the experts in its 14 R&D institutes in 12 different countries around the world.

In the future, collaboration will be a key factor towards advancing research into advanced technology. Samsung will continue to work towards a better future powered by innovation, inspired by daily routines and designed for users.

Intelligent Recommendations for Informed Cooking

For this year’s IFA Club des Chefs cooking show, Chef Michel Roux Jr. was in charge of preparing the appetizer and dessert while Chef Michel Troisgros took responsibility for the entrée.

Chef Roux Jr. used the Quick View Inside feature of the Samsung Family Hub refrigerator to quickly and easily check what ingredients were available inside the refrigerator. Quick View Inside is an upgraded feature of the latest version of the Family Hub that lets users use the refrigerator’s View Inside feature no matter what page or app they have been visiting. Chef Roux Jr. demonstrated to the audience that Quick View Inside was accessible simply by dragging down the upper scroll bar of the refrigerator’s screen with three fingers, an intuitive upgraded user interface feature.

Chef Roux Jr. (right) demonstrates the time-saving qualities of Family Hub, which allowed him to easily and quickly check what ingredients were inside the refrigerator ahead of preparing his dishes.

Minimized Cooking Time, Maximum Efficiency

Moving on to the preparation of his two dishes, Chef Roux Jr. took advantage of the flexibility offered by Samsung’s Dual Cook Flex Oven as he was able to simultaneously bake choux buns for the dessert whilst also cooking the appetizer, a pissaladière. Given that choux buns require a hotter cooking temperature than a pissaladière, Chef Roux Jr. was able to cook the two dishes separately in the upper and lower compartments of the Dual Cook Flex Oven, meaning they could cook independently yet at the same time at different temperatures. Chef Roux Jr. was also able to check on each dish in its separate compartment without disturbing the other, thanks to the Dual Cook Flex Oven’s versatile Flexible Door.

Chef Roux Jr. explains to the MC and the Club des Chefs audience how the Dual Cook Flex Oven saves users time thanks to its separate upper and lower compartments.

When it came to Chef Troisgros’ turn to prepare his entrée, Chef Troisgros was able to minimize food preparation time thanks to the Dual Flex Zone Plus’s cooktop, which meant that he could cook potatoes two different ways simultaneously – one in a pot and one in a wide steamer. The audience was impressed with how, thanks to its expansive, open dual-zone surface, the Dual Flex Zone Plus offers chefs maximum efficiency and flexibility when cooking with different sizes of cookware at the same time.

AI-Powered Kitchen Assistance with Bot Chef

For the preparation of the sauce of the entrée, the Club des Chef’s latest sous-chef made its appearance — Samsung Bot Chef. The audience was impressed to see how Samsung’s AI-powered kitchen assistant robot — also called a cobot — was able to independently work on a variety of tasks, including seasoning the sauce with salt, stirring the contents of the pan and even taking hold of a utensil and offering up a spoonful of the sauce for the Chefs to taste.

Bot Chef assists Chef Troisgros (center) in preparing a sauce by taking hold of a utensil and stirring, top image, and seasoning the pan’s contents with salt.

The audience was eager to hear from the event MC and Chefs how Bot Chef uses an extensible AI-powered system that users can command, with new skills available via download from Samsung’s robot skills ecosystem. Bot Chef can be instructed either by simple voice command or through app-based controls, and is able to assist with tasks throughout the cooking process, from preparation – cutting or stirring food – to plating and seasoning.

During Bot Chef’s presentation, the Club des Chefs cooking show audience witnessed not only how this innovative artificial intelligence-assisted technology helped with cooking, but also how Samsung’s mobile ecosystem capabilities create a seamless connectivity throughout the cooking process across Samsung’s home appliance devices.

Premium Connectivity for the Modern Kitchen

As the show was drawing to a close, the MC invited Chef Troisgros and Chef Roux Jr. to join her for a selfie to commemorate the event. The photo was then easily transferred from a Samsung smartphone to the Family Hub’s Family Board to be displayed for all.

The audience left the cooking event full of ideas as to how Samsung’s smart kitchen appliances, driven by AI and IoT functionality, can seamlessly integrate into their own kitchens. Thanks to their premium contemporary design and intelligent support capabilities, Samsung smart kitchens are creating brighter futures for busy modern families today.

The exhibition will reveal the company’s vision for the future, with a range of cutting-edge lifestyle concepts on display. At the event, Samsung is unveiling the newest addition to its intelligent robotic Bots lineup, alongside several devices first showcased in January at CES 2019.

Samsung Bot Chef

We can all appreciate an extra pair of hands in the kitchen, sharing the burden of the laborious and repetitive tasks that go into creating delicious meals. First unveiled at KBIS 2019, Samsung Bot Chef is an AI-powered chef’s assistant – a “cobot”, or collaborative robot – that assists them as they whip up gourmet dishes. Bot Chef is designed to be easy to use and highly versatile, with a sleek exterior that hides its advanced mechatronics.

Bot Chef is optimized for kitchen use and is capable of a wide range of kitchen tasks – from chopping and whisking, to pouring and cleaning. Based on SARAM, Samsung’s multi-purpose programmable robotic platform, the lightweight robotic manipulator arm has six degrees of freedom, with the diameter, reach and safety of a human arm. This allows it to perform with a payload just enough to lift common, everyday kitchen items, making it ideal for homes and small businesses. With advanced internal and external sensors and AI-based planning algorithms, the Bot Chef works alongside a person safely, even when they get in each other’s way.

Users can interact with the robotic arm using simple and intuitive voice commands. Meanwhile, the underlying AI and machine-learning skills platform allows users to easily program tasks to perform using voice control, physical manipulation and app-based controls, thereby creating new “skills”. New skills can also be downloaded, customized and shared as part of the online ecosystem, providing endless possibilities in the kitchen. To stir a pot of soup, for example, a user can download the “stirring” skill from the skills ecosystem. Bot Chef can autonomously understand the location of objects, so the user can tell it where to find the spoon, and which pot to stir.

Chef Garden

Many cooks will tell you that there is a certain kind of joy that only fresh, locally sourced ingredients can bring. When it comes to fruit and vegetables, the freshest and most flavorful are home-grown. Chef Garden, debuting at KBIS 2019, aims to provide a farm-to-table experience in the comfort of your own home, with an AI farming platform that effortlessly produces fresh, pesticide-free fruit and vegetables all year round.

Chef Garden integrates seamlessly with the next generation of Samsung Family Hub, using the power and water connections already provided to the refrigerator. To grow their plants, users can choose from a wide variety of seed capsules and place them in the Growing Zone, which applies fogponics technology to create a nutrient fog that delivers water and nutrients to the plants, saving water and keeping the inside of the Growing Zone clean. Chef Garden can control conditions to ensure an optimal environment for growth, making predictions about growth status and controlling harvest time. Thanks to its integration with Family Hub, the platform can also recommend recipes based on the fruit and vegetables that are ready to harvest, as well as the items already in the fridge, helping to minimize waste.

GEMS: Gait Enhancing and Muscle Strengthening

Our bodies are our true homes – and strong muscles are vital to ensure these homes run smoothly. Samsung GEMS was originally developed to help the elderly improve their mobility by applying assistive forces to their lower-limb joints, synchronized with their movements. GEMS has three versions, each assisting a different joint: GEMS-H, GEMS-K and GEMS-A target the hip, knee and ankle, respectively.

In-depth clinical studies for GEMS-H have shown that it can enhance the gait efficiency and stability resulting in more dynamic and balanced walking, as well as a dramatic reduction in energy consumption. With these insights, we decided to expand the scope of its applications from rehabilitation alone to providing proactive musculoskeletal healthcare for everyday users. GEMS-K was designed to reduce the impactive load at the knee joint, the main cause of knee pain when standing up or walking on the stairs. It is expected that it will provide particular relief to osteoarthritis patients as they perform their daily tasks. GEMS-A was devised to support the ankle joint, assisting push-off as the user moves forward and preventing foot-drop, thus helping those who are suffering from hemiplegia caused by stroke aftereffects.

The GEMS series is subject to continuous innovation, with constant improvements in comfort, affordability and personalization, thanks to a synergetic fusion of cutting-edge technologies in robotics, biomechanics and AI.

Samsung Bot Air and Bot Clean

Housework can seem like a never-ending task. What’s more, you might move your air purifier from room to room without knowing if you are targeting the right location or vacuum the same spot repeatedly while missing those hard-to-reach corners.

Making use of sensors throughout the house to gauge air quality, the Samsung Bot Air will move directly to the source of unclean air and begin purifying the area. Users can see the real-time improvement of indoor air quality through its intuitive lighting effects, which go from red for bad air quality to green for fresh air. Taking cleaning to the next level, the Bot Clean is an intelligent and interactive robotic vacuum cleaner that was newly revealed at KBIS 2019. The device can express its current status using its motions and eyes and utilizes LiDAR sensors to navigate a room autonomously, with SLAM (Simultaneous Localization and Mapping) functionality that scans the area to map out its route.