Samsung Electronics has long recognized the significance of these advanced technologies, and has been actively pursuing innovation in these fields. Expert researchers are working hard at Samsung Research’s¹ 14 R&D centers and 7 global AI centers all over the world in order to prepare for the future, innovate for users and create the next generation of cutting-edge technologies and services that the Samsung Electronics’ legacy is built on.

In this relay series, Samsung Newsroom is introducing tech experts from Samsung’s R&D centers around the globe to hear more about the work they do and the ways in which it is directly improving the lives of consumers.

The first expert in the series to be introduced is Sergii Lytvynenko, Head of the Visual Intelligence Team at Samsung R&D Institute Ukraine (SRK). Sergii has been working for SRK for more than decade since he first joined as a SW Engineer. Read on to hear more about the groundbreaking work Lytvynenko and his team undertake at SRK.

Q: Can you please briefly introduce the Samsung R&D Institute Ukraine and the kind of work that goes on there?

Our R&D center is located in Kyiv, situated in the heart of Ukraine. Since its inception in 2009, SRK has been focused on and has deep expertise in the AI, Augmented Reality (AR) / Virtual Reality (VR) and Security domains. SRK is composed of prominent industry professionals and is currently working on the study of intelligence security, computer vision, context-aware intelligent services, and more. Also, as part of industrial-educational cooperation initiatives, SRK actively co-operates with local universities and schools.

Q: What are you and the Visual Intelligence Team working on at the moment?

Our team is currently conducting fundamental research into the AI, Computer Vision and Computer Graphics domains. The main mission of our team is to transform research advancements into holistic user experiences, thereby enhancing the quality of people’s lives, simplifying their daily routines and delivering positive emotions and immersive experiences.

To do so, we are collaborating closely with various teams in other countries by conducting advanced research in our focal domains and working with different business units by contributing our core technologies to Samsung products.

Q: Your team covers two major technological domains – Computer Vision and Computer Graphics. How do these technologies contribute to innovating new user experiences?

Last year, we undertook extensive work on the Smart Trainer solution, which enables a totally new level of home fitness experiences. Through the USB camera connected to the Samsung Smart TV, the system can track your activities, keep track of the exercises you do and even offer recommendations on your form accuracy, all thanks to AI. We are now very happy that Samsung TV users can enjoy this feature in their homes.

Q: How are you incorporating the key technologies from your focal domains into your current projects, such as AR Glasses?

These days we are performing advanced R&D to tackle major challenges in the computer vision and graphics areas for AR Glasses. On the vision side, we are working on the essential solutions required for AR, including Simultaneous Localization and Mapping (SLAM), Depth Estimation, Environment Understanding and Human Computer Interaction (HCI). On the graphics side, we are conducting research into low-latency rendering for AR and Game Performance optimization.

▲ Visual Intelligence Team at Samsung R&D Institute Ukraine

Q: As well as AR, your team contributes to S Pen technology development. Can you give us a bit of background into the development of this technology?

One of our focal R&D areas and core solutions is handwriting recognition technology for S Pen enabled devices, which is being deployed and spread to Galaxy line up. While working on our handwriting recognition solution, we also developed a rich patent portfolio, thus contributing to Samsung’s core technology development.

Q: In what ways do you think the optimized S Pen technologies your team created for the Galaxy Z Fold3 will complement users’ experience of the device?

Galaxy Z Fold3 is a really unique product. Its large, flexible display expands the boundaries and opens up new possibilities for users to serve as a true productivity companion for daily business and education. Within this context, S Pen and handwriting recognition and low latency become of crucial importance and we are taking the very best of the conventional pen and paper to deliver those same types of experiences to the digital screen.

Q: In what ways do the technologies your team contributed to Galaxy Z Fold3 set to enhance the quality of users’ lives and simplify their routines?

We deployed our AI Based Point Prediction solution to minimize the latency input of S Pen in order to make the writing and drawing experience feel more like that of pen and paper. Furthermore, handwriting recognition technologies make digital writing smarter, easier and more enjoyable. Users can transform their notes to printed documents, recognize tables, diagrams, embed links, undertake math problems and more, simpler than ever before. Experiences like this are what make a real difference in our daily lives.

Q: What do you see as the main technology trends right now?

These days, we recognize Visual Modality as the next big thing: how to transform a note into a smart note, how to make a video into a smart video, and how much useful context information we can extract from these processes. For this technology, AR opens up tons of possibilities, as well as challenges to be resolved. For example, “Digital Eyes” that would fully explore an environment for a user and provide well-organized contextual information could totally change our lives.

Another big trend right now is HCI. Here we think multi-modal interaction, which is a crucial part of HCI, would be essential. Multi-modal interactions are user-machine interactions that encapsulate vision, language and knowledge, and this technology can help a Samsung device understand the world in which it’s situated.

Q: What has been your most memorable achievement at SRK so far?

June 2021 was a really special month for us as we won the CVPR (Conference on Computer Vision and Pattern Recognition) 2021 Chart Question Answering Challenge. CVPR is the world’s biggest conference on computer vision and AI. We are really proud of what we achieved.

Q: Visual intelligence technologies are crucial when it comes to innovating new mobile experiences for users. In what ways do language-related technologies also contribute to these experiences?

Natural Language Processing (NLP) is one of the most challenging research areas. We really wish that every single person around the world were able to use and experience our solutions, and to achieve this, language expansion and support are of crucial importance. In S Pen Handwriting recognition, we are continuously working to extend the language coverage. Our solution now supports more than 80 languages, and more are on the way.

An interview with Lukasz, a natural language processing expert from Samsung Research Institute Poland can be found in the following episode.

¹ Samsung Research is the advanced research and development (R&D) hub of Samsung’s Consumer Electronics (CE) Division and IT & Mobile Communications (IM) Division.

Samsung Electronics shared its vision for the future at the 2021 Consumer Electronics Show (CES). The company introduced new products at its virtual press conference, showing how it’s continuing to innovate by creating solutions that are flexible, intelligently connected, and use AI to understand context to make daily life seamless.

“Our world looks different, and many of you have been faced with a new reality–one where, among other things, your home has taken on a greater significance,” said Sebastian Seung, President and Head of Samsung Research, the advanced R&D hub of the company that leads the development of future technologies for Samsung Electronics’ SET (end-products), addressing the company’s efforts to enhance the at-home technology experience. “Our innovations are designed to provide more personal and more intuitive experiences that express your personality. We’re hard at work to bring you next-generation innovation, with AI as the core enabler, for your better tomorrow.”

Creating More Intimate and Personalized User Experiences

As people spent more time at home last year, the tech they wanted became the tech they needed. In relying on that tech, people came to understand many ways connected solutions could make life easier. TVs were vital for entertainment—and sometimes, fitness. Chromebooks became crucial home learning tools. Smart fridges helped with remote shopping and meal prep. Smartphones were indispensable.

Samsung’s continued investments in innovations that adapt to evolving lifestyles enable the company to keep pace with rapid changes. Key innovations unveiled include:

• Samsung Bespoke 4-Door Flex: The latest version of the Bespoke refrigerator features changeable panels that come with a choice of colors and materials that made the original a success, allowing consumers to adjust their fridge for form and function. The new 4-Door model, available in North America this spring, also includes a brand-new Beverage Center, which gives quick access to a water dispenser and an automatically-filled water pitcher. Samsung Bespoke 4-Door Flex also comes equipped with a Dual Auto Ice Maker which makes not only regular cubed ice but also smaller ‘ice bites’ to suit different preferences for cold beverages.

• 110-inch MICRO LED: This new screen features self-lit inorganic LED with a slim and nearly bezel-less Infinity Screen design that seamlessly blends into the living space. The result is a spectacularly immersive viewing experience with astounding picture quality. The 110-inch MICRO LED also adds “4Vue” (Quad View), a four-way viewing option—so you can keep up with multiple sports at once, or stream a tutorial while playing a video game. Also, for US consumers, more than 160 free channels¹ are available through Samsung TV Plus. MICRO LED will be rolled out globally beginning this spring.²

• Lifestyle TV: Samsung’s cutting-edge lifestyle TV lineup includes The Serif, The Frame, The Sero and The Terrace—a recently launched 4K QLED outdoor TV—as well as Samsung The Premiere, a cinema-like quality 4K laser projector.

• SmartThings Cooking: A new service from Samsung SmartThings is designed to make your culinary journey seamless. An automatic Meal Planner powered by Whisk’s Food AI recommends meals for the whole week, makes shopping lists with the ingredients you need, and connects to grocery retailers for one-stop shopping straight from the Family Hub refrigerator or your mobile screen. Recipe instructions can be sent directly to synced Samsung cooking devices to minimize hassles and mistakes.³

• Samsung Health Smart Trainer on Samsung 2021 TVs: Samsung Health seamlessly transforms the home into a personal gym, and the new Smart Trainer⁴ feature tracks and analyzes posture in real time, just like a personal trainer. During and post-workout, Smart Trainer provides feedback on form, helps you count your reps, and estimates calories burned. With video and interactive training via Bixby-enabled voice control, Samsung Health Smart Trainer elevates and personalizes the home workout experience.

Bringing AI and Robots to Daily Life Including Housekeeping

Samsung has long been at the forefront of AI and robotics innovation, leveraging its seven global AI research centers to advance technology. By bringing AI to its products, Samsung is creating new home experiences—from washing machines that optimize water usage, detergents, and wash cycles, to TVs with a Quantum AI Processor that can upscale HD content into pristine 8K resolution.

The major technologies featured during Samsung’s press conference include the following:

• JetBot 90 AI+: Coming to the US 1H 2021, this new vacuum cleaner uses object recognition technology to identify and classify objects to decide the best cleaning path. LiDAR and 3D sensors allow JetBot 90 AI+ to avoid cables and small objects, while still cleaning hard-to-reach corners in your home. Also outfitted with a camera, JetBot 90 AI+ is integrated with the SmartThings app to assist you with home monitoring.

• Samsung Bot Care⁵: The latest development in Samsung’s growing robotics lineup, Samsung Bot Care is designed use AI to recognize and respond to your behavior. It will be able to act as both a robotic assistant and companion, helping to take care of the details in your life. It will also learn your schedule and habits and send you reminders to help guide you throughout your busy day.

• Samsung Bot Handy⁶: Also in development, Samsung Bot Handy will rely on advanced AI to recognize and pick up objects of varying sizes, shapes and weights, becoming an extension of you and helping you with work around the house. Samsung Bot Handy will be able to tell the difference between the material composition of various objects, utilizing the appropriate amount of force to grab and move around household items and objects, working as your trusted partner to help with house chores like cleaning up messy rooms or sorting out the dishes after a meal.

Looking to the future, Samsung is also developing other tangible applications of AI technologies for daily life. Samsung is continuing to build on its Samsung Bot Retail, which would guide users outside the home in retail environments, and GEMS, Samsung’s health-focused exoskeleton as mobility aide. These innovations allow robots and robotic devices to co-exist with humans, enhancing their lives and catering to a variety of lifestyles and different environments.

Technology for a Better Future

Samsung closed its CES 2021 press conference by sharing more on the brand’s vision for how technology can enable progress and create a better tomorrow for all. Samsung believes that there is no single all-encompassing solution; a sustainable future requires investments across a broad spectrum. That’s why Samsung is focused on three key areas that are poised to benefit the most from technological innovation: sustainability, education, and accessibility.

“At Samsung, we are always looking for ways to build a better, more sustainable future,” said Sandeep Rana, Senior Manager, Environmental Sustainability Specialist at Samsung, who shared Samsung’s sustainability message at the press conference. “With a focus on the environment, people and society, we will continue to provide our consumers with responsible tools and technologies to address our challenges today and enable a better future for all.”

As part of its commitment to creating a circular economy, Samsung is developing programs and packaging solutions that promote a sustainable product lifecycle—from beginning to end:

• Galaxy Upcycling at Home: At the event, Samsung announced an update to its Galaxy Upcycling program called Galaxy Upcycling at Home. The new program reimagines the lifecycle of an older Galaxy phone and offers consumers options on how they might be able to repurpose their device to create a variety of convenient IoT tools.

• TV Eco Packaging: Additionally, Samsung announced that its eco-packaging would expand to QLED, UHD TV, monitor, and audio products in 2021. As part of an ongoing commitment to eco-consciousness, Samsung is creating products and solutions with sustainability at the core. For example, Samsung’s new Solar Cell Remote Control—made in part with recycled plastic—can be charged via solar or indoor lighting, reducing battery waste.

Believing that future generations will play a critical role in building a sustainable future, Samsung has also prioritized technology education through Samsung Solve for Tomorrow, which encourages young people to creatively use technology to solve community challenges, and Samsung Innovation Campus, which helps students develop technical and personal skills for future jobs.

Samsung also highlighted at the event how it is making its products even more inclusive and accessible for all. The SeeColors application is designed to help those with color vision deficiency (CVD) to adjust the settings on their 2021 QLED TVs for a better viewing experience, and the Sign Language Zoom feature on TVs ensures that everyone, regardless of ability, can easily enjoy cutting-edge technology.

Lastly, understanding how challenging it can be to strike a balance between our digital and analog lives, Samsung presented the Digital Wellbeing feature, designed to help people take control and manage their digital time—so they can use technology to better our lives and empower ourselves.

For more information, images or video on the products Samsung announced at CES 2021, please visit http://news.samsung.com.

¹ 160 free channels for Samsung TV Plus are available for US users only. The Service availability may vary by region and product.

² Availability of MICRO LED will vary by region.

³ Available on Android and iOS devices. A Wi-Fi connection and a Samsung account are required.

⁴ Specific features available may vary by region and product.

⁵ Samsung Bot Care is not currently available for sale. The described functions and technical capabilities have been demonstrated in test settings.

⁶ Samsung Bot Handy is not currently available for sale. The described functions and technical capabilities have been demonstrated in test settings.

Disclaimer: All functionality, features, specifications, and other product information provided in this document including, but not limited to, the benefits, design, pricing, components, performance, availability, and capabilities of the product are subject to change without notice or obligation. Availability of the above-mentioned products, services, and features may vary by region.