But Samsung’s cutting-edge new foldable solutions couldn’t have been realized without the hard work and collaborative efforts of Samsung’s global network of engineers. Among other accomplishments, these employees were able to incorporate advanced technologies like AI to enable drawing and writing on the Galaxy Z Fold3’s large screen, and to allow the device to convert handwriting in a range of languages into on-screen text with S Pen.

To find out more about just what went into the development of the innovative new S Pen, Newsroom spoke to Olga Radyvonenko from the Visual Intelligence Team at Samsung Research Ukraine.

Q: Can you briefly introduce the Ukrainian Research Institute and the kind of work that goes on there?

Artificial intelligence (AI) is one of the key technologies of our R&D center. Our lab is focused on bringing Samsung users human-centric AI technologies that make input and interaction easy and enjoyable.

Q: The S Pen has now been applied to the Galaxy Z Fold3, making it the first of its kind among foldable smartphones. What role did you play in this process?

The major contributions of our lab to the Z Fold3 were the development of on-device deep learning technologies for handwritten content recognition and the technology for reducing S Pen input latency that combines advances in graphics and AI.

Q: What is the S Pen Software Development Kit? What particular advantages does it bring?

The S Pen Software Development Kit (SDK) is being developed by Samsung researchers and developers from many countries around the world, and we are very proud to be part of this international team. The S Pen SDK provides ways to speed up rendering and to reduce and optimize S Pen input latency. In addition, it comprises smart AI-powered features, such as recognition of handwritten text, shapes, diagrams and handwriting straightening.

Q: Given the unique function of the S Pen on the Z Fold3, was there any function of the S Pen that was developed or expanded specifically for the foldable smartphone?

As the new Z Fold3 has a massive and unique foldable display, we’ve focused on using our AI technology to optimize the S Pen experience as well as improve the latency. Thanks to our point prediction technology powered by deep learning, we are able to predict the next movement of the S Pen to minimize the delay between input and output on the screen of the Z Fold3. This, in combination with the front buffer rendering technology and unique large screen, means that our users can enjoy the most natural handwriting, note-taking and drawing on their Z Fold3.

Q: How many languages does the S Pen now support? What were some of the difficulties of expanding the number of recognized languages for the Z Fold3’s S Pen?

At this moment, the S Pen supports 88 languages in total, 14 of which were newly added in the past year.

Adding a new language for handwriting recognition requires a lot of effort. In order to achieve the world’s best level of recognition quality and speed, we have developed an advanced AI technology. This technology is based on deep learning across millions of users’ samples and language modeling for natural language processing (NLP).

Q: Do you have any special tips for making the most out of the S Pen on the Z Fold3 to share with users?

For those who love writing by hand, the S Pen To Text feature now supports writing with the S Pen directly into text fields. The handwriting is then automatically converted to text. Also, you can edit any printed text with your S Pen – for example, when correcting spacing or inserting words.

One feature that will also interest many users is Neat Shape mode. It can help you to not only draw shapes, but also build a flowchart.

Make sure to check the Tips application and not skip usage tips – there you will discover the newest S Pen and input features, and also learn how to use many other Samsung apps to their fullest potential.

Q: Looking forward, what do you think the future of the S Pen looks like? How might it evolve?

We hope to introduce more ways to support better user experiences by providing even more personalization in the near future. Our mission is to make S Pen input like writing in your notepad with a pen, but better and smarter.

Most of all, I want for as many people as possible to experience the power and convenience of the S Pen and its handwritten input capabilities. It has the potential to enhance input on a wide range of devices. We view the S Pen not only as a cool gadget, but first and foremost as an accessible and useful tool. I hope I will see the S Pen and its features casually used in education, in the workplace and in everyday life. I do believe we are set to see a revolution in input technology.

The full interview video can be viewed here:

The event took place in New York at Samsung 837, Samsung’s marketing experience center. It happened at a time when discussion around AI is increasing in importance, with the technology expected to bring about dynamic changes.

Twenty renowned scholars including Professor Zoubin Ghahramani of the University of Cambridge, Professor Barry Smyth of University College Dublin, Professor Alexander Rush of Harvard University and Professor Rob Fergus of New York University were invited to the Samsung Global AI Forum. Along with Samsung Electronics executives and staff, they talked about the major areas of AI application and how the hindrances in advanced AI algorithms can be overcome.

The forum consisted of four separate sessions: Natural Language Processing; Computer Vision; Data Analytics/Recommendation System; and Overcoming Limitations in AI.

• In the first session on Natural Language Processing, participants reviewed the current stage of development in voice recognition and voice command technology and discussed ways to achieve a higher level of language processing in the future.

• The next session on Computer Vision covered the advancement of deep learning technology in image recognition, including its application in areas such as shopping, search, and autonomous driving.

• In the session on Data Analytics/Recommendation System, the participants discussed on how to drive consumer innovation and production efficiencies by leveraging data, which has been growing rapidly due to increasing smart devices and the Internet of Things (IoT).

• The last session addressed the future of deep learning technology and its limitations to apply in many areas which relate to interpreting the vast amount of quality data and the flexible application of algorithms.

“Today’s forum will provide an important opportunity for us to gain insights from global experts on how to prepare for the AI era. Samsung is determined to navigate through any challenges facing us along the way,” BK Yoon, President and CEO of Samsung Electronics told the attendants at the event. “Our passion and commitment together with your ideas and insights will help us take the lead in the age of AI.”

“Samsung has a significant number of consumer devices, and it puts the company in a unique position to apply its technology in smartphones as well as home appliances,” said Rob Fergus, Assistant Professor at New York University. “I think Samsung has a lot of potential for deploying AI in practice for a number of interesting applications.”

Samsung announced its plans to make the forum an annual event so that the direction of AI technology development and technology innovation can continue to be discussed, and the company will continue to strengthen its global network of AI scholars to speed up the rollout of the technology in the future.