CVPR is a world-renowned international Artificial Intelligence (AI) conference co-hosted by the Institute of Electrical and Electronics Engineers (IEEE) and the Computer Vision Foundation (CVF) which has been running since 1983. CVPR is widely considered to be one of the three most significant international conferences in the field of computer vision, alongside the International Conference on Computer Vision (ICCV) and the European Conference on Computer Vision (ECCV). CVPR 2022 will be held as a hybrid event, both in-person and online, from June 19 to 24 in New Orleans, Louisiana, U.S..

Of the thesis papers submitted by Samsung Research, two papers submitted by its Toronto AI Center were selected for oral presentations. Opportunities to give oral presentations at CVPR 2022 are extended to the top 4-5% of the total number of papers submitted. For Samsung’s Toronto AI Center, this is the second time in two years they have earned such a chance, as they were also selected for oral presentation in 2020.

The first of these two oral presentations from the Toronto AI Center will focus on their paper “P³IV: Probabilistic Procedure Planning from Instructional Videos with Weak Supervision”, a study done on how to build next-level AI systems capable of analyzing and mimicking human behavior. Procedure planning is gaining attention in the field, as it could lead to technologies capable of assisting humans in solving goal-directed problems, such as cooking food or installing and repairing devices.

The research team’s approach undercuts the previous requirement of costly data annotations that the start and end times of each intermediate instructional step were labeled with. Instead, the new approach allows AI to learn from natural language instructions, sourced from the internet for example, and predict the intermediate steps. Additionally, the model is enhanced with a probabilistic generative module to handle the uncertainty inherent to procedural planning.

▲ A section from the presentation for “P³IV: Probabilistic Procedure Planning from Instructional Videos with Weak Supervision” by the Toronto AI Center

The second oral presentation to be given by the Toronto AI Center is a study on “Day-to-Night Image Synthesis for Training Nighttime Neural ISPs”. This study is focused on how to synthesize the nighttime image data needed to train Night Mode using neural Image Signal Processors (ISPs) on smartphone cameras. This technology converts clear daytime images — which are much easier to capture than nighttime images — into nighttime image pairs, a strategy that demonstrates performance on par with training on real data captured at night.

▲ A visual from “Day-to-Night Image Synthesis for Training Nighttime Neural ISPs” by the Toronto AI Center

Innovative Approaches Spanning Samsung Research’s Global AI Centers

As well as the two thesis papers submitted by the Toronto AI Center, other global Samsung AI centers — such as the Moscow AI Center, the Cambridge AI Center and the New York AI Center — have also attracted academic attention in anticipation of the conference.

Two papers submitted by the Moscow AI Center were accepted into the conference. The first is a study on what is currently the world’s most competitive Single-View Depth Estimation (SVDE). This study on depth estimation — a research area that concerns many forms of image manipulation, generation and analysis — has gained attention due to its high accuracy. Unlike its predecessors which require resource-intensive post-processing, the proposed GP2 (General-Purpose and Geometry-Preserving) SVDE approach demonstrates outstanding capabilities without the need for this post-processing.

The second paper, “Stereo Magnification with Multi-Layer Images”, is a study of a novel method of 3D photo synthesis. Unlike existing methods of 3D photo synthesis, which necessitate high-capacity memory and processing abilities, the method championed in this paper can be applied to mobile devices as well, thanks to a drastically increased memory efficiency which has not caused accuracy or processing effectiveness to suffer.

The Cambridge AI Center’s paper on “Gaussian Process Modeling of Approximate Inference Errors for Variational Autoencoders” achieves state-of-the-art performance by proposing a novel Gaussian Process (GP) modeling method. This enables test time inference using a single feed forward pass in Variational Autoencoder (VAE).

They also introduced the paper titled “Pushing the Limits of Simple Pipelines for Few-Shot Learning: External Data and Fine-Tuning Make a Difference”. In this paper, the research team proposes a novel neural network based on innovative transformer architecture for the few-shot learning which is a representative method for dealing with situations where data has scarce labelling.

These achievements, among many others, help emphasize Samsung Research’s position in the world of AI research and in the field of computer vision. Other papers that were accepted for CVPR 2022 include works submitted by Samsung Research’s Platform Team and Samsung R&D Institute India’s Virtual Intelligence Team.

Samsung Research operates AI centers in seven different regions; Korea (Seoul), the U.S. (Silicon Valley and New York), Canada (Toronto and Montreal), the U.K. (Cambridge) and Russia (Moscow). Going forward, Samsung will continue to conduct advanced research and actively push innovation in the AI field.

¹ Samsung Research, acting as Samsung Electronics’ advanced R&D hub, leads the development of future technologies for the company’s Device eXperience (DX) Division.

Samsung Electronics’ Global Research & Development (R&D) Centers have presented their studies to the CVPR (Computer Vision and Pattern Recognition) introducing new computer vision, deep learning and AI related technical researches.

CVPR is the world’s biggest conference on computer engineering and AI. At this year’s conference, held online from June 14 to 19, Samsung Research, an advanced R&D hub within Samsung Electronics’ SET Business and its advanced R&D centers gave presentations on a total of 11 thesis papers. Researchers from Samsung Moscow AI center and Samsung Toronto AI center were invited to oral presentations, an opportunity given to only 5% of the entire attendees.

At the oral presentation, Pavel Solovev of Samsung Moscow AI Center introduced ‘High Resolution Daytime Translation without Domain Labels’, which is a technology that changes a high resolution landscape photograph into scenes from various times of the day using data without domain label. Konstantin Sofiiuk also introduced ‘f-BRS: Rethinking Backpropagating Refinement for Interactive Segmentation’, which is a technology that allows a user to simply click an object in a photograph to precisely select and separate it.

‘High Resolution Daytime Translation without Domain Labels’

Joining from the Toronto AI Center, researcher Michael Brown and his team introduced the paper titled ‘Deep White-Balance Editing’, which was also selected for an oral presentation. This AI technology corrects white-balance mistakes made in a captured photograph much more accurately than existing photo editing programs. This technology also allows users to accurately adjust the photo’s white-balance color temperature.

Deep White-Balance Editing

Researchers from Samsung Research America also presented interesting findings at the conference. Eric Luo’s study titled ‘Wavelet Synthesis Net: An Efficient Architecture for Disparity Estimation to Synthesize DSLR Calibre Bokeh Effect on Smartphones’ focused on key enablers to narrow the gap between DSLR and smartphone camera in terms of bokeh, the narrow depth of field (DoF).

Yilin Shen from Samsung Research America’s AI Center introduced a study on out-of-distribution (OoD) benchmarks for deep neural networks research. Shen’s study titled ‘Generalized ODIN: Detecting Out-Of-Distribution Image Without Learning From Out-Of-Distribution Data’ proposed the key machine learning algorithm of drastically improving the detection rate, one of major challenges in AI technology.

Additionally, the studies proposed by researchers from the Samsung Research’s Visual Technology team and Samsung R&D Institute India-Bangalore were also selected by CVPR.

Samsung AI Center Cambridge Director Dr. Nicholas Lane has been awarded the Association for Computing Machinery (ACM) SIGMOBILE Rockstar Award for 2020. The international award was founded in 2013, and recognizes an individual who has produced outstanding research or product contributions in the area of mobile computing early in their career.

The ACM is the high-profile academic organization that presents the prestigious Turing Award, and SIGMOBILE is ACM’s international special interest group on the topics of mobile computing, wireless networks and wearable computing. SIGMOBILE’s Rockstar Award is awarded to an individual who makes outstanding contributions to the field of mobile computing within 10 years of receiving their highest university degree or at an age younger than 40. Entrants are nominated by their peers, before an award committee that includes past award winners screens nominees and selects that year’s recipient. 2020 marks the first time that a professor from a university outside of the United States has won SIGMOBILE’s Rockstar Award.

Dr. Nicholas Lane is a Director at the Samsung AI Center in Cambridge where he oversees teams that work on on-device and distributed machine learning. In recent years he has focused primarily on developing deep learning for modeling human activity and context, as well as enabling state-of-the-art signal processing and modeling algorithms to work efficiently on embedded and mobile-class hardware. Commenting on why Dr. Lane was chosen to receive the honor, the committee said the award recognized “his contributions to the understanding of how resource-constrained mobile devices can robustly understand, reason and react to complex user behaviors and environments through new paradigms in learning algorithms and system design.”

Speaking about receiving SIGMOBILE’s 2020 Rockstar Award, Dr. Lane said, “I am delighted and honored to have received this award. I feel very fortunate to have worked with many wonderfully talented researchers over the years, and this award is a reflection of these collaborative efforts. I believe it is also a signal of the growing importance of on-device machine learning, and I look forward to doing my part to keep Samsung at the forefront of this rapidly evolving technology.”

“Fellow” is the highest grade awarded by the IEEE, and is conferred by the institute’s Board of Directors when a person demonstrates an outstanding record of accomplishment in any of the IEEE’s fields of interest. The total number of fellows selected in any one year cannot exceed 0.1 percent of the organization’s total voting membership.

Dr. Liu is Vice President and Chief Scientist at Samsung’s Montreal AI Center. He develops novel AI solutions for real-world applications like the intelligent automation of 5G networks and has made tremendous contributions to performance and power management advancements in the areas of computing and networking systems.

In his work, Dr. Liu applies insights from a wide range of disciplines, including feedback control, optimization, and machine learning, to lead research in the fields of performance and power management of data centers and networked servers. This line of pioneering research, which helps complex systems adapt to highly dynamic workloads and diverse conditions, has influenced many subsequent research efforts in both academia and industry.

“I am both honored and humbled to be granted the grade of IEEE Fellow,” said Dr. Liu, “I am grateful to my colleagues and the community members who have recognized my contributions, and will continue to devote myself to the development of the AI technologies that will enable intelligent systems to better complement people’s everyday lives.”

Dr. Liu received a B.S. in Applied Mathematics and M.E. in Automatic Control from Tsinghua University in 1996 and 1999 respectively, then completed his Ph.D. in Computer Science at the University of Illinois at Urbana-Champaign in 2006. He has been a professor of Computer Science at McGill University since 2007, and a William Dawson Scholar since 2014. Dr. Liu is a recipient of several awards, including the Mitacs Award for Exceptional Leadership – Professor from MITACS, the Outstanding Young Computer Science Researcher Award from the Canadian Association of Computer Science, and the Tomlinson Science Award from McGill University.

Other members of Samsung Electronics who have been named as IEEE Fellows in the past include: Vice Chairman, CEO and head of the Device Solutions Division Kinam Kim, Executive Vice President and Samsung Research New York AI Center head Daniel Lee, Senior Vice President and Head of the Advanced Communications Research Center Sunghyun Choi, Vice President Jianzhong (Charlie) Zhang of Samsung Research America (SRA), and Fellow Juho Lee of Samsung Research.

They say the world is getting smaller. But with more and more family and friends living far apart and business being done across long distances, it can still feel very big. What a personal and professional boost then for people to have the ability to project their realistic presences into a place thousands of miles away.

This lesser-known application of AI, known as ‘telepresence’¹, is one of the focuses of the Samsung AI Center in Moscow. Samsung Newsroom spoke to Dr. Victor Lempitsky, who leads Samsung’s Moscow AI Center, to find out more about what his center is working on and what he sees for the future of AI.

AI and Telepresence

The overarching research area that Samsung’s Moscow AI Center works on is that of ‘machine learning’, which is a core AI capability that encompasses the areas of language understanding, computer vision and data analysis. Specifically, the center works on next-generation core technology for machine learning, as well as the solutions that will allow them to apply this technology to image and video creation.

“We work on vision learning and telepresence,” Lempitsky relates, “which means developing new experiences that make users who are far apart feel as much like they’re physically together as possible.” It may not initially be clear how exactly AI is pertinent to the area of ‘telepresence’, but Lempitsky explains. “We use computer vision and machine learning to recognize and learn human motions. Then, we use those learned motions to complete a realistic simulation of a person.”

“The ‘Neural Network Rendering’ project has been one of the Moscow AI Center’s major achievements,” Lempitsky continues, “It involved using neural networks to render humans as so-called ‘neural avatars’.”

Asked about other applications for this technology, Lempitsky explains that, “It has made it possible for us to create a 3D digital version of a person’s head from just a single image,” and that “It can also be utilized for a wide range of telepresence applications.”

Awards Success

Russia is globally prominent when it comes to foundational subject areas such as mathematics, physics and source technology. As such, the Moscow AI Center is expected to be a driving force when it comes to leading the AI developments of the future.

Due to the groundbreaking work done by its expert personnel, the Moscow AI Center has accrued an impressive list of awards and nominations.

In 2018, the Moscow AI Center won a competition initiated by NeurIPS (Neural Information Processing Systems), the world’s biggest AI conference. The center also achieved meaningful results at the ECCV (European Conference on Computer Vision) and ICCV (International Conference on Computer Vision).

Lempitsky himself was also awarded a Scopus Award in 2018 for his contributions to the industry. Scopus is the world’s largest global base of peer-reviewed scientific literature, and the award is given to highly cited Russian researchers.

Samsung Devices to Optimize Computer Vision

Samsung has announced that it plans to connect the more than 500 million devices it sells each year and make them intelligent. Asked how the Moscow AI Center is contributing to this vision, Lempitsky remarks that Samsung has broad expertise when it comes to hardware. He points out that an increasing number of devices now incorporate in-device cameras and visual sensors, illustrating that computer vision technology is only set to become more and more instrumental to the industry going forward.

“Our center’s mission is to provide state-of-the-art computer vision software to underpin Samsung’s top-of-the-line hardware and service. I think that’s how we fit in to the wider vision and provide convenience to consumers.”

The Future of AI

Asked what he expects to see over the next 50 years of AI development, Lempitsky notes that 50 years ago people wouldn’t have come close to accurately guessing where we’d be now, but issues some predictions regardless. He says, “I think AI technology will most likely plateau and become the kind of very commonplace technology that people just take for granted. Computers will be able to scan and process visual information just like humans do, and that too will no longer be something that people marvel at.”

Speaking about short-term goals, Lempitsky explains that the Moscow AI Center wants to develop outstanding telepresence capabilities that can create value for people and provide them with new experiences. He outlines how he expects these capabilities will change people’s lives, saying that, “People won’t have to fly, say, from Seoul to Moscow, because they will instead put on devices equipped with AR or VR and be transported to a common environment where the feeling of being present is as strong as being physically together.” Lempitsky is confident that ‘telepresence as real as physical presence’ won’t exist only in sci-fi scenarios for much longer either, saying, “I’m optimistic that this can become a reality.”

Speaking about the business applications of the technology, Lempitsky says, “We anticipate that telepresence developments will transform our behaviors quite significantly.” He also comments that the introduction of telepresence technology will likely lead to an increased uptake in, and different perception of, people working remotely.

Finally, Lempitsky speaks to how developments in this area will help friends and families separated by some distance feel closer to one another, saying that, “Soon, we hope that people will come to see video chats as being as obsolete as sending a message by telegraph seems to us right now.”

¹ Telepresence refers to technologies that allow a person to feel as if they were present, to give the appearance of being present, or to otherwise have an effect at a place other than their actual location.

This summer saw a second AI Bootcamp Summer School take place in Moscow; Samsung’s DesArrolladoras program, designed to help women break into technological fields, launched its third phase in Spain this September; and Samsung Innovation Campus (SIC) launched this October following several successful similar initiatives around the world.

Where Technology Meets Business

The second annual AI Bootcamp Summer School for local students took place at the Samsung Artificial Intelligence Center in Moscow (SAIC-Moscow) this summer. Following a competitive application process, sixteen students from local higher education institutions took part in two weeks of classes on big data for business organized by Samsung Research Russia.

Along with a program similar to the 2018 AI Bootcamp Summer School, which familiarized students with approaches to solving non-trivial tasks related to computer vision, this year’s AI Bootcamp Summer School saw the inclusion of an introduction course to the data-analyst and contemporary marketing professions, providing students with the opportunity to learn how to work with big data and machine learning in business contexts.

The AI Bootcamp Summer School program was also supplemented by additional lectures on biometrics, computer vision and the management of research projects. These lectures were given by Samsung Research Russia employees, providing students with first-hand, practical perspectives on these subjects. The program ended with student presentations and an awards ceremony during which participants received certificates of completion.

Shifting the Balance for Women in Technology

Samsung’s DesArrolladoras program launched its third phase this September in Madrid following the successful outcome of its previous iterations, which have already helped over 1,000 women of various ages expand their technological knowledge to the end of personal and professional development. The initiative, in partnership with Bejob, provides training to women in programming and other technological fields regardless of previous experience.

Participants from a variety of different stages in their careers took part in the DesArrolladoras program. 60% were employed while attending, 22% were seeking work and 11% were pursuing other studies while undertaking the program, which provided its students with the opportunity to acquire highly valuable skills and knowledge for today’s competitive job market. Furthermore, the top 40 achievers of the program’s 300 students will have the opportunity to take a course on AI and big data, valued at 7,000 euros.

The program’s curriculum provides an introduction to the fundamentals of AI, natural language processing, machine learning and deep learning, as well as programming languages required in the field of AI. Furthermore, in the Big Data unit, students are trained in data analysis, management and storage, as well as programming and Business Intelligence.

With this latest edition of the DesArrolladoras program, Samsung is strengthening its commitment to helping women succeed in the field of technology, providing the students with the necessary expertise for a constantly shifting, increasingly demanding sector.

An Education for Future Generations

Samsung Innovation Campus (SIC), a new program that builds on the positive outcomes of Samsung’s previous youth education programs including the Samsung Tech Institute, which has been running since 2013 and provided valuable experience for over 110,000 young people in over 30 countries, launched this October in Russia. SIC offers young people the chance to enhance both their practical technological knowledge as well as their employment prospects through both classroom learning and online-based courses (the latter from 2020).

Culminating in official certification, the skills at the core of the SIC curriculum are rooted in future technologies such as artificial intelligence (AI), Internet of Things (IoT), big data, cloud and mobile platforms, and more. On top of this core syllabus, additional programs, tailored to each country where SIC takes place, will offer students a uniquely customized course experience. Some examples of Samsung Innovation Campus programs include the educational projects “SAMSUNG IT SCHOOL” and “Samsung IT Academy”, which have been successfully developed in Russia.

Having launched in Russia, the initiative will gradually expand to Vietnam, Spain, Indonesia, Chile, Thailand and other countries in the near future, so as to empower and enable a new generation of problem solvers by equipping them with the future-proof skills and localized knowledge to work towards a brighter future.

With initiatives such as these, Samsung is bringing activities to educate people on the skills and technologies key to the Fourth Industrial Revolution, such as AI, to people all over the world. Since the launch of its new vision for its global citizenship initiative, “Enabling People”, Samsung has been working continuously towards helping people succeed within the field of technology for the betterment of their own, as well as the global, community.

One of the most exciting things about the times we live in is the fact that we stand on the precipice of several major technological shifts. What’s more, the individual innovations that make up these seismic changes are not happening independently, but rather are interweaving to inform and empower one another. As we stand here at the edge, no two innovations are enlivening and empowering the tech industry more than those of AI and 5G.

While AI is making technology smarter across the board, 5G is ensuring that connection speeds are fast enough to allow platforms to interface in real-time. So how exactly do AI and 5G work hand-in-hand to make each other, and the entire tech industry, stronger? Dr. Gregory Dudek, Head of the Samsung AI Center in Montreal, extrapolates on the interplay between the two innovations, and how they stand to change things for consumers.

AI for 5G and Beyond

The Montreal AI Center’s primary area of focus is ‘AI-for-5G’, as Dudek explains. “Bringing the strength of AI to bear in order to make use of the full potential of 5G is the key focus of our research in Montreal,” he relates, “The area is a natural fit for our center, since Montreal is one of the world’s hotbeds for AI research, as well as having a telecommunications research community that has a decades-long history.”

Dudek relates that 5G (and beyond 5G) telecommunications systems are very flexible, and can outperform older systems such as 4G, but that extensive configuration is required to take full advantage of them. “In order to exploit 5G networks’ full potential, and make them applicable for a wide range of users, devices and needs, extensive automated reconfigurability is required,” Dudek says, “And that is where AI comes into the picture.”

The impact that AI stands to have on almost every aspect of our lives, and on our technologies as a whole, is immense – there are few areas of the industry that aren’t expected to be revamped by the introduction of artificial intelligence. “Telecommunication systems have been getting steadily more complex since they were first developed,” Dudek says, “In almost all areas of digital communication, complex optimization problems arise and are solved by increasingly sophisticated solutions that I would often call AI.” According to Dudek, the main thing that AI allows devices to do is “adapt to changing conditions”, and this can lead to those devices being optimized in ways that have rarely been seen before. He says that 5G’s “richer protocols and abundance of cells” provide an opportunity to enhance performance with learning-based AI algorithms, and claims that using AI for 5G is likely to prove “more of a fundamental requirement for state-of-the-art performance than just an opportunity.”

5G Underpins the Future

Just as AI can be used to optimize 5G networks, the enhanced performance characteristics of 5G will be important for many key applications of AI in our daily lives.

Dudek outlines that 5G will be an essential component in many of the key applications of AI. Among those applications, Dudek highlights the automotive, edge computing, robotics and medicine sectors, among others.

“One of the clearest needs is in the domain of autonomous cars and delivery vehicles,” he expands, “Efficient coordination of automotive vehicles will depend on the reduced latency that 5G networks offer.”

Dudek also touches on the area of ‘edge computing’, which is additionally set to empower and be empowered by the introduction of 5G. Edge computing means computing that is done very close to data sources, so that relays back and forth from the cloud are minimized. 5G will accelerate communication speeds between ‘the edge’ and the public cloud, while edge computing, in turn, will improve cybersecurity on 5G networks, reduce the burden on the public cloud and lead to savings on storage and processing costs. Eventually, edge computing is expected to prove helpful in realizing AI’s potential because it will allow computation to be distributed across more devices. 5G is also expected to contribute in this area by accommodating a potentially very large number of edge clients and allowing them to do real-time processing with low latency.

Dudek says that robotics will be another important application of 5G, pointing out that areas such as robotic vision, reasoning and action will all depend on the high-quality connectivity it provides. “Many of the most exciting applications of robotics will combine edge computing, sensing, big data in the cloud and interactions between multiple devices,” Dudek reports, “The combination of bandwidth and low-latency will be critical for things like robotic telemedicine solutions as well.”

Touching on other sectors that 5G connectivity is set to empower, Dudek highlights those of medicine (telemedicine, smart diagnostic capabilities and therapeutic technologies), leisure (multiple streams and viewpoints for sports events and rich, multi-person VR), public transportation and factory automation. “In fact,” Dudek concludes, “if we see 5G and AI as a combined package, then there are very few areas of human activity where there will not be some impact.”

Samsung’s Position

The versatility and power of 5G networks will allow for new kinds of connectivity and the emergence of an all-new family of devices. And, as the dual rise of AI and 5G expands capabilities, the degree to which companies and individuals will be able to take advantage of the resulting solutions will depend on their access. That is why Samsung’s position is such a promising one.

“Samsung has successfully exploited successive waves of the most modern and rapidly-changing technologies,” Dudek imparts, “When it comes to these upcoming innovations – not just AI and 5G, but edge computing, multi-device interactions, machine learning, robotics and personalized devices, – a fluid combination of hardware and software will be crucial. The company’s extensive experience in creating individual devices that can talk to one another and have overlapping functions means that Samsung is in an exceptional position to lead this family of emerging technologies.”

Touching on other areas that are expected to change things for consumers, Dudek says that he also expects Samsung to be instrumental in the sphere of robotics, as well as when it comes to innovations regarding original technologies such as AI and 5G. “It cannot be overlooked that robotics is expected to have a huge influence on our lives,” Dudek says, “And, as robotics is a synthesis of AI and mechatronics, this area is very well-matched to Samsung’s strengths as well.”

What AI and 5G Will Bring to Consumers

For most of us, our devices have already become crucial to going through our daily routines, but the dual inception of 5G and AI is set to make our devices even more complementary to our day to day lives.

“In the near term, AI will make our home lives healthier, safer and more fun,” Dudek says, “But AI also has the potential to help people communicate more easily across linguistic and geographic boundaries.”

“A world where 5G-dependent smart devices become woven into our lives is much closer than most people expect,” Dudek continues, “I expect this embedded, interconnected intelligence to start playing a role in our lives in the near future.”

When one considers the entirety of the picture that Dudek paints, it is hard not to start seeing AI and 5G as more of a necessity than a luxury. Asked how he envisions the combined inception of 5G and AI will change our lives, he says “One of my goals as a researcher is to have a positive impact on the world, and to play a role in bringing important new technologies to life.” He also says that he expects the introduction of 5G and AI to progressively take away more and more of the mundane tasks that people deal with day-to-day. “People will expect much more from the objects around them,” he says, “and this will allow them to focus more on the aspects of their lives that they find more rewarding.”

There is much anticipation these days around the field of robotics with its immense potential and promising future applications. However, a large gap exists between public expectations and what is actually deemed technically feasible by scientists and engineers today. Fortunately, Samsung’s New York AI Center is buoyed by the presence of a team of highly skilled researchers, led by robotics and AI expert Dr. Daniel D. Lee, who are working to close this gap. Samsung Newsroom spoke with Dr. Lee about the work being done at the center, as well as the facility’s ability to foster collaboration in a range of areas and attract top talent.

Challenges to Overcome

Asked about his center’s mandate, Lee explains that the New York AI Center focuses on “fundamental research at the intersection of AI, robotics and neuroscience.” The center’s objective is to “solve challenging problems” at this intersection, and one good example is the problem of robotic manipulation¹.

Put simply, robots need to become far more skillful before they are ready to help humans with physical tasks in their daily lives. The first step involves endowing robots with the intelligence to perceive and understand their surroundings. Next, they must be able to make swift decisions in unpredictable situations. Finally, robots should be dexterous and nimble enough to perform the appropriate actions. However, it is impossible for robot designers to anticipate every contingency robots will encounter in real world environments. Thus, robots need to be able to learn from experience just as humans do.

At this time, most common machine learning methods are not suitable for teaching robots since enormous amounts of training data are required. Lee explained that there are several challenges that need to be addressed regarding machine learning for robotics.

“Dealing with the physical world is much more difficult for AI than playing video games or Go,” he explains, “We are currently developing AI learning methods that can deal with the uncertainty and diversity of the physical world so that robots become more prevalent in homes and workplaces. I would compare the state of robots today to computers in the 1980’s during the transformation from mainframes to personal computers.”

The New York AI Center is addressing such challenges to provide a richer AI and robotics experience. For instance, the center has recently developed novel AI methods that are able to efficiently teach robots using limited data. One recently-developed method trains a neural network to generate motion trajectories for a robot arm directly from camera images.

Getting a Handle on Robotic Manipulation

In order to allow robots to handle things for people, robots need to learn how to touch, grasp, and move a variety of everyday objects. Lee explains how the problem of dexterous robotic manipulation is an area of focus for the New York AI Center.

Lee comments that “the ability of humans and some animals to manipulate household objects is currently unmatched by machines. That’s why we are investigating how AI-based solutions can be applied to make breakthroughs in this area.” Extrapolating further, Lee explains that ‘dexterous’ robotic manipulation “requires the ability to precisely and robustly handle objects exhibiting uncertain material properties.”

“Manipulation is relatively easy if the objects and environments are carefully controlled, such as on a factory floor,” Lee reports, “But it becomes much more difficult in unknown, cluttered environments when faced with a diverse array of objects.”

By way of an example, Lee lays out the capabilities that would be required for a robot to serve a chilled glass of wine in a restaurant. “How heavy is the glass, and how slippery is it due to condensation?” He adds, “It’s impossible to completely model all the possible physical characteristics of the glass of wine, so machine learning is critical in training robots to handle the difficult situations.”

Collaborative Innovation

As the AI sector has grown more sophisticated, it has become increasingly clear that collaborative solutions are critical for researchers to overcome the challenges they face. In an area as complex and multi-faceted as robotic manipulation, contributions from and collaborations with “the world’s best and brightest” will be instrumental, comments Lee. He highlights the value of working with both other Samsung AI Centers and academic institutions, saying that, “solving fundamental problems in AI to positively impact society requires drawing upon the ability and skills of numerous experts globally.”

He added, “The Samsung AI Centers invite collaborations with researchers who can help address these difficult challenges. We currently have a number of faculty from leading academic institutions who are collaborating with us in New York.”

Attracting Talent

Lee highlights just how beneficial being located in New York has been for his team, saying that “certainly, New York City is one of the greatest and most diverse cities in the world. It is a magnet for world-class research and engineering talent.”

Attracting the very best in talent is extremely important to remain on the bleeding edge of future AI advancements, and Lee reports that the center has been fortunate in this area, saying, “We have benefited from being able to attract and recruit some outstanding researchers since we started the Center.”

“Our team is composed of expert scientists and engineers who are creating innovative theories and algorithms and state-of-the-art technological developments,” Lee adds, “It’s been great working with them to publish in leading academic conferences and journals as well.”

How Robotics Could Revolutionize Our Lives

Speaking about how he envisions robots will fit into society in the future, Lee points out that, in their infancy, some robots drew attention because they were cute and fun, but that people tended to use them less as the novelty wore off. In order for people to see robots as valuable and relevant, new systems need to have enough intelligence that they become indispensable in our daily lives.

“Intelligent robotic systems have the potential to completely revolutionize how people go about their activities in the future,” Lee extrapolates, “In the near term, we will see modest improvements on simple tasks in constrained environments. But more complete systems that can handle a variety of chores and complex tasks will require further research breakthroughs. The Samsung AI Centers are helping to generate those new advances.”

Asked to outline what he sees as the ultimate vision for AI and robotic intelligence, Lee says, “I grew up reading and watching science fiction stories that envisaged amazing robots helping humans. It would be incredible to see some of those positive visions actually come to life.”

¹ The ability for robots to interact with and move physical objects in a range of environments.

As technology has evolved, it has changed our lives dramatically. It’s truly startling to think just how different life was before the invention of innovations like smartphones, the internet and PCs.

Recently, AI has emerged as a hot topic in this regard based on its potential impact both on technology and on society. Especially with on-device AI¹, AI will be embedded in devices that we use in everyday lives without necessarily connecting to processors in the cloud. To learn more about this exciting subject, Samsung Newsroom met with the head of the Samsung AI Center Cambridge, Dr. Andrew Blake.

Dr. Blake was formerly the Director of both the Alan Turing Institute (which he also helped found) and, before that, Microsoft’s Cambridge Research Laboratory. As a pioneer in the development of the theory and algorithms that make it possible for computers to behave as seeing machines, he explained how Samsung’s AI and hardware innovations will enrich people’s lives in fundamental ways.

Taking IoT to the Next Level

“AI is what is going to breathe life into IoT,” begins Blake.

On-device AI realizes AI functions by processing AI algorithms on the device itself, without necessarily connecting to the cloud, and that is advantageous for privacy and personal information protection, as well as for security.

Unlocking AI-powered smart devices’ true potential will require a combination of two factors: seamlessly connected hardware and an approach to AI that is human-centric above all else.

“One key area is health and fitness – for example, linking exercise, food and mental wellbeing. Another is communication and memories – especially via photography and video. For that, we have to move past the academic world of prototypes working on high-powered computer systems, and get AI working in a leaner fashion – on the everyday devices that people are using.”

The Right Tools

As Blake notes, Samsung’s wide-ranging device portfolio makes it uniquely qualified to deliver this human-centric future for AI.

“This is a great time to be adding new dimensions to Samsung’s AI capabilities, given the company’s leading market position in devices of all sorts,” says Blake. “On-device AI begins with hardware, and this is why working for Samsung is such a fabulous opportunity for AI researchers.”

“Hardware is the channel that moves us beyond simply smart algorithms, to put those algorithms in everyone’s pockets and homes. The big challenge that Cambridge is addressing is moving high-quality embedded AI beyond specialists’ research labs, where people with PhDs in machine learning and in systems work for several months to implement a new embedded system. We envisage a world where advanced tools enable the world’s software developers to move their AI models, simply and effectively, onto Samsung devices, and we are working hard on those tools.”

As Blake explains, on-device AI, in which AI algorithms are processed on a device itself, rather than sent to the cloud, offers significant advantages here by providing a safe and reliable means to protect users’ privacy and data. “We also need to do that in a way that holds the data close, to reassure people that their data is being held safely and privately,” he added.

A Multi-Disciplinary Approach

When it comes to AI, what exactly does the Cambridge AI center want to bring to consumers? The answer to that question is what Samsung has described as a human-centric approach to AI innovation, which Blake describes in further detail.

“Human-centric AI is about homing in on the areas of life that people really care about,” says Blake. “I believe this will require a multi-disciplinary approach. I am not so excited about a future designed solely by engineers. Instead, we need to collaborate with other disciplines, especially design – hardware, user interfaces, and above all, system design – and with human disciplines such as psychology, to achieve a technological future that really helps people live better.”

Taking this multi-disciplinary approach, the Cambridge AI center endeavors to better understand human behavior by exploring areas like communication of emotion, and further expand the boundaries of user-centric communication.²

Drawing from a Diverse Team

Samsung AI Center Cambridge employs a team of experts of various disciplines, and emphasizes collaboration between them.

“We work together a lot as a team,” says Blake. “Our two Program Directors, Maja Pantic and Nic Lane, are world-experts in non-verbal human behavior and in embedded AI, respectively. We also have quite a few senior specialists in machine learning, in machine vision, in networks and devices, and in computing and cognition. We now have a team of very talented people, and new ideas are flowing freely!” says Blake.

As Blake notes, what makes the Cambridge AI center unique is not just its team’s wide-ranging expertise, but its location as well.

“Cambridge is a very special place,” begins Blake. “The university is one of the strongest in the world in research, and that is coupled with an extraordinary culture of research ventures, and a whole constellation of startups in robotics, medicine, AI, self-driving, and many other areas.”

“Being in this environment is important to us for several reasons. It is a stimulating ecosystem and an extraordinary network; it is a rich source of expert talent; it is well connected to the ‘Golden Triangle’ with London and Oxford.”

Of course, in addition to taking full advantage of the benefits that come with its location, the Cambridge center draws strength from its connections to other AI centers in Samsung Research’s global network.

“I am especially pleased to be connected with Samsung’s other AI centers around the world, where I know some of their internationally renowned scientists well,” says Blake. “I believe that, as we begin to work together, we can bring something special to consumers.”

Having more than 40 years of experience working in the field of AI, Blake added, “I was born in the same year as AI – 1956 – the year the Dartmouth conference famously coined the term AI – and I have been studying AI vision for 40 years. I have been lucky to have such an extraordinary career.”

¹ AI that processes information on a device itself, rather than sending that information to the cloud. Because on-device AI does not rely on outside networks, it is regarded as safer and more reliable than cloud-based AI.

² https://internetofbusiness.com/samsung-uk-to-open-new-ai-centre/

Sven Dickinson, Head of Samsung’s Toronto AI Center

Can you imagine a world where the personal AI assistant on your smartphone is able to understand as much about the world as you do? What about a scenario where communicating with that AI assistant is as natural and easy as interacting with another human? Developing those kinds of capabilities is exactly what the team at Samsung’s AI Center in Toronto are putting their minds to.

Samsung Newsroom sat down with Sven Dickinson, Head of Samsung’s Toronto AI Center to learn more about these exciting fields, and what they could mean for the future.

The Vision for Vision

The second Samsung AI center established in North America, Samsung’s Toronto AI Center is led by Dr. Sven Dickinson, an expert in computer vision and former chair of the Department of Computer Science at the University of Toronto.

At the epicenter of AI research and development, Samsung’s Toronto AI Center is mainly focused on developing the visual understanding capabilities that allow a Samsung device to understand the world in which it’s situated. In addition, the team is working on multi-modal interactions, which are user-machine interactions that encapsulate vision, language and knowledge.

“Allowing Samsung devices to ‘see the world’ through computer vision enables them to ‘visually ground’ their dialog with the user, providing an integrated, multimodal experience that’s far more natural than one that’s solely vision or dialog-based” says Dickinson, whose expertise includes exploring problems surrounding shape perception and object recognition.

Touching on the benefits of multimodal technology, Dickinson claims that, “I should not have to read manuals to figure out which buttons to push on my device and in which order. Rather, I should be able to show my device what I want, and tell it what I want, in natural language that is understandable, and situated in the world that I live in.”

Extrapolating on the interplay between computer vision and multimodal inputs, he goes on to say that, “To achieve this breadth of comprehension, the device has to have a model of my understanding of the world, the capacity to communicate robustly and naturally with me, and the ability to see and understand the same world that I see.”

Remarking on applications for this technology, Dickinson identifies the most compelling as being “a personal assistant that you not only speak to, but that sees the world the same way that you do.” Speaking to the importance of multi-modal device interactions, Dickinson points out how much cancelling out one of the modes of communication (audio, speech, sight etc.) would hamper communication between two people, and says that also applies to personal devices.

A Truly Enhanced User Experience is Key

At the 2019 Consumer Electronics Show (CES), Samsung unveiled its vision for Connected Living, which involves connecting the 500 million devices the company sells every year, and making them intelligent. Dickinson highlights that Samsung’s broad product portfolio will be instrumental in fulfilling this vision, saying that, “What differentiates Samsung is that it makes a multitude of devices in the home, including digital appliances, TVs, and mobile phones. Samsung has a unique opportunity to leverage these devices to yield a multi-device experience which follows the user from one device to another, and one room to another. This will help realize the full potential of each device to effectively communicate, to help the user execute device-specific tasks, and to learn the user’s habits and preferences so that subsequent communication is not intrusive but instead ‘always helpful.’”

Speaking about what his center will need to do to truly realize computer vision and multimodal interaction, Dickinson comments that, “Vision is not about understanding images; vision is about understanding the world. Truly capable AI systems must possess an understanding of our world, of its physics and causality, of its geometry and dynamics. They must also be able to model and understand human behavior.” He extrapolates on this by pointing out that, “If our devices can see the 3D world that we live in the same way as we do, i.e., understand the 3D shapes, positions and identities of objects in our shared environment, then our devices can visually experience the world as we do. Such a shared visual context will be crucial in developing fully realized personal assistants.”

Dickinson says that Samsung is leading the charge when it comes to truly intelligent visual understanding, and identifies ‘visual grounding’ as an essential pre-requisite for well-rounded visual understanding capabilities. “Samsung is leading the way when it comes to developing human-device interaction that closely mimics human-human interaction,” Dickinson says, “We aim to provide visual grounding and knowledge representation scaffolding for dialog-based interaction services. Without these components in place, users become disappointed with services, and quickly tune out.”

Human-device Interactions Based on Open Information Sharing

Dickinson goes on to explain that AI also needs to be able to explain itself to the user. He remarks that, after failing to carry out a task or provide an appropriate response, “A device should be able to reflect to the user precisely how and why it came up with that response (or lack thereof). Ideally, it should be able to follow up with the user by asking a question or asking the user to adjust its camera or other input modes so that it can gather more information and formulate an appropriate response.” Dickinson relates that this kind of openness and information sharing will be key to the further sophistication of human-device interactions, noting that “What we call the domain of ‘active dialog and active vision’ is where the system can construct a mental model of what the user understands, and can, in turn, open up its own mental model so that the user can understand the thought processes of the device.”

The Benefits of Being Based in Toronto

Asked about how being based in Toronto affects the AI center, Dickinson remarks that the center enjoys a lot of benefits due to its close proximity to various world-class AI-related institutions, including the University of Toronto, York University and Ryerson University. “Being in Toronto offers us a tremendous regional advantage,” Dickinson comments, “We are across the street from the University of Toronto, home to the Department of Computer Science (DCS), which is one of the top-10 international computer science departments. Over half the members of our AI Center are either active faculty, graduates or current students at DCS.”

On the topic of collaboration between Samsung’s global AI centers, Dickinson relates that, “The seven global AI centers are working to create industry-leading solutions in their respective areas of focus, while coordinating to achieve the common goal that is realizing Samsung’s ultimate AI vision.” Dickinson touches on the topic of the Toronto AI center collaborating with other AI centers further afield, saying that, “We are starting to explore possible research collaborations with other global AI centers, and hope to converge on some use cases of value to Samsung and its products and services.”

Sebastian Seung, Executive Vice President & Chief Research Scientist, Samsung Electronics

There’s no denying that the age of AI is upon us and that the ways we engage and interact are set to change in big ways. In anticipation of this, Samsung Electronics has opened AI centers across the world to ensure that the company leads the charge on AI. 2019 marks the 50^th anniversary of Samsung Electronics, and the company has forecast another 50 years of ingenuity ahead, with AI set to be at the heart of future innovation.

To gain a great insight into what AI means for the future of society, as well as the work being done at the Samsung AI Centers, Samsung Newsroom sat down with Executive Vice President & Chief Research Scientist, Dr. Sebastian Seung.

Seung joined Samsung Electronics in 2018. He is also a professor at the Princeton Neuroscience Institute and Department of Computer Science. Seung is one of the most influential scientists in the world when it comes to AI research based on neuroscience.

Artificial Neural Networks and AI

Based on his extensive experience and insights into the field of artificial neural networks¹, Seung is working on developing future growth engines for Samsung Electronics by establishing an AI strategy and providing advice on advanced research.

Artificial neural networks are mathematical models or computer simulations of the biological neural networks in the brain. “Convolutional networks, now the dominant approach to computer vision, were inspired by Nobel Prize-winning neuroscience of the 1960s,” according to Seung. His research at Princeton focuses on mapping the neuronal “wiring diagram” of the cerebral cortex. “I hope that our 21^st century studies of the cortex will finally reveal how it learns, and that this new understanding will lead to more powerful artificial neural networks,” says Seung.

In his work for Samsung, Dr. Seung travels back and forth between the U.S. and Korea. His recent work is especially focused on advanced research regarding robots, which is the New York AI Center’s main field of research.

Deep Learning and Robotics

These days, robots are already present in society in the forms of robot vacuum cleaners in our homes and robotic arms being used in factories and by shipping companies. Seung acknowledges that these robots already represent an early stage of this technology, but says that what he is aiming for is something much more sophisticated. “In order to develop robots that can, for instance, reach out to pick something up and put it away,” Seung says, “we have to equip them with computer vision so they can see what’s in front of them, and with brains so that they know what all these objects in your house are and what they should do with them.”

Seung acknowledges that labs have tried in the past to achieve these capabilities through the classical approach of programming, but that that hasn’t really worked out. “We have realized that we have to somehow allow the robot to learn to perform the required actions itself,” says Seung, “and a lot of that involves the deep-learning approach.”

Seung points to the area of home automation as a primary application for their work. “In the future, you can imagine robots that won’t just give you weather information or change the temperature – they’ll perform far more helpful tasks in your home. They’ll pick up the toys, wash the dishes and even take the laundry up and down the stairs.”

AI in Society

No discussion of AI would be complete without addressing the apprehensions some people feel when it comes to the technology and the ways in which it stands to change our way of life. Seung addresses this question first with regards to the prospect of people losing their jobs to automation. “I think this issue of robots taking our jobs is exaggerated,” he relates. “Firstly, in the last 20 years, the U.S. and many other developed countries have lost a lot of jobs to offshoring, not just to automation. As in the first industrial revolution, many jobs were eliminated, but that didn’t mean that there were fewer jobs in total, because new jobs arose from the new circumstances.”

Seung went on to comment on the wider attitudes towards automation of industry, and the fact that the issue needs to be looked at through a different lens. “If robots really could do all of our work, why shouldn’t we be happy about that?” he said.

Asked the inevitable question about doomsday scenarios in which machine intelligence outstrips that of humans and robots take over the world, Seung claimed, “People don’t actually know what the real capabilities of AI are. And part of that is a public misconception based on science fiction movies that convince people that robots can do anything. In reality, robots are still really clumsy.”

Seung went on to point out that AI developments may well end up greatly helping us, instead of dooming us. “Are robots going to do something bad to us?” he said. “Well, the reason that I don’t worry about that is that of all the environmental and political threats to humanity, robots are not very high on the list. And not only that, I think that if humanity is to best equip itself to deal with any and all future threats, we need to be as smart as possible. And that involves having the most sophisticated technology. You could be a science-fiction pessimist and say maybe these robots could turn on us, but you could also argue that maybe we’ll use these robots to save us.”

Speaking to other misconceptions about AI, Seung pointed to the actual capabilities of the technology. “The public thinks that AI can do more than it really can,” he said. “To give you an example, I met someone who wanted AI to replace her doctor. But there are many things that no human doctor can fix. So, because our current approach to AI involves training machines based on the expertise of human practitioners, if the best human experts can’t solve it, then the AI can’t do it either. It’s not like AI will all of a sudden be able to perform tasks better than the human experts.”

The Next 50 Years of AI

Having reached its 50-year anniversary this year, Samsung is now looking to AI to spearhead the next 50 years of innovation. Asked what he expects for this period, Seung said, “In 20~30 years robots will be able to work in the home just as humans can. It will have happened the same way that the mobile phone revolution has happened. Everybody has a mobile phone now – billions of them are sold every year – and the same is going to be true of robots.”

Home automation and self-driving cars based on AI are other hot-button topics right now. Seung says he fully expects AI-equipped cars to become a reality, but that the timeline for their inception is hard to sketch out. “AI is going to lead to a lot of labor-saving things happening in people’s everyday lives, like autonomous cars for instance,” he said. “Are they going to be here next year, or will it take 20 years? Experts are realizing that full autonomy will take longer than the media originally portrayed, but most still believe that it will be achieved. I’d like to see Samsung have some part in that revolution, if not lead that revolution.”

The prospective benefits of AI are enormous in scale and diverse in focus. Outlining some of the applications of AI that the general population may not be aware of, Seung remarked that “The effect AI could have on scientific research is a major one. AI can be applied to accelerate scientific discovery, and in the long term, it will have a huge impact on areas like materials engineering and chemistry. Let’s say I want to design a new molecule with certain properties – AI might allow me to do that more easily. Then, that new molecule could have applications for a drug company, or really any company that creates materials. So AI is not only applied to technology – it’s also used for scientific discovery, which then accelerates the advancement of technology.”

¹An artificial neural network is an attempt to simulate the network of neurons that make up a human brain so that the computer will be able to learn things and make decisions in a humanlike manner. (https://www.forbes.com/sites/bernardmarr/2018/09/24/what-are-artificial-neural-networks-a-simple-explanation-for-absolutely-anyone/#1b4809251245)

Gary Geunbae Lee, Head of Samsung Research AI Center, Samsung Electronics

Nowadays, artificial intelligence (AI) has emerged as a leading global future technology trend. AI is so much at the center of the current technological revolution that it is expected to fundamentally alter not only the IT industry, but also the automobile, banking, and medical sectors. As a result, companies are making efforts to hire AI experts and are investing in research and other related business fields to ensure that they are fully prepared to integrate AI into products and services that can benefit people’s lives.

Samsung Electronics has long recognized the significance of AI and has been actively investing in the area. The company currently maintains seven AI centers in five different countries; South Korea (Seoul), the U.S. (Silicon Valley and New York), the U.K. (Cambridge), Canada (Toronto and Montreal) and Russia (Moscow). But the question remains – what is behind AI’s rapid evolution, and what forms is the technology expected to take going forward?

In a series of interviews, Samsung Newsroom sat down with leading experts from each of the AI centers around the world about the latest AI trends and what they believe the future holds for the technology. The first interviewee is Gary Geunbae Lee, Senior Vice President and Head of Samsung Research’s AI Center in Seoul. We asked him for his insights regarding AI development, and enquired about what makes Samsung’s approach to AI distinctive.

What is AI to Samsung?

AI is the realization of implementing the human capabilities of seeing, listening, decision making, moving and learning into computers,” said Gary Lee. “To think about it another way, AI is a combination of A (algorithms), B (big data), and C (computing power). These are the three key components that allow us to construct well-rounded artificial intelligence.”

Lee explained that Samsung Research’s AI Centers around the world conduct research that covers the full gamut of AI development, including computer vision, language understanding, data analytics, robotics and machine learning. Their research aims to bring the capabilities of AI closer to that of the human brain. “Compared to the time it took humanity to evolve to its current state, the history of AI is very short – only about 60 years,” noted Lee. “AI still isn’t quite there as far as fully realizing human actions, but I believe the technology will continue to improve quickly.”

Each of Samsung’s seven AI Centers spread across the world has its own specific fields of research. For example, the Cambridge AI Center is focused on On-Device AI and AI technology related to next-generation telecommunication networks, the Moscow Center is the Moscow Center is focusing on AI core technology such as data generation for machine learning and advanced deep learning, and the New York Center’s focus is centered on advanced research fields such as robotic manipulation. The Seoul Center works on language understanding, speech processing and big data and also coordinates the other centers, fostering collaboration and efficiency between them.

Samsung Seoul R&D Campus

Once AI products and services are ready for release, they are made available to consumers through various services, including Bixby. The majority of recent Samsung products come with Bixby already incorporated into the device, and Samsung is working hard to make all Samsung products AI-ready in the near future. “Once Samsung’s AI speaker is launched, it will enable seamless interconnection between smart devices,” affirmed Lee. “This will enable us to achieve our goal of implementing AI on all of our products to the goal of fostering environments led by connectivity.”

In 2018 the company revealed a software development kit (SDK) for Bixby and currently Bixby supports voice commands and translations in English, Chinese (Mandarin), German, Italian and Spanish. The diversity and global versatility of Samsung’s product portfolio is one of its key strengths, and Bixby will enable users around the world to connect their devices seamlessly.

When Samsung Products and AI Meet

“Samsung Electronics sells 500 million products every year. We strive each and every day to provide new services and features that are a fit for our devices in order to closely match our customers’ lifestyles,” noted Lee. One of the greatest strengths of the Samsung AI offering is the device integration; Lee highlighted the added value Samsung brings to users by assimilating a wide range of AI capabilities into the devices that have already become staples in users’ lives.

“For example, AI can recognize the ingredients in your refrigerator and automatically recommend recipes which it then displays on the refrigerator’s screen,” said Lee. “After you put the food in the oven, the AI-powered software sets the temperature and cooking time intelligently, according to the recipe.”

Samsung’s AI centers around the world focus on developing original technology such as natural language processing for Bixby and advanced research areas such as robotics; R&D teams at each business unit then work on merging and applying the new innovations to the products.

Teaching AI to Think for Itself

Despite its recent origin date, AI recognition technology has evolved considerably over time; in some areas, even, AI has proven to be more advanced when compared with the abilities of humans. Nevertheless, there are still some areas that AI needs to improve to become the ultimate assistance tool: performance and accuracy.

While current AI technology is capable of understanding spoken languages, there is still a long way to go until AI is able to fully process words within their context as a human does. In other words, what AI needs to develop is the comprehension of hypothetical reasoning, or the ability to read between the lines, of a user’s command or statement. There is also a need for more development regarding AI’s capability to taking different approaches to solve problems based on individual, or unpredictable, situations and daily scenarios. Future AI development currently rests on how fast improvements are made in these areas.

Samsung’s AI Philosophy

AI can provide a significant competitive edge for businesses – but if abused or mishandled, it can cause a social problem when not kept under control. Therefore, sustained ethical compliance is critical when conducting AI research. Samsung takes this responsibility very seriously and is constantly formulating ways to improve its practices and increase accountability.

“Although AI is meant to improve people’ lives, the possibility of its abuse cannot be ignored, so ethical compliance is very important,” stressed Lee. “There are three ethics that Samsung follows in regards to AI.” These are fairness, accountability and transparency. The development or use of AI must not lead to discrimination or prejudice, and the company assumes total responsibility for the technology and maintains transparency in its data collection and management process. In order to ensure that it keeps developing AI products and services that are worthy of consumers’ trust, Samsung became a member of the Partnership on AI (PAI) last year.

Privacy is another important part of Samsung’s AI policy. Since the more data an AI service is able to utilize, the more helpful it can be, all data harnessed must be processed transparently so that consumers feel safe when using Samsung products and services. “Samsung Electronics greatly prioritizes data security and privacy,” noted Lee. “We adhere to all the related laws and regulations on data security, including the GDPR¹ in Europe. We are working towards implementing technology that detects security vulnerabilities in our AI codes to ensure our customers can use AI-enabled products and services safely. At the same time, we are incorporating AI into our security software to develop identification capabilities that will provide further peace of mind for users.”

So, what is the final goal Samsung has in mind when pursuing an AI-enabled future? From Lee’s perspective, it is totally consumer centric. “For me, AI is about adding value to consumers’ lives thanks to the integration of AI into their everyday products and services. Which this in mind, Samsung Research operates, and will continue to operate, under the conviction that user-based AI is always there, must be helpful, must be safe, must be user-centric, and, finally, must always be learning.”

“At Samsung, we always keep our AI principles front of mind,” Lee emphasized. “Firstly, all of our AI-based products and services continuously learn by themselves and become smarter over time, improving performance and usability while interacting with consumers. Secondly, Samsung AI is always there across a range of devices for whenever a customer needs it. Thirdly, Samsung is committed to developing AI technologies that are always safe and offer consumers peace of mind. Finally, Samsung’s AI always pursues user-centric customization to provide as helpful and as personalized service as possible.”

“These principles are the fundamental technological base for all the AI products and services of Samsung,” added Lee. “They enable Samsung to provide meaningful and tangible user-oriented experience and values with our AI offering.”

¹ The EU General Data Protection Regulation

Pavel Ostyakov, one of the researchers from SAIC-Moscow, won first place among 110 teams in the “Inclusive Images Challenge,” a Kaggle¹ competition hosted by Neural Information Processing Systems (NeurIPS) 2018, which took place in Montreal, Canada from December 3^rd to the 8^th. NeurIPS, formerly known as NIPS, is the world’s largest conference in the field of AI and machine learning. In 2017, over 8,000 researchers and scientists attended the event. In addition to those in machine learning and neuroscience research, experts from many related fields such as cognitive science, computer vision, statistical linguistics, and information theory actively participate in the conference. Winning the challenge at this prestigious event was a significant achievement for both Pavel and Samsung. Pavel was also named a Kaggle Grandmaster, which is the highest tier possible for contributors, and is currently ranked as one of the world’s top five scientists on Kaggle.

In the “Inclusive Images Challenge,” participants developed image recognition and classifier models that can successfully perform image classification tasks even when the test images are geographically and culturally different from the previously shown images in the provided dataset.

(Left image, from left) Pavel Ostyakov, a Researcher at Samsung AI Center-Moscow (SAIC-Moscow), and Jin Wook Lee, the Head of Samsung R&D Institute Russia. (Right image) A snapshot of SAIC-Moscow’s opening ceremony, which took place in May of this year

For example, an image classifier may fail to properly apply “wedding” related labels, such as “bride,” “groom,” and “celebration,” to an image, if a couple is not wearing traditional western European wedding attires or colors. This challenge attempts to address the biases in previously existing algorithms and classifier models. Through the challenge, researchers can identify ways to teach image classifiers to generalize the accumulated data and apply them in new geographic and cultural contexts. The expectation is that the scientific community will make even more progress in inclusive machine learning that benefits everyone, everywhere.

Such competitions reflect the movement towards realization of ethical and non-discriminatory AI, which also aligns with Samsung’s goal and vision. To this end, Samsung has recently joined the Partnership on AI (PAI) to take part in the development of inclusive, fair, and responsible AI technology.

Last September, the SAIC-Moscow team also participated in “The 2nd YouTube-8M Video Understanding Challenge,” hosted by the European Conference on Computer Vision (ECCV) 2018. ECCV is one of the world’s top research conferences in the area of computer vision, held biennially. In this Kaggle competition, researchers were provided with public YouTube-8M training and validation datasets that consist of millions of videos with labels, and then asked to develop classification algorithms to accurately predict the labels for 700,000 previously unseen YouTube videos. The SAIC-Moscow team utilized a unique approach in its complex model and data analysis, taking second place by a very narrow margin.

An exterior view of the White Square Business Center, where SAIC-Moscow is located

“We love these competitions because they provide us with opportunities to measure ourselves against the best in the AI industry,” said Pavel Ostyakov at SAIC-Moscow. “Researchers at Samsung are obsessed with making AI a part of everyday lives. So, it is exciting to take part in the challenges where we can contribute our skills to develop the technology.”

The global AI centers reflect Samsung’s commitment to next-generation AI development. Besides Russia, there are six others located across the globe – Korea, Silicon Valley and New York in the U.S., Toronto and Montreal in Canada, and Cambridge, the U.K. Each location is conducting research, focused on a different area of unique strengths. These AI centers are playing a pivotal role in realizing Samsung’s vision for human-centric AI technologies and products.

¹An online community allowing users to find and publish datasets, explore and build models in a web-based data-science environment, work with others, and enter competitions to solve data science challenges.

Today’s announcement complements earlier news of multiple AI centers launched in North America and continues Samsung’s efforts in AI that include the development of Samsung’s virtual assistant, Bixby. The center is the fourth Samsung AI Center to be established in North America following the centers launched in Silicon Valley, New York and Toronto.

The opening of the AI center in Montreal will allow Samsung to expand its outpost for industry collaboration and talent recruitment in a major AI hub in Canada, dedicated to research and development of core AI technologies that entail machine learning, language, vision and other multi-modal interactions.

At the opening event, there were about 100 guests in attendance including Marc Garneau, Canada’s Minister of Transportation; Philippe Tomlinson, mayor of Outremont and member of Montreal’s Development Commission; Seunghwan Cho, Executive Vice President of Samsung Research; Geunbae Lee, Head of Samsung AI Center in Seoul and Larry Heck, Head of AI Centers for Samsung Research America.

“By leveraging the power of AI in Samsung’s products and services, we must focus on creating new values, never seen nor experienced before,” said Seunghwan Cho. “To do this, seven Global AI Centers, including the Montreal AI Center, will play a pivotal role.”

During the event, Samsung presented its future vision for AI as it aims to develop a highly personalized multi-device platform that empowers people to accomplish more in their lives. By providing multiple touchpoints where a user can interact with AI, not only through voice, Samsung’s multi-modal interaction platform (voice, vision, screen, touch) will make experiences more relevant and personal in the future.

“One key element that will move AI to being more widely adopted is multi-device systems—i.e., a variety of AI-enabled devices that communicate seamlessly with each other,” said Dr. Heck. “Samsung is uniquely positioned to be a leader in this regard. It’s not just how each device uses AI, it’s how they use it together.”

Key contributing factors for the location include the availability of key AI talent, including leading AI researchers at McGill University and the University of Montreal who have had longstanding relationships with Samsung. Greater Montreal has emerged as a powerhouse in AI research, housing 250 researchers and 9000 university students in related programs. Establishing an AI Center in Greater Montreal also enables Samsung to better collaborate with regional start-ups and expand the current ecosystem.

“This new research Center will help make artificial intelligence more accessible right here in Montreal. It will open the door for collaboration and grow opportunities for our universities and our businesses,” said Minister Garneau.

The mayor of Montreal also celebrated the news. “Montreal is a vibrant and creative city with well-known academic institutions and some of the highest standards in quality of life allowing us to attract some of the best talents around the world. Samsung’s AI Center reinforces Montreal’s unique position and reputation as a global leader in the field of artificial intelligence. Their work with faculties, students, and the broader academic community highlights the city’s international reach and the strengths of a business ecosystem that is firmly focused on collaboration, research and innovation,” said Valérie Plante, Mayor of Montreal International.

Dr. Gregory Dudek, Head of the Montreal AI Center

Dr. Gregory Dudek, of the McGill University School of Computer Science and an expert in a wide range of AI technologies from machine learning to human-robot interactions, will lead the Montreal AI Center. “We are excited to open a new Samsung AI Center in Montreal, which will conduct research in machine learning and robotics enabled multi-modal interactions,” said Dr. Dudek. “As the second AI research facility in Canada, we plan to research in tandem with the Toronto AI Center, established in May, and leverage the tremendous AI talent in Montreal. We are also looking forward to collaborating with the top universities and academic institutions in the region.”

Earlier this year, Samsung has announced plans to expand its advanced AI research capabilities to 1,000 by 2020 when all Samsung products and services are expected to be AI enabled. The global AI Centers, which are also located in Korea, Russia, and the U.K, will contribute to Samsung’s AI research with their unique regional strengths, and Samsung plans to expand its AI research centers to other technology and talent-rich areas.

Just ask the distinguished speakers at the “Samsung AI Forum 2018”. Held in Seoul from September 12^th to 13^th, the second edition of Samsung Electronics’ artificial intelligence (AI) forum featured accomplished AI experts, who discussed how groundbreaking advancements are not only helping to create technology that will make our lives more comfortable, convenient and efficient. They’re also teaching us more about how our own minds work.

Unsupervised Learning Takes Center Stage

Attendees of the Samsung AI Forum 2018 are listening intently to the opening address of Kinam Kim, Samsung Electronics’ President and CEO

The forum began with a presentation from the founding director of the New York University Center for Data Science, and one of the world’s leading minds in the field of deep learning, Yann LeCun.

LeCun’s speech set the stage for the exciting discussions on unsupervised learning that would follow over the course of the two-day event. LeCun explained why he and many of his peers believe that unsupervised learning, also known as self-supervised learning, represents the future of AI. He also delved into unsupervised learning algorithms’ potential applications (and limitations), and explained how they differ from supervised and reinforcement learning algorithms.

As LeCun explained, supervised learning algorithms learn utilizing labeled datasets and answer keys that allow them to evaluate their accuracy. This essentially means that each example in the training dataset includes the answer that the algorithm should produce. With reinforcement learning, an algorithm is trained using a reward system that offers feedback when it performs an optimal action for a given situation. It relies on this feedback, rather than labeled datasets, to make the choice that offers the greatest reward.

With unsupervised learning, the algorithm is tasked with making sense of an unlabeled dataset—a set of examples that doesn’t have a correct answer or desired outcome—on its own. While these algorithms can be more unpredictable than their counterparts, they can also perform more complex processing tasks.

LeCun used training self-driving cars as a key example of unsupervised learning’s potential. “A lot of people who are working on autonomous driving are hoping to use reinforcement learning to get cars to learn to drive by themselves by trial and error,” said LeCun. “The problem with this is that, because of [reinforcement learning’s inherent inefficiencies], you’d have to get a car to drive off a cliff several thousand times before it figures out how not to do that.”

LeCun explained how, unlike reinforcement learning models, which rely on trial and error, unsupervised learning models could potentially be capable of guessing what to do in a situation like this—demonstrating mental capabilities similar to what we’d call common sense.

He also discussed his experience developing artificial neural networks—specifically convolutional neural networks (ConvNets)—and demonstrated how they can be used to build not only self-driving cars but a wide variety of innovative devices, including technologies for medical signal and image analysis, bioinformatics, speech recognition, language translation, image restoration, robotics and physics.

LeCun’s presentation was followed by a lecture from another leading light in the field of deep learning: University of Montreal professor Yoshua Bengio. Professor Bengio’s lecture focused specifically on stochastic gradient descent (SGD)—an AI optimization method that’s used to minimize errors made by artificial neural networks.

As Bengio explained, “[SGD] is really the workhorse of deep learning. This is the optimization technique that is used everywhere for supervised learning, reinforcement learning and self-supervised learning. It’s been with us for many decades and it works incredibly well, but we don’t completely understand it yet.”

Bengio’s presentation allowed attendees to gain a better understanding of SGD, with specific focus on how SGD variants can affect neural network optimization and generalization. Bengio discussed how the traditional view of machine learning sees optimization and generalization as neatly separated, but that’s not actually the case. He also presented detailed research findings on the effects of SGD-based learning techniques on both aspects of network design.

(from the top, clockwise) NYU professor Yann LeCun, University of Montreal professor Yoshua Bengio, MIT Media Lab’s professor Cynthia Breazeal and Samsung Research’s Executive Vice President Sebastian Seung.

Could Unsupervised Learning Unlock the Secrets of the Brain?

Sebastian Seung, Executive Vice President of Samsung Research and Chief Research Scientist of Samsung Electronics, delivered a particularly illuminating presentation that outlined why unsupervised learning will be essential for developing AI with human-level mental capabilities.

Seung described how the convolutional neural networks that LeCun had examined in detail are in fact based on insights gained through the study of neuroscience. He also discussed how his research in both artificial and biological neural networks led him to study ways to apply AI to gain a better understanding of how our brains are wired.

Seung stressed that the model for designing unsupervised learning networks lies in the cortex of the brain, and highlighted a recent study that his team was involved in that used AI to map out all of the neurons contained in a one cubic millimeter of a mouse’s visual cortex—more than 100,000 in total.

The unsupervised learning algorithm that the researchers utilized allowed them to not only create a 3D reconstruction of the neural network’s wiring, but also made it possible to label and color in individual cells and their components. “That’s the magic of deep learning,” said Seung. “If a human had to color all that in, it would take about 100 years of work. And that’s with no coffee breaks or sleeping.”

Living with Social Robots in ‘10 to 20 Years’

The speech delivered by Cynthia Breazeal, the founder and Chief Scientist of Jibo, Inc., and the founding director of the Personal Robotics Group at MIT’s (the Massachusetts Institute of Technology’s) Media Lab, shifted focus to applying AI to develop advanced robotics.

Breazeal’s speech, entitled “Living and Flourishing with Social Robots,” discussed approaches needed to develop autonomous systems that utilize AI to enhance our quality of life. As Breazeal explained, autonomous, socially and emotionally intelligent technologies—robots with what’s known as ‘relational AI’—present a wide range of exciting benefits.

“I’m really excited to think about the next 10 to 20 years—of having these robots actually become a part of our daily lives,” said Breazeal.

The fascinating presentation highlighted helpful companion technologies in particular, and included specific examples of ways that robots could be used to assist children and older adults. Breazeal noted studies in which AI robotic companions were given to patients at a children’s hospital, as well as kindergarten-age students and senior citizens.

Videos of the studies showed how the children in the hospital drew comfort from having a peer-like companion by their side, and demonstrated how robots can be used to boost learning. As Breazeal explained, “This is about a different vision for AI. There’s so much emphasis right now on tools for professionals, and there’s not a lot of deep thinking around how AI is going to benefit everyone.” The studies, Breazeal added, “show that there’s a lot of promise with these technologies in the real world… making a real difference.”

This year’s forum also included a diverse array of speeches that offered an all-encompassing look at the state of artificial intelligence development today. These included presentations on topics covering advancements in reinforcement learning, mutual information neural estimation, socially and emotionally intelligent AI, personal assistant robots, and precision medicine via machine learning. The developments discussed at the Samsung AI Forum 2018 represent great strides toward creating an AI-connected future.

The new Center’s main research areas will be computer vision and basic algorithms for AI platform. The Center will also expand the field of AI for key areas such as robotics, intelligent driving assistance, as well as those for future projects by Samsung.

The Samsung AI Center in Russia will be led by Professor Dmitry Vetrov of the Higher School of Economics (HSE). With a Ph.D. degree in Physical and Mathematical Sciences, Vetrov is also the Head of Samsung’s laboratory at the Center for Deep Learning, and Bayesian Methods Research Group at the HSE.

Professor Dmitry Vetrov, the leader of Samsung AI Center Moscow, is delivering an introductory speech at the opening, May 29, 2018.

As the leader of the Center, Professor Vetrov will combine scientific work and administrative activities: interacting with Samsung’s divisions and third-party institutions, organizing the Center’s overall work, managing work groups, as well as controlling and participating in scientific research. Professor Viktor Lempitsky of the Skolkovo Institute of Science Technology will also join the team as the leader of the research group.

“Samsung has always been the first to introduce new products and solutions that change the way people interact. Considering Russia as one of the world’s biggest hubs in the field of technical sciences, it is only natural that we chose the country as one of the sites for our new AI Center,” said Ultack Kim, President of Samsung Electronics Headquarters in CIS countries. “A team of the world’s best scientists and IT specialists at the AI Center will help Samsung bring its robotics and AI technologies to a whole new level.”

While there currently are several joint AI laboratories at Moscow State University, the Higher School of Economics and the PDMI RAS, the new Center will establish additional joint labs with Russia’s leading universities. In addition, Samsung will conduct projects with the local universities and different regions of Russia including Kazan, Samara, Rostov-on-Don, Tomsk, and Novosibirsk. Considerations regarding cooperation with Russian start-ups to solve practical problems are also under discussion. Moving forward, these can be developed into full-fledged services related to AI and machine learning, as well as promising developments in the field of applications and components for the company’s products.

“Samsung plans to introduce AI technologies to all of its connected devices and services by 2020. AI will enhance our customer value by offering information and services under any circumstances,” commented Jin Wook Lee, the Head of Samsung R&D Institute Russia, at the opening of the new Center. “This will be a huge step for the world of technology, and will help simplify the implementation of everyday tasks,” he added.

As the popularity of Internet of things (IoT) devices grows, the area of application of AI-based solutions is expanding rapidly. According to the forecast of Samsung experts, IoT devices with built-in AI will generate enormous amounts of data in the coming years. By processing this information, the devices will be able to provide maximum personalization and full compliance with the users’ needs.

“Currently, AI is one of the most promising branches of technology. The opening of the Samsung AI Center in Russia will allow us to contribute to the development of the industry and to apply the achievements of the Russian mathematical school, which has a high level of practical training of research specialists,” said Professor Vetrov.

Earlier in May, Samsung has opened two new AI Centers in Cambridge (UK) and Toronto (Canada). More information about Samsung AI Center in Russia and global AI Centers is available at Samsung Research website.