COVID-19 has sped up digital transformation by several years, and advances in AI technology have sped right along with it, leading to a significant increase in demand for AI. Large-scale adoption of AI is already taking place in key industries, from the automotive sector and finance to healthcare and education, as seen through innovations like self-driving cars and chatbots. At the same time, the range of applications for AI is expanding fast, driving impressive advancements in areas like image processing, speech recognition, and natural language processing.

A New Memory Solution for the AI Era

In recent years, as data throughput has increased rapidly, it’s stressed the limits of existing computing systems. AI data throughput has been rising tenfold each year,¹ and current computing systems don’t offer memory capacities large enough to handle the sharp increase in data volumes.

Currently, a central processing unit (CPU) can hold up to 16 DRAMs (a maximum of 8 terabytes (TB)) — a number far smaller than what’s needed to handle the massive stores of data used in AI and machine learning. The need for a memory platform that supports fast interfaces and easy scalability is becoming all the more clear as the age of AI draws ever nearer. Recently, a new DRAM module based on Compute Express Link (CXL) has emerged as a promising memory solution for the AI era. So too have processing-in-memory (PIM) and computing storage equipped with a memory-based AI processor.

CXL: An Interface Pushing the Boundaries of Memory Capacity and Server Flexibility

What makes CXL a next-generation memory platform and such a promising solution to current computing limitations? In short: scalability.

CXL is a new interface that’s designed to enhance the efficiency of a computing system’s memory, CPU and graphics processing unit (GPU). In conventional platforms, devices like memory and storage have their own interfaces that link them to the CPU. But going through all these different interfaces to communicate with one another creates latency, slowing down operations. And with the massive growth in data being used for AI and machine learning, latency issues have only gotten worse.

CXL is part of a next-generation interface that will be applied to PCIe 5.0. By integrating multiple existing interfaces into one, directly connecting devices and enabling them to share memory, CXL addresses those limitations and creates new data pathways that are faster and more efficient. This next-generation memory solution is the reason why CXL has been receiving so much attention.

In line with this trend, in May of 2021, Samsung Electronics introduced the CXL Memory Expander, a first-of-its-kind CXL-based software development solution, and began promoting CXL memory solutions. The main advantages of CXL are as follows:

• Unrivaled Memory Expansion

Similar to a solid state drive (SSD), which is an external storage device, the CXL Memory Expander enables DRAM capacity to be expanded when installed in the location where the SSD is inserted. In other words, it enables an IT system’s DRAM capacity to be expanded simply by improving the interface and without having to modify the existing server structure or change it altogether.

• Streamlined Data Handling

A key benefit of the Memory Expander is its efficient data processing. By expanding higher bandwidth, it enables different devices to share memory and leverage their resources more effectively. They can then use the accelerator’s memory as if it were main memory by sharing common memory areas. Devices without their own internal memory can also take advantage of that main memory and use it as their own.

• Accelerated Computing Speed

Minimizing latency issues (or delays) caused by increases in data throughput is a key function of the CXL Memory Expander. The Memory Expander leverages both the accelerator and the CPU to improve system computing speeds, supporting much smoother and more rapid data processing.

As it stands, many in the industry are still unfamiliar with the concept of the CXL interface. Although the technology is still in its early stages, its potential to drive more efficient data processing is pushing it to be viewed as a driver of the Fourth Industrial Revolution.

In preparation for the rapidly approaching AI era, Samsung Electronics will help expand the CXL ecosystem by introducing everything from CXL memory hardware to a software solution called the Scalable Memory Development Kit (SMDK), and will lead the market with next-generation memory solutions that are capable of accommodating the fast-evolving landscape of data processing.

¹ Source: OpenAI (2019)

The CXL interconnect is an open, industry-backed standard that enables different types of devices such as accelerators, memory expanders and smart I/O devices to work more efficiently when processing high-performance computational workloads.

“In order for data center and enterprise systems to smoothly run next-generation memory solutions like CXL, development of corresponding software is a necessity,” said Cheolmin Park, Vice President of the Memory Product Planning Team at Samsung Electronics. “Today, Samsung is reinforcing its commitment toward delivering a total memory solution that encompasses hardware and software, so that IT OEMs can incorporate new technologies into their systems much more effectively.”

CXL Memory Adoption Made Easier Without the Need for Application Modifications

Samsung’s SMDK enables the main memory and the CXL memory expander to work together seamlessly in heterogeneous memory systems. The extensive software kit consists of libraries — sets of pre-built, reusable codes — and application programming interfaces (APIs) — the connections to access these software codes. Using the SMDK, system developers can easily incorporate CXL memory into advanced IT systems without having to modify existing application environments, or alternatively use it to optimize application software settings to suit special system needs.

Industry’s First Software-Defined Memory Management

The new SMDK also supports memory virtualization, allowing system designers to efficiently manage an expanded memory pool in shared memory architectures. Leveraging a proprietary Intelligent Tiering Engine, the SMDK can identify and configure the memory type, capacity and bandwidth that are most appropriate for each use case.

Samsung’s SMDK is now available on a limited basis for initial testing and optimization and will be open-sourced within the first half of next year. Samsung will continue to enhance its open-source SMDK as it closely collaborates with industry leaders in expanding adoption of the CXL memory platform for broad use in AI, edge and cloud applications.

In this relay series, Samsung Newsroom is introducing tech experts from Samsung’s R&D centers around the globe to hear more about the work they do and the ways in which it is directly improving the lives of consumers.

The third expert in the series to be introduced is Bin Dai, Staff Engineer at the Artificial Intelligence (AI) Lab in Samsung R&D Institute China – Beijing (SRC-B). Dai joined SRC-B in 2020 to join his colleagues in working on network compression and on-device model design and research. Read on to learn more about the groundbreaking technologies Dai and his team are developing at SRC-B.

Q: AI-based technologies, including NLP (Natural Language Processing) and acoustic intelligence, are cutting-edge research areas that are constantly breaking new ground. But what role does the core research offering provided by machine learning play as a background for these innovations?

Machine learning plays a crucial role into bringing all kinds of technologies directly to users. Computer vision and speech recognition are two of the most successful areas currently utilizing AI. However, existing AI algorithms require huge computation resources, making it difficult to deploy state-of-the-art algorithms on mobile devices. In order to fix this issue, our AI Lab is working on producing tiny models with powerful performance from both a theoretical and a practical perspective. In this way, our core research is set to innovate all kinds of AI-based technologies.

Q: Can you please briefly introduce the Beijing Research Institute, and the kind of work that goes on there?

SRC-B is one of Samsung’s Electronics’ advanced R&D centers and was established in 2000, the first Samsung R&D center to be established in China. SRC-B focuses on groundbreaking technologies and specializes in artificial intelligence (AI) and next-generation telecommunications, from machine learning, computer vision, language processing and voice intelligence through to 3GPP standardization and more. We also promote tight industrial-academic partnerships. In April 2019, the AI Lab was established to focus on fundamental research into machine learning, and we are continuously looking for ways to apply our research results to Samsung products.

Q: Following the success of your major research thesis and other accomplishments, what are you working on at the moment?

SRC-B is currently aiming to find the best possible way to enhance the accuracy of an AI algorithm while reducing the computation complexity and resources used to do so. In order to achieve these goals, we are currently working on two research topics that enable accurate predictions with less data required: equivariant networks, part of the broader topic of geometric deep learning, and dynamic inference. There are many kinds of symmetries in computer vision datasets which are able to provide accurate depth measurement like human eyes can, such as image and LiDAR point clouds. With an equivariant network, these symmetries are taken into consideration when designing the network. It is thus able to achieve better performance with fewer resources since we have specifically considered the intrinsic structure of the dataset.

Dynamic inference is also a very interesting research direction. Unlike conventional methods which harness a fixed architecture for all data samples, dynamic inference can adaptively decide how many resources to use for each data sample. Accordingly, it will use fewer computational resources for simple samples and more resources for difficult ones. By doing so, the average computation resource used can be significantly reduced.

Q: Fundamental research into AI has been empowering all kinds of user-forward application fields, from computer vision to speech recognition. Could you explain a bit more about why this is, and the direction of research you and the AI Lab have been taking in order to optimize mobile experiences?

In this era of the internet, data is flooding everywhere around us. Where there is data, there is knowledge. AI algorithms are the very best tool for uncovering the knowledge hidden behind the data and make use of this knowledge to make all of our lives better.

We have developed a network compression algorithm based on the information bottleneck theory – which posits that extraneous details can be removed from noisy input data as if squeezed through a bottleneck – which has been applied to multiple tasks including video recognition, image segmentation and machine translation. We also actively collaborate with other labs in SRC-B in order to develop more powerful AI algorithms, including the Neural Architecture Search (NAS) and Once-For-All (OFA) solutions.

Q: What do you see as the main user benefits from incorporating all base mobile technologies with machine learning-based AI technologies?

Machine learning-based AI technologies can dramatically improve users’ lives in three key ways. Firstly, there are many convenient functions that simply cannot work without AI technologies. For example, the automatic question and answering system on mobile devices has to be powered by AI algorithms. Other more traditional methods are only able to handle very limited, pre-defined questions.

Secondly, AI techniques can significantly improve the performance of many applications compared to their performance when harnessing conventional technologies only. For example, after applying deep neural networks to a camera’s neural image signal processing (ISP) function, the quality of photos taken on that camera becomes significantly better.

Thirdly, AI technologies are capable of providing services that users previously didn’t even know they needed. For example, AI is capable of developing a user-specific software based on that user’s specific preferences, meaning that the user’s device experience can continuously be improved.

▲ Researchers at Samsung R&D Institute China – Beijing

Q: How does the work you do synergize with the work undertaken by the rest of Samsung R&D Institute China – Beijing, or perhaps even other R&D Institutes around the world? How does it come together to make users’ lives more convenient?

We are constantly collaborating with the other teams within SRC-B. We have been collaborating recently with our Visual Computing team in order to apply our information bottleneck-based compression algorithm to video recognition tasks and human segmentation tasks, resulting in the significant reduction of model sizes without any performance drop. In 2021, we participated in the Conference on Computer Vision and Pattern Recognition (CVPR)’s Neural Architecture Search (NAS) competition as one team with this solution, and won 1^st place.

We have also been working with our Language Intelligence team to compress their machine translation model, which facilitates the commercialization of their application.

We also believe that we can produce better research and application results by further communication, discussion and collaboration with AI centers globally.

Q: What do you see as being the main trends within your industry right now? How have you been incorporating them into the research you do at Samsung R&D Institute China – Beijing?

There are a lot of trending topics within our field at this time. Efficient network architecture design, self-supervised learning and graph neural networks are just a few examples.

Our focus is on network compression and tiny model design, which is ultimately useful for applications on mobile devices. There are a lot of mobile devices, such as smartphones, that possess very limited computational resources, meaning that it is impossible to deploy the huge models designed for services to these devices. Therefore, my team is focused on designing models suitable for these devices.

There are different ways to achieve these kinds of light yet powerful models. For instance, network pruning, quantization, knowledge distillation, neural network architecture search and dynamic inference are just a few industry areas that we are focusing on right now to achieve this.

Q: What has been the achievement at Samsung R&D Institute China – Beijing that you are most proud of so far?

Developed together in collaboration with our Communication Research team, we engineered AI algorithms for wireless communication. This solution achieved first place at the Wireless Communication AI Competition (WAIC) this year, which is the official competition for 5G+AI in China with over 600 teams enter from around the world and is held by the China Academy of Information and Communication Technology (CAICT). I am proud of this achievement and feel that it validates my belief that 5G combined with AI is a research direction with great potential.

An interview with Evgeny Pavlov, a system software expert from Samsung R&D Institute Russia (SRR) can be found in the following episode.

The event, held today in Santa Clara, California, features top Samsung executives and industry experts reviewing progress on semiconductor technologies and foundry platform solutions that enable developments in artificial intelligence (AI), machine learning, 5G networking, automotive, the Internet of Things (IoT), advanced data centers and many other domains.

“We stand at the verge of the Fourth Industrial Revolution, a new era of high-performance computing and connectivity that will advance the daily lives of everyone on the planet,” said Dr. ES Jung, President and head of Foundry Business at Samsung Electronics.

“Samsung Electronics fully understands that achieving powerful and reliable silicon solutions requires not only the most advanced manufacturing and packaging processes as well as design solutions, but also collaborative foundry-customer relationships grounded on trust and shared vision. This year’s Foundry Forum is filled with compelling evidence of our commitment to progress in all those areas, and we’re honored to host and converse with our industry’s best and brightest,” Dr. Jung added.

Highlights from the U.S. Foundry Forum include:

The New 3nm GAE PDK Version 0.1 is Ready

Samsung’s 3nm Gate-All-Around (GAA) process, 3GAE, development is on track. The company noted today that its Process Design Kit (PDK) version 0.1 for 3GAE has been released in April to help customers get an early start on the design work and enable improved design competitiveness along with reduced turnaround time (TAT).

Compared to 7nm technology, Samsung’s 3GAE process is designed to provide up to a 45 percent reduction in chip area with 50 percent lower power consumption or 35 percent higher performance. The GAA-based process node is expected to be widely adopted in next-generation applications, such as mobile, network, automotive, Artificial Intelligence (AI) and IoT.

Conventional GAA based on nanowire requires a larger number of stacks due to its small effective channel width. On the other hand, Samsung’s patented version of GAA, MBCFET (Multi-Bridge-Channel FET), uses a nanosheet architecture, enabling greater current per stack.

While FinFET structures must modulate the number of fins in a discrete way, MBCFET provides greater design flexibility by controlling the nanosheet width. In addition, MBCFET’s compatibility with FinFET processes means the two can share the same manufacturing technology and equipment, which accelerates process development and production ramp-up.

Samsung recently taped out the 3GAE test vehicle design and will focus on improving its performance and power efficiency going forward.

For more information, please refer to the Newsroom links for GAA infographic and video clip.

The Launching of a New SAFE–Cloud Program

As part of its ongoing efforts to support and enhance customers’ entire design workflow, Samsung Electronics launched the Samsung Advanced Foundry Ecosystem Cloud (SAFE-Cloud) program. It will provide customers with a more flexible design environment through collaboration with major public cloud service providers, such as Amazon Web Services (AWS) and Microsoft Azure, as well as leading Electronic Design Automation (EDA) companies, including Cadence and Synopsys.

To date, most foundry customers have built and managed design infrastructure on their own servers. The SAFE-Cloud program reduces this burden and supports easier, faster and more efficient design efforts by providing an excellent turnkey design environment with extensive process information (PDK, design methodologies), EDA tools, design assets (IP, library) and design services.

Customers can be assured of as much server and storage space as they need, as well as a safe environment optimized for chip design, due to Samsung Electronics’ verification of SAFE-Cloud’s security, applicability and expandability.

Utilizing the SAFE-Cloud platform, Samsung was able to accelerate the development of its 7nm and 5nm cell libraries in collaboration with Synopsys. In addition, Samsung, Gaonchips – a fabless design company in Korea – and Cadence have successfully completed design verification based on the platform.

“Making up-front investments in high-performance computing (HPC) servers and systems can be a challenge for a company like us,” said Kyu Dong Jung, CEO of Gaonchips. “SAFE-Cloud offers us a very flexible design environment without requiring investment in additional infrastructure, as well as reduced design TAT. I expect this program to provide more tangible business and technical benefits to us and the entire fabless industry.”

Process Technology Roadmap and Advanced Packaging Updates  

Samsung’s roadmap includes four FinFET-based processes from 7nm down to 4nm that leverage extreme ultraviolet (EUV) technology as well as 3nm GAA, or MBCFET.

In the second half of this year, Samsung is scheduled to start the mass production of 6nm process devices and complete the development of 4nm process.

The product design of Samsung’s 5nm FinFET process, which was developed in April, is expected to be completed in the second half of this year and go under mass production in the first half of 2020.

Extensions of the company’s FD-SOI (FDS) process and eMRAM together with an expanded set of state-of-the-art package solutions were also unveiled at this year’s Foundry Forum. Development of the successor to the 28FDS process, 18FDS, and eMRAM with 1Gb capacity will be finished this year.

Samsung Research, the advanced R&D hub of Samsung Electronics’ SET (end-products) business, has ranked first in two of the world’s top global artificial intelligence (AI) machine reading comprehension competitions.

Samsung Research recently placed first in the MAchine Reading COmprehension (MS MARCO) Competition held by Microsoft (MS), as well as showing the best performance in TriviaQA^* hosted by the University of Washington, proving the excellence of its AI algorithm.

With intense competition in developing AI technologies globally, machine reading comprehension competitions such as MS MARCO are booming around the world. MS MARCO and TriviaQA are among the actively researched and used machine reading comprehension competitions along with SQuAD of Stanford University and NarrativeQA of DeepMind. Distinguished universities around the world and global AI firms including Samsung are competing in these challenges.

Machine reading comprehension is where an AI algorithm is tasked with analyzing data and finding an optimum answer to a query on its own accord. For MS MARCO and TriviaQA, AI algorithms are tested in their capabilities of processing natural language in human Q&As and also providing written text in various types of documents such as news articles and blog posts.

For example in MS MARCO, ten web documents are presented for a certain query to let an AI algorithm create an optimum answer. Queries are randomly selected from a million queries from Bing (MS search engine) users. Answers are evaluated statistically by estimating how close they are with human answers. This is a test designed to apply an AI algorithm to solve real-world problems.

Samsung Research took part in the competitions with ConZNet, an AI algorithm developed by the company’s AI Center. ConZNet features skillful capabilities through adopting the Reinforcement Learning^** technique, which advances machine intelligence by giving reasonable feedback for outcomes, similar to a stick-and-carrot strategy in a learning process.

With the recent acceleration in global competition to develop AI technologies, contests are widespread in areas of computer vision (technologies to analyze characters and images) and visual Q&A to solve problems using recognized images of characters as well as machine reading comprehension. The Beijing branch of Samsung Research won the International Conference on Document Analysis and Recognition (ICDAR) hosted by the International Association of Pattern Recognition (IAPR) in March, putting them in a top-tier group for global computer vision tests. The ICDAR is the most influential competition in Optical Character Recognition (OCR) technologies.

“We are developing an AI algorithm to provide answers to user queries in a simpler and more convenient manner, for real life purposes,” said Jihie Kim, Head of Language Understanding Lab at Samsung Research. “Active discussion is underway in Samsung to adopt the ConZNet AI algorithm for products, services, customer response and technological development.”

^* Competitions such as MS MARCO and TriviaQA allow contestants to participate at any time, and rankings are altered according to real-time test results.

^** The Reinforcement Learning is the most advanced Machine Learning AI algorithm, and cutting-edge AI technologies including AlphaGo are upgrading machine intelligence by applying this technique.

The new AI technology achieves close to 8K resolution and enhanced sound quality by aligning with the unique characteristics of the specific content, a step up from typical upscaling technology used to improve image quality. So how exactly does it work? Let’s take a closer look.

Enjoy any content in 8K through Machine Learning

The world’s first 8K AI Technology that realizes a definition nearly the equivalent of 8K is based on Machine Learning. Computers or smartphones run according to directive values that humans enter. In contrast, Machine Learning refers to the way AI learns certain patterns and gives optimized answers based on various examples.

Samsung’s Machine Learning Super Resolution (MLSR) utilizes AI technology to compare low and high-quality versions of the same content to learn the technological differences between the two and form a vast database. It analyzes millions of pieces of video content and finds a correlation. Based on its analysis, it can select the optimum filters that support brightness, the level of blackness, spread and other errors from all inputs, and transform low-definition content to close to 8K high definition.

The input content is recognized as ‘real-time’ based on a frame and is enhanced by scenes, which makes it possible to upgrade image and sound quality immediately, regardless of whether the video source is live streaming or OTT (Over The Top).

8K picture quality through AI, what’s the difference?

Other upscaling techniques require human input to compare low-resolution and high-resolution scenes and find ways to reconstruct them. However, Samsung’s AI Technology enables it to self-study millions of images on its own using MLSR, allowing much-improved accuracy compared to conventional technologies.

There are three elements to improving picture quality on displays. First is ‘Detail Creation’ that sharpens the detail of expression and improves the texture to areas with low definition that have become blurry after compressing the file. Second, ‘Edge Restoration’ defines the edges of text, people or objects in a video, moving pixels on the borders to thin them down to increase legibility and visibility. For example, if text context is spreaded along the edges, the video will be adjusted around the text for clarity. In a video that shows the moon, Edge Restoration improves details of the moon’s shadow and enhances the darkness of the background for a clear distinction. Lastly, ‘Noise Reduction’ gets rid of static noises generated during high compression or recompression of files. In order to transmit an image, it is necessary to compress the image. In this process, various ‘noises’ such as a jiggling point or a squared dot can be effectively removed according to the image characteristics.

AI delivers immersive sound effects

When watching dramas or movies, realistic, immersive sounds are as important as picture quality. Samsung’s AI technology not only transforms low-definition into high-definition, but also optimizes the sound quality of content.

Conventional TVs provide multiple view settings such as movie mode and sports mode according to the genre of content. With Samsung’s new AI technology, content can be automatically analyzed by characterizing scenes to provide optimum sounds.

For example, let’s say you were watching a movie that includes musical performances. AI technology can highlight the music in a way that allows you to experience the sound as the actual characters would. When the crowd applauds after the performance is over, you would hear the clapping the same way as if you were in the crowd in the movie. When characters are speaking, AI adjusts the sound to make sure the lines are communicated clearly.

Imagine you were watching a relay broadcast of the 2018 PyeongChang Winter Olympics that started last week. AI will enhance the voice of the announcer so that you don’t miss who’s up next. When a game has started, AI will increase the background sound to deliver the liveliness of the actual game. With this tailored sound adjustment scene by scene, audiences can enjoy the best sound quality for any genre of content.

Samsung developers plan to continue to improve sound quality according to the preferences of individual viewers so that each viewer can enjoy the best TV viewing experience, right for them. Because volume patterns differ for every user, and the viewing environment can change according to the time of day and other factors, the sound will be accordingly adjusted and optimized to provide the most enjoyable experience to each individual viewer.

Why does 8K AI technology matter?

As customer needs for high-definition TVs and content increases, some terrestrial broadcasting stations have committed to working towards UHD delivery, and various IPTV and cable channels have initiated 4K UHD (3,840 × 2,160) services. However, even as the TV industry begins to launch 8K (7,680 × 4,320) TVs, the reality is that 4K content is, as of yet, still not fully utilized in homes.

In this context, Samsung has proposed a new direction for TV technology by combining 8K UHD display technology and premiere content through AI. Samsung has developed an AI algorithm that automatically enhances picture quality to solve the problem of limited high-quality content. As example, AI technology plays a key role in the 85-inch 8K QLED TV technology that Samsung introduced at CES 2018.

Samsung will begin the process of applying AI technology to its 8K QLED TVs from the second half of this year, and viewers will soon be able to enjoy UHD quality video that is nearly 8K in resolution and delivers optimized sound for any type of content. A genuine 8K era is now on the horizon, and Samsung will continue to lead the way.