JOHANNESBURG, South Africa – 22 September, 2020 – The global market for large screen televisions is on a surge. According to global market research firm Omdia, global shipments of TVs with displays over 75-inches are expected to increase by 17% compared to 2019 and 8K TVs are set to increase by 110% this year as well. Given the market demands, Samsung is doubling down on its efforts to give consumers large screen TVs that not only offer the best picture quality but also the best sound experiences.

At the heart of that is Samsung’s AI Quantum Processor.

The AI Quantum Processor is a high-performing chip developed using Samsung’s proprietary technologies that can process complex algorithms quickly using machine learning-based upscaling. That gives consumers the ability to stream 8K content in real-time paired with dynamic, location-based acoustics. Samsung is peeling back the secrets of the AI Quantum Processor to share on how it brings these experiences together to get better resolution, better sound and better picture integrity.

One of the leading questions consumers continue to ask about large screen TVs is how to watch ultra HD content in 8K resolution given that it isn’t widely available yet. That’s where Samsung’s AI upscaling technology comes in.

Today, the technology transforms not only 4K content into 8K, but it also works for lower-resolution SD, HD and FHD-quality content as well. The AI Quantum Processor 8K combines the advantages of existing machine learning capabilities with new deep learning technologies to analyse and deliver content at some of the most granular levels possible.

This hybrid approach helps to remove unnecessary noise and creates life-like textures and smooth edges on up-scaled 8K content that only Samsung can deliver. It also adjusts content brightness and darkness levels according to the light and living room conditions the display is set up in.

AI Quantum Sound for the Most Suitable Audio Experiences

Adding a truly effective audio experience to high-quality picture quality serves to bring about an even deeper sense of immersion to viewers. Samsung’s AI Quantum Sound, bolstered by artificial intelligence, provides a listening experience similar to that of a movie theatre right in your living room.

Samsung’s new innovative sound technology Object Tracking Sound+ follows movement on-screen to match the direction of the sound to the action. The technology recognized and analyses the recorded audio synced to the content in real-time and evenly distributes the audio tracks to the six speakers embedded in the TV. And due in part to the Active Voice Amplifier (AVA), users will never miss a word of dialogue, regardless of what is happening around the TV. The feature monitors the sound environment so that if there is noise louder than the TV’s audio, it will increase the volume of just the dialogue audio so as not to overwhelm users while ensuring no word goes unheard. Also, sound will be delivered seamlessly thanks to real-time deep learning audio separation technology, high-performance NPU (Neural Processing Unit).

AI Streaming for Minimal Data Loss

Thanks to the widespread use of the Over the Top (OTT) streaming services, more and more users are taking advantage of their large screen TVs and stream 8K content. However, existing network infrastructures cannot always guarantee image quality that meets users’ high-resolution expectations. This is why at the Samsung Developers’ Conference in 2019, Samsung introduced its AI Scalenet technology, designed to overcome these infrastructure limitations and support effective 8K streaming and low-latency livestreaming.

AI Scalenet is a CES 2020 Innovation Award-winning AI codec technology that generates resolutions four times higher than conventional technologies without any change to the network infrastructure. It minimises the data loss that may occur during the compression and playing of videos through deep learning technologies, meaning users can enjoy 8K-quality content regardless of their network infrastructure. Even during peak usage, AI Scalenet dramatically improves image quality and content speed according to the network status and reduces picture breaking and ambient noise.

These days, more content than ever can be enjoyed on our TVs, meaning that the expectation for immersive viewing experiences in the home has increased, too. Samsung plans to continue to use a range of innovative technologies, starting with artificial intelligence, to bring the best experience 8K content and audio experiences to living rooms across the world.

With its 2020 QLED 8K TVs and AI sound technologies, Samsung has raised the bar for TV audio experiences. Object Tracking Sound+ uses AI-based software to match the movement of audio with movement on-screen; Active Voice Amplifier (AVA) tracks the user’s audio environment; and Q-Symphony creates a more realistic, three-dimensional sound.

Samsung Newsroom sat down with the sound developers of Samsung Electronics’ Visual Display Business to learn more about their extensive capabilities and the journey to fostering innovation in sound.

(From left to right) Youngtae Kim (Sound Lab), Jongbae Kim (Sound Lab), Yoonjae Lee (Sound Device Lab) and Sunmin Kim (Sound Lab)

Object Tracking Sound+: Reading the Room for More Realistic Audio Experiences

Action movies with amazing sound arrangements provide the most realistic experiences when experienced at movie theatres. This is because movie theatres have multi-channel speakers with 3D sound placed on almost all the walls (including the ceiling), as well as around the screen. Compared with a two-channel sound that features speakers only on the left- and right-hand sides, the multi-channel speakers in a movie theatre deliver a more refined sense of realism. So how can this realism be recreated in the home? Samsung’s sound developers came up with Object Tracking Sound+ technology, in which sound follows movement onscreen through six speakers built into the TV.

Thanks to this technology, a video’s audio follows the action on-screen in real-time. When a car moves from the left to the right-hand side of the screen, so will the sound it makes; and when a heavy object drops from the top to the bottom of the screen, so will the audio.

When developing the 2020 QLED 8K TVs, Samsung’s TV developers increased the number of QLED TV speakers from two to six to realise a sound that can mimic the action. “By placing two speakers on each side of the screen, as well as on the top and bottom, we enabled the dissemination of sound in all directions from a total of six speakers,” explained Jongbae Kim. “The distance between the two main speakers has been widened as much as possible, and the additional speakers have been installed in order to maximise sound across all axes to be as three-dimensional as possible. For example, we placed speakers on the upper side of the screen to enable the movement of sound in a vertical direction for a more immersive sound experience.” Additionally, Kim highlighted how, despite the complex nature of a TV structure that includes six embedded speakers, the team managed to keep the design of the TV slim and minimal.

In order to ensure audio will follow on-screen movements accurately, it is important to understand the original intentions of content creators. “The role of a sound engineer is to increase the consistency between the action onscreen and its accompanying audio track when mixing. The location information of sound in a piece of video content, including sound panning information, is subsequently audio-signalled into the audio channel by the sound engineer – something we must be able to track in order to reproduce the location and movement of a content’s audio accurately,” noted Jongbae Kim. “Our Object Tracking Sound+ technology analyses the location information contained in these audio signals as originally placed during mixing. This means the TV can then effectively distribute the sound amongst its six speakers by distinguishing between sound orientations and whether or not the audio source is on-screen, off-screen, close-up or distant.”

AVA: Intelligent Noise Detection to Enhance Voice Clarity

When that crucial scene in the show you are watching is overwhelmed by mixer sound or an important breaking news report is obscured by loud thunder, the act of reaching for the remote to adjust the TV’s volume can come too late. This is why the team developed their AVA technology, which recognises exterior noises and increases the volume of voices in content accordingly if surrounding conditions become too loud.

The way it works is intelligent. The TV’s sound sensor, attached to the bottom middle of the TV, takes in audio from the content onscreen as well as its surrounding environs. AI technology then compares the volume levels of the two types of sound, and if external sounds are found to be louder than that of the TV’s content, it will selectively raise the TV’s volume. “The system does not have one set definition of noise,” explained Sunmin Kim. “It considers any elements that disturb the enjoyment of content as noise. When exterior sounds persist above a certain decibel level, that is when the system registers it as ‘noise’.”

However, AVA technology does not just raise the volume of the TV when it recognises a louder environment, as this would only contribute to a boisterous room experience. “The system harnesses AI to keep sound effect and background audio levels consistent and to only raise the volume of voice audio,” highlighted Sunmin Kim. “Our research showed us that the majority of content is dialogue-heavy, so we believe that enhancing the delivery of dialogue would be most beneficial to aid comprehension.”

Q-Symphony: Creating a Multi-Dimensional Soundscape

One of the key elements to achieving a realistic sound during content playback on a TV is three-dimensionality, which encompasses both horizontal and vertical audio characteristics. Until recently, these perspectives had been being developed separately by the TV and Soundbar teams. However, with the inclusion of upper side speakers on Samsung’s 2020 QLED 8K TVs, the team developed an all-inclusive solution that utilises the capabilities of both the TV and the soundbar in perfect harmony. Q-Symphony is a feature that plays audio using both the TV speakers and the soundbar at the same time, and as an industry-first achievement, the Q-Symphony technology was even recognised with a Best of Innovation Award at CES 2020.

“The core of Q-symphony, which manages sound playback harmoniously using speakers with different characteristics, is the technology that follows the sound playback rules which are determined in advance and exchanges the necessary information between the TV and the soundbar when connected. This approach allows for a superior reproduced sound experience,” explained Yoonjae Lee. “A key element of the technology is a special algorithm that we created which divides and harmonises sounds seamlessly between the TV’s speakers and soundbar.”

During development, the challenge arose regarding the quality of dialogue reproduction. When both the soundbar and the TV speakers played dialogue simultaneously, the sound quality was noticeably diminished. However, the sound development team were able to resolve this issue by separating the main soundtrack, including dialogue, from the entire signal and assigning the different tracks to the TV speakers and soundbar respectively. “In the 2020 QLED 8K TV range, the voice signal is extracted and removed from the sound being reproduced by the TV’s embedded speakers, which are then assigned to play ambient sound signals such as sound effects,” explained Lee. “The soundbar then reproduces the main sound involving any dialogue. With this technology, Q-Symphony harnesses the advantages of both TV speakers and the soundbar in order to deliver the best, and most harmonious, sound experience to users.”

Working Together to Develop the Perfect Sound Experience

The sound development team agreed that realising the addition of speakers, the new placement of speakers and the AI harmonisation with the soundbar on the 2020 QLED 8K TV range was possible because of close coordination with a variety of other teams. “When developing new TVs, all areas need to be in sync with their innovations,” noted Youngtae Kim. “We came together in suggesting various solutions to overcome each technological hurdle with an open mind.”

Youngtae Kim (left) and Sunmin Kim introduce the innovative sound technologies of the 2020 QLED 8K TV range

The sound development team has always been and always will be dedicated to developing the best possible audio experiences for users. As well as working with Samsung’s Audio Lab based in the U.S. for the future audio technology, the team also works with Samsung Research’s R&D Centers, universities and start-up experts around the world. “We want to bring about sound experiences that are as natural and as real as possible,” explained Youngtae Kim. “To achieve this, we will continue to work hard to understand the end-to-end process of the sound and realise sounds in our TVs.”

Samsung’s sound innovation is  paving the path forward. “In the future, we will bolster the use of AI so that users do not even need to use a remote control to find the perfect audio balance when enjoying their favourite content,” said Nivash Ramsern, Head: Visual Display at Samsung South Africa. “As time goes by, TV installation environments, lifestyles and age groups will be diversified. We want users to enjoy the sound of their content as intended, regardless of content type or listening environment.”

Allan Devantier, Audio Lab Head, Samsung Research America

Devantier started working for Samsung in October of 2013. By January of 2014, Samsung had acquired lab space and recruited four staff members, and by October of that year the staff had grown to number 10 engineers and research scientists and the Audio Lab had been made fully functional. From there, the lab was ready to begin working towards its mandate of ‘taking Samsung’s audio technology to the top of the industry.’

Devantier says that “The first product we worked on was the Radiant 360 loudspeaker – it was the first mass-market speaker to feature 360-degree sound. We were the industry leader, and now 360-degree speakers have grown very popular in the market.”

Evolutionary Change

Cut to the present, when Samsung’s LA Audio Lab is staffed by more than 20 people with over 300 collective years of experience in the audio field between them. Eleven members of the team hold Master’s or PhD degrees in audio technology, while eight are both engineers and musicians who continue to play in bands. The lab also collaborates extensively with Samsung’s audio engineering teams in Korea.

“Our focus starts at the ear of the listener,” Devantier explains, “then expands out to the listening environment – the transducers (woofers and tweeters), amplifiers, and digital signal processors (DSPs).” And this emphasis has led to a steady stream of important contributions coming out of the lab. Whereas, in 2014, industry publications awarded Samsung’s Soundbar solutions a rating of three-and-a-half stars out of five, by 2019 the average score for Samsung Soundbars had been lifted to four-and-a-half stars. What’s more, in 2019 the HW-Q90R premium Soundbar model received a perfect five-star rating from several industry publications. In addition to improvements in the pure audio sphere, the Samsung Audio Lab has contributed to remarkable advancements in TV sound quality as well.

The Samsung Audio Lab has proven its ability to discern changes in the audio market and act upon them swiftly, as is demonstrated by the nature of its recent innovations. The Q-Symphony solution announced at this year’s CES enables a TV equipped with a Soundbar to provide surround sound that closely reproduces the spatial feeling of original sound fields. Devantier explains that Q-Symphony “uses the top-firing speakers in the TV to improve envelopment and immersion.”

Another example of this cutting-edge innovation is Samsung’s OTS (Object Tracking Sound) technology. To improve upon Soundbar technology that allows sound to be delivered in fixed directions, the lab developed TVs that incorporate speakers not just on the bottom and sides of the screen, but in the top left and right sections as well. This allows OTS technology to make use of deep learning that can recognise what kind of content is being displayed on the screen and deliver multichannel sound accordingly. Thus, users can enjoy actual, physical surround sound that is not only immersive but is specifically tailored to whatever they are watching at the time.

What’s more, in 2020 Samsung’s Q-Symphony and OTS technologies are coming together to truly maximise the potential of both solutions. While OTS delivers multichannel surround sound through the TV speakers, Q-Symphony makes the TV-speaker-Soundbar combination possible, enabling the TV’s speakers to provide rich, lively surround sound while the Soundbar delivers powerful, absorbing bass.

Taking Audio into the Future

The inception of AI is set to reshape the audio industry, and Samsung is working to anticipate the changes it will bring about. Asked to outline what he sees as AI’s role in the audio industry, Devantier says, “I think we can use AI to help our loudspeakers play louder, with more perceived bass and less perceived distortion. AI will also help make old recordings sound better and allow our loudspeakers to adapt to different listening environments.”

Another way in which AI is empowering future solutions is by enabling features like Active Voice Amplifier (AVA). Samsung’s AVA feature makes it possible for the TV to recognise the audiovisual space environment and provide sound that is optimised not only for the content being displayed on the screen but also for the viewing space that the consumer is in. In 2020, sensors embedded in QLED solutions will make optimised viewing possible, even in rooms with high levels of noise. For instance, if a user is watching something and a member of their family switches a blender on in the adjoining kitchen, the system will adjust to ensure that the dialogue remains audible to the viewer in real-time despite the unexpected peripheral noise.

As far as how his lab will contribute to delivering better audio going forward, Devantier reports that it will continue to meet the needs of customers who are looking for “high quality and portable sound” while improving and expanding its OTS technology. “Maybe one day we will develop a loudspeaker that sounds great, plays very loud and is nearly invisible or exceptionally small and portable,” Devantier concludes, “Until that time, we will continue to produce solutions that make incremental improvements to sound quality.

Samsung Audio Lab