‘All In’ on AI, Part 2: Driving the Evolution of 8K Picture Quality and Advanced Sound on TV through AI

What do you consider important when watching the Olympic Games on TV? Vivid picture and sound quality on your TV would be able to provide you a lifelike experience just as if you were onsite at the Games. The artificial intelligence (AI) technology which Samsung Electronics recently unveiled at CES 2018 promises to deliver this kind of experience, with picture quality nearly the equivalent of 8K (7,680 × 4,320) resolution, as well as optimized sound, for real-time and other video content.

The new AI technology achieves close to 8K resolution and enhanced sound quality by aligning with the unique characteristics of the specific content, a step up from typical upscaling technology used to improve image quality. So how exactly does it work? Let’s take a closer look.

Enjoy any content in 8K through Machine Learning

The world’s first 8K AI Technology that realizes a definition nearly the equivalent of 8K is based on Machine Learning. Computers or smartphones run according to directive values that humans enter. In contrast, Machine Learning refers to the way AI learns certain patterns and gives optimized answers based on various examples.

Samsung’s Machine Learning Super Resolution (MLSR) utilizes AI technology to compare low and high-quality versions of the same content to learn the technological differences between the two and form a vast database. It analyzes millions of pieces of video content and finds a correlation. Based on its analysis, it can select the optimum filters that support brightness, the level of blackness, spread and other errors from all inputs, and transform low-definition content to close to 8K high definition.

The input content is recognized as ‘real-time’ based on a frame and is enhanced by scenes, which makes it possible to upgrade image and sound quality immediately, regardless of whether the video source is live streaming or OTT (Over The Top).

8K picture quality through AI, what’s the difference?

Other upscaling techniques require human input to compare low-resolution and high-resolution scenes and find ways to reconstruct them. However, Samsung’s AI Technology enables it to self-study millions of images on its own using MLSR, allowing much-improved accuracy compared to conventional technologies.

There are three elements to improving picture quality on displays. First is ‘Detail Creation’ that sharpens the detail of expression and improves the texture to areas with low definition that have become blurry after compressing the file. Second, ‘Edge Restoration’ defines the edges of text, people or objects in a video, moving pixels on the borders to thin them down to increase legibility and visibility. For example, if text context is spreaded along the edges, the video will be adjusted around the text for clarity. In a video that shows the moon, Edge Restoration improves details of the moon’s shadow and enhances the darkness of the background for a clear distinction. Lastly, ‘Noise Reduction’ gets rid of static noises generated during high compression or recompression of files. In order to transmit an image, it is necessary to compress the image. In this process, various ‘noises’ such as a jiggling point or a squared dot can be effectively removed according to the image characteristics.

AI delivers immersive sound effects

When watching dramas or movies, realistic, immersive sounds are as important as picture quality. Samsung’s AI technology not only transforms low-definition into high-definition, but also optimizes the sound quality of content.

Conventional TVs provide multiple view settings such as movie mode and sports mode according to the genre of content. With Samsung’s new AI technology, content can be automatically analyzed by characterizing scenes to provide optimum sounds.

For example, let’s say you were watching a movie that includes musical performances. AI technology can highlight the music in a way that allows you to experience the sound as the actual characters would. When the crowd applauds after the performance is over, you would hear the clapping the same way as if you were in the crowd in the movie. When characters are speaking, AI adjusts the sound to make sure the lines are communicated clearly.

Imagine you were watching a relay broadcast of the 2018 PyeongChang Winter Olympics that started last week. AI will enhance the voice of the announcer so that you don’t miss who’s up next. When a game has started, AI will increase the background sound to deliver the liveliness of the actual game. With this tailored sound adjustment scene by scene, audiences can enjoy the best sound quality for any genre of content.

Samsung developers plan to continue to improve sound quality according to the preferences of individual viewers so that each viewer can enjoy the best TV viewing experience, right for them. Because volume patterns differ for every user, and the viewing environment can change according to the time of day and other factors, the sound will be accordingly adjusted and optimized to provide the most enjoyable experience to each individual viewer.

Why does 8K AI technology matter?

As customer needs for high-definition TVs and content increases, some terrestrial broadcasting stations have committed to working towards UHD delivery, and various IPTV and cable channels have initiated 4K UHD (3,840 × 2,160) services. However, even as the TV industry begins to launch 8K (7,680 × 4,320) TVs, the reality is that 4K content is, as of yet, still not fully utilized in homes.

In this context, Samsung has proposed a new direction for TV technology by combining 8K UHD display technology and premiere content through AI. Samsung has developed an AI algorithm that automatically enhances picture quality to solve the problem of limited high-quality content. As example, AI technology plays a key role in the 85-inch 8K QLED TV technology that Samsung introduced at CES 2018.

Samsung will begin the process of applying AI technology to its 8K QLED TVs from the second half of this year, and viewers will soon be able to enjoy UHD quality video that is nearly 8K in resolution and delivers optimized sound for any type of content. A genuine 8K era is now on the horizon, and Samsung will continue to lead the way.