Samsung Electronics Big Data Solution, No. 1 in Sort Benchmark

Summary:

• Samsung Electronics’ sorting engine, DeepSort, finished No. 1 in this year’s Sort Benchmark competition, often known as the “World Cup” of data sorting.

Samsung Electronics Big Data Solution, No. 1 in Sort Benchmark

New Samsung sorting engine, DeepSort, shatters previous data processing record.

Commercialization planned for Samsung Electronics software services and mobile apps.

Samsung Electronics announced its record-breaking sorting engine, DeepSort, finished No. 1 in this year’s Sort Benchmark competition, often known as the “World Cup” of data sorting. The competition, which pitted Samsung against other major technology companies and research institutions, saw Samsung’s DeepSort analyze and reorganize 3.7TB of data in 60 seconds, more than doubling the previous record of 1.5TB in the MinuteSort benchmark, an industry benchmark on the amount of data a system can sort through in a minute.

“Billions of people across the world continue to generate an incredible amount of data,” said Juhan Lee, Vice President at the Samsung Electronics Software Center in Seoul. “As a result, data processing capabilities have become fundamental in developing next-generation services for consumers, such as those related to the Internet of Things and digital health. DeepSort and other innovative solutions currently in the pipeline will form the foundation upon which a new generation of such services may reach their full potential.”

DeepSort, technically referred to as a distributed general sorting engine, was tested in a networked environment composed of 384 servers. In order to more accurately gauge the software’s sheer performance, hardware and software of moderate specifications were used throughout the test. For instance, the number of servers, the 64GB memory capacity and 2.1GHz processing power of each server were significantly lower relative to those of the main competitors in this year’s Sort Benchmark contest, while hard disk drives (HDD) were used in place of significantly faster solid state drives (SSD).

The record-breaking performance of DeepSort was attributed to fine-tuning a range of key elements in accelerating data processing speed, which occur throughout the overall sorting process. They include, among others, preventing additional wait time for servers with relatively higher data loads by first evening out the sorting load among all servers; enabling simultaneous processing, communication and storage of data by concurrently implementing thousands of threads; and minimizing buffering time in the servers’ HDDs by maximizing memory cache usage.

Developed jointly by Samsung’s Software Center in Korea and Samsung Research America (SRA) in the Silicon Valley, DeepSort is expected to be implemented into a number of Samsung services and mobile apps that incorporate search functions.

DeepSort’s achievement comes in the midst of Samsung’s ongoing push into software innovation, an initiative for which the company has dedicated significant financial investments and more than 40,000 software developers with diverse backgrounds and capabilities.