ICCE is the flagship conference of the IEEE Consumer Technology Society (formerly the IEEE Consumer Electronics Society), held annually in conjunction with the Consumer Electronics Show (CES) in Las Vegas, USA. A total of 190 research papers were accepted this year, with five finalists nominated for the virtual conference’s Best Paper Award. The competition consisted of video-recorded paper presentations and a live Q&A session, and just three papers were selected as winners.

“The work portrayed in this paper connects domains such as AI, UX and affective computing with the goal of providing an emotionally enriching user experience by bridging artworks and music in an unconventional manner,” said Anant Baijal, a Samsung Staff Engineer (Advanced R&D Group) and the lead author of the paper. “We propose a method that considers both an artwork’s style (classical, baroque, impressionist etc.) and the emotion it conveys (calm, joyful, mellow, etc.) to automatically curate music that is well-suited to accompany the image.”

The research highlighted in the paper is under review for possible commercialization in Samsung’s flagship products, including the recently announced Neo QLED TVs, along with lifestyle products such as The Frame. Currently, through Ambient Mode on QLED and Art Mode on The Frame, consumers can display pictures of their choice when they are not watching TV. The research team wants to take the user experience to the next level by giving consumers the option to play music that is curated based on automatic analysis of the image displayed on the screen. The results obtained from initial tests conducted during the research support the efficacy of the proposed approach. 

“When a user uploads a picture, such as artwork or a photograph, our proprietary AI algorithms evaluate the image to recommend music through a user-subscribed music streaming provider,” added Baijal. “Future work on this technology would include integrating other parameters into the algorithm, such as the weather, time, geolocation and user activity, in addition to the image being displayed. We are designing the algorithm to be capable of adapting to individual users’ preferences.” 

“At Samsung’s Advanced R&D Group, we’re continually striving to provide a more dynamic screen experience by developing innovative technologies that complement consumers’ ever-changing lifestyles,” said Dr. Danny Hyun, Corporate Vice President and Head of Advanced R&D Group at Samsung Electronics and a co-author of the paper. 

The Advanced R&D Group’s research spans multidisciplinary areas such as AI, Big Data and connectivity, and the Group is also responsible for creating futuristic concepts and product prototypes. Multiple international patent applications covering the technology’s concept, components and user scenarios have been filed by Samsung Electronics. 

Check out Samsung’s First Look 2021 event (highlights available here) for more details on the company’s vision for the future of screens, including its latest display innovations and 2021 product lineup.

The paper is derived from the Digital Health Lab’s Lung Health project, where the team is developing innovative technologies to help patients with chronic respiratory diseases such as Chronic Obstructive Pulmonary Disease (COPD) or Asthma. The paper particularly focuses on algorithms that allow users to evaluate their lung function on their own without clinical supervision, simply by recording breathing sounds on their smartphones.

“Winning the Best Paper Award at such a competitive venue as the IEEE PerCom testifies to the recognition of our research from leading experts in pervasive computing. I feel fortunate to work in a lung health project that has potential to have an impact on respiratory patients, to support the control over their illness and improve the quality of life,” said Mahbubur Rahman, the first author of the paper who is working as Research Engineer in Digital Health Lab.

Acknowledging the Best Paper Award, PerCom 2020 General Chair and the University of Texas at Austin Professor Christine Julien shared an encouraging note emphasizing the potential impact of the Digital Health Lab’s work on real-world patients. She commented, “Having much more accessible tools for treating this [respiratory] disease would be so incredible. I love that there is this real-world impact of what you’re working on.”

The Digital Health Lab at SRA is partnering with world-leading physicians and advanced health research institutions to develop novel algorithms to make the health monitoring tools accessible anywhere, anytime through smartphones and smartwatches. The Digital Health Lab’s papers have also previously received the Best Poster Award at the IEEE Body Sensor Network (BSN) 2019 conference.