Like so many great inventions, Expert RAW was a group effort — driven by Samsung’s experts and Galaxy users’ valuable feedback. To learn more about the Expert RAW experience, Samsung Newsroom spoke with the experts at Samsung Research² who collaborated on the app.

Read on to see what Hamid Sheikh, Head of the Intelligent Imaging Team at Samsung Research America’s (SRA) Mobile Processor Innovation (MPI) Lab, and Girish Kulkarni, Head of the Camera Solutions Group at the Samsung R&D Institute India-Bangalore (SRI-B), had to say about this innovative camera collaboration.

Q: What does the Expert RAW app do? What are the main benefits for users?

Hamid: Expert RAW really gives users the ability to take full control of their mobile photography experience much like a professional photographer would. The app captures much more comprehensive data for each image, which can be used to both improve photos and artistically enhance them with editing software like Adobe Lightroom — the go-to app for professional photographers.

To improve the overall camera experience, we worked closely with Adobe to tightly couple Expert RAW with Lightroom. The result is that photographers have full control over their creative experience. Novice photographers can also utilize automatic set points for camera controls, and then use the Samsung-specific default values in Lightroom. In both cases, the tight integration of the two apps makes for an enjoyable experience.

Girish: Expert RAW is an epic leap forward for photography. It essentially places a professional studio in your pocket. Whether you are a pro photographer or you’re just a user who wants to capture amazing content, anyone can take full advantage of the Galaxy S22’s pro-grade camera with Expert RAW. It captures high-quality, high dynamic range pictures in the multi-frame RAW format that enables precise post-capture editing and professional results.

With Expert RAW, people can capture much more information in a single picture — from dark details to bright highlights. It also drastically reduces noise and increases sharpness and detail. If you’re looking for the ultimate creative controls, you can choose your own settings for aspects like ISO, shutter speed, white balance, EV and focus, and even choose which lenses you want to use for the subject and the scene. Finally, because both JPEG and DNG files are saved when shooting, you can hop right into the Adobe Lightroom app on your smartphone or PC and edit with ease.

Q: What was the original inspiration for Expert RAW?

Hamid: Samsung camera technologies have always relied on sophisticated computational photography algorithms to produce the best picture quality. However, the standard camera app makes many decisions about how the image will be rendered in terms of color, tone, detail and noise. The user has very limited say in these decisions.

Our inspiration came from the confluence of user experience limitations and the leaps in computational and AI capabilities of today’s mobile processors. We asked ourselves, “How can we make the photography experience more inspiring for users?” So really, our inspiration came from the expert and enthusiast photography community, which yearned for an easier mobile photography experience, without sacrificing the creative control that DSLR and post-capture editing allows. Expert RAW was born out of a desire to transform that inspiration into reality.

Girish: As we heard the input of professional photographers who were mostly utilizing DSLR cameras, they wanted to achieve a pro-grade experience with smartphones. Our consumer research also showed that after using their smartphone’s auto camera for a long time, typical users were very passionate about exploring Samsung’s pro-grade camera capabilities in-depth to become better photographers.

The more we talked to professional and amateur photographers, the more we understood their needs in terms of RAW image capturing, camera controls, the post-editing experience, creative capabilities and more. That’s when we set out to create a new pro-grade experience based around RAW capturing so that every level of photographer could merge our camera controls and their creativity to enjoy professional-looking photos.

Q: What makes Expert RAW such an important release for mobile photography in general?

Hamid: The camera experience is one of the most important experiences that a user has with his or her smartphone. Samsung has been at the cutting edge of camera quality, and the gap between consumers and professional/enthusiast photographers has always been a tempting one to bridge.

I think with the release of Expert RAW, users will find that their photography experience will take a quantum leap in terms of creative control and artistic expression. I expect more and more people will find this new experience rewarding because it lets them personalize their camera based on their tastes — exactly how they want them — and not based on what some other engineers chose for them.

Girish: As an app, this platform has simplified the RAW capturing process with precise camera and lens controls and post-editing with Adobe Lightroom. It’s for every user who is willing to go beyond — to unleash the true power of an Expert RAW camera, and explore their creative side.

Q: What role did the MPI Lab play in Expert RAW’s development?

Hamid: Any time we are creating a brand-new user experience, we have to consider and overcome many tough engineering challenges. The MPI Lab developed the main idea behind Expert RAW and its technologies, and we knew it was challenge worth pursuing.

We developed a brand-new computational photography pipeline that can produce both DNG as well as processed JPEG files. Developing a complex computational photography pipeline from scratch was not easy. We no longer had the benefit of an older, mature technology base to build upon. But our team successfully developed this new, AI-based 3DISP technology to produce raw data that looks very natural when users load it in editing apps like Adobe Lightroom.

Q: What role did SRI-B play in Expert RAW’s development? What were the team’s main areas of focus?

Girish: Expert RAW was a complex R&D project, and SRI-B’s role was to think in terms of end-to-end for the best consumer experience. This required multi-domain expertise and deep collaboration among multiple teams.

SRI-B is equipped with strong multi-domain expertise teams, so we formed a task force of talented experts. They all worked together as a single team within SRI-B and also worked with a larger team of experts based in Korea and at the MPI Lab.

The multi-domain expertise and close collaboration with all key stakeholders helped SRI-B continuously focus on improving Expert RAW’s product experience, end-to-end pipeline, etc.

Q: Could you offer more details on your collaboration with one another?

Hamid: The MPI Lab has been working with SRI-B throughout our entire history at Samsung. SRI-B contributed its amazing skillset and resources towards the development of the software technologies, as well as key AI blocks for the Expert RAW pipeline. They also owned the UI piece of the Expert RAW application and provided many innovation points on the Expert RAW experience. It’s always fun working with them.

Girish: The MPI Lab has world-class expertise in computational imaging domain, and has a proven record of developing successful IPs for Samsung flagship products. That expertise was one of the key requirements for Expert RAW’s development. The MPI Lab came up with the multi-frame pipeline building blocks implementation, which helped SRI-B integrate and enhance the same for the end-to-end architecture and pipeline.

One of the major expectations for Expert RAW was that it would enable the best image quality in different lighting conditions, which requires the tuning of the multi-frame processing pipeline. MPI’s strong expertise in this area helped us continuously improve image quality before the commercial launch. In summary, MPI’s expertise in computational imaging and SRI-B’s breadth of knowledge in multiple subdomains helped SRI-B prepare Expert RAW’s end-to-end experience for commercial launch.

SRI-B and the MPI Lab used their 12-hour time zone difference effectively to plan and collaborate on Expert RAW. SRI-B’s day was MPI’s night, and vice versa. It helped to have 24/7 development to finish this complex feature in record time. We are proud of Samsung’s culture that enables respectful collaboration among different overseas R&D centers and we always enjoy working with MPI Lab.

Q: How do the MPI Lab and SRI-B plan to build on this collaboration to develop even more innovative camera technologies?

Hamid: Our journey with the new RAW camera experience is just beginning. I am excited to think about what new improvements and enhancements we can make. One of the best parts of the Expert RAW experience is that the photographer is front and center. We’ll preserve that as we actively look into the experiences of different users and continuously implement improvements to make Expert RAW even better than before.

Girish: The MPI Lab and SRI-B have been working together for the last few years developing the world’s best night mode for Samsung flagship smartphones. This collaboration helped lead us to the success story that is Expert RAW on the Galaxy S22. We’re excited to continue that momentum for the next innovative products to come.

In the future, we would like to continuously enhance Expert RAW with a focus on creating a new ecosystem for pro photography that fully utilizes pro-grade camera capabilities.

Collaborating for Creativity 

Features like Expert RAW underline Samsung’s commitment to developing tools that empower users to unleash their creativity.

Going forward, the company will continue to strengthen collaborations that leverage the knowledge and capabilities of its world-class network of R&D centers. Doing so will help unlock technologies that change the way we live, work and create.

¹ Expert RAW must be downloaded separately from the Galaxy Store, for free, before use. Expert RAW app availability is currently limited to the Galaxy S21 Ultra and S22 series. Samsung Members notified Expert RAW will become available on Galaxy Z Fold3 in April 2022 and Galaxy S20 Ultra, Galaxy Note20 Ultra and Galaxy Z Fold2 later this year.

² Samsung Electronics’ advanced R&D hub, which leads the development of future technologies for its Device eXperience division

In this relay series, Samsung Newsroom is introducing tech experts from Samsung’s R&D centers around the globe to hear more about the work they do and the ways in which it is directly improving the lives of consumers.

The fifth expert interviewed for the series is Ratnakar Rao V R, who heads the Beyond 5G Team at Samsung R&D Institute India – Bangalore (SRI-B). Rao is soon to complete a decade at SRI-B, and the bulk of his experience has been in the research and development of wireless communication technologies like 4G and 5G. Check out the interview below to find out more about the promising technologies Rao and his team have been working on.

Q: Advanced system software plays a crucial role in activating all kinds of technologies developed to provide better user experiences. How has research into applied AI been factoring into your work within the communications field?

Traditionally, all cellular communication systems were implemented using mathematical models and were strictly rule-based. However, this is now changing in the 5G era due to a few key factors.

Firstly, since a single network has to cater to a number of diverse use cases simultaneously, such systems cannot operate to the best of their capabilities if implemented based only on traditional modeling. Secondly, advances in computation algorithms and processor architectures are making it easier to run AI and machine learning models on a wider range of devices. Thirdly, wireless networks are being virtualized and are getting split into micro-services that run in the cloud. On-Device AI capabilities are also being added to wireless terminals. From 5G onwards, networks will be closely integrated with applications, making it necessary for them to be more contextually aware of users and applications in order to deliver personalized network experiences to all.

All these factors enable and necessitate broader use of AI and machine learning in next-generation wireless networks and terminals.

Q: Can you please briefly introduce SRI-B and the kind of work that goes on there?

The Samsung R&D Institute in Bangalore (SRI-B) has established five Centers of Excellence (CoEs) with the focus areas of Communication, Camera and Multimedia, On-Device AI, IoT and Services. SRI-B has experience executing projects from the research to market stage in each of these areas, and makes impactful contributions to Samsung product lines on the backs of these CoEs every year.

At the Communication CoE, SRI-B has dedicated teams working on mobile terminals, network RAN/core development and wireless standards. Strong synergy between these teams has resulted in the establishment of end-to-end domain expertise. In addition to this, we have recently seeded advanced communication research in a bid to make impactful contributions to Beyond 5G and 6G evolution.

Q: What kind of communication-related work are you and your team engaged in now?

Firstly, our team specializes in radio, data networking protocols and embedded modem system software. We craft the 5G radio experience for different markets around the world. Our team is engaged in the product development of 5G mobile terminals for a range of world-wide markets.

Secondly, we are engaged in advanced research and development surrounding communication protocols. Some of this work makes it into Samsung products as differentiating features and solutions. The rest of our work is aimed at creating standards and implementation IP (intellectual property) pertaining to Beyond 5G and 6G systems.

Q: How do you expect the Beyond 5G era to change the way users interact with technology in their day-to-day lives?

The early launches of non-standalone 5G have unlocked a lot of hitherto unused spectrum in mid and high bands and also enabled the re-farming of the existing 4G spectrum for use with 5G. Thus, the transition to a very high-capacity communication system is underway.

This massive addition of capacity will enable more users to connect more devices to the internet and bring the benefits of connectivity to the masses who live in rural areas. For regular users, these benefits will be evident in terms of very high-resolution streaming, faster downloads and uploads and real-time interactive gaming. They will also see mixed-reality experiences in video streaming and video calls.

The subsequent enhancements to 5G in the coming years will unlock consumer infotainment and a lot of other use cases. For example, the low-power, low-bandwidth features of IoT devices will help public services, the agricultural industry and factories automate for better efficiency. Likewise, satellite-based 5G will provide ubiquitous coverage all over the globe. The highly reliable, low-latency optimizations applied to 5G networks will also enable better remote delivery of services like healthcare and education.

Q: Which of SRI-B’s achievements in the communications field are you most proud of?

We at SRI-B are proud to have played a critical role in the launch of the world’s first 3G, 4G and 5G smartphones. Recently, SRI-B has enabled standalone 5G and 5G carrier aggregation in mobile terminals for commercial use, developed 4G and 5G network software and helped establish MC-PTT (Mission Critical Push to Talk) capabilities.

SRI-B has also been an IP powerhouse for several years. Every year, a number of IPs are created by SRI-B engineers from across the various domains. We have created more than 200 implementation IPs in the area of wireless communication, and more than 100 standard essential IPs in the areas of 4G and 5G.

Q: How does collaborating with other institutes like Samsung Research America, Samsung R&D Institute UK and Samsung Research in Korea complement your work and research capabilities?

We have worked closely with Samsung Research on early technology development and realization of 5G, and are now collaborating on nascent 6G technology development. I strongly believe that a lot of potentials can be unlocked by further collaboration between SRI-B and the teams at Samsung Research America and Samsung R&D Institute UK.

SRI-B has a very large pool of communication engineers, including innovators and domain experts. It is therefore possible to build high-quality teams and execute research promptly. We are actively exploring these possibilities by interacting with R&D leaders at global research centers to enable breakthrough innovations.

Q: How are AI and machine learning being applied to Beyond 5G and 6G wireless communication technology? How do you expect these technology combinations to evolve going forward?

It is widely agreed that AI and machine learning will have a significant influence on network management and radio resource management for Beyond 5G and 6G networks. We envisage AI and machine learning applications being present in block-level AI, procedural AI and system software AI, and are actively researching along these lines.

• • Block-level AI: A specific block in the terminal or network could be added with AI/machine learning without impacting the rest of the system, resulting in performance improvements and/or computation savings. For example, a channel decoder could decide to terminate the decoder iterations early if it is able to predict whether the block decoding will eventually pass or fail.
  • Procedural AI: This is where at least two entities in the end-to-end system exchange information to enable accurate use of AI and machine learning techniques. For example, meta-data needs to be exchanged between the terminal and network for an auto encoder or decoder to work within a margin of error. Another example is mobility management for terminals.
  • System software AI: Most entities in next-gen communication systems will have to operate in several modes. The embedded system software should be able to scale up or scale down system resources very dynamically. AI-assisted embedded system software is expected to learn context-specific requirements and adapt accordingly.

Q: You are a senior member of the Institute of Electrical and Electronics Engineers (IEEE). What kind of activities are you involved with in this role? How does your position in the IEEE inform your other work?

In this role, I deliver invited lectures and talks on various technology topics for the student communities and teach fraternities at regional engineering colleges. The role has also enabled me to seed new study items internally, allowing us to initiate new collaborations with student communities and faculties from reputed universities.

The aim is to influence as many people as possible to improve their domain expertise and pursue advanced research in the communications field. I also represent Samsung in various talks and discussions with industry, government and academia. My interactions help me stay in touch with the latest trends in various areas adjacent to my area of expertise.

I also encourage my team members to publish their results in outstanding conferences and journals. This year and last year, we have published more than 20 papers in various outstanding forums.