Like so many great inventions, Expert RAW was a group effort — driven by Samsung’s experts and Galaxy users’ valuable feedback. To learn more about the Expert RAW experience, Samsung Newsroom spoke with the experts at Samsung Research² who collaborated on the app.

Read on to see what Hamid Sheikh, Head of the Intelligent Imaging Team at Samsung Research America’s (SRA) Mobile Processor Innovation (MPI) Lab, and Girish Kulkarni, Head of the Camera Solutions Group at the Samsung R&D Institute India-Bangalore (SRI-B), had to say about this innovative camera collaboration.

Q: What does the Expert RAW app do? What are the main benefits for users?

Hamid: Expert RAW really gives users the ability to take full control of their mobile photography experience much like a professional photographer would. The app captures much more comprehensive data for each image, which can be used to both improve photos and artistically enhance them with editing software like Adobe Lightroom — the go-to app for professional photographers.

To improve the overall camera experience, we worked closely with Adobe to tightly couple Expert RAW with Lightroom. The result is that photographers have full control over their creative experience. Novice photographers can also utilize automatic set points for camera controls, and then use the Samsung-specific default values in Lightroom. In both cases, the tight integration of the two apps makes for an enjoyable experience.

Girish: Expert RAW is an epic leap forward for photography. It essentially places a professional studio in your pocket. Whether you are a pro photographer or you’re just a user who wants to capture amazing content, anyone can take full advantage of the Galaxy S22’s pro-grade camera with Expert RAW. It captures high-quality, high dynamic range pictures in the multi-frame RAW format that enables precise post-capture editing and professional results.

With Expert RAW, people can capture much more information in a single picture — from dark details to bright highlights. It also drastically reduces noise and increases sharpness and detail. If you’re looking for the ultimate creative controls, you can choose your own settings for aspects like ISO, shutter speed, white balance, EV and focus, and even choose which lenses you want to use for the subject and the scene. Finally, because both JPEG and DNG files are saved when shooting, you can hop right into the Adobe Lightroom app on your smartphone or PC and edit with ease.

Q: What was the original inspiration for Expert RAW?

Hamid: Samsung camera technologies have always relied on sophisticated computational photography algorithms to produce the best picture quality. However, the standard camera app makes many decisions about how the image will be rendered in terms of color, tone, detail and noise. The user has very limited say in these decisions.

Our inspiration came from the confluence of user experience limitations and the leaps in computational and AI capabilities of today’s mobile processors. We asked ourselves, “How can we make the photography experience more inspiring for users?” So really, our inspiration came from the expert and enthusiast photography community, which yearned for an easier mobile photography experience, without sacrificing the creative control that DSLR and post-capture editing allows. Expert RAW was born out of a desire to transform that inspiration into reality.

Girish: As we heard the input of professional photographers who were mostly utilizing DSLR cameras, they wanted to achieve a pro-grade experience with smartphones. Our consumer research also showed that after using their smartphone’s auto camera for a long time, typical users were very passionate about exploring Samsung’s pro-grade camera capabilities in-depth to become better photographers.

The more we talked to professional and amateur photographers, the more we understood their needs in terms of RAW image capturing, camera controls, the post-editing experience, creative capabilities and more. That’s when we set out to create a new pro-grade experience based around RAW capturing so that every level of photographer could merge our camera controls and their creativity to enjoy professional-looking photos.

Q: What makes Expert RAW such an important release for mobile photography in general?

Hamid: The camera experience is one of the most important experiences that a user has with his or her smartphone. Samsung has been at the cutting edge of camera quality, and the gap between consumers and professional/enthusiast photographers has always been a tempting one to bridge.

I think with the release of Expert RAW, users will find that their photography experience will take a quantum leap in terms of creative control and artistic expression. I expect more and more people will find this new experience rewarding because it lets them personalize their camera based on their tastes — exactly how they want them — and not based on what some other engineers chose for them.

Girish: As an app, this platform has simplified the RAW capturing process with precise camera and lens controls and post-editing with Adobe Lightroom. It’s for every user who is willing to go beyond — to unleash the true power of an Expert RAW camera, and explore their creative side.

Q: What role did the MPI Lab play in Expert RAW’s development?

Hamid: Any time we are creating a brand-new user experience, we have to consider and overcome many tough engineering challenges. The MPI Lab developed the main idea behind Expert RAW and its technologies, and we knew it was challenge worth pursuing.

We developed a brand-new computational photography pipeline that can produce both DNG as well as processed JPEG files. Developing a complex computational photography pipeline from scratch was not easy. We no longer had the benefit of an older, mature technology base to build upon. But our team successfully developed this new, AI-based 3DISP technology to produce raw data that looks very natural when users load it in editing apps like Adobe Lightroom.

Q: What role did SRI-B play in Expert RAW’s development? What were the team’s main areas of focus?

Girish: Expert RAW was a complex R&D project, and SRI-B’s role was to think in terms of end-to-end for the best consumer experience. This required multi-domain expertise and deep collaboration among multiple teams.

SRI-B is equipped with strong multi-domain expertise teams, so we formed a task force of talented experts. They all worked together as a single team within SRI-B and also worked with a larger team of experts based in Korea and at the MPI Lab.

The multi-domain expertise and close collaboration with all key stakeholders helped SRI-B continuously focus on improving Expert RAW’s product experience, end-to-end pipeline, etc.

Q: Could you offer more details on your collaboration with one another?

Hamid: The MPI Lab has been working with SRI-B throughout our entire history at Samsung. SRI-B contributed its amazing skillset and resources towards the development of the software technologies, as well as key AI blocks for the Expert RAW pipeline. They also owned the UI piece of the Expert RAW application and provided many innovation points on the Expert RAW experience. It’s always fun working with them.

Girish: The MPI Lab has world-class expertise in computational imaging domain, and has a proven record of developing successful IPs for Samsung flagship products. That expertise was one of the key requirements for Expert RAW’s development. The MPI Lab came up with the multi-frame pipeline building blocks implementation, which helped SRI-B integrate and enhance the same for the end-to-end architecture and pipeline.

One of the major expectations for Expert RAW was that it would enable the best image quality in different lighting conditions, which requires the tuning of the multi-frame processing pipeline. MPI’s strong expertise in this area helped us continuously improve image quality before the commercial launch. In summary, MPI’s expertise in computational imaging and SRI-B’s breadth of knowledge in multiple subdomains helped SRI-B prepare Expert RAW’s end-to-end experience for commercial launch.

SRI-B and the MPI Lab used their 12-hour time zone difference effectively to plan and collaborate on Expert RAW. SRI-B’s day was MPI’s night, and vice versa. It helped to have 24/7 development to finish this complex feature in record time. We are proud of Samsung’s culture that enables respectful collaboration among different overseas R&D centers and we always enjoy working with MPI Lab.

Q: How do the MPI Lab and SRI-B plan to build on this collaboration to develop even more innovative camera technologies?

Hamid: Our journey with the new RAW camera experience is just beginning. I am excited to think about what new improvements and enhancements we can make. One of the best parts of the Expert RAW experience is that the photographer is front and center. We’ll preserve that as we actively look into the experiences of different users and continuously implement improvements to make Expert RAW even better than before.

Girish: The MPI Lab and SRI-B have been working together for the last few years developing the world’s best night mode for Samsung flagship smartphones. This collaboration helped lead us to the success story that is Expert RAW on the Galaxy S22. We’re excited to continue that momentum for the next innovative products to come.

In the future, we would like to continuously enhance Expert RAW with a focus on creating a new ecosystem for pro photography that fully utilizes pro-grade camera capabilities.

Collaborating for Creativity 

Features like Expert RAW underline Samsung’s commitment to developing tools that empower users to unleash their creativity.

Going forward, the company will continue to strengthen collaborations that leverage the knowledge and capabilities of its world-class network of R&D centers. Doing so will help unlock technologies that change the way we live, work and create.

¹ Expert RAW must be downloaded separately from the Galaxy Store, for free, before use. Expert RAW app availability is currently limited to the Galaxy S21 Ultra and S22 series. Samsung Members notified Expert RAW will become available on Galaxy Z Fold3 in April 2022 and Galaxy S20 Ultra, Galaxy Note20 Ultra and Galaxy Z Fold2 later this year.

² Samsung Electronics’ advanced R&D hub, which leads the development of future technologies for its Device eXperience division

Gary Geunbae Lee, Head of Samsung Research AI Center, Samsung Electronics

Nowadays, artificial intelligence (AI) has emerged as a leading global future technology trend. AI is so much at the center of the current technological revolution that it is expected to fundamentally alter not only the IT industry, but also the automobile, banking, and medical sectors. As a result, companies are making efforts to hire AI experts and are investing in research and other related business fields to ensure that they are fully prepared to integrate AI into products and services that can benefit people’s lives.

Samsung Electronics has long recognized the significance of AI and has been actively investing in the area. The company currently maintains seven AI centers in five different countries; South Korea (Seoul), the U.S. (Silicon Valley and New York), the U.K. (Cambridge), Canada (Toronto and Montreal) and Russia (Moscow). But the question remains – what is behind AI’s rapid evolution, and what forms is the technology expected to take going forward?

In a series of interviews, Samsung Newsroom sat down with leading experts from each of the AI centers around the world about the latest AI trends and what they believe the future holds for the technology. The first interviewee is Gary Geunbae Lee, Senior Vice President and Head of Samsung Research’s AI Center in Seoul. We asked him for his insights regarding AI development, and enquired about what makes Samsung’s approach to AI distinctive.

What is AI to Samsung?

AI is the realization of implementing the human capabilities of seeing, listening, decision making, moving and learning into computers,” said Gary Lee. “To think about it another way, AI is a combination of A (algorithms), B (big data), and C (computing power). These are the three key components that allow us to construct well-rounded artificial intelligence.”

Lee explained that Samsung Research’s AI Centers around the world conduct research that covers the full gamut of AI development, including computer vision, language understanding, data analytics, robotics and machine learning. Their research aims to bring the capabilities of AI closer to that of the human brain. “Compared to the time it took humanity to evolve to its current state, the history of AI is very short – only about 60 years,” noted Lee. “AI still isn’t quite there as far as fully realizing human actions, but I believe the technology will continue to improve quickly.”

Each of Samsung’s seven AI Centers spread across the world has its own specific fields of research. For example, the Cambridge AI Center is focused on On-Device AI and AI technology related to next-generation telecommunication networks, the Moscow Center is the Moscow Center is focusing on AI core technology such as data generation for machine learning and advanced deep learning, and the New York Center’s focus is centered on advanced research fields such as robotic manipulation. The Seoul Center works on language understanding, speech processing and big data and also coordinates the other centers, fostering collaboration and efficiency between them.

Samsung Seoul R&D Campus

Once AI products and services are ready for release, they are made available to consumers through various services, including Bixby. The majority of recent Samsung products come with Bixby already incorporated into the device, and Samsung is working hard to make all Samsung products AI-ready in the near future. “Once Samsung’s AI speaker is launched, it will enable seamless interconnection between smart devices,” affirmed Lee. “This will enable us to achieve our goal of implementing AI on all of our products to the goal of fostering environments led by connectivity.”

In 2018 the company revealed a software development kit (SDK) for Bixby and currently Bixby supports voice commands and translations in English, Chinese (Mandarin), German, Italian and Spanish. The diversity and global versatility of Samsung’s product portfolio is one of its key strengths, and Bixby will enable users around the world to connect their devices seamlessly.

When Samsung Products and AI Meet

“Samsung Electronics sells 500 million products every year. We strive each and every day to provide new services and features that are a fit for our devices in order to closely match our customers’ lifestyles,” noted Lee. One of the greatest strengths of the Samsung AI offering is the device integration; Lee highlighted the added value Samsung brings to users by assimilating a wide range of AI capabilities into the devices that have already become staples in users’ lives.

“For example, AI can recognize the ingredients in your refrigerator and automatically recommend recipes which it then displays on the refrigerator’s screen,” said Lee. “After you put the food in the oven, the AI-powered software sets the temperature and cooking time intelligently, according to the recipe.”

Samsung’s AI centers around the world focus on developing original technology such as natural language processing for Bixby and advanced research areas such as robotics; R&D teams at each business unit then work on merging and applying the new innovations to the products.

Teaching AI to Think for Itself

Despite its recent origin date, AI recognition technology has evolved considerably over time; in some areas, even, AI has proven to be more advanced when compared with the abilities of humans. Nevertheless, there are still some areas that AI needs to improve to become the ultimate assistance tool: performance and accuracy.

While current AI technology is capable of understanding spoken languages, there is still a long way to go until AI is able to fully process words within their context as a human does. In other words, what AI needs to develop is the comprehension of hypothetical reasoning, or the ability to read between the lines, of a user’s command or statement. There is also a need for more development regarding AI’s capability to taking different approaches to solve problems based on individual, or unpredictable, situations and daily scenarios. Future AI development currently rests on how fast improvements are made in these areas.

Samsung’s AI Philosophy

AI can provide a significant competitive edge for businesses – but if abused or mishandled, it can cause a social problem when not kept under control. Therefore, sustained ethical compliance is critical when conducting AI research. Samsung takes this responsibility very seriously and is constantly formulating ways to improve its practices and increase accountability.

“Although AI is meant to improve people’ lives, the possibility of its abuse cannot be ignored, so ethical compliance is very important,” stressed Lee. “There are three ethics that Samsung follows in regards to AI.” These are fairness, accountability and transparency. The development or use of AI must not lead to discrimination or prejudice, and the company assumes total responsibility for the technology and maintains transparency in its data collection and management process. In order to ensure that it keeps developing AI products and services that are worthy of consumers’ trust, Samsung became a member of the Partnership on AI (PAI) last year.

Privacy is another important part of Samsung’s AI policy. Since the more data an AI service is able to utilize, the more helpful it can be, all data harnessed must be processed transparently so that consumers feel safe when using Samsung products and services. “Samsung Electronics greatly prioritizes data security and privacy,” noted Lee. “We adhere to all the related laws and regulations on data security, including the GDPR¹ in Europe. We are working towards implementing technology that detects security vulnerabilities in our AI codes to ensure our customers can use AI-enabled products and services safely. At the same time, we are incorporating AI into our security software to develop identification capabilities that will provide further peace of mind for users.”

So, what is the final goal Samsung has in mind when pursuing an AI-enabled future? From Lee’s perspective, it is totally consumer centric. “For me, AI is about adding value to consumers’ lives thanks to the integration of AI into their everyday products and services. Which this in mind, Samsung Research operates, and will continue to operate, under the conviction that user-based AI is always there, must be helpful, must be safe, must be user-centric, and, finally, must always be learning.”

“At Samsung, we always keep our AI principles front of mind,” Lee emphasized. “Firstly, all of our AI-based products and services continuously learn by themselves and become smarter over time, improving performance and usability while interacting with consumers. Secondly, Samsung AI is always there across a range of devices for whenever a customer needs it. Thirdly, Samsung is committed to developing AI technologies that are always safe and offer consumers peace of mind. Finally, Samsung’s AI always pursues user-centric customization to provide as helpful and as personalized service as possible.”

“These principles are the fundamental technological base for all the AI products and services of Samsung,” added Lee. “They enable Samsung to provide meaningful and tangible user-oriented experience and values with our AI offering.”

¹ The EU General Data Protection Regulation