Dreem-ing of Helping Developers Create a More Connected World

Last week, Samsung held the 3rd Samsung Developer Conference in San Francisco. Developers from across the board – some 4,000 from different regions, various platforms – came together to “Connect the Future Everywhere You Look.”

Members of the Interaction Group at Visual Display (Samsung) and TEEM

Among the many SDKs, projects, and chips, one of the things that stood out was the Dreem Project (No, that’s not a typo; there are two e’s). Produced in collaboration between Samsung’s Interaction Group and TEEM, DreemGL is an opensource framework that helps developers build a room-scale, high-performance graphics experience. What does that mean? Well…

Frameworks Help Developers Simplify the Process of Composing Apps

Frameworks provide developers with support programs, compilers, code libraries, tools, etc. For example, if you want to make a switch to turn on a certain function within an app, you can refer to the code library and use previously set components instead of having to design and code the same structure over and over again.

A good framework can help dramatically boost the volume and quality of apps available in the market by enabling and encouraging developers to code more efficiently. There are many different frameworks for all kinds of platforms and devices when it comes to features and functions for the native environment (within the app, that is).

But the requirements change with the changing times. More and more apps now run across several screens and devices; and with the IoT era around the corner, the need for cross-device connectivity is growing. Another increasingly important factor is how to visualize the massive datasets coming from this network of devices and services in a way that satisfies users’ demand for rich graphical output. That’s where DreemGL comes in to play.

Beyond Mirroring – Multiple Screens, Users, and Roles

While most multi-screen solutions currently focus on mirroring – sharing content on one device to another – the future pushes for more than that. Instead of a simple host-client relationship, future services will involve multiple users on multiple screens, each playing a different role.

For example, say you want to play Texas Holdem across several screens, you could have the chips and community cards on the TV while each player has his/her hand on their private smartphones and tablets.

MusicSquare is another good example of how newer services may work: One TV is connected to multiple phones and tablets, allowing each user to assume the DJ role. Each connected user can provide feedback on the music and ultimately collectively create a whole new music experience.

MusicSquare was built using Dreem

These are dynamic environments involving many additional components including multiple devices, roles and customized UI design for each device. Dreem is meant to enable developers to describe such room-scale experiences all from a single source file. Thanks to DreemGL’s unique architecture you can manipulate the source code easily without disrupting the structure with a visual data flow editor. Once developers build a complex user scenario with DreemGL, they can easily change the flow of events with this visual editor or repurpose the same interaction pattern to interact with different types of data sets easily.

Prototyping for GPU-Accelerated Multi-Screen Experiences

Another aspect to take into consideration is that the UI on each screen is evolving to take advantage of faster Graphical Processing Unit (GPU) capabilities. Dreem is built to enable teams of designers and developers to collaborate on real, functional prototypes that explore not only multi-screen user experiences, but also the power of GPU-accelerated graphical UIs. Currently, to implement high-performance graphic UIs, developers need to work with OpenGL Shader Language (GLSL), a specialized complex language. With this low-level programming method, it is very difficult to build graphic experience that responses to lots of data coming from the network of devices and services. Dreem’s architectural breakthrough is that it lets developers to write Shader effect – a complex pixel-level graphical expression – with Javascript, a much approachable scripting language. The Dreem team thinks that this will lower the barrier for many developers to build high-quality graphic visualization projects.

Connecting Designers and Developers with Tools Like Flowgraph and Graphy

There are many different groups of people involved when developing apps and services; like planners, designers, developers, etc. Often times it is a challenge to get everyone to understand what’s going on. The features they need often differ as well. Dreem takes each of these players into consideration and aims to help better facilitate communication.

Flowgraph visualizes the connection between data and events

Flowgraph visualizes the connections between data and events, helping designers, developers and planners better span out their ideas. These connections can be changed on the spot. Graphy reflects the changes UX designers make in the code in real time, so developers don’t need to re-code everything all over.

Graphy allows designers to directly influence the code with its GUI tool

In building and supporting an open-source development framework for GPU-accelerated multi-screen prototyping, Samsung hopes to invite more creativity from across the developer community while preparing them for the upcoming requirements for cross-device, multi-screen, multi-user services as well as the hyper connected IoT.

Hungry for more? Below is a recorded session straight from SDC 2016, featuring the masters; Jinha Lee and Rik Arends:

Dreem project is an on-going effort rather than a finished stable framework. For more information on the project, visit: https://dreemproject.org/.