A New Rendering Engine for Darkroom

We’re releasing a major rebuild of Darkroom’s rendering engine that will dramatically improve the performance, stability, and reliability of your photo editing experience.

Portrait editing is more reliable and accurate, RAW photos are more stable, and every slider, transition, and interaction is smoother. This is the result of a yearlong investment that touched much of the app. Although there aren’t any new features, every existing feature is better and faster.

Why take this on?

In 2015 we supported a single image type on two iPhones with one engineer. Making Darkroom fast, stable, and reliable was a manageable task given the simplicity. Since then, we dramatically increased the number of devices, platforms, and image types we support.

We couldn’t have predicted this growth in complexity up front. Some technical decisions were made early on that made supporting the full weight of advanced mobile photography a challenge. If you ever encountered stutters, crashes, or jitters while editing, you know what we’re talking about. This is counter to our core values of delivery the highest quality editing experience possible.

We decided that the time had come to modernize the app’s infrastructure. We got back to basics, brought on graphics specialists to help us dig deep into the world of Metal to build a fine-tuned rendering engine optimized for Portrait and RAW photos, and delivers an unparalleled editing experience.

Throughout the entire app’s experience, you’ll notice the app respond faster and more predictably, and with smoother transitions. Darkroom now uses dramatically less memory, providing a much more responsive and reliable editing experience

Portrait Photos

Portrait images now load faster, and our blur pipeline is much more performant and accurate, matching the Photos app in its rendition, while retaining all our depth editing features

We also made the Blur slider consistent across all devices and moved it so it’s always available, regardless of whether you have the Foreground or Background selected.

Pro-tip: If you’re seeing some unexpected results with the blur, try long-pressing on the Portrait badge in the top left of the photo to view the raw depth image. You can even hold it down while you adjust the Depth slider to view the impact of your changes.

Note: As part of our work to deliver the highest quality portrait blur effect, we decided to eliminate the “Portrait” and “Portrait HD” distinction. We now always use all available depth maps to produce a single high quality blur.

RAW Photos

For the past year, RAW editing stability has been our top reported issue across all platforms. Fixing it required a massive, yearlong effort, but that’s exactly what we’ve been doing. Starting now, the entire RAW editing experience is revamped end-to-end to be much faster, smoother, and more stable.

We’ve made the transition into and out-of RAW images much lighter and more smooth, improved the performance of all editing operations, and added support for pixel-level zoom on all RAW images. Shadow recovery is also improved; RAW images no longer appear as dark as they used to in Darkroom.

Note: Although the quality and performance of the RAW engine itself is vastly improved, we still have some more work to do to refine the recovery range of our Highlights/Shadows sliders as you drag them to the edge of their range. This is something we’re actively working on, and we’ll have more to share when it’s ready.

Library Navigation

These improvements also extend to the rest of the app experience. We’ve dramatically improved the performance of scrolling through your photo grid. You’ll no longer see black tiles as you quickly scroll through your library.

We’ve also updated the logic for generating edited thumbnails to make them faster and much more accurate. Now, Portrait and RAW photos will look pixel-accurate once you make any edits. Try pasting edits while in batch mode and watch them all cascade and populate dropping any frames.

We’ve also improved the experience and performance of switching between albums.

Photo Grid Pro Tip: If you have a lot of assets in your library that need to be downloaded from iCloud, pasting edits is a really fast way to trigger the iCloud download on many photos before your editing session. This is particularly helpful if you’re capturing a lot of photos on your phone, but editing them later on your iPad.

How is all this done?

We’ve traditionally relied heavily on Apple’s image processing toolkit to provide the foundation of Darkroom’s editing experience. This served us really well in our early days since it meant we could deliver a pro-level editing experience quickly, with a single engineer. However, as our feature complexity grew, we started stepping beyond what the toolkit is built to designed to handle.

For example. If you open an 88MB RAW file and zoom in, we need to process only the visible part of the image to keep performance fast. But if we also have a Histogram, we’ll need to also process the entire image every frame. Being able to process the image at these two different resolutions simultaneously turns out to be a non-trivial task at 120 frames per second.

Moreover, as the complexity of the rendering pipeline grows, it can start affecting the responsiveness of the interface. Ensuring that the rendering pipeline can operate independently from the app’s interface can also be challenging.

Our new rendering engine is built right on top of Metal, Apple’s low-level graphics engine which games, photo, and video apps are all built on top of. It gives us freedom, but meant we had to build our own toolkit on top. We had to build our own caching layer, our own scheduling system, and our own processing and rendering systems to ensure we never allocate too much memory, starve out the graphics system from updating the interface, or do more work that is necessary.

It’s a dynamic rendering system that responds to the capability of your device as you edit. It starts out high resolution, and will reduce the editing resolution while you’re interacting with the app to ensure everything stays fast and responsive.

Our work here isn’t done, and we’ll continue to iterate on this in the coming months.

What’s Next?

In our announcement when we switched to Subscriptions, we promised you a lot more frequent updates, with a lot of features you’ve been asking for for a long time. This is one step, but in many ways, it’s the most important. This new infrastructure provides us the memory, performance, and technical headspace to start tackling much more challenging projects that have been out of reach for us.

Although this version has been a huge team endurance effort by Majd Taby and Igor Lipovac, we owe a big big thank you to Andrew Pouliot for his help in building the initial version of the engine, and his continued support throughout the process. Couldn’t have done it without you!

Until soon, The Darkroom Team