Adobe Teases AI That Can Create Impressive Composite Photos

Adobe is working on an artificially intelligent (AI) feature that will automatically blend different photos into one convincing composite with a single button press.

Called “Project Clever Composites” and revealed during the company’s annual “Sneaks” presentation at Adobe MAX, it will harness AI to predict the object’s scale, blend the colors of the objects, estimate the lighting conditions, and generate the necessary shadows to make it look realistic.

“Image compositing lets you add yourself in to make it look like you were there. Or maybe you want to create a photo of yourself camping under a starry sky but only have mages of the starry sky and yourself sampling during the daytime,” Adobe tells Tech Crunch.

The example that Adobe gave is of a starry sky above mountains. If the photo editor wants to add in a full moon, a tent, and a person from separate photos, then Clever Composites will blend these three elements to make them appear as if they were actually in the original scene.

In the video presentation at MAX, Zhifei Zhang, an Adobe research engineer on the project, shows how the tech is smart enough to recognize that a van needs a shadow in order to appear seamlessly on a road background.

“We developed a more intelligent and automated technique for image object compositing with a new compositing-aware search technology,” Zhang tells TechCrunch.

“Our compositing-aware search technology uses multiple deep learning models and millions of data points to determine semantic segregation, compositing-aware search, scale-location prediction for object compositing, color and tone harmonization, lighting estimation, shadow generation, and others.”

The models that power the overall feature are trained separately on their given tasks. For example, the AI that looks for an object that is consistent within the image is different from the AI that is tasked with deciding the object’s scale and location.

“Achieving automatic object compositing is challenging, as there are several components of the process that need to be composed. Our technology serves as the ‘glue’ as it allows all these components to work together,” Zhang says.

Since it’s a Sneak, Clever Composites might be added to an existing Adobe product in the future, or it might not. It and other innovations shown during the Sneaks presentation may be left on the factory floor and may never find their way into a commercial product.

However, Zhang believes it would make a fantastic addition to Photoshop and Lightroom, and anyone who has labored intensively to get different photos with different lighting to fit harmoniously into a single image might agree.