Nvidia’s new text-to-image tool shows how AI could render stock photos obsolete

Stock photos are the bane of many marketers and designers. The heavyweight image providers are prohibitively pricey, while the affordable ones could lead to a concerned call from HR.

At TNW Towers, we smoothly circumvent these barriers by creating custom masterpieces — like these glorious pictures of Mark Zuckerberg:

Sadly, not everyone is blessed with such virtuosity — including yours truly. Just check out the monstrosity I made atop this article.

Thankfully, AI could soon conceal our artistic ineptitudes, by enabling photorealistic image generation.

The 💜 of EU tech

The latest rumblings from the EU tech scene, a story from our wise ol' founder Boris, and some questionable AI art. It's free, every week, in your inbox. Sign up now!

Nvidia’s new GauGAN2 demo gives a glimpse of the potential. The system can translate text into realist-looking pictures, which you can then edit to your heart’s content.

The model, which is powered by generative adversarial networks (GANs), combines segmentation mapping, inpainting, and text-to-image generation.

You can use these features to create something magical, like unicorns and rainbows, or something more realistic, like the apocalypse.

The demo works best with simple descriptions of nature, such as “sunset at a rocky beach” or “snow in the mountains.” More complicated text can produce bizarre outputs — but that gives the system a chance to flaunt its editing power.

You can generate a segmentation map of objects in the scene, then switch to drawing mode to doodle any tweaks. GauGAN2’s smart paintbrush will then convert the sketches into the style of a photo.

This feature allows you to make realistic edits, like adding a tree or enlarging a mountain. It can also be used to create otherworldly images.

“Imagine, for instance, recreating a landscape from the iconic planet of Tatooine in the Star Wars franchise, which has two suns,” said Nvidia’s Ishan Salian in a blogpost.

“All that’s needed is the text ‘desert hills sun’ to create a starting point, after which users can quickly sketch in a second sun.”

The results aren’t always perfect, but the approach has enormous potential. It may not be long before we ditch stock photos for an endlessly customizable AI image generator.