Dall-E 3, its latest AI art tool. It uses OpenAI’s smash-hit chatbot, ChatGPT, to help create more complex and carefully composed works of art by automatically expanding on a prompt in a way that gives the generator more detailed and coherent instruction.
What’s new with Dall-E 3 is how it removes some of the complexity required with refining the text that is fed to the program—what’s known as “prompt engineering”—and how it allows users to make refinements through ChatGPT’s conversational interface. The new tool could help lower the bar for generating sophisticated AI artwork, and it could help OpenAI stay ahead of the competition thanks to the superior abilities of its chatbot.
This kind of quirky AI-generated art has become commonplace on social media thanks to a number of tools that turn a text prompt into a visual composition. But this one was created with a significant amount of artistic assistance from ChatGPT, which took a short prompt and turned it into a more detailed one, including instructions about how to compose it correctly.
That’s a big step forward not just for Dall-E, but for generative AI art as a whole. Dall-E, a portmanteau of the Pixar character Wall-E and the artist Salvador Dalí that was announced in 2021 and launched in 2022, consists of an algorithm that’s fed huge quantities of labeled images scraped from the web and other sources. It uses what’s known as a diffusion model to predict how to render an image for a given prompt. With sufficiently huge quantities of data this can produce complex, coherent, and aesthetically pleasing imagery. What’s different with Dall-E 3 is in the way humans and machines interact.
This image, rendered by Dall-E 3, shows how using ChatGPT to fill in a prompt produces a more coherent and sophisticated image. It might normally require a huge amount of prompt engineering, whereby a user tries increasingly complex prompts to create something sophisticated. But with Dall-E 3, ChatGPT takes on the work of crafting that more sophisticated prompt.
Dall-E 3 produced this image in response to the following prompt: “An illustration of a human heart made of translucent glass, standing on a pedestal amid a stormy sea. Rays of sunlight pierce the clouds, illuminating the heart, revealing a tiny universe within. The quote 'Find the universe within you' is etched in bold letters across the horizon.”
Dall-E 3 will also let users refine a creation through ChatGPT, as if they were asking a real artist to make changes. “You won't really have to worry about fussing around with really long prompts,” says Aditya Ramesh, lead researcher and head of the Dall-E team. “Instead, you can just interact with ChatGPT as if you were talking to a coworker.”
Gabriel Goh, lead researcher on the Dall-E team, demonstrated the trick to WIRED by asking Dall-E 3 to create several promotional posters for an imaginary noodle restaurant. After being presented with a few options, Goh asked Dall-E 3, through ChatGPT, to take one of them and turn it into an illustration of a sign hanging outside a restaurant.
In 2022, the emergence of numerous AI art generators heralded the start of a broader generative AI boom. Many of the early generators were crude and unable to refine or modify images. Besides OpenAI, startups including Midjourney, Stable Diffusion, and Ideogram have attracted significant funding and public interest. But the use of these AI art systems has also prompted concern that professional human artists could be displaced, and about how AI companies use copyrighted material to train their algorithms.
Reacting to this controversy, OpenAI also announced today that it will launch a way for artists to have their works removed from future training runs. Dall-E 3 will also prevent users from trying to generate a piece of art in the style of a well-known artist by detecting when that is included in a prompt.
OpenAI also has safeguards in place to block users from generating pornographic or graphically violent art, or images featuring public figures. Sandhini Agarwal, a policy researcher at OpenAI, says the new version of Dall-E has gone through even more “red teaming,” which involves researchers trying to break its guardrails. “A very explicit request for racy content, that's something that will have to go through a classifier and will be rejected,” Agarwal says. Some programmers have, of course, created open source image-generators without any restrictions.
One of the most notable results of the boom in AI-generated art is the surreal imagery that has flooded social media. The following image was generated using Dall-E 3 with help from ChatGPT, using this prompt: “A vast landscape made entirely of various meats spreads out before the viewer. Tender, succulent hills of roast beef, chicken drumstick trees, bacon rivers, and ham boulders create a surreal, yet appetizing scene. The sky is adorned with a pepperoni sun and salami clouds.”
