DALL-E 3 Inpainting Update: New Features coming to their Editor

Looks like OpenAI is moving too fast with all the new updates and technologies they are revealing! Recently, they have been working on Voice Engine to clone voices, and now the new features coming to DALL-E 3 Edtior Interface.

Highlights:

• OpenAI unveiled new features to the DALL-E 3 Editor Interface, improving Inpainting capabilities.
• Allows to update ChatGPT-generated images; can also add, remove and update parts of the generated image.
• Comes with a few limitations which may be resolved soon.

DALL-E 3 Editor Interface Update

The latest update to OpenAI’s help article for DALL-E 3 Editor Interface revealed that inpainting features are coming to their AI Image Tool.

Using the DALL·E editor interface, we can now edit an image by selecting a specific area and then prompting about the modifications we need. We can also just use the prompting in the conversational panel, without utilizing the selection tool.

With the aid of these upgrades for inpainting and outpainting, the interface can now modify photographs more creatively and with greater control.

Currently, the updated Editor Interface feature is being rolled out to Desktop users at the moment. OpenAI plans to release advanced features to smartphones, tablets, etc soon.

Desktop users who wish to access this tool can perform either of the following steps:

• Editing a Generated Image: Generate an image using GPT-4’s DALL-E 3, and after clicking on it, we will be taken to the image editor interface as shown below:

• Editing from a Blank Canvas: We can also choose to generate and edit an image from scratch. Note that we will need credits to generate and edit images here. Each prompt you give will cost a credit.

A ChatGPT Plus subscription which offers DALL-E 3 through GPT-4. though mobile users aren’t able to use sophisticated editing functions like outpainting, they can still inpaint photographs by selecting “Edit” when they’ve already created or uploaded an image.

Exploring the Inpainting in DALL-3

A variety of choices are available in the editor interface to help pinpoint areas of the created image that we would like to improve. Let’s explore these features in detail:

The Editor Interface offers a selection tool on the top right corner of the editor. We can use it to select/highlight any parts of the generated image you wish to edit.

We can adjust the selection tool’s size in the upper-left corner of the editor to make it easier to choose the area that has to be edited. To improve the outcome, it is advisable to choose a sizable region surrounding the section you want to alter.

The Undo and Redo buttons above the image can also be used to undo and redo selections. Alternatively, you can choose Clear Selection to start over from scratch.

The below video from Tibor Blaho, one of the few people who got access to the updated interface:

As we can see, parts of the generated image can be updated, deleted, and added to using the editor interface.

1) Adding an Object

To add an object to the generated image simply, we can give the prompt “add <desired object>’”. And the editor will do the rest.

For example, the editor successfully adds cherry blossoms to highlighted parts of a generated image when given the prompt “Add cherry blossoms”.

2) Removing an Object

The editor interface can also remove an object from parts of a generated image. All we have to do is simply give the command “remove ‘your desired object’”.

In the image below, we can see that the highlighted birds have been removed by the editor interface when given the prompt “remove birds”.

3) Updating an Object

We can also update parts of a generated image with the help of the editor interface. In the example image below, the kitten’s face was highlighted and the prompt “change the cat’s expression to happy” was given. The result was amazing:

Make sure to click on the Save button in the upper-right corner of the editor, as currently, expanded photos are not automatically stored. Users must be sure to frequently download the incremental work to prevent losing any data.

We can also just use prompts to edit the images, without the need for highlighting specific parts in them. Simply include the precise location of the edit in the prompt, or just apply it to the desired part of the image.

OpenAI also recommends using the inpainting only for a relatively smaller area in the original image, and using muted colors if inpainting in the corners.

Are there any Limitations?

OpenAI has accepted some of the limitations of the Editor Interface feature and has asked users to keep them in mind.

Firstly, users can’t yet entirely view the extended image in their history or save it to a collection. This is quite a drawback as till now ChatGPT has stored all data of previous conversations entirely in the left side “History” panel, but not yet with edited images.

OpenAI has stated that they will come up with a fix to this in the days to come.

Secondly, they also stated that users can experience freezes in their browsers while editing and working with large images.

They didn’t provide any upcoming solution to this problem instead they advised users to download the edited images instantly, to avoid losing track of their work.

Whenever a new technology arrives, it’s bound to have bugs and shortcomings, so we can be hassle-free and expect OpenAI to come up with solutions to these problems soon.

The Future of Editing Images With AI

All things considered, the use of AI for image editing—whether with DALL-E or other models—shows promise for developing more potent and user-friendly tools that will increase creative possibilities.

To make them even more appropriate for image editing tasks, future updates of DALL-E might concentrate on producing more realistic photos with more attention to detail, texture, and lighting.

Users may be able to edit several parts of the image, such as object placement, size, orientation, and style, with greater control over the image-generating process thanks to AI models.

More sophisticated AI models might be able to comprehend the semantic meaning of textual descriptions more fully, which would improve their ability to analyze user input accurately and produce images that more closely represent the intended idea.

It might become possible to combine AI image editing capabilities with currently available image editing software so that users could take advantage of AI support in well-known programs like GIMP or Adobe Photoshop.

However, when looked at the other side of the coin, an advanced editing tool like OpenAI’s Editor Interface and Midjourney may give rise to more such advanced tools in the future that can fully encapsulate the editing details with enhanced natural language processing capabilities.

This raises the question of deepfakes, a highly concerning matter in the world of AI today. When such a highly powerful tool gets widespread access, it indeed raises an eyebrow regarding ethics and safety for the society.

Conclusion

All these new improvements to OpenAI’s DALL-E 3 Editor Interface are here to disrupt the image editing landscape. The tool lays a strong foundation for more advanced image editing tools in the days to come. Only time will tell, how the tool performs in the days to come!