The video editing workflow will change forever with all these new AI tools for Premiere Pro in their arsenal.
Highlights:
- Adobe announced a set of generative AI features within their video editing platform Premiere Pro.
- These include object addition, object removal and generative video extension using their new AI video model for Firefly.
- They are also exploring ways to use third-party models from OpenAI, Runway and Pika Labs within Premiere Pro.
Adobe Premiere Pro’s Upcoming AI Upgrades
Adobe announced new generative AI tools within Adobe Premiere Pro, which will be available for users starting May 2024.
New generative AI tools for Adobe Premiere Pro will enable editors to add or remove objects and extend existing videos. These tools will work with a new video AI model developed for Firefly.
All videos generated using Premiere Pro will prominently indicate the AI model used for production, if any. This is how Adobe will ensure content credentials for every generated video.
In addition to these video innovations, Adobe showcased advancements in audio workflows, including interactive fade handles, essential sound badges categorizing audio elements, effect badges, and a redesigned waveform display in the timeline.
But let’s just focus on the 3 main new AI features first!
1) Object Addition
This Object Addition feature helps to transform footage with AI-generated elements. Simply, select an area in the video, and input a text prompt on what we have to add in that area. In the below example by Adobe, more diamonds are added to the suitcase.
We can add scenery or swap out props, like turning a trash can into a mailbox or inserting a scenic view of a city in the background.
2) Object Removal
The Object Removal feature helps to quickly and easily eliminate unwanted objects from the clips. We have to select the specific object from the video and click ‘Remove Object’. In the below example, the Cafeteria signboard is removed.
Users can remove boom mics, light stands, logos, or license plates to avoid branding concerns hassle-free.
3) Generative Extend
Users can easily extend clip length by dragging to create seamless transitions or add pacing to videos. In the below example, we extended the clip by 1 second using Generative Extend.
It also lets users extend silent sections to create an ambient room tone for smoothing out edits.
“Adobe is reimagining every step of video creation and production workflow to give creators new power and flexibility to realize their vision.By bringing generative AI innovations deep into core Premiere Pro workflows, we are solving real pain points that video editors experience every day, while giving them more space to focus on their craft.”
Ashley Still, Senior Vice President, Creative Product Group, Adobe
The newly introduced Generative Extend feature expands video duration by seamlessly adding frames, enhancing editing precision and facilitating smoother transitions between footage segments.
How Will it Work with Third-Party Video Models?
While Adobe’s new Video model is always there to do all the generative AI work, they are also bringing third-party generative AI models in future to give more choices to video editors.
OpenAI’s text-to-video tool SORA will help generate B-rolls for any scene. SORA can generate multiple videos based on a simple text prompt. Users can then choose to add any video variation to their timeline as per their preferences.
While OpenAI showcased its Sora model creating lifelike videos from textual prompts, the technology remains unreleased to the public, with no specified timeline for its availability. Adobe, who presented a demo of Sora integrated into Premiere Pro for video generation, referred to it as an “experiment” and “early exploration,” also without providing a release schedule.
There is also Pika. Pika will help Generative Extend to make the scene flow better. Using video generation provided by Pika, users can extend scenes to enhance videos and improve transitions.
Users can also generate videos and add them to their timelines using Runway AI’s Gen2 model. The working is similar to SORA wherein a simple text prompt is provided Runway generates multiple videos and users can select the best choice.
They also mentioned that the revenue-sharing scheme for third-party AI tools is still in process. However, users will receive alerts when they utilize these third-party AI-based tools, thus ensuring safety and transparency.
Conclusion
Adobe’s latest generative AI tools in Adobe Premiere Pro mark significant progress in the world of video editing. With more tools and more models coming soon, a lot is changing in the future!
