If you’re a content material creator trying to make your movies extra partaking then you know the way essential b-roll footage is. However discovering the appropriate B-roll footage to enhance your authentic content material could be time-consuming and irritating. However what if I advised you there’s a method to automate this course of utilizing AI? In the event you want to study from a video tutorial as an alternative here’s a video information
In right now’s information, I’ll present you methods to seamlessly use AI know-how to rapidly discover and incorporate the proper B-roll, saving you time and enhancing your movies effortlessly. First let’s perceive what’s a B-roll.
B-roll refers to supplementary footage that’s intercut with the principle footage in a video. It enhances the storytelling by offering further context, supporting the narrative, or illustrating factors being made within the main footage (often called A-roll). B-roll can embody quite a lot of clips, comparable to cutaways, transitions, or background photographs, that assist to create a extra partaking and visually interesting video. For content material creators, efficient use of B-roll can considerably elevate the manufacturing high quality, making movies extra dynamic and charming for the viewers.
The method of figuring out B-roll for a video entails both capturing them individually or selecting related footage from on-line sources comparable to inventory suppliers like Pexels and many others. manually. So let’s now attempt to automate this course of
This AI B-Roll generator is an open-source instrument accessible on GitHub. It makes use of AI to automate the method of discovering and integrating B-roll footage into your movies. Beneath is the workflow we observe so as to add b-roll to our video
- Generate captions for enter video
- Establish key phrases which signify these captions
- Instance : AI is used to automate most of the human duties -> Automation
3. Fetch pexels movies for these key phrases to make use of as b-roll movies
4. Sew collectively the b-roll movies with authentic video
Let’s do these step-by-step
First, you’ll want to put in the required dependencies. These embody:
- Whisper Library: Used to generate captions in your audio.
- Pytube: Used to obtain movies from YouTube.
- OpenAI: Used for AI-related duties, together with key phrase era.
- MoviePy: Used for video enhancing.
We begin by downloading the enter video utilizing Pytube. You may both enter a YouTube URL or add a video on to Google Colab. We will modify video_url from the under code to obtain any video of our alternative. The video might be obtain to file named video.mp4
from pytube import YouTube
import os# Perform to obtain a YouTube video
def download_youtube_video(url, output_path='.', filename=None):
attempt:
# Create a YouTube object
yt = YouTube(url)
# Get the best decision stream accessible
video_stream = yt.streams.get_highest_resolution()
# Obtain the video with the desired filename
downloaded_file_path = video_stream.obtain(output_path=output_path, filename=filename)
# If a filename is specified, rename the file
if filename:
base, ext = os.path.splitext(downloaded_file_path)
new_file_path = os.path.be a part of(output_path, f"{filename}")
os.rename(downloaded_file_path, new_file_path)
print(f"Downloaded and renamed to: {new_file_path}")
else:
print(f"Downloaded: {downloaded_file_path}")
besides Exception as e:
print(f"An error occurred: {e}")
# Instance utilization
if __name__ == "__main__":
# URL of the YouTube video to be downloaded
video_url = 'https://www.youtube.com/watch?v=8ZyShHwF_g0'
# Output path the place the video might be saved
output_path = '.'
# Obtain the video
download_youtube_video(video_url, output_path, filename="video.mp4")
Subsequent, we extract the audio from the video utilizing FFmpeg and move it to the Whisper library to generate captions utilizing the under command. If we’re operating the code in Google colab, ffmpeg is put in by default and therefore there isn’t any want to put in it individually. We then load whisper medium and run by way of our audio to generate the captions
!ffmpeg -i video.mp4 -ab 160k -ac 2 -ar 44100 -vn audio.wavimport whisper
# Load the mannequin
mannequin = whisper.load_model("medium")
consequence = mannequin.transcribe("audio.wav")
As soon as we’ve got the captions, we divide them into teams of 20 sentences. We do that as we noticed that if we’ve got loads sentences the key phrases usually are not recognized for each sentence. As soon as the division is full we move it to the OpenAI API to generate related key phrases for every sentence in a bunch. These key phrases signify the principle concepts within the captions and are used to search out matching B-roll footage.
segments = consequence["segments"]
extracted_data = [{'start': item['start'], 'finish': merchandise['end'], 'textual content': merchandise['text']} for merchandise in segments]
information = [x["text"] for x in extracted_data]def split_array(arr, max_size=20):
# Listing to retailer the cut up arrays
consequence = []
# Iterate over the array in chunks of measurement max_size
for i in vary(0, len(arr), max_size):
consequence.append(arr[i:i + max_size])
return consequence
# Instance utilization
my_array = record(vary(100)) # Instance array with 100 parts
split_arrays = split_array(information, max_size=20)
Now every of those group of 20 sentences is handed to a immediate as could be seen within the code under to establish visible key phrases which can be utilized to fetch related pexels movies. Lastly we merge the key phrases from every group to get the entire broll information which comprises key phrases for every sentence of enter video.
from openai import OpenAI
import jsonOPENAI_API_KEY = "openai-api-key"
broll_info = []
os.environ["OPENAI_API_KEY"] = OPENAI_API_KEY
consumer = OpenAI(
api_key=OPENAI_API_KEY,
)
for i, x in enumerate(split_arrays):
immediate = """This can be a transcript from a shorts video with 20 sublists. Every sublist represents a section of the dialog. Your job is to establish a key phrase from every sublist that can be utilized to seek for related b-roll footage. B-roll footage ought to complement the dialog subjects and must be recognized such that it may give related outcomes when searched in pexels api. Please present one key phrase per sublist. By no means skip any sublist and at all times give so as i.e from 0 to 19. Want output with key phrase and record index. Strictly give jsonnn**Enter**nn"""+str(x)+"""nn**Output format**nn[{"k": keyword1, "i":0},{"k":keyword2, "i":1}]"""
chat_completion = consumer.chat.completions.create(
messages=[
{
"role": "user",
"content": prompt,
}
],
mannequin="gpt-4o",
)
broll_data = chat_completion.selections[0].message.content material
print("Information", broll_data)
attempt:
broll_data = json.hundreds(broll_data)
besides:
broll_data = broll_data.cut up('```json')[1].cut up('```')[0].exchange('n', '')
broll_data = json.hundreds(broll_data)
broll_data = [{"k":x["k"], "i":20*i+x["i"]} for x in broll_data]
broll_info.prolong(broll_data)
Now that we’ve got key phrase for every sentence of the dialog in a video, we are going to choose 50% of the key phrases for fetching B-roll movies. This fashion 50% of the unique video might be lined with b-rolls. Primarily based on requirement you possibly can select a better or decrease proportion of b-rolls.
import random
num_to_select = int(len(broll_info) * 0.5)
enumerated_list = record(enumerate(broll_info))
selected_with_indices = random.pattern(enumerated_list, num_to_select)
selected_elements = [elem for index, elem in selected_with_indices]
selected_indices = [index for index, elem in selected_with_indices]
for x in selected_indices:
aspect = broll_info[x]
extracted_data[x]["video"] = fetch_pexels_video(aspect["k"])
With the key phrases in hand, we use the Pexels API to fetch related movies. First you’ll want to create an account and Pexels and from documentation want to repeat the Pexels api key and paste within the under code. If we don’t discover any movies for a key phrase we return “Invalid key phrase”
import requestsPEXELS_API_KEY = "pexels-api-key"
def fetch_pexels_video(key phrase, orientation="panorama"):
url = f"https://api.pexels.com/movies/search?question={key phrase}&orientation={orientation}&measurement=medium"
headers = {
"Authorization": PEXELS_API_KEY
}
response = requests.get(url, headers=headers)
information = response.json()
if information['total_results'] > 0:
video_info = information['videos'][0]
video_url = video_info['video_files'][0]['link']
thumbnail_url = video_info['image']
video_url = information['videos'][0]['video_files'][0]['link']
return {'video': video_url, 'thumbnail': thumbnail_url}
else:
return "Invalid key phrase"
Now that we’ve got all of the content material available, we are able to sew all the things collectively. We iterate over all of the sentences within the captions, creating clips for every sentence. If a sentence is chosen for a B-roll, we incorporate the corresponding B-roll video; in any other case, we retain the unique clip. The B-roll clips are resized and trimmed to match the period of the respective sentences.
Now we add the audio again to the created video above utilizing concatenate_clips_with_audio operate to sync each and generate the ultimate video.
import os
import requests
from moviepy.editor import VideoFileClip, concatenate_videoclips, concatenate_audioclips
from tempfile import TemporaryDirectory
from moviepy.video.fx.all import resizedef download_video(url, temp_dir):
local_filename = os.path.be a part of(temp_dir, url.cut up('/')[-1])
with requests.get(url, stream=True) as r:
r.raise_for_status()
with open(local_filename, 'wb') as f:
for chunk in r.iter_content(chunk_size=8192):
f.write(chunk)
return local_filename
def process_broll_clip(b_roll_clip, segment_duration, original_audio, begin):
b_roll_duration = b_roll_clip.period
if b_roll_duration < segment_duration:
num_loops = int(segment_duration / b_roll_duration) + 1
b_roll_clip = concatenate_videoclips([b_roll_clip] * num_loops)
b_roll_clip = b_roll_clip.subclip(0, segment_duration)
else:
b_roll_clip = b_roll_clip.subclip(0, segment_duration)
b_roll_clip = resize(b_roll_clip, newsize=(original_clip.w, original_clip.h))
# Set audio from the unique video to the b-roll clip
b_roll_clip = b_roll_clip.set_audio(original_audio.subclip(begin, begin + segment_duration))
return b_roll_clip
def concatenate_clips_with_audio(clips):
audio_clips = [clip.audio for clip in clips if clip.audio is not None]
video_clips = [clip for clip in clips]
final_video = concatenate_videoclips(video_clips, methodology="compose")
if audio_clips:
final_audio = concatenate_audioclips(audio_clips)
final_video = final_video.set_audio(final_audio)
return final_video
# Load the unique video
original_video_path = 'video.mp4'
original_video = VideoFileClip(original_video_path)
original_audio = original_video.audio
with TemporaryDirectory() as temp_dir:
final_clips = []
for section in extracted_data:
begin = section['start']
finish = section['end']
segment_duration = finish - begin
original_clip = original_video.subclip(begin, finish)
if 'video' in section and section["video"] != "Invalid key phrase":
print("Phase", section)
b_roll_video_url = section['video']['video']
b_roll_video_path = download_video(b_roll_video_url, temp_dir)
b_roll_clip = VideoFileClip(b_roll_video_path)
b_roll_clip = process_broll_clip(b_roll_clip, segment_duration, original_audio, begin)
final_clips.append(b_roll_clip)
else:
final_clips.append(original_clip)
final_video = concatenate_clips_with_audio(final_clips)
final_video.write_videofile('final_video_with_broll.mp4', audio_codec='aac')
Beneath is a demo video with AI b-roll added. Right here is the unique video https://github.com/Anil-matcha/AI-B-roll/blob/main/video.mp4
Utilizing AI to automate the method of discovering and incorporating B-roll footage can prevent a major quantity of effort and time. This information confirmed you methods to use the AI B-Roll generator to reinforce your movies effortlessly. To get entry to the entire code, try the GitHub repository linked under.
Related Hyperlinks: