Let’s face it. Prompting AI-generated videos is 10x harder than prompting ChatGPT.
If you’ve ever used an AI video generator, you know how hard it is to get the exact video you want.
What’s worse, you often have to spend time waiting for your videos to finish, just to iterate on your prompt.
It’s not uncommon to spend hours just to create one video, not to mention all the credits you waste doing so.
Getting better at prompting can save you time, effort, and money, and in this guide, we’ll demystify the mystery of prompting by sharing our top 10 tips to prompting AI videos.
These tips are drawn from my experience creating 50,000 AI-generated videos and posting the best ones on TikTok, Instagram, and Youtube, where I‘ve received hundreds of millions of views.
Tip 1: Finding Inspiration
Like art, the easiest way to prompt is to steal, tastefully.
The great thing about prompting videos is that you can use both image and video prompts as inspiration.
The key is to find the prompts that others have used, and then customize them based on your vision.
One place you can browse is our explore page, which has video templates with prompts that we’ve used. The Video-to-Video section has some of our most battle-tested and intricate prompts, including fictional characters, marble statues, and art styles.
Another solid place to browse is civitai.com, an AI image repository, where you find prompts for nearly any idea you have in mind. I like to use the search feature and enter one word at a time to see what types of variations people have created.
Once you find something you like, simply copy and paste the prompt into your preferred AI video generator and try it out. You’re already off to a great start.
Tip 2: Understanding Prompt Structure
The way to think about video prompts is that they’re like a bowl of soup.
The more ingredients you throw in, the less influence each ingredient will have.
Moreover, the ingredients you put in first, the words in the beginning, have more weight than the words you put in at the end (so maybe it’s more like a cake? I’m getting hungry).
So when crafting a prompt, include the most important things you’d like to see in the beginning, and then add supporting details toward the end like the background.
Note that there are diminishing returns. Below is a diagram that shows roughly when the words you’re adding aren’t likely to be impactful:
Tip 3: Be Specific
Prompts aren’t sentient. They can’t understand what you have in your head. They can’t take lyrics or poetry and turn it into a movie. There’s no smart human on the other end deciphering your words, at least not yet.
Thus, you need to be specific. This means including anything and everything that you’d like to see, unless you’d like the AI to fill in the rest by chance (which is okay as well).
Below are categories of words you may want to include to achieve a rich and complex video:
- Subject: Person, animal, character, location, or object.
- Medium: Photo, painting, sculpture, etc.
- Artist: In the style of Studio Ghibli, Van Gogh, Greg Rutkowski, etc.
- Environment: Iindoors, outdoors, another planet, a fictional realm, etc.
- Lighting: Cinematic, bokeh, God rays, etc.
- Color: Dim, colorful, vibrant, flat, etc.
- Composition: Wide shot, close up, aerial view, etc.
Tip 4: Use a Few Quality Modifiers
If you’ve ever seen prompts for images and videos, you might have seen words like:
masterpiece, best quality, extremely detailed 8k wallpaper, ultra-detailed, high contrast, trending on artstation, award-winning, professional
Do these words matter, or are they just decorative and don’t impact the video?
The truth is somewhere in between. Some of these words do matter, but some have a negligible effect on your video.
In net, however, they generally improve the quality of your video.
The reason is that these words are often used to tag high-quality images in the training data, an artifact of websites like Artstation and DeviantArt, so by including them, you’re getting the benefit of nudging your output towards these images.
Note, each modifier has a different effect and degree of effect, so you’ll want to play with which ones you like best. Some modifiers are better suited to certain art styles as well. For example “trending on artstation” tends to work well with illustrations and digital art.
However, when it comes to using them, it’s perfectly safe to throw them into your prompt by default to get a general enhancement.
On Magic Hour, our art style templates come pre-loaded with quality modifiers, so you don’t have to worry about this.
Tip 5: Add Some Flair
There are certain words that I love including in prompts because they add they extra bit of touch that really brings the video to life.
Here are some examples:
energy swirls, aura, glowing runes, motion lines, motion blur, night, glow, particles, light trail, glowing eyes, speed of light, superpowers, ki charge
The reason is that they give your video a lifelike, post-production quality, that really makes it visually engaging. For example, I used modifiers such as “energy swirls” and “aura” to achieve the effects in this Spiderman video:
When it comes to choosing modifiers,I recommend trying them all out and seeing which ones you like. Different modifiers suit different purposes and there isn’t a right or wrong. Sometimes I even stuff them all into a prompt and see what happens (hint: it’s awesome).
Tip 6: Specify Face Directions
AI loves to render faces on the backs of peoples heads. It sucks.
The reason is because most of the training data uses front-facing images.
This issue is most painful in Video-to-Video mode, where faces on the back of people’s heads really ruins the reality of the video.
The best current workaround is to include the modifier “from behind, facing away from viewer” if a substantial part of your video includes people facing away.
Even if only half of your video or less involves people facing away, I’d still consider including this modifier, as it’s more rare for the opposite issue to occur, i.e. AI rendering the back of someone’s head on their face.
This is an area we’re working on improving, and we plan on launching some features soon that help reduce the extent of this issue.
Tip 7: Test Before Rendering
You can get a pretty accurate representation of your video by entering it into an image generator.
We recommend Civit’s Dreamshaper Image generator because that’s the default base model Magic Hour uses.
You can also generate shorter versions of the video in AI video generator.
On Magic Hour, you can also watch the video as it renders, frame-by-frame, in our Animation and Video-to-Video modes.
Note that when you use image generators, the results won’t be identical to your chosen AI video generator, as they often use different settings and models.
That being said, it’s a very fast and easy way to test, and so there’s little cost to testing with images first.
Tip 8: Store Your Favorite Prompts
I have an Apple note with all of my favorite prompts, as well as ones that I’m testing and new ideas I have. I iterate on this list almost every day, and sometimes remix past prompts that I liked.
This is a helpful way to build your own “taste” while also creating a repository that you can draw on.
Now when I make videos I just reference my note, saving me time, while allowing me to pump out high quality videos. Fortunately for you, most of these prompts are also templates in Magic Hour in our Video-to-Video section.
Tip 9: Choose the Right AI Video Generator
Each prompt is dependent on the tool you use to run it. For example, you might have a great prompt in Runway that doesn’t look as great in Pika. The reason is that each AI video generator uses different models and settings. For example, Pika is more oriented towards anime.
Therefore, you want to make sure you’re using the AI video generator that’s best for your personal style and vision.
Each AI video generator has pros and cons, and suits different styles, so I recommend experimenting with each of the free ones.
One thing I like to do is enter the same prompt on multiple AI video generators as a way to benchmark them and see which one is best for my current project.
Below are some AI video generators with a free tier that I recommend checking out:
Tip 10: Share Your Work
I highly recommend sharing your videos on social media. The main reason is to see what types of content viewers like, which can help you inform which prompts worked really well.
Then, you create more content with a similar prompt, and craft variations of it.
For example, I learned that viewers love marble statues and superheroes in Video-to-Video content, and I wouldn’t have learned that unless I posted our content regularly.
In Magic Hour, you can also submit your videos as templates. We review all of them manually, and if accepted, you get 100 free frames every time they’re used. Pretty cooly right?
Conclusion:
Prompting AI videos is really, really hard. Hopefully with these pro tips, you’re a little better at prompting and can now uplevel your prompting game. If you make any awesome content using these tips, or have any personal tips of your own, let me know at runbo@magichour.ai or on our Discord!
