How to prepare audio for AI lip sync

This guide prepares the audio input for an existing-video lip-sync job. It does not report generation tests or promise to fix a rendered mouth. The timing examples and filenames are illustrative. Current Magic Hour routes and API documentation were checked on October 12, 2026.

For workflow, pricing and setup, see Magic Hour Lip Sync: workflow and troubleshooting.

Separate the original recording from the replacement track

The original audio is what was recorded with the picture. It helps you understand the performance, existing pauses and where a sentence begins. The replacement audio is the approved speech or vocal you want the visible person to perform. Keep both with clear names; do not overwrite the original while preparing the new file.

If you only need to change volume or clean the existing soundtrack, decide whether ordinary audio editing meets the goal. If words, pronunciation or delivery change and the visible mouth must follow them, prepare the new track for a lip-sync trial. For translation, finish and approve the translated wording and pronunciation before this stage. Lip Sync accepts a supplied audio track; uploading source speech is not a request to translate it or clone its voice.

Keep music and ambience as separate finishing elements where possible. In a dialogue replacement, the original spoken line is a timing reference rather than a second voice to layer under the new line. Plan how the final edit will avoid doubled speech. If you need to preserve background sound, obtain or prepare an authorized background track and review that mix separately.

1. Approve the words and the visible speaker

Read the exact sentence with the person responsible for facts and pronunciation. Check names, dates, prices, numbers, abbreviations and the intended language. A clean recording with the wrong number is still the wrong input. Correct the script or performance before attempting to animate the mouth; lip alignment cannot establish whether the statement is true.

Confirm you are authorized to use the footage and supplied voice for this purpose. Choose a segment with a visible speaker and record who the voice is meant to represent. For a simple first trial, use one person’s clear line. If an interview has two people speaking at once, isolate an appropriate turn or redesign the edit; do not ask a single replacement track to resolve an ambiguous speaker assignment.

2. Listen to the untouched audio and mark defects

Listen from beginning to end using headphones at a comfortable level, then check the important words through the speakers likely to be used for review. Mark timestamps for rumble, hiss, traffic, plosive bursts, crackling, echo, competing speech and abrupt level changes. Also mark unclear word endings and rushed phrases. This log tells you what needs attention without treating every quiet pause as a defect.

Check the waveform and recording meters for clipped peaks, but make the listening decision as well. A loud-looking waveform alone is not proof that speech is unusable. If a consonant sounds broken or a vowel crackles in the original, compare another take if available. Preserve a short untreated reference around the problem so you can judge whether processing helps or merely sounds different.

3. Clean cautiously, then compare

Try the smallest treatment that addresses the marked problem. For steady noise, a noise-reduction tool may help; irregular interference or speech buried under noise may remain difficult. Audacity’s official manual describes profiling noise from a noise-only region and warns that stronger reduction can damage the wanted audio. Preview the difficult words before applying a treatment to the whole track.

Use the Audacity Noise Reduction manual for that tool’s actual controls. Compare processed and original speech at a similar listening level. If cleanup makes the voice watery, removes consonants or swallows word endings, reduce the processing, use another take or rerecord. Do not sacrifice intelligibility to make every gap sound perfectly silent. In Audacity, select a region containing only the unwanted noise. Open Effect → Noise Removal and Repair → Noise Reduction, then choose Get Noise Profile. Select the speech region, reopen Noise Reduction and use Preview before applying it. Listen to Residue as well: recognizable words or syllables there indicate that wanted speech may be removed. Use the lowest effective treatment and compare the exported result with the untouched recording.

Clipping needs a separate decision. Lowering an already distorted recording’s playback level does not supply the lost signal. Audacity’s Clip Fix attempts reconstruction and is described as likely to help only lightly clipped audio. Treat any repair as a candidate to inspect, not a restored original. If important words remain distorted, a clean authorized rerecording may be the more direct route.

See the official Clip Fix limitations. Avoid copying arbitrary noise-reduction, compression or loudness numbers from another project. Record what you changed and confirm the final export sounds clear without clipping. This article sets no universal sample rate, loudness or effect preset for every tool and recording.

4. Fit the delivery to the intended picture

Place the proposed voice track beside the selected video in an editor. Check whether the first word, important phrases and final word fit the usable picture. Match the emotional delivery to the intended scene: a calm instructional line and an excited sales line can require different pauses even when they contain the same words. Listen to the performance rather than judging only total file length.

When a line is too long, consider rewriting or rerecording it, choosing a longer usable video segment, or splitting at a natural phrase boundary. Do not compress all pauses merely to force a long script into a short shot. If you change speaking speed, listen for altered articulation and unnatural delivery, then check the timeline again. An audio edit that moves a phrase also changes when its mouth movement must occur.

5. Trim without losing the sentence

Remove unintended material before the chosen opening and after the intended ending. Leave the actual beginning and ending of words intact. Retain purposeful breaths and pauses when they belong to the delivery. Inspect edit boundaries for clicks and listen across them; a waveform selection that looks neat can still cut a sound.

Keep original-video time and exported-segment time distinct. If you extract seconds 32–40 from a longer recording, the new eight-second file starts at its own zero. Prepare the replacement audio for that exported eight-second segment and record the original range for reference. This makes the upload pair understandable without guessing which offset applies. Do not assume that a clip-selection field also shifts a separate audio track by the same amount.

An illustrative eight-second handoff

Suppose an authorized talking-head video has a usable eight-second segment at original time 32–40. The planned replacement line lasts within that segment. The following is a planning example, not a measured recording or generated result. Export the selected picture as speaker-segment-v1.mp4 and the approved replacement as replacement-voice-v2.wav, both covering the intended eight-second handoff.

Listen to the exact exported files together before upload. If the line ends past the picture, revise the wording, recording or chosen segment rather than appending silence and calling the pair matched. Equal file durations do not prove that phrase timing or delivery is appropriate; they simply make one boundary easier to verify. Save the original range and all edits with the pair.

Music, singing and overlapping voices

For a spoken line with music underneath, keep a clean approved speech version when possible and add the music during final assembly. For singing, prefer an authorized vocal stem for a focused trial if one exists. An isolated stem still needs listening: separation can leave instrumental bleed or remove syllables. The aim is an intelligible intended vocal, not a claim that every mixed song is unsupported or that every stem will work.

When two voices overlap, decide who should visibly speak at each moment. Use separate clear turns or an edit that preserves that assignment. Keep untouched mixed audio for reference and final production needs; do not claim that removing one voice perfectly is always possible. If the necessary performance cannot be separated clearly, obtain another take or simplify the segment before generation.

Export the prepared version and check it again

Choose an accepted audio format for the route you will use. Magic Hour’s current Lip Sync documentation lists WAV and MP3 among its supported API formats. Do not assume that renaming an extension converts a file. Export from your editor, reopen the actual output and verify that it plays, contains the correct track and covers the intended segment. Keep the untouched source separately.

Check the first and last words, the marked difficult sounds, the pauses and any level changes after export. Confirm you did not accidentally export only a selection, an instrumental track or the original dialogue instead of the replacement. Use descriptive version names, such as replacement-voice-v2.wav, and attach the approved script to the handoff. A screenshot of an editor session does not identify the file another operator uploads.

Upload the matching pair to Lip Sync

Open Magic Hour’s Lip Sync creator, which currently exposes Add Video and Add Audio. Upload the confirmed picture segment and prepared replacement track, inspect the selected mode and available controls, and submit only after the pair matches your handoff record. This workflow uses existing video plus supplied audio, rather than a portrait-only animation.

For the API equivalent, POST /v1/lip-sync accepts assets.video_source, assets.video_file_path and assets.audio_file_path, with start_seconds and end_seconds selecting the video clip. With the pretrimmed eight-second example, set video_source to file, start_seconds to 0 and end_seconds to 8, using your real uploaded file paths. This example avoids relying on an assumed separate audio-offset control. Read the current linked reference for other settings.

Keep a compact audio-preparation receipt

Record the untouched source name, exported audio version, approved script, source-video range, exported segment duration, cleanup edits and the reviewer’s unresolved notes. Include the intended speaker and any music/ambience plan. Write specific findings, such as “final word intact after trim” or “traffic noise remains under second phrase,” instead of a general “cleaned.” Unknowns should remain visible to the next operator.

After a trial, review the downloaded result with sound around word starts, pauses and sentence endings. A usable prepared track is an input gate; it does not establish that teeth, identity or timing in the output are acceptable. This page owns source listening, cleanup, timing and the upload pair. The separate lip-sync troubleshooting guide owns diagnosis of defects that remain in the generated video.

Official sources and product references

https://docs.magichour.ai/tools/video/lip-sync

https://docs.magichour.ai/api-reference/video-projects/lip-sync