Free to try — 5 transcriptions, clips up to 2 minutesPro $19.99/year

Audio

Auto captions with background music, and how to mix for them

Background music is a common reason auto captions go wrong. Why music competes with speech, and how to order and mix a clip so the captions come out clean.

By Arjesh Vadadoriya · · 5 min read

You added a track under your voiceover, ran the clip through auto captions, and the transcript came back with holes in it. Small words are missing, a few lines have swapped in words you never said, and somewhere in the middle there’s half a chorus you didn’t sing. The captions aren’t broken. The audio gave the transcription model two things to listen to, and it guessed which one you meant.

The short answer. Background music competes with your voice for the same part of the sound, so auto captions drop or swap words wherever the music is loud, and songs with vocals can leak lyrics into the transcript. The cleanest fix is to caption the voice-only clip first and add the music afterwards. If the music has to be in the file, use an instrumental track, keep it well under the voice and dip it whenever you speak.

Why music trips up speech recognition

It covers the parts of speech that carry meaning

When two sounds share the same pitch range, the louder one hides the quieter one. Audio engineers call this masking. Vowels are loud and fairly easy to pick out, but the consonants that tell “bat” from “pat” or “sit” from “fit” are quiet, short and sit in the upper middle of the range. That’s the same territory where guitars, synths, hi-hats and most vocals live.

So the model hears the vowels, loses some of the consonants and fills the gaps with its best guess. Short words like “a”, “to” and “the” go first, then word endings, then whole words at moments where the music swells.

A song with vocals is more speech

Speech models are trained to turn human voices into text. They have no reliable way to know that one voice is the person on camera and the other is a singer on a backing track. If the vocal is clear, parts of the lyrics get transcribed alongside your words, or stitched into the same line.

Music-only stretches invite invented text

An intro with music and nobody talking looks, to a model, like audio that ought to contain words. Some models fill those stretches with a plausible phrase rather than leaving them empty. It’s a common source of captions that appear over the opening shot before anyone has said anything. The earlier post on why auto captions get words wrong covers this and the other usual causes in more detail.

The best fix: caption first, add music last

The order you do things in matters more than any mixing trick. Auto captions only need your voice, and burned-in captions are fixed in the picture once you export. So:

  1. Record or edit the clip with voice only.
  2. Transcribe it, read the captions and fix any wrong words.
  3. Export with the captions burned in.
  4. Add the music afterwards, in an editor or in the app you’re posting to.

Capster transcribes the audio track of whatever clip you import, so this is how to get the cleanest result from it. The guide to adding captions to a video on iPhone walks through steps 2 and 3. At the time of writing, TikTok and Instagram both let you pick a sound for a post before you publish, so a voice-only export can get its music at the last step without a second edit.

When the music has to be in the file

Sometimes the edit is cut to the track, or the music was mixed in before you thought about captions. You can still get a usable transcript.

Pick instrumental over vocal tracks

This is the biggest single choice. An instrumental under a voiceover gives the model one voice to follow. A pop song with a lead vocal gives it two. If you like a particular song, many tracks are also released as instrumental versions.

Keep the music well under the voice

A level that sounds right with your eyes on the screen is usually too loud for transcription. Your brain fills in words from lip movement and context; the model only has the sound. A rough test: play the clip without looking and see whether you can follow every word without effort. If you have to lean in, the music is too loud.

Dip the music while you speak

Ducking means pulling the music down whenever the voice comes in and letting it rise again in the gaps. Most video editors let you set the volume of a music clip in sections, or draw a volume line that drops under speech. It keeps the track present in pauses and cuts and out of the way when the words matter.

Watch for loud moments

A beat drop, a crash or a chorus landing on top of a sentence is where words vanish. Move the sentence, or move the music, so the loud moment falls in a pause.

When the music was in the room

A café speaker, a car radio or a venue PA is the hardest case, because the music is recorded on the same track as your voice. You can’t take it out cleanly afterwards. What helps:

  • Move the mic, not the volume. Getting the phone or a clip-on mic closer to your mouth raises your voice against the room. Turning the file up later raises both by the same amount.
  • Turn the music down or move away from the speaker. Even a few metres changes the balance.
  • Try a voice-focused audio mode. At the time of writing, recent iPhones offer an Audio Mix option when editing a video in Photos, with settings that bring the voice forward over background sound. Results vary with how loud the music was.

Check the lines most likely to be wrong

You don’t need to proofread every word equally. Music causes errors in predictable places:

Where to look What tends to go wrong
Lines over a chorus or beat drop Dropped or swapped words
Any caption during an intro or outro Text nobody said
Moments a vocal comes in Lyrics mixed into your line
Quiet asides and the last word of a sentence Missing words

In Capster you tap a caption to fix a word before you export. If you’re comparing tools, the auto subtitle app guide covers how easy different apps make that correction step, which counts for more on music-heavy clips than on quiet ones.

Get Capster for iPhone

Questions people ask

Can auto captions tell the difference between song lyrics and someone speaking?

Not reliably. A speech model is trained to turn any human voice into words, and a sung lyric is still a human voice. If the vocal in the track is clear enough, it will often turn up in your captions mixed in with what you said. Instrumental music avoids the problem entirely, which is why it is the safer choice under talking.

Does a voice isolation or noise reduction tool fix music under speech?

It helps, sometimes a lot, but it rarely removes music cleanly. These tools estimate which parts of the sound are voice and suppress the rest, and music overlaps speech in pitch and rhythm far more than hiss or traffic does. Expect some artefacts and a few missed words. Recording the voice without music is still the cleaner route.

Should I add music before or after I burn in the captions?

After, if you can. Captions are timed against the voice, and the voice is easiest to hear on its own. Caption the voice-only clip, export it with the captions burned in, then add the music in your editor or in the platform's sound picker. The captions are already fixed in the picture, so the music can't affect them.

Read next

Captions that read, on every platform

Free to try — 5 transcriptions and clips up to 2 minutes. No account.

Download on the App Store

Free to try: 5 transcriptions, clips up to 2 minutes, exports carry a small Capster badge. iPhone, iOS 16.0+. No account.