Free to try — 5 transcriptions, clips up to 2 minutesPro $19.99/year

Accuracy

Why auto captions get words wrong, and what actually helps

Auto captions miss words for a few predictable reasons, like accents, music, names and crosstalk. What causes each one, and what really improves accuracy.

By Arjesh Vadadoriya · · 5 min read

You ran a clip through auto captions and most of it came back right. Then you hit “Kay Tea” where you said “KT”, a brand name turned into two random words, and a line in the middle that bears no relation to anything you said. It’s tempting to blame the app. Most of the time, though, the errors follow a few predictable patterns, and once you know them you can stop half of them before you record.

The short answer. Auto captions get words wrong when the audio is ambiguous: music or noise under the voice, two people talking at once, a mic that’s too far away, or words the model rarely sees, like names and slang. The model always picks the most likely word it knows, so unclear audio turns into confident mistakes. Cleaner audio fixes most of them; a quick read-through fixes the rest.

How speech recognition decides what you said

A speech-to-text model doesn’t look words up in a dictionary as it hears them. It turns the audio into a pattern of sound and then asks which string of words most likely produced that pattern, given everything it learned from the speech it was trained on.

That second part matters. The model leans on context and on how common each word is. When the sound is clear, the audio wins and you get what you said. When the sound is muddy, the model’s sense of what people usually say takes over. That’s why errors are rarely gibberish. They’re real, common words that sound roughly like what you said and fit the sentence well enough to look right at a glance.

So the useful question for any wrong caption is: what made the audio ambiguous at that point?

The five usual causes

Background music

Music and speech share much of the same frequency range, and a song with vocals is worse still, because it’s literally more speech. When a track sits under your voice, the model hears two sets of sounds and has to separate them. It often gets most words right and then drops or swaps the quiet ones: “the”, “to”, word endings, the last word before a beat drop.

The easiest fix is to keep music off the clip you caption. On TikTok and Instagram you can add a sound inside the app after you upload (at the time of writing, both still let you do this), so the file you transcribe can stay voice only.

Distance and echo

A phone held at arm’s length in a kitchen records you plus every hard surface in the room. Echo smears the end of one word into the start of the next, and the model hears a blur where there were two clean words. The closer the mic is to your mouth, the more of the recording is your voice and the less is the room.

Accents, dialects and fast speech

Models learn from the speech they were trained on, and some accents and dialects are better represented in that material than others. The less a model has heard a way of speaking, the more often it reaches for a more familiar word. Speaking fast compounds it, because short words get swallowed and boundaries between words blur.

This isn’t a reason to change how you talk. It’s a reason to read the transcript more carefully if your accent, or your guest’s, tends to trip up voice assistants.

Names, brands and jargon

Proper nouns trip up speech recognition more reliably than anything else here, for the reason above. A model has heard “Katie” thousands of times and your product name perhaps never. When the sound is close enough, the familiar word wins. Niche terms, product codes, usernames and new slang all fail the same way.

You can’t really record your way out of this one. You can say the name clearly and not rush it, but expect to check every name by hand.

People talking over each other

Interviews, duets and reaction videos are the hardest case. When two voices overlap, the model gets a mix of both and usually follows the louder one, dropping or garbling the other. It can also stitch half a sentence from each speaker into one line.

Mixed languages and wrong detection

Most modern caption apps work out the language from the audio. That works well when a clip is in one language. When you switch mid-sentence, the transcript usually follows what was actually said, but words at the switch point are where errors cluster. If a clip opens with a few seconds in a different language from the rest, detection can also lean the wrong way. The languages page covers how detection and mixed-language clips behave in Capster.

What actually improves accuracy

Ranked roughly by how much difference each makes:

  1. Get the mic closer. A clip-on mic, or the phone held near your face, usually does more than any other change.
  2. Record voice and music separately. Caption the voice-only file, add the music later.
  3. Pick a quieter, softer room. Curtains, a sofa and a rug cut echo. Bathrooms and empty rooms are the worst.
  4. One speaker at a time. Leave a beat between turns in an interview.
  5. Say names and numbers clearly. Slow down a touch on the words you most need to be right.
  6. Trim dead air. Long silences and music-only stretches at either end are where invented text tends to appear.

What doesn’t help much: boosting the volume afterwards (it raises noise too), running the same file again, or exporting at a higher quality. Transcription only cares about the sound.

Proofread before you burn it in

Once captions are burned into the video, a typo is permanent in every copy you post. So a read-through is worth the minute it takes, and you don’t need to check everything equally. Focus on:

Check Why
Every name and brand Rare words lose to familiar ones
Numbers, prices and dates Easy to mishear, costly to get wrong
The first two lines Your hook, read by everyone who stops scrolling
Lines over music or crosstalk Where swapped words cluster
Any line during silence May be text nobody said

In Capster you tap any caption to correct a word before you export, and the step-by-step guide to adding captions on iPhone shows where that fits. If you’re still choosing a tool, the auto subtitle app guide covers what to look for, including how easy each app makes that correction step.

Get Capster for iPhone

Questions people ask

Will auto captions get better if I transcribe the same clip again?

Usually not by much. The model hears the same audio and makes mostly the same guesses, so a second pass tends to repeat the same mistakes. You'll get a better result by fixing the recording, such as moving the mic closer or removing the music, or by correcting the few wrong words by hand, which takes less time than another round.

Why do auto captions write words nobody said?

Speech recognition always produces its most likely guess, even when the audio is unclear. During long silences, applause or music-only stretches, some models fill the gap with a plausible phrase rather than leaving it empty. Trim dead air at the start and end of a clip and read any caption that appears where nobody is talking.

Does turning up the volume make captions more accurate?

Only if the speech was very quiet to begin with. Boosting a recording raises the voice and the background noise by the same amount, so the balance between them stays the same. What helps is a better ratio of voice to everything else, and that comes from mic placement and a quieter room, not a louder file.

Read next

Captions that read, on every platform

Free to try — 5 transcriptions and clips up to 2 minutes. No account.

Download on the App Store

Free to try: 5 transcriptions, clips up to 2 minutes, exports carry a small Capster badge. iPhone, iOS 16.0+. No account.