AI video dubbing and translation, free.

At a glance

How to dub a video into another language for free

The soundtrack is decoded in the browser and handed to Whisper, which returns the speech as timed phrases, translated into English if you asked for that. Fragments are joined into natural lines, but never across a pause or a full stop. Whisper only translates into English, so a dub into Spanish, French, Italian, Portuguese or Hindi takes one more step: the English lines go through OPUS-MT, the University of Helsinki's open translation models, one sentence at a time (given two sentences at once, the model sometimes drops one). That runs on the processor at about a second a line, after a one-time download of the language's model. You then see the lines in a list with their times, and in a translated dub each one has the English it came from underneath. This is the moment to fix a name or soften a clumsy translation. Machine translation of speech is good enough to follow a video, not good enough to publish unread.

Then each line is read by Kokoro, an open neural voice, and placed where the original phrase began. A translation often runs longer than the language it came from, so a line that would overrun the start of the next one is read again slightly faster, up to 1.3×, which is about as fast as a voice stays natural. Past that, the line is allowed to run on and is marked in the list, so you can shorten its wording and dub again. The original soundtrack is turned down by 18 dB under each line and left alone in between, so music, effects and the presence of the original speakers stay; that is how most documentary voice-over is mixed. Finally the new soundtrack is muxed onto the video with the picture copied bit for bit, so nothing is re-compressed.

Translating a video to English or Spanish: good results

Clear speech with modest background works best; a single presenter, an interview, a lecture, a cooking or how-to video. For a translation, choose Whisper small: its translations are noticeably better than base's, at the cost of a bigger download and more time. Pick the spoken language yourself if the video starts with music. Choose a voice that suits the speaker; Michael and Fenrir are steady male narrators, Heart and Bella the most natural female voices. If the original speech is still too present under the new voice, choose "Replace it", or separate the voice from the music first with the AI vocal remover and dub the instrumental.

What it can't do

The target languages are the six that Kokoro speaks: English, Spanish, French, Italian, Brazilian Portuguese and Hindi. German is out because Kokoro has no German voice; its Japanese and Chinese voices need a pronunciation engine that has no browser version yet. Going into a language other than English, the speech is translated twice (Whisper into English, then OPUS-MT out of it), so a subtlety lost in the first step stays lost; for a Spanish video you want in French, check the lines against the original. On a four-line how-to script, the Spanish, French, Italian and Portuguese translations were all right; the Hindi one got the meaning of one line wrong, so a Hindi dub especially needs a reader who knows the language. French has a single voice, so in a French dub every speaker shares it. Re-voicing without translating works in the same six languages: a Spanish video can be given a new Spanish voice (measured: a Spanish clip re-voiced in Spanish transcribes back with every key word). With "a different voice for each speaker" ticked, a speaker model (pyannote with WeSpeaker voiceprints, the same one behind the speaker labels on audio to text) works out who says each line, and every person gets their own voice, chosen per speaker; by default they alternate a man's and a woman's voice so two people stay easy to tell apart. Without it, one narrator reads everything. With "use each speaker's own voice" ticked, an English dub is read in voices cloned from the speakers themselves: about five seconds of each person's own lines is the reference, and a line too long for its slot is tightened at the same pitch. On a test clip of a woman's voice, the dub came out in her register (a median of 171 Hz, against about 120 for the default male voice) and Whisper read it back nearly word for word. It needs WebGPU, downloads the 560 MB cloning model once, takes about four and a half times the speech's length, and must only be used with the speakers' permission; music under the voice muddies the clone, so separate it first. Without it, the voice does not match the original speaker's voice. Either way the dub is not lip-synced: this is voice-over dubbing, as on news and documentaries, not the studio kind where actors match mouth movements. Cloud services such as ElevenLabs and Rask do voice cloning and lip-sync by uploading your video to their servers and charging by the minute. Long videos take a while: transcription runs at a fraction of the video's length on a GPU, and the voice at about the length of the speech. Trim to the part you need with the cutter or your editor first.

If you only need English subtitles, audio to text writes them directly. The SRT from this page is in the dub's language and loads into YouTube, Premiere or CapCut alongside the dub.