15

I am currently using ffmpeg to convert videos in various formats to flv files. One request has also come up and that is to get closed caption info out o the file as well. Does anyone have any experience with this or know it can even be done. I don't see any options for it but thought I would ask and see.

7 Answers 7

41

If anyone, like me, ends up on this thread, here's a bit more detailed explanation on ffmpeg command that worked for me.

ffmpeg -f lavfi -i movie=input.ts[out+subcc]  -map 0:1  output.srt

There seems a hard requirement on source to be of mpegts format (file extension .ts). Otherwise the lavfi filter does not seem to work. The spec out+subcc forces ffmpeg to treat closed captions (which are embedded into frame data) as separate stream. Later -map 0:1 makes ffmpeg map only that stream and discard everything else. Result is saved to output.srt. Depending on your input the mapping might be different. One easy way to figure out the closed captions mapping is to run ffprobe command, like so

$ ffprobe -f lavfi -i movie=input.ts[out+subcc]
ffprobe version N-79653-g4efd3ec Copyright (c) 2007-2016 the FFmpeg developers
  libavutil      55. 22.101 / 55. 22.101
  libavcodec     57. 38.100 / 57. 38.100
  libavformat    57. 34.103 / 57. 34.103
  libavdevice    57.  0.101 / 57.  0.101
  libavfilter     6. 44.100 /  6. 44.100
  libswscale      4.  1.100 /  4.  1.100
  libswresample   2.  0.101 /  2.  0.101
  libpostproc    54.  0.100 / 54.  0.100
[h264 @ 0x7fe869826200] Increasing reorder buffer to 1
Input #0, lavfi, from 'movie=input.ts[out+subcc]':
  Duration: N/A, start: 1562.233011, bitrate: N/A
    Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], 90k fps, 30 tbr, 90k tbn
    Stream #0:1: Subtitle: eia_608

Stream Subtitle: eia_608 has "index" 0:1, so that is what should be mapped.

Few parting notes, order of arguments matters for ffmpeg, -f lavfi must go before -i move=..., otherwise the spec will not be recognized. Also this feature is pretty recent, so double check your ffmpeg version and upgrade if needed.

1
  • worked for me also with MKV (ffmpeg version 4.2). Also, "-map s" saves the need to figure out the CC stream id. Nov 20, 2019 at 4:17
15

Closed caption are of 2 format
1) ATSC American standard (support is there in ffmpeg)
2) ISDB Japanese standard (support is yet not there in ffmpeg)

you can use following command

ffmpeg -f lavfi -i "movie=test.ts[out0+subcc]" -map s output.srt

This thing has been recently developed so please check out your version of ffmpeg.

0
2

For getting just the subtitles and not any meta junk, I've found that

ffmpeg -i input.mov -an -vn -bsf:s mov2textsub -scodec copy -f rawvideo sub.txt 

works best for me.

0
2

I use this to extract CC608 Closed Captioning from .mp4 files:

FOR %%F IN (*.mp4) DO ffmpeg -f lavfi -i movie="%%F"[out+subcc] -map 0:1 -y "%%~nF.srt"
1

If the caption are included as a separate stream, then extracting them is (relatively) straightforward:

ffmpeg -i input.mov -an -vn -c:s copy -f rawvideo -map 0:s sub.txt

If it's "burned in" on the video file, then you're probably out of luck, but I that would be more common for subtitles than closed captions.

1

Closed Captions are not separate streams, nor are they burned into the picture, they are interlaced in scanlines IN the picture - this a strange American idea, not really used elsewhere. Though you can buy DVD's which has this kind of subtitling, usually people outside the US can't see the subtitles, and may not even know there are some in the file.

CCExtractor can extract this information from VOB's by looking at the TS metadata: http://ccextractor.sourceforge.net/

2
  • 1
    appears that for ATSC streams the captions are somehow "contained but separate" and aren't images, so are extractable, see Anshul's answer. Now if ffmpeg can "overlay them" appropriately and burn them in, not sure, but at least extraction is possible...
    – rogerdpack
    Jan 13, 2016 at 18:48
  • 1
    No CCExtractor usage for this case, so the answer is useless.
    – beroal
    Mar 28, 2019 at 6:58
1

It depends on what sort of input file you're working with. In my case I had Final Cut Pro MOV Master files with a Scenarist (.scc) style eia-608 track. While ffmpeg can extract embeded closed captions from mpeg2 transport stream files, it can't deal with the eia-608 data on a separate track.

ccextractor should work out the box, but MOV revolves around Apple products and sometimes moves with complete disregard to third parties... as such you should normalize MOV files before dealing with them... a simple ffmpeg -i my.mov -c copy -map 0 out.mov should do the trick. Then CCExtractor's built in MP4Box build should be able to pass it the eia-track and decode it into something useable... like vtt.

Source: https://trac.ffmpeg.org/ticket/7694

Not the answer you're looking for? Browse other questions tagged or ask your own question.