1

I have an ffmpeg command recording video and audio as one output and I'm trying to add a second output to pipe out only the raw audio to python so that I can run additional processing on the audio chunks.

from subprocess import Popen, PIPE, STDOUT

record_audio_video_command = f"ffmpeg -i small_bunny_1080p_60fps.mp4 \ 
                             -vcodec libx264 output.mp4 -f s16le pipe:1"

record_command = Popen(record_audio_video_command, 
                        shell=True, 
                        stdout=PIPE, 
                        stderr=STDOUT)

f = open("output.txt", "w")
data = record_command.stdout.read(320)
f.write(str(data))
while len(data) > 0:
    data = record_command.stdout.read(320)
    f.write(str(data))

Is there a way to have ffmpeg pipe out only the raw audio bytes as one output, while still recording video and audio as the other output?

4
  • It is possible, but it's too difficult to answer your question in its current form. 1. The FFmpeg command line reads audio and video from audio and video devices that we don't have (that makes in unreproducible). Since audio/video source is not relevant to the question, please use a sample file as input. For example, use small_bunny_1080p_60fps.mp4. 2. m3u8 output format is complicated and not relevant, please use other format as MP4. 3. Post an example that uses two outputs
    – Rotem
    Jun 9 at 9:01
  • 4. Using pyaudio doesn't seem relevant (makes things more complicated). 5. Try to post a reproducible (and executable) code sample that includes the relevant import statements and that we can actually execute. I expect to see two code samples: one that records the video, and one that reads the audio from FFmpeg subprocess stdout pipe and stores the output of the pipe to an output file (saving the output from the pipe to a file is important for making the code reproducible).
    – Rotem
    Jun 9 at 9:08
  • I updated the question with an example. The output file with the stdout data seems to have more than just the raw audio bytes.
    – urico12
    Jun 9 at 14:14
  • Can you please respond my answer?
    – Rotem
    Jun 13 at 13:45

1 Answer 1

0

We may split the solution into stages that are easier to test:

First we better try using FFmpeg CLI for encoding two MP4 files, one with video and audio stream, and one with audio only.
The input file small_bunny_1080p_60fps.mp4 has 6 audio channels, we may convert the audio to stereo as described here:

ffmpeg -y -i small_bunny_1080p_60fps.mp4 -af "pan=stereo|c0=c2+0.30*c0+0.30*c4|c1=c2+0.30*c1+0.30*c5" -ac 2 -channel_layout stereo -vcodec libx264 -acodec aac output.mp4 -vn -ac 2 -channel_layout stereo -acodec aac output2.mp4

Note that I didn't notice that small_bunny_1080p_60fps.mp4 has 6 audio channel when I recommended to use it as input.

We can verify that output.mp4 and output2.mp4 are valid MP4 files.


Second test: pass raw audio with s16le format to the second file:
ffmpeg -y -i small_bunny_1080p_60fps.mp4 -af "pan=stereo|c0=c2+0.30*c0+0.30*c4|c1=c2+0.30*c1+0.30*c5" -ac 2 -channel_layout stereo -vcodec libx264 -acodec aac output.mp4 -vn -ar 48000 -ac 2 -channel_layout stereo -f s16le output2.s16le

Convert output2.s16le to MP4 with AAC codec for testing:

ffmpeg -y -f s16le -ar 48000 -ac 2 -channel_layout stereo -i output2.s16le -acodec aac output3.mp4

Verify that output3.mp4 is valid MP4 files (with valid audio stream).


Third test: replacing the second file with a PIPE (write the output of stdout to output3.s16le):

ffmpeg -y -i small_bunny_1080p_60fps.mp4 -af "pan=stereo|c0=c2+0.30*c0+0.30*c4|c1=c2+0.30*c1+0.30*c5" -ac 2 -channel_layout stereo -vcodec libx264 -acodec aac output.mp4 -vn -ar 48000 -ac 2 -channel_layout stereo -f s16le pipe:1 > output3.s16le

Verify that output2.s16le and output3.s16le have the same content.


Python implementation:

from subprocess import Popen, PIPE, STDOUT
import shlex

# In case we are using Linux, we have to use shlex.split (in Windows we don't).
record_audio_video_command = shlex.split('ffmpeg -y -i small_bunny_1080p_60fps.mp4 -af "pan=stereo|c0=c2+0.30*c0+0.30*c4|c1=c2+0.30*c1+0.30*c5" \
                                          -ac 2 -channel_layout stereo -vcodec libx264 -acodec aac output.mp4 -vn -ar 48000 -ac 2 \
                                          -channel_layout stereo -f s16le pipe:1')

record_command = Popen(record_audio_video_command, stdout=PIPE)

with open("output4.s16le", "wb") as f:  # Open file as binary file
    data = record_command.stdout.read(320)
    f.write(data)
    while len(data) > 0:
        data = record_command.stdout.read(320)
        f.write(data)

Verify that output4.s16le and output3.s16le have the same content.


Notes:

  • We have to open the output file as binary file - use "wb" argument.
    Write binary data to the file - don't convert the data to string.
  • We may either use with open(...) as f in a nested code block or use f.close() for closing the file.
  • It is recommended not to use shell=True (it considered to be unsafe).
  • Use shlex.split for splitting the command into a list of command and arguments (in Windows it may work without splitting).

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.