0

How to direct the output of a process to stdout and pipe to another process in bash?

I am using bash shell in mac

I have two python programs generate.py and sum.py

generate.py prints integers to stdout.

when run using python3 generate.py it generates

1
2
3
4
...
...

sum.py just consumes the output of generate.py and keeps the counter and once the generate.py is done sum.py outputs the final value.

however when I run using the following bash command

python3 generate.py >&2 | python3 sum.py I cannot see the output of python3 sum.py

but if I do this python3 generate.py | python3 sum.py I can see the result from sum.py is printed to stdout but I dont see the output from generate.py in stdout.

so what I am looking is to see the output of both generate.py and sum.py while piping the data between them. any idea?

3 Answers 3

5

Use the tee command to split your output between the tty and a program stdin

echo "hello" | tee /dev/tty | pbcopy

In your case it would be

python3 generate.py | tee /dev/tty | python3 sum.py
2

I prefer this command:

( command1 | tee /dev/fd/3 | command2 ) 3>&1

The advantage is, that the complete output is combined to stdout, and stderr as second channel is still intact. So this can be used as function or script with usual redirects outside.

Translated to your commands:

( python3 generate.py | tee /dev/fd/3 | python3 sum.py ) 3>&1
1

You can do it this way

python3 generate.py | tee /dev/stderr | python3 sum.py

But both "/dev/stderr" and "/dev/tty" are special "device file" and we should avoid using it this way.

This is easier to illustrate with an example. If we construct a chained command with pipeline, the normal expectation is, the whole chained pipeline will behave like a single coherent command (whole pipeline takes input form stdin, and write all output to stdout, and errors goes to stderr).

If I append "|wc -l" at the end (counting number of lines), it will expect it return the number of line for both "generate.py" and "sum.py".

But it will not work this way.

python3 generate.py | tee /dev/stderr | python3 sum.py | wc -l

The wc output will only be "1", which refers to the single line from "sum.py".

About "/dev/tty", it is referring to a "controlling terminal" device file. You can refers to this for more information. But in essence this is not a "device file" which should be used to generate output

2
  • why is it hacky? Commented Feb 20, 2020 at 11:29
  • I updated my response to illustrate why it is hacky
    – Zarick Lau
    Commented Feb 21, 2020 at 4:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.