533

I want to execute a long running command in Bash, and both capture its exit status, and tee its output.

So I do this:

command | tee out.txt
ST=$?

The problem is that the variable ST captures the exit status of tee and not of command. How can I solve this?

Note that command is long running and redirecting the output to a file to view it later is not a good solution for me.

2
  • 1
    [[ "${PIPESTATUS[@]}" =~ [^0\ ] ]] && echo -e "Match - error found" || echo -e "No match - all good" This will test all the values of the array at once and give an error message if any of the pipe values returned are not zero. This is a pretty robust generalized solution for detecting errors in a piped situation. Commented Mar 31, 2015 at 19:08

17 Answers 17

626

There is an internal Bash variable called $PIPESTATUS; it’s an array that holds the exit status of each command in your last foreground pipeline of commands.

<command> | tee out.txt ; test ${PIPESTATUS[0]} -eq 0

Or another alternative which also works with other shells (like zsh) would be to enable pipefail:

set -o pipefail
...

The first option does not work with zsh due to a little bit different syntax.

11
  • 28
    There is a good explanation with examples of PIPESTATUS AND Pipefail here: unix.stackexchange.com/a/73180/7453.
    – slm
    Commented Apr 21, 2013 at 14:01
  • 23
    Note: $PIPESTATUS[0] holds the exit status of the first command in the pipe, $PIPESTATUS[1] the exit status of the second command, and so on.
    – simpleuser
    Commented Sep 24, 2013 at 18:56
  • 22
    Of course, we have to remember that this is Bash-specific: if I were to (for example) write a script to run on BusyBox's "sh" implementation on my Android device, or on some other embedded platform using some other "sh" variant, this would not work. Commented Mar 31, 2014 at 9:09
  • 5
    For those concerned about the unquoted variable expansion: The exit status is always unsigned 8-bit integer in Bash, therefore there is no need to quote it. This holds under Unix generally, too, where exit status is defined to be 8-bit explicitly, and it is assumed to be unsigned even by POSIX itself, e.g. when defining its logical negation.
    – Palec
    Commented Jun 15, 2016 at 9:06
  • 8
    You can also use exit ${PIPESTATUS[0]}.
    – Chaoran
    Commented Sep 9, 2016 at 20:03
177

Dumb solution: Connecting them through a named pipe (mkfifo). Then the command can be run second.

 mkfifo pipe
 tee out.txt < pipe &
 command > pipe
 echo $?

EDIT: Since I had to look this up, I'm adding the decoration to use a randomly named pipe and closing it. Also sends stdout to the named pipe. I needed multiple such structures in a single script. - LM

pipename="pipe-$RANDOM" ; mkfifo "$pipename" ; tee -a "$LOG_FILE" < "$pipename" &
command > "$pipename" 2>&1
echo $?
rm "$pipename"
7
  • 37
    This is the only answer in this question that also works for the simple sh Unix shell. Thanks! Commented Sep 30, 2014 at 4:08
  • 4
    @DaveKennedy: Dumb as in "obvious, not requiring intricate knowledge of bash syntax"
    – EFraim
    Commented Mar 2, 2016 at 18:18
  • 15
    While the bash answers are more elegant when you have the advantage of bash's extra capabilities, this is the more cross platform solution. It's also something that's worth thinking about in general as any time you're doing a long-running command a name pipe is often the most flexible way. It's worth noting that some systems don't have mkfifo and may instead require mknod -p if I remember right.
    – Haravikk
    Commented May 21, 2016 at 10:21
  • 2
    In case someone has a problem with mkfifo or mknod -p: in my case proper command to create pipe file was mknod FILE_NAME p.
    – Karol Gil
    Commented Jun 7, 2019 at 14:31
  • 3
    mkfifo seems more portable pubs.opengroup.org/onlinepubs/9699919799
    – MarcH
    Commented Apr 14, 2020 at 23:13
156

using bash's set -o pipefail is helpful

pipefail: the return value of a pipeline is the status of the last command to exit with a non-zero status, or zero if no command exited with a non-zero status

4
  • 29
    In case you don't want to modify the pipefail setting of the whole script, you can set the option only locally: ( set -o pipefail; command | tee out.txt ); ST=$?
    – Jaan
    Commented Oct 20, 2015 at 18:15
  • 11
    @Jaan This would run a subshell. If you want to avoid that, you could do set -o pipefail and then do the command, and immediately afterwards do set +o pipefail to unset the option. Commented Sep 27, 2016 at 0:21
  • 2
    Note: the question poster doesn't want a "general exit code" of the pipe, he wants the return code of 'command'. With -o pipefail he would know if the pipe fails, but if both 'command' and 'tee' fail, he would receive the exit code from 'tee'.
    – t0r0X
    Commented Mar 7, 2018 at 9:24
  • @LinusArver wouldn't that clear the exit code since it's a command that succeeds? Commented Apr 23, 2018 at 0:12
40

There's an array that gives you the exit status of each command in a pipe.

$ cat x| sed 's///'
cat: x: No such file or directory
$ echo $?
0
$ cat x| sed 's///'
cat: x: No such file or directory
$ echo ${PIPESTATUS[*]}
1 0
$ touch x
$ cat x| sed 's'
sed: 1: "s": substitute pattern can not be delimited by newline or backslash
$ echo ${PIPESTATUS[*]}
0 1
0
34

This solution works without using bash specific features or temporary files. Bonus: in the end the exit status is actually an exit status and not some string in a file.

Situation:

someprog | filter

you want the exit status from someprog and the output from filter.

Here is my solution:

((((someprog; echo $? >&3) | filter >&4) 3>&1) | (read xs; exit $xs)) 4>&1

echo $?

See my answer for the same question on unix.stackexchange.com for a detailed explanation and an alternative without subshells and some caveats.

26

By combining PIPESTATUS[0] and the result of executing the exit command in a subshell, you can directly access the return value of your initial command:

command | tee ; ( exit ${PIPESTATUS[0]} )

Here's an example:

# the "false" shell built-in command returns 1
false | tee ; ( exit ${PIPESTATUS[0]} )
echo "return value: $?"

will give you:

return value: 1

4
  • 6
    Thanks, this allowed me to use the construct: VALUE=$(might_fail | piping) which won't set PIPESTATUS in the master shell but will set its errorlevel. By using: VALUE=$(might_fail | piping; exit ${PIPESTATUS[0]}) I get want I wanted.
    – vaab
    Commented Jun 23, 2014 at 8:47
  • @vaab, that syntax looks really nice but I am confused on what 'piping' means in your context? Is that just where one would do 'tee' or whatever processing on the output of might_fail? ty! Commented Jul 24, 2014 at 18:41
  • 1
    @AnneTheAgile 'piping' in my example stands for commands from which you don't want to see the errlvl. For instance: one of or any piped combination of 'tee', 'grep', 'sed', ... It's not so uncommon that these piping commands are meant to format or extract info from a larger output or log output of the main command: you are then more interested in the errlevel of the main command (the one I've called 'might_fail' in my example) but without my construct the whole assignation returns the last piped command's errlvl which is here meaningless. Is this clearer ?
    – vaab
    Commented Jul 25, 2014 at 8:20
  • 1
    command_might_fail | grep -v "line_pattern_to_exclude" || exit ${PIPESTATUS[0]} in case not tee but grep filtering Commented Apr 6, 2020 at 18:14
15

So I wanted to contribute an answer like lesmana's, but I think mine is perhaps a little simpler and slightly more advantageous pure-Bourne-shell solution:

# You want to pipe command1 through command2:
exec 4>&1
exitstatus=`{ { command1; printf $? 1>&3; } | command2 1>&4; } 3>&1`
# $exitstatus now has command1's exit status.

I think this is best explained from the inside out - command1 will execute and print its regular output on stdout (file descriptor 1), then once it's done, printf will execute and print icommand1's exit code on its stdout, but that stdout is redirected to file descriptor 3.

While command1 is running, its stdout is being piped to command2 (printf's output never makes it to command2 because we send it to file descriptor 3 instead of 1, which is what the pipe reads). Then we redirect command2's output to file descriptor 4, so that it also stays out of file descriptor 1 - because we want file descriptor 1 free for a little bit later, because we will bring the printf output on file descriptor 3 back down into file descriptor 1 - because that's what the command substitution (the backticks), will capture and that's what will get placed into the variable.

The final bit of magic is that first exec 4>&1 we did as a separate command - it opens file descriptor 4 as a copy of the external shell's stdout. Command substitution will capture whatever is written on standard out from the perspective of the commands inside it - but since command2's output is going to file descriptor 4 as far as the command substitution is concerned, the command substitution doesn't capture it - however once it gets "out" of the command substitution it is effectively still going to the script's overall file descriptor 1.

(The exec 4>&1 has to be a separate command because many common shells don't like it when you try to write to a file descriptor inside a command substitution, that is opened in the "external" command that is using the substitution. So this is the simplest portable way to do it.)

You can look at it in a less technical and more playful way, as if the outputs of the commands are leapfrogging each other: command1 pipes to command2, then the printf's output jumps over command 2 so that command2 doesn't catch it, and then command 2's output jumps over and out of the command substitution just as printf lands just in time to get captured by the substitution so that it ends up in the variable, and command2's output goes on its merry way being written to the standard output, just as in a normal pipe.

Also, as I understand it, $? will still contain the return code of the second command in the pipe, because variable assignments, command substitutions, and compound commands are all effectively transparent to the return code of the command inside them, so the return status of command2 should get propagated out - this, and not having to define an additional function, is why I think this might be a somewhat better solution than the one proposed by lesmana.

Per the caveats lesmana mentions, it's possible that command1 will at some point end up using file descriptors 3 or 4, so to be more robust, you would do:

exec 4>&1
exitstatus=`{ { command1 3>&-; printf $? 1>&3; } 4>&- | command2 1>&4; } 3>&1`
exec 4>&-

Note that I use compound commands in my example, but subshells (using ( ) instead of { } will also work, though may perhaps be less efficient.)

Commands inherit file descriptors from the process that launches them, so the entire second line will inherit file descriptor four, and the compound command followed by 3>&1 will inherit the file descriptor three. So the 4>&- makes sure that the inner compound command will not inherit file descriptor four, and the 3>&- will not inherit file descriptor three, so command1 gets a 'cleaner', more standard environment. You could also move the inner 4>&- next to the 3>&-, but I figure why not just limit its scope as much as possible.

I'm not sure how often things use file descriptor three and four directly - I think most of the time programs use syscalls that return not-used-at-the-moment file descriptors, but sometimes code writes to file descriptor 3 directly, I guess (I could imagine a program checking a file descriptor to see if it's open, and using it if it is, or behaving differently accordingly if it's not). So the latter is probably best to keep in mind and use for general-purpose cases.

1
  • Nice explanation!
    – selurvedu
    Commented Aug 1, 2019 at 15:02
11
(command | tee out.txt; exit ${PIPESTATUS[0]})

Unlike @cODAR's answer this returns the original exit code of the first command and not only 0 for success and 127 for failure. But as @Chaoran pointed out you can just call ${PIPESTATUS[0]}. It is important however that all is put into brackets.

2
  • I like this approach. We can also do ( set +e ; command | tee ... ) to prevent the script from exiting if tee fails and the outer script has set -e. Commented Sep 1, 2022 at 5:41
  • For that reason, ( ... exit ) is preferable to { ... return } because although our child shell does inherit the set -e from the parent, our set +e does not affect the parent shell. Commented Sep 1, 2022 at 5:45
8

In Ubuntu and Debian, you can apt-get install moreutils. This contains a utility called mispipe that returns the exit status of the first command in the pipe.

8

Outside of bash, you can do:

bash -o pipefail  -c "command1 | tee output"

This is useful for example in ninja scripts where the shell is expected to be /bin/sh.

6

The simplest way to do this in plain bash is to use process substitution instead of a pipeline. There are several differences, but they probably don't matter very much for your use case:

  • When running a pipeline, bash waits until all processes complete.
  • Sending Ctrl-C to bash makes it kill all the processes of a pipeline, not just the main one.
  • The pipefail option and the PIPESTATUS variable are irrelevant to process substitution.
  • Possibly more

With process substitution, bash just starts the process and forgets about it, it's not even visible in jobs.

Mentioned differences aside, consumer < <(producer) and producer | consumer are essentially equivalent.

If you want to flip which one is the "main" process, you just flip the commands and the direction of the substitution to producer > >(consumer). In your case:

command > >(tee out.txt)

Example:

$ { echo "hello world"; false; } > >(tee out.txt)
hello world
$ echo $?
1
$ cat out.txt
hello world

$ echo "hello world" > >(tee out.txt)
hello world
$ echo $?
0
$ cat out.txt
hello world

As I said, there are differences from the pipe expression. The process may never stop running, unless it is sensitive to the pipe closing. In particular, it may keep writing things to your stdout, which may be confusing.

1
  • This is my preferred solution. Commented Aug 25, 2021 at 17:01
3

PIPESTATUS[@] must be copied to an array immediately after the pipe command returns. Any reads of PIPESTATUS[@] will erase the contents. Copy it to another array if you plan on checking the status of all pipe commands. "$?" is the same value as the last element of "${PIPESTATUS[@]}", and reading it seems to destroy "${PIPESTATUS[@]}", but I haven't absolutely verified this.

declare -a PSA  
cmd1 | cmd2 | cmd3  
PSA=( "${PIPESTATUS[@]}" )

This will not work if the pipe is in a sub-shell. For a solution to that problem,
see bash pipestatus in backticked command?

2

Base on @brian-s-wilson 's answer; this bash helper function:

pipestatus() {
  local S=("${PIPESTATUS[@]}")

  if test -n "$*"
  then test "$*" = "${S[*]}"
  else ! [[ "${S[@]}" =~ [^0\ ] ]]
  fi
}

used thus:

1: get_bad_things must succeed, but it should produce no output; but we want to see output that it does produce

get_bad_things | grep '^'
pipeinfo 0 1 || return

2: all pipeline must succeed

thing | something -q | thingy
pipeinfo || return
1

Pure shell solution:

% rm -f error.flag; echo hello world \
| (cat || echo "First command failed: $?" >> error.flag) \
| (cat || echo "Second command failed: $?" >> error.flag) \
| (cat || echo "Third command failed: $?" >> error.flag) \
; test -s error.flag  && (echo Some command failed: ; cat error.flag)
hello world

And now with the second cat replaced by false:

% rm -f error.flag; echo hello world \
| (cat || echo "First command failed: $?" >> error.flag) \
| (false || echo "Second command failed: $?" >> error.flag) \
| (cat || echo "Third command failed: $?" >> error.flag) \
; test -s error.flag  && (echo Some command failed: ; cat error.flag)
Some command failed:
Second command failed: 1
First command failed: 141

Please note the first cat fails as well, because it's stdout gets closed on it. The order of the failed commands in the log is correct in this example, but don't rely on it.

This method allows for capturing stdout and stderr for the individual commands so you can then dump that as well into a log file if an error occurs, or just delete it if no error (like the output of dd).

1

It may sometimes be simpler and clearer to use an external command, rather than digging into the details of bash. pipeline, from the minimal process scripting language execline, exits with the return code of the second command*, just like a sh pipeline does, but unlike sh, it allows reversing the direction of the pipe, so that we can capture the return code of the producer process (the below is all on the sh command line, but with execline installed):

$ # using the full execline grammar with the execlineb parser:
$ execlineb -c 'pipeline { echo "hello world" } tee out.txt'
hello world
$ cat out.txt
hello world

$ # for these simple examples, one can forego the parser and just use "" as a separator
$ # traditional order
$ pipeline echo "hello world" "" tee out.txt 
hello world

$ # "write" order (second command writes rather than reads)
$ pipeline -w tee out.txt "" echo "hello world"
hello world

$ # pipeline execs into the second command, so that's the RC we get
$ pipeline -w tee out.txt "" false; echo $?
1

$ pipeline -w tee out.txt "" true; echo $?
0

$ # output and exit status
$ pipeline -w tee out.txt "" sh -c "echo 'hello world'; exit 42"; echo "RC: $?"
hello world
RC: 42
$ cat out.txt
hello world

Using pipeline has the same differences to native bash pipelines as the bash process substitution used in answer #43972501.

* Actually pipeline doesn't exit at all unless there is an error. It executes into the second command, so it's the second command that does the returning.

0

Why not use stderr? Like so:

(
     # Our long-running process that exits abnormally
    ( for i in {1..100} ; do echo ploop ; sleep 0.5 ; done ; exit 5 )
    echo $? 1>&2 # We pass the exit status of our long-running process to stderr (fd 2).
) | tee ploop.out

So ploop.out receives the stdout. stderr receives the exit status of the long running process. This has the benefit of being completely POSIX-compatible.

(Well, with the exception of the range expression in the example long-running process, but that's not really relevant.)

Here's what this looks like:

...
ploop
ploop
ploop
ploop
ploop
ploop
ploop
ploop
ploop
ploop
5

Note that the return code 5 does not get output to the file ploop.out.

0

Shortest way of doing this is:

(set -o pipefail && command | tee out.txt)

Another option

command | tee out.txt
exit ${PIPESTATUS[0]}

See the full answer here

Not the answer you're looking for? Browse other questions tagged or ask your own question.