736

I know how to use tee to write the output (standard output) of aaa.sh to bbb.out, while still displaying it in the terminal:

./aaa.sh | tee bbb.out

How would I now also write standard error to a file named ccc.out, while still having it displayed?

2
  • 3
    To clarify -- do you want stderr to go to the screen as well as the file? Mar 28, 2009 at 2:32
  • I did, I will edit my post to clarify that. I do believe lhunath's solution will suffice. Thanks for the help all!
    – jparanich
    Mar 28, 2009 at 16:28

12 Answers 12

993

I'm assuming you want to still see standard error and standard output on the terminal. You could go for Josh Kelley's answer, but I find keeping a tail around in the background which outputs your log file very hackish and cludgy. Notice how you need to keep an extra file descriptor and do cleanup afterward by killing it and technically should be doing that in a trap '...' EXIT.

There is a better way to do this, and you've already discovered it: tee.

Only, instead of just using it for your standard output, have a tee for standard output and one for standard error. How will you accomplish this? Process substitution and file redirection:

command > >(tee -a stdout.log) 2> >(tee -a stderr.log >&2)

Let's split it up and explain:

> >(..)

>(...) (process substitution) creates a FIFO and lets tee listen on it. Then, it uses > (file redirection) to redirect the standard output of command to the FIFO that your first tee is listening on.

The same thing for the second:

2> >(tee -a stderr.log >&2)

We use process substitution again to make a tee process that reads from standard input and dumps it into stderr.log. tee outputs its input back on standard output, but since its input is our standard error, we want to redirect tee's standard output to our standard error again. Then we use file redirection to redirect command's standard error to the FIFO's input (tee's standard input).

See Input And Output

Process substitution is one of those really lovely things you get as a bonus of choosing Bash as your shell as opposed to sh (POSIX or Bourne).


In sh, you'd have to do things manually:

out="${TMPDIR:-/tmp}/out.$$" err="${TMPDIR:-/tmp}/err.$$"
mkfifo "$out" "$err"
trap 'rm "$out" "$err"' EXIT
tee -a stdout.log < "$out" &
tee -a stderr.log < "$err" >&2 &
command >"$out" 2>"$err"
18
  • 8
    I tried this: $ echo "HANG" > >(tee stdout.log) 2> >(tee stderr.log >&2) which works, but waits for input. Is there a simple reason why this happens?
    – Justin
    Aug 23, 2011 at 18:35
  • 1
    @SillyFreak I don't understand what you want to do or what the problem is you're having. echo test; exit doesn't produce any output on stdout, so err will remain empty.
    – lhunath
    Aug 25, 2013 at 0:46
  • 1
    thanks for that comment; I figured out what my logical error was afterwards: when invoked as an interactive shell, bash prints a command prompt and echoes exit to stderr. However, if stderr is redirected, bash starts as noninteractive by default; compare /bin/bash 2> err and /bin/bash -i 2> err Aug 27, 2013 at 16:12
  • 27
    And for those who "seeing is believing", a quick test: (echo "Test Out";>&2 echo "Test Err") > >(tee stdout.log) 2> >(tee stderr.log >&2) Jan 27, 2015 at 10:58
  • 6
    Can this be used to write both to the same file, but in a synchronized way, so the order of stderr and stdout messages is preserved, while each stream of the supbrocess is still directed to the corresponding stream of the calling process? I don't know, but I can imagine if tee does some buffering, the subprocess may finish writing something to stdout and write to stderr, but the second tee receiving the stderr message may finish writing it to the file earlier than the first.
    – Larry
    Jul 15, 2020 at 13:21
930

Simply:

./aaa.sh 2>&1 | tee -a log

This simply redirects standard error to standard output, so tee echoes both to log and to the screen. Maybe I'm missing something, because some of the other solutions seem really complicated.

Note: Since Bash version 4 you may use |& as an abbreviation for 2>&1 |:

./aaa.sh |& tee -a log
14
  • 125
    That works fine if you want both stdout (channel 1) and stderr (channel 2) logged to the same file (a single file containing the mixture of both stdout and sterr). The other, more complicated solution allows you to separate stdout and stderr into 2 different files (stdout.log and stderr.log, respectively). Sometimes that is important, sometimes it's not.
    – Tyler Rick
    Nov 17, 2011 at 18:55
  • 30
    The other solutions are far more complicated than necessary in many cases. This one works perfectly for me.
    – dkamins
    Nov 30, 2011 at 5:50
  • 24
    The problem with this method is that you lose the exit/status code from the aaa.sh process, which can be important (e.g. when using in a makefile). You don't have this problem with the accepted answer.
    – Stefaan
    Jun 28, 2013 at 15:46
  • 16
    if you don't mind merged stdout/stderr then ./aaa.sh |& tee aaa.log works (in bash).
    – jfs
    Sep 3, 2013 at 6:50
  • 6
    @Stefaan I believe you can retain exit status if you prepend the command chain with set -o pipefail followed by ; or && if I'm not mistaken.
    – David
    Nov 25, 2015 at 21:24
84

This may be useful for people finding this via Google. Simply uncomment the example you want to try out. Of course, feel free to rename the output files.

#!/bin/bash

STATUSFILE=x.out
LOGFILE=x.log

### All output to screen
### Do nothing, this is the default


### All Output to one file, nothing to the screen
#exec > ${LOGFILE} 2>&1


### All output to one file and all output to the screen
#exec > >(tee ${LOGFILE}) 2>&1


### All output to one file, STDOUT to the screen
#exec > >(tee -a ${LOGFILE}) 2> >(tee -a ${LOGFILE} >/dev/null)


### All output to one file, STDERR to the screen
### Note you need both of these lines for this to work
#exec 3>&1
#exec > >(tee -a ${LOGFILE} >/dev/null) 2> >(tee -a ${LOGFILE} >&3)


### STDOUT to STATUSFILE, stderr to LOGFILE, nothing to the screen
#exec > ${STATUSFILE} 2>${LOGFILE}


### STDOUT to STATUSFILE, stderr to LOGFILE and all output to the screen
#exec > >(tee ${STATUSFILE}) 2> >(tee ${LOGFILE} >&2)


### STDOUT to STATUSFILE and screen, STDERR to LOGFILE
#exec > >(tee ${STATUSFILE}) 2>${LOGFILE}


### STDOUT to STATUSFILE, STDERR to LOGFILE and screen
#exec > ${STATUSFILE} 2> >(tee ${LOGFILE} >&2)


echo "This is a test"
ls -l sdgshgswogswghthb_this_file_will_not_exist_so_we_get_output_to_stderr_aronkjegralhfaff
ls -l ${0}
4
  • 9
    No, and I guess exec can use some explaining. exec > means, move the target of a file descriptor to a certain destination. The default is 1, so, exec > /dev/null moves the output of stdout to /dev/null from now on in this session. The current file descriptors for this session can be seen by doing ls -l /dev/fd/. Try it! Then see what happens when you issue exec 2>/tmp/stderr.log. Additionally, exec 3>&1 means, create a new file descriptor with number 3, and redirect it to the target of file descriptor 1. In the example, the target was the screen when the command was issued.
    – drumfire
    May 20, 2016 at 19:00
  • 2
    The example to show stdout and stderr both in screen and separate files is awesome!!! Thanks a lot! Jan 2, 2020 at 7:31
  • 1
    The "All output to one file, STDERR to the screen" part can be put on one line (just put 3>&1 at the front): exec 3>&1 > >(tee -a ${LOGFILE} >/dev/null) 2> >(tee -a ${LOGFILE} >&3) Note if you want stderr to print to screen as 2 still, just switch to 3>&2 instead of 3>&1
    – aarowman
    Apr 22, 2021 at 19:35
  • You also don't need the tee command if you're not wanting the output on screen. Instead of using tee with /dev/null, you can just output to the file: just do exec 3>&2 1>${LOGFILE} 2> >(tee -a ${LOGFILE} >&3)
    – aarowman
    Apr 22, 2021 at 19:40
24

To redirect standard error to a file, display standard output to the screen, and also save standard output to a file:

./aaa.sh 2>ccc.out | tee ./bbb.out

To display both standard error and standard output to screen and also save both to a file, you can use Bash's I/O redirection:

#!/bin/bash

# Create a new file descriptor 4, pointed at the file
# which will receive standard error.
exec 4<>ccc.out

# Also print the contents of this file to screen.
tail -f ccc.out &

# Run the command; tee standard output as normal, and send standard error
# to our file descriptor 4.
./aaa.sh 2>&4 | tee bbb.out

# Clean up: Close file descriptor 4 and kill tail -f.
exec 4>&-
kill %1
2
  • 1
    I expect that the user wants stderr to go to their console in addition to the file, though such wasn't explicitly specified. Mar 28, 2009 at 2:37
  • 2
    I should have been clearer, I did want stderr to the screen too. I still enjoyed Josh Kelley's solution but find lhunath's to suit my needs more. Thanks guys!
    – jparanich
    Mar 28, 2009 at 16:24
23

In other words, you want to pipe stdout into one filter (tee bbb.out) and stderr into another filter (tee ccc.out). There is no standard way to pipe anything other than stdout into another command, but you can work around that by juggling file descriptors.

{ { ./aaa.sh | tee bbb.out; } 2>&1 1>&3 | tee ccc.out; } 3>&1 1>&2

See also How to grep standard error stream (stderr)? and When would you use an additional file descriptor?

In bash (and ksh and zsh), but not in other POSIX shells such as dash, you can use process substitution:

./aaa.sh > >(tee bbb.out) 2> >(tee ccc.out)

Beware that in bash, this command returns as soon as ./aaa.sh finishes, even if the tee commands are still executed (ksh and zsh do wait for the subprocesses). This may be a problem if you do something like ./aaa.sh > >(tee bbb.out) 2> >(tee ccc.out); process_logs bbb.out ccc.out. In that case, use file descriptor juggling or ksh/zsh instead.

2
  • 4
    This seems like the only answer that allows keeping the stdout/stderr streams as-is (eg. not merging them). Sweet! Oct 17, 2015 at 8:03
  • The simplest approach for sh, useful for cron jobs, where process substitution is not available. Apr 17, 2020 at 15:30
17

If using Bash:

# Redirect standard out and standard error separately
% cmd >stdout-redirect 2>stderr-redirect

# Redirect standard error and out together
% cmd >stdout-redirect 2>&1

# Merge standard error with standard out and pipe
% cmd 2>&1 |cmd2

Credit (not answering from the top of my head) goes here: Re: bash : stderr & more (pipe for stderr)

1
  • 1
    The question was "How would I now also write standard error to a file named ccc.out, while still having it displayed?"
    – jberryman
    Sep 21, 2022 at 19:15
12

If you're using Z shell (zsh), you can use multiple redirections, so you don't even need tee:

./cmd 1>&1 2>&2 1>out_file 2>err_file

Here you're simply redirecting each stream to itself and the target file.


Full example

% (echo "out"; echo "err">/dev/stderr) 1>&1 2>&2 1>/tmp/out_file 2>/tmp/err_file
out
err
% cat /tmp/out_file
out
% cat /tmp/err_file
err

Note that this requires the MULTIOS option to be set (which is the default).

MULTIOS

Perform implicit tees or cats when multiple redirections are attempted (see Redirection).

2
  • In contrast to Bash, presumably(?). What other shells can do it as well? Fish? Feb 17, 2022 at 21:07
  • This reorders the output and so is useless
    – jberryman
    Sep 21, 2022 at 19:11
7

Like the accepted answer well explained by lhunath, you can use

command > >(tee -a stdout.log) 2> >(tee -a stderr.log >&2)

Beware than if you use bash you could have some issue.

Let me take the matthew-wilcoxson example.

And for those who "seeing is believing", a quick test:

(echo "Test Out";>&2 echo "Test Err") > >(tee stdout.log) 2> >(tee stderr.log >&2)

Personally, when I try, I have this result:

user@computer:~$ (echo "Test Out";>&2 echo "Test Err") > >(tee stdout.log) 2> >(tee stderr.log >&2)
user@computer:~$ Test Out
Test Err

Both messages do not appear at the same level. Why does Test Out seem to be put like if it is my previous command?

The prompt is on a blank line letting me think the process is not finished, and when I press Enter this fix it. When I check the content of the files, it is ok, and redirection works.

Let’s take another test.

function outerr() {
  echo "out"     # stdout
  echo >&2 "err" # stderr
}
user@computer:~$ outerr
out
err

user@computer:~$ outerr >/dev/null
err

user@computer:~$ outerr 2>/dev/null
out

Trying again the redirection, but with this function:

function test_redirect() {
  fout="stdout.log"
  ferr="stderr.log"
  echo "$ outerr"
  (outerr) > >(tee "$fout") 2> >(tee "$ferr" >&2)
  echo "# $fout content: "
  cat "$fout"
  echo "# $ferr content: "
  cat "$ferr"
}

Personally, I have this result:

user@computer:~$ test_redirect
$ outerr
# stdout.log content:
out
out
err
# stderr.log content:
err
user@computer:~$

No prompt on a blank line, but I don't see normal output. The stdout.log content seem to be wrong, and only stderr.log seem to be ok.

If I relaunch it, the output can be different...

So, why?

Because, like explained here:

Beware that in bash, this command returns as soon as [first command] finishes, even if the tee commands are still executed (ksh and zsh do wait for the subprocesses)

So, if you use Bash, prefer use the better example given in this other answer:

{ { outerr | tee "$fout"; } 2>&1 1>&3 | tee "$ferr"; } 3>&1 1>&2

It will fix the previous issues.

Now, the question is, how to retrieve exit status code?

$? does not work.

I have no found better solution than switch on pipefail with set -o pipefail (set +o pipefail to switch off) and use ${PIPESTATUS[0]} like this:

function outerr() {
  echo "out"
  echo >&2 "err"
  return 11
}

function test_outerr() {
  local - # To preserve set option
  ! [[ -o pipefail ]] && set -o pipefail; # Or use second part directly
  local fout="stdout.log"
  local ferr="stderr.log"
  echo "$ outerr"
  { { outerr | tee "$fout"; } 2>&1 1>&3 | tee "$ferr"; } 3>&1 1>&2
  # First save the status or it will be lost
  local status="${PIPESTATUS[0]}" # Save first, the second is 0, perhaps tee status code.
  echo "==="
  echo "# $fout content :"
  echo "<==="
  cat "$fout"
  echo "===>"
  echo "# $ferr content :"
  echo "<==="
  cat "$ferr"
  echo "===>"
  if (( status > 0 )); then
    echo "Fail $status > 0"
    return "$status" # or whatever
  fi
}
user@computer:~$ test_outerr
$ outerr
err
out
===
# stdout.log content:
<===
out
===>
# stderr.log content:
<===
err
===>
Fail 11 > 0
6

In my case, a script was running command while redirecting both stdout and stderr to a file, something like:

cmd > log 2>&1

I needed to update it such that when there is a failure, take some actions based on the error messages. I could of course remove the dup 2>&1 and capture the stderr from the script, but then the error messages won't go into the log file for reference. While the accepted answer from lhunath is supposed to do the same, it redirects stdout and stderr to different files, which is not what I want, but it helped me to come up with the exact solution that I need:

(cmd 2> >(tee /dev/stderr)) > log

With the above, log will have a copy of both stdout and stderr and I can capture stderr from my script without having to worry about stdout.

2
  • 1
    This reorders the output
    – jberryman
    Sep 21, 2022 at 19:14
  • @jberryman Good point! I am not sure how to completely avoid it, though disabling buffering on stdout could help.
    – haridsv
    Sep 22, 2022 at 5:28
5

The following will work for KornShell (ksh) where the process substitution is not available,

# create a combined (standard input and standard output) collector
exec 3 <> combined.log

# stream standard error instead of standard output to tee, while draining all standard output to the collector
./aaa.sh 2>&1 1>&3 | tee -a stderr.log 1>&3

# cleanup collector
exec 3>&-

The real trick here, is the sequence of the 2>&1 1>&3 which in our case redirects the standard error to standard output and redirects the standard output to file descriptor 3. At this point the standard error and standard output are not combined yet.

In effect, the standard error (as standard input) is passed to tee where it logs to stderr.log and also redirects to file descriptor 3.

And file descriptor 3 is logging it to combined.log all the time. So the combined.log contains both standard output and standard error.

2
  • Nifty! Pity that's underrated because it's a nice option. I do have KSH, btw :)
    – runlevel0
    Nov 8, 2019 at 15:26
  • Is file descriptor 3 special in some way? Or is it just any number higher than 2? Feb 17, 2022 at 21:05
1

Thanks lhunath for the answer in POSIX.

Here's a more complex situation I needed in POSIX with the proper fix:

# Start script main() function
# - We redirect standard output to file_out AND terminal
# - We redirect standard error to file_err, file_out AND terminal
# - Terminal and file_out have both standard output and standard error, while file_err only holds standard error

main() {
    # my main function
}

log_path="/my_temp_dir"
pfout_fifo="${log_path:-/tmp}/pfout_fifo.$$"
pferr_fifo="${log_path:-/tmp}/pferr_fifo.$$"

mkfifo "$pfout_fifo" "$pferr_fifo"
trap 'rm "$pfout_fifo" "$pferr_fifo"' EXIT

tee -a "file_out" < "$pfout_fifo" &
    tee -a "file_err" < "$pferr_fifo" >>"$pfout_fifo" &
    main "$@" >"$pfout_fifo" 2>"$pferr_fifo"; exit
0

Compilation errors which are sent to standard error (STDERR) can be redirected or save to a file by:

Bash:

gcc temp.c &> error.log

C shell (csh):

% gcc temp.c |& tee error.log

See: How can I redirect compilation/build error to a file?

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.