251

I know how to redirect stdout to a file:

exec > foo.log
echo test

this will put the 'test' into the foo.log file.

Now I want to redirect the output into the log file AND keep it on stdout

i.e. it can be done trivially from outside the script:

script | tee foo.log

but I want to do declare it within the script itself

I tried

exec | tee foo.log

but it didn't work.

3
  • 4
    Your question is poorly phrased. When you invoke 'exec > foo.log', the stdout of the script is the file foo.log. I think you mean that you want the output to go to foo.log and to the tty, since going to foo.log is going to stdout. Jul 5, 2010 at 5:07
  • what I'd like to do is to use the | on the 'exec'. that would be perfect for me, i.e. "exec | tee foo.log", unfortunately you can not use pipe redirection on the exec call Jul 7, 2010 at 1:32

9 Answers 9

316
#!/usr/bin/env bash

# Redirect stdout ( > ) into a named pipe ( >() ) running "tee"
exec > >(tee -i logfile.txt)

# Without this, only stdout would be captured - i.e. your
# log file would not contain any error messages.
# SEE (and upvote) the answer by Adam Spiers, which keeps STDERR
# as a separate stream - I did not want to steal from him by simply
# adding his answer to mine.
exec 2>&1

echo "foo"
echo "bar" >&2

Note that this is bash, not sh. If you invoke the script with sh myscript.sh, you will get an error along the lines of syntax error near unexpected token '>'.

If you are working with signal traps, you might want to use the tee -i option to avoid disruption of the output if a signal occurs. (Thanks to JamesThomasMoon1979 for the comment.)


Tools that change their output depending on whether they write to a pipe or a terminal (ls using colors and columnized output, for example) will detect the above construct as meaning that they output to a pipe.

There are options to enforce the colorizing / columnizing (e.g. ls -C --color=always). Note that this will result in the color codes being written to the logfile as well, making it less readable.

37
  • 5
    Tee on most systems is buffered, so output may not arrive until after the script has finished. Also, since this tee is running in a subshell, not a child process, wait cannot be used to synchronize output to the calling process. What you want is an unbuffered version of tee similar to bogomips.org/rainbows.git/commit/…
    – user246672
    Feb 13, 2012 at 15:15
  • 14
    @Barry: POSIX specifies that tee should not buffer its output. If it does buffer on most systems, it's broken on most systems. That's a problem of the tee implementations, not of my solution.
    – DevSolar
    Feb 16, 2012 at 8:21
  • 3
    @Sebastian: exec is very powerful, but also very involved. You can "back up" the current stdout to a different filedescriptor, then recover it later on. Google "bash exec tutorial", there's lots of advanced stuff out there.
    – DevSolar
    Apr 7, 2012 at 7:27
  • 2
    @AdamSpiers: I'm not sure what Barry was about, either. Bash's exec is documented not to start new processes, >(tee ...) is a standard named pipe / process substitution, and the & in the redirection of course has nothing to do with backgrounding... ?:-)
    – DevSolar
    Aug 10, 2012 at 10:56
  • 13
    I suggest passing -i to tee. Otherwise, signal interrupts (traps) will disrupt stdout in the main script. For example, if you have a trap 'echo foo' EXIT and then press ctrl+c, you will not see "foo". So I would modify the answer to exec &> >(tee -ia file). May 23, 2015 at 9:23
188

The accepted answer does not preserve STDERR as a separate file descriptor. That means

./script.sh >/dev/null

will not output bar to the terminal, only to the logfile, and

./script.sh 2>/dev/null

will output both foo and bar to the terminal. Clearly that's not the behaviour a normal user is likely to expect. This can be fixed by using two separate tee processes both appending to the same log file:

#!/bin/bash

# See (and upvote) the comment by JamesThomasMoon1979 
# explaining the use of the -i option to tee.
exec >  >(tee -ia foo.log)
exec 2> >(tee -ia foo.log >&2)

echo "foo"
echo "bar" >&2

(Note that the above does not initially truncate the log file - if you want that behaviour you should add

>foo.log

to the top of the script.)

The POSIX.1-2008 specification of tee(1) requires that output is unbuffered, i.e. not even line-buffered, so in this case it is possible that STDOUT and STDERR could end up on the same line of foo.log; however that could also happen on the terminal, so the log file will be a faithful reflection of what could be seen on the terminal, if not an exact mirror of it. If you want the STDOUT lines cleanly separated from the STDERR lines, consider using two log files, possibly with date stamp prefixes on each line to allow chronological reassembly later on.

4
  • For some reason, in my case, when the script is executed from a c-program system() call, the two tee sub-processes continue to exist even after the main script exits. So I had to add traps like this: exec > >(tee -a $LOG) trap "kill -9 $! 2>/dev/null" EXIT exec 2> >(tee -a $LOG >&2) trap "kill -9 $! 2>/dev/null" EXIT
    – alveko
    Jul 15, 2014 at 20:38
  • 22
    I suggest passing -i to tee. Otherwise, signal interrupts (traps) will disrupt stdout in the script. For example, if you trap 'echo foo' EXIT and then press ctrl+c, you will not see "foo". So I would modify the answer to exec > >(tee -ia foo.log). May 23, 2015 at 9:32
  • I made some little "sourceable" scripts based on this. Can use them in a script like . log or . log foo.log: sam.nipl.net/sh/log sam.nipl.net/sh/log-a Jul 16, 2015 at 3:22
  • 3
    The problem with this method is that messages going to STDOUT appear first as a batch, and then messages going to STDERR appear. They are not interleaved as usually expected. Feb 10, 2016 at 8:24
29

Solution for busybox, macOS bash, and non-bash shells

The accepted answer is certainly the best choice for bash. I'm working in a Busybox environment without access to bash, and it does not understand the exec > >(tee log.txt) syntax. It also does not do exec >$PIPE properly, trying to create an ordinary file with the same name as the named pipe, which fails and hangs.

Hopefully this would be useful to someone else who doesn't have bash.

Also, for anyone using a named pipe, it is safe to rm $PIPE, because that unlinks the pipe from the VFS, but the processes that use it still maintain a reference count on it until they are finished.

Note the use of $* is not necessarily safe.

#!/bin/sh

if [ "$SELF_LOGGING" != "1" ]
then
    # The parent process will enter this branch and set up logging

    # Create a named piped for logging the child's output
    PIPE=tmp.fifo
    mkfifo $PIPE

    # Launch the child process with stdout redirected to the named pipe
    SELF_LOGGING=1 sh $0 $* >$PIPE &

    # Save PID of child process
    PID=$!

    # Launch tee in a separate process
    tee logfile <$PIPE &

    # Unlink $PIPE because the parent process no longer needs it
    rm $PIPE    

    # Wait for child process, which is running the rest of this script
    wait $PID

    # Return the error code from the child process
    exit $?
fi

# The rest of the script goes here
1
  • This is the only solution I've seen so far that works on mac Nov 29, 2018 at 15:07
21

Inside your script file, put all of the commands within parentheses, like this:

(
echo start
ls -l
echo end
) | tee foo.log
5
  • 5
    pedantically, could also use braces ({}) Jul 4, 2010 at 4:14
  • well yeah, I considered that, but this is not redirection of the current shell stdout, its kind of a cheat, you actually running a subshell and doing a regular piper redirection on it. works thought. I'm split with this and the "tail -f foo.log &" solution. will wait a little to see if may be a better one surfaces. if not probably going to settle ;) Jul 7, 2010 at 1:34
  • 8
    { } executes a list in the current shell environment. ( ) executes a list in a subshell environment.
    – user246672
    Feb 13, 2012 at 16:38
  • 1
    Damn. Thank you. The accepted answer up there didn't work for me, trying to schedule a script to run under MingW on a Windows system. It complained, I believe, about unimplemented process substitution. This answer worked just fine, after the following change, to capture both stderr and stdout: ``` -) | tee foo.log +) 2>&1 | tee foo.log
    – Jon Carter
    Sep 30, 2015 at 17:26
  • For me this answer is way simpler and easier to understand than the accepted one, and also doesn't keep redirecting output after the script finishes like the accepted answer does!
    – Ben Farmer
    Feb 17, 2021 at 1:06
15

Easy way to make a bash script log to syslog. The script output is available both through /var/log/syslog and through stderr. syslog will add useful metadata, including timestamps.

Add this line at the top:

exec &> >(logger -t myscript -s)

Alternatively, send the log to a separate file:

exec &> >(ts |tee -a /tmp/myscript.output >&2 )

This requires moreutils (for the ts command, which adds timestamps).

1
  • It seems your solutions sends only stdout to a separate file. How do I send stdout and stderr to a separate file?
    – mles
    Oct 18, 2020 at 15:18
13

Using the accepted answer my script kept returning exceptionally early (right after 'exec > >(tee ...)') leaving the rest of my script running in the background. As I couldn't get that solution to work my way I found another solution/work around to the problem:

# Logging setup
logfile=mylogfile
mkfifo ${logfile}.pipe
tee < ${logfile}.pipe $logfile &
exec &> ${logfile}.pipe
rm ${logfile}.pipe

# Rest of my script

This makes output from script go from the process, through the pipe into the sub background process of 'tee' that logs everything to disc and to original stdout of the script.

Note that 'exec &>' redirects both stdout and stderr, we could redirect them separately if we like, or change to 'exec >' if we just want stdout.

Even thou the pipe is removed from the file system in the beginning of the script it will continue to function until the processes finishes. We just can't reference it using the file name after the rm-line.

3
  • Similar answer as the second idea from David Z. Have a look at its comments. +1 ;-)
    – oHo
    Jul 31, 2013 at 14:32
  • Works well. I'm not understanding the $logfile part of tee < ${logfile}.pipe $logfile &. Specifically, I tried to alter this to capture full expanded command log lines (from set -x) to file while only showing lines without leading '+' in stdout by changing to (tee | grep -v '^+.*$') < ${logfile}.pipe $logfile & but received an error message regarding $logfile. Can you explain the tee line in a little more detail? Jun 19, 2015 at 12:47
  • I tested this out and it seems this answer doesn't preserve STDERR (it is merged with STDOUT), so if you rely on the streams being separate for error detection or other redirection, you should look at Adam's answer.
    – HeroCC
    Jun 16, 2020 at 15:15
2

Bash 4 has a coproc command which establishes a named pipe to a command and allows you to communicate through it.

1

Can't say I'm comfortable with any of the solutions based on exec. I prefer to use tee directly, so I make the script call itself with tee when requested:

# my script: 

check_tee_output()
{
    # copy (append) stdout and stderr to log file if TEE is unset or true
    if [[ -z $TEE || "$TEE" == true ]]; then 
        echo '-------------------------------------------' >> log.txt
        echo '***' $(date) $0 $@ >> log.txt
        TEE=false $0 $@ 2>&1 | tee --append log.txt
        exit $?
    fi 
}

check_tee_output $@

rest of my script

This allows you to do this:

your_script.sh args           # tee 
TEE=true your_script.sh args  # tee 
TEE=false your_script.sh args # don't tee
export TEE=false
your_script.sh args           # tee

You can customize this, e.g. make tee=false the default instead, make TEE hold the log file instead, etc. I guess this solution is similar to jbarlow's, but simpler, maybe mine has limitations that I have not come across yet.

-1

Neither of these is a perfect solution, but here are a couple things you could try:

exec >foo.log
tail -f foo.log &
# rest of your script

or

PIPE=tmp.fifo
mkfifo $PIPE
exec >$PIPE
tee foo.log <$PIPE &
# rest of your script
rm $PIPE

The second one would leave a pipe file sitting around if something goes wrong with your script, which may or may not be a problem (i.e. maybe you could rm it in the parent shell afterwards).

4
  • 2
    tail will leave a running process behind in the 2nd script tee will block, or you will need to run it with & in which case it will leave process as in 1st one. Jul 3, 2010 at 23:29
  • @Vitaly: oops, forgot to background tee - I've edited. As I said, neither is a perfect solution, but the background processes will get killed when their parent shell terminates, so you don't have to worry about them hogging resources forever.
    – David Z
    Jul 3, 2010 at 23:44
  • 1
    Yikes: these look appealing, but the output of tail -f is also going to foo.log. You can fix that by running tail -f before the exec, but the tail is still left running after the parent terminates. You need to explicitly kill it, probably in a trap 0. Jul 5, 2010 at 5:04
  • Yeap. If the script is backgrounded, it leaves processes all over.
    – user246672
    Feb 29, 2012 at 9:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.