184

I am using the following command to run a python script in the background:

nohup ./cmd.py > cmd.log &

But it appears that nohup is not writing anything to the log file. cmd.log is created but is always empty. In the python script, I am using sys.stdout.write instead of print to print to standard output. Am I doing anything wrong?

3
  • Which variant of nohup are you using? The BSD version writes to a file called nohup.out in the current directory (or $HOME/nohup.out if the current directory isn't writable). I don't see a way to change the output file name...
    – wulong
    Oct 16, 2012 at 17:10
  • @wulong That's only if if stdout is a terminal. Oct 16, 2012 at 17:11
  • I also tried the command without redirection and it didn't create the nohup.out file at all. I don't know which variant it is but I am on SunOS 5.10 if that helps.
    – user1642513
    Oct 16, 2012 at 17:17

7 Answers 7

517

You can run Python with the -u flag to avoid output buffering:

nohup python -u ./cmd.py > cmd.log &
2
  • @kommradHomer I guess it depends on the amount of output on stdout/stderr your program produces.
    – vz0
    Dec 10, 2014 at 9:58
  • 1
    Warning: it doesn't always work. I don't know why. Do you?
    – Basj
    Jan 12, 2016 at 21:41
115

It looks like you need to flush stdout periodically (e.g. sys.stdout.flush()). In my testing Python doesn't automatically do this even with print until the program exits.

4
  • 22
    python as well as other C stdio-based programs uses line-buffering in interactive case (stdout is connected to a tty) and block-buffering when redirected to a file. If python -u doesn't work; nohup might have introduced its own buffering.
    – jfs
    Oct 16, 2012 at 17:37
  • 16
    @J.F.Sebastian As of today, nohup doesn't buffer output and python -u works just fine. (just an update for people)
    – Pijusn
    Nov 19, 2014 at 8:31
  • 1
    @Pius: nohup is a POSIX utility there could be different implementations on different platforms. btw, python3 I/O is no longer C stdio-based but it has similar buffering behavior.
    – jfs
    Dec 1, 2014 at 16:30
  • @jfs stderr buffering behavior changed in Python 3.9, it is now always line-buffered. See here.
    – jdhao
    Mar 16, 2021 at 17:07
77
  • Using -u with nohup worked for me. Using -u will force the stdout, stderr streams to be unbuffered. It will not affect stdin. Everything will be saved in "nohup.out " file. Like this-

    nohup python -u your_code.py &
    

    You can also save it into your directory. This way-

    nohup python -u your_code.py > your_directory/nohup.out &
    
  • Also, you can use PYTHONUNBUFFERED. If you set it to a non-empty string it will work same as the -u option. For using this run below commands before running python code.

    export PYTHONUNBUFFERED=1
    

    or

    export PYTHONUNBUFFERED=TRUE
    

P.S.- I will suggest using tools like cron-job to run things in the background and scheduled execution.

2
  • 2
    What's the difference with the answer from @vz0?
    – Deqing
    Dec 4, 2017 at 0:13
  • 1
    @Deqing there is no difference.
    – Overcode
    Jan 24, 2018 at 15:21
21
export PYTHONUNBUFFERED=1
nohup ./cmd.py > cmd.log &

or

nohup python -u ./cmd.py > cmd.log &

https://docs.python.org/2/using/cmdline.html#cmdoption-u

0
5

Python 3.3 and above has a flush argument to print and this is the only method that worked for me.

print("number to train = " + str(num_train), flush=True)
print("Using {} evaluation batches".format(num_evals), flush=True)
0

I had a similar issue, but not connected with a Python process. I was running a script which did a nohup and the script ran periodically via cron.

I was able to resolve the problem by:

  1. redirecting the stdin , stdout and stderr
  2. ensuring the the script being invoked via nohup didn't run anything else in the background

PS: my scripts were written in ksh running on RHEL

-1

I run my scripts in the following way and I have no problem at all:

nohup python my_script.py &> my_script.out &

comparing with your syntax looks like you are only missing a "&" symbol after your input...

1
  • how did you find this workaround? Dec 16, 2022 at 12:46

Your Answer

Reminder: Answers generated by Artificial Intelligence tools are not allowed on Stack Overflow. Learn more

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.