73

Is there any way to get the effect of running python -u from within my code? Failing that, can my program check if it is running in -u mode and exit with an error message if not? This is on Linux (Ubuntu 8.10 Server).

2
  • 7
    Another workaround is set PYTHONUNBUFFERED env var to any non-empty string. This is NOT in your code — must be set before python interpreter starts — but is easier than modifying whatever launches your script to add -u. May 11, 2020 at 5:53
  • 2
    Perfect, thank you! This helped me immensely today. I was starting a FastAPI service from a bash file (running uvicorn), which is itself started from a SystemD .service file. I realized that some of my stdout was being buffered and only spilling out when the program terminated, not at the beginning when I wanted to see it. Since the Python startup is a couple of layers deep, I was trying to figure out how to convince uvicorn to add the "-u" when it started Python, but with this method, I was able to simply add a Environment=PYTHONUNBUFFERED=anystringhere to the service definition! Nice!
    – Jeff Saxe
    Sep 2, 2022 at 16:00

4 Answers 4

52

The best I could come up with:

>>> import os
>>> import sys
>>> unbuffered = os.fdopen(sys.stdout.fileno(), 'w', 0)
>>> unbuffered.write('test')
test>>> 
>>> sys.stdout = unbuffered
>>> print 'test'
test

Tested on GNU/Linux. It seems it should work on Windows too. If I knew how to reopen sys.stdout, it would be much easier:

sys.stdout = open('???', 'w', 0)

References:
http://docs.python.org/library/stdtypes.html#file-objects
http://docs.python.org/library/functions.html#open
http://docs.python.org/library/os.html#file-object-creation

[Edit]

Note that it would be probably better to close sys.stdout before overwriting it.

3
  • 1
    stdout still lives in sys.__stdout__ Feb 28, 2014 at 10:35
  • 23
    Note that this doesn't work in Py3K. ValueError: can't have unbuffered text I/O
    – vbo
    Sep 12, 2014 at 14:27
  • 2
    The answers on this question are terribly out of date. Please refer to the linked question for Python 3.3+ answers: stackoverflow.com/q/107705/2111778
    – xjcl
    Nov 21, 2022 at 4:04
40

You could always pass the -u parameter in the shebang line:

#!/usr/bin/python -u
7
  • 25
    See title: 1. OP knows about it; 2. wants to do it programmatically.
    – Tobu
    Jan 23, 2011 at 1:32
  • 21
    The author implies command-line usage, not indicating familiarity with the alternate invocation offered by the shell (shebang). Thanks for your helpful feedback though.
    – mikewaters
    Jan 25, 2011 at 7:31
  • 15
    This doesn't seem to work with the "env trick". #!/usr/bin/env python -u. I get the following error /usr/bin/env: python -u: No such file or directory. If I remove the -u it works again. Sep 25, 2014 at 8:44
  • 2
    (me again!). Here is a solution to that problem I discussed in my two comments above: stackoverflow.com/questions/17458528/why-does-this-snippet-work . (But yes, this isn't related to the original question here) Sep 26, 2014 at 9:51
  • 2
    As Aaron says, plainly adding the option to the shebang doesn't work. Instead, pass the -S option to the env command, like so: #!/usr/bin/env -S python3 -u Oct 9, 2020 at 14:10
9

Assuming you're on Windows:

msvcrt.setmode(sys.stdout.fileno(), os.O_BINARY)

... and on Unix:

fl = fcntl.fcntl(sys.stdout.fileno(), fcntl.F_GETFL)
fl |= os.O_SYNC
fcntl.fcntl(sys.stdout.fileno(), fcntl.F_SETFL, fl)

(Unix copied in from commented solution, rather than linking.)

4
  • sorry, forgot to add that. on linux (ubuntu) May 19, 2009 at 10:27
  • 5
    @Martin DeMello: Please do not add new facts in comments. Please update your question with new facts. New facts in comments are hard to find.
    – S.Lott
    May 19, 2009 at 10:37
  • 1
    Supported on unix, linux, and anything posix: stackoverflow.com/questions/107705/python-output-buffering/…
    – Tobu
    Jan 23, 2011 at 1:41
  • Note that this does not work with python2 (no error, output is still buffered). Found out when running a script with the wrong interpreter Mar 31, 2021 at 14:22
8

EDIT (Oct 2020). As pointed out in a note to this answer, in Python3, stderr is buffered too.

You might use the fact that stderr is never buffered and try to redirect stdout to stderr:

import sys
#buffered output is here
doStuff()

oldStdout = sys.stdout
sys.stdout = sys.stderr
#unbuffered output from here on

doMoreStuff()
sys.stdout = oldStdout

#the output is buffered again
doEvenMoreStuff()
3
  • 6
    a hack, but a very cute one :) doesn't really work if you need your stderr separate but nice idea May 20, 2009 at 7:21
  • thanks. as to the separation: you can't get it all, can you :) May 20, 2009 at 7:55
  • 2
    "stderr is never buffered" - no longer true; in python3, stderr is line buffered if to a tty, fully buffered if to a file or pipe.
    – Don Hatch
    Oct 4, 2020 at 1:53

Not the answer you're looking for? Browse other questions tagged or ask your own question.