3

I wrote this little test class, based upon a Python issue - closed/fixed, and it seems to be occurring in Python 2.7.1 on Fedora 15.

from subprocess import Popen, PIPE
from threading import Thread

OUTPUT = "asl;dkfhqwiouethnjzxvnhsldfhuyasdhofuiweqamchuisemfawepfhuaipwemhfuaehfclkuehnflw ehfcoiuwehfiuwmhefuiwehfoiuhSfcl hfulh fuiqhuif huiwafh uiahf iUH fhh flkJH fl HASLFuhSAIULFhSUA HFulSAHfOI SUFChiuwqhncriouweycriuowanbyoUIWCryu iWyruawyrouiWYRcoiu YCRoiuNr uyr oUIAWryocIUWRNyc owuroiuNr cuWyrnawueitcnoy U IuiR yiuowaYnorc oWIUAr coiury iuoAW rnuoi asdfsdfd\n"


class X(Thread):
    def __init__(self):
        Thread.__init__(self)

    def run(self):
        print("Running")
        for i in xrange(10):
            s = Popen(
                "cat /tmp/junk",
                shell=True,
                stdout=PIPE,
                universal_newlines=True
            )
            output = s.communicate()[0]
            if not output == OUTPUT:
                print("Error: %r" % output)


XThreads = set([])

for i in xrange(1000):
    XThreads.add(X())

for x in XThreads:
    x.start()

Just create a file, /tmp/junk in this case, that has the contents of OUTPUT, less the last newline.

Running this, you would expect to see "Running" on every line. However, sometimes it shows " Running" or "RunningRunning\n\nRunning".

(Removed citation to actual problem, as this was a false-symptom, thanks to @phihag's answer).

The actual issue: https://stackoverflow.com/questions/9338409/python-subprocess-popen-corrupts-binary-streams

1 Answer 1

2

The behavior you're seeing has nothing to do with subprocess; I can reproduce it with:

import threading
def run(): print("Running")
for x in [threading.Thread(target=run) for i in range(1000)]:
    x.start()

This is because Python's print is not thread-safe. To avoid the race condition between printing out text and newline, directly write to stdout, like this:

import threading,sys
def run(): sys.stdout.write("Running\n")
for x in [threading.Thread(target=run) for i in range(1000)]:
    x.start()

This assumes the underlying write calls to stdout are thread-safe. That depends on the platform.

3
  • thanks, I think I've found my issue, but require uploading multiple files to provide a working example - not sure how to do that here...can you help? Feb 18, 2012 at 3:35
  • more interestingly, the real issue - stackoverflow.com/questions/9338409/… Feb 18, 2012 at 4:18
  • @CrackerJack9 With a few modifications, you should be able to concatenate any multi-file Python program into one. If that's not possible (or the program is really long), upload the files to gist or pastebin
    – phihag
    Feb 18, 2012 at 9:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.