1

I tried both pexpect and subprocess.Popen from python to call an external long term background process (this process use socket to communicate with external applications), with following details.

  1. subprocess.Popen(launchcmd, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE) This works fine. I do not need to do anything else. However, because I have to get the output immediately, I choose pexpect to avoid the pipe file buffer problem.

  2. obj= pexpect.spawn(launchcmd, timeout=None) after launching external process, I use a separate thread to do "readline" to read the output of the launched process "obj", and everything is ok.

  3. obj= pexpect.spawn(launchcmd, timeout=None) after launching external process, I did nothing further, i.e., just leave it there. Although, by using the "ps -e" command I can find the launched process, but the launched process seems blocked and cannot communicate in sockets with other applications.

OK. To be more specific, I put some sample code to formulate my question.

import subprocess
import pexpect
import os

t=1
while(True):
    if(t==1):
        background_process="./XXX.out"
        launchcmd = [background_process]
        #---option 3--------
        p=pexpect.spawn(launchcmd, timeout=None) # process launced, problem with socket.
        #---option 1--------
        p=subprocess.Popen(launchcmd, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE) # process launced, everything fine
        t=0 

Could anyone tell me what's wrong with the 3rd option? And if it is due to the fact that I did not use a separate thread to manipulate the output, why 1st option works with subprocess.popen? I suspect there is something wrong with pexpect to launch a process using socket, but I am not sure, especially considering option 2 works well.

4
  • I think you may be mistaking the difference between piping and buffers - a pipe is just a conduit, if it's buffered or not depends on the application thats writing to it (before you read). Readline is buffering -- until you get a newline. read() is buffered because it reads until it gets an EOF. The first example you provide should be fine, if there's data to read you'll get it, but if your launchcmd is buffering it won't be instant. Dec 7, 2010 at 12:41
  • hi, synthesizerpatel, thank you for your comment. but I think it does not really help my question.
    – pepero
    Dec 7, 2010 at 14:02
  • @synthesizerpatel: on Linux, a pipe is buffered, by the kernel. This buffering is independent of the in-process buffering and so far, I haven't found a way of turning it off.
    – Fred Foo
    Apr 15, 2011 at 15:48
  • Can you explain what you mean with sockets? I cannot see them in your code.
    – pyfex
    May 11, 2011 at 14:25

1 Answer 1

1

I think that you are making this too complicated.

Yes, it is a good idea to use a pty instead of a pipe to communicate with the background process because most applications recognize tty/pty devices and switch to using unbuffered output, (or at least line-buffered).

But why pexpect? Just use Python's pty module. First call openpty to get some filehandles and then use Popen to spawn the process. Example code is found in the following question (the answer with the green checkmark) Python Run a daemon sub-process & read stdout

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.