12

I spawn a child process like this:

var child = require('child_process');
var proc = child.spawn('python', ['my_script.py', '-p', 'example']);

I also set some data handling:

proc.stdin.setEncoding('utf8');
proc.stdout.setEncoding('utf8');
proc.stderr.setEncoding('utf8');

proc.stdout.on('data', function (data) {
  console.log('out: ' + data);
});

proc.stderr.on('data', function (data) {
  console.log('err: ' + data);
});

proc.on('close', function (code) {
  console.log('subprocess exited with status ' + code);
  proc.stdin.end();
});

My Python script reads lines from stdin and for each line does some operations and prints to stdout. It works fine in the shell (I write a line and I get the output immediately) but when I do this in Node:

for (var i = 0; i < 10; i++) {
  proc.stdin.write('THIS IS A TEST\n');
}

I get nothing.

I got to fix it calling proc.stdin.end() but that also terminates the child process (which I want to stay in background, streaming data).

I also triggered a flush filling the buffer with lots of writes, but that's not really an option.

Is there any way to manually flush the stream?

5
  • If write() returns true, then it has been flushed. If it returns false, you can listen for the 'drain' event to verify when the write buffer has been completely flushed. You're saying you can write data, and it never gets flushed at all? I would suspect the child process is chunking the data and looking for something other than \n to breakup the input. When you overflow its input buffer, or end the stream, the child is forced to process the data at that time. Commented Jun 17, 2013 at 5:14
  • @BretCopeland yup, write() returns true. I thought maybe it wasn't the write being queued... but perhaps the read? It's weird, but I'm sure the children is not chunking the data since it works in the shell, and I can verify it's looking for '\n'. Running the script and just writing lines and pressing enter works as expected (data is written to stdout as enter is pressed and the process doesn't exit). I'm really stuck here.
    – kaoD
    Commented Jun 17, 2013 at 5:30
  • 1
    Same problem here buddy. spawned grep never outputs.
    – kilianc
    Commented Sep 16, 2013 at 17:02
  • @kilianc sorry to hear that. I ended up using Python against my wishes. It did the job anyways. Perhaps it's time to file a Node bug report.
    – kaoD
    Commented Sep 16, 2013 at 20:37
  • @kaoD the problem in my case was grep. The guy wants a --line-buffered option otherwise it will buffer ~3k lines before outputting something.
    – kilianc
    Commented Sep 16, 2013 at 23:12

1 Answer 1

4

You are not flushing the output from Python after print statement. I had similar problem and @Alfe answered my question. Take a look at this:

Stream child process output in flowing mode

1
  • Unfortunately I discarded the code so I can't check if it solves the problem. Thanks for the answer!
    – kaoD
    Commented Oct 4, 2013 at 0:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.