I spawn a child process like this:
var child = require('child_process');
var proc = child.spawn('python', ['my_script.py', '-p', 'example']);
I also set some data handling:
proc.stdin.setEncoding('utf8');
proc.stdout.setEncoding('utf8');
proc.stderr.setEncoding('utf8');
proc.stdout.on('data', function (data) {
console.log('out: ' + data);
});
proc.stderr.on('data', function (data) {
console.log('err: ' + data);
});
proc.on('close', function (code) {
console.log('subprocess exited with status ' + code);
proc.stdin.end();
});
My Python script reads lines from stdin
and for each line does some operations and prints to stdout
. It works fine in the shell (I write a line and I get the output immediately) but when I do this in Node:
for (var i = 0; i < 10; i++) {
proc.stdin.write('THIS IS A TEST\n');
}
I get nothing.
I got to fix it calling proc.stdin.end()
but that also terminates the child process (which I want to stay in background, streaming data).
I also triggered a flush filling the buffer with lots of writes, but that's not really an option.
Is there any way to manually flush the stream?
write()
returns true, then it has been flushed. If it returns false, you can listen for the'drain'
event to verify when the write buffer has been completely flushed. You're saying you can write data, and it never gets flushed at all? I would suspect the child process is chunking the data and looking for something other than\n
to breakup the input. When you overflow its input buffer, or end the stream, the child is forced to process the data at that time.write()
returnstrue
. I thought maybe it wasn't the write being queued... but perhaps the read? It's weird, but I'm sure the children is not chunking the data since it works in the shell, and I can verify it's looking for '\n'. Running the script and just writing lines and pressing enter works as expected (data is written to stdout as enter is pressed and the process doesn't exit). I'm really stuck here.grep
. The guy wants a--line-buffered
option otherwise it will buffer ~3k lines before outputting something.