178

I have a script that outputs 'hi', sleeps for a second, outputs 'hi', sleeps for 1 second, and so on and so forth. Now I thought I would be able to tackle this problem with this model.

var spawn = require('child_process').spawn,
temp    = spawn('PATH TO SCRIPT WITH THE ABOVE BEHAVIOUR');

temp.stdout.pipe(process.stdout);

Now the problem is that the task needs to be finished in order for the output to be displayed. As I am understanding it, this is due to the fact that the newly spawned process takes execution control. Obviously node.js does not support threads so any solutions? My idea was to possibly run two instances, first one for the specific purpose of creating the task and have it pipe the output to process of the second instance, considering this can be achieved.

3
  • 6
    If child process is written python then do not forget to pass -u flag for it to not buffer console output, otherwise it will look like script is not live stackoverflow.com/a/49947671/906265
    – Ivar
    Nov 13, 2018 at 7:21
  • Use npmjs.com/package/cross-spawn instead of anything else. It's just better. Mar 17, 2020 at 23:06
  • @foklepoint Did you manage to get it right in the end? I am currently facing a similar problem and none of the answers seem to work for me May 28, 2022 at 17:59

11 Answers 11

226

It's much easier now (6 years later)!

Spawn returns a childObject, which you can then listen for events with. The events are:

  • Class: ChildProcess
    • Event: 'error'
    • Event: 'exit'
    • Event: 'close'
    • Event: 'disconnect'
    • Event: 'message'

There are also a bunch of objects from childObject, they are:

  • Class: ChildProcess
    • child.stdin
    • child.stdout
    • child.stderr
    • child.stdio
    • child.pid
    • child.connected
    • child.kill([signal])
    • child.send(message[, sendHandle][, callback])
    • child.disconnect()

See more information here about childObject: https://nodejs.org/api/child_process.html

Asynchronous

If you want to run your process in the background while node is still able to continue to execute, use the asynchronous method. You can still choose to perform actions after your process completes, and when the process has any output (for example if you want to send a script's output to the client).

child_process.spawn(...); (Node v0.1.90)

var spawn = require('child_process').spawn;
var child = spawn('node ./commands/server.js');

// You can also use a variable to save the output 
// for when the script closes later
var scriptOutput = "";

child.stdout.setEncoding('utf8');
child.stdout.on('data', function(data) {
    //Here is where the output goes

    console.log('stdout: ' + data);

    data=data.toString();
    scriptOutput+=data;
});

child.stderr.setEncoding('utf8');
child.stderr.on('data', function(data) {
    //Here is where the error output goes

    console.log('stderr: ' + data);

    data=data.toString();
    scriptOutput+=data;
});

child.on('close', function(code) {
    //Here you can get the exit code of the script

    console.log('closing code: ' + code);

    console.log('Full output of script: ',scriptOutput);
});

Here's how you would use a callback + asynchronous method:

var child_process = require('child_process');

console.log("Node Version: ", process.version);

run_script("ls", ["-l", "/home"], function(output, exit_code) {
    console.log("Process Finished.");
    console.log('closing code: ' + exit_code);
    console.log('Full output of script: ',output);
});

console.log ("Continuing to do node things while the process runs at the same time...");

// This function will output the lines from the script 
// AS is runs, AND will return the full combined output
// as well as exit code when it's done (using the callback).
function run_script(command, args, callback) {
    console.log("Starting Process.");
    var child = child_process.spawn(command, args);

    var scriptOutput = "";

    child.stdout.setEncoding('utf8');
    child.stdout.on('data', function(data) {
        console.log('stdout: ' + data);

        data=data.toString();
        scriptOutput+=data;
    });

    child.stderr.setEncoding('utf8');
    child.stderr.on('data', function(data) {
        console.log('stderr: ' + data);

        data=data.toString();
        scriptOutput+=data;
    });

    child.on('close', function(code) {
        callback(scriptOutput,code);
    });
}

Using the method above, you can send every line of output from the script to the client (for example using Socket.io to send each line when you receive events on stdout or stderr).

Synchronous

If you want node to stop what it's doing and wait until the script completes, you can use the synchronous version:

child_process.spawnSync(...); (Node v0.11.12+)

Issues with this method:

  • If the script takes a while to complete, your server will hang for that amount of time!
  • The stdout will only be returned once the script has finished running. Because it's synchronous, it cannot continue until the current line has finished. Therefore it's unable to capture the 'stdout' event until the spawn line has finished.

How to use it:

var child_process = require('child_process');

var child = child_process.spawnSync("ls", ["-l", "/home"], { encoding : 'utf8' });
console.log("Process finished.");
if(child.error) {
    console.log("ERROR: ",child.error);
}
console.log("stdout: ",child.stdout);
console.log("stderr: ",child.stderr);
console.log("exist code: ",child.status);
10
  • 19
    +1, this should be chosen as the right answer now. Just a note, the data variable in the callback comes in as Buffer object. You can use child.stdout.setEncoding('utf8') if you want utf8 strings coming in.
    – Ashish
    Feb 17, 2016 at 16:54
  • 2
    This doesn't work if you need the information from stdout asynchronously, that is, while the remaining program continues, if the process continues. Nov 18, 2018 at 15:45
  • 2
    Hey @ChristianHujer ! I updated my answer to include both async and sync :D
    – Katie
    Aug 9, 2019 at 18:00
  • 1
    FYI, it's even easier if you use pipe or pipeline or pass in the appropriate options to spawn.
    – RichS
    May 10, 2020 at 20:17
  • 1
    If you're using spawnSync and want the values to come back as a string instead of a buffer you'll want to feed in { encoding: 'utf-8' } as part of the options (3rd param).
    – K. Waite
    Jul 18, 2020 at 19:08
119

I'm still getting my feet wet with Node.js, but I have a few ideas. first, I believe you need to use execFile instead of spawn; execFile is for when you have the path to a script, whereas spawn is for executing a well-known command that Node.js can resolve against your system path.

1. Provide a callback to process the buffered output:

var child = require('child_process').execFile('path/to/script', [ 
    'arg1', 'arg2', 'arg3', 
], function(err, stdout, stderr) { 
    // Node.js will invoke this callback when process terminates.
    console.log(stdout); 
});  

2. Add a listener to the child process' stdout stream (9thport.net)

var child = require('child_process').execFile('path/to/script', [ 
    'arg1', 'arg2', 'arg3' ]); 
// use event hooks to provide a callback to execute when data are available: 
child.stdout.on('data', function(data) {
    console.log(data.toString()); 
});

Further, there appear to be options whereby you can detach the spawned process from Node's controlling terminal, which would allow it to run asynchronously. I haven't tested this yet, but there are examples in the API docs that go something like this:

child = require('child_process').execFile('path/to/script', [ 
    'arg1', 'arg2', 'arg3', 
], { 
    // detachment and ignored stdin are the key here: 
    detached: true, 
    stdio: [ 'ignore', 1, 2 ]
}); 
// and unref() somehow disentangles the child's event loop from the parent's: 
child.unref(); 
child.stdout.on('data', function(data) {
    console.log(data.toString()); 
});
4
  • 9
    Bonus points if you can explain how to do this with exec() as I need to execute a shell cmd.
    – DynamicDan
    Apr 10, 2014 at 19:35
  • 3
    You can use child.spawn() with the shell option set to true. nodejs.org/api/…
    – CedX
    Jun 2, 2016 at 14:35
  • 12
    You can also pipe child.stdout directly to process.stdout with child.stdout.pipe(process.stdout);
    – darkadept
    Oct 19, 2016 at 20:43
  • @DynamicDan javascript let childProcess = exec ( './script-to-run --arg1 arg1value', ( error, stdout, stderror ) => { console.log( '[CALLBACK]: ' + error ); // or stdout or stderror } ); // Same as with spawn: childProcess.stdout.on ( 'data', ( data ) => { console.log( '[LIVE]: ' + data ); // Here's your live data! } );
    – Rik
    Jun 8, 2018 at 14:05
60

Here is the cleanest approach I've found:

require("child_process").spawn('bash', ['./script.sh'], {
  cwd: process.cwd(),
  detached: true,
  stdio: "inherit"
});
5
  • 48
    What is it doing, exactly? Why does it work? Why is this the cleaner approach? Sep 16, 2019 at 5:42
  • @raisinrising It is setting the process to inherit the stdio handles (including stdout). It is cleaner because there is only one function invocation (sorry for the partial comment that I deleted, a pack of index cards was dropped on my keyboard while I was away) Apr 16, 2022 at 2:14
  • Simple and worked for me. I didnt need the detached option and my process was actually attached 😉
    – Pierre
    Jan 6, 2023 at 7:54
  • Agreed, this is clean, simple, and allows processes to output directly to console (i.e. if there's a fancy spinning character or whatever). Aug 20, 2023 at 17:17
  • SHOULD BE ACCEPTED ANSWER +1
    – danday74
    Dec 6, 2023 at 18:40
19

I had a little trouble getting logging output from the "npm install" command when I spawned npm in a child process. The realtime logging of dependencies did not show in the parent console.

The simplest way to do what the original poster wants seems to be this (spawn npm on windows and log everything to parent console):

var args = ['install'];

var options = {
    stdio: 'inherit' //feed all child process logging into parent process
};

var childProcess = spawn('npm.cmd', args, options);
childProcess.on('close', function(code) {
    process.stdout.write('"npm install" finished with code ' + code + '\n');
});
0
9

PHP-like passthru

import { spawn } from 'child_process';

export default async function passthru(exe, args, options) {
    return new Promise((resolve, reject) => {
        const env = Object.create(process.env);
        const child = spawn(exe, args, {
            ...options,
            env: {
                ...env,
                ...options.env,
            },
        });
        child.stdout.setEncoding('utf8');
        child.stderr.setEncoding('utf8');
        child.stdout.on('data', data => console.log(data));
        child.stderr.on('data', data => console.log(data));
        child.on('error', error => reject(error));
        child.on('close', exitCode => {
            console.log('Exit code:', exitCode);
            resolve(exitCode);
        });
    });
}

Usage

const exitCode = await passthru('ls', ['-al'], { cwd: '/var/www/html' })
0
4

child:

setInterval(function() {
    process.stdout.write("hi");
}, 1000); // or however else you want to run a timer

parent:

require('child_process').fork('./childfile.js');
// fork'd children use the parent's stdio
4

I found myself requiring this functionality often enough that I packaged it into a library called std-pour. It should let you execute a command and view the output in real time. To install simply:

npm install std-pour

Then it's simple enough to execute a command and see the output in realtime:

const { pour } = require('std-pour');
pour('ping', ['8.8.8.8', '-c', '4']).then(code => console.log(`Error Code: ${code}`));

It's promised based so you can chain multiple commands. It's even function signature-compatible with child_process.spawn so it should be a drop in replacement anywhere you're using it.

1
  • 1
    @KodieGrantham glad it's working for you! Y'all seem like you're doing some cool work so I hope it keeps you running.
    – Joel B
    Sep 25, 2019 at 15:40
4

Adding a sample for exec as I too had needed live feedback and wasn't getting any until after the script finished. exec does return an EventEmitter, contrary to the many claims that only spawn works in such a way.

This supplements the comment I made to the accepted answer more thoroughly.

The interface for exec is similar to spawn:

// INCLUDES
import * as childProcess from 'child_process'; // ES6 Syntax
    
    
// DEFINES
let exec = childProcess.exec; // Use 'var' for more proper 
                              // semantics, or 'const' it all
                              // if that's your thing; though 'let' is 
                              // true-to-scope;

// Return an EventEmitter to work with, though
// you can also chain stdout too: 
// (i.e. exec( ... ).stdout.on( ... ); )
let childProcess = exec
(
    './binary command -- --argument argumentValue',
    ( error, stdout, stderr ) =>
    {    // When the process completes:
        if( error )
        {   
            console.log( `${error.name}: ${error.message}` );
            console.log( `[STACK] ${error.stack}` );
        }
            
        console.log( stdout );
        console.log( stderr );
        callback();                // Gulp stuff
    }
);

Now its as simple as registering an event handler for stdout:

childProcess.stdout.on( 'data', data => console.log( data ) );

And for stderr:

childProcess.stderr.on( 'data', data => console.log( `[ERROR]: ${data}` ) );

You can also pipe stdout to the main process' stdout:

childProcess.stdout.pipe( process.stdout );

Not too bad at all - HTH

1

I was interested into running a script that gets the input and outputs from my terminal, and that will close my process once the child script finishes.

import { spawn } from 'node:child_process'
import process from 'node:process'

const script = spawn('path/to/script', { stdio: 'inherit' })
script.on('close', process.exit)
0

Read subprocess stdout line-by-line with readline

Reading line by line live is a common use case since many CLI utilities produce line-oriented output.

Sample program:

main.js

#!/usr/bin/env node

const childProcess = require('child_process')
const readline = require('readline')

const p = childProcess.spawn('./count.js', ['2'])
;(async function() {
const rl = readline.createInterface({ input: p.stdout })
for await (const line of rl) {
  console.log('read: ' + line)
}
})()

Test helper:

count.js

#!/usr/bin/env node
(async function() {
const sleep = ms => new Promise(resolve => setTimeout(resolve, ms))
let i = 0
let max
if (process.argv.length > 2) {
  max = parseInt(process.argv[2])
}
while (true) {
  console.log(i)
  if (i === max) break
  await sleep(1000)
  i++
}
})()

Output of ./main.js:

read: 0
read: 1
read: 2

and each line appears one second after the previous one. Therefore we understand that this is reading lines immediately as soon as they become available as desired, and not waiting until the subprocess terminates to read everything at once.

A more dedicated question for this specific case is present at: Parse output of spawned node.js child process line by line

Tested on Node.js v16.14.2, Ubuntu 23.04.

-1

I ran into a situation where none of the above worked when I was spawning a Python 3 script. I would get data from stdout, but only once the child terminated.

As it turns out, Python buffers stdout by default. It's possible to disable stdout buffering by including -u as a command line parameter to python3.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.