5

I have my app running on http://talkwithstranger.com/ and I have deployed it on AWS EC2. I use this command

sudo nohup node index.js &

To continue running my Node JS server even if I close my terminal and exit my SSH.

However, after 2 days everytime I wake up and I find out that the node server itself stops automatically. I checked the running processes by using

ps -ef

and my node script is not there.

Google Chrome say site DNS not found, because NodeJS Express is not running of course to serve my html file, but why it stops itself?

What is causing this unexpected shutdown of my server after every 2 days? I have to manually run nohup again to run it again.

Does nohup has a time to expire or something ?

5
  • Probably an uncaught exception, possibly caused by a memory leak. Check the bottom of your nohup.out
    – Paul
    Commented Apr 28, 2016 at 7:16
  • @Paulpro I checked my nohup.out file,. Does the nohup,out file records errors as well ? How can I track such an uncaught exception because it only happens during late night when I am sleeping. Is there a way to restart NodeJS automatically by pinging the site URL and it no response comes?
    – Faizan
    Commented Apr 28, 2016 at 7:21
  • @Paulpro Just found this statement in my nohup,out file which sends the node console output to a file FATAL ERROR: Committing semi space failed. Allocation failed - process out of memory Could this be a reason ? How to allocate more memory to my Node process?
    – Faizan
    Commented Apr 28, 2016 at 7:22
  • That would be the reason, yes. You can use a tool like forever to restart the process if it crashes, but ideally you should fix the problem (memory leak?) that causes the process to run out of memory. You may need to do some profiling to find it.
    – Paul
    Commented Apr 28, 2016 at 7:23
  • Run it as sudo nohup node index.js > /var/log/server.log & and see what is the final crash logs look like Commented Sep 7, 2019 at 15:50

2 Answers 2

17
+50

You should run node.js using a service / process manager. You can use something basic such as forever or supervisord but I would actually advise you to take a look at PM2.

It can do a lot of things - one of them being that it manages your process, makes sure it keeps running, restarts it when it fails, manages the logs, etc. You can also have it autostart when you restart the server.

It becomes really powerful in combination with https://pm2.io, because this enables you to monitor your server's metrics such as CPU and memory remotely and see whether exceptions happened, and much more (such as even remotely updating the software by pulling from git). However, they no longer offer a free plan unfortunately - their plans now start at $79/month, which is a pity. But don't worry, the PM2 application itself is still free and open source, only the monitoring costs money.

Basic usage of PM2:

npm install -g pm2

...to install PM2.

pm2 start my_script.js

Starts a script and lets it run in background.

pm2 status

Shows the status of any running scripts.

pm2 restart all

Restarts all running scripts.

pm2 kill

Stops all scripts and completely shuts down the PM2 daemon.

pm2 monit

Monitors CPU/RAM usage and shows it.

pm2 logs

Shows the last 20 output and error log lines and starts streaming live logs to the console. The logs are stored in the folder ~/.pm2/logs.

Using PM2, your script will not stop - at most, it will restart. And if it does you will be able to more easily understand why because you can easily access logs and watch what happenes with memory usage, etc.

Extra tips:

To avoid filling up the harddisk with logfiles, I recommend installing the module pm2-logrotate:

pm2 install pm2-logrotate

To automatically launch PM2 with the same script on startup when the server starts, you can first save the current configuration:

pm2 save

...and then use the following command to install a startup script - follow the instructions displayed, which will be different based on the exact OS you are using:

pm2 startup

To use PM2 in a more advanced way with multiple processes, custom environment variables, etc., take a look at ecosystem files.

8
  • Just found this statement in my nohup,out file which sends the node console output to a file FATAL ERROR: Committing semi-space failed. Allocation failed - process out of memory Could this be a reason ? How to allocate more memory to my Node process? Or how to rectify this problem?
    – Faizan
    Commented Apr 28, 2016 at 7:25
  • 1
    You have to get rid of your memory leak. See toptal.com/nodejs/debugging-memory-leaks-node-js-applications for example.
    – CherryDT
    Commented Apr 28, 2016 at 7:34
  • 1
    It should work the same with Amazon Linux. Personally I prefer Ubuntu instances though. It's a matter of taste I guess. (and yes these warnings are perfectly normal.)
    – CherryDT
    Commented Sep 8, 2019 at 13:03
  • 2
    I have updated the post a bit to reflect the situation in 2019.
    – CherryDT
    Commented Sep 11, 2019 at 14:16
  • 1
    This is the best explanation, particularly the extra tips. They should paste this to the pm2 docs because theirs are much less clear. Commented Jan 3, 2020 at 8:21
3

You can try forever.Install using the following command.

npm install -g forever

Then just start forever:

forever start index.js

Another better option for production use is pm2.You can install pm2 with below command

npm install -g pm2

# or

yarn global add pm2

start server

pm2 start index.js

The best thing is you can achieve load balancing with pm2(utilize all available CPU)

pm2 start index.js -i max

For more info, you can visit pm2 documentation page.

6
  • How forever is different from using nohup ??
    – Faizan
    Commented Apr 28, 2016 at 7:24
  • 1
    In case of forever, it will restart Node in the event of a crash. I think your node application generating any exception, that's why its getting break. Commented Apr 28, 2016 at 7:36
  • After running forever start index.js how can I stop my script if I want to stop it myself?
    – Faizan
    Commented Apr 28, 2016 at 7:37
  • forever stop index.js Commented Apr 28, 2016 at 7:41
  • 1
    These are actions for forever: start Start SCRIPT as a daemon stop Stop the daemon SCRIPT by Id|Uid|Pid|Index|Script stopall Stop all running forever scripts restart Restart the daemon SCRIPT restartall Restart all running forever scripts list List all running forever scripts config Lists all forever user configuration set <key> <val> Sets the specified forever config <key> clear <key> Clears the specified forever config <key> Commented Apr 28, 2016 at 7:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.