Under Linux, how do I find out which process is using the swap space more?
-
30Your accepted answer is wrong. Consider changing it to lolotux's answer, which is actually correct. – jterrace Jul 25 '12 at 17:12
-
@jterrace is correct, I don't have as much swap space as the sum of the values in the SWAP column in top. – akostadinov Aug 30 '12 at 16:58
-
1iotop is a very useful command which will show live stats of io and swap usage per process/thread – sunil Sep 1 '15 at 7:22
-
@jterrace, consider stating whose accepted-answer-of-the-day is wrong. Six years later, the rest of us have no idea whether you were referring to David Holm's answer (the currently-accepted one as of today) or some other answer. (Well, I see you also said David Holm's answer is wrong, as a comment on his answer... so I guess you probably meant his.) – Don Hatch Jul 10 '18 at 11:08
Run top then press OpEnter. Now processes should be sorted by their swap usage.
Here is an update as my original answer does not provide an exact answer to the problem as pointed out in the comments. From the htop FAQ:
It is not possible to get the exact size of used swap space of a process. Top fakes this information by making SWAP = VIRT - RES, but that is not a good metric, because other stuff such as video memory counts on VIRT as well (for example: top says my X process is using 81M of swap, but it also reports my system as a whole is using only 2M of swap. Therefore, I will not add a similar Swap column to htop because I don't know a reliable way to get this information (actually, I don't think it's possible to get an exact number, because of shared pages).
-
140From the docs, the SWAP column in top seems to just show how much swap would be necessary if the entire process was swapped out, rather than how much of the process is actually swapped at present. From what I can tell after a short search, there is no way to determine how much of each process is swapped out at the moment. The author of htop refuses to put in such a column because of this (I do see CNSWAP and NSWAP columns, but they don't seem to do anything on my machine): htop.sourceforge.net/index.php?page=faq – yukondude Nov 11 '09 at 19:25
-
6@yukondude is right, the SWAP column in top is just VIRT - RES and this information is a kind of useless in this context. There is no compensation for e.g. shared memory of mapped video RAM. Also not all memory might have been referenced by the process yet. In this case it's not necessary for the OS to read the complete binary from disk into memory, and thus the value of RES doesn't include this part of memory. – Bart Mar 5 '12 at 14:53
-
-
Thankfully that's what the comments are for @jterrace :) (although admittedly you have to read them :S... not sure what atrain is referring to, I hope it's yukondude) – AJP Feb 21 '14 at 16:15
-
11Regarding the comment not working anymore: It seems more recent versions of top no longer have 'O' set as the key for choosing the sort fields. When using the ? key you can see the actual program name and version, procps-ng being the latest version. This is a fork by Debian, Fedora and openSUSE: gitorious.org/procps . If you would still like to do a sort on the SWAP column: Use the 'f' key to see the fields, use the arrow keys to go to SWAP and use 's' to set the sort, then 'q'. – Pieter VN Jan 20 '15 at 7:48
The best script I found is on this page : http://northernmost.org/blog/find-out-what-is-using-your-swap/
Here's one variant of the script and no root needed:
#!/bin/bash
# Get current swap usage for all running processes
# Erik Ljungstrom 27/05/2011
# Modified by Mikko Rantalainen 2012-08-09
# Pipe the output to "sort -nk3" to get sorted output
# Modified by Marc Methot 2014-09-18
# removed the need for sudo
SUM=0
OVERALL=0
for DIR in `find /proc/ -maxdepth 1 -type d -regex "^/proc/[0-9]+"`
do
PID=`echo $DIR | cut -d / -f 3`
PROGNAME=`ps -p $PID -o comm --no-headers`
for SWAP in `grep VmSwap $DIR/status 2>/dev/null | awk '{ print $2 }'`
do
let SUM=$SUM+$SWAP
done
if (( $SUM > 0 )); then
echo "PID=$PID swapped $SUM KB ($PROGNAME)"
fi
let OVERALL=$OVERALL+$SUM
SUM=0
done
echo "Overall swap used: $OVERALL KB"
-
Here is the copy, in case the link dies: gitorious.org/dolanormisc/scripts/blobs/master/getswapused – TautrimasPajarskas Apr 29 '12 at 8:50
-
4funny though, I get
Overall swap used: 260672 KB
, while free shows738932
as used... – Doncho Gunchev Dec 18 '12 at 16:13 -
29Same output ten times faster:
for file in /proc/*/status ; do awk '/Tgid|VmSwap|Name/{printf $2 " " $3}END{ print ""}' $file; done | grep kB | sort -k 3 -n
for Debian/RH 6x+, Arch, Ubuntu (RH 5x hasVmSize
) (source). Like @dgunchev it does gives much less total swap thanfree
. @Tensibai does not work on Arch; your awk may lack something. – tuk0z Nov 19 '15 at 12:41 -
1
-
3The author has a follow-up post about how to do it using
top
: northernmost.org/blog/swap-usage-5-years-later – Jack Valmadre Feb 22 '17 at 15:27
Here's another variant of the script, but meant to give more readable output (you need to run this as root to get exact results):
#!/bin/bash
# find-out-what-is-using-your-swap.sh
# -- Get current swap usage for all running processes
# --
# -- rev.0.3, 2012-09-03, Jan Smid - alignment and intendation, sorting
# -- rev.0.2, 2012-08-09, Mikko Rantalainen - pipe the output to "sort -nk3" to get sorted output
# -- rev.0.1, 2011-05-27, Erik Ljungstrom - initial version
SCRIPT_NAME=`basename $0`;
SORT="kb"; # {pid|kB|name} as first parameter, [default: kb]
[ "$1" != "" ] && { SORT="$1"; }
[ ! -x `which mktemp` ] && { echo "ERROR: mktemp is not available!"; exit; }
MKTEMP=`which mktemp`;
TMP=`${MKTEMP} -d`;
[ ! -d "${TMP}" ] && { echo "ERROR: unable to create temp dir!"; exit; }
>${TMP}/${SCRIPT_NAME}.pid;
>${TMP}/${SCRIPT_NAME}.kb;
>${TMP}/${SCRIPT_NAME}.name;
SUM=0;
OVERALL=0;
echo "${OVERALL}" > ${TMP}/${SCRIPT_NAME}.overal;
for DIR in `find /proc/ -maxdepth 1 -type d -regex "^/proc/[0-9]+"`;
do
PID=`echo $DIR | cut -d / -f 3`
PROGNAME=`ps -p $PID -o comm --no-headers`
for SWAP in `grep Swap $DIR/smaps 2>/dev/null| awk '{ print $2 }'`
do
let SUM=$SUM+$SWAP
done
if (( $SUM > 0 ));
then
echo -n ".";
echo -e "${PID}\t${SUM}\t${PROGNAME}" >> ${TMP}/${SCRIPT_NAME}.pid;
echo -e "${SUM}\t${PID}\t${PROGNAME}" >> ${TMP}/${SCRIPT_NAME}.kb;
echo -e "${PROGNAME}\t${SUM}\t${PID}" >> ${TMP}/${SCRIPT_NAME}.name;
fi
let OVERALL=$OVERALL+$SUM
SUM=0
done
echo "${OVERALL}" > ${TMP}/${SCRIPT_NAME}.overal;
echo;
echo "Overall swap used: ${OVERALL} kB";
echo "========================================";
case "${SORT}" in
name )
echo -e "name\tkB\tpid";
echo "========================================";
cat ${TMP}/${SCRIPT_NAME}.name|sort -r;
;;
kb )
echo -e "kB\tpid\tname";
echo "========================================";
cat ${TMP}/${SCRIPT_NAME}.kb|sort -rh;
;;
pid | * )
echo -e "pid\tkB\tname";
echo "========================================";
cat ${TMP}/${SCRIPT_NAME}.pid|sort -rh;
;;
esac
rm -fR "${TMP}/";
-
2Very nice script. It gives the same information as lolotux's one, but in a better readable way. – Philipp Wendler Oct 31 '12 at 12:53
-
2The only thing I changed was using
args
instead ofcomm
in theps
command since I have a lot of processes with the same name but different arguments (a bunch of python gunicorn processes). I.e.:ps -p $PID -o args --no-headers
– mgalgs Sep 19 '14 at 19:33 -
1Side note the
grep VmSwap $DIR/status 2>/dev/null | awk '{ print $2 }'
could be simplified asawk ' /VmSwap/ { print $2 }'
– Tensibai Sep 1 '15 at 7:58
I did notice this thread is rather old, but if you happen to stumble upon it, as I just did, another answer is: use smem.
Here is a link which tells you both how to install it and how to use it:
http://www.cyberciti.biz/faq/linux-which-process-is-using-swap/
-
This one is good. Here's adapted version from that article to show procs sorted by swap usage with PID added: $ for file in /proc/*/status ; do awk '/^Pid|VmSwap|Name/{printf $2 " " $3}END{ print ""}' $file; done | sort -k 3 -n -r | less – Stan Brajewski Feb 7 '17 at 13:49
-
You should glob /proc/[1-9]*/status to exclude a couple of special /proc entries, and you can combine the sort args as -rnk3 – dland Sep 19 '17 at 12:11
It's not entirely clear if you mean you want to find the process who has most pages swapped out or process who caused most pages to be swapped out.
For the first you may run top
and order by swap (press 'Op'), for the latter you can run vmstat
and look for non-zero entries for 'so'.
Another script variant avoiding the loop in shell:
#!/bin/bash
grep VmSwap /proc/[0-9]*/status | awk -F':' -v sort="$1" '
{
split($1,pid,"/") # Split first field on /
split($3,swp," ") # Split third field on space
cmdlinefile = "/proc/"pid[3]"/cmdline" # Build the cmdline filepath
getline pname[pid[3]] < cmdlinefile # Get the command line from pid
swap[pid[3]] = sprintf("%6i %s",swp[1],swp[2]) # Store the swap used (with unit to avoid rebuilding at print)
sum+=swp[1] # Sum the swap
}
END {
OFS="\t" # Change the output separator to tabulation
print "Pid","Swap used","Command line" # Print header
if(sort) {
getline max_pid < "/proc/sys/kernel/pid_max"
for(p=1;p<=max_pid;p++) {
if(p in pname) print p,swap[p],pname[p] # print the values
}
} else {
for(p in pname) { # Loop over all pids found
print p,swap[p],pname[p] # print the values
}
}
print "Total swap used:",sum # print the sum
}'
Standard usage is script.sh
to get the usage per program with random order (down to how awk
stores its hashes) or script.sh 1
to sort the output by pid.
I hope I've commented the code enough to tell what it does.
-
1Note that
bash
expands the directories in a sorted way (lexical, not numerical). The random order is down to howawk
stores its arrays (hash table) and howfor p in pname
retrieves them. – Stephane Chazelas Aug 5 '16 at 15:06 -
@StephaneChazelas Well, that's not even lexicaly, that's an ascii code sort (as
/proc/1/status
comes after/proc/1992/status
and that the/
has an ascii code above the 9 ascii code. this gives a "random order" look and feel too. I agree with the awk hash table, I took a shortcut here. Feel free to edit the answer to keep the attribution in edit history. – Tensibai Aug 22 '16 at 9:51 -
1
/proc/1/status
would not come after/proc/1992/status
in the C locale where the order is based on the byte value. It does in your locale (or in myen_GB.UTF-8
on a GNU system), because/
there is ignored in the first instance in the collation algorithm (ands
sorts after9
). Compareprintf '/proc/%s/status\n' 1 1992 | LC_ALL=en_GB.UTF-8 sort
withprintf '/proc/%s/status\n' 1 1992 | LC_ALL=C sort
. In locales other thanC
, the sort order is generally not based on the byte value. – Stephane Chazelas Aug 22 '16 at 10:04 -
@StephaneChazelas Nice point, didn't though about the locale. Again feel free to edit to add the precision so the credits will be yours (at least in history edit). – Tensibai Aug 22 '16 at 11:05
-
3Done. This answer is a lot better than the most voted one here. It does deserve more upvotes. That and other answers here were discussed at Why is using a shell loop to process text considered bad practice? which is what brought me here. – Stephane Chazelas Aug 22 '16 at 11:42
The top command also contains a field to display the number of page faults for a process. The process with maximum page faults would be the process which is swapping most. For long running daemons it might be that they incur large number of page faults at the beginning and the number does not increase later on. So we need to observe whether the page faults is increasing.
Yet two more variants:
Because top
or htop
could be not installed on small systems, browsing /proc
stay always possible.
Even on small systems, you will found a shell
...
A shell variant! (Not bash only)
This is exactly same than lolotux script, but without any fork to grep
, awk
or ps
. This is a lot quicker!
And as bash is one of the poorest shell regarding performance, a little work was done to ensure this script will run well under dash, busybox and some other. Then, (thanks to Stéphane Chazelas,) become a lot quicker again!
#!/bin/sh
# Get current swap usage for all running processes
# Felix Hauri 2016-08-05
# Rewritted without fork. Inspired by first stuff from
# Erik Ljungstrom 27/05/2011
# Modified by Mikko Rantalainen 2012-08-09
# Pipe the output to "sort -nk3" to get sorted output
# Modified by Marc Methot 2014-09-18
# removed the need for sudo
OVERALL=0
rifs=`printf ': \t'`
for FILE in /proc/[0-9]*/status ;do
SUM=0
while IFS="$rifs" read FIELD VALUE ;do
case $FIELD in
Pid ) PID=$VALUE ;;
Name ) PROGNAME="$VALUE" ;;
VmSwap ) SUM=$((SUM=${VALUE% *})) ;;
esac
done <$FILE
[ $SUM -gt 0 ] &&
printf "PID: %9d swapped: %11d KB (%s)\n" $PID $SUM "$PROGNAME"
OVERALL=$((OVERALL+SUM))
done
printf "Total swapped memory: %14u KB\n" $OVERALL
Don't forgot to double quote "$PROGNAME"
! See Stéphane Chazelas's comment:
read FIELD PROGNAME < <(
perl -ne 'BEGIN{$0="/*/*/../../*/*"} print if /^Name/' /proc/self/status
)
echo $FIELD "$PROGNAME"
Don't try echo $PROGNAME
without double quote on sensible system, and be ready to kill current shell before!
And a perl version
As this become a not so simple script, time is comming to write a dedicated tool by using more efficient language.
#!/usr/bin/perl -w
use strict;
use Getopt::Std;
my ($tot,$mtot)=(0,0);
my %procs;
my %opts;
getopt('', \%opts);
sub sortres {
return $a <=> $b if $opts{'p'};
return $procs{$a}->{'cmd'} cmp $procs{$b}->{'cmd'} if $opts{'c'};
return $procs{$a}->{'mswap'} <=> $procs{$b}->{'mswap'} if $opts{'m'};
return $procs{$a}->{'swap'} <=> $procs{$b}->{'swap'};
};
opendir my $dh,"/proc";
for my $pid (grep {/^\d+$/} readdir $dh) {
if (open my $fh,"</proc/$pid/status") {
my ($sum,$nam)=(0,"");
while (<$fh>) {
$sum+=$1 if /^VmSwap:\s+(\d+)\s/;
$nam=$1 if /^Name:\s+(\S+)/;
}
if ($sum) {
$tot+=$sum;
$procs{$pid}->{'swap'}=$sum;
$procs{$pid}->{'cmd'}=$nam;
close $fh;
if (open my $fh,"</proc/$pid/smaps") {
$sum=0;
while (<$fh>) {
$sum+=$1 if /^Swap:\s+(\d+)\s/;
};
};
$mtot+=$sum;
$procs{$pid}->{'mswap'}=$sum;
} else { close $fh; };
};
};
map {
printf "PID: %9d swapped: %11d (%11d) KB (%s)\n",
$_, $procs{$_}->{'swap'}, $procs{$_}->{'mswap'}, $procs{$_}->{'cmd'};
} sort sortres keys %procs;
printf "Total swapped memory: %14u (%11u) KB\n", $tot,$mtot;
could by run with one of
-c sort by command name
-p sort by pid
-m sort by swap values
by default, output is sorted by status's vmsize
-
It assumes process names don't contain space, tab,
:
, backslash, wildcard or control characters. – Stephane Chazelas Aug 5 '16 at 15:01 -
@StephaneChazelas Thanks! I've added
[1-9]
before*
for counting only numbered paths (noself
, northread-self
) – F. Hauri Aug 5 '16 at 16:00 -
1The syntax is known, but the process names are not. At least quote your variables. (in any case, your script is a lot less bad than loloxux'). – Stephane Chazelas Aug 8 '16 at 9:04
-
1Process names on Linux can contain any byte value but 0 but are limited to 15 bytes in length. The
Name
entry in/proc/*/status
encodes some of those byte values. Try for instanceperl -ne 'BEGIN{$0="\n\t\\"} print if /^Name/' /proc/self/status
. Because it's so short, the damage that can be done with things likeperl -ne 'BEGIN{$0="/*/*/../../*/*"} print if /^Name/' /proc/self/status
is limited when you forget to quote your variables. – Stephane Chazelas Aug 8 '16 at 15:18 -
1This (at least the perl version which I just tried) is hugely faster than the other answers. – David Gardner Apr 8 '19 at 9:57
I adapted a different script on the web to this long one-liner:
{ date;for f in /proc/[0-9]*/status; do
awk '{k[$1]=$2} END { if (k["VmSwap:"]) print k["Pid:"],k["Name:"],k["VmSwap:"];}' $f 2>/dev/null;
done | sort -n ; }
Which I then throw into a cronjob and redirect output to a logfile. The information here is the same as accumulating the Swap:
entries in the smaps file, but if you want to be sure, you can use:
{ date;for m in /proc/*/smaps;do
awk '/^Swap/ {s+=$2} END { if (s) print FILENAME,s }' $m 2>/dev/null;
done | tr -dc ' [0-9]\n' |sort -k 1n; }
The output of this version is in two columns: pid, swap amount. In the above version, the tr
strips the non-numeric components. In both cases, the output is sorted numerically by pid.
-
2This is good, but the first one sorts by pid ascending (sort -n). The better usage is to have it sorted by swap usage in descending order (the most using in front of the list). To get it change "sort -n" to "sort -n -k 3 -r" – Stan Brajewski Feb 7 '17 at 13:53
On MacOSX, you run top command as well but need to type "o" then "vsize" then ENTER.
Gives totals and percentages for process using swap
smem -t -p
Source : https://www.cyberciti.biz/faq/linux-which-process-is-using-swap/
-
Oh, I didn't know
smem
can do this, thanks! FTR, I modified a command I usually use to look like thissmem -kc "name user pid vss pss rss swap"
, so now I can see the amount of swap used in the last column. – Hi-Angel Mar 3 at 22:08
Since the year 2015 kernel patch that adds SwapPss
(https://lore.kernel.org/patchwork/patch/570506/) one can finally get proportional swap count meaning that if a process has swapped a lot and then it forks, both forked processes will be reported to swap 50% each. And if either then forks, each process is counted 33% of the swapped pages so if you count all those swap usages together, you get real swap usage instead of value multiplied by process count.
In short:
(cd /proc; for pid in [0-9]*; do printf "%5s %6s %s\n" "$pid" "$(awk 'BEGIN{sum=0} /SwapPss:/{sum+=$2} END{print sum}' $pid/smaps)" "$(cat $pid/comm)"; done | sort -k2n,2 -k1n,1)
First column is pid, second column is swap usage in KiB and rest of the line is command being executed. Identical swap counts are sorted by pid.
Above may emit lines such as
awk: cmd. line:1: fatal: cannot open file `15407/smaps' for reading (No such file or directory)
which simply means that process with pid 15407 ended between seeing it in the list for /proc/
and reading the process smaps
file. If that matters to you, simply add 2>/dev/null
to the end. Note that you'll potentially lose any other possible diagnostics as well.
In real world example case, this changes other tools reporting ~40 MB swap usage for each apache child running on one server to actual usage of between 7-3630 KB really used per child.
I suppose you could get a good guess by running top
and looking for active processes using a lot of memory. Doing this programatically is harder---just look at the endless debates about the Linux OOM killer heuristics.
Swapping is a function of having more memory in active use than is installed, so it is usually hard to blame it on a single process. If it is an ongoing problem, the best solution is to install more memory, or make other systemic changes.
Here's a version that outputs the same as the script by @loolotux, but is much faster(while less readable). That loop takes about 10 secs on my machine, my version takes 0.019 s, which mattered to me because I wanted to make it into a cgi page.
join -t / -1 3 -2 3 \
<(grep VmSwap /proc/*/status |egrep -v '/proc/self|thread-self' | sort -k3,3 --field-separator=/ ) \
<(grep -H '' --binary-files=text /proc/*/cmdline |tr '\0' ' '|cut -c 1-200|egrep -v '/proc/self|/thread-self'|sort -k3,3 --field-separator=/ ) \
| cut -d/ -f1,4,7- \
| sed 's/status//; s/cmdline//' \
| sort -h -k3,3 --field-separator=:\
| tee >(awk -F: '{s+=$3} END {printf "\nTotal Swap Usage = %.0f kB\n",s}') /dev/null
iotop
is a very useful tool. It gives live stats of I/O and swap usage per process/thread. By default it shows per thread but you can do iotop -P
to get per process info. This is not available by default. You may have to install via rpm/apt.
You can use Procpath (author here), to simplify parsing of VmSwap
from /proc/$PID/status
.
$ procpath record -f stat,cmdline,status -r 1 -d db.sqlite
$ sqlite3 -column db.sqlite \
'SELECT status_name, status_vmswap FROM record ORDER BY status_vmswap DESC LIMIT 5'
Web Content 192136
okular 186872
thunderbird 183692
Web Content 143404
MainThread 86300
You can also plot VmSwap
of processes of interest over time like this. Here I'm recording my Firefox process tree while opening a couple tens of tabs along with statrting a memory-hungry application to try to cause it to swap (which wasn't convincing for Firefox, but your kilometrage may vary).
$ procpath record -f stat,cmdline,status -i 1 -d db2.sqlite \
'$..children[?(@.stat.pid == 6029)]'
# interrupt by Ctrl+C
$ procpath plot -d db2.sqlite -q cpu --custom-value-expr status_vmswap \
--title "CPU usage, % vs Swap, kB"
The same answer as @lolotux, but with sorted output:
printf 'Computing swap usage...\n';
swap_usages="$(
SUM=0
OVERALL=0
for DIR in `find /proc/ -maxdepth 1 -type d -regex "^/proc/[0-9]+"`
do
PID="$(printf '%s' "$DIR" | cut -d / -f 3)"
PROGNAME=`ps -p $PID -o comm --no-headers`
for SWAP in `grep VmSwap $DIR/status 2>/dev/null | awk '{ print $2 }'`
do
let SUM=$SUM+$SWAP
done
if (( $SUM > 0 )); then
printf "$SUM KB ($PROGNAME) swapped PID=$PID\\n"
fi
let OVERALL=$OVERALL+$SUM
SUM=0
break
done
printf '9999999999 Overall swap used: %s KB\n' "$OVERALL"
)"
printf '%s' "$swap_usages" | sort -nk1
Example output:
Computing swap usage...
2064 KB (systemd) swapped PID=1
59620 KB (xfdesktop) swapped PID=21405
64484 KB (nemo) swapped PID=763627
66740 KB (teamviewerd) swapped PID=1618
68244 KB (flameshot) swapped PID=84209
763136 KB (plugin_host) swapped PID=1881345
1412480 KB (java) swapped PID=43402
3864548 KB (sublime_text) swapped PID=1881327
9999999999 Overall swap used: 2064 KB