I want a bash command that I can pipe into that will sum a column of numbers. I just want a quick one liner that will do something essentially like this:
cat FileWithColumnOfNumbers.txt | sum
I want a bash command that I can pipe into that will sum a column of numbers. I just want a quick one liner that will do something essentially like this:
cat FileWithColumnOfNumbers.txt | sum
paste -sd+ infile | bc
<cmd> | paste -sd+ | bc
Edit: With some paste implementations you need to be more explicit when reading from stdin:
<cmd> | paste -sd+ - | bc
-s
option is in GNU paste
; it is not supported by Mac OS X 10.7.4 paste
. However, since the POSIX 2008 specification of paste
supports -s
, this is a deficiency in the Mac OS X version.
– Jonathan Leffler
Aug 15 '12 at 18:56
bc -l
you can even add up float numbers. With cut
you can select columns from the input: cat input.txt | cut -d ' ' -f 8 | paste -sd+ | bc -l
will add all float numbers in column 8 of the input (space being the field separator).
– Arne
Sep 24 '13 at 14:59
awk 'END { print s } { s += $8 }' infile
:)
– Dimitre Radoulov
Sep 24 '13 at 15:31
I like the chosen answer. However, it tends to be slower than awk since 2 tools are needed to do the job.
$ wc -l file
49999998 file
$ time paste -sd+ file | bc
1448700364
real 1m36.960s
user 1m24.515s
sys 0m1.772s
$ time awk '{s+=$1}END{print s}' file
1448700364
real 0m45.476s
user 0m40.756s
sys 0m0.287s
s
to 0, by doing awk 'BEGIN{s=0}{s+=$1}END{print s}'
. In this case, if an empty column is piped into paste | bc
, the latter pipe will return 0
instead of null. (There may be cases where and empty column is a legitimate input).
– gbgnv
Mar 31 '15 at 22:12
Does two lines count?
awk '{ sum += $1; }
END { print sum; }' "$@"
You can then use it without the superfluous 'cat':
sum < FileWithColumnOfNumbers.txt
sum FileWithColumnOfNumbers.txt
FWIW: on MacOS X, you can do it with a one-liner:
awk '{ sum += $1; } END { print sum; }' "$@"
awk
any more. I don't know whether the one-line would have worked there, but I didn't try it back then when I learned awk
. You're probably right; all current versions of awk
are likely to support the one-liner.
– Jonathan Leffler
May 24 '13 at 6:59
"$@"
is used inside a script to represent all the arguments to the script, or nothing/none if there were no arguments. If you're using the awk
in pbpaste | awk …
, you simply omit the "$@"
(though it would usually do no damage; most interactive shells at a terminal have no 'positional parameters' so "$@"
is nothing). But if you have a shell script sumcol1.sh
, adding the "$@"
would be sensible — you'd then use pbpaste | sumcol1.sh
or similar and it would work correctly, and so would sumcol1.sh file1 file2
.
– Jonathan Leffler
Aug 10 '16 at 14:51
The following command will add all the lines(first field of the awk output)
awk '{s+=$1} END {print s}' filename
[a followup to ghostdog74s comments]
bash-2.03$ uname -sr
SunOS 5.8
bash-2.03$ perl -le 'print for 1..49999998' > infile
bash-2.03$ wc -l infile
49999998 infile
bash-2.03$ time paste -sd+ infile | bc
bundling space exceeded on line 1, teletype
Broken Pipe
real 0m0.062s
user 0m0.010s
sys 0m0.010s
bash-2.03$ time nawk '{s+=$1}END{print s}' infile
1249999925000001
real 2m0.042s
user 1m59.220s
sys 0m0.590s
bash-2.03$ time /usr/xpg4/bin/awk '{s+=$1}END{print s}' infile
1249999925000001
real 2m27.260s
user 2m26.230s
sys 0m0.660s
bash-2.03$ time perl -nle'
$s += $_; END { print $s }
' infile
1.249999925e+15
real 1m34.663s
user 1m33.710s
sys 0m0.650s
perl -nle '$s += (split)[2]; END { print $s }' foo.txt
or using pipes: cat foo.txt | perl -nle '$s += (split)[2]; END { print $s }'
.
– Ben
May 14 '14 at 13:17
You can use bc (calculator). Assuming your file with #s is called "n":
$ cat n
1
2
3
$ (cat n | tr "\012" "+" ; echo "0") | bc
6
The tr
changes all newlines to "+"; then we append 0 after the last plus, then we pipe the expression (1+2+3+0
) to the calculator
Or, if you are OK with using awk or perl, here's a Perl one-liner:
$perl -nle '$sum += $_ } END { print $sum' n
6
perl -nle '$s+=$_}{print $s'
It is a bit simpler. :)
– jm666
May 8 '13 at 20:26
while read -r num; do ((sum += num)); done < inputfile; echo $sum
Use a for
loop to iterate over your file …
sum=0; for x in `cat <your-file>`; do let sum+=x; done; echo $sum
cat
:: for s in $(< infile); do let sum+=$s ; done && echo $sum
– Rahul Patil
Apr 8 '13 at 4:13
If you have ruby installed
cat FileWithColumnOfNumbers.txt | xargs ruby -e "puts ARGV.map(&:to_i).inject(&:+)"
[root@pentest3r ~]# (find / -xdev -size +1024M) | (while read a ; do aa=$(du -sh $a | cut -d "." -f1 ); o=$(( $o+$aa )); done; echo "$o";)
du
on each file separately, which is both painful and largely irrelevant. Another, older answer provides the same 'sum it using just shell' solution. Given the absence of explanation and repetition of another answer and off-topic coding, there really isn't any benefit to keeping this answer.
– Jonathan Leffler
Jul 30 '15 at 19:50
awk
solution is both easier to remember and about 2x faster (see here and here). – Trevor Boyd Smith Apr 10 at 14:03