240

Given a list of files in files.txt, I can get a list of their sizes like this:

cat files.txt | xargs ls -l | cut -c 23-30

which produces something like this:

  151552
  319488
 1536000
  225280

How can I get the total of all those numbers?

0

24 Answers 24

449
... | paste -sd+ - | bc

is the shortest one I've found (from the UNIX Command Line blog).

Edit: added the - argument for portability, thanks @Dogbert and @Owen.

10
  • Nice. Need the last - on Solaris too
    – Owen B
    Commented May 2, 2014 at 9:38
  • 9
    alias sum="paste -sd+ - | bc" added to shell completion, thanks mate
    – slf
    Commented Sep 30, 2014 at 20:59
  • 18
    @slf, watch out, you just overloaded /usr/bin/sum
    – qneill
    Commented Apr 3, 2015 at 23:35
  • 3
    Beware, bc is not available on some systems! awk, on the other hand is (I believe) required for POSIX compliance.
    – squirl
    Commented Jan 1, 2017 at 20:23
  • 2
    @donbright, make sure every line of the input file contains exactly one number, and nothing else. You can debug it by leaving off the | bc, and visually inspecting the output to spot the syntax error (it should be in the format "a + b + c + ...").
    – Todd Owen
    Commented Jun 28, 2020 at 7:19
172

Here goes

cat files.txt | xargs ls -l | cut -c 23-30 | 
  awk '{total += $1}END{print total}'
4
  • 40
    Using awk is a fine idea, but why keep the cut? That's a predictable column number, so use ... | xargs ls -l | awk '{total = total + $5}{END{print total}' Commented May 29, 2009 at 14:36
  • 3
    You are correct of course - it was easier just to append on to the end of what was already there :-) Commented Jun 1, 2009 at 10:05
  • 3
    One bracket too much in @dmckee's answer :) Commented Apr 10, 2013 at 11:04
  • 11
    To make this a bit shorter, you could use total+=$1 instead of total = total + $1
    – squirl
    Commented Jan 1, 2017 at 20:22
11

Instead of using cut to get the file size from output of ls -l, you can use directly:

$ cat files.txt | xargs ls -l | awk '{total += $5} END {print "Total:", total, "bytes"}'

Awk interprets "$5" as the fifth column. This is the column from ls -l that gives you the file size.

11

cat will not work if there are spaces in filenames. here is a perl one-liner instead.

perl -nle 'chomp; $x+=(stat($_))[7]; END{print $x}' files.txt
1
  • With the -l option to perl, the chomp; is unnecessary. The -l takes care of that. Commented Sep 5, 2023 at 14:48
10

The whole ls -l and then cut is rather convoluted when you have stat. It is also vulnerable to the exact format of ls -l (it didn't work until I changed the column numbers for cut)

Also, fixed the useless use of cat.

<files.txt xargs stat -c %s | paste -sd+ - | bc
1
  • 3
    Huh. Been using Unix for 32 years, and never knew that <infile command is the same as (and in a better order than) command <infile. Commented Dec 5, 2016 at 21:50
9
python3 -c"import os; print(sum(os.path.getsize(f) for f in open('files.txt').read().split()))"

Or if you just want to sum the numbers, pipe into:

python3 -c"import sys; print(sum(int(x) for x in sys.stdin))"
2
  • 1
    ... | python -c'import sys; print(sum(int(x) for x in sys.stdin))' when python 2 disappears at the end of this year.
    – Eponymous
    Commented May 21, 2019 at 18:22
  • don@oysters:~/Documents$ cat tax | python3 -c"import sys; print(sum(int(x) for x in sys.stdin))" Traceback (most recent call last): File "<string>", line 1, in <module> File "<string>", line 1, in <genexpr> ValueError: invalid literal for int() with base 10: '\n'
    – don bright
    Commented Jun 28, 2020 at 1:10
8

if you don't have bc installed, try

echo $(( $(... | paste -sd+ -) ))

instead of

... | paste -sd+ - | bc

$( ) <-- return the value of executing the command

$(( 1+2 )) <-- return the evaluated results

echo <-- echo it to the screen

6
cat files.txt | awk '{ total += $1} END {print total}'

You can use the awk to do the same it even skips the non integers

$ cat files.txt
1
2.3
3.4
ew
1

$ cat files.txt | awk '{ total += $1} END {print total}'
7.7

or you can use ls command and calculate human readable output

$ ls -l | awk '{ sum += $5} END  {hum[1024^3]="Gb"; hum[1024^2]="Mb"; hum[1024]="Kb"; for (x=1024^3; x>=1024; x/=1024) { if (sum>=x) { printf "%.2f %s\n",sum/x,hum[x]; break; } } if (sum<1024) print "1kb"; }'
15.69 Mb

$ ls -l *.txt | awk '{ sum += $5} END  {hum[1024^3]="Gb"; hum[1024^2]="Mb"; hum[1024]="Kb"; for (x=1024^3; x>=1024; x/=1024) { if (sum>=x) { printf "%.2f %s\n",sum/x,hum[x]; break; } } if (sum<1024) print "1kb"; }'
2.10 Mb
1
  • 2
    You don't even need pipe: awk '{ total += $1} END {print total}' files.txt is quicker
    – bmv
    Commented May 19, 2020 at 5:50
5

You can use the following script if you just want to use shell scripting without awk or other interpreters:

#!/bin/bash

total=0

for number in `cat files.txt | xargs ls -l | cut -c 23-30`; do
   let total=$total+$number
done

echo $total
5

TMTWWTDI: Perl has a file size operator (-s)

perl -lne '$t+=-s;END{print $t}' files.txt
1
  • 9 years later... some perl golf: perl -lne '$t+=-s}{print $t' files.txt Commented Jun 9, 2022 at 13:58
3

I would use "du" instead.

$ cat files.txt | xargs du -c | tail -1
4480    total

If you just want the number:

cat files.txt | xargs du -c | tail -1 | awk '{print $1}'
5
  • 5
    Disk usage != size of file. du reports disk usage.
    – 0x6adb015
    Commented May 29, 2009 at 14:07
  • 4
    I think the -b switch makes du do what I need. Commented May 29, 2009 at 14:16
  • @0x6adb015 Good knowledge. Thanks I hadn't realised. Commented May 29, 2009 at 16:34
  • 3
    That's a useful answer for the specific reason why the OP wanted the column of numbers added, but for the general number-adding case, it falls short. (I use "du" all the time myself, but I came here looking for command-line math. :-) )
    – Michael H.
    Commented Jun 29, 2011 at 19:55
  • 12
    This will not work when files.txt is large. If the number of arguments piped to xargs reaches certain threshold, it breaks them up over multiple calls to du. The total shown at the end is the total for just the last call to du, not the whole list. Commented Sep 7, 2011 at 21:31
3

In ksh:

echo " 0 $(ls -l $(<files.txt) | awk '{print $5}' | tr '\n' '+') 0" | bc
1
  • 1
    Good for picking up on skipping the cut, but you ignore awks ability to do the math... Commented May 29, 2009 at 14:38
3
... |xargs|tr \  +|bc
... |paste -sd+ -|bc

The first command is just one symbol longer (note, it must have two spaces after the backslash!), but it handles the cases with empty lines in a column, whereas the second command results in an invalid expression with extra pluses.

E.g.:

echo "2
3
5

" | paste -sd+ -

results in

2+3+5++

which bc cannot handle, whereas

echo "2
3
5

" | xargs | tr \  +

gives a valid expression

 2+3+5 

which can be piped into bc to get the final result

2

Here's mine

cat files.txt | xargs ls -l | cut -c 23-30 | sed -e :a -e '$!N;s/\n/+/;ta' | bc
1
  • 6
    +1 for proving once and for all that there are uglier languages than perl :)
    – bdonlan
    Commented Aug 12, 2009 at 16:17
2

I find myself reaching for jq for things like this, even when not working with JSON:

$ echo -e '1\n2\n3' | jq --slurp add
6

it's become a standard dependency for almost every script I've written recently.

1

Pipe to gawk:

 cat files.txt | xargs ls -l | cut -c 23-30 | gawk 'BEGIN { sum = 0 } // { sum = sum + $0 } END { print sum }'
1
#
#       @(#) addup.sh 1.0 90/07/19
#
#       Copyright (C) <heh> SjB, 1990
#       Adds up a column (default=last) of numbers in a file.
#       95/05/16 updated to allow (999) negative style numbers.


case $1 in

-[0-9])

        COLUMN=`echo $1 | tr -d -`

        shift

;;

*)

        COLUMN="NF"

;;

esac

echo "Adding up column .. $COLUMN .. of file(s) .. $*"

nawk  ' OFMT="%.2f"                                       # 1 "%12.2f"

        { x = '$COLUMN'                                   # 2

          neg = index($x, "$")                            # 3

          if (neg > 0) X = gsub("\\$", "", $x)

          neg = index($x, ",")                            # 4

          if (neg > 1) X = gsub(",", "", $x)

          neg = index($x, "(")                            # 8 neg (123 & change

          if (neg > 0) X = gsub("\\(", "", $x)

          if (neg > 0) $x = (-1 * $x)                     # it to "-123.00"

          neg = index($x, "-")                            # 5

          if (neg > 1) $x = (-1 * $x)                     # 6

          t += $x                                         # 7

          print "x is <<<", $x+0, ">>> running balance:", t

        } ' $*


# 1.  set numeric format to eliminate rounding errors
# 1.1 had to reset numeric format from 12.2f to .2f 95/05/16
#     when a computed number is assigned to a variable ( $x = (-1 * $x) )
#     it causes $x to use the OFMT so -1.23 = "________-1.23" vs "-1.23"
#     and that causes my #5 (negative check) to not work correctly because
#     the index returns a number >1 and to the neg neg than becomes a positive
#     this only occurs if the number happened to b a "(" neg number
# 2.  find the field we want to add up (comes from the shell or defaults
#     to the last field "NF") in the file
# 3.  check for a dollar sign ($) in the number - if there get rid of it
#     so we may add it correctly - $12 $1$2 $1$2$ $$1$$2$$ all = 12
# 4.  check for a comma (,) in the number - if there get rid of it so we
#     may add it correctly - 1,2 12, 1,,2 1,,2,, all = 12   (,12=0)
# 5.  check for negative numbers
# 6.  if x is a negative number in the form 999- "make" it a recognized
#     number like -999 - if x is a negative number like -999 already
#     the test fails (y is not >1) and this "true" negative is not made
#     positive
# 7.  accumulate the total
# 8.  if x is a negative number in the form (999) "make it a recognized
#     number like -999
# * Note that a (-9) (neg neg number) returns a postive
# * Mite not work rite with all forms of all numbers using $-,+. etc. *
1

I like to use....

echo "
1
2
3 " | sed -e 's,$, + p,g' | dc 

they will show the sum of each line...

applying over this situation:

ls -ld $(< file.txt) | awk '{print $5}' | sed -e 's,$, + p,g' | dc 

Total is the last value...

1

Pure bash

total=0; for i in $(cat files.txt | xargs ls -l | cut -c 23-30); do 
total=$(( $total + $i )); done; echo $total
0

In my opinion, the simplest solution to this is "expr" unix command:

s=0; 
for i in `cat files.txt | xargs ls -l | cut -c 23-30`
do
   s=`expr $s + $i`
done
echo $s
0
sizes=( $(cat files.txt | xargs ls -l | cut -c 23-30) )
total=$(( $(IFS="+"; echo "${sizes[*]}") ))

Or you could just sum them as you read the sizes

declare -i total=0
while read x; total+=x; done < <( cat files.txt | xargs ls -l | cut -c 23-30 )

If you don't care about bite sizes and blocks is OK, then just

declare -i total=0
while read s junk; total+=s; done < <( cat files.txt | xargs ls -s )
0

If you have R, you can use:

> ... | Rscript -e 'print(sum(scan("stdin")));'
Read 4 items
[1] 2232320

Since I'm comfortable with R, I actually have several aliases for things like this so I can use them in bash without having to remember this syntax. For instance:

alias Rsum=$'Rscript -e \'print(sum(scan("stdin")));\''

which let's me do

> ... | Rsum
Read 4 items
[1] 2232320

Inspiration: Is there a way to get the min, max, median, and average of a list of numbers in a single command?

0

The most popular answer doesn't work right when the start of the pipe can produce 0 lines, because it ends up outputting nothing rather than 0. You can get correct behavior by always adding 0:

... | (cat && echo 0) | paste -sd+ - | bc

0

The - is not required for paste. The following will do as long as files.txt contains one or more valid file names:

<files.txt xargs stat -c %s | paste -sd+ | bc

cat is not required to insert 0 in case there is no file. Without a pipe, perhaps more convenient in a script, you could use:

(xargs -a files.txt stat -c %s || echo 0) | paste -sd+ | bc

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.