For the average, median & standard deviation you can use awk
. This will generally be faster than R
solutions. For instance the following will print the average :
awk '{a+=$1} END{print a/NR}' myfile
(NR
is an awk
variable for the number of records, $1
means the first (space-separated) argument of the line ($0
would be the whole line, which would also work here but in principle would be less secure, although for the computation it would probably just take the first argument anyway) and END
means that the following commands will be executed after having processed the whole file (one could also have initialized a
to 0
in a BEGIN{a=0}
statement)).
Here is a simple awk
script which provides more detailed statistics (takes a CSV file as input, otherwise change FS
) :
#!/usr/bin/awk -f
BEGIN {
FS=",";
}
{
a += $1;
b[++i] = $1;
}
END {
m = a/NR; # mean
for (i in b)
{
d += (b[i]-m)^2;
e += (b[i]-m)^3;
f += (b[i]-m)^4;
}
va = d/NR; # variance
sd = sqrt(va); # standard deviation
sk = (e/NR)/sd^3; # skewness
ku = (f/NR)/sd^4-3; # standardized kurtosis
print "N,sum,mean,variance,std,SEM,skewness,kurtosis"
print NR "," a "," m "," va "," sd "," sd/sqrt(NR) "," sk "," ku
}
It is straightforward to add min/max to this script, but it is as easy to pipe sort
& head
/tail
:
sort -n myfile | head -n1
sort -n myfile | tail -n1
jp
, a CLI utility for making plots.