15

I have a long column of ones and zeroes:

0
0
0
1
0
0
0
0
0
1
0
0
0
0
0
1
0
0
1
....

I can easily get the average number of zeroes between ones (just total/ones):

ones=$(grep -c 1 file.txt)
lines=$(wc -l < file.txt)
echo "$lines / $ones" | bc -l

But how can I get the length of strings of zeroes between the ones? In the short example above it would be:

3
5
5
2
4
  • Are you familiar with awk?
    – lurker
    Commented Feb 9, 2015 at 17:01
  • No, I'm very new to this. But, based on the answers (which I'm currently looking through), it looks like it would be very helpful to learn.
    – userABC123
    Commented Feb 9, 2015 at 18:37
  • 1
    tutorialspoint.com/awk/index.htm :)
    – lurker
    Commented Feb 9, 2015 at 18:38
  • 1
    Thanks lurker, and everyone for all of the answers. Looks like I should learn some awk :)
    – userABC123
    Commented Feb 9, 2015 at 20:00

14 Answers 14

17

I'd include uniq for a more easily read approach:

uniq -c file.txt | awk '/ 0$/ {print $1}'
2
  • 3
    Nice and upvoted, although it will not report any string of no 0s (i.e. two consecutive 1s). Not clear from the problem statement if that is needed.
    – rici
    Commented Feb 9, 2015 at 22:32
  • 2
    This is the best and the most unixy answer here. You can catch zero-gaps by awk-ing ones with count more than 1; $1-1 then tells you the number of zero-length strings of zero.
    – orion
    Commented Feb 10, 2015 at 7:52
10

Edit: fixed for the case where the last line is a 0

Easy in awk:

awk '/1/{print NR-prev-1; prev=NR;}END{if (NR>prev)print NR-prev;}'

Not so difficult in bash, either:

i=0
for x in $(<file.txt); do
  if ((x)); then echo $i; i=0; else ((++i)); fi
done
((i)) && echo $i 
4
  • That math seems excessive. awk '/^0$/{n++;next}{print n;n=0}END{if(n)print n}'
    – Kevin
    Commented Feb 9, 2015 at 22:27
  • 1
    @Kevin: I'm assuming that most lines are not going to match /1/, so it will do a lot less math than yours (or the bash solution). Yours is conceptually simpler, I grant you.
    – rici
    Commented Feb 9, 2015 at 22:30
  • Just wondering, does ((x)) convert a "1" as a string to a boolean, or does it do the check a different way?
    – TMH
    Commented Feb 10, 2015 at 14:02
  • @TomHart: ((...)) evaluates its contents as an arithmetic expression, and succeeds if the result is non-zero.
    – rici
    Commented Feb 10, 2015 at 14:34
7

Using awk, I would use the fact that a field with the value 0 evaluates as False:

awk '!$1{s++; next} {if (s) print s; s=0} END {if (s) print s}' file

This returns:

3
5
5
2

Also, note the END block to print any "remaining" zeroes appearing after the last 1.

Explanation

  • !$1{s++; next} if the field is not True, that is, if the field is 0, increment the counter. Then, skip to the next line.
  • {if (s) print s; s=0} otherwise, print the value of the counter and reset it, but just if it contains some value (to avoid printing 0 if the file starts with a 1).
  • END {if (s) print s} print the remaining value of the counter after processing the file, but just if it wasn't printed before.
2
  • It should be if(s) print s - in both print actions. Otherwise you'll output (an) empty line(s) if the input starts with 1(s). (I also had to realize that :)
    – hek2mgl
    Commented Feb 9, 2015 at 17:20
  • @hek2mgl fair enough, good one! I just added it into the answer, many thanks :)
    – fedorqui
    Commented Feb 9, 2015 at 17:25
5

If your file.txt is just a column of ones and zeros, you can use awk and change the record separator to "1\n". This makes each "record" a sequence of "0\n", and the count of 0's in the record is the length of the record divided by 2. Counts will be correct for leading and trailing ones and zeros.

awk 'BEGIN {RS="1\n"} { print length/2 }' file.txt
5

This seems to be pretty popular question today. Joining the party late, here is another short gnu-awk command to do the job:

awk -F '\n' -v RS='(1\n)+' 'NF{print NF-1}' file
3
5
5
2

How it works:

-F '\n'           # set input field separator as \n (newline)
-v RS='(1\n)+'    # set input record separator as multipled of 1 followed by newline
NF                # execute the block if minimum one field is found
print NF-1        # print num of field -1 to get count of 0
1
  • 2
    My apologies - there was a trailing space on my "1 " line which caused this method to fail while some of the others succeeded.
    – Ben Grimm
    Commented Feb 9, 2015 at 20:52
3

Pure bash:

sum=0
while read n ; do
    if ((n)) ; then
        echo $sum
        sum=0
    else
        ((++sum))
    fi
done < file.txt
((sum)) && echo $sum # Don't forget to output the last number if the file ended in 0.
3

Another way:

perl -lnE 'if(m/1/){say $.-1;$.=0}' < file

"reset" the line counter when 1.

prints

3
5
5
2
3

You can use awk:

awk '$1=="0"{s++} $1=="1"{if(s)print s;s=0} END{if(s)print(s)}'

Explanation:

The special variable $1 contains the value of the first field (column) of a line of text. Unless you specify the field delimiter using the -F command line option it defaults to a widespace - meaning $1 will contain 0 or 1 in your example.

If the value of $1 equals 0 a variable called s will get incremented but if $1 is equal to 1 the current value of s gets printed (if greater than zero) and re-initialized to 0. (Note that awk initializes s with 0 before the first increment operation)

The END block gets executed after the last line of input has been processed. If the file ends with 0(s) the number of 0s between the file's end and the last 1 will get printed. (Without the END block they wouldn't printed)

Output:

3
5
5
2
3
  • 2
    Note s=0 is the default value, so there is no need to set it
    – fedorqui
    Commented Feb 9, 2015 at 17:04
  • Oh! I thought it would have been 1 by default (in awk).. Thank you!
    – hek2mgl
    Commented Feb 9, 2015 at 17:05
  • 1
    Need an END block (what if the file ends with some zeroes?) Commented Feb 9, 2015 at 18:47
3

if you can use perl:

perl -lne 'BEGIN{$counter=0;} if ($_ == 1){ print $counter; $counter=0; next} $counter++' file
3
5
5
2

It actually looks better with awk same logic:

awk '$1{print c; c=0} !$1{c++}' file 
3
5
5
2
2

My attempt. Not so pretty but.. :3

grep -n 1 test.txt | gawk '{y=$1-x; print y-1; x=$1}' FS=":"

Out:

3
5
5
2
1
  • Here's a sillier version of the same idea: bc <<<$(echo '-(1';grep -n 1 file.txt|sed -r s'/(.*):.*/-\1);-(\1+1/';printf -- '-1-%s)' $(wc -l < file.txt);)
    – rici
    Commented Feb 9, 2015 at 17:32
2

A funny one, in pure Bash:

while read -d 1 -a u || ((${#u[@]})); do
    echo "${#u[@]}"
done < file

This tells read to use 1 as a delimiter, i.e., to stop reading as soon as a 1 is encountered; read stores the 0's in the fields of the array u. Then we only need to count the number of fields in u with ${#u[@]}. The || ((${#u[@]})) is here just in case your file doesn't end with a 1.

2

More strange (and not fully correct) way:

perl -0x31 -laE 'say @F+0' <file

prints

3
5
5
2
0

It

  • reads the file with the record separator is set to character 1 the -0x31
  • with autosplit -a (splits the record into array @F)
  • and prints the number of elements in @F e.g. say @F+0 or could use say scalar @F

Unfortunately, after the final 1 (as record separator) it prints an empty record - therefore prints the last 0.

It is incorrect solution, showing it only as alternative curiosity.

2

Expanding erickson's excellent answer, you can say:

$ uniq -c file | awk '!$2 {print $1}'
3
5
5
2

From man uniq we see that the purpose of uniq is to:

Filter adjacent matching lines from INPUT (or standard input), writing to OUTPUT (or standard output).

So uniq groups the numbers. Using the -c option we get a prefix with the number of occurrences:

$ uniq -c file
      3 0
      1 1
      5 0
      1 1
      5 0
      1 1
      2 0
      1 1

Then it is a matter of printing those the counters before the 0. For this we can use awk like: awk '!$2 {print $1}'. That is: print the second field if the field is 0.

2

The simplest solution would be to use sed together with awk, like this:

sed -n '$bp;/0/{:r;N;/0$/{h;br}};/1/{x;bp};:p;/.\+/{s/\n//g;p}' input.txt \
  | awk '{print length}'

Explanation:

The sed command separates the 0s and creates output like this:

000
00000
00000
00

Piped into awk '{print length}' you can get the count of 0 for each interval:

Output:

3
5
5
2

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.