34

I am iterating through a file line by line and put each word into a array and that works. But it also picks up blank lines and puts it as an item in the array, how can I skip the blank lines?

example file

      Line 1
line 2

line 3
        line 4 

line 5
   line 6

My code

while read line ; do
            myarray[$index]="$line"
            index=$(($index+1))
    done < $inputfile

Possible psuedo code

while read line ; do
           if (line != space);then
            myarray[$index]="$line"
             fi
            index=$(($index+1))
    done < $inputfile

7 Answers 7

40

Be more elegant:

echo "\na\nb\n\nc" | grep -v "^$"

cat $file | grep -v "^$" | next transformations...
7
  • 17
    Why the cat to grep? Wouldn't grep -v "^$" "$inputfile" do just as well?
    – go2null
    Nov 16, 2015 at 18:00
  • 3
    Since the output is meant to be piped to another operation, it's nicer to pipe in the input file than to supply it as an argument. That way, you can read the data flow straight from left to right, rather than start-in-the-middle-go-left-then-jump-right. Admittedly, it wastes 4 characters and a few milliseconds, but I don't think that's a big deal. Also, presenting it this way makes it easy to just swap out the first stage of the pipeline if the input is coming from somewhere else.
    – pyrocrasty
    May 1, 2017 at 9:30
  • 1
    $file should really be double-quoted here. There's no guarantee the filename won't contain spaces.
    – pyrocrasty
    May 1, 2017 at 9:41
  • It's not clear to me how the OP would actually use this answer. Substitute the last line for $inputfile?
    – Noumenon
    May 4, 2017 at 11:43
  • 1
    the echo "\na\nb\n\nc" is probably confusing in this example, as echo doesn't handle control chars by default (but it depends on which echo you're using: the shell-built-in, or one found in the $PATH); since we're talking bash, a better example would be: printf '\na\nb\n\nc' | grep -v '^$' (or equivalently, ... | grep '.'
    – michael
    Mar 6, 2018 at 22:05
29

Implement the same test as in your pseudo-code:

while read line; do
    if [ -n "$line" ]; then
        myarray[$index]="$line"
        index=$(($index+1))
    fi
done < $inputfile

The -n test means true if not empty.

You can also use expressions like [ "x$line" = x ] or test "x$line" = x to test if the line is empty.

However, any line which contains whitespace will not be considered empty. If that is a problem, you can use sed to remove such lines from the input (including empty lines), before they are passed to the while loop, as in:

sed '/^[ \t]*$/d' $inputfile | while read line; do
    myarray[$index]="$line"
    index=$(($index+1))
done
9
  • 3
    It could be simply: if [ "$line" ]; then
    – SzG
    Feb 27, 2014 at 21:48
  • 1
    yes, that works on Linux/bash... not sure about other / older Unix and shells. The autoconf tools specifically use the test "x$line" = x form for max. portability.
    – isedev
    Feb 27, 2014 at 21:52
  • Could I add something to sed? If there is a blank space before the first letter in a line to remove that space or spaces up to the first letter in the line?
    – MAXGEN
    Feb 27, 2014 at 22:01
  • well, you could use: sed -e '/^[ \t]*$/d' -e 's/^[ \t]*//' to delete blank or all whitespace lines and to remove all leading whitespace before non-whitespace in a line.
    – isedev
    Feb 27, 2014 at 22:03
  • Copy your code up top, is that correct? I'm not getting good results.
    – MAXGEN
    Feb 27, 2014 at 22:11
12

Remove the blank lines first with sed.

for word in `sed '/^$/d' $inputfile`; do
    myarray[$index]="$word"
    index=$(($index+1))
done
5

cat -b -s file |grep -v '^$'

I know it's solved but, I needed to output numbered lines while ignoring empty lines, so I thought of putting it right here in case someone needs it. :)

1
  • as an alternative to cat (which reformats the output more than one typically wants), grep can also add line numbers. If matching every line, no numbers are skipped: e.g., grep -v '^$' file | grep -n . (you can convert the colon (:) to something else via sed)
    – michael
    Mar 6, 2018 at 22:12
3

Use grep to remove the blank lines:

for word in $(cat ${inputfile} | grep -v "^$"); do
   myarray[$index]="${word}"
   index=$(($index+1))
done
2
  • 2
    Why the cat to grep? Wouldn't grep -v "^$" "$inputfile" do just as well?
    – go2null
    Nov 16, 2015 at 17:59
  • Both are correct. I just used to use 'cat' because sometimes I need to use 'echo' instead 'cat'.
    – NFTX
    Nov 17, 2015 at 19:54
2

This version is very fast compared to solutions that invoke external commands like sed and grep. Also it skips lines that contain only spaces, the lines don't need to be empty to be skipped.

#!/bin/bash

myarray=()
while read line
do
    if [[ "$line" =~ [^[:space:]] ]]; then
        myarray+=("${line}")
    fi
done < test.txt

for((i = 0; i < ${#myarray[@]}; ++i))
do
    echo ${myarray[$i]}
done
2

Here is the way I've been doing it. Doesn't require calling grep, and you don't need to have two levels of indentation either.

while read line; do
 # skip  empty lines
 [ -z "$line" ] && continue
 echo "processing $line"
done < "$inputfile"

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.