40

I found this question and answer on how to remove triple empty lines. However, I need the same only for double empty lines. Ie. all double blank lines should be deleted completely, but single blank lines should be kept.

I know a bit of sed, but the proposed command for removing triple blank lines is over my head:

sed '1N;N;/^\n\n$/d;P;D'

2
  • 2
    what about triple and more blank lines?
    – xdazz
    Sep 26, 2012 at 9:47
  • @xdazz: yes it would be ok for triple or more blank lines.
    – marlar
    Sep 26, 2012 at 10:46

8 Answers 8

76

This would be easier with cat:

cat -s
1
  • Nice one, but it doesn't do exactly the same. However, I didn't know this option and it is very useful, so thanks anyway.
    – marlar
    Apr 23, 2013 at 9:04
20

I've commented the sed command you don't understand:

sed '
    ## In first line: append second line with a newline character between them.
    1N;
    ## Do the same with third line.
    N;
    ## When found three consecutive blank lines, delete them. 
    ## Here there are two newlines but you have to count one more deleted with last "D" command.
    /^\n\n$/d;
    ## The combo "P+D+N" simulates a FIFO, "P+D" prints and deletes from one side while "N" appends
    ## a line from the other side.
    P;
    D
'

Remove 1N because we need only two lines in the 'stack' and it's enought with the second N, and change /^\n\n$/d; to /^\n$/d; to delete all two consecutive blank lines.

A test:

Content of infile:

1


2
3

4



5

6


7

Run the sed command:

sed '
    N;
    /^\n$/d;
    P;
    D
' infile

That yields:

1
2
3

4

5

6
7
3
  • For all wanting to use this, don't forget the sed -i option to change that immediately in the file instead of sending the result to the out stream.
    – jan
    Apr 29, 2014 at 6:43
  • 1
    on macOS, this deletes the last line of the file (even when that line is non-blank) May 15, 2018 at 0:10
  • Good Explanation, but the regex part can be more clear. "Here there are two newlines but you have to count one more deleted", the /^\n\n$/d; has already delete 3 blank lines.(note 3 blank lines has two \n) The last 'D' operates on 'pattern space'. D-command removes pattern space and left only one line(without \n). -- Then sed start next cyclic, jump to the beginning and what the N does, is appending \n to pattern space then loading next line.
    – DoorWay
    Sep 1, 2021 at 7:21
9
sed '/^$/{N;/^\n$/d;}'

It will delete only two consecutive blank lines in a file. You can use this expression only in file then only you can fully understand. When a blank line will come that it will enter into braces.

Normally sed will read one line. N will append the second line to pattern space. If that line is empty line. the both lines are separated by newline.

/^\n$/ this pattern will match that time only the d will work. Else d not work. d is used to delete the pattern space whole content then start the next cycle.

2
  • I've tried this command and it removes excess newlines (some of the multiple newlines are removed altogether rather than replaced by a single newline). I'm not really sure why this happens, so beware.
    – waldyrious
    Jan 9, 2017 at 19:53
  • 5
    sed 'N;/^\n$/D;P;D;' seems to work better and deletes only consecutive newlines
    – Goblinhack
    Nov 6, 2017 at 15:22
8

This would be easier with awk:

awk -v RS='\n\n\n' 1
1

BUT the above solution only deletes first search of 3 consecutive blank line. To delete all, 3 consecutive blank lines use below command

sed '1N;N;/^\n\n$/ { N;s/^\n\n//;N;D; };P;D' filename
0

As far as I can tell none of the solutions here work. cat -s as suggested by @DerMike isn't POSIX compliant (and it's less convenient if you're already using sed for another transformation), and sed 'N;/^\n$/d;P;D' as suggested by @Birei sometimes deletes more newlines than it should.

Instead, sed ':L;N;s/^\n$//;t L' works. For POSIX compliance use sed -e :L -e N -e 's/^\n$//' -e 't L', since POSIX doesn't specify using ; to separate commands.

Example:

$ S='foo\nbar\n\nbaz\n\n\nqux\n\n\n\nquxx\n';\
> paste <(printf "$S")\
>       <(printf "$S" | sed -e 'N;/^\n$/d;P;D')\
>       <(printf "$S" | sed -e ':L;N;s/^\n$//;t L')
foo     foo     foo
bar     bar     bar
                
baz     baz     baz
        qux     
                qux
qux     quxx    
                quxx
                
                
quxx            
$ 

Here we can see the original file, @Birei's solution, and my solution side-by-side. @Birei's solution deletes all blank lines separating baz and qux, while my solution removes all but one as intended.

Explanation:

:L        Create a new label called L.

N         Read the next line into the current pattern space,
          separated by an "embedded newline."

s/^\n$//  Replace the pattern space with the empty pattern space,
          corresponding to a single non-embedded newline in the output,
          if the current pattern space only contains a single embedded newline,
          indicating that a blank line was read into the pattern space by `N`
          after a blank line had already been read from the input.

t L       Branch to label L if the previous `s` command successfully
          substituted text in the pattern space.

In effect, this deletes one recurrent blank line at a time, reading each into the pattern space as an embedded newline with N and deleting them with s.

-1

BUT the above solution only deletes first search of 3 consecutive blank line. To delete all, 3 consecutive blank lines use below command

sed '1N;N;/^\n\n$/ { N;s/^\n\n//;N;D; };P;D' filename

1
  • 1
    I understand that you have created 2 accounts. Now I suggest you keep the other one and have that one deleted. hint: you can change account name after creation Feb 28, 2019 at 14:14
-2

Just pipe it to 'uniq' command and all empty lines regardless the number of them will be shrank to just one. Simpler is better.

Clarification: As Marlar stated this is not a solution if you have "other non-blank consecutive duplicated lines" that you do not want to get rid of. This is a solution in other cases like when trying to cleanup configuration files which was the solution I was after when I saw this question. I solved my problem indeed just using 'uniq'.

1
  • 5
    Sometimes it can be too simple, though! If the source contains duplicate entries they would be removed too. That is not an acceptable solution.
    – marlar
    Oct 7, 2013 at 19:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.