-1

I have a bibtex file that is the merging of several other .bib files. In the merging process all but one repeated entries were commented out so that all cases with repeated entries are just like below. Some of them have 20~30 entries commented out, making a 100 references file 30k lines of text long.

@Article{goodnight2005,
  author    = {Goodnight, N. and Wang, R. and Humphreys, G.},
  journal   = {{IEEE Computer Graphics and Applications}},
  title     = {{Computation on programmable graphics hardware}},
  year      = {2005},
  volume    = {25},
  number    = {5},
  pages     = {12-15}
}

###Article{goodnight2005,
  author    = {Goodnight, N. and Wang, R. and Humphreys, G.},
  journal   = {{IEEE Computer Graphics and Applications}},
  title     = {{Computation on programmable graphics hardware}},
  year      = {2005},
  volume    = {25},
  number    = {5},
  pages     = {12-15}
}

@INPROCEEDINGS{Llosa-pact96,
    author = {Josep Llosa and Antonio González and Eduard Ayguadé and Mateo Valero},
    title = {Swing Modulo Scheduling: A Lifetime-Sensitive Approach},
    booktitle = {In IFIP WG10.3 Working Conference on Parallel Architectures and Compilation Techniques (PACT'96},
    year = {1996},
    pages = {80--86}

    }

How do I remove all lines starting with ### inclusive until the next line with a @ exclusive? In essence, my result file would be:

@Article{goodnight2005,
      author    = {Goodnight, N. and Wang, R. and Humphreys, G.},
      journal   = {{IEEE Computer Graphics and Applications}},
      title     = {{Computation on programmable graphics hardware}},
      year      = {2005},
      volume    = {25},
      number    = {5},
      pages     = {12-15}
    }

@INPROCEEDINGS{Llosa-pact96,
        author = {Josep Llosa and Antonio González and Eduard Ayguadé and Mateo Valero},
        title = {Swing Modulo Scheduling: A Lifetime-Sensitive Approach},
        booktitle = {In IFIP WG10.3 Working Conference on Parallel Architectures and Compilation Techniques (PACT'96},
        year = {1996},
        pages = {80--86}

        }

For instance sed '/###/,/@/{//!d}' bibliography.bib keeps the line starting with ###, but sed '/###/,/@/d' bibliography.bib makes the line starting with @ go away.

Thank you so much for the help.

5
  • 1
    what do you mean with "a @ exclusive"? what have you tried so far? Try to read How to Ask to have a better experience here.
    – fedorqui
    Jul 27, 2015 at 14:39
  • No problems with understanding inclusive? Exclusive is the opposite. I mean remove all lines between the first ### until it reaches the @. The line starting with @ is to be preserved. I've seen a couple of sed examples, which I use on a regular basis to remove/replace strings, but not to remove lines, and none suits my needs. Jul 27, 2015 at 14:41
  • 3
    What have you tried? See How much research effort is expected of Stack Overflow users? Jul 27, 2015 at 14:43
  • A BibTeX parser can also be used to strip comments, see for example: Text::BibTeX Jul 27, 2015 at 14:49
  • This is like a one time job for which Text::BibTeX might be overkill, no? Jul 27, 2015 at 14:54

3 Answers 3

2

A simple solution using a $skip sentinel value:

use strict;
use warnings; 

my $skip = 0;
while ( <> ) {
   $skip = 1 if /^###/;
   $skip = 0 if /^@/;
   next if $skip;

   print;
}

output:

[hmcmillen]$ perl test.pl < test.txt 
@Article{goodnight2005,
  author    = {Goodnight, N. and Wang, R. and Humphreys, G.},
  journal   = {{IEEE Computer Graphics and Applications}},
  title     = {{Computation on programmable graphics hardware}},
  year      = {2005},
  volume    = {25},
  number    = {5},
  pages     = {12-15}
}

@INPROCEEDINGS{Llosa-pact96,
    author = {Josep Llosa and Antonio González and Eduard Ayguadé and Mateo Valero},
    title = {Swing Modulo Scheduling: A Lifetime-Sensitive Approach},
    booktitle = {In IFIP WG10.3 Working Conference on Parallel Architectures and Compilation Techniques (PACT'96},
    year = {1996},
    pages = {80--86}
}

If you really want it to be a single command:

perl -ne 'BEGIN { $SKIP = 1 } $SKIP = 1 if /^###/; $SKIP = 0 if /^@/; print unless $SKIP;' < test.txt
5
  • Well, I was hoping for a shorter sed command or so. But this does it for sure. Sorry, hadn't seen the output box when the answer came through. Super. Jul 27, 2015 at 14:59
  • 2
    You never asked for a sed solution. You just mentioned that you had tried a sed command as a solution. Jul 27, 2015 at 15:00
  • I had just refreshed the page to see the first code box but not the rest and I wrongly assumed it as java. Then after I posted the comment I the perl command and the output box displayed on the answer. Sed was not a requirement, you're right. Jul 27, 2015 at 15:05
  • I'm always a tad slow. But really, perl -i -ne is the way to go IMO.
    – dlamblin
    Jul 27, 2015 at 15:09
  • I usually like to verify that the correct substitution was performed before replacing so I usually use -i.bak I just omitted it for this question Jul 27, 2015 at 15:11
1

Suppose your input files are all your *.bib files somewhere in the current directory or lower.

Lemme be your find perl magician for the day:

find . -name '*.bib' -exec \
perl -i -ne '$o=1if/^@/;$o=0if/^###/;print if$o' \{} \;

If you can't read this, do not use it. E.G. it will strip anything before the first @ line and won't consider indented @ or ### lines.

There's also a good module called File::Find, read all about it with perldoc File::Find. Personally that won't keep this as one-liner-y.

0

With awk:

$ awk '/###/{p=0} /@/{p=1} p' bib.text

@Article{goodnight2005,
  author    = {Goodnight, N. and Wang, R. and Humphreys, G.},
  journal   = {{IEEE Computer Graphics and Applications}},
  title     = {{Computation on programmable graphics hardware}},
  year      = {2005},
  volume    = {25},
  number    = {5},
  pages     = {12-15}
}

@INPROCEEDINGS{Llosa-pact96,
    author = {Josep Llosa and Antonio González and Eduard Ayguadé and Mateo Valero},
    title = {Swing Modulo Scheduling: A Lifetime-Sensitive Approach},
    booktitle = {In IFIP WG10.3 Working Conference on Parallel Architectures and Compilation Techniques (PACT'96},
    year = {1996},
    pages = {80--86}

    }

Not the answer you're looking for? Browse other questions tagged or ask your own question.