2

I wonder if there is an efficient way to delete the first line in a file if it matches a specified pattern. For example, I have a file with data of the following form:

Date,Open,High,Low,Close,Volume,Adj.Volume
2012-01-27,42.38,42.95,42.27,42.68,2428000,42.68
2012-01-26,44.27,44.85,42.48,42.66,5785700,42.66
.
.
.

I want to delete the first line, only if it contains the text (as shown in the example in the first line), and leave it unchanged if it contains only numbers(as in the rest of the lines). This task is quite easy and I've accomplished it by applying the following peace of code which writes each line to a $newFile as long as it does not include Date pattern:

while( <$origFile> )
    {
        chomp($_);
        print $newFile $_ unless ($_  =~ m/Date/g)
    }

So as I mentioned, that makes the job done. However it seems that it's a great waste of resources to read each line in a whole file when it is known that the text can appear only in the first line..

Is there any way to accomplish this task more efficiently?

NOTE: I already found an almost similar question here, but since I want my code to be available on Linux and Windows as well, using sed will not help me here.

Thanks in advance!

2
  • 4
    Your code will delete this line of text regardless of position in the file. As for inefficiency: you have to read all lines in the file, you cannot remove bytes from the beginning of a file (that's how file systems work). Note: you will find your answer in Perl: How do I remove the first line of a file without reading and copying whole file – simply combine with your regex.
    – knittl
    Feb 5, 2012 at 10:27
  • 1
    (1) You don't want to be chomping without then adding back a newline, because this will put your entire input file on one line! (2) You don't need to specify "$_ =~" because m// works on $_ by default. (3) You don't need the 'g' flag on the m//; it's not doing anything here.
    – zgpmax
    Feb 7, 2012 at 0:12

3 Answers 3

3

$. can be used to determine if are processing the first line of the file.

perl -i.bak -ne'print if $. != 1 || !/^Date/;' file
However it seems that it's a great waste of resources to read each line in a whole file

It's impossible to delete from anywhere but the end of a file. To delete from the start or middle, everything that follows in the file needs to be shifted, which means it must be both read and written.

You can only avoid work if the first line doesn't match (by doing nothing at all). If you need to remove the line, you must copy the whole file.

0
2

The Tie::File module is ideal for this. It is very efficient as it does block IO instead of reading a line at a time, and it makes the program very simple to write.

use strict;
use warnings;

use Tie::File;

tie my @data, 'Tie::File', 'mydatafile' or die $!;
shift @data if $data[0] =~ /Date/;
untie @data;
1

Only do the test on the first line, then just run through the rest of the file without checking:

if (defined( $_ = <$origFile> )) {
    if ( ! m/Date/o ) { print $newFile $_ }

    my $data;

    for (;;) {
        my $readRes = read($origFile, $data, 0x10000);

        if (!defined $readRes) { die "Can't read: $!" }

        if ($readRes == 0) { last }

        print $newFile $data;
    }
}
2
  • Hi and thanks for your response! However I'm not sure I fully understand you r code... Could you please provide some explanation, specifically for (;;) and read parameters. Thank you!
    – Eugene S
    Feb 8, 2012 at 13:22
  • 1
    for (;;) is shorthand for while (1), i.e. loop without specifying an exit condition (we exit the loop in the middle using last). The builtin function read is documented at perldoc.perl.org/functions/read.html
    – zgpmax
    Feb 8, 2012 at 13:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.