1

I am a beginner in perl, so please bear with me.

I have 2 files:

1
2
3

and

2
4
5
6

I want to create a new file that is the sum of the above 2 files:

output file:

3
6
8
6

What I am doing right now is reading the files as arrays and adding them element by element.

To add the arrays I am using the following:

$asum[@asum] = $array1[@asum] + $array2[@asum] while defined $array1[@asum] or defined $array2[@asum];

But this is giving the following error:

Argument "M-oM-;M-?3" isn't numeric in addition (+) at perl_ii.pl line 30.
Argument "M-oM-;M-?1" isn't numeric in addition (+) at perl_ii.pl line 30.
Use of uninitialized value in addition (+) at perl_ii.pl line 30.

I am using the following code to read files as arrays:

use strict;
use warnings;

my @array1;
open(my $fh, "<", "file1.txt") or die "Failed to open file1\n";
while(<$fh>) { 
    chomp; 
    push @array1, $_;
} 
close $fh;

my @array2;
open(my $fh1, "<", "file2.txt") or die "Failed to open file2\n";
while(<$fh1>) {
    chomp;
    push @array2, $_;
}
close $fh1 ;

Anyone could tell me how to fix this, or give a better approach altogether?

6 Answers 6

3

Here is another Perl solution that makes use of the diamond, <>, file read operator. This reads in files specified on the command line, (rather than explicitly opening them within the program). Sorry, I can't find the part of the docs that explains this for a read.

The command line for this program would look like:

perl myprogram.pl file1 file2 > outputfile

Where file1 and file2 are the 2 input files and outputfile is the file you want to print the results of the addition.

#!/usr/bin/perl
use strict;
use warnings;

my @sums;
my $i = 0;
while (my $num = <>) {
    $sums[$i++] += $num;
    $i = 0 if eof;
}

print "$_\n" for @sums;

Note: $i is reset to zero at the end of file, (in this case after the first file is read). Actually, it is also reset to 0 after the second file is read. This has no effect on the program however, because there are no files to be read after the second file in your example.

2
  • Thanks a lot! Seems like a shorter solution to my problem :)
    – user324
    Jun 23, 2016 at 4:24
  • 1
    This is an improvement over holding all files in memory, but its memory footprint is still determined by the size of the largest file. See How to sum data from multiple files in Perl?. Jun 30, 2016 at 16:35
3

The following solution makes the memory footprint of the program independent of the sizes of the files. Instead, now the memory footprint only depends on the number of files:

#!/usr/bin/env perl

use strict;
use warnings;

use Carp qw( croak );
use List::Util qw( sum );
use Path::Tiny;

run(@ARGV);

sub run {
    my @readers = map make_reader($_), @_;

    while (my @obs = grep defined, map $_->(), @readers) {
        print sum(@obs), "\n";
    }

    return;
}

sub make_reader {
    my $fname = shift;
    my $fhandle = path( $fname )->openr;
    my $is_readable = 1;
    sub {
        return unless $is_readable;

        my $line = <$fhandle>;
        return $line if defined $line;

        close $fhandle
            or croak "Cannot close '$fname': $!";

        $is_readable = 0;
        return;
    }
}
6
  • Thanks, Sinan. I have added your program to my saved programs. I have seen other problems that needed to read 2 or more files one line each at a time and then process them. I hadn't thought about the possible memory problem when I offered my solution here. Just wondering - why did you use croak from the Carp module instead of die. Is it usual to use Carp when building a module and you want to use croak then? I've never been too informed about when to use Carp instead of dying. I see you edited your make_reader sub - guess it does the same as the original definition. Jun 30, 2016 at 20:41
  • The main benefit of croak is that you can force a stack dump without changing the script. I edited my answer to avoid the if/else. I think a linear flows read much better. I have written countless variations of this, some with logging, better error recovery, formatting etc, but I tried to avoid all of that to distill everything to its essence here. Jun 30, 2016 at 21:12
  • 2
    Wow, this is a treasure-trove. Great use of closure, rare to see it in real life. Thank you.
    – zdim
    Jul 1, 2016 at 1:48
  • @SinanÜnür You mentioned other versions of your code with better error recovery (and I assume error handling). I find it inconsistent that you check for errors on close, but not on <>. I can't tell about checking for errors on open since it depends on the error handling of path and readr. Would you consider modifying your answer to either ignore the close errors or, preferably add error handling for <>?
    – jrw32982
    Jul 4, 2016 at 15:17
  • @JohnWiersba The kind of error handling I am referring to is about checking that the input conforms to a spec. A quick glance at Path::Tiny will show, openr croaks on failure. As for errors from readline, see perldoc -f readline: "It can be helpful to check $! when you are reading from filehandles you don't trust, such as a tty or a socket". In normal circumstances, checking for errors in readline is rather superfluous. In this case, if readline returns undef, we immediately try to close the file. If something went wrong, that'll tell us. Jul 4, 2016 at 20:37
2

You have two different problems with your script now:

  1. First error

    Argument "M-oM-;M-?3" isn't numeric in addition (+) at perl_ii.pl line 30

    happens because your input files are saved in Unicode and first line is read with "\xFF\xFE" BOM bytes. To fix it simply, just resave the files as ANSI text. If Unicode is required, then remove these bytes from first string you read from file.

  2. Second error

    Use of uninitialized value in addition (+) at perl_ii.pl line 30.

    happens because you access 4th element in first array that doesn't exist. Remember, you select maximal input array length as index limit. To fix it just add following condition for input element:

    $asum[@asum] = (@asum < @array1 ? $array1[@asum] : 0)  + (@asum < @array2 ? $array2[@asum] : 0) while defined $array1[@asum] or defined $array2[@asum];
    
0
2

The logic of reading your two files is the same, and I suggest using a subroutine for that and calling it twice:

#!/usr/bin/env perl

use strict;
use warnings;

my @array1 = read_into_array('file1.txt');
my @array2 = read_into_array('file2.txt');

sub read_into_array
{
    my $filename = shift;
    my @array;
    open(my $fh, "<", $filename) or die "Failed to open $filename: $!\n";
    while(<$fh>) { 
        chomp; 
        push @array, $_;
    } 
    close $fh;
    return @array;
}

But that's just an observation I made and not a solution to your problem. As CodeFuller already said, you should re-save your files as plain ASCII instead of UTF-8.

The second problem, Use of uninitialized value in addition (+), can also be solved with the Logical Defined Or operator // which was introduced in Perl 5.10:

my @asum;
$asum[@asum] = ($array1[@asum] // 0) 
             + ($array2[@asum] // 0) 
             while defined $array1[@asum] or defined $array2[@asum];

No, this is not a comment, but an operator very similar to ||. The difference is that it triggers when the left-hand-side (lhs) is undef while the || triggers when the lhs is falsy (i.e. 0, '' or undef). Thus

$array1[@asum] // 0

gives 0 if $array1[@asum] is undef. It's the same as

defined($array1[@asum]) ? $array1[@asum] : 0
1
  • Thanks for the help!
    – user324
    Jun 23, 2016 at 4:19
1

A different approach altogether:

$ paste -d '+' file1 file2 | sed 's/^+//;s/+$//' | bc
3
6
8
6

The paste command prints the files next to each other, separated by a + sign:

$ paste -d '+' file1 file2
1+2
2+4
3+5
+6

The sed command removes leading and trailing + signs, because those trip up bc:

$ paste -d '+' file1 file2 | sed 's/^+//;s/+$//'
1+2
2+4
3+5
6

And bc finally calculates the sums.

1
  • Thanks! But I was looking for a solution specifically in Perl. :)
    – user324
    Jun 23, 2016 at 4:20
1

Here’s a rendition of Sinan’s approach in a more Perlish form:

#!/usr/bin/env perl
use 5.010; use strict; use warnings;

use autodie;
use List::Util 'sum';

my @fh = map { open my $fh, '<', $_; $fh } @ARGV;

while ( my @value = grep { defined } map { scalar readline $_ } @fh ) {
    say sum @value;
    @fh = grep { not eof $_ } @fh if @value < @fh;
}

Your Answer

Reminder: Answers generated by Artificial Intelligence tools are not allowed on Stack Overflow. Learn more

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.