2

I don't know enough about Perl to even know what I'm asking for exactly, but I'm writing a series of subroutines to be available for many individual scripts that all process different incoming flat files. The process is far from perfect, but it's what I've got to deal with and I'm trying to build myself a small library of subs that make it easier for me to manage it all. Each script handles a different incoming flat file with it's own formatting, sorting, grouping and outputting requirements. One common aspect is that we have small text files that house counters that are used to name the output files so that we have no duplicate file names.

Because the processing of the data is different for each file, I need to open the file to get my counter value, because this is a common operation, I'd like to put it in a sub to retrieve the counter. But then need to write specific code to process the data. And would like a second sub that allows me to update the counter with the counter once I've processed the data.

Is there a way to make the second sub call a requirement if the first one is called? Ideally if it could even be an error that would prevent the script from running at all much like a syntax error.

EDIT: Here is a little [ugly and simplified] psuedo-code to give a better feel for what the current process is:

require "importLibrary.plx";

#open data source file
open DataIn, $filename;

# call getCounterInfo from importLibrary.plx to get 
# the counter value from counter file
$counter = &getCounterInfo($counterFileName);

while (<DataIn>) {
  # Process data based on unique formatting and requirements
  # output to task files based on requirements and name files 
  # using the $counter increment $counter
}

#update counter file with new value of $counter
&updateCounterInfo($counter);
2
  • 1
    You could call the second subroutine from the first (after all error handling in the first subroutine is done). Or, you could use a global flag that the second sub would check to see if the first sub ran successfully. Or you could pass the indicator from the first sub as one of the arguments into the second sub. TIMTOWTDI! :)
    – user554546
    Sep 10, 2012 at 20:08
  • The problem is that I need the first sub to return the counter so that my script can process and output files using the counter. I can't call the second to update the file until I'm done processing the data. I want to throw an error if the first is called in the script without the second being called at some point further in the script. I can't have a test in the second, because if it's testing a global variable, that means the second was called thereby making the global variable test moot.
    – MitchelWB
    Sep 10, 2012 at 20:15

2 Answers 2

4

I don't quite get what you are trying, but you can always make your subs pluggable:

We have a sub process_file. It takes a subroutine as argument that will do the main processing:

our $counter;
sub process_file {
   my ($subroutine, @args) = @_;
   local $counter = get_counter();
   my @return_value = $subroutine->(@args);
   set_counter($counter);
   return @return_value;
}
# Here are other sub definitions for the main processing
# They can see $counter and always magically have the right value.
# If they assign to it, the counter file will be updated afterwards.

Assuming we have a sub process_type_A, we can then do

my @return_values = process_file(\&process_type_A, $arg1, $arg2, $arg3);

This behaves just like process_type_A($arg1, $arg2, $arg3), except for the extra call stack frame and the $counter setting.

If you prefer passing names instead of coderefs, we can arrange for that too.

package MitchelWB::FileParsingLib;
our $counter;
our %file_type_processing_hash = (
  "typeA" => \&process_type_A,
  "typeB" => \&process_type_B,
  "countLines" => sub { # anonymous sub
     open my $fh, '<', "./dir/$counter.txt" or die "cant open $counter file";
     my $lines = 0;
     $lines++ while <$fh>;
     return $lines;
  },
);

sub process_file {
   my ($filetype, @args) = @_;
   local $counter = get_counter();
   # fetch appropriate subroutine:
   my $subroutine = $file_type_processing_hash{$filetype};
   die "$filetype is not registered" if not defined $subroutine;  # check for existence
   die "$filetype is not assigned to a sub" if ref $subroutine ne 'CODE';  # check that we have a sub
   # execute
   my @return_value = $subroutine->(@args);
   set_counter($counter);
   return @return_value;
}
...;
my $num_of_lines = process_file('countLines');

Edit: The Most Elegant Solution

or: Attributes are really neat

Why stupid callbacks? Why extra code? Why calling conventions? Why dispatch tables? While they all are very interesting and flexible, there is a more elegant solution. I had just forgotten a tiny little piece of information, but now it has all fallen into place. Perl has "Attributes", known as "Annotations" in other languages, that allow us to, well, annotate code or variables.

Defining a new Perl attribute is easy. We use Attribute::Handlers and define a sub with the same name as the attribute you want to use:

sub file_processor :ATTR(CODE) {
  my (undef, $glob, $subroutine) = @_;
  no strict 'refs';
  ${$glob} = sub {
    local $counter = get_counter();
    my @return_value = $subroutine->(@_);
    set_counter($counter);
    return @return_value;
}

We use the attribute :ATTR(CODE) to denote that this is a attribute applicable for subroutines. We only need two arguments, the full name of the subroutine we want to annotate, and a coderef to the sub.

We then turn off a part of the strictness to redefine the sub with ${$glob}. This is a bit advanced, but it essentially just accesses the internal symbol table.

We replace the annotated sub with a dumbed-down version of process_file as given above. We can pass all arguments (@_) right through without further processing.

After all that, we add a tiny litte piece of information to the subs you used before:

sub process_type_A :file_processor {
  print "I can haz $counter\n";
}

… and it just does the replacement without further modifications. The changes are invisible when using the library. I am aware of the restrictions of this approach, but you are unlikely to run into them when writing ordinary code.

5
  • I didn't edit my comment fast enough: So what you're saying is that I would need to also define a few more sub-routines. get_counter() would retrieve that my starting value, set_counter() would write my end result back to my file and in between, I'd call a unique sub to process the data in a sub named to match a string in $subroutine? The content in the $subroutine sub would then be what is unique to the data I'm processing for a given file?
    – MitchelWB
    Sep 10, 2012 at 20:47
  • @MitchelWB Perl is a functional language, so we can pass coderefs/functions as arguments to another subroutine. Inside of process_file, we assign a local value to a global variable - this local values is seen from all subs called from within this scope. Yes, that is what I am suggesting. But we shouldn't pass the function name, but a coderef to that function (we take a coderef with the \& prefix). This allows you to create plugin-functions for a new filetype anywhere and just call it indirectly. This pattern is known as a callback or wrapper.
    – amon
    Sep 10, 2012 at 20:53
  • @MitchelWB I added an example for a "function dispatch table". That is, a hash that associates names with functions. You can extend the hash at runtime to add new methods if needed.
    – amon
    Sep 10, 2012 at 21:09
  • @MitchelWB I added another solution. This is even less comprehensible, but once implemented can be used easily and intuetively. I use attributes to modify the symbol table. You can consult the Attribute::Handlers documentation for further information. To use this attribute outside your module, it has to be subclassed with use base MODULE or use parent MODULE.
    – amon
    Sep 11, 2012 at 21:03
  • wow, I don't get it at all! :) But I seriously appreciate your help.
    – MitchelWB
    Oct 1, 2012 at 18:54
0

Well, you could set a global flag and use an END block.

Perhaps neater is something like @amon's proposal or even just putting your file processing in a standard named sub and calling it from your counter code.

my ($fh, counter) = get_counter(...);
my $ok = process_file($fh, $counter);
update_counter($counter) if $ok;

Your process_file will be exported from a module, and if you want to keep it really simple, use perl's -Μ to load the module with you process_file sub.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.