143
votes

What are some really useful but esoteric language features in Perl that you've actually been able to employ to do useful work?

Guidelines:

  • Try to limit answers to the Perl core and not CPAN
  • Please give an example and a short description

Hidden Features also found in other languages' Hidden Features:

(These are all from Corion's answer)

  • C
    • Duff's Device
    • Portability and Standardness
  • C#
    • Quotes for whitespace delimited lists and strings
    • Aliasable namespaces
  • Java
    • Static Initalizers
  • JavaScript
    • Functions are First Class citizens
    • Block scope and closure
    • Calling methods and accessors indirectly through a variable
  • Ruby
    • Defining methods through code
  • PHP
    • Pervasive online documentation
    • Magic methods
    • Symbolic references
  • Python
    • One line value swapping
    • Ability to replace even core functions with your own functionality

Other Hidden Features:

Operators:

Quoting constructs:

Syntax and Names:

Modules, Pragmas, and command-line options:

Variables:

Loops and flow control:

Regular expressions:

Other features:

Other tricks, and meta-answers:


See Also:

1
  • Most of these features are in everyday use, some occur in the majority of Perl scripts, and most listed under "Other" still stem from other languages, calling these "hidden" changes the intent of the question. May 26, 2010 at 9:24

78 Answers 78

9
votes

The input record separator can be set to a reference to a number to read fixed length records:

$/ = \3; print $_,"\n" while <>; # output three chars on each line
7
votes

I don't know how esoteric it is, but one of my favorites is the hash slice. I use it for all kinds of things. For example to merge two hashes:

my %number_for = (one => 1, two => 2, three => 3);
my %your_numbers = (two => 2, four => 4, six => 6);
@number_for{keys %your_numbers} = values %your_numbers;
print sort values %number_for; # 12346
1
  • %number_for = ( %number_for, %your_numbers );
    – ijw
    Nov 18, 2009 at 13:36
7
votes

This one isn't particularly useful, but it's extremely esoteric. I stumbled on this while digging around in the Perl parser.

Before there was POD, perl4 had a trick to allow you to embed the man page, as nroff, straight into your program so it wouldn't get lost. perl4 used a program called wrapman (see Pink Camel page 319 for some details) to cleverly embed an nroff man page into your script.

It worked by telling nroff to ignore all the code, and then put the meat of the man page after an END tag which tells Perl to stop processing code. Looked something like this:

#!/usr/bin/perl
'di';
'ig00';

...Perl code goes here, ignored by nroff...

.00;        # finish .ig

'di         \" finish the diversion
.nr nl 0-1  \" fake up transition to first page
.nr % 0     \" start at page 1
'; __END__

...man page goes here, ignored by Perl...

The details of the roff magic escape me, but you'll notice that the roff commands are strings or numbers in void context. Normally a constant in void context produces a warning. There are special exceptions in op.c to allow void context strings which start with certain roff commands.

              /* perl4's way of mixing documentation and code
                 (before the invention of POD) was based on a
                 trick to mix nroff and perl code. The trick was
                 built upon these three nroff macros being used in
                 void context. The pink camel has the details in
                 the script wrapman near page 319. */
                const char * const maybe_macro = SvPVX_const(sv);
                if (strnEQ(maybe_macro, "di", 2) ||
                    strnEQ(maybe_macro, "ds", 2) ||
                    strnEQ(maybe_macro, "ig", 2))
                        useless = NULL;

This means that 'di'; doesn't produce a warning, but neither does 'die'; 'did you get that thing I sentcha?'; or 'ignore this line';.

In addition, there are exceptions for the numeric constants 0 and 1 which allows the bare .00;. The code claims this was for more general purposes.

            /* the constants 0 and 1 are permitted as they are
               conventionally used as dummies in constructs like
                    1 while some_condition_with_side_effects;  */
            else if (SvNIOK(sv) && (SvNV(sv) == 0.0 || SvNV(sv) == 1.0))
                useless = NULL;

And what do you know, 2 while condition does warn!

1
  • This one is really a hidden feature of Perl!
    – dolmen
    Mar 29, 2011 at 21:00
7
votes

You can use @{[...]} to get an interpolated result of complex perl expressions

$a = 3;
$b = 4;

print "$a * $b = @{[$a * $b]}";

prints: 3 * 4 = 12

6
votes
sub load_file
{
    local(@ARGV, $/) = shift;
    <>;
}

and a version that returns an array as appropriate:

sub load_file
{
    local @ARGV = shift;
    local $/ = wantarray? $/: undef;
    <>;
}
0
6
votes
use diagnostics;

If you are starting to work with Perl and have never done so before, this module will save you tons of time and hassle. For almost every basic error message you can get, this module will give you a lengthy explanation as to why your code is breaking, including some helpful hints as to how to fix it. For example:

use strict;
use diagnostics;

$var = "foo";

gives you this helpful message:

Global symbol "$var" requires explicit package name at - line 4.
Execution of - aborted due to compilation errors (#1)
    (F) You've said "use strict vars", which indicates that all variables
    must either be lexically scoped (using "my"), declared beforehand using
    "our", or explicitly qualified to say which package the global variable
    is in (using "::").

Uncaught exception from user code:
        Global symbol "$var" requires explicit package name at - line 4.
Execution of - aborted due to compilation errors.
 at - line 5
use diagnostics;
use strict;

sub myname {
    print { " Some Error " };
};

you get this large, helpful chunk of text:

syntax error at - line 5, near "};"
Execution of - aborted due to compilation errors (#1)
(F) Probably means you had a syntax error.  Common reasons include:

    A keyword is misspelled.
    A semicolon is missing.
    A comma is missing.
    An opening or closing parenthesis is missing.
    An opening or closing brace is missing.
    A closing quote is missing.

Often there will be another error message associated with the syntax
error giving more information.  (Sometimes it helps to turn on -w.)
The error message itself often tells you where it was in the line when
it decided to give up.  Sometimes the actual error is several tokens
before this, because Perl is good at understanding random input.
Occasionally the line number may be misleading, and once in a blue moon
the only way to figure out what's triggering the error is to call
perl -c repeatedly, chopping away half the program each time to see
if the error went away.  Sort of the cybernetic version of S.

Uncaught exception from user code:
    syntax error at - line 5, near "};"
Execution of - aborted due to compilation errors.
at - line 7

From there you can go about deducing what might be wrong with your program (in this case, print is formatted entirely wrong). There's a large number of known errors with diagnostics. Now, while this would not be a good thing to use in production, it can serve as a great learning aid for those who are new to Perl.

5
votes

($x, $y) = ($y, $x) is what made me want to learn Perl.

The list constructor 1..99 or 'a'..'zz' is also very nice.

5
votes

There also is $[ the variable which decides at which index an array starts. Default is 0 so an array is starting at 0. By setting

$[=1;

You can make Perl behave more like AWK (or Fortran) if you really want to.

3
  • 3
    Although to quote from the perlvar documentuation: "Its use is highly discouraged.". Not many people expect the starting subscript of an array to change.
    – pjf
    Oct 2, 2008 at 13:17
  • 2
    I would only use this feature in a one-liner, if ever. Oct 2, 2008 at 22:05
  • be warned: if you do this in a CPAN module, chromatic will find you and spray-paint your car. (this is a joke; chromatic has not been consulted regarding it.)
    – davidnicol
    Oct 3, 2008 at 21:08
5
votes

@Schwern mentioned turning warnings into errors by localizing $SIG{__WARN__}. You can do also do this (lexically) with use warnings FATAL => "all";. See perldoc lexwarn.

On that note, since Perl 5.12, you've been able to say perldoc foo instead of the full perldoc perlfoo. Finally! :)

4
votes

Safe compartments.

With the Safe module you can build your own sandbox-style environment using nothing but perl. You would then be able to load perl scripts into the sandbox.

Best regards,

4
votes

How about the ability to use

my @symbols = map { +{ 'key' => $_ } } @things;

to generate an array of hashrefs from an array -- the + in front of the hashref disambiguates the block so the interpreter knows that it's a hashref and not a code block. Awesome.

(Thanks to Dave Doyle for explaining this to me at the last Toronto Perlmongers meeting.)

4
votes

Use lvalues to make your code really confusing:

my $foo = undef ;
sub bar:lvalue{ return $foo ;}

# Then later

bar = 5 ;
print bar ;
4
votes

Core IO::Handle module. Most important thing for me is that it allows autoflush on filehandles. Example:

use IO::Handle;    
$log->autoflush(1);
1
  • Not like we didn’t know how to do it before IO::Handle, you know.
    – tchrist
    Nov 20, 2010 at 2:46
4
votes

The Schwartzian Transform is a technique that allows you to efficiently sort by a computed, secondary index. Let's say that you wanted to sort a list of strings by their md5 sum. The comments below are best read backwards (that's the order I always end up writing these anyways):

my @strings = ('one', 'two', 'three', 'four');

my $md5sorted_strings = 
    map { $_->[0] }               # 4) map back to the original value
    sort { $a->[1] cmp $b->[1] }  # 3) sort by the correct element of the list
    map { [$_, md5sum_func($_)] } # 2) create a list of anonymous lists
    @strings                      # 1) take strings

This way, you only have to do the expensive md5 computation N times, rather than N log N times.

4
votes

All right. Here is another. Dynamic Scoping. It was talked about a little in a different post, but I didn't see it here on the hidden features.

Dynamic Scoping like Autovivification has a very limited amount of languages that use it. Perl and Common Lisp are the only two I know of that use Dynamic Scoping.

1
  • Ohh ya, "local" designates a dynamically scoped variable, while "my" designates a variable in a static scope.
    – J.J.
    Oct 5, 2008 at 15:14
4
votes

One useful composite operator for conditionally adding strings or lists into other lists is the x!!operator:

 print 'the meaning of ', join ' ' =>  
     'life,'                x!! $self->alive,
     'the universe,'        x!! ($location ~~ Universe),
     ('and', 'everything.') x!! 42; # this is added as a list

this operator allows for a reversed syntax similar to

 do_something() if test();
4
votes

This one-liner illustrates how to use glob to generate all word combinations of an alphabet (A, T, C, and G -> DNA) for words of a specified length (4):

perl -MData::Dumper -e '@CONV = glob( "{A,T,C,G}" x 4 ); print Dumper( \@CONV )'
3
votes

My favorite semi-hidden feature of Perl is the eof function. Here's an example pretty much directly from perldoc -f eof that shows how you can use it to reset the file name and $. (the current line number) easily across multiple files loaded up at the command line:

while (<>) {
  print "$ARGV:$.\t$_";
} 
continue {
  close ARGV if eof
}
3
votes

I personally love the /e modifier to the s/// operation:

while(<>) {
  s/(\w{0,4})/reverse($1);/e; # reverses all words between 0 and 4 letters
  print;
}

Input:

This is a test of regular expressions
^D

Output (I think):

sihT si a tset fo regular expressions
3
votes

You can replace the delimiter in regexes and strings with just about anything else. This is particularly useful for "leaning toothpick syndrome", exemplified here:

$url =~ /http:\/\/www\.stackoverflow\.com\//;

You can eliminate most of the back-whacking by changing the delimiter. /bar/ is shorthand for m/bar/ which is the same as m!bar!.

$url =~ m!http://www\.stackoverflow\.com/!;

You can even use balanced delimiters like {} and []. I personally love these. q{foo} is the same as 'foo'.

$code = q{
    if( this is awesome ) {
        print "Look ma, no escaping!";
    }
};

To confuse your friends (and your syntax highlighter) try this:

$string = qq'You owe me $1,000 dollars!';
1
  • You should explicitly mention that, when using {} (and friends) as quote delimiters, Perl will balance the delimiters.
    – Chris Lutz
    Aug 25, 2009 at 0:05
3
votes

Very late to the party, but: attributes.

Attributes essentially let you define arbitrary code to be associated with the declaration of a variable or subroutine. The best way to use these is with Attribute::Handlers; this makes it easy to define attributes (in terms of, what else, attributes!).

I did a presentation on using them to declaratively assemble a pluggable class and its plugins at YAPC::2006, online here. This is a pretty unique feature.

3
votes

Quantum::Superpositions

use Quantum::Superpositions;

if ($x == any($a, $b, $c)) { ...  }
3
votes

There is a more powerful way to check program for syntax errors:

perl -w -MO=Lint,no-context myscript.pl

The most important thing that it can do is reporting for 'unexistant subroutine' errors.

3
votes

use re debug
Doc on use re debug

and

perl -MO=Concise[,OPTIONS]
Doc on Concise

Besides being exquisitely flexible, expressive and amenable to programing in the style of C, Pascal, Python and other languages, there are several pragmas command switches that make Perl my 'goto' language for initial kanoodling on an algorithm, regex, or quick problems that needs to be solved. These two are unique to Perl I believe, and are among my favorites.

use re debug: Most modern flavors of regular expressions owe their current form and function to Perl. While there are many Perl forms of regex that cannot be expressed in other languages, there are almost no forms of other languages' flavor of regex that cannot be expressed in Perl. Additionally, Perl has a wonderful regex debugger built in to show how the regex engine is interpreting your regex and matching against the target string.

Example: I recently was trying to write a simple CSV routine. (Yes, yes, I know, I should have been using Text::CSV...) but the CSV values were not quoted and simple.

My first take was /^(^(?:(.*?),){$i}/ to extract the i record on n CSV records. That works fine -- except for the last record or n of n. I could see that without the debugger.

Next I tried /^(?:(.*?),|$){$i}/ This did not work, and I could not see immediately why. I thought I was saying (.*?) followed by a comma or EOL. Then I added use re debug at the top of a small test script. Ahh yes, the alteration between ,|$ was not being interpreted that way; it was being interpreted as ((.*?),) | ($) -- not what I wanted.

A new grouping was needed. So I arrived at the working /^(?:(.*?)(?:,|$)){$i}/. While I was in the regex debugger, I was surprised how many loops it took for a match towards the end of the string. It is the .*? term that is quite ambiguous and requires excessive backtracking to satisfy. So I tried /^(?:(?:^|,)([^,]*)){$i}/ This does two things: 1) reduces backtracking because of the greedy match of all but a comma 2) allowed the regex optimizer to only use the alteration once on the first field. Using Benchmark, this is 35% faster than the first regex. The regex debugger is wonderful and few use it.

perl -MO=Concise[,OPTIONS]: The B and Concise frameworks are tremendous tools to see how Perl is interpreting your masterpiece. Using the -MO=Concise prints the result of the Perl interpreters translation of your source code. There are many options to Concise and in B, you can write your own presentation of the OP codes.

As in this post, you can use Concise to compare different code structures. You can interleave your source lines with the OP codes those lines generate. Check it out.

3
votes

You can use different quotes on HEREDOCS to get different behaviors.

my $interpolation = "We will interpolated variables";
print <<"END";
With double quotes, $interpolation, just like normal HEREDOCS.
END

print <<'END';
With single quotes, the variable $foo will *not* be interpolated.
(You have probably seen this in other languages.)
END

## this is the fun and "hidden" one
my $shell_output = <<`END`;
echo With backticks, these commands will be executed in shell.
echo The output is returned.
ls | wc -l
END

print "shell output: $shell_output\n";
2
votes

The following are just as short but more meaningful than "~~" since they indicate what is returned, and there's no confusion with the smart match operator:

print "".localtime;   # Request a string

print 0+@array;       # Request a number
2
votes

Axeman reminded me of how easy it is to wrap some of the built-in functions.

Before Perl 5.10 Perl didn't have a pretty print(say) like Python.

So in your local program you could do something like:

sub print {
     print @_, "\n";
}

or add in some debug.

sub print {
    exists $ENV{DEVELOPER} ?
    print Dumper(@_) :
    print @_;
}
5
  • It's also very easy to accidentally change the context! Your print subroutine (using . to concatenate) will print the number of items to be printed, rather than the items themselves. Using print @_, "\n" (note the comma) will preserve the context.
    – pjf
    Oct 2, 2008 at 22:22
  • :-D tks for the clarification. I will edit accordingly. :-D teach me to write code without running it :-P
    – J.J.
    Oct 3, 2008 at 1:55
  • Er...except I don't think you can override print this way. Most other builtins I think you can, but not print. :|
    – Robert P
    Apr 3, 2009 at 22:57
  • This is true - print cannot be overridden.
    – Chris Lutz
    Aug 25, 2009 at 0:16
  • @Chris: It's true print cannot be overridden in this manner. But not that it cannot be overridden absolutely. Using some sub modules of B and some tricks you can find on perlmonks.com you can override it.
    – J.J.
    Sep 9, 2009 at 17:23
2
votes

Two things that work well together: IO handles on in-core strings, and using function prototypes to enable you to write your own functions with grep/map-like syntax.

sub with_output_to_string(&) {           # allows compiler to accept "yoursub {}" syntax.
  my $function = shift;
  my $string   = '';
  my $handle   = IO::Handle->new();
  open($handle, '>', \$string) || die $!; # IO handle on a plain scalar string ref
  my $old_handle = select $handle;
  eval { $function->() };
  select $old_handle;
  die $@ if $@;
  return $string;
}

my $greeting = with_output_to_string {
  print "Hello, world!";
};

print $greeting, "\n";
2
votes

The ability to use a hash as a seen filter in a loop. I have yet to see something quite as nice in a different language. For example, I have not been able to duplicate this in python.

For example, I want to print a line if it has not been seen before.

my %seen;

for (<LINE>) {
  print $_ unless $seen{$_}++;
}
2
votes

The new -E option on the command line:

> perl -e "say 'hello"" # does not work 

String found where operator expected at -e line 1, near "say 'hello'"
        (Do you need to predeclare say?)
syntax error at -e line 1, near "say 'hello'"
Execution of -e aborted due to compilation errors.

> perl -E "say 'hello'" 
hello

Not the answer you're looking for? Browse other questions tagged or ask your own question.