24

I need to handle URI (i.e. percent) encoding and decoding in my Perl script. How do I do that?


This is a question from the official perlfaq. We're importing the perlfaq to Stack Overflow.

4 Answers 4

33

This is the official FAQ answer minus subsequent edits.

Those % encodings handle reserved characters in URIs, as described in RFC 2396, Section 2. This encoding replaces the reserved character with the hexadecimal representation of the character's number from the US-ASCII table. For instance, a colon, :, becomes %3A.

In CGI scripts, you don't have to worry about decoding URIs if you are using CGI.pm. You shouldn't have to process the URI yourself, either on the way in or the way out.

If you have to encode a string yourself, remember that you should never try to encode an already-composed URI. You need to escape the components separately then put them together. To encode a string, you can use the URI::Escape module. The uri_escape function returns the escaped string:

my $original = "Colon : Hash # Percent %";

my $escaped = uri_escape( $original );

print "$escaped\n"; # 'Colon%20%3A%20Hash%20%23%20Percent%20%25'

To decode the string, use the uri_unescape function:

my $unescaped = uri_unescape( $escaped );

print $unescaped; # back to original

If you wanted to do it yourself, you simply need to replace the reserved characters with their encodings. A global substitution is one way to do it:

# encode
$string =~ s/([^^A-Za-z0-9\-_.!~*'()])/ sprintf "%%%0x", ord $1 /eg;

#decode
$string =~ s/%([A-Fa-f\d]{2})/chr hex $1/eg;
2
  • 1
    Mark Stosberg has a relevant blog post: mark.stosberg.com/blog/2010/12/… on the topic. Dec 22, 2010 at 15:17
  • 1
    Here's a different use case for those who find themselves on this post while doing bash one-liner where you need to encode in a substitute $ echo "Colon : Hash # Percent %" | perl -MURI::Escape -pe 's/.*/uri_escape($&)/e' Output will look something likes this: Colon%20%3A%20Hash%20%23%20Percent%20%25
    – Unantsika
    Aug 26, 2022 at 4:31
5

DIY encode (improving above version):

$string =~ s/([^^A-Za-z0-9\-_.!~*'()])/ sprintf "%%%02x", ord $1 /eg;

(note the '%02x' rather than only '%0x')

DIY decode (adding '+' -> ' '):

$string =~ s/\+/ /g; $string =~ s/%([A-Fa-f\d]{2})/chr hex $1/eg;

Coders helping coders - bartering knowledge!

3

Maybe this will help deciding which method to choose.

Benchmarks on perl 5.32. Every function returns same result for given $input.

Code:

#!/usr/bin/env perl

my $input = "ala ma 0,5 litra 40%'owej vodki :)";

use Net::Curl::Easy;
my $easy = Net::Curl::Easy->new();
use URI::Encode qw( uri_encode );
use URI::Escape qw( uri_escape );
use Benchmark(cmpthese);

cmpthese(-3, {
    'a' => sub {
        my $string = $input;
        $string =~ s/([^^A-Za-z0-9\-_.!~*'()])/ sprintf "%%%0x", ord $1 /eg;
    },
    'b' => sub {
        my $string = $input;
        $string = $easy->escape( $string );
    },
    'c' => sub {
        my $string = $input;
        $string = uri_encode( $string, {encode_reserved => 1} ); 
    },
    'd' => sub {
        my $string = $input;
        $string = uri_escape( $string );
    },
});

And results:

       Rate      c      d      a      b
c    5618/s     --   -98%   -99%  -100%
d  270517/s  4716%     --   -31%   -80%
a  393480/s  6905%    45%     --   -71%
b 1354747/s 24016%   401%   244%     --

Not surprising. A specialized C solution is the fast. An in-place regex with no sub calls is quite fast, followed closely by a copying regex with a sub call. I didn't look into why uri_encode was so much worse than uri_escape.

0
-1

use URI and it will make URLs that just work.

2
  • I'm not sure why this was down voted but a one liner example may be needed. $ perl -MURI -E 'my $u=URI->new("https://host:123/"); $u->query_form("foo/slash"=>"1 x", "bar:colon"=>2, foo=>3); say $u;' https://host:123/?foo%2Fslash=1+x&bar%3Acolon=2&foo=3 Apr 3, 2023 at 14:37
  • and the reverse perl -MURI -E 'my $u=URI->new(q{https://host:123/?foo%2Fslash=1+x&bar%3Acolon=2&foo=3}); say join ", ", $u->query_form;' foo/slash, 1 x, bar:colon, 2, foo, 3 Apr 3, 2023 at 14:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.