2

I would like to rewrite a small Perl programm to Python. I am processing text files with it as follows:

Input:

00000001;Root;;
00000002;  Documents;;
00000003;    oracle-advanced_plsql.zip;file;
00000004;  Public;;
00000005;  backup;;
00000006;    20110323-JM-F.7z.001;file;
00000007;    20110426-JM-F.7z.001;file;
00000008;    20110603-JM-F.7z.001;file;
00000009;    20110701-JM-F-via-summer_school;;
00000010;      20110701-JM-F-yyy.7z.001;file;

Desired output:

00000001;;Root;;
00000002;  ;Documents;;
00000003;    ;oracle-advanced_plsql.zip;file;
00000004;  ;Public;;
00000005;  ;backup;;
00000006;    ;20110323-JM-F.7z.001;file;
00000007;    ;20110426-JM-F.7z.001;file;
00000008;    ;20110603-JM-F.7z.001;file;
00000009;    ;20110701-JM-F-via-summer_school;;
00000010;      ;20110701-JM-F-yyy.7z.001;file;

Here is the working Perl code:

#filename: perl_regex.pl
#/usr/bin/perl -w
while(<>) {                                                           
  s/^(.*?;.*?)(\w)/$1;$2/;                                            
  print $_;                                                           
}      

It call it from the command line: perl_regex.pl input.txt

Explanation of the Perl-style regex:

s/        # start search-and-replace regexp
  ^       # start at the beginning of this line
  (       # save the matched characters until ')' in $1
    .*?;  # go forward until finding the first semicolon
    .*?   # go forward until finding... (to be continued below)
  )
  (       # save the matched characters until ')' in $2
    \w    # ... the next alphanumeric character.
  )
/         # continue with the replace part
  $1;$2   # write all characters found above, but insert a ; before $2
/         # finish the search-and-replace regexp.

Could anyone tell me, how to get the same result in Python? Especially for the $1 and $2 variables I couldn't find something alike.

0

2 Answers 2

2

The replace instruction for s/pattern/replace/ in python regexes is the re.sub(pattern, replace, string) function, or re.compile(pattern).sub(replace, string). In your case, you will do it so:

_re_pattern = re.compile(r"^(.*?;.*?)(\w)")
result = _re_pattern.sub(r"\1;\2", line)

Note that $1 becomes \1. As for perl, you need to iterate over your lines the way you want to do it (open, inputfile, splitlines, ...).

6
  • I saw this several times: what is this compiling of a regex about? Is it just saving a string into a variable or does it optimize it the Regex somehow performancewise?
    – royskatt
    Commented Jan 30, 2014 at 14:59
  • 2
    @royskatt Before the regex can be executed, it's parsed, optimized, and transformed into efficient instructions. If you don't manually compile it in Python, it will be compiled on each match, which is unnecessary overhead. Perl has regex literals which mostly hide this.
    – amon
    Commented Jan 30, 2014 at 15:04
  • 1
    I do not understand regexes that well this deep, but I believe this converts the pattern to the Non Finite Automata. This is a step that will be done the first time you call re.sub instead of compiled_reg.sub() and it will then be stored in a cache so that subsequent calls are quicker. I took the habit to compile regexes and I imagine this saves the cache search at each iteration.
    – Cilyan
    Commented Jan 30, 2014 at 15:06
  • 1
    @Cilyan, According to re.compile documentation NOTE, only patterns passed re.search, re.match, re.compile are cached.
    – falsetru
    Commented Jan 30, 2014 at 15:09
  • 1
    @falsetru, I was surprised by your notice, so I looked a little bit into it, and actually the cache works for sub too, at least from 2.6 (I did not look earlier): hg.python.org/cpython/file/7f649ff0756c/Lib/re.py#l168 I guess that makes sense ;)
    – Cilyan
    Commented Jan 31, 2014 at 16:36
1

Python regular expression is very similar to Perl's, except:

  • In Python there's no regular expression literal. It should be expressed using string. I used r'raw string literal' in the following code.
  • Backreferences are expressed as \1, \2, .. or \g<1>, \g<2>, ..
  • ...

Use re.sub to replace.

import re
import sys

for line in sys.stdin: # Explicitly iterate standard input line by line
    # `line` contains trailing newline!
    line = re.sub(r'^(.*?;.*?)(\w)', r'\1;\2', line)
    #print(line) # This print trailing newline
    sys.stdout.write(line) # Print the replaced string back.
4
  • Cool! That's almost as compact and readable as the Perl version. Fit's my brain! Just the backslah in \1 and \2 doesn't look so pythonic to me, but I'm new to python, though. Thanks for your answer!
    – royskatt
    Commented Jan 30, 2014 at 15:02
  • 1
    @royskatt - The Perl re could have been written as s/;\s*\K(?=\w)/;/, using the \Keep (all to the left) assertion and a positive look-ahead, substituting just the first instance. A close Python equivalent is re.sub(r'(;\s*)(?=\w)', r'\1;', line, 1) which avoids one capture and limits the substitution to only once.
    – Kenosis
    Commented Jan 30, 2014 at 17:59
  • @Kenosis: So there is no real PCRE in Python, right? The only think I found was github.com/awahlig/python-pcre, but that doesn't look too official to me.
    – royskatt
    Commented Jan 30, 2014 at 20:22
  • @royskatt - See tchrist's response to Perl Compatible Regular Expression (PCRE) in Python.
    – Kenosis
    Commented Jan 31, 2014 at 6:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.