10

In a text editor, I want to replace a given word with the number of the line number on which this word is found. Is this is possible with Regex?

4
  • Any text editor? Some of them don't support regexes at all.
    – choroba
    Commented May 18, 2014 at 22:26
  • Short answer: No. Regular expressions don't replace or do anything else. They simply define a pattern
    – ikegami
    Commented May 18, 2014 at 22:27
  • 5
    StackOverflow is for programming questions. Questions about how to use your editor belong on Super User.
    – ikegami
    Commented May 18, 2014 at 22:29
  • Do you mean a text editor like Vim? Commented Sep 8, 2023 at 21:18

4 Answers 4

21
+150

Recursion, Self-Referencing Group (Qtax trick), Reverse Qtax or Balancing Groups

Introduction

The idea of adding a list of integers to the bottom of the input is similar to a famous database hack (nothing to do with regex) where one joins to a table of integers. My original answer used the @Qtax trick. The current answers use either Recursion, the Qtax trick (straight or in a reversed variation), or Balancing Groups.

Yes, it is possible... With some caveats and regex trickery.

  1. The solutions in this answer are meant as a vehicle to demonstrate some regex syntax more than practical answers to be implemented.
  2. At the end of your file, we will paste a list of numbers preceded with a unique delimiter. For this experiment, the appended string is :1:2:3:4:5:6:7 This is a similar technique to a famous database hack that uses a table of integers.
  3. For the first two solutions, we need an editor that uses a regex flavor that allows recursion (solution 1) or self-referencing capture groups (solutions 2 and 3). Two come to mind: Notepad++ and EditPad Pro. For the third solution, we need an editor that supports balancing groups. That probably limits us to EditPad Pro or Visual Studio 2013+.

Input file:

Let's say we are searching for pig and want to replace it with the line number.

We'll use this as input:

my cat
dog
my pig
my cow
my mouse

:1:2:3:4:5:6:7

First Solution: Recursion

Supported languages: Apart from the text editors mentioned above (Notepad++ and EditPad Pro), this solution should work in languages that use PCRE (PHP, R, Delphi), in Perl, and in Python using Matthew Barnett's regex module (untested).

The recursive structure lives in a lookahead, and is optional. Its job is to balance lines that don't contain pig, on the left, with numbers, on the right: think of it as balancing a nested construct like {{{ }}}... Except that on the left we have the no-match lines, and on the right we have the numbers. The point is that when we exit the lookahead, we know how many lines were skipped.

Search:

(?sm)(?=.*?pig)(?=((?:^(?:(?!pig)[^\r\n])*(?:\r?\n))(?:(?1)|[^:]+)(:\d+))?).*?\Kpig(?=.*?(?(2)\2):(\d+))

Free-Spacing Version with Comments:

(?xsm)             # free-spacing mode, multi-line
(?=.*?pig)        # fail right away if pig isn't there

(?=               # The Recursive Structure Lives In This Lookahead
(                 # Group 1
   (?:               # skip one line 
      ^              
      (?:(?!pig)[^\r\n])*  # zero or more chars not followed by pig
      (?:\r?\n)      # newline chars
    ) 
    (?:(?1)|[^:]+)   # recurse Group 1 OR match all chars that are not a :
    (:\d+)           # match digits
)?                 # End Group 
)                 # End lookahead. 
.*?\Kpig                # get to pig
(?=.*?(?(2)\2):(\d+))   # Lookahead: capture the next digits

Replace: \3

In the demo, see the substitutions at the bottom. You can play with the letters on the first two lines (delete a space to make pig) to move the first occurrence of pig to a different line, and see how that affects the results.


Second Solution: Group that Refers to Itself ("Qtax Trick")

Supported languages: Apart from the text editors mentioned above (Notepad++ and EditPad Pro), this solution should work in languages that use PCRE (PHP, R, Delphi), in Perl, and in Python using Matthew Barnett's regex module (untested). The solution is easy to adapt to .NET by converting the \K to a lookahead and the possessive quantifier to an atomic group (see the .NET Version a few lines below.)

Search:

(?sm)(?=.*?pig)(?:(?:^(?:(?!pig)[^\r\n])*(?:\r?\n))(?=[^:]+((?(1)\1):\d+)))*+.*?\Kpig(?=[^:]+(?(1)\1):(\d+))

.NET version: Back to the Future

.NET does not have \K. It its place, we use a "back to the future" lookbehind (a lookbehind that contains a lookahead that skips ahead of the match). Also, we need to use an atomic group instead of a possessive quantifier.

(?sm)(?<=(?=.*?pig)(?=(?>(?:^(?:(?!pig)[^\r\n])*(?:\r?\n))(?=[^:]+((?(1)\1):\d+)))*).*)pig(?=[^:]+(?(1)\1):(\d+))

Free-Spacing Version with Comments (Perl / PCRE Version):

(?xsm)             # free-spacing mode, multi-line
(?=.*?pig)        # lookahead: if pig is not there, fail right away to save the effort
(?:               # start counter-line-skipper (lines that don't include pig)
   (?:               # skip one line 
      ^              # 
      (?:(?!pig)[^\r\n])*  # zero or more chars not followed by pig
      (?:\r?\n)      # newline chars
    )   
   # for each line skipped, let Group 1 match an ever increasing portion of the numbers string at the bottom
   (?=             # lookahead
      [^:]+           # skip all chars that are not colons
      (               # start Group 1
        (?(1)\1)      # match Group 1 if set
        :\d+          # match a colon and some digits
      )               # end Group 1
   )               # end lookahead
)*+               # end counter-line-skipper: zero or more times
.*?               # match
\K                # drop everything we've matched so far
pig               # match pig (this is the match!)
(?=[^:]+(?(1)\1):(\d+))   # capture the next number to Group 2

Replace:

\2

Output:

my cat
dog
my 3
my cow
my mouse

:1:2:3:4:5:6:7

In the demo, see the substitutions at the bottom. You can play with the letters on the first two lines (delete a space to make pig) to move the first occurrence of pig to a different line, and see how that affects the results.

Choice of Delimiter for Digits

In our example, the delimiter : for the string of digits is rather common, and could happen elsewhere. We can invent a UNIQUE_DELIMITER and tweak the expression slightly. But the following optimization is even more efficient and lets us keep the :


Optimization on Second Solution: Reverse String of Digits

Instead of pasting our digits in order, it may be to our benefit to use them in the reverse order: :7:6:5:4:3:2:1

In our lookaheads, this allows us to get down to the bottom of the input with a simple .*, and to start backtracking from there. Since we know we're at the end of the string, we don't have to worry about the :digits being part of another section of the string. Here's how to do it.

Input:

my cat pi g
dog p ig
my pig
my cow
my mouse

:7:6:5:4:3:2:1

Search:

(?xsm)             # free-spacing mode, multi-line
(?=.*?pig)        # lookahead: if pig is not there, fail right away to save the effort
(?:               # start counter-line-skipper (lines that don't include pig)
   (?:               # skip one line that doesn't have pig
      ^              # 
      (?:(?!pig)[^\r\n])*  # zero or more chars not followed by pig
      (?:\r?\n)      # newline chars
    )   
   # Group 1 matches increasing portion of the numbers string at the bottom
   (?=             # lookahead
      .*           # get to the end of the input
      (               # start Group 1
        :\d+          # match a colon and some digits
        (?(1)\1)      # match Group 1 if set
      )               # end Group 1
   )               # end lookahead
)*+               # end counter-line-skipper: zero or more times
.*?               # match
\K                # drop match so far
pig               # match pig (this is the match!)
(?=.*(\d+)(?(1)\1))   # capture the next number to Group 2

Replace: \2

See the substitutions in the demo.

Third Solution: Balancing Groups

This solution is specific to .NET.

Search:

(?m)(?<=\A(?<c>^(?:(?!pig)[^\r\n])*(?:\r?\n))*.*?)pig(?=[^:]+(?(c)(?<-c>:\d+)*):(\d+))

Free-Spacing Version with Comments:

(?xm)                # free-spacing, multi-line
(?<=                 # lookbehind
   \A                # 
   (?<c>               # skip one line that doesn't have pig
                       # The length of Group c Captures will serve as a counter
     ^                    # beginning of line
     (?:(?!pig)[^\r\n])*  # zero or more chars not followed by pig
     (?:\r?\n)            # newline chars
   )                   # end skipper
   *                   # repeat skipper
   .*?                 # we're on the pig line: lazily match chars before pig
   )                # end lookbehind
pig                 # match pig: this is the match
(?=                 # lookahead
   [^:]+               # get to the digits
   (?(c)               # if Group c has been set
     (?<-c>:\d+)         # decrement c while we match a group of digits
     *                   # repeat: this will only repeat as long as the length of Group c captures > 0 
   )                   # end if Group c has been set
   :(\d+)              # Match the next digit group, capture the digits
)                    # end lokahead

Replace: $1


Reference

7
  • You should probably specify the solution is .NET specific, because... 1. Most default regex flavors do not support variable length lookbehind. Which led me to try to port this to PHP PCRE, which led to... 2. .NET and PHP apparently don't support backtracking in the same way regarding capturing groups. [\r\n]*?'s laziness makes the regex do one useless call to the following look ahead. I'm guessing that is not an issue in .NET since it keeps track of capturing groups value, but PHP does not. This means the line number will be 2 * <correct> - 2 in PHP.
    – Robin
    Commented May 29, 2014 at 10:20
  • @Robin I hear you, but he wanted to do it in an editor, and I did specify that we need an editor that allows infinite-width lookbehinds To my knowledge restricts us to .NET, the Matthew Garnett regex lib for Python, or JGSoft... But in an editor, the only one I know is EditPad Pro, which uses JGSoft not .NET :)
    – zx81
    Commented May 29, 2014 at 10:25
  • And that is very fine by me :) However the wrong result (which quite surprised me) that PHP outputs is not linked to the variable look behind feature, but to subtle differences in implementation between PHP and .NET when handling already matched capturing group in case of backtracking: this is much more specific (and hard to check) than just allowing variable length lookbehind
    – Robin
    Commented May 29, 2014 at 10:34
  • 2
    That's a nice trick/thought-exercise, though! If you want to broaden the spectrum I believe using (?m)\A((?:^.*?(?:(?=pig)|$)[\r\n]*(?=[^:]+(\2?+:(\d+))))+)pig (so, removing the laziness) and replacing with \1\3 would be more "Text editor friendly" :) Don't think OP cares anymore though.
    – Robin
    Commented May 29, 2014 at 10:46
  • 1
    Really nice tricks, but this answer definitely needs a bigger "don't try this at home" warning :) Commented Jul 20, 2014 at 12:58
2

Because you didn't specify which text editor, in vim it would be:

:%s/searched_word/\=printf('%-4d', line('.'))/g (read more)

But as somebody mentioned it's not a question for SO but rather Super User ;)

1

I don't know of an editor that does that short of extending an editor that allows arbitrary extensions.

You could easily use perl to do the task, though.

perl -i.bak -e"s/word/$./eg" file

Or if you want to use wildcards,

perl -MFile::DosGlob=glob -i.bak -e"BEGIN { @ARGV = map glob($_), @ARGV } s/word/$./eg" *.txt
4
  • @Miller, Cause the OP is on Windows
    – ikegami
    Commented May 19, 2014 at 2:50
  • 2 Questions. Why do you suspect/know the OP's on windows, is it based off of previous questions? And if the OP is, are there any features of File::DosGlob that you consider it likely that they would need? I've never needed anything beyond *.ext which normal glob will handle just fine, but then I suspect I don't know what the additional features of dos glob are.
    – Miller
    Commented May 19, 2014 at 3:08
  • @Miller, Yes, the OP posted two related questions in a row, and the other concerned UltraEdit. // "Normal" glob are very different than Windows globs. It would fail miserably with common expressions. There's nothing DosGlob glob can do that the builtin one can't. It just does it differently.
    – ikegami
    Commented May 19, 2014 at 3:23
  • To give the OP another option (without overwriting the original file) the code ~$ perl -pe 's/word/$./g;' file > tmp works in my hands. I'm guessing these answers become more important as WSL usage becomes more prevalent: learn.microsoft.com/en-us/windows/wsl/about Commented Sep 10, 2023 at 7:21
0

Using Raku (formerly known as Perl_6)

To replace an entire line where a target word is found with linenumber:

~$ raku -ne 'state $i; ++$i; put m/word/ ?? $i !! $_;'  file

To replace every instance of a target word with linenumber (global replace):

~$ raku -pe 'state $i; ++$i; s:g/word/{$i}/;' file

This answer is intended to complement the excellent Perl answer posted by @ikegami. Raku, like Perl, is cross-platform. Above is an answer for Unix/Linux systems. Use double-quotes instead of single-quotes for Windows (although per @ikegami, WSL uses single-quotes. Thank you!).

The first code example is read as follows: Using the -ne non-autoprinting linewise flags, state a counter variable $i. Increment the variable using ++i. Using Raku's ternary operator Test ?? True !! False, output (i.e. put) the incremented $i variable if a match is found to word, otherwise output $_ the original line.

The second code example is read as follows: Using the -pe autoprinting linewise flags, state a counter variable $i. Increment the variable using ++i. Using Raku's s:g/// global substitution operator replace every of match of word with the $i counter.

Sample Input:

my cat
dog
my pig
my cow
my mouse
my pig also

Sample Output replacing pig globally (second code example above):

my cat
dog
my 3
my cow
my mouse
my 6 also

Note: post-incrementation can be done with $++, which will 0-index the lines instead of 1-indexing. The Regex matcher can actually be written / … /, i.e. without the m if using forward-slashes, or even m{ … }, if you're trying to match forward-slashes within the Regex.

Additionally, you can add a number of Regex "adverbs" other than :g global to the m/ … / or s/// matchers, the most useful of which might be :i for case-insensitive match, as follows:

m:i/ … /; or s:i:g/…/…/;

More info on Regex adverbs at bottom.


Finally, if you have reasonably fluency with Vim you can open the file at your shell command line with ~$ vim file and then move to command-line mode with : colon. Once on the Vim command-line type %! raku -pe 'state $i; ++$i; s:g/pig/{$i}/;' to run the Raku command over the file. Save to a new file or over-write the original at your discretion.

https://docs.raku.org/language/regexes
https://raku.org

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.