In a text editor, I want to replace a given word with the number of the line number on which this word is found. Is this is possible with Regex?
-
Any text editor? Some of them don't support regexes at all.– chorobaCommented May 18, 2014 at 22:26
-
Short answer: No. Regular expressions don't replace or do anything else. They simply define a pattern– ikegamiCommented May 18, 2014 at 22:27
-
5StackOverflow is for programming questions. Questions about how to use your editor belong on Super User.– ikegamiCommented May 18, 2014 at 22:29
-
Do you mean a text editor like Vim?– jubilatious1Commented Sep 8, 2023 at 21:18
4 Answers
Recursion, Self-Referencing Group (Qtax trick), Reverse Qtax or Balancing Groups
Introduction
The idea of adding a list of integers to the bottom of the input is similar to a famous database hack (nothing to do with regex) where one joins to a table of integers. My original answer used the @Qtax trick. The current answers use either Recursion, the Qtax trick (straight or in a reversed variation), or Balancing Groups.
Yes, it is possible... With some caveats and regex trickery.
- The solutions in this answer are meant as a vehicle to demonstrate some regex syntax more than practical answers to be implemented.
- At the end of your file, we will paste a list of numbers preceded with a unique delimiter. For this experiment, the appended string is
:1:2:3:4:5:6:7
This is a similar technique to a famous database hack that uses a table of integers. - For the first two solutions, we need an editor that uses a regex flavor that allows recursion (solution 1) or self-referencing capture groups (solutions 2 and 3). Two come to mind: Notepad++ and EditPad Pro. For the third solution, we need an editor that supports balancing groups. That probably limits us to EditPad Pro or Visual Studio 2013+.
Input file:
Let's say we are searching for pig
and want to replace it with the line number.
We'll use this as input:
my cat
dog
my pig
my cow
my mouse
:1:2:3:4:5:6:7
First Solution: Recursion
Supported languages: Apart from the text editors mentioned above (Notepad++ and EditPad Pro), this solution should work in languages that use PCRE (PHP, R, Delphi), in Perl, and in Python using Matthew Barnett's regex
module (untested).
The recursive structure lives in a lookahead, and is optional. Its job is to balance lines that don't contain pig
, on the left, with numbers, on the right: think of it as balancing a nested construct like {{{ }}}
... Except that on the left we have the no-match lines, and on the right we have the numbers. The point is that when we exit the lookahead, we know how many lines were skipped.
Search:
(?sm)(?=.*?pig)(?=((?:^(?:(?!pig)[^\r\n])*(?:\r?\n))(?:(?1)|[^:]+)(:\d+))?).*?\Kpig(?=.*?(?(2)\2):(\d+))
Free-Spacing Version with Comments:
(?xsm) # free-spacing mode, multi-line
(?=.*?pig) # fail right away if pig isn't there
(?= # The Recursive Structure Lives In This Lookahead
( # Group 1
(?: # skip one line
^
(?:(?!pig)[^\r\n])* # zero or more chars not followed by pig
(?:\r?\n) # newline chars
)
(?:(?1)|[^:]+) # recurse Group 1 OR match all chars that are not a :
(:\d+) # match digits
)? # End Group
) # End lookahead.
.*?\Kpig # get to pig
(?=.*?(?(2)\2):(\d+)) # Lookahead: capture the next digits
Replace: \3
In the demo, see the substitutions at the bottom. You can play with the letters on the first two lines (delete a space to make pig
) to move the first occurrence of pig
to a different line, and see how that affects the results.
Second Solution: Group that Refers to Itself ("Qtax Trick")
Supported languages: Apart from the text editors mentioned above (Notepad++ and EditPad Pro), this solution should work in languages that use PCRE (PHP, R, Delphi), in Perl, and in Python using Matthew Barnett's regex
module (untested). The solution is easy to adapt to .NET by converting the \K
to a lookahead and the possessive quantifier to an atomic group (see the .NET Version a few lines below.)
Search:
(?sm)(?=.*?pig)(?:(?:^(?:(?!pig)[^\r\n])*(?:\r?\n))(?=[^:]+((?(1)\1):\d+)))*+.*?\Kpig(?=[^:]+(?(1)\1):(\d+))
.NET version: Back to the Future
.NET does not have \K
. It its place, we use a "back to the future" lookbehind (a lookbehind that contains a lookahead that skips ahead of the match). Also, we need to use an atomic group instead of a possessive quantifier.
(?sm)(?<=(?=.*?pig)(?=(?>(?:^(?:(?!pig)[^\r\n])*(?:\r?\n))(?=[^:]+((?(1)\1):\d+)))*).*)pig(?=[^:]+(?(1)\1):(\d+))
Free-Spacing Version with Comments (Perl / PCRE Version):
(?xsm) # free-spacing mode, multi-line
(?=.*?pig) # lookahead: if pig is not there, fail right away to save the effort
(?: # start counter-line-skipper (lines that don't include pig)
(?: # skip one line
^ #
(?:(?!pig)[^\r\n])* # zero or more chars not followed by pig
(?:\r?\n) # newline chars
)
# for each line skipped, let Group 1 match an ever increasing portion of the numbers string at the bottom
(?= # lookahead
[^:]+ # skip all chars that are not colons
( # start Group 1
(?(1)\1) # match Group 1 if set
:\d+ # match a colon and some digits
) # end Group 1
) # end lookahead
)*+ # end counter-line-skipper: zero or more times
.*? # match
\K # drop everything we've matched so far
pig # match pig (this is the match!)
(?=[^:]+(?(1)\1):(\d+)) # capture the next number to Group 2
Replace:
\2
Output:
my cat
dog
my 3
my cow
my mouse
:1:2:3:4:5:6:7
In the demo, see the substitutions at the bottom. You can play with the letters on the first two lines (delete a space to make pig
) to move the first occurrence of pig
to a different line, and see how that affects the results.
Choice of Delimiter for Digits
In our example, the delimiter :
for the string of digits is rather common, and could happen elsewhere. We can invent a UNIQUE_DELIMITER
and tweak the expression slightly. But the following optimization is even more efficient and lets us keep the :
Optimization on Second Solution: Reverse String of Digits
Instead of pasting our digits in order, it may be to our benefit to use them in the reverse order: :7:6:5:4:3:2:1
In our lookaheads, this allows us to get down to the bottom of the input with a simple .*
, and to start backtracking from there. Since we know we're at the end of the string, we don't have to worry about the :digits
being part of another section of the string. Here's how to do it.
Input:
my cat pi g
dog p ig
my pig
my cow
my mouse
:7:6:5:4:3:2:1
Search:
(?xsm) # free-spacing mode, multi-line
(?=.*?pig) # lookahead: if pig is not there, fail right away to save the effort
(?: # start counter-line-skipper (lines that don't include pig)
(?: # skip one line that doesn't have pig
^ #
(?:(?!pig)[^\r\n])* # zero or more chars not followed by pig
(?:\r?\n) # newline chars
)
# Group 1 matches increasing portion of the numbers string at the bottom
(?= # lookahead
.* # get to the end of the input
( # start Group 1
:\d+ # match a colon and some digits
(?(1)\1) # match Group 1 if set
) # end Group 1
) # end lookahead
)*+ # end counter-line-skipper: zero or more times
.*? # match
\K # drop match so far
pig # match pig (this is the match!)
(?=.*(\d+)(?(1)\1)) # capture the next number to Group 2
Replace: \2
See the substitutions in the demo.
Third Solution: Balancing Groups
This solution is specific to .NET.
Search:
(?m)(?<=\A(?<c>^(?:(?!pig)[^\r\n])*(?:\r?\n))*.*?)pig(?=[^:]+(?(c)(?<-c>:\d+)*):(\d+))
Free-Spacing Version with Comments:
(?xm) # free-spacing, multi-line
(?<= # lookbehind
\A #
(?<c> # skip one line that doesn't have pig
# The length of Group c Captures will serve as a counter
^ # beginning of line
(?:(?!pig)[^\r\n])* # zero or more chars not followed by pig
(?:\r?\n) # newline chars
) # end skipper
* # repeat skipper
.*? # we're on the pig line: lazily match chars before pig
) # end lookbehind
pig # match pig: this is the match
(?= # lookahead
[^:]+ # get to the digits
(?(c) # if Group c has been set
(?<-c>:\d+) # decrement c while we match a group of digits
* # repeat: this will only repeat as long as the length of Group c captures > 0
) # end if Group c has been set
:(\d+) # Match the next digit group, capture the digits
) # end lokahead
Replace: $1
Reference
-
You should probably specify the solution is .NET specific, because... 1. Most default regex flavors do not support variable length lookbehind. Which led me to try to port this to PHP PCRE, which led to... 2. .NET and PHP apparently don't support backtracking in the same way regarding capturing groups.
[\r\n]*?
's laziness makes the regex do one useless call to the following look ahead. I'm guessing that is not an issue in .NET since it keeps track of capturing groups value, but PHP does not. This means the line number will be2 * <correct> - 2
in PHP.– RobinCommented May 29, 2014 at 10:20 -
@Robin I hear you, but he wanted to do it in an editor, and I did specify that
we need an editor that allows infinite-width lookbehinds
To my knowledge restricts us to .NET, the Matthew Garnettregex
lib for Python, or JGSoft... But in an editor, the only one I know is EditPad Pro, which uses JGSoft not .NET :)– zx81Commented May 29, 2014 at 10:25 -
And that is very fine by me :) However the wrong result (which quite surprised me) that PHP outputs is not linked to the variable look behind feature, but to subtle differences in implementation between PHP and .NET when handling already matched capturing group in case of backtracking: this is much more specific (and hard to check) than just allowing variable length lookbehind– RobinCommented May 29, 2014 at 10:34
-
2That's a nice trick/thought-exercise, though! If you want to broaden the spectrum I believe using
(?m)\A((?:^.*?(?:(?=pig)|$)[\r\n]*(?=[^:]+(\2?+:(\d+))))+)pig
(so, removing the laziness) and replacing with\1\3
would be more "Text editor friendly" :) Don't think OP cares anymore though.– RobinCommented May 29, 2014 at 10:46 -
1Really nice tricks, but this answer definitely needs a bigger "don't try this at home" warning :) Commented Jul 20, 2014 at 12:58
Because you didn't specify which text editor, in vim it would be:
:%s/searched_word/\=printf('%-4d', line('.'))/g
(read more)
But as somebody mentioned it's not a question for SO but rather Super User ;)
I don't know of an editor that does that short of extending an editor that allows arbitrary extensions.
You could easily use perl
to do the task, though.
perl -i.bak -e"s/word/$./eg" file
Or if you want to use wildcards,
perl -MFile::DosGlob=glob -i.bak -e"BEGIN { @ARGV = map glob($_), @ARGV } s/word/$./eg" *.txt
-
-
2 Questions. Why do you suspect/know the OP's on windows, is it based off of previous questions? And if the OP is, are there any features of
File::DosGlob
that you consider it likely that they would need? I've never needed anything beyond*.ext
which normalglob
will handle just fine, but then I suspect I don't know what the additional features of dos glob are.– MillerCommented May 19, 2014 at 3:08 -
@Miller, Yes, the OP posted two related questions in a row, and the other concerned UltraEdit. // "Normal"
glob
are very different than Windows globs. It would fail miserably with common expressions. There's nothing DosGlobglob
can do that the builtin one can't. It just does it differently.– ikegamiCommented May 19, 2014 at 3:23 -
To give the OP another option (without overwriting the original file) the code
~$ perl -pe 's/word/$./g;' file > tmp
works in my hands. I'm guessing these answers become more important as WSL usage becomes more prevalent: learn.microsoft.com/en-us/windows/wsl/about Commented Sep 10, 2023 at 7:21
Using Raku (formerly known as Perl_6)
To replace an entire line where a target word
is found with linenumber:
~$ raku -ne 'state $i; ++$i; put m/word/ ?? $i !! $_;' file
To replace every instance of a target word
with linenumber (global replace):
~$ raku -pe 'state $i; ++$i; s:g/word/{$i}/;' file
This answer is intended to complement the excellent Perl answer posted by @ikegami. Raku, like Perl, is cross-platform. Above is an answer for Unix/Linux systems. Use double-quotes instead of single-quotes for Windows (although per @ikegami, WSL uses single-quotes. Thank you!).
The first code example is read as follows: Using the -ne
non-autoprinting linewise flags, state
a counter variable $i
. Increment the variable using ++i
. Using Raku's ternary operator Test ??
True !!
False, output (i.e. put
) the incremented $i
variable if a match is found to word
, otherwise output $_
the original line.
The second code example is read as follows: Using the -pe
autoprinting linewise flags, state
a counter variable $i
. Increment the variable using ++i
. Using Raku's s:g///
global substitution operator replace every of match of word
with the $i
counter.
Sample Input:
my cat
dog
my pig
my cow
my mouse
my pig also
Sample Output replacing pig
globally (second code example above):
my cat
dog
my 3
my cow
my mouse
my 6 also
Note: post-incrementation can be done with $++
, which will 0-index the lines instead of 1-indexing. The Regex matcher can actually be written / … /
, i.e. without the m
if using forward-slashes, or even m{ … }
, if you're trying to match forward-slashes within the Regex.
Additionally, you can add a number of Regex "adverbs" other than :g
global to the m/ … /
or s///
matchers, the most useful of which might be :i
for case-insensitive match, as follows:
m:i/ … /;
or s:i:g/…/…/;
More info on Regex adverbs at bottom.
Finally, if you have reasonably fluency with Vim you can open the file at your shell command line with ~$ vim file
and then move to command-line mode with :
colon. Once on the Vim command-line type %! raku -pe 'state $i; ++$i; s:g/pig/{$i}/;'
to run the Raku command over the file. Save to a new file or over-write the original at your discretion.