0

I want to see how many characters the regex has looked at before it succeeds or fails (to support incrementally re-lexing something). I haven't noticed a direct way to do this in the Rust regex library's API.

If I could somehow intercept the &str I give to library, I could record the furthest character it's had to observe. I'm not sure how to do this, as I am new to Rust. If this were Java, I would use a proxy/implement CharSequence.

Bonus points if it works with RegexSet.

  • Please define what you mean by character. There are many possible meanings. – Shepmaster Jan 2 at 19:35
5

This looks like an instance of the XY problem. Namely, it sounds like what you're asking is whether the regex crate supports incremental searching on streams. More specifically, whether you can use the regex crate to search text that does not necessarily live contiguously in memory.

In any case, the answer to both questions is: you can't, in general.

If you're curious about the complexities of supporting search on streams, issue 425 tracks that feature.

  • I'm not really asking whether the regex crate supports incremental searching on streams - what really want is how far the regex matcher has had to search. The suggestion of intercepting &str is just one (possibly infeasible) solution for how to make that work, I'm open to other suggestions. At the limit, I suspect this could be done by forking the library myself and keeping track of the furthest char, but that sounds like a lot of work. – Callum Rogers Jan 2 at 19:27
  • 1
    "how far the regex matcher has had to search" - The regex matcher might just decide to start from the end of the string. – Sebastian Redl Jan 2 at 19:40
  • @SebastianRedl: Honestly, that's fine/what I want for my usecase (but I suspect for most regexes that don't have a $ in them it won't do that) – Callum Rogers Jan 2 at 19:50
  • @CallumRogers This still seems to be an instance of the XY problem. What are you planning on doing with this information? Which characters looked at depends heavily on the regex engine implementation. For instance, some parts of the regex engine use SIMD instructions to match multiple bytes at a time with a single instruction. This may look further ahead in the string than you were expecting. Or it could do reverse search. And it can't be intercepted; the regex engine matches on raw byte sequences directly. With more information on your eventual goal, we may be able to provide an alternative. – Brian Campbell Jan 2 at 20:30
  • 1
    Input is only used in the slower engines, so the result will not be fast, although you might consider it fast enough. The approach of forking and modifying the slower engines is exactly what I suggested in the linked issue. Regardless, your question is answered. You cannot do what you want with the regex crate as of today. – BurntSushi5 Jan 2 at 21:19

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.