4

I have run into headaches trying to coerce a grammar to match the last line of a file if it is not followed by a newline:

Line 1
Line 2 EOF

This attempted solution, which makes the newline optional, causes an infinite loop:

my grammar HC4 {
    token TOP {  <line>+ }
    token line { [ <header> | <not-header> ] \n? } # optional newline

    token header { <header-start> <header-content> }
    token not-header { <not-header-content> }
    token header-start { \s* '#' ** 1..6 }
    token header-content { \N* }
    token not-header-content { \N* }
}

The \N* bits will match the '' string after the last character in the last line forever.

I have tried using <[\n\Z]> but then the compiler complains and suggests using \n?$ which I tried but that does not work either. After a lot of trial and error, the only solution I discovered that works requires me to create a new <blank> capture and to change the \N* to \N+:

my grammar HC3 {
    token TOP {  <line>+ }
    token line { [ <header> | <blank> | <not-header> ] \n? }

    token header { <header-start> <header-content> }
    token blank { \h* <?[\n]> }
    token not-header { <not-header-content> }
    token header-start { \s* '#' ** 1..6 }
    token header-content { \N+ }
    token not-header-content { \N+ }
}

I'd like to know if there is a more straightforward accomplishing this, though. Thanks.

6
  • 1
    Perhaps this is helpful: docs.raku.org/language/regexes#Summary_of_anchors Mar 27, 2022 at 8:54
  • Thanks. I have tried various combinations of $ and $$ but I still ran into the same problems with \N* matching nothing and causing infinite loops.
    – StevieD
    Mar 27, 2022 at 9:17
  • Ooh, wait. May have something. Will post an answer.
    – StevieD
    Mar 27, 2022 at 9:20
  • 1
    May be relevant: unix.stackexchange.com/questions/263869/… Mar 28, 2022 at 18:15
  • 1
    @jubilatious1 I found the issue. So IntelliJ/CommaIDE was the culprit. vim was saving the files just fine but when you save a file with intellij with no blank line showing at the end of the file, it does not add in the newline as it should. I confirmed this by dumping out the hex code of the file.
    – StevieD
    Mar 29, 2022 at 19:08

3 Answers 3

3

I think I may have found something that can work and is simple:

my grammar G {
    token TOP {  (^^ <line>)+ }
    token line { \N* \n? }
}

The ^^ symbol, for the beginning of a line, stops the infinite loop.

2

OK, after some investigation, I discovered the root cause of my woes:

enter image description here

This screenshot is from the IntelliJ IDE's Editor -> General settings. By default, the "Ensure every saved file ends with a line break" is not checked off. So if I saved a file with the very last line deleted to clean it up, it was stripping the last \n character. Check that setting on to avoid my pain, suffering and deep psychological trauma.

1

I believe the simplest solution is something like this:

grammar LineOriented {
    token TOP {
        <line>* %% \n
    }

    token line {
        ^^ \N*
    }
}

Using %% allows, but not requires, the last trailing line.

Your Answer

Reminder: Answers generated by Artificial Intelligence tools are not allowed on Stack Overflow. Learn more

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.