2

I'm trying to add a support for UnrealScript language (OOP, with C-style syntax) for IntelliJ Idea according to this tutorial. However, I'm having some difficulties with grammar and lexer generation.

Specifically, what's the purpose of :

  • extends(".*expr")=expr in this .bnf file (line 5)?
  • name(".*expr")='expression' in this sample.bnf file at the end of the article?

I'm adapting classical C syntax in the .bnf file so that once I have a working C-style grammar, I can tune it according to the aforementioned language. Here is what I've done so far. I'm adapting it from this page, which makes heavy use of left-recursion, which is unsupported by the generator. Therefore how can one convert, for instance, the following rule to non-recursive?

logical-or-expression ::= logical-and-expression
                      | logical-or-expression '||' logical-and-expression

What about this slightly more sophisticated one?

direct-abstract-declarator ::=  '(' abstract-declarator ')'
                           | {direct-abstract-declarator}? '[' {constant-expression}? ']'
                           | {direct-abstract-declarator}? '(' {parameter-type-list}? ')'

But apart from this, I feel an immense lack of theory of what I'm trying to accomplish, so more generally, what are the theoretical prerequisites for adding language support in IntelliJ Platform? So far, I've got myself acquainted only with general bnf/ebnf notation. Do I need to read the dragon book? If yes, then which chapters should I focus on (since I do not intend to write a compiler)?

1
  • Hello! Have you managed to implement US support? If yes, will you be willing to share your implementation?
    – Xymanek
    Sep 12, 2018 at 17:57

1 Answer 1

4

The extends(".*expr")=expr line changes how the AST classes are generated, making the AST more shallow, which in turn makes the PSI hierarchy nicer. Essentially, it means that any rules that match the regular expression .*expr will be treated as extending the expr rule, rather than being children.

This means that e.g. the types generated for add_expr or mul_expr would extend the Expr interface, so you'd be able to use AddExpr and MulExpr wherever you can use Expr. Without this, Expr would have getter methods for AddExpr and `MulExpr, etc. Check out this section of the HOWTO.

The name(".*expr")='expression' line means that "<expression>" will be used to describe any rule that matches the regular expression. This is mostly surfaced in error messages - "<expression> expected, got XX". This is useful to group all expression rules together with a name, rather than list every variation.

Unfortunately, I can't give much advice on your other questions. GrammarKit does provide support some left recursion, as described in this section on parsing expressions, but I don't really know details. Other than that, it is possible to rewrite left recursive rules, and it is best to search for examples of how to do this.

Similarly, I'd suggest looking for tutorials on writing parsers, and reading as much as possible. There is little written about how to write a parser for an IDE, but the big difference between writing a parser for a compiler and an IDE is error handling. An IDE needs to assume that your code is always broken (it usually is, when you're typing) and needs to recover gracefully. Similarly, the design of your AST is important, but there is no guidance available for this. It essentially boils down to your requirements - be prepared to tweak your grammar and AST structure to suit the features you're building.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.