5

I think this is a stupid question, but I'm just starting out with ANTLR. I put together the "SimpleCalc" grammar from their tutorials, and generated it with C as the target language. I got SimpleCalcParser.c/.h and SimpleCalcLexer.c/.h as the output, and I was able to compile these and build successfuly. But now, how do I actually use the code that's generated? I'm having trouble finding anything in the docs that's helpful.

Below is my main() function. This is also from the tutorial.

 #include "SimpleCalcLexer.h"

 int main(int argc, char * argv[])
 {

    pANTLR3_INPUT_STREAM           input;
    pSimpleCalcLexer               lex;
    pANTLR3_COMMON_TOKEN_STREAM    tokens;
    pSimpleCalcParser              parser;

    input  = antlr3AsciiFileStreamNew          ((pANTLR3_UINT8)argv[1]);
    lex    = SimpleCalcLexerNew                (input);
    tokens = antlr3CommonTokenStreamSourceNew  (ANTLR3_SIZE_HINT, TOKENSOURCE(lex));
    parser = SimpleCalcParserNew               (tokens);

    parser  ->expr(parser);

    // Must manually clean up
    //
    parser ->free(parser);
    tokens ->free(tokens);
    lex    ->free(lex);
    input  ->close(input);

    return 0;
 }

EDIT: Per the first response, I should say that I ran the program like this: "./testantlr test.txt", where test.txt contained "4+1". There was no output.

From here, how would I, for example, access the "4" in the generated syntax tree, or print out the entire syntax tree? Basically, how do I access stuff in the syntax tree that ANTLR generates?

4 Answers 4

4

I faced the same perplexment when I first took a crack at it. It's a pretty obvious question/issue, which makes it more weird that it doesn't seem to be explicitly and straightforwardly addressed in tutorials.

The way out of the perplexment that I found is the 'returns' keyword:

token returns [TreeNode value]
    :    WORD { $value = new TreeNode( "word", $WORD.Text ); }
    |    INT { $value = new TreeNode( "int", $INT.Text ); }
    ;

WORD:    ('a'..'z'|'A'..'Z')+;
INT :    ('0'..'9')+;

TreeNode is a class that I made. Where it got tricky was how to do this with a sequence of say, multiple tokens. The solution I came up with was recursion:

expr returns [Accumulator value]
    :   a=token  (WS+ b=expr)?
    {
        if( b != null )
        {
            $value = new Accumulator( "expr", a.value, b.value );
        } else
        {
            $value = new Accumulator( "expr", a.value );
        }
    }
    ;

Accumulator is a class that I made that has two different constructors. One constructor encapsulates a single token, and the other encapsulates a single token and another Accumulator instance. Notice the rule itself is defined recursively, and that b.value is an Accumulator instance. Why? Because b is an expr, and the definition of expr has returns [Accumulator value].

The final resulting tree is a single Accumulator instance that has grouped up all the tokens. To actually use that tree, you do some setup and then call the method that has the same name as the rule with respect to which you're parsing your content:

Antlr.Runtime.ANTLRStringStream stringstream =  new Antlr.Runtime.ANTLRStringStream( script );
TokenLexer lexer = new TokenLexer( stringstream );
Antlr.Runtime.CommonTokenStream tokenstream = new Antlr.Runtime.CommonTokenStream( lexer );
TokenParser parser = new TokenParser( tokenstream );

Accumulator grandtree = parser.expr().value;

Hope this helps people who encounter this perplexion.


Update

There's a more straightforward way to collect items into lists, due to how the system allows you to intersperse target-language code at what appears to be arbitrary pattern locations. The idiom is:

sequence returns [String k]
    :   (e=atom { $k = $e.k; })
        (e=atom { $k += ", " + $e.k; })*
        { $k = "sequence (" + $k + ")"; } ;

A string k gets initialized to the k value of the first atom, and subsequent atoms get += to k. The snippet $e.k is referring to a atom returns [String k] rule defined elsewhere. If there isn't such a rule, you can use the text property (i.e. $e.text, which tokens have. I'm not sure if non-tokens have this property. If not, you can just do:

nonToken returns [String whatever] : e=TOKEN { $whatever = $e.text; } ;

Which you would then use in higher rules by e.g.

e=nonToken { System.out.println($e.whatever); }
0

So when you run this program you gave here the first command line argument will be the name of the file to parse.

Step one, try that (run it and give it a file).

Step two, come back and edit your question but change the direction slightly. Instead of asking "how do I use the code" try asking "how do I do __________ with this" where the blank is replaced by some description of what you are trying to accomplish.

So parser->expr(parser) appears to be parsing the token stream coming from your file, which should produce a AST. Guessing my way through lots of details I'd suggest looking at what it returns, esp. the value member if it has one. There seem to be a slew of tutorials on line that look similar to what you are doing, no two identical.

If all else fails, continue on with the tutorial and either 1) it will answer your questions or 2) it won't, and you can try another.

0

Take a look at Scott Stanchfield's part 8 video http://vimeo.com/groups/29150/videos/8377479. He's doing it in Java, but same principle can be applied in C(++).

-1

I don't mean to be rude, but it seems like you don't really know what the purpose of ANTLR is. I think you need to understand this before you can attempt to use the files it generates.

The very short answer is that ANTLR is a parser generator, which means that it generates code to parse text. A common usage is to parse the text of a programming language. I haven't read the tutorial you're referring to, but I'd guess the text this parser parses is a set of calculator instructions, something like

ADD 2 4
MULTIPLY 4 8

In order to use the program you've shown above, you would execute it like any other C program. The first argument (argc) should be the number of arguments, and the second (argv) should be the text to be parsed.

In order to learn ANTLR from the ground up, I recommend you read the book published by Terence Parr, the author of ANTLR.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.