mirror of https://github.com/yasm/yasm.git
svn path=/trunk/yasm/; revision=6950.3
parent
320a83294e
commit
c0eee0f5b8
6 changed files with 0 additions and 5442 deletions
@ -1 +0,0 @@ |
||||
lemon |
@ -1,12 +0,0 @@ |
||||
# $IdPath$
|
||||
CFLAGS = @ANSI_CFLAGS@
|
||||
|
||||
noinst_PROGRAMS = lemon
|
||||
|
||||
lemon_SOURCES = \
|
||||
lemon.c
|
||||
|
||||
EXTRA_DIST = \
|
||||
lemon.html \
|
||||
lempar.c \
|
||||
README
|
@ -1,14 +0,0 @@ |
||||
$IdPath$ |
||||
$Id: README,v 1.3 2002/04/12 04:12:11 peter Exp $ |
||||
|
||||
The Lemon Parser Generator's home page is: |
||||
|
||||
http://www.hwaci.com/sw/lemon/index.html |
||||
|
||||
The file in this directory, lemon.html, was obtained from: |
||||
|
||||
http://www.hwaci.com/sw/lemon/lemon.html |
||||
|
||||
lemon.c has been modified to include the t= and o= command-line |
||||
arguments. These changes are thanks to the FreeBSD project ports |
||||
collection. |
File diff suppressed because it is too large
Load Diff
@ -1,861 +0,0 @@ |
||||
<html> |
||||
<head> |
||||
<title>The Lemon Parser Generator</title> |
||||
</head> |
||||
<body bgcolor=white> |
||||
<h1 align=center>The Lemon Parser Generator</h1> |
||||
|
||||
<p>Lemon is an LALR(1) parser generator for C or C++. |
||||
It does the same job as ``bison'' and ``yacc''. |
||||
But lemon is not another bison or yacc clone. It |
||||
uses a different grammar syntax which is designed to |
||||
reduce the number of coding errors. Lemon also uses a more |
||||
sophisticated parsing engine that is faster than yacc and |
||||
bison and which is both reentrant and thread-safe. |
||||
Furthermore, Lemon implements features that can be used |
||||
to eliminate resource leaks, making is suitable for use |
||||
in long-running programs such as graphical user interfaces |
||||
or embedded controllers.</p> |
||||
|
||||
<p>This document is an introduction to the Lemon |
||||
parser generator.</p> |
||||
|
||||
<h2>Theory of Operation</h2> |
||||
|
||||
<p>The main goal of Lemon is to translate a context free grammar (CFG) |
||||
for a particular language into C code that implements a parser for |
||||
that language. |
||||
The program has two inputs: |
||||
<ul> |
||||
<li>The grammar specification. |
||||
<li>A parser template file. |
||||
</ul> |
||||
Typically, only the grammar specification is supplied by the programmer. |
||||
Lemon comes with a default parser template which works fine for most |
||||
applications. But the user is free to substitute a different parser |
||||
template if desired.</p> |
||||
|
||||
<p>Depending on command-line options, Lemon will generate between |
||||
one and three files of outputs. |
||||
<ul> |
||||
<li>C code to implement the parser. |
||||
<li>A header file defining an integer ID for each terminal symbol. |
||||
<li>An information file that describes the states of the generated parser |
||||
automaton. |
||||
</ul> |
||||
By default, all three of these output files are generated. |
||||
The header file is suppressed if the ``-m'' command-line option is |
||||
used and the report file is omitted when ``-q'' is selected.</p> |
||||
|
||||
<p>The grammar specification file uses a ``.y'' suffix, by convention. |
||||
In the examples used in this document, we'll assume the name of the |
||||
grammar file is ``gram.y''. A typical use of Lemon would be the |
||||
following command: |
||||
<pre> |
||||
lemon gram.y |
||||
</pre> |
||||
This command will generate three output files named ``gram.c'', |
||||
``gram.h'' and ``gram.out''. |
||||
The first is C code to implement the parser. The second |
||||
is the header file that defines numerical values for all |
||||
terminal symbols, and the last is the report that explains |
||||
the states used by the parser automaton.</p> |
||||
|
||||
<h3>Command Line Options</h3> |
||||
|
||||
<p>The behavior of Lemon can be modified using command-line options. |
||||
You can obtain a list of the available command-line options together |
||||
with a brief explanation of what each does by typing |
||||
<pre> |
||||
lemon -? |
||||
</pre> |
||||
As of this writing, the following command-line options are supported: |
||||
<ul> |
||||
<li><tt>-b</tt> |
||||
<li><tt>-c</tt> |
||||
<li><tt>-g</tt> |
||||
<li><tt>-m</tt> |
||||
<li><tt>-q</tt> |
||||
<li><tt>-s</tt> |
||||
<li><tt>-x</tt> |
||||
</ul> |
||||
The ``-b'' option reduces the amount of text in the report file by |
||||
printing only the basis of each parser state, rather than the full |
||||
configuration. |
||||
The ``-c'' option suppresses action table compression. Using -c |
||||
will make the parser a little larger and slower but it will detect |
||||
syntax errors sooner. |
||||
The ``-g'' option causes no output files to be generated at all. |
||||
Instead, the input grammar file is printed on standard output but |
||||
with all comments, actions and other extraneous text deleted. This |
||||
is a useful way to get a quick summary of a grammar. |
||||
The ``-m'' option causes the output C source file to be compatible |
||||
with the ``makeheaders'' program. |
||||
Makeheaders is a program that automatically generates header files |
||||
from C source code. When the ``-m'' option is used, the header |
||||
file is not output since the makeheaders program will take care |
||||
of generated all header files automatically. |
||||
The ``-q'' option suppresses the report file. |
||||
Using ``-s'' causes a brief summary of parser statistics to be |
||||
printed. Like this: |
||||
<pre> |
||||
Parser statistics: 74 terminals, 70 nonterminals, 179 rules |
||||
340 states, 2026 parser table entries, 0 conflicts |
||||
</pre> |
||||
Finally, the ``-x'' option causes Lemon to print its version number |
||||
and copyright information |
||||
and then stop without attempting to read the grammar or generate a parser.</p> |
||||
|
||||
<h3>The Parser Interface</h3> |
||||
|
||||
<p>Lemon doesn't generate a complete, working program. It only generates |
||||
a few subroutines that implement a parser. This section describes |
||||
the interface to those subroutines. It is up to the programmer to |
||||
call these subroutines in an appropriate way in order to produce a |
||||
complete system.</p> |
||||
|
||||
<p>Before a program begins using a Lemon-generated parser, the program |
||||
must first create the parser. |
||||
A new parser is created as follows: |
||||
<pre> |
||||
void *pParser = ParseAlloc( malloc ); |
||||
</pre> |
||||
The ParseAlloc() routine allocates and initializes a new parser and |
||||
returns a pointer to it. |
||||
The actual data structure used to represent a parser is opaque -- |
||||
its internal structure is not visible or usable by the calling routine. |
||||
For this reason, the ParseAlloc() routine returns a pointer to void |
||||
rather than a pointer to some particular structure. |
||||
The sole argument to the ParseAlloc() routine is a pointer to the |
||||
subroutine used to allocate memory. Typically this means ``malloc()''.</p> |
||||
|
||||
<p>After a program is finished using a parser, it can reclaim all |
||||
memory allocated by that parser by calling |
||||
<pre> |
||||
ParseFree(pParser, free); |
||||
</pre> |
||||
The first argument is the same pointer returned by ParseAlloc(). The |
||||
second argument is a pointer to the function used to release bulk |
||||
memory back to the system.</p> |
||||
|
||||
<p>After a parser has been allocated using ParseAlloc(), the programmer |
||||
must supply the parser with a sequence of tokens (terminal symbols) to |
||||
be parsed. This is accomplished by calling the following function |
||||
once for each token: |
||||
<pre> |
||||
Parse(pParser, hTokenID, sTokenData, pArg); |
||||
</pre> |
||||
The first argument to the Parse() routine is the pointer returned by |
||||
ParseAlloc(). |
||||
The second argument is a small positive integer that tells the parse the |
||||
type of the next token in the data stream. |
||||
There is one token type for each terminal symbol in the grammar. |
||||
The gram.h file generated by Lemon contains #define statements that |
||||
map symbolic terminal symbol names into appropriate integer values. |
||||
(A value of 0 for the second argument is a special flag to the |
||||
parser to indicate that the end of input has been reached.) |
||||
The third argument is the value of the given token. By default, |
||||
the type of the third argument is integer, but the grammar will |
||||
usually redefine this type to be some kind of structure. |
||||
Typically the second argument will be a broad category of tokens |
||||
such as ``identifier'' or ``number'' and the third argument will |
||||
be the name of the identifier or the value of the number.</p> |
||||
|
||||
<p>The Parse() function may have either three or four arguments, |
||||
depending on the grammar. If the grammar specification file request |
||||
it, the Parse() function will have a fourth parameter that can be |
||||
of any type chosen by the programmer. The parser doesn't do anything |
||||
with this argument except to pass it through to action routines. |
||||
This is a convenient mechanism for passing state information down |
||||
to the action routines without having to use global variables.</p> |
||||
|
||||
<p>A typical use of a Lemon parser might look something like the |
||||
following: |
||||
<pre> |
||||
01 ParseTree *ParseFile(const char *zFilename){ |
||||
02 Tokenizer *pTokenizer; |
||||
03 void *pParser; |
||||
04 Token sToken; |
||||
05 int hTokenId; |
||||
06 ParserState sState; |
||||
07 |
||||
08 pTokenizer = TokenizerCreate(zFilename); |
||||
09 pParser = ParseAlloc( malloc ); |
||||
10 InitParserState(&sState); |
||||
11 while( GetNextToken(pTokenizer, &hTokenId, &sToken) ){ |
||||
12 Parse(pParser, hTokenId, sToken, &sState); |
||||
13 } |
||||
14 Parse(pParser, 0, sToken, &sState); |
||||
15 ParseFree(pParser, free ); |
||||
16 TokenizerFree(pTokenizer); |
||||
17 return sState.treeRoot; |
||||
18 } |
||||
</pre> |
||||
This example shows a user-written routine that parses a file of |
||||
text and returns a pointer to the parse tree. |
||||
(We've omitted all error-handling from this example to keep it |
||||
simple.) |
||||
We assume the existence of some kind of tokenizer which is created |
||||
using TokenizerCreate() on line 8 and deleted by TokenizerFree() |
||||
on line 16. The GetNextToken() function on line 11 retrieves the |
||||
next token from the input file and puts its type in the |
||||
integer variable hTokenId. The sToken variable is assumed to be |
||||
some kind of structure that contains details about each token, |
||||
such as its complete text, what line it occurs on, etc. </p> |
||||
|
||||
<p>This example also assumes the existence of structure of type |
||||
ParserState that holds state information about a particular parse. |
||||
An instance of such a structure is created on line 6 and initialized |
||||
on line 10. A pointer to this structure is passed into the Parse() |
||||
routine as the optional 4th argument. |
||||
The action routine specified by the grammar for the parser can use |
||||
the ParserState structure to hold whatever information is useful and |
||||
appropriate. In the example, we note that the treeRoot field of |
||||
the ParserState structure is left pointing to the root of the parse |
||||
tree.</p> |
||||
|
||||
<p>The core of this example as it relates to Lemon is as follows: |
||||
<pre> |
||||
ParseFile(){ |
||||
pParser = ParseAlloc( malloc ); |
||||
while( GetNextToken(pTokenizer,&hTokenId, &sToken) ){ |
||||
Parse(pParser, hTokenId, sToken); |
||||
} |
||||
Parse(pParser, 0, sToken); |
||||
ParseFree(pParser, free ); |
||||
} |
||||
</pre> |
||||
Basically, what a program has to do to use a Lemon-generated parser |
||||
is first create the parser, then send it lots of tokens obtained by |
||||
tokenizing an input source. When the end of input is reached, the |
||||
Parse() routine should be called one last time with a token type |
||||
of 0. This step is necessary to inform the parser that the end of |
||||
input has been reached. Finally, we reclaim memory used by the |
||||
parser by calling ParseFree().</p> |
||||
|
||||
<p>There is one other interface routine that should be mentioned |
||||
before we move on. |
||||
The ParseTrace() function can be used to generate debugging output |
||||
from the parser. A prototype for this routine is as follows: |
||||
<pre> |
||||
ParseTrace(FILE *stream, char *zPrefix); |
||||
</pre> |
||||
After this routine is called, a short (one-line) message is written |
||||
to the designated output stream every time the parser changes states |
||||
or calls an action routine. Each such message is prefaced using |
||||
the text given by zPrefix. This debugging output can be turned off |
||||
by calling ParseTrace() again with a first argument of NULL (0).</p> |
||||
|
||||
<h3>Differences With YACC and BISON</h3> |
||||
|
||||
<p>Programmers who have previously used the yacc or bison parser |
||||
generator will notice several important differences between yacc and/or |
||||
bison and Lemon. |
||||
<ul> |
||||
<li>In yacc and bison, the parser calls the tokenizer. In Lemon, |
||||
the tokenizer calls the parser. |
||||
<li>Lemon uses no global variables. Yacc and bison use global variables |
||||
to pass information between the tokenizer and parser. |
||||
<li>Lemon allows multiple parsers to be running simultaneously. Yacc |
||||
and bison do not. |
||||
</ul> |
||||
These differences may cause some initial confusion for programmers |
||||
with prior yacc and bison experience. |
||||
But after years of experience using Lemon, I firmly |
||||
believe that the Lemon way of doing things is better.</p> |
||||
|
||||
<h2>Input File Syntax</h2> |
||||
|
||||
<p>The main purpose of the grammar specification file for Lemon is |
||||
to define the grammar for the parser. But the input file also |
||||
specifies additional information Lemon requires to do its job. |
||||
Most of the work in using Lemon is in writing an appropriate |
||||
grammar file.</p> |
||||
|
||||
<p>The grammar file for lemon is, for the most part, free format. |
||||
It does not have sections or divisions like yacc or bison. Any |
||||
declaration can occur at any point in the file. |
||||
Lemon ignores whitespace (except where it is needed to separate |
||||
tokens) and it honors the same commenting conventions as C and C++.</p> |
||||
|
||||
<h3>Terminals and Nonterminals</h3> |
||||
|
||||
<p>A terminal symbol (token) is any string of alphanumeric |
||||
and underscore characters |
||||
that begins with an upper case letter. |
||||
A terminal can contain lower class letters after the first character, |
||||
but the usual convention is to make terminals all upper case. |
||||
A nonterminal, on the other hand, is any string of alphanumeric |
||||
and underscore characters than begins with a lower case letter. |
||||
Again, the usual convention is to make nonterminals use all lower |
||||
case letters.</p> |
||||
|
||||
<p>In Lemon, terminal and nonterminal symbols do not need to |
||||
be declared or identified in a separate section of the grammar file. |
||||
Lemon is able to generate a list of all terminals and nonterminals |
||||
by examining the grammar rules, and it can always distinguish a |
||||
terminal from a nonterminal by checking the case of the first |
||||
character of the name.</p> |
||||
|
||||
<p>Yacc and bison allow terminal symbols to have either alphanumeric |
||||
names or to be individual characters included in single quotes, like |
||||
this: ')' or '$'. Lemon does not allow this alternative form for |
||||
terminal symbols. With Lemon, all symbols, terminals and nonterminals, |
||||
must have alphanumeric names.</p> |
||||
|
||||
<h3>Grammar Rules</h3> |
||||
|
||||
<p>The main component of a Lemon grammar file is a sequence of grammar |
||||
rules. |
||||
Each grammar rule consists of a nonterminal symbol followed by |
||||
the special symbol ``::='' and then a list of terminals and/or nonterminals. |
||||
The rule is terminated by a period. |
||||
The list of terminals and nonterminals on the right-hand side of the |
||||
rule can be empty. |
||||
Rules can occur in any order, except that the left-hand side of the |
||||
first rule is assumed to be the start symbol for the grammar (unless |
||||
specified otherwise using the <tt>%start</tt> directive described below.) |
||||
A typical sequence of grammar rules might look something like this: |
||||
<pre> |
||||
expr ::= expr PLUS expr. |
||||
expr ::= expr TIMES expr. |
||||
expr ::= LPAREN expr RPAREN. |
||||
expr ::= VALUE. |
||||
</pre> |
||||
</p> |
||||
|
||||
<p>There is one non-terminal in this example, ``expr'', and five |
||||
terminal symbols or tokens: ``PLUS'', ``TIMES'', ``LPAREN'', |
||||
``RPAREN'' and ``VALUE''.</p> |
||||
|
||||
<p>Like yacc and bison, Lemon allows the grammar to specify a block |
||||
of C code that will be executed whenever a grammar rule is reduced |
||||
by the parser. |
||||
In Lemon, this action is specified by putting the C code (contained |
||||
within curly braces <tt>{...}</tt>) immediately after the |
||||
period that closes the rule. |
||||
For example: |
||||
<pre> |
||||
expr ::= expr PLUS expr. { printf("Doing an addition...\n"); } |
||||
</pre> |
||||
</p> |
||||
|
||||
<p>In order to be useful, grammar actions must normally be linked to |
||||
their associated grammar rules. |
||||
In yacc and bison, this is accomplished by embedding a ``$$'' in the |
||||
action to stand for the value of the left-hand side of the rule and |
||||
symbols ``$1'', ``$2'', and so forth to stand for the value of |
||||
the terminal or nonterminal at position 1, 2 and so forth on the |
||||
right-hand side of the rule. |
||||
This idea is very powerful, but it is also very error-prone. The |
||||
single most common source of errors in a yacc or bison grammar is |
||||
to miscount the number of symbols on the right-hand side of a grammar |
||||
rule and say ``$7'' when you really mean ``$8''.</p> |
||||
|
||||
<p>Lemon avoids the need to count grammar symbols by assigning symbolic |
||||
names to each symbol in a grammar rule and then using those symbolic |
||||
names in the action. |
||||
In yacc or bison, one would write this: |
||||
<pre> |
||||
expr -> expr PLUS expr { $$ = $1 + $3; }; |
||||
</pre> |
||||
But in Lemon, the same rule becomes the following: |
||||
<pre> |
||||
expr(A) ::= expr(B) PLUS expr(C). { A = B+C; } |
||||
</pre> |
||||
In the Lemon rule, any symbol in parentheses after a grammar rule |
||||
symbol becomes a place holder for that symbol in the grammar rule. |
||||
This place holder can then be used in the associated C action to |
||||
stand for the value of that symbol.<p> |
||||
|
||||
<p>The Lemon notation for linking a grammar rule with its reduce |
||||
action is superior to yacc/bison on several counts. |
||||
First, as mentioned above, the Lemon method avoids the need to |
||||
count grammar symbols. |
||||
Secondly, if a terminal or nonterminal in a Lemon grammar rule |
||||
includes a linking symbol in parentheses but that linking symbol |
||||
is not actually used in the reduce action, then an error message |
||||
is generated. |
||||
For example, the rule |
||||
<pre> |
||||
expr(A) ::= expr(B) PLUS expr(C). { A = B; } |
||||
</pre> |
||||
will generate an error because the linking symbol ``C'' is used |
||||
in the grammar rule but not in the reduce action.</p> |
||||
|
||||
<p>The Lemon notation for linking grammar rules to reduce actions |
||||
also facilitates the use of destructors for reclaiming memory |
||||
allocated by the values of terminals and nonterminals on the |
||||
right-hand side of a rule.</p> |
||||
|
||||
<h3>Precedence Rules</h3> |
||||
|
||||
<p>Lemon resolves parsing ambiguities in exactly the same way as |
||||
yacc and bison. A shift-reduce conflict is resolved in favor |
||||
of the shift, and a reduce-reduce conflict is resolved by reducing |
||||
whichever rule comes first in the grammar file.</p> |
||||
|
||||
<p>Just like in |
||||
yacc and bison, Lemon allows a measure of control |
||||
over the resolution of paring conflicts using precedence rules. |
||||
A precedence value can be assigned to any terminal symbol |
||||
using the %left, %right or %nonassoc directives. Terminal symbols |
||||
mentioned in earlier directives have a lower precedence that |
||||
terminal symbols mentioned in later directives. For example:</p> |
||||
|
||||
<p><pre> |
||||
%left AND. |
||||
%left OR. |
||||
%nonassoc EQ NE GT GE LT LE. |
||||
%left PLUS MINUS. |
||||
%left TIMES DIVIDE MOD. |
||||
%right EXP NOT. |
||||
</pre></p> |
||||
|
||||
<p>In the preceding sequence of directives, the AND operator is |
||||
defined to have the lowest precedence. The OR operator is one |
||||
precedence level higher. And so forth. Hence, the grammar would |
||||
attempt to group the ambiguous expression |
||||
<pre> |
||||
a AND b OR c |
||||
</pre> |
||||
like this |
||||
<pre> |
||||
a AND (b OR c). |
||||
</pre> |
||||
The associativity (left, right or nonassoc) is used to determine |
||||
the grouping when the precedence is the same. AND is left-associative |
||||
in our example, so |
||||
<pre> |
||||
a AND b AND c |
||||
</pre> |
||||
is parsed like this |
||||
<pre> |
||||
(a AND b) AND c. |
||||
</pre> |
||||
The EXP operator is right-associative, though, so |
||||
<pre> |
||||
a EXP b EXP c |
||||
</pre> |
||||
is parsed like this |
||||
<pre> |
||||
a EXP (b EXP c). |
||||
</pre> |
||||
The nonassoc precedence is used for non-associative operators. |
||||
So |
||||
<pre> |
||||
a EQ b EQ c |
||||
</pre> |
||||
is an error.</p> |
||||
|
||||
<p>The precedence of non-terminals is transferred to rules as follows: |
||||
The precedence of a grammar rule is equal to the precedence of the |
||||
left-most terminal symbol in the rule for which a precedence is |
||||
defined. This is normally what you want, but in those cases where |
||||
you want to precedence of a grammar rule to be something different, |
||||
you can specify an alternative precedence symbol by putting the |
||||
symbol in square braces after the period at the end of the rule and |
||||
before any C-code. For example:</p> |
||||
|
||||
<p><pre> |
||||
expr = MINUS expr. [NOT] |
||||
</pre></p> |
||||
|
||||
<p>This rule has a precedence equal to that of the NOT symbol, not the |
||||
MINUS symbol as would have been the case by default.</p> |
||||
|
||||
<p>With the knowledge of how precedence is assigned to terminal |
||||
symbols and individual |
||||
grammar rules, we can now explain precisely how parsing conflicts |
||||
are resolved in Lemon. Shift-reduce conflicts are resolved |
||||
as follows: |
||||
<ul> |
||||
<li> If either the token to be shifted or the rule to be reduced |
||||
lacks precedence information, then resolve in favor of the |
||||
shift, but report a parsing conflict. |
||||
<li> If the precedence of the token to be shifted is greater than |
||||
the precedence of the rule to reduce, then resolve in favor |
||||
of the shift. No parsing conflict is reported. |
||||
<li> If the precedence of the token it be shifted is less than the |
||||
precedence of the rule to reduce, then resolve in favor of the |
||||
reduce action. No parsing conflict is reported. |
||||
<li> If the precedences are the same and the shift token is |
||||
right-associative, then resolve in favor of the shift. |
||||
No parsing conflict is reported. |
||||
<li> If the precedences are the same the the shift token is |
||||
left-associative, then resolve in favor of the reduce. |
||||
No parsing conflict is reported. |
||||
<li> Otherwise, resolve the conflict by doing the shift and |
||||
report the parsing conflict. |
||||
</ul> |
||||
Reduce-reduce conflicts are resolved this way: |
||||
<ul> |
||||
<li> If either reduce rule |
||||
lacks precedence information, then resolve in favor of the |
||||
rule that appears first in the grammar and report a parsing |
||||
conflict. |
||||
<li> If both rules have precedence and the precedence is different |
||||
then resolve the dispute in favor of the rule with the highest |
||||
precedence and do not report a conflict. |
||||
<li> Otherwise, resolve the conflict by reducing by the rule that |
||||
appears first in the grammar and report a parsing conflict. |
||||
</ul> |
||||
|
||||
<h3>Special Directives</h3> |
||||
|
||||
<p>The input grammar to Lemon consists of grammar rules and special |
||||
directives. We've described all the grammar rules, so now we'll |
||||
talk about the special directives.</p> |
||||
|
||||
<p>Directives in lemon can occur in any order. You can put them before |
||||
the grammar rules, or after the grammar rules, or in the mist of the |
||||
grammar rules. It doesn't matter. The relative order of |
||||
directives used to assign precedence to terminals is important, but |
||||
other than that, the order of directives in Lemon is arbitrary.</p> |
||||
|
||||
<p>Lemon supports the following special directives: |
||||
<ul> |
||||
<li><tt>%destructor</tt> |
||||
<li><tt>%extra_argument</tt> |
||||
<li><tt>%include</tt> |
||||
<li><tt>%left</tt> |
||||
<li><tt>%name</tt> |
||||
<li><tt>%nonassoc</tt> |
||||
<li><tt>%parse_accept</tt> |
||||
<li><tt>%parse_failure </tt> |
||||
<li><tt>%right</tt> |
||||
<li><tt>%stack_overflow</tt> |
||||
<li><tt>%stack_size</tt> |
||||
<li><tt>%start_symbol</tt> |
||||
<li><tt>%syntax_error</tt> |
||||
<li><tt>%token_destructor</tt> |
||||
<li><tt>%token_prefix</tt> |
||||
<li><tt>%token_type</tt> |
||||
<li><tt>%type</tt> |
||||
</ul> |
||||
Each of these directives will be described separately in the |
||||
following sections:</p> |
||||
|
||||
<h4>The <tt>%destructor</tt> directive</h4> |
||||
|
||||
<p>The %destructor directive is used to specify a destructor for |
||||
a non-terminal symbol. |
||||
(See also the %token_destructor directive which is used to |
||||
specify a destructor for terminal symbols.)</p> |
||||
|
||||
<p>A non-terminal's destructor is called to dispose of the |
||||
non-terminal's value whenever the non-terminal is popped from |
||||
the stack. This includes all of the following circumstances: |
||||
<ul> |
||||
<li> When a rule reduces and the value of a non-terminal on |
||||
the right-hand side is not linked to C code. |
||||
<li> When the stack is popped during error processing. |
||||
<li> When the ParseFree() function runs. |
||||
</ul> |
||||
The destructor can do whatever it wants with the value of |
||||
the non-terminal, but its design is to deallocate memory |
||||
or other resources held by that non-terminal.</p> |
||||
|
||||
<p>Consider an example: |
||||
<pre> |
||||
%type nt {void*} |
||||
%destructor nt { free($$); } |
||||
nt(A) ::= ID NUM. { A = malloc( 100 ); } |
||||
</pre> |
||||
This example is a bit contrived but it serves to illustrate how |
||||
destructors work. The example shows a non-terminal named |
||||
``nt'' that holds values of type ``void*''. When the rule for |
||||
an ``nt'' reduces, it sets the value of the non-terminal to |
||||
space obtained from malloc(). Later, when the nt non-terminal |
||||
is popped from the stack, the destructor will fire and call |
||||
free() on this malloced space, thus avoiding a memory leak. |
||||
(Note that the symbol ``$$'' in the destructor code is replaced |
||||
by the value of the non-terminal.)</p> |
||||
|
||||
<p>It is important to note that the value of a non-terminal is passed |
||||
to the destructor whenever the non-terminal is removed from the |
||||
stack, unless the non-terminal is used in a C-code action. If |
||||
the non-terminal is used by C-code, then it is assumed that the |
||||
C-code will take care of destroying it if it should really |
||||
be destroyed. More commonly, the value is used to build some |
||||
larger structure and we don't want to destroy it, which is why |
||||
the destructor is not called in this circumstance.</p> |
||||
|
||||
<p>By appropriate use of destructors, it is possible to |
||||
build a parser using Lemon that can be used within a long-running |
||||
program, such as a GUI, that will not leak memory or other resources. |
||||
To do the same using yacc or bison is much more difficult.</p> |
||||
|
||||
<h4>The <tt>%extra_argument</tt> directive</h4> |
||||
|
||||
The %extra_argument directive instructs Lemon to add a 4th parameter |
||||
to the parameter list of the Parse() function it generates. Lemon |
||||
doesn't do anything itself with this extra argument, but it does |
||||
make the argument available to C-code action routines, destructors, |
||||
and so forth. For example, if the grammar file contains:</p> |
||||
|
||||
<p><pre> |
||||
%extra_argument { MyStruct *pAbc } |
||||
</pre></p> |
||||
|
||||
<p>Then the Parse() function generated will have an 4th parameter |
||||
of type ``MyStruct*'' and all action routines will have access to |
||||
a variable named ``pAbc'' that is the value of the 4th parameter |
||||
in the most recent call to Parse().</p> |
||||
|
||||
<h4>The <tt>%include</tt> directive</h4> |
||||
|
||||
<p>The %include directive specifies C code that is included at the |
||||
top of the generated parser. You can include any text you want -- |
||||
the Lemon parser generator copies to blindly. If you have multiple |
||||
%include directives in your grammar file, their values are concatenated |
||||
before being put at the beginning of the generated parser.</p> |
||||
|
||||
<p>The %include directive is very handy for getting some extra #include |
||||
preprocessor statements at the beginning of the generated parser. |
||||
For example:</p> |
||||
|
||||
<p><pre> |
||||
%include {#include <unistd.h>} |
||||
</pre></p> |
||||
|
||||
<p>This might be needed, for example, if some of the C actions in the |
||||
grammar call functions that are prototyed in unistd.h.</p> |
||||
|
||||
<h4>The <tt>%left</tt> directive</h4> |
||||
|
||||
The %left directive is used (along with the %right and |
||||
%nonassoc directives) to declare precedences of terminal |
||||
symbols. Every terminal symbol whose name appears after |
||||
a %left directive but before the next period (``.'') is |
||||
given the same left-associative precedence value. Subsequent |
||||
%left directives have higher precedence. For example:</p> |
||||
|
||||
<p><pre> |
||||
%left AND. |
||||
%left OR. |
||||
%nonassoc EQ NE GT GE LT LE. |
||||
%left PLUS MINUS. |
||||
%left TIMES DIVIDE MOD. |
||||
%right EXP NOT. |
||||
</pre></p> |
||||
|
||||
<p>Note the period that terminates each %left, %right or %nonassoc |
||||
directive.</p> |
||||
|
||||
<p>LALR(1) grammars can get into a situation where they require |
||||
a large amount of stack space if you make heavy use or right-associative |
||||
operators. For this reason, it is recommended that you use %left |
||||
rather than %right whenever possible.</p> |
||||
|
||||
<h4>The <tt>%name</tt> directive</h4> |
||||
|
||||
<p>By default, the functions generated by Lemon all begin with the |
||||
five-character string ``Parse''. You can change this string to something |
||||
different using the %name directive. For instance:</p> |
||||
|
||||
<p><pre> |
||||
%name Abcde |
||||
</pre></p> |
||||
|
||||
<p>Putting this directive in the grammar file will cause Lemon to generate |
||||
functions named |
||||
<ul> |
||||
<li> AbcdeAlloc(), |
||||
<li> AbcdeFree(), |
||||
<li> AbcdeTrace(), and |
||||
<li> Abcde(). |
||||
</ul> |
||||
The %name directive allows you to generator two or more different |
||||
parsers and link them all into the same executable. |
||||
</p> |
||||
|
||||
<h4>The <tt>%nonassoc</tt> directive</h4> |
||||
|
||||
<p>This directive is used to assign non-associative precedence to |
||||
one or more terminal symbols. See the section on precedence rules |
||||
or on the %left directive for additional information.</p> |
||||
|
||||
<h4>The <tt>%parse_accept</tt> directive</h4> |
||||
|
||||
<p>The %parse_accept directive specifies a block of C code that is |
||||
executed whenever the parser accepts its input string. To ``accept'' |
||||
an input string means that the parser was able to process all tokens |
||||
without error.</p> |
||||
|
||||
<p>For example:</p> |
||||
|
||||
<p><pre> |
||||
%parse_accept { |
||||
printf("parsing complete!\n"); |
||||
} |
||||
</pre></p> |
||||
|
||||
|
||||
<h4>The <tt>%parse_failure</tt> directive</h4> |
||||
|
||||
<p>The %parse_failure directive specifies a block of C code that |
||||
is executed whenever the parser fails complete. This code is not |
||||
executed until the parser has tried and failed to resolve an input |
||||
error using is usual error recovery strategy. The routine is |
||||
only invoked when parsing is unable to continue.</p> |
||||
|
||||
<p><pre> |
||||
%parse_failure { |
||||
fprintf(stderr,"Giving up. Parser is hopelessly lost...\n"); |
||||
} |
||||
</pre></p> |
||||
|
||||
<h4>The <tt>%right</tt> directive</h4> |
||||
|
||||
<p>This directive is used to assign right-associative precedence to |
||||
one or more terminal symbols. See the section on precedence rules |
||||
or on the %left directive for additional information.</p> |
||||
|
||||
<h4>The <tt>%stack_overflow</tt> directive</h4> |
||||
|
||||
<p>The %stack_overflow directive specifies a block of C code that |
||||
is executed if the parser's internal stack ever overflows. Typically |
||||
this just prints an error message. After a stack overflow, the parser |
||||
will be unable to continue and must be reset.</p> |
||||
|
||||
<p><pre> |
||||
%stack_overflow { |
||||
fprintf(stderr,"Giving up. Parser stack overflow\n"); |
||||
} |
||||
</pre></p> |
||||
|
||||
<p>You can help prevent parser stack overflows by avoiding the use |
||||
of right recursion and right-precedence operators in your grammar. |
||||
Use left recursion and and left-precedence operators instead, to |
||||
encourage rules to reduce sooner and keep the stack size down. |
||||
For example, do rules like this: |
||||
<pre> |
||||
list ::= list element. // left-recursion. Good! |
||||
list ::= . |
||||
</pre> |
||||
Not like this: |
||||
<pre> |
||||
list ::= element list. // right-recursion. Bad! |
||||
list ::= . |
||||
</pre> |
||||
|
||||
<h4>The <tt>%stack_size</tt> directive</h4> |
||||
|
||||
<p>If stack overflow is a problem and you can't resolve the trouble |
||||
by using left-recursion, then you might want to increase the size |
||||
of the parser's stack using this directive. Put an positive integer |
||||
after the %stack_size directive and Lemon will generate a parse |
||||
with a stack of the requested size. The default value is 100.</p> |
||||
|
||||
<p><pre> |
||||
%stack_size 2000 |
||||
</pre></p> |
||||
|
||||
<h4>The <tt>%start_symbol</tt> directive</h4> |
||||
|
||||
<p>By default, the start-symbol for the grammar that Lemon generates |
||||
is the first non-terminal that appears in the grammar file. But you |
||||
can choose a different start-symbol using the %start_symbol directive.</p> |
||||
|
||||
<p><pre> |
||||
%start_symbol prog |
||||
</pre></p> |
||||
|
||||
<h4>The <tt>%token_destructor</tt> directive</h4> |
||||
|
||||
<p>The %destructor directive assigns a destructor to a non-terminal |
||||
symbol. (See the description of the %destructor directive above.) |
||||
This directive does the same thing for all terminal symbols.</p> |
||||
|
||||
<p>Unlike non-terminal symbols which may each have a different data type |
||||
for their values, terminals all use the same data type (defined by |
||||
the %token_type directive) and so they use a common destructor. Other |
||||
than that, the token destructor works just like the non-terminal |
||||
destructors.</p> |
||||
|
||||
<h4>The <tt>%token_prefix</tt> directive</h4> |
||||
|
||||
<p>Lemon generates #defines that assign small integer constants |
||||
to each terminal symbol in the grammar. If desired, Lemon will |
||||
add a prefix specified by this directive |
||||
to each of the #defines it generates. |
||||
So if the default output of Lemon looked like this: |
||||
<pre> |
||||
#define AND 1 |
||||
#define MINUS 2 |
||||
#define OR 3 |
||||
#define PLUS 4 |
||||
</pre> |
||||
You can insert a statement into the grammar like this: |
||||
<pre> |
||||
%token_prefix TOKEN_ |
||||
</pre> |
||||
to cause Lemon to produce these symbols instead: |
||||
<pre> |
||||
#define TOKEN_AND 1 |
||||
#define TOKEN_MINUS 2 |
||||
#define TOKEN_OR 3 |
||||
#define TOKEN_PLUS 4 |
||||
</pre> |
||||
|
||||
<h4>The <tt>%token_type</tt> and <tt>%type</tt> directives</h4> |
||||
|
||||
<p>These directives are used to specify the data types for values |
||||
on the parser's stack associated with terminal and non-terminal |
||||
symbols. The values of all terminal symbols must be of the same |
||||
type. This turns out to be the same data type as the 3rd parameter |
||||
to the Parse() function generated by Lemon. Typically, you will |
||||
make the value of a terminal symbol by a pointer to some kind of |
||||
token structure. Like this:</p> |
||||
|
||||
<p><pre> |
||||
%token_type {Token*} |
||||
</pre></p> |
||||
|
||||
<p>If the data type of terminals is not specified, the default value |
||||
is ``int''.</p> |
||||
|
||||
<p>Non-terminal symbols can each have their own data types. Typically |
||||
the data type of a non-terminal is a pointer to the root of a parse-tree |
||||
structure that contains all information about that non-terminal. |
||||
For example:</p> |
||||
|
||||
<p><pre> |
||||
%type expr {Expr*} |
||||
</pre></p> |
||||
|
||||
<p>Each entry on the parser's stack is actually a union containing |
||||
instances of all data types for every non-terminal and terminal symbol. |
||||
Lemon will automatically use the correct element of this union depending |
||||
on what the corresponding non-terminal or terminal symbol is. But |
||||
the grammar designer should keep in mind that the size of the union |
||||
will be the size of its largest element. So if you have a single |
||||
non-terminal whose data type requires 1K of storage, then your 100 |
||||
entry parser stack will require 100K of heap space. If you are willing |
||||
and able to pay that price, fine. You just need to know.</p> |
||||
|
||||
<h3>Error Processing</h3> |
||||
|
||||
<p>After extensive experimentation over several years, it has been |
||||
discovered that the error recovery strategy used by yacc is about |
||||
as good as it gets. And so that is what Lemon uses.</p> |
||||
|
||||
<p>When a Lemon-generated parser encounters a syntax error, it |
||||
first invokes the code specified by the %syntax_error directive, if |
||||
any. It then enters its error recovery strategy. The error recovery |
||||
strategy is to begin popping the parsers stack until it enters a |
||||
state where it is permitted to shift a special non-terminal symbol |
||||
named ``error''. It then shifts this non-terminal and continues |
||||
parsing. But the %syntax_error routine will not be called again |
||||
until at least three new tokens have been successfully shifted.</p> |
||||
|
||||
<p>If the parser pops its stack until the stack is empty, and it still |
||||
is unable to shift the error symbol, then the %parse_failed routine |
||||
is invoked and the parser resets itself to its start state, ready |
||||
to begin parsing a new file. This is what will happen at the very |
||||
first syntax error, of course, if there are no instances of the |
||||
``error'' non-terminal in your grammar.</p> |
||||
|
||||
</body> |
||||
</html> |
@ -1,600 +0,0 @@ |
||||
/* Driver template for the LEMON parser generator.
|
||||
** Copyright 1991-1995 by D. Richard Hipp. |
||||
** |
||||
** This library is free software; you can redistribute it and/or |
||||
** modify it under the terms of the GNU Library General Public |
||||
** License as published by the Free Software Foundation; either |
||||
** version 2 of the License, or (at your option) any later version. |
||||
**
|
||||
** This library is distributed in the hope that it will be useful, |
||||
** but WITHOUT ANY WARRANTY; without even the implied warranty of |
||||
** MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU |
||||
** Library General Public License for more details. |
||||
**
|
||||
** You should have received a copy of the GNU Library General Public |
||||
** License along with this library; if not, write to the |
||||
** Free Software Foundation, Inc., 59 Temple Place - Suite 330, |
||||
** Boston, MA 02111-1307, USA. |
||||
** |
||||
** Modified 1997 to make it suitable for use with makeheaders. |
||||
** |
||||
** $IdPath$ |
||||
*/ |
||||
/* First off, code is include which follows the "include" declaration
|
||||
** in the input file. */ |
||||
#include <stdio.h> |
||||
%% |
||||
/* Next is all token values, in a form suitable for use by makeheaders.
|
||||
** This section will be null unless lemon is run with the -m switch. |
||||
*/ |
||||
/*
|
||||
** These constants (all generated automatically by the parser generator) |
||||
** specify the various kinds of tokens (terminals) that the parser |
||||
** understands.
|
||||
** |
||||
** Each symbol here is a terminal symbol in the grammar. |
||||
*/ |
||||
%% |
||||
/* Make sure the INTERFACE macro is defined.
|
||||
*/ |
||||
#ifndef INTERFACE |
||||
# define INTERFACE 1 |
||||
#endif |
||||
/* The next thing included is series of defines which control
|
||||
** various aspects of the generated parser. |
||||
** YYCODETYPE is the data type used for storing terminal |
||||
** and nonterminal numbers. "unsigned char" is |
||||
** used if there are fewer than 250 terminals |
||||
** and nonterminals. "int" is used otherwise. |
||||
** YYNOCODE is a number of type YYCODETYPE which corresponds |
||||
** to no legal terminal or nonterminal number. This |
||||
** number is used to fill in empty slots of the hash
|
||||
** table. |
||||
** YYACTIONTYPE is the data type used for storing terminal |
||||
** and nonterminal numbers. "unsigned char" is |
||||
** used if there are fewer than 250 rules and |
||||
** states combined. "int" is used otherwise. |
||||
** ParseTOKENTYPE is the data type used for minor tokens given
|
||||
** directly to the parser from the tokenizer. |
||||
** YYMINORTYPE is the data type used for all minor tokens. |
||||
** This is typically a union of many types, one of |
||||
** which is ParseTOKENTYPE. The entry in the union |
||||
** for base tokens is called "yy0". |
||||
** YYSTACKDEPTH is the maximum depth of the parser's stack. |
||||
** ParseARGDECL is a declaration of a 3rd argument to the |
||||
** parser, or null if there is no extra argument. |
||||
** ParseKRARGDECL A version of ParseARGDECL for K&R C. |
||||
** ParseANSIARGDECL A version of ParseARGDECL for ANSI C. |
||||
** YYNSTATE the combined number of states. |
||||
** YYNRULE the number of rules in the grammar |
||||
** YYERRORSYMBOL is the code number of the error symbol. If not |
||||
** defined, then do no error processing. |
||||
*/ |
||||
%% |
||||
#define YY_NO_ACTION (YYNSTATE+YYNRULE+2) |
||||
#define YY_ACCEPT_ACTION (YYNSTATE+YYNRULE+1) |
||||
#define YY_ERROR_ACTION (YYNSTATE+YYNRULE) |
||||
/* Next is the action table. Each entry in this table contains
|
||||
** |
||||
** + An integer which is the number representing the look-ahead |
||||
** token |
||||
** |
||||
** + An integer indicating what action to take. Number (N) between |
||||
** 0 and YYNSTATE-1 mean shift the look-ahead and go to state N. |
||||
** Numbers between YYNSTATE and YYNSTATE+YYNRULE-1 mean reduce by |
||||
** rule N-YYNSTATE. Number YYNSTATE+YYNRULE means that a syntax |
||||
** error has occurred. Number YYNSTATE+YYNRULE+1 means the parser |
||||
** accepts its input. |
||||
** |
||||
** + A pointer to the next entry with the same hash value. |
||||
** |
||||
** The action table is really a series of hash tables. Each hash |
||||
** table contains a number of entries which is a power of two. The |
||||
** "state" table (which follows) contains information about the starting |
||||
** point and size of each hash table. |
||||
*/ |
||||
struct yyActionEntry { |
||||
YYCODETYPE lookahead; /* The value of the look-ahead token */ |
||||
YYACTIONTYPE action; /* Action to take for this look-ahead */ |
||||
struct yyActionEntry *next; /* Next look-ahead with the same hash, or NULL */ |
||||
}; |
||||
static struct yyActionEntry yyActionTable[] = { |
||||
%% |
||||
}; |
||||
|
||||
/* The state table contains information needed to look up the correct
|
||||
** action in the action table, given the current state of the parser. |
||||
** Information needed includes: |
||||
** |
||||
** + A pointer to the start of the action hash table in yyActionTable. |
||||
** |
||||
** + A mask used to hash the look-ahead token. The mask is an integer |
||||
** which is one less than the size of the hash table.
|
||||
** |
||||
** + The default action. This is the action to take if no entry for |
||||
** the given look-ahead is found in the action hash table. |
||||
*/ |
||||
struct yyStateEntry { |
||||
struct yyActionEntry *hashtbl; /* Start of the hash table in yyActionTable */ |
||||
int mask; /* Mask used for hashing the look-ahead */ |
||||
YYACTIONTYPE actionDefault; /* Default action if look-ahead not found */ |
||||
}; |
||||
static struct yyStateEntry yyStateTable[] = { |
||||
%% |
||||
}; |
||||
|
||||
/* The following structure represents a single element of the
|
||||
** parser's stack. Information stored includes: |
||||
** |
||||
** + The state number for the parser at this level of the stack. |
||||
** |
||||
** + The value of the token stored at this level of the stack. |
||||
** (In other words, the "major" token.) |
||||
** |
||||
** + The semantic value stored at this level of the stack. This is |
||||
** the information used by the action routines in the grammar. |
||||
** It is sometimes called the "minor" token. |
||||
*/ |
||||
struct yyStackEntry { |
||||
int stateno; /* The state-number */ |
||||
int major; /* The major token value. This is the code
|
||||
** number for the token at this stack level */ |
||||
YYMINORTYPE minor; /* The user-supplied minor token value. This
|
||||
** is the value of the token */ |
||||
}; |
||||
|
||||
/* The state of the parser is completely contained in an instance of
|
||||
** the following structure */ |
||||
struct yyParser { |
||||
int idx; /* Index of top element in stack */ |
||||
int errcnt; /* Shifts left before out of the error */ |
||||
struct yyStackEntry *top; /* Pointer to the top stack element */ |
||||
struct yyStackEntry stack[YYSTACKDEPTH]; /* The parser's stack */ |
||||
}; |
||||
typedef struct yyParser yyParser; |
||||
|
||||
#ifndef NDEBUG |
||||
#include <stdio.h> |
||||
static FILE *yyTraceFILE = 0; |
||||
static char *yyTracePrompt = 0; |
||||
|
||||
/*
|
||||
** Turn parser tracing on by giving a stream to which to write the trace |
||||
** and a prompt to preface each trace message. Tracing is turned off |
||||
** by making either argument NULL
|
||||
** |
||||
** Inputs: |
||||
** <ul> |
||||
** <li> A FILE* to which trace output should be written. |
||||
** If NULL, then tracing is turned off. |
||||
** <li> A prefix string written at the beginning of every |
||||
** line of trace output. If NULL, then tracing is |
||||
** turned off. |
||||
** </ul> |
||||
** |
||||
** Outputs: |
||||
** None. |
||||
*/ |
||||
void ParseTrace(FILE *TraceFILE, char *zTracePrompt){ |
||||
yyTraceFILE = TraceFILE; |
||||
yyTracePrompt = zTracePrompt; |
||||
if( yyTraceFILE==0 ) yyTracePrompt = 0; |
||||
else if( yyTracePrompt==0 ) yyTraceFILE = 0; |
||||
} |
||||
|
||||
/* For tracing shifts, the names of all terminals and nonterminals
|
||||
** are required. The following table supplies these names */ |
||||
static char *yyTokenName[] = {
|
||||
%% |
||||
}; |
||||
#define YYTRACE(X) if( yyTraceFILE ) fprintf(yyTraceFILE,"%sReduce [%s].\n",yyTracePrompt,X); |
||||
#else |
||||
#define YYTRACE(X) |
||||
#endif |
||||
|
||||
/*
|
||||
** This function allocates a new parser. |
||||
** The only argument is a pointer to a function which works like |
||||
** malloc. |
||||
** |
||||
** Inputs: |
||||
** A pointer to the function used to allocate memory. |
||||
** |
||||
** Outputs: |
||||
** A pointer to a parser. This pointer is used in subsequent calls |
||||
** to Parse and ParseFree. |
||||
*/ |
||||
void *ParseAlloc(void *(*mallocProc)(size_t)){ |
||||
yyParser *pParser; |
||||
pParser = (yyParser*)(*mallocProc)( sizeof(yyParser) ); |
||||
if( pParser ){ |
||||
pParser->idx = -1; |
||||
} |
||||
return pParser; |
||||
} |
||||
|
||||
/* The following function deletes the value associated with a
|
||||
** symbol. The symbol can be either a terminal or nonterminal. |
||||
** "yymajor" is the symbol code, and "yypminor" is a pointer to |
||||
** the value. |
||||
*/ |
||||
static void yy_destructor(YYCODETYPE yymajor, YYMINORTYPE *yypminor){ |
||||
switch( yymajor ){ |
||||
/* Here is inserted the actions which take place when a
|
||||
** terminal or non-terminal is destroyed. This can happen |
||||
** when the symbol is popped from the stack during a |
||||
** reduce or during error processing or when a parser is
|
||||
** being destroyed before it is finished parsing. |
||||
** |
||||
** Note: during a reduce, the only symbols destroyed are those |
||||
** which appear on the RHS of the rule, but which are not used |
||||
** inside the C code. |
||||
*/ |
||||
%% |
||||
default: break; /* If no destructor action specified: do nothing */ |
||||
} |
||||
} |
||||
|
||||
/*
|
||||
** Pop the parser's stack once. |
||||
** |
||||
** If there is a destructor routine associated with the token which |
||||
** is popped from the stack, then call it. |
||||
** |
||||
** Return the major token number for the symbol popped. |
||||
*/ |
||||
static int yy_pop_parser_stack(yyParser *pParser){ |
||||
YYCODETYPE yymajor; |
||||
|
||||
if( pParser->idx<0 ) return 0; |
||||
#ifndef NDEBUG |
||||
if( yyTraceFILE && pParser->idx>=0 ){ |
||||
fprintf(yyTraceFILE,"%sPopping %s\n", |
||||
yyTracePrompt, |
||||
yyTokenName[pParser->top->major]); |
||||
} |
||||
#endif |
||||
yymajor = pParser->top->major; |
||||
yy_destructor( yymajor, &pParser->top->minor); |
||||
pParser->idx--; |
||||
pParser->top--; |
||||
return yymajor; |
||||
} |
||||
|
||||
/*
|
||||
** Deallocate and destroy a parser. Destructors are all called for |
||||
** all stack elements before shutting the parser down. |
||||
** |
||||
** Inputs: |
||||
** <ul> |
||||
** <li> A pointer to the parser. This should be a pointer |
||||
** obtained from ParseAlloc. |
||||
** <li> A pointer to a function used to reclaim memory obtained |
||||
** from malloc. |
||||
** </ul> |
||||
*/ |
||||
void ParseFree( |
||||
void *p, /* The parser to be deleted */ |
||||
void (*freeProc)(void*) /* Function used to reclaim memory */ |
||||
){ |
||||
yyParser *pParser = (yyParser*)p; |
||||
if( pParser==0 ) return; |
||||
while( pParser->idx>=0 ) yy_pop_parser_stack(pParser); |
||||
(*freeProc)((void*)pParser); |
||||
} |
||||
|
||||
/*
|
||||
** Find the appropriate action for a parser given the look-ahead token. |
||||
** |
||||
** If the look-ahead token is YYNOCODE, then check to see if the action is |
||||
** independent of the look-ahead. If it is, return the action, otherwise |
||||
** return YY_NO_ACTION. |
||||
*/ |
||||
static int yy_find_parser_action( |
||||
yyParser *pParser, /* The parser */ |
||||
int iLookAhead /* The look-ahead token */ |
||||
){ |
||||
struct yyStateEntry *pState; /* Appropriate entry in the state table */ |
||||
struct yyActionEntry *pAction; /* Action appropriate for the look-ahead */ |
||||
|
||||
/* if( pParser->idx<0 ) return YY_NO_ACTION; */ |
||||
pState = &yyStateTable[pParser->top->stateno]; |
||||
if( iLookAhead!=YYNOCODE ){ |
||||
pAction = &pState->hashtbl[iLookAhead & pState->mask]; |
||||
while( pAction ){ |
||||
if( pAction->lookahead==iLookAhead ) return pAction->action; |
||||
pAction = pAction->next; |
||||
} |
||||
}else if( pState->mask!=0 || pState->hashtbl->lookahead!=YYNOCODE ){ |
||||
return YY_NO_ACTION; |
||||
} |
||||
return pState->actionDefault; |
||||
} |
||||
|
||||
/*
|
||||
** Perform a shift action. |
||||
*/ |
||||
static void yy_shift( |
||||
yyParser *yypParser, /* The parser to be shifted */ |
||||
int yyNewState, /* The new state to shift in */ |
||||
int yyMajor, /* The major token to shift in */ |
||||
YYMINORTYPE *yypMinor /* Pointer ot the minor token to shift in */ |
||||
){ |
||||
yypParser->idx++; |
||||
yypParser->top++; |
||||
if( yypParser->idx>=YYSTACKDEPTH ){ |
||||
yypParser->idx--; |
||||
yypParser->top--; |
||||
#ifndef NDEBUG |
||||
if( yyTraceFILE ){ |
||||
fprintf(yyTraceFILE,"%sStack Overflow!\n",yyTracePrompt); |
||||
} |
||||
#endif |
||||
while( yypParser->idx>=0 ) yy_pop_parser_stack(yypParser); |
||||
/* Here code is inserted which will execute if the parser
|
||||
** stack every overflows */ |
||||
%% |
||||
return; |
||||
} |
||||
yypParser->top->stateno = yyNewState; |
||||
yypParser->top->major = yyMajor; |
||||
yypParser->top->minor = *yypMinor; |
||||
#ifndef NDEBUG |
||||
if( yyTraceFILE && yypParser->idx>0 ){ |
||||
int i; |
||||
fprintf(yyTraceFILE,"%sShift %d\n",yyTracePrompt,yyNewState); |
||||
fprintf(yyTraceFILE,"%sStack:",yyTracePrompt); |
||||
for(i=1; i<=yypParser->idx; i++) |
||||
fprintf(yyTraceFILE," %s",yyTokenName[yypParser->stack[i].major]); |
||||
fprintf(yyTraceFILE,"\n"); |
||||
} |
||||
#endif |
||||
} |
||||
|
||||
/* The following table contains information about every rule that
|
||||
** is used during the reduce. |
||||
*/ |
||||
static struct { |
||||
YYCODETYPE lhs; /* Symbol on the left-hand side of the rule */ |
||||
unsigned char nrhs; /* Number of right-hand side symbols in the rule */ |
||||
} yyRuleInfo[] = { |
||||
%% |
||||
}; |
||||
|
||||
static void yy_accept(yyParser * ParseANSIARGDECL); /* Forward Declaration */ |
||||
|
||||
/*
|
||||
** Perform a reduce action and the shift that must immediately |
||||
** follow the reduce. |
||||
*/ |
||||
static void yy_reduce( |
||||
yyParser *yypParser, /* The parser */ |
||||
int yyruleno /* Number of the rule by which to reduce */ |
||||
ParseANSIARGDECL |
||||
){ |
||||
int yygoto; /* The next state */ |
||||
int yyact; /* The next action */ |
||||
YYMINORTYPE yygotominor; /* The LHS of the rule reduced */ |
||||
struct yyStackEntry *yymsp; /* The top of the parser's stack */ |
||||
int yysize; /* Amount to pop the stack */ |
||||
yymsp = yypParser->top; |
||||
switch( yyruleno ){ |
||||
/* Beginning here are the reduction cases. A typical example
|
||||
** follows: |
||||
** case 0: |
||||
** YYTRACE("<text of the rule>"); |
||||
** #line <lineno> <grammarfile> |
||||
** { ... } // User supplied code
|
||||
** #line <lineno> <thisfile> |
||||
** break; |
||||
*/ |
||||
%% |
||||
}; |
||||
yygoto = yyRuleInfo[yyruleno].lhs; |
||||
yysize = yyRuleInfo[yyruleno].nrhs; |
||||
yypParser->idx -= yysize; |
||||
yypParser->top -= yysize; |
||||
yyact = yy_find_parser_action(yypParser,yygoto); |
||||
if( yyact < YYNSTATE ){ |
||||
yy_shift(yypParser,yyact,yygoto,&yygotominor); |
||||
}else if( yyact == YYNSTATE + YYNRULE + 1 ){ |
||||
yy_accept(yypParser ParseARGDECL); |
||||
} |
||||
} |
||||
|
||||
/*
|
||||
** The following code executes when the parse fails |
||||
*/ |
||||
static void yy_parse_failed( |
||||
yyParser *yypParser /* The parser */ |
||||
ParseANSIARGDECL /* Extra arguments (if any) */ |
||||
){ |
||||
#ifndef NDEBUG |
||||
if( yyTraceFILE ){ |
||||
fprintf(yyTraceFILE,"%sFail!\n",yyTracePrompt); |
||||
} |
||||
#endif |
||||
while( yypParser->idx>=0 ) yy_pop_parser_stack(yypParser); |
||||
/* Here code is inserted which will be executed whenever the
|
||||
** parser fails */ |
||||
%% |
||||
} |
||||
|
||||
/*
|
||||
** The following code executes when a syntax error first occurs. |
||||
*/ |
||||
static void yy_syntax_error( |
||||
yyParser *yypParser, /* The parser */ |
||||
int yymajor, /* The major type of the error token */ |
||||
YYMINORTYPE yyminor /* The minor type of the error token */ |
||||
ParseANSIARGDECL /* Extra arguments (if any) */ |
||||
){ |
||||
#define TOKEN (yyminor.yy0) |
||||
%% |
||||
} |
||||
|
||||
/*
|
||||
** The following is executed when the parser accepts |
||||
*/ |
||||
static void yy_accept( |
||||
yyParser *yypParser /* The parser */ |
||||
ParseANSIARGDECL /* Extra arguments (if any) */ |
||||
){ |
||||
#ifndef NDEBUG |
||||
if( yyTraceFILE ){ |
||||
fprintf(yyTraceFILE,"%sAccept!\n",yyTracePrompt); |
||||
} |
||||
#endif |
||||
while( yypParser->idx>=0 ) yy_pop_parser_stack(yypParser); |
||||
/* Here code is inserted which will be executed whenever the
|
||||
** parser accepts */ |
||||
%% |
||||
} |
||||
|
||||
/* The main parser program.
|
||||
** The first argument is a pointer to a structure obtained from |
||||
** "ParseAlloc" which describes the current state of the parser. |
||||
** The second argument is the major token number. The third is |
||||
** the minor token. The fourth optional argument is whatever the |
||||
** user wants (and specified in the grammar) and is available for |
||||
** use by the action routines. |
||||
** |
||||
** Inputs: |
||||
** <ul> |
||||
** <li> A pointer to the parser (an opaque structure.) |
||||
** <li> The major token number. |
||||
** <li> The minor token number. |
||||
** <li> An option argument of a grammar-specified type. |
||||
** </ul> |
||||
** |
||||
** Outputs: |
||||
** None. |
||||
*/ |
||||
void Parse( |
||||
void *yyp, /* The parser */ |
||||
int yymajor, /* The major token code number */ |
||||
ParseTOKENTYPE yyminor /* The value for the token */ |
||||
ParseANSIARGDECL |
||||
){ |
||||
YYMINORTYPE yyminorunion; |
||||
int yyact; /* The parser action. */ |
||||
int yyendofinput; /* True if we are at the end of input */ |
||||
int yyerrorhit = 0; /* True if yymajor has invoked an error */ |
||||
yyParser *yypParser; /* The parser */ |
||||
|
||||
/* (re)initialize the parser, if necessary */ |
||||
yypParser = (yyParser*)yyp; |
||||
if( yypParser->idx<0 ){ |
||||
if( yymajor==0 ) return; |
||||
yypParser->idx = 0; |
||||
yypParser->errcnt = -1; |
||||
yypParser->top = &yypParser->stack[0]; |
||||
yypParser->top->stateno = 0; |
||||
yypParser->top->major = 0; |
||||
} |
||||
yyminorunion.yy0 = yyminor; |
||||
yyendofinput = (yymajor==0); |
||||
|
||||
#ifndef NDEBUG |
||||
if( yyTraceFILE ){ |
||||
fprintf(yyTraceFILE,"%sInput %s\n",yyTracePrompt,yyTokenName[yymajor]); |
||||
} |
||||
#endif |
||||
|
||||
do{ |
||||
yyact = yy_find_parser_action(yypParser,yymajor); |
||||
if( yyact<YYNSTATE ){ |
||||
yy_shift(yypParser,yyact,yymajor,&yyminorunion); |
||||
yypParser->errcnt--; |
||||
if( yyendofinput && yypParser->idx>=0 ){ |
||||
yymajor = 0; |
||||
}else{ |
||||
yymajor = YYNOCODE; |
||||
} |
||||
}else if( yyact < YYNSTATE + YYNRULE ){ |
||||
yy_reduce(yypParser,yyact-YYNSTATE ParseARGDECL); |
||||
}else if( yyact == YY_ERROR_ACTION ){ |
||||
#ifndef NDEBUG |
||||
if( yyTraceFILE ){ |
||||
fprintf(yyTraceFILE,"%sSyntax Error!\n",yyTracePrompt); |
||||
} |
||||
#endif |
||||
#ifdef YYERRORSYMBOL |
||||
/* A syntax error has occurred.
|
||||
** The response to an error depends upon whether or not the |
||||
** grammar defines an error token "ERROR".
|
||||
** |
||||
** This is what we do if the grammar does define ERROR: |
||||
** |
||||
** * Call the %syntax_error function. |
||||
** |
||||
** * Begin popping the stack until we enter a state where |
||||
** it is legal to shift the error symbol, then shift |
||||
** the error symbol. |
||||
** |
||||
** * Set the error count to three. |
||||
** |
||||
** * Begin accepting and shifting new tokens. No new error |
||||
** processing will occur until three tokens have been |
||||
** shifted successfully. |
||||
** |
||||
*/ |
||||
if( yypParser->errcnt<0 ){ |
||||
yy_syntax_error(yypParser,yymajor,yyminorunion ParseARGDECL); |
||||
} |
||||
if( yypParser->top->major==YYERRORSYMBOL || yyerrorhit ){ |
||||
#ifndef NDEBUG |
||||
if( yyTraceFILE ){ |
||||
fprintf(yyTraceFILE,"%sDiscard input token %s\n", |
||||
yyTracePrompt,yyTokenName[yymajor]); |
||||
} |
||||
#endif |
||||
yy_destructor(yymajor,&yyminorunion); |
||||
yymajor = YYNOCODE; |
||||
}else{ |
||||
while( |
||||
yypParser->idx >= 0 && |
||||
yypParser->top->major != YYERRORSYMBOL && |
||||
(yyact = yy_find_parser_action(yypParser,YYERRORSYMBOL)) >= YYNSTATE |
||||
){ |
||||
yy_pop_parser_stack(yypParser); |
||||
} |
||||
if( yypParser->idx < 0 || yymajor==0 ){ |
||||
yy_destructor(yymajor,&yyminorunion); |
||||
yy_parse_failed(yypParser ParseARGDECL); |
||||
yymajor = YYNOCODE; |
||||
}else if( yypParser->top->major!=YYERRORSYMBOL ){ |
||||
YYMINORTYPE u2; |
||||
u2.YYERRSYMDT = 0; |
||||
yy_shift(yypParser,yyact,YYERRORSYMBOL,&u2); |
||||
} |
||||
} |
||||
yypParser->errcnt = 3; |
||||
yyerrorhit = 1; |
||||
#else /* YYERRORSYMBOL is not defined */ |
||||
/* This is what we do if the grammar does not define ERROR:
|
||||
** |
||||
** * Report an error message, and throw away the input token. |
||||
** |
||||
** * If the input token is $, then fail the parse. |
||||
** |
||||
** As before, subsequent error messages are suppressed until |
||||
** three input tokens have been successfully shifted. |
||||
*/ |
||||
if( yypParser->errcnt<=0 ){ |
||||
yy_syntax_error(yypParser,yymajor,yyminorunion ParseARGDECL); |
||||
} |
||||
yypParser->errcnt = 3; |
||||
yy_destructor(yymajor,&yyminorunion); |
||||
if( yyendofinput ){ |
||||
yy_parse_failed(yypParser ParseARGDECL); |
||||
} |
||||
yymajor = YYNOCODE; |
||||
#endif |
||||
}else{ |
||||
yy_accept(yypParser ParseARGDECL); |
||||
yymajor = YYNOCODE; |
||||
} |
||||
}while( yymajor!=YYNOCODE && yypParser->idx>=0 ); |
||||
return; |
||||
} |
Loading…
Reference in new issue