@end direntry
@c %**end of header
-@node Top
+@node Top, Copyright, (dir), (dir)
@top flex
This manual describes @code{flex}, a tool for generating programs that
Scanner Options
-* Options for Specifing Filenames::
-* Options Affecting Scanner Behavior::
-* Code-Level And API Options::
-* Options for Scanner Speed and Size::
-* Debugging Options::
-* Miscellaneous Options::
+* Options for Specifing Filenames::
+* Options Affecting Scanner Behavior::
+* Code-Level And API Options::
+* Options for Scanner Speed and Size::
+* Debugging Options::
+* Miscellaneous Options::
Reentrant C Scanners
Serialized Tables
-* Creating Serialized Tables::
-* Loading and Unloading Serialized Tables::
-* Tables File Format::
+* Creating Serialized Tables::
+* Loading and Unloading Serialized Tables::
+* Tables File Format::
FAQ
* Why doesnt flex have non-greedy operators like perl does?::
* Memory leak - 16386 bytes allocated by malloc.::
* How do I track the byte offset for lseek()?::
-* unnamed-faq-16::
+* How do I use my own I/O classes in a C++ scanner?::
* How do I skip as many chars as possible?::
* unnamed-faq-33::
* unnamed-faq-42::
Appendices
* Makefiles and Flex::
-* Bison Bridge::
+* Bison Bridge::
Indices
* Index of Data Types::
* Index of Hooks::
* Index of Examples::
+* Index of Scanner Options::
@end detailmenu
@end menu
-@node Copyright
+@node Copyright, Reporting Bugs, Top, Top
@chapter Copyright
@cindex copyright of flex
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
PURPOSE.
-@node Reporting Bugs
+@node Reporting Bugs, Introduction, Copyright, Top
@chapter Reporting Bugs
@cindex bugs, reporting
please send mail detailing your problem to
@email{help-flex@@gnu.org}. Patches are always welcome.
-@node Introduction
+@node Introduction, Simple Examples, Reporting Bugs, Top
@chapter Introduction
@cindex scanner, definition of
input for occurrences of the regular expressions. Whenever it finds
one, it executes the corresponding C code.
-@node Simple Examples
+@node Simple Examples, Format, Introduction, Top
@chapter Some Simple Examples
First some simple examples to get the flavor of how one uses
The details of this example will be explained in the following
sections.
-@node Format
+@node Format, Patterns, Simple Examples, Top
@chapter Format of the Input File
* Comments in the Input::
@end menu
-@node Definitions Section
+@node Definitions Section, Rules Section, Format, Format
@section Format of the Definitions Section
@cindex input file, Definitions section
removed). The %@{ and %@} symbols must appear unindented on lines by
themselves.
-@node Rules Section
+@node Rules Section, User Code Section, Definitions Section, Format
@section Format of the Rules Section
@cindex input file, Rules Section
The %@{ and %@} symbols must appear unindented on lines by themselves.
@c proofread edit stopped here
-@node User Code Section
+@node User Code Section, Comments in the Input, Rules Section, Format
@section Format of the User Code Section
@cindex format, User Code Section
@cindex sections, User Code Section
@samp{%%}
in the input file may be skipped, too.
-@node Comments in the Input
+@node Comments in the Input, , User Code Section, Format
@section Comments in the Input
@cindex comments, syntax
@end verbatim
@end example
-@node Patterns
+@node Patterns, Matching, Format, Top
@chapter Patterns
@cindex Patterns
bar-at-the-beginning-of-a-line.
@end itemize
-@node Matching
+@node Matching, Actions, Patterns, Top
@chapter How the Input Is Matched
@cindex patterns, how the input is matched
Also note that you cannot use @code{%array} with C++ scanner classes
(@pxref{Cxx}).
-@node Actions
+@node Actions, Generated Scanner, Matching, Top
@chapter Actions
@cindex actions, explanation
is also called when an end-of-file is encountered. It is a macro and
may be redefined.
-@node Generated Scanner
+@node Generated Scanner, Start Conditions, Actions, Top
@chapter The Generated Scanner
@cindex yylex(), in generated scanner
@code{FILE}
pointer.
-@node Start Conditions
+@node Start Conditions, Multiple Input Buffers, Generated Scanner, Top
@chapter Start Conditions
@cindex start conditions, explanation
To use start condition stacks, your scanner must include a @code{%option
stack} directive (@pxref{Scanner Options}).
-@node Multiple Input Buffers
+@node Multiple Input Buffers, EOF, Start Conditions, Top
@chapter Multiple Input Buffers
@cindex multiple input streams
reflecting the size of the buffer.
@end deftp
-@node EOF
+@node EOF, Misc Macros, Multiple Input Buffers, Top
@chapter End-of-File Rules
@cindex EOF, explanation
@end verbatim
@end example
-@node Misc Macros
+@node Misc Macros, User Values, EOF, Top
@chapter Miscellaneous Macros
@hkindex YY_USER_ACTION
where because a rule's action ends with @code{return}, the
@code{YY_BREAK} is inaccessible.
-@node User Values
+@node User Values, Yacc, Misc Macros, Top
@chapter Values Available To the User
This chapter summarizes the various values available to the user in the
start condition.
@end table
-@node Yacc
+@node Yacc, Scanner Options, User Values, Top
@chapter Interfacing with Yacc
@cindex yacc, interface
@end verbatim
@end example
-@node Scanner Options
+@node Scanner Options, Performance, Yacc, Top
@chapter Scanner Options
@cindex command-line options
menu. If you want to lookup a particular option by name, @xref{Index of Scanner Options}.
@menu
-* Options for Specifing Filenames::
-* Options Affecting Scanner Behavior::
-* Code-Level And API Options::
-* Options for Scanner Speed and Size::
-* Debugging Options::
-* Miscellaneous Options::
+* Options for Specifing Filenames::
+* Options Affecting Scanner Behavior::
+* Code-Level And API Options::
+* Options for Scanner Speed and Size::
+* Debugging Options::
+* Miscellaneous Options::
@end menu
Even though there are many scanner options, a typical scanner might only
(though @code{yy_push_state()} and friends won't appear anyway unless
you use @code{%option stack)}.
-@node Options for Specifing Filenames
+@node Options for Specifing Filenames, Options Affecting Scanner Behavior, Scanner Options, Scanner Options
@section Options for Specifing Filenames
@table @samp
@end table
-@node Options Affecting Scanner Behavior
+@node Options Affecting Scanner Behavior, Code-Level And API Options, Options for Specifing Filenames, Scanner Options
@section Options Affecting Scanner Behavior
@table @samp
@end table
-@node Code-Level And API Options
+@node Code-Level And API Options, Options for Scanner Speed and Size, Options Affecting Scanner Behavior, Scanner Options
@section Code-Level And API Options
@table @samp
@end table
-@node Options for Scanner Speed and Size
+@node Options for Scanner Speed and Size, Debugging Options, Code-Level And API Options, Scanner Options
@section Options for Scanner Speed and Size
@table @samp
@end table
-@node Debugging Options
+@node Debugging Options, Miscellaneous Options, Options for Scanner Speed and Size, Scanner Options
@section Debugging Options
@table @samp
@end table
-@node Miscellaneous Options
+@node Miscellaneous Options, , Debugging Options, Scanner Options
@section Miscellaneous Options
@table @samp
@end table
-@node Performance
+@node Performance, Cxx, Scanner Options, Top
@chapter Performance Considerations
@cindex performance, considerations
``huge'' quantities, where the cutoff between the two is at about 8K
characters per token.
-@node Cxx
+@node Cxx, Reentrant, Performance, Top
@chapter Generating C++ Scanners
@cindex c++, experimental form of scanner class
if, for example, you used @code{%option prefix="xx"} for one of your
scanners and @code{%option prefix="zz"} for the other.
-@node Reentrant
+@node Reentrant, Lex and Posix, Cxx, Top
@chapter Reentrant C Scanners
@cindex reentrant, explanation
* Reentrant Functions::
@end menu
-@node Reentrant Uses
+@node Reentrant Uses, Reentrant Overview, Reentrant, Reentrant
@section Uses for Reentrant Scanners
However, there are other uses for a reentrant scanner. For example, you
@end verbatim
@end example
-@node Reentrant Overview
+@node Reentrant Overview, Reentrant Example, Reentrant Uses, Reentrant
@section An Overview of the Reentrant API
@cindex reentrant, API explanation
User-specific data can be stored in @code{yyextra}.
@end itemize
-@node Reentrant Example
+@node Reentrant Example, Reentrant Detail, Reentrant Overview, Reentrant
@section Reentrant Example
First, an example of a reentrant scanner:
@end verbatim
@end example
-@node Reentrant Detail
+@node Reentrant Detail, Reentrant Functions, Reentrant Example, Reentrant
@section The Reentrant API in Detail
Here are the things you need to do or know to use the reentrant C API of
* About yyscan_t::
@end menu
-@node Specify Reentrant
+@node Specify Reentrant, Extra Reentrant Argument, Reentrant Detail, Reentrant Detail
@subsection Declaring a Scanner As Reentrant
%option reentrant (--reentrant) must be specified.
you do @emph{not} want a reentrant scanner, although it is not
necessary. The default is to generate a non-reentrant scanner.
-@node Extra Reentrant Argument
+@node Extra Reentrant Argument, Global Replacement, Specify Reentrant, Reentrant Detail
@subsection The Extra Argument
@cindex reentrant, calling functions
@code{BEGIN}, @code{ECHO}, and @code{REJECT}, do not take this
additional argument.
-@node Global Replacement
+@node Global Replacement, Init and Destroy Functions, Extra Reentrant Argument, Reentrant Detail
@subsection Global Variables Replaced By Macros
@cindex reentrant, accessing flex variables
@code{yyget_text},
to accomplish this. (See below).
-@node Init and Destroy Functions
+@node Init and Destroy Functions, Accessor Methods, Global Replacement, Reentrant Detail
@subsection Init and Destroy Functions
@cindex memory, considerations for reentrant scanners
@end verbatim
@end example
-@node Accessor Methods
+@node Accessor Methods, Extra Data, Init and Destroy Functions, Reentrant Detail
@subsection Accessing Variables with Reentrant Scanners
@cindex reentrant, accessor functions
You may find that @code{%option header-file} is particularly useful for generating
prototypes of all the accessor functions. @xref{option-header}.
-@node Extra Data
+@node Extra Data, About yyscan_t, Accessor Methods, Reentrant Detail
@subsection Extra Data
@cindex reentrant, extra data
@end example
-@node About yyscan_t
+@node About yyscan_t, , Extra Data, Reentrant Detail
@subsection About yyscan_t
@tindex yyscan_t (reentrant only)
directly. In particular, you should never attempt to free it
(use @code{yylex_destroy()} instead.)
-@node Reentrant Functions
+@node Reentrant Functions, , Reentrant Detail, Reentrant
@section Functions and Macros Available in Reentrant C Scanners
The following Functions are available in a reentrant scanner:
by @code{bison}, in a .h file, and are included in section 1 of the
@code{flex} input.
-@node Lex and Posix
+@node Lex and Posix, Memory Management, Reentrant, Top
@chapter Incompatibilities with Lex and Posix
@cindex POSIX and lex
@code{flex} does not truncate the action. Actions that are not enclosed
in braces are simply terminated at the end of the line.
-@node Memory Management
+@node Memory Management, Serialized Tables, Lex and Posix, Top
@chapter Memory Management
@cindex memory management
* A Note About yytext And Memory::
@end menu
-@node The Default Memory Management
+@node The Default Memory Management, Overriding The Default Memory Management, Memory Management, Memory Management
@section The Default Memory Management
Flex allocates dynamic memory during initialization, and once in a while from
@end table
-@node Overriding The Default Memory Management
+@node Overriding The Default Memory Management, A Note About yytext And Memory, The Default Memory Management, Memory Management
@section Overriding The Default Memory Management
@cindex yyalloc, overriding
@end example
-@node A Note About yytext And Memory
+@node A Note About yytext And Memory, , Overriding The Default Memory Management, Memory Management
@section A Note About yytext And Memory
@cindex yytext, memory considerations
somehow. Our experience has shown that a garbage collection mechanism or a
pooled memory mechanism will save you a lot of grief when writing parsers.
-@node Serialized Tables
+@node Serialized Tables, Diagnostics, Memory Management, Top
@chapter Serialized Tables
@cindex serialization
@cindex memory, serialized tables
scanning begins. The tables may be discarded when scanning is finished.
@menu
-* Creating Serialized Tables::
-* Loading and Unloading Serialized Tables::
-* Tables File Format::
+* Creating Serialized Tables::
+* Loading and Unloading Serialized Tables::
+* Tables File Format::
@end menu
-@node Creating Serialized Tables
+@node Creating Serialized Tables, Loading and Unloading Serialized Tables, Serialized Tables, Serialized Tables
@section Creating Serialized Tables
@cindex tables, creating serialized
@cindex serialization of tables
runtime, we will open the file and tell flex to load the tables from it. Flex
will find the correct tables automatically. (See next section).
-@node Loading and Unloading Serialized Tables
+@node Loading and Unloading Serialized Tables, Tables File Format, Creating Serialized Tables, Serialized Tables
@section Loading and Unloading Serialized Tables
@cindex tables, loading and unloading
@cindex loading tables at runtime
After the tables are loaded, they are never written to, and no thread
protection is required thereafter -- until you destroy them.
-@node Tables File Format
+@node Tables File Format, , Loading and Unloading Serialized Tables, Serialized Tables
@section Tables File Format
@cindex tables, file format
@cindex file format, serialized tables
calculated from the beginning of this table.
@end table
-@node Diagnostics
+@node Diagnostics, Limitations, Serialized Tables, Top
@chapter Diagnostics
@cindex error reporting, diagnostic messages
least one of them twice).
@end itemize
-@node Limitations
+@node Limitations, Bibliography, Diagnostics, Top
@chapter Limitations
@cindex limitations of flex
The @code{flex} internal algorithms need documentation.
-@node Bibliography
+@node Bibliography, FAQ, Limitations, Top
@chapter Additional Reading
You may wish to read more about the following programs:
pattern-matching techniques used by @code{flex} (deterministic finite
automata).
-@node FAQ
+@node FAQ, Appendices, Bibliography, Top
@unnumbered FAQ
From time to time, the @code{flex} maintainer receives certain
* Why doesnt flex have non-greedy operators like perl does?::
* Memory leak - 16386 bytes allocated by malloc.::
* How do I track the byte offset for lseek()?::
-* unnamed-faq-16::
+* How do I use my own I/O classes in a C++ scanner?::
* How do I skip as many chars as possible?::
* unnamed-faq-33::
* unnamed-faq-42::
* unnamed-faq-101::
@end menu
-@node When was flex born?
+@node When was flex born?, How do I expand \ escape sequences in C-style quoted strings?, FAQ, FAQ
@unnumberedsec When was flex born?
Vern Paxson took over
was written in Ratfor. Around 1987 or so, Paxson translated it into C, and
a legend was born :-).
-@node How do I expand \ escape sequences in C-style quoted strings?
+@node How do I expand \ escape sequences in C-style quoted strings?, Why do flex scanners call fileno if it is not ANSI compatible?, When was flex born?, FAQ
@unnumberedsec How do I expand \ escape sequences in C-style quoted strings?
A key point when scanning quoted strings is that you cannot (easily) write
meaning of the escape sequence rather than the literal text in @code{yytext}.
In this way, @code{yytext} does not need to be modified at all.
-@node Why do flex scanners call fileno if it is not ANSI compatible?
+@node Why do flex scanners call fileno if it is not ANSI compatible?, Does flex support recursive pattern definitions?, How do I expand \ escape sequences in C-style quoted strings?, FAQ
@unnumberedsec Why do flex scanners call fileno if it is not ANSI compatible?
Flex scanners call @code{fileno()} in order to get the file descriptor
call, you must specify one of @code{%option always-interactive} or
@code{%option never-interactive}.
-@node Does flex support recursive pattern definitions?
+@node Does flex support recursive pattern definitions?, How do I skip huge chunks of input (tens of megabytes) while using flex?, Why do flex scanners call fileno if it is not ANSI compatible?, FAQ
@unnumberedsec Does flex support recursive pattern definitions?
e.g.,
as '@}'s. For more powerful pattern matching, you need a parser, such
as @cite{GNU bison}.
-@node How do I skip huge chunks of input (tens of megabytes) while using flex?
+@node How do I skip huge chunks of input (tens of megabytes) while using flex?, Flex is not matching my patterns in the same order that I defined them., Does flex support recursive pattern definitions?, FAQ
@unnumberedsec How do I skip huge chunks of input (tens of megabytes) while using flex?
Use @code{fseek()} (or @code{lseek()}) to position yyin, then call @code{yyrestart()}.
-@node Flex is not matching my patterns in the same order that I defined them.
+@node Flex is not matching my patterns in the same order that I defined them., My actions are executing out of order or sometimes not at all., How do I skip huge chunks of input (tens of megabytes) while using flex?, FAQ
@unnumberedsec Flex is not matching my patterns in the same order that I defined them.
@code{flex} picks the
identifier rule so it no longer matches @samp{data_}. (Of course, you might
also not have the option of changing the input language.)
-@node My actions are executing out of order or sometimes not at all.
+@node My actions are executing out of order or sometimes not at all., How can I have multiple input sources feed into the same scanner at the same time?, Flex is not matching my patterns in the same order that I defined them., FAQ
@unnumberedsec My actions are executing out of order or sometimes not at all.
Most likely, you have (in error) placed the opening @samp{@{} of the action
@end verbatim
@end example
-@node How can I have multiple input sources feed into the same scanner at the same time?
+@node How can I have multiple input sources feed into the same scanner at the same time?, Can I build nested parsers that work with the same input file?, My actions are executing out of order or sometimes not at all., FAQ
@unnumberedsec How can I have multiple input sources feed into the same scanner at the same time?
If @dots{}
interpreter I wrote that both reads keyboard input using a @code{flex} scanner and
IPC traffic from sockets, and it works fine.
-@node Can I build nested parsers that work with the same input file?
+@node Can I build nested parsers that work with the same input file?, How can I match text only at the end of a file?, How can I have multiple input sources feed into the same scanner at the same time?, FAQ
@unnumberedsec Can I build nested parsers that work with the same input file?
This is not going to work without some additional effort. The reason is
combine all of your scanners into one large scanner, using a different
exclusive start condition for each.
-@node How can I match text only at the end of a file?
+@node How can I match text only at the end of a file?, How can I make REJECT cascade across start condition boundaries?, Can I build nested parsers that work with the same input file?, FAQ
@unnumberedsec How can I match text only at the end of a file?
There is no way to write a rule which is ``match this text, but only if
@end verbatim
@end example
-@node How can I make REJECT cascade across start condition boundaries?
+@node How can I make REJECT cascade across start condition boundaries?, Why cant I use fast or full tables with interactive mode?, How can I match text only at the end of a file?, FAQ
@unnumberedsec How can I make REJECT cascade across start condition boundaries?
You can do this as follows. Suppose you have a start condition @samp{A}, and
@end verbatim
@end example
-@node Why cant I use fast or full tables with interactive mode?
+@node Why cant I use fast or full tables with interactive mode?, How much faster is -F or -f than -C?, How can I make REJECT cascade across start condition boundaries?, FAQ
@unnumberedsec Why can't I use fast or full tables with interactive mode?
One of the assumptions
might be another reason, though, why fast scanners don't support the
interactive option.
-@node How much faster is -F or -f than -C?
+@node How much faster is -F or -f than -C?, If I have a simple grammar cant I just parse it with flex?, Why cant I use fast or full tables with interactive mode?, FAQ
@unnumberedsec How much faster is -F or -f than -C?
Much faster (factor of 2-3).
-@node If I have a simple grammar cant I just parse it with flex?
+@node If I have a simple grammar cant I just parse it with flex?, Why doesnt yyrestart() set the start state back to INITIAL?, How much faster is -F or -f than -C?, FAQ
@unnumberedsec If I have a simple grammar can't I just parse it with flex?
Is your grammar recursive? That's almost always a sign that you're
better off using a parser/scanner rather than just trying to use a scanner
alone.
-@node Why doesnt yyrestart() set the start state back to INITIAL?
+@node Why doesnt yyrestart() set the start state back to INITIAL?, How can I match C-style comments?, If I have a simple grammar cant I just parse it with flex?, FAQ
@unnumberedsec Why doesn't yyrestart() set the start state back to INITIAL?
There are two reasons. The first is that there might
The second is that beginning with @code{flex} version 2.4, use of @code{yyrestart()} is no longer required,
so fixing the problem there doesn't solve the more general problem.
-@node How can I match C-style comments?
+@node How can I match C-style comments?, The period isnt working the way I expected., Why doesnt yyrestart() set the start state back to INITIAL?, FAQ
@unnumberedsec How can I match C-style comments?
You might be tempted to try something like this:
@end verbatim
@end example
-@node The period isnt working the way I expected.
+@node The period isnt working the way I expected., Can I get the flex manual in another format?, How can I match C-style comments?, FAQ
@unnumberedsec The '.' isn't working the way I expected.
Here are some tips for using @samp{.}:
Finally, if you want to match a literal @samp{.} (a period), then use @samp{[.]} or @samp{"."}
@end itemize
-@node Can I get the flex manual in another format?
+@node Can I get the flex manual in another format?, Does there exist a "faster" NDFA->DFA algorithm?, The period isnt working the way I expected., FAQ
@unnumberedsec Can I get the flex manual in another format?
The @code{flex} source distribution includes a texinfo manual. You are
free to convert that texinfo into whatever format you desire. The
@code{texinfo} package includes tools for conversion to a number of formats.
-@node Does there exist a "faster" NDFA->DFA algorithm?
+@node Does there exist a "faster" NDFA->DFA algorithm?, How does flex compile the DFA so quickly?, Can I get the flex manual in another format?, FAQ
@unnumberedsec Does there exist a "faster" NDFA->DFA algorithm?
There's no way around the potential exponential running time - it
In practice, though, the running time is closer to linear, or sometimes
quadratic.
-@node How does flex compile the DFA so quickly?
+@node How does flex compile the DFA so quickly?, How can I use more than 8192 rules?, Does there exist a "faster" NDFA->DFA algorithm?, FAQ
@unnumberedsec How does flex compile the DFA so quickly?
There are two big speed wins that @code{flex} uses:
state can be done very quickly, by first comparing hash values.
@end enumerate
-@node How can I use more than 8192 rules?
+@node How can I use more than 8192 rules?, How do I abandon a file in the middle of a scan and switch to a new file?, How does flex compile the DFA so quickly?, FAQ
@unnumberedsec How can I use more than 8192 rules?
@code{Flex} is compiled with an upper limit of 8192 rules per scanner.
with 16-bit ints. I've given this advice to others in the past but haven't
heard back from them whether it worked okay or not...
-@node How do I abandon a file in the middle of a scan and switch to a new file?
+@node How do I abandon a file in the middle of a scan and switch to a new file?, How do I execute code only during initialization (only before the first scan)?, How can I use more than 8192 rules?, FAQ
@unnumberedsec How do I abandon a file in the middle of a scan and switch to a new file?
Just call @code{yyrestart(newfile)}. Be sure to reset the start state if you want a
``fresh start, since @code{yyrestart} does NOT reset the start state back to @code{INITIAL}.
-@node How do I execute code only during initialization (only before the first scan)?
+@node How do I execute code only during initialization (only before the first scan)?, How do I execute code at termination?, How do I abandon a file in the middle of a scan and switch to a new file?, FAQ
@unnumberedsec How do I execute code only during initialization (only before the first scan)?
You can specify an initial action by defining the macro @code{YY_USER_INIT} (though
@end verbatim
@end example
-@node How do I execute code at termination?
+@node How do I execute code at termination?, Where else can I find help?, How do I execute code only during initialization (only before the first scan)?, FAQ
@unnumberedsec How do I execute code at termination?
You can specify an action for the @code{<<EOF>>} rule.
-@node Where else can I find help?
+@node Where else can I find help?, Can I include comments in the "rules" section of the file?, How do I execute code at termination?, FAQ
@unnumberedsec Where else can I find help?
The @code{help-flex} email list is served by GNU. See @uref{http://www.gnu.org/} for
details on how to subscribe or search the archives.
-@node Can I include comments in the "rules" section of the file?
+@node Can I include comments in the "rules" section of the file?, I get an error about undefined yywrap()., Where else can I find help?, FAQ
@unnumberedsec Can I include comments in the "rules" section of the file?
Yes, just about anywhere you want to. See the manual for the specific syntax.
-@node I get an error about undefined yywrap().
+@node I get an error about undefined yywrap()., How can I change the matching pattern at run time?, Can I include comments in the "rules" section of the file?, FAQ
@unnumberedsec I get an error about undefined yywrap().
You must supply a @code{yywrap()} function of your own, or link to @file{libfl.a}
in your source to say you don't want a @code{yywrap()} function.
-@node How can I change the matching pattern at run time?
+@node How can I change the matching pattern at run time?, How can I expand macros in the input?, I get an error about undefined yywrap()., FAQ
@unnumberedsec How can I change the matching pattern at run time?
You can't, it's compiled into a static table when flex builds the scanner.
-@node How can I expand macros in the input?
+@node How can I expand macros in the input?, How can I build a two-pass scanner?, How can I change the matching pattern at run time?, FAQ
@unnumberedsec How can I expand macros in the input?
The best way to approach this problem is at a higher level, e.g., in the parser.
You probably will want a stack of expansion buffers to allow nested macros.
From the above though hopefully the idea is clear.
-@node How can I build a two-pass scanner?
+@node How can I build a two-pass scanner?, How do I match any string not matched in the preceding rules?, How can I expand macros in the input?, FAQ
@unnumberedsec How can I build a two-pass scanner?
One way to do it is to filter the first pass to a temporary file,
smaller, since everything is already classified, in binary format, and
residing in memory.
-@node How do I match any string not matched in the preceding rules?
+@node How do I match any string not matched in the preceding rules?, I am trying to port code from AT&T lex that uses yysptr and yysbuf., How can I build a two-pass scanner?, FAQ
@unnumberedsec How do I match any string not matched in the preceding rules?
One way to assign precedence, is to place the more specific rules first. If
@code{flex} scanner will pick the first rule listed in your scanner as the
one to match.
-@node I am trying to port code from AT&T lex that uses yysptr and yysbuf.
+@node I am trying to port code from AT&T lex that uses yysptr and yysbuf., Is there a way to make flex treat NULL like a regular character?, How do I match any string not matched in the preceding rules?, FAQ
@unnumberedsec I am trying to port code from AT&T lex that uses yysptr and yysbuf.
Those are internal variables pointing into the AT&T scanner's input buffer. I
@code{YY_INPUT}. You shouldn't need to (and must not) replace
@code{flex}'s @code{unput()} function.
-@node Is there a way to make flex treat NULL like a regular character?
+@node Is there a way to make flex treat NULL like a regular character?, Whenever flex can not match the input it says "flex scanner jammed"., I am trying to port code from AT&T lex that uses yysptr and yysbuf., FAQ
@unnumberedsec Is there a way to make flex treat NULL like a regular character?
Yes, @samp{\0} and @samp{\x00} should both do the trick. Perhaps you have an ancient
version of @code{flex}. The latest release is version @value{VERSION}.
-@node Whenever flex can not match the input it says "flex scanner jammed".
+@node Whenever flex can not match the input it says "flex scanner jammed"., Why doesnt flex have non-greedy operators like perl does?, Is there a way to make flex treat NULL like a regular character?, FAQ
@unnumberedsec Whenever flex can not match the input it says "flex scanner jammed".
You need to add a rule that matches the otherwise-unmatched text.
See @code{%option default} for more information.
-@node Why doesnt flex have non-greedy operators like perl does?
+@node Why doesnt flex have non-greedy operators like perl does?, Memory leak - 16386 bytes allocated by malloc., Whenever flex can not match the input it says "flex scanner jammed"., FAQ
@unnumberedsec Why doesn't flex have non-greedy operators like perl does?
A DFA can do a non-greedy match by stopping
This approach also has much better error-reporting properties.
-@node Memory leak - 16386 bytes allocated by malloc.
+@node Memory leak - 16386 bytes allocated by malloc., How do I track the byte offset for lseek()?, Why doesnt flex have non-greedy operators like perl does?, FAQ
@unnumberedsec Memory leak - 16386 bytes allocated by malloc.
@anchor{faq-memory-leak}
Note: @code{yy_init} is an "internal variable", and hasn't been tested in this
situation. It is possible that some other globals may need resetting as well.
-@node How do I track the byte offset for lseek()?
+@node How do I track the byte offset for lseek()?, How do I use my own I/O classes in a C++ scanner?, Memory leak - 16386 bytes allocated by malloc., FAQ
@unnumberedsec How do I track the byte offset for lseek()?
@example
(You need to be careful to update your bookkeeping if you use @code{yymore(}),
@code{yyless()}, @code{unput()}, or @code{input()}.)
-@c faq edit stopped here
-@c TODO: Evaluate this faq.
-@node unnamed-faq-16
-@unnumberedsec unnamed-faq-16
-@example
-@verbatim
-To: steves@telebase.com
-Subject: Re: flex C++ question
-In-reply-to: Your message of Thu, 08 Dec 94 13:10:58 EST.
-Date: Wed, 14 Dec 94 16:40:47 PST
-From: Vern Paxson <vern>
+@node How do I use my own I/O classes in a C++ scanner?, How do I skip as many chars as possible?, How do I track the byte offset for lseek()?, FAQ
+@section How do I use my own I/O classes in a C++ scanner?
-> We'd like to override the provided LexerInput() and LexerOutput()
-> functions, but we'd like to *not* use iostreams. Instead, we'd like
-> to use some of our own I/O classes. Is this possible?
+When the flex C++ scanning class rewrite finally happens, then this sort of thing should become much easier.
-You can do this by passing the various functions nil iostream*'s, and then
+@cindex LexerOutput, overriding
+@cindex LexerInput, overriding
+@cindex overriding LexerOutput
+@cindex overriding LexerInput
+@cindex customizing I/O in C++ scanners
+@cindex C++ I/O, customizing
+You can do this by passing the various functions (such as @code{LexerInput()}
+and @code{LexerOutput()}) nil @code{iostream*}'s, and then
dealing with your own I/O classes surreptitiously (i.e., stashing them in
special member variables). This works because the only assumption about
the lexer regarding what's done with the iostream's is that they're
-ultimately passed to LexerInput and LexerOutput, which then do whatever
-necessary with them.
+ultimately passed to @code{LexerInput()} and @code{LexerOutput}, which then do whatever
+is necessary with them.
-When the flex C++ scanning class rewrite finally happens (no date for this
-in sight), then this sort of thing should become much easier.
-
- Vern
-@end verbatim
-@end example
-
-@node How do I skip as many chars as possible?
+@c faq edit stopped here
+@node How do I skip as many chars as possible?, unnamed-faq-33, How do I use my own I/O classes in a C++ scanner?, FAQ
@unnumberedsec How do I skip as many chars as possible?
How do I skip as many chars as possible -- without interfering with the other
@end example
@c TODO: Evaluate this faq.
-@node unnamed-faq-33
+@node unnamed-faq-33, unnamed-faq-42, How do I skip as many chars as possible?, FAQ
@unnumberedsec unnamed-faq-33
@example
@verbatim
@end example
@c TODO: Evaluate this faq.
-@node unnamed-faq-42
+@node unnamed-faq-42, unnamed-faq-43, unnamed-faq-33, FAQ
@unnumberedsec unnamed-faq-42
@example
@verbatim
@end example
@c TODO: Evaluate this faq.
-@node unnamed-faq-43
+@node unnamed-faq-43, unnamed-faq-44, unnamed-faq-42, FAQ
@unnumberedsec unnamed-faq-43
@example
@verbatim
@end example
@c TODO: Evaluate this faq.
-@node unnamed-faq-44
+@node unnamed-faq-44, unnamed-faq-45, unnamed-faq-43, FAQ
@unnumberedsec unnamed-faq-44
@example
@verbatim
@end example
@c TODO: Evaluate this faq.
-@node unnamed-faq-45
+@node unnamed-faq-45, unnamed-faq-46, unnamed-faq-44, FAQ
@unnumberedsec unnamed-faq-45
@example
@verbatim
@end example
@c TODO: Evaluate this faq.
-@node unnamed-faq-46
+@node unnamed-faq-46, unnamed-faq-47, unnamed-faq-45, FAQ
@unnumberedsec unnamed-faq-46
@example
@verbatim
@end example
@c TODO: Evaluate this faq.
-@node unnamed-faq-47
+@node unnamed-faq-47, unnamed-faq-48, unnamed-faq-46, FAQ
@unnumberedsec unnamed-faq-47
@example
@verbatim
@end example
@c TODO: Evaluate this faq.
-@node unnamed-faq-48
+@node unnamed-faq-48, unnamed-faq-49, unnamed-faq-47, FAQ
@unnumberedsec unnamed-faq-48
@example
@verbatim
@end example
@c TODO: Evaluate this faq.
-@node unnamed-faq-49
+@node unnamed-faq-49, unnamed-faq-50, unnamed-faq-48, FAQ
@unnumberedsec unnamed-faq-49
@example
@verbatim
@end example
@c TODO: Evaluate this faq.
-@node unnamed-faq-50
+@node unnamed-faq-50, unnamed-faq-51, unnamed-faq-49, FAQ
@unnumberedsec unnamed-faq-50
@example
@verbatim
@end example
@c TODO: Evaluate this faq.
-@node unnamed-faq-51
+@node unnamed-faq-51, unnamed-faq-52, unnamed-faq-50, FAQ
@unnumberedsec unnamed-faq-51
@example
@verbatim
@end example
@c TODO: Evaluate this faq.
-@node unnamed-faq-52
+@node unnamed-faq-52, unnamed-faq-53, unnamed-faq-51, FAQ
@unnumberedsec unnamed-faq-52
@example
@verbatim
@end example
@c TODO: Evaluate this faq.
-@node unnamed-faq-53
+@node unnamed-faq-53, unnamed-faq-54, unnamed-faq-52, FAQ
@unnumberedsec unnamed-faq-53
@example
@verbatim
@end example
@c TODO: Evaluate this faq.
-@node unnamed-faq-54
+@node unnamed-faq-54, unnamed-faq-55, unnamed-faq-53, FAQ
@unnumberedsec unnamed-faq-54
@example
@verbatim
@end example
@c TODO: Evaluate this faq.
-@node unnamed-faq-55
+@node unnamed-faq-55, unnamed-faq-56, unnamed-faq-54, FAQ
@unnumberedsec unnamed-faq-55
@example
@verbatim
@end example
@c TODO: Evaluate this faq.
-@node unnamed-faq-56
+@node unnamed-faq-56, unnamed-faq-57, unnamed-faq-55, FAQ
@unnumberedsec unnamed-faq-56
@example
@verbatim
@end example
@c TODO: Evaluate this faq.
-@node unnamed-faq-57
+@node unnamed-faq-57, unnamed-faq-58, unnamed-faq-56, FAQ
@unnumberedsec unnamed-faq-57
@example
@verbatim
@end example
@c TODO: Evaluate this faq.
-@node unnamed-faq-58
+@node unnamed-faq-58, unnamed-faq-59, unnamed-faq-57, FAQ
@unnumberedsec unnamed-faq-58
@example
@verbatim
@end example
@c TODO: Evaluate this faq.
-@node unnamed-faq-59
+@node unnamed-faq-59, unnamed-faq-60, unnamed-faq-58, FAQ
@unnumberedsec unnamed-faq-59
@example
@verbatim
@end example
@c TODO: Evaluate this faq.
-@node unnamed-faq-60
+@node unnamed-faq-60, unnamed-faq-61, unnamed-faq-59, FAQ
@unnumberedsec unnamed-faq-60
@example
@verbatim
@end example
@c TODO: Evaluate this faq.
-@node unnamed-faq-61
+@node unnamed-faq-61, unnamed-faq-62, unnamed-faq-60, FAQ
@unnumberedsec unnamed-faq-61
@example
@verbatim
@end example
@c TODO: Evaluate this faq.
-@node unnamed-faq-62
+@node unnamed-faq-62, unnamed-faq-63, unnamed-faq-61, FAQ
@unnumberedsec unnamed-faq-62
@example
@verbatim
@end example
@c TODO: Evaluate this faq.
-@node unnamed-faq-63
+@node unnamed-faq-63, unnamed-faq-64, unnamed-faq-62, FAQ
@unnumberedsec unnamed-faq-63
@example
@verbatim
@end example
@c TODO: Evaluate this faq.
-@node unnamed-faq-64
+@node unnamed-faq-64, unnamed-faq-65, unnamed-faq-63, FAQ
@unnumberedsec unnamed-faq-64
@example
@verbatim
@end example
@c TODO: Evaluate this faq.
-@node unnamed-faq-65
+@node unnamed-faq-65, unnamed-faq-66, unnamed-faq-64, FAQ
@unnumberedsec unnamed-faq-65
@example
@verbatim
@end example
@c TODO: Evaluate this faq.
-@node unnamed-faq-66
+@node unnamed-faq-66, unnamed-faq-67, unnamed-faq-65, FAQ
@unnumberedsec unnamed-faq-66
@example
@verbatim
@end example
@c TODO: Evaluate this faq.
-@node unnamed-faq-67
+@node unnamed-faq-67, unnamed-faq-68, unnamed-faq-66, FAQ
@unnumberedsec unnamed-faq-67
@example
@verbatim
@end example
@c TODO: Evaluate this faq.
-@node unnamed-faq-68
+@node unnamed-faq-68, unnamed-faq-69, unnamed-faq-67, FAQ
@unnumberedsec unnamed-faq-68
@example
@verbatim
@end example
@c TODO: Evaluate this faq.
-@node unnamed-faq-69
+@node unnamed-faq-69, unnamed-faq-70, unnamed-faq-68, FAQ
@unnumberedsec unnamed-faq-69
@example
@verbatim
@end example
@c TODO: Evaluate this faq.
-@node unnamed-faq-70
+@node unnamed-faq-70, unnamed-faq-71, unnamed-faq-69, FAQ
@unnumberedsec unnamed-faq-70
@example
@verbatim
@end example
@c TODO: Evaluate this faq.
-@node unnamed-faq-71
+@node unnamed-faq-71, unnamed-faq-72, unnamed-faq-70, FAQ
@unnumberedsec unnamed-faq-71
@example
@verbatim
@end example
@c TODO: Evaluate this faq.
-@node unnamed-faq-72
+@node unnamed-faq-72, unnamed-faq-73, unnamed-faq-71, FAQ
@unnumberedsec unnamed-faq-72
@example
@verbatim
@end example
@c TODO: Evaluate this faq.
-@node unnamed-faq-73
+@node unnamed-faq-73, unnamed-faq-74, unnamed-faq-72, FAQ
@unnumberedsec unnamed-faq-73
@example
@verbatim
@end example
@c TODO: Evaluate this faq.
-@node unnamed-faq-74
+@node unnamed-faq-74, unnamed-faq-75, unnamed-faq-73, FAQ
@unnumberedsec unnamed-faq-74
@example
@verbatim
@end example
@c TODO: Evaluate this faq.
-@node unnamed-faq-75
+@node unnamed-faq-75, unnamed-faq-76, unnamed-faq-74, FAQ
@unnumberedsec unnamed-faq-75
@example
@verbatim
@end example
@c TODO: Evaluate this faq.
-@node unnamed-faq-76
+@node unnamed-faq-76, unnamed-faq-77, unnamed-faq-75, FAQ
@unnumberedsec unnamed-faq-76
@example
@verbatim
@end example
@c TODO: Evaluate this faq.
-@node unnamed-faq-77
+@node unnamed-faq-77, unnamed-faq-78, unnamed-faq-76, FAQ
@unnumberedsec unnamed-faq-77
@example
@verbatim
@end example
@c TODO: Evaluate this faq.
-@node unnamed-faq-78
+@node unnamed-faq-78, unnamed-faq-79, unnamed-faq-77, FAQ
@unnumberedsec unnamed-faq-78
@example
@verbatim
@end example
@c TODO: Evaluate this faq.
-@node unnamed-faq-79
+@node unnamed-faq-79, unnamed-faq-80, unnamed-faq-78, FAQ
@unnumberedsec unnamed-faq-79
@example
@verbatim
@end example
@c TODO: Evaluate this faq.
-@node unnamed-faq-80
+@node unnamed-faq-80, unnamed-faq-81, unnamed-faq-79, FAQ
@unnumberedsec unnamed-faq-80
@example
@verbatim
@end example
@c TODO: Evaluate this faq.
-@node unnamed-faq-81
+@node unnamed-faq-81, unnamed-faq-82, unnamed-faq-80, FAQ
@unnumberedsec unnamed-faq-81
@example
@verbatim
@end example
@c TODO: Evaluate this faq.
-@node unnamed-faq-82
+@node unnamed-faq-82, unnamed-faq-83, unnamed-faq-81, FAQ
@unnumberedsec unnamed-faq-82
@example
@verbatim
@end example
@c TODO: Evaluate this faq.
-@node unnamed-faq-83
+@node unnamed-faq-83, unnamed-faq-84, unnamed-faq-82, FAQ
@unnumberedsec unnamed-faq-83
@example
@verbatim
@end example
@c TODO: Evaluate this faq.
-@node unnamed-faq-84
+@node unnamed-faq-84, unnamed-faq-85, unnamed-faq-83, FAQ
@unnumberedsec unnamed-faq-84
@example
@verbatim
@end example
@c TODO: Evaluate this faq.
-@node unnamed-faq-85
+@node unnamed-faq-85, unnamed-faq-86, unnamed-faq-84, FAQ
@unnumberedsec unnamed-faq-85
@example
@verbatim
@end example
@c TODO: Evaluate this faq.
-@node unnamed-faq-86
+@node unnamed-faq-86, unnamed-faq-87, unnamed-faq-85, FAQ
@unnumberedsec unnamed-faq-86
@example
@verbatim
@end example
@c TODO: Evaluate this faq.
-@node unnamed-faq-87
+@node unnamed-faq-87, unnamed-faq-88, unnamed-faq-86, FAQ
@unnumberedsec unnamed-faq-87
@example
@verbatim
@end example
@c TODO: Evaluate this faq.
-@node unnamed-faq-88
+@node unnamed-faq-88, unnamed-faq-90, unnamed-faq-87, FAQ
@unnumberedsec unnamed-faq-88
@example
@verbatim
@end example
@c TODO: Evaluate this faq.
-@node unnamed-faq-90
+@node unnamed-faq-90, unnamed-faq-91, unnamed-faq-88, FAQ
@unnumberedsec unnamed-faq-90
@example
@verbatim
@end example
@c TODO: Evaluate this faq.
-@node unnamed-faq-91
+@node unnamed-faq-91, unnamed-faq-92, unnamed-faq-90, FAQ
@unnumberedsec unnamed-faq-91
@example
@verbatim
@end example
@c TODO: Evaluate this faq.
-@node unnamed-faq-92
+@node unnamed-faq-92, unnamed-faq-93, unnamed-faq-91, FAQ
@unnumberedsec unnamed-faq-92
@example
@verbatim
@end example
@c TODO: Evaluate this faq.
-@node unnamed-faq-93
+@node unnamed-faq-93, unnamed-faq-94, unnamed-faq-92, FAQ
@unnumberedsec unnamed-faq-93
@example
@verbatim
@end example
@c TODO: Evaluate this faq.
-@node unnamed-faq-94
+@node unnamed-faq-94, unnamed-faq-95, unnamed-faq-93, FAQ
@unnumberedsec unnamed-faq-94
@example
@verbatim
@end example
@c TODO: Evaluate this faq.
-@node unnamed-faq-95
+@node unnamed-faq-95, unnamed-faq-96, unnamed-faq-94, FAQ
@unnumberedsec unnamed-faq-95
@example
@verbatim
@end example
@c TODO: Evaluate this faq.
-@node unnamed-faq-96
+@node unnamed-faq-96, unnamed-faq-97, unnamed-faq-95, FAQ
@unnumberedsec unnamed-faq-96
@example
@verbatim
@end example
@c TODO: Evaluate this faq.
-@node unnamed-faq-97
+@node unnamed-faq-97, unnamed-faq-98, unnamed-faq-96, FAQ
@unnumberedsec unnamed-faq-97
@example
@verbatim
@end example
@c TODO: Evaluate this faq.
-@node unnamed-faq-98
+@node unnamed-faq-98, unnamed-faq-99, unnamed-faq-97, FAQ
@unnumberedsec unnamed-faq-98
@example
@verbatim
@end example
@c TODO: Evaluate this faq.
-@node unnamed-faq-99
+@node unnamed-faq-99, unnamed-faq-100, unnamed-faq-98, FAQ
@unnumberedsec unnamed-faq-99
@example
@verbatim
@end example
@c TODO: Evaluate this faq.
-@node unnamed-faq-100
+@node unnamed-faq-100, unnamed-faq-101, unnamed-faq-99, FAQ
@unnumberedsec unnamed-faq-100
@example
@verbatim
@end example
@c TODO: Evaluate this faq.
-@node unnamed-faq-101
+@node unnamed-faq-101, , unnamed-faq-100, FAQ
@unnumberedsec unnamed-faq-101
@example
@verbatim
@end verbatim
@end example
-@node Appendices
+@node Appendices, Indices, FAQ, Top
@appendix Appendices
@menu
* Bison Bridge::
@end menu
-@node Makefiles and Flex
+@node Makefiles and Flex, Bison Bridge, Appendices, Appendices
@appendixsec Makefiles and Flex
@cindex Makefile, syntax
For more details on writing Makefiles, see @ref{Top, , , make, The
GNU Make Manual}.
-@node Bison Bridge
+@node Bison Bridge, , Makefiles and Flex, Appendices
@section C Scanners with Bison Parsers
@cindex bison, bridging with flex
@end example
-@node Indices
+@node Indices, , Appendices, Top
@unnumbered Indices
@menu
* Index of Data Types::
* Index of Hooks::
* Index of Examples::
-* Index of Scanner Options::
+* Index of Scanner Options::
@end menu
-@node Concept Index
+@node Concept Index, Index of Functions and Macros, Indices, Indices
@unnumberedsec Concept Index
@cindex beginning of line -- see BOL
@cindex ^ -- see BOL
@cindex bison, see also yacc
@printindex cp
-@node Index of Functions and Macros
+@node Index of Functions and Macros, Index of Variables, Concept Index, Indices
@unnumberedsec Index of Functions and Macros
This is an index of functions and preprocessor macros that look like functions.
@printindex fn
-@node Index of Variables
+@node Index of Variables, Index of Data Types, Index of Functions and Macros, Indices
@unnumberedsec Index of Variables
This is an index of variables, constants, and preprocessor macros
@printindex vr
-@node Index of Data Types
+@node Index of Data Types, Index of Hooks, Index of Variables, Indices
@unnumberedsec Index of Data Types
@printindex tp
-@node Index of Hooks
+@node Index of Hooks, Index of Examples, Index of Data Types, Indices
@unnumberedsec Index of Hooks
This is an index of "hooks" that the user may define. These hooks typically correspond
@printindex hk
-@node Index of Examples
+@node Index of Examples, Index of Scanner Options, Index of Hooks, Indices
@unnumberedsec Index of Examples
@printindex ex
-@node Index of Scanner Options
+@node Index of Scanner Options, , Index of Examples, Indices
@unnumberedsec Index of Scanner Options
@printindex op
projects / flex / commitdiff