From 8f85e8e46305393f63426ba1b3a4a42f7c5c9816 Mon Sep 17 00:00:00 2001 From: Vern Paxson Date: Thu, 28 Jun 1990 00:40:34 +0000 Subject: [PATCH] Fixed bug in mini-scanner examle Fixed bug in YY_INPUT redefinition yylineno defense reentrancy documentation Something else which I forget. --- flex.1 | 158 ++++++++++++++++++++++++++++++++++++++++----------------- 1 file changed, 111 insertions(+), 47 deletions(-) diff --git a/flex.1 b/flex.1 index ffa589a..cef3d7d 100644 --- a/flex.1 +++ b/flex.1 @@ -105,7 +105,7 @@ A somewhat more complicated example: } {DIGIT}+"."{DIGIT}* { - printf( "A float: %s (%d)\\n", yytext, + printf( "A float: %s (%g)\\n", yytext, atof( yytext ) ); } @@ -786,7 +786,10 @@ section of the input file): %{ #undef YY_INPUT #define YY_INPUT(buf,result,max_size) \\ - result = ((buf[0] = getchar()) == EOF) ? YY_NULL : 1; + { \\ + int c = getchar(); \\ + result = (c == EOF) ? YY_NULL : (buf[0] = c, 1); \\ + } %} .fi @@ -2041,6 +2044,99 @@ is fully compatible with .I lex with the following exceptions: .IP - +The undocumented +.I lex +scanner internal variable +.B yylineno +is not supported. It is difficult to support this option efficiently, +since it requires examining every character scanned and reexamining +the characters when the scanner backs up. +Things get more complicated when the end of buffer or file is reached or a +NUL is scanned (since the scan must then be restarted with the proper line +number count), or the user uses the yyless(), unput(), or REJECT actions, +or the multiple input buffer functions. +.IP +The fix is to add rules which, upon seeing a newline, increment +yylineno. This is usually an easy process, though it can be a drag if some +of the patterns can match multiple newlines along with other characters. +.IP +yylineno is not part of the POSIX draft. +.IP - +The +.B input() +routine is not redefinable, though it may be called to read characters +following whatever has been matched by a rule. If +.B input() +encounters an end-of-file the normal +.B yywrap() +processing is done. A ``real'' end-of-file is returned by +.B input() +as +.I EOF. +.IP +Input is instead controlled by redefining the +.B YY_INPUT +macro. +.IP +The +.I flex +restriction that +.B input() +cannot be redefined is in accordance with the POSIX draft, but +.B YY_INPUT +has not yet been accepted into the draft (and probably won't; it looks +like the draft will simply not specify any way of controlling the +scanner's input other than by making an initial assignment to +.I yyin). +.IP - +.I flex +scanners do not use stdio for input. Because of this, when writing an +interactive scanner one must explicitly call fflush() on the +stream associated with the terminal after writing out a prompt. +With +.I lex +such writes are automatically flushed since +.I lex +scanners use +.B getchar() +for their input. Also, when writing interactive scanners with +.I flex, +the +.B -I +flag must be used. +.IP - +.I flex +scanners are not as reentrant as +.I lex +scanners. In particular, if you have an interactive scanner and +an interrupt handler which long-jumps out of the scanner, and +the scanner is subsequently called again, you may get the following +message: +.nf + + fatal flex scanner internal error--end of buffer missed + +.fi +To reenter the scanner, first use +.nf + + yyrestart( yyin ); + +.fi +.IP - +.B output() +is not supported. +Output from the +.B ECHO +macro is done to the file-pointer +.I yyout +(default +.I stdout). +.IP +The POSIX draft mentions that an +.B output() +routine exists but currently gives no details as to what it does. +.IP - .I lex does not support exclusive start conditions (%x), though they are in the current POSIX draft. @@ -2084,49 +2180,6 @@ one must use "[^\\]]". The latter works with .I lex, too. .IP - -The undocumented -.I lex -scanner internal variable -.B yylineno -is not supported. (The variable is not part of the POSIX draft.) -.IP - -The -.B input() -routine is not redefinable, though it may be called to read characters -following whatever has been matched by a rule. If -.B input() -encounters an end-of-file the normal -.B yywrap() -processing is done. A ``real'' end-of-file is returned by -.B input() -as -.I EOF. -.IP -Input is instead controlled by redefining the -.B YY_INPUT -macro. -.IP -The -.I flex -restriction that -.B input() -cannot be redefined is in accordance with the POSIX draft, but -.B YY_INPUT -has not yet been accepted into the draft. -.IP - -.B output() -is not supported. -Output from the -.B ECHO -macro is done to the file-pointer -.I yyout -(default -.I stdout). -.IP -The POSIX draft mentions that an -.B output() -routine exists but currently gives no details as to what it does. -.IP - The .I lex .B %r @@ -2138,7 +2191,7 @@ If you are providing your own yywrap() routine, you must include a the "#undef" will have to be enclosed in %{}'s. .IP The POSIX draft -specifies that yywrap() is a function and this is unlikely to change; so +specifies that yywrap() is a function and this is very unlikely to change; so .I flex users are warned that .B yywrap() @@ -2313,6 +2366,17 @@ Your scanner specification includes recognizing 8-bit characters and you did not specify the -8 flag (and your site has not installed flex with -8 as the default). .LP +.I +fatal flex scanner internal error--end of buffer missed - +This can occur in an scanner which is reentered after a long-jump +has jumped out (or over) the scanner's activation frame. Before +reentering the scanner, use: +.nf + + yyrestart( yyin ); + +.fi +.LP .I too many %t classes! - You managed to put every single character into its own %t class. .I flex @@ -2334,7 +2398,7 @@ Jacobson. The implementation was done by Kevin Gong and Vern Paxson. Thanks to the many .I flex beta-testers, feedbackers, and contributors, especially Casey -Leedom, benson@odi.com, +Leedom, benson@odi.com, Keith Bostic, Frederic Brehm, Nick Christopher, Jason Coughlin, Scott David Daniels, Leo Eskin, Chris Faylor, Eric Goldman, Eric -- 2.40.0