&& { git archive --remote=. remotes/$remote/master doc/manual \
| tar -C src/ --strip-components=1 -xpf -; } \
&& { git archive --remote=. remotes/$remote/master examples/*.re \
- | tar -C src/ --strip-components=0 -xpf -; }
+ | tar -C src/ -xpf -; } \
+ && { git archive --remote=. remotes/$remote/master CHANGELOG \
+ | tar -C src/releases/changelog/ -xpf -; } \
+ && mv src/releases/changelog/CHANGELOG src/releases/changelog/changelog.rst
sphinx-build -b html src obj
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
extensions = [
- 'sphinx.ext.imgmath'
+ 'sphinx.ext.imgmath',
+ 'sphinx.ext.autosectionlabel'
]
imgmath_use_preview = True # to find the baseline of surrounding text
+autosectionlabel_prefix_document = True
# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']
.. toctree::
:hidden:
- User manual <manual/manual>
- Download & build <install/install>
- Releases <news/news>
+ User manual </manual/manual>
+ Build instructions </install/install>
+ Release notes </releases/release_notes>
+ Changelog </releases/changelog/changelog>
Re2c is a free and open-source lexer generator for C and C++.
The main goal of the project is to generate *very fast* lexers that match or
which does not requre the overhead on full parsing --- a feature that is rarely found in the wild.
Re2c is used by many other projects
(such as
-`PHP <http://php.net/>`_,
+`php <http://php.net/>`_,
`ninja <https://ninja-build.org/>`_,
`yasm <http://yasm.tortall.net/>`_,
-`spamassassin <https://spamassassin.apache.org/>`_ and
-`BRL-CAD <http://brlcad.org/>`_)
+`spamassassin <https://spamassassin.apache.org/>`_,
+`BRL-CAD <http://brlcad.org/>`_
+and `wake <https://github.com/sifive/wake>`_)
and aims at being fully backward compatible.
On the other hand, it is a research project and a playground for the development of new algorithms in the field of formal grammars and automata.
See the `user manual <manual/manual.html>`_ for a complete overview with
examples.
+Download
+--------
+
+You can get the
+`latest release <https://github.com/skvadrik/re2c/releases/latest>`_ on Github,
+as well as the `older releases <https://github.com/skvadrik/re2c/releases>`_
+(make sure you download the latest minor version in each series).
+Many Linux distributions and other systems provide their own packages.
+Re2c source code is hosted on both Github
+(`<https://github.com/skvadrik/re2c>`_) and SourceForge
+(`<https://sourceforge.net/p/re2c>`_).
+Github serves as the main repository, bugtracker and tarball hosting.
+SourceForge is used as a backup repository and email hosting.
+
Bugs & patches
--------------
-The best place to report a bug is `on github <https://github.com/skvadrik/re2c>`_
-or in `re2c-devel <re2c-devel@lists.sourceforge.net>`_ and `re2c-general <re2c-general@lists.sourceforge.net>`_ mailing lists.
-Note that re2c is hosted on both `github <https://github.com/skvadrik/re2c>`_
-and `sourceforge <https://sourceforge.net/p/re2c>`_:
-github serves as the main repository, bugtracker, and tarball hosting;
-sourceforge is used as a backup repository and for mail.
-All contributions are welcome!
+Please send bugs reports, patches and other feedback to `github issue tracker
+<https://github.com/skvadrik/re2c>`_ or email them to
+`re2c-devel@lists.sourceforge.net <re2c-devel@lists.sourceforge.net>`_ and
+`re2c-general@lists.sourceforge.net <re2c-general@lists.sourceforge.net>`_
+mailing lists.
+Re2c has an IRC channel ``#re2c`` on `freenode <https://freenode.net>`_.
+Re2c developers are happy to answer questions and provide help.
+Contributions are always welcome!
Papers
------
-Re2c is a research project. It is described in the following papers:
-
- `"RE2C: a more versatile scanner generator"
<1994_bumbulis_cowan_re2c_a_more_versatile_scanner_generator.ps>`_
by Peter Bumbulis and Donald D. Cowan,
- `"Efficient POSIX submatch extraction on NFA"
<2019_borsotti_trofimovich_efficient_posix_submatch_extraction_on_nfa.pdf>`_
by Angelo Borsotti and Ulya Trofimovich,
- 2017
+ 2019
Authors
-------
-Re2c was originally written by Peter Bumbulis (peter@csg.uwaterloo.ca)
-and described in research article `"RE2C: a more versatile scanner generator"
-<1994_bumbulis_cowan_re2c_a_more_versatile_scanner_generator.ps>`_
-by Peter Bumbulis and Donald Cowan, 1994.
-Since then re2c has been maintained and developed by multiple volunteers,
+Re2c was originally written by Peter Bumbulis (peter@csg.uwaterloo.ca) in 1993.
+Since then it has been maintained and developed by multiple volunteers,
most notably,
Brian Young (bayoung@acm.org),
Markus Boerger (helly@users.sourceforge.net),
Dan Nuffer (nuffer@users.sourceforge.net)
and Ulya Trofimovich (skvadrik@gmail.com).
-Here is a list of re2c contributors (sorry if anyone is missing):
-Brian Young,
-Dan Nuffer,
+Other re2c contributors are
Derick Rethans,
Emmanuel Mogenet,
Hartmut Kaiser,
jcfp,
joscherl,
-Markus Boerger,
Mike Gilbert,
Nerd,
nuno-lopes,
Ross Burton,
Ryan Mast,
Serghei Iakovlev,
-Sergei Trofimovich,
-Tim Kelly,
-Ulya Trofimovich.
+Sergei Trofimovich
+and Tim Kelly
+(apologies if someone is missing).
License
-------
-================
-Download & build
-================
+==================
+Build instructions
+==================
.. toctree::
:hidden:
-Download
-========
-
-Below is the list of the latest (stable) tarballs for each branch.
-You can find other releases `here <https://sourceforge.net/projects/re2c/files/old/>`_
-(but be aware that they are hidden for a good reason:
-most of them contain bugs that have been fixed in the next minor release).
-
-* `re2c-1.1.1.tar.gz <https://github.com/skvadrik/re2c/releases/download/1.1.1/re2c-1.1.1.tar.gz>`_
-* `re2c-1.0.3.tar.gz <https://github.com/skvadrik/re2c/releases/download/1.0.3/re2c-1.0.3.tar.gz>`_
-* `re2c-0.16.tar.gz <https://github.com/skvadrik/re2c/releases/download/0.16/re2c-0.16.tar.gz>`_
-* `re2c-0.15.3.tar.gz <https://github.com/skvadrik/re2c/releases/download/0.15.3/re2c-0.15.3.tar.gz>`_
-* `re2c-0.14.3.tar.gz <https://github.com/skvadrik/re2c/releases/download/0.14.3/re2c-0.14.3.tar.gz>`_
-* `re2c-0.13.7.5.tar.gz <https://github.com/skvadrik/re2c/releases/download/0.13.7.5/re2c-0.13.7.5.tar.gz>`_
-* `re2c-0.13.6.tar.gz <https://github.com/skvadrik/re2c/releases/download/0.13.6/re2c-0.13.6.tar.gz>`_
-* `re2c-0.13.5.tar.gz <http://sourceforge.net/projects/re2c/files/re2c/0.13.5/re2c-0.13.5.tar.gz/download>`_
+Dependencies
+============
-Source files are `on github <https://github.com/skvadrik/re2c>`_:
+Re2c users need only a C++ compiler to build re2c from a release tarball, and
+optionally Bash if they want to run the tests.
-.. code-block:: bash
+Re2c developers also need Autotools, Bison (if they change parser code), rst2man
+(if they change documentation), Sphinx (if they change this website), Mingw and
+Wine (if they test builds for Windows) and Libtool (if they change the
+experimental libre2c library). A few occasional helper scripts are written in
+Haskell (but they are not necessary for re2c development).
+Re2c is a bootstrapping project with all the consequences.
- $ git clone https://github.com/skvadrik/re2c.git re2c
+Building
+========
-There's also a mirror on `sourceforge <https://sourceforge.net/p/re2c/code-git/ci/master/tree/>`_
-(should be in sync with github):
+If you are building re2c from repository, *not* from a release tarball, first of
+all you should run Autotools:
.. code-block:: bash
- $ git clone https://git.code.sf.net/p/re2c/code-git re2c
-
-Many distributions and systems provide their own packages:
-`Alt <http://www.sisyphus.ru/ru/srpm/Sisyphus/re2c>`_,
-`Apple Mac OS X <http://macappstore.org/re2c/>`_,
-`Arch <https://www.archlinux.org/packages/extra/x86_64/re2c>`_,
-`Debian <https://packages.debian.org/search?keywords=re2c>`_,
-`Gentoo <https://packages.gentoo.org/packages/dev-util/re2c>`_,
-`Fedora <http://rpmfind.net/linux/rpm2html/search.php?query=re2c&system=fedora>`_,
-`FreeBSD <http://www.freebsd.org/cgi/ports.cgi?query=re2c>`_,
-`Mageia <http://rpmfind.net/linux/rpm2html/search.php?query=re2c&system=mageia>`_,
-`Mandriva <http://rpmfind.net/linux/rpm2html/search.php?query=re2c&system=mandriva>`_,
-`NetBSD <ftp://ftp.netbsd.org/pub/pkgsrc/current/pkgsrc/devel/re2c/README.html>`_,
-`OpenSuSE <http://rpmfind.net/linux/rpm2html/search.php?query=re2c&system=opensuse>`_,
-`RedHat <http://rpmfind.net/linux/rpm2html/search.php?query=re2c&system=dag>`_,
-`Slackware <http://slackbuilds.org/repository/14.1/development/re2c/?search=re2c>`_,
-`Ubuntu <http://packages.ubuntu.com/search?keywords=re2c>`_.
-
-Build
-=====
-
-The only required build-time dependency for re2c is a C++98 compiler.
-Optional build-time dependencies are the following:
-autotools (to build re2c from source, not from distribution tarball),
-bison (to rebuild re2c parser)
-and rst2man (to rebuild documentation).
-
-If you are building re2c from source, first of all you should run autoconf and automake:
+ $ autoreconf -i -W all
+
+The simplest possible build is an in-tree build (will install re2c to
+``$RE2C_PATH`` location):
.. code-block:: bash
- $ ./autogen.sh
+ $ ./configure --prefix=$RE2C_PATH
+ $ make
+ $ make install
-The simplest possible build is an in-tree build.
-The following instructions will install re2c (binary and manpage) to ``_p_r_e_f_i_x_`` (by default ``/usr/local``):
+Out-of-tree build (puts build artifacts in a separate directory):
.. code-block:: bash
- $ ./configure --prefix=_p_r_e_f_i_x_
+ $ mkdir $BUILD_DIR
+ $ cd $BUILD_DIR
+ $ ../configure --prefix=$RE2C_PATH
$ make
$ make install
-Since re2c is a self-hosting lexer generator (some parts of re2c are written in re2c), it needs to be bootstrapped.
-Bootstrapping files are packaged into the re2c distribution, so that re2c can be built on a system without re2c.
-The following instructions bootstrap re2c from scratch and update precompiled files:
+Bootstrapping build (rebuilds re2c with the freshly built re2c and updates
+precompiled bootstrap files):
.. code-block:: bash
$ make bootstrap
-re2c supports out-of-tree builds:
+Building on Windows is more tricky. The best option is to use
+`Cygwin <https://cygwin.com/>`_, or build re2c with `Mingw <http://mingw.org/>`_
+(Mingw builds are supported and tested regularly).
+For some Mingw versions you might have to use ``-std=gnu++11`` compiler option
+to avoid spurious build errors (re2c uses ``-std=c++98`` by default).
.. code-block:: bash
- $ mkdir builddir && cd builddir
- $ ../configure
- $ make
- $ make install
+ $ ./configure --host i686-w64-ming 32 CXXFLAGS="-std=gnu++11"
-If you intend to use re2c on Windows, you can either
-use `cygwin <https://cygwin.com/>`_
-or build re2c with `mingw <http://mingw.org/>`_
-(mingw builds are supported and tested regularly).
-For some mingw versions you might have to use ``-std=gnu++11`` compiler option to avoid spurious build errors
-(re2c uses ``-std=c++98`` by default).
+To build documentation:
.. code-block:: bash
- $ ./configure --host i686-w64-mingw32 CXXFLAGS="-std=gnu++11"
- $ make
- $ make install
+ $ ./configure --enable-docs
-To rebuild re2c documentation:
+To enable debug:
.. code-block:: bash
- $ ./configure --enable-docs
- $ make docs
+ $ ./configure --enable-debug
+
+To build the experimental libre2c library:
+
+.. code-block:: bash
+ $ ./configure --enable-libs
-Test
-====
+Re2c provides a bunch of build scripts ``__build_*.sh`` that can be used
+for specialized builds with ``GLIBCXX_DEBUG``, AddressSanitizer, etc.
-re2c has a main test suite and a couple of small tests. Run them all:
+Testing
+=======
+
+Re2c has a main test suite and a couple of small unit tests. Run them all:
.. code-block:: bash
$ make check
-Or run only the main test suite (and watch progress dumped to ``stdout``):
+Run only the main test suite and watch the progress dumped on ``stdout``:
+
+.. code-block:: bash
+
+ $ ./run_tests.sh -j<N>
+
+Run the main test suite with ``--skeleton`` re2c option:
.. code-block:: bash
- $ make tests
+ $ ./run_tests.sh --skeleton
-Run the test suite under `valgrind <http://valgrind.org/>`_ (takes a long time):
+Run the test suite under Valgrind (takes some time and memory):
.. code-block:: bash
$ make vtests
-Test mingw builds with `wine <https://www.winehq.org/>`_:
+Test mingw builds with Wine:
.. code-block:: bash
$ make distcheck
+Re2c provides a helper script ``__alltest.sh`` that builds and tests various
+re2c build flavours with ``GLIBCXX_DEBUG``, AddressSanitizer, etc.
+There is a couple of fuzz-testing Haskell scripts in the ``fuzz`` subdirectory;
+they are based on the QuickCheck library and can be easily modified to fuzz-test
+various aspects of re2c by comparing current re2c version against older versions
+or against other regular expression libraries.
``-? -h --help``
Show help message.
+``-1 --single-pass``
+ Deprecated. Does nothing (single pass is the default now).
+
+``-8 --utf-8``
+ Generate a lexer that reads input in UTF-8 encoding.
+ re2c assumes that character range is 0 -- 0x10FFFF and character size is
+ 1 byte.
+
``-b --bit-vectors``
Optimize conditional jumps using bit masks. Implies ``-s``.
``-c --conditions --start-conditions``
- Enable support of Flex-like "conditions": multiple interrelated lexers within one block.
- Option ``--start-conditions`` is a legacy alias; use ``--conditions`` instead.
+ Enable support of Flex-like "conditions": multiple interrelated lexers
+ within one block. Option ``--start-conditions`` is a legacy alias; use
+ ``--conditions`` instead.
-``-d --debug-output``
- Emit ``YYDEBUG`` in the generated code.
- ``YYDEBUG`` should be defined by the user in the form of a void function with two parameters:
- ``state`` (lexer state or -1) and ``symbol`` (current input symbol of type ``YYCTYPE``).
+``--case-insensitive``
+ Treat single-quoted and double-quoted strings as case-insensitive.
+
+``--case-inverted``
+ Invert the meaning of single-quoted and double-quoted strings:
+ treat single-quoted strings as case-sensitive and double-quoted strings
+ as case-insensitive.
``-D --emit-dot``
- Instead of normal output generate lexer graph in DOT format.
- The output can be converted to PNG with the help of Graphviz (something like ``dot -Tpng -odfa.png dfa.dot``).
- Note that large graphs may crash Graphviz.
+ Instead of normal output generate lexer graph in .dot format.
+ The output can be converted to an image with the help of Graphviz
+ (e.g. something like ``dot -Tpng -odfa.png dfa.dot``).
-``-e --ecb``
- Generate a lexer that reads input in EBCDIC encoding.
- ``re2c`` assumes that character range is 0 -- 0xFF an character size is 1 byte.
+``-d --debug-output``
+ Emit ``YYDEBUG`` in the generated code.
+ ``YYDEBUG`` should be defined by the user in the form of a void function
+ with two parameters: ``state`` (lexer state or -1) and ``symbol`` (current
+ input symbol of type ``YYCTYPE``).
-``-f --storable-state``
- Generate a lexer which can store its inner state.
- This is useful in push-model lexers which are stopped by an outer program when there is not enough input,
- and then resumed when more input becomes available.
- In this mode users should additionally define
- ``YYGETSTATE ()`` and ``YYSETSTATE (state)`` macros
- and variables ``yych``, ``yyaccept`` and the ``state`` as part of the lexer state.
+``--dfa-minimization <moore | table>``
+ The internal algorithm used by re2c to minimize the DFA: ``moore`` (the
+ default) is Moore algorithm, and ``table`` is the "table filling" algorithm.
+ Both algorithms should produce the same DFA up to states relabeling; table
+ filling is simpler and much slower and serves as a reference implementation.
-``-F --flex-syntax``
- Partial support for Flex syntax:
- in this mode named definitions don't need the equal sign and the terminating semicolon,
- and when used they must be surrounded by curly braces.
- Names without curly braces are treated as double-quoted strings.
+``--dump-adfa``
+ Debug option: output DFA after tunneling (in .dot format).
-``-g --computed-gotos``
- Optimize conditional jumps using non-standard "computed goto" extension (must be supported by C/C++ compiler).
- ``re2c`` generates jump tables only in complex cases with a lot of conditional branches.
- Complexity threshold can be configured with ``cgoto:threshold`` configuration.
- This option implies ``-b``.
+``--dump-cfg``
+ Debug option: output control flow graph of tag variables (in .dot format).
-``-i --no-debug-info``
- Do not output ``#line`` information.
- This is useful when the generated code is tracked by some version control system.
+``--dump-closure-stats``
+ Debug option: output statistics on the number of states in closure.
-``-o OUTPUT --output=OUTPUT``
- Specify the ``OUTPUT`` file.
+``--dump-dfa-det``
+ Debug option: output DFA immediately after determinization (in .dot format).
-``-r --reusable``
- Allows reuse of ``re2c`` rules with ``/*!rules:re2c */`` and ``/*!use:re2c */`` blocks.
- In this mode simple ``/*!re2c */`` blocks are not allowed
- and exactly one ``/*!rules:re2c */`` block must be present.
- The rules are saved and used by every ``/*!use:re2c */`` block that follows (which may add rules of their own).
- This option allows to reuse the same set of rules with different configurations.
+``--dump-dfa-min``
+ Debug option: output DFA after minimization (in .dot format).
-``-s --nested-ifs``
- Use nested ``if`` statements instead of ``switch`` statements in conditional jumps.
- This usually results in more efficient code with non-optimizing C/C++ compilers.
+``--dump-dfa-tagopt``
+ Debug option: output DFA after tag optimizations (in .dot format).
-``-t HEADER --type-header=HEADER``
- Generate a ``HEADER`` file that contains enum with condition names.
- Requires ``-c`` option.
+``--dump-dfa-raw``
+ Debug option: output DFA under construction with expanded state-sets
+ (in .dot format).
-``-T --tags``
- Enable submatch extraction with tags.
+``--dump-interf``
+ Debug option: output interference table produced by liveness analysis of tag
+ variables.
-``-P --posix-captures``
- Enable submatch extraction with POSIX-style capturing groups.
+``--dump-nfa``
+ Debug option: output NFA (in .dot format).
-``-u --unicode``
- Generate a lexer that reads input in UTF-32 encoding.
- ``re2c`` assumes that character range is 0 -- 0x10FFFF and character size is 4 bytes.
- Implies ``-s``.
+``-e --ecb``
+ Generate a lexer that reads input in EBCDIC encoding.
+ re2c assumes that character range is 0 -- 0xFF an character size is 1 byte.
-``-v --version``
- Show version information.
+``--eager-skip``
+ Make the generated lexer advance the input position "eagerly":
+ immediately after reading input symbol.
+ By default this happens after transition to the next state.
+ Implied by ``--no-lookahead``.
-``-V --vernum``
- Show version information in ``MMmmpp`` format (major, minor, patch).
+``--empty-class <match-empty | match-none | error>``
+ Define the way re2c treats empty character classes. With ``match-empty``
+ (the default) empty class matches empty input (which is illogical, but
+ backwards-compatible). With``match-none`` empty class always fails to match.
+ With ``error`` empty class raises a compilation error.
-``-w --wide-chars``
- Generate a lexer that reads input in UCS-2 encoding.
- ``re2c`` assumes that character range is 0 -- 0xFFFF and character size is 2 bytes.
- Implies ``-s``.
+``--encoding-policy <fail | substitute | ignore>``
+ Define the way re2c treats Unicode surrogates.
+ With ``fail`` re2c aborts with an error when a surrogate is encountered.
+ With ``substitute`` re2c silently replaces surrogates with the error code
+ point 0xFFFD. With ``ignore`` (the default) re2c treats surrogates as
+ normal code points. The Unicode standard says that standalone surrogates
+ are invalid, but real-world libraries and programs behave in different ways.
-``-x --utf-16``
- Generate a lexer that reads input in UTF-16 encoding.
- ``re2c`` assumes that character range is 0 -- 0x10FFFF and character size is 2 bytes.
- Implies ``-s``.
+``-f --storable-state``
+ Generate a lexer which can store its inner state.
+ This is useful in push-model lexers which are stopped by an outer program
+ when there is not enough input, and then resumed when more input becomes
+ available. In this mode users should additionally define ``YYGETSTATE()``
+ and ``YYSETSTATE(state)`` macros and variables ``yych``, ``yyaccept``
+ and ``state`` as part of the lexer state.
-``-8 --utf-8``
- Generate a lexer that reads input in UTF-8 encoding.
- ``re2c`` assumes that character range is 0 -- 0x10FFFF and character size is 1 byte.
+``-F --flex-syntax``
+ Partial support for Flex syntax: in this mode named definitions don't need
+ the equal sign and the terminating semicolon, and when used they must be
+ surrounded by curly braces. Names without curly braces are treated as
+ double-quoted strings.
-``--case-insensitive``
- Treat single-quoted and double-quoted strings as case-insensitive.
+``-g --computed-gotos``
+ Optimize conditional jumps using non-standard "computed goto" extension
+ (which must be supported by the C/C++ compiler). re2c generates jump tables
+ only in complex cases with a lot of conditional branches. Complexity
+ threshold can be configured with ``cgoto:threshold`` configuration. This
+ option implies ``-b``.
+
+``-I PATH``
+ Add ``PATH`` to the list of locations which are used when searching for
+ include files. This option is useful in combination with
+ ``/*!include:re2c ... */`` directive. Re2c looks for ``FILE`` in the
+ directory of including file and in the list of include paths specified by
+ ``-I`` option.
-``--case-inverted``
- Invert the meaning of single-quoted and double-quoted strings:
- treat single-quoted strings as case-sensitive and double-quoted strings as case-insensitive.
+``-i --no-debug-info``
+ Do not output ``#line`` information. This is useful when the generated code
+ is tracked by some version control system or IDE.
+
+``--input <default | custom>``
+ Specify re2c input API.
+ Option ``default`` is the default API composed of pointer-like primitives
+ ``YYCURSOR``, ``YYMARKER``, ``YYLIMIT`` etc.
+ Option ``custom`` is the generic API composed of function-like primitives
+ ``YYPEEK()``, ``YYSKIP()``, ``YYBACKUP()``, ``YYRESTORE()`` etc.
+
+``--input-encoding <ascii | utf8>``
+ Specify the way re2c parses regular expressions.
+ With ``ascii`` (the default) re2c handles input as ASCII-encoded: any
+ sequence of code units is a sequence of standalone 1-byte characters.
+ With ``utf8`` re2c handles input as UTF8-encoded and recognizes multibyte
+ characters.
+
+``--location-format <gnu | msvc>``
+ Specify location format in messages.
+ With ``gnu`` locations are printed as 'filename:line:column: ...'.
+ With ``msvc`` locations are printed as 'filename(line,column) ...'.
+ Default is ``gnu``.
``--no-generation-date``
Suppress date output in the generated file.
``--no-lookahead``
Use TDFA(0) instead of TDFA(1).
- This option only has effect with ``--tags`` or ``--posix-captures`` options.
+ This option has effect only with ``--tags`` or ``--posix-captures`` options.
``--no-optimize-tags``
- Suppress optimization of tag variables (useful for debugging or benchmarking).
+ Suppress optimization of tag variables (useful for debugging).
``--no-version``
Suppress version output in the generated file.
-``--encoding-policy POLICY``
- Define the way ``re2c`` treats Unicode surrogates.
- ``POLICY`` can be one of the following: ``fail`` (abort with an error when a surrogate is encountered),
- ``substitute`` (silently replace surrogates with the error code point 0xFFFD),
- ``ignore`` (default, treat surrogates as normal code points).
- The Unicode standard says that standalone surrogates are invalid,
- but real-world libraries and programs behave in different ways.
+``-o OUTPUT --output=OUTPUT``
+ Specify the ``OUTPUT`` file.
+
+``-P --posix-captures``
+ Enable submatch extraction with POSIX-style capturing groups.
+
+``--posix-closure <gor1 | gtop>``
+ Specify shortest-path algorithm used for construction of epsilon-closure
+ with POSIX disambiguation semantics: ``gor1`` (the default) stands for
+ Goldberg-Radzik algorithm, and ``gtop`` stands for "global topological
+ order" algorithm.
-``--input INPUT``
- Specify ``re2c`` input API. ``INPUT`` can be either ``default`` or ``custom`` (enables the use of generic API).
+``-r --reusable``
+ Allows reuse of re2c rules with ``/*!rules:re2c */`` and ``/*!use:re2c */``
+ blocks. Exactly one rules-block must be present. The rules are saved and
+ used by every use-block that follows, which may add its own rules and
+ configurations.
``-S --skeleton``
- Ignore user-defined interface code and generate a self-contained "skeleton" program.
- Additionally, generate input files with strings derived from the regular grammar
- and compressed match results that are used to verify "skeleton" behavior on all inputs.
- This option is useful for finding bugs in optimizations and code generation.
-
-``--empty-class POLICY``
- Define the way ``re2c`` treats empty character classes.
- ``POLICY`` can be one of the following: ``match-empty`` (match empty input: illogical, but default behavior for backwards compatibility reasons),
- ``match-none`` (fail to match on any input),
- ``error`` (compilation error).
-
-``--dfa-minimization ALGORITHM``
- The internal algorithm used by re2c to minimize the DFA.
- ``ALGORITHM`` can be either ``moore`` (Moore algorithm, the default) or ``table`` (table filling algorithm).
- Both algorithms should produce the same DFA up to states relabeling;
- table filling is much slower and serves as a reference implementation.
+ Ignore user-defined interface code and generate a self-contained "skeleton"
+ program. Additionally, generate input files with strings derived from the
+ regular grammar and compressed match results that are used to verify
+ "skeleton" behavior on all inputs. This option is useful for finding bugs
+ in optimizations and code generation.
-``--eager-skip``
- Make the generated lexer advance the input position "eagerly":
- immediately after reading input symbol.
- By default this happens after transition to the next state.
- Implied by ``--no-lookahead``.
+``-s --nested-ifs``
+ Use nested ``if`` statements instead of ``switch`` statements in conditional
+ jumps. This usually results in more efficient code with non-optimizing C/C++
+ compilers.
-``--dump-nfa``
- Generate representation of NFA in DOT format and dump it on stderr.
+``-T --tags``
+ Enable submatch extraction with tags.
-``--dump-dfa-raw``
- Generate representation of DFA in DOT format under construction and dump it on stderr.
+``-t HEADER --type-header=HEADER``
+ Generate a ``HEADER`` file that contains enum with condition names.
+ Requires ``-c`` option.
-``--dump-dfa-det``
- Generate representation of DFA in DOT format immediately after determinization and dump it on stderr.
+``-u --unicode``
+ Generate a lexer that reads UTF32-encoded input. Re2c assumes that character
+ range is 0 -- 0x10FFFF and character size is 4 bytes. This option implies
+ ``-s``.
-``--dump-dfa-tagopt``
- Generate representation of DFA in DOT format after tag optimizations and dump it on stderr.
+``-V --vernum``
+ Show version information in ``MMmmpp`` format (major, minor, patch).
-``--dump-dfa-min``
- Generate representation of DFA in DOT format after minimization and dump it on stderr.
+``--verbose``
+ Output a short message in case of success.
-``--dump-adfa``
- Generate representation of DFA in DOT format after tunneling and dump it on stderr.
+``-v --version``
+ Show version information.
-``-1 --single-pass``
- Deprecated. Does nothing (single pass is the default now).
+``-w --wide-chars``
+ Generate a lexer that reads UCS2-encoded input. Re2c assumes that character
+ range is 0 -- 0xFFFF and character size is 2 bytes. This option implies
+ ``-s``.
+
+``-x --utf-16``
+ Generate a lexer that reads UTF16-encoded input. Re2c assumes that character
+ range is 0 -- 0x10FFFF and character size is 2 bytes. This option implies
+ ``-s``.
+++ /dev/null
-----
-0.9x
-----
-
-0.9.12 (2005-12-28)
-~~~~~~~~~~~~~~~~~~~
-
-- Fixed bug #1390174 re2c cannot accept ``{0,}``.
-
-0.9.11 (2005-12-18)
-~~~~~~~~~~~~~~~~~~~
-
-- Fixed #1313083 ``-e`` (EBCDIC cross compile) broken.
-- Fixed #1297658 underestimation of ``n`` in ``YYFILL(n)``.
-- Applied #1339483 Avoid rebuilds of re2c when running subtargets.
-- Implemented #1335305 symbol table reimplementation, just slightly modifed.
-
-0.9.10 (2005-09-04)
-~~~~~~~~~~~~~~~~~~~
-
-- Add ``-i`` switch to avoid generating ``#line`` information.
-- Fixed bug #1251653 re2c generate some invalid ``#line`` on WIN32.
-
-0.9.9 (2005-07-21)
-~~~~~~~~~~~~~~~~~~~
-
-- Implemented #1232777 negated char classes ``[^...]`` and the dot operator ``.``.
-- Added hexadecimal character definitions.
-- Added consistency check for octal character definitions.
-
-0.9.8 (2005-06-26)
-~~~~~~~~~~~~~~~~~~~
-
-- Fixed code generation for ``-b`` switch.
-- Added Microsoft Visual C .NET build files.
-
-0.9.7 (2005-04-30)
-~~~~~~~~~~~~~~~~~~~
-
-- Applied #1181535 storable state patch.
-- Added ``-d`` flag which outputs a debugable parser.
-- Fixed generation of ``#line`` directives (according to ISO-C99).
-- Fixed bug #1187785 Re2c fails to generate valid code.
-- Fixed bug #1187452 unused variable ``yyaccept``.
-
-0.9.6 (2005-04-14)
-~~~~~~~~~~~~~~~~~~~
-
-- Fix build with gcc >= 3.4.
-
-0.9.5 (2005-04-08)
-~~~~~~~~~~~~~~~~~~~
-
-- Added ``/*!max:re2c */`` which emits ``#define YYMAXFILL <max>``
- line. This allows to define buffers of the minimum required length.
- Occurence must follow ``/*re2c */`` and cannot preceed it.
-- Changed re2c to two pass generation to output warning free code.
-- Fixed bug #1163046 re2c hangs when processing valid re-file.
-- Fixed bug #1022799 re2c scanner has buffering bug.
-
-0.9.4 (2005-03-12)
-~~~~~~~~~~~~~~~~~~~
-
-- Added ``--vernum`` support.
-- Fixed bug #1054496 incorrect code generated with ``-b`` option.
-- Fixed bug #1012748 re2c does not emit last line if ``\n`` missing.
-- Fixed bug #999104 ``--output=output`` option does not work as documented.
-- Fixed bug #999103 Invalid options prefixed with two dashes cause program crash.
-
-0.9.3 (2004-05-26)
-~~~~~~~~~~~~~~~~~~~
-
-- Fixes one small possible bug in the generated output. ``ych`` instead of ``yych`` is output in certain circumstances.
-
-0.9.2 (2004-05-26)
-~~~~~~~~~~~~~~~~~~~
-
-- Added ``-o`` option to specify the output file which also will set the ``#line`` directives to something useful.
-- Print version to ``cout`` instead of ``cerr``.
-- Added ``-h`` and ``--`` style options.
-- Moved development to http://sourceforge.net/projects/re2c
-- Fixed bug #960144 minor cosmetic problem.
-- Fixed bug #953181 cannot compile with.
-- Fixed bug #939277 Windows support.
-- Fixed bug #914462 automake build patch
-- Fixed bug #891940 braced quantifiers: ``{\d+(,|,\d+)?}`` style.
-- Fixed bug #869298 Add case insensitive string literals.
-- Fixed bug #869297 Input buffer overrun.
-
-0.9.1 (2003-12-13)
-~~~~~~~~~~~~~~~~~~~
-
-- Removed rcs comments in source files.
+++ /dev/null
------
-0.10x
------
-
-0.10.8 (2007-04-01)
-~~~~~~~~~~~~~~~~~~~
-
-- Fixed issue with long form of switches.
-
-0.10.7 (2007-02-20)
-~~~~~~~~~~~~~~~~~~~
-
-- Applied #1647875 Add ``const`` to ``yybm`` vector.
-
-0.10.6 (2006-08-05)
-~~~~~~~~~~~~~~~~~~~
-
-- Fixed #1529351 Segv bug on unterminated code blocks.
-- Fixed #1528269 Invalid code generation.
-
-0.10.5 (2006-06-11)
-~~~~~~~~~~~~~~~~~~~
-
-- Fixed long form of ``-1`` switch to ``--single-pass`` as noted in man page and help.
-- Added MSVC 2003 project files and renamed old 2002 ones.
-
-0.10.4 (2006-06-01)
-~~~~~~~~~~~~~~~~~~~
-
-- Fix whitespace in generated code.
-
-0.10.3 (2006-05-14)
-~~~~~~~~~~~~~~~~~~~
-
-- Fixed issue with ``-wb`` and ``-ws``.
-- Added ``-g`` switch to support gcc's computed goto's.
-- Changed to use nested ``if``'s instead of ``switch(yyaccept)`` in ``-s`` mode.
-
-0.10.2 (2006-05-01)
-~~~~~~~~~~~~~~~~~~~
-
-- Changed to generate ``YYMARKER`` only when needed or in single pass mode.
-- Added ``-1`` switch to force single pass generation and make two pass the default.
-- Fixed ``-i`` switch.
-- Added configuration ``yyfill:enable`` to allow suppression of ``YYFILL()`` blocks.
-- Added tutorial like lessons to re2c.
-- Added ``/*!ignore:re2c */`` to support documenting of re2c source.
-- Fixed issue with multiline re2c comments (``/*!max:re2c ... */`` and alike).
-- Fixed generation of ``YYDEBUG()`` when using ``-d`` switch.
-- Added ``/*!getstate:re2c */`` which triggers generation of the ``YYGETSTATE()`` block.
-- Added configuration ``state:abort``.
-- Changed to not generate ``yyNext`` unless configuration ``state:nextlabel`` is used.
-- Changed to not generate ``yyaccept`` code unless needed.
-- Changed to use ``if`` instead of ``switch`` expression when ``yyaccpt`` has only one case.
-- Added docu, examples and tests to ``.src.zip`` package (0.10.1 zip was repackaged).
-- Fixed #1479044 incorrect code generated when using ``-b``.
-- Fixed #1472770 re2c creates an infinite loop.
-- Fixed #1454253 Piece of code saving a backtracking point not generated.
-- Fixed #1463639 Missing forward declaration.
-- Implemented #1187127 savable state support for multiple re2c blocks.
-- re2c 0.10.2 has been tested with the following compilers:
- + gcc (GCC) 4.1.0 (Gentoo 4.1.0)
- + gcc version 4.0.3 (4.0.3-0.20060215.2mdk for Mandriva Linux release 2006.1)
- + gcc version 4.0.2 20050901 (prerelease) (SUSE Linux)
- + gcc (GCC) 3.4.5 (Gentoo 3.4.5, ssp-3.4.5-1.0, pie-8.7.9)
- + gcc version 3.4.4 [FreeBSD] 20050518
- + gcc version 3.4.4 (cygming special) (gdc 0.12, using dmd 0.125)
- + gcc version 3.4.2 20041017 (Red Hat 3.4.2-6.fc3)
- + gcc-Version 3.3.5 (Debian 1:3.3.5-13)
- + gcc-Version 3.3.0 (mips-sgi-irix6.5/3.3.0/specs)
- + MIPSpro Compilers: Version 7.4.4m
- + Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.42 for 80x86 (Microsoft Visual C++ 2005)
- + Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 13.10.3077 for 80x86 (Mictosoft Visual C++ 2003)
- + Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 13.00.9466 for 80x86 (Microsoft Visual C++ 2002)
- + Intel(R) C++ Compiler for Intel(R) EM64T-based applications, Version 9.0 Build 20050430 Package ID: l_cc_p_9.0.021
- + CC: Sun C++ 5.8 2005/10/13 (``CXXFLAGS='-library=stlport4'``)
- + bison 2.1, 1.875d, 1.875b, 1.875
-
-0.10.1 (2006-02-28)
-~~~~~~~~~~~~~~~~~~~
-
-- Added support for Solaris and native SUN compiler.
-- Applied #1438160 expose ``YYCTXMARKER``.
-- re2c 0.10.1 has been tested with the following compilers:
- + gcc version 4.0.3 (4.0.3-0.20060215.2mdk for Mandriva Linux release 2006.1)
- + gcc version 4.0.2 (4.0.2-1mdk for Mandriva Linux release 2006.1)
- + gcc version 4.0.2 20050901 (prerelease) (SUSE Linux)
- + gcc version 3.4.4 (cygming special) (gdc 0.12, using dmd 0.125)
- + gcc-Version 3.3.5 (Debian 1:3.3.5-13)
- + gcc-Version 3.3.0 (mips-sgi-irix6.5/3.3.0/specs)
- + MIPSpro Compilers: Version 7.4.4m
- + Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.42 for 80x86 (Microsoft Visual C 2005)
- + Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 13.00.9466 for 80x86 (Microsoft Visual C 2002)
- + Intel(R) C++ Compiler for 32-bit applications, Version 9.0 Build 20051130Z Package ID: W_CC_C_9.0.028
- + CC: Sun C++ 5.8 2005/10/13 (``CXXFLAGS='-compat5 -library=stlport4'``)
- + bison 2.1, 1.875d, 1.875b, 1.875
-
-0.10.0 (2006-02-18)
-~~~~~~~~~~~~~~~~~~~
-
-- Added make target ``zip`` to create windows source packages as zip files.
-- Added ``re2c:startlabel`` configuration.
-- Fixed code generation to not generate unreachable code for initial state.
-- Added support for c/c++ compatible ``\u`` and ``\U`` unicode notation.
-- Added ability to control indendation.
-- Made scanner error out in case an ambiguous ``/*`` is found.
-- Fixed indendation of generated code.
-- Added support for DOS line endings.
-- Added experimental unicode support.
-- Added ``config_w32.h`` to build out of the box on windows (using msvc 2002+).
-- Added Microsoft Visual C .NET 2005 build files.
-- Applied #1411087 variable length trailing context.
-- Applied #1408326 do not generate ``goto`` next state.
-- Applied #1408282 ``CharSet`` initialization fix.
-- Applied #1408278 ``readsome`` with MSVC.
-- Applied #1307467 Unicode patch for 0.9.7.
+++ /dev/null
------
-0.11x
------
-
-0.11.3 (2007-04-01)
-~~~~~~~~~~~~~~~~~~~
-
-- Added support for underscores in named definitions.
-- Added new option ``--no-generation-date``.
-- Fixed issue with long form of switches.
-
-0.11.2 (2007-03-01)
-~~~~~~~~~~~~~~~~~~~
-
-- Added inplace configuration ``re2c:yyfill:parameter``.
-- Added inplace configuration ``re2c:yych:conversion``.
-- Fixed ``-u`` switch code generation.
-- Added ability to avoid defines and overwrite generated variable names.
-
-0.11.1 (2007-02-20)
-~~~~~~~~~~~~~~~~~~~
-
-- Applied #1647875 Add ``const`` to ``yybm`` vector.
-
-0.11.0 (2007-01-01)
-~~~~~~~~~~~~~~~~~~~
-
-- Added ``-u`` switch to support unicode.
+++ /dev/null
------
-0.12x
------
-
-0.12.3 (2007-08-24)
-~~~~~~~~~~~~~~~~~~~
-
-- Fixed issue with some compilers.
-- Fixed #1776177 Build on AIX.
-- Fixed #1743180 ``fwrite`` with 0 length crashes on OS X.
-
-0.12.2 (2007-06-26)
-~~~~~~~~~~~~~~~~~~~
-
-- Fixed #1743180 ``fwrite`` with 0 length crashes on OS X.
-
-0.12.1 (2007-05-23)
-~~~~~~~~~~~~~~~~~~~
-
-- Fixed #1711240 problem with ``"`` and ``7F`` on EBCDIC plattforms.
-
-0.12.0 (2007-05-01)
-~~~~~~~~~~~~~~~~~~~
-
-- Re-release of 0.11.3 as new stable branch.
-- Fixed issue with short form of switches and parameter if not first switch.
-- Fixed #1708378 segfault in ``actions.cc``.
-- re2c 0.12.0 has been tested with the following compilers:
- + gcc version 4.1.2 (Gentoo 4.1.2)
- + gcc version 4.1.2 20070302 (prerelease) (4.1.2-1mdv2007.1)
- + gcc version 4.1.2 20061115 (prerelease) (Debian 4.1.1-21)
- + gcc version 4.1.1 20070105 (Red Hat 4.1.1-51)
- + gcc version 4.1.0 (SUSE Linux 10)
- + gcc version 4.0.3 (4.0.3-0.20060215.2mdk for Mandriva Linux release 2006.1)
- + gcc version 4.0.2 20050901 (prerelease) (SUSE Linux) (32 + 64 bit)
- + MacPPC, gcc version 4.0.1 (Apple Computer, Inc. build 5367)
- + MacIntel, gcc version 4.0.1 (Apple Computer, Inc. build 5250)
- + gcc version 3.4.4 [FreeBSD] 20050518 (32 + 64 bit)
- + gcc version 3.4.4 (cygming special) (gdc 0.12, using dmd 0.125)
- + gcc version 3.4.2 [FreeBSD]
- + gcc version 3.3.5 20050117 (prerelease) (SUSE Linux)
- + gcc version 3.3.3 (PPC, 32 + 64 bit)
- + Microsoft (R) C/C++ Optimizing Compiler Version 14.00.50727.762 for x64 (64 bit)
- + Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.42 for 80x86 (Microsoft Visual C++ 2005)
- + Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 13.10.3077 for 80x86 (Mictosoft Visual C++ 2003)
- + Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 13.00.9466 for 80x86 (Microsoft Visual C++ 2002)
- + Intel(R) C++ Compiler for 32-bit applications, Version 9.1 Build 20070322Z Package ID: W_CC_C_9.1.037
- + Intel(R) C++ Compiler for Intel(R) EM64T-based applications, Version 9.1 (64 bit)
- + icpcbin (ICC) 9.1 20070215
- + CC: Sun C++ 5.8 2005/10/13 (``CXXFLAGS='-library=stlport4'``)
- + MIPSpro Compilers: Version 7.4.4m (32 + 64 bit)
- + aCC: HP C/aC++ B3910B A.06.15 [Mar 28 2007] (HP-UX IA64)
+++ /dev/null
------
-0.13x
------
-
-0.13.7.5 (2014-08-22)
-~~~~~~~~~~~~~~~~~~~~~
-
-- Fixed `Gentoo bug with PHP lexer <https://bugs.gentoo.org/show_bug.cgi?id=518904>`_
-
-0.13.7.4 (2014-07-29)
-~~~~~~~~~~~~~~~~~~~~~
-
-- Enabled ``make docs`` only if configured with ``--enable-docs``
-- Disallowed to use yacc/byacc instead of bison to build parser
-- Removed non-portable sed feature in script that runs tests
-
-0.13.7.3 (2014-07-27)
-~~~~~~~~~~~~~~~~~~~~~
-
-- Fixed CXX warning
-- Got rid of asciidoc build-time dependency
-
-0.13.7.2 (2014-07-27)
-~~~~~~~~~~~~~~~~~~~~~
-
-- Included man page into dist, respect users CXXFLAGS.
-
-0.13.7.1 (2014-07-26)
-~~~~~~~~~~~~~~~~~~~~~
-
-- Added missing files to tarball
-
-0.13.7 (2014-07-25)
-~~~~~~~~~~~~~~~~~~~
-
-- Added UTF-8 support
-- Added UTF-16 support
-- Added default rule
-- Added option to control ill-formed Unicode
-
-0.13.6 (2013-07-04)
-~~~~~~~~~~~~~~~~~~~
-
-- Fixed #2535084 uint problem with Sun C 5.8
-- #3308400: allow Yacc-style ``%{`` code brackets ``}%``
-- #2506253: allow C++ ``//`` comments
-- Fixed inplace configuration in ``-e`` mode.
-- Applied #2482572 Typos in error messages.
-- Applied #2482561 Error in manual section on ``-r`` mode.
-- Fixed #2478216 Wrong ``start_label`` in ``-c`` mode.
-- Fixed #2186718 Unescaped backslash in file name of ``#line`` directive.
-- Fixed #2102138 Duplicate case labels on EBCDIC.
-- Fixed #2088583 Compile problem on AIX.
-- Fixed #2038610 Ebcdic problem.
-- improve dot support: make char intervals (e.g. ``[A-Z]``) instead of one edge per char
-
-0.13.5 (2008-05-25)
-~~~~~~~~~~~~~~~~~~~
-
-- Fixed #1952896 Segfault in ``re2c::Scanner::scan``.
-- Fixed #1952842 Regression.
-
-0.13.4 (2008-04-05)
-~~~~~~~~~~~~~~~~~~~
-
-- Added transparent handling of ``#line`` directives in input files.
-- Added ``re2c:yyfill:check`` inplace configuration.
-- Added ``re2c:define:YYSETSTATE:naked`` inplace configuration.
-- Added ``re2c:flags:w`` and ``re2c:flags:u`` inplace configurations.
-- Added the ability to add rules in ``use:re2c`` blocks.
-- Changed ``-r`` flag to accept only ``rules:re2c`` and ``use:re2c`` blocks.
-
-0.13.3 (2008-03-14)
-~~~~~~~~~~~~~~~~~~~
-
-- Added ``-r`` flag to allow reuse of scanner definitions.
-- Added ``-F`` flag to support flex syntax in rules.
-- Fixed SEGV in scanner that occurs with very large blocks.
-- Fixed issue with unused ``yybm``.
-- Partial support for flex syntax.
-- Changed to allow ``/*`` comments with ``-c`` switch.
-- Added flag ``-D/--emit-dot``.
-
-0.13.2 (2008-02-14)
-~~~~~~~~~~~~~~~~~~~
-
-- Added flag ``--case-inverted``.
-- Added flag ``--case-insensitive``.
-- Added support for ``<!...>`` to enable rule setup.
-- Added support for ``=>`` style rules.
-- Added support for ``:=`` style rules.
-- Added support for ``:=>`` style rules.
-- Added ``re2c:cond:divider`` and ``re2c:cond:goto`` inplace configuration.
-- Fixed code generation to emit space after ``if``.
-
-0.13.1 (2007-08-24)
-~~~~~~~~~~~~~~~~~~~
-
-- Added custom build rules for Visual Studio 2005 (``re2c.rules``). (William Swanson)
-- Fixed issue with some compilers.
-- Fixed #1776177 Build on AIX.
-- Fixed #1743180 ``fwrite`` with 0 length crashes on OS X.
-
-0.13.0 (2007-06-24)
-~~~~~~~~~~~~~~~~~~~
-
-- Added ``-c`` and ``-t`` to generate scanners with (f)lex-like condition support.
-- Fixed issue with short form of switches and parameter if not first switch.
-- Fixed #1708378 segfault ``in actions.cc``.
+++ /dev/null
------
-0.14x
------
-
-0.14.3 (2015-05-20)
-~~~~~~~~~~~~~~~~~~~
-
-- applied patch '#27 re2c crashes reading files containing %{ %}' by Rui
-- dropped distfiles for MSVC (they are broken anyway)
-
-0.14.2 (2015-03-25)
-~~~~~~~~~~~~~~~~~~~
-
-- fixed #57 Wrong result only if another rule is present
-
-0.14.1 (2015-02-27)
-~~~~~~~~~~~~~~~~~~~
-
-- fixed #55 re2c-0.14: re2c -V outputs null byte
-
-0.14 (2015-02-23)
-~~~~~~~~~~~~~~~~~
-
-- Added generic input API (#21 "Support to configure how re2c code interfaced with the symbol buffer?")
-- fixed #46 "re2c generates an infinite loop, depends on existence of previous parser"
-- fixed #47 "Dot output label escaped characters"
-
+++ /dev/null
------
-0.15x
------
-
-
-0.15.3 (2015-12-02)
-~~~~~~~~~~~~~~~~~~~
-
-- Fixed bugs and applied patches:
- + #122 "clang does not compile re2c 0.15.x" (reported and fixed by Oleksii Taran).
- + #124 "Get rid of UINT32_MAX and friends" (patch by Sergei Trofimovich, fixes FreeBSD builds).
- + #125 "[OS X] git reports changes not staged for commit in newly cloned repository" (by Oleksii Taran, this fix also applies to Windows).
-- Added option ``--no-version`` that allows to omit version information.
-- Reduced memory and time consumed with ``-Wundefined-control-flow``.
-- Improved coverage of input data generated with ``-S --skeleton``.
-
-
-0.15.2 (2015-11-23)
-~~~~~~~~~~~~~~~~~~~
-
-- Fixed build system: lexer depends on bison-generated header
- (Gentoo bug: https://bugs.gentoo.org/show_bug.cgi?id=566620)
-
-
-0.15.1 (2015-11-22)
-~~~~~~~~~~~~~~~~~~~
-
-- Fixed test failures caused by locale-sensitive 'sort'.
-
-
-0.15 (2015-11-22)
-~~~~~~~~~~~~~~~~~
-
-.. include:: 0_15_list.rst
+++ /dev/null
-- Updated website http://re2c.org:
- + added examples
- + updated docs
- + added news
- + added web feed (Atom 1.0)
-- Added options:
- + ``-S, --skeleton``
- + ``--empty-class <match-empty | match-none | error>``
-- Added warnings:
- + ``-W``
- + ``-Werror``
- + ``-W<warning>``
- + ``-Wno-<warning>``
- + ``-Werror-<warning>``
- + ``-Wno-error-<warning>``
-- Added specific warnings:
- + ``-Wundefined-control-flow``
- + ``-Wunreachable-rules``
- + ``-Wcondition-order``
- + ``-Wuseless-escape``
- + ``-Wempty-character-class``
- + ``-Wswapped-range``
- + ``-Wmatch-empty-string``
-- Fixed options:
- + ``--`` (interpret remaining arguments as non-options)
-- Deprecated options:
- + ``-1 --single-pass`` (single pass is the default now)
-- Reduced size of the generated ``.dot`` files.
-- Fixed bugs:
- + #27 re2c crashes reading files containing ``%{ %}`` (patch by Rui)
- + #51 default rule doesn't work in reuse mode
- + #52 eliminate multiple passes
- + #59 bogus ``yyaccept`` in ``-c`` mode
- + #60 redundant use of ``YYMARKER``
- + #61 empty character class ``[]`` matches empty string
- + #115 flex-style named definitions cause ambiguity in re2c grammar
- + #119 ``-f`` with ``-b``/``-g`` generates incorrect dispatch on fill labels
- + #116 empty string with non-empty trailing context consumes code units
-- Added test options:
- + ``-j``, ``-j <N>`` (run tests in ``N`` threads, defaults to the number of CPUs)
- + ``--wine`` (test windows builds using ``wine``)
- + ``--skeleton`` (generate skeleton programs, compile and execute them)
- + ``--keep-tmp-files`` (don't delete intermediate files for successful tests)
-- Updated build system:
- + support out of source builds
- + support ```make distcheck```
- + added ```make bootstrap``` (rebuild re2c after building with precompiled ``.re`` files)
- + added ```make tests``` (run tests with ``-j``)
- + added ```make vtests``` (run tests with ``--valgrind -j``)
- + added ```make wtests``` (run tests with ``--wine -j 1``)
- + added Autoconf tests for ``CXXFLAGS``. By default try the following options:
- ``-W -Wall -Wextra -Weffc++ -pedantic -Wformat=2 -Wredundant-decls -Wsuggest-attribute=format -Wconversion -Wsign-conversion -O2 -Weverything``),
- respect user-defined ``CXXFLAGS``
- + support Mingw builds: ```configure -host i686-w64-mingw32```
- + structured source files
- + removed old MSVC files
-- Moved development to github (https://github.com/skvadrik/re2c), keep a mirror on sourceforge.
-
+++ /dev/null
------
-0.16x
------
-
-
-0.16 (2016-01-21)
-~~~~~~~~~~~~~~~~~
-
-- Fixed bug #127 "code generation error with wide chars and bitmaps (omitted ``goto`` statement)"
-- Added DFA minimization and option ``--dfa-minimization <table | moore>``
-- Fixed bug #128 "very slow DFA construction (resulting in a very large DFA)"
-- Fixed bug #132 "test failure on big endian archs with 0.15.3"
-
+++ /dev/null
-----
-1.0x
-----
-
-1.0.3 (2017-11-08)
-~~~~~~~~~~~~~~~~~~
-
-.. include:: 1_0_3_list.rst
-
-1.0.2 (2017-08-26)
-~~~~~~~~~~~~~~~~~~
-
-.. include:: 1_0_2_list.rst
-
-1.0.1 (2017-08-11)
-~~~~~~~~~~~~~~~~~~
-
-.. include:: 1_0_1_list.rst
-
-1.0 (2017-08-11)
-~~~~~~~~~~~~~~~~
-
-.. include:: 1_0_list.rst
-
+++ /dev/null
-- Fixed bug #193 "1.0 build failure on macOS: error: calling a private constructor of class 're2c::Rule'"
-- Added paper "Tagged Deterministic Finite Automata with Lookahead" to distribution
+++ /dev/null
-- Fixed bug #194 "Build with ``--enable-docs``"
-- Updated documentation.
+++ /dev/null
-- Fixed bug #198 (build error on MacOS with GCC-4.2.1)
+++ /dev/null
-- Added options:
- + ``-P --posix-captures`` (POSIX-compliant capturing groups)
- + ``-T --tags`` (standalone tags with leftmost greedy disambiguation)
- + ``--no-lookahead``
- + ``--no-optimize-tags``
- + ``--eager-skip``
- + ``--dump-nfa``
- + ``--dump-dfa-raw``
- + ``--dump-dfa-det``
- + ``--dump-dfa-tagopt``
- + ``--dump-dfa-min``
- + ``--dump-adfa``
-- Added new syntax:
- + ``@<stag>``
- + ``#<mtag>``
-- Added new directives:
- + ``/*!stags:re2c ... */``
- + ``/*!mtags:re2c ... */``
- + ``/*!maxnmatch:re2c ... */``
-- Added new API:
- + ``YYSTAGN (t)``
- + ``YYSTAGP (t)``
- + ``YYMTAGN (t)``
- + ``YYMTAGP (t)``
- + ``YYRESTORETAG (t)``
- + ``YYMAXNMATCH``
- + ``yynmatch``
- + ``yypmatch``
-- Added inplace confgurations:
- + ``re2c:define:YYSTAGN``
- + ``re2c:define:YYSTAGP``
- + ``re2c:define:YYMTAGN``
- + ``re2c:define:YYMTAGP``
- + ``re2c:define:YYRESTORETAG``
- + ``re2c:flags:8`` or ``re2c:flags:utf-8````
- + ``re2c:flags:b`` or ``re2c:flags:bit-vectors``
- + ``re2c:flags:case-insensitive``
- + ``re2c:flags:case-inverted``
- + ``re2c:flags:d`` or ``re2c:flags:debug-output``
- + ``re2c:flags:dfa-minimization``
- + ``re2c:flags:eager-skip``
- + ``re2c:flags:e`` or ``re2c:flags:ecb``
- + ``re2c:flags:empty-class``
- + ``re2c:flags:encoding-policy``
- + ``re2c:flags:g`` or ``re2c:flags:computed-gotos``
- + ``re2c:flags:i`` or ``re2c:flags:no-debug-info``
- + ``re2c:flags:input``
- + ``re2c:flags:lookahead``
- + ``re2c:flags:optimize-tags``
- + ``re2c:flags:P`` or ``re2c:flags:posix-captures``
- + ``re2c:flags:s`` or ``re2c:flags:nested-ifs``
- + ``re2c:flags:T`` or ``re2c:flags:tags``
- + ``re2c:flags:u`` or ``re2c:flags:unicode``
- + ``re2c:flags:w`` or ``re2c:flags:wide-chars``
- + ``re2c:flags:x`` or ``re2c:flags:utf-16``
- + ``re2c:tags:expression``
- + ``re2c:tags:prefix``
-- Added warning ``-Wnondeterministic-tags``
-- Added fuzz-testing scripts
-- Added paper "Tagged Deterministic Finite Automata with Lookahead"
-- Fixed bugs:
- + #121 "trailing contexts are fundamentally broken"
- + #135 "In installation ``make check`` give syntax error"
- + #137 "run_tests.sh fail when running configure script with absolute path"
- + #138 "website improvement"
- + #141 "Tests under Windows"
- + #142 "segvault with null terminated input"
- + #145 "Values for enum YYCONDTYPE are not generated when default rules with conditions are used"
- + #147 "Please add symbol name to "can't find symbol" error message"
- + #152 "Line number in #line directive after enum YYCONDTYPE is 0-based"
- + #156 "Build with Visual Studio 14 2015: symbol name conflict"
- + #158 "Inconsistent forward declaration of struct/class vs definition"
- + #160 "Open text files with "wb" causes issues on Windows"
- + #162 "Reading files with "rb" causes issues in Windows"
- + #165 "Trailing context consumed if initial expression matches it"
- + #176 "re2c help message is too wide for most terminals"
- + #184 "Small documentation issue"
- + #186 "Difference operator sometimes doesn't work with utf-8"
-- Merged pull requests:
- + #131 "Use bash-specific ``[[`` builtin"
- + #136 "Added basic support for travis-ci.org integration"
- + #171 "Typo fix"
- + #172 "Grammar fixes in the docs"
- + #173 "Grammar fixes in the manpage"
- + #174 "more documentation fixes"
- + #175 "more manpage fixes"
- + #177 "sync --help output w/ manpage"
- + #178 "Moves rts used in the manpage to master"
- + #179 "compose manpage out of rsts from gh-pages-gen"
- + #189 "Typo fix and small grammatical change"
- + #191 "Makefile.am: create target directory before writing into it"
+++ /dev/null
-----
-1.1x
-----
-
-1.1.1 (2018-08-30)
-~~~~~~~~~~~~~~~~~~
-
-.. include:: 1_1_1_list.rst
-
-1.1 (2018-08-27)
-~~~~~~~~~~~~~~~~~~
-
-.. include:: 1_1_list.rst
+++ /dev/null
-- Fixed bug #211 re2c ``-V`` throws ``std::out_of_range`` (version to vernum conversion).
+++ /dev/null
-- Replaced Kuklewicz POSIX disambiguation algorithm with Okui algorithm.
-- Optimized GOR1 algorithm (computation of tagged epsilon-closure).
-- Added option ``--conditions`` (an alias for ``-c --start-conditions``).
-- Fixed bug #201 Bugs with option: ``re2c:flags:no-debug-info``.
-- Reworked first part of TDFA paper.
+++ /dev/null
--------------------------
-re2c adopted (2003-12-09)
--------------------------
-
-- Version 0.9.1 README::
-
- Originally written by Peter Bumbulis (peter@csg.uwaterloo.ca)
- Currently maintained by Brian Young (bayoung@acm.org)
-
- The re2c distribution can be found at:
- http://www.tildeslash.org/re2c/index.html
-
- The source distribution is available from:
- http://www.tildeslash.org/re2c/re2c-0.9.1.tar.gz
-
- This distribution is a cleaned up version of the 0.5 release
- maintained by me (Brian Young). Several bugs were fixed as well
- as code cleanup for warning free compilation. It has been
- developed and tested with egcs 1.0.2 and gcc 2.7.2.3 on Linux x86.
- Peter Bumbulis' original release can be found at:
- ftp://csg.uwaterloo.ca/pub/peter/re2c.0.5.tar.gz
-
- re2c is a great tool for writing fast and flexible lexers.
- It has served many people well for many years and it deserves
- to be maintained more actively. re2c is on the order of 2-3
- times faster than a flex based scanner, and its input model
- is much more flexible.
-
- Patches and requests for features will be entertained. Areas
- of particular interest to me are porting (a Solaris and an NT
- version will be forthcoming) and wide character support. Note
- that the code is already quite portable and should be buildable
- on any platform with minor makefile changes.
-
-- Version 0.5 Peter's original ANNOUNCE and README::
-
- re2c is a tool for generating C-based recognizers from regular
- expressions. re2c-based scanners are efficient: for programming
- languages, given similar specifications, an re2c-based scanner
- is typically almost twice as fast as a flex-based scanner with
- little or no increase in size (possibly a decrease on cisc
- architectures). Indeed, re2c-based scanners are quite competitive
- with hand-crafted ones.
-
- Unlike flex, re2c does not generate complete scanners: the user
- must supply some interface code. While this code is not bulky
- (about 50-100 lines for a flex-like scanner; see the man page
- and examples in the distribution) careful coding is required for
- efficiency (and correctness). One advantage of this arrangement
- is that the generated code is not tied to any particular input
- model. For example, re2c generated code can be used to scan
- data from a null-byte terminated buffer as illustrated below.
-
- Given the following source:
-
- #define NULL ((char*) 0)
- char *scan(char *p) {
- char *q;
- #define YYCTYPE char
- #define YYCURSOR p
- #define YYLIMIT p
- #define YYMARKER q
- #define YYFILL(n)
- /*!re2c
- [0-9]+ {return YYCURSOR;}
- [\000-\377] {return NULL;}
- */
- }
-
- re2c will generate:
-
- /* Generated by re2c on Sat Apr 16 11:40:58 1994 */
- #line 1 "simple.re"
- #define NULL ((char*) 0)
- char *scan(char *p) {
- char *q;
- #define YYCTYPE char
- #define YYCURSOR p
- #define YYLIMIT p
- #define YYMARKER q
- #define YYFILL(n)
- {
- YYCTYPE yych;
- unsigned int yyaccept;
- goto yy0;
- yy1: ++YYCURSOR;
- yy0:
- if((YYLIMIT - YYCURSOR) < 2) YYFILL(2);
- yych = *YYCURSOR;
- if(yych <= '/') goto yy4;
- if(yych >= ':') goto yy4;
- yy2: yych = *++YYCURSOR;
- goto yy7;
- yy3:
- #line 10
- {return YYCURSOR;}
- yy4: yych = *++YYCURSOR;
- yy5:
- #line 11
- {return NULL;}
- yy6: ++YYCURSOR;
- if(YYLIMIT == YYCURSOR) YYFILL(1);
- yych = *YYCURSOR;
- yy7: if(yych <= '/') goto yy3;
- if(yych <= '9') goto yy6;
- goto yy3;
- }
- #line 12
-
- }
-
- Note that most compilers will perform dead-code elimination to
- remove all YYCURSOR, YYLIMIT comparisions.
-
- re2c was developed for a particular project (constructing a fast
- REXX scanner of all things!) and so while it has some rough edges,
- it should be quite usable. More information about re2c can be
- found in the (admittedly skimpy) man page; the algorithms and
- heuristics used are described in an upcoming LOPLAS article
- (included in the distribution). Probably the best way to find out
- more about re2c is to try the supplied examples. re2c is written in
- C++, and is currently being developed under Linux using gcc 2.5.8.
-
- Peter
+++ /dev/null
-=========
-Changelog
-=========
-
-.. toctree::
- :hidden:
-
-.. include:: 1_1.rst
-.. include:: 1_0.rst
-.. include:: 0_16.rst
-.. include:: 0_15.rst
-.. include:: 0_14.rst
-.. include:: 0_13.rst
-.. include:: 0_12.rst
-.. include:: 0_11.rst
-.. include:: 0_10.rst
-.. include:: 0_09.rst
-.. include:: beginning.rst
-
+++ /dev/null
-====
-News
-====
-
-.. toctree::
- :maxdepth: 1
-
- Release 1.1.1 <release_notes/1_1_1>
- Release 1.1 <release_notes/1_1>
- Release 1.0.3 <release_notes/1_0_3>
- Release 1.0.2 <release_notes/1_0_2>
- Release 1.0.1 <release_notes/1_0_1>
- Release 1.0 <release_notes/1_0>
- Release 0.16 <release_notes/0_16>
- Release 0.15.3 <release_notes/0_15_3>
- Release 0.15.2 <release_notes/0_15_2>
- Release 0.15.1 <release_notes/0_15_1>
- Release 0.15 <release_notes/0_15>
- Changelog <changelog/changelog>
-
+++ /dev/null
-==============
-Release 0.15.1
-==============
-
-This release fixes an error in the testing script:
-it used the locale-sensitive ``sort`` utility,
-which resulted in different order of files on different platforms.
-
-Thanks to Sergei Trofimovich who reported the bug and suggested a quick fix. ``:)``
-
+++ /dev/null
-==============
-Release 0.15.2
-==============
-
-This release fixes a bug in the build system (`reported <https://bugs.gentoo.org/show_bug.cgi?id=566620>`_ on Gentoo bugtracker).
-The fix adds a missing dependency (the lexer depends on a bison-generated header).
-It seems that the dependency has always been missing, and the build order just happened to be correct.
-
+++ /dev/null
-==============
-Release 0.15.3
-==============
-
-This release fixes multiple build-time and run-time failures on OS X, FreeBSD, and Windows.
-Most of the problems were reported and fixed by Oleksii Taran (on OS X)
-and Sergei Trofimovich (on FreeBSD and Windows).
-Thank you for your help!
-
-`Skeleton <../../manual/features/skeleton/skeleton.html>`_ coverage has been improved.
-See section `Generating data <../../manual/features/skeleton/skeleton.html#generating-data>`_ for details.
-
-See `changelog <../changelog/changelog.html>`_ for the list of all changes.
-
+++ /dev/null
-=============
-Release 1.0.1
-=============
-
-*by Ulya Trofimovich*
-
-This is a minor bug-fix release; see `release 1.0 <./1_0.html>`_ for details.
-This release addresses the following issues:
-
-.. include:: ../changelog/1_0_1_list.rst
-
-Thanks to ilovezf for bug report.
-
+++ /dev/null
-=============
-Release 1.0.2
-=============
-
-*by Ulya Trofimovich*
-
-This is another minor bug-fix release in the 1.0.x series; see `release 1.0 <./1_0.html>`_ for details.
-This release addresses the following issues:
-
-.. include:: ../changelog/1_0_2_list.rst
-
-Thanks to Ziris85 for the bug report.
-
+++ /dev/null
-=============
-Release 1.0.3
-=============
-
-*by Ulya Trofimovich*
-
-Yet another minor bug-fix release in the 1.0.x series; see `release 1.0 <./1_0.html>`_ for details.
-This release fixes build failure on Mac OS X 10.5.8 with with GCC version 4.2.1.
-Thanks to Ryan Shmidt for the bug report.
-
-.. include:: ../changelog/1_0_3_list.rst
-
--- /dev/null
+Changelog
+=========
+
+----
+1.2x
+----
+
+1.2 (2019-08-01)
+~~~~~~~~~~~~~~~~
+
+- Added EOF rule ``$`` and configuration ``re2c:eof``.
+
+- Added ``/*!include:re2c ... */`` directive and ``-I`` option.
+
+- Added ``/*!header:re2c:on*/`` and ``/*!header:re2c:off*/`` directives.
+
+- Added ``--input-encoding <ascii | utf8>`` option.
+
+ + `#237 <https://github.com/skvadrik/re2c/issues/237>`_:
+ Handle non-ASCII encoded characters in regular expressions
+ + `#250 <https://github.com/skvadrik/re2c/issues/250>`_
+ UTF8 enoding
+
+- Added include file with a list of definitions for Unicode character classes.
+
+ + `#235 <https://github.com/skvadrik/re2c/issues/235>`_:
+ Unicode character classes
+
+- Added ``--location-format <gnu | msvc>`` option.
+
+ + `#195 <https://github.com/skvadrik/re2c/issues/195>`_:
+ Please consider using Gnu format for error messages
+
+- Added ``--verbose`` option that prints "success" message if re2c exits
+ without errors.
+
+- Added configurations for options:
+
+ + ``-o --output`` (specify output file)
+ + ``-t --type-header`` (specify header file)
+
+- Removed configurations for internal/debug options.
+
+- Extended ``-r`` option: allow to mix multiple ``/*!rules:re2c*/``,
+ ``/*!use:re2c*/`` and ``/*!re2c*/`` blocks.
+
+ + `#55 <https://github.com/skvadrik/re2c/issues/55>`_:
+ allow standard re2c blocks in reuse mode
+
+- Fixed ``-F --flex-support`` option: parsing and operator precedence.
+
+ + `#229 <https://github.com/skvadrik/re2c/issues/229>`_:
+ re2c option -F (flex syntax) broken
+ + `#242 <https://github.com/skvadrik/re2c/issues/242>`_:
+ Operator precedence with --flex-syntax is broken
+
+- Changed difference operator ``/`` to apply before encoding expansion of
+ operands.
+
+ + `#236 <https://github.com/skvadrik/re2c/issues/236>`_:
+ Support range difference with variable-length encodings
+
+- Changed output generation of output file to be atomic.
+
+ + `#245 <https://github.com/skvadrik/re2c/issues/245>`_:
+ re2c output is not atomic
+
+- Authored research paper "Efficient POSIX Submatch Extraction on NFA"
+ together with Dr Angelo Borsotti.
+
+- Added experimental libre2c library (``--enable-libs`` configure option) with
+ the following algorithms:
+
+ + TDFA with leftmost-greedy disambiguation
+ + TDFA with POSIX disambiguation (Okui-Suzuki algorithm)
+ + TNFA with leftmost-greedy disambiguation
+ + TNFA with POSIX disambiguation (Okui-Suzuki algorithm)
+ + TNFA with lazy POSIX disambiguation (Okui-Suzuki algorithm)
+ + TNFA with POSIX disambiguation (Kuklewicz algorithm)
+ + TNFA with POSIX disambiguation (Cox algorithm)
+
+- Added debug subsystem (``--enable-debug`` configure option) and new debug
+ options:
+
+ + ``-dump-cfg`` (dump control flow graph of tag variables)
+ + ``-dump-interf`` (dump interference table of tag variables)
+ + ``-dump-closure-stats`` (dump epsilon-closure statistics)
+
+- Added internal options:
+
+ + ``--posix-closure <gor1 | gtop>`` (switch between shortest-path algorithms
+ used for the construction of POSIX closure)
+
+- Fixed a number of crashes found by American Fuzzy Lop fuzzer:
+
+ + `#226 <https://github.com/skvadrik/re2c/issues/226>`_,
+ `#227 <https://github.com/skvadrik/re2c/issues/227>`_,
+ `#228 <https://github.com/skvadrik/re2c/issues/228>`_,
+ `#231 <https://github.com/skvadrik/re2c/issues/231>`_,
+ `#232 <https://github.com/skvadrik/re2c/issues/232>`_,
+ `#233 <https://github.com/skvadrik/re2c/issues/233>`_,
+ `#234 <https://github.com/skvadrik/re2c/issues/234>`_,
+ `#238 <https://github.com/skvadrik/re2c/issues/238>`_
+
+- Fixed handling of newlines:
+
+ + correctly parse multi-character newlines CR LF in ``#line`` directives
+ + consistently convert all newlines in the generated file to Unix-style LF
+
+- Changed default tarball format from .gz to .xz.
+
+ + `#221 <https://github.com/skvadrik/re2c/issues/221>`_:
+ big source tarball
+
+- Fixed a number of other bugs and resolved issues:
+
+ + `#2 <https://github.com/skvadrik/re2c/issues/2>`_: abort
+ + `#6 <https://github.com/skvadrik/re2c/issues/6>`_: segfault
+ + `#10 <https://github.com/skvadrik/re2c/issues/10>`_:
+ lessons/002_upn_calculator/calc_002 doesn't produce a useful example program
+ + `#44 <https://github.com/skvadrik/re2c/issues/44>`_:
+ Access violation when translating the attached file
+ + `#49 <https://github.com/skvadrik/re2c/issues/49>`_:
+ wildcard state \000 rules makes lexer behave weard
+ + `#98 <https://github.com/skvadrik/re2c/issues/98>`_:
+ Transparent handling of #line directives in input files
+ + `#104 <https://github.com/skvadrik/re2c/issues/104>`_:
+ Improve const-correctness
+ + `#105 <https://github.com/skvadrik/re2c/issues/105>`_:
+ Conversion of pointer parameters into references
+ + `#114 <https://github.com/skvadrik/re2c/issues/114>`_:
+ Possibility of fixing bug 2535084
+ + `#120 <https://github.com/skvadrik/re2c/issues/120>`_:
+ condition consisting of default rule only is ignored
+ + `#167 <https://github.com/skvadrik/re2c/issues/167>`_:
+ Add word boundary support
+ + `#168 <https://github.com/skvadrik/re2c/issues/168>`_:
+ Wikipedia's article on re2c
+ + `#180 <https://github.com/skvadrik/re2c/issues/180>`_:
+ Comment syntax?
+ + `#182 <https://github.com/skvadrik/re2c/issues/182>`_:
+ yych being set by YYPEEK () and then not used
+ + `#196 <https://github.com/skvadrik/re2c/issues/196>`_:
+ Implicit type conversion warnings
+ + `#198 <https://github.com/skvadrik/re2c/issues/198>`_:
+ no match for ‘operator!=’ in ‘i != std::vector<_Tp, _Alloc>::rend() [with _Tp = re2c::bitmap_t, _Alloc = std::allocator<re2c::bitmap_t>]()’
+ + `#210 <https://github.com/skvadrik/re2c/issues/210>`_:
+ How to build re2c in windows?
+ + `#215 <https://github.com/skvadrik/re2c/issues/215>`_:
+ A memory read overrun issue in s_to_n32_unsafe.cc
+ + `#220 <https://github.com/skvadrik/re2c/issues/220>`_:
+ src/dfa/dfa.h: simplify constructor to avoid g++-3.4 bug
+ + `#223 <https://github.com/skvadrik/re2c/issues/223>`_:
+ Fix typo
+ + `#224 <https://github.com/skvadrik/re2c/issues/224>`_:
+ src/dfa/closure_posix.cc: pack() tweaks
+ + `#225 <https://github.com/skvadrik/re2c/issues/225>`_:
+ Documentation link is broken in libre2c/README
+ + `#230 <https://github.com/skvadrik/re2c/issues/230>`_:
+ Changes for upcoming Travis' infra migration
+ + `#239 <https://github.com/skvadrik/re2c/issues/239>`_:
+ Push model example has wrong re2c invocation, breaks guide
+ + `#241 <https://github.com/skvadrik/re2c/issues/241>`_:
+ Guidance on how to use re2c for full-duplex command & response protocol
+ + `#243 <https://github.com/skvadrik/re2c/issues/243>`_:
+ A code generated for period (.) requires 4 bytes
+ + `#246 <https://github.com/skvadrik/re2c/issues/246>`_:
+ Please add a license to this repo
+ + `#247 <https://github.com/skvadrik/re2c/issues/247>`_:
+ Build failure on current Cygwin, probably caused by force-fed c++98 mode
+ + `#248 <https://github.com/skvadrik/re2c/issues/248>`_:
+ distcheck still looks for README
+ + `#251 <https://github.com/skvadrik/re2c/issues/251>`_:
+ Including what you use is find, but not without inclusion guards
+
+- Updated documentation and website.
+
+
+----
+1.1x
+----
+
+1.1.1 (2018-08-30)
+~~~~~~~~~~~~~~~~~~
+
+- Fixed bug `#211 <https://github.com/skvadrik/re2c/issues/211>`_:
+ re2c ``-V`` throws ``std::out_of_range`` (version to vernum conversion).
+
+1.1 (2018-08-27)
+~~~~~~~~~~~~~~~~
+
+- Replaced Kuklewicz POSIX disambiguation algorithm with Okui algorithm.
+- Optimized GOR1 algorithm (computation of tagged epsilon-closure).
+- Added option ``--conditions`` (an alias for ``-c --start-conditions``).
+- Fixed bug `#201 <https://github.com/skvadrik/re2c/issues/201>`_:
+ Bugs with option: ``re2c:flags:no-debug-info``.
+- Reworked first part of TDFA paper.
+
+----
+1.0x
+----
+
+1.0.3 (2017-11-08)
+~~~~~~~~~~~~~~~~~~
+
+- Fixed bug `#198 <https://github.com/skvadrik/re2c/issues/198>`_:
+ build error on MacOS with GCC-4.2.1
+
+1.0.2 (2017-08-26)
+~~~~~~~~~~~~~~~~~~
+
+- Fixed bug `#194 <https://github.com/skvadrik/re2c/issues/194>`_:
+ Build with ``--enable-docs``
+- Updated documentation.
+
+1.0.1 (2017-08-11)
+~~~~~~~~~~~~~~~~~~
+
+- Fixed bug `#193 <https://github.com/skvadrik/re2c/issues/193>`_:
+ 1.0 build failure on macOS: error: calling a private constructor of class
+ 're2c::Rule'
+
+- Added paper "Tagged Deterministic Finite Automata with Lookahead" to the
+ distribution files.
+
+1.0 (2017-08-11)
+~~~~~~~~~~~~~~~~
+
+- Added options:
+
+ + ``-P --posix-captures`` (POSIX-compliant capturing groups)
+ + ``-T --tags`` (standalone tags with leftmost greedy disambiguation)
+ + ``--no-lookahead``
+ + ``--no-optimize-tags``
+ + ``--eager-skip``
+ + ``--dump-nfa``
+ + ``--dump-dfa-raw``
+ + ``--dump-dfa-det``
+ + ``--dump-dfa-tagopt``
+ + ``--dump-dfa-min``
+ + ``--dump-adfa``
+
+- Added new syntax:
+
+ + ``@<stag>``
+ + ``#<mtag>``
+
+- Added new directives:
+
+ + ``/*!stags:re2c ... */``
+ + ``/*!mtags:re2c ... */``
+ + ``/*!maxnmatch:re2c ... */``
+
+- Added new API:
+
+ + ``YYSTAGN (t)``
+ + ``YYSTAGP (t)``
+ + ``YYMTAGN (t)``
+ + ``YYMTAGP (t)``
+ + ``YYRESTORETAG (t)``
+ + ``YYMAXNMATCH``
+ + ``yynmatch``
+ + ``yypmatch``
+
+- Added inplace confgurations:
+
+ + ``re2c:define:YYSTAGN``
+ + ``re2c:define:YYSTAGP``
+ + ``re2c:define:YYMTAGN``
+ + ``re2c:define:YYMTAGP``
+ + ``re2c:define:YYRESTORETAG``
+ + ``re2c:flags:8`` or ``re2c:flags:utf-8````
+ + ``re2c:flags:b`` or ``re2c:flags:bit-vectors``
+ + ``re2c:flags:case-insensitive``
+ + ``re2c:flags:case-inverted``
+ + ``re2c:flags:d`` or ``re2c:flags:debug-output``
+ + ``re2c:flags:dfa-minimization``
+ + ``re2c:flags:eager-skip``
+ + ``re2c:flags:e`` or ``re2c:flags:ecb``
+ + ``re2c:flags:empty-class``
+ + ``re2c:flags:encoding-policy``
+ + ``re2c:flags:g`` or ``re2c:flags:computed-gotos``
+ + ``re2c:flags:i`` or ``re2c:flags:no-debug-info``
+ + ``re2c:flags:input``
+ + ``re2c:flags:lookahead``
+ + ``re2c:flags:optimize-tags``
+ + ``re2c:flags:P`` or ``re2c:flags:posix-captures``
+ + ``re2c:flags:s`` or ``re2c:flags:nested-ifs``
+ + ``re2c:flags:T`` or ``re2c:flags:tags``
+ + ``re2c:flags:u`` or ``re2c:flags:unicode``
+ + ``re2c:flags:w`` or ``re2c:flags:wide-chars``
+ + ``re2c:flags:x`` or ``re2c:flags:utf-16``
+ + ``re2c:tags:expression``
+ + ``re2c:tags:prefix``
+
+- Added warning ``-Wnondeterministic-tags``.
+
+- Added fuzz-testing scripts
+
+- Added paper "Tagged Deterministic Finite Automata with Lookahead".
+
+- Fixed bugs:
+
+ + `#121 <https://github.com/skvadrik/re2c/issues/121>`_:
+ trailing contexts are fundamentally broken
+ + `#135 <https://github.com/skvadrik/re2c/issues/135>`_:
+ In installation ``make check`` give syntax error
+ + `#137 <https://github.com/skvadrik/re2c/issues/137>`_:
+ run_tests.sh fail when running configure script with absolute path
+ + `#138 <https://github.com/skvadrik/re2c/issues/138>`_:
+ website improvement
+ + `#141 <https://github.com/skvadrik/re2c/issues/141>`_:
+ Tests under Windows
+ + `#142 <https://github.com/skvadrik/re2c/issues/142>`_:
+ segvault with null terminated input
+ + `#145 <https://github.com/skvadrik/re2c/issues/145>`_:
+ Values for enum YYCONDTYPE are not generated when default rules with conditions are used
+ + `#147 <https://github.com/skvadrik/re2c/issues/147>`_:
+ Please add symbol name to "can't find symbol" error message
+ + `#152 <https://github.com/skvadrik/re2c/issues/152>`_:
+ Line number in #line directive after enum YYCONDTYPE is 0-based
+ + `#156 <https://github.com/skvadrik/re2c/issues/156>`_:
+ Build with Visual Studio 14 2015: symbol name conflict
+ + `#158 <https://github.com/skvadrik/re2c/issues/158>`_:
+ Inconsistent forward declaration of struct/class vs definition
+ + `#160 <https://github.com/skvadrik/re2c/issues/160>`_:
+ Open text files with "wb" causes issues on Windows
+ + `#162 <https://github.com/skvadrik/re2c/issues/162>`_:
+ Reading files with "rb" causes issues in Windows
+ + `#165 <https://github.com/skvadrik/re2c/issues/165>`_:
+ Trailing context consumed if initial expression matches it
+ + `#176 <https://github.com/skvadrik/re2c/issues/176>`_:
+ re2c help message is too wide for most terminals
+ + `#184 <https://github.com/skvadrik/re2c/issues/184>`_:
+ Small documentation issue
+ + `#186 <https://github.com/skvadrik/re2c/issues/186>`_:
+ Difference operator sometimes doesn't work with utf-8
+
+- Merged pull requests:
+
+ + `#131 <https://github.com/skvadrik/re2c/issues/131>`_:
+ Use bash-specific ``[[`` builtin
+ + `#136 <https://github.com/skvadrik/re2c/issues/136>`_:
+ Added basic support for travis-ci.org integration
+ + `#171 <https://github.com/skvadrik/re2c/issues/171>`_:
+ Typo fix
+ + `#172 <https://github.com/skvadrik/re2c/issues/172>`_:
+ Grammar fixes in the docs
+ + `#173 <https://github.com/skvadrik/re2c/issues/173>`_:
+ Grammar fixes in the manpage
+ + `#174 <https://github.com/skvadrik/re2c/issues/174>`_:
+ more documentation fixes
+ + `#175 <https://github.com/skvadrik/re2c/issues/175>`_:
+ more manpage fixes
+ + `#177 <https://github.com/skvadrik/re2c/issues/177>`_:
+ sync --help output w/ manpage
+ + `#178 <https://github.com/skvadrik/re2c/issues/178>`_:
+ Moves rts used in the manpage to master
+ + `#179 <https://github.com/skvadrik/re2c/issues/179>`_:
+ compose manpage out of rsts from gh-pages-gen
+ + `#189 <https://github.com/skvadrik/re2c/issues/189>`_:
+ Typo fix and small grammatical change
+ + `#191 <https://github.com/skvadrik/re2c/issues/191>`_:
+ Makefile.am: create target directory before writing into it
+
+
+-----
+0.16x
+-----
+
+0.16 (2016-01-21)
+~~~~~~~~~~~~~~~~~
+
+- Fixed bug `#127 <https://github.com/skvadrik/re2c/issues/127>`_:
+ code generation error with wide chars and bitmaps (omitted ``goto`` statement)
+- Added DFA minimization and option ``--dfa-minimization <table | moore>``
+- Fixed bug `#128 <https://github.com/skvadrik/re2c/issues/128>`_:
+ very slow DFA construction (resulting in a very large DFA)
+- Fixed bug `#132 <https://github.com/skvadrik/re2c/issues/132>`_:
+ test failure on big endian archs with 0.15.3
+
+
+-----
+0.15x
+-----
+
+0.15.3 (2015-12-02)
+~~~~~~~~~~~~~~~~~~~
+
+- Fixed bugs and applied patches:
+
+ + `#122 <https://github.com/skvadrik/re2c/issues/122>`_:
+ clang does not compile re2c 0.15.x
+ + `#124 <https://github.com/skvadrik/re2c/issues/124>`_:
+ Get rid of UINT32_MAX and friends
+ + `#125 <https://github.com/skvadrik/re2c/issues/125>`_:
+ [OS X] git reports changes not staged for commit in newly cloned repository
+
+- Added option ``--no-version`` that allows to omit version information.
+- Reduced memory and time consumed with ``-Wundefined-control-flow``.
+- Improved coverage of input data generated with ``-S --skeleton``.
+
+
+0.15.2 (2015-11-23)
+~~~~~~~~~~~~~~~~~~~
+
+- Fixed build system: lexer depends on bison-generated header
+ (Gentoo bug: https://bugs.gentoo.org/show_bug.cgi?id=566620)
+
+
+0.15.1 (2015-11-22)
+~~~~~~~~~~~~~~~~~~~
+
+- Fixed test failures caused by locale-sensitive 'sort'.
+
+
+0.15 (2015-11-22)
+~~~~~~~~~~~~~~~~~
+
+- Updated website http://re2c.org:
+
+ + added examples
+ + updated docs
+ + added news
+ + added web feed (Atom 1.0)
+
+- Added options:
+
+ + ``-S, --skeleton``
+ + ``--empty-class <match-empty | match-none | error>``
+
+- Added warnings:
+
+ + ``-W``
+ + ``-Werror``
+ + ``-W<warning>``
+ + ``-Wno-<warning>``
+ + ``-Werror-<warning>``
+ + ``-Wno-error-<warning>``
+
+- Added specific warnings:
+
+ + ``-Wundefined-control-flow``
+ + ``-Wunreachable-rules``
+ + ``-Wcondition-order``
+ + ``-Wuseless-escape``
+ + ``-Wempty-character-class``
+ + ``-Wswapped-range``
+ + ``-Wmatch-empty-string``
+
+- Fixed options:
+
+ + ``--`` (interpret remaining arguments as non-options)
+
+- Deprecated options:
+
+ + ``-1 --single-pass`` (single pass is the default now)
+
+- Reduced size of the generated ``.dot`` files.
+
+- Fixed bugs:
+
+ + `#27 <https://github.com/skvadrik/re2c/issues/27>`_:
+ re2c crashes reading files containing ``%{ %}`` (patch by Rui)
+ + `#51 <https://github.com/skvadrik/re2c/issues/51>`_:
+ default rule doesn't work in reuse mode
+ + `#52 <https://github.com/skvadrik/re2c/issues/52>`_:
+ eliminate multiple passes
+ + `#59 <https://github.com/skvadrik/re2c/issues/59>`_:
+ bogus ``yyaccept`` in ``-c`` mode
+ + `#60 <https://github.com/skvadrik/re2c/issues/60>`_:
+ redundant use of ``YYMARKER``
+ + `#61 <https://github.com/skvadrik/re2c/issues/61>`_:
+ empty character class ``[]`` matches empty string
+ + `#115 <https://github.com/skvadrik/re2c/issues/115>`_:
+ flex-style named definitions cause ambiguity in re2c grammar
+ + `#119 <https://github.com/skvadrik/re2c/issues/119>`_:
+ ``-f`` with ``-b``/``-g`` generates incorrect dispatch on fill labels
+ + `#116 <https://github.com/skvadrik/re2c/issues/116>`_:
+ empty string with non-empty trailing context consumes code units
+
+- Added test options:
+
+ + ``-j``, ``-j <N>`` (run tests in ``N`` threads, defaults to the number of CPUs)
+ + ``--wine`` (test windows builds using ``wine``)
+ + ``--skeleton`` (generate skeleton programs, compile and execute them)
+ + ``--keep-tmp-files`` (don't delete intermediate files for successful tests)
+
+- Updated build system:
+
+ + support out of source builds
+ + support ```make distcheck```
+ + added ```make bootstrap``` (rebuild re2c after building with precompiled
+ ``.re`` files)
+ + added ```make tests``` (run tests with ``-j``)
+ + added ```make vtests``` (run tests with ``--valgrind -j``)
+ + added ```make wtests``` (run tests with ``--wine -j 1``)
+ + added Autoconf tests for ``CXXFLAGS``. By default try the following options:
+ ``-W -Wall -Wextra -Weffc++ -pedantic -Wformat=2 -Wredundant-decls
+ -Wsuggest-attribute=format -Wconversion -Wsign-conversion -O2 -Weverything``),
+ respect user-defined ``CXXFLAGS``
+ + support Mingw builds: ```configure -host i686-w64-mingw32```
+ + structured source files
+ + removed old MSVC files
+
+- Moved development to github (https://github.com/skvadrik/re2c),
+ keep a mirror on sourceforge.
+
+
+-----
+0.14x
+-----
+
+0.14.3 (2015-05-20)
+~~~~~~~~~~~~~~~~~~~
+
+- applied patch `#27 <https://github.com/skvadrik/re2c/issues/27>`_:
+ re2c crashes reading files containing %{ %}
+- dropped distfiles for MSVC (they are broken anyway)
+
+0.14.2 (2015-03-25)
+~~~~~~~~~~~~~~~~~~~
+
+- fixed `#57 <https://github.com/skvadrik/re2c/issues/57>`_:
+ Wrong result only if another rule is present
+
+0.14.1 (2015-02-27)
+~~~~~~~~~~~~~~~~~~~
+
+- fixed `#55 <https://github.com/skvadrik/re2c/issues/55>`_:
+ re2c-0.14: re2c -V outputs null byte
+
+0.14 (2015-02-23)
+~~~~~~~~~~~~~~~~~
+
+- Added generic input API
+
+ + `#21 <https://github.com/skvadrik/re2c/issues/21>`_:
+ Support to configure how re2c code interfaced with the symbol buffer?"
+
+- fixed `#46 <https://github.com/skvadrik/re2c/issues/46>`_:
+ re2c generates an infinite loop, depends on existence of previous parser
+- fixed `#47 <https://github.com/skvadrik/re2c/issues/47>`_:
+ Dot output label escaped characters
+
+
+-----
+0.13x
+-----
+
+0.13.7.5 (2014-08-22)
+~~~~~~~~~~~~~~~~~~~~~
+
+- Fixed `Gentoo bug with PHP lexer <https://bugs.gentoo.org/show_bug.cgi?id=518904>`_
+
+0.13.7.4 (2014-07-29)
+~~~~~~~~~~~~~~~~~~~~~
+
+- Enabled ``make docs`` only if configured with ``--enable-docs``
+- Disallowed to use yacc/byacc instead of bison to build parser
+- Removed non-portable sed feature in script that runs tests
+
+0.13.7.3 (2014-07-27)
+~~~~~~~~~~~~~~~~~~~~~
+
+- Fixed CXX warning
+- Got rid of asciidoc build-time dependency
+
+0.13.7.2 (2014-07-27)
+~~~~~~~~~~~~~~~~~~~~~
+
+- Included man page into dist, respect users CXXFLAGS.
+
+0.13.7.1 (2014-07-26)
+~~~~~~~~~~~~~~~~~~~~~
+
+- Added missing files to tarball
+
+0.13.7 (2014-07-25)
+~~~~~~~~~~~~~~~~~~~
+
+- Added UTF-8 support
+- Added UTF-16 support
+- Added default rule
+- Added option to control ill-formed Unicode
+
+0.13.6 (2013-07-04)
+~~~~~~~~~~~~~~~~~~~
+
+- Fixed #2535084 uint problem with Sun C 5.8
+- #3308400: allow Yacc-style ``%{`` code brackets ``}%``
+- #2506253: allow C++ ``//`` comments
+- Fixed inplace configuration in ``-e`` mode.
+- Applied #2482572 Typos in error messages.
+- Applied #2482561 Error in manual section on ``-r`` mode.
+- Fixed #2478216 Wrong ``start_label`` in ``-c`` mode.
+- Fixed #2186718 Unescaped backslash in file name of ``#line`` directive.
+- Fixed #2102138 Duplicate case labels on EBCDIC.
+- Fixed #2088583 Compile problem on AIX.
+- Fixed #2038610 Ebcdic problem.
+- improve dot support: make char intervals (e.g. ``[A-Z]``) instead of one edge
+ per char
+
+0.13.5 (2008-05-25)
+~~~~~~~~~~~~~~~~~~~
+
+- Fixed #1952896 Segfault in ``re2c::Scanner::scan``.
+- Fixed #1952842 Regression.
+
+0.13.4 (2008-04-05)
+~~~~~~~~~~~~~~~~~~~
+
+- Added transparent handling of ``#line`` directives in input files.
+- Added ``re2c:yyfill:check`` inplace configuration.
+- Added ``re2c:define:YYSETSTATE:naked`` inplace configuration.
+- Added ``re2c:flags:w`` and ``re2c:flags:u`` inplace configurations.
+- Added the ability to add rules in ``use:re2c`` blocks.
+- Changed ``-r`` flag to accept only ``rules:re2c`` and ``use:re2c`` blocks.
+
+0.13.3 (2008-03-14)
+~~~~~~~~~~~~~~~~~~~
+
+- Added ``-r`` flag to allow reuse of scanner definitions.
+- Added ``-F`` flag to support flex syntax in rules.
+- Fixed SEGV in scanner that occurs with very large blocks.
+- Fixed issue with unused ``yybm``.
+- Partial support for flex syntax.
+- Changed to allow ``/*`` comments with ``-c`` switch.
+- Added flag ``-D/--emit-dot``.
+
+0.13.2 (2008-02-14)
+~~~~~~~~~~~~~~~~~~~
+
+- Added flag ``--case-inverted``.
+- Added flag ``--case-insensitive``.
+- Added support for ``<!...>`` to enable rule setup.
+- Added support for ``=>`` style rules.
+- Added support for ``:=`` style rules.
+- Added support for ``:=>`` style rules.
+- Added ``re2c:cond:divider`` and ``re2c:cond:goto`` inplace configuration.
+- Fixed code generation to emit space after ``if``.
+
+0.13.1 (2007-08-24)
+~~~~~~~~~~~~~~~~~~~
+
+- Added custom build rules for Visual Studio 2005 (``re2c.rules``).
+ (William Swanson)
+- Fixed issue with some compilers.
+- Fixed #1776177 Build on AIX.
+- Fixed #1743180 ``fwrite`` with 0 length crashes on OS X.
+
+0.13.0 (2007-06-24)
+~~~~~~~~~~~~~~~~~~~
+
+- Added ``-c`` and ``-t`` to generate scanners with (f)lex-like condition
+ support.
+- Fixed issue with short form of switches and parameter if not first switch.
+- Fixed #1708378 segfault ``in actions.cc``.
+
+
+-----
+0.12x
+-----
+
+0.12.3 (2007-08-24)
+~~~~~~~~~~~~~~~~~~~
+
+- Fixed issue with some compilers.
+- Fixed #1776177 Build on AIX.
+- Fixed #1743180 ``fwrite`` with 0 length crashes on OS X.
+
+0.12.2 (2007-06-26)
+~~~~~~~~~~~~~~~~~~~
+
+- Fixed #1743180 ``fwrite`` with 0 length crashes on OS X.
+
+0.12.1 (2007-05-23)
+~~~~~~~~~~~~~~~~~~~
+
+- Fixed #1711240 problem with ``"`` and ``7F`` on EBCDIC plattforms.
+
+0.12.0 (2007-05-01)
+~~~~~~~~~~~~~~~~~~~
+
+- Re-release of 0.11.3 as new stable branch.
+- Fixed issue with short form of switches and parameter if not first switch.
+- Fixed #1708378 segfault in ``actions.cc``.
+- re2c 0.12.0 has been tested with the following compilers:
+
+ + gcc version 4.1.2 (Gentoo 4.1.2)
+ + gcc version 4.1.2 20070302 (prerelease) (4.1.2-1mdv2007.1)
+ + gcc version 4.1.2 20061115 (prerelease) (Debian 4.1.1-21)
+ + gcc version 4.1.1 20070105 (Red Hat 4.1.1-51)
+ + gcc version 4.1.0 (SUSE Linux 10)
+ + gcc version 4.0.3 (4.0.3-0.20060215.2mdk for Mandriva Linux release 2006.1)
+ + gcc version 4.0.2 20050901 (prerelease) (SUSE Linux) (32 + 64 bit)
+ + MacPPC, gcc version 4.0.1 (Apple Computer, Inc. build 5367)
+ + MacIntel, gcc version 4.0.1 (Apple Computer, Inc. build 5250)
+ + gcc version 3.4.4 [FreeBSD] 20050518 (32 + 64 bit)
+ + gcc version 3.4.4 (cygming special) (gdc 0.12, using dmd 0.125)
+ + gcc version 3.4.2 [FreeBSD]
+ + gcc version 3.3.5 20050117 (prerelease) (SUSE Linux)
+ + gcc version 3.3.3 (PPC, 32 + 64 bit)
+ + Microsoft (R) C/C++ Optimizing Compiler Version 14.00.50727.762 for x64 (64 bit)
+ + Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.42 for 80x86 (Microsoft Visual C++ 2005)
+ + Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 13.10.3077 for 80x86 (Mictosoft Visual C++ 2003)
+ + Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 13.00.9466 for 80x86 (Microsoft Visual C++ 2002)
+ + Intel(R) C++ Compiler for 32-bit applications, Version 9.1 Build 20070322Z Package ID: W_CC_C_9.1.037
+ + Intel(R) C++ Compiler for Intel(R) EM64T-based applications, Version 9.1 (64 bit)
+ + icpcbin (ICC) 9.1 20070215
+ + CC: Sun C++ 5.8 2005/10/13 (``CXXFLAGS='-library=stlport4'``)
+ + MIPSpro Compilers: Version 7.4.4m (32 + 64 bit)
+ + aCC: HP C/aC++ B3910B A.06.15 [Mar 28 2007] (HP-UX IA64)
+
+
+-----
+0.11x
+-----
+
+0.11.3 (2007-04-01)
+~~~~~~~~~~~~~~~~~~~
+
+- Added support for underscores in named definitions.
+- Added new option ``--no-generation-date``.
+- Fixed issue with long form of switches.
+
+0.11.2 (2007-03-01)
+~~~~~~~~~~~~~~~~~~~
+
+- Added inplace configuration ``re2c:yyfill:parameter``.
+- Added inplace configuration ``re2c:yych:conversion``.
+- Fixed ``-u`` switch code generation.
+- Added ability to avoid defines and overwrite generated variable names.
+
+0.11.1 (2007-02-20)
+~~~~~~~~~~~~~~~~~~~
+
+- Applied #1647875 Add ``const`` to ``yybm`` vector.
+
+0.11.0 (2007-01-01)
+~~~~~~~~~~~~~~~~~~~
+
+- Added ``-u`` switch to support unicode.
+
+
+-----
+0.10x
+-----
+
+0.10.8 (2007-04-01)
+~~~~~~~~~~~~~~~~~~~
+
+- Fixed issue with long form of switches.
+
+0.10.7 (2007-02-20)
+~~~~~~~~~~~~~~~~~~~
+
+- Applied #1647875 Add ``const`` to ``yybm`` vector.
+
+0.10.6 (2006-08-05)
+~~~~~~~~~~~~~~~~~~~
+
+- Fixed #1529351 Segv bug on unterminated code blocks.
+- Fixed #1528269 Invalid code generation.
+
+0.10.5 (2006-06-11)
+~~~~~~~~~~~~~~~~~~~
+
+- Fixed long form of ``-1`` switch to ``--single-pass`` as noted in man page
+ and help.
+- Added MSVC 2003 project files and renamed old 2002 ones.
+
+0.10.4 (2006-06-01)
+~~~~~~~~~~~~~~~~~~~
+
+- Fix whitespace in generated code.
+
+0.10.3 (2006-05-14)
+~~~~~~~~~~~~~~~~~~~
+
+- Fixed issue with ``-wb`` and ``-ws``.
+- Added ``-g`` switch to support gcc's computed goto's.
+- Changed to use nested ``if``'s instead of ``switch(yyaccept)`` in ``-s`` mode.
+
+0.10.2 (2006-05-01)
+~~~~~~~~~~~~~~~~~~~
+
+- Changed to generate ``YYMARKER`` only when needed or in single pass mode.
+- Added ``-1`` switch to force single pass generation and make two pass the
+ default.
+- Fixed ``-i`` switch.
+- Added configuration ``yyfill:enable`` to allow suppression of ``YYFILL()``
+ blocks.
+- Added tutorial like lessons to re2c.
+- Added ``/*!ignore:re2c */`` to support documenting of re2c source.
+- Fixed issue with multiline re2c comments (``/*!max:re2c ... */`` and alike).
+- Fixed generation of ``YYDEBUG()`` when using ``-d`` switch.
+- Added ``/*!getstate:re2c */`` which triggers generation of the
+ ``YYGETSTATE()`` block.
+- Added configuration ``state:abort``.
+- Changed to not generate ``yyNext`` unless configuration ``state:nextlabel`` is
+ used.
+- Changed to not generate ``yyaccept`` code unless needed.
+- Changed to use ``if`` instead of ``switch`` expression when ``yyaccpt`` has
+ only one case.
+- Added docu, examples and tests to ``.src.zip`` package (0.10.1 zip was
+ repackaged).
+- Fixed #1479044 incorrect code generated when using ``-b``.
+- Fixed #1472770 re2c creates an infinite loop.
+- Fixed #1454253 Piece of code saving a backtracking point not generated.
+- Fixed #1463639 Missing forward declaration.
+- Implemented #1187127 savable state support for multiple re2c blocks.
+- re2c 0.10.2 has been tested with the following compilers:
+
+ + gcc (GCC) 4.1.0 (Gentoo 4.1.0)
+ + gcc version 4.0.3 (4.0.3-0.20060215.2mdk for Mandriva Linux release 2006.1)
+ + gcc version 4.0.2 20050901 (prerelease) (SUSE Linux)
+ + gcc (GCC) 3.4.5 (Gentoo 3.4.5, ssp-3.4.5-1.0, pie-8.7.9)
+ + gcc version 3.4.4 [FreeBSD] 20050518
+ + gcc version 3.4.4 (cygming special) (gdc 0.12, using dmd 0.125)
+ + gcc version 3.4.2 20041017 (Red Hat 3.4.2-6.fc3)
+ + gcc-Version 3.3.5 (Debian 1:3.3.5-13)
+ + gcc-Version 3.3.0 (mips-sgi-irix6.5/3.3.0/specs)
+ + MIPSpro Compilers: Version 7.4.4m
+ + Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.42 for
+ 80x86 (Microsoft Visual C++ 2005)
+ + Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 13.10.3077 for 80x86
+ (Mictosoft Visual C++ 2003)
+ + Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 13.00.9466 for 80x86
+ (Microsoft Visual C++ 2002)
+ + Intel(R) C++ Compiler for Intel(R) EM64T-based applications, Version 9.0
+ Build 20050430 Package ID: l_cc_p_9.0.021
+ + CC: Sun C++ 5.8 2005/10/13 (``CXXFLAGS='-library=stlport4'``)
+ + bison 2.1, 1.875d, 1.875b, 1.875
+
+0.10.1 (2006-02-28)
+~~~~~~~~~~~~~~~~~~~
+
+- Added support for Solaris and native SUN compiler.
+- Applied #1438160 expose ``YYCTXMARKER``.
+- re2c 0.10.1 has been tested with the following compilers:
+
+ + gcc version 4.0.3 (4.0.3-0.20060215.2mdk for Mandriva Linux release 2006.1)
+ + gcc version 4.0.2 (4.0.2-1mdk for Mandriva Linux release 2006.1)
+ + gcc version 4.0.2 20050901 (prerelease) (SUSE Linux)
+ + gcc version 3.4.4 (cygming special) (gdc 0.12, using dmd 0.125)
+ + gcc-Version 3.3.5 (Debian 1:3.3.5-13)
+ + gcc-Version 3.3.0 (mips-sgi-irix6.5/3.3.0/specs)
+ + MIPSpro Compilers: Version 7.4.4m
+ + Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.42 for
+ 80x86 (Microsoft Visual C 2005)
+ + Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 13.00.9466 for 80x86
+ (Microsoft Visual C 2002)
+ + Intel(R) C++ Compiler for 32-bit applications, Version 9.0 Build 20051130Z
+ Package ID: W_CC_C_9.0.028
+ + CC: Sun C++ 5.8 2005/10/13 (``CXXFLAGS='-compat5 -library=stlport4'``)
+ + bison 2.1, 1.875d, 1.875b, 1.875
+
+0.10.0 (2006-02-18)
+~~~~~~~~~~~~~~~~~~~
+
+- Added make target ``zip`` to create windows source packages as zip files.
+- Added ``re2c:startlabel`` configuration.
+- Fixed code generation to not generate unreachable code for initial state.
+- Added support for c/c++ compatible ``\u`` and ``\U`` unicode notation.
+- Added ability to control indendation.
+- Made scanner error out in case an ambiguous ``/*`` is found.
+- Fixed indendation of generated code.
+- Added support for DOS line endings.
+- Added experimental unicode support.
+- Added ``config_w32.h`` to build out of the box on windows (using msvc 2002+).
+- Added Microsoft Visual C .NET 2005 build files.
+- Applied #1411087 variable length trailing context.
+- Applied #1408326 do not generate ``goto`` next state.
+- Applied #1408282 ``CharSet`` initialization fix.
+- Applied #1408278 ``readsome`` with MSVC.
+- Applied #1307467 Unicode patch for 0.9.7.
+
+
+----
+0.9x
+----
+
+0.9.12 (2005-12-28)
+~~~~~~~~~~~~~~~~~~~
+
+- Fixed bug #1390174 re2c cannot accept ``{0,}``.
+
+0.9.11 (2005-12-18)
+~~~~~~~~~~~~~~~~~~~
+
+- Fixed #1313083 ``-e`` (EBCDIC cross compile) broken.
+- Fixed #1297658 underestimation of ``n`` in ``YYFILL(n)``.
+- Applied #1339483 Avoid rebuilds of re2c when running subtargets.
+- Implemented #1335305 symbol table reimplementation, just slightly modifed.
+
+0.9.10 (2005-09-04)
+~~~~~~~~~~~~~~~~~~~
+
+- Add ``-i`` switch to avoid generating ``#line`` information.
+- Fixed bug #1251653 re2c generate some invalid ``#line`` on WIN32.
+
+0.9.9 (2005-07-21)
+~~~~~~~~~~~~~~~~~~~
+
+- Implemented #1232777 negated char classes ``[^...]`` and the dot operator ``.``.
+- Added hexadecimal character definitions.
+- Added consistency check for octal character definitions.
+
+0.9.8 (2005-06-26)
+~~~~~~~~~~~~~~~~~~~
+
+- Fixed code generation for ``-b`` switch.
+- Added Microsoft Visual C .NET build files.
+
+0.9.7 (2005-04-30)
+~~~~~~~~~~~~~~~~~~~
+
+- Applied #1181535 storable state patch.
+- Added ``-d`` flag which outputs a debugable parser.
+- Fixed generation of ``#line`` directives (according to ISO-C99).
+- Fixed bug #1187785 Re2c fails to generate valid code.
+- Fixed bug #1187452 unused variable ``yyaccept``.
+
+0.9.6 (2005-04-14)
+~~~~~~~~~~~~~~~~~~~
+
+- Fix build with gcc >= 3.4.
+
+0.9.5 (2005-04-08)
+~~~~~~~~~~~~~~~~~~~
+
+- Added ``/*!max:re2c */`` which emits ``#define YYMAXFILL <max>``
+ line. This allows to define buffers of the minimum required length.
+ Occurence must follow ``/*re2c */`` and cannot preceed it.
+- Changed re2c to two pass generation to output warning free code.
+- Fixed bug #1163046 re2c hangs when processing valid re-file.
+- Fixed bug #1022799 re2c scanner has buffering bug.
+
+0.9.4 (2005-03-12)
+~~~~~~~~~~~~~~~~~~~
+
+- Added ``--vernum`` support.
+- Fixed bug #1054496 incorrect code generated with ``-b`` option.
+- Fixed bug #1012748 re2c does not emit last line if ``\n`` missing.
+- Fixed bug #999104 ``--output=output`` option does not work as documented.
+- Fixed bug #999103 Invalid options prefixed with two dashes cause program
+ crash.
+
+0.9.3 (2004-05-26)
+~~~~~~~~~~~~~~~~~~~
+
+- Fixes one small possible bug in the generated output. ``ych`` instead of
+ ``yych`` is output in certain circumstances.
+
+0.9.2 (2004-05-26)
+~~~~~~~~~~~~~~~~~~~
+
+- Added ``-o`` option to specify the output file which also will set the
+ ``#line`` directives to something useful.
+- Print version to ``cout`` instead of ``cerr``.
+- Added ``-h`` and ``--`` style options.
+- Moved development to http://sourceforge.net/projects/re2c
+- Fixed bug #960144 minor cosmetic problem.
+- Fixed bug #953181 cannot compile with.
+- Fixed bug #939277 Windows support.
+- Fixed bug #914462 automake build patch
+- Fixed bug #891940 braced quantifiers: ``{\d+(,|,\d+)?}`` style.
+- Fixed bug #869298 Add case insensitive string literals.
+- Fixed bug #869297 Input buffer overrun.
+
+0.9.1 (2003-12-13)
+~~~~~~~~~~~~~~~~~~~
+
+- Removed rcs comments in source files.
+
+-------------------------
+re2c adopted (2003-12-09)
+-------------------------
+
+- Version 0.9.1 README::
+
+ Originally written by Peter Bumbulis (peter@csg.uwaterloo.ca)
+ Currently maintained by Brian Young (bayoung@acm.org)
+
+ The re2c distribution can be found at:
+ http://www.tildeslash.org/re2c/index.html
+
+ The source distribution is available from:
+ http://www.tildeslash.org/re2c/re2c-0.9.1.tar.gz
+
+ This distribution is a cleaned up version of the 0.5 release
+ maintained by me (Brian Young). Several bugs were fixed as well
+ as code cleanup for warning free compilation. It has been
+ developed and tested with egcs 1.0.2 and gcc 2.7.2.3 on Linux x86.
+ Peter Bumbulis' original release can be found at:
+ ftp://csg.uwaterloo.ca/pub/peter/re2c.0.5.tar.gz
+
+ re2c is a great tool for writing fast and flexible lexers.
+ It has served many people well for many years and it deserves
+ to be maintained more actively. re2c is on the order of 2-3
+ times faster than a flex based scanner, and its input model
+ is much more flexible.
+
+ Patches and requests for features will be entertained. Areas
+ of particular interest to me are porting (a Solaris and an NT
+ version will be forthcoming) and wide character support. Note
+ that the code is already quite portable and should be buildable
+ on any platform with minor makefile changes.
+
+- Version 0.5 Peter's original ANNOUNCE and README::
+
+ re2c is a tool for generating C-based recognizers from regular
+ expressions. re2c-based scanners are efficient: for programming
+ languages, given similar specifications, an re2c-based scanner
+ is typically almost twice as fast as a flex-based scanner with
+ little or no increase in size (possibly a decrease on cisc
+ architectures). Indeed, re2c-based scanners are quite competitive
+ with hand-crafted ones.
+
+ Unlike flex, re2c does not generate complete scanners: the user
+ must supply some interface code. While this code is not bulky
+ (about 50-100 lines for a flex-like scanner; see the man page
+ and examples in the distribution) careful coding is required for
+ efficiency (and correctness). One advantage of this arrangement
+ is that the generated code is not tied to any particular input
+ model. For example, re2c generated code can be used to scan
+ data from a null-byte terminated buffer as illustrated below.
+
+ Given the following source:
+
+ #define NULL ((char*) 0)
+ char *scan(char *p) {
+ char *q;
+ #define YYCTYPE char
+ #define YYCURSOR p
+ #define YYLIMIT p
+ #define YYMARKER q
+ #define YYFILL(n)
+ /*!re2c
+ [0-9]+ {return YYCURSOR;}
+ [\000-\377] {return NULL;}
+ */
+ }
+
+ re2c will generate:
+
+ /* Generated by re2c on Sat Apr 16 11:40:58 1994 */
+ #line 1 "simple.re"
+ #define NULL ((char*) 0)
+ char *scan(char *p) {
+ char *q;
+ #define YYCTYPE char
+ #define YYCURSOR p
+ #define YYLIMIT p
+ #define YYMARKER q
+ #define YYFILL(n)
+ {
+ YYCTYPE yych;
+ unsigned int yyaccept;
+ goto yy0;
+ yy1: ++YYCURSOR;
+ yy0:
+ if((YYLIMIT - YYCURSOR) < 2) YYFILL(2);
+ yych = *YYCURSOR;
+ if(yych <= '/') goto yy4;
+ if(yych >= ':') goto yy4;
+ yy2: yych = *++YYCURSOR;
+ goto yy7;
+ yy3:
+ #line 10
+ {return YYCURSOR;}
+ yy4: yych = *++YYCURSOR;
+ yy5:
+ #line 11
+ {return NULL;}
+ yy6: ++YYCURSOR;
+ if(YYLIMIT == YYCURSOR) YYFILL(1);
+ yych = *YYCURSOR;
+ yy7: if(yych <= '/') goto yy3;
+ if(yych <= '9') goto yy6;
+ goto yy3;
+ }
+ #line 12
+
+ }
+
+ Note that most compilers will perform dead-code elimination to
+ remove all YYCURSOR, YYLIMIT comparisions.
+
+ re2c was developed for a particular project (constructing a fast
+ REXX scanner of all things!) and so while it has some rough edges,
+ it should be quite usable. More information about re2c can be
+ found in the (admittedly skimpy) man page; the algorithms and
+ heuristics used are described in an upcoming LOPLAS article
+ (included in the distribution). Probably the best way to find out
+ more about re2c is to try the supplied examples. re2c is written in
+ C++, and is currently being developed under Linux using gcc 2.5.8.
+
+ Peter
--- /dev/null
+=============
+Release notes
+=============
+
+.. toctree::
+ :hidden:
+
+ /releases/changelog/changelog
+
+.. include:: /releases/release_notes/1_2.rst
+.. include:: /releases/release_notes/1_1_1.rst
+.. include:: /releases/release_notes/1_1.rst
+.. include:: /releases/release_notes/1_0_3.rst
+.. include:: /releases/release_notes/1_0_2.rst
+.. include:: /releases/release_notes/1_0_1.rst
+.. include:: /releases/release_notes/1_0.rst
+.. include:: /releases/release_notes/0_16.rst
+.. include:: /releases/release_notes/0_15_3.rst
+.. include:: /releases/release_notes/0_15_2.rst
+.. include:: /releases/release_notes/0_15_1.rst
+.. include:: /releases/release_notes/0_15.rst
-============
Release 0.15
============
This release started out in the spring of 2015 as a relatively simple code cleanup.
-I focused on the following problem: re2c used to repeat the whole generation process multiple times.
+It focused on the following problem: re2c used to repeat the whole generation process multiple times.
Some parts of the generated program depend on the overall input statistics;
they cannot be generated until the whole input has been processed.
The usual strategy is to make stubs for all such things and fix them later.
Moreover, each generation pass further duplicated certain calculations (for similar reasons).
As a result, the code generation phase was repeated four times.
-The problem here is not inefficiency: re2c is fast enough to allow the 4x overhead.
-The real problem is the complexity of the code: you have to think of multiple execution layers all the time.
+The problem here was not inefficiency: re2c is fast enough to allow the 4x overhead.
+The real problem was the complexity of the code: the developer has to think of multiple execution layers all the time.
Some parts of code are only executed on certain layers and affect each other.
Simple reasoning gets really hard.
So the main purpose of this release was to simplify the code and make it easier to fix bugs and add new features.
-However, very soon I realized that some of the changes in code generation are hard to verify by hand.
+Very soon it became clear that some of the changes in code generation are hard to verify by hand.
For example, even a minor rebalancing of ``if`` and ``switch`` statements
may change the generated code significantly.
-In search of an automatic verification tool, I encountered the idea of generating `skeleton <../../manual/features/skeleton/skeleton.html>`_ programs.
+It was clear that re2c needed an automatic verification tool,
+which lead to the idea of generating :ref:`skeleton programs <manual/manual:skeleton programs>`.
-Meanwhile I just couldn't help adding `warnings <../../manual/warnings/warnings.html>`_,
-updating the build system, and fixing various bugs.
+Meanwhile some work was done on adding :ref:`warnings <manual/manual:warnings>`,
+updating the build system and fixing various bugs.
A heart-warming consequence of the code simplification is that re2c now uses re2c more extensively:
not only the main program, but also the command-line options, inplace configurations,
and all kinds of escaped strings are parsed with re2c.
-I also updated the website (feel free to suggest improvements).
-
-Here is the list of all the changes:
+Website has also been updated (feel free to suggest improvements).
-.. include:: ../changelog/0_15_list.rst
+See the changelog :ref:`releases/changelog/changelog:0.15 (2015-11-22)` for details.
--- /dev/null
+Release 0.15.1
+==============
+
+This release fixes an error in the testing script
+(thanks to Sergei Trofimovich who reported the bug and suggested a quick fix).
+See the changelog :ref:`releases/changelog/changelog:0.15.1 (2015-11-22)` for details.
+
--- /dev/null
+Release 0.15.2
+==============
+
+This release fixes a bug in the build system:
+it adds a missing dependency of the lexer on the bison-generated header
+(`reported <https://bugs.gentoo.org/show_bug.cgi?id=566620>`_ on Gentoo bugtracker).
+See the changelog :ref:`releases/changelog/changelog:0.15.2 (2015-11-23)` for details.
--- /dev/null
+Release 0.15.3
+==============
+
+This release fixes multiple build-time and run-time failures on OS X, FreeBSD, and Windows.
+Most of the problems were reported and fixed by Oleksii Taran (on OS X)
+and Sergei Trofimovich (on FreeBSD and Windows).
+Thank you for your help!
+
+:ref:`Skeleton <manual/manual:skeleton programs>` coverage has been improved.
+
+See the changelog :ref:`releases/changelog/changelog:0.15.3 (2015-12-02)` for details.
+
-==============
Release 0.16
==============
However, in the case of DFA minimization the generated code changes dramatically.
It is impossible to verify the changes manually.
-One possible verification tool is `skeleton <../../manual/features/skeleton/skeleton.html>`_.
+One possible verification tool is the :ref:`skeleton<manual/manual:skeleton programs>` feature.
Because skeleton is constructed prior to DFA minimization, it cannot be affected by any errors in its implementation.
Another way to verify DFA minimization is by implementing two different algorithms
of code generation (see `this issue <https://github.com/skvadrik/re2c/issues/128>`_ for example).
The test suite now runs twice as fast (at least).
-See `changelog <../changelog/changelog.html>`_ for the list of changes.
+See the changelog :ref:`releases/changelog/changelog:0.16 (2016-01-21)` for details.
-===========
Release 1.0
===========
-*by Ulya Trofimovich*
-
This release is an important milestone in the history of re2c: it adds *submatch extraction*.
It is a long-awaited feature; even as Peter Bumbulis released the first version of re2c back in 1993,
he mentioned submatch extraction in his paper together with Donald D. Cowan
-`"RE2C: a more versatile scanner generator" <../../1994_bumbulis_cowan_re2c_a_more_versatile_scanner_generator.pdf>`_
+`"RE2C: a more versatile scanner generator" <../../1994_bumbulis_cowan_re2c_a_more_versatile_scanner_generator.ps>`_
as a useful feature that soon should be added to re2c.
It seemed back then that submatch extraction is relatively easy to implement;
like many other useful extensions, it was deemed to be no more than syntactic sugar on top of canonical regular expressions.
But what is most important, the added complexity depends on submatch detalization:
on submatch-free regular expressions Laurikari automaton reduces to a simple DFA.
-As I started working on the implementation of submatch extraction in re2c (in early 2016),
-at first I didn't appreciate the elegance of Laurikari approach; it still seemed overly complicated for re2c.
-I studied many papers and tried to invent an algorithm that would support only *partial* submatch extraction,
+When the work on submatch extraction feature started in early 2016,
+at first it seemed that Laurikari algorithm was still overly complicated.
+Re2c needed an algorithm that would support only *partial* submatch extraction,
but the supported cases would be handled very efficiently.
-As I developed the algorithm, I came to realize that it was in fact but a special case of Laurikari algorithm.
-Naturally, I turned back to the original paper and started to adapt my implementation.
-To my surprise, I discovered a strange deficiency in Laurikari algorithm compared to my early implementation:
-when recording submatches, Laurikari automata totally ignore the lookahead symbol
+A lot of effort was spent on studying papers,
+and finally this lead to the development of a new algorithm.
+It soon became clear that the self-invented algorithm was in fact a special case of Laurikari algorithm,
+with one significant performance improvement: the use of lookahead symbol.
+Surprisingly, the original Laurikari algorithm has a strange deficiency:
+when recording submatches, the automata totally ignore the lookahead symbol
and perform a lot of redundant operations that could be easily avoided.
-It was only logical to fix Laurikari algorithm;
-but as I worked on it, a couple of other issues showed up,
-and so it happened that I ended up writing a whole paper on the subject:
+It was only logical to fix Laurikari algorithm.
+The new improved algorithm is described in the paper
`"Tagged Deterministic Finite Automata with Lookahead" <../../2017_trofimovich_tagged_deterministic_finite_automata_with_lookahead.pdf>`_
Submatch extraction in re2c comes in two flavors:
POSIX-compliant capturing groups (with true POSIX semantics)
and standalone *tags* that use leftmost greedy disambiguation.
Both features require a number of new API primitives and configurations;
-and they are barely documented yet (though I managed to add a `section <../../manual/features/submatch/submatch.html>`_ in the manual).
+and they are barely documented yet (though I managed to add a section in the manual).
The paper cited above has some interesting benchmark results;
articles and examples will follow in the nearest future.
As usually, you may expect a number of small bug-fix releases in the 1.0.x series.
* Many thanks to the early user of the new features João Paredes for his patience and encouraging comments on the subject.
-Below is a coarse list of all changes (a raw excerpt from `changelog <../changelog/changelog.html>`_):
-
-.. include:: ../changelog/1_0_list.rst
+See the changelog :ref:`releases/changelog/changelog:1.0 (2017-08-11)` for details.
--- /dev/null
+Release 1.0.1
+=============
+
+This is a minor bug-fix release.
+It fixes a build failure on Mac OS X (thanks to ilovezf for the bug report).
+See the changelog :ref:`releases/changelog/changelog:1.0.1 (2017-08-11)` for details.
+
--- /dev/null
+Release 1.0.2
+=============
+
+This is another minor bug-fix release.
+It addresses some documentation issues (thanks to Ziris85 for the bug report).
+See the changelog :ref:`releases/changelog/changelog:1.0.2 (2017-08-26)` for details.
+
--- /dev/null
+Release 1.0.3
+=============
+
+Yet another minor bug-fix release.
+It fixes a build failure on Mac OS X (thanks to Ryan Shmidt for the bug report).
+See the changelog :ref:`releases/changelog/changelog:1.0.3 (2017-11-08)` for details.
-=============
Release 1.1
=============
-*by Ulya Trofimovich*
-
This release doesn't bring in any serious user-visible changes;
most of the work concentrated on the internal POSIX disambiguation algorithm
used for submatch extraction with ``--posix-captures`` option.
based on the work of Satoshi Okui and Taro Suzuki (described in the paper
"Disambiguation in Regular Expression Matching via Position Automata with Augmented Transitions", year 2013).
The main effort in this release was in developing proper theoretical foundations for the new algorithm.
+As usual, there has been a couple of bug fixes.
-As usual, there has been a couple of bug fixes. The list of changes:
-
-.. include:: ../changelog/1_1_list.rst
+See the changelog :ref:`releases/changelog/changelog:1.1 (2018-08-27)` for details.
-=============
Release 1.1.1
=============
-*by Ulya Trofimovich*
-
-This is a minor bug-fix release in the 1.1.x series.
+This is a minor bug-fix release.
It fixes a crash with ``-V`` ``--vernum`` option.
The crash was reported and fixed by Mike Gilbert,
and later rediscovered by Daniel J. Luke and Perry E. Metzger.
Sergei Trofimovich made a unit test, as the ``-V`` ``--vernum`` option has already caused some troubles in the past.
+See the changelog :ref:`releases/changelog/changelog:1.1.1 (2018-08-30)` for details.
Thanks everyone!
-
--- /dev/null
+Release 1.2
+===========
+
+This release adds a handful of useful features and finalizes two-year
+research effort on POSIX disambiguation.
+
+First and foremost, :ref:`EOF rule <manual/manual:eof rule>` has been added to
+re2c. It is a new method of checking for the end of input, which aims at being
+both generic and simple to use. Historically re2c has a
+:ref:`number of ways <manual/manual:eof handling>` to handle EOF situation. The
+simplest and the most efficient method is
+:ref:`using a sentinel symbol <manual/manual:using sentinel symbol>`, but it
+has limited applicability. The most widely used and generic method is
+:ref:`bounds checking with padding <manual/manual:bounds checking with padding>`,
+but it is rather complex and requires input buffering.
+Finally, it is possible to define custom EOF handling methods
+by :ref:`using the generic API <manual/manual:using generic api>`. However, such
+methods are not necessarily correct and are likely to be slower than other
+methods. The new method, EOF rule, tries to combine the simplicity and
+efficiency of the sentinel symbol and the generality of bounds checking.
+Essentially, all the user is required to do is to specify an arbitrary "fake"
+sentinel using ``re2c:eof`` configuration (zero is usually a good choice), to
+define a special EOF rule ``$``, and to make sure that the fake sentinel is
+always the last symbol in the input buffer. The lexer will then run without end
+of input checks until it hits the branch corresponding to the fake sentinel, at
+which point it will perform additional bounds checking. If indeed it is the end,
+the lexer will try to get more input. If this attempt fails, the lexer will
+either rollback to the last matched rule (or the default rule, if it has
+consumed some input but did not match), or else it will go to EOF rule.
+Depending on each particular case and the choice of sentinel, the performance of
+EOF rule may be slightly better or worse than bounds checking with padding.
+See the user manual for more details and examples.
+
+Another important new feature is the possibility to :ref:`include other files
+<manual/manual:include files>`. Re2c provides a directive
+``/*!include:re2c "file.re" */``, where ``file.re`` is the name of the included
+file.
+Re2c looks for included files in the directory of the including file and in the
+include locations, which can be specified with ``-I`` option. Include files
+may have further includes of their own. Re2c provides some predefined include
+files that can be found in the ``include/`` sub-directory of the project. These
+files contain useful definitions and form something like a standard library
+for re2c. Currently there is one available include, ``unicode_categories.re``.
+
+Re2c now allows to :ref:`generate header file <manual/manual:header files>`
+from the input file using option ``-t --type-header`` or
+``re2c:flags:t`` and ``re2c:flags:type-header`` configurations and the newly
+added directives ``/*!header:re2c:on*/`` and ``/*!header:re2c:off*/``.
+Header files may be needed in cases when re2c is used to generate definitions
+of constants, variables and structs that must be visible from other translation
+units.
+
+Re2c can now understand UTF8-encoded string literals and character classes in
+regular expressions. By default, re2c parses regular expressions like
+``"∀x ∃y"`` as a sequence of 1-byte ASCII code points
+``e2 88 80 78 20 e2 88 83 79`` (hexadecimal), and the users have to escape
+Unicode symbols manually: ``"\u2200x \u2203y"``. This is not what most users
+would expect, as demonstrated by multiple issues on the bugtracker. The new
+option ``--input-encoding <ascii | utf8>`` allows to change the default
+behavior and parse ``"∀x ∃y"`` as a sequence of 4-byte Unicode code points
+``2200 78 20 2203 79`` (hexadecimal). Note that the new option affects only
+parsing, and not the code generation.
+
+Re2c now allows to mix :ref:`reusable blocks <manual/manual:reusable blocks>`
+with normal blocks when ``-r --reuse`` option is used. This is very useful in
+cases when the input file contains many re2c blocks, and only some of them need
+to be reused (a good example is re2c own lexer, which has to use certain rules
+twice in order to generate ASCII and UTF8 lexer variants for
+``--input-encoding <ascii | utf8>`` option).
+
+It is now possible to specify location format in re2c-generated messages with
+``--location-format <gnu | msvc>`` option. As one might guess, possible
+arguments are GNU location format ``filename:line:column:`` (the default) and
+MSVC format ``filename(line,column)``. This option might help IDE users.
+Another new option is ``--verbose`` --- it prints a short "success" message in
+case re2c exits successfully.
+
+Flex compatibility mode (enabled with ``-F --flex-support`` option) has been
+improved: parsing errors and incorrect operator precedence in some rare cases
+have been fixed. Historically re2c allows to mix flex-style code with re2c-style
+code, which creates certain difficulties for the parser. Such difficulties can
+be resolved by passing parts of the parser context to the lexer.
+
+The difference operator ``/`` is now applied to its operands before encoding
+expansion. Re2c supports difference only for character classes, and encoding
+expansion in case of a variable-length encoding may transform even a simple
+character class into a complex graph (see
+`the graph <../../_images/utf8_any.png>`_ for ``[^]`` in UTF8 for example).
+Applying expansion after difference makes ``/`` applicable in more cases.
+
+Re2c now generates the output file atomically: it first creates a temporary file
+and writes the generated output into it, and then renames the temporary file to
+the output file.
+
+Documentation has been rearranged and updated, and the website has been
+fixed to look somewhat better on mobile devices (hopefully).
+
+From developers standpoint, re2c now has better debugging capabilities. Debug
+code has been moved in a separate subsystem and is no longer built in release
+binaries (debug builds are enabled with ``--enable-debug`` configure option).
+New debug options and passes have been added.
+
+This release has taken almost a year, mostly because of the time needed to
+formalize the extended algorithm for POSIX disambiguation and write a formal
+paper `Efficient POSIX Submatch Extraction on NFA
+<../../2019_borsotti_trofimovich_efficient_posix_submatch_extraction_on_nfa.pdf>`_.
+The algorithms described in the paper are implemented in an experimental
+library ``libre2c``. The library is intended as a playground for experiments
+and research, not a replacement for existing libraries like RE2. Having re2c
+as a library is convenient for benchmarking, testing and creating bindings to
+other languages.
+To build the library and benchmarks, configure re2c with ``--enable-libs`` option.
+
+Many people have contributed to this release.
+Angelo Borsotti co-authored the paper; his hard work has revealed multiple
+shortcomings and deficiencies in the algorithms used in re2c.
+Henri Salo helped with fuzz-testing re2c using the American Fuzzy Lop.
+Denis Naumov helped with Windows portability issues.
+Sergei Trofimovich fixed a number of bugs;
+he and Serghei Iakovlev helped with the Travis CI infrastructure.
+Wesley Terpstra used re2c in the open-source
+`wake build tool <https://github.com/sifive/wake>`_.
+Many thanks to them and all the others who contributed to this release!
+
+A lot of bugs have been fixed.
+See the changelog :ref:`releases/changelog/changelog:1.2 (2019-08-01)` for details.
div.document code {
font-size: 85%;
padding: 1px;
- border: 1px solid lightgrey;
+ border: 1px solid darkgray;
background-color: aliceblue;
}
/* :download: role; should look like a normal link rather than inline code block */
overflow: hidden;
}
div.re2c-toc-global {
- font-size: 110%;
+ font-size: 115%;
}
div.re2c-toc-local {
padding: 0.5em;
background-color: linen;
border: 1px solid darkgray;
overflow: auto;
- font-size: 110%;
+ font-size: 115%;
}
/* searchbox */
margin: 0;
}
body {
- font-size: 20px;
+ font-size: 150%;
}
}
+
+::selection {
+ background: lightsteelblue;
+}
projects / re2c / commitdiff