granicus.if.org Git - onig/blob - README.md

   1 [![Build Status](https://travis-ci.org/kkos/oniguruma.svg?branch=master)](https://travis-ci.org/kkos/oniguruma)
   2 [![Code Quality: Cpp](https://img.shields.io/lgtm/grade/cpp/g/kkos/oniguruma.svg?logo=lgtm&logoWidth=18)](https://lgtm.com/projects/g/kkos/oniguruma/context:cpp)
   3 [![Total Alerts](https://img.shields.io/lgtm/alerts/g/kkos/oniguruma.svg?logo=lgtm&logoWidth=18)](https://lgtm.com/projects/g/kkos/oniguruma/alerts)
   4
   5 Oniguruma
   6 =========
   7
   8 https://github.com/kkos/oniguruma
   9
  10 Oniguruma is a modern and flexible regular expressions library. It
  11 encompasses features from different regular expression implementations
  12 that traditionally exist in different languages.
  13
  14 Character encoding can be specified per regular expression object.
  15
  16 Supported character encodings:
  17
  18   ASCII, UTF-8, UTF-16BE, UTF-16LE, UTF-32BE, UTF-32LE,
  19   EUC-JP, EUC-TW, EUC-KR, EUC-CN,
  20   Shift_JIS, Big5, GB18030, KOI8-R, CP1251,
  21   ISO-8859-1, ISO-8859-2, ISO-8859-3, ISO-8859-4, ISO-8859-5,
  22   ISO-8859-6, ISO-8859-7, ISO-8859-8, ISO-8859-9, ISO-8859-10,
  23   ISO-8859-11, ISO-8859-13, ISO-8859-14, ISO-8859-15, ISO-8859-16
  24
  25 * GB18030: contributed by KUBO Takehiro
  26 * CP1251:  contributed by Byte
  27 * doc/SYNTAX.md: contributed by seanofw
  28
  29
  30 Master branch
  31 -------------
  32
  33 * NEW API: RegSet (set of regexes)
  34
  35
  36 Version 6.9.3 (security fix release)
  37 ------------------------------------
  38
  39 * Fixed CVE-2019-13224
  40 * Fixed CVE-2019-13225
  41 * Fixed CVE-2019-16163
  42 * Fixed many problems (found by libFuzzer programs)
  43
  44
  45 Version 6.9.2 (Reiwa)
  46 ---------------------
  47
  48 * add doc/SYNTAX.md
  49 * Direct threaded code (for GCC and Clang)
  50 * Update Unicode version 12.1.0
  51 * NEW: Unicode Text Segment mode option (?y{g}) (?y{w})  (*original)
  52
  53
  54 Version 6.9.1
  55 -------------
  56
  57 * Speed improvement (* especially UTF-8)
  58
  59
  60 Version 6.9.0
  61 -------------
  62
  63 * Update Unicode version 11.0.0
  64 * NEW: add Emoji properties
  65
  66
  67 Version 6.8.2
  68 -------------
  69
  70 * Fix: #80 UChar in header causes issue
  71 * NEW API: onig_set_callout_user_data_of_match_param()  (* omission in 6.8.0)
  72 * add doc/CALLOUTS.API and doc/CALLOUTS.API.ja
  73
  74
  75 Version 6.8.1
  76 -------------
  77
  78 * Update shared library version to 5.0.0 for API incompatible changes from 6.7.1
  79
  80
  81 Version 6.8.0
  82 -------------
  83
  84 * Retry-limit-in-match function enabled by default
  85 * NEW: configure option --enable-posix-api=no  (* enabled by default)
  86 * NEW API: onig_search_with_param(), onig_match_with_param()
  87 * NEW: Callouts of contents  (?{...contents...}) (?{...}\[tag]\[X<>]) (?{{...}})
  88 * NEW: Callouts of name      (*name) (*name\[tag]{args...})
  89 * NEW: Builtin callouts  (*FAIL) (*MISMATCH) (*ERROR{n}) (*COUNT) (*MAX{n}) etc..
  90 * Examples of Callouts program: [callout.c](sample/callout.c), [count.c](sample/count.c), [echo.c](sample/echo.c)
  91
  92
  93 Version 6.7.1
  94 -------------
  95
  96 * NEW: Mechanism of retry-limit-in-match (* disabled by default)
  97
  98
  99 Version 6.7.0
 100 -------------
 101
 102 * NEW: hexadecimal codepoint \uHHHH
 103 * NEW: add ONIG_SYNTAX_ONIGURUMA (== ONIG_SYNTAX_DEFAULT)
 104 * Disabled \N and \O on ONIG_SYNTAX_RUBY
 105 * Reduced size of object file
 106
 107
 108 Version 6.6.0
 109 -------------
 110
 111 * NEW: ASCII only mode options for character type/property (?WDSP)
 112 * NEW: Extended Grapheme Cluster boundary \y, \Y
 113 * NEW: Extended Grapheme Cluster \X
 114 * Range-clear (Absent-clear) operator restores previous range in retractions.
 115
 116
 117 Version 6.5.0
 118 -------------
 119
 120 * NEW: \K (keep)
 121 * NEW: \R (general newline) \N (no newline)
 122 * NEW: \O (true anychar)
 123 * NEW: if-then-else   (?(...)...\|...)
 124 * NEW: Backreference validity checker (?(xxx)) (*original)
 125 * NEW: Absent repeater (?~absent)  \[is equal to (?\~\|(?:absent)|\O*)]
 126 * NEW: Absent expression   (?~|absent|expr)  (*original)
 127 * NEW: Absent stopper (?~|absent)     (*original)
 128
 129
 130 Version 6.4.0
 131 -------------
 132
 133 * Fix fatal problem of endless repeat on Windows
 134 * NEW: call zero (call the total regexp) \g<0>
 135 * NEW: relative backref/call by positive number \k<+n>, \g<+n>
 136
 137
 138 Version 6.3.0
 139 -------------
 140
 141 * NEW: octal codepoint \o{.....}
 142 * Fixed CVE-2017-9224
 143 * Fixed CVE-2017-9225
 144 * Fixed CVE-2017-9226
 145 * Fixed CVE-2017-9227
 146 * Fixed CVE-2017-9228
 147 * Fixed CVE-2017-9229
 148
 149
 150 Version 6.1.2
 151 -------------
 152
 153 * allow word bound, word begin and word end in look-behind.
 154 * NEW option: ONIG_OPTION_CHECK_VALIDITY_OF_STRING
 155
 156 Version 6.1
 157 -----------
 158
 159 * improved doc/RE
 160 * NEW API: onig_scan()
 161
 162 Version 6.0
 163 -----------
 164
 165 * Update Unicode 8.0 Property/Case-folding
 166 * NEW API: onig_unicode_define_user_property()
 167
 168
 169 License
 170 -------
 171
 172   BSD license.
 173
 174
 175 Install
 176 -------
 177
 178 ### Case 1: Unix and Cygwin platform
 179
 180    1. autoreconf -vfi   (* case: configure script is not found.)
 181
 182    2. ./configure
 183    3. make
 184    4. make install
 185
 186    * uninstall
 187
 188      make uninstall
 189
 190    * configuration check
 191
 192      onig-config --cflags
 193      onig-config --libs
 194      onig-config --prefix
 195      onig-config --exec-prefix
 196
 197
 198
 199 ### Case 2: Windows 64/32bit platform (Visual Studio)
 200
 201    Execute make_win.bat
 202
 203       onig_s.lib:  static link library
 204       onig.dll:    dynamic link library
 205
 206    * test (ASCII/Shift_JIS)
 207
 208       1. cd src
 209       2. copy ..\windows\testc.c .
 210       3. nmake -f Makefile.windows ctest
 211
 212    (I have checked by Visual Studio Community 2015)
 213
 214
 215
 216 Regular Expressions
 217 -------------------
 218
 219   See [doc/RE](doc/RE) or [doc/RE.ja](doc/RE.ja) for Japanese.
 220
 221
 222 Usage
 223 -----
 224
 225   Include oniguruma.h in your program. (Oniguruma API)
 226   See doc/API for Oniguruma API.
 227
 228   If you want to disable UChar type (== unsigned char) definition
 229   in oniguruma.h, define ONIG_ESCAPE_UCHAR_COLLISION and then
 230   include oniguruma.h.
 231
 232   If you want to disable regex_t type definition in oniguruma.h,
 233   define ONIG_ESCAPE_REGEX_T_COLLISION and then include oniguruma.h.
 234
 235   Example of the compiling/linking command line in Unix or Cygwin,
 236   (prefix == /usr/local case)
 237
 238     cc sample.c -L/usr/local/lib -lonig
 239
 240
 241   If you want to use static link library(onig_s.lib) in Win32,
 242   add option -DONIG_EXTERN=extern to C compiler.
 243
 244
 245
 246 Sample Programs
 247 ---------------
 248
 249 |File                  |Description                               |
 250 |:---------------------|:-----------------------------------------|
 251 |sample/callout.c      |example of callouts                       |
 252 |sample/count.c        |example of built-in callout *COUNT        |
 253 |sample/echo.c         |example of user defined callouts of name  |
 254 |sample/encode.c       |example of some encodings                 |
 255 |sample/listcap.c      |example of the capture history            |
 256 |sample/names.c        |example of the named group callback       |
 257 |sample/posix.c        |POSIX API sample                          |
 258 |sample/regset.c       |example of using RegSet API               |
 259 |sample/scan.c         |example of using onig_scan()              |
 260 |sample/simple.c       |example of the minimum (Oniguruma API)    |
 261 |sample/sql.c          |example of the variable meta characters   |
 262 |sample/user_property.c|example of user defined Unicode property  |
 263
 264
 265 Test Programs
 266
 267 |File               |Description                            |
 268 |:------------------|:--------------------------------------|
 269 |sample/syntax.c    |Perl, Java and ASIS syntax test.       |
 270 |sample/crnl.c      |--enable-crnl-as-line-terminator test  |
 271
 272
 273
 274 Source Files
 275 ------------
 276
 277 |File               |Description                                             |
 278 |:------------------|:-------------------------------------------------------|
 279 |oniguruma.h        |Oniguruma API header file (public)                      |
 280 |onig-config.in     |configuration check program template                    |
 281 |regenc.h           |character encodings framework header file               |
 282 |regint.h           |internal definitions                                    |
 283 |regparse.h         |internal definitions for regparse.c and regcomp.c       |
 284 |regcomp.c          |compiling and optimization functions                    |
 285 |regenc.c           |character encodings framework                           |
 286 |regerror.c         |error message function                                  |
 287 |regext.c           |extended API functions (deluxe version API)             |
 288 |regexec.c          |search and match functions                              |
 289 |regparse.c         |parsing functions.                                      |
 290 |regsyntax.c        |pattern syntax functions and built-in syntax definitions|
 291 |regtrav.c          |capture history tree data traverse functions            |
 292 |regversion.c       |version info function                                   |
 293 |st.h               |hash table functions header file                        |
 294 |st.c               |hash table functions                                    |
 295 |oniggnu.h          |GNU regex API header file (public)                      |
 296 |reggnu.c           |GNU regex API functions                                 |
 297 |onigposix.h        |POSIX API header file (public)                          |
 298 |regposerr.c        |POSIX error message function                            |
 299 |regposix.c         |POSIX API functions                                     |
 300 |mktable.c          |character type table generator                          |
 301 |ascii.c            |ASCII encoding                                          |
 302 |euc_jp.c           |EUC-JP encoding                                         |
 303 |euc_tw.c           |EUC-TW encoding                                         |
 304 |euc_kr.c           |EUC-KR, EUC-CN encoding                                 |
 305 |sjis.c             |Shift_JIS encoding                                      |
 306 |big5.c             |Big5      encoding                                      |
 307 |gb18030.c          |GB18030   encoding                                      |
 308 |koi8.c             |KOI8      encoding                                      |
 309 |koi8_r.c           |KOI8-R    encoding                                      |
 310 |cp1251.c           |CP1251    encoding                                      |
 311 |iso8859_1.c        |ISO-8859-1 (Latin-1)                                    |
 312 |iso8859_2.c        |ISO-8859-2 (Latin-2)                                    |
 313 |iso8859_3.c        |ISO-8859-3 (Latin-3)                                    |
 314 |iso8859_4.c        |ISO-8859-4 (Latin-4)                                    |
 315 |iso8859_5.c        |ISO-8859-5 (Cyrillic)                                   |
 316 |iso8859_6.c        |ISO-8859-6 (Arabic)                                     |
 317 |iso8859_7.c        |ISO-8859-7 (Greek)                                      |
 318 |iso8859_8.c        |ISO-8859-8 (Hebrew)                                     |
 319 |iso8859_9.c        |ISO-8859-9 (Latin-5 or Turkish)                         |
 320 |iso8859_10.c       |ISO-8859-10 (Latin-6 or Nordic)                         |
 321 |iso8859_11.c       |ISO-8859-11 (Thai)                                      |
 322 |iso8859_13.c       |ISO-8859-13 (Latin-7 or Baltic Rim)                     |
 323 |iso8859_14.c       |ISO-8859-14 (Latin-8 or Celtic)                         |
 324 |iso8859_15.c       |ISO-8859-15 (Latin-9 or West European with Euro)        |
 325 |iso8859_16.c       |ISO-8859-16 (Latin-10)                                  |
 326 |utf8.c             |UTF-8    encoding                                       |
 327 |utf16_be.c         |UTF-16BE encoding                                       |
 328 |utf16_le.c         |UTF-16LE encoding                                       |
 329 |utf32_be.c         |UTF-32BE encoding                                       |
 330 |utf32_le.c         |UTF-32LE encoding                                       |
 331 |unicode.c          |common codes of Unicode encoding                        |
 332 |unicode_fold_data.c|Unicode folding data                                    |
 333 |windows/testc.c    |Test program for Windows (VC++)                        |