2 .. role:: file(literal)
12 if socket.gethostname() == 'phenix':
15 bpi = '.progiciels-bpi.ca'
28 Here is version 3.6 for the Recode program and library. Hereafter,
29 Recode means the whole package, :code:`recode` means the executable
30 program. Glance through this :file:`README` file before starting
31 configuration. Make sure you read files :file:`ABOUT-NLS` and
32 :file:`INSTALL` if you are not familiar with them already.
34 The Recode library converts files between character sets and usages.
35 It recognises or produces over 200 different character sets (or about
36 300 if combined with an :code:`iconv` library) and transliterates files
37 between almost any pair. When exact transliteration are not possible,
38 it gets rid of offending characters or falls back on approximations.
39 The :code:`recode` program is a handy front-end to the library.
41 The Recode program and library have been written by François Pinard,
42 yet it significantly reuses tabular works from Keld Simonsen. It is an
43 evolving package, and specifications might change in future releases.
45 On various Unix systems, Recode is usually compiled from sources,
46 see the `Installation`_ section below. On Linux, it often comes
47 bundled. Recode had been ported to other popular systems. See both
48 `contrib/README`__ and the `Non-Unix ports`_ section below, to find
49 some more information about these.
53 Reports and collaboration
54 -------------------------
56 Send bug reports to mailto:recode-bugs@iro.umontreal.ca' . A bug
57 report is an adequate description of the problem: your input, what you
58 expected, what you got, and why this is wrong. Diffs are welcome, but
59 they only describe a solution, from which the problem might be uneasy
60 to infer. If needed, submit actual data files with your report. Small
61 data files are preferred. Big files may sometimes be necessary, but do
62 not send them on the mailing list; rather take special arrangement with
65 Your feedback will help us to make a better and more portable
66 package. Consider documentation errors as bugs, and report them
67 as such. If you develop anything pertaining to Recode or have
68 suggestions, let us know and share your findings by writing at
69 mailto:recode-forum@iro.umontreal.ca . You may also choose to directly
70 write at mailto:pinard@iro.umontreal.ca, yet be warned that such
71 correspondence is often visible for a while through the Recode Web site.
73 If you feel like receiving releases and pretest announcements for the
74 Recode package, send a message to mailto:majordomo@iro.umontreal.ca
75 having, in its body, a line saying::
77 subscribe recode-announce
79 If you rather want to participate actively in discussions, pretesting
80 and development for Recode, do just as above, but this time, use::
82 subscribe recode-forum
84 Visit http://recode%(bpi)s/ for releases or pretests, and related
85 files. In particular, button ``Browse`` gives access to a weekly mirror
86 of the current unpackaged work files, while button ``Folders`` gives
87 access to saved or pending correspondence.
89 Please *do not* widely redistribute releases having a letter after the
90 version numbers, as these are meant for pretesting only, and might not
91 be stable enough for other usages.
96 My plan has long been to end the 3.x series of this package, rather
97 aiming 4.0 as a major internal rewrite. As there is still a long
98 way before 4.0 gets ready, and *especially* because some of my good
99 collaborators insisted that I do so, there will be a Recode 3.7. That
100 release is meant to provide a selection of user-contributed patches.
102 For prototyping what Recode will become and experimenting new concepts
103 more easily, I created a subsidiary and standalone project named
104 Recodec, meant to receive the best part of my development efforts in
105 this particular area. Once I'll be happy with the prototype, the plan
106 is to rewrite it from Python to C, somehow. Visit the Web pages for
107 this `Recodec project`__ for more information and details. For now at
108 least, new features go to Recodec only.
110 __ http://recodec.progiciels-bpi.ca
112 Notes for version 3.7-beta2
113 ---------------------------
115 Here are a few notes related to the beta2 pre-test release for the incoming
116 Recode 3.7. I publish it to ease later exchanges of patches with testers.
118 + The name has been changed from Free recode to Recode -- as "Free" was
119 a four letter word to some people :-). :code:`recode` (no capital) still
120 names the executable program specifically, or the distribution archive
123 + Recode does not itself include :code:`libiconv` anymore. However,
124 it uses an external :code:`iconv` library if one is available at
125 installation time, like :code:`libiconv` or the one provided within GNU
126 :code:`libc`. The ``-x:`` option to the program, or a new flag to the
127 library :code:`recode_new_outer` function, inhibits the initialisation
128 and usage of :code:`iconv`.
130 + The bug about loosing a few characters, here and there, when recoding
131 big files in :code:`iconv` context, seems to have been corrected. A
132 patch for this problem has been floating around for years, but it was
133 not solving all cases.
135 + Recode installation now uses Python. In particular, it creates
136 file :file:`build/src/iconvdecl.h` from local ``iconv -l`` output.
137 Recode testing through ``make check`` also needs what people
138 :code:`python-devel`, providing C header files for Python and
139 :code:`distutils`. The :file:`Makemore` file has been merged within
140 regular Makefiles and is not distributed separately anymore.
142 + It is likely that new bugs have been introduced through the above
143 changes. In particular, not everything is cosy on the side of release
144 engineering. A few files are either spuriously remade, or remade late.
145 I'm a bit surprised by the difficulty to get this right.
147 + ``make check`` accepts a ``LIMIT=`` option, for limiting tests to one or
148 a few cases. See :file:`tests/Makefile` for more information.
150 + PO files have been updated from the Translation Project.
152 Notes for version 3.7-beta1
153 ---------------------------
155 The beta 1 pre-test release for the incoming Recode 3.7 has been made
156 available for those needing it right away. While it solves some serious
157 bugs and portability problems, others are meant to be addressed only in
158 later pre-tests. In particular, none of charset or surface issues, user
159 requests, and various suggestions appear in this pre-test, and will not
160 either in later pretests, until all real show-stoppers are solved first.
161 So this is in no way a candidate for a Recode 3.7 release.
163 The test suite is worth more comments:
165 + The suite is very partial, and may not be thought as a validation
166 suite. Before it could be used to ascertain confidence, it would need
167 much more tests than it has already.
169 + Testing is notably more speedy than it used to be. For example, the
170 previous :code:`bigauto` test, which was not run by default because it
171 ran for too long, is now executed within the standard test suite, once
172 in non-strict mode, and a second time in strict mode.
174 + It does not use Autotest anymore, but rather a home grown test driver
175 much inspired from the Codespeak project. The link between the test and
176 the Recode library is established through a Pyrex interface, so you need
177 to have :code:`python` and :code:`python-devel` installed first.
179 + Beware that the Pyrex interface to the Recode library is only meant
180 for testing. for now at least. While you may play with it, it would not
181 be wise relying on it, as the specifications might change at any time.
189 Simple installation of Recode requires the usual tools and facilities as
190 those needed for most GNU packages. If not already bundled with your
191 system, you also need to pre-install Python, version 2.2 or better. You
194 http://www.python.org
196 It is also convenient to have some :code:`iconv` library already present
197 on your system, this much extends Recode capabilities, especially in
198 the area of Asiatic character sets. GNU :code:`libc`, as found on
199 Linux systems and a few others, already has such an :code:`iconv`
200 library. Otherwise, you might consider pre-installing the portable
201 :code:`libiconv`, written by Bruno Haible. You may get it from:
203 http://www.gnu.org/software/libiconv/
205 Getting your own copy
206 ---------------------
208 The canonical distribution point for this version is:
210 http://recode%(bpi)s/archives/recode.tar.gz
212 GNU mirrors usually hold a copy of non-pretest releases, the canonical
213 distribution point for the last such release is:
215 ftp://ftp.gnu.org/pub/gnu/recode/recode-3.6.tar.gz
217 Some older distributions, and maybe beta pre-releases for testers,
218 *might* be available in this directory:
220 http://recode%(bpi)s/archives/
222 There is a reachable copy of my own sandbox repository for Recode. I
223 experiment with it, and branches may be reset at any time. The server
224 might not even be up (then just tell me!). To get your own copy, try
227 git clone git://recode%(bpi)s/recode
231 (or ``python after-patch.py`` if you miss either :code:`sh` or GNU
234 Once you have an unpacked distribution, see files:
236 =================== =======================================================
237 File name Description
238 =================== =======================================================
239 :file:`ABOUT-NLS` how to customise this program to your language
240 :file:`COPYING` copying conditions for the program
241 :file:`COPYING.LIB` copying conditions for the library
242 :file:`INSTALL` compilation and installation instructions
243 :file:`NEWS` major changes in the current release
244 :file:`THANKS` partial list of contributors
245 =================== =======================================================
250 Besides those configure options documented in files :file:`INSTALL`
251 and :file:`ABOUT-NLS`, a few extra options may be accepted after
254 + Options ``--disable-shared`` or ``--disable-static``
256 to inhibit the building of shared libraries or static libraries; the
257 default is to always build static libraries, and to attempt building
258 shared libraries if there is some known recipe for this.
260 + Option ``--with-gnu-ld``
262 to force the assumption that the C compiler uses GNU ld.
264 + Option ``--with-dmalloc``
266 to trigger a debugging feature for looking at memory management
267 problems, it pre-requires Gray Watson's package, which is available as
268 ftp://ftp.letters.com/src/dmalloc/dmalloc.tar.gz .
273 For simple modifications to Recode, you should not need special tools
274 beyond those usual for installing GNU packages. However, if you modify
275 any :file:`.l` source file, Python and Flex are both needed for remaking
278 For more comprehensive modifications, you might need more tools. If not
279 done already, make sure you have a copy of the packages listed in the
280 following table. You may also choose to establish a link in your build
281 :file:`doc/` directory, as explained within :file:`doc/Makemore`.
283 ================ ========== ========== =============
284 Package name Current Minimum Install after
285 ================ ========== ========== =============
286 :code:`autoconf` 2.61 2.12 :code:`m4`
287 :code:`automake` 1.10 1.9 :code:`Perl`
288 :code:`Flex` 2.5.33 2.5.4a
289 :code:`gettext` 0.16 0.16
290 :code:`Help2man` 1.36 1.020 :code:`Perl`
291 :code:`libtool` 1.5.24 1.3.4
292 :code:`m4` 1.4.10 1.4n
294 :code:`Perl` 5.8.8 5.005.03
295 :code:`Python` 2.5.1 2.2
296 :code:`tar` 1.17 1.12
298 ================ ========== ========== =============
300 The *current* version numbers just happen to be those used for
301 development, it is often likely that older versions would work just as
302 well. The *minimum* version numbers were once acceptable, they might
303 not be anymore, this has not been verified; any updating information is
309 Here are a few hints which might help installing Recode on some systems.
310 Many may be applied by temporary presetting environment variables while
311 calling ``./configure``. File :file:`INSTALL` explains this.
315 Some C compilers, like Apollo's, have a hard time compiling
316 :file:`merged.c`. If this is your case, avoid compiler
317 optimisation. From within the Bourne shell, you may use::
321 But if you want to give a real hard time to your C optimiser on
322 :file:`merged.c`, to get code that runs only a bit faster, merely
325 CPPFLAGS=-DINLINE_HARDER ./configure
329 For 80286 based systems (do some still exist?!), it has been
330 reported that some compilers generate wrong code while optimising
331 for *small* models. So, from within the Bourne shell, do::
333 CFLAGS=-Ml LDFLAGS=-Ml ./configure
335 to force large memory model. For 80286 Xenix compiler, the last time
336 it was tried a while ago, one ought to use::
338 CFLAGS='-Ml -F2000' LDFLAGS=-Ml ./configure
340 Other systems have poor :code:`pipe`/:code:`popen` support or thrash
341 heavily when processes fork. In this case, just before doing
342 ``make``, edit :file:`config.h` and ensure :code:`HAVE_PIPE` is
353 + Character Mnemonics & Character Sets
355 + ftp://nic.ddn.mil/rfc/rfc1345.txt
357 Keld Simonsen <keld@dkuug.dk>, 1992-06.
359 + UTF-7 - A Mail-Safe Transformation Format of Unicode
361 + ftp://nic.ddn.mil/rfc/rfc1642.txt
363 David Goldsmith <david_goldsmith@taligent.com> and Mark Davis
364 <ark_davis@taligent.com>, 1994-07.
366 + UTF-8, a transformation format of Unicode and ISO 10646
368 + ftp://nic.ddn.mil/rfc/rfc2044.txt
370 François Yergeau <yergeau@alis.com>, 1997-10.
374 + Unicode charset mappings
376 + ftp://ftp.unicode.org:/Public/MAPPINGS/
378 The Unicode consortium makes available plenty of charset mappings
379 for converting "legacy" charsets to Unicode.
381 + Normalisation et internationalisation: Inventaire et prospectives des
382 normes clefs pour le traitement informatique du français. (392p.)
384 This is a report, written in French, discussing charset issues and many
385 other topics as well. Laurent Bourbeau <bourbeau@progiciels-bpi.ca>
386 and François Pinard <pinard@iro.umontreal.ca>, 1995-10.
388 + ftp://ftp.iro.umontreal.ca/pub/contrib/pinard/accents/oqil-tome1.ps.gz
389 + http://www.ceveil.qc.ca/Normes
395 In 1999, the organisers of the `m17n99 conference`__ in Tsukuba,
396 Japan, were kind enough to invite me. This has been for me a
397 fabulous trip and experience, and I met many extraordinary people in
398 there. At the conference, I presented the Translation Project, and
399 Recode. The Recode `presentation slides`__ are available.
401 __ http://www.m17n.org/conference/m17n99_all_but_registration/welcome.en.html
409 This comprehensive charset converter library revolves around Unicode,
410 and support Asian encodings among many others. Even Recode uses it!
412 + http://www.gnu.org/software/libiconv/
414 Bruno Haible <haible@ilog.fr>
418 Here is the main recoding tool from the Plan9 project.
420 + ftp://research.att.com/dist/tcs.shar.Z
424 This GUI editor handles many encodings, among which UTF-8. It also
425 installs uniconv, a recoding program, and uniprint, a printing tool.
427 + ftp://sunsite.unc.edu/pub/Linux/apps/editors/X/yudit-1.2.tar.gz
429 Gaspar Sinai <gsinai@iname.com>, 1999-01.
433 These 6x13 fonts, covering Unicode characters besides the Asian sets,
434 merely replace the Linux fixed 6x13 font. Works nicely with yudit.
436 + http://www.cl.cam.ac.uk/~mgk25/download/ucs-fonts.tar.gz
438 Markus Kuhn <Markus.Kuhn@cl.cam.ac.uk>, 1998-11.
442 This charset converter is oriented towards SGML text manipulation. It
443 may be freely downloaded for non-commercial, non-military use from:
445 + http://www.lpl.univ-aix.fr/projects/multext/MtRecode/
447 Pointer given by Jean Véronis <veronis@univ-aix.fr>, 1996-06.
451 This quite nice SGML structure analyser contains internal C++ modules
452 for handling many charsets.
454 + ftp://ftp.jclark.com/pub/sp/sp-1.3.tar.gz
456 James Clark <jjc@jclark.com>
460 This program is able to generate interpreted
461 character dumps, but properly embedded within complete C header files.
463 + http://research.de.uu.net:8080/~gnu/b2c/b2c-2.1.tar.gz
465 Jörg Heitkötter <Joerg.Heitkoetter@de.uu.net>, 1997-11.
469 This wrapper provides Recode functionality to Python programs.
471 + http://www.suxers.de/PyRecode.tgz
473 Andreas Jung <ajung@server.python.net>
477 + http://www.vex.net/parnassus/apyllo.py?find=recode
478 + http://www.suxers.de/python/pyrecode.htm
483 Please mailto:recode-bugs@iro.umontreal.ca if you are aware of various
484 ports to non-Unix systems not listed here, or for corrections. Please
485 provide the goal system, a complete and stable URL, the maintainer name
486 and address, the Recode version used as a base, and your comments.
490 Juan Manuel Guerrero <juan.guerrero@gmx.de> maintains this port,
491 dated 2001-03 and based on Recode 3.5. The following archives hold
492 binaries, docs and sources respectively.
494 + ftp://ftp.simtel.net/pub/simtelnet/gnu/djgpp/v2gnu/rcode35b.zip
495 + ftp://ftp.simtel.net/pub/simtelnet/gnu/djgpp/v2gnu/rcode35d.zip
496 + ftp://ftp.simtel.net/pub/simtelnet/gnu/djgpp/v2gnu/rcode35s.zip
498 See `contrib/DJGPP/README`__ in the Recode distribution for more
499 information about compiling this port.
505 Darrel Hankerson <hankedr@mail.auburn.edu> maintains this port, dated
506 1994-11 and based on Recode 3.4. You get many GNU tools, not only
507 Recode. The GNUish project is described in :file:`gnuish_t.htm`.
509 + http://www.simtel.net/simtel.net/
510 + http://www.leo.org/pub/comp/platforms/pc/gnuish (Germany)
511 + ftp://ftp.simtel.net/simtelnet/gnu
512 + ftp://ftp.leo.org/pub/comp/platforms/pc/gnuish
514 + OS/2 (using emx/gcc)
516 Maintainer unknown (maybe Kai Uwe Rommel <rommel@ars.de>), dated
517 1994-11 and based on Recode 3.4.
519 + http://hobbes.nmsu.edu/pub/os2/util/convert/gnurcode.zip