From: Serhiy Storchaka <storchaka@gmail.com>
Date: Wed, 1 Oct 2014 21:11:21 +0000 (+0300)
Subject: Issue #20076: Added non derived UTF-8 aliases to locale aliases table.
X-Git-Tag: v3.5.0a1~791
X-Git-Url: https://granicus.if.org/sourcecode?a=commitdiff_plain;h=c383ad8d9c3a7866ea27667f14ffa99f18e905a3;p=python

Issue #20076: Added non derived UTF-8 aliases to locale aliases table.

The makelocalealias.py script no longer ignores UTF-8 mapping.
---

c383ad8d9c3a7866ea27667f14ffa99f18e905a3
diff --cc Misc/NEWS
index 8e94cdc329,182032afa9..97899c95e3
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@@ -156,17 -50,6 +156,19 @@@ Core and Builtin
  Library
  -------
  
++- Issue #20076: Added non derived UTF-8 aliases to locale aliases table.
++
 +- Issue #20079: Added locales supported in glibc 2.18 to locale alias table.
 +
 +- Issue #20218: Added convenience methods read_text/write_text and read_bytes/
 +  write_bytes to pathlib.Path objects.
 +
 +- Issue #22437: Number of capturing groups in regular expression is no longer
 +  limited by 100.
 +
 +- Issue #17442: InteractiveInterpreter now displays the full chained traceback
 +  in its showtraceback method, to match the built in interactive interpreter.
 +
  - Issue #10510: distutils register and upload methods now use HTML standards
    compliant CRLF line endings.
  
@@@ -1188,23 -871,6 +1190,25 @@@ Test
  Tools/Demos
  -----------
  
++- Issue #20076: The makelocalealias.py script no longer ignores UTF-8 mapping.
++
 +- Issue #20079: The makelocalealias.py script now can parse the SUPPORTED file
 +  from glibc sources and supports command line options for source paths.
 +
 +- Issue #22201: Command-line interface of the zipfile module now correctly
 +  extracts ZIP files with directory entries.  Patch by Ryan Wilson.
 +
 +- Issue #22120: For functions using an unsigned integer return converter,
 +  Argument Clinic now generates a cast to that type for the comparison
 +  to -1 in the generated code.  (This supresses a compilation warning.)
 +
 +- Issue #18974: Tools/scripts/diff.py now uses argparse instead of optparse.
 +
 +- Issue #21906: Make Tools/scripts/md5sum.py work in Python 3.
 +  Patch by Zachary Ware.
 +
 +- Issue #21629: Fix Argument Clinic's "--converters" feature.
 +
  - Add support for ``yield from`` to 2to3.
  
  - Add support for the PEP 465 matrix multiplication operator to 2to3.
diff --cc Tools/i18n/makelocalealias.py
index ca69daa7a8,10887ce338..980465b42c
--- a/Tools/i18n/makelocalealias.py
+++ b/Tools/i18n/makelocalealias.py
@@@ -45,41 -44,10 +45,37 @@@ def parse(filename)
              encoding = encoding.replace('-', '')
              encoding = encoding.replace('_', '')
              locale = lang + '.' + encoding
--            if encoding.lower() == 'utf8':
--                # Ignore UTF-8 mappings - this encoding should be
--                # available for all locales
--                continue
 +        data[locale] = alias
 +    return data
 +
 +def parse_glibc_supported(filename):
 +
 +    with open(filename, encoding='latin1') as f:
 +        lines = list(f)
 +    data = {}
 +    for line in lines:
 +        line = line.strip()
 +        if not line:
 +            continue
 +        if line[:1] == '#':
 +            continue
 +        if '/' not in line:
 +            continue
 +        line = line.rstrip('\\').rstrip()
 +        alias, _, alias_encoding = line.partition('/')
 +        # Lower-case locale
 +        locale = alias.lower()
 +        # Normalize encoding, if given
 +        if '.' in locale:
 +            lang, encoding = locale.split('.')[:2]
 +            encoding = encoding.replace('-', '')
 +            encoding = encoding.replace('_', '')
 +            locale = lang + '.' + encoding
 +        # Add an encoding to alias
 +        alias, _, modifier = alias.partition('@')
 +        alias = _locale._replace_encoding(alias, alias_encoding)
 +        if modifier and not (modifier == 'euro' and alias_encoding == 'ISO-8859-15'):
 +            alias += '@' + modifier
          data[locale] = alias
      return data