]> granicus.if.org Git - clang/commit
UTF-8 support for clang-format.
authorAlexander Kornienko <alexfh@google.com>
Wed, 5 Jun 2013 14:09:10 +0000 (14:09 +0000)
committerAlexander Kornienko <alexfh@google.com>
Wed, 5 Jun 2013 14:09:10 +0000 (14:09 +0000)
commit00895106f9ed602af67984ec4d225a0cdc8c12af
tree071d223dd44a5c81a04ce01eebd1129bc00a8571
parent2aae5ba15d07e9aa37ff9b03ad0024c0707068f5
UTF-8 support for clang-format.

Summary:
Detect if the file is valid UTF-8, and if this is the case, count code
points instead of just using number of bytes in all (hopefully) places, where
number of columns is needed. In particular, use the new
FormatToken.CodePointCount instead of TokenLength where appropriate.
Changed BreakableToken implementations to respect utf-8 character boundaries
when in utf-8 mode.

Reviewers: klimek, djasper

Reviewed By: djasper

CC: cfe-commits, rsmith, gribozavr
Differential Revision: http://llvm-reviews.chandlerc.com/D918

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@183312 91177308-0d34-0410-b5e6-96231b3b80d8
lib/Format/BreakableToken.cpp
lib/Format/BreakableToken.h
lib/Format/Encoding.h [new file with mode: 0644]
lib/Format/Format.cpp
lib/Format/FormatToken.h
lib/Format/TokenAnnotator.cpp
lib/Format/TokenAnnotator.h
unittests/Format/FormatTest.cpp