Lex: Don't restrict legal UCNs when preprocessing assembly

The C and C++ standards disallow using universal character names to
refer to some characters, such as basic ascii and control characters,
so we reject these sequences in the lexer. However, when the
preprocessor isn't being used on C or C++, it doesn't make sense to
apply these restrictions.

Notably, accepting these characters avoids issues with unicode escapes
when GHC uses the compiler as a preprocessor on haskell sources.

Fixes rdar://problem/14742289

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@193067 91177308-0d34-0410-b5e6-96231b3b80d8

diff --git a/lib/Lex/Lexer.cpp b/lib/Lex/Lexer.cpp
index 3817afe75d1383947d067975f513f76e22e69bf8..c071455da6623173b2d9b4b96946ef58658dad51 100644 (file)
--- a/lib/Lex/Lexer.cpp
+++ b/lib/Lex/Lexer.cpp
@@ -2730,6 +2730,10 @@ uint32_t Lexer::tryReadUCN(const char *&StartPtr, const char *SlashLoc,
     StartPtr = CurPtr;
   }
 
+  // Don't apply C family restrictions to UCNs in assembly mode
+  if (LangOpts.AsmPreprocessor)
+    return CodePoint;
+
   // C99 6.4.3p2: A universal character name shall not specify a character whose
   //   short identifier is less than 00A0 other than 0024 ($), 0040 (@), or
   //   0060 (`), nor one in the range D800 through DFFF inclusive.)

diff --git a/test/Preprocessor/assembler-with-cpp.c b/test/Preprocessor/assembler-with-cpp.c
index 65be564786129bb226e1cc8bda93eab10c764459..f03cb06ea115877404cda6fbb7117609e71880ae 100644 (file)
--- a/test/Preprocessor/assembler-with-cpp.c
+++ b/test/Preprocessor/assembler-with-cpp.c
@@ -72,6 +72,9 @@
 11: T11(b)
 // CHECK-Identifiers-True: 11: #0
 
+// Universal character names can specify basic ascii and control characters
+12: \u0020\u0030\u0080\u0000
+// CHECK-Identifiers-False: 12: \u0020\u0030\u0080\u0000
 
 // This should not crash
 // rdar://8823139