[MSVC] Mimic MSVC whitespace collapse for incompatible token pasting

In public MS headers for XAudio, clang would fail to generate a valid UUID due to the UUID components being combined with the '-' UUID separators. Clang would attempting to recover but would preserve the leading whitespace from the tokens after each failed paste leading to spaces creeping into the UUID and causing an error in the __declspace(uuid()) parsing.

Reference: Microsoft DirectX SDK (June 2010)\Include\XAudio2.h(51)

Resolves http://llvm.org/pr23071

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@235186 91177308-0d34-0410-b5e6-96231b3b80d8

diff --git a/lib/Lex/TokenLexer.cpp b/lib/Lex/TokenLexer.cpp
index 23d72814f31bfeb9d0807bf24e954217c2c25f4b..83efbab76139ac1a36551a22c2ff013ce8ba9f7f 100644 (file)
--- a/lib/Lex/TokenLexer.cpp
+++ b/lib/Lex/TokenLexer.cpp
@@ -521,6 +521,13 @@ bool TokenLexer::Lex(Token &Tok) {
 /// are more ## after it, chomp them iteratively.  Return the result as Tok.
 /// If this returns true, the caller should immediately return the token.
 bool TokenLexer::PasteTokens(Token &Tok) {
+  // MSVC: If previous token was pasted, this must be a recovery from an invalid
+  // paste operation. Ignore spaces before this token to mimic MSVC output.
+  // Required for generating valid UUID strings in some MS headers.
+  if (PP.getLangOpts().MicrosoftExt && (CurToken >= 2) &&
+      Tokens[CurToken - 2].is(tok::hashhash))
+    Tok.clearFlag(Token::LeadingSpace);
+  
   SmallString<128> Buffer;
   const char *ResultTokStrPtr = nullptr;
   SourceLocation StartLoc = Tok.getLocation();

diff --git a/test/Preprocessor/macro_paste_msextensions.c b/test/Preprocessor/macro_paste_msextensions.c
index afdcdbd493f04749879643a6649a35440e6e9fc0..aa5f41f9ee4fc5863876acf8984c09137a6e8fd6 100644 (file)
--- a/test/Preprocessor/macro_paste_msextensions.c
+++ b/test/Preprocessor/macro_paste_msextensions.c
@@ -32,3 +32,10 @@ nested(baz)  rise of the dead tokens
 bar(q)
 
 // CHECK: abc(baz(q))
+
+
+#define str(x) #x
+#define collapse_spaces(a, b, c, d) str(a ## - ## b ## - ## c ## d)
+collapse_spaces(1a, b2, 3c, d4)
+
+// CHECK: "1a-b2-3cd4"