From: Joel E. Denny <jdenny.ornl@gmail.com>
Date: Sat, 12 Oct 2019 14:58:43 +0000 (+0000)
Subject: [lit] Try again to fix new tests that fail on Windows bots
X-Git-Url: https://granicus.if.org/sourcecode?a=commitdiff_plain;h=383b3a14b606715a72e9f6d3e3b29a4974692c17;p=llvm

[lit] Try again to fix new tests that fail on Windows bots

Based on the bot logs, when lit's internal diff runs on Windows, it
looks like binary diffs must be decoded also for Python 2.7.
Otherwise, writing the diff to stdout fails with:

```
UnicodeEncodeError: 'ascii' codec can't encode characters in position 7-8: ordinal not in range(128)
```

I did not need to decode using Python 2.7.15 under Ubuntu.  When I do
it anyway in that case, `errors="backslashreplace"` fails for me:

```
TypeError: don't know how to handle UnicodeDecodeError in error callback
```

However, `errors="ignore"` works, so this patch uses that, hoping
it'll work on Windows as well.

This patch leaves `errors="backslashreplace"` for Python >= 3.5 as
there's no evidence yet that doesn't work and it produces more
informative binary diffs.  This patch also adjusts some lit tests to
succeed for either error handler.

This patch adjusts changes introduced by D68664.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@374657 91177308-0d34-0410-b5e6-96231b3b80d8
---

diff --git a/utils/lit/lit/builtin_commands/diff.py b/utils/lit/lit/builtin_commands/diff.py
index f413f87425c..338a01474ac 100644
--- a/utils/lit/lit/builtin_commands/diff.py
+++ b/utils/lit/lit/builtin_commands/diff.py
@@ -62,6 +62,7 @@ def compareTwoBinaryFiles(flags, filepaths, filelines):
             func = difflib.context_diff
         diffs = func(filelines[0], filelines[1], filepaths[0], filepaths[1],
                      n = flags.num_context_lines)
+        diffs = [diff.decode(errors="ignore") for diff in diffs]
 
     for diff in diffs:
         sys.stdout.write(diff)
diff --git a/utils/lit/tests/shtest-shell.py b/utils/lit/tests/shtest-shell.py
index 9a6f2aac0a4..fb290f52b5f 100644
--- a/utils/lit/tests/shtest-shell.py
+++ b/utils/lit/tests/shtest-shell.py
@@ -48,7 +48,7 @@
 # CHECK-NEXT: @@
 # CHECK-NEXT: {{^ .f.o.o.$}}
 # CHECK-NEXT: {{^-.b.a.r.$}}
-# CHECK-NEXT: {{^\+.b.a.r..}}
+# CHECK-NEXT: {{^\+.b.a.r.}}
 # CHECK-NEXT: {{^ .b.a.z.$}}
 # CHECK: error: command failed with exit status: 1
 # CHECK: $ "true"
@@ -62,7 +62,7 @@
 # CHECK-NEXT: -bar
 # CHECK-NEXT: -baz
 # CHECK-NEXT: {{^\+.f.o.o.$}}
-# CHECK-NEXT: {{^\+.b.a.r..}}
+# CHECK-NEXT: {{^\+.b.a.r.}}
 # CHECK-NEXT: {{^\+.b.a.z.$}}
 # CHECK: error: command failed with exit status: 1
 # CHECK: $ "true"
@@ -73,7 +73,7 @@
 # CHECK-NEXT: +++
 # CHECK-NEXT: @@
 # CHECK-NEXT: {{^\-.f.o.o.$}}
-# CHECK-NEXT: {{^\-.b.a.r..}}
+# CHECK-NEXT: {{^\-.b.a.r.}}
 # CHECK-NEXT: {{^\-.b.a.z.$}}
 # CHECK-NEXT: +foo
 # CHECK-NEXT: +bar
@@ -100,7 +100,7 @@
 # CHECK-NEXT: @@
 # CHECK-NEXT: {{^ .f.o.o.$}}
 # CHECK-NEXT: {{^-.b.a.r.$}}
-# CHECK-NEXT: {{^\+.b.a.r..}}
+# CHECK-NEXT: {{^\+.b.a.r.}}
 # CHECK-NEXT: {{^ .b.a.z.$}}
 # CHECK: error: command failed with exit status: 1
 # CHECK: $ "true"
@@ -116,7 +116,7 @@
 # CHECK-NEXT: -bar
 # CHECK-NEXT: -baz
 # CHECK-NEXT: {{^\+.f.o.o.$}}
-# CHECK-NEXT: {{^\+.b.a.r..}}
+# CHECK-NEXT: {{^\+.b.a.r.}}
 # CHECK-NEXT: {{^\+.b.a.z.$}}
 # CHECK: error: command failed with exit status: 1
 # CHECK: $ "true"
@@ -127,7 +127,7 @@
 # CHECK-NEXT: +++
 # CHECK-NEXT: @@
 # CHECK-NEXT: {{^\-.f.o.o.$}}
-# CHECK-NEXT: {{^\-.b.a.r..}}
+# CHECK-NEXT: {{^\-.b.a.r.}}
 # CHECK-NEXT: {{^\-.b.a.z.$}}
 # CHECK-NEXT: +foo
 # CHECK-NEXT: +bar