[Driver][Bundler] Improve bundling of object files.

Summary:
Previously, object files were bundled using partial linking. It resulted
in the following structure of the bundled objects:
```
<host_code>
clang-offload-bundle
__CLANG_OFFLOAD_BUNDLE__<target>
<target_code>
```
But when we tried to unbundle object files, it worked correctly only for
the target objects. The host object remains bundled. It produced a lot of
junk sections in the host object files and in some cases may caused
incorrect linking.

Patch improves bundling of the object files. After this patch the
bundled object looks like this:

```
<host_code>
clang-offload-bundle
__CLANG_OFFLOAD_BUNDLE__<target>
<target_code>
__CLANG_OFFLOAD_BUNDLE__<host>
<host_code>
```

With this structure we are able to unbundle the host object files too so
that after unbundling they are the same as were before.
The host section is bundled twice. The bundled section is used to
unbundle the original host section.

Reviewers: yaxunl, tra, jlebar, hfinkel, jdoerfert

Subscribers: caomhin, kkwli0, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D65819

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@369019 91177308-0d34-0410-b5e6-96231b3b80d8

diff --git a/test/Driver/clang-offload-bundler.c b/test/Driver/clang-offload-bundler.c
index f0f570bb198a3df952664640fef9b25eb917a02c..620a5553f47e3cee34d53b0bf45eb240c0a0461d 100644 (file)
--- a/test/Driver/clang-offload-bundler.c
+++ b/test/Driver/clang-offload-bundler.c
@@ -231,18 +231,18 @@
 
 // RUN: clang-offload-bundler -type=o -targets=host-%itanium_abi_triple,openmp-powerpc64le-ibm-linux-gnu,openmp-x86_64-pc-linux-gnu -inputs=%t.o,%t.tgt1,%t.tgt2 -outputs=%t.bundle3.o -### -dump-temporary-files 2>&1 \
 // RUN: | FileCheck %s --check-prefix CK-OBJ-CMD
-// CK-OBJ-CMD: private constant [1 x i8] zeroinitializer, section "__CLANG_OFFLOAD_BUNDLE__host-[[HOST:.+]]"
+// CK-OBJ-CMD: private constant [{{[0-9]+}} x i8] c"{{.+}}", section "__CLANG_OFFLOAD_BUNDLE__host-[[HOST:.+]]"
 // CK-OBJ-CMD: private constant [{{[0-9]+}} x i8] c"Content of device file 1{{.+}}", section "__CLANG_OFFLOAD_BUNDLE__openmp-powerpc64le-ibm-linux-gnu"
 // CK-OBJ-CMD: private constant [{{[0-9]+}} x i8] c"Content of device file 2{{.+}}", section "__CLANG_OFFLOAD_BUNDLE__openmp-x86_64-pc-linux-gnu"
 // CK-OBJ-CMD: clang{{(.exe)?}}" "-r" "-target" "[[HOST]]" "-o" "{{.+}}.o" "{{.+}}.o" "{{.+}}.bc" "-nostdlib"
 
 // RUN: clang-offload-bundler -type=o -targets=host-%itanium_abi_triple,openmp-powerpc64le-ibm-linux-gnu,openmp-x86_64-pc-linux-gnu -inputs=%t.o,%t.tgt1,%t.tgt2 -outputs=%t.bundle3.o
 // RUN: clang-offload-bundler -type=o -targets=host-%itanium_abi_triple,openmp-powerpc64le-ibm-linux-gnu,openmp-x86_64-pc-linux-gnu -outputs=%t.res.o,%t.res.tgt1,%t.res.tgt2 -inputs=%t.bundle3.o -unbundle
-// RUN: diff %t.bundle3.o %t.res.o
+// RUN: diff %t.o %t.res.o
 // RUN: diff %t.tgt1 %t.res.tgt1
 // RUN: diff %t.tgt2 %t.res.tgt2
 // RUN: clang-offload-bundler -type=o -targets=openmp-powerpc64le-ibm-linux-gnu,host-%itanium_abi_triple,openmp-x86_64-pc-linux-gnu -outputs=%t.res.tgt1,%t.res.o,%t.res.tgt2 -inputs=%t.bundle3.o -unbundle
-// RUN: diff %t.bundle3.o %t.res.o
+// RUN: diff %t.o %t.res.o
 // RUN: diff %t.tgt1 %t.res.tgt1
 // RUN: diff %t.tgt2 %t.res.tgt2
 

diff --git a/tools/clang-offload-bundler/ClangOffloadBundler.cpp b/tools/clang-offload-bundler/ClangOffloadBundler.cpp
index a4ee282c4bd92c76c7d41e0176dfbd59ce1218c2..8f26687aa7c97d8f84e7a659bcd44808a22771b0 100644 (file)
--- a/tools/clang-offload-bundler/ClangOffloadBundler.cpp
+++ b/tools/clang-offload-bundler/ClangOffloadBundler.cpp
@@ -370,13 +370,9 @@ public:
 /// designated name.
 ///
 /// In order to bundle we create an IR file with the content of each section and
-/// use incremental linking to produce the resulting object. We also add section
-/// with a single byte to state the name of the component the main object file
-/// (the one we are bundling into) refers to.
+/// use incremental linking to produce the resulting object.
 ///
-/// To unbundle, we use just copy the contents of the designated section. If the
-/// requested bundle refer to the main object file, we just copy it with no
-/// changes.
+/// To unbundle, we just copy the contents of the designated section.
 class ObjectFileHandler final : public FileHandler {
 
   /// The object file we are currently dealing with.
@@ -471,10 +467,7 @@ public:
       return;
     }
 
-    if (Content->size() < 2)
-      OS.write(Input.getBufferStart(), Input.getBufferSize());
-    else
-      OS.write(Content->data(), Content->size());
+    OS.write(Content->data(), Content->size());
   }
 
   void WriteHeader(raw_fd_ostream &OS,
@@ -592,22 +585,14 @@ public:
     std::string SectionName = OFFLOAD_BUNDLER_MAGIC_STR;
     SectionName += CurrentTriple;
 
-    // Create the constant with the content of the section. For the input we are
-    // bundling into (the host input), this is just a place-holder, so a single
-    // byte is sufficient.
-    assert(HostInputIndex != ~0u && "Host input index undefined??");
-    Constant *Content;
-    if (NumberOfProcessedInputs == HostInputIndex + 1) {
-      uint8_t Byte[] = {0};
-      Content = ConstantDataArray::get(VMContext, Byte);
-    } else
-      Content = ConstantDataArray::get(
-          VMContext, ArrayRef<uint8_t>(reinterpret_cast<const uint8_t *>(
-                                           Input.getBufferStart()),
-                                       Input.getBufferSize()));
-
-    // Create the global in the desired section. We don't want these globals in
-    // the symbol table, so we mark them private.
+    // Create the constant with the content of the section.
+    auto *Content = ConstantDataArray::get(
+        VMContext, ArrayRef<uint8_t>(reinterpret_cast<const uint8_t *>(
+                                         Input.getBufferStart()),
+                                     Input.getBufferSize()));
+
+    // Create the global in the desired section. We don't want these globals
+    // in the symbol table, so we mark them private.
     auto *GV = new GlobalVariable(*M, Content->getType(), /*IsConstant=*/true,
                                   GlobalVariable::PrivateLinkage, Content);
     GV->setSection(SectionName);