Add a flag to remap manglings when reading profile data information.

This can be used to preserve profiling information across codebase
changes that have widespread impact on mangled names, but across which
most profiling data should still be usable. For example, when switching
from libstdc++ to libc++, or from the old libstdc++ ABI to the new ABI,
or even from a 32-bit to a 64-bit build.

The user can provide a remapping file specifying parts of mangled names
that should be treated as equivalent (eg, std::__1 should be treated as
equivalent to std::__cxx11), and profile data will be treated as
applying to a particular function if its name is equivalent to the name
of a function in the profile data under the provided equivalences. See
the documentation change for a description of how this is configured.

Remapping is supported for both sample-based profiling and instruction
profiling. We do not support remapping indirect branch target
information, but all other profile data should be remapped
appropriately.

Support is only added for the new pass manager. If someone wants to also
add support for this for the old pass manager, doing so should be
straightforward.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@344199 91177308-0d34-0410-b5e6-96231b3b80d8

diff --git a/docs/ReleaseNotes.rst b/docs/ReleaseNotes.rst
index b3eb9d5e1084b2a5c8064c21d71c27cfb5ee4986..7e798492bb0f6ff4eaa3001367efc53dc481a9c1 100644 (file)
--- a/docs/ReleaseNotes.rst
+++ b/docs/ReleaseNotes.rst
@@ -46,6 +46,11 @@ sections with improvements to Clang's support for those languages.
 Major New Features
 ------------------
 
+- Clang supports use of a profile remapping file, which permits
+  profile data captured for one version of a program to be applied
+  when building another version where symbols have changed (for
+  example, due to renaming a class or namespace).
+  See the :doc:`UsersManual` for details.
 
 Improvements to Clang's diagnostics
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

diff --git a/docs/UsersManual.rst b/docs/UsersManual.rst
index e0dc31f432fa9c10527895fa969b9e4db33ceef2..438ef46a45e020397237816a0d1a6130938a22d8 100644 (file)
--- a/docs/UsersManual.rst
+++ b/docs/UsersManual.rst
@@ -1799,6 +1799,69 @@ In these cases, you can use the flag ``-fno-profile-instr-generate`` (or
 Note that these flags should appear after the corresponding profile
 flags to have an effect.
 
+Profile remapping
+^^^^^^^^^^^^^^^^^
+
+When the program is compiled after a change that affects many symbol names,
+pre-existing profile data may no longer match the program. For example:
+
+ * switching from libstdc++ to libc++ will result in the mangled names of all
+   functions taking standard library types to change
+ * renaming a widely-used type in C++ will result in the mangled names of all
+   functions that have parameters involving that type to change
+ * moving from a 32-bit compilation to a 64-bit compilation may change the
+   underlying type of ``size_t`` and similar types, resulting in changes to
+   manglings
+
+Clang allows use of a profile remapping file to specify that such differences
+in mangled names should be ignored when matching the profile data against the
+program.
+
+.. option:: -fprofile-remapping-file=<file>
+
+  Specifies a file containing profile remapping information, that will be
+  used to match mangled names in the profile data to mangled names in the
+  program.
+
+The profile remapping file is a text file containing lines of the form
+
+.. code-block::
+
+  fragmentkind fragment1 fragment2
+
+where ``fragmentkind`` is one of ``name``, ``type``, or ``encoding``,
+indicating whether the following mangled name fragments are
+<`name <http://itanium-cxx-abi.github.io/cxx-abi/abi.html#mangle.name>`>s,
+<`type <http://itanium-cxx-abi.github.io/cxx-abi/abi.html#mangle.type>`>s, or
+<`encoding <http://itanium-cxx-abi.github.io/cxx-abi/abi.html#mangle.encoding>`>s,
+respectively.
+
+Blank lines and lines starting with ``#`` are ignored.
+
+For example, to specify that ``absl::string_view`` and ``std::string_view``
+should be treated as equivalent when matching profile data, the following
+remapping file could be used:
+
+.. code-block::
+
+  # absl::string_view is considered equivalent to std::string_view
+  type N4absl11string_viewE St17basic_string_viewIcSt11char_traitsIcEE
+
+  # std:: might be std::__1:: in libc++ or std::__cxx11:: in libstdc++
+  name 3std St3__1
+  name 3std St7__cxx11
+
+Matching profile data using a profile remapping file is supported on a
+best-effort basis. For example, information regarding indirect call targets is
+currently not remapped. For best results, you are encouraged to generate new
+profile data matching the updated program.
+
+.. note::
+
+  Profile data remapping is currently only supported for C++ mangled names
+  following the Itanium C++ ABI mangling scheme. This covers all C++ targets
+  supported by Clang other than Windows.
+
 Controlling Debug Information
 -----------------------------
 

diff --git a/include/clang/Driver/Options.td b/include/clang/Driver/Options.td
index e006187c0e34fcafe017b224f62d64ab7d1a4ee4..166f913211c9de2fd927eb03999308e0b79a55df 100644 (file)
--- a/include/clang/Driver/Options.td
+++ b/include/clang/Driver/Options.td
@@ -735,6 +735,11 @@ def fprofile_instr_use : Flag<["-"], "fprofile-instr-use">, Group<f_Group>,
 def fprofile_instr_use_EQ : Joined<["-"], "fprofile-instr-use=">,
     Group<f_Group>, Flags<[CoreOption]>,
     HelpText<"Use instrumentation data for profile-guided optimization">;
+def fprofile_remapping_file_EQ : Joined<["-"], "fprofile-remapping-file=">,
+    Group<f_Group>, Flags<[CC1Option, CoreOption]>, MetaVarName<"<file>">,
+    HelpText<"Use the remappings described in <file> to match the profile data against names in the program">;
+def fprofile_remapping_file : Separate<["-"], "fprofile-remapping-file">,
+    Group<f_Group>, Flags<[CoreOption]>, Alias<fprofile_remapping_file_EQ>;
 def fcoverage_mapping : Flag<["-"], "fcoverage-mapping">,
     Group<f_Group>, Flags<[CC1Option, CoreOption]>,
     HelpText<"Generate coverage mapping to enable code coverage analysis">;

diff --git a/include/clang/Frontend/CodeGenOptions.h b/include/clang/Frontend/CodeGenOptions.h
index d876344a57c97435e493a34deab0ba2e71903c3f..14e6039372afbb971be6747e6de4cceba90d280e 100644 (file)
--- a/include/clang/Frontend/CodeGenOptions.h
+++ b/include/clang/Frontend/CodeGenOptions.h
@@ -200,6 +200,10 @@ public:
   /// Name of the profile file to use as input for -fprofile-instr-use
   std::string ProfileInstrumentUsePath;
 
+  /// Name of the profile remapping file to apply to the profile data supplied
+  /// by -fprofile-sample-use or -fprofile-instr-use.
+  std::string ProfileRemappingFile;
+
   /// Name of the function summary index file to use for ThinLTO function
   /// importing.
   std::string ThinLTOIndexFile;

diff --git a/lib/CodeGen/BackendUtil.cpp b/lib/CodeGen/BackendUtil.cpp
index 45be10f005b8ccf3b78a58e513a062e85c4dab76..fd9a95b71998b0758ffa468be5265af388b3c202 100644 (file)
--- a/lib/CodeGen/BackendUtil.cpp
+++ b/lib/CodeGen/BackendUtil.cpp
@@ -930,18 +930,21 @@ void EmitAssemblyHelper::EmitAssemblyWithNewPassManager(
     PGOOpt = PGOOptions(CodeGenOpts.InstrProfileOutput.empty()
                             ? DefaultProfileGenName
                             : CodeGenOpts.InstrProfileOutput,
-                        "", "", true, CodeGenOpts.DebugInfoForProfiling);
+                        "", "", "", true,
+                        CodeGenOpts.DebugInfoForProfiling);
   else if (CodeGenOpts.hasProfileIRUse())
     // -fprofile-use.
-    PGOOpt = PGOOptions("", CodeGenOpts.ProfileInstrumentUsePath, "", false,
+    PGOOpt = PGOOptions("", CodeGenOpts.ProfileInstrumentUsePath, "",
+                        CodeGenOpts.ProfileRemappingFile, false,
                         CodeGenOpts.DebugInfoForProfiling);
   else if (!CodeGenOpts.SampleProfileFile.empty())
     // -fprofile-sample-use
-    PGOOpt = PGOOptions("", "", CodeGenOpts.SampleProfileFile, false,
+    PGOOpt = PGOOptions("", "", CodeGenOpts.SampleProfileFile,
+                        CodeGenOpts.ProfileRemappingFile, false,
                         CodeGenOpts.DebugInfoForProfiling);
   else if (CodeGenOpts.DebugInfoForProfiling)
     // -fdebug-info-for-profiling
-    PGOOpt = PGOOptions("", "", "", false, true);
+    PGOOpt = PGOOptions("", "", "", "", false, true);
 
   PassBuilder PB(TM.get(), PGOOpt);
 
@@ -1130,6 +1133,7 @@ static void runThinLTOBackend(ModuleSummaryIndex *CombinedIndex, Module *M,
                               const LangOptions &LOpts,
                               std::unique_ptr<raw_pwrite_stream> OS,
                               std::string SampleProfile,
+                              std::string ProfileRemapping,
                               BackendAction Action) {
   StringMap<DenseMap<GlobalValue::GUID, GlobalValueSummary *>>
       ModuleToDefinedGVSummaries;
@@ -1202,6 +1206,7 @@ static void runThinLTOBackend(ModuleSummaryIndex *CombinedIndex, Module *M,
   Conf.CGOptLevel = getCGOptLevel(CGOpts);
   initTargetOptions(Conf.Options, CGOpts, TOpts, LOpts, HeaderOpts);
   Conf.SampleProfile = std::move(SampleProfile);
+  Conf.ProfileRemapping = std::move(ProfileRemapping);
   Conf.UseNewPM = CGOpts.ExperimentalNewPassManager;
   Conf.DebugPassManager = CGOpts.DebugPassManager;
   Conf.RemarksWithHotness = CGOpts.DiagnosticsWithHotness;
@@ -1268,7 +1273,7 @@ void clang::EmitBackendOutput(DiagnosticsEngine &Diags,
       if (!CombinedIndex->skipModuleByDistributedBackend()) {
         runThinLTOBackend(CombinedIndex.get(), M, HeaderOpts, CGOpts, TOpts,
                           LOpts, std::move(OS), CGOpts.SampleProfileFile,
-                          Action);
+                          CGOpts.ProfileRemappingFile, Action);
         return;
       }
       // Distributed indexing detected that nothing from the module is needed

diff --git a/lib/CodeGen/CodeGenModule.cpp b/lib/CodeGen/CodeGenModule.cpp
index 12ed65cf2675198ac7c583d4fd53de77fd1bdc0d..eebf7c23f3b56964d46424475657dabdaea6bffa 100644 (file)
--- a/lib/CodeGen/CodeGenModule.cpp
+++ b/lib/CodeGen/CodeGenModule.cpp
@@ -154,7 +154,7 @@ CodeGenModule::CodeGenModule(ASTContext &C, const HeaderSearchOptions &HSO,
 
   if (CodeGenOpts.hasProfileClangUse()) {
     auto ReaderOrErr = llvm::IndexedInstrProfReader::create(
-        CodeGenOpts.ProfileInstrumentUsePath);
+        CodeGenOpts.ProfileInstrumentUsePath, CodeGenOpts.ProfileRemappingFile);
     if (auto E = ReaderOrErr.takeError()) {
       unsigned DiagID = Diags.getCustomDiagID(DiagnosticsEngine::Error,
                                               "Could not read profile %0: %1");

diff --git a/lib/Driver/ToolChains/Clang.cpp b/lib/Driver/ToolChains/Clang.cpp
index 070abd82508e2793e61af0a431e8ed7f6d138117..127f80696220d22bec89d1aac783a7e22f84d818 100644 (file)
--- a/lib/Driver/ToolChains/Clang.cpp
+++ b/lib/Driver/ToolChains/Clang.cpp
@@ -4355,6 +4355,7 @@ void Clang::ConstructJob(Compilation &C, const JobAction &JA,
     else
       A->render(Args, CmdArgs);
   }
+  Args.AddLastArg(CmdArgs, options::OPT_fprofile_remapping_file_EQ);
 
   RenderBuiltinOptions(TC, RawTriple, Args, CmdArgs);
 

diff --git a/lib/Frontend/CompilerInvocation.cpp b/lib/Frontend/CompilerInvocation.cpp
index e848e50ced45c6a0cf76a43afaeda88218fc5854..5dcd23e1ad417161ed34dd92d3345bd3b2b03295 100644 (file)
--- a/lib/Frontend/CompilerInvocation.cpp
+++ b/lib/Frontend/CompilerInvocation.cpp
@@ -657,6 +657,13 @@ static bool ParseCodeGenArgs(CodeGenOptions &Opts, ArgList &Args, InputKind IK,
       Args.getLastArgValue(OPT_fprofile_instrument_use_path_EQ);
   if (!Opts.ProfileInstrumentUsePath.empty())
     setPGOUseInstrumentor(Opts, Opts.ProfileInstrumentUsePath);
+  Opts.ProfileRemappingFile =
+      Args.getLastArgValue(OPT_fprofile_remapping_file_EQ);
+  if (!Opts.ProfileRemappingFile.empty() && !Opts.ExperimentalNewPassManager) {
+    Diags.Report(diag::err_drv_argument_only_allowed_with)
+      << Args.getLastArg(OPT_fprofile_remapping_file_EQ)->getAsString(Args)
+      << "-fexperimental-new-pass-manager";
+  }
 
   Opts.CoverageMapping =
       Args.hasFlag(OPT_fcoverage_mapping, OPT_fno_coverage_mapping, false);

diff --git a/test/CodeGenCXX/Inputs/profile-remap.map b/test/CodeGenCXX/Inputs/profile-remap.map
new file mode 100644 (file)
index 0000000..50b812a
--- /dev/null
+++ b/test/CodeGenCXX/Inputs/profile-remap.map
@@ -0,0 +1,2 @@
+name 3Foo 3Bar
+type N3Foo1XE N3Baz1YE

diff --git a/test/CodeGenCXX/Inputs/profile-remap.proftext b/test/CodeGenCXX/Inputs/profile-remap.proftext
new file mode 100644 (file)
index 0000000..a1f90cf
--- /dev/null
+++ b/test/CodeGenCXX/Inputs/profile-remap.proftext
@@ -0,0 +1,7 @@
+:ir
+_ZN3Foo8functionENS_1XE
+29667547796
+2
+10
+90
+

diff --git a/test/CodeGenCXX/Inputs/profile-remap.samples b/test/CodeGenCXX/Inputs/profile-remap.samples
new file mode 100644 (file)
index 0000000..45e5994
--- /dev/null
+++ b/test/CodeGenCXX/Inputs/profile-remap.samples
@@ -0,0 +1,3 @@
+_ZN3Bar8functionEN3Baz1YE:100:0
+ 2: 10
+ 4: 90

diff --git a/test/CodeGenCXX/profile-remap.cpp b/test/CodeGenCXX/profile-remap.cpp
new file mode 100644 (file)
index 0000000..e122db1
--- /dev/null
+++ b/test/CodeGenCXX/profile-remap.cpp
@@ -0,0 +1,27 @@
+// RUN: %clang_cc1 -triple x86_64-linux-gnu -fprofile-sample-use=%S/Inputs/profile-remap.samples -fprofile-remapping-file=%S/Inputs/profile-remap.map -fexperimental-new-pass-manager -O2 %s -emit-llvm -o - | FileCheck %s --check-prefixes=CHECK,CHECK-SAMPLES
+// RUN: llvm-profdata merge -output %T.profdata %S/Inputs/profile-remap.proftext
+// RUN: %clang_cc1 -triple x86_64-linux-gnu -fprofile-instrument-use-path=%T.profdata -fprofile-remapping-file=%S/Inputs/profile-remap.map -fexperimental-new-pass-manager -O2 %s -emit-llvm -o - | FileCheck %s --check-prefixes=CHECK,CHECK-INSTR
+
+namespace Foo {
+  struct X {};
+  bool cond();
+  void bar();
+  void baz();
+  void function(X x) {
+    if (cond())
+      bar();
+    else
+      baz();
+  }
+}
+
+// CHECK: define {{.*}} @_ZN3Foo8functionENS_1XE() {{.*}} !prof [[FUNC_ENTRY:![0-9]*]]
+// CHECK: br i1 {{.*}} !prof [[BR_WEIGHTS:![0-9]*]]
+//
+// FIXME: Laplace's rule of succession is applied to sample profiles...
+// CHECK-SAMPLES-DAG: [[FUNC_ENTRY]] = !{!"function_entry_count", i64 1}
+// CHECK-SAMPLES-DAG: [[BR_WEIGHTS]] = !{!"branch_weights", i32 11, i32 91}
+//
+// ... but not to instruction profiles.
+// CHECK-INSTR-DAG: [[FUNC_ENTRY]] = !{!"function_entry_count", i64 100}
+// CHECK-INSTR-DAG: [[BR_WEIGHTS]] = !{!"branch_weights", i32 10, i32 90}

diff --git a/test/Driver/clang_f_opts.c b/test/Driver/clang_f_opts.c
index f1836980c1c33f909491d43df1fca9a6ff4e4bc8..6536605c4c58ee043ed42280de512997316d7ea9 100644 (file)
--- a/test/Driver/clang_f_opts.c
+++ b/test/Driver/clang_f_opts.c
@@ -116,6 +116,7 @@
 // RUN: %clang -### -S -fcoverage-mapping %s 2>&1 | FileCheck -check-prefix=CHECK-COVERAGE-AND-GEN %s
 // RUN: %clang -### -S -fcoverage-mapping -fno-coverage-mapping %s 2>&1 | FileCheck -check-prefix=CHECK-DISABLE-COVERAGE %s
 // RUN: %clang -### -S -fprofile-instr-generate -fcoverage-mapping -fno-coverage-mapping %s 2>&1 | FileCheck -check-prefix=CHECK-DISABLE-COVERAGE %s
+// RUN: %clang -### -S -fprofile-remapping-file foo/bar.txt %s 2>&1 | FileCheck -check-prefix=CHECK-PROFILE-REMAP %s
 // CHECK-PROFILE-GENERATE: "-fprofile-instrument=clang"
 // CHECK-PROFILE-GENERATE-LLVM: "-fprofile-instrument=llvm"
 // CHECK-PROFILE-GENERATE-DIR: "-fprofile-instrument-path=/some/dir{{/|\\\\}}{{.*}}"
@@ -126,6 +127,7 @@
 // CHECK-DISABLE-USE-NOT: "-fprofile-instr-use"
 // CHECK-COVERAGE-AND-GEN: '-fcoverage-mapping' only allowed with '-fprofile-instr-generate'
 // CHECK-DISABLE-COVERAGE-NOT: "-fcoverage-mapping"
+// CHECK-PROFILE-REMAP: "-fprofile-remapping-file=foo/bar.txt"
 
 // RUN: %clang -### -S -fprofile-use %s 2>&1 | FileCheck -check-prefix=CHECK-PROFILE-USE %s
 // RUN: %clang -### -S -fprofile-instr-use %s 2>&1 | FileCheck -check-prefix=CHECK-PROFILE-USE %s