Serge Guelton [Tue, 18 Dec 2018 16:04:21 +0000 (16:04 +0000)]
Portable Python script across Python version
In Python3, dict.items, dict.keys, dict.values, zip, map and filter no longer return lists, they create generator instead.
The portability patch consists in forcing an extra `list` call if the result is actually used as a list.
`map` are replaced by list comprehension and `filter` by filtered list comprehension.
Aaron Ballman [Tue, 18 Dec 2018 15:54:38 +0000 (15:54 +0000)]
Emit -Wformat properly for bit-field promotions.
Only explicitly look through integer and floating-point promotion where the result type is actually a promotion, which is not always the case for bit-fields in C.
Haojian Wu [Tue, 18 Dec 2018 15:29:12 +0000 (15:29 +0000)]
[AST] Unify the code paths of traversing lambda expressions.
Summary:
This supposes to be a non-functional change. We have two code paths when
traversing lambda expressions:
1) traverse the function proto typeloc when parameters and return type
are explicit;
2) otherwise fallback to traverse parameter decls and return type loc
individually;
This patch unifies the code path to always traverse parameters and
return type, rather than relying on traversing the full type-loc.
Serge Guelton [Tue, 18 Dec 2018 08:38:50 +0000 (08:38 +0000)]
Portable Python script across Python version
In Python2, division between integer yields an integer, while it yields a float in Python3.
Use a combination of from __future__ import division and // operator to get a portable behavior.
Martin Storsjo [Tue, 18 Dec 2018 08:36:10 +0000 (08:36 +0000)]
[Driver] Automatically enable -munwind-tables if -fseh-exceptions is enabled
For targets where SEH exceptions are used by default (on MinGW,
only x86_64 so far), -munwind-tables are added automatically. If
-fseh-exeptions is enabled on a target where SEH exeptions are
availble but not enabled by default yet (aarch64), we need to
pass -munwind-tables if -fseh-exceptions was specified.
JF Bastien [Tue, 18 Dec 2018 05:12:21 +0000 (05:12 +0000)]
Automatic variable initialization
Summary:
Add an option to initialize automatic variables with either a pattern or with
zeroes. The default is still that automatic variables are uninitialized. Also
add attributes to request uninitialized on a per-variable basis, mainly to disable
initialization of large stack arrays when deemed too expensive.
This isn't meant to change the semantics of C and C++. Rather, it's meant to be
a last-resort when programmers inadvertently have some undefined behavior in
their code. This patch aims to make undefined behavior hurt less, which
security-minded people will be very happy about. Notably, this means that
there's no inadvertent information leak when:
- The compiler re-uses stack slots, and a value is used uninitialized.
- The compiler re-uses a register, and a value is used uninitialized.
- Stack structs / arrays / unions with padding are copied.
This patch only addresses stack and register information leaks. There's many
more infoleaks that we could address, and much more undefined behavior that
could be tamed. Let's keep this patch focused, and I'm happy to address related
issues elsewhere.
To keep the patch simple, only some `undef` is removed for now, see
`replaceUndef`. The padding-related infoleaks are therefore not all gone yet.
This will be addressed in a follow-up, mainly because addressing padding-related
leaks should be a stand-alone option which is implied by variable
initialization.
There are three options when it comes to automatic variable initialization:
0. Uninitialized
This is C and C++'s default. It's not changing. Depending on code
generation, a programmer who runs into undefined behavior by using an
uninialized automatic variable may observe any previous value (including
program secrets), or any value which the compiler saw fit to materialize on
the stack or in a register (this could be to synthesize an immediate, to
refer to code or data locations, to generate cookies, etc).
1. Pattern initialization
This is the recommended initialization approach. Pattern initialization's
goal is to initialize automatic variables with values which will likely
transform logic bugs into crashes down the line, are easily recognizable in
a crash dump, without being values which programmers can rely on for useful
program semantics. At the same time, pattern initialization tries to
generate code which will optimize well. You'll find the following details in
`patternFor`:
- Integers are initialized with repeated 0xAA bytes (infinite scream).
- Vectors of integers are also initialized with infinite scream.
- Pointers are initialized with infinite scream on 64-bit platforms because
it's an unmappable pointer value on architectures I'm aware of. Pointers
are initialize to 0x000000AA (small scream) on 32-bit platforms because
32-bit platforms don't consistently offer unmappable pages. When they do
it's usually the zero page. As people try this out, I expect that we'll
want to allow different platforms to customize this, let's do so later.
- Vectors of pointers are initialized the same way pointers are.
- Floating point values and vectors are initialized with a negative quiet
NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff).
NaNs are nice (here, anways) because they propagate on arithmetic, making
it more likely that entire computations become NaN when a single
uninitialized value sneaks in.
- Arrays are initialized to their homogeneous elements' initialization
value, repeated. Stack-based Variable-Length Arrays (VLAs) are
runtime-initialized to the allocated size (no effort is made for negative
size, but zero-sized VLAs are untouched even if technically undefined).
- Structs are initialized to their heterogeneous element's initialization
values. Zero-size structs are initialized as 0xAA since they're allocated
a single byte.
- Unions are initialized using the initialization for the largest member of
the union.
Expect the values used for pattern initialization to change over time, as we
refine heuristics (both for performance and security). The goal is truly to
avoid injecting semantics into undefined behavior, and we should be
comfortable changing these values when there's a worthwhile point in doing
so.
Why so much infinite scream? Repeated byte patterns tend to be easy to
synthesize on most architectures, and otherwise memset is usually very
efficient. For values which aren't entirely repeated byte patterns, LLVM
will often generate code which does memset + a few stores.
2. Zero initialization
Zero initialize all values. This has the unfortunate side-effect of
providing semantics to otherwise undefined behavior, programs therefore
might start to rely on this behavior, and that's sad. However, some
programmers believe that pattern initialization is too expensive for them,
and data might show that they're right. The only way to make these
programmers wrong is to offer zero-initialization as an option, figure out
where they are right, and optimize the compiler into submission. Until the
compiler provides acceptable performance for all security-minded code, zero
initialization is a useful (if blunt) tool.
I've been asked for a fourth initialization option: user-provided byte value.
This might be useful, and can easily be added later.
Why is an out-of band initialization mecanism desired? We could instead use
-Wuninitialized! Indeed we could, but then we're forcing the programmer to
provide semantics for something which doesn't actually have any (it's
uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`,
or whether it's just there to shut that warning up. It's also way easier to use
a compiler flag than it is to manually and intelligently initialize all values
in a program.
Why not just rely on static analysis? Because it cannot reason about all dynamic
code paths effectively, and it has false positives. It's a great tool, could get
even better, but it's simply incapable of catching all uses of uninitialized
values.
Why not just rely on memory sanitizer? Because it's not universally available,
has a 3x performance cost, and shouldn't be deployed in production. Again, it's
a great tool, it'll find the dynamic uses of uninitialized variables that your
test coverage hits, but it won't find the ones that you encounter in production.
What's the performance like? Not too bad! Previous publications [0] have cited
2.7 to 4.5% averages. We've commmitted a few patches over the last few months to
address specific regressions, both in code size and performance. In all cases,
the optimizations are generally useful, but variable initialization benefits
from them a lot more than regular code does. We've got a handful of other
optimizations in mind, but the code is in good enough shape and has found enough
latent issues that it's a good time to get the change reviewed, checked in, and
have others kick the tires. We'll continue reducing overheads as we try this out
on diverse codebases.
Is it a good idea? Security-minded folks think so, and apparently so does the
Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this
feature would have killed 49 MSRC cases that involved uninitialized struct data
leaking across a trust boundary. It would have also mitigated a number of bugs
involving uninitialized struct data being used directly.". They seem to use pure
zero initialization, and claim to have taken the overheads down to within noise.
Don't just trust Microsoft though, here's another relevant person asking for
this [2]. It's been proposed for GCC [3] and LLVM [4] before.
What are the caveats? A few!
- Variables declared in unreachable code, and used later, aren't initialized.
This goto, Duff's device, other objectionable uses of switch. This should
instead be a hard-error in any serious codebase.
- Volatile stack variables are still weird. That's pre-existing, it's really
the language's fault and this patch keeps it weird. We should deprecate
volatile [5].
- As noted above, padding isn't fully handled yet.
I don't think these caveats make the patch untenable because they can be
addressed separately.
Should this be on by default? Maybe, in some circumstances. It's a conversation
we can have when we've tried it out sufficiently, and we're confident that we've
eliminated enough of the overheads that most codebases would want to opt-in.
Let's keep our precious undefined behavior until that point in time.
How do I use it:
1. On the command-line:
-ftrivial-auto-var-init=uninitialized (the default)
-ftrivial-auto-var-init=pattern
-ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang
2. Using an attribute:
int dont_initialize_me __attribute((uninitialized));
Reid Kleckner [Mon, 17 Dec 2018 23:16:43 +0000 (23:16 +0000)]
Fix ms-layout_version declspec test and add missing new test
Now that MSVC compatibility versions are stored as a four digit number
(1912) instead of a two digit number (19), we need to adjust how we
handle this attribute.
Also add a new test that was intended to be part of r349414.
Reid Kleckner [Mon, 17 Dec 2018 23:10:43 +0000 (23:10 +0000)]
Update Microsoft name mangling scheme for exception specifiers in the type system
Summary:
The msvc exception specifier for noexcept function types has changed
from the prior default of "Z" to "_E" if the function cannot throw when
compiling with /std:C++17.
Alex Lorenz [Mon, 17 Dec 2018 21:01:04 +0000 (21:01 +0000)]
Make test/Driver/darwin-sdk-version.c pass on hosts < macOS10.14
The test test/Driver/darwin-sdk-version.c from r349380 checks if the macOS
deployment target can be correctly inferred from the SDK version. When the
SDK version is > host version, the driver will pick the host version, so
the old test failed on macOS < 10.14. This commit makes this test more
resilient by using an older SDK version.
Alex Lorenz [Mon, 17 Dec 2018 19:30:46 +0000 (19:30 +0000)]
[darwin][arm64] use the "cyclone" CPU for Darwin even when `-arch`
is not specified
The -target option allows the user to specify the build target using LLVM
triple. The triple includes the arch, and so the -arch option is redundant.
This should work just as well without the -arch. However, the driver has a bug
in which it doesn't target the "Cyclone" CPU for darwin if -target is used
without -arch. This commit fixes this issue.
Alex Lorenz [Mon, 17 Dec 2018 19:19:15 +0000 (19:19 +0000)]
[darwin] parse the SDK settings from SDKSettings.json if it exists and
pass in the -target-sdk-version to the compiler and backend
This commit adds support for reading the SDKSettings.json file in the Darwin
driver. This file is used by the driver to determine the SDK's version, and it
uses that information to pass it down to the compiler using the new
-target-sdk-version= option. This option is then used to set the appropriate
SDK Version module metadata introduced in r349119.
Note: I had to adjust the two ast tests as the SDKROOT environment variable
on macOS caused SDK version to be picked up for the compilation of source file
but not the AST.
Gabor Marton [Mon, 17 Dec 2018 13:53:12 +0000 (13:53 +0000)]
[ASTImporter] Add importer specific lookup
Summary:
There are certain cases when normal C/C++ lookup (localUncachedLookup)
does not find AST nodes. E.g.:
Example 1:
template <class T>
struct X {
friend void foo(); // this is never found in the DC of the TU.
};
Example 2:
// The fwd decl to Foo is not found in the lookupPtr of the DC of the
// translation unit decl.
struct A { struct Foo *p; };
In these cases we create a new node instead of returning with the old one.
To fix it we create a new lookup table which holds every node and we are
not interested in any C++ specific visibility considerations.
Simply, we must know if there is an existing Decl in a given DC.
Gabor Marton [Mon, 17 Dec 2018 12:42:12 +0000 (12:42 +0000)]
[ASTImporter] Fix redecl chain of classes and class templates
Summary:
The crux of the issue that is being fixed is that lookup could not find
previous decls of a friend class. The solution involves making the
friend declarations visible in their decl context (i.e. adding them to
the lookup table).
Also, we simplify `VisitRecordDecl` greatly.
This fix involves two other repairs (without these the unittests fail):
(1) We could not handle the addition of injected class types properly
when a redecl chain was involved, now this is fixed.
(2) DeclContext::removeDecl failed if the lookup table in Vector form
did not contain the to be removed element. This caused troubles in
ASTImporter::ImportDeclContext. This is also fixed.
Kristof Umann [Mon, 17 Dec 2018 12:25:48 +0000 (12:25 +0000)]
Revert rC349281 '[analyzer][MallocChecker][NFC] Document and reorganize some functions'
Accidentally commited earlier with the same commit title, but really it
should've been
"Revert rC349283 '[analyzer][MallocChecker] Improve warning messages on double-delete errors'"
Carey Williams [Mon, 17 Dec 2018 10:11:35 +0000 (10:11 +0000)]
[Docs] Expand -fstack-protector and -fstack-protector-all
Improve the description of these command line options
by providing specific heuristic information, as outlined
for the ssp function attribute(s) in LLVM's documentation.
Artem Dergachev [Mon, 17 Dec 2018 06:19:32 +0000 (06:19 +0000)]
[analyzer] MoveChecker: Add an option to suppress warnings on locals.
Re-using a moved-from local variable is most likely a bug because there's
rarely a good motivation for not introducing a separate variable instead.
We plan to keep emitting such warnings by default.
Introduce a flag that allows disabling warnings on local variables that are
not of a known move-unsafe type. If it doesn't work out as we expected,
we'll just flip the flag.
We still warn on move-unsafe objects and unsafe operations on known move-safe
objects.
Artem Dergachev [Sun, 16 Dec 2018 23:44:06 +0000 (23:44 +0000)]
[analyzer] Fix some expressions staying live too long. Add a debug checker.
StaticAnalyzer uses the CFG-based RelaxedLiveVariables analysis in order to,
in particular, figure out values of which expressions are still needed.
When the expression becomes "dead", it is garbage-collected during
the dead binding scan.
Expressions that constitute branches/bodies of control flow statements,
eg. `E1' in `if (C1) E1;' but not `E2' in `if (C2) { E2; }', were kept alive
for too long. This caused false positives in MoveChecker because it relies
on cleaning up loop-local variables when they go out of scope, but some of those
live-for-too-long expressions were keeping a reference to those variables.
Fix liveness analysis to correctly mark these expressions as dead.
Add a debug checker, debug.DumpLiveStmts, in order to test expressions liveness.
Aaron Puchert [Sun, 16 Dec 2018 14:15:30 +0000 (14:15 +0000)]
Thread safety analysis: Allow scoped releasing of capabilities
Summary:
The pattern is problematic with C++ exceptions, and not as widespread as
scoped locks, but it's still used by some, for example Chromium.
We are a bit stricter here at join points, patterns that are allowed for
scoped locks aren't allowed here. That could still be changed in the
future, but I'd argue we should only relax this if people ask for it.
Kristof Umann [Sat, 15 Dec 2018 18:34:00 +0000 (18:34 +0000)]
[analyzer][MallocChecker][NFC] Document and reorganize some functions
This patch merely reorganizes some things, and features no functional change.
In detail:
* Provided documentation, or moved existing documentation in more obvious
places.
* Added dividers. (the //===----------===// thing).
* Moved getAllocationFamily, printAllocDeallocName, printExpectedAllocName and
printExpectedDeallocName in the global namespace on top of the file where
AllocationFamily is declared, as they are very strongly related.
* Moved isReleased and MallocUpdateRefState near RefState's definition for the
same reason.
* Realloc modeling was very poor in terms of variable and structure naming, as
well as documentation, so I renamed some of them and added much needed docs.
* Moved function IdentifierInfos to a separate struct, and moved isMemFunction,
isCMemFunction adn isStandardNewDelete inside it. This makes the patch affect
quite a lot of lines, should I extract it to a separate one?
* Moved MallocBugVisitor out of MallocChecker.
* Preferred switches to long else-if branches in some places.
* Neatly organized some RUN: lines.
Kristof Umann [Sat, 15 Dec 2018 18:11:49 +0000 (18:11 +0000)]
[analyzer][NFC] Merge ClangCheckerRegistry to CheckerRegistry
Now that CheckerRegistry lies in Frontend, we can finally eliminate
ClangCheckerRegistry. Fortunately, this also provides us with a
DiagnosticsEngine, so I went ahead and removed some parameters from it's
methods.
Kristof Umann [Sat, 15 Dec 2018 16:23:51 +0000 (16:23 +0000)]
[analyzer][NFC] Move CheckerRegistry from the Core directory to Frontend
ClangCheckerRegistry is a very non-obvious, poorly documented, weird concept.
It derives from CheckerRegistry, and is placed in lib/StaticAnalyzer/Frontend,
whereas it's base is located in lib/StaticAnalyzer/Core. It was, from what I can
imagine, used to circumvent the problem that the registry functions of the
checkers are located in the clangStaticAnalyzerCheckers library, but that
library depends on clangStaticAnalyzerCore. However, clangStaticAnalyzerFrontend
depends on both of those libraries.
One can make the observation however, that CheckerRegistry has no place in Core,
it isn't used there at all! The only place where it is used is Frontend, which
is where it ultimately belongs.
This move implies that since
include/clang/StaticAnalyzer/Checkers/ClangCheckers.h only contained a single function:
it had to re purposed, as CheckerRegistry is no longer available to
clangStaticAnalyzerCheckers. It was renamed to BuiltinCheckerRegistration.h,
which actually describes it a lot better -- it does not contain the registration
functions for checkers, but only those generated by the tblgen files.
Kristof Umann [Sat, 15 Dec 2018 15:44:05 +0000 (15:44 +0000)]
[analyzer] Prefer returns values to out-params in CheckerRegistry.cpp
Renaming collectCheckers to getEnabledCheckers
Changing the functionality to acquire all enabled checkers, rather then collect
checkers for a specific CheckerOptInfo (for example, collecting all checkers for
{ "core", true }, which meant enabling all checkers from the core package, which
was an unnecessary complication).
Removing CheckerOptInfo, instead of storing whether the option was claimed via a
field, we handle errors immediately, as getEnabledCheckers can now access a
DiagnosticsEngine. Realize that the remaining information it stored is directly
accessible through AnalyzerOptions.CheckerControlList.
Fix a test with -analyzer-disable-checker -verify accidentally left in.
Fangrui Song [Sat, 15 Dec 2018 08:54:06 +0000 (08:54 +0000)]
[libclang] Add dependency on clangSerialization to unbreak -DBUILD_SHARED_LIBS=1 build after rC349237
Frontend headers have undefined reference on the symbol `clang::PCHContainerOperations::PCHContainerOperations()` through some shared_ptr usage. Any dependents will get the undefined reference which can only be resolved by explicit dependency on clangSerialization (due to -z defs).
Martin Storsjo [Sat, 15 Dec 2018 08:08:11 +0000 (08:08 +0000)]
[MinGW] Produce a vtable and RTTI for dllexported classes without a key function
This matches what GCC does in these situations.
This fixes compiling Qt in debug mode. In release mode, references to
the vtable of this particular class ends up optimized away, but in debug
mode, the compiler creates references to the vtable, which is expected
to be dllexported from a different DLL. Make sure the dllexported
version actually ends up emitted.
Artem Dergachev [Sat, 15 Dec 2018 02:13:26 +0000 (02:13 +0000)]
[analyzer] Fix unknown block calls to have zero parameters.
Right now they report to have one parameter with null decl,
because initializing an ArrayRef of pointers with a nullptr
yields an ArrayRef to an array of one null pointer.
Fixes a crash in the OSObject section of RetainCountChecker.
Artem Dergachev [Sat, 15 Dec 2018 02:09:02 +0000 (02:09 +0000)]
[analyzer] ObjCDealloc: Fix a crash when a class attempts to deallocate a class.
The checker wasn't prepared to see the dealloc message sent to the class itself
rather than to an instance, as if it was +dealloc.
Additionally, it wasn't prepared for pure-unknown or undefined self values.
The new guard covers that as well, but it is annoying to test because
both kinds of values shouldn't really appear and we generally want to
get rid of all of them (by modeling unknown values with symbols and
by warning on use of undefined values before they are used).
The CHECK: directive for FileCheck at the end of the test looks useless,
so i removed it.
Artem Dergachev [Sat, 15 Dec 2018 02:06:13 +0000 (02:06 +0000)]
[analyzer] ObjCContainers: Track index values.
Use trackExpressionValue() (previously known as trackNullOrUndefValue())
to track index value in the report, so that the user knew
what Static Analyzer thinks the index is.
Additionally, implement printState() to help debugging the checker later.
Artem Dergachev [Sat, 15 Dec 2018 01:53:38 +0000 (01:53 +0000)]
[analyzer] MoveChecker: Add checks for dereferencing a smart pointer after move.
Calling operator*() or operator->() on a null STL smart pointer is
undefined behavior.
Smart pointers are specified to become null after being moved from.
So we can't warn on arbitrary method calls, but these two operators
definitely make no sense.
The new bug is fatal because it's an immediate UB,
unlike other use-after-move bugs.
The work on a more generic null smart pointer dereference checker
is still pending.
Reid Kleckner [Fri, 14 Dec 2018 23:42:59 +0000 (23:42 +0000)]
Mangle calling conventions into function pointer types where GCC does
Summary:
GCC 5.1 began mangling these Windows calling conventions into function
types, since they can be used for overloading. They've always been
mangled in the MS ABI, but they are new to the Itanium mangler. Note
that the calling convention doesn't appear as part of the main
declaration, it only appears on function parameter types and other
types.
Erich Keane [Fri, 14 Dec 2018 23:17:34 +0000 (23:17 +0000)]
Add AddressSpace mangling to MS mode
All of the symbols demangle on llvm-undname and demangler.com. This
address space qualifier is useful for when we want to use opencl C++ in
Windows mode. Additionally, C++ address-space using functions will now
be usable on windows.
Erich Keane [Fri, 14 Dec 2018 22:22:29 +0000 (22:22 +0000)]
Add extension to always default-initialize nullptr_t.
Core issue 1013 suggests that having an uninitialied std::nullptr_t be
UB is a bit foolish, since there is only a single valid value. This DR
reports that DR616 fixes it, which does so by making lvalue-to-rvalue
conversions from nullptr_t be equal to nullptr.
However, just implementing that results in warnings/etc in many places.
In order to fix all situations where nullptr_t would seem uninitialized,
this patch instead (as an otherwise transparent extension) default
initializes uninitialized VarDecls of nullptr_t.
Eric Fiselier [Fri, 14 Dec 2018 21:11:28 +0000 (21:11 +0000)]
[Clang] Add __builtin_launder
Summary:
This patch adds `__builtin_launder`, which is required to implement `std::launder`. Additionally GCC provides `__builtin_launder`, so thing brings Clang in-line with GCC.
I'm not exactly sure what magic `__builtin_launder` requires, but based on previous discussions this patch applies a `@llvm.invariant.group.barrier`. As noted in previous discussions, this may not be enough to correctly handle vtables.
Alexey Bataev [Fri, 14 Dec 2018 21:00:58 +0000 (21:00 +0000)]
[OPENMP][NVPTX]Improved interwarp copy function.
Inlined runtime with the current implementation of the interwarp copy
function leads to the undefined behavior because of the not quite
correct implementation of the barriers. Start using generic
__kmpc_barier function instead of the custom made barriers.
Artem Dergachev [Fri, 14 Dec 2018 20:52:57 +0000 (20:52 +0000)]
[analyzer] MoveChecker Pt.6: Suppress the warning for the move-safe STL classes.
Some C++ standard library classes provide additional guarantees about their
state after move. Suppress warnings on such classes until a more precise
behavior is implemented. Warnings for locals are not suppressed anyway
because it's still most likely a bug.
If a moved-from object is passed into a conservatively evaluated function
by pointer or by reference, we assume that the function may reset its state.
Make sure it doesn't apply to const pointers and const references. Add a test
that demonstrates that it does apply to rvalue references.
Additionally, make sure that the object is invalidated when its contents change
for reasons other than invalidation caused by evaluating a call conservatively.
In particular, when the object's fields are manipulated directly, we should
assume that some sort of reset may be happening.
Aaron Ballman [Fri, 14 Dec 2018 20:34:23 +0000 (20:34 +0000)]
Update our SARIF support from 10-10 to 11-28.
Functional changes include:
* The run.files property is now an array instead of a mapping.
* fileLocation objects now have a fileIndex property specifying the array index into run.files.
* The resource.rules property is now an array instead of a mapping.
* The result object was given a ruleIndex property that is an index into the resource.rules array.
* rule objects now have their "id" field filled out in addition to the name field.
* Updated the schema and spec version numbers to 11-28.
Implement options in clang to enable recording the driver command-line
in an ELF section.
Implement a new special named metadata, llvm.commandline, to support
frontends embedding their command-line options in IR/ASM/ELF.
This differs from the GCC implementation in some key ways:
* In GCC there is only one command-line possible per compilation-unit,
in LLVM it mirrors llvm.ident and multiple are allowed.
* In GCC individual options are separated by NULL bytes, in LLVM entire
command-lines are separated by NULL bytes. The advantage of the GCC
approach is to clearly delineate options in the face of embedded
spaces. The advantage of the LLVM approach is to support merging
multiple command-lines unambiguously, while handling embedded spaces
with escaping.
Richard Trieu [Fri, 14 Dec 2018 03:35:10 +0000 (03:35 +0000)]
Fix up diagnostics.
Move some diagnostics around between Diagnostic*Kinds.td files. Diagnostics
used in multiple places were moved to DiagnosticCommonKinds.td. Diagnostics
listed in the wrong place (ie, Sema diagnostics listed in
DiagnosticsParseKinds.td) were moved to the correct places. One diagnostic
split into two so that the diagnostic string is in the .td file instead of in
code. Cleaned up the diagnostic includes after all the changes.
Craig Topper [Fri, 14 Dec 2018 00:21:02 +0000 (00:21 +0000)]
[Builltins][X86] Provide implementations of __lzcnt16, __lzcnt, __lzcnt64 for MS compatibility. Remove declarations from intrin.h and implementations from lzcntintrin.h
intrin.h had forward declarations for these and lzcntintrin.h had implementations that were only available with -mlzcnt or a -march that supported the lzcnt feature.
For MS compatibility we should always have these builtins available regardless of X86 being the target or the CPU support the lzcnt instruction. The backends should be able to gracefully fallback to something support even if its just shifts and bit ops.
Unfortunately, gcc also implements 2 of the 3 function names here on X86 when lzcnt feature is enabled.
This patch adds builtins for these for MSVC compatibility and drops the forward declarations from intrin.h. To keep the gcc compatibility the two intrinsics that collided have been turned into macros that use the X86 specific builtins with the lzcnt feature check. These macros are only defined when _MSC_VER is not defined. Without them being macros we can get a redefinition error because -ms-extensions doesn't seem to set _MSC_VER but does make the MS builtins available.
Artem Belevich [Thu, 13 Dec 2018 21:43:04 +0000 (21:43 +0000)]
[CUDA] Make all host-side shadows of device-side variables undef.
The host-side code can't (and should not) access the values that may
only exist on the device side. E.g. address of a __device__ function
does not exist on the host side as we don't generate the code for it there.
Aaron Ballman [Thu, 13 Dec 2018 20:55:34 +0000 (20:55 +0000)]
Update the scan-build to generate SARIF.
This updates the scan-build perl script to allow outputting to sarif in a more natural fashion by specifying -sarif as a command line argument, similar to how -plist is already supported.
Adrian Prantl [Thu, 13 Dec 2018 17:53:29 +0000 (17:53 +0000)]
Reinstate DW_AT_comp_dir support after D55519.
The DIFile used by the CU is special and distinct from the main source
file. Its directory part specifies what becomes the DW_AT_comp_dir
(the compilation directory), even if the source file was specified
with an absolute path.
To support the .dwo workflow, a valid DW_AT_comp_dir is necessary even
if source files were specified with an absolute path.
Found the case in the clang codebase where the assertion fires.
To avoid crashing assertion-enabled builds before I re-add the missing
operation.
Will restore the assertion alongside the upcoming fix.
David Green [Thu, 13 Dec 2018 17:20:06 +0000 (17:20 +0000)]
Fix CodeCompleteTest.cpp for older gcc plus ccache builds
Some versions of gcc, especially when invoked through ccache (-E), can have
trouble with raw string literals inside macros. This moves the string out of
the macro.
Vitaly Buka [Thu, 13 Dec 2018 09:47:39 +0000 (09:47 +0000)]
[asan] Don't check ODR violations for particular types of globals
Summary:
private and internal: should not trigger ODR at all.
unnamed_addr: current ODR checking approach fail and rereport false violation if
a linker merges such globals
linkonce_odr, weak_odr: could cause similar problems and they are already not
instrumented for ELF.
Artem Dergachev [Thu, 13 Dec 2018 01:30:47 +0000 (01:30 +0000)]
[analyzer] RunLoopAutoreleaseLeakChecker: Come up with a test for r348822.
Statement memoization was removed in r348822 because it was noticed to cause
memory corruption. This was happening because a reference to an object
in a DenseMap was used after being invalidated by inserting a new key
into the map.
This test case crashes reliably under ASan (i.e., when Clang is built with
-DLLVM_USE_SANITIZER="Address") on at least some machines before r348822
and doesn't crash after it.
Haibo Huang [Wed, 12 Dec 2018 22:04:12 +0000 (22:04 +0000)]
Declares __cpu_model as dso local
__builtin_cpu_supports and __builtin_cpu_is use information in __cpu_model to decide cpu features. Before this change, __cpu_model was not declared as dso local. The generated code looks up the address in GOT when reading __cpu_model. This makes it impossible to use these functions in ifunc, because at that time GOT entries have not been relocated. This change makes it dso local.
Eric Fiselier [Wed, 12 Dec 2018 21:50:55 +0000 (21:50 +0000)]
[AST] Store "UsesADL" information in CallExpr.
Summary:
Currently the Clang AST doesn't store information about how the callee of a CallExpr was found. Specifically if it was found using ADL.
However, this information is invaluable to tooling. Consider a tool which renames usages of a function. If the originally CallExpr was formed using ADL, then the tooling may need to additionally qualify the replacement.
Without information about how the callee was found, the tooling is left scratching it's head. Additionally, we want to be able to match ADL calls as quickly as possible, which means avoiding computing the answer on the fly.
This patch changes `CallExpr` to store whether it's callee was found using ADL. It does not change the size of any AST nodes.
Erich Keane [Wed, 12 Dec 2018 20:30:53 +0000 (20:30 +0000)]
Teach __builtin_unpredictable to work through implicit casts.
The __builtin_unpredictable implementation is confused by any implicit
casts, which happen in C++. This patch strips those off so that
if/switch statements now work with it in C++.
Erich Keane [Wed, 12 Dec 2018 18:11:36 +0000 (18:11 +0000)]
Change CallGraph print to show the fully qualified name
CallGraph previously would just show the normal name of a function,
which gets really confusing when using it on large C++ projects. This
patch switches the printName call to a printQualifiedName, so that the
namespaces are included.
When multiple loop transformation are defined in a loop's metadata, their order of execution is defined by the order of their respective passes in the pass pipeline. For instance, e.g.
and will try to loop-distribute before Unroll-And-Jam because the LoopDistribute pass is scheduled after UnrollAndJam pass. UnrollAndJamPass only supports one inner loop, i.e. it will necessarily fail after loop distribution. It is not possible to specify another execution order. Also,t the order of passes in the pipeline is subject to change between versions of LLVM, optimization options and which pass manager is used.
This patch adds 'followup' attributes to various loop transformation passes. These attributes define which attributes the resulting loop of a transformation should have. For instance,
defines a loop ID (!0) to be unrolled-and-jammed (!1) and then the attribute !3 to be added to the jammed inner loop, which contains the instruction to distribute the inner loop.
Currently, in both pass managers, pass execution is in a fixed order and UnrollAndJamPass will not execute again after LoopDistribute. We hope to fix this in the future by allowing pass managers to run passes until a fixpoint is reached, use Polly to perform these transformations, or add a loop transformation pass which takes the order issue into account.
For mandatory/forced transformations (e.g. by having been declared by #pragma omp simd), the user must be notified when a transformation could not be performed. It is not possible that the responsible pass emits such a warning because the transformation might be 'hidden' in a followup attribute when it is executed, or it is not present in the pipeline at all. For this reason, this patche introduces a WarnMissedTransformations pass, to warn about orphaned transformations.
Since this changes the user-visible diagnostic message when a transformation is applied, two test cases in the clang repository need to be updated.
To ensure that no other transformation is executed before the intended one, the attribute `llvm.loop.disable_nonforced` can be added which should disable transformation heuristics before the intended transformation is applied. E.g. it would be surprising if a loop is distributed before a #pragma unroll_and_jam is applied.
With more supported code transformations (loop fusion, interchange, stripmining, offloading, etc.), transformations can be used as building blocks for more complex transformations (e.g. stripmining+stripmining+interchange -> tiling).
projects / clang / log