Yes, i erroneously assumed that the "after" was meant,
but i was wrong:
> I really meant "performed before", for cases like 4u / -2,
> where -2 is implicitly converted to UINT_MAX - 2 before
> the computation. Conversions that are performed after
> a computation aren't part of the computation at all,
> so I think it's much clearer that they're not in scope
> for this sanitizer.
David Greene [Mon, 30 Jul 2018 19:08:20 +0000 (19:08 +0000)]
Make test/Driver/baremetal.cpp work with linkers other than lld
This test fails if clang is configure with, for example, gold as the
default linker. It does not appear that this test really relies on lld
so make the checks accept ld, ld.gold and ld.bfd too.
Roman Lebedev [Mon, 30 Jul 2018 18:58:30 +0000 (18:58 +0000)]
[clang][ubsan] Implicit Conversion Sanitizer - integer truncation - clang part
Summary:
C and C++ are interesting languages. They are statically typed, but weakly.
The implicit conversions are allowed. This is nice, allows to write code
while balancing between getting drowned in everything being convertible,
and nothing being convertible. As usual, this comes with a price:
```
unsigned char store = 0;
bool consume(unsigned int val);
void test(unsigned long val) {
if (consume(val)) {
// the 'val' is `unsigned long`, but `consume()` takes `unsigned int`.
// If their bit widths are different on this platform, the implicit
// truncation happens. And if that `unsigned long` had a value bigger
// than UINT_MAX, then you may or may not have a bug.
// Similarly, integer addition happens on `int`s, so `store` will
// be promoted to an `int`, the sum calculated (0+768=768),
// and the result demoted to `unsigned char`, and stored to `store`.
// In this case, the `store` will still be 0. Again, not always intended.
store = store + 768; // before addition, 'store' was promoted to int.
}
// But yes, sometimes this is intentional.
// You can either make the conversion explicit
(void)consume((unsigned int)val);
// or mask the value so no bits will be *implicitly* lost.
(void)consume((~((unsigned int)0)) & val);
}
```
Yes, there is a `-Wconversion`` diagnostic group, but first, it is kinda
noisy, since it warns on everything (unlike sanitizers, warning on an
actual issues), and second, there are cases where it does **not** warn.
So a Sanitizer is needed. I don't have any motivational numbers, but i know
i had this kind of problem 10-20 times, and it was never easy to track down.
The logic to detect whether an truncation has happened is pretty simple
if you think about it - https://godbolt.org/g/NEzXbb - basically, just
extend (using the new, not original!, signedness) the 'truncated' value
back to it's original width, and equality-compare it with the original value.
The most non-trivial thing here is the logic to detect whether this
`ImplicitCastExpr` AST node is **actually** an implicit conversion, //or//
part of an explicit cast. Because the explicit casts are modeled as an outer
`ExplicitCastExpr` with some `ImplicitCastExpr`'s as **direct** children.
https://godbolt.org/g/eE1GkJ
Nowadays, we can just use the new `part_of_explicit_cast` flag, which is set
on all the implicitly-added `ImplicitCastExpr`'s of an `ExplicitCastExpr`.
So if that flag is **not** set, then it is an actual implicit conversion.
As you may have noted, this isn't just named `-fsanitize=implicit-integer-truncation`.
There are potentially some more implicit conversions to be warned about.
Namely, implicit conversions that result in sign change; implicit conversion
between different floating point types, or between fp and an integer,
when again, that conversion is lossy.
One thing i know isn't handled is bitfields.
This is a clang part.
The compiler-rt part is D48959.
[ARM, AArch64]: Use unadjusted alignment when passing composites as arguments
The "Procedure Call Procedure Call Standard for the ARM® Architecture"
(https://static.docs.arm.com/ihi0042/f/IHI0042F_aapcs.pdf), specifies that
composite types are passed according to their "natural alignment", i.e. the
alignment before alignment adjustment on the entire composite is applied.
The same applies for AArch64 ABI.
Clang, however, used the adjusted alignment.
GCC already implements the ABI correctly. With this patch Clang becomes
compatible with GCC and passes such arguments in accordance with AAPCS.
Reka Kovacs [Mon, 30 Jul 2018 15:43:45 +0000 (15:43 +0000)]
[analyzer] Add support for more invalidating functions in InnerPointerChecker.
According to the standard, pointers referring to the elements of a
`basic_string` may be invalidated if they are used as an argument to
any standard library function taking a reference to non-const
`basic_string` as an argument. This patch makes InnerPointerChecker warn
for these cases.
Adam Balogh [Mon, 30 Jul 2018 08:52:21 +0000 (08:52 +0000)]
[Analyzer] Iterator Checker Hotfix: Defer deletion of container data until its last iterator is cleaned up
The analyzer may consider a container region as dead while it still has live
iterators. We must defer deletion of the data belonging to such containers
until all its iterators are dead as well to be able to compare the iterator
to the begin and the end of the container which is stored in the container
data.
Revert r337456: [CodeGen] Disable aggressive structor optimizations at -O0, take 3
This commit increases the number of sections and overall output size of
.o files by 10% and sometimes a bit more. This alone is challenging for
some users, but it also appears to trigger an as-yet unexplained
behavior in the Gold linker where the memory usage increases
considerably more than 10% (we think).
The increase is also frustrating because in many (if not all) cases we
end up with almost all of the growth coming from the ELF overhead of
-ffunction-sections and such, not from actual extra code being emitted.
Richard Smith and Eric Christopher are both going to investigate this
and try to get to the bottom of what is triggering this and whether the
kinds of increases here are sustainable or what options we might have to
minimize the impact they have. However, this is currently breaking
a pretty large number of our users' builds so reverting it while we sort
out how to make progress here. I've seen a longer and more detailed
update to the commit thread.
[UBSan] Strengthen pointer checks in 'new' expressions
With this change compiler generates alignment checks for wider range
of types. Previously such checks were generated only for the record types
with non-trivial default constructor. So the types like:
did not get checks when allocated by 'new' expression.
This change also optimizes the checks generated for the arrays created
in 'new' expressions. Previously the check was generated for each
invocation of type constructor. Now the check is generated only once
for entire array.
CUDA 8.0 E.3.9.4 says: Within the body of a __device__ or __global__
function, only __shared__ variables or variables without any device
memory qualifiers may be declared with static storage class.
It is unclear how a function-scope non-const static variable
without device memory qualifier is implemented, therefore only static
const variable without device memory qualifier is allowed, which
can be emitted as a global variable in constant address space.
Currently clang only allows function-scope static variable with
__shared__ qualifier.
This patch also allows function-scope static const variable without
device memory qualifier and emits it as a global variable in constant
address space.
Nicolas Lesser [Fri, 27 Jul 2018 21:55:12 +0000 (21:55 +0000)]
Parse a possible trailing postfix expression suffix after a fold expression
Summary:
This patch allows the parsing of a postfix expression involving a fold expression, which is legal as a fold-expression is a primary-expression.
Erik Pilkington [Fri, 27 Jul 2018 21:23:48 +0000 (21:23 +0000)]
[Sema] Use a TreeTransform to extract deduction guide parameter types
Previously, we just canonicalized the type, but this lead to crashes with
parameter types that referred to ParmVarDecls of the constructor. There may be
more cases that this TreeTransform needs to handle though, such as a constructor
parameter type referring to a member in an unevaluated context. Canonicalization
doesn't address these cases either though, so we can address them as-needed in
follow-up commits.
[DEBUGINFO] Disable unsupported debug info options for NVPTX target.
Summary:
Some targets support only default set of the debug options and do not
support additional debug options, like NVPTX target. Patch introduced
virtual function supportsDebugInfoOptions() that can be overloaded
by the toolchain, checks if the target supports some debug
options and emits warning when an unsupported debug option is
found.
George Karpenkov [Fri, 27 Jul 2018 18:26:40 +0000 (18:26 +0000)]
[analyzer] Extend NoStoreFuncVisitor to insert a note on IVars
The note is added in the following situation:
- We are throwing a nullability-related warning on an IVar
- The path goes through a method which *could have* (syntactically
determined) written into that IVar, but did not
George Karpenkov [Fri, 27 Jul 2018 17:40:59 +0000 (17:40 +0000)]
[ASTMatchers] Introduce a matcher for `ObjCIvarExpr`, support getting it's declaration
ObjCIvarExpr is *not* a subclass of MemberExpr, and a separate matcher
is required to support it.
Adding a hasDeclaration support as well, as it's not very useful without
it.
George Karpenkov [Fri, 27 Jul 2018 17:26:11 +0000 (17:26 +0000)]
[ASTMatchers] Introduce a matcher for `ObjCIvarExpr`, support getting it's declaration.
ObjCIvarExpr is *not* a subclass of MemberExpr, and a separate matcher
is required to support it.
Adding a hasDeclaration support as well, as it's not very useful without
it.
Roman Lebedev [Fri, 27 Jul 2018 07:27:14 +0000 (07:27 +0000)]
[AST] Sink 'part of explicit cast' down into ImplicitCastExpr
Summary:
As discussed in IRC with @rsmith, it is slightly not good to keep that in the `CastExpr` itself:
Given the explicit cast, which is represented in AST as an `ExplicitCastExpr` + `ImplicitCastExpr`'s,
only the `ImplicitCastExpr`'s will be marked as `PartOfExplicitCast`, but not the `ExplicitCastExpr` itself.
Thus, it is only ever `true` for `ImplicitCastExpr`'s, so we don't need to write/read/dump it for `ExplicitCastExpr`'s.
We don't need to worry that we write the `PartOfExplicitCast` in PCH after `CastExpr::path_iterator`,
since the `ExprImplicitCastAbbrev` is only used when the `NumBaseSpecs == 0`, i.e. there is no 'path'.
Clang already has L__FUNCTION__ as a workaround for dealing with
pre-processor code that expects to be able to do L##__FUNCTION__ in a
macro. This patch implements the same logic for __FUNCSIG__.
Updated llvm-proto-fuzzer to execute the compiled code
Summary:
Made changes to the llvm-proto-fuzzer
- Added loop vectorizer optimization pass in order to have two IR versions
- Updated old fuzz target to handle two different IR versions
- Wrote code to execute both versions in memory
[ARM64] [Windows] Follow MS X86_64 C++ ABI when passing structs
Summary: Microsoft's C++ object model for ARM64 is the same as that for X86_64.
For example, small structs with non-trivial copy constructors or virtual
function tables are passed indirectly. Currently, they are passed in registers
when compiled with clang.
Simon Marchi [Thu, 26 Jul 2018 18:55:02 +0000 (18:55 +0000)]
[VirtualFileSystem] InMemoryFileSystem::status: Return a Status with the requested name
Summary:
InMemoryFileSystem::status behaves differently than
RealFileSystem::status. The Name contained in the Status returned by
RealFileSystem::status will be the path as requested by the caller,
whereas InMemoryFileSystem::status returns the normalized path.
For example, when requested the status for "../src/first.h",
RealFileSystem returns a Status with "../src/first.h" as the Name.
InMemoryFileSystem returns "/absolute/path/to/src/first.h".
The reason for this change is that I want to make a unit test in the
clangd testsuite (where we use an InMemoryFileSystem) to reproduce a
bug I get with the clangd program (where a RealFileSystem is used).
This difference in behavior "hides" the bug in the unit test version.
Richard Smith [Thu, 26 Jul 2018 18:41:30 +0000 (18:41 +0000)]
Refactor checking of switch conditions and case values.
Check each case value in turn while parsing it, performing the
conversion to the switch type within the context of the expression
itself. This will become necessary in order to properly handle cleanups
for temporaries created as part of the case label (in an upcoming
patch). For now it's just good hygiene.
This necessitates moving the checking for the switch condition itself to
earlier, so that the destination type is available when checking the
case labels.
As a nice side-effect, we get slightly improved diagnostic quality and
error recovery by separating the case expression checking from the case
statement checking and from tracking whether there are discarded case
labels.
[COFF, ARM64] Decide when to mark struct returns as SRet
Summary:
Refer the MS ARM64 ABI Convention for the behavior for struct returns:
https://docs.microsoft.com/en-us/cpp/build/arm64-windows-abi-conventions#return-values
[Sema][ObjC] Do not propagate the nullability specifier on the receiver
to the result type of a message send if the result type cannot have a
nullability specifier.
Previously, clang would print the following message when the code in
nullability.m was compiled:
"incompatible integer to pointer conversion initializing 'int *' with
an expression of type 'int _Nullable'"
This is wrong as 'int' isn't supposed to have any nullability
specifiers.
Ana Pazos [Thu, 26 Jul 2018 17:37:45 +0000 (17:37 +0000)]
[RISCV] Add support for interrupt attribute
Summary:
Clang supports the GNU style ``__attribute__((interrupt))`` attribute on RISCV targets.
Permissible values for this parameter are user, supervisor, and machine.
If there is no parameter, then it defaults to machine.
Reference: https://gcc.gnu.org/onlinedocs/gcc/RISC-V-Function-Attributes.html
Based on initial patch by Zhaoshi Zheng.
[CodeGen][ObjC] Make block copy/dispose helper functions exception-safe.
When an exception is thrown in a block copy helper function, captured
objects that have previously been copied should be destructed or
released. Similarly, captured objects that are yet to be released should
be released when an exception is thrown in a dispose helper function.
Summary:
This patch replaces the current method of getting an `APSInt` from Z3's model by calling generic API method `getBitvector` instead of `Z3_get_numeral_uint64`.
By calling `getBitvector`, there's no need to handle bitvectors with bit width == 128 separately.
And, as a bonus, clang now compiles correctly with Z3 4.7.1.
[OPENMP] ThreadId in serialized parallel regions is 0.
The first argument for the parallel outlined functions, called as
serialized parallel regions, should be a pointer to the global thread id
that always is 0.
[analyzer] Update SMT API documentation and methods
Summary:
Update the documentation of all the classes introduced with the new generic SMT API, most of them were referencing Z3 and how previous operations were being done (like including the context as parameter in a few methods).
Renamed the following methods, so it's clear that the operate on bitvectors:
*`mkSignExt` -> `mkBVSignExt`
*`mkZeroExt` -> `mkBVZeroExt`
*`mkExtract` -> `mkBVExtract`
*`mkConcat` -> `mkBVConcat`
Removed the unecessary methods:
* `getDataExpr`: it was an one line method that called `fromData`
* `mkBitvector(const llvm::APSInt Int)`: it was not being used anywhere
[OPENMP] Exclude service expressions/statements from the list of
the children.
Special internal helper expressions/statements for the OpenMP directives
should not be exposed as children, only the main substatement must be
represented as the child.
[analyzer] Removed API used by the Refutation Manager from SMTConstraintManager and replace by proper calls to SMTSolver
Summary:
Third patch in the refactoring series, to decouple the SMT Solver from the Refutation Manager (1st: D49668, 2nd: D49767).
The refutation API in the `SMTConstraintManager` was a hack to allow us to create an SMT solver and verify the constraints; it was conceptually wrong from the start. Now, we don't actually need to use the `SMTConstraintManager` and can create an SMT object directly, add the constraints and check them.
While updating the Falsification visitor, I inlined the two functions that were used to collect the constraints and add them to the solver.
As a result of this patch, we could move the SMT API elsewhere and as it's not really dependent on the CSA anymore. Maybe we can create a new dir (utils/smt) for Z3 and future solvers?
[analyzer] Try to minimize the number of equivalent bug reports evaluated by the refutation manager
Summary:
This patch changes how the SMT bug refutation runs in an equivalent bug report class.
Now, all other visitor are executed until they find a valid bug or mark all bugs as invalid. When the one valid bug is found (and crosscheck is enabled), the SMT refutation checks the satisfiability of this single bug.
If the bug is still valid after checking with Z3, it is returned and a bug report is created. If the bug is found to be invalid, the next bug report in the equivalent class goes through the same process, until we find a valid bug or all bugs are marked as invalid.
Massive speedups when verifying redis/src/rax.c, from 1500s to 10s.
[analyzer] Moved non solver specific code from Z3ConstraintManager to SMTConstraintManager
Summary:
This patch moves a lot of code from `Z3ConstraintManager` to `SMTConstraintManager`, leaving only the necessary:
* `canReasonAbout` which returns if a Solver can handle a given `SVal` (should be moved to `SMTSolver` in the future).
* `removeDeadBindings`, `assumeExpr` and `print`: methods that need to use `ConstraintZ3Ty`, can probably be moved to `SMTConstraintManager` in the future.
The patch creates a new file, `SMTConstraintManager.cpp` with the moved code. Conceptually, this is move in the right direction and needs further improvements: `SMTConstraintManager` still does a lot of things that are not required by a `ConstraintManager`.
We ought to move the unrelated to `SMTSolver` and remove everything that's not related to a `ConstraintManager`. In particular, we could remove `addRangeConstraints` and `isModelFeasible`, and make the refutation manager create an Z3Solver directly.
Summary:
Although it is a big patch, the changes are simple:
1. There is one `Z3_Context` now, member of the `SMTConstraintManager` class.
2. `Z3Expr`, `Z3Sort`, `Z3Model` and `Z3Solver` are constructed with a reference to the `Z3_Context` in `SMTConstraintManager`.
3. All static functions are now members of `Z3Solver`, e.g, the `SMTConstraintManager` now calls `Solver.fromBoolean(false)` instead of `Z3Expr::fromBoolean(false)`.
Most of the patch only move stuff around except:
1. New method `Z3Sort MkSort(const QualType &Ty, unsigned BitWidth)`, that creates a sort based on the `QualType` and its width. Used to simplify the `fromData` method.
Unfortunate consequence of this patch:
1. `getInterpretation` was moved from `Z3Model` class to `Z3Solver`, because it needs to create a `Z3Sort` before returning the interpretation. This can be fixed by changing both `toAPFloat` and `toAPSInt` by removing the dependency of `Z3Sort` (it's only used to check which Sort was created and to retrieve the type width).
Summary:
This patch creates `SMTContext` which will wrap a specific SMT context, through `SMTSolverContext`.
The templated `SMTSolverContext` class it's a simple wrapper around a SMT specific context (currently only used in the Z3 backend), while `Z3Context` inherits `SMTSolverContext<Z3_context>` and implements solver specific operations like initialization and destruction of the context.
This separation was done because:
1. We might want to keep one single context, shared across different `SMTConstraintManager`s. It can be achieved by constructing a `SMTContext`, through a function like `CreateSMTContext(Z3)`, `CreateSMTContext(BOOLECTOR)`, etc. The rest of the CSA only need to know about `SMTContext`, so maybe it's a good idea moving `SMTSolverContext` to a separate header in the future.
2. Any generic SMT operation will only require one `SMTSolverContext`object, which can access the specific context by calling `getContext()`.
Summary:
Extend the Clang-Format IncludeCategories documentation by adding a link to the supported regular expression standard (POSIX).
And extenting the example with a system header regex.
[[ https://bugs.llvm.org/show_bug.cgi?id=35041 | bug 35041]]
CodeGen: use non-zero memset when possible for automatic variables
Summary:
Right now automatic variables are either initialized with bzero followed by a few stores, or memcpy'd from a synthesized global. We end up encountering a fair amount of code where memcpy of non-zero byte patterns would be better than memcpy from a global because it touches less memory and generates a smaller binary. The optimizer could reason about this, but it's not really worth it when clang already knows.
This code could definitely be more clever but I'm not sure it's worth it. In particular we could track a histogram of bytes seen and figure out (as we do with bzero) if a memset could be followed by a handful of stores. Similarly, we could tune the heuristics for GlobalSize, but using the same as for bzero seems conservatively OK for now.
Petr Hosek [Wed, 25 Jul 2018 03:01:35 +0000 (03:01 +0000)]
[profile] Support profiling runtime on Fuchsia
This ports the profiling runtime on Fuchsia and enables the
instrumentation. Unlike on other platforms, Fuchsia doesn't use
files to dump the instrumentation data since on Fuchsia, filesystem
may not be accessible to the instrumented process. We instead use
the data sink to pass the profiling data to the system the same
sanitizer runtimes do.
Petr Hosek [Wed, 25 Jul 2018 01:44:22 +0000 (01:44 +0000)]
[CMake] Link static libunwind and libc++abi into libc++ in Fuchsia toolchain
When building libc++ for Fuchsia, we want to distribute shared libc++,
libc++abi and libunwind as separate libraries, but for static versions
we would like to link all of them into libc++ so -lc++ flag has the same
effect whether shared or static library is being used.
George Karpenkov [Tue, 24 Jul 2018 23:14:29 +0000 (23:14 +0000)]
[analyzer] Extend NoStoreFuncVisitor to insert a note on IVars
The note is added in the following situation:
- We are throwing a nullability-related warning on an IVar
- The path goes through a method which *could have* (syntactically
determined) written into that IVar, but did not
Sam McCall [Tue, 24 Jul 2018 16:00:55 +0000 (16:00 +0000)]
[VFS] Cleanups to VFS interfaces.
Summary:
- add comments clarifying semantics
- Status::copyWithNewName(Status, Name) --> instance method
- Status::copyWithNewName(fs::file_status, Name) --> constructor (it's not a copy)
- File::getName() -> getRealPath(), reflecting its actual behavior/function
and stop returning status().getName() in the base class (callers can do this
fallback if they want to, it complicates the contracts).
This is mostly NFC, but the behavior of File::getName() affects FileManager's
FileEntry::tryGetRealPathName(), which now fails in more cases:
- non-real file cases
- real-file cases where the underlying vfs::File was opened in a way that
doesn't call realpath().
(In these cases we don't know a distinct real name, so in principle it seems OK)
projects / clang / log