PTX 6.3 extends `wmma` instruction to support s8/u8/s4/u4/b1 -> s32.
All of the new instructions are still handled mostly by tablegen. I've slightly
refactored the code to drive intrinsic/instruction generation from a master
list of supported variants, so all irregularities have to be implemented in one place only.
The test generation script wmma.py has been refactored in a similar way.
[NVPTX] generate correct MMA instruction mnemonics with PTX63+.
PTX 6.3 requires using ".aligned" in the MMA instruction names.
In order to generate correct name, now we pass current
PTX version to each instruction as an extra constant operand
and InstPrinter adjusts its output accordingly.
[NVPTX] Refactor generation of MMA intrinsics and instructions. NFC.
Generalized constructions of 'fragments' of MMA operations to provide
common primitives for construction of the ops. This will make it easier
to add new variants of the instructions that operate on integer types.
Use nested foreach loops which makes it possible to better control
naming of the intrinsics.
This patch does not affect LLVM's output, so there are no test changes.
[CMake][PowerPC] Recognize LLVM_NATIVE_TARGET="ppc64le" as PowerPC
Summary:
This value is derived from the host triple, which on the machine
I'm currently using is `ppc64le-linux-redhat`. This change makes
LLVM compile.
David Blaikie [Thu, 25 Apr 2019 20:09:00 +0000 (20:09 +0000)]
Assigning to a local object in a return statement prevents copy elision. NFC.
I added a diagnostic along the lines of `-Wpessimizing-move` to detect `return x = y` suppressing copy elision, but I don't know if the diagnostic is really worth it. Anyway, here are the places where my diagnostic reported that copy elision would have been possible if not for the assignment.
P1155R1 in the post-San-Diego WG21 (C++ committee) mailing discusses whether WG21 should fix this pitfall by just changing the core language to permit copy elision in cases like these.
(Kona update: The bulk of P1155 is proceeding to CWG review, but specifically *not* the parts that explored the notion of permitting copy-elision in these specific cases.)
[ObjC][ARC] Let ARC optimizer bail out if the number of pointer states
it keeps track of becomes too large
ARC optimizer does a top-down and a bottom-up traversal of the whole
function to pair up retain and release instructions and remove them.
This can be expensive if the number of instructions in the function and
pointer states it tracks are large since it has to look at each pointer
state and determine whether the instruction being visited can
potentially use the pointer.
This patch adds a command line option that sets a limit to the number of
pointers it tracks.
[GlobalISel][AArch64] Add generic legalization rule for extends
This adds a legalization rule for G_ZEXT, G_ANYEXT, and G_SEXT which allows
extends whenever the types will fit in registers (or the source is an s1).
Update tests. Add GISel checks throughout all of arm64-vabs.ll,
where we now select a good portion of the code. Add GISel checks to
arm64-subvector-extend.ll, which has a good number of vector extends in it.
[SelectionDAG][X86] Use stack load/store in PromoteIntRes_BITCAST when the input needs to be be split and the output type is a vector.
We had special case handling here, but it uses a scalar any_extend for the
promotion then bitcasts to the final type. This won't split up the input data
into multiple promoted elements like we need.
This patch falls back to doing the conversion through memory.
Fixes PR41594 which I believe was reflected in the bitcast-vector-bool.ll
changes. The changes to vector-half-conversions.ll are fixing a previously
unknown miscompile from this issue.
Robert Lougher [Thu, 25 Apr 2019 17:00:01 +0000 (17:00 +0000)]
[Evaluator] Walk initial elements when handling load through bitcast
When evaluating a store through a bitcast, the evaluator tries to move the
bitcast from the pointer onto the stored value. If the cast is invalid, it
tries to "introspect" the type to get a valid cast by obtaining a pointer to
the initial element (if the type is nested, this may require walking several
initial elements).
In some situations it is possible to get a bitcast on a load (e.g. with
unions, where the bitcast may not be the same type as the store). However,
equivalent logic to the store to introspect the type is missing. This patch
add this logic.
Note, when developing the patch I was unhappy with adding similar logic
directly to the load case as it could get out of step. Instead, I have
abstracted the "introspection" into a helper function, with the specifics
being handled by a passed-in lambda function.
Sam McCall [Thu, 25 Apr 2019 12:51:42 +0000 (12:51 +0000)]
[Support] Add JSON streaming output API, faster where the heavy value types aren't needed.
Summary:
There's still a little bit of constant factor that could be trimmed (e.g.
more overloads to avoid round-tripping primitives through json::Value).
But this solves the memory scaling problem, and greatly improves the performance
constant factor, and the API should leave room for optimization if needed.
Adapt TimeProfiler to use it, eliminating almost all the performance regression
from r358476.
Performance test on my machine:
perf stat -r 5 ~/llvmbuild-opt/bin/clang++ -w -S -ftime-trace -mllvm -time-trace-granularity=0 spirit.cpp
Handcrafted JSON (HEAD=r358532 with r358476 reverted): 2480ms
json::Value (HEAD): 2757ms (+11%)
After this patch: 2520 ms (+1.6%)
Fangrui Song [Thu, 25 Apr 2019 11:33:30 +0000 (11:33 +0000)]
Parallel: only allow the first TaskGroup to run tasks parallelly
Summary:
Concurrent (e.g. nested) llvm::parallel::for_each() may lead to dead
locks. See PR35788 (fixed by rLLD322041) and PR41508 (fixed by D60757).
When parallel_for_each() is about to return, in ~Latch() called by
~TaskGroup(), a thread (in the default executor) may block in
Latch::sync() waiting for Count to become zero. If all threads in the
default executor are blocked, it is a dead lock.
To fix this, force serial execution if the current TaskGroup is not the
first one. For a nested llvm::parallel::for_each(), this parallelizes
the outermost loop and serializes inner loops.
[Testing] Move clangd::Annotations to llvm testing support
Summary:
Annotations allow writing nice-looking unit test code when one needs
access to locations from the source code, e.g. running code completion
at particular offsets in a file. See comments in Annotations.cpp for
more details on the API.
Also got rid of a duplicate annotations parsing code in clang's code
complete tests.
George Rimar [Thu, 25 Apr 2019 09:59:55 +0000 (09:59 +0000)]
[yaml2obj] - Don't crash on invalid inputs.
yaml2obj might crash on invalid input when unable to parse the YAML.
Recently a crash with a very similar nature was fixed for an empty files.
This patch revisits the fix and does it in yaml::Input instead.
It seems to be more correct way to handle such situation.
With that crash for invalid inputs is also fixed now.
Simon Atanasyan [Thu, 25 Apr 2019 07:47:28 +0000 (07:47 +0000)]
[MIPS] Use custom bitcast lowering to avoid excessive instructions
On Mips32r2 bitcast can be expanded to two sw instructions and an ldc1
when using bitcast i64 to double or an sdc1 and two lw instructions when
using bitcast double to i64. By introducing custom lowering that uses
mtc1/mthc1 we can avoid excessive instructions.
[X86] Remove part of an if condition that should always be true.
The IndexReg will always be non-null at this point. Earlier in the function, if
IndexReg was null we set it to CurDAG->getRegister(0, VT) which made it
non-null.
Summary:
When refactoring vectorization flags, vectorization was disabled by default in the new pass manager.
This patch re-enables is for both managers, and changes the assumptions opt makes, based on the new defaults.
Comments in opt.cpp should clarify the intended use of all flags to enable/disable vectorization.
Do not wrap the contents of printFusionCandidates in the LLVM_DEBUG macro. This
fixes an unused variable warning generated when compiling without asserts but
with -DENABLE_LLVM_DUMP.
llvm-cvtres: Make new dupe resource error a bit friendlier
For well-known type IDs, include the name of the type.
To not duplicate the ID->name map, make llvm-readobj call this new
function as well. It has slightly different output, so this also
requires updating a few tests.
Summary:
We've seen cases of bots failing with:
clang: error: unable to execute command: posix_spawn failed: Interrupted system call
Add a small retry loop to posix_spawn in case this happens. Don't retry too much in case there's some systemic problem going on, but retry a few times.
<rdar://problem/50181448>
Recommitting r358783 and r358786 "[MS] Emit S_HEAPALLOCSITE debug info" with fixes for buildbot error (undefined assembler label).
Summary:
This emits labels around heapallocsite calls and S_HEAPALLOCSITE debug
info in codeview. Currently only changes FastISel, so emitting labels still
needs to be implemented in SelectionDAG.
[DAGCombiner] scale repeated FP divisor by splat factor
If we have a vector FP division with a splatted divisor, use the existing transform
that converts 'x/y' into 'x * (1.0/y)' to allow more conversions. This can then
potentially be converted into a scalar FP division by existing combines (rL358984)
as seen in the tests here.
That can be a potentially big perf difference if scalar fdiv has better timing
(including avoiding possible frequency throttling for vector ops).
Using initial-exec TLS variables is a reasonable performance
optimisation for system libraries. Use the correct PIC mechanism to get
hold of the GOT to avoid text relocations.
Try once more to ensure constant initializaton of ManagedStatics
First, use the old style of linker initialization for MSVC 2019 in
addition to 2017. MSVC 2019 emits a dynamic initializer for
ManagedStatic when compiled in debug mode, and according to zturner,
also sometimes in release mode. I wasn't able to reproduce that, but it
seems best to stick with the old code that works.
When clang is using the MSVC STL, we have to give ManagedStatic a
constexpr constructor that fully zero initializes all fields, otherwise
it emits a dynamic initializer. The MSVC STL implementation of
std::atomic has a non-trivial (but constexpr) default constructor that
zero initializes the atomic value. Because one of the fields has a
non-trivial constructor, ManagedStatic ends up with a non-trivial ctor.
The ctor is not constexpr, so clang ends up emitting a dynamic
initializer, even though it simply does zero initialization. To make it
constexpr, we must initialize all fields of the ManagedStatic.
However, while the constructor that takes a pointer is marked constexpr,
clang says it does not evaluate to a constant because it contains a cast
from a pointer to an integer. I filed this as:
https://developercommunity.visualstudio.com/content/problem/545566/stdatomic-value-constructor-is-not-actually-conste.html
Once we do that, we can add back the
LLVM_REQUIRE_CONSTANT_INITIALIZATION marker, and so far as I'm aware it
compiles successfully on all supported targets.
David Blaikie [Wed, 24 Apr 2019 18:09:44 +0000 (18:09 +0000)]
DebugInfo: Emit only declarations (not whole definitions) of non-unit user defined types into type units
While this doesn't come up in reasonable cases currently (the only user
defined types not in type units are ones without linkage - which makes
for near-ODR violations, because it'd be a type with linkage referencing
a type without linkage - such a type can't be validly defined in more
than one TU, so arguably it shouldn't be in a type unit to begin with -
but it's a convenient way to demonstrate an issue that will become more
revalent with homed modular debug info type definitions - which also
don't need to be in type units but more legitimately so).
Precursor to the Clang change to de-type-unit (by omitting the
'identifier') types homed due to strong linkage vtables. (making that
change without this one would lead to major type duplication in type
units)
[X86] Remove dead nodes left after ReplaceAllUsesWith calls during address matching
ReplaceAllUsesWith doesn't remove the node that was replaced. So its left around in the graph messing up use counts on other nodes.
One thing to note, is that this isn't valid if the node being deleted is the root node of an LEA match that gets rejected. In that case the node needs to stay alive because the isel table walking code would still have a reference to it that its going to try to match next. I don't think that's the case here though because the nodes being deleted here should be "and", "srl", and "zero_extend" none of which can be the root node of an LEA match.
Robert Widmann [Wed, 24 Apr 2019 17:05:08 +0000 (17:05 +0000)]
[LLVM-C] Deprecate the LLVMValueRef-returning metadata creation functions
Summary: There is still some value in using these functions while the remaining LLVMValueRef-based accessors are still around, but LLVMMDNodeInContext in particular has some wonky semantics that make it worth replacing outright.
Simon Pilgrim [Wed, 24 Apr 2019 16:53:17 +0000 (16:53 +0000)]
[InstCombine][X86] Use generic expansion of PACKSS/PACKUS for constant folding. NFCI.
This patch rewrites the existing PACKSS/PACKUS constant folding code to expand as a generic expansion.
This is a first NFCI step toward expanding PACKSS/PACKUS intrinsics which are acting as non-saturating truncations (although technically the expansion could be used in all cases - but we'll probably want to be conservative).
Lang Hames [Wed, 24 Apr 2019 15:15:55 +0000 (15:15 +0000)]
[JITLink] Refer to FDE's CIE (not the most recent CIE) when parsing eh-frame.
Frame Descriptor Entries (FDEs) have a pointer back to a Common Information
Entry (CIE) that describes how the rest FDE should be parsed. JITLink had been
assuming that FDEs always referred to the most recent CIE encountered, but the
spec allows them to point back to any previously encountered CIE. This patch
fixes JITLink to look up the correct CIE for the FDE.
The testcase is a MachO binary with an FDE that refers to a CIE that is not the
one immediately proceeding it (the layout can be viewed wit
'dwarfdump --eh-frame <testcase>'. This test case had to be a binary as llvm-mc
now sorts FDEs (as of r356216) to ensure FDEs *do* point to the most recent CIE.
George Rimar [Wed, 24 Apr 2019 15:03:53 +0000 (15:03 +0000)]
[obj2yamp] - Simplify and cleanup the code in ELFDumper<ELFT>::dumpGroup a bit. NFC.
This makes the variables naming to match LLVM style,
simplifies the code used to extract the group members,
simplifies the loop and reorders the code around a bit.
[x86] make sure horizontal op and broadcast types match to simplify (PR41414)
If the types don't match, we can't just remove the shuffle.
There may be some other opportunity for optimization here,
but this should prevent the crashing seen in:
https://bugs.llvm.org/show_bug.cgi?id=41414
[LLVM-C] Use dyn_cast instead of unwrap in LLVMGetDebugLoc functions
Summary:
The `unwrap<Type>` calls can assert with:
```
Assertion failed: (isa<X>(Val) && "cast<Ty>() argument of incompatible type!"), function cast
```
so replace them with `dyn_cast`.
Let llvm-cvtres (and lld-link) report duplicate resources
If two .res files contain the same resource, cvtres.exe (and hence
link.exe) reject the input with this message:
CVTRES : fatal error CVT1100: duplicate resource. type:STRING, name:101, language:0x0409
LINK : fatal error LNK1123: failure during conversion to COFF: file invalid or corrupt
llvm-cvtres (and lld-link) used to silently pick one of the duplicate
resources instead. This patch makes them report an error as well.
We slightly improve on cvtres by printing the name of two .res files
containing duplicate entries as well.
Summary:
Both the input Value pointer and the returned Value
pointers in GetUnderlyingObjects are now declared as
const.
It turned out that all current (in-tree) uses of
GetUnderlyingObjects were trivial to update, being
satisfied with have those Value pointers declared
as const. Actually, in the past several of the users
had to use const_cast, just because of ValueTracking
not providing a version of GetUnderlyingObjects with
"const" Value pointers. With this patch we get rid
of those const casts.
[Mips][CodeGen] Remove MachineFunction::setSubtarget. Change Mips to just copy the subtarget from the MachineFunction instead of recalculating it.
Summary:
The MachineFunction should have been created with the correct subtarget. As
long as there is no way to change it, MipsTargetMachine can just capture it
directly from the MachineFunction without calling getSubtargetImpl again.
While there, const correct the Subtarget pointer to avoid a const_cast.
I believe the Mips16Subtarget and NoMips16Subtarget members are never used, but
I'll leave there removal for a separate patch.
And the string table in the .remarks/__remarks section containing:
```
inline\0NoDefinition\0printArgsNoRet\0
test-suite/SingleSource/UnitTests/2002-04-17-PrintfChar.c\0printf\0
will not be inlined into \0 because its definition is unavailable\0
```
This is mostly supposed to be used for testing purposes, but it gives us
a 2x reduction in the remark size, and is an incremental change for the
updates to the remarks file format.
Alex Langford [Tue, 23 Apr 2019 21:59:07 +0000 (21:59 +0000)]
[CMake] Use add_dependencies in add_llvm_install_targets
Summary:
The CMake documentation says that the `DEPENDS` field of
add_custom_target is for files and output of custom commands. Adding a
dependency on a target should be done with `add_dependency`.
projects / llvm / log