Sjoerd Meijer [Thu, 1 Aug 2019 18:21:44 +0000 (18:21 +0000)]
[LV] Tail-Loop Folding
This allows folding of the scalar epilogue loop (the tail) into the main
vectorised loop body when the loop is annotated with a "vector predicate"
metadata hint. To fold the tail, instructions need to be predicated (masked),
enabling/disabling lanes for the remainder iterations.
[WebAssembly] Assembler/InstPrinter: support call_indirect type index.
A TYPE_INDEX operand (as used by call_indirect) used to be represented
by the InstPrinter as a symbol (e.g. .Ltype_index0@TYPE_INDEX) which
was a bit of a mismatch with the WasmObjectWriter which expects an
unnamed symbol, to receive the signature from and then turn into a
reloc.
There was really no good way to round-trip this information. An earlier
version of this patch tried to attach the signature information using
a .functype, but that ran into trouble when the symbol was re-emitted
without a name. Removing the name was a giant hack also.
The current version changes the assembly syntax to have an inline
signature spec for TYPEINDEX operands that is always unnamed, which
is much more elegant both in syntax and in implementation (as now the
assembler is able to follow the same path as the regular backend)
[Attributor][FIX] Indicate a missing update change
User of AAReturnedValues need to know if HasOverdefinedReturnedCalls
changed from false to true as it will impact the result of the return
value traversal (calls are not ignored anymore).
Simon Atanasyan [Thu, 1 Aug 2019 16:04:29 +0000 (16:04 +0000)]
[mips] Fix lowering load/store instruction in PIC case
If an operand of the `lw/sw` instructions is a symbol, these instructions
incorrectly lowered using not-position-independent chain of commands.
For PIC code we should use `lw/addiu` instructions with the `R_MIPS_GOT16`
and `R_MIPS_LO16` relocations respectively. Instead of that LLVM generates
position dependent code with the `R_MIPS_HI16` and `R_MIPS_LO16`
relocations.
This patch provides a fix for the bug by handling PIC case separately in
the `MipsAsmParser::expandMemInst`. The main idea is to generate a chain
of PIC instructions to load a symbol address into a register and then
load the address content.
The fix is not optimal and does not fix all PIC-related problems. This
is a task for subsequent patches.
Kuba Mracek [Thu, 1 Aug 2019 15:51:14 +0000 (15:51 +0000)]
[llvm-objdump] Fix jumptable detection when disassembling Mach-O binaries
- Add LC_SEGMENT_64 handling in getSectionsAndSymbols to be able to find the base segment address from 64-bit Mach-O binaries.
- Add "data in code" detection into the !symbolTableWorked case, extract it into a separate function.
- Fix uninitialized variable usage on BaseSegmentAddress (initialize to 0).
- Add test.
This adds SimplifyMultipleUseDemandedBitsForTargetNode X86 support and uses it to allow us to peek through vector insertions to avoid dependencies on entire insertion chains.
Sam Elliott [Thu, 1 Aug 2019 12:42:31 +0000 (12:42 +0000)]
[RISCV] Add Custom Parser for Atomic Memory Operands
Summary:
GCC Accepts both (reg) and 0(reg) for atomic instruction memory
operands. These instructions do not allow for an offset in their
encoding, so in the latter case, the 0 is silently dropped.
Due to how we have structured the RISCVAsmParser, the easiest way to add
support for parsing this offset is to add a custom AsmOperand and
parser. This parser drops all the parens, and just keeps the register.
This commit also adds a custom printer for these operands, which matches
the GCC canonical printer, printing both `(a0)` and `0(a0)` as `(a0)`.
David Green [Thu, 1 Aug 2019 11:22:03 +0000 (11:22 +0000)]
[ARM] Fix for MVE VREV64
The VREV64 instruction is apparently unpredictable if Qd == Qm, due to the
cross-beat nature of the instruction. This adds an earlyclobber to Qd, which
seems to be the same way we deal with this on other instructions like the
write-back on loads and stores.
[AArch64] Do not allocate unnecessary emergency slot.
Fix an issue where the compiler still allocates an emergency spill slot even
though it already decided to spill an extra callee-save register to use
as a scratch register.
Zi Xuan Wu [Thu, 1 Aug 2019 05:26:02 +0000 (05:26 +0000)]
recommit:[PowerPC] Eliminate loads/swap feeding swap/store for vector type by using big-endian load/store
In PowerPC, there is instruction to load vector in big endian element order when it's in little endian target.
So we can combine vector load + reverse into big endian load to eliminate the swap instruction.
Also combine vector reverse + store into big endian store.
JF Bastien [Thu, 1 Aug 2019 03:30:45 +0000 (03:30 +0000)]
[NFC] Remove obsolete LLVM_GNUC_PREREQ
The current minimum GCC version is 4.8 (soon to be 5.1), we there don't need to check for older versions. While I'm around Compiler.h, also update some of the doxygen comment.
Matt Arsenault [Thu, 1 Aug 2019 03:25:52 +0000 (03:25 +0000)]
AMDGPU: Start redefining atomic PatFrags
Start migrating to a form that will be compatible with the global isel
emitter. Also should fix some overly lax checks on the memory type,
which allowed mis-selecting some illegal atomics.
Create unique, but identically-named ELF sections for explicitly-sectioned functions and globals when using -function-sections and -data-sections.
This allows functions and globals to to be reordered later in the linking phase
(using the -symbol-ordering-file) even though reordering will be limited to
the scope of the explicit section.
[ScalarizeMaskedMemIntrin] Bitcast the mask to the scalar domain and use scalar bit tests for the branches.
X86 at least is able to use movmsk or kmov to move the mask to the scalar
domain. Then we can just use test instructions to test individual bits.
This is more efficient than extracting each mask element
individually.
I special cased v1i1 to use the previous behavior. This avoids
poor type legalization of bitcast of v1i1 to i1.
I've skipped expandload/compressstore as I think we need to
handle constant masks for those better first.
Many tests end up with duplicate test instructions due to tail
duplication in the branch folding pass. But the same thing
happens when constructing similar code in C. So its not unique
to the scalarization.
Not sure if this lowering code will also be good for other targets,
but we're only testing X86 today.
[X86] Add DAG combine to fold any_extend_vector_inreg+truncstore to an extractelement+store
We have custom code that ignores the normal promoting type legalization on less than 128-bit vector types like v4i8 to emit pavgb, paddusb, psubusb since we don't have the equivalent instruction on a larger element type like v4i32. If this operation appears before a store, we can be left with an any_extend_vector_inreg followed by a truncstore after type legalization. When truncstore isn't legal, this will normally be decomposed into shuffles and a non-truncating store. This will then combine away the any_extend_vector_inreg and shuffle leaving just the store. On avx512, truncstore is legal so we don't decompose it and we had no combines to fix it.
This patch adds a new DAG combine to detect this case and emit either an extract_store for 64-bit stoers or a extractelement+store for 32 and 16 bit stores. This makes the avx512 codegen match the avx2 codegen for these situations. I'm restricting to only when -x86-experimental-vector-widening-legalization is false. When we're widening we're not likely to create this any_extend_inreg+truncstore combination. This means we should be able to remove this code when we flip the default. I would like to flip the default soon, but I need to investigate some performance regressions its causing in our branch that I wasn't seeing on trunk.
Michael Berg [Wed, 31 Jul 2019 21:57:28 +0000 (21:57 +0000)]
Migrate some more fadd and fsub cases away from UnsafeFPMath control to utilize NoSignedZerosFPMath options control
Summary: Honoring no signed zeroes is also available as a user control through clang separately regardless of fastmath or UnsafeFPMath context, DAG guards should reflect this context.
Philip Reames [Wed, 31 Jul 2019 21:15:21 +0000 (21:15 +0000)]
[IndVars, RLEV] Support rewriting exit values in loops without known exits (prep work)
This is a prepatory patch for future work on support exit value rewriting in loops with a mixture of computable and non-computable exit counts. The intention is to be "mostly NFC" - i.e. not enable any interesting new transforms - but in practice, there are some small output changes.
The test differences are caused by cases wherewhere getSCEVAtScope can simplify a single entry phi without needing any knowledge of the loop.
This feature instructs the backend to allow locally defined global variable
addresses to contain a pointer tag in bits 56-63 that will be ignored by
the hardware (i.e. TBI), but may be used by an instrumentation pass such
as HWASAN. It works by adding a MOVK instruction to the regular ADRP/ADD
sequence that sets bits 48-63 to the corresponding bits of the global, with
the linker bounds check disabled on the ADRP instruction to prevent the tag
from causing a link failure.
This implementation of the feature omits the MOVK when loading from or storing
to a global, which is sufficient for TBI. If the same approach is extended
to MTE, assuming that 0 is not configured as a catch-all tag, we will most
likely also need the MOVK in this case in order to avoid a tag mismatch.
SelectionDAG, MI, AArch64: Widen target flags fields/arguments from unsigned char to unsigned.
This makes the field wider than MachineOperand::SubReg_TargetFlags so that
we don't end up silently truncating any higher bits. We should still catch
any bits truncated from the MachineOperand field as a consequence of the
assertion in MachineOperand::setTargetFlags().
Wei Mi [Wed, 31 Jul 2019 19:59:24 +0000 (19:59 +0000)]
[DAGCombine] Limit the number of times for the same store and root nodes
to bail out in store merging dependence check.
We run into a case where dependence check in store merging bail out many times
for the same store and root nodes in a huge basicblock. That increases compile
time by almost 100x. The patch add a map to track how many times the bailing
out happen for the same store and root, and if it is over a limit, stop
considering the store with the same root as a merging candidate.
Summary:
Verify that the incoming defs into phis are the last defs from the
respective incoming blocks.
When moving blocks, insertDef must RenameUses. Adding this verification
makes GVNHoist tests fail that uncovered this issue.
gn build: Fix redundant object files in builtin lib.
compiler-rt's builtin library has generic implementations of many
functions, and then per-arch optimized implementations of some.
In the CMake build, both filter_builtin_sources() and an explicit loop
at the end of the build file (see D37166) filter out the generic
versions if a per-arch file is present.
The GN build wasn't doing this filtering. Just do the filtering manually
and explicitly, instead of being clever.
While here, also remove files from the mingw/arm build that are
redundantly listed after D39938 / r318139 (both from the CMake and the
GN build).
While here, also fix a target_os -> target_cpu typo.
Reverse the canonicalization of fneg relative to fmul/fdiv. That makes it
easier to implement the transforms (and possibly other fneg transforms) in
1 place because we can always start the pattern match from fneg (either the
legacy binop or the new unop).
There's a secondary practical benefit seen in PR21914 and PR42681:
https://bugs.llvm.org/show_bug.cgi?id=21914
https://bugs.llvm.org/show_bug.cgi?id=42681
...hoisting fneg rather than sinking seems to play nicer with LICM in IR
(although this change may expose analysis holes in the other direction).
1. The instcombine test changes show the expected neutral IR diffs from
reversing the order.
2. The reassociation tests show that we were missing an optimization
opportunity to fold away fneg-of-fneg. My reading of IEEE-754 says
that all of these transforms are allowed (regardless of binop/unop
fneg version) because:
"For all other operations [besides copy/abs/negate/copysign], this
standard does not specify the sign bit of a NaN result."
In all of these transforms, we always have some other binop
(fadd/fsub/fmul/fdiv), so we are free to flip the sign bit of a
potential intermediate NaN operand.
(If that interpretation is wrong, then we must already have a bug in
the existing transforms?)
3. The clang tests shouldn't exist as-is, but that's effectively a
revert of rL367149 (the test broke with an extension of the
pre-existing fneg canonicalization in rL367146).
Philip Reames [Wed, 31 Jul 2019 16:24:20 +0000 (16:24 +0000)]
[docs] Reword documentation in terms of SCCs not cycles
Given the example:
header:
br i1 %c, label %next, label %header
next:
br i1 %c2, label %exit, label %header
We end up with a loop containing both header and next. Given that, the describing the loop in terms of cycles is confusing since we have multiple distinct cycles within a single Loop. Standardize on the SCC to clarify.
Roman Lebedev [Wed, 31 Jul 2019 15:20:33 +0000 (15:20 +0000)]
[NFC][InstCombine] Add xor-or-icmp tests with icmp having extra uses
Currently InstCombiner::foldXorOfICmps() bailouts if the
ICMP it wants to invert has extra uses. As it can be seen
in the tests in previous commit, this is super unfortunate,
this is the single pattern that is left non-canonicalized.
We could analyze if we can also invert all the uses if said ICMP
at the same time, thus not bailing out there.
I'm not seeing any nicer alternative.
Peter Smith [Wed, 31 Jul 2019 14:42:57 +0000 (14:42 +0000)]
[AARCH64] Switch relocations R_AARCH64_TLS_TPREL64 and R_AARCH64_DTPMOD64
The ELF for the Arm 64-bit Architecture document originally specified
R_AARCH64_TLS_DTPREL64 = 0x404
R_AARCH64_TLS_DTPMOD64 = 0x405
LLVM correctly followed the document. Unfortunately in binutils these
two codes were reversed:
R_AARCH64_TLS_DTPMOD64 = 0x404
R_AARCH64_TLS_DTPREL64 = 0x405
Given that binaries had shipped this change has become the defacto standard
interpretation of these relocation codes for any toolchain that wanted to
remain compatible with GNU.
To recognize this the latest version of the ABI document has renamed
the relocations to R_AARCH64_TLS_IMPDEF1 and R_AARCH64_TLS_IMPDEF2
permitting a toolchain to choose between the two relocation types, and
recommending that toolchains follow the GNU interpretation for maximum
compatibility.
Given that upstream llvm has never implemented the standard TLS model for
AArch64 so we have no binary legacy, synchronize with GCC so that we don't
create incompatible objects in the future. So far the only visible change
is in llvm-readobj as it can decode these relocations. Tthis change will
mean that llvm-readobj decodes the same way as GNU readelf.
[IPSCCP] Move callsite check to the beginning of the loop.
We have some code marks instructions with struct operands as overdefined,
but if the instruction is a call to a function with tracked arguments,
this breaks the assumption that the lattice values of all call sites
are not overdefined and will be replaced by a constant.
This also re-adds the assertion from D65222, with additionally skipping
non-callsite uses. This patch should address the cases reported in which
the assertion fired.
Simon Pilgrim [Wed, 31 Jul 2019 12:55:39 +0000 (12:55 +0000)]
[X86][AVX] Ensure chained subvector insertions are the same size (PR42833)
Before combining insert_subvector(insert_subvector(vec, sub0, c0), sub1, c1) patterns, ensure that the subvectors are all the same type. On AVX512 targets especially we might have a mixture of 128/256 subvector insertions.
Simon Pilgrim [Wed, 31 Jul 2019 12:27:47 +0000 (12:27 +0000)]
[X86] Regenerate alias-static-alloca test checks to make D65354 diff easier
I've manually added the stack offsets back as these are worth keeping - we really need a way for update_llc_test_checks.py not to mask out useful address math
projects / llvm / log