We have code to split vector splats (of zero and non-zero) for performance
reasons, but it ignores the fact that a store might be truncating.
Actually, truncating stores are formed for vNi8 and vNi16 types. Since the
truncation is from a legal type, the size of the store is always <= 64-bits and
so they don't actually benefit from being split up anyway, so this patch just
disables that transformation.
Sam Parker [Tue, 8 Jan 2019 10:12:36 +0000 (10:12 +0000)]
[ARM] Add missing patterns for DSP muls
Using a PatLeaf for sext_16_node allowed matching smulbb and smlabb
instructions once the operands had been sign extended. But we also
need to use sext_inreg operands along with sext_16_node to catch a
few more cases that enable use to remove the unnecessary sxth.
Matt Arsenault [Tue, 8 Jan 2019 06:30:53 +0000 (06:30 +0000)]
AMDGPU/GlobalISel: Introduce vcc reg bank
I'm not entirely sure this is the correct thing
to do with the global isel philosophy, but I think
this is necessary to handle how differently SGPRs
are used normally vs. from a condition.
For example, it makes sense to allow a copy
from a VGPR to an SGPR, but it makes no sense
to allow a copy from VGPRs to SGPRs used as
select mask.
This avoids regbankselecting strange code with
a truncate feeding directly into a condition field.
Now a copy is forced from sgpr(s1) to vcc, which is
more sensible to handle.
Some of these issues could probably avoided with making enough
operations resulting in i1 illegal. I think we can't avoid
this register bank for legality.
For example, an i1 and where one source is from a truncate, and
one source is a compare needs some kind of copy inserted to
make sure both are in condition registers.
Thomas Lively [Tue, 8 Jan 2019 06:25:55 +0000 (06:25 +0000)]
[WebAssembly] Massive instruction renaming
Summary:
An automated renaming of all the instructions listed at
https://github.com/WebAssembly/spec/issues/884#issuecomment-426433329
as well as some similarly-named identifiers.
Robert Widmann [Tue, 8 Jan 2019 06:24:19 +0000 (06:24 +0000)]
[LLVM-C] Allow For Creating a BasicBlock without a Parent Function
Summary: Add a utility function for creating a basic block without a parent function. A useful operation for compilers that need to synthesize and conditionally insert code without having to bother with appending and immediately unlinking a block.
Robert Widmann [Tue, 8 Jan 2019 06:23:22 +0000 (06:23 +0000)]
[LLVM-C] Allow Specifying Signedness in Int Cast
Summary: Fix an old outstanding problem with the int cast builder binding always assuming the cast is signed by introducing a new LLVMBuildIntCast2 operation and deprecating the old prototype.
Heejin Ahn [Tue, 8 Jan 2019 01:25:12 +0000 (01:25 +0000)]
[WebAssembly] Move CFG-changing passes before RegStackify
Summary:
FixIrreducibleControlFlow and LateEHPrepare both possibly modify CFG and
create new registers. There seems to be no reason these passes go after
register-related optimization passes (PrepareForLiveIntervals,
OptimizeLiveIntervals, StoreResults, RegStackify, and RegColoring), and
this also possibly create new optimization opportunities. I think we
should put all current and future optimization passes before RegStackify
(and related passes) unless there's a reason not to.
Matt Arsenault [Tue, 8 Jan 2019 01:22:47 +0000 (01:22 +0000)]
RegBankSelect: Fix copy insertion point for terminators
If a copy was needed to handle the condition of brcond, it was being
inserted before the defining instruction. Add tests for iterator edge
cases.
I find the existing code here suspect for the case where it's looking
for terminators that modify the register. It's going to insert a copy
in the middle of the terminators, which isn't allowed (it might be
necessary to have a COPY_terminator if anybody actually needs this).
[dsymutil] Fix assertion triggered by empty address range.
An assertion was hit when running dsymutil on a gcc generated binary
that contained an empty address range. Address ranges are stored in an
interval map of half open intervals. Since the interval is empty and
therefore meaningless, we simply don't add it to the map.
Wei Mi [Tue, 8 Jan 2019 00:26:11 +0000 (00:26 +0000)]
[RegisterCoalescer] dst register's live interval needs to be updated when
merging a src register in ToBeUpdated set.
This is to fix PR40061 related with https://reviews.llvm.org/rL339035.
In https://reviews.llvm.org/rL339035, live interval of source pseudo register
in rematerialized copy may be saved in ToBeUpdated set and its update may be
postponed.
In PR40061, %t2 = %t1 is rematerialized and %t1 is added into toBeUpdated set
to postpone its live interval update. After the rematerialization, the live
interval of %t1 is larger than necessary. Then %t1 is merged into %t3 and %t1
gets removed. After the merge, %t3 contains live interval larger than necessary.
Because %t3 is not in toBeUpdated set, its live interval is not updated after
register coalescing and it will break some assumption in regalloc.
The patch requires the live interval of destination register in a merge to be
updated if the source register is in ToBeUpdated.
The unobufscation support for BCSymbolMaps was the last piece of code
that hasn't been upstreamed yet. This patch contains a reworked version
of the existing code and relevant tests.
Rong Xu [Mon, 7 Jan 2019 23:25:56 +0000 (23:25 +0000)]
[PGO] Use SourceFileName rather module name in PGOFuncName
In LTO or Thin-lto mode (though linker plugin), the module
names are of temp file names which are different for
different compilations. Using SourceFileName avoids the issue.
This should not change any functionality for current PGO as
all the current callers of getPGOFuncName() is before LTO.
Craig Topper [Mon, 7 Jan 2019 20:13:45 +0000 (20:13 +0000)]
[X86][AutoUpgrade] Make some tweaks to reduce the number of nested if/else in the intrinsic upgrade code to avoid an MSVC compiler limit.
MSVC has a nesting limit of around 110-130. An if/else if/else if counts against this next level. The autoupgrade code consists a long chain of these checking matches against strings.
This commit moves some code to a helper function to move out a large if/else chain that was inside of one of the blocks into a separate function. There are more of these we could move or we could change some to lookup tables.
I've also merged together a few similar blocks in the outer chain. This should buy us some margin for a little bit.
Craig Topper [Mon, 7 Jan 2019 19:30:43 +0000 (19:30 +0000)]
[TargetLowering][AMDGPU] Remove the SimplifyDemandedBits function that takes a User and OpIdx. Stop using it in AMDGPU target for simplifyI24.
As we saw in D56057 when we tried to use this function on X86, it's unsafe. It allows the operand node to have multiple users, but doesn't prevent recursing past the first node when it does have multiple users. This can cause other simplifications earlier in the graph without regard to what bits are needed by the other users of the first node. Ideally all we should do to the first node if it has multiple uses is bypass it when its not needed by the user we started from. Doing any other transformation that SimplifyDemandedBits can do like turning ZEXT/SEXT into AEXT would result in an increase in instructions.
Fortunately, we already have a function that can do just that, GetDemandedBits. It will only make transformations that involve bypassing a node.
This patch changes AMDGPU's simplifyI24, to use a combination of GetDemandedBits to handle the multiple use simplifications. And then uses the regular SimplifyDemandedBits on each operand to handle simplifications allowed when the operand only has a single use. Unfortunately, GetDemandedBits simplifies constants more aggressively than SimplifyDemandedBits. This caused the -7 constant in the changed test to be simplified to remove the upper bits. I had to modify computeKnownBits to account for this by ignoring the upper 8 bits of the input.
Alina Sbirlea [Mon, 7 Jan 2019 18:40:27 +0000 (18:40 +0000)]
[MemorySSA] Extend the clobber walker with the option to skip the starting access.
Summary:
The option enables loop transformations to hoist accesses that do not
have clobbers in the loop. If the clobber queries skips the starting
access, the result may be outside the loop instead of the header Phi.
Adding the walker that uses this option in a separate patch.
Nikita Popov [Mon, 7 Jan 2019 18:03:36 +0000 (18:03 +0000)]
[DemandedBits] Use SetVector for Worklist
DemandedBits currently uses a simple vector for the worklist, which
means that instructions may be inserted multiple times into it.
Especially in combination with the deep lattice, this may cause
instructions too be recomputed very often. To avoid this, switch
to a SetVector.
[elfabi] Add option to manually specify file read format
Although llvm-elfabi will attempt to read input files without needing the format to be manually specified, doing so has the potential to introduce extraneous errors that can hinder debugging (since multiple readers may fail in attempts to read the file). This change allows the input file format to be manually specified to force elfabi to use a single reader. This makes it easier to test and debug errors specific to a given reader.
Summary:
The -O flag is currently being mostly ignored; it's only checked whether or not the output format is "binary". This adds support for a few formats (e.g. elf64-x86-64), so that when specified, the output can change between 32/64 bit and sizes/alignments are updated accordingly.
David Greene [Mon, 7 Jan 2019 16:24:37 +0000 (16:24 +0000)]
[lit] Respect PYTHONPATH
If a user has PYTHONPATH set in the environment, append new entries to
it rather than blindly setting PYTHONPATH to a fixed string. This
allows tests to, for example, find psutil if it is in
PYTHONPATH. Without this change, lit will detect psutil but then
various tests will fail because PYTHONPATH has been overwritten and
psutil cannot be found.
Rhys Perry [Mon, 7 Jan 2019 15:52:28 +0000 (15:52 +0000)]
AMDGPU: test for uniformity of branch instruction, not its condition
Summary:
If a divergent branch instruction is marked as divergent by propagation
rule 2 in DivergencePropagator::exploreSyncDependency() and its condition
is uniform, that branch would incorrectly be assumed to be uniform.
[CallSite removal] Migrate all Alias Analysis APIs to use the newly
minted `CallBase` class instead of the `CallSite` wrapper.
This moves the largest interwoven collection of APIs that traffic in
`CallSite`s. While a handful of these could have been migrated with
a minorly more shallow migration by converting from a `CallSite` to
a `CallBase`, it hardly seemed worth it. Most of the APIs needed to
migrate together because of the complex interplay of AA APIs and the
fact that converting from a `CallBase` to a `CallSite` isn't free in its
current implementation.
Out of tree users of these APIs can fairly reliably migrate with some
combination of `.getInstruction()` on the `CallSite` instance and
casting the resulting pointer. The most generic form will look like `CS`
-> `cast_or_null<CallBase>(CS.getInstruction())` but in most cases there
is a more elegant migration. Hopefully, this migrates enough APIs for
users to fully move from `CallSite` to the base class. All of the
in-tree users were easily migrated in that fashion.
[CallSite removal] Add `CallBase` support to the `InstVisitor` in such
a way that it still supports `CallSite` but users can be ported to rely
on `CallBase` instead.
This will unblock the ports across the analysis and transforms libraries
(and out-of-tree users) and once done we can clean this up by removing
the `CallSite` layer.
Nico Weber [Mon, 7 Jan 2019 01:26:12 +0000 (01:26 +0000)]
[gn build] Add build files for llvm/lib/Target/ARM + tests
The ARM target itself is similar to the X86 target in https://reviews.llvm.org/rL348903
The llvm-exegesis unittests ARM bits are similar to the X86 bits in https://reviews.llvm.org/rL350413
Both are similar to the corresponding AArch64 bits in https://reviews.llvm.org/rL350499 too
After this, everything in my local GN branch is upstreamed to LLVM.
Nico Weber [Mon, 7 Jan 2019 01:23:11 +0000 (01:23 +0000)]
[gn build] Add build files for llvm/lib/Target/AArch64 + tests
The AArch64 target itself is similar to the X86 target in https://reviews.llvm.org/rL348903
The llvm-exegesis AArch64 bits are similar to the X86 bits in http://reviews.llvm.org/rL350184
The llvm-exegesis unittests AArch64 bits are similar to the X86 bits in https://reviews.llvm.org/rL350413
llvm/unittests/Target/AArch64 doesn't have an equivalent since the X86 Target
only has lit tests, no unittests.
Sanjay Patel [Sun, 6 Jan 2019 16:21:42 +0000 (16:21 +0000)]
[x86] explicitly set cost of integer add/sub
There are no test changes here in the existing cost model
regression tests because integer add/sub have a default
legal cost of 1 already. This would break, however, if
we custom lower those ops because the default cost model
assumes that custom-lowered ops are more expensive.
This is similar to the change in rL350403. See discussion
in D56011 for more details. When we enhance that patch to
handle integer ops, we need this cost model change to avoid
unintended diffs here from the custom lowering.
Lama Saba [Sun, 6 Jan 2019 15:45:40 +0000 (15:45 +0000)]
Resubmit rL345008 "Split MachinePipeliner code into header and cpp files"
Resubmitted in rL345290 and reverted in rL350345 due to failures in
http://green.lab.llvm.org/green/job/lldb-cmake/
Resubmitting after a workaround to lldb-cmake failure was
committed in rL350346, more info in https://reviews.llvm.org/D56084
Craig Topper [Sat, 5 Jan 2019 23:30:28 +0000 (23:30 +0000)]
[X86][AsmParser] Don't allow X86::DX in CheckBaseRegAndIndexRegAndScale.
This was here because out and in instructions allow '(%dx)' even though its not a memory reference. To handle this we build a special operand for the DX register reference before we get to the call to CheckBaseRegAndIndexRegAndScale. So we no longer need this special case.
Craig Topper [Sat, 5 Jan 2019 21:40:07 +0000 (21:40 +0000)]
[X86] Allow combinevxi1Bitcast to use pmovmskb on avx512 targets if the input is a truncate from v16i8/v32i8.
This is especially helpful on targets without avx512bw since we don't have a good way to convert from v16i8/v32i8 to v16i1/v32i1 for the truncate anyway. If we're just going to convert it to a GPR we might as well use pmovmskb to accomplish both.
Nikita Popov [Sat, 5 Jan 2019 09:48:16 +0000 (09:48 +0000)]
[InstCombine] Relax cttz/ctlz with select on zero
The cttz/ctlz intrinsics have a parameter specifying whether the
result is undefined for zero. cttz(x, false) can be relaxed to
cttz(x, true) if x is known non-zero, and in fact such an optimization
is already performed. However, this currently doesn't work if x is
non-zero as a result of a select rather than an explicit branch.
This patch adds handling for this case, thus allowing
x != 0 ? cttz(x, false) : y to simplify to x != 0 ? cttz(x, true) : y.
Easwaran Raman [Sat, 5 Jan 2019 02:26:29 +0000 (02:26 +0000)]
[Inliner] Optimize shouldBeDeferred
This has some minor optimizations to shouldBeDeferred. This is not
strictly NFC because the early exit inside the loop assumes
TotalSecondaryCost is monotonically non-decreasing, which is not true if
the threshold used by CostAnalyzer is negative. AFAICT the thresholds do
not go below 0 for the default values of the various options we use.
Craig Topper [Sat, 5 Jan 2019 01:40:29 +0000 (01:40 +0000)]
[X86] Require second operand of X86vshiftuniform to be an integer. NFC
We don't need to require the first operand to be an integer because we already said it was the same type as the result which we also constrained to an integer.
Nico Weber [Sat, 5 Jan 2019 01:39:18 +0000 (01:39 +0000)]
[gn build] Add build files for unittests that load shared libraries
This is slightly ugly for three reasons:
- The shlib needs to go next to the binary to be found on all platforms, so the
build files refer to target_out_dir
- The explicit -fPIC flag needed on the shared lib side, and the -rdynamic flag
needed on the host side, on Linux
- Plugins that refer to LLVM code and assume that the host will resolve them
don't work on Windows -- PluginsTests won't test anything on Windows (but
DynamicLibraryTests will, since the dll here doesn't call LLVM code)
If we get lots more of these plugin / plugin host targets it might make sense
to add a template for them. But for now, these are the last ones we need.
(We're at 6 plugin hosts, 2 of them tests, and at 6 shared libraries, 2 of them
tests as well. clang is a plugin host by default in the CMake build but not
(yet?) in the GN build.)
Nico Weber [Sat, 5 Jan 2019 00:14:37 +0000 (00:14 +0000)]
[gn build] Add build file for DebugInfoPDBTests
I'm pretty unhappy this patch: DebugInfoPDBTests uses an API that requires some
magic txt file to be next to the unit test executable that stores the absolute
path to the LLVM source root.
The choices here are:
1. Don't use the unittest() template for DebugInfoPDBTests and set output_dir
for unit tests in two places (the gni file for every test but this one, and the
BUILD.gn file for this specific test).
2. Add another unittest_foo() template variation for this one test.
I went with the former, and added a comment to the template to look out for
this.
Rong Xu [Fri, 4 Jan 2019 22:54:03 +0000 (22:54 +0000)]
[PGO] Use SourceFileName rather module name in PGOFuncName
In LTO or Thin-lto mode (though linker plugin), the module
names are of temp file names which are different for
different compilations. Using SourceFileName avoids the issue.
This should not change any functionality for current PGO as
all the current callers of getPGOFuncName() is before LTO.
Nikita Popov [Fri, 4 Jan 2019 21:21:43 +0000 (21:21 +0000)]
[BDCE] Remove dead uses of arguments
In addition to finding dead uses of instructions, also find dead uses
of function arguments, and replace them with zero as well.
I'm changing the way the known bits are computed here to remove the
coupling between the transfer function and the algorithm. It previously
relied on the first op being visited first and computing known bits --
unless the first op is not an instruction, in which case they're computed
on the second op. I could have adjusted this to check for "instruction
or argument", but I think it's better to avoid the repeated calculation
with an explicit flag.
hwasan: Implement lazy thread initialization for the interceptor ABI.
The problem is similar to D55986 but for threads: a process with the
interceptor hwasan library loaded might have some threads started by
instrumented libraries and some by uninstrumented libraries, and we
need to be able to run instrumented code on the latter.
The solution is to perform per-thread initialization lazily. If a
function needs to access shadow memory or add itself to the per-thread
ring buffer its prologue checks to see whether the value in the
sanitizer TLS slot is null, and if so it calls __hwasan_thread_enter
and reloads from the TLS slot. The runtime does the same thing if it
needs to access this data structure.
This change means that the code generator needs to know whether we
are targeting the interceptor runtime, since we don't want to pay
the cost of lazy initialization when targeting a platform with native
hwasan support. A flag -fsanitize-hwaddress-abi={interceptor,platform}
has been introduced for selecting the runtime ABI to target. The
default ABI is set to interceptor since it's assumed that it will
be more common that users will be compiling application code than
platform code.
Because we can no longer assume that the TLS slot is initialized,
the pthread_create interceptor is no longer necessary, so it has
been removed.
Ideally, lazy initialization should only cost one instruction in the
hot path, but at present the call may cause us to spill arguments
to the stack, which means more instructions in the hot path (or
theoretically in the cold path if the spills are moved with shrink
wrapping). With an appropriately chosen calling convention for
the per-thread initialization function (TODO) the hot path should
always need just one instruction and the cold path should need two
instructions with no spilling required.
Teresa Johnson [Fri, 4 Jan 2019 19:04:54 +0000 (19:04 +0000)]
[ThinLTO] Handle chains of aliases
At -O0, globalopt is not run during the compile step, and we can have a
chain of an alias having an immediate aliasee of another alias. The
summaries are constructed assuming aliases in a canonical form
(flattened chains), and as a result only the base object but no
intermediate aliases were preserved.
Fix by adding a pass that canonicalize aliases, which ensures each
alias is a direct alias of the base object.
Sanjay Patel [Fri, 4 Jan 2019 17:48:13 +0000 (17:48 +0000)]
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try
The 1st try for this was at rL350369, but it caused IR-level diffs because
our cost models differentiate custom vs. legal/promote lowering. So that was
reverted at rL350373. The cost models were fixed independently at rL350403,
so this is effectively the same patch as last time.
Original commit message:
This would show up if we fix horizontal reductions to narrow as they go along,
but it's an improvement for size and/or Jaguar (fast-hops) independent of that.
We need to do this late to not interfere with other pattern matching of larger
horizontal sequences.
We can extend this to integer ops in a follow-up patch.
Prior to extraction, the stack coloring pass sees that the slots for 'x'
and 'y' are in-use at the same time. After extraction, the coloring pass
infers that 'x' and 'y' are *not* in-use concurrently, because markers
from 'rhs' are no longer available to help decide otherwise.
This leads to a miscompile, because the stack slots actually are in-use
concurrently in the extracted function.
Fix this by moving lifetime start/end markers for memory regions defined
in the calling function around the call to the extracted function.
Sanjay Patel [Fri, 4 Jan 2019 17:38:12 +0000 (17:38 +0000)]
[InstCombine] reduce raw IR narrowing rotate patterns to funnel shift
Similar to rL350199 - there are no known analysis/codegen holes for
funnel shift intrinsics now, so we can canonicalize the 6+ regular
instructions to funnel shift to improve vectorization, inlining,
unrolling, etc.
projects / llvm / log