MergeFunc: preserve COMDAT information when creating a thunk
We would previously drop the COMDAT on the thunk we generated when replacing a
function body with the forwarding thunk. This would result in a function that
may have been multiply emitted and multiply merged to be emitted with the same
name without the COMDAT. This is a hard error with PE/COFF where the COMDAT is
used for the deduplication of Value Witness functions for Swift.
Adrian Prantl [Thu, 18 Apr 2019 21:22:50 +0000 (21:22 +0000)]
Implement sys::fs::copy_file using the macOS copyfile(3) API
to support APFS clones.
This patch adds a Darwin-specific implementation of
llvm::sys::fs::copy_file() that uses the macOS copyfile(3) API to
support APFS copy-on-write clones, which should be faster and much
more space efficient.
[GlobalISel][AArch64] Legalize/select G_(S/Z/ANY)_EXT for v8s8s
This adds legalization for G_SEXT, G_ZEXT, and G_ANYEXT for v8s8s.
We were falling back on G_ZEXT in arm64-vabs.ll before, preventing us from
selecting the @llvm.aarch64.neon.sabd.v8i8 intrinsic.
This adds legalizer support for those 3, which gives us selection via the
importer. Update the relevant tests (legalize-ext.mir, select-int-ext.mir) and
add a GISel line to arm64-vabs.ll.
[BlockExtractor] Extend the file format to support the grouping of basic blocks
Prior to this patch, each basic block listed in the extrack-blocks-file
would be extracted to a different function.
This patch adds the support for comma separated list of basic blocks
to form group.
When the region formed by a group is not extractable, e.g., not single
entry, all the blocks of that group are left untouched.
Let us see this new format in action (comments are not part of the
file format):
;; funcName bbName[,bbName...]
foo bb1 ;; Extract bb1 in its own function
foo bb2,bb3 ;; Extract bb2,bb3 in their own function
bar bb1,bb4 ;; Extract bb1,bb4 in their own function
bar bb2 ;; Extract bb2 in its own function
Assuming all regions are extractable, this will create one function and
thus one call per region.
Simon Pilgrim [Thu, 18 Apr 2019 17:23:09 +0000 (17:23 +0000)]
[X86] combineVectorTruncationWithPACKUS - remove split/concatenation of mask
combineVectorTruncationWithPACKUS is currently splitting the upper bit bit masking into 128-bit subregs and then concatenating them back together.
This was originally done to avoid regressions that caused existing subregs to be concatenated to the larger type just for the AND masking before being extracted again. This was fixed by @spatel (notably rL303997 and rL347356).
This also lets SimplifyDemandedBits do some further improvements before it hits the recursive depth limit.
My only annoyance with this is that we were broadcasting some xmm masks but we seem to have lost them by moving to ymm - but that's a known issue as the logic in lowerBuildVectorAsBroadcast isn't great.
Philip Reames [Thu, 18 Apr 2019 17:01:19 +0000 (17:01 +0000)]
[LoopPred] Fix a blatantly obvious bug in r358684
The bug is that I didn't check whether the operand of the invariant_loads were themselves invariant. I don't know how this got missed in the patch and review. I even had an unreduced test case locally, and I remember handling this case, but I must have lost it in one of the rebases. Oops.
Stefan Granitz [Thu, 18 Apr 2019 16:37:07 +0000 (16:37 +0000)]
[CMake] Allow custom extensions for externalized debug info
Summary:
Extra flexibility for emitting debug info to external files (remains Darwin only for now).
LLDB needs this functionality to emit a LLDB.framework.dSYM instead of LLDB.dSYM when building the framework, because the latter could conflict with the driver's lldb.dSYM when emitted in the same directory on case-insensitive file systems.
Philip Reames [Thu, 18 Apr 2019 16:33:17 +0000 (16:33 +0000)]
[LoopPredication] Allow predication of loop invariant computations (within the loop)
The purpose of this patch is to eliminate a pass ordering dependence between LoopPredication and LICM. To understand the purpose, consider the following snippet of code inside some loop 'L' with IV 'i'
A = _a.length;
guard (i < A)
a = _a[i]
B = _b.length;
guard (i < B);
b = _b[i];
...
Z = _z.length;
guard (i < Z)
z = _z[i]
accum += a + b + ... + z;
Today, we need LICM to hoist the length loads, LoopPredication to make the guards loop invariant, and TrivialUnswitch to eliminate the loop invariant guard to establish must execute for the next length load. Today, if we can't prove speculation safety, we'd have to iterate these three passes 26 times to reduce this example down to the minimal form.
Using the fact that the array lengths are known to be invariant, we can short circuit this iteration. By forming the loop invariant form of all the guards at once, we remove the need for LoopPredication from the iterative cycle. At the moment, we'd still have to iterate LICM and TrivialUnswitch; we'll leave that part for later.
As a secondary benefit, this allows LoopPred to expose peeling oppurtunities in a much more obvious manner. See the udiv test changes as an example. If the udiv was not hoistable (i.e. we couldn't prove speculation safety) this would be an example where peeling becomes obviously profitable whereas it wasn't before.
A couple of subtleties in the implementation:
- SCEV's isSafeToExpand guarantees speculation safety (i.e. let's us expand at a new point). It is not a precondition for expansion if we know the SCEV corresponds to a Value which dominates the requested expansion point.
- SCEV's isLoopInvariant returns true for expressions which compute the same value across all iterations executed, regardless of where the original Value is located. (i.e. it can be in the loop) This implies we have a speculation burden to prove before expanding them outside loops.
- invariant_loads and AA->pointsToConstantMemory are two cases that SCEV currently does not handle, but meets the SCEV definition of invariance. I plan to sink this part into SCEV once this has baked for a bit.
[SDA] Bug fix: Use IPD outside the loop as divergence bound
Summary:
The immediate post dominator of the loop header may be part of the divergent loop.
Since this /was/ the divergence propagation bound the SDA would not detect joins of divergent paths outside the loop.
Philip Reames [Thu, 18 Apr 2019 16:10:21 +0000 (16:10 +0000)]
Fix a bug in SCEV's isSafeToExpand around speculation safety
isSafeToExpand was making a common, but dangerously wrong, mistake in assuming that if any instruction within a basic block executes, that all instructions within that block must execute. This can be trivially shown to be false by considering the following small example:
bb:
add x, y <-- InsertionPoint
call @throws()
udiv x, y <-- SCEV* S
br ...
It's clearly not legal to expand S above the throwing call, but the previous logic would do so since S dominates (but not properlyDominates) the block containing the InsertionPoint.
Since iterating instructions w/in a block is expensive, this change special cases two cases: 1) S is an operand of InsertionPoint, and 2) InsertionPoint is the terminator of it's block. These two together are enough to keep all current optimizations triggering while fixing the latent correctness issue.
As best I can tell, this is a silent bug in current ToT. Given that, there's no tests with this change. It was noticed in an upcoming optimization change (D60093), and was reviewed as part of that. That change will include the test which caused me to notice the issue. I'm submitting this seperately so that anyone bisecting a problem gets a clear explanation.
Pavel Labath [Thu, 18 Apr 2019 14:57:31 +0000 (14:57 +0000)]
MinidumpYAML: Add support for ModuleList stream
Summary:
This patch adds support for yaml (de)serialization of the minidump
ModuleList stream. It's a fairly straight forward-application of the
existing patterns to the ModuleList structures defined in previous
patches.
One thing, which may be interesting to call out explicitly is the
addition of "new" allocation functions to the helper BlobAllocator
class. The reason for this was, that there was an emerging pattern of a
need to allocate space for entities, which do not have a suitable
lifetime for use with the existing allocation functions. A typical
example of that was the "size" of various lists, which is only available
as a temporary returned by the .size() method of some container. For
these cases, one can use the new set of allocation functions, which
will take a temporary object, and store it in an allocator-managed
buffer until it is written to disk.
with the more general icmp lowering so it can pick up more cases through bitcasts - notably vXi8 cases which use vXi16 shifts+masks, this patch can remove the mask and use pcmpgtb(0,x) for the sra.
James Henderson [Thu, 18 Apr 2019 09:13:30 +0000 (09:13 +0000)]
[llvm-objcopy][llvm-strip] Add switch to allow removing referenced sections
llvm-objcopy currently emits an error if a section to be removed is
referenced by another section. This is a reasonable thing to do, but is
different to GNU objcopy. We should allow users who know what they are
doing to have a way to produce the invalid ELF. This change adds a new
switch --allow-broken-links to both llvm-strip and llvm-objcopy to do
precisely that. The corresponding sh_link field is then set to 0 instead
of an error being emitted.
I cannot use llvm-readelf/readobj to test the link fields because they
emit an error if any sections, like the .dynsym, cannot be properly
loaded.
Kang Zhang [Thu, 18 Apr 2019 07:24:15 +0000 (07:24 +0000)]
[PowerPC] Fix wrong ElemSIze when calling isConsecutiveLS()
Summary:
This issue from the bugzilla: https://bugs.llvm.org/show_bug.cgi?id=41177
When the two operands for BUILD_VECTOR are same, we will get assert error.
llvm::SDValue combineBVOfConsecutiveLoads(llvm::SDNode*, llvm::SelectionDAG&):
Assertion `!(InputsAreConsecutiveLoads && InputsAreReverseConsecutive) &&
"The loads cannot be both consecutive and reverse consecutive."' failed.
This error caused by the wrong ElemSIze when calling isConsecutiveLS(). We
should use `getScalarType().getStoreSize();` to get the ElemSize instread of
`getScalarSizeInBits() / 8`.
Tim Renouf [Thu, 18 Apr 2019 05:27:01 +0000 (05:27 +0000)]
[AMDGPU] Avoid DAG combining assert with fneg(fadd(A,0))
fneg combining attempts to turn it into fadd(fneg(A), fneg(0)), but
creating the new fadd folds to just fneg(A). When A has multiple uses,
this confuses it and you get an assert. Fixed.
Richard Trieu [Thu, 18 Apr 2019 01:39:45 +0000 (01:39 +0000)]
Fix bad compare function over FusionCandidate.
Reverse the checking of the domiance order so that when a self compare happens,
it returns false. This makes compare function have strict weak ordering.
Adrian Prantl [Thu, 18 Apr 2019 00:01:05 +0000 (00:01 +0000)]
Implement sys::fs::copy_file using the macOS copyfile(3) API
to support APFS clones.
This patch adds a Darwin-specific implementation of
llvm::sys::fs::copy_file() that uses the macOS copyfile(3) API to
support APFS copy-on-write clones, which should be faster and much
more space efficient.
...so if we can fold the shift op into LEA in the 1st pattern, then we
should be able to do the same in the 2nd pattern (unnecessary 'movzbl'
is a separate bug I think).
We don't want to do this any sooner though because that would conflict
with generic transforms that try to narrow the width of the shift.
Nick Desaulniers [Wed, 17 Apr 2019 22:21:10 +0000 (22:21 +0000)]
[AsmPrinter] hoist %a output template to base class for ARM+Aarch64
Summary:
X86 is quite complicated; so I intend to leave it as is. ARM+Aarch64 do
basically the same thing (Aarch64 did not correctly handle immediates,
ARM has a test llvm/test/CodeGen/ARM/2009-04-06-AsmModifier.ll that uses
%a with an immediate) for a flag that should be target independent
anyways.
This patch adds a basic loop fusion pass. It will fuse loops that conform to the
following 4 conditions:
1. Adjacent (no code between them)
2. Control flow equivalent (if one loop executes, the other loop executes)
3. Identical bounds (both loops iterate the same number of iterations)
4. No negative distance dependencies between the loop bodies.
The pass does not make any changes to the IR to create opportunities for fusion.
Instead, it checks if the necessary conditions are met and if so it fuses two
loops together.
The pass has not been added to the pass pipeline yet, and thus is not enabled by
default. It can be run stand alone using the -loop-fusion option.
Nick Desaulniers [Wed, 17 Apr 2019 18:22:48 +0000 (18:22 +0000)]
[AsmPrinter] defer %c to base class for ARM, PPC, and Hexagon. NFC
Summary:
None of these derived classes do anything that the base class cannot.
If we remove these case statements, then the base class can handle them
just fine.
Steven Wu [Wed, 17 Apr 2019 17:38:09 +0000 (17:38 +0000)]
[ThinLTO] Fix ThinLTOCodegenerator to export llvm.used symbols
Summary:
Reapply r357931 with fixes to ThinLTO testcases and llvm-lto tool.
ThinLTOCodeGenerator currently does not preserve llvm.used symbols and
it can internalize them. In order to pass the necessary information to the
legacy ThinLTOCodeGenerator, the input to the code generator is
rewritten to be based on lto::InputFile.
Now ThinLTO using the legacy LTO API will requires data layout in
Module.
"internalize" thinlto action in llvm-lto is updated to run both
"promote" and "internalize" with the same configuration as
ThinLTOCodeGenerator. The old "promote" + "internalize" option does not
produce the same output as ThinLTOCodeGenerator.
Philip Reames [Wed, 17 Apr 2019 17:37:58 +0000 (17:37 +0000)]
[InstCombine] Factor out unreachable inst idiom creation [NFC]
In InstCombine, we use an idiom of "store i1 true, i1 undef" to indicate we've found a path which we've proven unreachable. We can't actually insert the unreachable instruction since that would require changing the CFG. We leave that to simplifycfg later.
This just factors out that idiom creation so we don't duplicate the same mostly undocument idiom creation in multiple places.
[LVI][CVP] Constrain values in with.overflow branches
If a branch is conditional on extractvalue(op.with.overflow(%x, C), 1)
then we can constrain the value of %x inside the branch based on
makeGuaranteedNoWrapRegion(). We do this by extending the edge-value
handling in LVI. This allows CVP to then fold comparisons against %x,
as illustrated in the tests.
Rhys Perry [Wed, 17 Apr 2019 16:31:52 +0000 (16:31 +0000)]
AMDGPU: Force skip over SMRD, VMEM and s_waitcnt instructions
Summary: This fixes a large Dawn of War 3 performance regression with RADV from Mesa 19.0 to master which was caused by creating less code in some branches.
This will change with the proposal in D60214.
Unfortunately, the triple is not supported for auto-generation
via script, and the multiple RUN lines have diffs on this test,
but I can't tell exactly what is required by this test.
PR7162 was an assert/crash, so hopefully, this is good enough.
[LoopUnroll] Allow unrolling if the unrolled size does not exceed loop size.
Summary:
In the following cases, unrolling can be beneficial, even when
optimizing for code size:
1) very low trip counts
2) potential to constant fold most instructions after fully unrolling.
We can unroll in those cases, by setting the unrolling threshold to the
loop size. This might highlight some cost modeling issues and fixing
them will have a positive impact in general.
Lang Hames [Wed, 17 Apr 2019 15:38:27 +0000 (15:38 +0000)]
[Support] Add LEB128 support to BinaryStreamReader/Writer.
Summary:
This patch adds support for ULEB128 and SLEB128 encoding and decoding to
BinaryStreamWriter and BinaryStreamReader respectively.
Support for ULEB128/SLEB128 will be used for eh-frame parsing in the JITLink
library currently under development (see https://reviews.llvm.org/D58704).
[ScheduleDAGRRList] Recompute topological ordering on demand.
Currently there is a single point in ScheduleDAGRRList, where we
actually query the topological order (besides init code). Currently we
are recomputing the order after adding a node (which does not have
predecessors) and then we add predecessors edge-by-edge.
We can avoid adding edges one-by-one after we added a new node. In that case, we can
just rebuild the order from scratch after adding the edges to the DAG
and avoid all the updates to the ordering.
Also, we can delay updating the DAG until we query the DAG, if we keep a
list of added edges. Depending on the number of updates, we can either
apply them when needed or recompute the order from scratch.
This brings down the geomean compile time for of CTMark with -O1 down 0.3% on X86,
with no regressions.
On pre-AVX512 targets we can use MOVMSK to extract reduced boolean results. This is properly optimized, annoyingly AVX512 isn't and produces code that is almost as bad as the (unchanged) costs suggest......
Jordan Rupprecht [Wed, 17 Apr 2019 07:42:31 +0000 (07:42 +0000)]
[llvm-objcopy] Support full list of bfd targets that lld uses.
Summary:
This change takes the full list of bfd targets that lld supports (see `ScriptParser.cpp`), including generic handling for `*-freebsd` targets (which uses the same settings but with a FreeBSD OSABI). In particular this adds mips support for `--output-target` (but not yet via `--binary-architecture`).
lld and llvm-objcopy use their own different custom data structures, so I'd prefer to check this in as-is (add support directly in llvm-objcopy, including all the test coverage) and do a separate NFC patch(s) that consolidate the two by putting this mapping into libobject.
See [[ https://bugs.llvm.org/show_bug.cgi?id=41462 | PR41462 ]].
[X86] In CopyToFromAsymmetricReg, use VR128 instead of FR32 instructions for GR32<->XMM register copies.
We have two versions of some instructions, VR128 versions and FR32 versions that
are marked as CodeGenOnly.
This change switches to using the VR128 versions for these copies. It's after
register allocation so the class size no longer matters. This matches how GR64
works.
This patch adds a basic loop fusion pass. It will fuse loops that conform to the
following 4 conditions:
1. Adjacent (no code between them)
2. Control flow equivalent (if one loop executes, the other loop executes)
3. Identical bounds (both loops iterate the same number of iterations)
4. No negative distance dependencies between the loop bodies.
The pass does not make any changes to the IR to create opportunities for fusion.
Instead, it checks if the necessary conditions are met and if so it fuses two
loops together.
The pass has not been added to the pass pipeline yet, and thus is not enabled by
default. It can be run stand alone using the -loop-fusion option.
Sam McCall [Tue, 16 Apr 2019 23:53:28 +0000 (23:53 +0000)]
[ADT] llvm::bsearch, binary search for mere mortals
Summary:
Add to STLExtras a binary search function with a simple mental model:
You provide a range and a predicate which is true above a certain point.
bsearch() tells you that point.
Overloads are provided for integers, iterators, and containers.
This is more suitable than std:: alternatives in many cases:
- std::binary_search only indicates presence/absence
- upper_bound/lower_bound give you the opportunity to pick the wrong one
- all of the options have confusing names and definitions when your predicate
doesn't have simple "less than" semantics
- all of the options require iterators
- we plumb around a useless `value` parameter that should be a lambda capture
The API is inspired by Go's standard library, but we add an extra parameter as
well as some overloads and templates to show how clever C++ is.
Robert Widmann [Tue, 16 Apr 2019 21:39:48 +0000 (21:39 +0000)]
[LLVM-C] Add Accessors For Global Variable Metadata Properties
Summary: Metadata for a global variable is really a (GlobalVariable, Expression) tuple. Allow access to these, then allow retrieving the file, scope, and line for a DIVariable, whether global or local. This should be the last of the accessors required for uniform access to location and file information metadata.
Nick Desaulniers [Tue, 16 Apr 2019 21:04:34 +0000 (21:04 +0000)]
[NVPTXAsmPrinter] clean up dead code. NFC
Summary:
The printOperand function takes a default parameter, for which there are
zero call sites that explicitly pass such a parameter. As such, there
is no case to support. This means that the method
printVecModifiedImmediate is purly dead code, and can be removed.
The eventual goal for some of these AsmPrinter refactoring is to have
printOperand be a virtual method; making it easier to print operands
from the base class for more generic Asm printing. It will help if all
printOperand methods have the same function signature (ie. no Modifier
argument when not needed).
[EarlyCSE] detect equivalence of selects with inverse conditions and commuted operands (PR41101)
This is 1 of the problems discussed in the post-commit thread for:
rL355741 / http://lists.llvm.org/pipermail/llvm-commits/Week-of-Mon-20190311/635516.html
and filed as:
https://bugs.llvm.org/show_bug.cgi?id=41101
Instcombine tries to canonicalize some of these cases (and there's room for improvement
there independently of this patch), but it can't always do that because of extra uses.
So we need to recognize these commuted operand patterns here in EarlyCSE. This is similar
to how we detect commuted compares and commuted min/max/abs.
Anton Afanasyev [Tue, 16 Apr 2019 20:36:56 +0000 (20:36 +0000)]
Time profiler: optimize json output time
Summary:
Use llvm::json::Array.reserve() to optimize json output time. Here is motivation:
https://reviews.llvm.org/D60609#1468941. In short: for the json array
with ~32K entries, pushing back each entry takes ~4% of whole time compared
to the method of preliminary memory reservation: (3995-3845)/3995 = 3.75%.
[CVP] Simplify umulo and smulo that cannot overflow
If a umul.with.overflow or smul.with.overflow operation cannot
overflow, simplify it to a simple mul nuw / mul nsw. After the
refactoring in D60668 this is just a matter of removing an
explicit check against multiplications.
Anton Afanasyev [Tue, 16 Apr 2019 19:43:18 +0000 (19:43 +0000)]
[Support][JSON] Add reserve() to json Array
Summary:
Space reservation increases json lib performance for the arrays with large number of entries.
Here is the example and discussion: https://reviews.llvm.org/D60609#1468941
Simon Pilgrim [Tue, 16 Apr 2019 19:27:00 +0000 (19:27 +0000)]
[SLP] Refactoring of the operand reordering code.
This is a refactoring patch which should have all the functionality of the current code. Its goal is twofold:
i. Cleanup and simplify the reordering code, and
ii. Generalize reordering so that it will work for an arbitrary number of operands, not just 2.
This is the second patch in a series of patches that will enable operand reordering across chains of operations. An example of this was presented in EuroLLVM'18 https://www.youtube.com/watch?v=gIEn34LvyNo .
Committed on behalf of @vporpo (Vasileios Porpodas)
Simon Pilgrim [Tue, 16 Apr 2019 19:18:53 +0000 (19:18 +0000)]
[X86][AVX] X86ISD::PERMV/PERMV3 node types can never fold index ops
Improves codegen demonstrated by D60512 - instructions represented by X86ISD::PERMV/PERMV3 can never memory fold the operand used for their index register.
This patch updates the 'isUseOfShuffle' helper into the more capable 'isFoldableUseOfShuffle' that recognises that the op is used for a X86ISD::PERMV/PERMV3 index mask and can't be folded - allowing us to use broadcast/subvector-broadcast ops to reduce the size of the mask constant pool data.
[InstCombine] Prune fshl/fshr with masked operands
If a constant shift amount is used, then only some of the LHS/RHS
operand bits are demanded and we may be able to simplify based on
that. InstCombineSimplifyDemanded already had the necessary support
for that, we just weren't calling it with fshl/fshr as root.
In particular, this allows us to relax some masked funnel shifts
into simple shifts, as shown in the tests.
This adds a WithOverflowInst class with a few helper methods to get
the underlying binop, signedness and nowrap type and makes use of it
where sensible. There will be two more uses in D60650/D60656.
The refactorings are all NFC, though I left some TODOs where things
could be improved. In particular we have two places where add/sub are
handled but mul isn't.
Luis Marques [Tue, 16 Apr 2019 15:09:18 +0000 (15:09 +0000)]
[DAGCombiner] Add missing flag to addressing mode check
The checks in `canFoldInAddressingMode` tested for addressing modes that have a
base register but didn't set the `HasBaseReg` flag to true (it's false by
default). This patch fixes that. Although the omission of the flag was
technically incorrect it had no known observable impact, so no tests were
changed by this patch.
projects / llvm / log