Jessica Paquette [Mon, 13 Mar 2017 18:39:33 +0000 (18:39 +0000)]
[Outliner] Add tail call support
This commit adds tail call support to the MachineOutliner pass. This allows
the outliner to insert jumps rather than calls in areas where tail calling is
possible. Outlined tail calls include the return or terminator of the basic
block being outlined from.
Tail call support allows the outliner to take returns and terminators into
consideration while finding candidates to outline. It also allows the outliner
to save more instructions. For example, in the X86-64 outliner, a tail called
outlined function saves one instruction since no return has to be inserted.
Craig Topper [Mon, 13 Mar 2017 18:34:46 +0000 (18:34 +0000)]
[X86] Lower AVX2 gather intrinsics similar to AVX-512. Apply the same input source optimizations to break execution dependencies.
For AVX-512 we force the input to zero if the input is undef or the mask is all ones to break an execution dependency. This patch brings the same behavior to AVX2.
Craig Topper [Mon, 13 Mar 2017 18:17:46 +0000 (18:17 +0000)]
[AVX-512] If gather mask is all ones, force the input to a zero vector.
We were already forcing undef inputs to become a zero vector, this now catches an all ones mask too.
Ideally we'd use undef and let execution dep fix handle picking the best register/clearance for the undef, but I don't think it can handle the early clobber today.
Craig Topper [Mon, 13 Mar 2017 17:37:14 +0000 (17:37 +0000)]
[SelectionDAG] Enhance SDTCisSameNumEltsAs to work with scalar types and use it on extend/trunc/round operations.
Currently we don't enforce that ISD::ANY_EXTEND, ZERO_EXTEND, SIGN_EXTEND, TRUNC, FP_ROUND, FP_EXTEND have the same number of elements(including scalar) between their input and output. Though we have them documented as such. Up until a few months ago x86 created nodes that violated this rule. That's all been fixed now, and we should enforce the rule going forward.
In order to do this we need to allow SDTCisSameNumEltsAs to support scalar types and not enforce being a vector. If one type is scalar we will force the other type to also be scalar.
Zachary Turner [Mon, 13 Mar 2017 16:24:10 +0000 (16:24 +0000)]
[ADT] Improve the genericity of llvm::enumerate().
There were some issues in the implementation of enumerate()
preventing it from being used in various contexts. These were
all related to the fact that it did not supporter llvm's
iterator_facade_base class. So this patch adds support for that
and additionally exposes a new helper method to_vector() that
will evaluate an entire range and store the results in a
vector.
Zachary Turner [Mon, 13 Mar 2017 14:57:45 +0000 (14:57 +0000)]
[llvm-pdbdump] Add support for dumping symbols from Yaml -> PDB.
Previously we could round-trip type records from PDB -> Yaml ->
PDB, but for symbols we could only go from PDB -> Yaml. This
completes the round-tripping for symbols as well.
Rafael Espindola [Mon, 13 Mar 2017 14:45:06 +0000 (14:45 +0000)]
Fix crash when multiple raw_fd_ostreams to stdout are created.
If raw_fd_ostream is constructed with the path of "-", it claims
ownership of the stdout file descriptor. This means that it closes
stdout when it is destroyed. If there are multiple users of
raw_fd_ostream wrapped around stdout, then a crash can occur because
of operations on a closed stream.
An example of this would be running something like "clang -S -o - -MD
-MF - test.cpp". Alternatively, using outs() (which creates a local
version of raw_fd_stream to stdout) anywhere combined with such a
stream usage would cause the crash.
The fix duplicates the stdout file descriptor when used within
raw_fd_ostream, so that only that particular descriptor is closed when
the stream is destroyed.
Gil Rapaport [Mon, 13 Mar 2017 10:23:46 +0000 (10:23 +0000)]
[LV] Set memcheck metadata also for VF==1
This commit is a follow-up on r297580. It fixes the FIXME added temporarily
by that commit to keep the removal of Unroller's specialized version of
scalarizeInstruction() an NFC. See https://reviews.llvm.org/D30715 for details.
Craig Topper [Mon, 13 Mar 2017 05:34:03 +0000 (05:34 +0000)]
Revert "[AVX-512] EVEX2VEX, don't reject intrinsic instructions when both have a memory operand. We should just continue to check other operands instead."
This reverts r297596.
There were other issues that were making this not work that have been fixed now. Reverting this results in a more accurate table.
Craig Topper [Mon, 13 Mar 2017 00:36:49 +0000 (00:36 +0000)]
[AVX-512] EVEX2VEX, don't reject intrinsic instructions when both have a memory operand. We should just continue to check other operands instead.
This exposed that we have several intrinsic instructions that have identical TSFlags to other instructions. We should merge their patterns and kill of the duplicate. I'll fix that in a follow up patch.
Craig Topper [Sun, 12 Mar 2017 22:29:12 +0000 (22:29 +0000)]
[AVX-512] Fix the valid immediates for the scatter/gather prefetch intrinsics.
The immediate should be 1 or 2, not 0 or 1. This was found while adding bounds checking to clang. In fact the existing clang builtin test failed if we ran it all the way to assembly.
Sanjay Patel [Sun, 12 Mar 2017 18:28:48 +0000 (18:28 +0000)]
[x86] don't blindly transform SETB into SBB
I noticed unnecessary 'sbb' instructions in D30472 and while looking at 'ptest' codegen recently.
This happens because we were transforming any 'setb' - even when we only wanted a single-bit result.
This patch moves those transforms under visitAdd/visitSub, so we we're only creating sbb/adc when it
is a win. I don't know why we need a SETCC_CARRY node type, but I'm not proposing to change that
existing behavior in this patch.
Also, I'm skeptical that sbb/adc are a win for all micro-arches, so I added comments to the test files
where this transform still fires.
The test changes here are all cases where we no longer produce sbb/adc. Avoiding partial register
stalls (generating an xor to clear a register) is not handled in some cases, but that's a separate
issue.
Gil Rapaport [Sun, 12 Mar 2017 12:31:38 +0000 (12:31 +0000)]
[LV] A unified scalarizeInstruction() for Vectorizer and Unroller; NFC
Unroller's specialized scalarizeInstruction() is mostly duplicating Vectorizer's
variant. OTOH Vectorizer's scalarizeInstruction() already supports the special
case of VF==1 except for avoiding mask-bit extraction in that case. This patch
removes Unroller's specialized version in favor of a unified method.
The only functional difference between the two variants seems to be setting
memcheck metadata for loads and stores only in Vectorizer's variant, which is a
bug in Unroller. To keep this patch an NFC the unified method doesn't set
memcheck metadata for VF==1.
Craig Topper [Sun, 12 Mar 2017 03:37:37 +0000 (03:37 +0000)]
[AVX-512] Fix a bad use of a high GR8 register after copying from a mask register during fast isel. This ends up extracting from bits 15:8 instead of the lower bits of the mask.
I'm pretty sure there are more problems lurking here. But I think this fixes PR32241.
I've added the test case from that bug and added asserts that will fail if we ever try to copy between high registers and mask registers again.
Simon Pilgrim [Sat, 11 Mar 2017 20:42:31 +0000 (20:42 +0000)]
[X86][SSE] Improve extraction of elements from v16i8 (pre-SSE41)
Without SSE41 (pextrb) we currently extract byte elements from a vector by spilling to stack and reloading the byte.
This patch is an initial attempt at using MOVD/PEXTRW to extract the relevant DWORD/WORD from the vector and then shift+truncate to collect the correct byte.
Extraction of multiple bytes this way would result in code bloat, but as explained in the patch we could probably afford to be more aggressive with the supported extractions before again falling back on spilling - possibly through counting the number of extracts and which DWORD/WORD they originate?
Matt Arsenault [Sat, 11 Mar 2017 05:40:40 +0000 (05:40 +0000)]
AMDGPU: Keep track of modifiers when converting v_mac to v_mad
Since v_max_f32_e64/v_max_f16_e64 can be folded if the target
instruction supports the clamp bit, we also need to maintain
modifiers when converting v_mac to v_mad.
This fixes a rendering issue with Dirt Rally because a v_mac
instruction with the clamp bit set was converted to a v_mad
but that bit was lost during the conversion.
Fixes: e184e01dd79 ("AMDGPU: Fold FP clamp as modifier bit")
Patch by Samuel Pitoiset <samuel.pitoiset@gmail.com>
Eric Fiselier [Sat, 11 Mar 2017 02:24:13 +0000 (02:24 +0000)]
Revert r297516 - Respect CMAKE_INSTALL_MANDIR for sphinx generated manpages
When CMAKE_INSTALL_MANDIR isn't defined it ends up attempting to install
the man pages under "/man1" and we really don't want to accidentally install
stuff at the filesystem root.
Daniel Berlin [Sat, 11 Mar 2017 01:41:03 +0000 (01:41 +0000)]
Remove opt-bisect support for "cases" in favor of debug counters
Summary:
Ths "cases" support was not quite finished, is unused, and is really just debug counters.
(well, almost, debug counters are slightly more powerful, in that they can skip things at the start, too).
Note, opt-bisect itself could also be implemented as a wrapper around
debug counters, but not sure it's worth it ATM.
Jordan Rose [Sat, 11 Mar 2017 01:24:56 +0000 (01:24 +0000)]
[unittest] Explicitly specify alignment when using BumpPtrAllocator.
r297310 began inserting red zones around allocations under ASan, which
perturbs the alignment of subsequent allocations. Deliberately specify
this in two places where it matters.
Fixes failures when these tests are run under ASan and UBSan together.
Reviewed by Duncan Exon Smith.
Sanjoy Das [Sat, 11 Mar 2017 01:15:48 +0000 (01:15 +0000)]
Use a WeakVH for UnknownInstructions in AliasSetTracker
Summary:
This change solves the same problem as D30726, except that this only
throws out the bathwater.
AST was not correctly tracking and deleting UnknownInstructions via
handles. The existing code only tracks "pointers" in its
`ASTCallbackVH`, so an UnknownInstruction (that isn't also def'ing a
pointer used by another memory instruction) never gets a
`ASTCallbackVH`.
There are two other ways to solve this problem:
- Use the `PointerRec` scheme for both known and unknown instructions.
- Use a `CallbackVH` that erases the offending Instruction from the
UnknownInstruction list.
Both of the above changes seemed to be significantly (and unnecessarily
IMO) more complex than this.
This method inverts the Reason field of a scheduling candidate.
It does right comparison between RegCritical and RegExcess, but
everything else is broken. In fact it can prefer less strong reason
such as Weak over RegCritical because Weak > -RegCritical.
The CandReason enum is properly sorted, so just remove artificial
ranking.
Quentin Colombet [Sat, 11 Mar 2017 00:28:33 +0000 (00:28 +0000)]
[IRTranslator] Simplify error handling for translating constants. NFC.
We don't need to check whether the fallback path is enabled to return
false. Just do that all the time on error cases, the caller knows (or
at least should know!) how to handle the failing case.
The problem can occur in presence of subregs. If we are swapping two
instructions defining different subregs of the same register we will
get a new liveout from a block. We need to preserve value number for
block's liveout for successor block's livein to match.
This function will find the closest ref node aliased to Reg that is
in an instruction preceding Inst. This could be used to identify the
hypothetical reaching def of Reg, if Reg was a member of Inst.
Evandro Menezes [Fri, 10 Mar 2017 20:20:04 +0000 (20:20 +0000)]
[AArch64, X86] Additional debug information for MacroFusion
In order to make it easier to parse information about the performance of
MacroFusion, this patch adds the function and the instruction names to the
debug output of this pass.
[SLP] Revert everything that has to do with memory access sorting.
This reverts r293386, r294027, r294029 and r296411.
Turns out the SLP tree isn't actually a "tree" and we don't handle
accessing the same packet of loads in several different orders well,
causing miscompiles.