Commit Graph

811 Commits

Author SHA1 Message Date
Kazu Hirata 343de6856e [Transforms] Use std::nullopt instead of None (NFC)
This patch mechanically replaces None with std::nullopt where the
compiler would warn if None were deprecated.  The intent is to reduce
the amount of manual work required in migrating from Optional to
std::optional.

This is part of an effort to migrate from llvm::Optional to
std::optional:

https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716
2022-12-02 21:11:37 -08:00
Krzysztof Parzyszek 86fe4dfdb6 TargetTransformInfo: convert Optional to std::optional
Recommit: added missing "#include <cstdint>".
2022-12-02 11:42:15 -08:00
Krzysztof Parzyszek 4e12d1836a Revert "TargetTransformInfo: convert Optional to std::optional"
This reverts commit b83711248c.

Some buildbots are failing.
2022-12-02 11:34:04 -08:00
Krzysztof Parzyszek b83711248c TargetTransformInfo: convert Optional to std::optional 2022-12-02 11:27:12 -08:00
Krzysztof Parzyszek 467432899b MemoryLocation: convert Optional to std::optional 2022-12-01 15:36:20 -08:00
William Huang be4b1dd35b [InstCombine] Revert D125845
Reverting D125845 `[InstCombine] Canonicalize GEP of GEP by swapping constant-indexed GEP to the back` because multiple users reported performance regression

Reviewed By: davidxl

Differential Revision: https://reviews.llvm.org/D138950
2022-11-29 22:02:40 +00:00
Kazu Hirata 3da96e0361 [InstCombine] Use std::optional in InstructionCombining.cpp (NFC)
This is part of an effort to migrate from llvm::Optional to
std::optional:

https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716
2022-11-25 23:49:50 -08:00
Thomas Symalla 470aea5ed4 [InstCombine] Fold extractelt with select of constants
An extractelt with a constant index which extracts an element from the
two vector operands of a select can be directly folded into a select.

extractelt (select %x, %vec1, %vec2), %const ->
select %x, %vec1[%const], %vec2[%const]

Note: the implementation currently only works for constant vector operands.

Reviewed By: foad, spatel

Differential Revision: https://reviews.llvm.org/D137934
2022-11-22 14:07:06 +01:00
OCHyams 4ba08d512c [Assignment Tracking][24/*] Always RemoveRedundantDbgInstrs in instcombine in assignment tracking builds
The Assignment Tracking debug-info feature is outlined in this RFC:

https://discourse.llvm.org/t/
rfc-assignment-tracking-a-better-way-of-specifying-variable-locations-in-ir

This reduces peak memory overhead by 15% when building CTMark's tramp3d-v4 with
-O2 -g with assignment tracking enabled.

Reviewed By: jmorse

Differential Revision: https://reviews.llvm.org/D133321
2022-11-18 12:36:41 +00:00
OCHyams fcd5098a03 [Assignment Tracking][14/*] Account for assignment tracking in instcombine
The Assignment Tracking debug-info feature is outlined in this RFC:

https://discourse.llvm.org/t/
rfc-assignment-tracking-a-better-way-of-specifying-variable-locations-in-ir

Most of the updates here are just to ensure DIAssignID attachments are
maintained and propagated correctly.

Reviewed By: jmorse

Differential Revision: https://reviews.llvm.org/D133307
2022-11-18 09:25:33 +00:00
Sanjay Patel 362c23500a Revert "[InstCombine] allow more folds for multi-use selects (2nd try)"
This reverts commit 6eae6b3722.

This version of the patch results in the same DFSAN bot failure as before,
so my guess about the SimplifyQuery context instruction was wrong.
I don't know what the real bug is.
2022-11-13 11:47:21 -05:00
Sanjay Patel 6eae6b3722 [InstCombine] allow more folds for multi-use selects (2nd try)
The 1st try ( 681a6a3990 ) was reverted because it caused
a DataFlowSanitizer bot failure.

This try modifies the existing calls to simplifyBinOp() to
not use a query that sets the context instruction because
that seems like a likely source of failure. Since we already
try those simplifies with multi-use patterns in some cases,
that means the bug is likely present even without this patch.

However, I have not been able to reduce a test to prove that
this was the bug, so if we see any bot failures with this patch,
then it should be reverted again.

The reduced simplify power does not affect any optimizations
in existing, motivating regression tests.

Original commit message:

The 'and' case showed up in a recent bug report and prevented
more follow-on transforms from happening.

We could handle more patterns (for example, the select arms
simplified, but not to constant values), but this seems
like a safe, conservative enhancement. The backend can
convert select-of-constants to math/logic in many cases
if it is profitable.

There is a lot of overlapping logic for these kinds of patterns
(see SimplifySelectsFeedingBinaryOp() and FoldOpIntoSelect()),
so there may be some opportunity to improve efficiency.

There are also optimization gaps/inconsistency because we do
not call this code for all bin-opcodes (see TODO for ashr test).
2022-11-13 10:28:06 -05:00
Michał Górny f6f1fd443f Revert "[InstCombine] allow more folds more multi-use selects"
This reverts commit 681a6a3990.
It broke sanitizer tests (as seen on buildbots), see:
https://reviews.llvm.org/rG681a6a399022#1143137
2022-11-13 07:27:01 +01:00
Sanjay Patel 681a6a3990 [InstCombine] allow more folds more multi-use selects
The 'and' case showed up in a recent bug report and prevented
more follow-on transforms from happening.

We could handle more patterns (for example, the select arms
simplified, but not to constant values), but this seems
like a safe, conservative enhancement. The backend can
convert select-of-constants to math/logic in many cases
if it is profitable.

There is a lot of overlapping logic for these kinds of patterns
(see SimplifySelectsFeedingBinaryOp() and FoldOpIntoSelect()),
so there may be some opportunity to improve efficiency.

There are also optimization gaps/inconsistency because we do
not call this code for all bin-opcodes (see TODO for ashr test).
2022-11-11 15:26:54 -05:00
William Huang bd2b5ec803 [InstCombine] PR58901 - fix bug with swapping GEP of different types
Fix https://github.com/llvm/llvm-project/issues/58901 by adding stricter check whether non-opaque GEP can be swapped. This will not affect GEP swapping optimization in the future since we are switching to opaque GEP

Reviewed By: clin1

Differential Revision: https://reviews.llvm.org/D137752
2022-11-10 20:24:41 +00:00
Sanjay Patel 1f8ac37e2d [InstCombine] adjust branch on logical-and fold
The transform was just added with:
115d2f69a5
...but as noted in post-commit feedback, it was
confusingly coded. Now, we create the final
expected canonicalized form directly and put
an extra use check on the match, so we should
not ever end up with more instructions.
2022-10-31 17:39:29 -04:00
Sanjay Patel 115d2f69a5 [InstCombine] canonicalize branch with logical-and-not condition
https://alive2.llvm.org/ce/z/EfHlWN

In the motivating case from issue #58313,
this allows forming a duplicate 'not' op
which then gets CSE'd and simplifyCFG'd
and combined into the expected 'xor'.
2022-10-31 15:51:45 -04:00
zhongyunde f58311796c [InstCombine] refactor the SimplifyUsingDistributiveLaws NFC
Precommit for D136015
Reviewed By: spatel
Differential Revision: https://reviews.llvm.org/D137019
2022-10-30 21:04:06 +08:00
David Green 5dd7d2ce67 [InstCombine] Don't change switch table from desirable to illegal types
In InstCombine we treat i8/i16 as desirable, even if they are not legal.
The current logic in shouldChangeType will decide to convert from an
illegal but desirable type (such as an i8) to an illegal and undesirable
type (such as i3). This patch prevents changing the switch conditions to
an irregular type on like Arm/AArch64 where i8/i16 are not legal.

This is the same issue as https://reviews.llvm.org/D54115. In the case I
was looking it is was converting an i32 switch to an i8 switch, which
then became a i3 switch.

Differential Revision: https://reviews.llvm.org/D136763
2022-10-28 10:15:41 +01:00
William Huang 6c767cef5a [InstCombine] Canonicalize GEP of GEP by swapping constant-indexed GEP to the back
Canonicalize GEP of GEP by swapping GEP with some suffix constant indices to the back (and GEP with all constant indices to the back of that), this allows more constant index GEP merging to happen. Exceptions are: If swapping violates use-def relations, or anti-optimizes LICM

For constant indexed GEP of GEP, if they cannot be merged directly, they will be casted to i8* and merged.

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D125845
2022-10-20 17:41:26 +00:00
Sanjay Patel d16989607b [InstCombine] reduce code duplication in visitBranchInst(); NFCI 2022-10-18 11:34:02 -04:00
Nikita Popov 779fd39684 Reapply [InstCombine] Switch foldOpIntoPhi() to use InstSimplify
Relative to the previous attempt, this is rebased over the
InstSimplify fix in ac74e7a780,
which addresses the miscompile reported in PR58401.

-----

foldOpIntoPhi() currently only folds operations into the phi if all
but one operands constant-fold. The two exceptions to this are freeze
and select, where we allow more general simplification.

This patch makes foldOpIntoPhi() generally simplification based and
removes all the instruction-specific logic. We just try to simplify
the instruction for each operand, and for the (potentially) one
non-simplified operand, we move it into the new block with adjusted
operands.

This fixes https://github.com/llvm/llvm-project/issues/57448, which
was my original motivation for the change.

Differential Revision: https://reviews.llvm.org/D134954
2022-10-17 16:11:05 +02:00
Florian Hahn 699396131f
Revert "Reapply [InstCombine] Switch foldOpIntoPhi() to use InstSimplify"
This reverts commit 333246b48e.

It looks like this patch causes a mis-compile:
https://github.com/llvm/llvm-project/issues/58401

Fixes #58401.
2022-10-17 12:56:28 +01:00
Nikita Popov 333246b48e Reapply [InstCombine] Switch foldOpIntoPhi() to use InstSimplify
Relative to the previous attempt, this adjusts simplification to
use the correct context instruction: We need to use the terminator
of the incoming block, not the original instruction.

-----

foldOpIntoPhi() currently only folds operations into the phi if all
but one operands constant-fold. The two exceptions to this are freeze
and select, where we allow more general simplification.

This patch makes foldOpIntoPhi() generally simplification based and
removes all the instruction-specific logic. We just try to simplify
the instruction for each operand, and for the (potentially) one
non-simplified operand, we move it into the new block with adjusted
operands.

This fixes https://github.com/llvm/llvm-project/issues/57448, which
was my original motivation for the change.

Differential Revision: https://reviews.llvm.org/D134954
2022-10-07 11:04:19 +02:00
Alina Sbirlea b9898e7ed1 Revert "Reapply [InstCombine] Switch foldOpIntoPhi() to use InstSimplify"
This reverts commit e94619b955.
2022-10-06 13:12:24 -07:00
Nikita Popov e94619b955 Reapply [InstCombine] Switch foldOpIntoPhi() to use InstSimplify
The infinite loop seen on buildbots should be fixed by
11897708c0 (assuming there are not
multiple infinite combine loops...)

-----

foldOpIntoPhi() currently only folds operations into the phi if all
but one operands constant-fold. The two exceptions to this are freeze
and select, where we allow more general simplification.

This patch makes foldOpIntoPhi() generally simplification based and
removes all the instruction-specific logic. We just try to simplify
the instruction for each operand, and for the (potentially) one
non-simplified operand, we move it into the new block with adjusted
operands.

This fixes https://github.com/llvm/llvm-project/issues/57448, which
was my original motivation for the change.

Differential Revision: https://reviews.llvm.org/D134954
2022-10-05 14:00:19 +02:00
Gulfem Savrun Yeniceri d7592bbb03 Revert "Reapply [InstCombine] Switch foldOpIntoPhi() to use InstSimplify"
This reverts commit e1dd2cd063 because
the original commit b20e34b39f had
a dramatic increase in the build time of RTfuzzer, which caused Fuchsia
Clang toolchain builders to timeout:
https://luci-milo.appspot.com/ui/p/fuchsia/builders/toolchain.ci/clang-linux-x64/b8801248587754572961/overview
2022-10-04 20:57:34 +00:00
Nikita Popov e1dd2cd063 Reapply [InstCombine] Switch foldOpIntoPhi() to use InstSimplify
Reapply with a fix for the case where an operand simplified back
to the original phi: We need to map this case to the new phi node.

-----

foldOpIntoPhi() currently only folds operations into the phi if all
but one operands constant-fold. The two exceptions to this are freeze
and select, where we allow more general simplification.

This patch makes foldOpIntoPhi() generally simplification based and
removes all the instruction-specific logic. We just try to simplify
the instruction for each operand, and for the (potentially) one
non-simplified operand, we move it into the new block with adjusted
operands.

This fixes https://github.com/llvm/llvm-project/issues/57448, which
was my original motivation for the change.
2022-10-04 15:18:34 +02:00
Nikita Popov 0f32f0e147 Revert "[InstCombine] Switch foldOpIntoPhi() to use InstSimplify"
This reverts commit b20e34b39f.

This causes RAUW type mismatch assertions on some buildbots,
reverting for now.
2022-10-04 11:17:09 +02:00
Nikita Popov b20e34b39f [InstCombine] Switch foldOpIntoPhi() to use InstSimplify
foldOpIntoPhi() currently only folds operations into the phi if all
but one operands constant-fold. The two exceptions to this are freeze
and select, where we allow more general simplification.

This patch makes foldOpIntoPhi() generally simplification based and
removes all the instruction-specific logic. We just try to simplify
the instruction for each operand, and for the (potentially) one
non-simplified operand, we move it into the new block with adjusted
operands.

This fixes https://github.com/llvm/llvm-project/issues/57448, which
was my original motivation for the change.
2022-10-04 10:12:14 +02:00
Sebastian Neubauer c7750c522e Add helper func to get first non-alloca position
The LLVM performance tips suggest that allocas should be placed at the
beginning of the entry block. So far, llvm doesn’t provide any helper to
find that position.

Add BasicBlock::getFirstNonPHIOrDbgOrAlloca and IRBuilder::SetInsertPointPastAllocas(Function*)
that get an insert position after the (static) allocas at the start of a
function and use it in ShadowStackGCLowering.

Differential Revision: https://reviews.llvm.org/D132554
2022-09-09 15:39:53 +02:00
Chenbing Zheng 01cea7ac10 [InstCombine] extractvalue (any_mul_with_overflow X, 2^n), 0 -> X << n
Alive2: https://alive2.llvm.org/ce/z/JLmabt (umul)
        https://alive2.llvm.org/ce/z/J_ruXR  (smul)
        https://alive2.llvm.org/ce/z/o9SVSz (vector)

Reviewed By: spatel, RKSimon

Differential Revision: https://reviews.llvm.org/D133188
2022-09-08 11:12:55 +08:00
Sanjay Patel 7c57180900 [InstCombine] fold add+negate through select into sub
This transform came up as a potential DAGCombine in D133282,
so I wanted to see how it escaped in IR too.

We do general folds in InstCombiner::SimplifySelectsFeedingBinaryOp()
by checking if either arm of a select simplifies when the trailing
binop is threaded into the select.

So as long as one side simplifies, it's a good fold to combine a
negate and add into 1 subtract.

This is an example with a zero arm in the select:
https://alive2.llvm.org/ce/z/Hgu_Tj

And this models the tests with a cancelling 'not' op:
https://alive2.llvm.org/ce/z/BuzVV_

Differential Revision: https://reviews.llvm.org/D133369
2022-09-07 08:23:35 -04:00
Chenbing Zheng d30cf77cb1 [InstCombine] complete fold extractvalue (any_mul_with_overflow X, -1)
When we do extractvalue (any_mul_with_overflow X, -1) --> (-X and icmp),
which left partly failed to match vector constant with poison element.
This patch try to fix it.

Alive2: https://alive2.llvm.org/ce/z/2rGp_3

Reviewed By: spatel

Differential Revision: https://reviews.llvm.org/D132996
2022-09-02 10:58:42 +08:00
Nikita Popov 43e7d9af1d [InstCombine] Fold extractvalue of phi
Just as we do for most other operations, we should push
extractvalue instructions through phis, if this does not increase
unfolded instruction count.
2022-09-01 10:51:54 +02:00
Nikita Popov ad66bc42b0 [InstCombine] Use getInsertionPointAfterDef() in freeze fold
This simplifies the code and fixes handling of catchswitch, in
which case we have no insertion point for the freeze.

Originally part of D129660.
2022-08-31 11:32:57 +02:00
Kazu Hirata 56ea4f9bd3 [Transforms] Qualify auto in range-based for loops (NFC)
Identified with readability-qualified-auto.
2022-08-27 21:21:02 -07:00
Sanjay Patel 7c2f93c04a [InstCombine] use isa instead of dyn_cast for unused value; NFC 2022-08-24 17:58:20 -04:00
Sanjay Patel 0cfc651032 [InstCombine] ease use constraint in tryFactorization()
The stronger one-use checks prevented transforms like this:
(x * y) + x --> x * (y + 1)
(x * y) - x --> x * (y - 1)

https://alive2.llvm.org/ce/z/eMhvQa

This is one of the IR transforms suggested in issue #57255.

This should be better in IR because it removes a use of a
variable operand (we already fold the case with a constant
multiply operand).
The backend should be able to re-distribute the multiply if
that's better for the target.

Differential Revision: https://reviews.llvm.org/D132412
2022-08-24 12:10:54 -04:00
Sanjay Patel 4391351463 [InstCombine] improve readability in tryFactorization(); NFC
Added/removed braces, reduced indents, and renamed a variable.
2022-08-24 11:31:18 -04:00
Danila Malyutin 4a9ff289fb [InstCombine] Fix freeze instruction getting inserted before landingpad
The code would use first non-phi instruction as an insertion point, however
this could lead to freeze getting inserted between phi and landingpad
causing a verifier assert.

Differential Revision: https://reviews.llvm.org/D132105
2022-08-18 17:43:42 +03:00
Kazu Hirata 50724716cd [Transforms] Qualify auto in range-based for loops (NFC)
Identified with readability-qualified-auto.
2022-08-14 12:51:58 -07:00
Sanjay Patel 926e7312b2 [InstCombine] fold usub.with.overflow to icmp when there's no use of the math value
https://alive2.llvm.org/ce/z/UE48FH

This is part of solving issue #56926.
2022-08-09 13:13:48 -04:00
Sanjay Patel 6bfe5361b7 [InstCombine] add helper function for extract of with-overflow-intrinsic; NFC
We can do more with these patterns, so this block is going to grow.
2022-08-09 12:38:11 -04:00
Fangrui Song de9d80c1c5 [llvm] LLVM_FALLTHROUGH => [[fallthrough]]. NFC
With C++17 there is no Clang pedantic warning or MSVC C5051.
2022-08-08 11:24:15 -07:00
Nikita Popov 1f69503107 [MemoryBuiltins] Add getReallocatedOperand() function (NFC)
Replace the value-accepting isReallocLikeFn() overload with a
getReallocatedOperand() function, which returns which operand is
the one being reallocated. Currently, this is always the first one,
but once allockind(realloc) is respected, the reallocated operand
will be determined by the allocptr parameter attribute.
2022-07-21 14:54:16 +02:00
Nikita Popov 5e856a8578 [InstCombine] Use getFreedOperand() (NFC)
Use getFreedOperand() instead of isFreeCall() to remove the
implicit assumption that any pointer operand to a free function
is the operand being freed. This won't actually matter until we
handle allockind(free).
2022-07-21 14:33:55 +02:00
Nikita Popov c81dff3c30 [MemoryBuiltins] Add getFreedOperand() function (NFCI)
We currently assume in a number of places that free-like functions
free their first argument. This is true for all hardcoded free-like
functions, but with the new attribute-based design, the freed
argument is supposed to be indicated by the allocptr attribute.

To make sure we handle this correctly once allockind(free) is
respected, add a getFreedOperand() helper which returns the freed
argument, rather than just indicating whether the call frees *some*
argument.

This migrates most but not all users of isFreeCall() to the new
API. The remaining users are a bit more tricky.
2022-07-21 12:39:35 +02:00
Nikita Popov f45ab43332 [MemoryBuiltins] Avoid isAllocationFn() call before checking removable alloc
Alloc directly checking whether a given call is a removable
allocation, instead of first checking whether it is an allocation
first.
2022-07-21 09:39:19 +02:00
Nikita Popov 8a519b3c21 [InstCombine] Ensure constant folding in binop of select fold
When folding a binop into a select, we need to ensure that one
of the select arms actually does constant fold, otherwise we'll
create two binop instructions and perform the reverse transform.

Ensure this by performing an explicit constant folding attempt,
and failing the transform if neither side simplifies.

A simple alternative here would have been to limit the fold to
ImmConstants, but given the current representation of scalable
vector splats, this wouldn't be ideal.
2022-07-15 11:03:10 +02:00