Commit Graph

374 Commits

Author SHA1 Message Date
chenglin.bi 6703290361 [InstCombine] fold `sub + and` pattern with specific const value
`C1 - ((C3 - X) & C2) --> (X & C2) + (C1 - (C2 & C3))`
when:
    (C3 - ((C2 & C3) - 1)) is pow2 &&
    ((C2 + C3) & ((C2 & C3) - 1)) == ((C2 & C3) - 1) &&
    C2 is negative pow2 || (C3 - X) is nuw

https://alive2.llvm.org/ce/z/HXQJV-

Fix: #58523

Reviewed By: spatel

Differential Revision: https://reviews.llvm.org/D136582
2022-11-05 12:58:45 +08:00
zhongyunde f58311796c [InstCombine] refactor the SimplifyUsingDistributiveLaws NFC
Precommit for D136015
Reviewed By: spatel
Differential Revision: https://reviews.llvm.org/D137019
2022-10-30 21:04:06 +08:00
zhongyunde 620cff096a [InstCombine] Fold series of instructions into mull for more types
Relax the constraint of wide/vectors types.
Address the comment https://reviews.llvm.org/D136015?id=469189#inline-1314520

Reviewed By: spatel, chfast
Differential Revision: https://reviews.llvm.org/D136661
2022-10-25 23:04:46 +08:00
zhongyunde 81713e893a [InstCombine] Fold series of instructions into mull
The following sequence should be folded into in0 * in1
      In0Lo = in0 & 0xffffffff; In0Hi = in0 >> 32;
      In1Lo = in1 & 0xffffffff; In1Hi = in1 >> 32;
      m01 = In1Hi * In0Lo; m10 = In1Lo * In0Hi; m00 = In1Lo * In0Lo;
      addc = m01 + m10;
      ResLo = m00 + (addc >> 32);

Reviewed By: spatel, RKSimon
Differential Revision: https://reviews.llvm.org/D136015
2022-10-25 01:09:37 +08:00
Sanjay Patel ee0bf64722 [InstCombine] try to fold mul by neg-power-of-2 to shl
`(A * -2**C) + B --> B - (A << C)`

https://alive2.llvm.org/ce/z/A6BWkf

This inverts what Negator was doing before:
D134310 / 0f32a5dea0

Analysis and codegen are generally better without multiply,
so we should favor this form even if we trade add for sub
(because those are generally equivalent cost operations).
2022-09-21 15:09:39 -04:00
Marc Auberer f52dd920d4 [InstCombine] Fix bug when folding x + (x | -x) to x & (x - 1)
Addresses concern: https://reviews.llvm.org/rG09cdddea0c4d284c2c22f5dfade40a60850c5ea7

There was a copy/paste mistake in the code. Updated code and test ref.

Differential Revision: https://reviews.llvm.org/D134135
2022-09-18 13:16:12 -04:00
Marc Auberer 09cdddea0c [InstCombine] Fold x + (x | -x) to x & (x - 1)
Fixes #57531

This transformation may be particularly useful on x86-64,
because x & (x - 1) can be performed by a single blsr instruction.

Differential Revision: https://reviews.llvm.org/D133362
2022-09-11 06:14:24 -04:00
Joe Loser 5e96cea1db [llvm] Use std::size instead of llvm::array_lengthof
LLVM contains a helpful function for getting the size of a C-style
array: `llvm::array_lengthof`. This is useful prior to C++17, but not as
helpful for C++17 or later: `std::size` already has support for C-style
arrays.

Change call sites to use `std::size` instead.

Differential Revision: https://reviews.llvm.org/D133429
2022-09-08 09:01:53 -06:00
Sanjay Patel ae117e1c1b [InstCombine] remove dead code for add (select cond, (sub), 0); NFC
This pattern is handled more generally in SimplifySelectsFeedingBinaryOp().
Tests to confirm that added to the add.ll test file in the previous commit.
2022-09-06 12:19:50 -04:00
Sanjay Patel 6c39a3aae1 [InstCombine] fold not-shift of signbit to icmp+zext
https://alive2.llvm.org/ce/z/j_8Wz9

The arithmetic shift was converted to logical shift with:
246078604c

That does not seem to uncover any other missing/conflicting folds,
so convert directly to signbit test + cast.

We still need to fold the pattern with logical shift to test + cast.

This allows reducing patterns where the output type is not
the same as the input value:
https://alive2.llvm.org/ce/z/nydwFV

Fixes #57394
2022-08-29 10:06:31 -04:00
Sanjay Patel 246078604c [InstCombine] fold inc-of-signbit-splat to not+lshr
(iN X s>> (N - 1)) + 1 --> (~X) u>> (N - 1)

https://alive2.llvm.org/ce/z/wzS474
2022-08-29 08:48:22 -04:00
Sanjay Patel f7ab70cf8d [InstCombine] reduce disguised mul+add factorization
~(A * C1) + A --> (A * (1 - C1)) - 1

This is a non-obvious mix of bitwise logic and math:
https://alive2.llvm.org/ce/z/U7ACVT

The pattern may be produced by Negator from the more typical
code seen in issue #57255.
2022-08-24 16:02:12 -04:00
Simon Pilgrim 2f217c1214 [InstCombine] Canonicalize ((X & -X) - 1) --> ((X - 1) & ~X) (PR51784)
Enables the ctpop((x & -x ) - 1) -> cttz(x, false) fold

Alive2: https://alive2.llvm.org/ce/z/EDk4h7 (((X & -X) - 1) --> (~X & (X - 1)) )

Alive2: https://alive2.llvm.org/ce/z/8Yr3XG (CTPOP -> CTTZ)

Fixes #51126

Differential Revision: https://reviews.llvm.org/D110488
2022-08-24 16:50:43 +01:00
Simon Pilgrim 80cc8f0f62 Revert rGc360955c4804e9b25017372cb4c6be7adcb216ce "[InstCombine] Canonicalize ((X & -X) - 1) --> (~X & (X - 1)) (PR51784)"
The test changes are failing on some buildbots (but not others.....).
2022-08-24 16:26:28 +01:00
Simon Pilgrim c360955c48 [InstCombine] Canonicalize ((X & -X) - 1) --> (~X & (X - 1)) (PR51784)
Enables the ctpop((x & -x ) - 1) -> cttz(x, false) fold

Alive2: https://alive2.llvm.org/ce/z/EDk4h7 (((X & -X) - 1) --> (~X & (X - 1)) )

Alive2: https://alive2.llvm.org/ce/z/8Yr3XG (CTPOP -> CTTZ)

Fixes #51126

Differential Revision: https://reviews.llvm.org/D110488
2022-08-24 15:31:15 +01:00
Jay Foad f82c55fa08 [InstCombine] Change order of canonicalization of ADD and AND
Canonicalize ((x + C1) & C2) --> ((x & C2) + C1) for suitable constants
C1 and C2, instead of the other way round. This should allow more
constant ADDs to be matched as part of addressing modes for loads and
stores.

Differential Revision: https://reviews.llvm.org/D130080
2022-08-22 20:03:53 +01:00
Sanjay Patel fa68d93d54 [InstCombine] fold reassociative fadd with negated operand
We manage to iteratively achieve this result with no extra
uses, and the reassociate pass can also do this, but this
pattern falls through the cracks in the example from
issue #57053.
2022-08-11 11:43:36 -04:00
Nikita Popov 5eaeeed8cb [InstCombine] Avoid ConstantExpr::getFNeg() calls (NFCI)
Instead call the constant folding API, which can fail. For now,
this should be NFC, as we still allow the creation of fneg
constant expressions.
2022-07-29 16:01:46 +02:00
Sanjay Patel 08091a99ae Revert "[InstCombine] enhance fold for subtract-from-constant -> xor"
This reverts commit 79bb915fb6.
This caused regressions because SCEV works better with sub.
2022-07-22 15:56:24 -04:00
Nikita Popov fc18a88231 [InstCombine] Avoid creating float binop ConstantExprs
Replace ConstantExpr:getFAdd etc with call to
ConstantFoldBinaryOpOperands(). I'm using the constant folding API
rather than IRBuilder here to ensure that this does actually
constant fold. These transforms don't use m_ImmConstant(), so this
would not otherwise be guaranteed (and apparently, they can't use
m_ImmConstant because they want to handle scalable vector splats).

There is an opportunity here to further migrate these to the
ConstantFoldFPInstOperands() API, which would respect the denormal
mode. I've held off on doing so here, because some of this code
explicitly checks for denormal results, and I don't want to touch
it in a mostly NFC change.
2022-07-08 16:36:04 +02:00
Sanjay Patel 79bb915fb6 [InstCombine] enhance fold for subtract-from-constant -> xor
A low-bit mask is not required:
https://alive2.llvm.org/ce/z/yPShss

This matches the SDAG implementation that was updated at:
8b75671314
2022-07-08 10:02:19 -04:00
Sanjay Patel 142aca7741 [InstCombine] fold sub of min/max of sub with common operand
x - max(x - y, 0) --> min(x, y)
  x - min(x - y, 0) --> max(x, y)

https://alive2.llvm.org/ce/z/2YkqFe

issue #55470
2022-07-04 18:55:24 -04:00
Sanjay Patel 4276d00b12 [InstCombine] add helper function for sub-of-min/max folds; NFC
The test diffs are cosmetic -- but improvements -- because we
let instcombine handle replacement. Instead of dropping the
old value name, it propagates to the new instruction.
2022-07-04 17:43:18 -04:00
Simon Moll b8c2781ff6 [NFC] format InstructionSimplify & lowerCaseFunctionNames
Clang-format InstructionSimplify and convert all "FunctionName"s to
"functionName".  This patch does touch a lot of files but gets done with
the cleanup of InstructionSimplify in one commit.

This is the alternative to the less invasive clang-format only patch: D126783

Reviewed By: spatel, rengolin

Differential Revision: https://reviews.llvm.org/D126889
2022-06-09 16:10:08 +02:00
Nikita Popov e6e0eb3bc8 [InstCombine] Strip bitcasts in GEP diff fold
Bitcasts were stripped in one case, but not the other. Of course,
this no longer really matters with opaque pointers, but as I went
through the trouble of tracking this down, we may as well remove
one typed vs opaque pointer optimization discrepancy.
2022-05-24 16:12:01 +02:00
Sanjay Patel be7f09f7b2 [IR] create and use helper functions that test the signbit; NFCI 2022-05-16 11:26:23 -04:00
Chenbing Zheng 2a0837aab1 [InstCombine] fix sub(add(X,Y),umin(Y,Z)) --> add(X,usub.sat(Y,Z))
This patch fix bug left in D124503. We should do
sub(add(X,Z),umin(Y,Z)) --> add(X,usub.sat(Z,Y)) instead of
sub(add(X,Z),umin(Y,Z)) --> add(X,usub.sat(Y,Z)).

Reviewed By: spatel

Differential Revision: https://reviews.llvm.org/D125352
2022-05-13 09:54:10 +08:00
Chenbing Zheng 394c683d40 [InstCombine] sub(add(X,Y),umin(Y,Z)) --> add(X,usub.sat(Y,Z))
Alive2: https://alive2.llvm.org/ce/z/2UNVbp

Reviewed By: RKSimon, spatel

Differential Revision: https://reviews.llvm.org/D124503
2022-05-07 17:17:48 +08:00
Simon Pilgrim ffe13960b5 [InstCombine] Fold (A & 2^C1) + A => A & (2^C1 - 1) iff bit C1 in A is a sign bit (PR21929)
Alive2: https://alive2.llvm.org/ce/z/Ygq26C

This is the final missing fold to handle the modulo2 simplification: https://github.com/llvm/llvm-project/issues/22303

Fixes #22303

Differential Revision: https://reviews.llvm.org/D123374
2022-04-22 16:59:02 +01:00
Simon Pilgrim 431e93f4f5 [InstCombine] Fold sub(add(x,y),min/max(x,y)) -> max/min(x,y) (PR38280)
As discussed on Issue #37628, we can flip a min/max node if we're subtracting from the sum of the node's operands

Alive2: https://alive2.llvm.org/ce/z/W_KXfy

Differential Revision: https://reviews.llvm.org/D123399
2022-04-11 11:32:56 +01:00
Dávid Bolvanský 4397504c2d [NFCI] Fix set-but-unused warning in InstCombineAddSub.cpp 2022-03-24 08:33:40 +01:00
Sanjay Patel 598721f866 [InstCombine] try harder to propagate 'nsz' through fneg-of-select
This can be viewed as swapping the select arms:
https://alive2.llvm.org/ce/z/jUvFMJ
...so we don't have the 'nsz' problem with the more general fold.

This unlocks other folds for the motivating fabs example.
This was discussed in issue #38828.
2022-03-15 11:05:29 -04:00
Nikita Popov d5ea3b2f33 [InstCombine] Remove sub of SPF min/max fold (NFCI)
This isn't necessary anymore, now that we canonicalize SPF min/max
to intrinsics. Might not be strictly NFC due to worklist order
changes.
2022-02-28 10:57:24 +01:00
Sanjay Patel 5379f76e63 [InstCombine] try harder to preserve 'nsz' in fneg-of-select transform
The corner case where 'nsz' needs to be removed is very narrow
as discussed here:
https://reviews.llvm.org/rG3cdd05e519dd

If the select condition is not undef, there's no problem with
propagating 'nsz':
https://alive2.llvm.org/ce/z/4GWJdq
2022-02-24 10:43:53 -05:00
Nikita Popov e2f627e5e3 [InstCombine] Fold sub of umin to usub.sat
We were handling sub of umax, but not the conjugated umin case.

https://alive2.llvm.org/ce/z/4fdZfy
https://alive2.llvm.org/ce/z/BhUQBM
2022-02-23 12:00:34 +01:00
Benjamin Kramer f15014ff54 Revert "Rename llvm::array_lengthof into llvm::size to match std::size from C++17"
This reverts commit ef82063207.

- It conflicts with the existing llvm::size in STLExtras, which will now
  never be called.
- Calling it without llvm:: breaks C++17 compat
2022-01-26 16:55:53 +01:00
serge-sans-paille ef82063207 Rename llvm::array_lengthof into llvm::size to match std::size from C++17
As a conquence move llvm::array_lengthof from STLExtras.h to
STLForwardCompat.h (which is included by STLExtras.h so no build
breakage expected).
2022-01-26 16:17:45 +01:00
Sanjay Patel 39e602b6c4 [InstCombine] try to fold binop with phi operands
This is an alternate version of D115914 that handles/tests all binary opcodes.

I suspect that we don't see these patterns too often because -simplifycfg
would convert the minimal cases into selects rather than leave them in phi form
(note: instcombine has logic holes for combining the select patterns too though,
so that's another potential patch).

We only create a new binop in a predecessor that unconditionally branches to
the final block.
https://alive2.llvm.org/ce/z/C57M2F
https://alive2.llvm.org/ce/z/WHwAoU (not safe to speculate an sdiv for example)
https://alive2.llvm.org/ce/z/rdVUvW (but it is ok on this path)

Differential Revision: https://reviews.llvm.org/D117110
2022-01-22 15:00:06 -05:00
Sanjay Patel 2d50630efb [InstCombine] reduce code duplication; NFC 2022-01-18 12:13:45 -05:00
Daniil Kovalev d8e0e125a2 [InstCombine] Simplify addends reordering logic
Previously some constants were not pushed to the top of the resulting
expression tree as intended by the algorithm. We can remove the logic
from simplifyFAdd and rely on SimplifyAssociativeOrCommutative to do
that.

Differential Revision: https://reviews.llvm.org/D117302
2022-01-18 16:00:47 +03:00
Sanjay Patel 4cdf30d9d3 [InstCombine] FP with reassoc FMF: (X * C) + X --> X * (MulC + 1.0)
This fold already exists for scalars via FAddCombine (and that's
why 2 of the tests are only changed cosmetically), but that code
misses vectors and has largely been replaced by simpler folds
over time, so this is another step towards removing it.
2022-01-17 10:38:05 -05:00
Sanjay Patel d5c002bdc7 [InstCombine] fix code comment to match code; NFC 2021-11-09 14:27:29 -05:00
Sanjay Patel 2a88d00cf2 [InstCombine] fold sub-of-umax to 0-usubsat
Op0 - umax(X, Op0) --> 0 - usub.sat(X, Op1)

I'm not sure if this is really an improvement in IR because
we probably have better recognition/analysis for min/max,
but this lines up with the fold we do for the icmp+select
idiom and removes another diff from D98152.

This is similar to the previous fold in the code that was
added with:
83c2fb9f66
baa6a85130

https://alive2.llvm.org/ce/z/5MrVB9
2021-11-09 12:46:03 -05:00
Sanjay Patel baa6a85130 [InstCombine] allow commute in sub-of-umax fold
This fold was added with:
83c2fb9f66
...but missed the commuted pattern:
https://alive2.llvm.org/ce/z/_tYEGy
2021-11-09 10:50:11 -05:00
Sanjay Patel 83c2fb9f66 [InstCombine] match usub.sat from umax intrinsic
umax(X, Op1) - Op1 --> usub.sat(X, Op1)

https://alive2.llvm.org/ce/z/HpcGiJ

This happens in 2 or more steps with an icmp-select idiom
instead of an intrinsic. This is another step towards
canonicalization of the min/max intrinsics. See:
D98152
2021-11-06 08:32:52 -04:00
Jay Foad a9bceb2b05 [APInt] Stop using soft-deprecated constructors and methods in llvm. NFC.
Stop using APInt constructors and methods that were soft-deprecated in
D109483. This fixes all the uses I found in llvm, except for the APInt
unit tests which should still test the deprecated methods.

Differential Revision: https://reviews.llvm.org/D110807
2021-10-04 08:57:44 +01:00
Dávid Bolvanský 5f2f611880 Fixed more warnings in LLVM produced by -Wbitwise-instead-of-logical 2021-10-03 13:58:10 +02:00
Sanjay Patel 75e8eb2b10 [InstCombine] update code/test comments; NFC
Follow-up for post-commit suggestion on:
28afaed691

The comments were partly copied from the original
code, but not updated to match the new code.
2021-09-11 10:53:53 -04:00
Sanjay Patel 28afaed691 [InstCombine] fold sub of min/max intrinsics with invertible ops
This is a translation of the existing code to handle the intrinsics
and another step towards D98152.

https://alive2.llvm.org/ce/z/jA7eBC

This pattern is already handled by underlying folds if there are
less uses, so the minimal tests in this case have extra uses.

The larger cmyk tests show the motivation - when combined with
other folds, we invert a larger sequence and eliminate 'not' ops.
2021-09-11 09:18:46 -04:00
Sanjay Patel cc9c545fb4 [InstCombine] generalize subtract with 'not' operands; 2nd try
This is a re-try of 3aa009cc87 which was reverted at
9577fac0fd because it caused an infinite loop.

For the extra test case, either re-ordering the transforms
or adding the extra clause to avoid sub-of-sub is enough
to prevent the infinite compile, but I'm doing both to be
safer.

Original commit message:
The motivation was to get min/max intrinsics to parity
with cmp+select idioms, but this unlocks a few more
folds because isFreeToInvert recognizes add/sub with
constants too.

In the min/max example, we have too many extra uses
for smaller folds to improve things, but this fold
is able to eliminate uses even though we can't reduce
the number of instructions.
2021-08-23 17:06:51 -04:00