Commit Graph

8863 Commits

Author SHA1 Message Date
Florian Hahn 1da30c659d [LoopInterchange] Use getExitBlock()/getExitingBlock instead of manual impl.
This also means we have to check if the latch is the exiting block now,
as `transform` expects the latches to be the exiting blocks too.

https://bugs.llvm.org/show_bug.cgi?id=36586

Reviewers: efriedma, davide, karthikthecool

Reviewed By: efriedma

Differential Revision: https://reviews.llvm.org/D45279

llvm-svn: 330806
2018-04-25 09:35:54 +00:00
Bjorn Pettersson bec2a7c4eb [DebugInfo] Invalidate debug info in ReassociatePass::RewriteExprTree
Summary:
When Reassociate is rewriting an expression tree it may
reuse old binary expression nodes, for new expressions.
Whenever an expression node is reused, but with a non-trivial
change in the result, we need to invalidate any debug info
that is associated with the node.

If for example rewriting
  x = mul a, b
  y = mul c, x
into
  x = mul c, b
  y = mul a, x
we still get the same result for 'y', but 'x' is a new expression.
All debug info referring to 'x' must be invalidated (marked as
optimized out) since we no longer calculate the expected value.

As a side-effect this patch avoid (at least some) problems where
reassociate could end up creating IR with debug-use before def.
Earlier the dbg.value nodes where left untouched in the IR, while
the reused binary nodes where sinked to just before the root node
of the rewritten expression tree. See PR27273 for more info about
such problems.

Reviewers: dblaikie, aprantl, dexonsmith

Reviewed By: aprantl

Subscribers: JDevlieghere, llvm-commits

Tags: #debug-info

Differential Revision: https://reviews.llvm.org/D45975

llvm-svn: 330804
2018-04-25 09:23:56 +00:00
Geoff Berry 2af5f3c1e5 [DivRemPairs] Fix non-determinism in use list order.
Summary:
Use a MapVector instead of a DenseMap for RemMap since it is iteratated
over and the order of iteration can effect the order that new
instructions are created.  This can in turn effect the use list order of
div/rem input values if multiple new instructions are created that share
any input values.

Reviewers: spatel

Subscribers: mcrosier, llvm-commits

Differential Revision: https://reviews.llvm.org/D45858

llvm-svn: 330792
2018-04-25 02:17:56 +00:00
Chandler Carruth 69e68f8468 [PM/LoopUnswitch] Begin teaching SimpleLoopUnswitch to use the new
update API for dominators rather than doing manual, hacky updates.

This is just the first step, but in some ways the most important as it
moves the non-trivial unswitching to update the domtree rather than
fully recalculating it each time.

Subsequent patches should remove the custom update logic used by the
trivial unswitch and replace it with uses of the update API.

This also fixes a number of bugs I was seeing when testing non-trivial
unswitch due to it querying the quasi-correct dominator tree. Now the
tree is 100% correct and safe to query. That said, there are still more
bugs I can see with non-trivial unswitch just running over the test
suite, so more bugfix patches are needed as well.

Thanks to both Sanjoy and Fedor for reviews and testing!

Differential Revision: https://reviews.llvm.org/D45943

llvm-svn: 330787
2018-04-25 00:18:07 +00:00
Florian Hahn ceee788947 [LoopInterchange] Make isProfitableForVectorization slightly more conservative.
After D43236, we started interchanging loops with empty dependence
matrices.  In isProfitableForVectorization, we try to determine if
interchanging makes the loop dependences more friendly to the
vectorizer. If there are no dependences, we should not interchange,
based on that heuristic.

Reviewers: efriedma, mcrosier, karthikthecool, blitz.opensource

Reviewed By: mcrosier

Differential Revision: https://reviews.llvm.org/D45208

llvm-svn: 330738
2018-04-24 16:55:32 +00:00
David Blaikie ba47dd16c5 Fix some layering in AggressiveInstCombine (avoiding inclusion of Scalar.h)
llvm-svn: 330726
2018-04-24 15:40:07 +00:00
Chandler Carruth 43acdb35bc [PM/LoopUnswitch] Fix a bug in the loop block set formation of the new
loop unswitch.

This code incorrectly added the header to the loop block set early. As
a consequence we would incorrectly conclude that a nested loop body had
already been visited when the header of the outer loop was the preheader
of the nested loop. In retrospect, adding the header eagerly doesn't
really make sense. It seems nicer to let the cycle be formed naturally.
This will catch crazy bugs in the CFG reconstruction where we can't
correctly form the cycle earlier rather than later, and makes the rest
of the logic just fall out.

I've also added various asserts that make these issues *much* easier to
debug.

llvm-svn: 330707
2018-04-24 10:33:08 +00:00
Chandler Carruth 0ace148ca6 [PM/LoopUnswitch] Remove another over-aggressive assert.
This code path can very clearly be called in a context where we have
baselined all the cloned blocks to a particular loop and are trying to
handle nested subloops. There is no harm in this, so just relax the
assert. I've added a test case that will make sure we actually exercise
this code path.

llvm-svn: 330680
2018-04-24 03:27:00 +00:00
David Blaikie a27771b62f InstCombine: Fix layering by not including Scalar.h in InstCombine
(notionally Scalar.h is part of libLLVMScalarOpts, so it shouldn't be
included by InstCombine which doesn't/shouldn't need to depend on
ScalarOpts)

llvm-svn: 330669
2018-04-24 00:48:59 +00:00
Craig Topper 1bcb258ba3 [AggressiveInstCombine] Add aggressive inst combiner to the LLVM C API.
I just tried to copy what was done for regular InstCombine. Hopefully I didn't miss anything.

llvm-svn: 330668
2018-04-24 00:39:29 +00:00
Florian Hahn 7441818560 [LoopInterchange] Do not change LI for BBs in child loops.
If a loop with child loops becomes our new inner loop after
interchanging, we only need to update LoopInfo for the blocks defined in
the old outer loop. BBs in child loops will stay there.

Reviewers: efriedma, karthikthecool, mcrosier

Reviewed By: efriedma

Differential Revision: https://reviews.llvm.org/D45970

llvm-svn: 330653
2018-04-23 21:38:19 +00:00
Xin Tong 8edff27923 [CallSiteSplit] Make sure we remove nonnull if the parameter turns out to be a constant.
Summary: We do not need nonull attribute if we know an argument is going to be constant.

Reviewers: junbuml, davide, fhahn

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D45608

llvm-svn: 330641
2018-04-23 20:09:08 +00:00
Bjorn Pettersson 8e484dc531 [MemCpyOpt] Skip optimizing basic blocks not reachable from entry
Summary:
Skip basic blocks not reachable from the entry node
in MemCpyOptPass::iterateOnFunction.

Code that is unreachable may have properties that do not exist
for reachable code (an instruction in a basic block can for
example be dominated by a later instruction in the same basic
block, for example if there is a single block loop).
MemCpyOptPass::processStore is only safe to use for reachable
basic blocks, since it may iterate past the basic block
beginning when used for unreachable blocks. By simply skipping
to optimize unreachable basic blocks we can avoid asserts such
as "Assertion `!NodePtr->isKnownSentinel()' failed."
in MemCpyOptPass::processStore.

The problem was detected by fuzz tests.

Reviewers: eli.friedman, dneilson, efriedma

Reviewed By: efriedma

Subscribers: efriedma, llvm-commits

Differential Revision: https://reviews.llvm.org/D45889

llvm-svn: 330635
2018-04-23 19:55:04 +00:00
Daniel Neilson cc45e923c5 [DSE] Teach the pass that atomic memory intrinsics are stores.
Summary:
This change teaches DSE that the atomic memory intrinsics are stores
that can be eliminated, and can allow other stores to be eliminated.
This change specifically does not teach DSE that these intrinsics
can be partially eliminated (i.e. length reduced, and dest/src changed);
that will be handled in another change.

Reviewers: mkazantsev, skatkov, apilipenko, efriedma, rsmith

Reviewed By: efriedma

Subscribers: dmgreen, llvm-commits

Differential Revision: https://reviews.llvm.org/D45535

llvm-svn: 330629
2018-04-23 19:06:49 +00:00
Chandler Carruth bf7190a154 [PM/LoopUnswitch] Remove a buggy assert in the new loop unswitch.
The condition this was asserting doesn't actually hold. I've added
comments to explain why, removed the assert, and added a fun test case
reduced out of 403.gcc.

llvm-svn: 330564
2018-04-23 06:58:36 +00:00
Chandler Carruth b525424118 [PM/LoopUnswitch] Fix comment typo. NFC.
llvm-svn: 330560
2018-04-23 00:48:42 +00:00
Michael Zolotukhin e268304122 Revert r330431.
There are still stage3/stage4 miscompares :(

llvm-svn: 330446
2018-04-20 16:57:10 +00:00
Florian Hahn 773872fd67 [NewGVN] Split OpPHI detection and creation.
It also adds a check making sure PHIs for operands are all in the same
block.

Patch by Daniel Berlin <dberlin@dberlin.org>

Reviewers: dberlin, davide

Differential Revision: https://reviews.llvm.org/D43865

llvm-svn: 330444
2018-04-20 16:37:13 +00:00
Michael Zolotukhin a2c9af0209 Revert "Revert r330403 and r330413."
Reapply the patches with a fix. Thanks Ilya and Hans for the reproducer!
This reverts commit r330416.

The issue was that removing predecessors invalidated uses that we stored
for rewrite. The fix is to finish manipulating with CFG before we select
uses for rewrite.

llvm-svn: 330431
2018-04-20 13:34:32 +00:00
Ilya Biryukov afe822bd6d Revert r330403 and r330413.
Revert r330413: "[SSAUpdaterBulk] Use SmallVector instead of DenseMap for storing rewrites."
Revert r330403 "Reapply "[PR16756] Use SSAUpdaterBulk in JumpThreading." one more time."

r330403 commit seems to crash clang during our integrate while doing PGO build with the following stacktrace:
      #2 llvm::SSAUpdaterBulk::RewriteAllUses(llvm::DominatorTree*, llvm::SmallVectorImpl<llvm::PHINode*>*)
      #3 llvm::JumpThreadingPass::ThreadEdge(llvm::BasicBlock*, llvm::SmallVectorImpl<llvm::BasicBlock*> const&, llvm::BasicBlock*)
      #4 llvm::JumpThreadingPass::ProcessThreadableEdges(llvm::Value*, llvm::BasicBlock*, llvm::jumpthreading::ConstantPreference, llvm::Instruction*)
      #5 llvm::JumpThreadingPass::ProcessBlock(llvm::BasicBlock*)
The crash happens while compiling 'lib/Analysis/CallGraph.cpp'.

r3340413 is reverted due to conflicting changes.

llvm-svn: 330416
2018-04-20 10:52:54 +00:00
Michael Zolotukhin 9dea079315 [SSAUpdaterBulk] Use SmallVector instead of DenseMap for storing rewrites.
llvm-svn: 330413
2018-04-20 10:31:06 +00:00
Michael Zolotukhin 79e4f7fadb Reapply "[PR16756] Use SSAUpdaterBulk in JumpThreading." one more time.
Hopefully, changing set to vector removes nondeterminism detected by
some bots, or the new assert will catch something.

This reverts commit r330180.

llvm-svn: 330403
2018-04-20 08:01:08 +00:00
Jin Lin 585f2699cf Refine the loop rotation's API
Summary:
The following changes addresses the following two issues.

1) The existing loop rotation pass contains both loop latch simplification and loop rotation. So one flag RotationOnly is added to be passed to the loop rotation pass.
2) The threshold value is initialized with MAX_UINT since the loop rotation utility should not have threshold limit.

Reviewers: dmgreen, efriedma

Reviewed By: efriedma

Differential Revision: https://reviews.llvm.org/D45582

llvm-svn: 330362
2018-04-19 20:29:43 +00:00
Chandler Carruth 32e62f9c5b [PM/LoopUnswitch] Detect irreducible control flow within loops and skip unswitching non-trivial edges.
Summary:
This fixes the bug pointed out in review with non-trivial unswitching.

This also provides a basis that should make it pretty easy to finish
fleshing out a routine to scan an entire function body for irreducible
control flow, but this patch remains minimal for disabling loop
unswitch.

Reviewers: sanjoy, fedor.sergeev

Subscribers: mcrosier, hiraditya, llvm-commits

Differential Revision: https://reviews.llvm.org/D45754

llvm-svn: 330357
2018-04-19 18:44:25 +00:00
Sanjay Patel a201787fd7 [Reassociate] fix formatting; NFC
llvm-svn: 330348
2018-04-19 17:56:36 +00:00
Florian Hahn b789165e6b [NewGVN] Add ops as dependency if we cannot find a leader for ValueOp.
If those operands change, we might find a leader for ValueOp, which
could enable new phi-of-op creation.

This fixes a case where we missed creating a phi-of-ops node. With D43865
and this patch, bootstrapping clang/llvm works with -enable-newgvn, whereas
without it, the "value changed after iteration" assertion is triggered.

Reviewers: dberlin, davide

Reviewed By: dberlin

Differential Revision: https://reviews.llvm.org/D42180

llvm-svn: 330334
2018-04-19 15:05:47 +00:00
Sam Parker 3c19051bf0 [IRCE] Only check for NSW on equality predicates
After investigation discussed in D45439, it would seem that the nsw
flag restriction is unnecessary in most cases. So the IsInductionVar
lambda has been removed, the functionality extracted, and now only
require nsw when using eq/ne predicates.

Differential Revision: https://reviews.llvm.org/D45617

llvm-svn: 330256
2018-04-18 13:50:28 +00:00
Michael Zolotukhin 21458fdc55 Revert "Reapply "[PR16756] Use SSAUpdaterBulk in JumpThreading." again."
This reverts r330175. There are still stage3/stage4 miscompares.

llvm-svn: 330180
2018-04-17 07:31:27 +00:00
Michael Zolotukhin 3f5fd1b129 Reapply "[PR16756] Use SSAUpdaterBulk in JumpThreading." again.
One more, hopefully the last, bug is fixed: when forming UsesToRewrite
we should ignore phi operands coming from edges that we want to delete.

This reverts r329910.

llvm-svn: 330175
2018-04-17 04:45:22 +00:00
Roman Lebedev 25cbb62d18 [NFC] ConstantOffsetExtractor::CanTraceInto(): add FIXME: no tests
As suggested in https://reviews.llvm.org/D45631#1068338,
looking at haveNoCommonBitsSet() users, and *trying* to
show the change effect elsewhere.

llvm-svn: 330100
2018-04-15 18:59:27 +00:00
Hiroshi Inoue ae17900997 [NFC] fix trivial typos in document and comments
"not not" -> "not" etc

llvm-svn: 330083
2018-04-14 08:59:00 +00:00
Mandeep Singh Grang 636d94db3b [Transforms] Change std::sort to llvm::sort in response to r327219
Summary:
r327219 added wrappers to std::sort which randomly shuffle the container before sorting.
This will help in uncovering non-determinism caused due to undefined sorting
order of objects having the same key.

To make use of that infrastructure we need to invoke llvm::sort instead of std::sort.

Note: This patch is one of a series of patches to replace *all* std::sort to llvm::sort.
Refer the comments section in D44363 for a list of all the required patches.

Reviewers: kcc, pcc, danielcdh, jmolloy, sanjoy, dberlin, ruiu

Reviewed By: ruiu

Subscribers: ruiu, llvm-commits

Differential Revision: https://reviews.llvm.org/D45142

llvm-svn: 330059
2018-04-13 19:47:57 +00:00
Xin Tong d83c883d29 [CallSiteSplit] Fix comment. NFC
llvm-svn: 329987
2018-04-13 04:35:38 +00:00
Benjamin Kramer b4ba3988bb Revert "Reapply "[PR16756] Use SSAUpdaterBulk in JumpThreading." one more time."
This reverts commit r329865. Causes stage2/stage3 miscompare.

llvm-svn: 329910
2018-04-12 13:52:02 +00:00
Sam Parker 9737535943 [IRCE] isKnownNonNegative helper function
Created a helper function to query for non negative SCEVs. Uses the
SGE predicate to catch constants that could be interpreted as
negative.

Differential Revision: https://reviews.llvm.org/D45481

llvm-svn: 329907
2018-04-12 12:49:40 +00:00
Hiroshi Inoue bcadfee2ad [NFC] fix trivial typos in documents and comments
"is is" -> "is", "if if" -> "if", "or or" -> "or"

llvm-svn: 329878
2018-04-12 05:53:20 +00:00
Michael Zolotukhin 815f453f76 Reapply "[PR16756] Use SSAUpdaterBulk in JumpThreading." one more time.
This reapplies commit r329644.

llvm-svn: 329865
2018-04-11 23:37:53 +00:00
Sanjay Patel 3b6d46761f [CVP] simplify phi with constant incoming values that match common variable edge values
This is based on an example that was recently posted on llvm-dev:

void *propagate_null(void* b, int* g) {
  if (!b) {
    return 0;
  }
  (*g)++;
  return b;
}

https://godbolt.org/g/xYk3qG

The original code or constant propagation in other passes has obscured the fact 
that the phi can be removed completely.

Differential Revision: https://reviews.llvm.org/D45448

llvm-svn: 329755
2018-04-10 20:42:39 +00:00
Michael Zolotukhin d6beefd5d3 Revert "[PR16756] Use SSAUpdaterBulk in JumpThreading." one more time.
This reverts r329661. Bots are still unhappy.

llvm-svn: 329666
2018-04-10 03:40:29 +00:00
Michael Zolotukhin 8a13f6d4a7 Revert "Revert "[PR16756] Use SSAUpdaterBulk in JumpThreading.""
This reapplies commit r329644.

llvm-svn: 329661
2018-04-10 02:16:45 +00:00
Michael Zolotukhin 0274632ee6 Revert "[PR16756] Use SSAUpdaterBulk in JumpThreading."
This reverts commit r329644.

llvm-svn: 329650
2018-04-10 00:42:43 +00:00
Michael Zolotukhin c6d2d65f37 [PR16756] Use SSAUpdaterBulk in JumpThreading.
Summary:
SSAUpdater is a bottleneck in JumpThreading, and this patch improves the
situation by using SSAUpdaterBulk instead.

Compile time impact: no noticable changes on CTMark, a big improvement
on the test from PR16756.

Reviewers: dberlin, davide, MatzeB

Subscribers: llvm-commits, hiraditya

Differential Revision: https://reviews.llvm.org/D44282

llvm-svn: 329644
2018-04-09 23:37:37 +00:00
Xin Tong fdad23bc36 [MergeICmp] Update debug msg.NFC
llvm-svn: 329572
2018-04-09 14:29:13 +00:00
Xin Tong 0efadbbcde [MergeICmp] Split blocks that do other work.
Summary:
We do not try to move the instructions and split the block till we
know the blocks can be split, i.e. BCE-cmp-insts can be separated from
non-BCE-cmp-insts.

Reviewers: davide, courbet

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D44443

llvm-svn: 329564
2018-04-09 13:14:06 +00:00
Max Kazantsev 8624a4786a [IRCE] Relax restriction on collected range checks
In IRCE, we have a very old legacy check that works when we collect comparisons that we
treat as range checks. It ensures that the value against which the indvar is compared is
loop invariant and is also positive.

This latter condition remained there since the times when IRCE was only able to handle
signed latch comparison. As the optimization evolved, it now learned how to intersect
signed or unsigned ranges, and this logic has no reliance on the fact that the right border
of each range should be positive.

The old implementation of this non-negativity check was also naive enough and just looked
into ranges (while most of other IRCE logic tries to use power of SCEV implications), so this
check did not allow to deal with the most simple case that looks like follows:

  int size; // not known non-negative
  int length; //known non-negative;
  i = 0;
  if (size != 0) {
    do {
      range_check(i < size);
      range_check(i < length);
    ++i;
    } while (i < size)
  }

In this case, even if from some dominating conditions IRCE could parse loop
structure, it could only remove the range check against `length` and simply
ignored the check against `size`.

In this patch we remove this obsolete check. It will allow IRCE to pick comparison
against `size` as a potential range check and then let Range Intersection logic
decide whether it is OK to eliminate it or not.

Differential Revision: https://reviews.llvm.org/D45362
Reviewed By: samparker

llvm-svn: 329547
2018-04-09 06:01:22 +00:00
Hiroshi Inoue 9ff2380ea6 [NFC] fix trivial typos in comments and error message
"is is" -> "is", "are are" -> "are"

llvm-svn: 329546
2018-04-09 04:37:53 +00:00
Xin Tong 99c4e2f364 [LIR] Reorder header. NFC
llvm-svn: 329530
2018-04-08 13:19:53 +00:00
Geoff Berry 5bf4a5eafa [EarlyCSE] Add debug counter for debugging mis-optimizations. NFC.
Reviewers: reames, spatel, davide, dberlin

Subscribers: mcrosier, llvm-commits

Differential Revision: https://reviews.llvm.org/D45162

llvm-svn: 329443
2018-04-06 18:47:33 +00:00
Max Kazantsev 832563a782 [NFC] Add missing end of line symbols
llvm-svn: 329383
2018-04-06 09:47:06 +00:00
Florian Hahn 6e0043365b [LoopInterchange] Add stats counter for number of interchanged loops.
Reviewers: samparker, karthikthecool, blitz.opensource

Reviewed By: samparker

Differential Revision: https://reviews.llvm.org/D45209

llvm-svn: 329269
2018-04-05 10:39:23 +00:00