llvm-project/llvm/test/Transforms
Bardia Mahjour a7e2c26939 [LV] Epilogue Vectorization with Optimal Control Flow (Recommit)
This is yet another attempt at providing support for epilogue
vectorization following discussions raised in RFC http://llvm.1065342.n5.nabble.com/llvm-dev-Proposal-RFC-Epilog-loop-vectorization-tt106322.html#none
and reviews D30247 and D88819.

Similar to D88819, this patch achieve epilogue vectorization by
executing a single vplan twice: once on the main loop and a second
time on the epilogue loop (using a different VF). However it's able
to handle more loops, and generates more optimal control flow for
cases where the trip count is too small to execute any code in vector
form.

Reviewed By: SjoerdMeijer

Differential Revision: https://reviews.llvm.org/D89566
2020-12-02 10:09:56 -05:00
..
ADCE
AddDiscriminators
AggressiveInstCombine Revert "[AggressiveInstCombine] Generalize foldGuardedRotateToFunnelShift to generic funnel shifts" 2020-11-04 08:39:32 +02:00
AlignmentFromAssumptions Reland [AssumeBundles] Use operand bundles to encode alignment assumptions 2020-09-12 15:36:06 +02:00
ArgumentPromotion OpaquePtr: Bulk update tests to use typed sret 2020-11-20 17:58:26 -05:00
AtomicExpand [AArch64] Out-of-line atomics (-moutline-atomics) implementation. 2020-11-20 13:30:12 +00:00
Attributor OpaquePtr: Update more tests to use typed sret 2020-11-20 20:08:43 -05:00
BDCE [DemandedBits][BDCE] Add support for min/max intrinsics 2020-09-10 22:13:31 +02:00
BlockExtractor [BlockExtract][NewPM] Port -extract-blocks to NPM 2020-10-21 12:51:11 -07:00
BranchFolding
CallSiteSplitting
CalledValuePropagation
CanonicalizeAliases [NPM] Move more O0 pass building into PassBuilder 2020-11-19 11:22:23 -08:00
CanonicalizeFreezeInLoops
CodeExtractor Revert "[IR] add fn attr for no_stack_protector; prevent inlining on mismatch" 2020-11-17 17:27:14 -08:00
CodeGenPrepare [CodeGenPrepare] Remove unused check-prefixes 2020-11-09 13:12:39 +00:00
ConstantHoisting [test] Fix const-addr-no-neg-offset.ll under NPM 2020-11-13 23:29:58 -08:00
ConstantMerge
ConstraintElimination [ConstraintElimination] Decompose GEP %ptr, ZEXT(SHL()). 2020-12-01 14:23:21 +00:00
Coroutines [coro] Async coroutines: Make sure we can handle control flow in suspend point dispatch function 2020-11-16 11:59:02 -08:00
CorrelatedValuePropagation Revert "Use uint64_t for branch weights instead of uint32_t" 2020-10-31 00:25:32 -07:00
CrossDSOCFI
DCE [DCE] Always get TargetLibraryInfo 2020-11-17 20:41:05 -08:00
DeadArgElim OpaquePtr: Bulk update tests to use typed sret 2020-11-20 17:58:26 -05:00
DeadStoreElimination [DSE] Precommit test case for PR48279. 2020-11-24 18:10:23 +00:00
DivRemPairs
EarlyCSE OpaquePtr: Bulk update tests to use typed sret 2020-11-20 17:58:26 -05:00
EliminateAvailableExternally
EntryExitInstrumenter
ExpandMemCmp
FixIrreducible [FixIrreducible][NewPM] Port -fix-irreducible to NPM 2020-10-09 09:22:09 -07:00
Float2Int
ForcedFunctionAttrs
FunctionAttrs [FunctionAttrs][NPM] Fix handling of convergent 2020-11-23 21:09:41 -08:00
FunctionImport [CGProfile] allows bitcast in metadata node storing function pointers 2020-11-13 09:28:21 -08:00
GCOVProfiling [gcov] Add spanning tree optimization 2020-09-13 00:07:31 -07:00
GVN OpaquePtr: Bulk update tests to use typed sret 2020-11-20 17:58:26 -05:00
GVNHoist [EarlyCSE] Verify hash code in regression tests 2020-09-04 10:40:35 -04:00
GVNSink [GVNSink][NewPM] Add GVNSinkPass to PassRegistry.def 2020-09-22 08:24:09 -07:00
GlobalDCE
GlobalMerge
GlobalOpt OpaquePtr: Bulk update tests to use typed sret 2020-11-20 17:58:26 -05:00
GlobalSplit
GuardWidening
HardwareLoops [ARM] Add a RegAllocHint for hinting t2DoLoopStart towards LR 2020-11-10 16:28:57 +00:00
HelloNew Reland [docs][NewPM] Add docs for writing NPM passes 2020-09-14 16:06:19 -07:00
HotColdSplit [test] HotColdSplit: cover use of opaque pointer type 2020-10-22 12:25:50 -07:00
IRCE [IRCE] Use the same min runtime iteration threshold for BPI and BFI checks 2020-11-16 09:21:50 +07:00
IndVarSimplify [Test] One more IndVars test 2020-12-02 13:16:34 +07:00
IndirectBrExpand
InferAddressSpaces [InferAddrSpace] Teach to handle assumed address space. 2020-11-16 17:06:33 -05:00
InferFunctionAttrs Make inferLibFuncAttributes() add SExt attribute on second arg to ldexp. 2020-11-10 18:32:15 +01:00
Inline [llvm][inliner] Reuse the inliner pass to implement 'always inliner' 2020-11-30 12:03:39 -08:00
InstCombine [InstCombine] canonicalize sign-bit-shift of difference to ext(icmp) 2020-12-01 09:58:11 -05:00
InstMerge OpaquePtr: Bulk update tests to use typed sret 2020-11-20 17:58:26 -05:00
InstNamer Port -instnamer to NPM 2020-10-22 12:08:36 -07:00
InstSimplify [InstSimplify] Add tests that fold instructions with poison operands (NFC) 2020-12-02 01:01:59 +09:00
InterleavedAccess [InterleavedAccess] Remove unused check-prefixes 2020-11-09 13:12:40 +00:00
Internalize
JumpThreading OpaquePtr: Bulk update tests to use typed sret 2020-11-20 17:58:26 -05:00
LCSSA
LICM Reland: Expand existing loopsink testing to also test loopsinking using new pass manager and fix LICM bug. 2020-11-20 10:26:33 -05:00
LoadStoreVectorizer Revert "Revert "[AMDGPU] Reorganize GCN subtarget features for unaligned access"" 2020-11-11 14:40:14 +00:00
LoopDataPrefetch
LoopDeletion Revert "[LoopDeletion] Allows deletion of possibly infinite side-effect free loops" 2020-11-07 00:32:12 -05:00
LoopDistribute [LoopVersioning] Form dedicated exits for versioned loop to preserve simplify form 2020-10-24 21:40:46 +08:00
LoopFlatten [LoopFlatten] Widen IV, support ZExt. 2020-11-23 08:57:19 +00:00
LoopFusion [SCEV] Fix nsw flags for GEP expressions 2020-11-13 18:19:32 +01:00
LoopIdiom [NFC][LoopIdiom] Left-shift-until-bittest: revisit test coverage 2020-11-18 21:22:27 +03:00
LoopInstSimplify
LoopInterchange [LoopInterchange] Skip non SCEV-able operands in cost function. 2020-11-08 11:41:19 +00:00
LoopLoadElim [LoopLoadElim] Make sure all loops are in simplify form. PR48150 2020-11-26 10:51:11 +07:00
LoopPredication
LoopReroll [SCEV] Strength nowrap flags after constant folding 2020-10-25 18:00:22 +01:00
LoopRotate [IR][LoopRotate] remove assertion that phi must have at least one operand 2020-11-30 11:32:42 -05:00
LoopSimplify [test] Fix tests using -analyze that fail under NPM 2020-10-21 21:54:30 -07:00
LoopSimplifyCFG [LoopSimplifyCFG][NewPM] Rename simplify-cfg -> loop-simplifycfg 2020-09-21 08:27:19 -07:00
LoopStrengthReduce Revert "[DebugInfo] Improve dbg preservation in LSR." 2020-11-27 08:52:32 +01:00
LoopTransformWarning
LoopUnroll Reland [SimplifyCFG] FoldBranchToCommonDest: lift use-restriction on bonus instructions 2020-11-27 12:47:15 +03:00
LoopUnrollAndJam [NPM] Share pass building options with legacy PM 2020-11-23 13:04:05 -08:00
LoopUnswitch OpaquePtr: Bulk update tests to use typed sret 2020-11-20 17:58:26 -05:00
LoopVectorize [LV] Epilogue Vectorization with Optimal Control Flow (Recommit) 2020-12-02 10:09:56 -05:00
LoopVersioning [LoopVersioning] Form dedicated exits for versioned loop to preserve simplify form 2020-10-24 21:40:46 +08:00
LoopVersioningLICM [NPM] Port -loop-versioning-licm to NPM 2020-10-24 21:51:18 +08:00
LowerAtomic
LowerConstantIntrinsics
LowerExpectIntrinsic Revert "clang-misexpect: Profile Guided Validation of Performance Annotations in LLVM" 2020-11-14 13:12:38 +03:00
LowerGuardIntrinsic
LowerInvoke
LowerMatrixIntrinsics [test] Fix multiply-minimal.ll 2020-11-19 18:16:35 -08:00
LowerSwitch [LowerSwitch][NewPM] Port lowerswitch to NPM 2020-09-15 18:18:31 -07:00
LowerTypeTests [AArch64] Add BTI to CFI jumptables. 2020-09-29 13:50:23 +02:00
LowerWidenableCondition
MakeGuardsExplicit
Mem2Reg
MemCpyOpt [MemCpyOpt] Port to MemorySSA 2020-12-01 17:57:41 +01:00
MergeFunc OpaquePtr: Bulk update tests to use typed sret 2020-11-20 17:58:26 -05:00
MergeICmps [MergeICmps] Fix missing split. 2020-12-01 16:50:55 +01:00
MetaRenamer OpaquePtr: Bulk update tests to use typed sret 2020-11-20 17:58:26 -05:00
NameAnonGlobals [NPM] Move more O0 pass building into PassBuilder 2020-11-19 11:22:23 -08:00
NaryReassociate [EarlyCSE] Verify hash code in regression tests 2020-09-04 10:40:35 -04:00
NewGVN OpaquePtr: Bulk update tests to use typed sret 2020-11-20 17:58:26 -05:00
ObjCARC OpaquePtr: Bulk update tests to use typed sret 2020-11-20 17:58:26 -05:00
OpenMP [test] Pin parallel_deletion_cg_update.ll to legacy PM 2020-11-24 10:23:37 -08:00
PGOProfile OpaquePtr: Bulk update tests to use typed sret 2020-11-20 17:58:26 -05:00
PartialInlining [PartialInliner]: Handle code regions in a switch stmt cases 2020-11-02 14:32:45 -05:00
PartiallyInlineLibCalls
PhaseOrdering [PassManager] Run Induction Variable Simplification pass *after* Recognize loop idioms pass, not before 2020-11-25 19:20:07 +03:00
PlaceSafepoints [PlaceSafepoints] Pin tests to legacy PM 2020-10-26 20:07:37 -07:00
PreISelIntrinsicLowering
PruneEH [PruneEH] Pin tests to legacy PM 2020-10-29 18:17:55 -07:00
Reassociate [Reassociate] Don't convert add-like-or's into add's if they appear to be part of load-combining idiom 2020-11-18 17:55:02 +03:00
Reg2Mem [Reg2Mem] add support for the new pass manager 2020-11-08 11:14:05 +00:00
RewriteStatepointsForGC GC-parseable element atomic memcpy/memmove 2020-10-23 14:06:09 -07:00
SCCP OpaquePtr: Bulk update tests to use typed byval 2020-11-20 14:00:46 -05:00
SLPVectorizer [AArch64][CostModel] Fix cost for mul <2 x i64> 2020-11-30 11:36:55 +00:00
SROA [ConstantFold] Fold more operations to poison 2020-11-29 21:19:48 +09:00
SafeStack OpaquePtr: Bulk update tests to use typed byval 2020-11-20 14:00:46 -05:00
SampleProfile [Remarks][2/2] Expand remarks hotness threshold option support in more tools 2020-11-30 21:55:50 -08:00
ScalarizeMaskedMemIntrin
Scalarizer [Scalarizer] Remove unused check-prefixes 2020-11-09 10:37:17 +00:00
SeparateConstOffsetFromGEP [NewPM] Port -separate-const-offset-from-gep 2020-11-09 17:42:36 -08:00
SimpleLoopUnswitch OpaquePtr: Bulk update tests to use typed sret 2020-11-20 17:58:26 -05:00
SimplifyCFG [SimplifyCFG] FoldBranchToCommonDest: don't require that cmp of br is last instruction 2020-12-01 15:13:06 +03:00
Sink Sink: Handle instruction sink when a user is dead 2020-10-09 16:20:26 -07:00
SpeculateAroundPHIs
SpeculativeExecution SpeculativeExecution: Allow speculating more instruction types 2020-11-18 17:00:19 +01:00
StraightLineStrengthReduce [NPM] Add remaining test of -slsr 2020-11-17 14:41:01 +08:00
StripDeadPrototypes
StripSymbols [NewPM] Port strip* passes to NPM 2020-09-15 18:25:12 -07:00
StructurizeCFG [StructurizeCFG][NewPM] Port -structurizecfg to NPM 2020-10-23 15:54:03 -07:00
SyntheticCountsPropagation
TailCallElim OpaquePtr: Bulk update tests to use typed sret 2020-11-20 17:58:26 -05:00
ThinLTOBitcodeWriter [test] Fix split-vfunc.ll under NPM 2020-11-19 14:59:05 -08:00
TypePromotion/ARM
UnifyFunctionExitNodes [NPM] Port -mergereturn to NPM 2020-10-20 10:33:58 -07:00
UnifyLoopExits [NPM] port -unify-loop-exits to NPM 2020-10-20 10:46:57 -07:00
UniqueInternalLinkageNames [NewPM] Add unique-internal-linkage-names to PassRegistry.def 2020-11-09 12:54:13 -08:00
Util Add pass to add !annotate metadata from @llvm.global.annotations. 2020-11-16 14:57:11 +00:00
VectorCombine [ConstantFold] Fold more operations to poison 2020-11-29 21:19:48 +09:00
WholeProgramDevirt [WholeProgramDevirt][NewPM] Add NPM testing path to match legacy pass 2020-09-30 17:27:37 -07:00