llvm-project/llvm/test/Transforms/LoopVectorize/AArch64
David Sherwood a08c7736a7 [LoopVectorize] Add support for scalable vectorization of induction variables
This patch adds support for the vectorization of induction variables when
using scalable vectors, which required the following changes:

1. Removed assert from InnerLoopVectorizer::getStepVector.
2. Modified InnerLoopVectorizer::createVectorIntOrFpInductionPHI to use
   a runtime determined value for VF and removed an assert.
3. Modified InnerLoopVectorizer::buildScalarSteps to work for scalable
   vectors. I did this by calculating the full vector value for each Part
   of the unroll factor (UF) and caching this in the VP state. This means
   that we are always able to extract an arbitrary element from the vector
   if necessary. In addition to this, I also permitted the caching of the
   individual lane values themselves for the known minimum number of elements
   in the same way we do for fixed width vectors. This is a further
   optimisation that improves the code quality since it avoids unnecessary
   extractelement operations when extracting the first lane.
4. Added an assert to InnerLoopVectorizer::widenPHIInstruction, since while
   testing some code paths I noticed this is currently broken for scalable
   vectors.

Various tests to support different cases have been added here:

  Transforms/LoopVectorize/AArch64/sve-inductions.ll

Differential Revision: https://reviews.llvm.org/D98715
2021-03-30 11:13:31 +01:00
..
Oz-and-forced-vectorize.ll [LoopRotation] Allow loop header duplication if vectorization is forced. 2020-10-27 09:28:01 +00:00
aarch64-predication.ll [SLP,LV] Use poison constant vector for shufflevector/initial insertelement 2021-01-06 11:22:50 +09:00
aarch64-unroll.ll
arbitrary-induction-step.ll [IR] Let IRBuilder's CreateVectorSplat/CreateShuffleVector use poison as placeholder 2020-12-30 04:21:04 +09:00
arm64-unroll.ll
backedge-overflow.ll
deterministic-type-shrinkage.ll
extend-vectorization-factor-for-unprofitable-memops.ll [LV] Add tests showing suboptimal vectorization for narrow types. 2021-02-11 17:24:28 +00:00
extractvalue-no-scalarization-required.ll [SLP,LV] Use poison constant vector for shufflevector/initial insertelement 2021-01-06 11:22:50 +09:00
gather-cost.ll
induction-trunc.ll
interleaved-vs-scalar.ll
interleaved_cost.ll
intrinsiccost.ll [CostModel] Remove VF from IntrinsicCostAttributes 2021-02-23 13:03:26 +00:00
lit.local.cfg
loop-vectorization-factors.ll
loopvectorize_pr33804_double.ll [AArch64][LV] Move vectorizer test to Transforms/LoopVectorize/AArch64. NFC 2020-10-10 10:15:43 +01:00
max-vf-for-interleaved.ll
no_vector_instructions.ll Revert "[LoopVectorize] Simplify scalar cost calculation in getInstructionCost" 2021-03-26 11:36:53 +00:00
nontemporal-load-store.ll
outer_loop_test1_no_explicit_vect_width.ll [VPlan] Manage pairs of incoming (VPValue, VPBB) in VPWidenPHIRecipe. 2021-02-22 09:44:25 +00:00
pr31900.ll
pr33053.ll [Vectorizers][TTI] remove option to bypass creation of vector reduction intrinsics 2021-02-12 08:13:50 -05:00
pr36032.ll
pr46950-load-cast-context-crash.ll
predication_costs.ll
reduction-small-size.ll
runtime-check-size-based-threshold.ll [LV] Add test cases that require a larger number of RT checks. 2021-03-02 10:49:38 +00:00
scalable-call.ll [TTI] Change getOperandsScalarizationOverhead to take Type args 2021-02-23 13:04:59 +00:00
scalable-reductions.ll [SVE] Add support for scalable vectorization of loops with int/fast FP reductions 2021-02-16 13:50:06 +00:00
scalable-vf-hint.ll [LV] Legalize scalable VF hints 2021-01-08 10:49:44 +00:00
sdiv-pow2.ll
select-costs.ll [LV] Add cost-model test for AArch64 select costs. 2020-10-26 13:43:31 +00:00
smallest-and-widest-types.ll
sve-basic-vec.ll [SVE] Add support for scalable vectorization of loops with selects and cmps 2021-01-22 09:48:13 +00:00
sve-cond-inv-loads.ll [SVE][LoopVectorize] Verify support for vectorizing loops with invariant loads 2021-03-25 14:10:21 +00:00
sve-extract-last-veclane.ll [SVE][LoopVectorize] Add support for extracting the last lane of a scalable vector 2021-03-05 09:57:56 +00:00
sve-gather-scatter.ll [SVE][LoopVectorize] Add masked load/store and gather/scatter support for SVE 2021-02-02 09:52:39 +00:00
sve-inductions.ll [LoopVectorize] Add support for scalable vectorization of induction variables 2021-03-30 11:13:31 +01:00
sve-inv-loads.ll [SVE][LoopVectorize] Verify support for vectorizing loops with invariant loads 2021-03-25 14:10:21 +00:00
sve-masked-loadstore.ll [SVE][LoopVectorize] Add masked load/store and gather/scatter support for SVE 2021-02-02 09:52:39 +00:00
sve-scalable-load-in-loop.ll [llvm][NFC] Disallow all warnings in TypeSize tests 2021-01-06 17:17:07 +00:00
sve-vector-reverse-mask4.ll [SVE][LoopVectorize] Add support for scalable vectorization of loops with vector reverse 2021-03-16 07:51:59 +00:00
sve-vector-reverse.ll [SVE][LoopVectorize] Add support for scalable vectorization of loops with vector reverse 2021-03-16 07:51:59 +00:00
type-shrinkage-insertelt.ll
unsafe-vf-hint-remark.ll [LV] Clamp VF hint when unsafe 2020-12-01 11:30:34 +00:00
vector-reverse-mask4.ll [SVE][LoopVectorize] Add support for scalable vectorization of loops with vector reverse 2021-03-16 07:51:59 +00:00
vector-reverse.ll [SVE][LoopVectorize] Add support for scalable vectorization of loops with vector reverse 2021-03-16 07:51:59 +00:00