After ISEL, the "valid" loop header which has two predecessors
(one is preheader and the other one is latch) may be transformed
to have more than two predecessors by some optimizations, like tail
duplicator, if the old header's successor(will be changed to new
header) is a sub loop.
The predecessors of the new loop header are preheader, loop latch
and the loop latch(es) of the sub loop(old header's successor).
Before the patch, ctrloop pass assumes two predecessors for candidate
loop header. This patch fixes this case.
Reviewed By: lkail
Differential Revision: https://reviews.llvm.org/D135846
In https://github.com/llvm/llvm-project/issues/57452, we found that IRTranslator is translating `i1 true` into `i32 -1`.
This is because IRTranslator uses SExt for indices.
In this fix, we change the expected behavior of extractelement's index, moving from SExt to ZExt.
This change includes both documentation, SelectionDAG and IRTranslator.
We also included a test for AMDGPU, updated tests for AArch64, Mips, PowerPC, RISCV, VE, WebAssembly and X86
This patch fixes issue #57452.
Differential Revision: https://reviews.llvm.org/D132978
This patch adds a test case for lowering vector shuffles to xxsplti32dx in
preparation for D135024. The test case added in this patch only adds the
64-bit CHECKs, as the 32-bit CHECKs cannot be generated (in which D135024
aims to fix).
Inputs to crnor can come from operands with chains so
if it is being used simply to negate such an operand,
the repeated input cannot be CSE'd. This patch just
adds a code-gen only instruction for this that takes
a single input and duplicates it in the encoding of
the underlying crnor.
Differential revision: https://reviews.llvm.org/D133577
When removing frame indices on PowerPC, we need to scavenge
a GPR to materialize a large constant if the stack offset
for the spill/reload cannot be reached by a D-Form
instruction. However, in a perfect storm of conditions,
we may not have GPR's available to scavenge, thereby
requiring an emergency spill. If such an emergency
spill also needs to be spilled to a location with a
large offset, it would itself require register scavenging
thereby creating an infinite loop.
This patch detects when the scavenger cannot scavenge
a register and the spill/reload is to a location with
a large offset. It then stashes a GPR into a VSR so
that it can use the GPR to materialize the constant
(rather than scavenging a GPR).
Fixes: https://github.com/llvm/llvm-project/issues/52894
Differential revision: https://reviews.llvm.org/D124841
(sra X, BW-1) is either 0 or -1. So the multiply is a conditional
negate of Y.
This pattern shows up when type legalizing wide multiplies involving
a sign extended value.
Fixes PR57549.
Reviewed By: RKSimon
Differential Revision: https://reviews.llvm.org/D133399
The LIT test cases were migrated with the script provided by
Nikita Popov. Due to the size of the change it is split into
several parts.
Reviewed By: nemanja, nikic
Differential Revision: https://reviews.llvm.org/D135474
The LIT test cases were migrated with the script provided by
Nikita Popov. Due to the size of the change it is split into
several parts.
Reviewed By: nemanja, amyk, nikic, PowerPC
Differential Revision: https://reviews.llvm.org/D135470
This is AIX part of update after https://reviews.llvm.org/D117225
Fixed the issue that AIX64 with vector pair enabled saw redundant
spill/reload of callee saved vector registers.
Based on original patch by: Kai Luo
Reviewed By: lkail
Differential Revision: https://reviews.llvm.org/D133466
The allocation hints for copies of ACC registers assumed that we would only be
copying between VSRp and UACC registers. In reality it is also possible to copy
between UACC and ACC registers.
This patch adds a new case for the ACC copy to fix that issue.
Note that the test case added with this patch will hit an assert without the
fix.
Reviewed By: lei, amyk
Differential Revision: https://reviews.llvm.org/D134501
https://alive2.llvm.org/ce/z/oShzr3
This was noted as a missing fold in D134876 (with additional
examples based on issue #58046).
I'm assuming that fmul with a zero operand is rare enough
that the use of ValueTracking will not noticeably increase
compile-time.
This adjusts a PowerPC codegen test that was added with D88388
because it would get folded away and no longer provide coverage
for the bug fix.
There are a few issues with the code we generate for atomic operations and the way we generate it:
- Hard coded CR0 for compares
- Order of operands for compares not conducive to
emitting compare-immediate or for CSE of compares
- Missing MachineMemOperand for st[bhwd]cx intrinsics
- Missing intrinsic properties for the same
- Unnecessary blocks with store conditional
instructions to clear reservation (which ends
up hindering performance)
- Move from CR instructions just to compare the
result of a store conditional with zero (even
though it is a record-form)
This patch aims to resolve all of those issues.
Differential revision: https://reviews.llvm.org/D134783
This seems to be beneficial overall, except for midpoint-int.ll .
The X86 backend seems to generate zeroing that are not necesary.
Reviewed By: shchenz
Differential Revision: https://reviews.llvm.org/D131260
This feature implements support for making entries in the exception section
on XCOFF on the direct assembly path using the ".except" pseudo-op. It also
provides functionality to lower entries (comprised of language and reason
codes) into the exception section through the use of annotation metadata
attached to llvm.ppc.trap/trapd/tw/tdw intrinsics. Integrated assembler
support will be provided in another review. https://reviews.llvm.org/D133030
needs to merge first for LIT tests
Reviewed By: shchenz, RKSimon
Differential Revision: https://reviews.llvm.org/D132146
The method of counting resource consumption is modified to be based on
"Cycles" value when DFA is not used.
The calculation of ResMII is modified to total "Cycles" and divide it
by the number of units for each resource. Previously, ResMII was
excessive because it was assumed that resources were consumed for
the cycles of "Latency" value.
The method of resource reservation is modified similarly. When a
value of "Cycles" is larger than 1, the resource is considered to be
consumed by 1 for cycles of its length from the scheduled cycle.
To realize this, ResourceManager maintains a resource table for all
slots. Previously, resource consumption was always 1 for 1 cycle
regardless of the value of "Cycles" or "Latency".
In addition, the number of micro operations per cycle is modified to
be constrained by "IssueWidth". To disable the constraint,
--pipeliner-force-issue-width=100 can be used.
For the case of using DFA, the scheduling results are unchanged.
Reviewed By: dpenry
Differential Revision: https://reviews.llvm.org/D133572
doesn't happened in peephole optimizer.
Summary: Converting a comparison against 1 or -1 into a comparison
against 0 can exploit record-form instructions for comparison optimization.
The conversion will happen only when a record-form instruction can be used
to replace the comparison during the peephole optimizer (see function optimizeCompareInstr).
In post-RA, we also want to optimize the comparison by using the record
form (see D131873) and it requires additional dataflow analysis to reliably
find uses of the CR register set.
It's reasonable to common the conversion for both peephole optimizer and
post-RA optimizer.
Converting to comparison against zero even when the optimization doesn't
happened in peephole optimizer may create additional opportunities for the
post-RA optimization.
Reviewed By: nemanjai
Differential Revision: https://reviews.llvm.org/D131374
Get some load-store forwarding cases for big-endian where a larger store covers
a smaller load, and the offset would be 0 and handled on little-endian but on
big-endian the offset is adjusted to be non-zero. The idea is just to shift the
data to make it look like the offset 0 case.
Differential Revision: https://reviews.llvm.org/D130115
This patch introduces the priority analysis and the priority advisor,
the default implementation, and the scaffolding for introducing the
other implementations of the advisor.
Reviewed By: mtrofin
Differential Revision: https://reviews.llvm.org/D132835
Add a visibility check for dllimport and dllexport. Note: dllimport with a
non-default visibility (implicit dso_local) is already rejected, but with a less
clear dso_local error.
The MC level visibility `MCSA_Exported` (D123951) is mapped from IR level
default visibility when dllexport is specified. The D123951 error is now very
difficult to trigger (needs to disable the IR verifier).
Reviewed By: mstorsjo
Differential Revision: https://reviews.llvm.org/D133267
IIUC, the conversion part is not part of atomic operations and fences should be put around converted atomic operations.
This also fixes atomic load of floating point values which requires fence on PowerPC.
Reviewed By: efriedma
Differential Revision: https://reviews.llvm.org/D127609
Add isNotDuplicable to CTRLoop pseudo instructions, to avoid other pass
such as early-tailduplication break the loop structure by duplicating
pseudo instructions.
Reviewed By: shchenz
Differential Revision: https://reviews.llvm.org/D132738
D132080 introduced a bug leading to `RegisterClassInfo` caches not
getting invalidated when there was exactly one more CSR register added.
Differential Revision: https://reviews.llvm.org/D132606
Summary: Currently, an error was reported when a thread local symbol has an invalid name. D100956 create a new symbol to prefix the TLS symbol name with a dot. When the symbol name is renamed, the error occurs. This patch uses the original symbol name (name in the symbol table) as the input for the symbol for TOC entry.
Reviewed By: shchenz, lkail
Differential Revision: https://reviews.llvm.org/D132348
`RegisterClassInfo` caches information like allocation orders and reuses
it for multiple machine functions where possible. However the `MCPhysReg
*CalleeSavedRegs` field used to test whether the set of callee saved
registers changed did not work: After D28566
`MachineRegisterInfo::getCalleeSavedRegs()` can return dynamically
computed CSR sets that are only valid while the `MachineRegisterInfo`
object of the current function exists.
This changes the code to make a copy of the CSR list instead of keeping
a possibly invalid pointer around.
Differential Revision: https://reviews.llvm.org/D132080
This patch fixes the following two bugs in `PPCInstrInfo::isSignOrZeroExtended` helper, which is used from sign-/zero-extension elimination in PPCMIPeephole pass.
- Registers defined by load with update (e.g. LBZU) were identified as already sign or zero-extended. But it is true only for the first def (loaded value) and not for the second def (i.e. updated pointer).
- Registers defined by ORIS/XORIS were identified as already sign-extended. But, it is not true for sign extension depending on the immediate (while it is ok for zero extension).
To handle the first case, the parameter for the helpers is changed from `MachineInstr` to a register number to distinguish first and second defs. Also, this patch moves the initialization of PPCMIPeepholePass to allow mir test case.
Reviewed By: nemanjai
Differential Revision: https://reviews.llvm.org/D40554
This patch adds additional test cases involving vector_shuffles where either its
left, right or both inputs are scalar_to_vector nodes. These test cases involve
v16i8, v2i64, v4i32 and v8i16 vector shuffles, and were generated in preparation
for D130487.
Differential Revision: https://reviews.llvm.org/D130485
TLS debug on AIX is not ready for now.
The location generated in no-integrated-as mode is wrong and
in integrated-as mode causes AIX linker error.
Reviewed By: Esme
Differential Revision: https://reviews.llvm.org/D130245
Summary: AIX XCOFF doesn't support the cold feature.
While it shouldn't be a function error when XCOFF catching the cold attribute.
As with the behavior of other formats, we just ignore the attribute for now.
Reviewed By: DiggerLin
Differential Revision: https://reviews.llvm.org/D131473
These changes to address issue
https://github.com/llvm/llvm-project/issues/55857.
Since R30/S30 is used as pointer (32 bits) for GOT Table in the ppc32 ABI,
remove it from the SPE callee save register when PIC is enabled.
This prevents emitting the SPE load and store for S30 and S31 regs.
Differential revision: https://reviews.llvm.org/D127495
SPE doesn't have a fmadd instruction, so don't bother hoisting a
multiply and add sequence to this, as it'd become just a library call.
Hoisting happens too late for the CTR usability test to veto using the
CTR in a loop, and results in an assert "Invalid PPC CTR loop!".
This intrinsic used a typed pointer for a call target operand. This
change updates the operand to be an opaque pointer and updates all
pointers in all test files that use the intrinsic.
Differential revision: https://reviews.llvm.org/D131261
The previous code overwrites VRMap for prologue stages during Phi
generation if a register spans many stages.
As a result, the wrong register is used as the one coming from
the prologue in Phis at later stages. (A process exists to correct
this, but it does not work in all cases.)
In addition, VRMap for prologue must be preserved until addBranches().
This patch fixes them by separating the map for Phis into a different
variable (VRMapPhi).
Reviewed By: bcahoon
Differential Revision: https://reviews.llvm.org/D127840
Map hardware loop intrinsics loop_decrement and set_loop_iteration
to the new PowerPC pseudo instructions, so that the hardware loop
intrinsics will be expanded to normal cmp+branch form or ctrloop
form based on the CTR register usage on MIR level.
Reviewed By: lkail
Differential Revision: https://reviews.llvm.org/D123366
Chen Zheng