Commit Graph

466 Commits

Author SHA1 Message Date
Kazu Hirata 20cde15415 [Target] Use std::nullopt instead of None (NFC)
This patch mechanically replaces None with std::nullopt where the
compiler would warn if None were deprecated.  The intent is to reduce
the amount of manual work required in migrating from Optional to
std::optional.

This is part of an effort to migrate from llvm::Optional to
std::optional:

https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716
2022-12-02 20:36:06 -08:00
Mateja Marjanovic 595a08847a [AMDGPU] Add support for new LLVM vector types
Add VReg, AReg and SReg on AMDGPU for bit widths: 288, 320, 352 and 384.

Differential Revision: https://reviews.llvm.org/D138205
2022-11-29 17:02:04 +01:00
Dmitry Preobrazhensky 9b8eb5fa8e [AMDGPU][MC][GFX11] Correct op_sel handling for permlane*16
Differential Revision: https://reviews.llvm.org/D137969
2022-11-29 18:45:22 +03:00
Dmitry Preobrazhensky 869fc7eabd [AMDGPU][MC][MI100+] Enable VOP3 variants of dot2c/dot4c/dot8c opcodes
Differential Revision: https://reviews.llvm.org/D138494
2022-11-29 17:38:18 +03:00
Ivan Kosarev 536b8c5377 [AMDGPU][AsmParser] Remove the now-unused OptionalOperand structure.
Became unused in https://reviews.llvm.org/D138492 .
2022-11-28 11:00:05 +00:00
Kazu Hirata 96cb34ab5a [AsmParser] Use std::optional in AMDGPUAsmParser.cpp (NFC)
This is part of an effort to migrate from llvm::Optional to
std::optional:

https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716
2022-11-25 22:36:37 -08:00
Ivan Kosarev fce7a7aa9f [AMDGPU][AsmParser] Refine parsing instruction operands.
Eliminates the need for working around optional and token operands being
mistakenly parsed as expressions.

Reviewed By: dp

Differential Revision: https://reviews.llvm.org/D138492
2022-11-24 10:50:26 +00:00
Pierre van Houtryve 220147d536 [AMDGPU] Make aperture registers 64 bit
Makes the SRC_(SHARED|PRIVATE)_(BASE|LIMIT) registers 64 bit instead of 32.
They're still usable as 32 bit operands by using the _LO suffix.

Preparation for D137542

Reviewed By: arsenm

Differential Revision: https://reviews.llvm.org/D137767
2022-11-22 09:17:59 +00:00
Dmitry Preobrazhensky 96155bf44b [AMDGPU][GFX11][NFC] Refactor VOPD operands handling (part 2)
Rename interface functions and operands to make code clearer.

Differential Revision: https://reviews.llvm.org/D138133
2022-11-18 14:15:05 +03:00
Dmitry Preobrazhensky e468b1b740 [AMDGPU][GFX11] Refactor VOPD operands handling
Differential Revision: https://reviews.llvm.org/D137952
2022-11-16 16:29:12 +03:00
Ivan Kosarev d55594bb5e [AMDGPU][AsmParser] Forbid TFE modifiers for MBUF stores.
Reviewed By: dp

Differential Revision: https://reviews.llvm.org/D137832
2022-11-14 16:10:36 +00:00
Ivan Kosarev 1b560e6ab7 [AMDGPU][MC] Support TFE modifiers in MUBUF loads and stores.
Reviewed By: dp, arsenm

Differential Revision: https://reviews.llvm.org/D137783
2022-11-14 15:36:18 +00:00
Dmitry Preobrazhensky ef6f3c76f7 [AMDGPU][MC][GFX11] Improve diagnostic messages for invalid VOPD syntax
Differential Revision: https://reviews.llvm.org/D137842
2022-11-14 16:20:20 +03:00
Ivan Kosarev 926acd2bb5 [AMDGPU][AsmParser] Remove extra checks on missing instruction modifiers.
https://reviews.llvm.org/D137549 makes them unnecessary.

Reviewed By: dp

Differential Revision: https://reviews.llvm.org/D137638
2022-11-10 12:49:27 +00:00
Pierre van Houtryve 7425077e31 [AMDGPU] Add & use `hasNamedOperand`, NFC
In a lot of places, we were just calling `getNamedOperandIdx` to check if the result was != or == to -1.
This is fine in itself, but it's verbose and doesn't make the intention clear, IMHO. I added a `hasNamedOperand` and replaced all cases I could find with regexes and manually.

Reviewed By: arsenm, foad

Differential Revision: https://reviews.llvm.org/D137540
2022-11-08 07:57:21 +00:00
Joe Nash 01b8140d3a [AMDGPU] Fix delay alu for VOPD with src2acc
V_FMAC_F32 and V_DOT2C_F32_F16 have a dummy src2 operand tied to vdst to
inform passes that the instructions read the dst operand. The VOPD
versions of these instructions lacked the dummy operand, which was a
problem for inserting s_delay_alu.
Introduce the dummy src2 operand on the VOPD versions, and fix the VOPD operand
tracking logic to account for it.

Reviewed By: dp

Differential Revision: https://reviews.llvm.org/D136629
2022-10-25 13:11:17 -04:00
Dmitry Preobrazhensky db3e858286 [AMDGPU][MC][GFX11][NFC] Improve error message when a VOPD opcode is used with WS64
Differential Revision: https://reviews.llvm.org/D136168
2022-10-20 15:14:25 +03:00
Dmitry Preobrazhensky 8f8e4e3b38 [AMDGPU][MC][GFX11] Correct v_fmac_.*_e64_dpp
Differential Revision: https://reviews.llvm.org/D134961
2022-10-07 16:21:55 +03:00
Dmitry Preobrazhensky fd7b0eeaf6 [AMDGPU][MC][GFX11] Add VOPD VGPR bank access validation
Differential Revision: https://reviews.llvm.org/D134960
2022-10-07 15:52:59 +03:00
Dmitry Preobrazhensky 485c539391 [AMDGPU][MC][GFX11] Disable non-null src0 for s_waitcnt_*cnt
Differential Revision: https://reviews.llvm.org/D134809
2022-09-29 19:56:03 +03:00
Jay Foad 5c7ee894f8 AMDGPU: Stop validating earlyclobber operands in assembler
This validation was introduced in D34003 for v_qsad/v_mqsad instructions
but it applies to all instructions with earlyclobber operands, which now
includes v_mad_i64/v_mad_u64.

In all these cases I do not think there is documentation saying that the
destination must not overlap the sources. Rather there are *some* cases
where the instruction may not function correctly if there is an overlap,
and we are using earlyclobber as a conservative way of preventing
codegen from generating those cases.

I think it is unhelpful for the assembler to enforce the earlyclobber
restriction because it prevents assembling cases where the programmer
knows that in fact the overlap is safe.

See also: https://github.com/llvm/llvm-project/issues/57610

Differential Revision: https://reviews.llvm.org/D134272
2022-09-21 21:46:59 +01:00
Joe Nash b982ba2a6e [AMDGPU][GFX11] Use VGPR_32_Lo128 for VOP1,2,C
Due to the encoding changes in GFX11, we had a hack in place that
    disables the use of VGPRs above 128. This patch removes the need for
    that hack.

    We introduce a new register class VGPR_32_Lo128 which is used for 16-bit
    operands of VOP1, VOP2, and VOPC instructions. This register class only has the
    low 128 VGPRs, but is otherwise identical to VGPR_32. Therefore, 16-bit VOP1,
    VOP2, and VOPC instructions are correctly limited to use the first 128
    VGPRs, while the other instructions can freely use all 256.

    We introduce new pseduo-instructions used on GFX11 which have the suffix
    t16 (True 16) to use the VGPR_32_Lo128 register class.

Reviewed By: foad, rampitec, #amdgpu

Differential Revision: https://reviews.llvm.org/D133723
2022-09-20 09:56:28 -04:00
Dmitry Preobrazhensky ef8feb6359 [AMDGPU][MC][NFC] Correct error message
Differential Revision: https://reviews.llvm.org/D134028
2022-09-16 18:22:08 +03:00
Jay Foad 3822a01e0b [AMDGPU] Add GFX11 ds_bvh_stack_rtn_b32 instruction
Differential Revision: https://reviews.llvm.org/D133928
2022-09-15 16:46:14 +01:00
Dmitry Preobrazhensky 0e868aff43 [AMDGPU][MC][GFX11] Add validation of constant bus limitations for VOPD
Differential Revision: https://reviews.llvm.org/D133881
2022-09-15 16:36:19 +03:00
Dmitry Preobrazhensky c89e60bf1f [AMDGPU][MC][GFX11] Add VOPD literals validation
Differential Revision: https://reviews.llvm.org/D133864
2022-09-15 16:29:53 +03:00
Dmitry Preobrazhensky 8bb5c89205 [AMDGPU][MC][NFC] Refactor AMDGPUAsmParser::validateVOPLiteral
Differential Revision: https://reviews.llvm.org/D133861
2022-09-15 16:26:14 +03:00
Dmitry Preobrazhensky 815ba49068 [AMDGPU][MC] Add detection of mandatory literals in parser
Differential Revision: https://reviews.llvm.org/D133606
2022-09-13 12:37:30 +03:00
Dmitry Preobrazhensky 6b79610fd5 [AMDGPU][MC][GFX11][NFC] Correct VOPD parsing
Differential Revision: https://reviews.llvm.org/D133492
2022-09-09 13:03:29 +03:00
Dmitry Preobrazhensky 3aae8cd842 [AMDGPU][MC] Verify selection of LDS MUBUF opcodes
Differential Revision: https://reviews.llvm.org/D130761
2022-08-01 16:44:39 +03:00
Dmitry Preobrazhensky 2b230d69ad [AMDGPU][MC][GFX90A] Correct MIMG dst size validation
Correct validator to enable MIMG dst size checks.

Differential Revision: https://reviews.llvm.org/D130512
2022-07-28 14:30:08 +03:00
Eli Friedman 1a6d82b93f Fix misc uses of "long" variables to use "int64_t".
I don't have any evidence these particular uses are actually causing any
issues, but we should avoid accidentally truncating immediate values
depending on the host.
2022-07-27 09:47:19 -07:00
Petar Avramovic 8de1f04c77 [AMDGPU] gfx11 Fix VOP3 dot instructions
Fix src modifiers for operands with bf16 type.
op_sel[0:1] are ignored.

Differential Revision: https://reviews.llvm.org/D129084
2022-07-22 11:43:35 +02:00
Joe Nash b28bb8cc9c [AMDGPU] Remove old operand from VOPC DPP
For most DPP instructions, the old operand stores the value that was in
the current lane before the DPP operation, and is tied to the
destination. For VOPC DPP, this is unnecessary and incorrect.

There appears to have been a latent bug related to D122737 with
SIInstrInfo::isOperandLegal. If you checked if a register operand was legal
when the InstructionDesc expected an immediate, it reported that is valid.
Its fix is necessary for and tested in this patch.

Reviewed By: foad, rampitec

Differential Revision: https://reviews.llvm.org/D130040
2022-07-19 09:35:05 -04:00
Stanislav Mekhanoshin 9fa5a6b7e8 [AMDGPU] Support for gfx940 fp8 conversions
Differential Revision: https://reviews.llvm.org/D129902
2022-07-18 11:48:43 -07:00
Petar Avramovic c287bc4841 [AMDGPU][MC][GFX11] AsmParser for op_sel for VOP3 dpp opcodes
Parse op_sel for *_e64_dpp VOP3 opcodes.
Depends on D129637 and setting of VOP3_OPSEL in dpp pseudos.

Differential Revision: https://reviews.llvm.org/D129767
2022-07-18 15:08:52 +02:00
Abinav Puthan Purayil d96361d714 [AMDGPU] Add the uses_dynamic_stack field to the kernel descriptor and the kernel metadata map
This change introduces the dynamic stack boolean field to code-object-v3
and above under the code properties of the kernel descriptor and under
the kernel metadata map of NT_AMDGPU_METADATA. This field corresponds to
the is_dynamic_callstack field of amd_kernel_code_t.

Differential Revision: https://reviews.llvm.org/D128344
2022-07-18 10:07:13 +05:30
Dmitry Preobrazhensky 185c36de73 [AMDGPU][MC][NFC] Remove unnecessary code
Differential Revision: https://reviews.llvm.org/D129766
2022-07-15 13:17:36 +03:00
Joe Nash 07b7fada73 [AMDGPU] gfx11 VOPD instructions MC support
VOPD is a new encoding for dual-issue instructions for use in wave32.
This patch includes MC layer support only.

A VOPD instruction is constituted of an X component (for which there are
13 possible opcodes) and a Y component (for which there are the 13 X
opcodes plus 3 more). Most of the complexity in defining and parsing
a VOPD operation arises from the possible different total numbers of
operands and deferred parsing of certain operands depending on the
constituent X and Y opcodes.

Reviewed By: dp

Differential Revision: https://reviews.llvm.org/D128218
2022-06-24 11:08:39 -04:00
Kazu Hirata 437f960062 [llvm] Call *set::insert without checking membership first (NFC) 2022-06-18 10:22:05 -07:00
Kazu Hirata 4271a1ff33 [llvm] Call *set::insert without checking membership first (NFC) 2022-06-18 10:17:22 -07:00
Dmitry Preobrazhensky 365d827f65 [AMDGPU][MC][GFX11] Correct ds_swizzle_b32
Enable offset parsing.

Differential Revision: https://reviews.llvm.org/D127404
2022-06-14 12:58:03 +03:00
Joe Nash ea3c9a87d3 [AMDGPU] gfx11 add bits to COMPUTE_PGM_RSRC3
Contributors:
Konstantin Zhuravlyov <kzhuravl_dev@outlook.com>

Patch 21/N for upstreaming of AMDGPU gfx11 architecture

Depends on D127143

Reviewed By: rampitec, #amdgpu, kzhuravl

Differential Revision: https://reviews.llvm.org/D127241
2022-06-10 13:07:14 -04:00
Joe Nash 78d8fdb88b [AMDGPU] NFC. Comment change to GFX10+ in AsmParser 2022-06-10 12:34:07 -04:00
Joe Nash 9175ab7746 [AMDGPU] gfx11 SRC_POPS_EXISTING_WAVE_ID is removed 2022-06-10 12:32:22 -04:00
Joe Nash fd3304ef85 [AMDGPU] gfx11 EXECZ and VCCZ are no longer allowed to be used as
sources to SALU and VALU instructions.

Contributors:
Baptiste Saleil <baptiste.saleil@amd.com>

Patch 20/N for upstreaming of AMDGPU gfx11 architecture

Depends on D126989

Reviewed By: rampitec, foad, #amdgpu

Differential Revision: https://reviews.llvm.org/D127143
2022-06-10 10:03:43 -04:00
Joe Nash be1082c6d5 [AMDGPU] gfx11 VOPC instructions
Supports encoding existing instrutions on gfx11 and MC support for the new VOPC
dpp instructions.

Patch 19/N for upstreaming of AMDGPU gfx11 architecture

Depends on D126978

Reviewed By: rampitec, #amdgpu

Differential Revision: https://reviews.llvm.org/D126989
2022-06-09 15:22:42 -04:00
Joe Nash 40f35cef89 [AMDGPU] gfx11 VOP3P instruction MC support
Includes dpp versions of VOP3P instructions.

Patch 18/N for upstreaming of AMDGPU gfx11 architecture

Depends on D126917

Reviewed By: rampitec, #amdgpu

Differential Revision: https://reviews.llvm.org/D126978
2022-06-08 13:32:01 -04:00
Joe Nash 086a9c1062 Reland [AMDGPU] gfx11 VOP1+VOP2 Instruction MC support
The reverted dependent commit is now relanded, so reland this.
Includes dpp instructions and vop1/vop2 promoted to vop3

Patch 17/N for upstreaming of AMDGPU gfx11 architecture

Depends on D126483

Reviewed By: rampitec, #amdgpu

Differential Revision: https://reviews.llvm.org/D126917
2022-06-08 11:10:57 -04:00
Joe Nash e243ead6fc Reland [AMDGPU] gfx11 vop3dpp instructions
There was an issue with encoding wide (>64 bit) instructions on
BigEndian hosts, which is fixed in D127195. Therefore reland this.

gfx11 adds the ability to use dpp modifiers on vop3 instructions.
This patch adds machine code layer support for that. The MCCodeEmitter
is changed to use APInt instead of uint64_t to support these wider
instructions.

Patch 16/N for upstreaming of AMDGPU gfx11 architecture

Differential Revision: https://reviews.llvm.org/D126483
2022-06-07 14:49:13 -04:00