Commit Graph

289 Commits

Author SHA1 Message Date
Dmitry Preobrazhensky 4e68834add [AMDGPU][MC][GFX11][NFC] Add tests for VOP1 and VOP2 16 bit opcodes
Differential Revision: https://reviews.llvm.org/D131588
2022-08-11 17:12:13 +03:00
Dmitry Preobrazhensky eda6e49aa8 [AMDGPU][MC][GFX11][NFC] Correct tests for 16-bit VOP2 opcodes which use v128 or higher VGPRs
Differential Revision: https://reviews.llvm.org/D131564
2022-08-11 15:55:58 +03:00
Dmitry Preobrazhensky 9d1eeefbfe [AMDGPU][MC][GFX11][NFC] Rename tests
Make test names more uniform.

Differential Revision: https://reviews.llvm.org/D131398
2022-08-09 13:56:05 +03:00
Dmitry Preobrazhensky fc4c1a86f3 [AMDGPU][MC][GFX11][NFC] Split large tests
Differential Revision: https://reviews.llvm.org/D131397
2022-08-09 13:24:23 +03:00
Dmitry Preobrazhensky 05b3aadfff [AMDGPU][MC][GFX11] Correct v_dot2_f16_f16 and v_dot2_bf16_bf16
Enable SGPRs for the following operands of these opcodes:

- src operands of VOP3 variant.
- src2 operand of DPP variants.

Differential Revision: https://reviews.llvm.org/D130989
2022-08-03 15:08:23 +03:00
Dmitry Preobrazhensky ae553f9e49 [AMDGPU][MC][GFX10] Correct encoding of VOP3 v_cmpx* opcodes
Encode dst=EXEC but allow disassembler accept any dst value.

Differential Revision: https://reviews.llvm.org/D130978
2022-08-03 15:03:44 +03:00
Dmitry Preobrazhensky bb901dcc5a [AMDGPU][MC][GFX940] Correct disassembly of MFMA opcodes
Add a decoder table for GFX940 MFMA opcodes.

Differential Revision: https://reviews.llvm.org/D130759
2022-08-01 16:00:47 +03:00
Petar Avramovic e8d260753e [AMDGPU] gfx11 allow dlc for MUBUF atomics
Add MC support for dlc in gfx11 MUBUF atomic instructions.

Differential Revision: https://reviews.llvm.org/D129075
2022-08-01 12:18:01 +02:00
Mirko Brkusanin 6a1aa627fa [AMDGPU] Enable image_gather4h instruction for gfx10 and gfx11
Differential Revision: https://reviews.llvm.org/D130764
2022-07-29 15:42:06 +02:00
Dmitry Preobrazhensky 0eb9f18520 [AMDGPU][MC][GFX11] Correct encoding of VOP3/VOP3_DPP v_cmpx* opcodes
Encode dst=EXEC but allow disassembler accept any dst value.

Differential Revision: https://reviews.llvm.org/D130345
2022-07-26 17:36:22 +03:00
Petar Avramovic 8de1f04c77 [AMDGPU] gfx11 Fix VOP3 dot instructions
Fix src modifiers for operands with bf16 type.
op_sel[0:1] are ignored.

Differential Revision: https://reviews.llvm.org/D129084
2022-07-22 11:43:35 +02:00
Stanislav Mekhanoshin 523a99c0eb [AMDGPU] Support for gfx940 fp8 smfmac
Differential Revision: https://reviews.llvm.org/D129908
2022-07-18 12:12:41 -07:00
Stanislav Mekhanoshin 2695f0a688 [AMDGPU] Support for gfx940 fp8 mfma
Differential Revision: https://reviews.llvm.org/D129906
2022-07-18 11:49:56 -07:00
Stanislav Mekhanoshin 9fa5a6b7e8 [AMDGPU] Support for gfx940 fp8 conversions
Differential Revision: https://reviews.llvm.org/D129902
2022-07-18 11:48:43 -07:00
Dmitry Preobrazhensky 2a6532d542 [AMDGPU][MC][GFX11] Correct disassembly of *_e64_dpp opcodes which support op_sel
These opcodes cannot be disassembled because op_sel operand is missing - it must be added manually.
See https://github.com/llvm/llvm-project/issues/56512 for detailed issue analysis.

Differential Revision: https://reviews.llvm.org/D129637
2022-07-15 13:11:59 +03:00
Piotr Sobczak 4874838a63 [AMDGPU] gfx11 WMMA instruction support
gfx11 introduces new WMMA (Wave Matrix Multiply-accumulate)
instructions.

Reviewed By: arsenm, #amdgpu

Differential Revision: https://reviews.llvm.org/D128756
2022-06-30 11:13:45 -04:00
Joe Nash 07b7fada73 [AMDGPU] gfx11 VOPD instructions MC support
VOPD is a new encoding for dual-issue instructions for use in wave32.
This patch includes MC layer support only.

A VOPD instruction is constituted of an X component (for which there are
13 possible opcodes) and a Y component (for which there are the 13 X
opcodes plus 3 more). Most of the complexity in defining and parsing
a VOPD operation arises from the possible different total numbers of
operands and deferred parsing of certain operands depending on the
constituent X and Y opcodes.

Reviewed By: dp

Differential Revision: https://reviews.llvm.org/D128218
2022-06-24 11:08:39 -04:00
Dmitry Preobrazhensky dcb24f93af [AMDGPU][MC][GFX11] Correct disassembly of VOP3.DPP8 opcodes
Fix bug #56163.
Add W32/W64 tests for all VOP3.DPP opcodes.

Differential Revision: https://reviews.llvm.org/D128369
2022-06-23 13:07:45 +03:00
Dmitry Preobrazhensky 485e8b4f63 [AMDGPU][MC][GFX11] Correct disassembly of DPP variants of VOPC64 opcodes
Fix bugs https://github.com/llvm/llvm-project/issues/56091, https://github.com/llvm/llvm-project/issues/56065.

Differential Revision: https://reviews.llvm.org/D128075
2022-06-20 14:23:07 +03:00
Dmitry Preobrazhensky b26afab9d1 [AMDGPU][MC][GFX11] Correct src0 for dpp variants of v_cvt_*_e64
Differential Revision: https://reviews.llvm.org/D127847
2022-06-16 13:48:43 +03:00
Joe Nash 989bd57f98 [AMDGPU] gfx11 support add_f16
The instruction was skipped in the earlier large patch adding
VOP2, https://reviews.llvm.org/D126917.

Reviewed By: rampitec, #amdgpu

Differential Revision: https://reviews.llvm.org/D127697
2022-06-14 08:59:45 -04:00
Dmitry Preobrazhensky 365d827f65 [AMDGPU][MC][GFX11] Correct ds_swizzle_b32
Enable offset parsing.

Differential Revision: https://reviews.llvm.org/D127404
2022-06-14 12:58:03 +03:00
Joe Nash fd3304ef85 [AMDGPU] gfx11 EXECZ and VCCZ are no longer allowed to be used as
sources to SALU and VALU instructions.

Contributors:
Baptiste Saleil <baptiste.saleil@amd.com>

Patch 20/N for upstreaming of AMDGPU gfx11 architecture

Depends on D126989

Reviewed By: rampitec, foad, #amdgpu

Differential Revision: https://reviews.llvm.org/D127143
2022-06-10 10:03:43 -04:00
Ivan Kosarev 60d6fbb621 [AMDGPU][GFX9][GFX10] Support base+soffset+offset SMEM atomics.
Resolves a part of
https://github.com/llvm/llvm-project/issues/38652

Reviewed By: dp

Differential Revision: https://reviews.llvm.org/D127314
2022-06-10 13:22:41 +01:00
Joe Nash be1082c6d5 [AMDGPU] gfx11 VOPC instructions
Supports encoding existing instrutions on gfx11 and MC support for the new VOPC
dpp instructions.

Patch 19/N for upstreaming of AMDGPU gfx11 architecture

Depends on D126978

Reviewed By: rampitec, #amdgpu

Differential Revision: https://reviews.llvm.org/D126989
2022-06-09 15:22:42 -04:00
Joe Nash 40f35cef89 [AMDGPU] gfx11 VOP3P instruction MC support
Includes dpp versions of VOP3P instructions.

Patch 18/N for upstreaming of AMDGPU gfx11 architecture

Depends on D126917

Reviewed By: rampitec, #amdgpu

Differential Revision: https://reviews.llvm.org/D126978
2022-06-08 13:32:01 -04:00
Joe Nash 086a9c1062 Reland [AMDGPU] gfx11 VOP1+VOP2 Instruction MC support
The reverted dependent commit is now relanded, so reland this.
Includes dpp instructions and vop1/vop2 promoted to vop3

Patch 17/N for upstreaming of AMDGPU gfx11 architecture

Depends on D126483

Reviewed By: rampitec, #amdgpu

Differential Revision: https://reviews.llvm.org/D126917
2022-06-08 11:10:57 -04:00
Joe Nash e243ead6fc Reland [AMDGPU] gfx11 vop3dpp instructions
There was an issue with encoding wide (>64 bit) instructions on
BigEndian hosts, which is fixed in D127195. Therefore reland this.

gfx11 adds the ability to use dpp modifiers on vop3 instructions.
This patch adds machine code layer support for that. The MCCodeEmitter
is changed to use APInt instead of uint64_t to support these wider
instructions.

Patch 16/N for upstreaming of AMDGPU gfx11 architecture

Differential Revision: https://reviews.llvm.org/D126483
2022-06-07 14:49:13 -04:00
Joe Nash eaed07eb7e Revert "[AMDGPU] gfx11 vop3dpp instructions"
This reverts commit 99a83b1286.
2022-06-06 17:12:09 -04:00
Joe Nash f617f89e5b Revert "[AMDGPU] gfx11 VOP1+VOP2 Instruction MC support"
This reverts commit 6079804498.
2022-06-06 17:11:35 -04:00
Ivan Kosarev facbfb121a [AMDGPU][GFX9+] Support base+soffset+offset s_atc_probe's.
Resolves part of
https://github.com/llvm/llvm-project/issues/38652

Reviewed By: dp

Differential Revision: https://reviews.llvm.org/D126791
2022-06-06 16:46:22 +01:00
Ivan Kosarev 79ec1e8fd6 [AMDGPU][GFX9][GFX10] Support base+soffset+offset s_dcache_discard's.
Resolves part of
https://github.com/llvm/llvm-project/issues/38652

Reviewed By: dp

Differential Revision: https://reviews.llvm.org/D126766
2022-06-06 16:32:16 +01:00
Joe Nash 6079804498 [AMDGPU] gfx11 VOP1+VOP2 Instruction MC support
Includes dpp instructions and vop1/vop2 promoted to vop3

Patch 17/N for upstreaming of AMDGPU gfx11 architecture

Depends on D126483

Reviewed By: rampitec, #amdgpu

Differential Revision: https://reviews.llvm.org/D126917
2022-06-06 09:57:59 -04:00
Joe Nash 99a83b1286 [AMDGPU] gfx11 vop3dpp instructions
gfx11 adds the ability to use dpp modifiers on vop3 instructions.
This patch adds machine code layer support for that. The MCCodeEmitter
is changed to use APInt instead of uint64_t to support these wider
instructions.

Patch 16/N for upstreaming of AMDGPU gfx11 architecture

Depends on D126475

Reviewed By: rampitec, #amdgpu

Differential Revision: https://reviews.llvm.org/D126483
2022-06-06 09:34:59 -04:00
Joe Nash 3732cd59be [AMDGPU] gfx11 vop3 and inherited vop instructions
This patch includes MC layer support for VOP3 encoded instructions and generic VOP support
classes.
Some VOP1 and VOP2 instructions which share an encoding with gfx10 and are using
the AssemblerPredicate = isGFX10Plus are also enabled. That predicate
will be changed to isGFX10Only in a later patch.

Patch 15/N for upstreaming of AMDGPU gfx11 architecture.

Depends on D126468

Reviewed By: dp

Differential Revision: https://reviews.llvm.org/D126475
2022-06-02 14:03:02 -04:00
Joe Nash e4870c8357 [AMDGPU] gfx11 ds instructions
MC layer support for ds instructions

Contributors:
Piotr Sobczak <Piotr.Sobczak@amd.com>

Patch 14/N for upstreaming of AMDGPU gfx11 architecture.

Depends on D126463

Reviewed By: arsenm, #amdgpu

Differential Revision: https://reviews.llvm.org/D126468
2022-06-02 13:36:56 -04:00
Joe Nash e8860bee28 [AMDGPU] gfx11 Image instructions
MC layer support for instructions in the MIMG encoding(Image
instructions).

Contributors:
Carl Ritson <carl.ritson@amd.com>

Patch 13/N for upstreaming of AMDGPU gfx11 architecture.

Depends on D125992

Reviewed By: rampitec, #amdgpu

Differential Revision: https://reviews.llvm.org/D126463
2022-05-31 10:53:35 -04:00
Ivan Kosarev 082822b381 [AMDGPU][GFX9] Support base+soffset+offset SMEM stores.
Reviewed By: dp

Differential Revision: https://reviews.llvm.org/D126388
2022-05-30 10:27:57 +01:00
Ivan Kosarev b0ccf38b01 [AMDGPU][GFX9] Support base+soffset+offset SMEM loads.
Resolves part of
https://github.com/llvm/llvm-project/issues/38652

Reviewed By: dp

Differential Revision: https://reviews.llvm.org/D125700
2022-05-26 12:42:33 +01:00
Joe Nash 835e09c4c3 [AMDGPU] gfx11 FLAT Instructions
MachineCode Support for FLAT type instructions

Contributors:
Sebastian Neubauer <sebastian.neubauer@amd.com>

Patch 12/N for upstreaming of AMDGPU gfx11 architecture.

Depends on D125989

Reviewed By: rampitec, #amdgpu

Differential Revision: https://reviews.llvm.org/D125992
2022-05-25 15:29:39 -04:00
Joe Nash ef1ea5ac01 [AMDGPU] gfx11 vinterp instructions MC support
A new instruction encoding. Some of these instructions were previously VOP3
encoded.

Contributors:
Carl Ritson <carl.ritson@amd.com>

Patch 11/N for upstreaming of AMDGPU gfx11 architecture.

Depends on D125824

Reviewed By: critson

Differential Revision: https://reviews.llvm.org/D125989
2022-05-25 14:59:16 -04:00
Joe Nash 1a51ab766f [AMDGPU] gfx11 export instructions
Contributors:
Jay Foad <jay.foad@amd.com>
Dmitry Preobrazhensky <d-pre@mail.ru>

Patch 10/N for upstreaming of AMDGPU gfx11 architecture.

Depends on D125822

Reviewed By: dp

Differential Revision: https://reviews.llvm.org/D125824
2022-05-25 14:44:09 -04:00
Ivan Kosarev 1586e1dc95 [AMDGPU][MC][GFX11] Support base+soffset+offset SMEM loads.
Reviewed By: dp

Differential Revision: https://reviews.llvm.org/D126207
2022-05-24 15:13:14 +01:00
Joe Nash ac2ff258d6 [AMDGPU] gfx11 scalar memory instructions
Contributors:
Mirko Brkusanin <Mirko.Brkusanin@amd.com>

Patch 9/N for upstreaming of AMDGPU gfx11 architecture.

Depends on D125820

Reviewed By: kosarev, #amdgpu, arsenm

Differential Revision: https://reviews.llvm.org/D125822
2022-05-19 10:27:47 -04:00
Joe Nash 729467acef [AMDGPU] gfx11 LDSDIR instructions MC support
Contributors:
Carl Ritson <carl.ritson@amd.com>

Patch 8/N for upstreaming of AMDGPU gfx11 architecture.

Depends on D125498

Reviewed By: critson, rampitec, #amdgpu

Differential Revision: https://reviews.llvm.org/D125820
2022-05-19 10:08:47 -04:00
Dmitry Preobrazhensky 32ca9bd7b5 [AMDGPU][MC][GFX940] Correct tied operand decoding for smfmac opcodes
Differential Revision: https://reviews.llvm.org/D125790
2022-05-18 15:39:30 +03:00
Ivan Kosarev 140ad30b24 [AMDGPU][MC][GFX10] Add missing s_scratch_load tests.
Completes
https://reviews.llvm.org/D125117

Reviewed By: dp, arsenm

Differential Revision: https://reviews.llvm.org/D125753
2022-05-18 11:11:10 +01:00
Stanislav Mekhanoshin a09af86693 [AMDGPU] Enable FLAT LDS DMA on gfx9/10 before gfx940
We always had global and scratch loads to LDS in the gfx9,
but did not handle it. These were available via the 'lds'
encoding bit. In gfx940 this bit was reused as 'svs' which
resulted in new '_lds' opcodes effectively pushing this
bit into the opcode, but functionally it is the same. These
instructions are also available on gfx10.

Differential Revision: https://reviews.llvm.org/D125126
2022-05-17 12:16:37 -07:00
Joe Nash d21b9b4946 [AMDGPU] gfx11 scalar alu instructions
MC layer support for SOP(scalar alu operations) including encoding
support for s_delay_alu and s_sendmsg_rtn.

Contributors:
Jay Foad <jay.foad@amd.com>

Patch 7/N for upstreaming of AMDGPU gfx11 architecture.

Depends on D125319

Reviewed By: #amdgpu, arsenm

Differential Revision: https://reviews.llvm.org/D125498
2022-05-17 13:35:41 -04:00
Joe Nash c70259405c [AMDGPU] gfx11 BUF Instructions
Includes MachineCode layer support and tests, and MIR tests not requiring
CodeGen pass changes.
Includes a small change in SMInstructions.td to correct encoded bits.

Contributors:
Petar Avramovic <Petar.Avramovic@amd.com>
Dmitry Preobrazhensky <dmitry.preobrazhensky@amd.com>

Depends on D125316

Patch 6/N for upstreaming of AMDGPU gfx11 architecture.

Reviewed By: dp, Petar.Avramovic

Differential Revision: https://reviews.llvm.org/D125319
2022-05-16 09:41:40 -04:00