692bd5ec2f
R600: Handle fnearbyint
...
The difference from rint isn't really relevant here,
so treat them as equivalent. OpenCL doesn't have nearbyint,
so this is sort of pointless other than for completeness.
llvm-svn: 211229
2014-06-18 22:03:45 +00:00
b55c68f171
Use LL suffix for literal that should be 64-bits.
...
This hopefully fixes Windows
llvm-svn: 211225
2014-06-18 21:40:43 +00:00
85f0dbce5c
R600: Expand vector fceil
...
Move fp64 fceil tests to fceil64.ll
v2: rebase
Signed-off-by: Jan Vesely <jan.vesely@rutgers.edu>
llvm-svn: 211194
2014-06-18 17:57:29 +00:00
d22626f6bb
Work around ridiculous warning.
...
Apparently C++ doesn't really have hex floating point constants.
llvm-svn: 211192
2014-06-18 17:45:58 +00:00
43160e7af2
R600/SI: Add intrinsics for brev instructions
...
llvm-svn: 211187
2014-06-18 17:13:57 +00:00
4601093267
R600: Implement f64 ftrunc, ffloor and fceil.
...
CI has instructions for these, so this fixes them for older hardware.
llvm-svn: 211183
2014-06-18 17:05:30 +00:00
e8208ec95b
R600: Custom lower f64 frint for pre-CI
...
llvm-svn: 211182
2014-06-18 17:05:26 +00:00
8579601050
R600/SI: Match ctlz_zero_undef
...
llvm-svn: 211115
2014-06-17 17:36:24 +00:00
880a80ad07
R600: Use LDS and vectors for private memory
...
llvm-svn: 211110
2014-06-17 16:53:14 +00:00
aad4659470
SelectionDAG: Expand i64 = FP_TO_SINT i32
...
llvm-svn: 211108
2014-06-17 16:53:07 +00:00
2a60de548a
Fix copy paste error
...
llvm-svn: 211003
2014-06-15 21:22:52 +00:00
717c1d0319
R600: Remove a few more things from AMDILISelLowering
...
Try to keep all the setOperationActions for integer ops
together.
llvm-svn: 211001
2014-06-15 21:08:58 +00:00
b5dff9ab50
R600: Fix assert on vector sdiv
...
llvm-svn: 211000
2014-06-15 21:08:54 +00:00
14d4645e46
R600: Move / cleanup more leftover AMDIL stuff.
...
llvm-svn: 210998
2014-06-15 20:23:38 +00:00
1578aa78d4
R600: Move division custom lowering out of AMDILISelLowering
...
llvm-svn: 210997
2014-06-15 20:08:02 +00:00
cf9a9a148e
R600: Report that integer division is expensive.
...
Divides by weird constants now emit much better code.
llvm-svn: 210995
2014-06-15 19:48:16 +00:00
e682a19a1c
R600: Fix asserts related to constant initializers
...
This would assert if a constant address space was extern
and therefore didn't have an initializer. If the initializer
was undef, it would hit the unreachable unhandled initializer case.
An extern global should never really occur since we don't have
machine linking, but bugpoint likes to remove initializers.
llvm-svn: 210967
2014-06-14 04:26:05 +00:00
41aa27c96b
R600: Use address space enum instead of value
...
llvm-svn: 210966
2014-06-14 04:26:01 +00:00
fd8c24ede8
R600: Cleanup some old AMDIL stuff.
...
Move / delete some of the more obviously wrong
setOperationAction calls. Most of these are setting Expand
for types that aren't legal which is the default anyway.
Leave stuff that might require more thought on whether it's
junk or not as it is.
No functionality change.
llvm-svn: 210922
2014-06-13 17:20:53 +00:00
825fb0b094
R600/SI: Fix selection error on i64 rotl / rotr.
...
Evergreen is still broken due to missing shl_parts.
llvm-svn: 210885
2014-06-13 04:00:30 +00:00
5d47d4ac7e
R600: Mostly remove remaining AMDIL intrinsics.
...
Delete all unused ones, and add new AMDGPU named intrinsics for
the ones that are. Handle the old AMDIL names for comptability (although
remove their GCCBuiltin names) and add tests since there weren't any
for these before.
llvm-svn: 210827
2014-06-12 21:15:44 +00:00
364a6747aa
R600/SI: Use v_cvt_f32_ubyte* instructions
...
This eliminates extra extract instructions when loading an i8 vector to
a float vector.
llvm-svn: 210666
2014-06-11 17:50:44 +00:00
ace0080a4a
Try to fix the msvc build.
...
llvm-svn: 210636
2014-06-11 04:41:37 +00:00
10da3b2516
Use cast instead of assert + dyn_cast
...
llvm-svn: 210628
2014-06-11 03:30:06 +00:00
c9df794042
R600: Add helper functions.
...
Extract these from some of my other patches, since this
is the only thing really making them dependent on each other.
llvm-svn: 210627
2014-06-11 03:29:54 +00:00
6042506b5c
R600: Use BCNT_INT for evergreen
...
llvm-svn: 210569
2014-06-10 19:18:28 +00:00
b5b5110b5c
R600/SI: Use bcnt instruction for ctpop
...
llvm-svn: 210567
2014-06-10 19:18:21 +00:00
6e43965fbc
R600: Handle fcopysign
...
llvm-svn: 210564
2014-06-10 19:00:20 +00:00
13ccc8f1bc
R600: Fix selection failure for vector bswap
...
llvm-svn: 210475
2014-06-09 16:20:25 +00:00
616a8e42b1
R600: Set all float vector expands in the same place
...
llvm-svn: 209988
2014-06-01 07:38:21 +00:00
05e96f4444
R600: Try to convert BFE back to standard bit ops when possible.
...
This allows existing DAG combines to work on them, and then
we can re-match to BFE if necessary during instruction selection.
llvm-svn: 209462
2014-05-22 18:09:12 +00:00
5565f65e13
R600: Add dag combine for BFE
...
llvm-svn: 209461
2014-05-22 18:09:07 +00:00
bf8694d36d
R600: Implement ComputeNumSignBitsForTargetNode for BFE
...
llvm-svn: 209460
2014-05-22 18:09:03 +00:00
af6df9d943
R600: Implement computeMaskedBitsForTargetNode for BFE
...
llvm-svn: 209459
2014-05-22 18:09:00 +00:00
eb260206c3
R600: Add intrinsics for mad24
...
llvm-svn: 209456
2014-05-22 18:00:15 +00:00
40100887b6
R600: Add comment describing problems with LowerConstantInitializer
...
llvm-svn: 209333
2014-05-21 22:59:17 +00:00
6a57fd8b47
R600: Partially fix constant initializers for structs and vectors.
...
This should extend the current workaround to work with structs
that only contain legal, scalar types.
llvm-svn: 209331
2014-05-21 22:42:42 +00:00
03df7eeda1
Use cast<> instead of unchecked dyn_cast
...
llvm-svn: 209310
2014-05-21 18:03:59 +00:00
d504a74e3c
Use range for
...
llvm-svn: 208922
2014-05-15 21:44:05 +00:00
a0653a3e6c
Rename ComputeMaskedBits to computeKnownBits. "Masked" has been
...
inappropriate since it lost its Mask parameter in r154011.
llvm-svn: 208811
2014-05-14 21:14:37 +00:00
62b1737081
R600: Add mul24 intrinsics
...
llvm-svn: 208604
2014-05-12 17:49:57 +00:00
46013d903f
Fix return before else
...
llvm-svn: 208510
2014-05-11 21:24:41 +00:00
a2acad785a
R600: Expand i64 SELECT_CC
...
llvm-svn: 208430
2014-05-09 16:42:19 +00:00
afa8b532b1
R600: Move MIN/MAX matching from LowerOperation() to PerformDAGCombine()
...
llvm-svn: 208429
2014-05-09 16:42:16 +00:00
e8a076a253
R600: Promote f64 vector load/stores to i64 for consistency
...
llvm-svn: 208344
2014-05-08 18:01:56 +00:00
45b3dcd35b
R600: Expand i64 ISD:SUB
...
llvm-svn: 208005
2014-05-05 21:47:15 +00:00
3dbf1f8df0
R600: Expand vector sin and cos.
...
v2: move code to AMDGPUISelLowering.cpp
squash with tests (both EG and SI)
Signed-off-by: Jan Vesely <jan.vesely@rutgers.edu>
llvm-svn: 207845
2014-05-02 15:41:47 +00:00
605e116e8e
R600: Expand TruncStore i64 -> {i16,i8}
...
llvm-svn: 207844
2014-05-02 15:41:46 +00:00
676f571999
R600: optimize the UDIVREM 64 algorithm
...
This is a squash of several optimization commits:
- calculate DIV_Lo and DIV_Hi separately
- use BFE_U32 if we are operating on 32bit values
- use precomputed constants instead of shifting in UDVIREM
- skip the first 32 iterations of udivrem
v2: Check whether BFE is supported before using it
Patch by: Jan Vesely
Signed-off-by: Jan Vesely <jan.vesely@rutgers.edu>
Reviewed-by: Tom Stellard <thomas.stellard@amd.com>
llvm-svn: 207589
2014-04-29 23:12:46 +00:00
bcd318fc76
R600: Implement iterative algorithm for udivrem
...
Initial implementation, rather slow
Patch by: Jan Vesely
Signed-off-by: Jan Vesely <jan.vesely@rutgers.edu>
Reviewed-by: Tom Stellard <thomas.stellard@amd.com>
llvm-svn: 207588
2014-04-29 23:12:45 +00:00
5f3378879f
R600: Change UDIV/UREM to UDIVREM when legalizing types
...
When legalizing ops, with UDIV/UREM set to expand, they automatically
expand to UDIVREM (if legal or custom).
We need to do this manually for legalize types.
v2:
SI should be set to Expand because the type is legal, and it is
automatically lowered to UDIVREM if UDIVREM is Legal/Custom
R600 should set to UDIV/UREM to Custom because it needs to lower them
during type legalization
Patch by: Jan Vesely
Signed-off-by: Jan Vesely <jan.vesely@rutgers.edu>
Reviewed-by: Tom Stellard <thomas.stellard@amd.com>
llvm-svn: 207587
2014-04-29 23:12:43 +00:00
df780303ef
R600: remove unused variable
...
Patch by: Jan Vesely
Signed-off-by: Jan Vesely <jan.vesely@rutgers.edu>
Reviewed-by: Tom Stellard <thomas.stellard@amd.com>
llvm-svn: 207586
2014-04-29 23:12:38 +00:00
8c0b4d0791
Convert more SelectionDAG functions to use ArrayRef.
...
llvm-svn: 207397
2014-04-28 05:57:50 +00:00
64941d9786
Convert SelectionDAG::getMergeValues to use ArrayRef.
...
llvm-svn: 207374
2014-04-27 19:20:57 +00:00
48d114bed1
Convert SelectionDAG::getNode methods to use ArrayRef<SDValue>.
...
llvm-svn: 207327
2014-04-26 18:35:24 +00:00
de1c3410c3
R600: Fix function name printing in LowerCall
...
v2: Check both ExternalSymbol and GlobalAddress
Patch by: Jan Vesely <jan.vesely@rutgers.edu>
llvm-svn: 207282
2014-04-25 22:22:01 +00:00
062a2baef0
[C++] Use 'nullptr'. Target edition.
...
llvm-svn: 207197
2014-04-25 05:30:21 +00:00
16353871c3
R600: Emit error instead of unreachable on function call
...
llvm-svn: 206904
2014-04-22 16:42:00 +00:00
a3c8cde77b
R600: Change how vector truncating stores are packed.
...
Don't introduce new operations on an illegal sub 32-bit type.
Do the operations on a 32-bit value, and then use a truncating store.
llvm-svn: 206864
2014-04-22 04:11:14 +00:00
5dbd5db518
R600: Make sign_extend_inreg legal.
...
Don't know why I didn't just do this in the first place.
llvm-svn: 206862
2014-04-22 03:49:30 +00:00
aeeea8a864
R600: Add comment clariying use of sext for result of MUL_U24
...
llvm-svn: 206501
2014-04-17 21:00:13 +00:00
4e46665a80
R600: Expand sign extension of vectors.
...
Setting vector types to expand will result in scalarization on pre SI hw,
as those gpus don't have vector shifts either.
Expand also i32 vectors, this helps llvm make the correct decision
about scalarizing the vector ops.
v2: move setOperation() calls to R600ISelLowering.cpp.
cleanup the SI code to make it obvious that this patch does is nop for SI
Patch by: Jan Vesely <jan.vesely@rutgers.edu>
llvm-svn: 206348
2014-04-16 01:41:30 +00:00
470acd81a8
R600/SI: Fix loads of i1
...
llvm-svn: 206330
2014-04-15 22:28:39 +00:00
aad475b324
Break PseudoSourceValue out of the Value hierarchy. It is now the root of its own tree containing FixedStackPseudoSourceValue (which you can use isa/dyn_cast on) and MipsCallEntry (which you can't). Anything that needs to use either a PseudoSourceValue* and Value* is strongly encouraged to use a MachinePointerInfo instead.
...
llvm-svn: 206255
2014-04-15 07:22:52 +00:00
9ec3cf2c8a
Move ExtractVectorElements to SelectionDAG.
...
This seems generally useful, and makes sense to
go along with SplitVector.
llvm-svn: 206041
2014-04-11 17:47:30 +00:00
50122a5890
R600: Match 24-bit arithmetic patterns in a Target DAGCombine
...
Moving these patterns from TableGen files to PerformDAGCombine()
should allow us to generate better code by eliminating unnecessary
shifts and extensions earlier.
This also fixes a bug where the MAD pattern was calling
SimplifyDemandedBits with a 24-bit mask on the first operand
even when the full pattern wasn't being matched. This occasionally
resulted in some instructions being incorrectly deleted from the
program.
v2:
- Fix bug with 64-bit mul
llvm-svn: 205731
2014-04-07 19:45:41 +00:00
7939acd7fa
Use .data() instead of &x[0]
...
llvm-svn: 205722
2014-04-07 16:44:24 +00:00
378bf9c68b
R600: Compute masked bits for min and max
...
llvm-svn: 205242
2014-03-31 19:35:33 +00:00
4c53717787
R600: Add BFE, BFI, and BFM intrinsics to help with writing tests.
...
llvm-svn: 205236
2014-03-31 18:21:18 +00:00
b34583661b
R600: Add target nodes for BFM and BFI
...
llvm-svn: 205235
2014-03-31 18:21:13 +00:00
b517c8128e
R600: Implement isZExtFree.
...
This allows 64-bit operations that are truncated to be reduced
to 32-bit ones.
llvm-svn: 204946
2014-03-27 17:23:31 +00:00
d125d74a73
R600/SI: Fix unreachable with a sext_in_reg to an illegal type.
...
llvm-svn: 204945
2014-03-27 17:23:24 +00:00
90b733a3cf
R600: Add a testcase for sext_in_reg I missed.
...
This sext_inreg i32 in i64 case was already handled, but not enabled.
llvm-svn: 204840
2014-03-26 18:31:06 +00:00
0c274feedf
R600: Move computeMaskedBitsForTargetNode out of AMDILISelLowering.cpp
...
Remove handling of select_cc, since it makes no sense to be there. This
now does nothing, but I'll be adding some handling of other target nodes
soon.
llvm-svn: 204743
2014-03-25 18:18:27 +00:00
a7f1e0c44f
R600: Implement isNarrowingProfitable.
...
llvm-svn: 204658
2014-03-24 19:43:31 +00:00
fae02989b7
R600: Match sign_extend_inreg to BFE instructions
...
llvm-svn: 204072
2014-03-17 18:58:11 +00:00
ea330fbe49
R600: Remove unnecessary attempt to zext a pointer.
...
Private pointers are now always 32-bits.
llvm-svn: 203989
2014-03-15 00:08:26 +00:00
74891cdefe
R600: Code cleanup.
...
Use sign_extend_inreg and getZeroExtendInReg instead of
using the bit operations they expand into.
llvm-svn: 203988
2014-03-15 00:08:22 +00:00
e389dd5d68
R600: Fix trunc store from i64 to i1
...
llvm-svn: 203695
2014-03-12 18:45:52 +00:00
0211714ecb
R600: Calculate store mask instead of using switch.
...
llvm-svn: 203527
2014-03-11 01:38:53 +00:00
9504d2f269
Use .data() instead of &x[0]
...
llvm-svn: 203516
2014-03-11 00:01:31 +00:00
f9a995d68c
R600: Fix extloads from i8 / i16 to i64.
...
This appears to only be working for global loads. Private
and local break for other reasons.
llvm-svn: 203135
2014-03-06 17:34:12 +00:00
9fe669c522
R600/SI: Expand selects on vectors.
...
llvm-svn: 203134
2014-03-06 17:34:03 +00:00
ca6dcfcf59
Fix typo
...
llvm-svn: 203013
2014-03-05 21:47:22 +00:00
41e2f2bacd
R600/SI - Add new CI arithmetic instructions.
...
Does not yet include larger part required
to match v_mad_i64_i32 / v_mad_u64_u32.
llvm-svn: 202077
2014-02-24 21:01:28 +00:00
21a3faaf25
Fix DOT4 missing from getTargetOpcodeName
...
llvm-svn: 202075
2014-02-24 21:01:21 +00:00
967bf5813f
R600/SI: Expand all v8[if]32 operations
...
llvm-svn: 201371
2014-02-13 23:34:15 +00:00
53f9df4c93
R600: Always implement both versions of isTruncateFree and add a sanity check.
...
llvm-svn: 201222
2014-02-12 10:17:54 +00:00
0cdcd961bf
R600: Implement isTruncateFree
...
Truncation is just accessing a subregister for any multiple of
the register size, so it's free.
llvm-svn: 201107
2014-02-10 19:57:42 +00:00
aeb456438c
R600/SI: Expand i1 BR_CC
...
This fixes a crashes in the OpenCV test suite and also the scrypt
kernel in bfgminer.
I was unable to come up with a reduced test case for this.
https://bugs.freedesktop.org/show_bug.cgi?id=72785
llvm-svn: 200776
2014-02-04 17:18:43 +00:00
bfebd1fc7e
R600: Enable vector fpow.
...
The OpenCL specs say: "The vector versions of the math functions operate
component-wise. The description is per-component."
Patch by: Jan Vesely
Reviewed-by: Tom Stellard <thomas.stellard@amd.com>
Signed-off-by: Jan Vesely <jan.vesely@rutgers.edu>
llvm-svn: 200773
2014-02-04 17:18:37 +00:00
04c0e9851b
R600: Add support for global addresses with constant initializers
...
llvm-svn: 199825
2014-01-22 19:24:21 +00:00
e93736057f
R600/SI: Add support for i8 and i16 private loads/stores
...
llvm-svn: 199823
2014-01-22 19:24:14 +00:00
eddfa69465
R600: Allow ftrunc
...
v2: Add ftrunc->TRUNC pattern instead of replacing int_AMDGPU_trunc
v3: move ftrunc pattern next to TRUNC definition, it's available since R600
Patch By: Jan Vesely
Reviewed-by: Tom Stellard <thomas.stellard@amd.com>
Signed-off-by: Jan Vesely <jan.vesely@rutgers.edu>
llvm-svn: 197783
2013-12-20 05:11:55 +00:00
52226f9a8e
Don't manually calculate size in bytes
...
llvm-svn: 197327
2013-12-14 18:21:59 +00:00
eaa3a7efab
Use llvm_unreachable instead of assert(0)
...
llvm-svn: 196971
2013-12-10 21:37:42 +00:00
175e7a8c97
R600: Expand vector FABS
...
NOTE: This is a candidate for the 3.4 branch.
llvm-svn: 195881
2013-11-27 21:23:39 +00:00
4d566b2edf
R600: Add support for ISD::FROUND
...
NOTE: This is a candidate for the 3.4 branch.
llvm-svn: 195878
2013-11-27 21:23:20 +00:00
c5559bb14b
Add target hook to prevent folding some bitcasted loads.
...
This is to avoid this transformation in some cases:
fold (conv (load x)) -> (load (conv*)x)
On architectures that don't natively support some vector
loads efficiently casting the load to a smaller vector of
larger types and loading is more efficient.
Patch by Micah Villmow.
llvm-svn: 194783
2013-11-15 04:42:23 +00:00
81d871dee3
R600/SI: Add support for private address space load/store
...
Private address space is emulated using the register file with
MOVRELS and MOVRELD instructions.
llvm-svn: 194626
2013-11-13 23:36:50 +00:00
Matt Arsenault