863d2d3235
Removed unused variable
...
llvm-svn: 159197
2012-06-26 10:50:07 +00:00
8ed44466c2
Rename to match other X86_64* names.
...
llvm-svn: 159196
2012-06-26 10:05:06 +00:00
26088d2e24
Shuffle optimization for AVX/AVX2.
...
The current patch optimizes frequently used shuffle patterns and gives these instruction sequence reduction.
Before:
vshufps $-35, %xmm1, %xmm0, %xmm2 ## xmm2 = xmm0[1,3],xmm1[1,3]
vpermilps $-40, %xmm2, %xmm2 ## xmm2 = xmm2[0,2,1,3]
vextractf128 $1, %ymm1, %xmm1
vextractf128 $1, %ymm0, %xmm0
vshufps $-35, %xmm1, %xmm0, %xmm0 ## xmm0 = xmm0[1,3],xmm1[1,3]
vpermilps $-40, %xmm0, %xmm0 ## xmm0 = xmm0[0,2,1,3]
vinsertf128 $1, %xmm0, %ymm2, %ymm0
After:
vshufps $13, %ymm0, %ymm1, %ymm1 ## ymm1 = ymm1[1,3],ymm0[0,0],ymm1[5,7],ymm0[4,4]
vshufps $13, %ymm0, %ymm0, %ymm0 ## ymm0 = ymm0[1,3,0,0,5,7,4,4]
vunpcklps %ymm1, %ymm0, %ymm0 ## ymm0 = ymm0[0],ymm1[0],ymm0[1],ymm1[1],ymm0[4],ymm1[4],ymm0[5],ymm1[5]
llvm-svn: 159188
2012-06-26 08:04:10 +00:00
bbcd09cc00
Make some ugly hacks for inline asm operands which name a specific register a bit more thorough. PR13196.
...
llvm-svn: 159176
2012-06-25 23:42:33 +00:00
2e22e6a361
%RCX is not a function live-out in eh.return functions.
...
The function live-out registers must be live at all function returns,
and %RCX is only used by eh.return. When a function also has a normal
return, only %RAX holds a return value.
This fixes PR13188.
llvm-svn: 159116
2012-06-24 15:53:01 +00:00
3c680dec8a
Remove code i'd been testing with but didn't mean to commit. Oops
...
llvm-svn: 159094
2012-06-24 00:08:36 +00:00
fe212e762f
DAG legalisation can now handle illegal fma vector types by scalarisation
...
llvm-svn: 159092
2012-06-24 00:05:44 +00:00
a3088f09b3
Handle aliases to tls variables in all architectures, not just x86.
...
llvm-svn: 159058
2012-06-23 00:30:03 +00:00
b9e8e18949
Don't insert 128-bit UNDEF into 256-bit vectors. Just keep the 256-bit vector. Original patch by Elena Demikhovsky. Tweaked by me to allow possibility of covering more cases.
...
llvm-svn: 158792
2012-06-20 05:39:26 +00:00
ca3e0ee8b3
Move the support for using .init_array from ARM to the generic
...
TargetLoweringObjectFileELF. Use this to support it on X86. Unlike ARM,
on X86 it is not easy to find out if .init_array should be used or not, so
the decision is made via TargetOptions and defaults to off.
Add a command line option to llc that enables it.
llvm-svn: 158692
2012-06-19 00:48:28 +00:00
a54893c662
Use XOP vpcom intrinsics in patterns instead of a target specific SDNode type. Remove the custom lowering code that selected the SDNode type.
...
llvm-svn: 158279
2012-06-09 17:02:24 +00:00
3352ba55b9
Replace XOP vpcom intrinsics with fewer intrinsics that take the immediate as an argument.
...
llvm-svn: 158278
2012-06-09 16:46:13 +00:00
6bc2d27073
Enable optimization for integer ABS on X86 if Subtarget has CMOV.
...
llvm-svn: 158220
2012-06-08 18:58:26 +00:00
2cdc8afccf
X86: optimize generated code for integer ABS
...
This patch will generate the following for integer ABS:
movl %edi, %eax
negl %eax
cmovll %edi, %eax
INSTEAD OF
movl %edi, %ecx
sarl $31, %ecx
leal (%rdi,%rcx), %eax
xorl %ecx, %eax
There exists a target-independent DAG combine for integer ABS, which converts
integer ABS to sar+add+xor. For X86, we match this pattern back to neg+cmov.
This is implemented in PerformXorCombine.
rdar://10695237
llvm-svn: 158175
2012-06-07 22:39:10 +00:00
bbd40f67d8
Do not optimize the used bits of the x86 vselect condition operand, when the condition operand is a vector of 1-bit predicates.
...
This may happen on MIC devices.
llvm-svn: 158168
2012-06-07 20:53:48 +00:00
746e4859d0
PR13046: we can't replace usage of SUB with CMP in the lowering phase.
...
It will cause assertion failure later on.
llvm-svn: 158160
2012-06-07 19:27:33 +00:00
ae02c5a93e
X86: replace SUB with CMP if possible
...
This patch will optimize the following
movq %rdi, %rax
subq %rsi, %rax
cmovsq %rsi, %rdi
movq %rdi, %rax
to
cmpq %rsi, %rdi
cmovsq %rsi, %rdi
movq %rdi, %rax
Perform this optimization if the actual result of SUB is not used.
rdar: 11540023
llvm-svn: 158126
2012-06-07 00:42:47 +00:00
bde9176663
Fix typos found by http://github.com/lyda/misspell-check
...
llvm-svn: 157885
2012-06-02 10:20:22 +00:00
789acfb63d
Implement the local-dynamic TLS model for x86 (PR3985)
...
This implements codegen support for accesses to thread-local variables
using the local-dynamic model, and adds a clean-up pass so that the base
address for the TLS block can be re-used between local-dynamic access on
an execution path.
llvm-svn: 157818
2012-06-01 16:27:21 +00:00
4f203ea34b
Add support for return value promotion in X86 calling conventions.
...
Patch by Yiannis Tsiouris!
llvm-svn: 157757
2012-05-31 17:28:20 +00:00
aa58397b3c
Change interface for TargetLowering::LowerCallTo and TargetLowering::LowerCall
...
to pass around a struct instead of a large set of individual values. This
cleans up the interface and allows more information to be added to the struct
for future targets without requiring changes to each and every target.
NV_CONTRIB
llvm-svn: 157479
2012-05-25 16:35:28 +00:00
53b4b73be9
Fix constant used for pshufb mask when lowering v16i8 shuffles. Bug introduced in r157043. Fixes PR12908.
...
llvm-svn: 157236
2012-05-22 06:09:38 +00:00
e88f2fd4f7
Allow 256-bit shuffles to still be split even if only half of the shuffle comes from two 128-bit pieces.
...
llvm-svn: 157175
2012-05-21 06:40:16 +00:00
c93e91da27
On Haswell, perfer storing YMM registers using a single instruction.
...
llvm-svn: 157129
2012-05-19 20:30:08 +00:00
900c7cb7ce
Add support for additional in-reg vbroadcast patterns
...
llvm-svn: 157127
2012-05-19 19:57:37 +00:00
0cf4038c59
Simplify code a bit. No functional change intended.
...
llvm-svn: 157044
2012-05-18 07:07:36 +00:00
92db928ee9
Simplify handling of v16i8 shuffles and fix a missed optimization.
...
llvm-svn: 157043
2012-05-18 06:42:06 +00:00
f9d0e44b82
Implement initial-exec TLS model for 32-bit PIC x86
...
This fixes a TODO from 2007 :) Previously, LLVM would emit the wrong
code here (see the update to test/CodeGen/X86/tls-pie.ll).
llvm-svn: 156611
2012-05-11 10:11:01 +00:00
1a65397017
Fix merge-typo and cleanup
...
llvm-svn: 156541
2012-05-10 12:50:02 +00:00
15946e50c1
AVX2: Add an additional broadcast idiom.
...
llvm-svn: 156540
2012-05-10 12:39:13 +00:00
b86a3fb8d0
Generate AVX/AVX2 shuffles even when there is a memory op somewhere else in the program.
...
Starting r155461 we are able to select patterns for vbroadcast even when the load op is used by other users.
Fix PR11900.
llvm-svn: 156539
2012-05-10 12:22:05 +00:00
d8287fec17
Fix a regression from r147481. This combine should only happen if there is a
...
single use.
rdar://11360370
llvm-svn: 156316
2012-05-07 18:47:44 +00:00
ef4e0479ec
X86: optimization for -(x != 0)
...
This patch will optimize -(x != 0) on X86
FROM
cmpl $0x01,%edi
sbbl %eax,%eax
notl %eax
TO
negl %edi
sbbl %eax %eax
In order to generate negl, I added patterns in Target/X86/X86InstrCompiler.td:
def : Pat<(X86sub_flag 0, GR32:$src), (NEG32r GR32:$src)>;
rdar: 10961709
llvm-svn: 156312
2012-05-07 18:06:23 +00:00
00a1e6d48b
Use MVT instead of EVT as the argument to all the shuffle decode functions. Simplify some of the decode functions.
...
llvm-svn: 156268
2012-05-06 19:46:21 +00:00
804be3b546
Add VPERMQ/VPERMPD to the list of target specific shuffles that can be looked through for DAG combine purposes.
...
llvm-svn: 156266
2012-05-06 18:54:26 +00:00
e31f31e5c0
Add a new target hook "predictableSelectIsExpensive".
...
This will be used to determine whether it's profitable to turn a select into a
branch when the branch is likely to be predicted.
Currently enabled for everything but Atom on X86 and Cortex-A9 devices on ARM.
I'm not entirely happy with the name of this flag, suggestions welcome ;)
llvm-svn: 156233
2012-05-05 12:49:14 +00:00
bdd2e34b1f
Fix some loops to match coding standards. No functional change intended.
...
llvm-svn: 156159
2012-05-04 06:39:13 +00:00
d4d3237bb8
Fix up some spacing. No functional change.
...
llvm-svn: 156158
2012-05-04 06:18:33 +00:00
e2ae413746
Simplify broadcast lowering code. No functional change intended.
...
llvm-svn: 156157
2012-05-04 05:49:51 +00:00
42f2182366
Allow v16i16 and v32i8 shuffles to be rewritten as narrower shuffles.
...
llvm-svn: 156156
2012-05-04 04:44:49 +00:00
59063c0a3d
Simplify shuffle narrowing code a bit. No functional change intended.
...
llvm-svn: 156154
2012-05-04 04:08:44 +00:00
242183834a
Use 'unsigned' instead of 'int' in a few places dealing with counts of vector elements.
...
llvm-svn: 156060
2012-05-03 07:26:59 +00:00
315a5cc789
Fix 256-bit vpshuflw and vpshufhw immediate encoding to handle undefs in the lower half correctly. Missed in r155982.
...
llvm-svn: 156059
2012-05-03 07:12:59 +00:00
926afd7401
For Intel Atom, use ILP scheduling always, instead of ILP for 64 bit
...
and Hybrid for 32 bit, since benchmarks show ILP scheduling is better
most of the time.
llvm-svn: 156028
2012-05-02 22:02:02 +00:00
f02efc8731
Revert r155853
...
The commit is intended to fix rdar://10961709.
But it is the root cause of PR12720.
Revert it for now.
llvm-svn: 155992
2012-05-02 15:24:32 +00:00
c73bc39c22
Add support for selecting AVX2 vpshuflw and vpshufhw. Add decoding support for AsmPrinter.
...
llvm-svn: 155982
2012-05-02 08:03:44 +00:00
425a55c1ce
X86: optimization for max-like struct
...
This patch will optimize the following cases on X86
(a > b) ? (a-b) : 0
(a >= b) ? (a-b) : 0
(b < a) ? (a-b) : 0
(b <= a) ? (a-b) : 0
FROM
movl %edi, %ecx
subl %esi, %ecx
cmpl %edi, %esi
movl $0, %eax
cmovll %ecx, %eax
TO
xorl %eax, %eax
subl %esi, %edi
cmovll %eax, %edi
movl %edi, %eax
rdar: 10734411
llvm-svn: 155919
2012-05-01 17:16:15 +00:00
4f4d5c8fc8
X86: optimization for -(x != 0)
...
This patch will optimize -(x != 0) on X86
FROM
cmpl $0x01,%edi
sbbl %eax,%eax
notl %eax
TO
negl %edi
sbbl %eax %eax
llvm-svn: 155853
2012-04-30 22:51:25 +00:00
d427d51c2b
Tidy up. No functional change intended.
...
llvm-svn: 155832
2012-04-30 17:47:15 +00:00
55b3990837
No need to normalize index before calling Extract128BitVector
...
llvm-svn: 155811
2012-04-30 05:17:10 +00:00
da03f3ba64
Remove unneeded casts. No functionality change.
...
llvm-svn: 155800
2012-04-29 20:52:53 +00:00
3b94fa63d6
Simplify code a bit. No functional change intended.
...
llvm-svn: 155798
2012-04-29 20:22:05 +00:00
0fa6c7e593
Use 'unsigned' instead of 'int' in several places when retrieving number of vector elements.
...
llvm-svn: 155742
2012-04-27 22:54:43 +00:00
32c2178ef3
Add x86-specific DAG combine to simplify:
...
x == -y --> x+y == 0
x != -y --> x+y != 0
On x86, the generated code goes from
negl %esi
cmpl %esi, %edi
je .LBB0_2
to
addl %esi, %edi
je .L4
This case is correctly handled for ARM with "cmn".
Patch by Manman Ren.
rdar://11245199
PR12545
llvm-svn: 155739
2012-04-27 22:33:25 +00:00
42cd8d2c00
Tidy up spacing.
...
llvm-svn: 155733
2012-04-27 21:05:09 +00:00
913da4b261
X86: Don't emit conditional floating point moves on when targeting pre-pentiumpro architectures.
...
* Model FPSW (the FPU status word) as a register.
* Add ISel patterns for the FUCOM*, FNSTSW and SAHF instructions.
* During Legalize/Lowering, build a node sequence to transfer the comparison
result from FPSW into EFLAGS. If you're wondering about the right-shift: That's
an implicit sub-register extraction (%ax -> %ah) which is handled later on by
the instruction selector.
Fixes PR6679. Patch by Christoph Erhardt!
llvm-svn: 155704
2012-04-27 12:07:43 +00:00
5ff6dc34b9
Use vector_shuffles instead of target specific unpack nodes for AVX ZERO_EXTEND/ANY_EXTEND combine. These will be converted to target specific nodes during lowering. This is more consistent with other code.
...
llvm-svn: 155537
2012-04-25 06:39:39 +00:00
7b7b99c74a
AVX2: The BLENDPW instruction selects between vectors of v16i16 using an i8
...
immediate. We can't use it here because the shuffle code does not check that
the lower part of the word is identical to the upper part.
llvm-svn: 155440
2012-04-24 11:27:53 +00:00
0b65c40821
Remove dangling spaces. Fix some other formatting.
...
llvm-svn: 155429
2012-04-24 06:36:35 +00:00
6f2a535de2
Simplify code a bit and make it compile better. Remove unused parameters.
...
llvm-svn: 155428
2012-04-24 06:02:29 +00:00
3f8acfc3c4
Optimize the vector UINT_TO_FP, SINT_TO_FP and FP_TO_SINT operations where the integer type is i8 (commonly used in graphics).
...
llvm-svn: 155397
2012-04-23 21:53:37 +00:00
153bb34a3c
Use MVT instead of EVT through all of LowerVECTOR_SHUFFLEtoBlend and not just the switch. Saves a little bit of binary size.
...
llvm-svn: 155339
2012-04-23 07:36:33 +00:00
0a2c809d09
Make getZeroVector and getOnesVector more alike as far as how they detect 128-bit versus 256-bit vectors. Be explicit about both sizes and use llvm_unreachable. Similar changes to getLegalSplat.
...
llvm-svn: 155337
2012-04-23 07:24:41 +00:00
2bbe8bcf4e
Tidy up by removing some 'else' after 'return'
...
llvm-svn: 155336
2012-04-23 06:57:04 +00:00
5c51eeecfc
Tidy up spacing in LowerVECTOR_SHUFFLEtoBlend. Remove code that checks if shuffle operand has a different type than the the shuffle result since it can never happen.
...
llvm-svn: 155333
2012-04-23 06:38:28 +00:00
a52f0d09b6
Add a couple llvm_unreachables.
...
llvm-svn: 155332
2012-04-23 03:42:40 +00:00
984dc015ae
Remove some tab characers.
...
llvm-svn: 155331
2012-04-23 03:28:34 +00:00
ea428fd79c
Remove some 'else' after 'return'. No functional change.
...
llvm-svn: 155330
2012-04-23 03:26:18 +00:00
bf7d5666f0
Make Extract128BitVector and Insert128BitVector take an unsigned instead of an ConstantNode SDValue. getConstant was almost always called just before only to have the functions take it apart and build a new ConstantSDNode.
...
llvm-svn: 155325
2012-04-22 20:55:18 +00:00
2d474d6d92
Convert getNode(UNDEF) to getUNDEF.
...
llvm-svn: 155321
2012-04-22 19:29:34 +00:00
860ed0d20a
Make calls to getVectorShuffle more consistent. Use shuffle VT for calls to getUNDEF instead of requerying. Use &Mask[0] instead of Mask.data().
...
llvm-svn: 155320
2012-04-22 19:17:57 +00:00
43397c0900
Tidy up. 80 columns and argument alignment.
...
llvm-svn: 155319
2012-04-22 18:51:37 +00:00
ad56a744f1
Simplify code by converting multiple places that were manually concatenating 128-bit vectors to use either CONCAT_VECTORS or a helper function. CONCAT_VECTORS will itself be lowered to the same pattern as before. The helper function is needed for concats of BUILD_VECTORs since getNode(CONCAT_VECTORS) will just return a large BUILD_VECTOR and we may be trying to lower large BUILD_VECTORS when this occurs.
...
llvm-svn: 155318
2012-04-22 18:15:59 +00:00
8d7e56c409
ZERO_EXTEND/SIGN_EXTEND/TRUNCATE optimization for AVX2
...
llvm-svn: 155309
2012-04-22 09:39:03 +00:00
6eadae8e60
Make some fixed arrays const. Use array_lengthof in a couple places instead of a hardcoded number.
...
llvm-svn: 155294
2012-04-21 18:58:38 +00:00
2568bf3089
Tidy up. 80 columns and some other spacing issues.
...
llvm-svn: 155291
2012-04-21 18:13:35 +00:00
abadc660e0
Convert some uses of XXXRegisterClass to &XXXRegClass. No functional change since they are equivalent.
...
llvm-svn: 155186
2012-04-20 06:31:50 +00:00
d3c9e404ba
Remove AVX vpermil intrinsics. I removed their uses from clang headers and builtins a while back.
...
llvm-svn: 154985
2012-04-18 05:24:00 +00:00
354103d8ca
Don't decode vperm2i128 or vperm2f128 into a shuffle if bit 3 or 7 of the immediate is set.
...
llvm-svn: 154907
2012-04-17 05:54:54 +00:00
12da79b859
Fix incorrect atomics codegen introduced in r154705, and extend test to catch it.
...
llvm-svn: 154845
2012-04-16 18:43:53 +00:00
4badeb3f0d
Replace vpermd/vpermps intrinic patterns with custom lowering to target specific nodes.
...
llvm-svn: 154801
2012-04-16 07:13:00 +00:00
26d7a94981
Change type profile for vpermv back to using operand type for the mask argument to match intrinsic behavior. Add a bitcast to the lowering code to convert mask from v8i32 to v8f32 for vpermps.
...
llvm-svn: 154798
2012-04-16 06:43:40 +00:00
b86fa404d3
Merge vpermps/vpermd and vpermpd/vpermq SD nodes.
...
llvm-svn: 154782
2012-04-16 00:41:45 +00:00
1f8c9eb925
Spacing fixes and 80 column fixes. Use 0 instead of 0x80 for undef indices in vpermps/vpermd. Hardware only looks at lower 3-bits.
...
llvm-svn: 154780
2012-04-15 23:48:57 +00:00
779a72b49e
Added VPERM optimization for AVX2 shuffles
...
llvm-svn: 154761
2012-04-15 11:18:59 +00:00
3e8f1f6aea
Fix X86 codegen for 'atomicrmw nand' to generate *x = ~(*x & y), not *x = ~*x & y.
...
llvm-svn: 154705
2012-04-13 22:47:00 +00:00
372cf15125
remove unused argument
...
llvm-svn: 154494
2012-04-11 11:05:21 +00:00
9bc178ac5c
Reapply 154396 after fixing a test.
...
Original message:
Modify the code that lowers shuffles to blends from using blendvXX to vblendXX.
blendV uses a register for the selection while Vblend uses an immediate.
On sandybridge they still have the same latency and execute on the same execution ports.
llvm-svn: 154483
2012-04-11 06:40:27 +00:00
f7345b027a
Whitespace.
...
llvm-svn: 154427
2012-04-10 19:42:07 +00:00
235a7a1746
Revert r154396, which looks to be the real culprit behind the bot failures.
...
llvm-svn: 154426
2012-04-10 19:39:18 +00:00
65ada95b84
Temporarily revert this patch to see if it brings the buildbots back.
...
llvm-svn: 154425
2012-04-10 19:33:16 +00:00
2735136655
Remove unused variable.
...
llvm-svn: 154398
2012-04-10 15:23:13 +00:00
f934f91709
Modify the code that lowers shuffles to blends from using blendvXX to vblendXX.
...
blendv uses a register for the selection while vblend uses an immediate.
On sandybridge they still have the same latency and execute on the same execution ports.
llvm-svn: 154396
2012-04-10 14:33:13 +00:00
f8bad08001
Fix a long standing tail call optimization bug. When a libcall is emitted
...
legalizer always use the DAG entry node. This is wrong when the libcall is
emitted as a tail call since it effectively folds the return node. If
the return node's input chain is not the entry (i.e. call, load, or store)
use that as the tail call input chain.
PR12419
rdar://9770785
rdar://11195178
llvm-svn: 154370
2012-04-10 01:51:00 +00:00
fb7e2ae53c
Lower some x86 shuffle sequences to the vblend family of instructions.
...
llvm-svn: 154313
2012-04-09 08:33:21 +00:00
b801ca3976
Fix a bug in the lowering of broadcasts: ConstantPools need to use the target pointer type.
...
Move NormalizeVectorShuffle and LowerVectorBroadcast into X86TargetLowering.
llvm-svn: 154310
2012-04-09 07:45:58 +00:00
16f0ebcbb5
Move the TLSModel information into the TargetMachine rather than hiding
...
in TargetLowering. There was already a FIXME about this location being
odd. The interface is simplified as a consequence. This will also make
it easier to change TLS models when compiling with PIE.
llvm-svn: 154292
2012-04-08 17:20:55 +00:00
82609df647
AVX2: Build splat vectors by broadcasting a scalar from the constant pool.
...
Previously we used three instructions to broadcast an immediate value into a
vector register.
On Sandybridge we continue to load the broadcasted value from the constant pool.
llvm-svn: 154284
2012-04-08 12:54:54 +00:00
3cacabfb04
Fix narrowing conversion.
...
llvm-svn: 154171
2012-04-06 13:33:52 +00:00
447417c932
Allow 256-bit shuffles to be split if a 128-bit lane contains elements from a single source. This is a rewrite of the 256-bit shuffle splitting code based on similar code from legalize types. Fixes PR12413.
...
llvm-svn: 154166
2012-04-06 07:45:23 +00:00
Elena Demikhovsky