Commit Graph

960 Commits

Author SHA1 Message Date
Artem Belevich 91cc00bde6 [CUDA] Added __hmma_m16n16k16_* builtins to support mma instructions on sm_70
Differential Revision: https://reviews.llvm.org/D38742

llvm-svn: 315624
2017-10-12 21:32:19 +00:00
Craig Topper 8c8e83a15f [X86] Add support for 'amdfam17h' to __builtin_cpu_is to match gcc.
The compiler-rt implementation already supported it, it just wasn't exposed.

llvm-svn: 315517
2017-10-11 21:42:02 +00:00
Matt Arsenault f12e3b848a AMDGPU: Add read_exec_lo/hi builtins
llvm-svn: 315238
2017-10-09 20:06:37 +00:00
Erich Keane 1fe643a6d7 Split X86::BI__builtin_cpu_init handling into own function[NFC]
The Cpu Init functionality is required for the target
attribute, so this patch simply splits it out into its own
function, exactly like CpuIs and CpuSupports.

llvm-svn: 315075
2017-10-06 16:40:45 +00:00
Akira Hatanaka a46381286f Fix check strings in test case and use llvm::to_string instead of
std::to_string.

These changes were needed to fix bots that started failing after
r315045.

llvm-svn: 315046
2017-10-06 07:47:47 +00:00
Akira Hatanaka 6b103bc18c [CodeGen] Emit a helper function for __builtin_os_log_format to reduce
code size.

Currently clang expands a call to __builtin_os_log_format into a long
sequence of instructions at the call site, causing code size to
increase in some cases.

This commit attempts to reduce code size by emitting a helper function
that can be shared by calls to __builtin_os_log_format with similar
formats and arguments. The helper function has linkonce_odr linkage to
enable the linker to merge identical functions across translation units.
Attribute 'noinline' is attached to the helper function at -Oz so that
the inliner doesn't inline functions that can potentially be merged.

This commit also fixes a bug where the generated IR writes past the end
of the buffer when "%m" is the last specifier appearing in the format
string passed to __builtin_os_log_format.

Original patch by Duncan Exon Smith.

rdar://problem/34065973
rdar://problem/34196543

Differential Revision: https://reviews.llvm.org/D38606

llvm-svn: 315045
2017-10-06 07:12:46 +00:00
Artem Belevich bab95c7087 [NVPTX] added match.{any,all}.sync instructions, intrinsics & builtins.
Differential Revision: https://reviews.llvm.org/D38191

llvm-svn: 314223
2017-09-26 17:07:23 +00:00
Justin Lebar d31d5e6aa2 Revert "[NVPTX] added match.{any,all}.sync instructions, intrinsics & builtins.", rL314135.
Causing assertion failures on macos:

> Assertion failed: (Num < NumOperands && "Invalid child # of SDNode!"),
> function getOperand, file
> /Users/buildslave/jenkins/workspace/clang-stage1-cmake-RA-incremental/llvm/include/llvm/CodeGen/SelectionDAGNodes.h,
> line 835.

http://green.lab.llvm.org/green/job/clang-stage1-cmake-RA-incremental/42739/testReport/LLVM/CodeGen_NVPTX/surf_read_cuda_ll/

llvm-svn: 314142
2017-09-25 19:41:56 +00:00
Artem Belevich 9941ee9529 [NVPTX] added match.{any,all}.sync instructions, intrinsics & builtins.
Differential Revision: https://reviews.llvm.org/D38191

llvm-svn: 314135
2017-09-25 18:53:57 +00:00
Heejin Ahn b29a17ba21 [WebAssembly] Restore __builtin_wasm_rethrow builtin
Summary:
Restore the `__builtin_wasm_rethrow` builtin deleted in D37931. On second
thought, it appears it can be used to implement `__cxa_rethrow`.

Reviewers: dschuff, sunfish

Reviewed By: dschuff

Subscribers: jfb, sbc100, jgravelle-google

Differential Revision: https://reviews.llvm.org/D37942

llvm-svn: 313430
2017-09-16 01:07:43 +00:00
Craig Topper 8cd7b0cd2c [X86] Use native shuffle vector for the perm2f128 intrinsics
This patch replaces the perm2f128 intrinsics with native shuffle vectors.

This uses a pretty simple approach to allocate source 0 to the lower half input and source 1 to the upper half input. Then its just a matter of filling in the indices to use either the lower or upper half of that specific source. This can result in the same source being used by both operands. InstCombine or SelectionDAGBuilder should be able to clean that up.

Differential Revision: https://reviews.llvm.org/D37892

llvm-svn: 313418
2017-09-15 23:00:59 +00:00
Heejin Ahn fa9e1fba8c Remove __builtin_wasm_rethrow builtin
Summary:
Remove `__builtin_wasm_rethrow` builtin. I thought it was required to implement
`__cxa_rethrow` function in libcxxabi, but it turned out it will be using
`__builtin_wasm_throw` instead.

Reviewers: dschuff, jgravelle-google

Reviewed By: jgravelle-google

Subscribers: jfb, sbc100, jgravelle-google

Differential Revision: https://reviews.llvm.org/D37931

llvm-svn: 313402
2017-09-15 22:01:22 +00:00
Uriel Korach 3fba3c3b0c [X86] [PATCH] [intrinsics] Lowering X86 ABS intrinsics to IR. (clang)
This patch, together with a matching llvm patch (https://reviews.llvm.org/D37693), implements the lowering of X86 ABS intrinsics to IR.

Differential Revision: https://reviews.llvm.org/D37694

llvm-svn: 313133
2017-09-13 09:02:02 +00:00
Jan Vesely 31ecb4bf60 [OpenCL] Add half load and store builtins
This enables load/stores of half type, without half being a legal type.

Differential Revision: https://reviews.llvm.org/D37231

llvm-svn: 312742
2017-09-07 19:39:10 +00:00
Reid Kleckner d53c39ba46 Commit changes missing from r312572
llvm-svn: 312573
2017-09-05 20:38:29 +00:00
Reid Kleckner 30701edf76 [ms] Implement the __annotation intrinsic
llvm-svn: 312572
2017-09-05 20:27:35 +00:00
Yaxun Liu 29a5ee358e [OpenCL] Do not use vararg in emitted functions for enqueue_kernel
Not all targets support vararg (e.g. amdgpu). Instead of using vararg in the emitted functions for enqueue_kernel,
this patch creates a temporary array of size_t, stores the size arguments in the temporary array
and passes it to the emitted functions for enqueue_kernel.

Differential Revision: https://reviews.llvm.org/D36678

llvm-svn: 312441
2017-09-03 13:52:24 +00:00
Erich Keane 9937b134c5 [CodeGen]Refactor CpuSupports/CPUIs Builtin Code Gen to better work with
"target" implementation

A small set of refactors that'll make it easier for me to implement 'target' 
support.

First, extract the CPUSupports functionality into its own function. 
THis has the advantage of not wasting time in this builtin to deal with 
arguments.
Second, pulls both CPUSupports and CPUIs implementation into a member-function, 
so that it can be called from the resolver generation that I'm working on.
Third, creates an overload that takes simply the feature/cpu name (rather than 
extracting it from a callexpr), since that info isn't available later.

Note that despite how the 'diff' looks, the EmitX86CPUSupports function simply 
takes the implementation out of the 'switch'.

llvm-svn: 312355
2017-09-01 19:42:45 +00:00
Craig Topper 2c03e53f4e [X86] Add support for __builtin_cpu_init
This adds builtin_cpu_init which will emit a call to cpu_indicator_init in libgcc or compiler-rt.

This is needed to support builtin_cpu_supports/builtin_cpu_is in an ifunc resolver.

Differential Revision: https://reviews.llvm.org/D36336

llvm-svn: 311874
2017-08-28 05:43:23 +00:00
John McCall de0fe07eef Extract IRGen's constant-emitter into its own helper class and clean up
the interface.

The ultimate goal here is to make it easier to do some more interesting
things in constant emission, like emit constant initializers that have
ignorable side-effects, or doing the majority of an initialization
in-place and then patching up the last few things with calls.  But for
now this is mostly just a refactoring.

llvm-svn: 310964
2017-08-15 21:42:52 +00:00
Craig Topper 699ae0c173 [X86] Implement __builtin_cpu_is
This patch adds support for __builtin_cpu_is. I've tried to match the strings supported to the latest version of gcc.

Differential Revision: https://reviews.llvm.org/D35449

llvm-svn: 310657
2017-08-10 20:28:30 +00:00
Craig Topper 41a550ccfa [X86] Support 'avx5124vnniw' and 'avx5124fmaps' for __builtin_cpu_supports.
They still need to be implemented in the intrinsics, the command line, and the backend. But this change isn't dependent on any of that and resolves a TODO.

llvm-svn: 310386
2017-08-08 17:43:44 +00:00
Joey Gouly fa76b49cef [OpenCL] Add missing subgroup builtins
This adds get_kernel_max_sub_group_size_for_ndrange and
get_kernel_sub_group_count_for_ndrange.

llvm-svn: 309678
2017-08-01 13:27:09 +00:00
Victor Leschuk 198357bbb9 Fix incorrect assertion condition.
llvm-svn: 309484
2017-07-29 08:18:38 +00:00
Vedant Kumar 10c3102071 [ubsan] Diagnose invalid uses of builtins (clang)
On some targets, passing zero to the clz() or ctz() builtins has undefined
behavior. I ran into this issue while debugging UB in __hash_table from libcxx:
the bug I was seeing manifested itself differently under -O0 vs -Os, due to a
UB call to clz() (see: libcxx/r304617).

This patch introduces a check which can detect UB calls to builtins.

llvm.org/PR26979

Differential Revision: https://reviews.llvm.org/D34590

llvm-svn: 309459
2017-07-29 00:19:51 +00:00
Martin Storsjo 022e782e75 [AArch64] Add support for __builtin_ms_va_list on aarch64
Move builtins from the x86 specific scope into the global
scope. Their use is still limited to x86_64 and aarch64 though.

This allows wine on aarch64 to properly handle variadic functions.

Differential Revision: https://reviews.llvm.org/D34475

llvm-svn: 308218
2017-07-17 20:49:45 +00:00
Ulrich Weigand cac24ab04c [SystemZ] Add support for IBM z14 processor (1/3)
This patch series adds support for the IBM z14 processor.  This part includes:
- Basic support for the new processor and its features.
- Support for low-level builtins mapped to new LLVM intrinsics.

Support for the -fzvector extension to vector float and the new
high-level vector intrinsics is provided by separate patches.

llvm-svn: 308197
2017-07-17 17:45:57 +00:00
Konstantin Zhuravlyov b0beb30fea Enhance synchscope representation (clang)
Relevant changes required for r307722.

Differential Revision: https://reviews.llvm.org/D33109

llvm-svn: 307723
2017-07-11 22:23:37 +00:00
Craig Topper e1ba5a3132 [X86] Move AVX512VPOPCNTDQ in __builtin_cpu_support's enum to match trunk gcc.
There are two other features before it that we don't currently support in the the frontend or backend so I left placeholders to keep the encoding correct.

I think the compiler-rt implementation of this feature is even further out of date.

llvm-svn: 307456
2017-07-08 00:47:44 +00:00
Sjoerd Meijer 98ee78578b This reverts r305820 (ARMv.2-A FP16 vector intrinsics) because it shows
problems in testing, see comments in D34161 for some more details.
A fix is in progres in D35011, but a revert seems better now as the fix will
probably take some more time to land.

llvm-svn: 307277
2017-07-06 16:37:31 +00:00
Heejin Ahn b92440eab0 [WebAssembly] Add throw/rethrow builtins for exception handling
Summary:
Add new builtins for throw/rethrow instructions. This follows exception handling
handling proposal in
https://github.com/WebAssembly/exception-handling/blob/master/proposals/Exceptions.md

Reviewers: sunfish, dschuff

Reviewed By: dschuff

Subscribers: jfb, dschuff, sbc100, jgravelle-google

Differential Revision: https://reviews.llvm.org/D34783

llvm-svn: 306775
2017-06-30 00:44:01 +00:00
Abderrazek Zaafrani f10ca93f34 [AArch64] ADD ARMv.2-A FP16 vector intrinsics
Differential Revision: https://reviews.llvm.org/D34161

llvm-svn: 305820
2017-06-20 18:54:57 +00:00
Dinar Temirbulatov 7b22425dff Expand vector oparation to as IR constants, PR28129.
llvm-svn: 305551
2017-06-16 12:09:52 +00:00
Galina Kistanova 0872d6c275 Added LLVM_FALLTHROUGH to address warning: this statement may fall through. NFC.
llvm-svn: 304649
2017-06-03 06:30:46 +00:00
Vedant Kumar a44a6ac81f Revert "[AArch64] Add ARMv8.2-A FP16 vefctor intrinsics"
This reverts commit r304493. It breaks all the Darwin bots:
http://green.lab.llvm.org/green/job/clang-stage1-cmake-RA-incremental_check/37168

Failure:
Failing Tests (2):
    Clang :: CodeGen/aarch64-v8.2a-neon-intrinsics.c
    Clang :: CodeGen/arm_neon_intrinsics.c

llvm-svn: 304509
2017-06-02 01:22:14 +00:00
Abderrazek Zaafrani a44e5f601d [AArch64] Add ARMv8.2-A FP16 vefctor intrinsics
llvm-svn: 304493
2017-06-01 23:22:29 +00:00
Oren Ben Simhon 140c1fb9ec [X86] Adding avx512_vpopcntdq feature set and its intrinsics
AVX512_VPOPCNTDQ is a new feature set that was published by Intel.
The patch represents the Clang side of the addition of six intrinsics for two new machine instructions (vpopcntd and vpopcntq).
It also includes the addition of the new feature set.

Differential Revision: https://reviews.llvm.org/D33170

llvm-svn: 303857
2017-05-25 13:44:11 +00:00
Tony Jiang 9aa2c0383d [PowerPC] Implement vec_xxsldwi builtin.
The vec_xxsldwi builtin is missing from altivec.h. This has been requested by
developers working on libvpx for VP9 support for Google.

The patch fixes PR: https://bugs.llvm.org/show_bug.cgi?id=32653
Differential Revision: https://reviews.llvm.org/D33236

llvm-svn: 303766
2017-05-24 15:54:13 +00:00
Tony Jiang bbc48e9164 [PowerPC] Implement vec_xxpermdi builtin.
The vec_xxpermdi builtin is missing from altivec.h. This has been requested by
developers working on libvpx for VP9 support for Google.

The patch fixes PR: https://bugs.llvm.org/show_bug.cgi?id=32653
Differential Revision: https://reviews.llvm.org/D33053

llvm-svn: 303760
2017-05-24 15:13:32 +00:00
Krzysztof Parzyszek 8f248234fa [CodeGen] Propagate LValueBaseInfo instead of AlignmentSource
The functions creating LValues propagated information about alignment
source. Extend the propagated data to also include information about
possible unrestricted aliasing. A new class LValueBaseInfo will
contain both AlignmentSource and MayAlias info.

This patch should not introduce any functional changes.

Differential Revision: https://reviews.llvm.org/D33284

llvm-svn: 303358
2017-05-18 17:07:11 +00:00
Serge Guelton 1d993270b3 Suppress all uses of LLVM_END_WITH_NULL. NFC.
Use variadic templates instead of relying on <cstdarg> + sentinel.

This enforces better type checking and makes code more readable.

Differential revision: https://reviews.llvm.org/D32550

llvm-svn: 302572
2017-05-09 19:31:30 +00:00
Dean Michael Berris 42af651358 [XRay] Add __xray_customeevent(...) as a clang-supported builtin
Summary:
We define the `__xray_customeevent` builtin that gets translated to
IR calls to the correct intrinsic. The default implementation of this is
a no-op function. The codegen side of this follows the following logic:

- When `-fxray-instrument` is not provided in the driver, we elide all
calls to `__xray_customevent`.
- When `-fxray-instrument` is enabled and a function is marked as "never
instrumented", we elide all calls to `__xray_customevent` in that
function; if either marked as "always instrumented" or subject to
threshold-based instrumentation, we emit a call to the
`llvm.xray.customevent` intrinsic from LLVM for each
`__xray_customevent` occurrence in the function.

This change depends on D27503 (to land in LLVM first).

Reviewers: echristo, rsmith

Subscribers: mehdi_amini, pelikan, lrl, cfe-commits

Differential Revision: https://reviews.llvm.org/D30018

llvm-svn: 302492
2017-05-09 00:45:40 +00:00
Nico Weber 050af67ea8 ANSIfy more. Still no behavior change.
llvm-svn: 302259
2017-05-05 17:16:58 +00:00
Nico Weber 0a234047eb ANSIfy. No behavior change.
llvm-svn: 302258
2017-05-05 17:15:08 +00:00
Hans Wennborg 5c3c51fe05 Implement _interlockedbittestandset as a builtin
It's used by MS headers in VS 2017 without including intrin.h, so we
can't implement it in the header anymore.

Differential Revision: https://reviews.llvm.org/D31736

llvm-svn: 299782
2017-04-07 16:41:47 +00:00
Michael Zuckerman 755a13db3d [X86][Clang] Converting __mm{|256|512}_movm_epi{8|16|32|64} LLVMIR call into generic intrinsics.
This patch is a part two of two reviews, one for the clang and the other for LLVM. 
In this patch, I covered the clang side, by introducing the intrinsic to the front end. 
This is done by creating a generic replacement.

Differential Revision: https://reviews.llvm.org/D31394a

llvm-svn: 299431
2017-04-04 13:29:53 +00:00
Hans Wennborg 043f402586 [X86] Implement __readgsqword (and the rest) as builtins (PR32373)
It seems MS headers have started using __readgsqword, and since it's
used in a header that doesn't include intrin.h, we can't implement it as
an inline function anymore.

That was already the case for __readfsdword, which Saleem added support
for in r220859. This patch reuses that codegen to implement all of
__read[fg]s{byte,word,dword,qword}.

Differential Revision: https://reviews.llvm.org/D31248

llvm-svn: 298538
2017-03-22 19:13:13 +00:00
George Burgess IV a63f91574f Let llvm.objectsize be conservative with null pointers
D28494 adds another parameter to @llvm.objectsize. Clang needs to be
sure to pass that third arg whenever applicable.

llvm-svn: 298431
2017-03-21 20:09:35 +00:00
Reid Kleckner de86482ce0 Update Clang for LLVM rename AttributeSet -> AttributeList
llvm-svn: 298394
2017-03-21 16:57:30 +00:00
Sanjay Patel e795daa55e [x86] these aren't the undefs you're looking for (PR32176)
x86 has undef SSE/AVX intrinsics that should represent a bogus register operand. 
This is not the same as LLVM's undef value which can take on multiple bit patterns.

There are better solutions / follow-ups to this discussed here:
https://bugs.llvm.org/show_bug.cgi?id=32176
...but this should prevent miscompiles with a one-line code change.

Differential Revision: https://reviews.llvm.org/D30834

llvm-svn: 297588
2017-03-12 19:15:10 +00:00
Yaxun Liu 4d86799219 [AMDGPU] Add builtin functions readlane ds_permute mov_dpp
Differential Revision: https://reviews.llvm.org/D30551

llvm-svn: 297436
2017-03-10 01:30:46 +00:00
Reid Kleckner b04cb9ab7a [MS] Add support for __ud2 and __int2c MSVC intrinsics
This was requested in PR31958 and elsewhere.

llvm-svn: 297057
2017-03-06 19:43:16 +00:00
George Burgess IV 0d6592a899 [CodeGen] Don't reemit expressions for pass_object_size params.
This fixes an assertion failure in cases where we had expression
statements that declared variables nested inside of pass_object_size
args. Since we were emitting the same ExprStmt twice (once for the arg,
once for the @llvm.objectsize call), we were getting issues with
redefining locals.

This also means that we can be more lax about when we emit
@llvm.objectsize for pass_object_size args: since we're reusing the
arg's value itself, we don't have to care so much about side-effects.

llvm-svn: 295935
2017-02-23 05:59:56 +00:00
Matt Arsenault a0c6dca15b AMDGPU: Add fmed3 half builtin
llvm-svn: 295874
2017-02-22 20:55:59 +00:00
George Burgess IV 8856aa9a54 Call the correct @llvm.objectsize.
The following code would crash clang:

void foo(unsigned *const __attribute__((pass_object_size(0))));
void bar(unsigned *i) { foo(i); }

This is because we were always selecting the version of
`@llvm.objectsize` that takes an i8* in CodeGen. Passing an i32* as an
i8* makes LLVM very unhappy.

(Yes, I'm surprised that this remained uncaught for so long, too. :) )

As an added bonus, we'll now also use the appropriate address space when
emitting @llvm.objectsize calls.

llvm-svn: 295805
2017-02-22 02:35:51 +00:00
Anastasia Stulova 58984e7087 [OpenCL] Correct ndrange_t implementation
Removed ndrange_t as Clang builtin type and added
as a struct type in the OpenCL header.

Use type name to do the Sema checking in enqueue_kernel
and modify IR generation accordingly.

Review: D28058

Patch by Dmitry Borisenkov!  
 

llvm-svn: 295311
2017-02-16 12:27:47 +00:00
Reid Kleckner 04f9f91da6 [MS] Implement the __fastfail intrinsic as a builtin
__fastfail terminates the process immediately with a special system
call. It does not run any process shutdown code or exception recovery
logic.

Fixes PR31854

llvm-svn: 294606
2017-02-09 18:31:06 +00:00
Matt Arsenault a274b209f5 AMDGPU: Add builtin for fmed3 intrinsic
llvm-svn: 293600
2017-01-31 03:42:07 +00:00
Arpith Chacko Jacob cdda3daa7f [OpenMP][NVPTX][CUDA] Adding support for printf for an NVPTX OpenMP device.
Support for CUDA printf is exploited to support printf for
an NVPTX OpenMP device.

To reflect the support of both programming models, the file
CGCUDABuiltin.cpp has been renamed to CGGPUBuiltin.cpp, and
the call EmitCUDADevicePrintfCallExpr has been renamed to
EmitGPUDevicePrintfCallExpr.

Reviewers: jlebar
Differential Revision: https://reviews.llvm.org/D17890

llvm-svn: 293444
2017-01-29 20:49:31 +00:00
Anastasia Stulova af0a7bbbe2 [OpenCL] Add missing address spaces in IR generation of blocks
Modify ObjC blocks impl wrt address spaces as follows:

- keep default private address space for blocks generated
as local variables (with captures);

- add global address space for global block literals (no captures);

- make the block invoke function and enqueue_kernel prototype with
the generic AS block pointer parameter to accommodate both 
private and global AS cases from above;

- add block handling into default AS because it's implemented as
a special pointer type (BlockPointer) in the frontend and therefore
it is used as a pointer everywhere. This is also needed to accommodate
both private and global AS blocks for the two cases above.

- removes ObjC RT specific symbols (NSConcreteStackBlock and
NSConcreteGlobalBlock) in the OpenCL mode.

Review: https://reviews.llvm.org/D28814
llvm-svn: 293286
2017-01-27 15:11:34 +00:00
Richard Smith 5e29dd3fe0 P0426: Make the library implementation of constexpr char_traits a little easier
by providing a memchr builtin that returns char* instead of void*.

Also add a __has_feature flag to indicate the presence of constexpr forms of
the relevant <string> functions.

llvm-svn: 292555
2017-01-20 00:45:35 +00:00
Chad Rosier c22abb3820 [ARM] Use generic bitreverse intrinsic, rather than ARM specific rbit.
The backend already supports lowering this intrinsic to a rbit instruction.

llvm-svn: 291582
2017-01-10 18:55:11 +00:00
Chad Rosier 5a4a1be690 [AArch64] Use generic bitreverse intrinsic, rather than AArch64 specific.
Differential Revision: https://reviews.llvm.org/D28400

llvm-svn: 291574
2017-01-10 17:20:28 +00:00
Sean Fertile 96d9e0ec05 Add vec_insert4b and vec_extract4b functions to altivec.h
Add builtins for the functions and custom codegen mapping the builtins to their
corresponding intrinsics and handling the endian related swapping.

https://reviews.llvm.org/D26546

llvm-svn: 291179
2017-01-05 21:43:30 +00:00
Saleem Abdulrasool 342beeb91e CodeGen: force builtins to be local
Unfortunately _setjmp3 can be both import or local.  The ASAN tests try to
emulate the flags which makes this harder to detect.  Rely on the linker
creating or using thunks here instead.  Should repair the ASAN windows bots.

llvm-svn: 289783
2016-12-15 07:29:04 +00:00
Zachary Turner 26dab1283f Update for clang after llvm::StringLiteral.
llvm-svn: 289553
2016-12-13 17:10:16 +00:00
Filipe Cabecinhas 322ecd901b [clang] Version support for UBSan handlers
This adds a way for us to version any UBSan handler by itself.
The patch overrides D21289 for a better implementation (we're able to
rev up a single handler).

After this, then we can land a slight modification of D19667+D19668.

We probably don't want to keep all the versions in compiler-rt (maybe we
want to deprecate on one release and remove the old handler on the next
one?), but with this patch we will loudly fail to compile when mixing
incompatible handler calls, instead of silently compiling and then
providing bad error messages.

Reviewers: kcc, samsonov, rsmith, vsk

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D21695

llvm-svn: 289444
2016-12-12 16:18:40 +00:00
Akira Hatanaka 6c299ca6e7 [CodeGen][ARM] Make sure the value and type used to create a bitcast
have the same size.

This fixes an asset that is triggered when an address of a boolean
variable is passed to __builtin_arm_ldrex or __builtin_arm_strex.

rdar://problem/29269006

llvm-svn: 288404
2016-12-01 19:25:14 +00:00
Peter Collingbourne b367c567d9 IRGen: Remove all uses of CreateDefaultAlignedLoad.
Differential Revision: https://reviews.llvm.org/D27157

llvm-svn: 288083
2016-11-28 22:30:21 +00:00
Konstantin Zhuravlyov 62ae8f671c [AMDGPU] Change frexp.exp builtin to return i16 for f16 input
Differential Revision: https://reviews.llvm.org/D26863

llvm-svn: 287390
2016-11-18 22:31:51 +00:00
Zaara Syeda c1d2952388 vector load store with length (left justified) clang portion
llvm-svn: 286994
2016-11-15 18:04:13 +00:00
Tony Jiang 6a49aad177 [PowerPC] Implement BE VSX load/store builtins - clang portion.
This patch implements all the overloads for vec_xl_be and vec_xst_be. On BE,
they behaves exactly the same with vec_xl and vec_xst, therefore they are
simply implemented by defining a matching macro. On LE, they are implemented
by defining new builtins and intrinsics. For int/float/long long/double, it
is just a load (lxvw4x/lxvd2x) or store(stxvw4x/stxvd2x). For char/char/short,
we also need some extra shuffling before or after call the builtins to get the
desired BE order. For int128, simply call vec_xl or vec_xst.

llvm-svn: 286971
2016-11-15 14:30:56 +00:00
Anastasia Stulova 0df4ac3f94 [OpenCL] Fix for integer parameters of enqueue_kernel
Make handling integer parameters more flexible:

- For the number of events argument allow to pass larger
integers than 32 bits as soon as compiler can prove that
the range fits in 32 bits. If not, the diagnostic will be given.

- Change type of the arguments specifying the sizes of
the corresponding block arguments to be size_t.

Review: https://reviews.llvm.org/D26509
llvm-svn: 286849
2016-11-14 17:39:58 +00:00
Anastasia Stulova 2b46120a09 [OpenCL] Change to clk_event parameter in enqueue_kernel.
- Accept NULL pointer as a valid parameter value for clk_event.
- Generate clk_event_t arguments of internal
__enqueue_kernel_XXX function as pointers in generic address space.

Review: https://reviews.llvm.org/D26507
llvm-svn: 286836
2016-11-14 15:34:01 +00:00
Konstantin Zhuravlyov 81a78bb864 [AMDGPU] Add f16 builtin functions (VI+)
Differential Revision: https://reviews.llvm.org/D26476

llvm-svn: 286741
2016-11-13 02:37:05 +00:00
Simon Pilgrim e47f2cd0c4 [X86] Merge (near) duplicate scalar non-temporal store code. NFCI.
llvm-svn: 286595
2016-11-11 14:38:34 +00:00
Ayman Musa e60a41ca28 [X86][AVX512][Clang] Add support for mask_{move|store|load}_s{s/d} and int2mask/mask2int intrinsics.
Differential Revision: https://reviews.llvm.org/D26021

llvm-svn: 286229
2016-11-08 12:00:30 +00:00
David Majnemer bb103d928e Use toCharUnitsFromBits instead of TargetInfo::getCharWidth
llvm-svn: 285595
2016-10-31 16:48:30 +00:00
David Majnemer 5116993f8e Add support for __builtin_alloca_with_align
__builtin_alloca always uses __BIGGEST_ALIGNMENT__ for the alignment of
the allocation.  __builtin_alloca_with_align allows the programmer to
specify the alignment of the allocation.

This fixes PR30658.

llvm-svn: 285544
2016-10-31 05:37:48 +00:00
David Majnemer 1878da43ea [CodeGen] Provide an appropriate alignment for dynamic allocas
GCC documents __builtin_alloca as aligning the storage to at least
__BIGGEST_ALIGNMENT__.

MSVC documents essentially the same for the x64 ABI:
https://msdn.microsoft.com/en-us/library/x9sx5da1.aspx

The 32-bit ABI follows the same rule: it emits a call to _alloca_probe_16

Differential Revision: https://reviews.llvm.org/D24378

llvm-svn: 285316
2016-10-27 17:18:24 +00:00
John McCall b92ab1afd5 Refactor call emission to package the function pointer together with
abstract information about the callee.  NFC.

The goal here is to make it easier to recognize indirect calls and
trigger additional logic in certain cases.  That logic will come in
a later patch; in the meantime, I felt that this was a significant
improvement to the code.

llvm-svn: 285258
2016-10-26 23:46:34 +00:00
Mehdi Amini 06d367c6c6 Add support for __builtin_os_log_format[_buffer_size]
This reverts commit r285007 and reapply r284990, with a fix for the
opencl test that I broke. Original commit message follows:

These new builtins support a mechanism for logging OS events, using a
printf-like format string to specify the layout of data in a buffer.
The _buffer_size version of the builtin can be used to determine the size
of the buffer to allocate to hold the data, and then __builtin_os_log_format
can write data into that buffer. This implements format checking to report
mismatches between the format string and the data arguments. Most of this
code was written by Chris Willmore.

Differential Revision: https://reviews.llvm.org/D25888

llvm-svn: 285019
2016-10-24 20:39:34 +00:00
Mehdi Amini 9c39fdceda Revert "Add support for __builtin_os_log_format[_buffer_size]"
This reverts commit r284990, two opencl test are broken

llvm-svn: 285007
2016-10-24 19:41:36 +00:00
Mehdi Amini 29034362ae Add support for __builtin_os_log_format[_buffer_size]
These new builtins support a mechanism for logging OS events, using a
printf-like format string to specify the layout of data in a buffer.
The _buffer_size version of the builtin can be used to determine the size
of the buffer to allocate to hold the data, and then __builtin_os_log_format
can write data into that buffer. This implements format checking to report
mismatches between the format string and the data arguments. Most of this
code was written by Chris Willmore.

Differential Revision: https://reviews.llvm.org/D25888

llvm-svn: 284990
2016-10-24 16:56:23 +00:00
Craig Topper 531ce28311 [AVX-512] Replace 64-bit element and 512-bit vector pmin/pmax builtins with native IR like we do for 128/256-bit, but with the addition of masking.
llvm-svn: 284956
2016-10-24 04:04:24 +00:00
Craig Topper f6373bc6fd [AVX-512] Remove masked 128/256-bit palignr builtins. We can just use a select in the header file with the older unmasked versions instead.
llvm-svn: 284920
2016-10-22 18:32:33 +00:00
Albert Gutowski 1deab38717 Implement __stosb intrinsic as a volatile memset
Summary: We need `__stosb` to be an intrinsic, because SecureZeroMemory function uses it without including intrin.h. Implementing it as a volatile memset is not consistent with MSDN specification, but it gives us target-independent IR while keeping the most important properties of `__stosb`.

Reviewers: rnk, hans, thakis, majnemer

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D25334

llvm-svn: 284253
2016-10-14 17:33:05 +00:00
Albert Gutowski 5e08df0266 Add 64-bit MS _Interlocked functions as builtins again
Summary: Previously global 64-bit versions of _Interlocked functions broke buildbots on i386, so now I'm adding them as builtins for x86-64 and ARM only (should they be also on AArch64? I had problems with testing it for AArch64, so I left it)

Reviewers: hans, majnemer, mstorsjo, rnk

Subscribers: cfe-commits, aemerson

Differential Revision: https://reviews.llvm.org/D25576

llvm-svn: 284172
2016-10-13 22:35:07 +00:00
Albert Gutowski 397d81bb9a Implement MS _ReturnAddress and _AddressOfReturnAddress intrinsics
Reviewers: rnk, thakis, majnemer, hans

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D25540

llvm-svn: 284131
2016-10-13 16:03:42 +00:00
Albert Gutowski 2a0621e58a Implement MS _BitScan intrinsics
Summary: _BitScan intrinsics (and some others, for example _Interlocked and _bittest) are supposed to work on both ARM and x86. This is an attempt to isolate them, avoiding repeating their code or writing separate function for each builtin.

Reviewers: hans, thakis, rnk, majnemer

Subscribers: RKSimon, cfe-commits, aemerson

Differential Revision: https://reviews.llvm.org/D25264

llvm-svn: 284060
2016-10-12 22:01:05 +00:00
Mehdi Amini 7186a4323e Revert "Change Builtins name to be stored as StringRef instead of raw pointers (NFC)"
This reverts commit r283802. It introduces temporarily static
initializers, because StringRef ctor isn't (yet) constexpr for
string literals.
I plan to get there this week, but apparently GCC is so terrible
with these static initializer right now (10 min+ extra codegen
time was reported) that I'll hold on to this patch till the
constexpr one is ready, and land these at the same time.

llvm-svn: 283920
2016-10-11 19:04:24 +00:00
Mehdi Amini b1bdc47309 Change Builtins name to be stored as StringRef instead of raw pointers (NFC)
llvm-svn: 283802
2016-10-10 21:34:29 +00:00
Albert Gutowski fcea61c563 Implement MS read/write barriers and __faststorefence intrinsic
Reviewers: hans, rnk, majnemer

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D25442

llvm-svn: 283793
2016-10-10 19:40:51 +00:00
Albert Gutowski 7216f17653 Implement __emul, __emulu, _mul128 and _umul128 MS intrinsics
Reviewers: rnk, thakis, majnemer, hans

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D25353

llvm-svn: 283785
2016-10-10 18:09:27 +00:00
Albert Gutowski f3a0bce155 Separate builtins for x84-64 and i386; implement __mulh and __umulh
Summary: We need x86-64-specific builtins if we want to implement some of the MS intrinsics - winnt.h contains definitions of some functions for i386, but not for x86-64 (for example _InterlockedOr64), which means that we cannot treat them as builtins for both i386 and x86-64, because then we have definitions of builtin functions in winnt.h on i386.

Reviewers: thakis, majnemer, hans, rnk

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D24598

llvm-svn: 283264
2016-10-04 22:29:49 +00:00
Gor Nishanov 97e3b6d895 [coroutines] Adding builtins for coroutine intrinsics and backendutil support.
Summary:
With this commit simple coroutines can be created in plain C using coroutine builtins.

Reviewers: rnk, EricWF, rsmith

Subscribers: modocache, mgorny, mehdi_amini, beanz, cfe-commits

Differential Revision: https://reviews.llvm.org/D24373

llvm-svn: 283155
2016-10-03 22:44:48 +00:00
Mehdi Amini b7fb124512 Use StringRef in Triple API (NFC)
llvm-svn: 282996
2016-10-01 01:16:22 +00:00
Martin Storsjo ed95a08ea4 [MS] Implement __iso_volatile loads/stores as builtins
These are supposed to produce the same as normal volatile
pointer loads/stores. When -volatile:ms is specified,
normal volatile pointers are forced to have atomic semantics
(as is the default on x86 in MSVC mode). In that case,
these builtins should still produce non-atomic volatile
loads/stores without acquire/release semantics, which
the new test verifies.

These are only available on ARM (and on AArch64,
although clang doesn't support AArch64/Windows yet).

This implements what is missing for PR30394, making it possible
to compile C++ for ARM in MSVC mode with MSVC headers.

Differential Revision: https://reviews.llvm.org/D24986

llvm-svn: 282900
2016-09-30 19:13:46 +00:00
Artem Belevich fda9905062 [CUDA] added __nvvm_atom_{sys|cta}_* builtins.
These builtins are available on sm_60+ GPU only.

Differential Revision: https://reviews.llvm.org/D24944

llvm-svn: 282609
2016-09-28 17:47:35 +00:00
Nemanja Ivanovic 10e2b5dcaa [Power9] Builtins for ELF v.2 ABI conformance - front end portion
This patch corresponds to review:
https://reviews.llvm.org/D24397

It adds the __POWER9_VECTOR__ macro and the -mpower9-vector option along with
a number of altivec.h functions (refer to the code review for a list).

llvm-svn: 282481
2016-09-27 10:45:22 +00:00
Alexey Bader 465c18973d [OpenCL] Augment pipe built-ins with pipe packet size and alignment.
Reviewers: Anastasia, vpykhtin

Subscribers: dmitry, cfe-commits

Differential Revision: https://reviews.llvm.org/D23992

llvm-svn: 282252
2016-09-23 14:20:00 +00:00
Albert Gutowski 727ab8a803 Add some MS aliases for existing intrinsics
Reviewers: thakis, compnerd, majnemer, rsmith, rnk

Subscribers: alexshap, cfe-commits

Differential Revision: https://reviews.llvm.org/D24330

llvm-svn: 281540
2016-09-14 21:19:43 +00:00
Dehao Chen 5d4f0be5b8 Convert finite to builtin
Summary: This patch converts finite/__finite to builtin functions so that it will be inlined by compiler.

Reviewers: hfinkel, davidxl, efriedma

Subscribers: efriedma, llvm-commits

Differential Revision: https://reviews.llvm.org/D24483

llvm-svn: 281509
2016-09-14 17:34:14 +00:00
Albert Gutowski fc19fa3721 Temporary fix for MS _Interlocked intrinsics
llvm-svn: 281401
2016-09-13 21:51:37 +00:00
Albert Gutowski 9918cb6573 Reverse commit 281375 (breaks building Chromium)
llvm-svn: 281399
2016-09-13 21:24:51 +00:00
Albert Gutowski ce7a9a47b2 Add bunch of _Interlocked builtins
Reviewers: compnerd, thakis, Prazek, majnemer, rnk

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D24153

llvm-svn: 281378
2016-09-13 19:43:33 +00:00
Albert Gutowski ae3fb3113f Add some MS aliases for existing intrinsics
Reviewers: thakis, compnerd, majnemer, rsmith, rnk

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D24330

llvm-svn: 281375
2016-09-13 19:26:42 +00:00
Albert Gutowski b6a11acb53 Implement MS _rot intrinsics
Reviewers: thakis, Prazek, compnerd, rnk

Subscribers: majnemer, cfe-commits

Differential Revision: https://reviews.llvm.org/D24311

llvm-svn: 280997
2016-09-08 22:32:19 +00:00
Changpeng Fang 03bdd8f797 AMDGPU: Add clang builtin for ds_swizzle.
Summary:
  int __builtin_amdgcn_ds_swizzle (int a, int imm);
while imm is a constant.

Differential Revision:
  http://reviews.llvm.org/D23682

llvm-svn: 279165
2016-08-18 22:04:54 +00:00
Reid Kleckner 66e7717b46 Revert "[X86] Add xgetbv/x[X86] Add xgetbv xsetbv intrinsics to non-windows platforms"
This reverts commit r278783.  It breaks usage of _xgetbv on Windows.

llvm-svn: 278814
2016-08-16 16:04:14 +00:00
Marina Yatsina 197b65f833 [X86] Add xgetbv/x[X86] Add xgetbv xsetbv intrinsics to non-windows platforms
commit on behalf of guyblank

Differential Revision: https://reviews.llvm.org/D21959

llvm-svn: 278783
2016-08-16 08:13:36 +00:00
Chandler Carruth 4c5e8ccf74 [x86] Fix a really nasty bug introduced in r276417 where alignment
constraints were added to _mm256_broadcast_{pd,ps} intel intrinsics.

The spec for these intrinics is ... pretty much silent on alignment.
This is especially frustrating considering the amount of discussion of
alignment in the load and store instrinsics. So I was forced to rely on
the specification for the VBROADCASTF128 instruction.

That instruction's spec is *also* completely silent on alignment.
Fortunately, when it comes to the instruction's spec, silence is enough.
There is no #GP fault option for an underaligned address so this
instruction, and by inference the intrinsic, can read any alignment.

As it happens, the old code worked exactly this way and in fact we have
plenty of code that hands pointers with less than 16-byte alignment to
these intrinsics. This code broke pretty spectacularly with this commit.

Fortunately, the fix is super simple! Change a 16 to a 1, and ta da!

Anyways, a lot of debugging for a really boring fix. =]

llvm-svn: 278202
2016-08-10 07:32:47 +00:00
Wei Ding 91c8450967 AMDGPU : Add Clang builtin intrinsics for compare with the full
wavefront result.

Differential Revision: http://reviews.llvm.org/D22934

llvm-svn: 277824
2016-08-05 15:38:46 +00:00
Alexey Bader d81623261a [OpenCL] Added underscores to the names of 'to_addr' OpenCL built-ins.
Summary:
In order to re-define OpenCL built-in functions
'to_{private,local,global}' in OpenCL run-time library LLVM names must
be different from the clang built-in function names.

Reviewers: yaxunl, Anastasia

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D23120

llvm-svn: 277743
2016-08-04 18:06:27 +00:00
Simon Pilgrim 2d8517303c [X86][AVX] Added support for lowering to VBROADCASTF128/VBROADCASTI128 with generic IR
As discussed on D22460, I've updated the vbroadcastf128 pd256/ps256 builtins to map directly to generic IR - load+splat a 128-bit vector to both lanes of a 256-bit vector.

Fix for PR28657.

llvm-svn: 276417
2016-07-22 13:58:56 +00:00
Matt Arsenault c7536a5d60 AMDGPU: Remove legacy ldexp builtin
llvm-svn: 275623
2016-07-15 21:33:06 +00:00
Matt Arsenault c86671da09 AMDGPU: Update for rsq intrinsic changes
llvm-svn: 275622
2016-07-15 21:33:02 +00:00
Wei Ding ea41f356bb AMDGPU: Add Clang Builtin for v_lerp_u8
Differential Revision: http://reviews.llvm.org/D22380

llvm-svn: 275577
2016-07-15 16:43:03 +00:00
Jan Vesely d7e03a5bd9 AMDGPU: Export workitem builtins
Reviewers: tstellardAMD

Differential Revision: http://reviews.llvm.org/D20299

llvm-svn: 275030
2016-07-10 22:38:04 +00:00
Craig Topper f2f1a099a7 [CodeGen] Use llvm::Type::getVectorNumElements instead of casting to llvm::VectorType and calling getNumElements. This is equivalent and shorter.
llvm-svn: 274823
2016-07-08 02:17:35 +00:00
Craig Topper 0160063aeb [X86] Reuse existing lambda and remove unnecessary argument from vector cmp builtin handling. NFC
llvm-svn: 274821
2016-07-08 01:57:24 +00:00
Craig Topper 925ef0a135 [X86] Remove a couple calls to create V2F64 and V4F32 types for builtin handling. Just get the type from the operand of the builtin instead. NFC
llvm-svn: 274820
2016-07-08 01:48:44 +00:00
Craig Topper 425d02d33e [X86] Use native IR for immediate values 0-7 of packed fp cmp builtins. This makes them the same as what is done when using the SSE builtins for these same encodings.
llvm-svn: 274608
2016-07-06 06:27:31 +00:00
Craig Topper 46e7555d4b [AVX512] Use the generic ctlz intrinsic to implement the vplzcntd/q builtins.
llvm-svn: 274603
2016-07-06 04:24:29 +00:00
Anastasia Stulova db7a31cce7 [OpenCL] An implementation of device side enqueue (DSE) from OpenCL v2.0 s6.13.17.
- Added new Builtins: enqueue_kernel, get_kernel_work_group_size
and get_kernel_preferred_work_group_size_multiple.

These Builtins use custom check to diagnose parameters of the passed Blocks
i. e. variable number of 'local void*' type params, and check different
overloads specified in Table 6.31 of OpenCL v2.0.

- IR is generated as an internal library call for each OpenCL Builtin,
reusing ObjC Block implementation.

Review: http://reviews.llvm.org/D20249
llvm-svn: 274540
2016-07-05 11:31:24 +00:00
Anastasia Stulova 7f8d6dc0ef [OpenCL] Make OpenCL Builtins added according to the right version.
Currently we only have OpenCL 2.0 Builtins i.e. pipes or address space conversions.

They have to be added only in the version 2.0 compilation mode to make the identifiers
available for use in the other versions.

Review: http://reviews.llvm.org/D20249
llvm-svn: 274509
2016-07-04 16:07:18 +00:00
Craig Topper ac1823f6e9 [AVX512] Modify what indices we emit for the zero vector we use for zero extension of the result of a v2i1 or v4i1 masked compare. This way we emit something that the backend easily interprets as a concatenation rather than a true shuffle. This delivers slightly better codegen with the current backend capabilities.
llvm-svn: 274484
2016-07-04 07:09:46 +00:00
Matt Arsenault f652caea65 Emit more intrinsics for builtin functions
This is important for building libclc. Since r273039 tests are failing
due to now emitting calls to these functions instead of emitting the
DAG node. The libm function names are implemented for OpenCL, and should
call the locally defined versions, so -fno-builtin is used. The IR
Some functions use the __builtins and expect the intrinsics to be
emitted. Without this we end up with nobuiltin calls to intrinsics
or to unsupported library calls.

llvm-svn: 274370
2016-07-01 17:38:14 +00:00
Igor Breger 2c880cf9b1 [AVX512] Zero extend cmp intrinsic return value.
Differential Revision: http://reviews.llvm.org/D21746

llvm-svn: 274110
2016-06-29 08:14:17 +00:00
Matt Arsenault 64665bc50d AMDGPU: Add builtin to read exec mask
llvm-svn: 273965
2016-06-28 00:13:17 +00:00
Craig Topper d1691c7026 [AVX512] Replace masked integer cmp and ucmp builtins with native IR.
llvm-svn: 273378
2016-06-22 04:47:58 +00:00
Simon Pilgrim d39d026324 [X86][SSE4A] Use native IR for mask movntsd/movntss intrinsics.
Depends on llvm side commit r273002.

llvm-svn: 273003
2016-06-17 14:28:16 +00:00
Ranjeet Singh ca2b3e7b5c [ARM] Add mrrc/mrrc2 intrinsics and update existing mcrr/mcrr2 intrinsics.
Reapplying patch in r272777 which was reverted
because the llvm patch which added support
for generating the mcrr/mcrr2 instructions
from the intrinsic was causing an assertion
failure. This has now been fixed in llvm.

llvm-svn: 272983
2016-06-17 00:59:41 +00:00
Sanjay Patel dbd68dd09d [x86] generate IR for AVX2 integer min/max builtins
Sibling patch to r272932:
http://reviews.llvm.org/rL272932

llvm-svn: 272933
2016-06-16 18:45:01 +00:00
Marcin Koscielnicki a46fade624 [Builtin] Make __builtin_thread_pointer target-independent.
This is now supported for ARM, AArch64, PowerPC, SystemZ, SPARC, Mips.

Differential Revision: http://reviews.llvm.org/D19589

llvm-svn: 272893
2016-06-16 13:41:54 +00:00
Sanjay Patel 280cfd1a69 [x86] translate SSE packed FP comparison builtins to IR
As noted in the code comment, a potential follow-on would be to remove
the builtins themselves. Other than ord/unord, this already works as 
expected. Eg:

  typedef float v4sf __attribute__((__vector_size__(16)));
  v4sf fcmpgt(v4sf a, v4sf b) { return a > b; }

Differential Revision: http://reviews.llvm.org/D21268

llvm-svn: 272840
2016-06-15 21:20:04 +00:00
Sanjay Patel 7495ec026e [x86] generate IR for SSE integer min/max builtins
Sibling patch to r272806:
http://reviews.llvm.org/rL272806

llvm-svn: 272807
2016-06-15 17:18:50 +00:00
Ranjeet Singh d48760da64 Reverting r272777 because one of the tests
added in the llvm patch is causing an assertion
to fail.

llvm-svn: 272790
2016-06-15 14:21:28 +00:00
Craig Topper a54c21e742 [AVX512] Use native IR for mask pcmpeq/pcmpgt intrinsics.
llvm-svn: 272787
2016-06-15 14:06:34 +00:00
Ranjeet Singh 8d5ad5bdf2 [ARM] Add mrrc/mrrc2 intrinsics and update existing mcrr/mcrr2 intrinsics.
Patch adds intrinsics for mrrc/mrrc2. The
intrinsics for mrrc/mrrc2 return a single
uint64_t to represent two 32 bit values.

The mcrr/mcrr2 intrinsic was changed to
accept a single uint64_t instead of two
32 bit values as the input for consistency.

Differential Revision: http://reviews.llvm.org/D21179

llvm-svn: 272777
2016-06-15 11:32:18 +00:00
Simon Pilgrim 532de1ceb9 Fix unused variable warning
llvm-svn: 272541
2016-06-13 10:05:19 +00:00
Simon Pilgrim beca5f295c [Clang][X86] Convert non-temporal store builtins to generic __builtin_nontemporal_store in headers
We can now use __builtin_nontemporal_store instead of target specific builtins for naturally aligned nontemporal stores which avoids the need for handling in CGBuiltin.cpp

The scalar integer nontemporal (unaligned) store builtins will have to wait as __builtin_nontemporal_store currently assumes natural alignment and doesn't accept the 'packed struct' trick that we use for normal unaligned load/stores.

The nontemporal loads require further backend support before we can safely convert them to __builtin_nontemporal_load

Differential Revision: http://reviews.llvm.org/D21272

llvm-svn: 272540
2016-06-13 09:57:52 +00:00
Craig Topper d1cb4ceacd [CodeGen] Update to use an ArrayRef of uint32_t instead of int in calls to CreateShuffleVector to match llvm interface change.
llvm-svn: 272492
2016-06-12 00:41:24 +00:00
Craig Topper 2769bb5753 [X86] Handle AVX2 pslldqi and psrldqi intrinsics shufflevector creation directly in the header file instead of in CGBuiltin.cpp. Simplify the sse2 equivalents as well.
llvm-svn: 272246
2016-06-09 05:15:12 +00:00
Craig Topper c1442973c8 [X86] Reuse the EmitX86Select routine to handle the select for masked palignr too.
llvm-svn: 272245
2016-06-09 05:15:00 +00:00
Igor Breger aadb876200 [AVX512] Emit select instruction instead of using x86 specific instrinsics.
This will allow us to remove the x86 instrinics from the backend.

Differential Revision: http://reviews.llvm.org/D21060

llvm-svn: 272141
2016-06-08 13:59:20 +00:00
Craig Topper f51cc07719 [AVX512] Convert masked palignr builtins directly to native IR similar to the other palignr builtins, but with a select to handle masking.
llvm-svn: 271873
2016-06-06 06:13:01 +00:00
Craig Topper 4b060e31c9 [AVX512] Convert masked load builtins to generic masked load intrinsics instead of the x86 specific ones.
This will allow the x86 intrinsics to be removed from the backend.

llvm-svn: 271253
2016-05-31 06:58:07 +00:00
Craig Topper 6e891fbdd2 [AVX512] Emit generic masked store instrinsics instead of using x86 specific intrinsics.
This will allow us to remove the x86 instrinics from the backend.

llvm-svn: 271246
2016-05-31 01:50:10 +00:00
Craig Topper b8b4b7eb01 [X86] Simplify alignr builtin support by recognizing that NumLaneElts is always 16. NFC
llvm-svn: 271176
2016-05-29 07:06:02 +00:00
Craig Topper 832caf041f [CodeGen] Use the ArrayRef form CreateShuffleVector instead of building ConstantVectors or ConstantDataVectors and calling the other form.
llvm-svn: 271165
2016-05-29 02:39:30 +00:00
Matt Arsenault 2d51059ebb AMDGPU: Add fract builtin
llvm-svn: 271080
2016-05-28 00:43:27 +00:00
David Majnemer e6abf3d29f [CodeGen] Don't crash when sizeof(long) != 4 for some intrins
_InterlockedIncrement and _InterlockedDecrement have 'long' in their
prototypes.  We assumed 'long' was the same size as an i32 which is
incorrect for other targets.

This fixes PR27892.

llvm-svn: 270953
2016-05-27 02:06:19 +00:00
Yaxun Liu f7449a179b [OpenCL] Add to_{global|local|private} builtin functions.
OpenCL builtin functions to_{global|local|private} accepts argument of pointer type to arbitrary pointee type, and return a pointer to the same pointee type in different addr space, i.e.

global gentype *to_global(gentype *p);
It is not desirable to declare it as

global void *to_global(void *);
in opencl header file since it misses diagnostics.

This patch implements these builtin functions as Clang builtin functions. In the builtin def file they are defined to have signature void*(void*). When handling call expressions, their declarations are re-written to have correct parameter type and return type corresponding to the call argument.

In codegen call to addr void *to_addr(void*) is generated with addrcasts or bitcasts to facilitate implementation in builtin library.

Differential Revision: http://reviews.llvm.org/D19932

llvm-svn: 270261
2016-05-20 19:54:38 +00:00
Benjamin Kramer f4c520d5d2 Add all the avx512 flavors to __builtin_cpu_supports's list.
This is matching what trunk gcc is accepting. Also adds a missing ssse3
case. PR27779. The amount of duplication here is annoying, maybe it
should be factored into a separate .def file?

llvm-svn: 270224
2016-05-20 15:21:08 +00:00
Justin Lebar 2e4ecfdebe [CUDA] Implement __ldg using intrinsics.
Summary:
Previously it was implemented as inline asm in the CUDA headers.

This change allows us to use the [addr+imm] addressing mode when
executing ld.global.nc instructions.  This translates into a 1.3x
speedup on some benchmarks that call this instruction from within an
unrolled loop.

Reviewers: tra, rsmith

Subscribers: jhen, cfe-commits, jholewinski

Differential Revision: http://reviews.llvm.org/D19990

llvm-svn: 270150
2016-05-19 22:49:13 +00:00
Derek Schuff dbd24b4593 [WebAssembly] Rename memory_size intrinsic to current_memory
This follows the recent change in the wasm spec.

llvm-svn: 268256
2016-05-02 17:26:19 +00:00
Marcin Koscielnicki 4005070e1b [AArch64] Fix D19098 fallout.
The intrinsic is now called llvm.thread.pointer, not
llvm.aarch64.thread.pointer.  Also, the code handling it in CGBuiltin.cpp
is dead - it's already covered by GCCBuiltin.  Remove it.

Differential Revision: http://reviews.llvm.org/D19099

llvm-svn: 266817
2016-04-19 20:51:00 +00:00
Ahmed Bougacha 1d9de10130 [ARM NEON] Define vfms_f32 on ARM, and all vfms using vfma.
r259537 added vfma/vfms to armv7, but the builtin was only lowered
on the AArch64 side. Instead of supporting it on ARM, get rid of it.

The vfms builtin lowered to:
  %nb = fsub float -0.0, %b
  %r = @llvm.fma.f32(%a, %nb, %c)

Instead, define the operation in terms of vfma, and swap the
multiplicands. It now lowers to:
  %na = fsub float -0.0, %a
  %r = @llvm.fma.f32(%na, %b, %c)

This matches the instruction more closely, and lets current LLVM
generate the "natural" operand ordering:
  fmls.2s v0, v1, v2
instead of the crooked (but equivalent):
  fmls.2s v0, v2, v1
Except for theses changes, assembly is identical.

LLVM accepts both commutations, and the LLVM tests in:
  test/CodeGen/AArch64/arm64-fmadd.ll
  test/CodeGen/AArch64/fp-dp3.ll
  test/CodeGen/AArch64/neon-fma.ll
  test/CodeGen/ARM/fusedMAC.ll
already check either the new one only, or both.

Also verified against the test-suite unittests.

llvm-svn: 266807
2016-04-19 19:44:45 +00:00
Sanjay Patel ae7a9df7bf make __builtin_isfinite more efficient (PR27145)
isinf (is infinite) and isfinite should be implemented with the same function
except we change the comparison operator.

See PR27145 for more details:
https://llvm.org/bugs/show_bug.cgi?id=27145

Ref: forked off of the discussion in D18513.

Differential Revision: http://reviews.llvm.org/D18648

llvm-svn: 265675
2016-04-07 14:29:05 +00:00
JF Bastien 92f4ef1017 NFC: make AtomicOrdering an enum class
Summary: See LLVM change D18775 for details, this change depends on it.

Reviewers: jyknight, reames

Subscribers: cfe-commits

Differential Revision: http://reviews.llvm.org/D18776

llvm-svn: 265569
2016-04-06 17:26:42 +00:00
Matt Arsenault 3fb963389e AMDGPU: Add frexp_mant + frexp_exp builtins
llvm-svn: 264960
2016-03-30 22:57:40 +00:00
Aaron Ballman abd466ed04 Silencing warnings from MSVC 2015 Update 2. Both of these changes silence "C4334 '<<': result of 32-bit shift implicitly converted to 64 bits (was 64-bit shift intended?)". NFC.
llvm-svn: 264932
2016-03-30 21:33:34 +00:00
Matt Arsenault 08087c52eb Add missing __builtin_bitreverse8
Also add documentation for bitreverse builtins

llvm-svn: 264203
2016-03-23 22:14:43 +00:00
Justin Lebar 717d2b0a0d [CUDA] Implement atomicInc and atomicDec builtins
These functions cannot be implemented as atomicrmw or cmpxchg
instructions, so they are implemented as a call to the NVVM intrinsics
@llvm.nvvm.atomic.load.inc.32.p0i32 and
@llvm.nvvm.atomic.load.dec.32.p0i32.

Patch by Jason Henline.

Reviewers: jlebar

Differential Revision: http://reviews.llvm.org/D18322

llvm-svn: 264009
2016-03-22 00:09:28 +00:00
John McCall c56a8b3284 Preserve ExtParameterInfos into CGFunctionInfo.
As part of this, make the function-arrangement interfaces
a little simpler and more semantic.

NFC.

llvm-svn: 263191
2016-03-11 04:30:31 +00:00
Kit Barton fbab158767 [PPC] FE support for generating VSX [negated] absolute value instructions
Includes new built-in, conversion of built-in to target-independent intrinsic
and update in the header file. Tests are also updated. There is a second part in
the backend for which I will post a separate code-review. BACKEND PART SHOULD BE
COMMITTED FIRST.

Phabricator: http://reviews.llvm.org/D17816
llvm-svn: 263051
2016-03-09 19:28:31 +00:00
Matt Arsenault 2d9339890f Add __builtin_canonicalize
llvm-svn: 262122
2016-02-27 09:06:18 +00:00
Matt Arsenault 9b277b4ad4 AMDGPU: Add sin/cos builtins
llvm-svn: 260783
2016-02-13 01:21:09 +00:00
Matt Arsenault f5c1f47181 AMDGPU: Update builtin for intrinsic change
llvm-svn: 260781
2016-02-13 01:03:09 +00:00
Matt Arsenault 105e892c2c Add builtins for bitreverse intrinsic
Follow the naming convention that bswap uses since it's a
similar sort of operation.

llvm-svn: 259671
2016-02-03 17:49:38 +00:00
Xiuli Pan bb4d8d30b1 Recommit: R258773 [OpenCL] Pipe builtin functions
Fix arc patch fuzz error.
Summary:
Support for the pipe built-in functions for OpenCL 2.0.
The pipe builtin functions may have infinite kinds of element types, one approach
would be to just generate calls that would always use generic types such as void*.
This patch is based on bader's opencl support patch on SPIR-V branch.

Reviewers: Anastasia, pekka.jaaskelainen

Subscribers: keryell, bader, cfe-commits

Differential Revision: http://reviews.llvm.org/D15914

llvm-svn: 258782
2016-01-26 04:03:48 +00:00
David Majnemer 747f168e8d Revert "[OpenCL] Pipe builtin functions"
This reverts commit r258773, it broke the build bots:
http://bb.pgr.jp/builders/cmake-clang-x86_64-linux/builds/43853

llvm-svn: 258775
2016-01-26 02:22:31 +00:00
Xiuli Pan 3a9952c9e7 [OpenCL] Pipe builtin functions
Summary:
Support for the pipe built-in functions for OpenCL 2.0.
The pipe builtin functions may have infinite kinds of element types, one approach
would be to just generate calls that would always use generic types such as void*.
This patch is based on bader's opencl support patch on SPIR-V branch.

Reviewers: Anastasia, pekka.jaaskelainen

Subscribers: keryell, bader, cfe-commits

Differential Revision: http://reviews.llvm.org/D15914

llvm-svn: 258773
2016-01-26 02:06:04 +00:00
Justin Lebar 3039a593db [CUDA] Make printf work.
Summary:
The code in CGCUDACall is largely based on a patch written by Eli
Bendersky:
http://lists.llvm.org/pipermail/llvm-commits/Week-of-Mon-20140324/210218.html

That patch implemented an LLVM pass lowering printf to vprintf; this
one does something similar, but in Clang codegen.

Reviewers: echristo

Subscribers: cfe-commits, jhen, tra, majnemer

Differential Revision: http://reviews.llvm.org/D16372

llvm-svn: 258642
2016-01-23 21:28:14 +00:00
Matt Arsenault 8a4078c741 AMDGPU: Rename builtins to use amdgcn prefix
Keep the ones still used by libclc around for now.

Emit the new amdgcn intrinsic name if not targeting r600,
in which case the old AMDGPU name is still used.

llvm-svn: 258560
2016-01-22 21:30:53 +00:00
Ben Craig cd7e9f143b Reordering fields to reduce padding in Clang. NFC
llvm-svn: 255552
2015-12-14 21:54:11 +00:00
George Burgess IV 3e3bb95b69 Add the `pass_object_size` attribute to clang.
`pass_object_size` is our way of enabling `__builtin_object_size` to
produce high quality results without requiring inlining to happen
everywhere.

A link to the design doc for this attribute is available at the
Differential review link below.

Differential Revision: http://reviews.llvm.org/D13263

llvm-svn: 254554
2015-12-02 21:58:08 +00:00
Eric Christopher fbfd97ed5c Move checkTargetFeatures to CodeGenFunction.cpp to make it
more obvious that it's generic.

llvm-svn: 252833
2015-11-12 00:44:07 +00:00
Eric Christopher c7e79dbec8 In preparation to use it in more places rename
checkBuiltinTargetFeatures to checkTargetFeatures and sink
the error handling into the function.

llvm-svn: 252832
2015-11-12 00:44:04 +00:00
Eric Christopher 2b90a64e31 Extract out a function onto CodeGenModule for getting the map of
features for a particular function, then use it to clean up some
code.

llvm-svn: 252819
2015-11-11 23:05:08 +00:00
Eric Christopher ed60b436d4 Fix a FIXME about using std::is_sorted.
llvm-svn: 252691
2015-11-11 02:04:08 +00:00
Petar Jovanovic 73d1044abe Fix __builtin_signbit for ppcf128 type
Function__builtin_signbit returns wrong value for type ppcf128 on big endian
machines. This patch fixes how value is generated in that case.

Patch by Aleksandar Beserminji.

Differential Revision: http://reviews.llvm.org/D14149

llvm-svn: 252307
2015-11-06 14:52:46 +00:00
Dan Gohman 24f0a08c1b [WebAssembly] Update wasm builtin functions to match spec changes.
The page_size operator has been removed from the spec, and the resize_memory
operator has been changed to grow_memory.

llvm-svn: 252201
2015-11-05 20:16:37 +00:00
John McCall 03107a4ef0 Add support for __builtin_{add,sub,mul}_overflow.
Patch by David Grayson!

llvm-svn: 251651
2015-10-29 20:48:01 +00:00
Benjamin Kramer e003ca2a03 Put global classes into the appropriate namespace.
Most of the cases belong into an anonymous namespace. No functionality
change intended.

llvm-svn: 251514
2015-10-28 13:54:16 +00:00
Eric Christopher 9d628c33b3 Reflow comment.
llvm-svn: 251501
2015-10-28 06:56:25 +00:00
Eric Christopher 99af5b2ea7 Handle target builtin options that are all required rather than
only one of a group of possibilities.

This changes the syntax in the builtin files to represent:

, as the and operator
| as the or operator

The former syntax matches how the backend tablegen files represent
multiple subtarget features being required.

Updated the builtin and intrinsic headers accordingly for the new
syntax.

llvm-svn: 251388
2015-10-27 06:11:03 +00:00
Eric Christopher 4a4367534b Use early exits to reduce indentation.
llvm-svn: 251371
2015-10-27 00:06:21 +00:00
Craig Topper 273dbc602f Make a bunch of static arrays const.
llvm-svn: 250647
2015-10-18 05:29:26 +00:00
Eric Christopher 15709991d0 Add an error when calling a builtin that requires features that don't
match the feature set of the function that they're being called from.

This ensures that we can effectively diagnose some[1] code that would
instead ICE in the backend with a failure to select message.

Example:

__m128d foo(__m128d a, __m128d b) {
  return __builtin_ia32_addsubps(b, a);
}

compiled for normal x86_64 via:

clang -target x86_64-linux-gnu -c

would fail to compile in the back end because the normal subtarget
features for x86_64 only include sse2 and the builtin requires sse3.

[1] We're still not erroring on:

__m128i bar(__m128i const *p) { return _mm_lddqu_si128(p); }

where we should fail and error on an always_inline function being
inlined into a function that doesn't support the subtarget features
required.

llvm-svn: 250473
2015-10-15 23:47:11 +00:00
Benjamin Kramer c2d2b4259c [CodeGen] Remove dead code. NFC.
llvm-svn: 250418
2015-10-15 15:29:40 +00:00
Amjad Aboud 2b9b8a5921 [X86] Add XSAVE intrinsic family
Add intrinsics for the
  XSAVE instructions (XSAVE/XSAVE64/XRSTOR/XRSTOR64)
  XSAVEOPT instructions (XSAVEOPT/XSAVEOPT64)
  XSAVEC instructions (XSAVEC/XSAVEC64)
  XSAVES instructions (XSAVES/XSAVES64/XRSTORS/XRSTORS64)

Differential Revision: http://reviews.llvm.org/D13014

llvm-svn: 250158
2015-10-13 12:29:35 +00:00
Dan Gohman 266b38ab56 [WebAssembly] Add a __builtin_wasm_resize_memory() intrinsic.
llvm-svn: 249179
2015-10-02 20:20:01 +00:00
Dan Gohman d4c5fb597d [WebAssembly] Add a __builtin_wasm_memory_size() intrinsic.
llvm-svn: 249176
2015-10-02 19:38:47 +00:00
Jingyue Wu f1eca25b16 [CUDA] fix codegen for __nvvm_atom_cas_*
Summary: __nvvm_atom_cas_* returns the old value instead of whether the swap succeeds.

Reviewers: eliben, tra

Subscribers: jholewinski, llvm-commits

Differential Revision: http://reviews.llvm.org/D13306

llvm-svn: 248951
2015-09-30 21:49:32 +00:00
Jeroen Ketema 55a8e80de8 [ARM][NEON] Use address space in vld([1234]|[234]lane) and vst([1234]|[234]lane) instructions
This is the clang commit associated with llvm r248887.

This commit changes the interface of the vld[1234], vld[234]lane, and vst[1234],
vst[234]lane ARM neon intrinsics and associates an address space with the
pointer that these intrinsics take. This changes, e.g.,

<2 x i32> @llvm.arm.neon.vld1.v2i32(i8*, i32)

to

<2 x i32> @llvm.arm.neon.vld1.v2i32.p0i8(i8*, i32)

This change ensures that address spaces are fully taken into account in the ARM
target during lowering of interleaved loads and stores.

Differential Revision: http://reviews.llvm.org/D13127

llvm-svn: 248888
2015-09-30 10:56:56 +00:00
Artem Belevich b5bc923af4 [CUDA] Allow parsing of host and device code simultaneously.
* adds -aux-triple option to specify target triple
 * propagates aux target info to AST context and Preprocessor
 * pulls in target specific preprocessor macros.
 * pulls in target-specific builtins from aux target.
 * sets appropriate host or device attribute on builtins.

Differential Revision: http://reviews.llvm.org/D12917

llvm-svn: 248299
2015-09-22 17:23:22 +00:00
Charles Davis c7d5c94f78 Support __builtin_ms_va_list.
Summary:
This change adds support for `__builtin_ms_va_list`, a GCC extension for
variadic `ms_abi` functions. The existing `__builtin_va_list` support is
inadequate for this because `va_list` is defined differently in the Win64
ABI vs. the System V/AMD64 ABI.

Depends on D1622.

Reviewers: rsmith, rnk, rjmccall

CC: cfe-commits

Differential Revision: http://reviews.llvm.org/D1623

llvm-svn: 247941
2015-09-17 20:55:33 +00:00
Steven Wu 0d22f2d57e Fix vld1_lane intrinsic generation
Fix a bug introduced in r246985 which causes assertion when generating
vld1_lane.

llvm-svn: 247117
2015-09-09 01:37:18 +00:00