Commit Graph

119 Commits

Author SHA1 Message Date
Craig Topper 5a43fdd313 [X86] Remove what little support we had for MPX
-Deprecate -mmpx and -mno-mpx command line options
-Remove CPUID detection of mpx for -march=native
-Remove MPX from all CPUs
-Remove MPX preprocessor define

I've left the "mpx" string in the backend so we don't fail on old IR, but its not connected to anything.

gcc has also deprecated these command line options. https://www.phoronix.com/scan.php?page=news_item&px=GCC-Patch-To-Drop-MPX

Differential Revision: https://reviews.llvm.org/D66669

llvm-svn: 370393
2019-08-29 18:09:02 +00:00
Vlad Tsyrklevich 57076d3199 Revert "Change the X86 datalayout to add three address spaces for 32 bit signed,"
This reverts commit r370083 because it caused check-lld failures on
sanitizer-x86_64-linux-fast.

llvm-svn: 370142
2019-08-28 01:08:54 +00:00
Amy Huang 1299945b81 Change the X86 datalayout to add three address spaces for 32 bit signed,
32 bit unsigned, and 64 bit pointers.

llvm-svn: 370083
2019-08-27 17:46:53 +00:00
Erich Keane 393094859e Remove CallingConvMethodType
This seems to be an old vestage of a previous implementation of getting
the default calling convention, and everything is now using
CXXABI/ASTContext's getDefaultCallingConvention.  Remove it, since it
isn't doing anything.

llvm-svn: 367039
2019-07-25 17:14:45 +00:00
Fangrui Song c46d78d1b7 [X86][PowerPC] Support -mlong-double-128
This patch makes the driver option -mlong-double-128 available for X86
and PowerPC. The CC1 option -mlong-double-128 is available on all targets
for users to test on unsupported targets.

On PowerPC, -mlong-double-128 uses the IBM extended double format
because we don't support -mabi=ieeelongdouble yet (D64283).

Reviewed By: rnk

Differential Revision: https://reviews.llvm.org/D64277

llvm-svn: 365866
2019-07-12 02:32:15 +00:00
Reid Kleckner 8777df5270 De-templatize non-dependent VS macro logic, NFC
These macro definitions don't depend on the template parameter, so they
don't need to be part of the template. Move them to a .cpp file.

llvm-svn: 365556
2019-07-09 20:57:28 +00:00
Fangrui Song 04615341e4 [ItaniumMangle] Refactor long double/__float128 mangling and fix the mangled code
In gcc PowerPC, long double has 3 mangling schemes:

-mlong-double-64: `e`
-mlong-double-128 -mabi=ibmlongdouble: `g`
-mlong-double-128 -mabi=ieeelongdouble: `u9__ieee128` (gcc <= 8.1: `U10__float128`)

The current useFloat128ManglingForLongDouble() bisection is not suitable
when we support -mlong-double-128 in clang (D64277). Replace
useFloat128ManglingForLongDouble() with getLongDoubleMangling() and
getFloat128Mangling() to allow 3 mangling schemes.

I also deleted the `getTriple().isOSBinFormatELF()` check (the Darwin
support has gone: https://reviews.llvm.org/D50988).

For x86, change the mangled code of __float128 from `U10__float128` to `g`. `U10__float128` was wrongly copied from PowerPC.
The test will be added to `test/CodeGen/x86-long-double.cpp` in D64277.

Reviewed By: erichkeane

Differential Revision: https://reviews.llvm.org/D64276

llvm-svn: 365480
2019-07-09 13:32:26 +00:00
Pengfei Wang 3a29f7c99c [X86] Add ENQCMD instructions
For more details about these instructions, please refer to the latest
ISE document:
https://software.intel.com/en-us/download/intel-architecture-instruction-set-extensions-programming-reference.

Patch by Tianqing Wang (tianqing)

Differential Revision: https://reviews.llvm.org/D62282

llvm-svn: 362685
2019-06-06 08:28:42 +00:00
Pengfei Wang cc3629d545 [X86] Add VP2INTERSECT instructions
Support intel AVX512 VP2INTERSECT instructions in clang

Patch by Xiang Zhang (xiangzhangllvm)

Differential Revision: https://reviews.llvm.org/D62367

llvm-svn: 362196
2019-05-31 06:09:35 +00:00
Luo, Yuanke 844f662932 Enable intrinsics of AVX512_BF16, which are supported for BFLOAT16 in Cooper Lake
Summary:
1. Enable infrastructure of AVX512_BF16, which is supported for BFLOAT16 in Cooper Lake;
2. Enable intrinsics for VCVTNE2PS2BF16, VCVTNEPS2BF16 and DPBF16PS instructions, which are Vector Neural Network Instructions supporting BFLOAT16 inputs and conversion instructions from IEEE single precision.
For more details about BF16 intrinsic, please refer to the latest ISE document: https://software.intel.com/en-us/download/intel-architecture-instruction-set-extensions-programming-reference

Patch by LiuTianle

Reviewers: craig.topper, smaslov, LuoYuanke, wxiao3, annita.zhang, spatel, RKSimon

Reviewed By: craig.topper

Subscribers: mgorny, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D60552

llvm-svn: 360018
2019-05-06 08:25:11 +00:00
Craig Topper 7339e61b89 [X86] Correct the value of MaxAtomicInlineWidth for pre-586 cpus
Use the new cx8 feature flag that was added to the backend to represent support for cmpxchg8b. Use this flag to set the MaxAtomicInlineWidth.

This also assumes all the cmpxchg instructions are enabled for CK_Generic which is what cc1 defaults to when nothing is specified.

Differential Revision: https://reviews.llvm.org/D59566

llvm-svn: 356709
2019-03-21 20:36:08 +00:00
Craig Topper 140f766f14 [X86] Remove getCPUKindCanonicalName which is unused.
Differential Revision: https://reviews.llvm.org/D59578

llvm-svn: 356580
2019-03-20 17:26:51 +00:00
Nirav Dave 91ecb69acd [X86] Prevent clang clobber checking for asm flag constraints.
Update getConstraintRegister as X86 Asm flag output constraints are no
longer fully alphanumeric,

llvm-svn: 354211
2019-02-17 03:53:23 +00:00
Chandler Carruth 2946cd7010 Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.

Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.

llvm-svn: 351636
2019-01-19 08:50:56 +00:00
Reid Kleckner 4dc0b1ac60 Fix clang -Wimplicit-fallthrough warnings across llvm, NFC
This patch should not introduce any behavior changes. It consists of
mostly one of two changes:
1. Replacing fall through comments with the LLVM_FALLTHROUGH macro
2. Inserting 'break' before falling through into a case block consisting
   of only 'break'.

We were already using this warning with GCC, but its warning behaves
slightly differently. In this patch, the following differences are
relevant:
1. GCC recognizes comments that say "fall through" as annotations, clang
   doesn't
2. GCC doesn't warn on "case N: foo(); default: break;", clang does
3. GCC doesn't warn when the case contains a switch, but falls through
   the outer case.

I will enable the warning separately in a follow-up patch so that it can
be cleanly reverted if necessary.

Reviewers: alexfh, rsmith, lattner, rtrieu, EricWF, bollu

Differential Revision: https://reviews.llvm.org/D53950

llvm-svn: 345882
2018-11-01 19:54:45 +00:00
Erich Keane 19a8adc9bd Implement Function Multiversioning for Non-ELF Systems.
Similar to how ICC handles CPU-Dispatch on Windows, this patch uses the
resolver function directly to forward the call to the proper function.
This is not nearly as efficient as IFuncs of course, but is still quite
useful for large functions specifically developed for certain
processors.

This is unfortunately still limited to x86, since it depends on
__builtin_cpu_supports and __builtin_cpu_is, which are x86 builtins.

The naming for the resolver/forwarding function for cpu-dispatch was
taken from ICC's implementation, which uses the unmodified name for this
(no mangling additions).  This is possible, since cpu-dispatch uses '.A'
for the 'default' version.

In 'target' multiversioning, this function keeps the '.resolver'
extension in order to keep the default function keeping the default
mangling.

Change-Id: I4731555a39be26c7ad59a2d8fda6fa1a50f73284

Differential Revision: https://reviews.llvm.org/D53586

llvm-svn: 345298
2018-10-25 18:57:19 +00:00
Hans Wennborg b4278895a4 Revert r323281 "Adjust MaxAtomicInlineWidth for i386/i486 targets."
As reported on http://lists.llvm.org/pipermail/cfe-dev/2018-August/058760.html,
this broke i386-freebsd11 due to its lack of atomic 64 bit primitives.

While that's not really this commit's fault, let's revert back to the old
behaviour until this can be fixed. This means generating cmpxchg8b etc for i386
and i486 which don't technically support those, but that's been the behaviour
for a long time, so a little longer probably doesn't hurt that much.

> Adjust MaxAtomicInlineWidth for i386/i486 targets.
>
> This is to fix the bug reported in https://bugs.llvm.org/show_bug.cgi?id=34347#c6.
> Currently, all  MaxAtomicInlineWidth of x86-32 targets are set to 64. However,
> i386 doesn't support any cmpxchg related instructions. i486 only supports cmpxchg.
> So in this patch MaxAtomicInlineWidth is reset as follows:
> For i386, the MaxAtomicInlineWidth should be 0 because no cmpxchg is supported.
> For i486, the MaxAtomicInlineWidth should be 32 because it supports cmpxchg.
> For others 32 bits x86 cpu, the MaxAtomicInlineWidth should be 64 because of cmpxchg8b.
>
> Differential Revision: https://reviews.llvm.org/D42154

llvm-svn: 340666
2018-08-24 22:46:33 +00:00
Chandler Carruth ae0cafece8 [x86/retpoline] Split the LLVM concept of retpolines into separate
subtarget features for indirect calls and indirect branches.

This is in preparation for enabling *only* the call retpolines when
using speculative load hardening.

I've continued to use subtarget features for now as they continue to
seem the best fit given the lack of other retpoline like constructs so
far.

The LLVM side is pretty simple. I'd like to eventually get rid of the
old feature, but not sure what backwards compatibility issues that will
cause.

This does remove the "implies" from requesting an external thunk. This
always seemed somewhat questionable and is now clearly not desirable --
you specify a thunk the same way no matter which set of things are
getting retpolines.

I really want to keep this nicely isolated from end users and just an
LLVM implementation detail, so I've moved the `-mretpoline` flag in
Clang to no longer rely on a specific subtarget feature by that name and
instead to be directly handled. In some ways this is simpler, but in
order to preserve existing behavior I've had to add some fallback code
so that users who relied on merely passing -mretpoline-external-thunk
continue to get the same behavior. We should eventually remove this
I suspect (we have never tested that it works!) but I've not done that
in this patch.

Differential Revision: https://reviews.llvm.org/D51150

llvm-svn: 340515
2018-08-23 06:06:38 +00:00
Fangrui Song 6907ce2f8f Remove trailing space
sed -Ei 's/[[:space:]]+$//' include/**/*.{def,h,td} lib/**/*.{cpp,h}

llvm-svn: 338291
2018-07-30 19:24:48 +00:00
Sanjin Sijaric 56391d6f84 [ARM64] [Windows] Follow MS X86_64 C++ ABI when passing structs
Summary: Microsoft's C++ object model for ARM64 is the same as that for X86_64.
For example, small structs with non-trivial copy constructors or virtual
function tables are passed indirectly.  Currently, they are passed in registers
when compiled with clang.

Reviewers: rnk, mstorsjo, TomTan, haripul, javed.absar

Reviewed By: rnk, mstorsjo

Subscribers: kristof.beyls, chrib, llvm-commits, cfe-commits

Differential Revision: https://reviews.llvm.org/D49770

llvm-svn: 338076
2018-07-26 22:18:28 +00:00
Erich Keane 3efe00206f Implement cpu_dispatch/cpu_specific Multiversioning
As documented here: https://software.intel.com/en-us/node/682969 and
https://software.intel.com/en-us/node/523346. cpu_dispatch multiversioning
is an ICC feature that provides for function multiversioning.

This feature is implemented with two attributes: First, cpu_specific,
which specifies the individual function versions. Second, cpu_dispatch,
which specifies the location of the resolver function and the list of
resolvable functions.

This is valuable since it provides a mechanism where the resolver's TU
can be specified in one location, and the individual implementions
each in their own translation units.

The goal of this patch is to be source-compatible with ICC, so this
implementation diverges from the ICC implementation in a few ways:
1- Linux x86/64 only: This implementation uses ifuncs in order to
properly dispatch functions. This is is a valuable performance benefit
over the ICC implementation. A future patch will be provided to enable
this feature on Windows, but it will obviously more closely fit ICC's
implementation.
2- CPU Identification functions: ICC uses a set of custom functions to identify
the feature list of the host processor. This patch uses the cpu_supports
functionality in order to better align with 'target' multiversioning.
1- cpu_dispatch function def/decl: ICC's cpu_dispatch requires that the function
marked cpu_dispatch be an empty definition. This patch supports that as well,
however declarations are also permitted, since the linker will solve the
issue of multiple emissions.

Differential Revision: https://reviews.llvm.org/D47474

llvm-svn: 337552
2018-07-20 14:13:28 +00:00
Reid Kleckner 89fbd55145 Revert r333791 "Cap "voluntary" vector alignment at 16 for all Darwin platforms."
Adding __attribute__((aligned(32))) to __m256 breaks the implementation
of _mm256_loadu_ps on Windows. On Windows, alignment attributes have
higher precedence than packing attributes.

We also might want to carefully consider the consequences of changing
our vector typedefs, since many users copy them and invent their own
new, non-Intel specific vector type names.

llvm-svn: 333958
2018-06-04 21:39:20 +00:00
John McCall 280c656031 Cap "voluntary" vector alignment at 16 for all Darwin platforms.
This fixes two major problems:
- We were not capping vector alignment as desired on 32-bit ARM.
- We were using different alignments based on the AVX settings on
  Intel, so we did not have a consistent ABI.

This is an ABI break, but we think we can get away with it because
vectors tend to be used mostly in inline code (which is why not having
a consistent ABI has not proven disastrous on Intel).

Intel's AVX types are specified as having 32-byte / 64-byte alignment,
so align them explicitly instead of relying on the base ABI rule.
Note that this sort of attribute is stripped from template arguments
in template substitution, so there's a possibility that code templated
over vectors will produce inadequately-aligned objects.  The right
long-term solution for this is for alignment attributes to be
interpreted as true qualifiers and thus preserved in the canonical type.

llvm-svn: 333791
2018-06-01 21:34:26 +00:00
Gabor Buella 078bb99a90 [x86] invpcid intrinsic
An intrinsic for an old instruction, as described in the Intel SDM.

Reviewers: craig.topper, rnk

Reviewed By: craig.topper, rnk

Differential Revision: https://reviews.llvm.org/D47142

llvm-svn: 333256
2018-05-25 06:34:42 +00:00
Alexander Ivchenko 0fb8c877c4 This patch aims to match the changes introduced
in gcc by https://gcc.gnu.org/ml/gcc-cvs/2018-04/msg00534.html.
The -mibt feature flag is being removed, and the -fcf-protection
option now also defines a CET macro and causes errors when used
on non-X86 targets, while X86 targets no longer check for -mibt
and -mshstk to determine if -fcf-protection is supported. -mshstk
is now used only to determine availability of shadow stack intrinsics.

Comes with an LLVM patch (D46882).

Patch by mike.dvoretsky

Differential Revision: https://reviews.llvm.org/D46881

llvm-svn: 332704
2018-05-18 11:56:21 +00:00
Gabor Buella 3a7571259e [X86] ptwrite intrinsic
Reviewers: craig.topper, RKSimon

Reviewed By: craig.topper

Differential Revision: https://reviews.llvm.org/D46540

llvm-svn: 331962
2018-05-10 07:28:54 +00:00
Adrian Prantl 9fc8faf9e6 Remove \brief commands from doxygen comments.
This is similar to the LLVM change https://reviews.llvm.org/D46290.

We've been running doxygen with the autobrief option for a couple of
years now. This makes the \brief markers into our comments
redundant. Since they are a visual distraction and we don't want to
encourage more \brief markers in new code either, this patch removes
them all.

Patch produced by

for i in $(git grep -l '\@brief'); do perl -pi -e 's/\@brief //g' $i & done
for i in $(git grep -l '\\brief'); do perl -pi -e 's/\\brief //g' $i & done

Differential Revision: https://reviews.llvm.org/D46320

llvm-svn: 331834
2018-05-09 01:00:01 +00:00
Gabor Buella b0f310d51d [x86] Introduce the pconfig intrinsic
Reviewers: craig.topper, zvi

Reviewed By: craig.topper

Differential Revision: https://reviews.llvm.org/D46431

llvm-svn: 331740
2018-05-08 06:49:41 +00:00
Gabor Buella a51e0c2243 [X86] directstore and movdir64b intrinsics
Reviewers: spatel, craig.topper, RKSimon

Reviewed By: craig.topper

Differential Revision: https://reviews.llvm.org/D45984

llvm-svn: 331249
2018-05-01 10:05:42 +00:00
Gabor Buella eba6c42e66 [X86] WaitPKG intrinsics
Reviewers: craig.topper, zvi

Reviewed By: craig.topper

Differential Revision: https://reviews.llvm.org/D45254

llvm-svn: 330463
2018-04-20 18:44:33 +00:00
Gabor Buella b220dd2b6c [X86] Introduce cldemote intrinsic
Reviewers: craig.topper, zvi

Reviewed By: craig.topper

Differential Revision: https://reviews.llvm.org/D45257

llvm-svn: 329993
2018-04-13 07:37:24 +00:00
Gabor Buella a052016ef2 [x86] wbnoinvd intrinsic
The WBNOINVD instruction writes back all modified
cache lines in the processor’s internal cache to main memory
but does not invalidate (flush) the internal caches.

Reviewers: craig.topper, zvi, ashlykov

Reviewed By: craig.topper

Differential Revision: https://reviews.llvm.org/D43817

llvm-svn: 329848
2018-04-11 20:09:09 +00:00
Saleem Abdulrasool 13aeee0d36 CodeGenCXX: support PreserveMostCC in MS ABI
Microsoft has reserved 'U' for the PreserveMostCC which is used in the
swift runtime.  Add support for this.  This allows the swift runtime to
be built for Windows again.

llvm-svn: 329025
2018-04-02 22:25:50 +00:00
Akira Hatanaka fcbe17c6be [ObjC++] Make parameter passing and function return compatible with ObjC
ObjC and ObjC++ pass non-trivial structs in a way that is incompatible
with each other. For example:
    
typedef struct {
  id f0;
  __weak id f1;
} S;
    
// this code is compiled in c++.
extern "C" {
  void foo(S s);
}
    
void caller() {
  // the caller passes the parameter indirectly and destructs it.
  foo(S());
}
    
// this function is compiled in c.
// 'a' is passed directly and is destructed in the callee.
void foo(S a) {
}
    
This patch fixes the incompatibility by passing and returning structs
with __strong or weak fields using the C ABI in C++ mode. __strong and
__weak fields in a struct do not cause the struct to be destructed in
the caller and __strong fields do not cause the struct to be passed
indirectly.
    
Also, this patch fixes the microsoft ABI bug mentioned here:
    
https://reviews.llvm.org/D41039?id=128767#inline-364710
    
rdar://problem/38887866
    
Differential Revision: https://reviews.llvm.org/D44908

llvm-svn: 328731
2018-03-28 21:13:14 +00:00
Dimitry Andric 2e3f23bbcc [X86] Add 'sahf' CPU feature to frontend
Summary:
Make clang accept `-msahf` (and `-mno-sahf`) flags to activate the
`+sahf` feature for the backend, for bug 36028 (Incorrect use of
pushf/popf enables/disables interrupts on amd64 kernels).  This was
originally submitted in bug 36037 by Jonathan Looney
<jonlooney@gmail.com>.

As described there, GCC also uses `-msahf` for this feature, and the
backend already recognizes the `+sahf` feature. All that is needed is to
teach clang to pass this on to the backend.

The mapping of feature support onto CPUs may not be complete; rather, it
was chosen to match LLVM's idea of which CPUs support this feature (see
lib/Target/X86/X86.td).

I also updated the affected test case (CodeGen/attr-target-x86.c) to
match the emitted output.

Reviewers: craig.topper, coby, efriedma, rsmith

Reviewed By: craig.topper

Subscribers: emaste, cfe-commits

Differential Revision: https://reviews.llvm.org/D43394

llvm-svn: 325446
2018-02-17 21:04:35 +00:00
Erich Keane d1d85f50d0 Add X86 Support to ValidCPUList (enabling march notes)
A followup to: https://reviews.llvm.org/D42978
This patch adds X86 and X86_64 support for
enabling the march notes.

Differential Revision: https://reviews.llvm.org/D43041

llvm-svn: 324674
2018-02-08 23:15:02 +00:00
Erich Keane 9176f669b4 [NFCi] Replace a couple of usages of const StringRef& with StringRef
No sense passing these by reference when a copy is about as free, and
saves on potential indirection later.

llvm-svn: 324540
2018-02-07 23:04:38 +00:00
Wei Mi d1621699dc Adjust MaxAtomicInlineWidth for i386/i486 targets.
This is to fix the bug reported in https://bugs.llvm.org/show_bug.cgi?id=34347#c6.
Currently, all  MaxAtomicInlineWidth of x86-32 targets are set to 64. However,
i386 doesn't support any cmpxchg related instructions. i486 only supports cmpxchg.
So in this patch MaxAtomicInlineWidth is reset as follows:
For i386, the MaxAtomicInlineWidth should be 0 because no cmpxchg is supported.
For i486, the MaxAtomicInlineWidth should be 32 because it supports cmpxchg.
For others 32 bits x86 cpu, the MaxAtomicInlineWidth should be 64 because of cmpxchg8b.

Differential Revision: https://reviews.llvm.org/D42154

llvm-svn: 323281
2018-01-23 23:27:57 +00:00
Chandler Carruth c58f2166ab Introduce the "retpoline" x86 mitigation technique for variant #2 of the speculative execution vulnerabilities disclosed today, specifically identified by CVE-2017-5715, "Branch Target Injection", and is one of the two halves to Spectre..
Summary:
First, we need to explain the core of the vulnerability. Note that this
is a very incomplete description, please see the Project Zero blog post
for details:
https://googleprojectzero.blogspot.com/2018/01/reading-privileged-memory-with-side.html

The basis for branch target injection is to direct speculative execution
of the processor to some "gadget" of executable code by poisoning the
prediction of indirect branches with the address of that gadget. The
gadget in turn contains an operation that provides a side channel for
reading data. Most commonly, this will look like a load of secret data
followed by a branch on the loaded value and then a load of some
predictable cache line. The attacker then uses timing of the processors
cache to determine which direction the branch took *in the speculative
execution*, and in turn what one bit of the loaded value was. Due to the
nature of these timing side channels and the branch predictor on Intel
processors, this allows an attacker to leak data only accessible to
a privileged domain (like the kernel) back into an unprivileged domain.

The goal is simple: avoid generating code which contains an indirect
branch that could have its prediction poisoned by an attacker. In many
cases, the compiler can simply use directed conditional branches and
a small search tree. LLVM already has support for lowering switches in
this way and the first step of this patch is to disable jump-table
lowering of switches and introduce a pass to rewrite explicit indirectbr
sequences into a switch over integers.

However, there is no fully general alternative to indirect calls. We
introduce a new construct we call a "retpoline" to implement indirect
calls in a non-speculatable way. It can be thought of loosely as
a trampoline for indirect calls which uses the RET instruction on x86.
Further, we arrange for a specific call->ret sequence which ensures the
processor predicts the return to go to a controlled, known location. The
retpoline then "smashes" the return address pushed onto the stack by the
call with the desired target of the original indirect call. The result
is a predicted return to the next instruction after a call (which can be
used to trap speculative execution within an infinite loop) and an
actual indirect branch to an arbitrary address.

On 64-bit x86 ABIs, this is especially easily done in the compiler by
using a guaranteed scratch register to pass the target into this device.
For 32-bit ABIs there isn't a guaranteed scratch register and so several
different retpoline variants are introduced to use a scratch register if
one is available in the calling convention and to otherwise use direct
stack push/pop sequences to pass the target address.

This "retpoline" mitigation is fully described in the following blog
post: https://support.google.com/faqs/answer/7625886

We also support a target feature that disables emission of the retpoline
thunk by the compiler to allow for custom thunks if users want them.
These are particularly useful in environments like kernels that
routinely do hot-patching on boot and want to hot-patch their thunk to
different code sequences. They can write this custom thunk and use
`-mretpoline-external-thunk` *in addition* to `-mretpoline`. In this
case, on x86-64 thu thunk names must be:
```
  __llvm_external_retpoline_r11
```
or on 32-bit:
```
  __llvm_external_retpoline_eax
  __llvm_external_retpoline_ecx
  __llvm_external_retpoline_edx
  __llvm_external_retpoline_push
```
And the target of the retpoline is passed in the named register, or in
the case of the `push` suffix on the top of the stack via a `pushl`
instruction.

There is one other important source of indirect branches in x86 ELF
binaries: the PLT. These patches also include support for LLD to
generate PLT entries that perform a retpoline-style indirection.

The only other indirect branches remaining that we are aware of are from
precompiled runtimes (such as crt0.o and similar). The ones we have
found are not really attackable, and so we have not focused on them
here, but eventually these runtimes should also be replicated for
retpoline-ed configurations for completeness.

For kernels or other freestanding or fully static executables, the
compiler switch `-mretpoline` is sufficient to fully mitigate this
particular attack. For dynamic executables, you must compile *all*
libraries with `-mretpoline` and additionally link the dynamic
executable and all shared libraries with LLD and pass `-z retpolineplt`
(or use similar functionality from some other linker). We strongly
recommend also using `-z now` as non-lazy binding allows the
retpoline-mitigated PLT to be substantially smaller.

When manually apply similar transformations to `-mretpoline` to the
Linux kernel we observed very small performance hits to applications
running typical workloads, and relatively minor hits (approximately 2%)
even for extremely syscall-heavy applications. This is largely due to
the small number of indirect branches that occur in performance
sensitive paths of the kernel.

When using these patches on statically linked applications, especially
C++ applications, you should expect to see a much more dramatic
performance hit. For microbenchmarks that are switch, indirect-, or
virtual-call heavy we have seen overheads ranging from 10% to 50%.

However, real-world workloads exhibit substantially lower performance
impact. Notably, techniques such as PGO and ThinLTO dramatically reduce
the impact of hot indirect calls (by speculatively promoting them to
direct calls) and allow optimized search trees to be used to lower
switches. If you need to deploy these techniques in C++ applications, we
*strongly* recommend that you ensure all hot call targets are statically
linked (avoiding PLT indirection) and use both PGO and ThinLTO. Well
tuned servers using all of these techniques saw 5% - 10% overhead from
the use of retpoline.

We will add detailed documentation covering these components in
subsequent patches, but wanted to make the core functionality available
as soon as possible. Happy for more code review, but we'd really like to
get these patches landed and backported ASAP for obvious reasons. We're
planning to backport this to both 6.0 and 5.0 release streams and get
a 5.0 release with just this cherry picked ASAP for distros and vendors.

This patch is the work of a number of people over the past month: Eric, Reid,
Rui, and myself. I'm mailing it out as a single commit due to the time
sensitive nature of landing this and the need to backport it. Huge thanks to
everyone who helped out here, and everyone at Intel who helped out in
discussions about how to craft this. Also, credit goes to Paul Turner (at
Google, but not an LLVM contributor) for much of the underlying retpoline
design.

Reviewers: echristo, rnk, ruiu, craig.topper, DavidKreitzer

Subscribers: sanjoy, emaste, mcrosier, mgorny, mehdi_amini, hiraditya, llvm-commits

Differential Revision: https://reviews.llvm.org/D41723

llvm-svn: 323155
2018-01-22 22:05:25 +00:00
Craig Topper 8cdb94901d [X86] Add rdpid command line option and intrinsics.
Summary: This patch adds -mrdpid/-mno-rdpid and the rdpid intrinsic. The corresponding LLVM commit has already been made.

Reviewers: RKSimon, spatel, zvi, AndreiGrischenko

Reviewed By: RKSimon

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D42272

llvm-svn: 323047
2018-01-20 18:36:52 +00:00
Oren Ben Simhon 57cc1a5d77 Added Control Flow Protection Flag
Cf-protection is a target independent flag that instructs the back-end to instrument control flow mechanisms like: Branch, Return, etc.
For example in X86 this flag will be used to instrument Indirect Branch Tracking instructions.

Differential Revision: https://reviews.llvm.org/D40478

Change-Id: I5126e766c0e6b84118cae0ee8a20fe78cc373dea
llvm-svn: 322063
2018-01-09 08:53:59 +00:00
Erich Keane 281d20b601 Implement Attribute Target MultiVersioning
GCC's attribute 'target', in addition to being an optimization hint,
also allows function multiversioning. We currently have the former
implemented, this is the latter's implementation.

This works by enabling functions with the same name/signature to coexist,
so that they can all be emitted. Multiversion state is stored in the
FunctionDecl itself, and SemaDecl manages the definitions.
Note that it ends up having to permit redefinition of functions so
that they can all be emitted. Additionally, all versions of the function
must be emitted, so this also manages that.

Note that this includes some additional rules that GCC does not, since
defining something as a MultiVersion function after a usage has been made illegal.

The only 'history rewriting' that happens is if a function is emitted before
it has been converted to a multiversion'ed function, at which point its name
needs to be changed.

Function templates and virtual functions are NOT yet supported (not supported
in GCC either).

Additionally, constructors/destructors are disallowed, but the former is 
planned.

llvm-svn: 322028
2018-01-08 21:34:17 +00:00
Coby Tayree a09663a5c1 [x86][icelake][vbmi2]
added vbmi2 feature recognition
added intrinsics support for vbmi2 instructions
_mm[128,256,512]_mask[z]_compress_epi[16,32]
_mm[128,256,512]_mask_compressstoreu_epi[16,32]
_mm[128,256,512]_mask[z]_expand_epi[16,32]
_mm[128,256,512]_mask[z]_expandloadu_epi[16,32]
_mm[128,256,512]_mask[z]_sh[l,r]di_epi[16,32,64]
_mm[128,256,512]_mask_sh[l,r]dv_epi[16,32,64]
matching a similar work on the backend (D40206)
Differential Revision: https://reviews.llvm.org/D41557

llvm-svn: 321487
2017-12-27 11:25:07 +00:00
Coby Tayree 3d9c88cfec [x86][icelake][vnni]
added vnni feature recognition
added intrinsics support for VNNI instructions
_mm256_mask_dpbusd_epi32
_mm256_maskz_dpbusd_epi32
_mm256_dpbusd_epi32
_mm256_mask_dpbusds_epi32
_mm256_maskz_dpbusds_epi32
_mm256_dpbusds_epi32
_mm256_mask_dpwssd_epi32
_mm256_maskz_dpwssd_epi32
_mm256_dpwssd_epi32
_mm256_mask_dpwssds_epi32
_mm256_maskz_dpwssds_epi32
_mm256_dpwssds_epi32
_mm128_mask_dpbusd_epi32
_mm128_maskz_dpbusd_epi32
_mm128_dpbusd_epi32
_mm128_mask_dpbusds_epi32
_mm128_maskz_dpbusds_epi32
_mm128_dpbusds_epi32
_mm128_mask_dpwssd_epi32
_mm128_maskz_dpwssd_epi32
_mm128_dpwssd_epi32
_mm128_mask_dpwssds_epi32
_mm128_maskz_dpwssds_epi32
_mm128_dpwssds_epi32
_mm512_mask_dpbusd_epi32
_mm512_maskz_dpbusd_epi32
_mm512_dpbusd_epi32
_mm512_mask_dpbusds_epi32
_mm512_maskz_dpbusds_epi32
_mm512_dpbusds_epi32
_mm512_mask_dpwssd_epi32
_mm512_maskz_dpwssd_epi32
_mm512_dpwssd_epi32
_mm512_mask_dpwssds_epi32
_mm512_maskz_dpwssds_epi32
_mm512_dpwssds_epi32
matching a similar work on the backend (D40208)
Differential Revision: https://reviews.llvm.org/D41558

llvm-svn: 321484
2017-12-27 10:37:51 +00:00
Coby Tayree 2268576fa0 [x86][icelake][bitalg]
added bitalg feature recognition
added intrinsics support for bitalg instructions
_mm512_popcnt_epi16
_mm512_mask_popcnt_epi16
_mm512_maskz_popcnt_epi16
_mm512_popcnt_epi8
_mm512_mask_popcnt_epi8
_mm512_maskz_popcnt_epi8
_mm512_mask_bitshuffle_epi64_mask
_mm512_bitshuffle_epi64_mask
_mm256_popcnt_epi16
_mm256_mask_popcnt_epi16
_mm256_maskz_popcnt_epi16
_mm128_popcnt_epi16
_mm128_mask_popcnt_epi16
_mm128_maskz_popcnt_epi16
_mm256_popcnt_epi8
_mm256_mask_popcnt_epi8
_mm256_maskz_popcnt_epi8
_mm128_popcnt_epi8
_mm128_mask_popcnt_epi8
_mm128_maskz_popcnt_epi8
_mm256_mask_bitshuffle_epi32_mask
_mm256_bitshuffle_epi32_mask
_mm128_mask_bitshuffle_epi16_mask
_mm128_bitshuffle_epi16_mask
matching a similar work on the backend (D40222)
Differential Revision: https://reviews.llvm.org/D41564

llvm-svn: 321483
2017-12-27 10:01:00 +00:00
Coby Tayree cf96c876c6 [x86][icelake][vpclmulqdq]
added vpclmulqdq feature recognition
added intrinsics support for vpclmulqdq instructions
  _mm256_clmulepi64_epi128
  _mm512_clmulepi64_epi128
matching a similar work on the backend (D40101)
Differential Revision: https://reviews.llvm.org/D41573

llvm-svn: 321480
2017-12-27 09:00:31 +00:00
Coby Tayree f4811ebc39 [x86][icelake][gfni]
added gfni feature recognition
added intrinsics support for gfni instructions
  _mm_gf2p8affineinv_epi64_epi8
  _mm_mask_gf2p8affineinv_epi64_epi8
  _mm_maskz_gf2p8affineinv_epi64_epi8
  _mm256_gf2p8affineinv_epi64_epi8
  _mm256_mask_gf2p8affineinv_epi64_epi8
  _mm256_maskz_gf2p8affineinv_epi64_epi8
  _mm512_gf2p8affineinv_epi64_epi8
  _mm512_mask_gf2p8affineinv_epi64_epi8
  _mm512_maskz_gf2p8affineinv_epi64_epi8
  _mm_gf2p8affine_epi64_epi8
  _mm_mask_gf2p8affine_epi64_epi8
  _mm_maskz_gf2p8affine_epi64_epi8
  _mm256_gf2p8affine_epi64_epi8
  _mm256_mask_gf2p8affine_epi64_epi8
  _mm256_maskz_gf2p8affine_epi64_epi8
  _mm512_gf2p8affine_epi64_epi8
  _mm512_mask_gf2p8affine_epi64_epi8
  _mm512_maskz_gf2p8affine_epi64_epi8
  _mm_gf2p8mul_epi8
  _mm_mask_gf2p8mul_epi8
  _mm_maskz_gf2p8mul_epi8
  _mm256_gf2p8mul_epi8
  _mm256_mask_gf2p8mul_epi8
  _mm256_maskz_gf2p8mul_epi8
  _mm512_gf2p8mul_epi8
  _mm512_mask_gf2p8mul_epi8
  _mm512_maskz_gf2p8mul_epi8
matching a similar work on the backend (D40373)
Differential Revision: https://reviews.llvm.org/D41582

llvm-svn: 321477
2017-12-27 08:37:47 +00:00
Coby Tayree a1e5f0c339 [x86][icelake][vaes]
added vaes feature recognition
added intrinsics support for vaes instructions, matching a similar work on the backend (D40078)
  _mm256_aesenc_epi128
  _mm512_aesenc_epi128
  _mm256_aesenclast_epi128
  _mm512_aesenclast_epi128
  _mm256_aesdec_epi128
  _mm512_aesdec_epi128
  _mm256_aesdeclast_epi128
  _mm512_aesdeclast_epi128

llvm-svn: 321474
2017-12-27 08:16:54 +00:00
Akira Hatanaka 502775a2ee [CodeGen][X86] Fix handling of __fp16 vectors.
This commit fixes a bug in IRGen where it generates completely broken
code for __fp16 vectors on X86. For example when the following code is
compiled:

half4 hv0, hv1, hv2; // these are vectors of __fp16.

void foo221() {
  hv0 = hv1 + hv2;
}

clang generates the following IR, in which two i16 vectors are added:

@hv1 = common global <4 x i16> zeroinitializer, align 8
@hv2 = common global <4 x i16> zeroinitializer, align 8
@hv0 = common global <4 x i16> zeroinitializer, align 8

define void @foo221() {
  %0 = load <4 x i16>, <4 x i16>* @hv1, align 8
  %1 = load <4 x i16>, <4 x i16>* @hv2, align 8
  %add = add <4 x i16> %0, %1
  store <4 x i16> %add, <4 x i16>* @hv0, align 8
  ret void
}

To fix the bug, this commit uses the code committed in r314056, which
modified clang to promote and truncate __fp16 vectors to and from float
vectors in the AST. It also fixes another IRGen bug where a short value
is assigned to an __fp16 variable without any integer-to-floating-point
conversion, as shown in the following example:

__fp16 a;
short b;

void foo1() {
  a = b;
}

@b = common global i16 0, align 2
@a = common global i16 0, align 2

define void @foo1() #0 {
  %0 = load i16, i16* @b, align 2
  store i16 %0, i16* @a, align 2
  ret void
}

rdar://problem/20625184

Differential Revision: https://reviews.llvm.org/D40112

llvm-svn: 320215
2017-12-09 00:02:37 +00:00
Martell Malone c950c651a4 Toolchain: Normalize dwarf, sjlj and seh eh
This is a re-apply of r319294.

adds -fseh-exceptions and -fdwarf-exceptions flags

clang will check if the user has specified an exception model flag,
in the absense of specifying the exception model clang will then check
the driver default and append the model flag for that target to cc1

-fno-exceptions has a higher priority then specifying the model

move __SEH__ macro definitions out of Targets into InitPreprocessor
behind the -fseh-exceptions flag

move __ARM_DWARF_EH__ macrodefinitions out of verious targets and into
InitPreprocessor behind the -fdwarf-exceptions flag and arm|thumb check

remove unused USESEHExceptions from the MinGW Driver

fold USESjLjExceptions into a new GetExceptionModel function that
gives the toolchain classes more flexibility with eh models

Reviewers: rnk, mstorsjo

Differential Revision: https://reviews.llvm.org/D39673

llvm-svn: 319297
2017-11-29 07:25:12 +00:00
Martell Malone 2fa25706ed Revert "Toolchain: Normalize dwarf, sjlj and seh eh"
This reverts rL319294.
The windows sanitizer does not like seh on x86.
Will re apply with None type for x86

llvm-svn: 319295
2017-11-29 06:51:27 +00:00
Martell Malone 390cfcb0b1 Toolchain: Normalize dwarf, sjlj and seh eh
adds -fseh-exceptions and -fdwarf-exceptions flags

clang will check if the user has specified an exception model flag,
in the absense of specifying the exception model clang will then check
the driver default and append the model flag for that target to cc1

clang cc1 assumes dwarf is the default if none is passed
and -fno-exceptions has a higher priority then specifying the model

move __SEH__ macro definitions out of Targets into InitPreprocessor
behind the -fseh-exceptions flag

move __ARM_DWARF_EH__ macrodefinitions out of verious targets and into
InitPreprocessor behind the -fdwarf-exceptions flag and arm|thumb check

remove unused USESEHExceptions from the MinGW Driver

fold USESjLjExceptions into a new GetExceptionModel function that
gives the toolchain classes more flexibility with eh models

Reviewers: rnk, mstorsjo

Differential Revision: https://reviews.llvm.org/D39673

llvm-svn: 319294
2017-11-29 06:25:13 +00:00
Oren Ben Simhon fec21ec0c6 Control-Flow Enforcement Technology - Shadow Stack and Indirect Branch Tracking support (Clang side)
Shadow stack solution introduces a new stack for return addresses only.
The stack has a Shadow Stack Pointer (SSP) that points to the last address to which we expect to return.
If we return to a different address an exception is triggered.
This patch includes shadow stack intrinsics as well as the corresponding CET header.
It includes CET clang flags for shadow stack and Indirect Branch Tracking.

For more information, please see the following:
https://software.intel.com/sites/default/files/managed/4d/2a/control-flow-enforcement-technology-preview.pdf

Differential Revision: https://reviews.llvm.org/D40224

Change-Id: I79ad0925a028bbc94c8ecad75f6daa2f214171f1
llvm-svn: 318995
2017-11-26 12:34:54 +00:00
Martell Malone 051e966e49 [MINGW] normalize WIN32 macros
move _WIN64 and _WIN32 defines to lib/Basic/Targets/OSTargets.h
move WIN32, WIN64 and __MINGW64__ to addMinGWDefines

fixes __MINGW64__ not being defined for aarch64
adds WIN32 definition for x64

Reviewers: mstorsjo

Differential Revision: https://reviews.llvm.org/D40285

llvm-svn: 318755
2017-11-21 11:28:29 +00:00
Erich Keane 6da1108659 Split x86 "Processor" info into its own def file. [NFC]
A first step toward removing the repetition of
features/CPU info in the x86 target info, this
patch pulls all the processor information out into
its own .def file.

Differential Revision: https://reviews.llvm.org/D40093

llvm-svn: 318343
2017-11-15 22:25:39 +00:00
Craig Topper a6021e3bc1 [X86] Make -march=i686 an alias of -march=pentiumpro
I think the only reason they are different is because we don't set tune_i686 for -march=i686 to match GCC. But GCC 4.9.0 seems to have changed this behavior and they do set it now. So I think they can aliases now.

Differential Revision: https://reviews.llvm.org/D39349

llvm-svn: 316712
2017-10-26 23:06:19 +00:00
Erich Keane 9ec60988cd Pull X86 "CPUKind" checking into .cpp file. [NFC]
Preparing to do a refactor of CPU/feature checking, this
patch pulls the one CPU implementation from the .h file
to the .cpp file.

llvm-svn: 316338
2017-10-23 16:20:15 +00:00
Craig Topper f8c10aa3a3 [X86] Add skeleton support for knm cpu
This adds support Knights Mill CPU. Preprocessor defines match gcc's implementation.

Differential Revision: https://reviews.llvm.org/D38813

llvm-svn: 315723
2017-10-13 18:14:24 +00:00
Saleem Abdulrasool 729379a1e1 Driver: hoist the `wchar_t` handling to the driver
Move the logic for determining the `wchar_t` type information into the
driver.  Rather than passing the single bit of information of
`-fshort-wchar` indicate to the frontend the desired type of `wchar_t`
through a new `-cc1` option of `-fwchar-type` and indicate the
signedness through `-f{,no-}signed-wchar`.  This replicates the current
logic which was spread throughout Basic into the
`RenderCharacterOptions`.

Most of the changes to the tests are to ensure that the frontend uses
the correct type.  Add a new test set under `test/Driver/wchar_t.c` to
ensure that we calculate the proper types for the various cases.

llvm-svn: 315126
2017-10-06 23:09:55 +00:00
Davide Italiano 64094f8621 [Targets/X86] Remove unneded `return` in setMaxAtomicWidth(). NFCI.
llvm-svn: 314367
2017-09-28 00:24:20 +00:00
Saleem Abdulrasool 4d321336d0 Basic: support Preserve{Most,All} CC on Windows
Add support for the `preserve_mostcc` and `preserve_allcc` on Windows
x86_64 and AArch64.  This is used by Swift.

llvm-svn: 314236
2017-09-26 19:26:01 +00:00
Wei Mi b086289787 [Atomic][X8664] set max atomic inline width according to the target
This is to fix PR31620. MaxAtomicInlineWidth is set to 128 for x86_64. However
for target without cx16 support, 128 atomic operation will generate __sync_*
libcalls. The patch set MaxAtomicInlineWidth to 64 if the target doesn't support
cx16.

Differential Revision: https://reviews.llvm.org/D38046

llvm-svn: 313992
2017-09-22 16:30:00 +00:00
Craig Topper 19a0a3ab68 [X86] Remove unnecessary extra encodings from the CPU name enum in clang
Summary:
For a lot of older CPUs we have a 1:1 mapping between CPU name and enum name. But many of them are effectively aliases of each other and as a result are always repeated together at every usage

This patch removes most of the duplication. It also uses StringSwitch::Cases to make the many to one mapping in the StringSwitch more obvious.

Reviewers: RKSimon, spatel, zvi, igorb

Reviewed By: RKSimon

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D37938

llvm-svn: 313462
2017-09-16 16:44:39 +00:00
Coby Tayree 7b49dc9c68 [Clang][x86][Inline Asm] support for GCC style inline asm - Y<x> constraints
This patch is intended to enable the use of basic double letter constraints used in GCC extended inline asm {Yi Y2 Yz Y0 Ym Yt}.
Supersedes D35205
llvm counterpart: D36369

Differential Revision: https://reviews.llvm.org/D36371

llvm-svn: 311643
2017-08-24 09:07:34 +00:00
Craig Topper 699ae0c173 [X86] Implement __builtin_cpu_is
This patch adds support for __builtin_cpu_is. I've tried to match the strings supported to the latest version of gcc.

Differential Revision: https://reviews.llvm.org/D35449

llvm-svn: 310657
2017-08-10 20:28:30 +00:00
Walter Lee 973131071a Move RTEMS to OSTargets.h
Differential Revision: https://reviews.llvm.org/D36106

llvm-svn: 309626
2017-07-31 21:00:16 +00:00
Martin Storsjo e2a247ccb0 [Targets] Move addCygMingDefines into the arch-independent Targets.cpp (NFC)
This fixes a dependency inconsistency, where addMinGWDefines in Targets.cpp
(used from other architectures than X86) called the addCygMingDefines function
in X86.h.

This was inconsistently split in SVN r308791 (D35701).

Differential Revision: https://reviews.llvm.org/D36072

llvm-svn: 309598
2017-07-31 18:17:38 +00:00
Erich Keane 2b9657b570 Remove Bitrig: Clang Changes
Bitrig code has been merged back to OpenBSD, thus the OS has been abandoned.

Differential Revision: https://reviews.llvm.org/D35708

llvm-svn: 308797
2017-07-21 22:46:31 +00:00
Erich Keane ebba592682 Break up Targets.cpp into a header/impl pair per target type[NFCI]
Targets.cpp is getting unwieldy, and even minor changes cause the entire thing 
to cause recompilation for everyone. This patch bites the bullet and breaks 
it up into a number of files.

I tended to keep function definitions in the class declaration unless it 
caused additional includes to be necessary. In those cases, I pulled it 
over into the .cpp file. Content is copy/paste for the most part, 
besides includes/format/etc.


Differential Revision: https://reviews.llvm.org/D35701

llvm-svn: 308791
2017-07-21 22:37:03 +00:00