Commit Graph

6079 Commits

Author SHA1 Message Date
Reid Kleckner 652d70f3e1 Fix most MSVC warnings in compiler-rt profiling library
Here's the warnings and how they were fixed:

- InstrProfilingUtil.c(110): warning C4013: '_open_osfhandle' undefined; assuming extern returning int

Include io.h to get the prototype.

- warning C4005: 'FILE_MAP_EXECUTE': macro redefinition

Stop trying to support pre-XP versions of Windows, don't attempt to
define this macro.

- InstrProfilingWriter.c(271): warning C4221: nonstandard extension used: 'Data': cannot be initialized using address of automatic variable 'Header'
- InstrProfilingWriter.c(275): warning C4221: nonstandard extension used: 'Data': cannot be initialized using address of automatic variable 'Zeroes'

Turn this warning off. This is definitely legal in C++, all compilers
accept it, and I only have room for half of one language standard in my
brain.

- InstrProfilingValue.c(320): warning C4113: 'uint32_t (__cdecl *)()' differs in parameter lists from 'uint32_t (__cdecl *)(void)'

Fix this with an explicit (void) in the prototype.

- InstrProfilingMerge.c.obj : warning LNK4006: _VPMergeHook already defined in InstrProfilingMergeFile.c.obj; second definition ignored

Last remaining warning. This is from linking a selectany definition with
a strong definition. We need to sort out weak symbols in compiler-rt in
general, though.

llvm-svn: 273026
2016-06-17 18:12:50 +00:00
Reid Kleckner d0680ad717 Fix some warnings in the MSVC build
- Fixes warnings about the ignored -fms-compatibility-version flag.
- Fixes warnings about overriding /W4 with /W3 and back.
- Fixes a warning where PREFETCH() expanded to nothing in a braceless if
  block.

llvm-svn: 273021
2016-06-17 17:48:52 +00:00
Qin Zhao e24bc365e8 [esan|cfrag] Add the struct field size array in StructInfo
Summary:
Adds the struct field size array in the struct StructInfo.

Prints struct field size info in the report.

Reviewers: aizatsky

Subscribers: vitalybuka, zhaoqin, kcc, eugenis, bruening, llvm-commits, kubabrecka

Differential Revision: http://reviews.llvm.org/D21342

llvm-svn: 272988
2016-06-17 04:50:11 +00:00
Evgeniy Stepanov 3c17c73924 [msan] Intercept send/sendto/sendmsg.
send/sendmsg moved from tsan to sanitizer_common; sendto is new.

llvm-svn: 272980
2016-06-17 00:43:11 +00:00
Kostya Serebryany bf6a04fde8 [sanitizers] introduce yet another API function: __sanitizer_install_malloc_and_free_hooks
llvm-svn: 272943
2016-06-16 20:06:06 +00:00
Derek Bruening 91c88e6558 [esan] Use internal_mmap during initialization
Fixes another interceptor issue where an app with a static tcmalloc
library that prevents our early-calloc handling from triggering yet
does not have a static mmap crashes in our mmap interceptor.  The
solution is to call internal_mmap when REAL(mmap) is not yet set up.

llvm-svn: 272870
2016-06-16 03:19:58 +00:00
Qin Zhao cd5fe14d89 [esan|cfrag] Minor cfrag report adjustment
Summary:
Reports the struct field access info only if it has been used at least
once.

Adds type printing size limit.

Formats the cache_frag.cpp with clang-format.

Reviewers: bruening

Subscribers: llvm-commits, eugenis, kcc, zhaoqin, vitalybuka, aizatsky, kubabrecka

Differential Revision: http://reviews.llvm.org/D21351

llvm-svn: 272810
2016-06-15 17:27:29 +00:00
Dmitry Vyukov 88f47d3521 tsan: intercept epoll_pwait
llvm-svn: 272779
2016-06-15 12:22:05 +00:00
Etienne Bergeron 89200ccefa In asan on Windows 64-bit, this is one of the broken things
that makes allocation fail. "UL" is 32-bit and shift by 40 will make
the value overflow and become 0.

Patch by Wei Wang

Differential Revision: http://reviews.llvm.org/D21310

llvm-svn: 272689
2016-06-14 18:05:45 +00:00
Vedant Kumar 8e2dd518af [profile] Update a warning message (NFC)
It's possible for a merge pool specifier to appear anywhere in a
filename pattern. Update the warning to reflect this.

llvm-svn: 272685
2016-06-14 17:23:13 +00:00
Adhemerval Zanella c0c6835096 [sanitizer] Revert new GLIBC msghdr/cmsghdr definitions
GLIBC reverted the POSIX conformance changes for msghdr/cmsghdr [1],
so there is no need to fix it on sanitizers.

[1] https://sourceware.org/git/?p=glibc.git;a=commit;h=78880cc185dc521855a58001a28e3059722d8e85

llvm-svn: 272678
2016-06-14 15:22:37 +00:00
Derek Bruening f6f149da29 [sanitizer][esan] Add internal_sigaction_syscall
Summary:
Adds a version of sigaction that uses a raw system call, to avoid circular
dependencies and support calling sigaction prior to setting up
interceptors.  The new sigaction relies on an assembly sigreturn routine
for its restorer, which is Linux x86_64-only for now.

Uses the new sigaction to initialize the working set tool's shadow fault
handler prior to libc interceptor being set up.  This is required to
support instrumentation invoked during interceptor setup, which happens
with an instrumented tcmalloc or other allocator compiled with esan.

Adds a test that emulates an instrumented allocator.

Reviewers: aizatsky

Subscribers: vitalybuka, tberghammer, zhaoqin, danalbert, kcc, srhines, eugenis, llvm-commits, kubabrecka

Differential Revision: http://reviews.llvm.org/D21083

llvm-svn: 272676
2016-06-14 15:15:38 +00:00
Nico Weber be7aa544f0 Revert r272591, http://lab.llvm.org:8011/builders/clang-x64-ninja-win7 has been broken since this landed.
llvm-svn: 272659
2016-06-14 12:39:22 +00:00
Derek Bruening c4c649e25a [sanitizer][esan] Add internal_sigaction_syscall
Summary:
Adds a version of sigaction that uses a raw system call, to avoid circular
dependencies and support calling sigaction prior to setting up
interceptors.  The new sigaction relies on an assembly sigreturn routine
for its restorer, which is Linux x86_64-only for now.

Uses the new sigaction to initialize the working set tool's shadow fault
handler prior to libc interceptor being set up.  This is required to
support instrumentation invoked during interceptor setup, which happens
with an instrumented tcmalloc or other allocator compiled with esan.

Adds a test that emulates an instrumented allocator.

Reviewers: aizatsky

Subscribers: vitalybuka, tberghammer, zhaoqin, danalbert, kcc, srhines, eugenis, llvm-commits, kubabrecka

Differential Revision: http://reviews.llvm.org/D21083

llvm-svn: 272591
2016-06-13 21:50:00 +00:00
Derek Bruening db176c11dc Revert "[sanitizer][esan] Add internal_sigaction_syscall"
This reverts commit r272553.

The iOS build fails to link.

llvm-svn: 272557
2016-06-13 16:16:49 +00:00
Derek Bruening 19a4b8d8f8 [sanitizer][esan] Add internal_sigaction_syscall
Summary:
Adds a version of sigaction that uses a raw system call, to avoid circular
dependencies and support calling sigaction prior to setting up
interceptors.  The new sigaction relies on an assembly sigreturn routine
for its restorer, which is Linux x86_64-only for now.

Uses the new sigaction to initialize the working set tool's shadow fault
handler prior to libc interceptor being set up.  This is required to
support instrumentation invoked during interceptor setup, which happens
with an instrumented tcmalloc or other allocator compiled with esan.

Adds a test that emulates an instrumented allocator.

Reviewers: aizatsky

Subscribers: vitalybuka, tberghammer, zhaoqin, danalbert, kcc, srhines, eugenis, llvm-commits, kubabrecka

Differential Revision: http://reviews.llvm.org/D21083

llvm-svn: 272553
2016-06-13 15:42:39 +00:00
Jeroen Ketema af0bbd92a2 [builtins] Make SOFT_FP versions of floatdidf and floatundidf compile
llvm-svn: 272551
2016-06-13 15:24:16 +00:00
Jeroen Ketema 0b748f220b [builtins] replace tabs by spaces and remove whitespace at end of line NFC
llvm-svn: 272550
2016-06-13 15:21:04 +00:00
Xinliang David Li 153e8b6c85 [profile] cleanup file setting code
Differential Revision: http://reviews.llvm.org/D21219

llvm-svn: 272428
2016-06-10 20:35:01 +00:00
Qin Zhao a4a7220db1 [esan|cfrag] Add the struct field offset array in StructInfo
Summary:
Adds the struct field offset array in the struct StructInfo.

Prints struct size and field offset info in the report.

Reviewers: aizatsky

Subscribers: vitalybuka, zhaoqin, kcc, eugenis, bruening, llvm-commits, kubabrecka

Differential Revision: http://reviews.llvm.org/D21191

llvm-svn: 272363
2016-06-10 02:10:34 +00:00
Xinliang David Li e2ce2e0020 [profile] in-process merging support part-3
Differential Revision: http://reviews.llvm.org/D21056

llvm-svn: 272227
2016-06-08 23:43:56 +00:00
Vedant Kumar 17af892d6f [profile] Hide some external symbols in InstrProfData.inc
Differential Revision: http://reviews.llvm.org/D21116

llvm-svn: 272167
2016-06-08 16:39:43 +00:00
Xinliang David Li f88d3b9afb [profile] Pass extra build flags (feature enabling macros) to Darwin build
Differential Revision: http://reviews.llvm.org/D21119

llvm-svn: 272162
2016-06-08 16:21:35 +00:00
Vedant Kumar 33b8b64aef [profile] Un-hide two symbols
- lprofCurFilename was intended to have external visibility. This is
  pending further discussion.
- The raw version number doesn't need to be hidden: hiding it may make
  it easier to accidentally combine FE/IR profiles.

See the mailing list discussion on r272081.

llvm-svn: 272089
2016-06-08 01:33:15 +00:00
Vedant Kumar b202ad6518 [profile] Hide a few external symbols (NFCI)
There are still a few external symbols visible from InstrProfData.inc.
The plan for dealing with those isn't as straightforward, so I'll try it
in a separate commit.

llvm-svn: 272081
2016-06-08 00:44:38 +00:00
Derek Bruening ff5cafa2ec [esan] Intercept calloc to avoid deadlocks with tcmalloc
Summary:
When tcmalloc initializes before esan, esan's initialization ends up
calling back into tcmalloc due to the calloc done by dlsym.  This results
in a deadlock.  We avoid this by special-casing this single allocation.

Intercepting calloc also gives us the opportunity to act on its zeroing as
stores by the application.

Reviewers: aizatsky

Subscribers: vitalybuka, zhaoqin, kcc, eugenis, llvm-commits, kubabrecka

Differential Revision: http://reviews.llvm.org/D21086

llvm-svn: 272076
2016-06-08 00:00:27 +00:00
Kuba Brecka 688c3d3bf1 [asan] Turn LSan-related #if’s into regular if’s in ASan initializer
Removing some preprocessor #if’s in favor of regular if’s. However, we need to declare empty stub functions to avoid linker errors.

Differential Revision: http://reviews.llvm.org/D20911

llvm-svn: 272047
2016-06-07 19:13:38 +00:00
Adhemerval Zanella 2c8fc6f8c9 [sanitizer] Fix build for new GLIBC msghdr/cmsghdr definition
GLIBC now follows POSIX [1] for both msghdr and cmsghdr definitions,
which means that msg_iovlen, msg_controllen, and cmsg_len are no
longer size_t but sockelen_t for 64-bits architectures. The final struct
size does not change, since paddings were added.

This patch fixes the build issue against GLIBC 2.24 socket.h header by
using the same definition for internal __sanitizer_msghdr and
__sanitizer_cmsghdr.

[1] http://pubs.opengroup.org/onlinepubs/9699919799/

llvm-svn: 272008
2016-06-07 13:19:38 +00:00
Kostya Serebryany 712fc9803a [sanitizer] Initial implementation of a Hardened Allocator
Summary:
This is an initial implementation of a Hardened Allocator based on Sanitizer Common's CombinedAllocator.
It aims at mitigating heap based vulnerabilities by adding several features to the base allocator, while staying relatively fast.
The following were implemented:
- additional consistency checks on the allocation function parameters and on the heap chunks;
- use of checksum protected chunk header, to detect corruption;
- randomness to the allocator base;
- delayed freelist (quarantine), to mitigate use after free and overall determinism.
Additional mitigations are in the works.

Reviewers: eugenis, aizatsky, pcc, krasin, vitalybuka, glider, dvyukov, kcc

Subscribers: kubabrecka, filcab, llvm-commits

Differential Revision: http://reviews.llvm.org/D20084

llvm-svn: 271968
2016-06-07 01:20:26 +00:00
Xinliang David Li 0891b6d59b [profile] code cleanup /NFC
Address review feedback for better
readability.

llvm-svn: 271922
2016-06-06 18:31:29 +00:00
Kuba Brecka 304b2c2a45 [tsan] Switch to InternalAlloc everywhere __libc_malloc is currently used
This patch replaces all uses of __libc_malloc and friends with the internal allocator.

It seems that the only reason why we have calls to __libc_malloc in the first place was the lack of the internal allocator at the time. Using the internal allocator will also make sure that the system allocator is never used (this is the same behavior as ASan), and we don’t have to worry about working with unknown pointers coming from the system allocator.

Differential Revision: http://reviews.llvm.org/D21025

llvm-svn: 271916
2016-06-06 18:18:47 +00:00
Etienne Bergeron 7a1bafd6ae This patch attempts to primitive support for Win64 asan
Some known issues are:

When "head" include instructions that involve branching, the "cut and paste" approach may break down in a way that function interception still work but calling back the original function does not work.
The jmp [rip -8] saves some bytes in the "head" but finding the safe zone of 0xCC is not implemented yet. So it may stomp on preceding codes.
The shadow offset is not working yet on Win64. More complexity maybe involved since there are some differences regarding virtual address space between Window 8 and Windows 8.1/10.

Patch by: Wang Wei

Differential Revision: http://reviews.llvm.org/D20884

llvm-svn: 271915
2016-06-06 18:09:54 +00:00
Kuba Brecka cdf3492191 [tsan] On OS X, optimize main thread’s ThreadState accesses
This is a very simple optimization that gets about 10% speedup for certain programs. We’re currently storing the pointer to the main thread’s ThreadState, but we can store the state directly in a static variable, which avoid the load acquire.

Differential Revision: http://reviews.llvm.org/D20910

llvm-svn: 271906
2016-06-06 16:27:38 +00:00
Xinliang David Li 5cd1f94d4f [profile] in-process mergeing support (part-2)
(Part-1 merging API is in profile runtime)

This patch implements a portable file opening API
with exclusive access for the process. In-process
profile merge requires profile file update to be
atomic/fully sychronized.

llvm-svn: 271864
2016-06-06 03:17:58 +00:00
Kuba Brecka 312071943c [tsan] On OS X, build Go runtime with -mmacosx-version-min
We're not building the Go runtime with -mmacosx-version-min, which means it'll have a minimum deployment target set to the system you're building on. Let's make the code compile (and link) with -mmacosx-version-min=10.7.

Differential Revision: http://reviews.llvm.org/D20670

llvm-svn: 271833
2016-06-05 15:15:36 +00:00
Derek Bruening c1c67d651d [esan] Initialize runtime during early interceptors
Summary:
Adds initialization of esan's runtime library during any early interceptors
that are sometimes called prior to the official __esan_init() invocation
(we see this with apps using tcmalloc).

Adds handling of interceptors called during interceptor initialization.

Reviewers: aizatsky

Subscribers: vitalybuka, zhaoqin, kcc, eugenis, llvm-commits, kubabrecka

Differential Revision: http://reviews.llvm.org/D20976

llvm-svn: 271744
2016-06-03 22:30:10 +00:00
Qin Zhao bc929e4765 [esan|cfrag] Compute the struct field access difference ratio
Summary:
Computes the struct field access variation based on each field access
count.

Adds a flag to control the report thresholds.

Updates struct-simple.cpp with variance report output.

Reviewers: aizatsky

Subscribers: kubabrecka, zhaoqin, llvm-commits, eugenis, vitalybuka, kcc, bruening

Differential Revision: http://reviews.llvm.org/D20914

llvm-svn: 271734
2016-06-03 20:48:17 +00:00
Derek Bruening 328e899301 [esan] Specify which tool via a global variable
Summary:
Adds a global variable to specify the tool, to support handling early
interceptors that invoke instrumented code, thus requiring shadow memory to
be initialized prior to __esan_init() being invoked.

Reviewers: aizatsky

Subscribers: vitalybuka, zhaoqin, kcc, eugenis, llvm-commits, kubabrecka

Differential Revision: http://reviews.llvm.org/D20974

llvm-svn: 271714
2016-06-03 19:40:08 +00:00
Derek Bruening e78e4a6e94 [esan|wset] Add 8-level working set snapshot accumulation
Summary:
Adds a new option -snapshot_step controlling the frequency distribution for
an 8-level series of samples using each bit of each shadow byte.
Implements accumulation from each level to the next higher level at the
specified frequency.

Adds storage of the 8 series of samples using CircularBuffer instances.
Fixes an error in the circular buffer data structure where a static
object's destructor will be called too early.

Prints the results out at the end in a simple manner to give us something
to start with.

Updates the workingset-samples test to test the new feature.

Reviewers: aizatsky

Subscribers: vitalybuka, zhaoqin, kcc, eugenis, llvm-commits, kubabrecka

Differential Revision: http://reviews.llvm.org/D20833

llvm-svn: 271683
2016-06-03 16:27:50 +00:00
Derek Bruening 07814769a8 [esan] Add sideline itimer support
Summary:
Adds support for creating a separate thread for performing "sideline"
actions on a periodic basis via an itimer.  A new class SidelineThread
implements this feature, exposing a sampling callback to the caller.

Adds initial usage of sideline sampling to the working set tool.  For now
it simply prints the usage at each snapshot at verbosity level 1.  Adds a
test of this behavior.  Adds a new option -record_snapshots to control
whether we sample and a new option -sample_freq to control the periodicity
of the sampling.

Reviewers: aizatsky

Subscribers: vitalybuka, zhaoqin, kcc, eugenis, llvm-commits, kubabrecka

Differential Revision: http://reviews.llvm.org/D20751

llvm-svn: 271682
2016-06-03 16:14:07 +00:00
Dmitry Vyukov 0f7e949a42 tsan: rely on AnnotateRWLockCreateStatic to detect linker-initialized mutexes
The new annotation was added a while ago, but was not actually used.
Use the annotation to detect linker-initialized mutexes instead
of the broken IsGlobalVar which has both false positives and false
negatives. Remove IsGlobalVar mess.

llvm-svn: 271663
2016-06-03 11:48:27 +00:00
Derek Bruening a601c2ada2 [esan] Ensure internal_sigaction() bypasses interceptors
Summary:
Implements real_sigaction() which it turns out is required for
internal_sigaction() to bypass the libc interceptors.

Without real_sigaction(), our internal_sigaction() calls during init happen
to work due to the EsanDuringInit check in COMMON_INTERCEPTOR_ENTER (though
even here it does not feel right for an "internal_" call to go through the
interceptor).  The real problem is when we call internal_sigaction() after
we're initialized, which only happens on an unhandled SIGSEGV for which the
app has no handler: then we'll spin in an infinite loop as our attempts to
remove our own handler repeatedly fail.  It's not easy to add a test for
that, unfortunately.

Reviewers: aizatsky

Subscribers: vitalybuka, zhaoqin, kcc, eugenis, llvm-commits, kubabrecka

Differential Revision: http://reviews.llvm.org/D20832

llvm-svn: 271626
2016-06-03 04:30:47 +00:00
Derek Bruening 0b04950116 [esan] Add support for log_exe_name
Summary:
Adds the call needed to cache the binary name to support the sanitizer
option log_exe_name for usable log file results when running multiple
applications.  Adds a test.

Reviewers: aizatsky

Subscribers: vitalybuka, zhaoqin, kcc, eugenis, llvm-commits, kubabrecka

Differential Revision: http://reviews.llvm.org/D20747

llvm-svn: 271586
2016-06-02 20:50:30 +00:00
Dmitry Vyukov 1ae44e6a2b tsan: clean up dynamic TLS memory between reuse
Currently the added test produces false race reports with glibc 2.19,
because DLTS memory is reused by pthread under the hood.

Use the DTLS machinery to intercept new DTLS ranges.
__tls_get_addr known to cause issues for tsan in the past,
so write the interceptor more carefully.

Reviewed in http://reviews.llvm.org/D20927

llvm-svn: 271568
2016-06-02 19:18:22 +00:00
Qin Zhao 4175a6d580 [esan|cfrag] Add struct info registration
Summary:
Adds StructInfo to CacheFragInfo to match the LLVM's EfficiencySanitizer
structs.

Uses StructHashMap to keep track of the struct info used by the app.

Adds registerStructInfo/unregisterStructInfo to add/remove struct infos
to/from StructHashMap.

updates test struct-simple.cpp with more C structs.

Reviewers: aizatsky, filcab

Subscribers: filcab, zhaoqin, llvm-commits, eugenis, vitalybuka, kcc, bruening, kubabrecka

Differential Revision: http://reviews.llvm.org/D20590

llvm-svn: 271564
2016-06-02 18:45:25 +00:00
Ivan Krasin 048155c399 UBSan: crash less often on corrupted Vtables.
Summary:
This CL adds a weak check for a Vtable prefix: for a well-formed
Vtable, we require the prefix to be within [-1<<20; 1<<20].

Practically, this solves most of the known cases when UBSan segfaults
without providing any useful diagnostics.

Reviewers: pcc

Subscribers: kubabrecka

Differential Revision: http://reviews.llvm.org/D19750

llvm-svn: 271560
2016-06-02 18:36:12 +00:00
Kostya Serebryany 3e7bf586f8 [asan] fix arm build
llvm-svn: 271474
2016-06-02 04:01:58 +00:00
Kostya Serebryany 5a7159c416 [asan] add an interface function __sanitizer_print_memory_profile (a basic memory profiler; asan/Linux-only for now)
llvm-svn: 271463
2016-06-02 01:21:52 +00:00
Xinliang David Li 96dad78963 [profile] Fix PR/27917
Skip the last (possibly) incomplete node from padding bytes.

llvm-svn: 271349
2016-05-31 23:12:13 +00:00
Qin Zhao 9e39638375 [esan|cfrag] Add the skeleton to handle the cfrag argument
Summary:
Adds the struct declaration for the cache-fragmentation tool variable
passed to the runtime library.

Updates test struct-simple.cpp.

Reviewers: aizatsky, bruening

Subscribers: filcab, kubabrecka, bruening, kcc, vitalybuka, eugenis, llvm-commits, zhaoqin

Differential Revision: http://reviews.llvm.org/D20542

llvm-svn: 271337
2016-05-31 21:27:39 +00:00
Chris Bieneman 5c3112cebe [CMake] Update to requiring CMake 3.4.3
Summary:
This is as per the discussions on developer lists:

http://lists.llvm.org/pipermail/llvm-dev/2016-April/098780.html
http://lists.llvm.org/pipermail/llvm-dev/2016-May/100058.html

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D20824

llvm-svn: 271327
2016-05-31 20:21:42 +00:00
Xinliang David Li b6d5f0b86f Local var name change /NFC
llvm-svn: 271304
2016-05-31 17:41:06 +00:00
Derek Bruening b7e38d8821 [esan] Add circular buffer data structure
Summary:
Adds a new class, CircularBuffer, for holding a wrap-around fixed-size
sequence of a primitive data type.  This will be used initially by the
working set tool.

Adds a unit test for CircularBuffer, including infrastructure support to
include esan headers and to link with the esan library by pretending to
want the working set tool.

Reviewers: aizatsky, filcab

Subscribers: vitalybuka, zhaoqin, kcc, eugenis, llvm-commits, kubabrecka

Differential Revision: http://reviews.llvm.org/D20579

llvm-svn: 271286
2016-05-31 14:44:49 +00:00
Derek Bruening 8ef3f0fa5b [esan|wset] Iterate all memory to compute the total working set
Summary:
Adds iteration of all application memory in an efficient manner using
shadow faults.  Shadow memory starts out inaccessible and we mark it
writable one page at a time on each fault when the instrumentation touches
it.  This allows iteration over just the mapped shadow memory, saving
significant time.

Adds a process-end iteration and pretty-printing of the final result.

Adds a new test and updates the existing tests.

Reviewers: aizatsky, filcab

Subscribers: vitalybuka, zhaoqin, kcc, eugenis, llvm-commits, kubabrecka

Differential Revision: http://reviews.llvm.org/D20578

llvm-svn: 271277
2016-05-31 13:41:07 +00:00
Derek Bruening 8e74c1084b [esan] Intercept and chain signal handlers
Summary:
In preparation for fault-based shadow memory iteration, we add support for
our own signal handler by adding app signal handler interception as well as
chaining for SIGSEGV.  This is done in a simple manner: we do not honor the
app's alternate stack nor any sigaction flags for SIGSEGV.

Adds a new test of transparency in app signal handling.

Reviewers: aizatsky

Subscribers: filcab, kubabrecka, vitalybuka, zhaoqin, kcc, eugenis, llvm-commits

Differential Revision: http://reviews.llvm.org/D20577

llvm-svn: 271272
2016-05-31 13:21:03 +00:00
Benjamin Kramer d53e3655e2 __builtin_ia32_storeups is no more. Replace it with xmmintrin.h intrinsic.
llvm-svn: 271257
2016-05-31 09:16:40 +00:00
Kuba Brecka 75b3da75b5 [asan] Fix a deadlock halt_on_error-signals.c when `reporting_thread_tid_` is 0
A signal can be delivered after TryLock but before StartReporting in ScopedInErrorReport, causing a deadlock.  Fixing this by statically initializing reporting_thread_tid_ to kInvalidTid.

Differential Revision: http://reviews.llvm.org/D20524

llvm-svn: 271256
2016-05-31 08:47:18 +00:00
Benjamin Kramer 1539cd326e Update asan_win_dll_thunk.cc to fix the windows buildbot.
llvm-svn: 271203
2016-05-30 10:00:54 +00:00
Benjamin Kramer 4acc312e47 Add __sanitizer_set_report_fd to the internal list of symbols.
Otherwise it will get the wrong visibility in the resulting library.

llvm-svn: 271202
2016-05-30 09:39:47 +00:00
Kostya Serebryany e31e778160 [sanitizers] introduce a common run-time option dedup_token_length to help with report deduplication, off by default for now. See https://github.com/google/sanitizers/issues/684
llvm-svn: 271085
2016-05-28 01:25:44 +00:00
Derek Bruening 909d32955b [esan] Add handling of large stack size rlimits
Summary:
Adds detection of large stack size rlimits (over 1 TB or unlimited), which
results in an mmap location that our shadow mapping does not support.  We
re-exec the application in this situation.  Adds a test of this behavior.

Adds general detection of mmap regions outside of our app regions.  In the
future we want to try to adaptively handle these but for now we abort.

Moves the existing Linux-specific mmap code into a platform-specific file
where the new rlimit code lives.

Reviewers: eugenis

Subscribers: vitalybuka, zhaoqin, kcc, aizatsky, llvm-commits, kubabrecka

Differential Revision: http://reviews.llvm.org/D20745

llvm-svn: 271079
2016-05-28 00:25:16 +00:00
Kostya Serebryany d9a451ab0d [sanitizer] Move termination functions in their own file
Summary:
As suggested by kcc@ in http://reviews.llvm.org/D20084#441418, move the CheckFailed and Die functions, and their associated callback functionalities in their own separate file.
I expended the build rules to include a new rule that would not include those termination functions, so that another project can define their own.
The tests check-{a,t,m,ub,l,e,df}san are all passing.

Reviewers: llvm-commits, kcc

Subscribers: kubabrecka

Differential Revision: http://reviews.llvm.org/D20742

llvm-svn: 271055
2016-05-27 21:57:17 +00:00
Kostya Serebryany 54b7162692 [sanitizers] trying to fix Windows build
llvm-svn: 271050
2016-05-27 21:35:58 +00:00
Etienne Bergeron 00f3f6e296 This patch is activating the build of Asan on Windows 64-bits.
It's fixing compilation errors. The runtime is not yet working.

Missing features:

OverrideFunction for x64
an equiv function for inline asm (atomic_compare_exchange_strong)
shadow memory offset needs to be adjusted
RoundUpToInstrBoundary for x64
They will be implemented by subsequent patches.

Patch by Wei Wang.

Differential revision: http://reviews.llvm.org/D20455

llvm-svn: 271049
2016-05-27 21:29:31 +00:00
Kostya Serebryany 1508f591b3 [sanitizers] introduce __sanitizer_set_report_fd so that we can re-route the sanitizer logging to another fd from inside the process
llvm-svn: 271046
2016-05-27 21:23:05 +00:00
Sean Silva e5e819b7d7 [profile] Avoid unused variable warning.
llvm-svn: 270969
2016-05-27 06:15:13 +00:00
Xinliang David Li 274cb1d224 Add portable macro in common header
llvm-svn: 270928
2016-05-26 22:15:12 +00:00
Xinliang David Li b033220085 Add segment prefix for darwin
llvm-svn: 270875
2016-05-26 17:21:24 +00:00
Vitaly Buka 132639120a Init tsan with .preinit_array section
Summary: Some libraries, like OpenSSL, runs code from .init section.

Reviewers: kcc, eugenis

Subscribers: kubabrecka, llvm-commits

Differential Revision: http://reviews.llvm.org/D20646

llvm-svn: 270873
2016-05-26 17:05:36 +00:00
Xinliang David Li a290c9b47a Fix windows bot failure
llvm-svn: 270864
2016-05-26 16:21:11 +00:00
Xinliang David Li 555cf5efbd [profile] pre-allocate a small counter set in profile runtime
Tested with relavant benchmarks in SPEC2006

Differential Revision: http://reviews.llvm.org/D20651

llvm-svn: 270862
2016-05-26 16:06:36 +00:00
Xinliang David Li 65971297a0 [profile] Add early checking to bypass node pointer update
llvm-svn: 270766
2016-05-25 21:27:02 +00:00
Sean Silva 5cf73b09cd [profile] Don't return `Node` when it is null.
The max warning check was masking the "return 0" codepath.

See the thread "Warnings and compile-time failure on 458.sjeng" for more
info.

llvm-svn: 270762
2016-05-25 21:08:38 +00:00
Qin Zhao 7e4933f430 [esan][cfrag] Add skeleton for cache fragmentation tool support
Summary:
Adds cache_frag.h and cache_frag.cpp for the cache fragmentation tool.

Updates test struct-simple.cpp.

Reviewers: aizatsky

Subscribers: filcab, zhaoqin, llvm-commits, eugenis, vitalybuka, kcc, bruening, kubabrecka

Differential Revision: http://reviews.llvm.org/D20538

llvm-svn: 270737
2016-05-25 17:49:00 +00:00
Xinliang David Li 7f08d12b0b [profile] Further cleanup/refactor file handling code
Also get rid of the redundant strncpy calls.

llvm-svn: 270730
2016-05-25 17:30:15 +00:00
Teresa Johnson c249167e90 Fix braced initializer list (NFC)
After r270617 I am getting an error when building:

projects/compiler-rt/lib/profile/InstrProfilingFile.c:33:36:
error: missing field 'PidChars' initializer
[-Werror,-Wmissing-field-initializers]
lprofFilename lprofCurFilename = {0};
                                   ^

Fix the aggregate initializer.

llvm-svn: 270697
2016-05-25 13:58:54 +00:00
Derek Bruening 88639859db Add working set base runtime library
Summary:
Adds the base runtime library for the working set tool.
Adds slowpath code for updating the shadow memory.

To be added in the future:
+ Scan memory and report the total size.
+ Take samples for intermediate values.

Reviewers: aizatsky

Subscribers: kubabrecka, vitalybuka, zhaoqin, kcc, eugenis, llvm-commits

Differential Revision: http://reviews.llvm.org/D20485

llvm-svn: 270650
2016-05-25 02:04:04 +00:00
Derek Bruening 7004e06b80 [esan] Add compilation unit init/exit routines.
Summary:
Adds processCompilationUnitInit and processCompilationUnitExit
for compilation unit init/exit.

Adds a tool-specific argument passed to esan_init.

Adds the dtor and esan_exit called from the dtor.

A test will be added separately (adding it here results in failure until
the corresponding compilation patch is in place).

Reviewers: aizatsky

Subscribers: kubabrecka, bruening, kcc, vitalybuka, eugenis, llvm-commits

Differential Revision: http://reviews.llvm.org/D20486

llvm-svn: 270624
2016-05-24 22:22:20 +00:00
Xinliang David Li 315e49d213 [profile] clean up profile file handling code
Differentianl Revision: http://reviews.llvm.org/D20572

llvm-svn: 270617
2016-05-24 21:29:18 +00:00
Evgeniy Stepanov 7b6f275b6a [asan] Workaround LD_PRELOAD bug in the Android linker.
llvm-svn: 270616
2016-05-24 21:25:00 +00:00
Xinliang David Li 41518945e3 Fix typo in comment
llvm-svn: 270524
2016-05-24 02:37:07 +00:00
Xinliang David Li 51fe002e67 [profile] clean up file initialization code
Also added more documentation.

llvm-svn: 270519
2016-05-24 01:23:09 +00:00
Xinliang David Li 66a891962b [profile] clean up runtime warnings.
o make warning message more meaningful to users.
 o add suggestion to fix the problem
 o limit the max number of output.

llvm-svn: 270483
2016-05-23 20:08:58 +00:00
Xinliang David Li 7b41393497 [profile] initialize static pool properly
Remove dependency on runtime initializer to avoid
issues related to initialization order.

llvm-svn: 270371
2016-05-22 16:36:03 +00:00
Xinliang David Li 9235d9f6d7 bug fix: trim section specifier name length
llvm-svn: 270350
2016-05-22 01:21:31 +00:00
Xinliang David Li 31023d3061 Fix a typo (darwin only)
llvm-svn: 270339
2016-05-21 23:27:18 +00:00
Xinliang David Li 2a99857e1a Fix typo
llvm-svn: 270338
2016-05-21 23:06:39 +00:00
Xinliang David Li 4e8754d2cb [profile] Static counter allocation for value profiling (part-2)
Differential Revision: http://reviews.llvm.org/D20460

llvm-svn: 270337
2016-05-21 22:55:45 +00:00
Kuba Brecka 0287e17d09 [tsan] Don't abort when a deadlock detector finds a mutex cycle longer than 10
In one of the already existing apps that I'm testing TSan on, I really see a mutex path that is longer than 10 (but not by much, something like 11-13 actually). Let's raise this to 20 and weaken the assertion so we don't crash.

Differential Revision: http://reviews.llvm.org/D20427

llvm-svn: 270319
2016-05-21 08:31:13 +00:00
Derek Bruening a85de06dd2 [esan] Add custom flag support
Summary:
Adds custom flag support to EfficiencySanitizer's runtime library.

Adds an initial flag cache_line_size which will be used by multiple tools.

Reviewers: aizatsky, vitalybuka

Subscribers: llvm-commits, eugenis, kcc, zhaoqin, aizatsky, kubabrecka

Differential Revision: http://reviews.llvm.org/D20478

llvm-svn: 270256
2016-05-20 19:26:52 +00:00
Derek Bruening c7f4922524 [esan] Extend shadow mapping to cover low libraries
Summary:
Adds support for app libraries starting slightly below 0x7f00'00000000 (the
mmap ASLR range extends down to 0x7efb'f8000000 for reasonable stack
limits) by switching to a shadow mapping offset of 0x1300'00000000.

Reviewers: aizatsky

Subscribers: vitalybuka, zhaoqin, kcc, eugenis, llvm-commits, kubabrecka

Differential Revision: http://reviews.llvm.org/D20479

llvm-svn: 270255
2016-05-20 19:19:06 +00:00
Sean Silva 357b0e3a7c Use C-style comment.
I think we allow use of C99 for libprofile, but use a C-style comment
for consistency.

llvm-svn: 270187
2016-05-20 06:13:07 +00:00
Xinliang David Li fb320a1137 Fix bot failure
llvm-svn: 270186
2016-05-20 05:40:07 +00:00
Xinliang David Li 690c31f104 [profile] PROF_ERR, PROF_WARN
1) Move common prefix to the macro def
 2) Introduced PROF_WARN
 3) Make error message unconditionally printed out.

llvm-svn: 270185
2016-05-20 05:15:42 +00:00
Xinliang David Li 71eddbf540 [profile] Remove anther malloc use
llvm-svn: 270181
2016-05-20 04:52:27 +00:00
Sean Silva b01e4b7727 Add #include for PS4 getenv shim.
llvm-svn: 270179
2016-05-20 03:43:39 +00:00
Sean Silva 29e434769e Small typo and whitespace fix.
llvm-svn: 270177
2016-05-20 03:25:15 +00:00
Xinliang David Li 5f153e686e [profile] entry eviction support in value profiler
Differential revision: http://reviews.llvm.org/D20408

llvm-svn: 270141
2016-05-19 21:35:34 +00:00
Benjamin Kramer f9679e89a1 Revert "[sanitizer] Move *fstat to the common interceptors"
This reverts commit r269981. Breaks msan tests on linux
http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux/builds/24019/steps/test%20standalone%20compiler-rt/logs/stdio

llvm-svn: 270076
2016-05-19 16:03:10 +00:00
Kuba Brecka ed29c21d5d [tsan] Add support for GCD's dispatch_after and dispatch_after_f
We're missing interceptors for dispatch_after and dispatch_after_f. Let's add them to avoid false positives. Added a test case.

Differential Revision: http://reviews.llvm.org/D20426

llvm-svn: 270071
2016-05-19 15:31:42 +00:00
Kuba Brecka 4087675948 Revert r270038 ("Change preprocessor `#if` to regular `if` for CAN_SANITIZE_LEAKS")
llvm-svn: 270044
2016-05-19 11:21:34 +00:00
Kuba Brecka b5cb227b98 [asan] Change preprocessor `#if` to regular `if` for CAN_SANITIZE_LEAKS
llvm-svn: 270038
2016-05-19 10:43:51 +00:00
Vedant Kumar 3afe657030 Reapply^3 "[ProfileData] (compiler-rt) Use Error in InstrProf and Coverage, NFC"
Sync up with "(llvm) Use Error in InstrProf and Coverage".

llvm-svn: 270022
2016-05-19 03:55:20 +00:00
Sean Silva aa705eccbe Fix typo.
llvm-svn: 270005
2016-05-19 00:12:02 +00:00
Xinliang David Li 4617aa7827 [profile] Allow max vals per site to be controllable at runtime
llvm-svn: 269993
2016-05-18 22:34:05 +00:00
Mike Aizatsky 522afdd77e [sanitizer] Move *fstat to the common interceptors
Summary:
Adds *fstat to the common interceptors.

Removes the now-duplicate fstat interceptor from msan/tsan
This adds fstat to asan/esan, which previously did not intercept it.

Resubmit of http://reviews.llvm.org/D20318 with ios build fixes.

Reviewers: eugenis, vitalybuka, aizatsky

Subscribers: zaks.anna, kcc, bruening, kubabrecka, srhines, danalbert, tberghammer

Differential Revision: http://reviews.llvm.org/D20350

llvm-svn: 269981
2016-05-18 20:49:49 +00:00
Xinliang David Li 32173153a4 [profile] add runtime variable documentation
llvm-svn: 269964
2016-05-18 17:44:57 +00:00
Francis Ricci 233a364bd8 [sanitizer] Allow dlopen/dlclose interception to be disabled from cmake
Summary:
dlopen and dlclose interception are broken when RUNPATH is used:
https://llvm.org/bugs/show_bug.cgi?id=27790

Reviewers: kutuzov.viktor.84, samsonov, dvyukov, eugenis, kcc

Subscribers: kcc, filcab, kubabrecka, compnerd, llvm-commits

Differential Revision: http://reviews.llvm.org/D20333

llvm-svn: 269947
2016-05-18 16:05:52 +00:00
Kuba Brecka daac6a0083 [sanitizer] Fix a crash when demangling Swift symbols, take 3
The previous patch (r269291) was reverted (commented out) because the patch caused leaks that
were detected by LSan and they broke some lit tests.  The actual reason was that dlsym allocates
an error string buffer in TLS, and some LSan lit tests are intentionally not scanning TLS for
root pointers.  This patch simply makes LSan ignore the allocation from dlsym, because it's
not interesting anyway.

llvm-svn: 269917
2016-05-18 13:00:20 +00:00
Sagar Thakur 7bec3a94c6 [LSAN] Fix test swapcontext.cc on MIPS
There is no frame validity check in the slow unwinder like there is in the fast unwinder due to which lsan reports a leak even for heap allocated coroutine in the test swapcontext.cc. Since mips/linux uses slow unwindwer instead of fast unwinder, the test fails for mips/linux. Therefore adding the checks before unwinding fixes the test for mips/linux.

Reviewed by aizatsky.
Differential: http://reviews.llvm.org/D19961

llvm-svn: 269882
2016-05-18 06:09:26 +00:00
Mike Aizatsky 41c2afe5d9 Revert "[sanitizer] Move *fstat to the common interceptors"
This reverts commit http://reviews.llvm.org/rL269856

llvm-svn: 269863
2016-05-17 23:28:56 +00:00
Mike Aizatsky 924acb50c9 [sanitizer] Move *fstat to the common interceptors
Summary:
Adds *fstat to the common interceptors.

Removes the now-duplicate fstat interceptor from msan/tsan
This adds fstat to asan/esan, which previously did not intercept it.

Reviewers: eugenis, vitalybuka, aizatsky

Subscribers: tberghammer, danalbert, srhines, kubabrecka, bruening, kcc

Differential Revision: http://reviews.llvm.org/D20318

llvm-svn: 269856
2016-05-17 22:26:50 +00:00
Anna Zaks 7d47c990e6 [tsan] Ensure mmap respects ignore_interceptors_accesses
The ignore_interceptors_accesses setting did not have an effect on mmap, so
let's change that. It helps in cases user code is accessing the memory
written to by mmap when the synchronization is ensured by the code that
does not get rebuilt.

(This effects Swift interoperability since it's runtime is mapping memory
which gets accessed by the code emitted into the Swift application by the
compiler.)

Differential Revision: http://reviews.llvm.org/D20294

llvm-svn: 269855
2016-05-17 22:24:55 +00:00
Derek Bruening 1f6d861ac2 [esan] Fix isShadowMem endpoint bug
Fixes a bug in checking the endpoint of a shadow region and removes an
invalid check (both introduced in http://reviews.llvm.org/rL269198).

llvm-svn: 269834
2016-05-17 20:44:09 +00:00
Mike Aizatsky 599eef49e5 [sanitizers] disabling LateInitialize call to fix the build.
http://reviews.llvm.org/rL269291 introduced a memory leak.
Disabling offending call temprorary rather than rolling back the chain
of CLs.

llvm-svn: 269799
2016-05-17 18:44:21 +00:00
Maxim Ostapenko 02c21b3ef9 [asan] Don't raise false alarm to recv/recvfrom when MSG_TRUNC is present.
Fix https://llvm.org/bugs/show_bug.cgi?id=27673.

Currenty ASan checks the return value of real recv/recvfrom to see if the written bytes fit in the buffer. That works fine most of time.
However, there is an exception: (from the RECV(2) man page)

MSG_TRUNC (since Linux 2.2)
    ... return the real length of the packet or datagram, even when it was longer than the passed buffer. ...

Some programs combine MSG_TRUNC, MSG_PEEK and a single-byte buffer to peek the incoming data size without reading (much of) them. In this case,
the return value is usually longer than what's been written and ASan raises a false alarm here. To avoid such false positive reports,
we can use min(res, len) in COMMON_INTERCEPTOR_WRITE_RANGE checks.

Differential Revision: http://reviews.llvm.org/D20280

llvm-svn: 269749
2016-05-17 07:38:27 +00:00
Sean Silva cfec6c6a28 Avoid leak. Free before resetting.
llvm-svn: 269724
2016-05-16 23:28:35 +00:00
Xinliang David Li 21d38c5eaf [profile] Add portability macro for atomic fetch_and_add
This is another enabler patch to support value profiling
without dynamic memory allocation.

llvm-svn: 269719
2016-05-16 23:01:03 +00:00
Vedant Kumar 533a88a5c9 Revert "Reapply^2 "[ProfileData] (compiler-rt) Use Error in InstrProf and Coverage, NFC""
This reverts commit r269696. The llvm commit does not pass the MSVC bot.

llvm-svn: 269702
2016-05-16 21:05:05 +00:00
Vedant Kumar cd5efa593b Reapply^2 "[ProfileData] (compiler-rt) Use Error in InstrProf and Coverage, NFC"
Sync up with "(llvm) Use Error in InstrProf and Coverage".

Differential Revision: http://reviews.llvm.org/D19903

llvm-svn: 269696
2016-05-16 20:50:36 +00:00
Xinliang David Li f90425e73b [profile] minor code restructuring /NFC
This is one of the enabler patch to allow value profiler to 
allocate counter statically.

llvm-svn: 269689
2016-05-16 20:33:30 +00:00
Paul Robinson 595b969c67 [PS4] Change the names of some "environmental" things to what our
licensees actually see in the toolchain we deliver to them.  This will
reduce the set of local patches we have to maintain.  The triple is
not changing.  (The term ORBIS is an internal code name for PS4.)

llvm-svn: 269672
2016-05-16 17:22:32 +00:00
Etienne Bergeron 3df2879232 [compiler-rt] Fix multi-configuration output paths
Summary:
When using a multi-configuration build (i.e. MSVC) the output path where
libraries are dropped is incorrect.

Example:
```
C:\src\llvm\examples>d:\src\llvm\build\Release\bin\clang-cl.exe -fsanitize=address test.cc
LINK : fatal error LNK1181: cannot open input file 'd:\src\llvm\build\Release\bin\..\lib\clang\3.9.0\lib\windows\clang_rt.asan-i386.lib'
```

The dropped executable path contains the configuration 'Release':
```
'd:\src\llvm\build\Release\bin\..\lib\clang\3.9.0\lib\windows\Release\clang_rt.asan-i386.lib'
```


The variable 'RUNTIME_OUTPUT_DIRECTORY' is used to specify the output directory.
But CMAKE is appending the current configuration (i.e. Debug, Release).

see: https://cmake.org/cmake/help/v3.0/prop_tgt/RUNTIME_OUTPUT_DIRECTORY.html
```
"Multi-configuration generators (VS, Xcode) append a per-configuration subdirectory to the specified directory."
```

To avoid this problem, the configuration specific variable must be set:
'RUNTIME_OUTPUT_DIRECTORY_DEBUG', 'RUNTIME_OUTPUT_DIRECTORY_RELEASE', and so on.

Reviewers: ddunbar, chapuni, rnk

Subscribers: kubabrecka, llvm-commits

Differential Revision: http://reviews.llvm.org/D20261

llvm-svn: 269658
2016-05-16 14:58:07 +00:00
Maxim Ostapenko 187043f61e [asan] Fix asan initialization failure with newer (2.23+) glibc in use.
This patch tries to fix https://llvm.org/bugs/show_bug.cgi?id=27310 by using the same hack for malloc as we use for calloc: allocate corresponding memory from internal buffer when ASan is not initialized.
This way we could avoid nasty '==6987==AddressSanitizer CHECK failed: ../../../../libsanitizer/asan/asan_rtl.cc:556 "((!asan_init_is_running && "ASan init calls itself!")) != (0)" (0x0, 0x0)' errors in
environments with glibc 2.23+ in use, where _dl_signal_error, called from dlsym for undefined symbols calls malloc in order to get a buffer for error message.

Differential Revision: http://reviews.llvm.org/D20235

llvm-svn: 269633
2016-05-16 07:20:53 +00:00
Xinliang David Li 6c7bddb28b Use __builtin_alloca with __GNUC__
llvm-svn: 269592
2016-05-15 16:41:58 +00:00
Xinliang David Li fedb0fd907 Fix FreeBSD build failure
llvm-svn: 269587
2016-05-15 04:26:17 +00:00
Xinliang David Li 23a66e45ed [profile] Eliminate dynamic memory allocation for vp writing
This is part-3 of the effort to eliminate dependency on
libc allocator in instr profiler runtime. With this change,
the profile dumper is completely free of malloc/calloc. 
Value profile instr API implementation is the only remaining
piece with calloc dependency.

llvm-svn: 269576
2016-05-14 20:12:42 +00:00
Chandler Carruth a3cd590644 Revert "Reapply "[ProfileData] (compiler-rt) Use Error in InstrProf and Coverage, NFC""
This reverts commit r269493 as the corresponding LLVM commit was
reverted due to lots of warnings. See the review thread for the original
LLVM commit (r269491) for details.

llvm-svn: 269550
2016-05-14 05:40:00 +00:00
Xinliang David Li a16c754e17 minor cleanup -- reset buffer pointer
llvm-svn: 269533
2016-05-14 03:16:47 +00:00
Vedant Kumar be1b4e4dc6 Reapply "[ProfileData] (compiler-rt) Use Error in InstrProf and Coverage, NFC"
Sync up with "(llvm) Use Error in InstrProf and Coverage".

Differential Revision: http://reviews.llvm.org/D19903

llvm-svn: 269493
2016-05-13 21:51:06 +00:00
Vedant Kumar bd3c2f25fa Revert "(HEAD -> master, origin/master, origin/HEAD) [ProfileData] (compiler-rt) Use Error in InstrProf and Coverage, NFC"
This reverts commit r269464. It fails two llvm-profdata tests.

llvm-svn: 269469
2016-05-13 20:10:49 +00:00
Vedant Kumar 6ecc10ab80 [ProfileData] (compiler-rt) Use Error in InstrProf and Coverage, NFC
Sync up with "(llvm) Use Error in InstrProf and Coverage".

llvm-svn: 269464
2016-05-13 20:01:38 +00:00
Xinliang David Li 609fae3893 [profile] Eliminate dynamic memory allocation for buffered writer
With this change, dynamic memory allocation is only used
for testing purpose. This change is one of the many steps to
make instrument profiler dynamic allocation free.

llvm-svn: 269453
2016-05-13 18:26:26 +00:00
Xinliang David Li d1dadf6bcf [profile] Code refactoring
Move runtime specific code from the common header file
to runtime source.

llvm-svn: 269357
2016-05-12 21:43:49 +00:00
Xinliang David Li 911af1ce15 Minor code refactoring /NFC
llvm-svn: 269351
2016-05-12 21:18:41 +00:00
Kuba Brecka a4c5bef998 More fixup for r269288 and r269296. Adding the missing 'return'.
llvm-svn: 269310
2016-05-12 16:58:19 +00:00
Kuba Brecka 11d3926cf6 [sanitizer] Allow nullptr in Swift demangler
The introduction of the Swift demangler now causes an assertion failure when we
try to demangle nullptr, but we used to allow that (and return nullptr back).
This situation is rare, but it can still happen.  Let's allow nullptr.

llvm-svn: 269302
2016-05-12 15:54:33 +00:00
Daniel Sanders 9ede03d4f2 [mips][sanitizer_common] Don't use `ld` in internal_clone() on 32-bit MIPS.
Summary:
On a 32-bit MIPS, the `ld` instruction does not exist. However, GAS has an `ld`
macro that expands to a pair of `lw` instructions which load to a pair of
registers (reg, and reg+1). This macro is not available in the Integrated
Assembler and its use causes -fintegrated-as builds to fail. Even if it were
available, the behaviour on 32-bit MIPS would be incorrect since the current
usage of `ld` causes the code to clobber $5 (which is supposed to hold
child_stack). It also clobbers $k0 which is reserved for kernel use.

Aside from enabling builds with the integrated assembler, there is no functional
change since internal_clone() is only used by StopTheWorld() which is only used
by 64-bit sanitizers.

Reviewers: kcc, sagar

Subscribers: mohit.bhakkad, jaydeep, sagar, llvm-commits

Differential Revision: http://reviews.llvm.org/D18753

llvm-svn: 269297
2016-05-12 14:21:33 +00:00
Kuba Brecka 00dde7563e Fixup for r269288. SleepForSeconds is not available in nolibc builds, let's add an internal_sleep.
llvm-svn: 269296
2016-05-12 14:08:56 +00:00
Kuba Brecka 3a0736e279 Fixup for r269291, which broke the Go TSan build. Let's not use the symbolizer in Go builds.
llvm-svn: 269293
2016-05-12 13:28:45 +00:00
Kuba Brecka 5bdf86ec7c [sanitizer] Fix a crash when demangling Swift symbols, take 2
To invoke the Swift demangler, we use dlsym to locate swift_demangle. However, dlsym malloc's storage and stores it in thread-local storage. Since allocations from the symbolizer are done with the system allocator (at least in TSan, interceptors are skipped when inside the symbolizer), we will crash when we try to deallocate later using the sanitizer allocator again.

To fix this, let's just not call dlsym from the demangler, and call it during initialization. The dlsym function calls malloc, so it needs to be only used after our allocator is initialized. Adding a Symbolizer::LateInitialize call that is only invoked after all other initializations.

Differential Revision: http://reviews.llvm.org/D20015

llvm-svn: 269291
2016-05-12 13:11:03 +00:00
Kuba Brecka 3f6f8de238 [sanitizer] On OS X, exit the forked process gracefully when login_tty fails
We're using forkpty to spawn the atos symbolizer. In some cases, login_tty (part of forkpty) can fail due to security measures (sandboxing). In this case, we should exit with a status code instead of completely crashing the spawned process. Even processing a failed CHECK() is problematic here, because we're post-fork and pre-exec where a lot of things don't work (for multithreaded processes, for OS X GUI apps, etc.).

Differential Revision: http://reviews.llvm.org/D20048

llvm-svn: 269289
2016-05-12 12:53:43 +00:00
Kuba Brecka e0c8256781 [sanitizer] Break infinite recursion in case of recursive failed CHECKs
While debugging ASan and TSan, I sometimes get a recursion during a failed CHECK processing.  CheckFailed can call a lot of code (printing, unwinding a stack trace, symbolicating, ...) and this can fail another CHECK. This means I sometimes see a crash due to a infinite recursion stack overflow. Let's stop after 10 failed CHECKs and just kill the process immediately. I also added a Sleep(2) call before the trap, so that other threads still get a chance to print their failed CHECKs.

Differential Revision: http://reviews.llvm.org/D20047

llvm-svn: 269288
2016-05-12 12:49:53 +00:00
Xinliang David Li 667de6a84e [profile] profile writing cleanup
Do not precompute value counts for all sites. This 
eliminates one more use of dynamic allocation 
in profiler writer.

llvm-svn: 269254
2016-05-11 23:21:22 +00:00
Xinliang David Li eb836a3dec cleanup: Enforce caller to set total size to avoid redundant size compute
llvm-svn: 269237
2016-05-11 21:16:11 +00:00
Evgeniy Stepanov 556e963e4e [asan] Don't use libcorkscrew on Android L+.
ASan runtime library used libcorkscrew from Android platform for
stack unwinding. Since Android L, this is both unnecessary (the
libgcc unwinder has been fixed) and impossible (the library is not
there any more). Don't even try.

This should have not effect on modern Android devices other than
removing a message about failing to open the library with
ASAN_OPTIONS=verbosity=1.

llvm-svn: 269233
2016-05-11 20:53:43 +00:00
Evgeniy Stepanov feb73c8566 [sanitizer] Move *stat to the common interceptors
Adds *stat to the common interceptors.

Removes the now-duplicate *stat interceptor from msan/tsan/esan.
This adds *stat to asan, which previously did not intercept it.

Patch by Qin Zhao.

llvm-svn: 269223
2016-05-11 20:02:15 +00:00
Derek Bruening 1658c089fd [esan] EfficiencySanitizer shadow memory
Summary:
Adds shadow memory mapping support common to all tools to the new
Efficiencysanitizer ("esan") family of tools.  This includes:

+ Shadow memory layout and mapping support for 64-bit Linux for any
  power-of-2 scale-down (1x, 2x, 4x, 8x, 16x, etc.) that ensures that
  shadow(shadow(address)) does not overlap shadow or application
  memory.

+ Mmap interception to ensure the application does not map on top of
  our shadow memory.

+ Init-time sanity checks for shadow regions.

+ A test of the mmap conflict mechanism.

Reviewers: aizatsky, filcab

Subscribers: filcab, kubabrecka, llvm-commits, vitalybuka, eugenis, kcc, zhaoqin

Differential Revision: http://reviews.llvm.org/D19921

llvm-svn: 269198
2016-05-11 15:47:54 +00:00
Dmitry Vyukov 2498629e34 tsan: fix another crash due to processors
Another stack where we try to free sync objects,
but don't have a processors is:

  //   ResetRange
  //   __interceptor_munmap
  //   __deallocate_stack
  //   start_thread
  //   clone

Again, it is a latent bug that lead to memory leaks.
Also, increase amount of memory we scan in MetaMap::ResetRange.
Without that the test does not fail, as we fail to free
the sync objects on stack.

llvm-svn: 269041
2016-05-10 11:19:50 +00:00
Xinliang David Li e9a8574da1 Reapply r268840: [profile] Simplify value profile writing
Revert r268864 that reverted 268840 after underlying problem
is fixed for arm bot.

llvm-svn: 268992
2016-05-10 00:17:31 +00:00
Marcin Koscielnicki a407f543c0 [MSan] Add a test for vararg with lots of non-vararg arguments.
This is a testcase for http://llvm.org/PR27646, hitting the bug on x86_64,
aarch64, mips.

Differential Revision: http://reviews.llvm.org/D19944

llvm-svn: 268981
2016-05-09 21:53:47 +00:00
Chris Bieneman c49e5e336b [CMake] Support platform building builtins without a full toolchain
Summary:
This patch adds support for building lib/builtins without a fully functioning toolchain. It allows you to bootstrap a cross-compiler, which previously couldn't be done with CMake.

This patch contains the following specific changes:

* Split builtin-specific code out of config-ix.cmake into builtin-config-ix.cmake
* Split some common CMake functionality needed by both builtins and sanitizers into base-config-ix.cmake
* Made lib/builtins/CMakeLists.txt able to be a top-level CMake configuration

I have tested this on Darwin targeting embedded Darwin, and on FreeBSD x86_64 targeting FreeBSD AArch64.

This patch depends on http://reviews.llvm.org/D19692, and is the last part of http://reviews.llvm.org/D16653.

Reviewers: samsonov, iains, jroelofs

Subscribers: compnerd, aemerson, tberghammer, danalbert, srhines, emaste, llvm-commits

Differential Revision: http://reviews.llvm.org/D19742

llvm-svn: 268977
2016-05-09 21:45:52 +00:00
Xinliang David Li b5a2b3a429 Fix variable visibility
llvm-svn: 268952
2016-05-09 19:01:19 +00:00
Renato Golin 26642b4035 Revert "[profile] Simplify value profile writing"
This reverts commit r268840, as it breaks Thumb2 self-hosting. There is something
unstable in the profiling for Thumb2 that needs to be sorted out before we continue
implementing these changes to the profiler. See PR27667.

llvm-svn: 268864
2016-05-07 20:07:09 +00:00
Xinliang David Li 4b30c79723 [profile] Simplify value profile writing
With this patch, value data are longer pre-collected
before writing. The code is simplified and requires
less heap space for dumping.

llvm-svn: 268840
2016-05-07 02:50:11 +00:00
Dmitry Vyukov 144eafd9ee tsan: fix a crash
Fixes crash reported in:
https://bugs.chromium.org/p/v8/issues/detail?id=4995

The problem is that we don't have a processor in a free interceptor
during thread exit.

The crash was introduced by introduction of Processors.
However, previously we silently leaked memory which wasn't any better.

llvm-svn: 268782
2016-05-06 19:35:22 +00:00
Kuba Brecka 12aaa1c955 Revert r268716. This has caused test failures in MSan on Linux.
llvm-svn: 268723
2016-05-06 10:23:10 +00:00
Kuba Brecka a609e37ebd [sanitizer] Fix a crash when demangling Swift symbols
To invoke the Swift demangler, we use dlsym to locate swift_demangle. However, dlsym malloc's storage and stores it in thread-local storage. Since allocations from the symbolizer are done with the system allocator (at least in TSan, interceptors are skipped when inside the symbolizer), we will crash when we try to deallocate later using the sanitizer allocator again.

To fix this, let's just not call dlsym from the demangler, and call it during initialization.

Differential Revision: http://reviews.llvm.org/D19974

llvm-svn: 268716
2016-05-06 08:54:58 +00:00
Maxim Ostapenko 6dccd5bc1f [asan] Bail out on stack overflow in recovery mode.
In recovery mode, when ASan detects stack overflow (say, when infinite recursion detected),
it tries to continue program execution and hangs on repetitive error reports. There isn't any
sense to do it, we can just bail out on stack overflow error, because the program would crash soon anyway.

Differential Revision: http://reviews.llvm.org/D19958

llvm-svn: 268713
2016-05-06 07:09:22 +00:00
Xinliang David Li ac96ea422b [profile] Remove another unneeded field in raw profile reader
DataValueSize is now removed. The change is consolidated
with previous raw version bump.

llvm-svn: 268704
2016-05-06 02:13:12 +00:00
Xinliang David Li 2cb6f0ce29 Sync up with master file
llvm-svn: 268603
2016-05-05 04:07:45 +00:00
Mike Aizatsky c826e634cc [sanitizer] Move stat/__xstat to the common interceptors
Summary:
Adds stat/__xstat to the common interceptors.

Removes the now-duplicate stat/__xstat interceptor from msan/tsan/esan.
This adds stat/__xstat to asan, which previously did not intercept it.

Resubmit of http://reviews.llvm.org/D19875 with win build fixes.

Reviewers: aizatsky, eugenis

Subscribers: tberghammer, llvm-commits, danalbert, vitalybuka, bruening, srhines, kubabrecka, kcc

Differential Revision: http://reviews.llvm.org/D19890

llvm-svn: 268466
2016-05-03 23:43:45 +00:00
Mike Aizatsky 7e72f66bf2 Revert "[sanitizer] Move stat/__xstat to the common interceptors"
This reverts commit 268440 because it breaks the windows bot.

http://lab.llvm.org:8011/builders/sanitizer-windows/builds/21425/steps/build%20compiler-rt/logs/stdio

llvm-svn: 268448
2016-05-03 21:49:56 +00:00
Mike Aizatsky 3eb521d417 [sanitizer] Move stat/__xstat to the common interceptors
Summary:
Adds stat/__xstat to the common interceptors.

Removes the now-duplicate stat/__xstat interceptor from msan/tsan/esan.
This adds stat/__xstat to asan, which previously did not intercept it.

Reviewers: aizatsky, eugenis

Subscribers: tberghammer, danalbert, srhines, kubabrecka, llvm-commits, vitalybuka, eugenis, kcc, bruening

Differential Revision: http://reviews.llvm.org/D19875

llvm-svn: 268440
2016-05-03 21:22:06 +00:00
Derek Bruening 2a14132285 [esan] Use do-while rather than braces for empty interceptors
Summary:
Replaces {} with a do..while sequence in esan's empty interceptors to allow
natural use with a trailing semicolon.  The sequence uses each argument to
avoid warnings.

Reviewers: filcab

Subscribers: kubabrecka, llvm-commits, zhaoqin

Differential Revision: http://reviews.llvm.org/D19832

llvm-svn: 268426
2016-05-03 19:44:32 +00:00
Dmitry Vyukov 7a1b7addf8 tsan: update tsan_analyze to what tip clang generates
We used to depend on host gcc. But some distributions got
new gcc recently which broke the check. Generally, we can't
depend that an arbitrary host gcc generates something stable.

Switch to clang.
This has an additional advantage of catching regressions in
clang codegen.

llvm-svn: 268382
2016-05-03 13:59:41 +00:00
Kostya Serebryany 9d91336a83 [compiler-rt] adjust platform_limits_linux.cc #include of posix_types
Summary:
Hello,

Building a recent gcc on a powerpc-linux system advertsing:

 Red Hat Enterprise Linux Server release 5.10 (Tikanga)

we stumbled on a compilation error on a file originating
from compiler-rt/lib/sanitizer-common.

sanitizer_platform_limits_linux.cc #includes asm/posix_types.h,
which, on our system, uses __kernel_fd_set and associated macros.
These aren't defined at the point of their use, and the compilation
fails with symptoms like:

  In file included from ../../../../src/libsanitizer/sanitizer_common/sanitizer_platform_limits_linux.cc:29:0:
  /usr/include/asm/posix_types.h:72:51: error: '__kernel_fd_set' has not been declared
  static __inline__ void __FD_SET(unsigned long fd, __kernel_fd_set *fdsetp)
  ...

The attached patch is a suggestion to fix this, by including linux/posix_types.h
instead of asm/posix_types.h. linux/posix_types defines the necessary types and
macros, then #includes asm/posix_types.h.

We have been using it locally for gcc without problems for a couple of years
on powerpc, x86 and x86_64-linux platforms. It is still needed for gcc-6 on
our powerpc host and applies cleanly on the compiler-rt trunk.

Comments ?

Thanks much in advance for your feedback,

With Kind Regards,

Olivier


Reviewers: llvm-commits, kcc

Subscribers: kcc, kubabrecka

Differential Revision: http://reviews.llvm.org/D19799

llvm-svn: 268283
2016-05-02 19:10:48 +00:00
Kostya Serebryany d83c96fa44 [scan-build] fix logic error warning emitted on compiler-rt code base
Summary:
Fix a "called c++ object pointer is null" warning emitted by Clang
Static Analyzer on the following file:
- lib/asan/asan_suppressions.cc.

Signed-off-by: Apelete Seketeli <apelete@seketeli.net>

Reviewers: kcc

Subscribers: Eugene.Zelenko, kubabrecka, llvm-commits

Differential Revision: http://reviews.llvm.org/D19627

llvm-svn: 268282
2016-05-02 19:07:20 +00:00
Kuba Brecka a90528bb89 [sanitizer] Fix a crash in SizeClassAllocator32 with an out-of-range pointer
This happens on a 64-bit platform that uses SizeClassAllocator32 (e.g. ASan on AArch64). When querying a large invalid pointer, `__sanitizer_get_allocated_size(0xdeadbeefdeadbeef)`, an assertion will fail.  This patch changes PointerIsMine to return false if the pointer is outside of [kSpaceBeg, kSpaceBeg + kSpaceSize).

Differential Revision: http://reviews.llvm.org/D15008

llvm-svn: 268243
2016-05-02 15:23:01 +00:00
Kuba Brecka d052a57900 [sanitizer] Don't reuse the main thread in ThreadRegistry
There is a hard-to-reproduce crash happening on OS X that involves terminating the main thread (dispatch_main does that, see discussion at http://reviews.llvm.org/D18496) and later reusing the main thread's ThreadContext. This patch disables reuse of the main thread. I believe this problem exists only on OS X, because on other systems the main thread cannot be terminated without exiting the process.

Differential Revision: http://reviews.llvm.org/D19722

llvm-svn: 268238
2016-05-02 15:06:08 +00:00
Marcin Koscielnicki ee587cba0e [ASan] Add shadow offset for SystemZ.
This is the compiler-rt counterpart to D19650.

Differential Revision: http://reviews.llvm.org/D19652

llvm-svn: 268162
2016-04-30 10:02:12 +00:00
Kuba Brecka 9ccde5ace4 [tsan] Return 0 from malloc_size for non-malloc'd pointers
In http://reviews.llvm.org/D19100, I introduced a bug: On OS X, existing programs rely on malloc_size() to detect whether a pointer comes from heap memory (malloc_size returns non-zero) or not. We have to distinguish between a zero-sized allocation (where we need to return 1 from malloc_size, due to other binary compatibility reasons, see http://reviews.llvm.org/D19100), and pointers that are not returned from malloc at all.

Differential Revision: http://reviews.llvm.org/D19653

llvm-svn: 268157
2016-04-30 07:14:41 +00:00
Filipe Cabecinhas 04d61050ea [asan] Assert in __sanitizer_ptr_{sub,cmp} if one of the pointers was freed.
Summary:
This (partially) implements the check mentioned at
http://kristerw.blogspot.co.uk/2016/04/dangling-pointers-and-undefined-behavior.html
(via John Regehr)

Quoting:
"That the behavior is undefined follows from C11 6.2.4 "Storage
durations of objects"
  The lifetime of an object is the portion of program execution during
  which storage is guaranteed to be reserved for it. An object exists, has
  a constant address, and retains its last-stored value throughout its
  lifetime. If an object is referred to outside of its lifetime, the
  behavior is undefined. The value of a pointer becomes indeterminate when
  the object it points to (or just past) reaches the end of its lifetime.
and 7.22.3 "Memory management functions" that says that free ends the
lifetime of objects
  The lifetime of an allocated object extends from the allocation until
  the deallocation.
"

We can probably implement this for stack variables too, but I think this
is a good start to see if there's interest in this check.
We can also hide this behind a flag, too.

Reviewers: samsonov, kcc, rsmith, regehr

Subscribers: kubabrecka, llvm-commits

Differential Revision: http://reviews.llvm.org/D19691

llvm-svn: 268097
2016-04-29 20:37:34 +00:00
Marcin Koscielnicki 14824b1c52 [sanitizers] [SystemZ] Mark kernel 3.12.58+ as safe from CVE-2016-2143.
llvm-svn: 268046
2016-04-29 14:59:19 +00:00
Evgeniy Stepanov 01c7450ec5 [msan] Tests for vector compare intrinsics.
llvm-svn: 267967
2016-04-29 01:20:05 +00:00
Marcin Koscielnicki 174f8b1981 [ASan] Reenable __builtin_setjmp test on PowerPC, disable on SystemZ.
Since __builtin_setjmp has been fixed by rL267943, the test now works
on PowerPC.  Enable it.

On the other hand, the SystemZ backend doesn't currently support
__builtin_setjmp.  Disable it.

Differential Revision: http://reviews.llvm.org/D19657

llvm-svn: 267946
2016-04-28 22:23:19 +00:00
Kuba Brecka dc7b607b09 [tsan] Fix Darwin GCD support after separation of Processor and ThreadState
Recent TSan changes (r267678) which factor out parts of ThreadState into a Processor structure broke worker threads on OS X.  This fixes it by properly calling ProcCreate for GCD worker threads and by replacing some CHECKs with RAW_CHECK in early process initialization.  CHECK() in TSan calls the allocator, which requires a valid Processor.

llvm-svn: 267864
2016-04-28 09:26:30 +00:00
Marcin Koscielnicki 0bec2fef8e [MSan] [PowerPC] Dereference function descriptors when recording stack origins.
Differential Revision: http://reviews.llvm.org/D19543

llvm-svn: 267795
2016-04-27 21:24:24 +00:00
Marcin Koscielnicki 66f0deacb5 [sanitizers] Get the proper symbol version when long double transition is involved.
On linux, some architectures had an ABI transition from 64-bit long double
(ie. same as double) to 128-bit long double.  On those, glibc symbols
involving long doubles come in two versions, and we need to pass the
correct one to dlvsym when intercepting them.

A few more functions we intercept are also versioned (all printf, scanf,
strtold variants), but there's no need to fix these, as the REAL() versions
are never called.

Differential Revision: http://reviews.llvm.org/D19555

llvm-svn: 267794
2016-04-27 21:24:21 +00:00
Derek Bruening 41939466d1 [sanitizer] Add early call handling to strchr + strrchr interceptors
Summary:
The strchr and strrchr interceptors are sometimes invoked too early
for their REAL() counterparts to be initialized.  We have seen this in
hooks invoked from tcmalloc on the dlsym() used in initializing
interceptors.  A special check is added to use internal_ routines for
this situation.

Reviewers: vitalybuka, aizatsky, filcab

Subscribers: filcab, llvm-commits, eugenis, kcc, zhaoqin, aizatsky, kubabrecka

Differential Revision: http://reviews.llvm.org/D19607

llvm-svn: 267793
2016-04-27 21:20:46 +00:00
Mike Aizatsky ccd318dc7e [sanitizers] read/write page fault detection on mac.
Summary: Resubmit of http://reviews.llvm.org/D19495 enabled only on intel.

Subscribers: kubabrecka

Differential Revision: http://reviews.llvm.org/D19561

llvm-svn: 267750
2016-04-27 18:02:21 +00:00
Marcin Koscielnicki b7b5ac60c4 [sanitizer] [SystemZ] Abort if the kernel might be vulnerable to CVE-2016-2143.
In short, CVE-2016-2143 will crash the machine if a process uses both >4TB
virtual addresses and fork().  ASan, TSan, and MSan will, by necessity, map
a sizable chunk of virtual address space, which is much larger than 4TB.
Even worse, sanitizers will always use fork() for llvm-symbolizer when a bug
is detected.  Disable all three by aborting on process initialization if
the running kernel version is not known to contain a fix.

Unfortunately, there's no reliable way to detect the fix without crashing
the kernel.  So, we rely on whitelisting - I've included a list of upstream
kernel versions that will work.  In case someone uses a distribution kernel
or applied the fix themselves, an override switch is also included.

Differential Revision: http://reviews.llvm.org/D19576

llvm-svn: 267747
2016-04-27 17:42:00 +00:00
Dmitry Vyukov 914b399736 tsan: fix windows support
UnmapOrDie used to do MEM_DECOMMIT and so worked
on partial regions. But r263160 changed it to use
MEM_RELEASE and MEM_RELEASE can only work with
whole regions mapped by VirtualAlloc. This broke
windows as:

FATAL: ThreadSanitizer CHECK failed: gotsan.cc:8296 "((mbi.AllocationBase == addr && "Windows cannot unmap part of a previous mapping")) != (0)" (0x0, 0x0)

Restore the previous behavior.

llvm-svn: 267730
2016-04-27 15:55:05 +00:00
Dmitry Vyukov 4592eb0534 tsan: fix darwin Go support
os_trace turns out to be a macro that creates static object.
Function-static objects use __cxa_atexit and __dso_handle
which are not present in Go runtime.

llvm-svn: 267720
2016-04-27 14:28:42 +00:00
Dmitry Vyukov 221e2c61ec tsan: fix windows Go support
Unmap can't unmap arbitrary regions on windows.

llvm-svn: 267716
2016-04-27 14:03:14 +00:00
Dmitry Vyukov 51c294a9e8 tsan: fix darwin Go build
syslog_lock is not defined in Go build.

llvm-svn: 267714
2016-04-27 13:40:05 +00:00
Dmitry Vyukov 3e0315c044 tsan: make windows shadow mapping a bijection
CheckShadowMapping function started catching that
mem->shadow->mem mapping is not bijection.
Make it bijection.

llvm-svn: 267713
2016-04-27 13:34:01 +00:00
Dmitry Vyukov 5aac22fc23 tsan: fix build
error: implicit declaration of function 'abort' is invalid in C99
llvm-svn: 267710
2016-04-27 12:59:35 +00:00
Dmitry Vyukov 30452f820b tsan: fix darwin Go build
Ifdef out global variables with destructors.
This requires runtime support that is not provided by Go runtime
(in particular _dso_handle symbol).

llvm-svn: 267709
2016-04-27 12:56:16 +00:00
Dmitry Vyukov 3efe395788 tsan: change tsan/Go interface for obtaining the current Processor
Current interface assumes that Go calls ProcWire/ProcUnwire
to establish the association between thread and proc.
With the wisdom of hindsight, this interface does not work
very well. I had to sprinkle Go scheduler with wire/unwire
calls, and any mistake leads to hard to debug crashes.
This is not something one wants to maintian.
Fortunately, there is a simpler solution. We can ask Go
runtime as to what is the current Processor, and that
question is very easy to answer on Go side.
Switch to such interface.

llvm-svn: 267703
2016-04-27 12:30:48 +00:00
Dmitry Vyukov a785640e83 tsan: fix windows build
warning: '__sanitizer::uptr __sanitizer::GetPreferredBase(const char*)' defined but not used
llvm-svn: 267682
2016-04-27 08:41:31 +00:00
Dmitry Vyukov f52c7c7abf tsan: fix windows build
llvm-svn: 267681
2016-04-27 08:39:32 +00:00
Dmitry Vyukov 3c65833a51 tsan: add missing file
llvm-svn: 267680
2016-04-27 08:34:55 +00:00
Dmitry Vyukov 066fefcbf3 tsan: fix compiler warning
tsan_debugging.cc: In function ‘void* __tsan_get_current_report()’:
tsan_debugging.cc:61:18: warning: cast from type ‘const __tsan::ReportDesc*’
                         to type ‘void*’ casts away qualifiers [-Wcast-qual]
   return (void *)rep;

llvm-svn: 267679
2016-04-27 08:28:08 +00:00
Dmitry Vyukov d87c7b321a tsan: split thread into logical and physical state
This is reincarnation of http://reviews.llvm.org/D17648 with the bug fix pointed out by Adhemerval (zatrazz).

Currently ThreadState holds both logical state (required for race-detection algorithm, user-visible)
and physical state (various caches, most notably malloc cache). Move physical state in a new
Process entity. Besides just being the right thing from abstraction point of view, this solves several
problems:

Cache everything on P level in Go. Currently we cache on a mix of goroutine and OS thread levels.
This unnecessary increases memory consumption.

Properly handle free operations in Go. Frees are issue by GC which don't have goroutine context.
As the result we could not do anything more than just clearing shadow. For example, we leaked
sync objects and heap block descriptors.

This will allow to get rid of libc malloc in Go (now we have Processor context for internal allocator cache).
This in turn will allow to get rid of dependency on libc entirely.

Potentially we can make Processor per-CPU in C++ mode instead of per-thread, which will
reduce resource consumption.
The distinction between Thread and Processor is currently used only by Go, C++ creates Processor per OS thread,
which is equivalent to the current scheme.

llvm-svn: 267678
2016-04-27 08:23:02 +00:00
Marcin Koscielnicki 3f9d7a217d [sanitizers] [NFC] Add defines for the various PowerPC ABIs.
Differential Revision: http://reviews.llvm.org/D19542

llvm-svn: 267586
2016-04-26 18:44:13 +00:00
Marcin Koscielnicki 7ecdeb7ac3 [sanitizer] [SystemZ] Add ptrace support bits.
Differential Revision: http://reviews.llvm.org/D19134

llvm-svn: 267548
2016-04-26 10:41:30 +00:00
Marcin Koscielnicki f98076b6a1 [sanitizer] [SystemZ] Implement internal_clone.
Differential Revision: http://reviews.llvm.org/D19159

llvm-svn: 267547
2016-04-26 10:41:19 +00:00
Mehdi Amini 9163f4997a Revert "[sanitizers] read/write page fault detection on mac."
This reverts commit r267477.
It broke our bots that enables the AArch64 backends, it seems that
this code is using a Darwin *X86 specific* field.

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 267526
2016-04-26 07:03:11 +00:00
Marcin Koscielnicki 9135ff9247 [MSan] Use COMMON_INTERCEPTOR_ENTER in libdl interceptors.
This fixes fails in test/msan/dlerror.cc - when real dlerror calls strcmp,
our strcmp interceptor now skips poison checking, since it's called in
interceptor context.  Strictly speaking, only the dlerror change is
necessary to fix the fail, but let's also change the other two just in case.

Differential Revision: http://reviews.llvm.org/D19499

llvm-svn: 267486
2016-04-25 22:25:49 +00:00
Mike Aizatsky f2bee298a0 [sanitizers] read/write page fault detection on mac.
Subscribers: kubabrecka

Differential Revision: http://reviews.llvm.org/D19495

llvm-svn: 267477
2016-04-25 21:44:43 +00:00
Derek Bruening 3441644245 [esan] Fix uninitialized warning from interception context
The interception context is not used by esan, but the compiler complains
about it being uninitialized all the same.  We set it to null to avoid the
warning.

llvm-svn: 267376
2016-04-25 03:56:20 +00:00
Derek Bruening 8d97011eb2 [esan] EfficiencySanitizer libc interceptors
Summary:
Adds libc interceptors to the runtime library for the new
EfficiencySanitizer ("esan") family of tools.  The interceptors cover
the memory operations in most common library calls and will be shared
among all esan tools.

Reviewers: aizatsky

Subscribers: zhaoqin, tberghammer, danalbert, srhines, llvm-commits, vitalybuka, eugenis, kcc

Differential Revision: http://reviews.llvm.org/D19411

llvm-svn: 267293
2016-04-23 16:41:24 +00:00
Kostya Serebryany 3884f1a5bd [sanitizer] add a function MmapNoAccess that mmaps a protected region *somewhere*; use MmapNoAccess in the Allocator when SpaceBeg is not a constant. In this mode the allocator will be a bit more hardened
llvm-svn: 267256
2016-04-23 00:05:24 +00:00
Kostya Serebryany 99ed605799 [sanitizer] rename MmapNoAccess to MmapFixedNoAccess; NFC
llvm-svn: 267253
2016-04-22 23:46:53 +00:00
Kostya Serebryany 58bc64432c [sanitizer] partially un-revert r267094: Allow the sanitizer allocator to use a non-fixed address range. An allocator with a non-fixed address range will be attack-resistan. NFC for the sanitizers at this point.
llvm-svn: 267252
2016-04-22 23:35:00 +00:00
Kostya Serebryany c2eb17db37 [sanitizer] revert r267203 as it breaks aarch64 bots
llvm-svn: 267251
2016-04-22 23:27:15 +00:00
Kostya Serebryany d4f275cecb [sanitizer] allow zero first parameter in MmapNoAccess; This is a part of r267094 that has been reverted
llvm-svn: 267203
2016-04-22 20:05:52 +00:00
Renato Golin e48f7e6f2d Revert "[LSAN] Fix test swapcontext.cc on MIPS"
This reverts commit r266716, as it breaks the self-hosting on Thumb2 buildbot.

llvm-svn: 267158
2016-04-22 14:56:30 +00:00
Renato Golin aa673486c4 Revert "[sanitizer] Allow the sanitizer allocator to use a non-fixed address range. An allocator with a non-fixed address range will be attack-resistan. NFC for the sanitizers at this point."
This reverts commit r267094, because it broke a lot of MSAN tests in AArch64.
Being NFC and all, this needs some deeper investigation before it goes in again.

llvm-svn: 267136
2016-04-22 11:18:14 +00:00
Sagar Thakur b1c51f6a94 [ASAN] Use struct instead of array in sancov.py
Summary: When using 32-bit python with 64-bit asan the pc array in sancov.py cannot fit in 64-bit pc's because the type-code 'L' for
arrays in python corresponds to the C type long which is only of 4 bytes. Because of this some of the coverage tool tests fail on
mips. To fix these test possible solutions are to use 64-bit python or use struct.unpack with the 'Q' type-code. We have used
struct.unpack with 'Q' type code since it is not appropriate to have a 64-bit python on all hosts.

Reviewed by kcc, aizatsky

Differential: http://reviews.llvm.org/D18817
llvm-svn: 267126
2016-04-22 09:20:22 +00:00
Kostya Serebryany d55d0875d2 [sanitizer] Allow the sanitizer allocator to use a non-fixed address range. An allocator with a non-fixed address range will be attack-resistan. NFC for the sanitizers at this point.
llvm-svn: 267094
2016-04-22 01:08:54 +00:00
Derek Bruening af7aaae1bc [esan] EfficiencySanitizer base runtime library
Summary:
Adds the initial version of a runtime library for the new
EfficiencySanitizer ("esan") family of tools.  The library includes:

+ Slowpath code via callouts from the compiler instrumentation for
  each memory access.

+ Registration of atexit() to call finalization code.

+ Runtime option flags controlled by the environment variable
  ESAN_OPTIONS.  The common sanitizer flags are supported such as
  verbosity and log_path.

+ An initial simple test.

Still TODO: common code for libc interceptors and shadow memory mapping,
and tool-specific code for shadow state updating.

Reviewers: eugenis, vitalybuka, aizatsky, filcab

Subscribers: filcab, vkalintiris, kubabrecka, llvm-commits, zhaoqin, kcc

Differential Revision: http://reviews.llvm.org/D19168

llvm-svn: 267060
2016-04-21 21:32:25 +00:00
Kuba Brecka bf8b5f8dd2 [tsan] Rename ReportThread->pid to ReportThread->os_id
The field "pid" in ReportThread is used to store the OS-provided thread ID (pthread_self or gettid). The name "pid" suggests it's a process ID, which it isn't. Let's rename it.

Differential Revision: http://reviews.llvm.org/D19365

llvm-svn: 266994
2016-04-21 14:49:25 +00:00
Kuba Brecka 586d93bd8b [sanitizer] Use pthread_threadid_np as thread ID on OS X
Let's use pthread_threadid_np which returns a more reasonable ID than pthread_self (which is actually a stack pointer). The numbers from pthread_threadid_np are already used in other tools, e.g. in LLDB, and often appear in logs, so it's much more useful than pthread_self.

Differential Revision: http://reviews.llvm.org/D18951

llvm-svn: 266991
2016-04-21 14:38:41 +00:00
Kostya Serebryany e63db09bea [sanitizer] update the sanitizer_allocator_testlib
llvm-svn: 266934
2016-04-20 23:28:06 +00:00
Evgeniy Stepanov 033c5d55c1 [asan] Increase LocatePcInTrace threshold.
Not sure what changed, but on my machine this is literally one byte
short. Only happens when malloc_context_size <= 2 due to the special
case in GET_STACK_TRACE definition (see asan_stack.h):
StackTrace::GetCurrentPc() on the right (context size > 2) branch
returns the address that is 200-something bytes from the return
address it is later matched to, while the same call on the left
branch is 321 bytes away from it.

This fixes the double-free test on my machine.

llvm-svn: 266932
2016-04-20 22:45:25 +00:00
Evgeniy Stepanov 5d71bc5252 [asan] Add __strdup interceptor.
This happens on Linux when building as C (not C++) with optimization.

llvm-svn: 266931
2016-04-20 22:45:23 +00:00
Evgeniy Stepanov 32773333cb [msan] Implement GetPageSize in the test.
Instead of calling a sanitizer_common function, implement GetPageSize in the
test directly. MSan runtime does not export __sanitizer::* symbols, and the
current code breaks when the test and the runtime library are in the separate
link units (ex. when the test is built as a shared library).

llvm-svn: 266910
2016-04-20 20:32:18 +00:00
Saleem Abdulrasool de120799a0 builtins: fix -Werror,-Wunused-variable warning
This removes the unused variable `flags`.  NFC

llvm-svn: 266892
2016-04-20 17:43:43 +00:00
Saleem Abdulrasool 28e1b977d4 builtins: remove use of __attribute__((pcs("aapcs"))) on Windows
Windows does not honour the __attribute__((pcs)) on ARM.  Although this will
result in ABI mismatches, compiler-rt should largely be unneeded for resolving
dependencies as we generate MS ABI compliant library calls now for the most
part.

llvm-svn: 266891
2016-04-20 17:43:40 +00:00
Chuang-Yu Cheng e480455fd6 Always inlining PrintCurrentStackSlow of tsan library to fix tail-call issue
The real problem is that sanitizer_print_stack_trace obtains current PC and
expects the PC to be in the stack trace after function calls. We don't
prevent tail calls in sanitizer runtimes, so this assumption does not
necessary hold.

We add "always inline" attribute on PrintCurrentStackSlow to address this
issue, however this solution is not reliable enough, but unfortunately, we
don't see any simple, reliable solution.

Reviewers: samsonov hfinkel kbarton tjablin dvyukov kcc

http://reviews.llvm.org/D19148

Thanks Hal, dvyukov, and kcc for invaluable discussion, I have even borrowed
part of dvyukov's summary as my commit message!

llvm-svn: 266869
2016-04-20 10:28:41 +00:00
Maxim Ostapenko 71630b9ea7 [sanitizer] Fix 'dyld: Symbol not found: _dyldVersionNumber' link error on old Darwin systems.
This patch fixes https://github.com/google/sanitizers/issues/669. On older Darwin systems (in particular, Darwin 10),
dyld doesn't export '_dyldVersionNumber' symbol so we would have 'undefined reference' error in sanitzer library. Although
sanitizers support was added to LLVM on OS X 10.7+ where '_dyldVersionNumber' symbol is already exported, GCC users still
may want use them on older systems.

Differential Revision: http://reviews.llvm.org/D19218

llvm-svn: 266868
2016-04-20 10:22:37 +00:00
Sagar Thakur 2e17dd5882 [LSAN] Fix test swapcontext.cc on MIPS
Summary: There is no frame validity check in the slow unwinder like there is in the fast unwinder due to which lsan reports a leak even for heap allocated coroutine in the test swapcontext.cc. Since mips/linux uses slow unwindwer instead of fast unwinder, the test fails for mips/linux. Therefore adding the checks before unwinding fixes the test for mips/linux.

Reviewers: samsonov, earthdok, kcc
Subscribers: llvm-commits, mohit.bhakkad, jaydeep
Differential: http://reviews.llvm.org/D18690
llvm-svn: 266716
2016-04-19 06:00:35 +00:00
Marcin Koscielnicki c5d2ff8099 [msan] Don't hardcode 4kiB page size in msan_test.cc.
This breaks the valloc test on PowerPC, which has 64kiB pages.  Since
getting page size portably is nontrivial, and there's already a function
for that in __sanitizer, just use it.  Unfortunately, sanitizer_common.h
conflicts with the interface headers inclucded by msan_test.cc (and a few
of its own macros), so we have to declare it manually.

Differential Revision: http://reviews.llvm.org/D19227

llvm-svn: 266688
2016-04-18 22:21:02 +00:00
Marcin Koscielnicki 3f89541a7d [MSan] [PowerPC] Add loadable library name for testing.
Differential Revision: http://reviews.llvm.org/D19217

llvm-svn: 266646
2016-04-18 18:21:41 +00:00
Marcin Koscielnicki 48f32510da [asan] [SystemZ] Add slop for stack address detection.
On s390, siginfo reports the faulting address with page granularity -
we need to mask off the low bits of sp before comparison.

Differential Revision: http://reviews.llvm.org/D19112

llvm-svn: 266593
2016-04-18 09:01:19 +00:00
Anna Zaks 8f8d3db940 [sanitizers] Fixup to r266494; the Unit test should not run on Windows
The test is failing on Windows because we do not have a definition for
DemangleSwiftAndCXX nor DemangleCXXABI, which I am replacing, on Windows.

llvm-svn: 266499
2016-04-16 00:32:16 +00:00
Anna Zaks 50154d4ec4 [sanitizers] Fixup r266494; make it build on Linux.
llvm-svn: 266497
2016-04-16 00:12:46 +00:00
Anna Zaks 60469e2ac0 [sanitizers] Teach the internal demangler about Swift names
Add support for Swift names when symbolicating sanitizer traces. This is
now relevant since TSan and ASan support have been added to Swift on OS X.

Differential Revision: http://reviews.llvm.org/D19135

llvm-svn: 266494
2016-04-15 23:51:00 +00:00
Marcin Koscielnicki b5e4804aee [sanitizer] [SystemZ] Enable UBSan.
sanitizer_common is now in good enough shape on s390x to support UBSan
- all tests passing.  Let's enable it.

Differential Revision: http://reviews.llvm.org/D19157

llvm-svn: 266483
2016-04-15 22:25:04 +00:00
Marcin Koscielnicki 604d873aa1 [sanitizers] [SystemZ] Introduce sanitizer_linux_s390.cc.
This file will contain s390-specific code.  For now, let's move the s390
version of internal_mmap here.

Differential Revision: http://reviews.llvm.org/D19174

llvm-svn: 266482
2016-04-15 22:11:10 +00:00
Marcin Koscielnicki 0a91cf8a84 Revert "[sanitizer] [SystemZ] Abort if the kernel might be vulnerable to CVE-2016-2143."
This reverts commit r266297.

llvm-svn: 266470
2016-04-15 20:00:12 +00:00
George Burgess IV 908dacf91d Fix StaticAnalyzer complaints. NFC.
Clang's StaticAnalyzer seems to (correctly) complain about code like:

    T *p = calloc(sizeof(U), N);

...Where T and U are different types.

This patch removes some instances of this pattern from compiler-rt.

Patch by Apelete Seketeli.
Differential Revision: http://reviews.llvm.org/D19085

llvm-svn: 266388
2016-04-14 23:58:26 +00:00
Marcin Koscielnicki 9e09355477 [sanitizer] [SystemZ] Fix stack traces.
On s390, the return address is in %r14, which is saved 14 words from
the frame pointer.

Unfortunately, there's no way to do a proper fast backtrace on SystemZ
with current LLVM - the saved %r15 in fixed-layout register save
area points to the containing frame itself, and not to the next one.
Likewise for %r11 - it's identical to %r15, unless alloca is used
(and even if it is, it's still useless).  There's just no way to
determine frame size / next frame pointer.  -mbackchain would fix that
(and make the current code just work), but that's not yet supported
in LLVM.  We will thus need to XFAIL some asan tests
(Linux/stack-trace-dlclose.cc, deep_stack_uaf.cc).

Differential Revision: http://reviews.llvm.org/D18895

llvm-svn: 266371
2016-04-14 21:19:27 +00:00
Marcin Koscielnicki 20bf94209e [sanitizer] [SystemZ] Add/fix kernel and libc type definitions.
This is the first part of upcoming asan support for s390 and s390x.
Note that there are bits for 31-bit support in this and subsequent
patches - while LLVM itself doesn't support it, gcc should be able
to make use of it just fine.

Differential Revision: http://reviews.llvm.org/D18888

llvm-svn: 266370
2016-04-14 21:17:19 +00:00
Ed Maste fe2b70637c [sanitizer] remove FreeBSD PS_STRINGS fallback
The PS_STRINGS constant can easily be incorrect with mismatched
kernel/userland - e.g. when building i386 sanitizers on FreeBSD/amd64
with -m32. The kern.ps_strings sysctl was introduced over 20 years ago
as the supported way to fetch the environment and argument string
addresses from the kernel, so the fallback is never used.

Differential Revision:	http://reviews.llvm.org/D19027

llvm-svn: 266305
2016-04-14 14:17:42 +00:00
Marcin Koscielnicki c8dda336bb [sanitizer] [SystemZ] Abort if the kernel might be vulnerable to CVE-2016-2143.
In short, CVE-2016-2143 will crash the machine if a process uses both >4TB
virtual addresses and fork().  ASan, TSan, and MSan will, by necessity, map
a sizable chunk of virtual address space, which is much larger than 4TB.
Even worse, sanitizers will always use fork() for llvm-symbolizer when a bug
is detected.  Disable all three by aborting on process initialization if
the running kernel version is not known to contain a fix.

Unfortunately, there's no reliable way to detect the fix without crashing
the kernel.  So, we rely on whitelisting - I've included a list of upstream
kernel versions that will work.  In case someone uses a distribution kernel
or applied the fix themselves, an override switch is also included.

Differential Revision: http://reviews.llvm.org/D18915

llvm-svn: 266297
2016-04-14 12:56:24 +00:00
Marcin Koscielnicki 0ffa9eaa4a [sanitizer] [SystemZ] Add virtual space size.
This teaches sanitizer_common about s390 and s390x virtual space size.
s390 is unusual in that it has 31-bit virtual space.

Differential Revision: http://reviews.llvm.org/D18896

llvm-svn: 266296
2016-04-14 12:56:15 +00:00
Marcin Koscielnicki 545e507c43 [sanitizer] [SystemZ] Implement internal_mmap.
mmap on s390 is quite a special snowflake: since it has too many
parameters to pass them in registers, it passes a pointer to a struct
with all the parameters instead.

Differential Revision: http://reviews.llvm.org/D18889

llvm-svn: 266295
2016-04-14 12:51:45 +00:00
Dmitry Vyukov 909d080d98 asan: fix build
Some bots failed with:

sanitizer_quarantine.h:104:7: error: unused typedef 'assertion_failed__104' [-Werror,-Wunused-local-typedef]
      COMPILER_CHECK(kPrefetch <= ARRAY_SIZE(b->batch));

Replace COMPILER_CHECK with CHECK.

llvm-svn: 266291
2016-04-14 11:40:08 +00:00
Dmitry Vyukov a7de697ae6 asan: fix out-of-bounds access in quarantine
llvm-svn: 266288
2016-04-14 09:52:33 +00:00
Kuba Brecka 173c690a61 [tsan] Fix size reporting for OS X zone allocator with 0-sized allocations
The custom zone implementation for OS X must not return 0 (even for 0-sized allocations). Returning 0 indicates that the pointer doesn't belong to the zone. This can break existing applications. The underlaying allocator allocates 1 byte for 0-sized allocations anyway, so returning 1 in this case is okay.

Differential Revision: http://reviews.llvm.org/D19100

llvm-svn: 266283
2016-04-14 09:05:19 +00:00
Evgeniy Stepanov b70a2dd157 [asan] Prefer alloc-dealloc-mismatch to new-delete-type-mismatch.
With -fsized-deallocation, new[] vs delete mismatch is reported as
new-delete-type-mismatch. This is technically true, but
alloc-dealloc-mismatch describes it better.

llvm-svn: 266246
2016-04-13 21:04:27 +00:00
Kuba Brecka 41dd5296b3 [tsan] Fix a crash with dispatch_source_set_cancel_handler(NULL) on OS X
We need to handle the case when handler is NULL in dispatch_source_set_cancel_handler and similar interceptors.

Differential Revision: http://reviews.llvm.org/D18968

llvm-svn: 266080
2016-04-12 15:18:11 +00:00
Kuba Brecka 036d060044 [sanitizer] Restore stderr when using forkpty() to spawn external symbolizer
In `AtosSymbolizer`, we're using `forkpty()` to create a new pseudo-terminal to communicate with the `atos` tool (we need that to avoid output buffering in interactive mode). This however redirects both stdout and stderr into a single stream, so when we read the output, we can't distinguish between errors and standard replies. Let's save&restore stderr to avoid that.

Differential Revision: http://reviews.llvm.org/D15073

llvm-svn: 265923
2016-04-11 09:27:09 +00:00
Derek Bruening a7685389c8 [sanitizer] Add early call handling to strlen interceptor
Summary:
The strlen interceptor is sometimes invoked too early for REAL(strlen) to
be initialized.  A special check is added to use internal_strlen for this
situation.

Reviewers: dim

Subscribers: llvm-commits, samsonov

Differential Revision: http://reviews.llvm.org/D18851

Change-Id: I3acc58f4abbae1904f25324abd84efea67aad0a2
llvm-svn: 265705
2016-04-07 18:07:09 +00:00
Kuba Brecka 74f7f399ac [tsan] Add support for OS X OSAtomic* functions
OS X provides atomic functions in libkern/OSAtomic.h. These provide atomic guarantees and they have alternatives which have barrier semantics. This patch adds proper TSan support for the functions from libkern/OSAtomic.h.

Differential Revision: http://reviews.llvm.org/D18500

llvm-svn: 265665
2016-04-07 12:05:09 +00:00
Kuba Brecka 06bbca9ec4 [tsan] Add TSan __tsan_atomic* functions to tsan_interface.h
To avoid using the public header (tsan_interface_atomic.h), which has different data types, let's add all the __tsan_atomic* functions to tsan_interface.h. 

Differential Revision: http://reviews.llvm.org/D18543

llvm-svn: 265663
2016-04-07 11:59:36 +00:00
Kuba Brecka 399af93242 [tsan] Add interceptors for dispatch_apply
Adding an interceptor with two more release+acquire pairs to avoid false positives with dispatch_apply.

Differential Revision: http://reviews.llvm.org/D18722

llvm-svn: 265662
2016-04-07 11:52:51 +00:00
Kuba Brecka e316bb61b3 [tsan] Add XPC support (OS X)
XPC APIs have async callbacks, and we need some more happen-before edges to avoid false positives. This patch add them, plus a test case (sorry for the long boilerplate code, but XPC just needs all that).

Differential Revision: http://reviews.llvm.org/D18493

llvm-svn: 265661
2016-04-07 11:47:11 +00:00
Kuba Brecka cecb7faea2 [tsan] Add support for dispatch event sources
GCD has APIs for event sources, we need some more release-acquire pairs to avoid false positives in TSan.

Differential Revision: http://reviews.llvm.org/D18515

llvm-svn: 265660
2016-04-07 11:38:53 +00:00
Kuba Brecka 33c15c91a6 [tsan] Fix synchronization in dispatch_sync
In the interceptor for dispatch_sync, we're currently missing synchronization between the callback and the code *after* the call to dispatch_sync. This patch fixes this by adding an extra release+acquire pair to dispatch_sync() and similar APIs. Added a testcase.

Differential Revision: http://reviews.llvm.org/D18502

llvm-svn: 265659
2016-04-07 11:33:44 +00:00
Xinliang David Li 5d2e06a34b sync up with master copy
llvm-svn: 265609
2016-04-06 22:30:14 +00:00
Yabin Cui c7db569cc1 [tsan] Fix freebsd build.
Summary:
Freebsd doesn't have <sys/personality.h>, so call personality() only in
SANITIZER_LINUX.

Reviewers: llvm-commits, dvyukov, zatrazz, rengolin, beanz

Subscribers: beanz, emaste

Differential Revision: http://reviews.llvm.org/D18785

llvm-svn: 265378
2016-04-05 05:37:48 +00:00
Yabin Cui e3f558ba14 [tsan] Disable randomized address space on aarch64 linux.
Summary:
After patch https://lkml.org/lkml/2015/12/21/340 is introduced in
linux kernel, the random gap between stack and heap is increased
from 128M to 36G on 39-bit aarch64. And it is almost impossible
to cover this big range. So we need to disable randomized virtual
space on aarch64 linux.

Reviewers: llvm-commits, zatrazz, dvyukov, rengolin

Subscribers: aemerson, rengolin, tberghammer, danalbert, srhines

Differential Revision: http://reviews.llvm.org/D18526

llvm-svn: 265366
2016-04-04 23:48:25 +00:00
Dmitry Vyukov 0d9b7bc9ad tsan: fix ignore handling in signal handlers
We've reset thr->ignore_reads_and_writes, but forget to do
thr->fast_state.ClearIgnoreBit(). So ignores were not effective
reset and fast_state.ignore_bit was corrupted if signal handler
itself uses ignores.

Properly reset/restore fast_state.ignore_bit around signal handlers.

llvm-svn: 265288
2016-04-04 10:52:59 +00:00
Kuba Brecka d0e83a6eb2 [sanitizer] Make AtosSymbolizer more resilient when symbolicating a zero address
llvm-svn: 265269
2016-04-03 19:13:03 +00:00
Reid Kleckner 1992ac75db Fix warnings uncovered by building with clang-cl
Move ifdefs to avoid unused static helpers. Move alignment attribute so
that it is respected in GCC and MSVC.

llvm-svn: 265153
2016-04-01 17:09:12 +00:00
Reid Kleckner a8e5c0c200 Fix a load of -Wunknown-pragma warnings from clang-cl
llvm-svn: 265152
2016-04-01 17:09:08 +00:00
Chris Bieneman ac400900da [Darwin] [Builtins] Cleaning up OS X exclude lists. NFC.
This just gets rid of a bunch of empty object file warnings. It doesn't impact the generated archives.

llvm-svn: 264751
2016-03-29 17:34:13 +00:00
Chris Bieneman 8de525696f [Darwin] [Builtins] Remove multi3 from the exclude list.
This addresses PR27077. For some historical reason Darwin wasn't shipping multi3 in the compiler builtin library or in the OS builtin library. This caused building ffmpeg to fail because Polly was generating calls to multi3. It is easy enough to just add the builtin.

llvm-svn: 264750
2016-03-29 17:24:23 +00:00
Chris Bieneman c4613f457a [Darwin] [Builtins] Cleaning up 10.4 exclude lists NFC
This just gets rid of a bunch of empty object file warnings. It doesn't impact the generated archives.

llvm-svn: 264749
2016-03-29 17:24:21 +00:00
Adam Nemet fadba5d4d4 [PGO] More comments how function pointers for indirect calls are mapped
to function names

Summary:
Hopefully this will make it easier for the next person to figure all
this out...

Reviewers: bogner, davidxl

Subscribers: davidxl, cfe-commits

Differential Revision: http://reviews.llvm.org/D18489

llvm-svn: 264680
2016-03-28 22:16:29 +00:00
Sean Silva c2feac75fd [libprofile] Handle '\\' in __llvm_profile_recursive_mkdir
This is implicitly needed at least by gcc-flag-compatibility.test

The thing that needs it is the `\` preceding the "default.profraw"
appended internally by clang when doing `-fprofile-use=`.

Clang uses `\` because is uses sys::path::append which will use `\` on a
Windows host. This is wrong, but I don't think there's an easy way to
solve it (maybe just always using `/` since places that accept `\` also
tend to accept `/`, but not the other way around).

llvm-svn: 264665
2016-03-28 21:32:46 +00:00
Ryan Govostes dc91fe5d8b [asan] Add runtime support for __asan_(un)register_image_globals
This change introduces routines that register and unregister all
instrumented globals in a loaded executable image.

These routines are only implemented on Darwin, where globals metadata
is expected to be placed in the __DATA,__asan_globals section.

Review: http://reviews.llvm.org/D16841
llvm-svn: 264644
2016-03-28 20:28:17 +00:00
Kuba Brecka 0d026d9e9e [tsan] Fix a crash when exiting the main thread (e.g. dispatch_main)
This patch fixes the custom ThreadState destruction on OS X to avoid crashing when dispatch_main calls pthread_exit which quits the main thread.

Differential Revision: http://reviews.llvm.org/D18496

llvm-svn: 264627
2016-03-28 19:36:25 +00:00
Adam Nemet a68d755774 [PGO] Update r264612 to use C-style comment
llvm-svn: 264615
2016-03-28 18:47:44 +00:00
Adam Nemet 42a6fe199f [PGO] Comment how function pointers for indirect calls are mapped to function names
Summary:
Hopefully this will make it easier for the next person to figure all
this out...

Reviewers: bogner, davidxl

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D18490

llvm-svn: 264612
2016-03-28 18:29:36 +00:00
Kostya Serebryany b48f8f09b1 [asan] bump the scariness score of read-after-frees (based on feedback from the Chrome security team)
llvm-svn: 264481
2016-03-26 00:00:19 +00:00
Derek Bruening e988af9073 [sanitizer] Add memset, memmove, and memcpy to the common interceptors
Summary:
Currently, sanitizer_common_interceptors.inc has an implicit, undocumented
assumption that the sanitizer including it has previously declared
interceptors for memset and memmove.  Since the memset, memmove, and memcpy
routines require interception by many sanitizers, we add them to the
set of common interceptions, both to address the undocumented assumption
and to speed future tool development.  They are intercepted under a new
flag intercept_intrin.

The tsan interceptors are removed in favor of the new common versions.  The
asan and msan interceptors for these are more complex (they incur extra
interception steps and their function bodies are exposed to the compiler)
so they opt out of the common versions and keep their own.

Reviewers: vitalybuka

Subscribers: zhaoqin, llvm-commits, kcc

Differential Revision: http://reviews.llvm.org/D18465

llvm-svn: 264451
2016-03-25 19:33:45 +00:00
Reid Kleckner 3b0290570b [asan] Intercept all Heap* related imports from ucrtbase.dll
ucrtbase.dll appears to be built with some kind of cross-module
inlining, because there are calls to imported Heap* routines sprinkled
throughout the code. This inlining defeats our attempts to hotpatch
malloc, _malloc_base, and related functions. Failing to intercept an
allocation or deallocation results in a crash when the program attempts
to deallocate or reallocate memory with the wrong allocator.

This change patches the IAT of ucrtbase.dll to replace the addresses of
the imported Heap* functions with implementations provided by ASan.  We
don't globally intercept the win32 Heap* functions because they are
typically used by system DLLs that run before ASan initializes.
Eventually, we may want to intercept them, but for now I think this is
the minimal change that will keep ASan stable.

Reviewers: samsonov

Differential Revision: http://reviews.llvm.org/D18413

llvm-svn: 264327
2016-03-24 20:19:48 +00:00
Kuba Brecka 3bab18d4af [tsan] Fix fork() and fork-based tests for OS X
On OS X, fork() under TSan asserts (in debug builds only) because REAL(fork) calls some intercepted functions, which check that no internal locks are held via CheckNoLocks(). But the wrapper of fork intentionally holds some locks. This patch fixes that by using ScopedIgnoreInterceptors during the call to REAL(fork). After that, all the fork-based tests seem to pass on OS X, so let's just remove all the UNSUPPORTED: darwin annotations we have.

Differential Revision: http://reviews.llvm.org/D18409

llvm-svn: 264261
2016-03-24 11:54:33 +00:00
Kuba Brecka 46b9363683 [tsan] Use direct syscalls for internal_mmap and internal_munmap on OS X
On OS X, internal_mmap just uses mmap, which can invoke callbacks into libmalloc (e.g. when MallocStackLogging is enabled). This can subsequently call other intercepted functions, and this breaks our Darwin-specific ThreadState initialization. Let's use direct syscalls in internal_mmap and internal_munmap. Added a testcase.

Differential Revision: http://reviews.llvm.org/D18431

llvm-svn: 264259
2016-03-24 11:50:21 +00:00
Derek Bruening b584410b80 [sanitizer] Add strnlen to the common interceptors
Summary:
Adds strnlen to the common interceptors, under the existing flag
intercept_strlen.

Removes the now-duplicate strnlen interceptor from asan and msan.
This adds strnlen to tsan, which previously did not intercept it.

Adds a new test of strnlen to the sanitizer_common test cases.

Reviewers: samsonov

Subscribers: zhaoqin, llvm-commits, kcc

Differential Revision: http://reviews.llvm.org/D18397

llvm-svn: 264195
2016-03-23 21:24:28 +00:00
Reid Kleckner 03d703bb62 [asan] Export new and delete operators on Windows
This is necessary to support the dynamic CRT (/MD) with VS2015. In
VS2015, these symbols are no longer imported from a DLL, they provided
statically by msvcrt.lib. This means our approach of hotpatching the DLL
no longer works.

By exporting the symbols, we end up relying on the same mechanism that
we use to intercept symbols in the static CRT (/MT) case. The ASan
runtime always needs to appear first on the link line, and the linker
searches for symbol definitions from left to right. This means we can
stop hotpatching operator new and delete in the CRT, which is nice.

I think that the only reason we weren't exporting the symbols already is
because MSVC doesn't allow you to do it directly with
__declspec(dllexport). Instead, we can use
`#pragma comment(linker, "/export:foo")`, which is most of what the
attribute does under the hood. It does mean we have to write down the
mangled names of the operators, but that's not too bad.

llvm-svn: 264190
2016-03-23 20:45:52 +00:00
Renato Golin c8622e8cde Revert "[tsan] Disable randomized address space on linux aarch64."
This reverts commits r264068 and r264079, and they were breaking the build and
weren't reverted in time, nor they exhibited expected behaviour from the
reviewers. There is more to discuss than just a test fix.

llvm-svn: 264150
2016-03-23 11:24:30 +00:00
Derek Bruening bffdcde03d Test commit to verify repository access and fix a typo.
llvm-svn: 264112
2016-03-22 22:32:03 +00:00
Yabin Cui 28f367d7b3 [tsan] Fix check-tsan build by using CHECK_NE.
Reviewers: llvm-commits, srhines, dvyukov

Subscribers: srhines

Differential Revision: http://reviews.llvm.org/D18361

llvm-svn: 264079
2016-03-22 18:12:18 +00:00
Yabin Cui 909e600abd [tsan] Disable randomized address space on linux aarch64.
Summary:
After patch https://lkml.org/lkml/2015/12/21/340 is introduced in
linux kernel, the random gap between stack and heap is increased
from 128M to 36G on 39-bit aarch64. And it is almost impossible
to cover this big range. So I think we need to disable randomized
virtual space on aarch64 linux.

Reviewers: kcc, llvm-commits, eugenis, zatrazz, dvyukov, rengolin

Subscribers: rengolin, aemerson, tberghammer, danalbert, srhines, enh

Differential Revision: http://reviews.llvm.org/D18003

llvm-svn: 264068
2016-03-22 17:16:26 +00:00
Reid Kleckner d83633f552 [asan] Add ucrtbase.dll to the list of DLLs to intercept
Reduces number of test failures in check-asan-dynamic with VS 2015.

llvm-svn: 264061
2016-03-22 15:46:50 +00:00
Reid Kleckner 2310c658d8 [asan] Add one more x86 encoding to the interceptor for strrchr
llvm-svn: 264060
2016-03-22 15:46:43 +00:00
Reid Kleckner f2be78c07e Move -fms-compatibility-version=19 into target cflags
This reduces cflags duplication and allows us to build
sanitizer_common/tests with clang and the VS 2015 STL.

llvm-svn: 264017
2016-03-22 01:14:36 +00:00
Reid Kleckner ea4ae806bb Wrap 81 character line to satisfy linter
llvm-svn: 264014
2016-03-22 00:53:04 +00:00
Reid Kleckner 23d0fde535 [asan] Intercept strdup on Windows
Some unit tests were failing because we didn't intercept strdup.  It
turns out it works just fine on 2013 and 2015 with a small patch to the
interception logic.

llvm-svn: 264013
2016-03-22 00:52:47 +00:00
Reid Kleckner 79d2451cd7 [asan] Also apply r264006 to asan_win_dynamic_runtime_thunk.cc
llvm-svn: 264011
2016-03-22 00:34:53 +00:00
Reid Kleckner 81f7729c3d [asan] Set the unhandled exception filter slightly later during startup
VS 2015 moved the priority of their exception filter initializer from
XIY to XCAA. We now set ours to XCAB, which makes it run after both CRT
versions but before user constructors, as it should.

Fixes null_deref.cc and a variety of related tests with VS 2015. Only 4
failures remain.

llvm-svn: 264006
2016-03-21 23:51:21 +00:00
Reid Kleckner ed051774ba [asan] Add new _*_base interceptors for VS 2015
There are some places in the CRT (such as mbctype) that directly call
_malloc_base. If you are incrementally linking a binary with ASan from
before this change, this change appears to result in a linker error.
Retrying the link succeeds for some reason.

llvm-svn: 264005
2016-03-21 23:51:17 +00:00
Evgeniy Stepanov 7923ce692e [asan] Add strrchr to asan_win_dll_thunk.
"dll_host.cc" test says there is a mismatch.

llvm-svn: 263998
2016-03-21 22:42:15 +00:00
Evgeniy Stepanov accf390a75 [sanitizer] Add strchr* to the common interceptors
Adds strchr, strchrnul, and strrchr to the common interceptors, under a new
common flag intercept_strchr.

Removes the now-duplicate strchr interceptor from asan and all 3
interceptors from tsan. Previously, asan did not intercept strchrnul, but
does now; previously, msan did not intercept strchr, strchrnul, or strrchr,
but does now.

http://reviews.llvm.org/D18329

Patch by Derek Bruening!

llvm-svn: 263992
2016-03-21 21:36:17 +00:00
Mike Aizatsky 5d17878101 [sanitizier] initializing common flags.
Summary: Without SetDefaults() CommonFlags contain garbage.

Differential Revision: http://reviews.llvm.org/D18326

llvm-svn: 263985
2016-03-21 20:53:20 +00:00
Reid Kleckner a4192a743a [asan] Pass -fms-compatibility-version=19 with VS 2015
This resolves errors about char16_t and char32_t when compiling 2015 STL
headers with clang.

llvm-svn: 263973
2016-03-21 19:37:30 +00:00
Reid Kleckner d2f05f52b4 [asan] Update hotpatch code for VS 2015 memset implementation
In VS 2015, the memset fill parameter is zero extended from one byte
instead of being copied wholesale.

The issue reproduces with existing tests if you use VS2015.

llvm-svn: 263966
2016-03-21 18:23:07 +00:00
Kuba Brecka 4b3833df7d [tsan] Add some NULL pointer checks into the debugging API
`__tsan_get_report_thread` and others can crash if a stack trace is missing, let's add the missing checks.

Differential Revision: http://reviews.llvm.org/D18306

llvm-svn: 263939
2016-03-21 12:12:44 +00:00
Saleem Abdulrasool 61e970607e builtins: make __clear_cache work on Linux-ARM
__clear_cache on Android is identical to the version on Linux.  Use __linux__
instead of __ANDROID__ as __linux__ is defined for Linux and Android.

llvm-svn: 263833
2016-03-18 21:06:06 +00:00
Saleem Abdulrasool fb21509509 builtins: port __clear_cache to Windows ARM
Support __clear_cache on Windows on ARM using the `FlushInstructionCache`
library call.

llvm-svn: 263832
2016-03-18 21:06:03 +00:00
Mike Aizatsky 6d8a876159 [sancov] common flags initialization.
Summary:
Introducing InitializeCommonFlags accross all sanitizers to simplify
common flags management.

Setting coverage=1 when html_cov_report is requested.

Differential Revision: http://reviews.llvm.org/D18273

llvm-svn: 263820
2016-03-18 19:28:07 +00:00
Kuba Brecka 911e229f30 [tsan] Add interceptor for pthread_cond_timedwait_relative_np
On OS X, we have pthread_cond_timedwait_relative_np. TSan needs to intercept this API to avoid false positives when using condition variables.

Differential Revision: http://reviews.llvm.org/D18184

llvm-svn: 263782
2016-03-18 10:54:11 +00:00
Kuba Brecka 493028e8e2 Removing a non-intentional debug output that got committed in r263695.
llvm-svn: 263698
2016-03-17 09:27:40 +00:00
Kuba Brecka 4c80867ecf [sanitizer] On OS X, verify that interceptors work and abort if not, take 2
On OS X 10.11+, we have "automatic interceptors", so we don't need to use DYLD_INSERT_LIBRARIES when launching instrumented programs. However, non-instrumented programs that load TSan late (e.g. via dlopen) are currently broken, as TSan will still try to initialize, but the program will crash/hang at random places (because the interceptors don't work). This patch adds an explicit check that interceptors are working, and if not, it aborts and prints out an error message suggesting to explicitly use DYLD_INSERT_LIBRARIES.

TSan unit tests run with a statically linked runtime, where interceptors don't work. To avoid aborting the process in this case, the patch replaces `DisableReexec()` with a weak `ReexecDisabled()` function which is defined to return true in unit tests.

Differential Revision: http://reviews.llvm.org/D18212

llvm-svn: 263695
2016-03-17 08:37:25 +00:00
Kuba Brecka 46bf454d18 [tsan] Detect uses of uninitialized, destroyed and invalid mutexes
This patch adds a new TSan report type, ReportTypeMutexInvalidAccess, which is triggered when pthread_mutex_lock or pthread_mutex_unlock returns EINVAL (this means the mutex is invalid, uninitialized or already destroyed).

Differential Revision: http://reviews.llvm.org/D18132

llvm-svn: 263641
2016-03-16 15:39:20 +00:00
Mohit K. Bhakkad beb155b4fa [ASAN] Add support for mips/mips64 android
Patch by Duane Sand

Reviewers: samsonov

Subscribers: duanesand, jaydeep, sagar, llvm-commits, filcab.

Differential Revision: http://reviews.llvm.org/D17883

llvm-svn: 263621
2016-03-16 08:23:10 +00:00
Alexey Samsonov 9b4bbeabcc [ASan] Make sure unittest works if sized-delete is enabled by default.
llvm-svn: 263575
2016-03-15 19:07:56 +00:00
Kuba Brecka 0fb87f77ae Revert r263551 due to a test failure.
llvm-svn: 263553
2016-03-15 15:53:39 +00:00
Kuba Brecka 69b5943a05 [sanitizer] On OS X, verify that interceptors work and abort if not
On OS X 10.11+, we have "automatic interceptors", so we don't need to use DYLD_INSERT_LIBRARIES when launching instrumented programs. However, non-instrumented programs that load TSan late (e.g. via dlopen) are currently broken, as TSan will still try to initialize, but the program will crash/hang at random places (because the interceptors don't work). This patch adds an explicit check that interceptors are working, and if not, it aborts and prints out an error message suggesting to explicitly use DYLD_INSERT_LIBRARIES.

Differential Revision: http://reviews.llvm.org/D18121

llvm-svn: 263551
2016-03-15 14:30:28 +00:00
Nico Weber 39a02a7bed Fix bad regression from r263077 when building with MSVC.
That change did:

  -#if defined(__BIG_ENDIAN__)
  +#if __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__

If __BYTE_ORDER__ and __ORDER_BIG_ENDIAN__ aren't defined, like
they are with MSVC, this condition is true (0 == 0).

Fixes PR26919.

llvm-svn: 263324
2016-03-12 01:57:31 +00:00
Mohit K. Bhakkad f4cc1fc7e9 [Sanitizer][MIPS] internal lstat and fstat for mips64
Reviewers: dvyukov

Subscribers: jaydeep, sagar, llvm-commits

Differential Revision: http://reviews.llvm.org/D18078

llvm-svn: 263218
2016-03-11 10:51:03 +00:00
Mohit K. Bhakkad ac4bd45f02 [Compiler-rt] Fix ioctl codes for MIPS in sanitizer_ioctl_test
Reviewers: samsonov

Subscribers: jaydeep, sagar, llvm-commits

Differential Revision: http://reviews.llvm.org/D17985

llvm-svn: 263199
2016-03-11 05:57:20 +00:00
Alexey Samsonov ed3d347e25 [sanitizer] Add strlen to the common interceptors
Summary:
Adds strlen to the common interceptors, under a new common flag
intercept_strlen.  This provides better sharing of interception code among
sanitizers and cleans up the inconsistent type declarations of the
previously duplicated interceptors.

Removes the now-duplicate strlen interceptor from asan, msan, and tsan.
The entry check semantics are normalized now for msan and asan, whose
private strlen interceptors contained multiple layers of checks that
included impossible-to-reach code.  The new semantics are identical to the
old: bypass interception if in the middle of init or if both on Mac and not
initialized; else, call the init routine and proceed.

Patch by Derek Bruening!

Reviewers: samsonov, vitalybuka

Subscribers: llvm-commits, kcc, zhaoqin

Differential Revision: http://reviews.llvm.org/D18020

llvm-svn: 263177
2016-03-11 00:45:49 +00:00
Alexey Samsonov 92a46991b5 [UBSan] Improve bufferization of UBSan error reports.
Summary:
Use InternalScopedString more extensively. This reduces the number of
write() syscalls, and reduces the chance that UBSan output will be
mixed with program output.

Reviewers: vitalybuka

Subscribers: kcc, llvm-commits

Differential Revision: http://reviews.llvm.org/D18068

llvm-svn: 263176
2016-03-11 00:37:02 +00:00
Reid Kleckner ad04914a53 [Windows] Fix UnmapOrDie and MmapAlignedOrDie
Now ASan can return virtual memory to the underlying OS. Portable
sanitizer runtime code needs to be aware that UnmapOrDie cannot unmap
part of previous mapping.

In particular, this required changing how we implement MmapAlignedOrDie
on Windows, which is what Allocator32 uses.

The new code first attempts to allocate memory of the given size, and if
it is appropriately aligned, returns early. If not, it frees the memory
and attempts to reserve size + alignment bytes. In this region there
must be an aligned address. We then free the oversized mapping and
request a new mapping at the aligned address immediately after. However,
a thread could allocate that virtual address in between our free and
allocation, so we have to retry if that allocation fails. The existing
thread creation stress test managed to trigger this condition, so the
code isn't totally untested.

Reviewers: samsonov

Differential Revision: http://reviews.llvm.org/D17431

llvm-svn: 263160
2016-03-10 20:47:26 +00:00
Dimitry Andric b45bf0af91 Retrieve command line arguments and environment correctly on FreeBSD
Summary:
Recently I saw the test `TestCases/Posix/print_cmdline.cc` failing on
FreeBSD, with "expected string not found in input".  This is because
asan could not retrieve the command line arguments properly.

In `lib/sanitizer_common/sanitizer_linux.cc`, this is taken care of by
the `GetArgsAndEnv()` function, but it uses `__libc_stack_end` to get at
the required data.  This variable does not exist on BSDs; the regular
way to retrieve the arguments and environment information is via the
`kern.ps_strings` sysctl.

I added this functionality in sanitizer_linux.cc, as a separate #ifdef
block in `GetArgsAndEnv()`.  Also, `ReadNullSepFileToArray()` becomes
unused due to this change.  (It won't work on FreeBSD anyway, since
`/proc` is not mounted by default.)

Reviewers: kcc, emaste, joerg, davide

Subscribers: llvm-commits, emaste

Differential Revision: http://reviews.llvm.org/D17832

llvm-svn: 263157
2016-03-10 20:22:02 +00:00
Kuba Brecka a1496f7c37 [tsan] Add TSan debugger APIs
Currently, TSan only reports everything in a formatted textual form. The idea behind this patch is to provide a consistent API that can be used to query information contained in a TSan-produced report. User can use these APIs either in a debugger (via a script or directly), or they can use it directly from the process (e.g. in the __tsan_on_report callback). ASan already has a similar API, see http://reviews.llvm.org/D4466.

Differential Revision: http://reviews.llvm.org/D16191

llvm-svn: 263126
2016-03-10 17:00:29 +00:00
Alexey Samsonov c1424fc7c8 sanitizer: Fix endianness checks for gcc
Summary:
__BIG_ENDIAN__ and __LITTLE_ENDIAN__ are not supported by gcc, which
eg. for ubsan Value::getFloatValue will silently fall through to
the little endian branch, breaking display of float values by ubsan.
Use __BYTE_ORDER__ == __ORDER_BIG/LITTLE_ENDIAN__ as the condition
instead, which is supported by both clang and gcc.

Noticed while porting ubsan to s390x.

Patch by Marcin Kościelnicki!

Differential Revision: http://reviews.llvm.org/D17660

llvm-svn: 263077
2016-03-09 23:39:40 +00:00
Xinliang David Li 5f58c27d79 Fix test failure on darwin-i386
llvm-svn: 263065
2016-03-09 21:45:37 +00:00
Logan Chien 5b7b9e946c Add ARM EHABI support to gcc_personality_v0.
Until now the only exception APIs supported by gcc_personality_v0
are DWARF EH and SJLJ. This adds support for ARM EHABI as well.

This is achieved by
a) changing the function signature on ARM EHABI,
b) unwinding the stack before returning _URC_CONTINUE_UNWIND.

See "Exception Handling ABI for the ARM Architecture" for details
(http://infocenter.arm.com/help/topic/com.arm.doc.ihi0038b/IHI0038B_ehabi.pdf).

Patch by Timon Van Overveldt.

llvm-svn: 263010
2016-03-09 13:20:17 +00:00
Mohit K. Bhakkad 9e168987b9 [Compiler-rt][MIPS] Defining macros for MIPS archs
Reviewers: samsonov

Subscribers: filcab, jaydeep, sagar, llvm-commits

Differential Revision: http://reviews.llvm.org/D17881

llvm-svn: 263000
2016-03-09 07:57:20 +00:00
Filipe Cabecinhas 19aeaf76df [Fix r262785] Fix missing declaration when COMPILER_RT_BOOL_CMPXCHG was used but InstrProfilingUtil.h wasn't included.
Also fixed declaration which still had the pre-r262788 name.

llvm-svn: 262827
2016-03-07 13:42:17 +00:00
Xinliang David Li 09fecac354 cleanup: remove dead preprocessing branch /nfc
llvm-svn: 262792
2016-03-06 04:56:55 +00:00
Xinliang David Li 44fd38856c [PGO] cleanup: move one support method into InstrProfilingUtil.h /NFC
llvm-svn: 262791
2016-03-06 04:52:45 +00:00
Xinliang David Li cf1a8d6912 [PGO] internal API name cleanups (for better consistency)
llvm-svn: 262788
2016-03-06 04:18:13 +00:00
Xinliang David Li cf2f829be6 Minor cleanup
move a function def to InstrProfilingUtil.c

llvm-svn: 262785
2016-03-06 00:55:20 +00:00
Vedant Kumar e55855f362 [GCDA] Unlock arc file before closing it
The GCDA writer closed the arc file before unlocking it. This causes an
EBADF while unlocking the file, and opens us up to racy behavior.

Fixes PR26847.

llvm-svn: 262779
2016-03-05 20:10:25 +00:00
Filipe Cabecinhas b08c76f5f8 [cmake] Address Samsonov's post-commit review of r262723
Reviewers: samsonov

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D17896

llvm-svn: 262770
2016-03-05 10:01:04 +00:00
Xinliang David Li 31f251f1f0 [PGO] Add API to check compatibility of profile data in buffer
This is needed by client which uses in-process merge API.

llvm-svn: 262736
2016-03-04 18:58:30 +00:00
Xinliang David Li 6534a8a837 Fix api visibility
llvm-svn: 262734
2016-03-04 18:33:49 +00:00
Filipe Cabecinhas 8144d3b091 [ubsan/cmake] Make sure we end up adding -frtti to ubsan_type_hash_itanium.cc, even if -fno-rtti is the default for the platform
llvm-svn: 262723
2016-03-04 17:02:06 +00:00
Alexey Samsonov 4362a994d2 [TSan] Fix compiler warning in Go sanity test.
llvm-svn: 262680
2016-03-04 00:56:15 +00:00
Xinliang David Li dd12e9a8c0 [PGO] Add API for profile merge from buffer
Differential Revision: http://reviews.llvm.org/D17831

llvm-svn: 262644
2016-03-03 18:54:46 +00:00
Sean Silva 6470f7615c Add some minimal portability code paths for PS4.
Summary:
Hi David, SCE folks,

What is implemented in this patch is enough for the upstream libprofile to
work for PGO with the PS4 game codebase I tested ("game7" for you SCE
folks; this is with a standalone build of compiler-rt).

The first change, which is simple, is to stub out gethostname. PS4
doesn't have a simple analog for this that doesn't bring in extra
OS libraries, so for now we do not support `%h` expansion.
This is consistent with internal B#136272.

The second change implies future work, but is a simple change at present.
PS4 does not have `getenv`, so for now we will introduce a shim.
This obviously makes it impossible for many of the tests to be run since
they require setting `LLVM_PROFILE_FILE=`.

I see two paths forward:

1. In the tests we are already wrapping execution with `%run` and so by
   setting a PS4-specific expansion for `%run` we can pass the information
   in another way We can adapt the getenv shim as appropriate.
   We will need to experiment with this internally.
   Maggie, Phillip, Filipe? Any ideas? Maybe ping me internally since we
   may need to get into some PS4 vagaries. I'm thinking a fake getenv
   library that uses some side channel for communication.

2. Another possibility which is more verbose is to use a separate clang
   invocation with `-profile-generate=<filename>` to set the filename in
   each test.
   This might require redundant clang invocations though which may be
   undesirable for upstream. David, thoughts?
   Also, this is a fairly libprofile-specific workaround, so it e.g.
   doesn't help Filipe's ASan work.
   Overall, this approach sounds like a bit of a hack to me.

Small detail:
InstrProfilingPort.h seems like the natural place for the getenv shim,
but GCDAProfiling.c needs it as well. InstrProfilingUtil.h is currently
the only header common between InstrProfilingFile.c and GCDAProfiling.c.
I can move the shim to InstrProfilingPort.h and add an include to
GCDAProfiling.c as per your preference David.

Reviewers: davidxl, MaggieYi, phillip.power, filcab

Subscribers: simon.f.whittaker, slingn, probinson, llvm-commits

Differential Revision: http://reviews.llvm.org/D17676

llvm-svn: 262527
2016-03-02 22:05:46 +00:00
Dmitry Vyukov a029b79e1a tsan: describe heap/data locations in Go
llvm-svn: 262343
2016-03-01 15:38:12 +00:00
Dmitry Vyukov f5fcdd268f sanitizer_common: silence compiler warning
llvm-svn: 262342
2016-03-01 15:36:42 +00:00
Mohit K. Bhakkad 1d1493219c [Compiler-rt][MIPS] Correct Dynamic Thread Vector offset for MIPS
Reviewers: samsonov

Subscribers: dsanders, jaydeep, sagar, llvm-commits

Differential Revision: http://reviews.llvm.org/D17703

llvm-svn: 262303
2016-03-01 05:56:33 +00:00
Filipe Cabecinhas b7692bc3e9 [UBSan] Fix isDerivedFromAtOffset on iOS ARM64
Summary:
iOS on ARM64 doesn't unique RTTI.
Ref: clang's iOS64CXXABI::shouldRTTIBeUnique()

Due to this, pointer-equality will not necessarily work in this
architecture, across dylib boundaries.

dynamic_cast<>() will (as expected) still work, since Apple ships with
one prepared for this, but we can't rely on the type names being
pointer-equal.

I've limited the expensive strcmp check to the specific architecture
which needs it.

Example which triggers this bug:

lib.h:
  struct X {
    virtual ~X() {}
  };
  X *libCall();

lib.mm:
  X *libCall() {
    return new X;
  }

prog.mm:
  int main() {
    X *px = libCall();
    delete px;
  }

Expected output: Nothing
Actual output:
<unknown>: runtime error: member call on address 0x00017001ef50 which does not point to an object of type 'X'
0x00017001ef50: note: object is of type 'X'
 00 00 00 00  60 00 0f 00 01 00 00 00  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  00 00 00 00
              ^~~~~~~~~~~~~~~~~~~~~~~
              vptr for ‘X’

Reviewers: kubabrecka, samsonov, eugenis, rsmith

Subscribers: aemerson, llvm-commits, rengolin

Differential Revision: http://reviews.llvm.org/D11502

llvm-svn: 262147
2016-02-27 19:57:44 +00:00
Dmitry Vyukov 7f022ae4c2 tsan: revert r262037
Broke aarch64 and darwin bots.

llvm-svn: 262046
2016-02-26 18:26:48 +00:00
Dmitry Vyukov b8868b9bea tsan: split thread into logical and physical state
Currently ThreadState holds both logical state (required for race-detection algorithm, user-visible)
and physical state (various caches, most notably malloc cache). Move physical state in a new
Process entity. Besides just being the right thing from abstraction point of view, this solves several
problems:
1. Cache everything on P level in Go. Currently we cache on a mix of goroutine and OS thread levels.
This unnecessary increases memory consumption.
2. Properly handle free operations in Go. Frees are issue by GC which don't have goroutine context.
As the result we could not do anything more than just clearing shadow. For example, we leaked
sync objects and heap block descriptors.
3. This will allow to get rid of libc malloc in Go (now we have Processor context for internal allocator cache).
This in turn will allow to get rid of dependency on libc entirely.
4. Potentially we can make Processor per-CPU in C++ mode instead of per-thread, which will
reduce resource consumption.
The distinction between Thread and Processor is currently used only by Go, C++ creates Processor per OS thread,
which is equivalent to the current scheme.

llvm-svn: 262037
2016-02-26 16:57:14 +00:00
Mohit K. Bhakkad fac2e248c4 [MSan] Endianness should not matter while printing a byte
Reviewers: eugenis

Subscribers: jaydeep, sagar, llvm-commits

Differential Revision: http://reviews.llvm.org/D17264
Differential Revision: http://reviews.llvm.org/D17563

llvm-svn: 261982
2016-02-26 06:44:10 +00:00
Sagar Thakur 63cecb3653 [MSAN] Fix test SmallPreAllocatedStackThread for MIPS
Summary: Msan was intercepting version 2.1 of the pthread_create function which was making it to crash in libc because __pthread_create_2_1 modifies the stack attributes of the thread. Intercepting the correct version fixes the test SmallPreAllocatedStackThread.

Reviewers: eugenis, samsonov
Subscribers: llvm-commits, mohit.bhakkad, jaydeep
Differential: http://reviews.llvm.org/D17603
llvm-svn: 261980
2016-02-26 05:56:54 +00:00
Vedant Kumar b850251d3e [profile] Compute number of data entries correctly
Compiler-rt miscalculates the number of entries in the __llvm_prf_data section
on i386 Darwin. This results in a number of test failures (which we started
catching after r261344).

The fix we attempted earlier is insufficient (r261683). It caused some tests to
start passing again, but that hid the fact that we drop some data entries.

This patch should fix the real problem. It fixes the way we compute DataSize by
taking into account the way the Darwin linker lays out __llvm_prf_data.

Differential Revision: http://reviews.llvm.org/D17623

llvm-svn: 261957
2016-02-26 02:49:41 +00:00
Xinliang David Li 1a436dd704 Sync up with master
llvm-svn: 261947
2016-02-26 00:56:31 +00:00
Alexey Samsonov 07f9c0f558 [Sanitizer] Protect against compiler-inserted memcpy() in InternalMmapVector::push_back().
llvm-svn: 261909
2016-02-25 18:12:30 +00:00
Maxim Ostapenko 50ca4464c4 [sanitizer] Fix third parameter in COMMON_INTERCEPTOR_WRITE_RANGE in recv and recvfrom interceptors.
Pass res instead of len as third parameter to COMMON_INTERCEPTOR_WRITE_RANGE,
because otherwise we can write to unrelated memory (in MSan) or get wrong report (in ASan).

Differential Revision: http://reviews.llvm.org/D17608

llvm-svn: 261898
2016-02-25 17:07:38 +00:00
Maxim Ostapenko b9fd2e0015 [sanitizer] Disable recv and recvfrom interceptors on Windows.
Trying to fix sanitizer-windows buildbot by disabling new interceptors on Windows.

llvm-svn: 261844
2016-02-25 09:36:20 +00:00
Maxim Ostapenko 7389936f57 [sanitizer] Move recvmsg and recv interceptors to sanitizer_common.
This patch moves recv and recvfrom interceptors from MSan and TSan to
sanitizer_common to enable them in ASan.

Differential Revision: http://reviews.llvm.org/D17479

llvm-svn: 261841
2016-02-25 08:44:25 +00:00
Mohit K. Bhakkad 72c3cce484 [Compiler-rt][MSan]Fix shmat testcase: Pass SHMLBA-alligned address to shmat
Reviewers: samsonov

Subscribers: jaydeep, sagar, llvm-commits

Differential Revision: http://reviews.llvm.org/D17262

llvm-svn: 261837
2016-02-25 08:07:44 +00:00
Saleem Abdulrasool 2b972dfebb builtins: adjust the clobbers list for android __clear_cache
Adjust the clobbers list.  This use to work with older versions of gcc, but now
will error on newer versions (tested against 5.3) (as well as clang).

Patch by Tee Hao Wei!

llvm-svn: 261821
2016-02-25 04:04:37 +00:00
Alexey Samsonov ee7023f79a [tests] Portability fixup for r261713.
llvm-svn: 261715
2016-02-24 01:28:51 +00:00
Alexey Samsonov 0dc3d24d5d [tests] Resolve old TODOs in ASan unit tests.
llvm-svn: 261713
2016-02-24 00:41:29 +00:00
Chris Bieneman 86792ea718 [CMake] Assign components and dependencies during add_compiler_rt_resource_file
This makes it so that component-based installations will include resource files (i.e. blacklists). My next patch will add support for component-based installations.

llvm-svn: 261699
2016-02-23 21:50:39 +00:00
Vedant Kumar 120f6301ed [profile] Fix iteration over profile data entries
Fix a crash when gathering value profile data on i386 Darwin.

The Darwin linker shrinks sections containing aligned structures when
padding is not explicitly added to the end of the structure. When
iterating over these structures, be sure to not walk past the end of the
section.

No tests added, since running `ninja check-profile` on i386 Darwin is
enough to reproduce the original crash.

llvm-svn: 261683
2016-02-23 20:46:14 +00:00