Commit Graph

400 Commits

Author SHA1 Message Date
Maxim Ostapenko f29aec76dd [sanitizer] Move fread and fwrite interceptors to sanitizer_common
{M, T, E}San have fread and fwrite interceptors, let's move them to sanitizer_common to enable ASan checks as well.

Differential Revision: https://reviews.llvm.org/D31456

llvm-svn: 299061
2017-03-30 07:25:33 +00:00
Dmitry Vyukov 8096a8c86f tsan: add new mutex annotations
There are several problems with the current annotations (AnnotateRWLockCreate and friends):
- they don't fully support deadlock detection (we need a hook _before_ mutex lock)
- they don't support insertion of random artificial delays to perturb execution (again we need a hook _before_ mutex lock)
- they don't support setting extended mutex attributes like read/write reentrancy (only "linker init" was bolted on)
- they don't support setting mutex attributes if a mutex don't have a "constructor" (e.g. static, Java, Go mutexes)
- they don't ignore synchronization inside of lock/unlock operations which leads to slowdown and false negatives
The new annotations solve of the above problems. See tsan_interface.h for the interface specification and comments.

Reviewed in https://reviews.llvm.org/D31093

llvm-svn: 298809
2017-03-26 15:27:04 +00:00
Kuba Mracek bba1d40589 [tsan] Properly describe GCD worker threads in reports
When dealing with GCD worker threads, TSan currently prints weird things like "created by thread T-1" and "[failed to restore the stack]" in reports. This patch avoids that and instead prints "Thread T3 (...) is a GCD worker thread".

Differential Revision: https://reviews.llvm.org/D29103

llvm-svn: 293882
2017-02-02 12:54:21 +00:00
Kuba Mracek e7709560ea [tsan] Implement a 'ignore_noninstrumented_modules' flag to better suppress false positive races
On Darwin, we currently use 'ignore_interceptors_accesses', which is a heavy-weight solution that simply turns of race detection in all interceptors. This was done to suppress false positives coming from system libraries (non-instrumented code), but it also silences a lot of real races. This patch implements an alternative approach that should allow us to enable interceptors and report races coming from them, but only if they are called directly from instrumented code.

The patch matches the caller PC in each interceptors. For non-instrumented code, we call ThreadIgnoreBegin.

The assumption here is that the number of instrumented modules is low. Most likely there's only one (the instrumented main executable) and all the other modules are system libraries (non-instrumented).

Differential Revision: https://reviews.llvm.org/D28264

llvm-svn: 291631
2017-01-11 00:54:26 +00:00
Sagar Thakur 19b84a0224 [TSan][MIPS] Implements setjmp assembly for MIPS64
Reviewed by dvyukov
Differential: https://reviews.llvm.org/D23494

llvm-svn: 278775
2016-08-16 05:06:56 +00:00
Dmitry Vyukov 246e0283d4 tsan: don't deliver signals when they are blocked
When we delay signals we can deliver them when the signal
is blocked. This can be surprising to the program.
Intercept signal blocking functions merely to process
pending signals. As the result, at worst we will delay
a signal till return from the signal blocking function.

llvm-svn: 276876
2016-07-27 14:34:21 +00:00
Evgeniy Stepanov 3c17c73924 [msan] Intercept send/sendto/sendmsg.
send/sendmsg moved from tsan to sanitizer_common; sendto is new.

llvm-svn: 272980
2016-06-17 00:43:11 +00:00
Dmitry Vyukov 88f47d3521 tsan: intercept epoll_pwait
llvm-svn: 272779
2016-06-15 12:22:05 +00:00
Kuba Brecka 304b2c2a45 [tsan] Switch to InternalAlloc everywhere __libc_malloc is currently used
This patch replaces all uses of __libc_malloc and friends with the internal allocator.

It seems that the only reason why we have calls to __libc_malloc in the first place was the lack of the internal allocator at the time. Using the internal allocator will also make sure that the system allocator is never used (this is the same behavior as ASan), and we don’t have to worry about working with unknown pointers coming from the system allocator.

Differential Revision: http://reviews.llvm.org/D21025

llvm-svn: 271916
2016-06-06 18:18:47 +00:00
Dmitry Vyukov 1ae44e6a2b tsan: clean up dynamic TLS memory between reuse
Currently the added test produces false race reports with glibc 2.19,
because DLTS memory is reused by pthread under the hood.

Use the DTLS machinery to intercept new DTLS ranges.
__tls_get_addr known to cause issues for tsan in the past,
so write the interceptor more carefully.

Reviewed in http://reviews.llvm.org/D20927

llvm-svn: 271568
2016-06-02 19:18:22 +00:00
Benjamin Kramer f9679e89a1 Revert "[sanitizer] Move *fstat to the common interceptors"
This reverts commit r269981. Breaks msan tests on linux
http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux/builds/24019/steps/test%20standalone%20compiler-rt/logs/stdio

llvm-svn: 270076
2016-05-19 16:03:10 +00:00
Mike Aizatsky 522afdd77e [sanitizer] Move *fstat to the common interceptors
Summary:
Adds *fstat to the common interceptors.

Removes the now-duplicate fstat interceptor from msan/tsan
This adds fstat to asan/esan, which previously did not intercept it.

Resubmit of http://reviews.llvm.org/D20318 with ios build fixes.

Reviewers: eugenis, vitalybuka, aizatsky

Subscribers: zaks.anna, kcc, bruening, kubabrecka, srhines, danalbert, tberghammer

Differential Revision: http://reviews.llvm.org/D20350

llvm-svn: 269981
2016-05-18 20:49:49 +00:00
Mike Aizatsky 41c2afe5d9 Revert "[sanitizer] Move *fstat to the common interceptors"
This reverts commit http://reviews.llvm.org/rL269856

llvm-svn: 269863
2016-05-17 23:28:56 +00:00
Mike Aizatsky 924acb50c9 [sanitizer] Move *fstat to the common interceptors
Summary:
Adds *fstat to the common interceptors.

Removes the now-duplicate fstat interceptor from msan/tsan
This adds fstat to asan/esan, which previously did not intercept it.

Reviewers: eugenis, vitalybuka, aizatsky

Subscribers: tberghammer, danalbert, srhines, kubabrecka, bruening, kcc

Differential Revision: http://reviews.llvm.org/D20318

llvm-svn: 269856
2016-05-17 22:26:50 +00:00
Anna Zaks 7d47c990e6 [tsan] Ensure mmap respects ignore_interceptors_accesses
The ignore_interceptors_accesses setting did not have an effect on mmap, so
let's change that. It helps in cases user code is accessing the memory
written to by mmap when the synchronization is ensured by the code that
does not get rebuilt.

(This effects Swift interoperability since it's runtime is mapping memory
which gets accessed by the code emitted into the Swift application by the
compiler.)

Differential Revision: http://reviews.llvm.org/D20294

llvm-svn: 269855
2016-05-17 22:24:55 +00:00
Evgeniy Stepanov feb73c8566 [sanitizer] Move *stat to the common interceptors
Adds *stat to the common interceptors.

Removes the now-duplicate *stat interceptor from msan/tsan/esan.
This adds *stat to asan, which previously did not intercept it.

Patch by Qin Zhao.

llvm-svn: 269223
2016-05-11 20:02:15 +00:00
Dmitry Vyukov 2498629e34 tsan: fix another crash due to processors
Another stack where we try to free sync objects,
but don't have a processors is:

  //   ResetRange
  //   __interceptor_munmap
  //   __deallocate_stack
  //   start_thread
  //   clone

Again, it is a latent bug that lead to memory leaks.
Also, increase amount of memory we scan in MetaMap::ResetRange.
Without that the test does not fail, as we fail to free
the sync objects on stack.

llvm-svn: 269041
2016-05-10 11:19:50 +00:00
Mike Aizatsky c826e634cc [sanitizer] Move stat/__xstat to the common interceptors
Summary:
Adds stat/__xstat to the common interceptors.

Removes the now-duplicate stat/__xstat interceptor from msan/tsan/esan.
This adds stat/__xstat to asan, which previously did not intercept it.

Resubmit of http://reviews.llvm.org/D19875 with win build fixes.

Reviewers: aizatsky, eugenis

Subscribers: tberghammer, llvm-commits, danalbert, vitalybuka, bruening, srhines, kubabrecka, kcc

Differential Revision: http://reviews.llvm.org/D19890

llvm-svn: 268466
2016-05-03 23:43:45 +00:00
Mike Aizatsky 7e72f66bf2 Revert "[sanitizer] Move stat/__xstat to the common interceptors"
This reverts commit 268440 because it breaks the windows bot.

http://lab.llvm.org:8011/builders/sanitizer-windows/builds/21425/steps/build%20compiler-rt/logs/stdio

llvm-svn: 268448
2016-05-03 21:49:56 +00:00
Mike Aizatsky 3eb521d417 [sanitizer] Move stat/__xstat to the common interceptors
Summary:
Adds stat/__xstat to the common interceptors.

Removes the now-duplicate stat/__xstat interceptor from msan/tsan/esan.
This adds stat/__xstat to asan, which previously did not intercept it.

Reviewers: aizatsky, eugenis

Subscribers: tberghammer, danalbert, srhines, kubabrecka, llvm-commits, vitalybuka, eugenis, kcc, bruening

Differential Revision: http://reviews.llvm.org/D19875

llvm-svn: 268440
2016-05-03 21:22:06 +00:00
Marcin Koscielnicki 66f0deacb5 [sanitizers] Get the proper symbol version when long double transition is involved.
On linux, some architectures had an ABI transition from 64-bit long double
(ie. same as double) to 128-bit long double.  On those, glibc symbols
involving long doubles come in two versions, and we need to pass the
correct one to dlvsym when intercepting them.

A few more functions we intercept are also versioned (all printf, scanf,
strtold variants), but there's no need to fix these, as the REAL() versions
are never called.

Differential Revision: http://reviews.llvm.org/D19555

llvm-svn: 267794
2016-04-27 21:24:21 +00:00
Dmitry Vyukov 3efe395788 tsan: change tsan/Go interface for obtaining the current Processor
Current interface assumes that Go calls ProcWire/ProcUnwire
to establish the association between thread and proc.
With the wisdom of hindsight, this interface does not work
very well. I had to sprinkle Go scheduler with wire/unwire
calls, and any mistake leads to hard to debug crashes.
This is not something one wants to maintian.
Fortunately, there is a simpler solution. We can ask Go
runtime as to what is the current Processor, and that
question is very easy to answer on Go side.
Switch to such interface.

llvm-svn: 267703
2016-04-27 12:30:48 +00:00
Dmitry Vyukov d87c7b321a tsan: split thread into logical and physical state
This is reincarnation of http://reviews.llvm.org/D17648 with the bug fix pointed out by Adhemerval (zatrazz).

Currently ThreadState holds both logical state (required for race-detection algorithm, user-visible)
and physical state (various caches, most notably malloc cache). Move physical state in a new
Process entity. Besides just being the right thing from abstraction point of view, this solves several
problems:

Cache everything on P level in Go. Currently we cache on a mix of goroutine and OS thread levels.
This unnecessary increases memory consumption.

Properly handle free operations in Go. Frees are issue by GC which don't have goroutine context.
As the result we could not do anything more than just clearing shadow. For example, we leaked
sync objects and heap block descriptors.

This will allow to get rid of libc malloc in Go (now we have Processor context for internal allocator cache).
This in turn will allow to get rid of dependency on libc entirely.

Potentially we can make Processor per-CPU in C++ mode instead of per-thread, which will
reduce resource consumption.
The distinction between Thread and Processor is currently used only by Go, C++ creates Processor per OS thread,
which is equivalent to the current scheme.

llvm-svn: 267678
2016-04-27 08:23:02 +00:00
Marcin Koscielnicki 3f9d7a217d [sanitizers] [NFC] Add defines for the various PowerPC ABIs.
Differential Revision: http://reviews.llvm.org/D19542

llvm-svn: 267586
2016-04-26 18:44:13 +00:00
Dmitry Vyukov 0d9b7bc9ad tsan: fix ignore handling in signal handlers
We've reset thr->ignore_reads_and_writes, but forget to do
thr->fast_state.ClearIgnoreBit(). So ignores were not effective
reset and fast_state.ignore_bit was corrupted if signal handler
itself uses ignores.

Properly reset/restore fast_state.ignore_bit around signal handlers.

llvm-svn: 265288
2016-04-04 10:52:59 +00:00
Derek Bruening e988af9073 [sanitizer] Add memset, memmove, and memcpy to the common interceptors
Summary:
Currently, sanitizer_common_interceptors.inc has an implicit, undocumented
assumption that the sanitizer including it has previously declared
interceptors for memset and memmove.  Since the memset, memmove, and memcpy
routines require interception by many sanitizers, we add them to the
set of common interceptions, both to address the undocumented assumption
and to speed future tool development.  They are intercepted under a new
flag intercept_intrin.

The tsan interceptors are removed in favor of the new common versions.  The
asan and msan interceptors for these are more complex (they incur extra
interception steps and their function bodies are exposed to the compiler)
so they opt out of the common versions and keep their own.

Reviewers: vitalybuka

Subscribers: zhaoqin, llvm-commits, kcc

Differential Revision: http://reviews.llvm.org/D18465

llvm-svn: 264451
2016-03-25 19:33:45 +00:00
Kuba Brecka 3bab18d4af [tsan] Fix fork() and fork-based tests for OS X
On OS X, fork() under TSan asserts (in debug builds only) because REAL(fork) calls some intercepted functions, which check that no internal locks are held via CheckNoLocks(). But the wrapper of fork intentionally holds some locks. This patch fixes that by using ScopedIgnoreInterceptors during the call to REAL(fork). After that, all the fork-based tests seem to pass on OS X, so let's just remove all the UNSUPPORTED: darwin annotations we have.

Differential Revision: http://reviews.llvm.org/D18409

llvm-svn: 264261
2016-03-24 11:54:33 +00:00
Evgeniy Stepanov accf390a75 [sanitizer] Add strchr* to the common interceptors
Adds strchr, strchrnul, and strrchr to the common interceptors, under a new
common flag intercept_strchr.

Removes the now-duplicate strchr interceptor from asan and all 3
interceptors from tsan. Previously, asan did not intercept strchrnul, but
does now; previously, msan did not intercept strchr, strchrnul, or strrchr,
but does now.

http://reviews.llvm.org/D18329

Patch by Derek Bruening!

llvm-svn: 263992
2016-03-21 21:36:17 +00:00
Kuba Brecka 911e229f30 [tsan] Add interceptor for pthread_cond_timedwait_relative_np
On OS X, we have pthread_cond_timedwait_relative_np. TSan needs to intercept this API to avoid false positives when using condition variables.

Differential Revision: http://reviews.llvm.org/D18184

llvm-svn: 263782
2016-03-18 10:54:11 +00:00
Kuba Brecka 46bf454d18 [tsan] Detect uses of uninitialized, destroyed and invalid mutexes
This patch adds a new TSan report type, ReportTypeMutexInvalidAccess, which is triggered when pthread_mutex_lock or pthread_mutex_unlock returns EINVAL (this means the mutex is invalid, uninitialized or already destroyed).

Differential Revision: http://reviews.llvm.org/D18132

llvm-svn: 263641
2016-03-16 15:39:20 +00:00
Alexey Samsonov ed3d347e25 [sanitizer] Add strlen to the common interceptors
Summary:
Adds strlen to the common interceptors, under a new common flag
intercept_strlen.  This provides better sharing of interception code among
sanitizers and cleans up the inconsistent type declarations of the
previously duplicated interceptors.

Removes the now-duplicate strlen interceptor from asan, msan, and tsan.
The entry check semantics are normalized now for msan and asan, whose
private strlen interceptors contained multiple layers of checks that
included impossible-to-reach code.  The new semantics are identical to the
old: bypass interception if in the middle of init or if both on Mac and not
initialized; else, call the init routine and proceed.

Patch by Derek Bruening!

Reviewers: samsonov, vitalybuka

Subscribers: llvm-commits, kcc, zhaoqin

Differential Revision: http://reviews.llvm.org/D18020

llvm-svn: 263177
2016-03-11 00:45:49 +00:00
Alexey Samsonov c1424fc7c8 sanitizer: Fix endianness checks for gcc
Summary:
__BIG_ENDIAN__ and __LITTLE_ENDIAN__ are not supported by gcc, which
eg. for ubsan Value::getFloatValue will silently fall through to
the little endian branch, breaking display of float values by ubsan.
Use __BYTE_ORDER__ == __ORDER_BIG/LITTLE_ENDIAN__ as the condition
instead, which is supported by both clang and gcc.

Noticed while porting ubsan to s390x.

Patch by Marcin Kościelnicki!

Differential Revision: http://reviews.llvm.org/D17660

llvm-svn: 263077
2016-03-09 23:39:40 +00:00
Dmitry Vyukov 7f022ae4c2 tsan: revert r262037
Broke aarch64 and darwin bots.

llvm-svn: 262046
2016-02-26 18:26:48 +00:00
Dmitry Vyukov b8868b9bea tsan: split thread into logical and physical state
Currently ThreadState holds both logical state (required for race-detection algorithm, user-visible)
and physical state (various caches, most notably malloc cache). Move physical state in a new
Process entity. Besides just being the right thing from abstraction point of view, this solves several
problems:
1. Cache everything on P level in Go. Currently we cache on a mix of goroutine and OS thread levels.
This unnecessary increases memory consumption.
2. Properly handle free operations in Go. Frees are issue by GC which don't have goroutine context.
As the result we could not do anything more than just clearing shadow. For example, we leaked
sync objects and heap block descriptors.
3. This will allow to get rid of libc malloc in Go (now we have Processor context for internal allocator cache).
This in turn will allow to get rid of dependency on libc entirely.
4. Potentially we can make Processor per-CPU in C++ mode instead of per-thread, which will
reduce resource consumption.
The distinction between Thread and Processor is currently used only by Go, C++ creates Processor per OS thread,
which is equivalent to the current scheme.

llvm-svn: 262037
2016-02-26 16:57:14 +00:00
Maxim Ostapenko 7389936f57 [sanitizer] Move recvmsg and recv interceptors to sanitizer_common.
This patch moves recv and recvfrom interceptors from MSan and TSan to
sanitizer_common to enable them in ASan.

Differential Revision: http://reviews.llvm.org/D17479

llvm-svn: 261841
2016-02-25 08:44:25 +00:00
Dmitry Vyukov 0c202893ae tsan: clean up code after r261658
llvm-svn: 261660
2016-02-23 17:58:23 +00:00
Dmitry Vyukov b2be098026 tsan: fix signal handling in ignored libraries
The first issue is that we longjmp from ScopedInterceptor scope
when called from an ignored lib. This leaves thr->in_ignored_lib set.
This, in turn, disables handling of sigaction. This, in turn,
corrupts tsan state since signals delivered asynchronously.
Another issue is that we can ignore synchronization in asignal
handler, if the signal is delivered into an IgnoreSync region.
Since signals are generally asynchronous, they should ignore
memory access/synchronization/interceptor ignores.
This could lead to false positives in signal handlers.

llvm-svn: 261658
2016-02-23 17:16:26 +00:00
Ed Maste cef252ea4c [tsan] Fix signal number definitions for FreeBSD
The change in r253983 for OS X also applies to FreeBSD.

llvm-svn: 261121
2016-02-17 18:25:27 +00:00
Ed Maste a36d555c33 [tsan] Fix build warnings on FreeBSD
The change in r252165 for OS X applies to FreeBSD as well.

llvm-svn: 261120
2016-02-17 18:22:50 +00:00
Dmitry Vyukov f8f2d46002 tsan: add back __tls_get_addr interceptor
Removal of the interceptor broke libtsan interface in gcc:
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=68824
Add back a simple interceptor.

llvm-svn: 258119
2016-01-19 12:28:00 +00:00
Yabin Cui f7ebaf889a [tsan] Do nothing in ScopedInterceptor's destructor if thr is not inited.
Summary:
It is part of http://reviews.llvm.org/D15301, but missed when I committed
that patch.

Reviewers: kubabrecka, kcc, eugenis, llvm-commits, dvyukov

Differential Revision: http://reviews.llvm.org/D16235

llvm-svn: 258021
2016-01-17 21:33:42 +00:00
Yabin Cui 19e8c0e2f7 [tsan] Fix some tiny errors.
Summary:
1. Fix spell error for sigaction.
2. Make line_length <= 80.

Reviewers: llvm-commits, eugenis, kcc, dvyukov

Subscribers: tberghammer, danalbert, srhines

Differential Revision: http://reviews.llvm.org/D16210

llvm-svn: 257872
2016-01-15 06:21:46 +00:00
Yabin Cui bd3a772e9f [tsan] Store the pointer to ThreadState in TLS slot on Android.
Summary:
1. Android doesn't support __thread keyword. So allocate ThreadState
dynamically and store its pointer in one TLS slot provided by Android.
2. On Android, intercepted functions can be called before ThreadState
is initialized. So add test of thr_->is_inited in some places.
3. On Android, intercepted functions can be called after ThreadState
is destroyed. So add a fake dead_thread_state to represent all
destroyed ThreadStates. And that is also why we don't store the pointer
to ThreadState in shadow memory of pthread_self().

Reviewers: kcc, eugenis, dvyukov

Subscribers: kubabrecka, llvm-commits, tberghammer, danalbert, srhines

Differential Revision: http://reviews.llvm.org/D15301

llvm-svn: 257866
2016-01-15 03:39:04 +00:00
Yabin Cui 66e5db99b3 [tsan] Use internal_sigfillset to replace REAL(sigfillset).
Summary:
Android doesn't intercept sigfillset, so REAL(sigfillset) is null.
And we can use internal_sigfillset() for all cases.

Reviewers: kcc, eugenis, kubabrecka, dvyukov

Subscribers: llvm-commits, tberghammer, danalbert

Differential Revision: http://reviews.llvm.org/D15296

llvm-svn: 257862
2016-01-15 02:59:23 +00:00
Kuba Brecka 0626dd0d3b [tsan] Introduce a "ignore_interceptors_accesses" option
On OS X, TSan already passes all unit and lit tests, but for real-world applications (even very simple ones), we currently produce a lot of false positive reports about data races. This makes TSan useless at this point, because the noise dominates real bugs. This introduces a runtime flag, "ignore_interceptors_accesses", off by default, which makes TSan ignore all memory accesses that happen from interceptors. This will significantly lower the coverage and miss a lot of bugs, but it eliminates most of the current false positives on OS X.

Differential Revision: http://reviews.llvm.org/D15189

llvm-svn: 257760
2016-01-14 12:24:37 +00:00
Kuba Brecka 9f5fde7081 [tsan] Fix the value of PTHREAD_MUTEX_RECURSIVE for OS X and FreeBSD
The value of the constant PTHREAD_MUTEX_RECURSIVE is not "1" on FreeBSD and OS X.

Differential Revision: http://reviews.llvm.org/D16075

llvm-svn: 257758
2016-01-14 12:12:58 +00:00
Kostya Serebryany abc2c998d2 [tsan] don't crash on closedir(0)
llvm-svn: 257223
2016-01-08 22:48:19 +00:00
Kuba Brecka 3aac36ad01 Revert r255996 ("[tsan] Add a DCHECK to verify __tsan_read* and __tsan_write function aren't called from ScopedInterceptor").
There are some test failures on the Linux buildbots.

llvm-svn: 255997
2015-12-18 13:52:08 +00:00
Kuba Brecka 15dd456236 [tsan] Add a DCHECK to verify __tsan_read* and __tsan_write function aren't called from ScopedInterceptor
Interceptors using ScopedInteceptor should never call into user's code before the ScopedInterceptor is out of scope (and its destructor is called). Let's add a DCHECK to enforce that.

Differential Revision: http://reviews.llvm.org/D15381

llvm-svn: 255996
2015-12-18 13:08:15 +00:00
Kuba Brecka f9cc9d7392 [tsan] Fix scoping of ScopedInteceptor in libdispatch support
Some interceptors in tsan_libdispatch_mac.cc currently wrongly use TSAN_SCOPED_INTERCEPTOR/ScopedInterceptor. Its constructor can start ignoring memory accesses, and the destructor the stops this -- however, e.g. dispatch_sync can call user's code, so the ignoring will extend to user's code as well. This is not expected and we should only limit the scope of ScopedInterceptor to TSan code.  This patch introduces annotations that mark the beginning and ending of a callback into user's code.

Differential Revision: http://reviews.llvm.org/D15419

llvm-svn: 255995
2015-12-18 11:16:24 +00:00
Alexey Samsonov c63c0d1cc0 [TSan] Try harder to avoid compiler-generated memcpy calls.
check_memcpy test added in r254959 fails on some configurations due to
memcpy() calls inserted by Clang. Try harder to avoid them by using
internal_memcpy() where applicable.

llvm-svn: 255287
2015-12-10 19:24:27 +00:00
Yabin Cui 7f48808882 [tsan] Move emptyset/oldset to ThreadSignalContext.
Summary:
Android doesn't support __thread keyword. So move emptyset/oldset
from THREADLOCAL to ThreadSignalContext.

Reviewers: kcc, eugenis, dvyukov

Subscribers: llvm-commits, tberghammer, danalbert

Differential Revision: http://reviews.llvm.org/D15299

llvm-svn: 255168
2015-12-09 22:40:31 +00:00
Yabin Cui 4ddbe8508a [tsan] Use REAL(malloc) instead of __libc_malloc for Android.
Summary:
Android doesn't have __libc_malloc and related allocation
functions. As its dynamic linker doesn't use malloc, so
we can use REAL(malloc) to replace __libc_malloc safely.

Reviewers: kcc, eugenis, dvyukov

Subscribers: llvm-commits, tberghammer, danalbert, srhines

Differential Revision: http://reviews.llvm.org/D15297

llvm-svn: 255167
2015-12-09 22:32:38 +00:00
Yabin Cui 903ab8a2a3 [tsan] Disable interceptors not supported in Android.
Reviewers: kcc, eugenis, dvyukov

Subscribers: llvm-commits, tberghammer, danalbert, srhines

Differential Revision: http://reviews.llvm.org/D15295

llvm-svn: 255164
2015-12-09 22:23:47 +00:00
Yabin Cui d28ffacc99 [tsan] Define sigaction_t for Android.
Reviewers: kcc, eugenis, dvyukov

Subscribers: llvm-commits, tberghammer, danalbert, srhines

Differential Revision: http://reviews.llvm.org/D15298

llvm-svn: 255135
2015-12-09 18:37:27 +00:00
Bill Schmidt 6b63d576c3 [PPC64, TSAN] Provide setjmp interceptor support for PPC64
This patch provides the assembly support for setjmp/longjmp for use
with the thread sanitizer.  This is a big more complicated than for
aarch64, because sibcalls are only legal under our ABIs if the TOC
pointer is unchanged.  Since the true setjmp function trashes the TOC
pointer, and we have to leave the stack in a correct state, we emulate
the setjmp function rather than branching to it.

We also need to materialize the TOC for cases where the _setjmp code
is called from libc.  This is done differently under the ELFv1 and
ELFv2 ABIs.

llvm-svn: 255059
2015-12-08 22:14:34 +00:00
Bill Schmidt 2979162732 [PPC64, TSAN] LLVM basic enablement of thread sanitizer for PPC64 (BE and LE)
This patch is by Simone Atzeni with portions by Adhemerval Zanella.

This contains the LLVM patches to enable the thread sanitizer for
PPC64, both big- and little-endian.  Two different virtual memory
sizes are supported:  Old kernels use a 44-bit address space, while
newer kernels require a 46-bit address space.

There are two companion patches that will be added shortly.  There is
a Clang patch to actually turn on the use of the thread sanitizer for
PPC64.  There is also a patch that I wrote to provide interceptor
support for setjmp/longjmp on PPC64.

Patch discussion at reviews.llvm.org/D12841.

llvm-svn: 255057
2015-12-08 21:54:39 +00:00
Dmitry Vyukov eee690b29a tsan: fix test invisible barrier
Another attempt at fixing tsan_invisible_barrier.
Current implementation causes:
https://llvm.org/bugs/show_bug.cgi?id=25643

There were several unsuccessful iterations for this functionality:

Initially it was implemented in user code using REAL(pthread_barrier_wait). But pthread_barrier_wait is not supported on MacOS. Futexes are linux-specific for this matter.
Then we switched to atomics+usleep(10). But usleep produced parasitic "as-if synchronized via sleep" messages in reports which failed some output tests.
Then we switched to atomics+sched_yield. But this produced tons of tsan- visible events, which lead to "failed to restore stack trace" failures.
Move implementation into runtime and use internal_sched_yield in the wait loop.
This way tsan should see no events from the barrier, so not trace overflows and
no "as-if synchronized via sleep" messages.

llvm-svn: 255030
2015-12-08 17:54:47 +00:00
Kuba Brecka 77ff411247 [tsan] Fix signals and setjmp/longjmp on OS X
1) There's a few wrongly defined things in tsan_interceptors.cc,
2) a typo in tsan_rtl_amd64.S which calls setjmp instead of sigsetjmp in the interceptor, and
3) on OS X, accessing an mprotected page results in a SIGBUS (and not SIGSEGV).

Differential Revision: http://reviews.llvm.org/D15052

llvm-svn: 254299
2015-11-30 14:11:02 +00:00
Kuba Brecka f94bb99adc [tsan] Port tsan_rtl_amd64.S to OS X to add support for setjmp/longjmp
This patch ports the assembly file tsan_rtl_amd64.S to OS X, where we need several changes:

* Some assembler directives are not available on OS X (.hidden, .type, .size)
* Symbol names need to start with an underscore (added a ASM_TSAN_SYMBOL macro for that).
* To make the interceptors work, we ween to name the function "_wrap_setjmp" (added ASM_TSAN_SYMBOL_INTERCEPTOR for that).
* Calling the original setjmp is done with a simple "jmp _setjmp".
* __sigsetjmp doesn't exist on OS X.

Differential Revision: http://reviews.llvm.org/D14947

llvm-svn: 254228
2015-11-28 12:44:23 +00:00
Kuba Brecka 8443d187e6 [tsan] Fix signal number definitions for OS X
On OS X, SIGBUS is 10 and SIGSYS is 12.

Differential Revision: http://reviews.llvm.org/D14946

llvm-svn: 253983
2015-11-24 13:44:54 +00:00
Kuba Brecka 6bbb8519e2 [tsan] Fix __cxa_guard_* interceptors on OS X
This patch fixes the __cxa_guard_acquire, __cxa_guard_release and __cxa_guard_abort interceptors on OS X. They apparently work on Linux just by having the same name, but on OS X, we actually need to use TSAN_INTERCEPTOR.

Differential Revision: http://reviews.llvm.org/D14868

llvm-svn: 253776
2015-11-21 12:53:34 +00:00
Kuba Brecka f503d51e5b [tsan] For OS X thread finalization, remove g_thread_finalize_key in favor of libpthread hooks
On OS X, the thread finalization is fragile due to thread-local variables destruction order. I've seen cases where the we destroy the ThreadState too early and subsequent thread-local values' destructors call interceptors again. Let's replace the TLV-based thread finalization method with libpthread hooks. The notification PTHREAD_INTROSPECTION_THREAD_TERMINATE is called *after* all TLVs have been destroyed.

Differential Revision: http://reviews.llvm.org/D14777

llvm-svn: 253560
2015-11-19 12:06:20 +00:00
Kuba Brecka efd92b3d16 [tsan] Handle dispatch_once on OS X
Reimplement dispatch_once in an interceptor to solve these issues that may produce false positives with TSan on OS X:

1) there is a racy load inside an inlined part of dispatch_once,
2) the fast path in dispatch_once doesn't perform an acquire load, so we don't properly synchronize the initialization and subsequent uses of whatever is initialized,
3) dispatch_once is already used in a lot of already-compiled code, so TSan doesn't see the inlined fast-path.

This patch uses a trick to avoid ever taking the fast path (by never storing ~0 into the predicate), which means the interceptor will always be called even from already-compiled code. Within the interceptor, our own atomic reads and writes are not written into shadow cells, so the race in the inlined part is not reported (because the accesses are only loads).

Differential Revision: http://reviews.llvm.org/D14811

llvm-svn: 253552
2015-11-19 10:35:35 +00:00
Adhemerval Zanella 2f7f5e3535 [compiler-rt] [tsan] Enable intercept setjmp/longjmp for AArch64
This patch adds assembly routines to enable setjmp/longjmp for aarch64
on linux.  It fixes:

 * test/tsan/longjmp2.cc
 * test/tsan/longjmp3.cc
 * test/tsan/longjmp4.cc
 * test/tsan/signal_longjmp.cc

I also checked with perlbench from specpu2006 (it fails to run
with missing setjmp/longjmp intrumentation).

llvm-svn: 253205
2015-11-16 13:55:19 +00:00
Kuba Brecka dccffd4fcc Trying to fix the FreeBSD build breakage due to r251916.
http://lab.llvm.org:8011/builders/sanitizer_x86_64-freebsd/builds/6395

llvm-svn: 252277
2015-11-06 08:16:24 +00:00
Kuba Brecka 537b6090d1 [tsan] Fix build warnings on OS X
Fixing `tsan_interceptors.cc`, which on OS X produces a bunch of warnings about unused constants and functions.

Differential Revision: http://reviews.llvm.org/D14381

llvm-svn: 252165
2015-11-05 14:32:56 +00:00
Kuba Brecka 9e0cf38d9b Lint warning fixup for r252160 ("[tsan] Fix pthread_once interceptor for OS X").
llvm-svn: 252163
2015-11-05 14:16:42 +00:00
Kuba Brecka 245bcf9eb9 [tsan] Fix the memcpy interceptor to be memmove compatible on OS X
On OS X, memcpy and memmove are actually aliases of the same implementation, which means the interceptor of memcpy is also invoked when memmove is called. The current implementation of the interceptor uses `internal_memcpy` to perform the actual memory operation, which can produce an incorrect result when memmove semantics are expected. Let's call `internal_memmove` instead.

Differential Revision: http://reviews.llvm.org/D14336

llvm-svn: 252162
2015-11-05 14:03:26 +00:00
Kuba Brecka 3d8536240a [tsan] Allow memmove interceptor to be used when TSan is not initialized
A call to memmove is used early during new thread initialization on OS X. This patch uses the `COMMON_INTERCEPTOR_NOTHING_IS_INITIALIZED` check, similarly to how we deal with other early-used interceptors.

Differential Revision: http://reviews.llvm.org/D14377

llvm-svn: 252161
2015-11-05 14:01:53 +00:00
Kuba Brecka 12bba1c2a0 [tsan] Fix pthread_once interceptor for OS X
TSan has a re-implementation of `pthread_once` in its interceptor, which assumes that the `pthread_once_t *once_control` pointer is actually pointing to a "storage" which is zero-initialized and used for the atomic operations. However, that's not true on OS X, where pthread_once_t is a structure, that contains a header (with a magic value) and the actual storage follows after that. This patch skips the header to make the interceptor work on OS X.

Differential Revision: http://reviews.llvm.org/D14379

llvm-svn: 252160
2015-11-05 13:59:07 +00:00
Kuba Brecka cd18f28751 [tsan] Alternative ThreadState storage for OS X
This implements a "poor man's TLV" to be used for TSan's ThreadState on OS X. Based on the fact that `pthread_self()` is always available and reliable and returns a valid pointer to memory, we'll use the shadow memory of this pointer as a thread-local storage. No user code should ever read/write to this internal libpthread structure, so it's safe to use it for this purpose. We lazily allocate the ThreadState object and store the pointer here.

Differential Revision: http://reviews.llvm.org/D14288

llvm-svn: 252159
2015-11-05 13:54:50 +00:00
Kuba Brecka 40ad607be6 [tsan] Use malloc zone interceptors on OS X, part 2
TSan needs to use a custom malloc zone on OS X, which is already implemented in ASan.  This patch uses the sanitizer_common implementation in `sanitizer_malloc_mac.inc` for TSan as well.

Reviewed at http://reviews.llvm.org/D14330

llvm-svn: 252155
2015-11-05 10:31:21 +00:00
Alexey Samsonov e5739981d5 [TSan] Fix mmap/mmap64 interceptor signature.
mmap() offset argument has type off_t, not unsigned. off_t is usually
64-bit on 64-bit Linux.

llvm-svn: 252103
2015-11-05 00:00:36 +00:00
Kuba Brecka a1186b828f [tsan] Handle libdispatch worker threads on OS X
On OS X, GCD worker threads are created without a call to pthread_create. We need to properly register these threads with ThreadCreate and ThreadStart. This patch uses a libpthread API (`pthread_introspection_hook_install`) to get notifications about new threads and about threads that are about to be destroyed.

Differential Revision: http://reviews.llvm.org/D14328

llvm-svn: 252049
2015-11-04 15:09:14 +00:00
Kuba Brecka 127f669913 Fixup for r251923 to fix a warning about an extra semicolon.
llvm-svn: 251924
2015-11-03 14:45:32 +00:00
Kuba Brecka 149012153c Reapply r251916 ("[tsan] Port TSan interceptors on OS X").
llvm-svn: 251923
2015-11-03 14:43:39 +00:00
Kuba Brecka 9d71caa9a4 Revert r251916 ("[tsan] Port TSan interceptors on OS X").
llvm-svn: 251922
2015-11-03 14:42:03 +00:00
Kuba Brecka 0ca1e2c3f9 [tsan] Port TSan interceptors on OS X
This patch modifies `tsan_interceptors.cc` to be buildable on OS X. Several of the intercepted methods are not available on OS X, so we need to `#if !SANITIZER_MAC` them. Plus a few other fixes, e.g. `pthread_yield` doesn't exist, let's use `internal_sched_yield` instead.

This is part of an effort to port TSan to OS X, and it's one the very first steps. Don't expect TSan on OS X to actually work or pass tests at this point.

Differential Revision: http://reviews.llvm.org/D14237

llvm-svn: 251916
2015-11-03 14:22:51 +00:00
Dmitry Vyukov 3464dac0ca tsan: speed up race deduplication
Race deduplication code proved to be a performance bottleneck in the past if suppressions/annotations are used, or just some races left unaddressed. And we still get user complaints about this:
https://groups.google.com/forum/#!topic/thread-sanitizer/hB0WyiTI4e4
ReportRace already has several layers of caching for racy pcs/addresses to make deduplication faster. However, ReportRace still takes a global mutex (ThreadRegistry and ReportMutex) during deduplication and also calls mmap/munmap (which take process-wide semaphore in kernel), this makes deduplication non-scalable.

This patch moves race deduplication outside of global mutexes and also removes all mmap/munmap calls.
As the result, race_stress.cc with 100 threads and 10000 iterations become 30x faster:

before:
real	0m21.673s
user	0m5.932s
sys	0m34.885s

after:
real	0m0.720s
user	0m23.646s
sys	0m1.254s

http://reviews.llvm.org/D12554

llvm-svn: 246758
2015-09-03 11:20:46 +00:00
Evgeniy Stepanov 533fde9f4d [sanitizer] Move sem_* to common interceptors.
llvm-svn: 246184
2015-08-27 18:51:18 +00:00
Alexey Samsonov eb4fe7883f [TSan] Support __sanitizer_set_death_callback().
llvm-svn: 245776
2015-08-22 01:07:02 +00:00
Adhemerval Zanella d7984710ae [tsan] Enable tsan for aarch64
This patch enabled TSAN for aarch64 with 39-bit VMA layout.  As defined by
tsan_platform.h the layout used is:

0000 4000 00 - 0200 0000 00: main binary
2000 0000 00 - 4000 0000 00: shadow memory
4000 0000 00 - 5000 0000 00: metainfo
5000 0000 00 - 6000 0000 00: -
6000 0000 00 - 6200 0000 00: traces
6200 0000 00 - 7d00 0000 00: -
7d00 0000 00 - 7e00 0000 00: heap
7e00 0000 00 - 7fff ffff ff: modules and main thread stack

Which gives it about 8GB for main binary, 4GB for heap and 8GB for
modules and main thread stack.

Most of tests are passing, with the exception of:

 * ignore_lib0, ignore_lib1, ignore_lib3 due a kernel limitation for
   no support to make mmap page non-executable.

 * longjmp tests due missing specialized assembly routines.

These tests are xfail for now.

The only tsan issue still showing is:

  rtl/TsanRtlTest/Posix.ThreadLocalAccesses

Which still required further investigation.  The test is disable for
aarch64 for now.

llvm-svn: 244055
2015-08-05 15:17:59 +00:00
Kostya Serebryany c69bf1b4c0 [asan,tsan,msan] move the memcmp interceptor from asan/tsan to sanitizer_common. This may potentially lead to more reports from msan as it now sees the reads inside memcmp. To disable, use the flag intercept_memcmp=0. Likewise, it may potentially cause new races to appear due to more strict memcmp checking (flag strict_memcmp=1)
llvm-svn: 243595
2015-07-29 23:53:08 +00:00
Ismail Pazarbasi 0193c3a40e __tsan::ThreadCreate takes incorrect value for detached argument
Summary:
PTHREAD_CREATE_DETACHED has a different value on Mac OS X. Since both
PTHREAD_CREATE_JOINABLE and PTHREAD_CREATE_DETACHED are non-zero,
`__tsan::ThreadCreate` always creates detached threads.

Reviewers: kcc, samsonov, glider

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D10606

llvm-svn: 243151
2015-07-24 20:00:49 +00:00
Alexey Samsonov f40c1076c9 [TSan] Fix dl_iterate_phdr callback for the case when info->dlpi_name is overwritten by user.
llvm-svn: 241876
2015-07-10 00:03:54 +00:00
Dmitry Vyukov 1d9b5e6ece tsan: fix handling of condition variable destruction
POSIX states that "It shall be safe to destroy an initialized condition
variable upon which no threads are currently blocked", and later clarifies
"A condition variable can be destroyed immediately after all the threads
that are blocked on it are awakened) (in examples section). Tsan reported
such destruction as a data race.
Fixes https://llvm.org/bugs/show_bug.cgi?id=23616
Reviewed in http://reviews.llvm.org/D10693

llvm-svn: 241082
2015-06-30 17:23:29 +00:00
Evgeniy Stepanov 856b11199f [asan] Fix SanitizerCommon.PthreadDestructorIterations test on Android L.
On Android L, TSD destructors run 8 times instead of 4.
Back to 4 times on the current master branch (as well as on K).

llvm-svn: 240992
2015-06-29 20:28:55 +00:00
Dmitry Vyukov 7c63340586 tsan: fix handling of dup2
Previously tsan modelled dup2(oldfd, newfd) as write on newfd.
We hit several cases where the write lead to false positives:
1. Some software dups a closed pipe in place of a socket before closing
   the socket (to prevent races actually).
2. Some daemons dup /dev/null in place of stdin/stdout.
On the other hand we have not seen cases when write here catches real bugs.
So model dup2 as read on newfd instead.

llvm-svn: 240687
2015-06-25 20:32:04 +00:00
Dmitry Vyukov d58df749c1 tsan: fix false positive between dlopen and dl_iterate_phdr
We see false reports between dlopen and dl_iterate_phdr.
This happens because tsan does not see dynamic linker
internal synchronization. Unpoison module names
in dl_iterate_phdr callback.

llvm-svn: 240576
2015-06-24 19:49:32 +00:00
Yury Gribov 0ca65fd83d [sanitizer] More string interceptors: strstr, strcasestr, strspn, strcspn, strpbrk.
Patch by Maria Guseva.

Differential Revision: http://reviews.llvm.org/D9017

llvm-svn: 238406
2015-05-28 09:24:33 +00:00
Evgeniy Stepanov 7f6290ca9a [sanitizer] Recognize static TLS in __tls_get_addr interceptor.
Current code tries to find the dynamic TLS header to the left of the
TLS block without checking that it's not a static TLS allocation.

llvm-svn: 237495
2015-05-16 00:34:15 +00:00
Alexey Samsonov 0eafe5df71 [TSan] Move new/delete interceptors into a separate source file. NFC.
llvm-svn: 235906
2015-04-27 19:33:55 +00:00
Dmitry Vyukov 8586e2352b tsan: fix handling of pthread_detach
Fixes https://llvm.org/bugs/show_bug.cgi?id=23235

If pthread_create is followed by pthread_detach,
the new thread may not acquire synchronize with
the parent thread.

llvm-svn: 235293
2015-04-20 10:35:10 +00:00
Dmitry Vyukov 8e39c404a0 tsan: handle async signals while blocked in pthread_cond_wait
Fixes https://code.google.com/p/thread-sanitizer/issues/detail?id=91

llvm-svn: 234394
2015-04-08 07:48:52 +00:00
Dmitry Vyukov 1e5b9f4131 sanitizer: new "strict_string_checks" run-time flag
This patch is related to Issue 346: moar string interceptors: strstr, strcasestr, strcspn, strpbrk
As was suggested in original review http://reviews.llvm.org/D6056 a new "strict_string_checks" run-time flag introduced.
The flag support applied for existing common, asan, msan and tsan interceptors. New asan tests added.

Change by Maria Guseva reviewed in http://reviews.llvm.org/D7123

llvm-svn: 234187
2015-04-06 18:00:26 +00:00
Dmitry Vyukov 19300f856b tsan: fix munmap interceptor
MetaMap::ResetRange/FreeRange rounds the range up to at least kMetaShadowSize.
This is requried for e.g. free(malloc(0)). However, munmap returns EINVAL
and do not unmap any memory when length arguments is equal to 0.
So don't free meta shadow in this case as well.

llvm-svn: 234145
2015-04-06 07:57:32 +00:00
Viktor Kutuzov eff2d98ecc [Tsan] Do not sanitize memset() and other functions during initialization
Differential Revision: http://reviews.llvm.org/D8544

llvm-svn: 233378
2015-03-27 14:12:28 +00:00
Viktor Kutuzov a7d323e996 [Tsan] Do not sanitize memcpy() during thread initialization on FreeBSD
Differential Revision: http://reviews.llvm.org/D8324

llvm-svn: 232381
2015-03-16 14:42:21 +00:00
Viktor Kutuzov 07a21dc46d [Tsan] Adjust SA_SIGINFO and SIG_SETMASK values on FreeBSD.
Differential Revision: http://reviews.llvm.org/D8176

llvm-svn: 232073
2015-03-12 18:12:43 +00:00