Commit Graph

105 Commits

Author SHA1 Message Date
Kuba Brecka 312071943c [tsan] On OS X, build Go runtime with -mmacosx-version-min
We're not building the Go runtime with -mmacosx-version-min, which means it'll have a minimum deployment target set to the system you're building on. Let's make the code compile (and link) with -mmacosx-version-min=10.7.

Differential Revision: http://reviews.llvm.org/D20670

llvm-svn: 271833
2016-06-05 15:15:36 +00:00
Kostya Serebryany d9a451ab0d [sanitizer] Move termination functions in their own file
Summary:
As suggested by kcc@ in http://reviews.llvm.org/D20084#441418, move the CheckFailed and Die functions, and their associated callback functionalities in their own separate file.
I expended the build rules to include a new rule that would not include those termination functions, so that another project can define their own.
The tests check-{a,t,m,ub,l,e,df}san are all passing.

Reviewers: llvm-commits, kcc

Subscribers: kubabrecka

Differential Revision: http://reviews.llvm.org/D20742

llvm-svn: 271055
2016-05-27 21:57:17 +00:00
Dmitry Vyukov 5aac22fc23 tsan: fix build
error: implicit declaration of function 'abort' is invalid in C99
llvm-svn: 267710
2016-04-27 12:59:35 +00:00
Dmitry Vyukov 3efe395788 tsan: change tsan/Go interface for obtaining the current Processor
Current interface assumes that Go calls ProcWire/ProcUnwire
to establish the association between thread and proc.
With the wisdom of hindsight, this interface does not work
very well. I had to sprinkle Go scheduler with wire/unwire
calls, and any mistake leads to hard to debug crashes.
This is not something one wants to maintian.
Fortunately, there is a simpler solution. We can ask Go
runtime as to what is the current Processor, and that
question is very easy to answer on Go side.
Switch to such interface.

llvm-svn: 267703
2016-04-27 12:30:48 +00:00
Dmitry Vyukov d87c7b321a tsan: split thread into logical and physical state
This is reincarnation of http://reviews.llvm.org/D17648 with the bug fix pointed out by Adhemerval (zatrazz).

Currently ThreadState holds both logical state (required for race-detection algorithm, user-visible)
and physical state (various caches, most notably malloc cache). Move physical state in a new
Process entity. Besides just being the right thing from abstraction point of view, this solves several
problems:

Cache everything on P level in Go. Currently we cache on a mix of goroutine and OS thread levels.
This unnecessary increases memory consumption.

Properly handle free operations in Go. Frees are issue by GC which don't have goroutine context.
As the result we could not do anything more than just clearing shadow. For example, we leaked
sync objects and heap block descriptors.

This will allow to get rid of libc malloc in Go (now we have Processor context for internal allocator cache).
This in turn will allow to get rid of dependency on libc entirely.

Potentially we can make Processor per-CPU in C++ mode instead of per-thread, which will
reduce resource consumption.
The distinction between Thread and Processor is currently used only by Go, C++ creates Processor per OS thread,
which is equivalent to the current scheme.

llvm-svn: 267678
2016-04-27 08:23:02 +00:00
Alexey Samsonov 4362a994d2 [TSan] Fix compiler warning in Go sanity test.
llvm-svn: 262680
2016-03-04 00:56:15 +00:00
Dmitry Vyukov a029b79e1a tsan: describe heap/data locations in Go
llvm-svn: 262343
2016-03-01 15:38:12 +00:00
Dmitry Vyukov 7f022ae4c2 tsan: revert r262037
Broke aarch64 and darwin bots.

llvm-svn: 262046
2016-02-26 18:26:48 +00:00
Dmitry Vyukov b8868b9bea tsan: split thread into logical and physical state
Currently ThreadState holds both logical state (required for race-detection algorithm, user-visible)
and physical state (various caches, most notably malloc cache). Move physical state in a new
Process entity. Besides just being the right thing from abstraction point of view, this solves several
problems:
1. Cache everything on P level in Go. Currently we cache on a mix of goroutine and OS thread levels.
This unnecessary increases memory consumption.
2. Properly handle free operations in Go. Frees are issue by GC which don't have goroutine context.
As the result we could not do anything more than just clearing shadow. For example, we leaked
sync objects and heap block descriptors.
3. This will allow to get rid of libc malloc in Go (now we have Processor context for internal allocator cache).
This in turn will allow to get rid of dependency on libc entirely.
4. Potentially we can make Processor per-CPU in C++ mode instead of per-thread, which will
reduce resource consumption.
The distinction between Thread and Processor is currently used only by Go, C++ creates Processor per OS thread,
which is equivalent to the current scheme.

llvm-svn: 262037
2016-02-26 16:57:14 +00:00
Dimitry Andric 042c4858b0 Fix lib/tsan/go/buildgo.sh for FreeBSD after r243051.
FreeBSD also needs to have sanitizer_linux_libcdep.cc included,
otherwise linking will fail with "undefined reference to
`__sanitizer::GetRSS()'".

While here, tabify the FreeBSD part, similar to the other parts.

llvm-svn: 260839
2016-02-14 00:26:32 +00:00
Anna Zaks 2ce9f9447c Reapply: [asan] On OS X, log reports to syslog and os_trace
When ASan currently detects a bug, by default it will only print out the text
of the report to stderr. This patch changes this behavior and writes the full
text of the report to syslog before we terminate the process. It also calls
os_trace (Activity Tracing available on OS X and iOS) with a message saying
that the report is available in syslog. This is useful, because this message
will be shown in the crash log.

For this to work, the patch makes sure we store the full report into
error_message_buffer unconditionally, and it also strips out ANSI escape
sequences from the report (they are used when producing colored reports).

I've initially tried to log to syslog during printing, which is done on Android
right now. The advantage is that if we crash during error reporting or the
produced error does not go through ScopedInErrorReport, we would still get a
(partial) message in the syslog. However, that solution is very problematic on
OS X. One issue is that the logging routine uses GCD, which may spawn a new
thread on its behalf. In many cases, the reporting logic locks threadRegistry,
which leads to deadlocks.

Reviewed at http://reviews.llvm.org/D13452

(In addition, add sanitizer_common_libcdep.cc to buildgo.sh to avoid
build failures on Linux.)

llvm-svn: 253688
2015-11-20 18:41:44 +00:00
Dmitry Vyukov 110094017c tsan: fix Go build on linux
librt is not linked in in Go build.

llvm-svn: 252877
2015-11-12 11:54:25 +00:00
Juergen Ributzka d7dcffaea4 Revert "Reapply: [asan] On OS X, log reports to syslog and os_trace"
Looks like this commit is deadlocking the ASAN tests on the green dragon bot
(http://lab.llvm.org:8080/green/job/clang-stage1-cmake-RA/).

llvm-svn: 252076
2015-11-04 21:03:12 +00:00
Anna Zaks 7920eb0b1f Reapply: [asan] On OS X, log reports to syslog and os_trace
When ASan currently detects a bug, by default it will only print out the text
        of the report to stderr. This patch changes this behavior and writes the full
        text of the report to syslog before we terminate the process. It also calls
        os_trace (Activity Tracing available on OS X and iOS) with a message saying
        that the report is available in syslog. This is useful, because this message
        will be shown in the crash log.

        For this to work, the patch makes sure we store the full report into
        error_message_buffer unconditionally, and it also strips out ANSI escape
        sequences from the report (they are used when producing colored reports).

        I've initially tried to log to syslog during printing, which is done on Android
        right now. The advantage is that if we crash during error reporting or the
        produced error does not go through ScopedInErrorReport, we would still get a
        (partial) message in the syslog. However, that solution is very problematic on
        OS X. One issue is that the logging routine uses GCD, which may spawn a new
        thread on its behalf. In many cases, the reporting logic locks threadRegistry,
        which leads to deadlocks.

        Reviewed at http://reviews.llvm.org/D13452

        (In addition, add sanitizer_common_libcdep.cc to buildgo.sh to avoid
         build failures on Linux.)

llvm-svn: 251577
2015-10-28 23:18:44 +00:00
Evgeniy Stepanov 5a268b10b3 [sanitizer] Implement logging to syslog.
Previously, Android target had a logic of duplicating all sanitizer
output to logcat. This change extends it to all posix platforms via
the use of syslog, controlled by log_to_syslog flag. Enabled by
default on Android, off everywhere else.

A bit of cmake magic is required to allow Printf() to call a libc
function. I'm adding a stub implementation to support no-libc builds
like dfsan and safestack.

This is a second attempt. I believe I've fixed all the issues that
prompted the revert: Mac build, and all kinds of non-CMake builds
(there are 3 of those).

llvm-svn: 243051
2015-07-23 22:05:20 +00:00
Evgeniy Stepanov 8e9c70be7f Add descriptive names to sanitizer entries in /proc/self/maps. Helps debugging.
This is done by creating a named shared memory region, unlinking it
and setting up a private (i.e. copy-on-write) mapping of that instead
of a regular anonymous mapping. I've experimented with regular
(sparse) files, but they can not be scaled to the size of MSan shadow
mapping, at least on Linux/X86_64 and ext3 fs.

Controlled by a common flag, decorate_proc_maps, disabled by default.

This patch has a few shortcomings:
* not all mappings are annotated, especially in TSan.
* our handling of memset() of shadow via mmap() puts small anonymous
  mappings inside larger named mappings, which looks ugly and can, in
  theory, hit the mapping number limit.

llvm-svn: 238621
2015-05-29 22:31:28 +00:00
Dmitri Gribenko 9cbc6f42aa Revert r230171 "TSan runtime: unbreak the build with ccache"
It broke a buildbot with a non-clang (?) compiler.

llvm-svn: 230174
2015-02-22 12:20:05 +00:00
Dmitri Gribenko bc4c42c345 TSan runtime: unbreak the build with ccache
llvm-svn: 230171
2015-02-22 11:04:59 +00:00
Dmitry Vyukov 2a80437038 tsan: add -Wno-maybe-uninitialized to mingw gcc build flags
As per discussion with David Blaikie here:
http://lists.cs.uiuc.edu/pipermail/llvm-commits/Week-of-Mon-20150216/260289.html

llvm-svn: 229657
2015-02-18 09:37:31 +00:00
Dmitry Vyukov 620ebaa9c1 tsan: remove everything related to rss/background thread in Go mode
In Go mode the background thread is not started (internal_thread_start is empty).
There is no sense in having this code compiled in.
Also removes dependency on sanitizer_linux_libcdep.cc which is good,
ideally Go runtime does not depend on libc at all.

llvm-svn: 229396
2015-02-16 14:49:34 +00:00
Dmitry Vyukov 1236868f85 tsan: fix windows build script
add missing source files
+ add -std=c++11 as it is now required to build compiler-rt

llvm-svn: 229393
2015-02-16 13:54:32 +00:00
Alexey Samsonov ec80d97c77 [TSan] Fix remaining bashism in buildgo.sh script.
llvm-svn: 227893
2015-02-02 23:14:46 +00:00
Alexey Samsonov 1270c530e8 [TSan] Fix Go sanity check script.
llvm-svn: 227870
2015-02-02 22:07:39 +00:00
Alexey Samsonov 46fe4a640d [TSan] Add Go runtime sanity check to CMake build.
llvm-svn: 227852
2015-02-02 20:37:28 +00:00
Alexey Samsonov a11a86b9ce [TSan] Fix Go build.
llvm-svn: 227634
2015-01-30 23:04:57 +00:00
Dmitry Vyukov cc5e40811b tsan: remove TSAN_SHADOW_COUNT
TSAN_SHADOW_COUNT is defined to 4 in all environments.
Other values of TSAN_SHADOW_COUNT were never tested and
were broken by recent changes to shadow mapping.
Remove it as there is no reason to fix nor maintain it.

llvm-svn: 226466
2015-01-19 15:01:07 +00:00
Alexey Samsonov 7eba6dfdaa [TSan] Fix Go build.
llvm-svn: 225336
2015-01-07 02:12:41 +00:00
Alexey Samsonov df3aeb8e71 Remove TSAN_DEBUG in favor of SANITIZER_DEBUG.
llvm-svn: 225111
2015-01-03 04:29:12 +00:00
Kostya Serebryany 83ed889bbd [tsan] remove TSAN_GO in favor of SANITIZER_GO
llvm-svn: 223732
2014-12-09 01:31:14 +00:00
Alexey Samsonov 0e90668f14 Simplify Symbolizer::SymbolizePC() interface.
Return a linked list of AddressInfo objects, instead of using an array of
these objects as an output parameter. This simplifies the code in callers
of this function (especially TSan).

Fix a few memory leaks from internal allocator, when the returned
AddressInfo objects were not properly cleared.

llvm-svn: 223145
2014-12-02 19:48:40 +00:00
Dmitry Vyukov 1f84559b7d tsan: fix Go build
llvm-svn: 222342
2014-11-19 10:43:11 +00:00
Alexey Samsonov e3754634c0 [TSan] Make ReportStack contain __sanitizer::AddressInfo object.
AddressInfo contains the results of symbolization. Store this object
directly in the symbolized stack, instead of copying data around and
making unnecessary memory allocations.

No functionality change.

llvm-svn: 221294
2014-11-04 20:52:37 +00:00
Alexey Samsonov 46ecdeecc4 [TSan} Build Go version with -std=c++11
llvm-svn: 221192
2014-11-03 22:18:12 +00:00
Alexey Samsonov 71b471fa09 [TSan] Try to fix Go runtime build
llvm-svn: 215000
2014-08-06 23:13:23 +00:00
Dmitry Vyukov a8df247f5a tsan: expose atomic operations in Go runtime
llvm-svn: 213382
2014-07-18 14:54:02 +00:00
Dmitry Vyukov 0eb1c6f2fb tsan: fix windows build script
llvm-svn: 211422
2014-06-20 23:22:31 +00:00
Dmitry Vyukov 73e1804c21 tsan: add Go build script for freebsd
llvm-svn: 211419
2014-06-20 22:53:43 +00:00
Dmitry Vyukov 9ff85e9674 tsan: fix argument passing in Go interface
Go calling convention does not support bools (it probably works, but uptrs are safer)

llvm-svn: 210365
2014-06-06 18:53:52 +00:00
Dmitry Vyukov 8744b1c1e3 tsan: allow to build Go runtime with clang + enable SSE3
llvm-svn: 210344
2014-06-06 15:49:49 +00:00
Dmitry Vyukov bde4c9c773 tsan: refactor storage of meta information for heap blocks and sync objects
The new storage (MetaMap) is based on direct shadow (instead of a hashmap + per-block lists).
This solves a number of problems:
 - eliminates quadratic behaviour in SyncTab::GetAndLock (https://code.google.com/p/thread-sanitizer/issues/detail?id=26)
 - eliminates contention in SyncTab
 - eliminates contention in internal allocator during allocation of sync objects
 - removes a bunch of ad-hoc code in java interface
 - reduces java shadow from 2x to 1/2x
 - allows to memorize heap block meta info for Java and Go
 - allows to cleanup sync object meta info for Go
 - which in turn enabled deadlock detector for Go

llvm-svn: 209810
2014-05-29 13:50:54 +00:00
Evgeniy Stepanov f77f2ece71 [tsan] Fix gotsan build.
llvm-svn: 209286
2014-05-21 09:42:56 +00:00
Dmitry Vyukov b7ebc53772 tsan: stop background thread when sandbox is enabled
Fixes https://code.google.com/p/thread-sanitizer/issues/detail?id=56

llvm-svn: 207114
2014-04-24 13:09:17 +00:00
Dmitry Vyukov bbbe6bba8a tsan: preliminary support for Go deadlock detector
llvm-svn: 204228
2014-03-19 12:50:47 +00:00
Alexey Samsonov 18adbc361a Remove sanitizer_linux_libcdep.cc from TSan-Go build
llvm-svn: 203238
2014-03-07 13:17:32 +00:00
Dmitry Vyukov dd419aee00 tsan: update Go windows build script
llvm-svn: 203121
2014-03-06 14:49:21 +00:00
Dmitry Vyukov 350e7ddd34 tsan: update interface for Go
this is required to fix:
https://code.google.com/p/go/issues/detail?id=7460

llvm-svn: 203116
2014-03-06 13:17:28 +00:00
Dmitry Vyukov 6cfab724ec tsan: refactor deadlock detector
Introduce DDetector interface between the tool and the DD itself.
It will help to experiment with other DD implementation,
as well as reuse DD in other tools.

llvm-svn: 202485
2014-02-28 10:48:13 +00:00
Dmitry Vyukov 9ba840865f tsan: simplify Go build script
we don't use assembly files

llvm-svn: 200916
2014-02-06 09:23:12 +00:00
Dmitry Vyukov 447bb46e03 tsan: remove unused functions
llvm-svn: 200915
2014-02-06 09:22:50 +00:00
Dmitry Vyukov 7799eb123b tsan: ensure proper alignment of mmaps on windows
llvm-svn: 200832
2014-02-05 11:37:20 +00:00