The new storage (MetaMap) is based on direct shadow (instead of a hashmap + per-block lists).
This solves a number of problems:
- eliminates quadratic behaviour in SyncTab::GetAndLock (https://code.google.com/p/thread-sanitizer/issues/detail?id=26)
- eliminates contention in SyncTab
- eliminates contention in internal allocator during allocation of sync objects
- removes a bunch of ad-hoc code in java interface
- reduces java shadow from 2x to 1/2x
- allows to memorize heap block meta info for Java and Go
- allows to cleanup sync object meta info for Go
- which in turn enabled deadlock detector for Go
llvm-svn: 209810
If a multi-threaded program calls fork(), TSan ignores all memory accesses
in the child to prevent deadlocks in TSan runtime. This is OK, as child is
probably going to call exec() as soon as possible. However, a rare deadlocks
could be caused by ThreadIgnoreBegin() function itself.
ThreadIgnoreBegin() remembers the current stack trace and puts it into the
StackDepot to report a warning later if a thread exited with ignores enabled.
Using StackDepotPut in a child process is dangerous: it locks a mutex on
a slow path, which could be already locked in a parent process.
The fix is simple: just don't put current stack traces to StackDepot in
ThreadIgnoreBegin() and ThreadIgnoreSyncBegin() functions if we're
running after a multithreaded fork. We will not report any
"thread exited with ignores enabled" errors in this case anyway.
Submitting this without a testcase, as I believe the standalone reproducer
is pretty hard to construct.
llvm-svn: 205534
Make vector clock operations O(1) for several important classes of use cases.
See comments for details.
Below are stats from a large server app, 77% of all clock operations are handled as O(1).
Clock acquire : 25983645
empty clock : 6288080
fast from release-store : 14917504
contains my tid : 4515743
repeated (fast) : 2141428
full (slow) : 2636633
acquired something : 1426863
Clock release : 2544216
resize : 6241
fast1 : 197693
fast2 : 1016293
fast3 : 2007
full (slow) : 1797488
was acquired : 709227
clear tail : 1
last overflow : 0
Clock release store : 3446946
resize : 200516
fast : 469265
slow : 2977681
clear tail : 0
Clock acquire-release : 820028
llvm-svn: 204656
Introduce DDetector interface between the tool and the DD itself.
It will help to experiment with other DD implementation,
as well as reuse DD in other tools.
llvm-svn: 202485
This is covered by existing ASan test.
This does not change anything for TSan by default (but provides a flag to
change the threshold size).
Based on a patch by florent.bruneau here:
https://code.google.com/p/address-sanitizer/issues/detail?id=256
llvm-svn: 201400
Currently correct programs can deadlock after fork, because atomic operations and async-signal-safe calls are not async-signal-safe under tsan.
With this change:
- if a single-threaded program forks, the child continues running with verification enabled (the tsan background thread is recreated as well)
- if a multi-threaded program forks, then the child runs with verification disabled (memory accesses, atomic operations and interceptors are disabled); it's expected that it will exec soon anyway
- if the child tries to create more threads after multi-threaded fork, the program aborts with error message
- die_after_fork flag is added that allows to continue running, but all bets are off
http://llvm-reviews.chandlerc.com/D2614
llvm-svn: 199993
This is intended to address the following problem.
Episodically we see CHECK-failures when recursive interceptors call back into user code. Effectively we are not "in_rtl" at this point, but it's very complicated and fragile to properly maintain in_rtl property. Instead get rid of it. It was used mostly for sanity CHECKs, which basically never uncover real problems.
Instead introduce ignore_interceptors flag, which is used in very few narrow places to disable recursive interceptors (e.g. during runtime initialization).
llvm-svn: 197979
Summary:
TSan and MSan need to know if interceptor was called by the
user code or by the symbolizer and use pre- and post-symbolization hooks
for that. Make Symbolizer class responsible for calling these hooks instead.
This would ensure the hooks are only called when necessary (during
in-process symbolization, they are not needed for out-of-process) and
save specific sanitizers from tracing all places in the code where symbolization
will be performed.
Reviewers: eugenis, dvyukov
Reviewed By: eugenis
CC: llvm-commits
Differential Revision: http://llvm-reviews.chandlerc.com/D2067
llvm-svn: 193807
This moves away from creating the symbolizer object and initializing the
external symbolizer as separate steps. Those steps now always take place
together.
Sanitizers with a legacy requirement to specify their own symbolizer path
should use InitSymbolizer to initialize the symbolizer with the desired
path, and GetSymbolizer to access the symbolizer. Sanitizers with no
such requirement (e.g. UBSan) can use GetOrInitSymbolizer with no need for
initialization.
The symbolizer interface has been made thread-safe (as far as I can
tell) by protecting its member functions with mutexes.
Finally, the symbolizer interface no longer relies on weak externals, the
introduction of which was probably a mistake on my part.
Differential Revision: http://llvm-reviews.chandlerc.com/D1985
llvm-svn: 193448
This allows to increase max shadow stack size to 64K,
and reliably catch shadow stack overflows instead of silently
corrupting memory.
llvm-svn: 192797
The annotations are AnnotateIgnoreSyncBegin/End,
may be useful to ignore some infrastructure synchronization
that introduces lots of false negatives.
llvm-svn: 192355
The flag allows to bound maximum process memory consumption (best effort).
If RSS reaches memory_limit_mb, tsan flushes all shadow memory.
llvm-svn: 191913
LibIgnore allows to ignore all interceptors called from a particular set
of dynamic libraries. LibIgnore remembers all "called_from_lib" suppressions
from the provided SuppressionContext; finds code ranges for the libraries;
and checks whether the provided PC value belongs to the code ranges.
Also make malloc and friends interceptors use SCOPED_INTERCEPTOR_RAW instead of
SCOPED_TSAN_INTERCEPTOR, because if they are called from an ignored lib,
then must call our internal allocator instead of libc malloc.
llvm-svn: 191897
Before, we had an unused internal_getpid function for Linux, and a
platform-independent GetPid function. To make the naming conventions
consistent for syscall-like functions, the GetPid syscall wrapper
in sanitizer_posix.cc is moved to sanitizer_mac.cc, and GetPid is
renamed to internal_getpid, bringing the Linux variant into use.
llvm-svn: 182132
This change moves to a model where the error value of a system call is
potentially contained in the return value itself rather than being
implicit in errno. The helper function internal_iserror can be used
to extract the error value from a return value. On platforms other
than Linux/x86_64 this still uses errno, but other platforms are free
to port their error handling to this new model.
Differential Revision: http://llvm-reviews.chandlerc.com/D756
llvm-svn: 181436
Dmitry Vyukov