Capture the computed shadow begin/end values at the point where the
shadow is first created and reuse those values on reset. Introduce new
windows-specific function "ZeroMmapFixedRegion" for zeroing out an
address space region previously returned by one of the MmapFixed*
routines; call this function (on windows) from DoResetImpl
tsan_rtl.cpp instead of MmapFixedSuperNoReserve.
See https://github.com/golang/go/issues/53539#issuecomment-1168778740
for context; intended to help with updating the syso for Go's
windows/amd64 race detector.
Differential Revision: https://reviews.llvm.org/D128909
This is a follow up to [Sanitizers][Darwin] Rename Apple macro SANITIZER_MAC -> SANITIZER_APPLE (D125816)
Performed a global search/replace as in title against LLVM sources
Differential Revision: https://reviews.llvm.org/D126263
The new tsan runtime has 2x more compact shadow.
Adjust shadow ranges accordingly.
Depends on D112603.
Reviewed By: vitalybuka, melver
Differential Revision: https://reviews.llvm.org/D113751
This change switches tsan to the new runtime which features:
- 2x smaller shadow memory (2x of app memory)
- faster fully vectorized race detection
- small fixed-size vector clocks (512b)
- fast vectorized vector clock operations
- unlimited number of alive threads/goroutimes
Depends on D112602.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D112603
This change switches tsan to the new runtime which features:
- 2x smaller shadow memory (2x of app memory)
- faster fully vectorized race detection
- small fixed-size vector clocks (512b)
- fast vectorized vector clock operations
- unlimited number of alive threads/goroutimes
Depends on D112602.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D112603
This change switches tsan to the new runtime which features:
- 2x smaller shadow memory (2x of app memory)
- faster fully vectorized race detection
- small fixed-size vector clocks (512b)
- fast vectorized vector clock operations
- unlimited number of alive threads/goroutimes
Depends on D112602.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D112603
This change switches tsan to the new runtime which features:
- 2x smaller shadow memory (2x of app memory)
- faster fully vectorized race detection
- small fixed-size vector clocks (512b)
- fast vectorized vector clock operations
- unlimited number of alive threads/goroutimes
Differential Revision: https://reviews.llvm.org/D112603
This change switches tsan to the new runtime which features:
- 2x smaller shadow memory (2x of app memory)
- faster fully vectorized race detection
- small fixed-size vector clocks (512b)
- fast vectorized vector clock operations
- unlimited number of alive threads/goroutimes
Depends on D112602.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D112603
This change switches tsan to the new runtime which features:
- 2x smaller shadow memory (2x of app memory)
- faster fully vectorized race detection
- small fixed-size vector clocks (512b)
- fast vectorized vector clock operations
- unlimited number of alive threads/goroutimes
Depends on D112602.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D112603
Building Go programs with the current runtime fails with:
loadelf: race_linux_amd64: malformed elf file:
_ZZN6__tsan15RestoreAddrImpl5ApplyINS_11MappingGo48EEEmmE6ranges: invalid symbol binding 10
Go linker does not understand ELF in all its generality.
Don't use static const data in inline methods.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D112434
Write uptime in real time seconds for every mem profile record.
Uptime is useful to make more sense out of the profile,
compare random lines, etc.
Depends on D110153.
Reviewed By: melver, vitalybuka
Differential Revision: https://reviews.llvm.org/D110154
We currently query number of threads before reading /proc/self/smaps.
But reading /proc/self/smaps can take lots of time for huge processes
and it's retries several times with different buffer sizes.
Overall it can take tens of seconds. This can make number of threads
significantly inconsistent with the rest of the stats.
So query it after reading /proc/self/smaps.
Depends on D110149.
Reviewed By: melver, vitalybuka
Differential Revision: https://reviews.llvm.org/D110150
We use kShadowCnt (number of shadow cells per application granule)
when computing shadow, but it's wrong. We need the ratio
between shadow and app memory (how much shadow is larger than app memory),
which is kShadowMultiplier.
Currently both are equal to 4, so it works fine.
Use the correct constant.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D108033
New tsan runtime will need to compress addresses/PCs to fewer bits.
Add CompressAddr/RestoreAddr functions that compress/restore
addresses to 44 bits.
Depends on D107744.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D107745
Rename mapping field selectors according to the code style.
Reuse the actual field names, there is no need to invent
second set of names.
Depends on D107743.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D107744
Currently there are 2 levels when selecting the active mapping:
the branchy ifdef tree + another ifdef tree in SelectMapping.
Moreover, there is an additional indirection for some platforms
via HAS_48_BIT_ADDRESS_SPACE define. This makes already complex
logic even more complex and almost impossible to read.
Remove one level of indirection and define the active mapping
in SelectMapping.
Depends on D107742.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D107743
Move the mapping checking logic from startup to unit tests
and test all mapping instead of just the active one.
This makes it much more feasible to make any global changes
to the mappings since we have 17 of them.
Depends on D107740.
Reviewed By: vitalybuka, melver
Differential Revision: https://reviews.llvm.org/D107741
Currently we have ifdef's for Go/C++ and Windows/non-Windows
in MemToShadow, MemToMeta, ShadowToMem. This does not allow
to test all mappings on a single platform.
Make all these functions support a superset of mappings for
all platforms by defining missing mapping consts to 0.
E.g. we always do ^A+B, but if A and B are defined to 0,
then these operations become no-op.
Depends on D107739.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D107740
Define all fields to 0 for all mappings.
This allows to write portable code and tests.
For all existing cases 0 values work out of the box
because we check if an address belongs to the range
and nothing belongs to [0, 0] range.
Depends on D107738.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D107739
Unify Go mapping naming with C++ naming to allow
writing portable code/tests that can work for both C++ and Go.
No functional changes.
Depends on D107737.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D107738
First, the define conflicts with definition/testing of all mappings,
since it's not a global property anymore. Second, macros/ifdefs are bad.
Define kMidAppMemBeg/End to 0 to denote that there is no "mid" range instead.
Depends on D107736.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D107737
Currently we have mapping selection duplicated 9 times.
Deduplicate it. No functional changes.
Depends on D107734.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D107735
Currently we define/compile the mapping for a platform
only on that platform. This makes it impossible to unit-test
them on a single platform, and even to build test.
We have 17 of them and the Go mappings will be tested
only after a manual episodic update of the Go runtime.
Define all mappings always with unique names.
This will allow to unit-test them.
No functional changes.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D107734
Currently we hardcode u64 type for shadow everywhere
and do lots of uptr<->u64* casts. It makes it hard to
change u64 to another type (e.g. u32) and makes it easy
to introduce bugs.
Introduce RawShadow type and use it in MemToShadow, ShadowToMem,
IsShadowMem and throughout the code base as u64 replacement.
This makes it possible to change u64 to something else in future
and generally improves static typing.
Depends on D107481.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D107482
MemToMeta returns u32*, so it's reasonable for IsMetaMem
to accept u32* as well.
Changing the argument type just removes few type casts.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D107481
Define the address ranges (similar to the C/C++ ones, but with the heap
range merged into the app range) and enable the sanity check.
Reviewed By: dvyukov
Differential Revision: https://reviews.llvm.org/D105629
The kernel supports a full 64-bit VMA, but we can use only 48 bits due
to the limitation imposed by SyncVar::GetId(). So define the address
ranges similar to the other architectures, except that the address
space "tail" needs to be made inaccessible in CheckAndProtect(). Since
it's for only one architecture, don't make an abstraction for this.
Reviewed By: dvyukov
Differential Revision: https://reviews.llvm.org/D105629
Go requires 47 bits VA for tsan.
Go will run race_detector testcases unless tsan warns about "unsupported VMA range"
Author: mzh (Meng Zhuo)
Reviewed-in: https://reviews.llvm.org/D98238
This reverts commit bde2e56071.
This patch produces a compile failure on linux amd64 environments, when
running:
ninja GotsanRuntimeCheck
I get various build errors:
../rtl/tsan_platform.h:608: error: use of undeclared identifier 'Mapping'
return MappingImpl<Mapping, Type>();
Here's a buildbot with the same failure during stage "check-tsan in gcc
build", there are other unrelated failures in there.
http://lab.llvm.org:8011/#/builders/37/builds/2831
When creating and destroying fibers in tsan a thread state is created and destroyed. Currently, a memory mapping is leaked with each fiber (in __tsan_destroy_fiber). This causes applications with many short running fibers to crash or hang because of linux vm.max_map_count.
The root of this is that ThreadState holds a pointer to ThreadSignalContext for handling signals. The initialization and destruction of it is tied to platform specific events in tsan_interceptors_posix and missed when destroying a fiber (specifically, SigCtx is used to lazily create the ThreadSignalContext in tsan_interceptors_posix). This patch cleans up the memory by makinh the ThreadState create and destroy the ThreadSignalContext.
The relevant code causing the leak with fibers is the fiber destruction:
void FiberDestroy(ThreadState *thr, uptr pc, ThreadState *fiber) {
FiberSwitchImpl(thr, fiber);
ThreadFinish(fiber);
FiberSwitchImpl(fiber, thr);
internal_free(fiber);
}
Author: Florian
Reviewed-in: https://reviews.llvm.org/D76073
Temporarily revert "tsan: fix leak of ThreadSignalContext for fibers"
because it breaks the LLDB bot on GreenDragon.
This reverts commit 93f7743851.
This reverts commit d8a0f76de7.
When creating and destroying fibers in tsan a thread state
is created and destroyed. Currently, a memory mapping is
leaked with each fiber (in __tsan_destroy_fiber).
This causes applications with many short running fibers
to crash or hang because of linux vm.max_map_count.
The root of this is that ThreadState holds a pointer to
ThreadSignalContext for handling signals. The initialization
and destruction of it is tied to platform specific events
in tsan_interceptors_posix and missed when destroying a fiber
(specifically, SigCtx is used to lazily create the
ThreadSignalContext in tsan_interceptors_posix). This patch
cleans up the memory by inverting the control from the
platform specific code calling the generic ThreadFinish to
ThreadFinish calling a platform specific clean-up routine
after finishing a thread.
The relevant code causing the leak with fibers is the fiber destruction:
void FiberDestroy(ThreadState *thr, uptr pc, ThreadState *fiber) {
FiberSwitchImpl(thr, fiber);
ThreadFinish(fiber);
FiberSwitchImpl(fiber, thr);
internal_free(fiber);
}
I would appreciate feedback if this way of fixing the leak is ok.
Also, I think it would be worthwhile to more closely look at the
lifecycle of ThreadState (i.e. it uses no constructor/destructor,
thus requiring manual callbacks for cleanup) and how OS-Threads/user
level fibers are differentiated in the codebase. I would be happy to
contribute more if someone could point me at the right place to
discuss this issue.
Reviewed-in: https://reviews.llvm.org/D76073
Author: Florian (Florian)
This #define is in the non-Go ppc64le build but not in the Go build.
Reviewed-in: https://reviews.llvm.org/D68046
Author: randall77 (Keith Randall)
llvm-svn: 374868
Factor out `ExtractLongJmpSp` helper function and move platform-specific
code to tsan_platform_{linux,mac}.cc.
Reviewed By: dvyukov
Differential Revision: https://reviews.llvm.org/D64050
llvm-svn: 364947
TSan needs to infer which calls to setjmp/longjmp are corresponding
pairs. My understanding is, that we can't simply use the jmp_buf
address, since this buffer is just a plain data structure storing the
environment (registers) with no additional semantics, i.e., it can be
copied around and is still expected to work. So we use the stack pointer
(SP) instead.
The setjmp interceptor stores some metadata, which is then consumed in
the corresponding call to longjmp. We use the SP as an "index" (stable
identifier) into the metadata table. So far so good.
However, when mangling is used, the setjmp interceptor observes the
UNmangled SP, but the longjmp interceptor only knows the mangled value
for SP. To still correlate corresponding pairs of calls, TSan currently
derives the mangled representation in setjmp and uses it as the stable
identifer, so that longjmp can do it's lookup.
Currently, this works since "mangling" simply means XOR with a secret
value. However, in the future we want to use operations that do not
allow us to easily go from unmangled -> mangled (pointer
authentication). Going from mangled -> unmangled should still be
possible (for pointer authentication it means zeroing a few bits).
This patch is part 1 of changing set/longjmp interceptors to use the
unmangled SP for metadata lookup. Instead of deriving the mangled SP in
setjmp, we will derive the unmangled SP in longjmp. Since this change
involves difficult-to-test code, it will be done in (at least) 2 parts:
This patch only replicates the existing behavior and checks that the
newly computed value for SP matches with what we have been doing so far.
This should help me to fix issues on architectures I cannot test
directly. I tested this patch on x86-64 (Linux/Darwin) and arm64
(Darwin).
This patch will also address an orthogonal issue: there is a lot of code
duplication in the assembly files, because the
`void __tsan_setjmp(uptr sp, uptr mangled_sp)` already demands the
mangled SP. This means that the code for computing the mangled SP is
duplicated at every call site (in assembly).
Reviewed By: dvyukov
Differential Revision: https://reviews.llvm.org/D60981
llvm-svn: 364662
to reflect the new license.
We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.
Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.
llvm-svn: 351636
This is a follow-up patch to r342541. After further investigations, only
48bits VMA size can be supported. As this is enforced in function
InitializePlatformEarly from lib/rt1/tsan_platform_linux.cc, the access
to the global variable vmaSize variable + switch can be removed. This
also addresses a comment from https://reviews.llvm.org/D52167.
vmaSize of 39 or 42bits are not compatible with a Go program memory
layout as the Go heap will not fit in the shadow memory area.
Patch by: Fangming Fang <Fangming.Fang@arm.com>
llvm-svn: 344329
Than McIntosh