Summary:
Hello,
Building a recent gcc on a powerpc-linux system advertsing:
Red Hat Enterprise Linux Server release 5.10 (Tikanga)
we stumbled on a compilation error on a file originating
from compiler-rt/lib/sanitizer-common.
sanitizer_platform_limits_linux.cc #includes asm/posix_types.h,
which, on our system, uses __kernel_fd_set and associated macros.
These aren't defined at the point of their use, and the compilation
fails with symptoms like:
In file included from ../../../../src/libsanitizer/sanitizer_common/sanitizer_platform_limits_linux.cc:29:0:
/usr/include/asm/posix_types.h:72:51: error: '__kernel_fd_set' has not been declared
static __inline__ void __FD_SET(unsigned long fd, __kernel_fd_set *fdsetp)
...
The attached patch is a suggestion to fix this, by including linux/posix_types.h
instead of asm/posix_types.h. linux/posix_types defines the necessary types and
macros, then #includes asm/posix_types.h.
We have been using it locally for gcc without problems for a couple of years
on powerpc, x86 and x86_64-linux platforms. It is still needed for gcc-6 on
our powerpc host and applies cleanly on the compiler-rt trunk.
Comments ?
Thanks much in advance for your feedback,
With Kind Regards,
Olivier
Reviewers: llvm-commits, kcc
Subscribers: kcc, kubabrecka
Differential Revision: http://reviews.llvm.org/D19799
llvm-svn: 268283
This happens on a 64-bit platform that uses SizeClassAllocator32 (e.g. ASan on AArch64). When querying a large invalid pointer, `__sanitizer_get_allocated_size(0xdeadbeefdeadbeef)`, an assertion will fail. This patch changes PointerIsMine to return false if the pointer is outside of [kSpaceBeg, kSpaceBeg + kSpaceSize).
Differential Revision: http://reviews.llvm.org/D15008
llvm-svn: 268243
There is a hard-to-reproduce crash happening on OS X that involves terminating the main thread (dispatch_main does that, see discussion at http://reviews.llvm.org/D18496) and later reusing the main thread's ThreadContext. This patch disables reuse of the main thread. I believe this problem exists only on OS X, because on other systems the main thread cannot be terminated without exiting the process.
Differential Revision: http://reviews.llvm.org/D19722
llvm-svn: 268238
In http://reviews.llvm.org/D19100, I introduced a bug: On OS X, existing programs rely on malloc_size() to detect whether a pointer comes from heap memory (malloc_size returns non-zero) or not. We have to distinguish between a zero-sized allocation (where we need to return 1 from malloc_size, due to other binary compatibility reasons, see http://reviews.llvm.org/D19100), and pointers that are not returned from malloc at all.
Differential Revision: http://reviews.llvm.org/D19653
llvm-svn: 268157
Summary:
This (partially) implements the check mentioned at
http://kristerw.blogspot.co.uk/2016/04/dangling-pointers-and-undefined-behavior.html
(via John Regehr)
Quoting:
"That the behavior is undefined follows from C11 6.2.4 "Storage
durations of objects"
The lifetime of an object is the portion of program execution during
which storage is guaranteed to be reserved for it. An object exists, has
a constant address, and retains its last-stored value throughout its
lifetime. If an object is referred to outside of its lifetime, the
behavior is undefined. The value of a pointer becomes indeterminate when
the object it points to (or just past) reaches the end of its lifetime.
and 7.22.3 "Memory management functions" that says that free ends the
lifetime of objects
The lifetime of an allocated object extends from the allocation until
the deallocation.
"
We can probably implement this for stack variables too, but I think this
is a good start to see if there's interest in this check.
We can also hide this behind a flag, too.
Reviewers: samsonov, kcc, rsmith, regehr
Subscribers: kubabrecka, llvm-commits
Differential Revision: http://reviews.llvm.org/D19691
llvm-svn: 268097
Since __builtin_setjmp has been fixed by rL267943, the test now works
on PowerPC. Enable it.
On the other hand, the SystemZ backend doesn't currently support
__builtin_setjmp. Disable it.
Differential Revision: http://reviews.llvm.org/D19657
llvm-svn: 267946
Recent TSan changes (r267678) which factor out parts of ThreadState into a Processor structure broke worker threads on OS X. This fixes it by properly calling ProcCreate for GCD worker threads and by replacing some CHECKs with RAW_CHECK in early process initialization. CHECK() in TSan calls the allocator, which requires a valid Processor.
llvm-svn: 267864
On linux, some architectures had an ABI transition from 64-bit long double
(ie. same as double) to 128-bit long double. On those, glibc symbols
involving long doubles come in two versions, and we need to pass the
correct one to dlvsym when intercepting them.
A few more functions we intercept are also versioned (all printf, scanf,
strtold variants), but there's no need to fix these, as the REAL() versions
are never called.
Differential Revision: http://reviews.llvm.org/D19555
llvm-svn: 267794
Summary:
The strchr and strrchr interceptors are sometimes invoked too early
for their REAL() counterparts to be initialized. We have seen this in
hooks invoked from tcmalloc on the dlsym() used in initializing
interceptors. A special check is added to use internal_ routines for
this situation.
Reviewers: vitalybuka, aizatsky, filcab
Subscribers: filcab, llvm-commits, eugenis, kcc, zhaoqin, aizatsky, kubabrecka
Differential Revision: http://reviews.llvm.org/D19607
llvm-svn: 267793
In short, CVE-2016-2143 will crash the machine if a process uses both >4TB
virtual addresses and fork(). ASan, TSan, and MSan will, by necessity, map
a sizable chunk of virtual address space, which is much larger than 4TB.
Even worse, sanitizers will always use fork() for llvm-symbolizer when a bug
is detected. Disable all three by aborting on process initialization if
the running kernel version is not known to contain a fix.
Unfortunately, there's no reliable way to detect the fix without crashing
the kernel. So, we rely on whitelisting - I've included a list of upstream
kernel versions that will work. In case someone uses a distribution kernel
or applied the fix themselves, an override switch is also included.
Differential Revision: http://reviews.llvm.org/D19576
llvm-svn: 267747
UnmapOrDie used to do MEM_DECOMMIT and so worked
on partial regions. But r263160 changed it to use
MEM_RELEASE and MEM_RELEASE can only work with
whole regions mapped by VirtualAlloc. This broke
windows as:
FATAL: ThreadSanitizer CHECK failed: gotsan.cc:8296 "((mbi.AllocationBase == addr && "Windows cannot unmap part of a previous mapping")) != (0)" (0x0, 0x0)
Restore the previous behavior.
llvm-svn: 267730
os_trace turns out to be a macro that creates static object.
Function-static objects use __cxa_atexit and __dso_handle
which are not present in Go runtime.
llvm-svn: 267720
Ifdef out global variables with destructors.
This requires runtime support that is not provided by Go runtime
(in particular _dso_handle symbol).
llvm-svn: 267709
Current interface assumes that Go calls ProcWire/ProcUnwire
to establish the association between thread and proc.
With the wisdom of hindsight, this interface does not work
very well. I had to sprinkle Go scheduler with wire/unwire
calls, and any mistake leads to hard to debug crashes.
This is not something one wants to maintian.
Fortunately, there is a simpler solution. We can ask Go
runtime as to what is the current Processor, and that
question is very easy to answer on Go side.
Switch to such interface.
llvm-svn: 267703
tsan_debugging.cc: In function ‘void* __tsan_get_current_report()’:
tsan_debugging.cc:61:18: warning: cast from type ‘const __tsan::ReportDesc*’
to type ‘void*’ casts away qualifiers [-Wcast-qual]
return (void *)rep;
llvm-svn: 267679
This is reincarnation of http://reviews.llvm.org/D17648 with the bug fix pointed out by Adhemerval (zatrazz).
Currently ThreadState holds both logical state (required for race-detection algorithm, user-visible)
and physical state (various caches, most notably malloc cache). Move physical state in a new
Process entity. Besides just being the right thing from abstraction point of view, this solves several
problems:
Cache everything on P level in Go. Currently we cache on a mix of goroutine and OS thread levels.
This unnecessary increases memory consumption.
Properly handle free operations in Go. Frees are issue by GC which don't have goroutine context.
As the result we could not do anything more than just clearing shadow. For example, we leaked
sync objects and heap block descriptors.
This will allow to get rid of libc malloc in Go (now we have Processor context for internal allocator cache).
This in turn will allow to get rid of dependency on libc entirely.
Potentially we can make Processor per-CPU in C++ mode instead of per-thread, which will
reduce resource consumption.
The distinction between Thread and Processor is currently used only by Go, C++ creates Processor per OS thread,
which is equivalent to the current scheme.
llvm-svn: 267678
This reverts commit r267477.
It broke our bots that enables the AArch64 backends, it seems that
this code is using a Darwin *X86 specific* field.
From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 267526
This fixes fails in test/msan/dlerror.cc - when real dlerror calls strcmp,
our strcmp interceptor now skips poison checking, since it's called in
interceptor context. Strictly speaking, only the dlerror change is
necessary to fix the fail, but let's also change the other two just in case.
Differential Revision: http://reviews.llvm.org/D19499
llvm-svn: 267486
The interception context is not used by esan, but the compiler complains
about it being uninitialized all the same. We set it to null to avoid the
warning.
llvm-svn: 267376
Summary:
Adds libc interceptors to the runtime library for the new
EfficiencySanitizer ("esan") family of tools. The interceptors cover
the memory operations in most common library calls and will be shared
among all esan tools.
Reviewers: aizatsky
Subscribers: zhaoqin, tberghammer, danalbert, srhines, llvm-commits, vitalybuka, eugenis, kcc
Differential Revision: http://reviews.llvm.org/D19411
llvm-svn: 267293
This reverts commit r267094, because it broke a lot of MSAN tests in AArch64.
Being NFC and all, this needs some deeper investigation before it goes in again.
llvm-svn: 267136
Summary: When using 32-bit python with 64-bit asan the pc array in sancov.py cannot fit in 64-bit pc's because the type-code 'L' for
arrays in python corresponds to the C type long which is only of 4 bytes. Because of this some of the coverage tool tests fail on
mips. To fix these test possible solutions are to use 64-bit python or use struct.unpack with the 'Q' type-code. We have used
struct.unpack with 'Q' type code since it is not appropriate to have a 64-bit python on all hosts.
Reviewed by kcc, aizatsky
Differential: http://reviews.llvm.org/D18817
llvm-svn: 267126
Summary:
Adds the initial version of a runtime library for the new
EfficiencySanitizer ("esan") family of tools. The library includes:
+ Slowpath code via callouts from the compiler instrumentation for
each memory access.
+ Registration of atexit() to call finalization code.
+ Runtime option flags controlled by the environment variable
ESAN_OPTIONS. The common sanitizer flags are supported such as
verbosity and log_path.
+ An initial simple test.
Still TODO: common code for libc interceptors and shadow memory mapping,
and tool-specific code for shadow state updating.
Reviewers: eugenis, vitalybuka, aizatsky, filcab
Subscribers: filcab, vkalintiris, kubabrecka, llvm-commits, zhaoqin, kcc
Differential Revision: http://reviews.llvm.org/D19168
llvm-svn: 267060
The field "pid" in ReportThread is used to store the OS-provided thread ID (pthread_self or gettid). The name "pid" suggests it's a process ID, which it isn't. Let's rename it.
Differential Revision: http://reviews.llvm.org/D19365
llvm-svn: 266994
Let's use pthread_threadid_np which returns a more reasonable ID than pthread_self (which is actually a stack pointer). The numbers from pthread_threadid_np are already used in other tools, e.g. in LLDB, and often appear in logs, so it's much more useful than pthread_self.
Differential Revision: http://reviews.llvm.org/D18951
llvm-svn: 266991
Kostya Serebryany