Attempts to fix a bot failure from
634da7a1c6 on an Android bot:
https://lab.llvm.org/buildbot#builders/77/builds/14339
It appears that the chmod is not making the directory unwritable as
expected on this system for some reason. Adopt an approach used in
compiler-rt/test/fuzzer/fuzzer-dirs.test for systems with
non-functioning chmod by including illegal characters in directory.
Add a DirExists mechanism, modeled after FileExists. Use it to guard
creation of the report path directory.
This should avoid failures running the sanitizer in a sandbox where the
file creation attempt causes hard failures, even for an existing
directory. Problem reported on D109794 for ChromeOS in sandbox
(https://issuetracker.google.com/209296420).
Differential Revision: https://reviews.llvm.org/D119495
All platforms return the main executable as the first dl_phdr_info.
FreeBSD, NetBSD, Solaris, and Linux-musl place the executable name
in the dlpi_name field of this entry. It appears that only Linux-glibc
uses the empty string.
To make this work generically on all platforms, unconditionally skip the first
object (like is currently done for FreeBSD and NetBSD). This fixes first DSO
detection on Linux-musl with clang -shared-libsan/-shared-libasan and GCC's
default. It also would likely fix detection on Solaris/Illumos if it were to
gain PIE support (since dlpi_addr would not be NULL).
Additionally, only skip the Linux VDSO on linux.
Finally, use the empty string as the "seen first dl_phdr_info"
marker rather than (char *)-1. If there was no other object, we
would try to dereference it for a string comparison.
Reviewed By: MaskRay, vitalybuka
Differential Revision: https://reviews.llvm.org/D119515
Follow-up to 458ead66dc, which replaced the bespoke CMakeLists.txt
file for building a custom instrumented libc++ with an invocation of the
runtimes build.
In the the bespoke CMakeLists.txt, the LIBCXX_CXX_ABI setting was forced
to libcxxabi, but this was not done for the CMake invocation for the
runtimes build. This would cause CMake configuration issues on platforms
where the default LIBCXX_CXX_ABI setting is not libcxxabi, such as
FreeBSD.
Add `-DLIBCXX_CXX_ABI=libcxxabi` to that invocation, to make sure the
custom instrumented libc++ always uses the expected ABI.
Reviewed By: phosek
Differential Revision: https://reviews.llvm.org/D119554
An infinite loop without any effects is illegal C++ and can be optimized
away by the compiler.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D119575
This is a follow up to 4f3f4d6722
("sanitizer_common: fix __sanitizer_get_module_and_offset_for_pc signature mismatch")
which fixes a similar problem for msan build.
I am getting the following error compiling a unit test for code that
uses sanitizer_common headers and googletest transitively includes
sanitizer interface headers:
In file included from third_party/gwp_sanitizers/singlestep_test.cpp:3:
In file included from sanitizer_common/sanitizer_common.h:19:
sanitizer_interface_internal.h:41:5: error: typedef redefinition with different types
('struct __sanitizer_sandbox_arguments' vs 'struct __sanitizer_sandbox_arguments')
} __sanitizer_sandbox_arguments;
common_interface_defs.h:39:3: note: previous definition is here
} __sanitizer_sandbox_arguments;
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D119546
We run bots on a shared machine and under high load
this test sometimes segfaults.
https://lab.llvm.org/buildbot/#/builders/185/builds/1368
==1952234==XRay FDR init successful.
==1952234==XRay FDR: Not flushing to file, 'no_file_flush=true'.
<...>fdr-reinit.cpp.script: line 4: 1952234 Segmentation fault
XRAY_OPTIONS="verbosity=1" <...>/fdr-reinit.cpp.tmp
Looking at the printed output I think it's happening at:
// Finally, we should signal the sibling thread to stop.
keep_going.clear(std::memory_order_release);
Disabling the test while I try to reproduce.
We are moving away from building the runtimes with LLVM_ENABLE_PROJECTS,
however the documentation was largely outdated. This commit updates all
the documentation I could find to use LLVM_ENABLE_RUNTIMES instead of
LLVM_ENABLE_PROJECTS for building runtimes.
Note that in the near future, libcxx, libcxxabi and libunwind will stop
supporting being built with LLVM_ENABLE_PROJECTS altogether. I don't know
what the plans are for other runtimes like libc, openmp and compiler-rt,
so I didn't make any changes to the documentation that would imply
something for those projects.
Once this lands, I will also cherry-pick this on the release/14.x branch
to make sure that LLVM's documentation is up-to-date and reflects what
we intend to support in the future.
Differential Revision: https://reviews.llvm.org/D119351
Summary:
This is neccessary to support solaris/sparc9 where some userspace
addresses have all top bits set, as well as, potentially, kernel memory
on aarch64.
This change does not update the compiler side (HWASan IR pass) which
needs to be done separately for the affected targets.
Reviewers: ro, vitalybuka
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D91827
Fix passing the port and IP address with the wrong endianness
in get_sock_peer_name() that causes the connect() to fail inside
without an outgoing network interface (it's trying to connect
to 1.0.0.127 instead of 127.0.0.1).
Differential Revision: https://reviews.llvm.org/D119461
We don't need precise control over the low-level behavior of these testcases so
C should be preferred for readability.
The new testcases test (1) the base dlopen case (running initializers and
deinitializers), and (2) the serial case of dlopen; dlclose; dlopen; dlclose,
where we expect the initializers and deinitializers to be run twice.
D116208 may cause a macro clash on older versions of linux, where
fs.h defines a READ macro. This is resolved by switching to a more
typical casing style for non-macro symbols.
Reapplying with changes to the symbol names in various platform
specific code, which I missed previously.
Differential Revision: https://reviews.llvm.org/D118783
D116208 may cause a macro clash on older versions of linux, where
fs.h defines a READ macro. This is resolved by switching to a more
typical casing style for non-macro symbols.
Differential Revision: https://reviews.llvm.org/D118783
As reported in Issue #41838, `clang` doesn't correctly implement `long
double` on 32-bit Solaris/SPARC: the psABI requires this to be an 128-bit
type. Four sanitizer tests currently `FAIL` for this reason.
While there is a WIP patch to fix `clang` (D89130
<https://reviews.llvm.org/D89130>), it isn't complete yet and I've hit so
many brick walls while trying to finish it that I'm unsure if I ever will.
This patch therefore `XFAIL`s those tests in the meantime.
Tested on `sparcv9-sun-solaris2.11`.
Differential Revision: https://reviews.llvm.org/D119016
Enabling `sanitizer_common` tests on Solaris (D91606
<https://reviews.llvm.org/D91606>) and SPARC (D91608
<https://reviews.llvm.org/D91608>) uncovered a sparcv9 failure
SanitizerCommon-Unit :: ./Sanitizer-sparcv9-Test/CompactRingBuffer.int64
like this:
[ RUN ] CompactRingBuffer.int64
==24576==ERROR: SanitizerTool failed to deallocate 0x2000 (8192) bytes at address 0xffffffff7f59b000
==24576==Sanitizer CHECK failed: /vol/llvm/src/llvm-project/local/compiler-rt/lib/sanitizer_common/sanitizer_posix.cpp:61 (("unable to unmap" && 0)) != (0) (0, 0)
The problem is that the original allocation via
`MmapAlignedOrDieOnFatalError` is for 4 kB, but the Solaris/sparcv9
pagesize is 8 kB. So the initial allocation is for 12 kB, rounded to a
multiple of the pagesize. Afterwards, the unneeded rest is unmapped again,
but this fails since the address is not pagesize-aligned.
This patch avoids this by aligning the end of the mapping to the pagesize.
With D91827 <https://reviews.llvm.org/D91827> added, the test `PASS`es on
`sparcv9-sun-solaris2.11`.
Differential Revision: https://reviews.llvm.org/D91615
As described in Issue #53523, the
`DenseMapCustomTest.DefaultMinReservedSizeTest` test FAILs on Solaris/SPARC
(both 32 and 64-bit):
/vol/llvm/src/llvm-project/local/compiler-rt/lib/sanitizer_common/tests/sanitizer_dense_map_test.cpp:399:
Failure
Expected: (MemorySize) != (Map.getMemorySize()), actual: 8192 vs 8192
This happens because SPARC, unlike many other CPUs, uses an 8 kB pagesize.
Fixed by incorporating the pagesize into the calculations of
`ExpectedInitialBucketCount` and derived values.
Tested on `sparcv9-sun-solaris2.11`, `amd64-pc-solaris2.11`, and
`x86_64-pc-linux-gnu`.
Differential Revision: https://reviews.llvm.org/D118771
The SANITIZER_GO code path reports an undefined symbol error for dlsym.
```
FAILED: projects/compiler-rt/lib/tsan/rtl/CMakeFiles/GotsanRuntimeCheck /tmp/RelA/projects/compiler-rt/lib/tsan/rtl/CMakeFiles/GotsanRuntimeCheck
```
This symbol has been exported (as an internal GLIBC_PRIVATE symbol) from libc.so.6 starting with glibc 2.34. glibc uses it internally for its libthread_db implementation to enable thread debugging on GDB, so it is unlikely to go away for now.
Fixes#52989.
Reviewed By: #sanitizers, MaskRay, vitalybuka
Differential Revision: https://reviews.llvm.org/D119007
This patch updates the MachO platform (both the ORC MachOPlatform class and the
ORC-Runtime macho_platform.* files) to use allocation actions, rather than EPC
calls, to transfer the initializer information scraped from each linked object.
Interactions between the ORC and ORC-Runtime sides of the platform are
substantially redesigned to accomodate the change.
The high-level changes in this patch are:
1. The MachOPlatform::setupJITDylib method now calls into the runtime to set up
a dylib name <-> header mapping, and a dylib state object (JITDylibState).
2. The MachOPlatformPlugin builds an allocation action that calls the
__orc_rt_macho_register_object_platform_sections and
__orc_rt_macho_deregister_object_platform_sections functions in the runtime
to register the address ranges for all "interesting" sections in the object
being allocated (TLS data sections, initializers, language runtime metadata
sections, etc.).
3. The MachOPlatform::rt_getInitializers method (the entry point in the
controller for requests from the runtime for initializer information) is
replaced by MachOPlatform::rt_pushInitializers. The former returned a data
structure containing the "interesting" section address ranges, but these are
now handled by __orc_rt_macho_register_object_platform_sections. The new
rt_pushInitializers method first issues a lookup to trigger materialization
of the "interesting" sections, then returns the dylib dependence tree rooted
at the requested dylib for dlopen to consume. (The dylib dependence tree is
returned by rt_pushInitializers, rather than being handled by some dedicated
call, because rt_pushInitializers can alter the dependence tree).
The advantage of these changes (beyond the performance advantages of using
allocation actions) is that it moves more information about the materialized
portions of the JITDylib into the executor. This tends to make the runtime
easier to reason about, e.g. the implementation of dlopen in the runtime is now
recursive, rather than relying on recursive calls in the controller to build a
linear data structure for consumption by the runtime. This change can also make
some operations more efficient, e.g. JITDylibs can be dlclosed and then
re-dlopened without having to pull all initializers over from the controller
again.
In addition to the high-level changes, there are some low-level changes to ORC
and the runtime:
* In ORC, at ExecutionSession teardown time JITDylibs are now destroyed in
reverse creation order. This is on the assumption that the ORC runtime will be
loaded into an earlier dylib that will be used by later JITDylibs. This is a
short-term solution to crashes that arose during testing when the runtime was
torn down before its users. Longer term we will likely destroy dylibs in
dependence order.
* toSPSSerializable(Expected<T> E) is updated to explicitly initialize the T
value, allowing it to be used by Ts that have explicit constructors.
* The ORC runtime now (1) attempts to track ref-counts, and (2) distinguishes
not-yet-processed "interesting" sections from previously processed ones. (1)
is necessary for standard dlopen/dlclose emulation. (2) is intended as a step
towards better REPL support -- it should enable future runtime calls that
run only newly registered initializers ("dlopen_more", "dlopen_additions",
...?).
Adds construction from std::string, an ostream &operator<< and std::hash
specialization. Also adds unit tests for each of these operations, as well as
tests for copy construction and assignment.
These new operations will be used in upcoming macho_platform patches.
Aligned new does not require size to be a multiple of alignment, so
memalign is the correct choice instead of aligned_alloc.
Fixes false reports for unaligned sizes.
Reviewed By: eugenis
Differential Revision: https://reviews.llvm.org/D119161
Similar to 60cc1d3218 for NetBSD, add aliases and interceptors for the
following pthread related functions:
- pthread_cond_init(3)
- pthread_cond_destroy(3)
- pthread_cond_signal(3)
- pthread_cond_broadcast(3)
- pthread_cond_wait(3)
- pthread_mutex_init(3)
- pthread_mutex_destroy(3)
- pthread_mutex_lock(3)
- pthread_mutex_trylock(3)
- pthread_mutex_unlock(3)
- pthread_rwlock_init(3)
- pthread_rwlock_destroy(3)
- pthread_rwlock_rdlock(3)
- pthread_rwlock_tryrdlock(3)
- pthread_rwlock_wrlock(3)
- pthread_rwlock_trywrlock(3)
- pthread_rwlock_unlock(3)
- pthread_once(3)
- pthread_sigmask(3)
In FreeBSD's libc, a number of internal aliases of the pthread functions
are invoked, typically with an additional prefixed underscore, e.g.
_pthread_cond_init() and so on.
ThreadSanitizer needs to intercept these aliases too, otherwise some
false positive reports about data races might be produced.
Reviewed By: dvyukov
Differential Revision: https://reviews.llvm.org/D119034
Fixes segfaults on x86_64 caused by instrumented code running before
shadow is set up.
Reviewed By: pcc
Differential Revision: https://reviews.llvm.org/D118171
This fixes the following error:
sanitizer_interface_internal.h:77:7: error: conflicting types for
'__sanitizer_get_module_and_offset_for_pc'
int __sanitizer_get_module_and_offset_for_pc(
common_interface_defs.h:349:5: note: previous declaration is here
int __sanitizer_get_module_and_offset_for_pc(void *pc, char *module_path,
I am getting it on a code that uses sanitizer_common (includes internal headers),
but also transitively gets includes of the public headers in tests
via an internal version of gtest.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D118910
We (Linaro) still have the occasional failure here due
to high load on a shared buildbot machine.
We are looking into general soloutions but perhaps this
will help in the meantime.
Instead of calling asan_report.* directly from assembly code they have been replaced with corresponding asan_report.*_asm function, which call asan_report.*. All asan_report.* are now undefined weak symbols, which allows DSOs to link when z defs is used.
Reviewed By: MaskRay, morehouse
Differential Revision: https://reviews.llvm.org/D118813
I was looking around and noticed that builtins for iossim, tvossim
and watchossim was missing arm64 builds, while apple's clang
toolchain ship with these. After a bit of searching around it just
seems like these are not listed correctly in CMake to be enabled.
I enabled just arm64 since I saw that Apple clang didn't include
arm64e.
Reviewed By: t.p.northover
Differential Revision: https://reviews.llvm.org/D118759
Unfortunately, the `sanitizer_common` tests are disabled on many targets
that are supported by `sanitizer_common`, making it easy to miss issues
with that support. This patch enables SPARC testing.
Beside the enabling proper, the patch fixes (together with D91607
<https://reviews.llvm.org/D91607>) the failures of the `symbolize_pc.cpp`,
`symbolize_pc_demangle.cpp`, and `symbolize_pc_inline.cpp` tests. They
lack calls to `__builtin_extract_return_addr`. When those are added, they
`PASS` when compiled with `gcc`. `clang` incorrectly doesn't implement a
non-default `__builtin_extract_return_addr` on several targets, SPARC
included.
Because `__builtin_extract_return_addr(__builtin_return_addr(0))` is quite
a mouthful and I'm uncertain if the code needs to compile with msvc which
appparently has it's own `_ReturnAddress`, I've introduced
`__sanitizer_return_addr` to hide the difference and complexity. Because
on 32-bit SPARC `__builtin_extract_return_addr` differs when the calling
function returns a struct, I've added a testcase for that.
There are a couple more tests failing on SPARC that I will deal with
separately.
Tested on `sparcv9-sun-solaris2.11`, `amd64-pc-solaris2.11`, and
`x86_64-pc-linux-gnu`.
Differential Revision: https://reviews.llvm.org/D91608
setjmp can return twice, but PostDominatorTree is unaware of this. as
such, it overestimates postdominance, leaving some cases (see attached
compiler-rt) where memory does not get untagged on return. this causes
false positives later in the program execution.
this is a crude workaround to unblock use-after-scope for now, in the
longer term PostDominatorTree should bemade aware of returns_twice
function, as this may cause problems elsewhere.
Reviewed By: eugenis
Differential Revision: https://reviews.llvm.org/D118647
These tests appear to be causing timeouts on our silent
Thumbv7 bot: https://lab.llvm.org/staging/#/builders/162/builds/260
It is possible they would complete given enough time. value-profile-switch
seems to take a long time even on a powerful Armv8 machine.
The definition of the MemInfoBlock is shared between the memprof
compiler-rt runtime and llvm/lib/ProfileData/. This change removes the
memprof_meminfoblock header and moves the struct to the shared include
file. To enable this sharing, the Print method is moved to the
memprof_allocator (the only place it is used) and the remaining uses are
updated to refer to the MemInfoBlock defined in the MemProfData.inc
file.
Also a couple of other minor changes which improve usability of the
types in MemProfData.inc.
* Update the PACKED macro to handle commas.
* Add constructors and equality operators.
* Don't initialize the buildid field.
Differential Revision: https://reviews.llvm.org/D116780
`__builtin_c[tl]z` accepts `unsigned int` argument that is not always the
same as uint32_t. For example, `unsigned int` is uint16_t on MSP430.
Reviewed By: aykevl
Differential Revision: https://reviews.llvm.org/D86547
Use s[iu]_int instead of `(unsigned) int` and d[ui]_int instead of
`(unsigned) long long` for LibCall arguments.
Note: the `*vfp` LibCall versions were NOT touched.
Reviewed By: aykevl
Differential Revision: https://reviews.llvm.org/D86546
As discussed in D118021 <https://reviews.llvm.org/D118021>, `clang -m32` on
Solaris/sparcv9 currently incorrectly doesn't inline atomics on 8-byte
operands, unlike `gcc`. With the workaround in that patch in place, we're
left with may undefined references to `__sync_val_compare_and_swap_8`,
which isn't provided by `libatomic`. This reference is due to the use of
`__sync_val_compare_and_swap` in `sanitizer_atomic_clang.h`'s
`atomic_compare_exchange_strong`. As is already done in
`scudo/standalone/atomic_helpers.h`, using `__atomic_compare_exchange`
instead avoids this problem.
Tested on `sparcv9-sun-solaris2.11`, `amd64-pc-solaris2.11`, and
`x86_64-pc-linux-gnu`.
Differential Revision: https://reviews.llvm.org/D118024
We previously had a few varied definitions of this floating around.
I had tried to make the one installed with LLVM handle all the cases, and then made the others use it, but this ran into issues with `HandleOutOfTreeLLVM` not working for compiler-rt, and also `CMAKE_EXE_LINKER_FLAGS` not working right without `CMP0056` set to the new behavior.
My compromise solution is this:
- No not completely deduplicate: the runtime libs will instead use a version that still exists as part of the internal and not installed common shared CMake utilities. This avoids `HandleOutOfTreeLLVM` or a workaround for compiler-rt.
- Continue to use `CMAKE_REQUIRED_FLAGS`, which effects compilation and linking. Maybe this is unnecessary, but it's safer to leave that as a future change. Also means we can avoid `CMP0056` for now, to try out later, which is good incrementality too.
- Call it `llvm_check_compiler_linker_flag` since it, in fact is about both per its implementation (before and after this patch), so there is no name collision.
In the future, we might still enable CMP0056 and make compiler-rt work with HandleOutOfTreeLLVM, which case we delete `llvm_check_compiler_flag` and go back to the old way (as these are, in fact, linking related flags), but that I leave for someone else as future work.
The original issue was reported to me in https://reviews.llvm.org/D116521#3248117 as
D116521 made clang and LLVM use the common cmake utils.
Reviewed By: sebastian-ne, phosek, #libunwind, #libc, #libc_abi, ldionne
Differential Revision: https://reviews.llvm.org/D117537
In glibc before 2.33, include/sys/stat.h defines fstat/fstat64 to
`__fxstat/__fxstat64` and provides `__fxstat/__fxstat64` in libc_nonshared.a.
The symbols are glibc specific and not needed on other systems.
Reviewed By: vitalybuka, #sanitizers
Differential Revision: https://reviews.llvm.org/D118423
Fixes a false positive that occurs when a user-implemented memmove is
instrumented by HWASan.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D118180
Use the llvm flag `-pgo-function-entry-coverage` to create single byte "counters" to track functions coverage. This mode has significantly less size overhead in both code and data because
* We mark a function as "covered" with a store instead of an increment which generally requires fewer assembly instructions
* We use a single byte per function rather than 8 bytes per block
The trade off of course is that this mode only tells you if a function has been covered. This is useful, for example, to detect dead code.
When combined with debug info correlation [0] we are able to create an instrumented Clang binary that is only 150M (the vanilla Clang binary is 143M). That is an overhead of 7M (4.9%) compared to the default instrumentation (without value profiling) which has an overhead of 31M (21.7%).
[0] https://groups.google.com/g/llvm-dev/c/r03Z6JoN7d4
Reviewed By: kyulee
Differential Revision: https://reviews.llvm.org/D116180
This is a follow up to D118101, that added bexpfull to the test on instrprof-get-filename-merge-mode.c AIX, in order to get the
necessary symbols exported. But unfortunately the extra dependent symbols this exports actually cause segfaults, which is why
this isn't really recommended in the first place, so just use an actual export list.
Differential Revision: https://reviews.llvm.org/D118310
Use the `llvm-profdata show` command to verify debug info for profile correlation using the `--debug-info` option.
Reviewed By: kyulee
Differential Revision: https://reviews.llvm.org/D118181
The "asan/asan_mapping.h" header relies on sanitizer_platform.h
macros, but doesn't directly include the header. All the existing
uses until recently happened to be in places where some other header
had indirectly included sanitizer_platform.h first. The addition of
asan_rtl_x86_64.S was the first place to use "asan/asan_mapping.h"
alone. It so happens that its uses of the macros make having no
macros defined equivalent to SANITIZER_LINUX, so this did not affect
Linux builds. But the assembly constants in asan_rtl_x86_64.S were
wrong for Fuchsia when SANITIZER_FUCHSIA was not properly defined.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D118296
When using the in-tree libc++, we should be using the full path to
ensure that we're using the right library and not accidentally pick up
the system library.
Differential Revision: https://reviews.llvm.org/D118200
We have been relying on the logic for C++ ABI library for sanitizer
tests but that's incorrect since most tests require a full C++ library
and not just C++ ABI. This change tries to address this by using the
dependency on libc++ if available.
Differential Revision: https://reviews.llvm.org/D118163
There is no need to compress the names string when correlating with
debug info since InstrProfReader will immediately uncompress it anyway.
This also removes the dependency on zlib in this case.
Reviewed By: kyulee
Differential Revision: https://reviews.llvm.org/D118176
The AIX linker doesn't export any symbols by default, so an export list is usually used. Since clang doesn't have the tools to auto-generate an export list yet, just pass the linker an extra opt to tell it to export everything. This is generally not recommended for real shared libs, but is fine for the purpose of this test.
Differential Revision: https://reviews.llvm.org/D118101
`_vfork` moved from libsystem_kernel.dylib to libsystem_c.dylib as part
of the below changes. The iOS simulator does not actually have
libsystem_kernel.dylib of its own, it only has the host Mac's. The
umbrella-nature of Libsystem makes this movement transparent to
everyone; except the simulator! So when we "back deploy", i.e., use the
current version of TSan with an older simulator runtime then this symbol
is now missing, when we run on the latest OS (but an older simulator
runtime).
Note we use `SANITIZER_IOS` because usage of vfork is forbidden on iOS
and the API is completely unavailable on watchOS and tvOS, even if this
problem is specific to the iOS simulator.
Caused by:
rdar://74818691 (Shim vfork() to fork syscall on iOS)
rdar://76762076 (Shim vfork() to fork syscall on macOS)
Radar-Id: rdar://8634734
This reverts commit 8c9f62ea90, which is causing build failures on
the bots because it inadvertently changes the output directory of the compiler-rt libs when
built as a runtime.
Differential Revision: https://reviews.llvm.org/D117815
We previously had a few varied definitions of this floating around. I made the one installed with LLVM handle all the cases, and then made the others use it.
This issue was reported to me in https://reviews.llvm.org/D116521#3248117 as
D116521 made clang and llvm use the common cmake utils.
Reviewed By: sebastian-ne, phosek, #libunwind, #libc, #libc_abi, ldionne
Differential Revision: https://reviews.llvm.org/D117537
This gives us the option of using CMake modules from LLVM, and other
things. We will use that to deduplicate code later.
Reviewed By: phosek
Differential Revision: https://reviews.llvm.org/D117815
https://reviews.llvm.org/D116179 introduced some changes to
`InstrProfData.inc` which broke some downstream builds. This commit
reverts those changes since they only changes two field names.
Reviewed By: phosek
Differential Revision: https://reviews.llvm.org/D117631
The kern.elf64.aslr.pie_enable and kern.elf32.aslr.pie_enable sysctls
control the default setting for PIE binary address randomization, but
it is possible to enable or disable ASLR on a per-process basis. So,
use procctl(2) to query whether ASLR is enabled.
(Note that with ASLR enabled but sysctl kern.elf64.aslr.pie_enable=0
a PIE binary will in effect have randomization disabled, and would be
functional with msan. This is not intended as as a user-facing control
though; proccontrol(1) should be used to disable aslr for the process.)
Reviewed By: devnexen
Differential Revision: https://reviews.llvm.org/D117521
1b1c8d changed `enable-noundef-analysis` flag to
`disable-noundef-analysis`. noundef_analysis.cpp was using old
`enable-noundef-analysis` flag and this patch fixes it.
This is the original patch in my GNUInstallDirs series, now last to merge as the final piece!
It arose as a new draft of D28234. I initially did the unorthodox thing of pushing to that when I wasn't the original author, but since I ended up
- Using `GNUInstallDirs`, rather than mimicking it, as the original author was hesitant to do but others requested.
- Converting all the packages, not just LLVM, effecting many more projects than LLVM itself.
I figured it was time to make a new revision.
I have used this patch series (and many back-ports) as the basis of https://github.com/NixOS/nixpkgs/pull/111487 for my distro (NixOS), which was merged last spring (2021). It looked like people were generally on board in D28234, but I make note of this here in case extra motivation is useful.
---
As pointed out in the original issue, a central tension is that LLVM already has some partial support for these sorts of things. Variables like `COMPILER_RT_INSTALL_PATH` have already been dealt with. Variables like `LLVM_LIBDIR_SUFFIX` however, will require further work, so that we may use `CMAKE_INSTALL_LIBDIR`.
These remaining items will be addressed in further patches. What is here is now rote and so we should get it out of the way before dealing more intricately with the remainder.
Reviewed By: #libunwind, #libc, #libc_abi, compnerd
Differential Revision: https://reviews.llvm.org/D99484
089acf2522 updated WrapperFunctionCall to carry arbitrary argument payloads
(rather than plain address ranges). This commit implements the corresponding
update for the ORC runtime.
Fixes:
[2587/4073] Building CXX object projects\compiler-rt\lib\sanitizer_common\CMakeFiles\RTSanitizerCommon.x86_64.dir\sanitizer_stoptheworld_win.cpp.obj
D:\git\llvm-project\compiler-rt\lib\sanitizer_common\sanitizer_stoptheworld_win.cpp(125,33): warning: comparison of integers of different signs: 'DWORD' (aka 'unsigned long') and 'int' [-Wsign-compare]
if (SuspendThread(thread) == -1) {
~~~~~~~~~~~~~~~~~~~~~ ^ ~~
1 warning generated.
This is the original patch in my GNUInstallDirs series, now last to merge as the final piece!
It arose as a new draft of D28234. I initially did the unorthodox thing of pushing to that when I wasn't the original author, but since I ended up
- Using `GNUInstallDirs`, rather than mimicking it, as the original author was hesitant to do but others requested.
- Converting all the packages, not just LLVM, effecting many more projects than LLVM itself.
I figured it was time to make a new revision.
I have used this patch series (and many back-ports) as the basis of https://github.com/NixOS/nixpkgs/pull/111487 for my distro (NixOS), which was merged last spring (2021). It looked like people were generally on board in D28234, but I make note of this here in case extra motivation is useful.
---
As pointed out in the original issue, a central tension is that LLVM already has some partial support for these sorts of things. Variables like `COMPILER_RT_INSTALL_PATH` have already been dealt with. Variables like `LLVM_LIBDIR_SUFFIX` however, will require further work, so that we may use `CMAKE_INSTALL_LIBDIR`.
These remaining items will be addressed in further patches. What is here is now rote and so we should get it out of the way before dealing more intricately with the remainder.
Reviewed By: #libunwind, #libc, #libc_abi, compnerd
Differential Revision: https://reviews.llvm.org/D99484
In C++20 compound assignment to volatile (here `LocalData[I]++`) is
deprecated, so `mutex_test.cpp` fails to compile.
Simply changing it to `LocalData[I] = LocalData[I] + 1` fixes it.
Differential Revision: https://reviews.llvm.org/D117359
Existing code tended to assume that counters had type `uint64_t` and
computed size from the number of counters. Fix this code to directly
compute the counters size in number of bytes where possible. When the
number of counters is needed, use `__llvm_profile_counter_entry_size()`
or `getCounterTypeSize()`. In a later diff these functions will depend
on the profile mode.
Change the meaning of `DataSize` and `CountersSize` to make them more clear.
* `DataSize` (`CountersSize`) - the size of the data (counter) section in bytes.
* `NumData` (`NumCounters`) - the number of data (counter) entries.
Reviewed By: kyulee
Differential Revision: https://reviews.llvm.org/D116179
If `pthread_create` is not available on a platform, we won't be able to check if interceptors work. Use `strcmp` instead.
Reviewed By: yln
Differential Revision: https://reviews.llvm.org/D116989
The test checks that an array of Obj-C literal integers (e.g. `@1`) gets a UBSan
warning when cast to an NSString, however the actual concrete Obj-C class of
literal integers doesn't always need to be __NSCFNumber. Let's relax the test
expectations to allow NSConstantIntegerNumber. Which exact subclass of NSNumber
is used is not actually important for the test (the test is just checking that
the invalid cast warning is thrown).
These tests are now green:
```
Trace.MultiPart
Trace.RestoreAccess
Trace.RestoreMutexLock
TraceAlloc.FinishedThreadReuse
TraceAlloc.FinishedThreadReuse2
TraceAlloc.SingleThread
```
rdar://82107856
D114294 <https://reviews.llvm.org/D114294> broke the Solaris buildbots:
/opt/llvm-buildbot/home/solaris11-amd64/clang-solaris11-amd64/llvm/compiler-rt/lib/sanitizer_common/sanitizer_linux_libcdep.cpp:613:29: error: use of undeclared identifier 'NT_GNU_BUILD_ID'
if (nhdr->n_type == NT_GNU_BUILD_ID && nhdr->n_namesz == kGnuNamesz) {
^
Like D107556 <https://reviews.llvm.org/D107556>, it forgot that
`NT_GNU_BUILD_ID` is an unportable GNU extension.
Fixed by making the code conditional on the definition of the macro.
Tested on `amd64-pc-solaris2.11` and `sparcv9-sun-solaris2.11`.
Differential Revision: https://reviews.llvm.org/D117051
indirect branch tracking(IBT) feature aiming to ensure the target address
of an indirect jump/call is not tampered.
When IBT is enabled, each function or target of any indirect jump/call will start
with an 'endbr32/64' instruction otherwise the program will crash during execution.
To build an application with CET enabled. we need to ensure:
1. build the source code with "-fcf-protection=full"
2. all the libraries linked with .o files must be CET enabled too
This patch aims to enable CET for compiler-rt builtins library, we add an option
"COMPILER_RT_ENABLE_CET" whose default value is OFF to enable CET for compiler-rt
in building time and when this option is "ON", "-fcf-protection=full" is added to
BUILTINS_CFLAG and the "endbr32/64" will be placed in the beginning of each assembly
function. We also enabled CET for crtbegin, crtend object files in this patch.
Reviewed by: MaskRay, compnerd, manojgupta, efriedma
Differential Revision: https://reviews.llvm.org/D109811
Signed-off-by: jinge90 <ge.jin@intel.com>
Inspired by LLVM_DEBUG, but using environment variables rather than command line
options.
Code can use ORC_RT_DEBUG(...) (if ORC_RT_DEBUG_TYPE is set), or
ORC_RT_DEBUG_WITH_TYPE(<type>, ...) (if ORC_RT_DEBUG_TYPE is not set. E.g. in
headers).
Debug logging is enabled in the executor by setting the ORC_RT_DEBUG environment
variable. Debug logging can be restricted by type by setting the
ORC_RT_DEBUG_TYPES environment variable to a comma separated list of types,
e.g. ORC_RT_DEBUG_TYPES=macho_platform,sps.
Differential Revision: https://reviews.llvm.org/D116139
When building libcxx, libcxxabi, and libunwind the build environment may
specify any number of sanitizers. For some build feature tests these
sanitizers must be disabled to prevent spurious linking errors. With
-fsanitize= this is straight forward with -fno-sanitize=all. With
-fsanitize-coverage= there is no -fno-sanitize-coverage=all, but there
is the equivalent undocumented but tested -fsanitize-coverage=0.
The current build rules fail to disable 'trace-pc-guard'. By disabling
all sanitize-coverage flags, including 'trace-pc-guard', possible
spurious linker errors are prevented. In particular, this allows libcxx,
libcxxabi, and libunwind to be built with HonggFuzz.
CMAKE_REQUIRED_FLAGS is extra compile flags when running CMake build
configuration steps (like check_cxx_compiler_flag). It does not affect
the compile flags for the actual build of the project (unless of course
these flags change whether or not a given source compiles and links or
not). So libcxx, libcxxabi, and libunwind will still be built with any
specified sanitize-coverage as before. The build configuration steps
(which are mostly checking to see if certain compiler flags are
available) will not try to compile and link "int main() { return 0;}"
(or other specified source) with sanitize-coverage (which can fail to
link at this stage in building, since the final compile flags required
are yet to be determined).
The change to LIBFUZZER_CFLAGS was done to keep it consistent with the
obvious intention of disabling all sanitize-coverage. This appears to
be intentional, preventing the fuzzer driver itself from showing up in
any coverage calculations.
Reviewed By: #libunwind, #libc, #libc_abi, ldionne, phosek
Differential Revision: https://reviews.llvm.org/D116050
Currently we use very common names for macros like ACQUIRE/RELEASE,
which cause conflicts with system headers.
Prefix all macros with SANITIZER_ to avoid conflicts.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D116652
This allows DFSan to find tainted values used to control program behavior.
Reviewed By: morehouse
Differential Revision: https://reviews.llvm.org/D116207
This allows their reuse across projects. The name of the module
is intentionally generic because we would like to move more platform
checks there.
Differential Revision: https://reviews.llvm.org/D115276
This reverts commit 640beb38e7.
That commit caused performance degradtion in Quicksilver test QS:sGPU and a functional test failure in (rocPRIM rocprim.device_segmented_radix_sort).
Reverting until we have a better solution to s_cselect_b64 codegen cleanup
Change-Id: Ibf8e397df94001f248fba609f072088a46abae08
Reviewed By: kzhuravl
Differential Revision: https://reviews.llvm.org/D115960
Change-Id: Id169459ce4dfffa857d5645a0af50b0063ce1105
This will allow linking in the callbacks directly instead of using PLT.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D116182
A signal handler can alter ucontext_t to affect execution after
the signal returns. Check that the contents are initialized.
Restoring unitialized values in registers can't be good.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D116209
ucontext_t can be larger than its static size if it contains
AVX state and YMM/ZMM registers.
Currently a signal handler that tries to access that state
can produce false positives with random origins on stack.
Account for the additional ucontext_t state.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D116208
This is a segmentation fault in INTERCEPTOR function on a special edge
case of strstr libc call. When 'Haystack'(main string to be examined) is
NULL and 'needle'(sub-string to be searched in 'Haystack') is an empty
string then it hits a SEGV while using sanitizers and as a 'string not
found' case otherwise.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D115919
In D116472 we created conditionally defined variables for the tools to
unbreak the legacy build where they are in `llvm/tools`.
The runtimes are not tools, so that flexibility doesn't matter. Still,
it might be nice to define (unconditionally) and use the variable for
the runtimes simply to make the code a bit clearer and document what is
going on.
Also, consistently put project dirs at the beginning, not end of `CMAKE_MODULE_PATH`. This ensures they will properly shadow similarly named stuff that happens to be later on the path.
Reviewed By: mstorsjo, #libunwind, #libc, #libc_abi, ldionne
Differential Revision: https://reviews.llvm.org/D116477
It is likely to become used again, if other projects want their own per-project
install directory variables. `install` is removed from the name since it is not inherently about installing.
Reviewed By: stephenneuendorffer
Differential Revision: https://reviews.llvm.org/D115746
When using debug info for profile correlation, avoid adding duplicate
functions in the synthetic Data section.
Before this patch, n duplicate function entries in the Data section would
cause counter values to be a factor of n larger. I built instrumented
clang with and without debug info correlation and got these summaries.
```
# With Debug Info Correlate
$ llvm-profdata show default.profdata
Instrumentation level: IR entry_first = 0
Total functions: 182530
Maximum function count: 52034
Maximum internal block count: 5763
# Without
$ llvm-profdata show default.profdata
Instrumentation level: IR entry_first = 0
Total functions: 183212
Maximum function count: 52034
Maximum internal block count: 5766
```
The slight difference in counts seem to be mostly from FileSystem and
Map functions and the difference in the number of instrumented functions
seems to come from missing debug info like destructors without source.
Reviewed By: kyulee
Differential Revision: https://reviews.llvm.org/D116051
This patch adds support to read all the PT_NOTE segments in the
executable to find the binary ids. Previously, it was only reading
the first PT_NOTE segment, and this was missing the cases where
binary id is in the following segments. As a result, binary-id.c
and binary-id-padding.c test were failing in the following cases:
1) sanitizer-x86_64-linux bot
https://lab.llvm.org/staging/#/builders/97
2) OpenSuse Tumbleweed
https://github.com/llvm/llvm-project/issues/52695
Differential Revision: https://reviews.llvm.org/D115830
This will allow linking in the callbacks directly instead of using PLT.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D116182
This option is per process anyway. I'd like to add more options, but
having them as parameters of __sanitizer_symbolize_code looks
inconvenient.
Reviewed By: browneee
Differential Revision: https://reviews.llvm.org/D116201
This will allow linking in the callbacks directly instead of using PLT.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D116182
After D116148 the memccpy gets optimized away and the expected
uninitialized memory access does not occur.
Make sure the call does not get optimized away.
The new tsan runtime has 2x more compact shadow.
Adjust shadow ranges accordingly.
Depends on D112603.
Reviewed By: vitalybuka, melver
Differential Revision: https://reviews.llvm.org/D113751
If there are multiple processes, it's hard to understand
what output comes from what process.
VReport prepends pid to the output. Use it.
Depends on D113982.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D113983
Update now after long operations so that we don't use
stale value in subsequent computations.
Depends on D113981.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D113982
Creating threads after a multi-threaded fork is semi-supported,
we don't give particular guarantees, but we try to not fail
on simple cases and we have die_after_fork=0 flag that enables
not dying on creation of threads after a multi-threaded fork.
This flag is used in the wild:
23c052e3e3/SConstruct (L3599)
fork_multithreaded.cpp test started hanging in debug mode
after the recent "tsan: fix deadlock during race reporting" commit,
which added proactive ThreadRegistryLock check in SlotLock.
But the test broke earlier after "tsan: remove quadratic behavior in pthread_join"
commit which made tracking of alive threads based on pthread_t stricter
(CHECK-fail on 2 threads with the same pthread_t, or joining a non-existent thread).
When we start a thread after a multi-threaded fork, the new pthread_t
can actually match one of existing values (for threads that don't exist anymore).
Thread creation started CHECK-failing on this, but the test simply
ignored this CHECK failure in the child thread and "passed".
But after "tsan: fix deadlock during race reporting" the test started hanging dead,
because CHECK failures recursively lock thread registry.
Fix this purging all alive threads from thread registry on fork.
Also the thread registry mutex somehow lost the internal deadlock detector id
and was excluded from deadlock detection. If it would have the id, the CHECK
wouldn't hang because of the nested CHECK failure due to the deadlock.
But then again the test would have silently ignore this error as well
and the bugs wouldn't have been noticed.
Add the deadlock detector id to the thread registry mutex.
Also extend the test to check more cases and detect more bugs.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D116091
If we miss both close of a file descriptor and a subsequent open
if the same file descriptor number, we report false positives
between operations on the old and on the new descriptors.
There are lots of ways to create new file descriptors, but for closing
there is mostly close call. So we try to handle at least it.
However, if the close happens in an ignored library, we miss it
and start reporting false positives.
Handle closing of file descriptors always, even in ignored libraries
(as we do for malloc/free and other critical functions).
But don't imitate memory accesses on close for ignored libraries.
FdClose checks validity of the fd (fd >= 0) itself,
so remove the excessive checks in the callers.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D116095
We used to use u64 as mutex id because it was some
tricky identifier built from address and reuse count.
Now it's just the mutex index in the report (0, 1, 2...),
so use int to represent it.
Depends on D112603.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D113980
These callbacks are used for SSE vector accesses.
In some computational programs these accesses dominate.
Currently we do 2 uninlined 8-byte accesses to handle them.
Inline and optimize them similarly to unaligned accesses.
This reduces the vector access benchmark time from 8 to 3 seconds.
Depends on D112603.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D114594
This also makes the sanitizer_stoptheworld_test cross-platform by using the STL, rather than pthread.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D115204
This attempts to adjust the test to still exercise the expected codepath after D115904. This test is fundementally rather fragile.
Unfortunately, I have not been able to confirm this workaround either does, or does not, work. Attempting check-all with compiler-rt blows through an additional 30GB of disk space so my build config which exceeds my local disk space.
There is a small chance that the slot may be not queued in TraceSwitchPart.
This can happen if the slot has kEpochLast epoch and another thread
in FindSlotAndLock discovered that it's exhausted and removed it from
the slot queue. kEpochLast can happen in 2 cases: (1) if TraceSwitchPart
was called with the slot locked and epoch already at kEpochLast,
or (2) if we've acquired a new slot in SlotLock in the beginning
of the function and the slot was at kEpochLast - 1, so after increment
in SlotAttachAndLock it become kEpochLast.
If this happens we crash on ctx->slot_queue.Remove(thr->slot).
Skip the requeueing if the slot is not queued.
The slot is exhausted, so it must not be ctx->slot_queue.
The existing stress test triggers this with very small probability.
I am not sure how to make this condition more likely to be triggered,
it evaded lots of testing.
Depends on D116040.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D116041
SlotPairLocker calls SlotLock under ctx->multi_slot_mtx.
SlotLock can invoke global reset DoReset if we are out of slots/epochs.
But DoReset locks ctx->multi_slot_mtx as well, which leads to deadlock.
Resolve the deadlock by removing SlotPairLocker/multi_slot_mtx
and only lock one slot for which we will do RestoreStack.
We need to lock that slot because RestoreStack accesses the slot journal.
But it's unclear why we need to lock the current slot.
Initially I did it just to be on the safer side (but at that time
we dit not lock the second slot, so it was easy just to lock the current slot).
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D116040
Profile merging is not supported when using debug info profile
correlation because the data section won't be in the binary at runtime.
Change the default profile name in this mode to `default_%p.proflite` so
we don't use profile merging.
Reviewed By: kyulee
Differential Revision: https://reviews.llvm.org/D115979
Previously we would crash in the TSan runtime if the user program passes
a pointer to `malloc_size()` that doesn't point into app memory.
In these cases, `malloc_size()` should return 0.
For ASan, we fixed a similar issue here:
https://reviews.llvm.org/D15008
Radar-Id: rdar://problem/86213149
Differential Revision: https://reviews.llvm.org/D115947
There was a build failure on the `instrprof-debug-info-correlate.c` test
because zlib was missing so we need to require it to run the test.
Differential Revision: https://reviews.llvm.org/D115970
Extend `llvm-profdata` to read in a `.proflite` file and also a debug info file to generate a normal `.profdata` profile. This reduces the binary size by 8.4% when building an instrumented Clang binary without value profiling (164 MB vs 179 MB).
This work is part of the "lightweight instrumentation" RFC: https://groups.google.com/g/llvm-dev/c/r03Z6JoN7d4
This was first landed in https://reviews.llvm.org/D114566 but had to be reverted due to build errors.
Reviewed By: kyulee
Differential Revision: https://reviews.llvm.org/D115915
This reverts commit 3f5f687e2e.
That commit broke building for mingw, where the sanitizers are
built with -nostdinc++, while the added source file includes
the C++ standard library's <algorithm>.
Additionally, the new code fails to build for i386, as it
unconditionally uses the CONTEXT member Rsp.
AARCH64_GET_REG() is used to initialize uptrs, and after D79132
the ptrauth branch of its implementation explicitly casts to uptr.
The non-ptrauth branch returns ucontext->uc_mcontext->__ss.__fp (etc),
which has either type void* or __uint64_t (ref usr/include/mach/arm/_structs.h)
where __uint64_t is a unsigned long long (ref usr/include/arm/_types.h).
uptr is an unsigned long (ref
compiler-rt/lib/sanitizer_common/sanitizer_internal_defs.h). So explicitly
cast to uptr in this branch as well, so that AARCH64_GET_REG() has a
well-defined type.
Then change DUMPREGA64() tu use %lx instead of %llx since that's the right type
for uptr. (Most other places in compiler-rt print uptrs as %p and cast the arg
to (void*), but there are explicit 0x%016 format strings in the surroundings,
so be locally consistent with that.)
No behavior change, in the end it's just 64-bit unsigneds by slightly different
names.
This also makes the sanitizer_stoptheworld_test cross-platform by using the STL, rather than pthread.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D115204
Extend `llvm-profdata` to read in a `.proflite` file and also a debug info file to generate a normal `.profdata` profile. This reduces the binary size by 8.4% when building an instrumented Clang binary without value profiling (164 MB vs 179 MB).
This work is part of the "lightweight instrumentation" RFC: https://groups.google.com/g/llvm-dev/c/r03Z6JoN7d4
Reviewed By: kyulee
Differential Revision: https://reviews.llvm.org/D114566
Add the llvm flag `-debug-info-correlate` to attach debug info to instrumentation counters so we can correlate raw profile data to their functions. Raw profiles are dumped as `.proflite` files. The next diff enables `llvm-profdata` to consume `.proflite` and debug info files to produce a normal `.profdata` profile.
Part of the "lightweight instrumentation" work: https://groups.google.com/g/llvm-dev/c/r03Z6JoN7d4
The original diff https://reviews.llvm.org/D114565 was reverted because of the `Instrumentation/InstrProfiling/debug-info-correlate.ll` test, which is fixed in this commit.
Reviewed By: kyulee
Differential Revision: https://reviews.llvm.org/D115693
There were changes made to the linux version of this test that were not made for darwin
(see https://reviews.llvm.org/D115837) and this caused downstream failures.
Adding comment to this test to remind people to edit interface_symbols_darwin.cpp.
There is the reverse of this comment in the darwin file to remind us to edit the linux version already.
Differential Revision: https://reviews.llvm.org/D115899
LSan (`ASAN_OPTIONS=detect_leaks=1`) is supported on macOS, but disabled
by default on Darwin (`SANITIZER_MAC`):
```
COMMON_FLAG(bool, detect_leaks, !SANITIZER_MAC, "Enable memory leak detection.")
```
We enable it here for ASan tests to prevent regressions (per comment).
However, LSan is not supported for the iOS simulator and the tests fail
when it is enabled.
Make this "Is macOS?" check more precise since the current one (`Darwin
&& x86_64`) has two issues:
* Includes the simulators
* Excludes macOS on Apple Silicon
This will allow us to (re)enable simulator testing on Green dragon to
give open source better feedback about sanitizer changes:
https://green.lab.llvm.org
rdar://86529234
Differential Revision: https://reviews.llvm.org/D115816
This test would hang when the system ran out of resources and we fail to
create all 300 threads.
Differential Revision: https://reviews.llvm.org/D115845
In the new TSan runtime refactoring this line was changed:
```
ProtectRange(MetaShadowEnd(), TraceMemBeg());
-->
ProtectRange(MetaShadowEnd(), HeapMemBeg());
```
But for `MappingAppleAarch64` the app heap comes before the shadow,
resulting in:
```
CHECK failed: tsan_platform_posix.cpp:83 "((beg)) <= ((end))" (0xe00000000, 0x200000000)
```
rdar://86521924
Differential Revision: https://reviews.llvm.org/D115834
A while ago we added some code to the sanitizer runtimes for iOS
simulators to allow `atos` (external process) to inspect the sanitized
process during report generation to enable symbolication. This was done
by setting the `__check_mach_ports_lookup` env var early during process
startup which came with a couple of complications.
This workaround is not required anymore and removing it fixes TSan in
the iOS simulator after the new TSan runtime landed.
(https://reviews.llvm.org/D112603)
Relevant/reverted revisions:
https://reviews.llvm.org/D78178https://reviews.llvm.org/D78179https://reviews.llvm.org/D78525
rdar://86472733
Differential Revision: https://reviews.llvm.org/D115767
Adds -L<search-path> and -l<library> options that are analogous to ld's
versions.
Each instance of -L<search-path> or -l<library> will apply to the most recent
-jd option on the command line (-jd <name> creates a JITDylib with the given
name). Library names will match against JITDylibs first, then llvm-jitlink will
look through the search paths for files named <search-path>/lib<library>.dylib
or <search-path>/lib<library>.a.
The default "main" JITDylib will link against all JITDylibs created by -jd
options, and all JITDylibs will link against the process symbols (unless
-no-process-symbols is specified).
The -dlopen option is renamed -preload, and will load dylibs into the JITDylib
for the ORC runtime only.
The effect of these changes is to make it easier to describe a non-trivial
program layout to llvm-jitlink for testing purposes. E.g. the following
invocation describes a program consisting of three JITDylibs: "main" (created
implicitly) containing main.o, "Foo" containing foo1.o and foo2.o, and linking
against library "bar" (not a JITDylib, so it must be a .dylib or .a on disk)
and "Baz" (which is a JITDylib), and "Baz" containing baz.o.
llvm-jitlink \
main.o \
-jd Foo foo1.o foo2.o -L${HOME}/lib -lbar -lBaz
-jd Baz baz.o
This is present in our assembly files. It should fix decorate_proc_maps.cpp failures because of shadow memory being allocated as executable.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D115552
This change moves optimized callbacks from each .o file to compiler-rt. Instead of using code generation it uses direct assembly implementation. Please note that the 'or' version is not implemented and it will produce unresolved external if somehow 'or' version is requested.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D114558
This reverts commit 800bf8ed29.
The `Instrumentation/InstrProfiling/debug-info-correlate.ll` test was
failing because I forgot the `llc` commands are architecture specific.
I'll follow up with a fix.
Differential Revision: https://reviews.llvm.org/D115689
Add the llvm flag `-debug-info-correlate` to attach debug info to instrumentation counters so we can correlate raw profile data to their functions. Raw profiles are dumped as `.proflite` files. The next diff enables `llvm-profdata` to consume `.proflite` and debug info files to produce a normal `.profdata` profile.
Part of the "lightweight instrumentation" work: https://groups.google.com/g/llvm-dev/c/r03Z6JoN7d4
Reviewed By: kyulee
Differential Revision: https://reviews.llvm.org/D114565
Adds a fallback to use the debuginfod client library (386655) in `findDebugBinary`.
Fixed a cast of Erorr::success() to Expected<> in debuginfod library.
Added Debuginfod to Symbolize deps in gn.
Updates compiler-rt/lib/sanitizer_common/symbolizer/scripts/build_symbolizer.sh to include Debuginfod library to fix sanitizer-x86_64-linux breakage.
Reviewed By: jhenderson, vitalybuka
Differential Revision: https://reviews.llvm.org/D113717
This change switches tsan to the new runtime which features:
- 2x smaller shadow memory (2x of app memory)
- faster fully vectorized race detection
- small fixed-size vector clocks (512b)
- fast vectorized vector clock operations
- unlimited number of alive threads/goroutimes
Depends on D112602.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D112603
Currently the test calls dlclose in the thread
concurrently with the main thread calling a function
from the dynamic library. This is not good.
Wait for the main thread to call the function
before calling dlclose.
Depends on D115612.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D115613
The test contains a race and checks that it's detected.
But the race may not be detected since we are doing aggressive flushes
and if the state flush happens between racing accesses, tsan won't
detect the race). So return 1 to make the test deterministic
regardless of the race.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D115612
This is present in our assembly files. It should fix decorate_proc_maps.cpp failures because of shadow memory being allocated as executable.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D115552
This change moves optimized callbacks from each .o file to compiler-rt. Instead of using code generation it uses direct assembly implementation. Please note that the 'or' version is not implemented and it will produce unresolved external if somehow 'or' version is requested.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D114558
Adds x-ray support for hexagon to llvm codegen, clang driver,
compiler-rt libs.
Differential Revision: https://reviews.llvm.org/D113638
Reapplying this after 543a9ad7c4,
which fixes the leak introduced there.
Adds a fallback to use the debuginfod client library (386655) in `findDebugBinary`.
Fixed a cast of Erorr::success() to Expected<> in debuginfod library.
Added Debuginfod to Symbolize deps in gn.
Updates compiler-rt/lib/sanitizer_common/symbolizer/scripts/build_symbolizer.sh to include Debuginfod library to fix sanitizer-x86_64-linux breakage.
Reviewed By: jhenderson, vitalybuka
Differential Revision: https://reviews.llvm.org/D113717
Adds a fallback to use the debuginfod client library (386655) in `findDebugBinary`.
Fixed a cast of Erorr::success() to Expected<> in debuginfod library.
Added Debuginfod to Symbolize deps in gn.
Adds new symbolizer symbols to `global_symbols.txt`.
Reviewed By: jhenderson
Differential Revision: https://reviews.llvm.org/D113717
This change moves optimized callbacks from each .o file to compiler-rt. Instead of using code generation it uses direct assembly implementation. Please note that the 'or' version is not implemented and it will produce unresolved external if somehow 'or' version is requested.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D114558
The test has been flaky for years, and I think we should remove it to
eliminate noise on the buildbot.
Neither me nor dokyungs have been able to fully deflake the test, and it
tests a non-default Entropic flag.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D115453
This removes the last use of StackDepot from StopTheWorld.
Depends on D115284.
Reviewed By: morehouse
Differential Revision: https://reviews.llvm.org/D115319
StackDepot locks some stuff. As is there is small probability to
deadlock if we stop thread which locked the Depot.
We need either Lock/Unlock StackDepot for StopTheWorld, or don't
interact with StackDepot from there.
This patch does not run LeakReport under StopTheWorld. LeakReport
contains most of StackDepot access.
As a bonus this patch will help to resolve kMaxLeaksConsidered FIXME.
Depends on D114498.
Reviewed By: morehouse, kstoimenov
Differential Revision: https://reviews.llvm.org/D115284
This reverts commit e5c2a46c5e as this
change introduced a linker error when building sanitizer runtimes:
ld.lld: error: undefined symbol: __sanitizer::internal_start_thread(void* (*)(void*), void*)
>>> referenced by sanitizer_stackdepot.cpp:133 (compiler-rt/lib/sanitizer_common/sanitizer_stackdepot.cpp:133)
>>> compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonSymbolizer.x86_64.dir/sanitizer_stackdepot.cpp.obj:(__sanitizer::(anonymous namespace)::CompressThread::NewWorkNotify())
This change switches tsan to the new runtime which features:
- 2x smaller shadow memory (2x of app memory)
- faster fully vectorized race detection
- small fixed-size vector clocks (512b)
- fast vectorized vector clock operations
- unlimited number of alive threads/goroutimes
Depends on D112602.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D112603
Fork the current version of tsan runtime before commiting
rewrite of the runtime (D112603). The old runtime can be
enabled with TSAN_USE_OLD_RUNTIME option.
This is a temporal measure for emergencies and is required
for Chromium rollout (for context see http://crbug.com/1275581).
The old runtime is supposed to be deleted soon.
Reviewed By: thakis
Differential Revision: https://reviews.llvm.org/D115223
This reverts commit 5c27740238.
Reverting due to Windows build issue:
sanitizer_stackdepot.cpp.obj : error LNK2005: "void __cdecl __sanitizer::StackDepotStopBackgroundThread(void)" (?StackDepotStopBackgroundThread@__sanitizer@@YAXXZ) already defined in sanitizer_common_libcdep.cpp.obj
LINK : fatal error LNK1181: cannot open input file 'projects\compiler-rt\lib\asan\CMakeFiles\RTAsan_dynamic.x86_64.dir\asan_rtl_x86_64.S.obj'
This change moves optimized callbacks from each .o file to compiler-rt. Instead of using code generation it uses direct assembly implementation. Please note that the 'or' version is not implemented and it will produce unresolved external if somehow 'or' version is requested.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D114558
Some of the compiler-rt runtimes use custom instrumented libc++ build.
Use the runtimes build for building this custom libc++.
Differential Revision: https://reviews.llvm.org/D114922
We don't run tests or benchmarks from this build anyway.
Benchmarks in custom libc++ break my local build.
Reviewed By: ldionne
Differential Revision: https://reviews.llvm.org/D115375
kIgnored didn't exist when the code was added, but it should be
equivalent to kReachable.
The goal is to refactor MarkInvalidPCCb to avoid StackDepotGet
in StopTheWorld.
Some of the compiler-rt runtimes use custom instrumented libc++ build.
Use the runtimes build for building this custom libc++.
Differential Revision: https://reviews.llvm.org/D114922
It is required for the [Leak Sanitizer port to Windows](https://reviews.llvm.org/D115103).
The currently used `unsigned long` type is 64 bits wide on UNIX like systems but only 32 bits wide on Windows.
Because of that, the literal `8UL << 30` causes an integer overflow on Windows.
By changing the type of the literals to `unsigned long long`, we have consistent behavior and no overflows on all Platforms.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D115186
That way, the build rules are closer to the source files they describe.
No intended behavior change.
Differential Revision: https://reviews.llvm.org/D115155
Building xray with recent clang on a 64-bit system results in a number
of -Wformat warnings:
compiler-rt/lib/xray/xray_allocator.h:70:11: warning: format specifies type 'int' but the argument has type '__sanitizer::uptr' (aka 'unsigned long') [-Wformat]
RoundedSize, B);
^~~~~~~~~~~
compiler-rt/lib/xray/xray_allocator.h:119:11: warning: format specifies type 'int' but the argument has type '__sanitizer::uptr' (aka 'unsigned long') [-Wformat]
RoundedSize, B);
^~~~~~~~~~~
Since `__sanitizer::uptr` has the same size as `size_t`, these can be
fixed by using the printf specifier `%zu`.
compiler-rt/lib/xray/xray_basic_logging.cpp:348:46: warning: format specifies type 'int' but the argument has type '__sanitizer::tid_t' (aka 'unsigned long long') [-Wformat]
Report("Cleaned up log for TID: %d\n", GetTid());
~~ ^~~~~~~~
%llu
compiler-rt/lib/xray/xray_basic_logging.cpp:353:62: warning: format specifies type 'int' but the argument has type '__sanitizer::tid_t' (aka 'unsigned long long') [-Wformat]
Report("Skipping buffer for TID: %d; Offset = %llu\n", GetTid(),
~~ ^~~~~~~~
%llu
Since `__sanitizer::tid_t` is effectively declared as `unsigned long
long`, these can be fixed by using the printf specifier `%llu`.
compiler-rt/lib/xray/xray_basic_logging.cpp:354:14: warning: format specifies type 'unsigned long long' but the argument has type 'size_t' (aka 'unsigned long') [-Wformat]
TLD.BufferOffset);
^~~~~~~~~~~~~~~~
Since `BufferOffset` is declared as `size_t`, this one can be fixed by
using `%zu` as a printf specifier.
compiler-rt/lib/xray/xray_interface.cpp:172:50: warning: format specifies type 'int' but the argument has type 'uint64_t' (aka 'unsigned long') [-Wformat]
Report("Unsupported sled kind '%d' @%04x\n", Sled.Address, int(Sled.Kind));
~~ ^~~~~~~~~~~~
%lu
Since ``xray::SledEntry::Address` is declared as `uint64_t`, this one
can be fixed by using `PRIu64`, and adding `<cinttypes>`.
compiler-rt/lib/xray/xray_interface.cpp:308:62: warning: format specifies type 'long long' but the argument has type 'size_t' (aka 'unsigned long') [-Wformat]
Report("System page size is not a power of two: %lld\n", PageSize);
~~~~ ^~~~~~~~
%zu
compiler-rt/lib/xray/xray_interface.cpp:359:64: warning: format specifies type 'long long' but the argument has type 'size_t' (aka 'unsigned long') [-Wformat]
Report("Provided page size is not a power of two: %lld\n", PageSize);
~~~~ ^~~~~~~~
%zu
Since `PageSize` is declared as `size_t`, these can be fixed by using
`%zu` as a printf specifier.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D114469
It's very simple, fast and efficient for the stack depot compression if used on entire pointers.
Reviewed By: morehouse, kstoimenov
Differential Revision: https://reviews.llvm.org/D114918
Using `_mkdir` of CRT in Asan Init leads to launch failure and hanging in Windows.
You can trigger it by calling:
> set ASAN_OPTIONS=log_path=a/a/a
> .\asan_program.exe
And their crash dump shows the following stack trace:
```
_guard_dispatch_icall_nop()
__acrt_get_utf8_acp_compatibility_codepage()
_mkdir(const char * path)
```
I guess there could be a cfg guard in CRT, which may lead to calling uninitialized cfg guard function address. Also, `_mkdir` supports UTF-8 encoding of the path and calls _wmkdir, but that's not necessary for this case since other file apis in sanitizer_win.cpp assumes only ANSI code case, so it makes sense to use CreateDirectoryA matching other file api calls in the same file.
Reviewed By: tejohnson
Differential Revision: https://reviews.llvm.org/D114760
Google-signed apexes appear on Android build servers' symbol files as
being under /apex/com.google.android.<foo>/. In reality, the apexes are
always installed as /apex/com.android.<foo>/ (note the lack of
'google'). In order for local symbolization under hwasan_symbolize to
work correctly, we also try the 'google' directory.
Reviewed By: eugenis
Differential Revision: https://reviews.llvm.org/D114919
According comments on D44404, something like that was the goal.
Reviewed By: morehouse, kstoimenov
Differential Revision: https://reviews.llvm.org/D114991
The goal is to identify the bot and try to fix it.
SetSoftRssLimitExceededCallback is AsanInitInternal as I assume
that only MaybeStartBackgroudThread needs to be delayed to constructors.
Later I want to move MaybeStartBackgroudThread call into sanitizer_common.
If it needs to be reverted please provide to more info, like bot, or details about setup.
Reviewed By: kstoimenov
Differential Revision: https://reviews.llvm.org/D114934
Compress by factor 4x, takes about 10ms per 8 MiB block.
Depends on D114498.
Reviewed By: morehouse
Differential Revision: https://reviews.llvm.org/D114503
It should be NFC, as they already intercept pthread_create.
This will let us to fix BackgroundThread for these sanitizerts.
In in followup patches I will fix MaybeStartBackgroudThread for them
and corresponding tests.
Reviewed By: kstoimenov
Differential Revision: https://reviews.llvm.org/D114935
We call UnmapShadow before the actual munmap, at that point we don't yet
know if the provided address/size are sane. We can't call UnmapShadow
after the actual munmap becuase at that point the memory range can
already be reused for something else, so we can't rely on the munmap
return value to understand is the values are sane.
While calling munmap with insane values (non-canonical address, negative
size, etc) is an error, the kernel won't crash. We must also try to not
crash as the failure mode is very confusing (paging fault inside of the
runtime on some derived shadow address).
Such invalid arguments are observed on Chromium tests:
https://bugs.chromium.org/p/chromium/issues/detail?id=1275581
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D114944
The added test demonstrates loading a dynamic library with static TLS.
Such static TLS is a hack that allows a dynamic library to have faster TLS,
but it can be loaded only iff all threads happened to allocate some excess
of static TLS space for whatever reason. If it's not the case loading fails with:
dlopen: cannot load any more object with static TLS
We used to produce a false positive because dlopen will write into TLS
of all existing threads to initialize/zero TLS region for the loaded library.
And this appears to be racing with initialization of TLS in the thread
since we model a write into the whole static TLS region (we don't what part
of it is currently unused):
WARNING: ThreadSanitizer: data race (pid=2317365)
Write of size 1 at 0x7f1fa9bfcdd7 by main thread:
0 memset
1 init_one_static_tls
2 __pthread_init_static_tls
[[ this is where main calls dlopen ]]
3 main
Previous write of size 8 at 0x7f1fa9bfcdd0 by thread T1:
0 __tsan_tls_initialization
Fix this by ignoring accesses during dlopen.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D114953
Broke the build on Windows, where MprotectReadOnly() isn't defined, see comment
on the code review.
> Compress by factor 4x, takes about 10ms per 8 MiB block.
>
> Depends on D114498.
>
> Reviewed By: morehouse
>
> Differential Revision: https://reviews.llvm.org/D114503
This reverts commit 1d8f295759.
Compress by factor 4x, takes about 10ms per 8 MiB block.
Depends on D114498.
Reviewed By: morehouse
Differential Revision: https://reviews.llvm.org/D114503
The first 8b of each raw profile section need to be aligned to 8b since
the first item in each section is a u64 count of the number of items in
the section.
Summary of changes:
* Assert alignment when reading counts.
* Update test to check alignment, relax some size checks to allow padding.
* Update raw binary inputs for llvm-profdata tests.
Differential Revision: https://reviews.llvm.org/D114826
Add Compression::Test type which just pretends packing,
but does nothing useful. It's only called from test for now.
Depends on D114493.
Reviewed By: kstoimenov
Differential Revision: https://reviews.llvm.org/D114494
We would like to use TLS to store the ThreadState object (or at least a
reference ot it), but on Darwin accessing TLS via __thread or manually
by using pthread_key_* is problematic, because there are several places
where interceptors are called when TLS is not accessible (early process
startup, thread cleanup, ...).
Previously, we used a "poor man's TLS" implementation, where we use the
shadow memory of the pointer returned by pthread_self() to store a
pointer to the ThreadState object.
The problem with that was that certain operations can populate shadow
bytes unbeknownst to TSan, and we later interpret these non-zero bytes
as the pointer to our ThreadState object and crash on when dereferencing
the pointer.
This patch changes the storage location of our reference to the
ThreadState object to "real" TLS. We make this work by artificially
keeping this reference alive in the pthread_key destructor by resetting
the key value with pthread_setspecific().
This change also fixes the issue were the ThreadState object is
re-allocated after DestroyThreadState() because intercepted functions
can still get called on the terminating thread after the
THREAD_TERMINATE event.
Radar-Id: rdar://problem/72010355
Reviewed By: dvyukov
Differential Revision: https://reviews.llvm.org/D110236
The memprof unit tests use an older version of gmock (included in the
repo) which does not build cleanly with -pedantic:
https://github.com/google/googletest/issues/2650
For now just silence the warning by disabling pedantic and add the
appropriate flags for gcc and clang.
This commit adds initial support to llvm-profdata to read and print
summaries of raw memprof profiles.
Summary of changes:
* Refactor shared defs to MemProfData.inc
* Extend show_main to display memprof profile summaries.
* Add a simple raw memprof profile reader.
* Add a couple of tests to tools/llvm-profdata.
Differential Revision: https://reviews.llvm.org/D114286
In multi-threaded application concurrent StackStore::Store may
finish in order different from assigned Id. So we can't assume
that after we switch writing the next block the previous is done.
The workaround is to count exact number of uptr stored into the block,
including skipped tail/head which were not able to fit entire trace.
Depends on D114490.
Reviewed By: morehouse
Differential Revision: https://reviews.llvm.org/D114493
Larger blocks are more convenient for compressions.
Blocks are allocated with MmapNoReserveOrDie to save some memory.
Also it's 15% faster on StackDepotBenchmarkSuite
Depends on D114464.
Reviewed By: morehouse
Differential Revision: https://reviews.llvm.org/D114488
Sometimes stacks for at_exit callbacks don't include any of the user functions/files.
For example, a race with a global std container destructor will only contain
the container type name and our at_exit_wrapper function. No signs what global variable
this is.
Remember and include in reports the function that installed the at_exit callback.
This should give glues as to what variable is being destroyed.
Depends on D114606.
Reviewed By: vitalybuka, melver
Differential Revision: https://reviews.llvm.org/D114607
Add a test for a common C++ bug when a global object is destroyed
while background threads still use it.
Depends on D114604.
Reviewed By: vitalybuka, melver
Differential Revision: https://reviews.llvm.org/D114605
[NFC] As part of using inclusive language within the llvm project, this patch
replaces master and slave with primary and secondary respectively in
`sanitizer_mac.cpp`.
Reviewed By: ZarkoCA
Differential Revision: https://reviews.llvm.org/D114255
This change switches tsan to the new runtime which features:
- 2x smaller shadow memory (2x of app memory)
- faster fully vectorized race detection
- small fixed-size vector clocks (512b)
- fast vectorized vector clock operations
- unlimited number of alive threads/goroutimes
Depends on D112602.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D112603
Linux/fork_deadlock.cpp currently hangs in debug mode in the following stack.
Disable memory access handling in OnUserAlloc/Free around fork.
1 0x000000000042c54b in __sanitizer::internal_sched_yield () at sanitizer_linux.cpp:452
2 0x000000000042da15 in __sanitizer::StaticSpinMutex::LockSlow (this=0x57ef02 <__sanitizer::internal_allocator_cache_mu>) at sanitizer_mutex.cpp:24
3 0x0000000000423927 in __sanitizer::StaticSpinMutex::Lock (this=0x57ef02 <__sanitizer::internal_allocator_cache_mu>) at sanitizer_mutex.h:32
4 0x000000000042354c in __sanitizer::GenericScopedLock<__sanitizer::StaticSpinMutex>::GenericScopedLock (this=this@entry=0x7ffcabfca0b8, mu=0x1) at sanitizer_mutex.h:367
5 0x0000000000423653 in __sanitizer::RawInternalAlloc (size=size@entry=72, cache=cache@entry=0x0, alignment=8, alignment@entry=0) at sanitizer_allocator.cpp:52
6 0x00000000004235e9 in __sanitizer::InternalAlloc (size=size@entry=72, cache=0x1, cache@entry=0x0, alignment=4, alignment@entry=0) at sanitizer_allocator.cpp:86
7 0x000000000043aa15 in __sanitizer::SymbolizedStack::New (addr=4802655) at sanitizer_symbolizer.cpp:45
8 0x000000000043b353 in __sanitizer::Symbolizer::SymbolizePC (this=0x7f578b77a028, addr=4802655) at sanitizer_symbolizer_libcdep.cpp:90
9 0x0000000000439dbe in __sanitizer::(anonymous namespace)::StackTraceTextPrinter::ProcessAddressFrames (this=this@entry=0x7ffcabfca208, pc=4802655) at sanitizer_stacktrace_libcdep.cpp:36
10 0x0000000000439c89 in __sanitizer::StackTrace::PrintTo (this=this@entry=0x7ffcabfca2a0, output=output@entry=0x7ffcabfca260) at sanitizer_stacktrace_libcdep.cpp:109
11 0x0000000000439fe0 in __sanitizer::StackTrace::Print (this=0x18) at sanitizer_stacktrace_libcdep.cpp:132
12 0x0000000000495359 in __sanitizer::PrintMutexPC (pc=4802656) at tsan_rtl.cpp:774
13 0x000000000042e0e4 in __sanitizer::InternalDeadlockDetector::Lock (this=0x7f578b1ca740, type=type@entry=2, pc=pc@entry=4371612) at sanitizer_mutex.cpp:177
14 0x000000000042df65 in __sanitizer::CheckedMutex::LockImpl (this=<optimized out>, pc=4) at sanitizer_mutex.cpp:218
15 0x000000000042bc95 in __sanitizer::CheckedMutex::Lock (this=0x600001000000) at sanitizer_mutex.h:127
16 __sanitizer::Mutex::Lock (this=0x600001000000) at sanitizer_mutex.h:165
17 0x000000000042b49c in __sanitizer::GenericScopedLock<__sanitizer::Mutex>::GenericScopedLock (this=this@entry=0x7ffcabfca370, mu=0x1) at sanitizer_mutex.h:367
18 0x000000000049504f in __tsan::TraceSwitch (thr=0x7f578b1ca980) at tsan_rtl.cpp:656
19 0x000000000049523e in __tsan_trace_switch () at tsan_rtl.cpp:683
20 0x0000000000499862 in __tsan::TraceAddEvent (thr=0x7f578b1ca980, fs=..., typ=__tsan::EventTypeMop, addr=4499472) at tsan_rtl.h:624
21 __tsan::MemoryAccessRange (thr=0x7f578b1ca980, pc=4499472, addr=135257110102784, size=size@entry=16, is_write=true) at tsan_rtl_access.cpp:563
22 0x000000000049853a in __tsan::MemoryRangeFreed (thr=thr@entry=0x7f578b1ca980, pc=pc@entry=4499472, addr=addr@entry=135257110102784, size=16) at tsan_rtl_access.cpp:487
23 0x000000000048f6bf in __tsan::OnUserFree (thr=thr@entry=0x7f578b1ca980, pc=pc@entry=4499472, p=p@entry=135257110102784, write=true) at tsan_mman.cpp:260
24 0x000000000048f61f in __tsan::user_free (thr=thr@entry=0x7f578b1ca980, pc=4499472, p=p@entry=0x7b0400004300, signal=true) at tsan_mman.cpp:213
25 0x000000000044a820 in __interceptor_free (p=0x7b0400004300) at tsan_interceptors_posix.cpp:708
26 0x00000000004ad599 in alloc_free_blocks () at fork_deadlock.cpp:25
27 __tsan_test_only_on_fork () at fork_deadlock.cpp:32
28 0x0000000000494870 in __tsan::ForkBefore (thr=0x7f578b1ca980, pc=pc@entry=4904437) at tsan_rtl.cpp:510
29 0x000000000046fcb4 in syscall_pre_fork (pc=1) at tsan_interceptors_posix.cpp:2577
30 0x000000000046fc9b in __sanitizer_syscall_pre_impl_fork () at sanitizer_common_syscalls.inc:3094
31 0x00000000004ad5f5 in myfork () at syscall.h:9
32 main () at fork_deadlock.cpp:46
Depends on D114595.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D114597
We currently use a wrong value for heap block
(only works for C++, but not for Java).
Use the correct value (we already computed it before, just forgot to use).
Depends on D114593.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D114595
Add a basic test that checks races between vector/non-vector
read/write accesses of different sizes/offsets in different orders.
This gives coverage of __tsan_read/write16 callbacks.
Depends on D114591.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D114592
Vector SSE accesses make compiler emit __tsan_[unaligned_]read/write16 callbacks.
Make it possible to test these.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D114591
Building cfi with recent clang on a 64-bit system results in the
following warnings:
compiler-rt/lib/cfi/cfi.cpp:233:64: warning: format specifies type 'void *' but the argument has type '__sanitizer::uptr' (aka 'unsigned long') [-Wformat]
VReport(1, "Can not handle: symtab > strtab (%p > %zx)\n", symtab, strtab);
~~ ^~~~~~
%lu
compiler-rt/lib/sanitizer_common/sanitizer_common.h:231:46: note: expanded from macro 'VReport'
if ((uptr)Verbosity() >= (level)) Report(__VA_ARGS__); \
^~~~~~~~~~~
compiler-rt/lib/cfi/cfi.cpp:253:59: warning: format specifies type 'void *' but the argument has type '__sanitizer::uptr' (aka 'unsigned long') [-Wformat]
VReport(1, "Can not handle: symtab %p, strtab %zx\n", symtab, strtab);
~~ ^~~~~~
%lu
compiler-rt/lib/sanitizer_common/sanitizer_common.h:231:46: note: expanded from macro 'VReport'
if ((uptr)Verbosity() >= (level)) Report(__VA_ARGS__); \
^~~~~~~~~~~
Since `__sanitizer::uptr` has the same size as `size_t`, consistently
use `%z` as a printf specifier.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D114466
Now that we lock the internal allocator around fork,
it's possible it will create additional deadlocks.
Add a fake mutex that substitutes the internal allocator
for the purposes of deadlock detection.
Depends on D114531.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D114532
There is a small chance that the internal allocator is locked
during fork and then the new process is created with locked
internal allocator and any attempts to use it will deadlock.
For example, if detected a suppressed race in the parent during fork
and then another suppressed race after the fork.
This becomes much more likely with the new tsan runtime
as it uses the internal allocator for more things.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D114531
The test tries to provoke internal allocator to be locked during fork
and then force the child process to use the internal allocator.
This test sometimes deadlocks with the new tsan runtime.
Depends on D114514.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D114515
It was introduced in:
9cffc9550b tsan: allow to force use of __libc_malloc in sanitizer_common
and used in:
512a18e518 tsan: add standalone deadlock detector
and later used for Go support.
But now both uses are gone. Nothing defines SANITIZER_USE_MALLOC.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D114514
Test size larger than clear_shadow_mmap_threshold,
which is handled differently.
Depends on D114348.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D114366
Nothing uses more than 8bit now. So the rest of the headers can store other data.
kStackTraceMax is 256 now, but all sanitizers by default store just 20-30 frames here.
Verified no diff exist between previous version, new version python 2, and python 3 for an example stack.
Reviewed By: eugenis
Differential Revision: https://reviews.llvm.org/D114404
The INSTR_PROF_RAW_MAGIC_* number in profraw files should match during
profile merging. This causes an error with 32-bit and 64-bit variants
of the same code. The module signatures for the two binaries are
identical but they use different INSTR_PROF_RAW_MAGIC_* causing a
failure when profile-merging is used. Including it when computing the
module signature yields different signatures for the 32-bit and 64-bit
profiles.
Differential Revision: https://reviews.llvm.org/D114054
This change switches tsan to the new runtime which features:
- 2x smaller shadow memory (2x of app memory)
- faster fully vectorized race detection
- small fixed-size vector clocks (512b)
- fast vectorized vector clock operations
- unlimited number of alive threads/goroutimes
Differential Revision: https://reviews.llvm.org/D112603
We dropped the printing of live on exit blocks in rG1243cef245f6 -
the commit changed the insertOrMerge logic. Remove the message since it
is no longer needed (all live blocks are inserted into the hashmap)
before serializing/printing the profile. Furthermore, the original
intent was to capture evicted blocks so it wasn't entirely correct.
Also update the binary format test invocation to remove the redundant
print_text directive now that it is the default.
Differential Revision: https://reviews.llvm.org/D114285
MemProfUnitTest now depends on libcxx but the dependency is not
explicitly expressed in build system, causing build races. This patch
addresses this issue.
Differential Revision: https://reviews.llvm.org/D114267
This change switches tsan to the new runtime which features:
- 2x smaller shadow memory (2x of app memory)
- faster fully vectorized race detection
- small fixed-size vector clocks (512b)
- fast vectorized vector clock operations
- unlimited number of alive threads/goroutimes
Depends on D112602.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D112603
All runtime callbacks must be non-instrumented with the new tsan runtime
(it's now more picky with respect to recursion into runtime).
Disable instrumentation in Darwin tests as we do in all other tests now.
Differential Revision: https://reviews.llvm.org/D114348
Add a fork test that models what happens on Mac
where fork calls malloc/free inside of our atfork
callbacks.
Reviewed By: vitalybuka, yln
Differential Revision: https://reviews.llvm.org/D114250
Using the out-of-line LSE atomics helpers for AArch64 on FreeBSD also
requires adding support for initializing __aarch64_have_lse_atomics
correctly. On Linux this is done with getauxval(3), on FreeBSD with
elf_aux_info(3), which has a slightly different interface.
Differential Revision: https://reviews.llvm.org/D109330
We need libfuzzer libraries on Arm32 so that we can fuzz
Arm32 binaries on Linux (Chrome OS). Android already
allows Arm32 for libfuzzer.
Reviewed By: morehouse
Differential Revision: https://reviews.llvm.org/D112091
pthread_join needs to map pthread_t of the joined thread to our Tid.
Currently we do this with linear search over all threads.
This has quadratic complexity and becomes much worse with the new
tsan runtime, which memorizes all threads that ever existed.
To resolve this add a hash map of live threads only (that are still
associated with pthread_t) and use it for the mapping.
With the new tsan runtime some programs spent 1/3 of time in this mapping.
After this change the mapping disappears from profiles.
Depends on D113996.
Reviewed By: vitalybuka, melver
Differential Revision: https://reviews.llvm.org/D113997
The __flags variable needs to be of type 'long' in order to get sign extended
properly.
internal_clone() uses an svc (Supervisor Call) directly (as opposed to
internal_syscall), and therefore needs to take care to set errno and return
-1 as needed.
Review: Ulrich Weigand
asan does not use user_id for anything,
so don't pass it to ThreadCreate.
Passing a random uninitialized field of AsanThread
as user_id does not make much sense anyway.
Depends on D113921.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D113922
memprof does not use user_id for anything,
so don't pass it to ThreadCreate.
Passing a random field of MemprofThread as user_id
does not make much sense anyway.
Depends on D113920.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D113921
They don't seem to do anything useful in lsan.
They are needed only if a tools needs to execute
some custom logic during detach/join, or if it uses
thread registry quarantine. Lsan does none of this.
And if a tool cares then it would also need to intercept
pthread_tryjoin_np and pthread_timedjoin_np, otherwise
it will mess thread states.
Fwiw, asan does not intercept these functions either.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D113920
Since libclang_rt.profile is added later in the command line, a
definition of __llvm_profile_raw_version is not included if it is
provided from an earlier object, e.g. from a shared dependency.
This causes an extra dependence edge where if libA.so depends on libB.so
and both are coverage-instrumented, libA.so uses libB.so's definition of
__llvm_profile_raw_version. This leads to a runtime link failure if the
libB.so available at runtime does not provide this symbol (but provides
the other dependent symbols). Such a scenario can occur in Android's
mainline modules.
E.g.:
ld -o libB.so libclang_rt.profile-x86_64.a
ld -o libA.so -l B libclang_rt.profile-x86_64.a
libB.so has a global definition of __llvm_profile_raw_version. libA.so
uses libB.so's definition of __llvm_profile_raw_version. At runtime,
libB.so may not be coverage-instrumented (i.e. not export
__llvm_profile_raw_version) so runtime linking of libA.so will fail.
Marking this symbol as hidden forces each binary to use the definition
of __llvm_profile_raw_version from libclang_rt.profile. The visiblity
is unchanged for Apple platforms where its presence is checked by the
TAPI tool.
Reviewed By: MaskRay, phosek, davidxl
Differential Revision: https://reviews.llvm.org/D111759
The new test started failing on bots with:
CHECK failed: tsan_rtl.cpp:327 "((addr + size)) <= ((TraceMemEnd()))"
(0xf06200e03010, 0xf06200000000) (tid=4073872)
https://lab.llvm.org/buildbot#builders/179/builds/1761
This is a latent bug in aarch64 virtual address space layout,
there is not enough address space to fit traces for all threads.
But since the trace space is going away with the new tsan runtime
(D112603), disable the test.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D113990
Use of gethostent provokes caching of some resources inside of libc.
They are freed in __libc_thread_freeres very late in thread lifetime,
after our ThreadFinish. __libc_thread_freeres calls free which
previously crashed in malloc hooks.
Fix it by setting ignore_interceptors for finished threads,
which in turn prevents malloc hooks.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D113989
Precisely specifying the unused parts of the bitfield is critical for
performance. If we don't specify them, compiler will generate code to load
the old value and shuffle it to extract the unused bits to apply to the new
value. If we specify the unused part and store 0 in there, all that
unnecessary code goes away (store of the 0 const is combined with other
constant parts).
I don't see a good way to ensure we cover all of u64 bits with fields.
So at least introduce named kUnusedBits consts and check that bits
sum up to 64.
Depends on D113978.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D113979
In the old runtime we used to use different number of trace parts
for C++ and Go to reduce trace memory consumption for Go.
But now it's easier and better to use smaller parts because
we already use minimal possible number of parts for C++ (3).
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D113978
man pthread_equal:
The pthread_equal() function is necessary because thread IDs
should be considered opaque: there is no portable way for
applications to directly compare two pthread_t values.
Depends on D113916.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D113919
pthread_setname_np does linear search over all thread descriptors
to map pthread_t to the thread descriptor. This has O(N^2) complexity
and becomes much worse in the new tsan runtime that keeps all ever
existed threads in the thread registry.
Replace linear search with direct access if pthread_setname_np
is called for the current thread (a very common case).
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D113916
We used to mmap C++ shadow stack as part of the trace region
before ed7f3f5bc9 ("tsan: move shadow stack into ThreadState"),
which moved the shadow stack into TLS. This started causing
timeouts and OOMs on some of our internal tests that repeatedly
create and destroy thousands of threads.
Allocate C++ shadow stack with mmap and small pages again.
This prevents the observed timeouts and OOMs.
But we now need to be more careful with interceptors that
run after thread finalization because FuncEntry/Exit and
TraceAddEvent all need the shadow stack.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D113786
Similar to how the other swift sections are registered by the ORC
runtime's macho platform, add the __swift5_types section, which contains
type metadata. Add a simple test that demonstrates that the swift
runtime recognized the registered types.
rdar://85358530
Differential Revision: https://reviews.llvm.org/D113811
Since glibc 2.34, dlsym does
1. malloc 1
2. malloc 2
3. free pointer from malloc 1
4. free pointer from malloc 2
These sequence was not handled by trivial dlsym hack.
This fixes https://bugs.llvm.org/show_bug.cgi?id=52278
Reviewed By: eugenis, morehouse
Differential Revision: https://reviews.llvm.org/D112588
The check was failing because it was matching against the end of the range, not
the start.
This bug wasn't causing the ORC-RT MachO TLV regression test to fail because
we were only logging deallocation errors (including TLV deregistration errors)
and not actually returning a failure code. This commit updates llvm-jitlink to
report the errors properly.
This change switches tsan to the new runtime which features:
- 2x smaller shadow memory (2x of app memory)
- faster fully vectorized race detection
- small fixed-size vector clocks (512b)
- fast vectorized vector clock operations
- unlimited number of alive threads/goroutimes
Depends on D112602.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D112603
Start the background thread only after fork, but not after clone.
For fork we did this always and it's known to work (or user code has adopted).
But if we do this for the new clone interceptor some code (sandbox2) fails.
So model we used to do for years and don't start the background thread after clone.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D113744
The compiler does not recognize HACKY_CALL as a call
(we intentionally hide it from the compiler so that it can
compile non-leaf functions as leaf functions).
To compensate for that hacky call thunk saves and restores
all caller-saved registers. However, it saves only
general-purposes registers and does not save XMM registers.
This is a latent bug that was masked up until a recent "NFC" commit
d736002e90 ("tsan: move memory access functions to a separate file"),
which allowed more inlining and exposed the 10-year bug.
Save and restore caller-saved XMM registers (all) as well.
Currently the bug manifests as e.g. frexp interceptor messes the
return value and the added test fails with:
i=8177 y=0.000000 exp=4
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D113742
Some compiler-rt tests are inherently incompatible with VE because..
* No consistent denormal support on VE. We skip denormal fp inputs in builtin tests.
* `madvise` unsupported on VE.
* Instruction alignment requirements.
Reviewed By: phosek
Differential Revision: https://reviews.llvm.org/D113093
Set the default memprof serialization format as binary. 9 tests are
updated to use print_text=true. Also fixed an issue with concatenation
of default and test specified options (missing separator).
Differential Revision: https://reviews.llvm.org/D113617
This change implements the raw binary format discussed in
https://lists.llvm.org/pipermail/llvm-dev/2021-September/153007.html
Summary of changes
* Add a new memprof option to choose binary or text (default) format.
* Add a rawprofile library which serializes the MIB map to profile.
* Add a unit test for rawprofile.
* Mark sanitizer procmaps methods as virtual to be able to mock them.
* Extend memprof_profile_dump regression test.
Differential Revision: https://reviews.llvm.org/D113317
The existing implementation uses a cache + eviction based scheme to
record heap profile information. This design was adopted to ensure a
constant memory overhead (due to fixed number of cache entries) along
with incremental write-to-disk for evictions. We find that since the
number to entries to track is O(unique-allocation-contexts) the overhead
of keeping all contexts in memory is not very high. On a clang workload,
the max number of unique allocation contexts was ~35K, median ~11K.
For each context, we (currently) store 64 bytes of data - this amounts
to 5.5MB (max). Given the low overheads for a complex workload, we can
simplify the implementation by using a hashmap without eviction.
Other changes:
* Memory map is dumped at the end rather than startup. The relative
order in the profile dump is unchanged since we no longer have evicted
entries at runtime.
* Added a test to check meminfoblocks are merged.
Differential Revision: https://reviews.llvm.org/D111676
Move the memprof MemInfoBlock struct to it's own header as requested
during the review of D111676.
Differential Revision: https://reviews.llvm.org/D113315
This change adds a ForEach method to the AddrHashMap class which can
then be used to iterate over all the key value pairs in the hash map.
I intend to use this in an upcoming change to the memprof runtime.
Added a unit test to cover basic insertion and the ForEach callback.
Differential Revision: https://reviews.llvm.org/D111368
Clone does not exist on Mac.
There are chances it will break on other OSes.
Enable it incrementally starting with Linux only,
other OSes can enable it later as needed.
Reviewed By: melver, thakis
Differential Revision: https://reviews.llvm.org/D113693
gtest uses clone for death tests and it needs the same
handling as fork to prevent deadlock (take runtime mutexes
before and release them after).
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D113677
Currently, SANITIZER_COMMON_SUPPORTED_OS is being used to enable many libraries.
Unfortunately this makes it impossible to selectively disable a library based on the OS.
This patch removes this limitation by adding a separate list of supported OSs for the lsan, ubsan, ubsan_minimal, and stats libraries.
Reviewed By: delcypher
Differential Revision: https://reviews.llvm.org/D113444
Entropic scheduling with exec-time option can be misled, if inputs
on the right path to become crashing inputs accidentally take more
time to execute before it's added to the corpus. This patch, by letting
more of such inputs added to the corpus (four inputs of size 7 to 10,
instead of a single input of size 2), reduces possibilities of being
influenced by timing flakiness.
A longer-term fix could be to reduce timing flakiness in the fuzzer;
one way could be to execute inputs multiple times and take average of
their execution time before they are added to the corpus.
Reviewed By: morehouse
Differential Revision: https://reviews.llvm.org/D113544
If async signal handler called when we MsanThread::Init
signal handler may trigger false reports.
I failed to reproduce this locally for a test.
Differential Revision: https://reviews.llvm.org/D113328
add tracing for loads and stores.
The primary goal is to have more options for data-flow-guided fuzzing,
i.e. use data flow insights to perform better mutations or more agressive corpus expansion.
But the feature is general puspose, could be used for other things too.
Pipe the flag though clang and clang driver, same as for the other SanitizerCoverage flags.
While at it, change some plain arrays into std::array.
Tests: clang flags test, LLVM IR test, compiler-rt executable test.
Reviewed By: morehouse
Differential Revision: https://reviews.llvm.org/D113447
Applying the same rules as for LLVM_BUILD_INSTRUMENTED build in the cmake files.
By having this patch, we are able to disable/enable instrument+coverage build
of the compiler-rt project when building instrumented LLVM.
Differential Revision: https://reviews.llvm.org/D108127
getting the tls base address. unlike linux arm64, the tpidr_el0 returns always 0 (aka unused)
thus using tpidrro_el0 instead clearing up the cpu id encoded in the lower bits.
Reviewed-By: yln
Differential Revision: https://reviews.llvm.org/D112866
D98452 introduced a mismatch between clang expectations for
builtin name for baremetal targets on arm. Fix it by
adding a case for baremetal. This now matches the output of
"clang -target armv7m-none-eabi -print-libgcc-file-name \
-rtlib=compiler-rt"
Reviewed By: mstorsjo
Differential Revision: https://reviews.llvm.org/D113357
https://sourceware.org/bugzilla/show_bug.cgi?id=22742
uc_mcontext.__reserved probably should not be considered user visible API but
unfortunate it is: it is the only way to access cpu states of some Linux
asm/sigcontext.h extensions. That said, the declaration may be
long double __reserved[256]; (used by musl)
instead of
unsigned char __reserved[4096] __attribute__((__aligned__(16))); (glibc)
to avoid dependency on a GNU variable attribute.
GCC introduced `__attribute__((mode(unwind_word)))` to work around
Cell Broadband Engine SPU (which was removed from GCC in 2019-09),
which is irrelevant to hwasan.
_Unwind_GetGR/_Unwind_GetCFA from llvm-project/libunwind don't use unwind_word.
Using _Unwind_Word can lead to build failures if libunwind's unwind.h is
preferred over unwind.h in the Clang resource directory (e.g. built with GCC).
Turning on `enable_noundef_analysis` flag allows better codegen by removing freeze instructions.
I modified clang by renaming `enable_noundef_analysis` flag to `disable-noundef-analysis` and turning it off by default.
Test updates are made as a separate patch: D108453
Reviewed By: eugenis
Differential Revision: https://reviews.llvm.org/D105169
[Clang/Test]: Rename enable_noundef_analysis to disable-noundef-analysis and turn it off by default (2)
This patch updates test files after D105169.
Autogenerated test codes are changed by `utils/update_cc_test_checks.py,` and non-autogenerated test codes are changed as follows:
(1) I wrote a python script that (partially) updates the tests using regex: {F18594904} The script is not perfect, but I believe it gives hints about which patterns are updated to have `noundef` attached.
(2) The remaining tests are updated manually.
Reviewed By: eugenis
Differential Revision: https://reviews.llvm.org/D108453
Resolve lit failures in clang after 8ca4b3e's land
Fix lit test failures in clang-ppc* and clang-x64-windows-msvc
Fix missing failures in clang-ppc64be* and retry fixing clang-x64-windows-msvc
Fix internal_clone(aarch64) inline assembly
Turning on `enable_noundef_analysis` flag allows better codegen by removing freeze instructions.
I modified clang by renaming `enable_noundef_analysis` flag to `disable-noundef-analysis` and turning it off by default.
Test updates are made as a separate patch: D108453
Reviewed By: eugenis
Differential Revision: https://reviews.llvm.org/D105169
It's called from ATTRIBUTE_NO_SANITIZE_MEMORY code.
It worked as expected if inlined and complained otherwise.
Reviewed By: eugenis
Differential Revision: https://reviews.llvm.org/D113323
Fixes:
sanitizer_stacktrace.h:212:5: warning: ISO C++ forbids braced-groups within expressions [-Wpedantic]
Differential Revision: https://reviews.llvm.org/D113292
Read each pointer in the argv and envp arrays before dereferencing
it; this correctly marks an error when these pointers point into
memory that has been freed.
Differential Revision: https://reviews.llvm.org/D113046
As pointed out in Bug 52371, the Solaris version of
`MemoryMappingLayout::Next` completely failed to handle `readlink` errors
or properly NUL-terminate the result.
This patch fixes this. Originally provided in the PR with slight
formatting changes.
Tested on `amd64-pc-solaris2.11`.
Differential Revision: https://reviews.llvm.org/D112998
I recently spent some extra time debugging a false positive because I
didn't realize the "real" tag was in the short granule. Adding the
short tag here makes it more obvious that we could be dealing with a
short granule.
Reviewed By: hctim, eugenis
Differential Revision: https://reviews.llvm.org/D112949
Previously we only applied it to the first one, which could allow
subsequent global tags to exceed the valid number of bits.
Reviewed By: hctim
Differential Revision: https://reviews.llvm.org/D112853
This is NOOP in x86_64.
On arch64 it avoids Data Memory Barrier with visible improvements on micro benchmarks.
Reviewed By: dvyukov
Differential Revision: https://reviews.llvm.org/D112391
MachOPlatform used to make an EPC-call (registerObjectSections) to register the
eh-frame and thread-data sections for each linked object with the ORC runtime.
Now that JITLinkMemoryManager supports allocation actions we can use these
instead of an EPC call. This saves us one EPC-call per object linked, and
manages registration/deregistration in the executor, rather than the controller
process. In the future we may use this to allow JIT'd code in the executor to
outlive the controller object while still being able to be cleanly destroyed.
Since the code for allocation actions must be available when the actions are
run, and since the eh-frame registration code lives in the ORC runtime itself,
this change required that MachO eh-frame support be split out of
macho_platform.cpp and into its own macho_ehframe_registration.cpp file that has
no other dependencies. During bootstrap we start by forcing emission of
macho_ehframe_registration.cpp so that eh-frame registration is guaranteed to be
available for the rest of the bootstrap process. Then we load the rest of the
MachO-platform runtime support, erroring out if there is any attempt to use
TLVs. Once the bootstrap process is complete all subsequent code can use all
features.
The ParseUnixMemoryProfile function is defined only for a subset
of platforms. Define the test for the same set of platforms.
Also disable the test for 32-bit platforms b/c the pointer
values used in the test are 64-bit and don't fit into 32-bit uptr.
Reported-by: Jan Svoboda (jansvoboda11)
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D112815
The new `Posix/mmap_write_exec.cpp` test FAILs on 32-bit Solaris/x86. This
happens because only `mmap` is intercepted, but not `mmap64` which is used
for largefile support.
Fixed by also intercepting `mmap64`.
Tested on `amd64-pc-solaris2.11`.
Differential Revision: https://reviews.llvm.org/D112810
I am hitting some cases where /proc/self/maps does not fit into 64MB.
256MB is lots of memory, but it's not radically more than the current 64MB.
Ideally we should read/parse these huge files incrementally,
but that's lots of work for a debugging/introspection interface.
So for now just bump the limit.
Depends on D112793.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D112794
ParseUnixMemoryProfile assumes well-formed input with \n at the end, etc.
It can over-read the input and crash on basically every line
in the case of malformed input.
ReadFileToBuffer has cap the max file size (64MB) and returns
truncated contents if the file is larger. Thus even if kernel behaves,
ParseUnixMemoryProfile crashes on too large /proc/self/smaps.
Fix input over-reading in ParseUnixMemoryProfile.
Depends on D112792.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D112793
stats_size argument is unnecessary in GetMemoryProfile and in the callback.
It just clutters code. The callback knowns how many stats to expect.
Depends on D112789.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D112790
Move parsing of /proc/self/smaps into a separate function
so that it can be tested.
Depends on D112788.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D112789
D112630 ("sanitizer_common: fix up onprint.cpp test")
added O_CREAT, but we also need O_TRUNC b/c the file
may not exist, or may exist as well.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D112788
WrapperFunctionCall represents a call to a wrapper function as a pair of a
target function (as an ExecutorAddr), and an argument buffer range (as an
ExecutorAddrRange). WrapperFunctionCall instances can be serialized via
SPS to send to remote machines (only the argument buffer address range is
copied, not any buffer content).
This utility will simplify the implementation of JITLinkMemoryManager
allocation actions in the ORC runtime.
tsan_rtl.cpp is huge and does lots of things.
Move everything related to memory access and tracing
to a separate tsan_rtl_access.cpp file.
No functional changes, only code movement.
Reviewed By: vitalybuka, melver
Differential Revision: https://reviews.llvm.org/D112625
Add `__c11_atomic_fetch_nand` builtin to language extensions and support `__atomic_fetch_nand` libcall in compiler-rt.
Reviewed By: theraven
Differential Revision: https://reviews.llvm.org/D112400
There's a lot of duplicated calls to find various compiler-rt libraries
from build of runtime libraries like libunwind, libc++, libc++abi and
compiler-rt. The compiler-rt helper module already implemented caching
for results avoid repeated Clang invocations.
This change moves the compiler-rt implementation into a shared location
and reuses it from other runtimes to reduce duplication and speed up
the build.
Differential Revision: https://reviews.llvm.org/D88458
We were writing a pointer to a selector string into the contents of a
string instead of overwriting the pointer to the string, leading to
corruption. This was causing non-deterministic failures of the
'trivial-objc-methods' test case.
Differential Revision: https://reviews.llvm.org/D112671
Enables the arm64 MachO platform, adds basic tests, and implements the
missing TLV relocations and runtime wrapper function. The TLV
relocations are just handled as GOT accesses.
rdar://84671534
Differential Revision: https://reviews.llvm.org/D112656
Commit D112602 ("sanitizer_common: tighten on_print hook test")
changed fopen to open in this test. fopen created the file
if if does not exist, but open does not. This was unnoticed
during local testing because lit is not hermetic and reuses
files from previous runs, but it started failing on bots.
Fix the open call.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D112630
The new tsan runtime does not support arbitrary forms
of recursing into the runtime from hooks.
Disable instrumentation of the hook and use write instead
of fwrite (calls malloc internally).
The new version still recurses (write is intercepted),
but does not fail now (the issue at hand was malloc).
Depends on D112601.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D112602
Gtest's EXPECT calls whole lot of libc functions
(mem*, malloc) even when EXPECT does not fail.
This does not play well with tsan runtime unit tests
b/c e.g. we call some EXPECTs with runtime mutexes locked.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D112601
Don't leak caller_pc var from the macro
(it's not supposed to be used by interceptors).
Use UNUSED instead of (void) cast.
Depends on D112540.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D112541
If the real function is not intercepted,
we are going to crash one way or another.
The question is just in the failure mode:
error message vs NULL deref. But the message
costs us a check in every interceptor and
they are not observed to be failing in real life
for a long time, also other sanitizers don't
have this check as well (also crash on
NULL deref if that happens).
Remove the check from non-debug mode.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D112540
This is a test-only failure. The test wrongly assumes that this gets us
a tagged pointer:
```
NSObject* num1 = @7;
assert(isTaggedPtr(num1));
```
However, on newer deployment targets that have “const data support” we
get a “normal” pointer to constant object.
Radar-Id: rdar://problem/83217293
All tsan interceptors check for initialization and/or initialize things
as necessary lazily, so we can pretend everything is initialized in the
COMMON_INTERCEPTOR_NOTHING_IS_INITIALIZED check to avoid double-checking
for initialization (this is only necessary for sanitizers that don't
handle initialization on common grounds).
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D112446
Print PC of the previous lock, not the current one.
The current one will be printed during unwind.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D112533
In TSan, we use the a function reference (`__tsan_stack_initialization`)
in a call to `StackTrace::GetNextInstructionPc(uptr pc)`. We sign
function pointers, so we need to strip the signature from this function
pointer.
Caused by: https://reviews.llvm.org/D111147
Radar-Id: rdar://problem/83940546
MutexSet is too large to be allocated on stack.
But we need local MutexSet objects in few places
and use various hacks to allocate them.
Add DynamicMutexSet helper that simplifies allocation
of such objects.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D112449
Trapping on CHECK failure makes it more convinient to use with gdb
(no need to set a breakpoint each time). Without a debugger attached
trap should terminate the program as well.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D112440
PPC64 bot failed with the following error.
The buildbot output is not particularly useful,
but looking at other similar tests, it seems
that there is something broken in free stacks on PPC64.
Use the same hack as other tests use to expect
an additional stray frame.
/home/buildbots/ppc64le-clang-lnt-test/clang-ppc64le-lnt/llvm/compiler-rt/test/tsan/free_race3.c:28:11: error: CHECK: expected string not found in input
// CHECK: Previous write of size 4 at {{.*}} by thread T1{{.*}}:
^
<stdin>:13:9: note: scanning from here
#1 main /home/buildbots/ppc64le-clang-lnt-test/clang-ppc64le-lnt/llvm/compiler-rt/test/tsan/free_race3.c:17:3 (free_race3.c.tmp+0x1012fab8)
^
<stdin>:17:2: note: possible intended match here
ThreadSanitizer: reported 1 warnings
^
Input file: <stdin>
Check file: /home/buildbots/ppc64le-clang-lnt-test/clang-ppc64le-lnt/llvm/compiler-rt/test/tsan/free_race3.c
-dump-input=help explains the following input dump.
Input was:
<<<<<<
.
.
.
8: Previous write of size 4 at 0x7ffff4d01ab0 by thread T1:
9: #0 Thread /home/buildbots/ppc64le-clang-lnt-test/clang-ppc64le-lnt/llvm/compiler-rt/test/tsan/free_race3.c:8:10 (free_race3.c.tmp+0x1012f9dc)
10:
11: Thread T1 (tid=3222898, finished) created by main thread at:
12: #0 pthread_create /home/buildbots/ppc64le-clang-lnt-test/clang-ppc64le-lnt/llvm/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:1001:3 (free_race3.c.tmp+0x100b9040)
13: #1 main /home/buildbots/ppc64le-clang-lnt-test/clang-ppc64le-lnt/llvm/compiler-rt/test/tsan/free_race3.c:17:3 (free_race3.c.tmp+0x1012fab8)
check:28'0 X~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ error: no match found
14:
check:28'0 ~
15: SUMMARY: ThreadSanitizer: data race /home/buildbots/ppc64le-clang-lnt-test/clang-ppc64le-lnt/llvm/compiler-rt/test/tsan/free_race3.c:19:3 in main
check:28'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
16: ==================
check:28'0 ~~~~~~~~~~~~~~~~~~~
17: ThreadSanitizer: reported 1 warnings
check:28'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
check:28'1 ? possible intended match
>>>>>>
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D112444
Add a test where a race with free is called during the free itself
(we only have tests where a race with free is caught during the other memory acces).
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D112433
Building Go programs with the current runtime fails with:
loadelf: race_linux_amd64: malformed elf file:
_ZZN6__tsan15RestoreAddrImpl5ApplyINS_11MappingGo48EEEmmE6ranges: invalid symbol binding 10
Go linker does not understand ELF in all its generality.
Don't use static const data in inline methods.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D112434
This is msan/dfsan data which does not need waste cache
of other sanitizers.
Depends on D111614.
Differential Revision: https://reviews.llvm.org/D111615
Based on post-commit review discussion on
2bd8493847 with Richard Smith.
Other uses of forcing HasEmptyPlaceHolder to false seem OK to me -
they're all around pointer/reference types where the pointer/reference
token will appear at the rightmost side of the left side of the type
name, so they make nested types (eg: the "int" in "int *") behave as
though there is a non-empty placeholder (because the "*" is essentially
the placeholder as far as the "int" is concerned).
This was originally committed in 277623f4d5
Reverted in f9ad1d1c77 due to breakages
outside of clang - lldb seems to have some strange/strong dependence on
"char [N]" versus "char[N]" when printing strings (not due to that name
appearing in DWARF, but probably due to using clang to stringify type
names) that'll need to be addressed, plus a few other odds and ends in
other subprojects (clang-tools-extra, compiler-rt, etc).
Reapply 5692ed0cce, but with the ORC runtime disabled explicitly on
CrossWinToARMLinux to match the other compiler-rt runtime libraries.
Differential Revision: https://reviews.llvm.org/D112229
---
Enable building the ORC runtime for 64-bit and 32-bit ARM architectures,
and for all Darwin embedded platforms (iOS, tvOS, and watchOS). This
covers building the cross-platform code, but does not add TLV runtime
support for the new architectures, which can be added independently.
Incidentally, stop building the Mach-O TLS support file unnecessarily on
other platforms.
Differential Revision: https://reviews.llvm.org/D112111
Add following interceptors on Linux: stat, lstat, fstat, fstatat.
This fixes use-of-uninitialized value on platforms with GLIBC 2.33+.
In particular: Arch Linux, Ubuntu hirsute/impish.
The tests should have also been failing during the release on the mentioned platforms, but I cannot find any related discussion.
Most likely, the regression was introduced by glibc commit [[ 8ed005daf0 | 8ed005daf0ab03e14250032 ]]:
all stat-family functions are now exported as shared functions.
Before, some of them (namely stat, lstat, fstat, fstatat) were provided as a part of libc_noshared.a and called their __xstat dopplegangers. This is still true for Debian Sid and earlier Ubuntu's. stat interceptors may be safely provided for them, no problem with that.
Closes https://github.com/google/sanitizers/issues/1452.
See also https://jira.mariadb.org/browse/MDEV-24841
Reviewed By: eugenis
Differential Revision: https://reviews.llvm.org/D111984
Enable building the ORC runtime for 64-bit and 32-bit ARM architectures,
and for all Darwin embedded platforms (iOS, tvOS, and watchOS). This
covers building the cross-platform code, but does not add TLV runtime
support for the new architectures, which can be added independently.
Incidentally, stop building the Mach-O TLS support file unnecessarily on
other platforms.
Differential Revision: https://reviews.llvm.org/D112111
Previously, when the fuzzing loop replaced an input in the corpus, it didn't update the execution time of the input. Therefore, some schedulers (e.g. Entropic) would adjust weights based on the incorrect execution time.
This patch updates the execution time of the input when replacing it.
Reviewed By: morehouse
Differential Revision: https://reviews.llvm.org/D111479
WrapperFunctionResult can already convey serialization errors as out-of-band
error values, so there's no need to wrap it in an Expected here. Removing the
wrapper simplifies the plumbing and call sites.
Allows us to use the small code model when we disable relocation
relaxation.
Reviewed By: eugenis
Differential Revision: https://reviews.llvm.org/D111344
There's a lot of duplicated calls to find various compiler-rt libraries
from build of runtime libraries like libunwind, libc++, libc++abi and
compiler-rt. The compiler-rt helper module already implemented caching
for results avoid repeated Clang invocations.
This change moves the compiler-rt implementation into a shared location
and reuses it from other runtimes to reduce duplication and speed up
the build.
Differential Revision: https://reviews.llvm.org/D88458
Instead of creating real threads for trace tests
create a new ThreadState in the main thread.
This makes the tests more unit-testy and will also
help with future trace tests that will need
more than 1 thread. Creating more than 1 real thread and
dispatching test actions across multiple threads in the
required deterministic order is painful.
This is resubmit of reverted D110546 with 2 changes:
1. The previous version patched ImitateTlsWrite to not
expect ThreadState to be allocated in TLS (the CHECK
failed for the fake test threads).
This added an ugly hack into production code and was still
logically wrong because we imitated write to the main
thread TLS/stack when we started the fake test thread
(which has nothing to do with the main thread TLS/stack).
This version uses ThreadType::Fiber instead of ThreadType::Regular
for the fake threads. This naturally makes ThreadStart skip
obtaining stack/tls and imitating writes to them.
2. This version still skips the tests on Darwin and PowerPC
to be on the safer side. Build bots reported failures for PowerPC
for the previous version.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D111156
Hide __llvm_profile_raw_version so as not to resolve reference from a
dependent shared object. Since libclang_rt.profile is added later in
the command line, a definition of __llvm_profile_raw_version is not
included if it is provided from an earlier object, e.g. from a shared
dependency.
This causes an extra dependence edge where if libA.so depends on libB.so
and both are coverage-instrumented, libA.so uses libB.so's definition of
__llvm_profile_raw_version. This leads to a runtime link failure if the
libB.so available at runtime does not provide this symbol (but provides
the other dependent symbols). Such a scenario can occur in Android's
mainline modules.
E.g.:
ld -o libB.so libclang_rt.profile-x86_64.a
ld -o libA.so -l B libclang_rt.profile-x86_64.a
libB.so has a global definition of __llvm_profile_raw_version. libA.so
uses libB.so's definition of __llvm_profile_raw_version. At runtime,
libB.so may not be coverage-instrumented (i.e. not export
__llvm_profile_raw_version) so runtime linking of libA.so will fail.
Marking this symbol as hidden forces each binary to use the definition
of __llvm_profile_raw_version from libclang_rt.profile.
Differential Revision: https://reviews.llvm.org/D111759
When built with hwasan, assume that the target architecture does not
support TBI. HWASan uses that byte for its own purpose, and changing it
breaks things.
Reviewed By: hctim
Differential Revision: https://reviews.llvm.org/D111842
On newer glibc, this test detects an extra match somewhere under
pthread_getattr_np. This results in Thread: lines getting spread out in
the report and failing to match the CHECKs.
Fix the CHECKs to allow this possibility.
Reviewed By: fmayer
Differential Revision: https://reviews.llvm.org/D111841
Add a default "/" prefix to the symbol search path in the
symbolization script. Without this, the binary itself is not considered
a valid source of symbol info.
Differential Revision: https://reviews.llvm.org/D111840
When LLVM_ENABLE_PER_TARGET_RUNTIME_DIR=on
Asan-i386-calls-Dynamic-Test and Asan-i386-inline-Dynamic-Test fail to
run on a x86_64 host. This is because asan's unit test lit files are
configured once, rather than per target arch as with the non-unit
tests. LD_LIBRARY_PATH ends up incorrect, and the tests try linking
against the x86_64 runtime which fails.
This changes the unit test CMake machinery to configure the default
and dynamic unit tests once per target arch, similar to the other asan
tests. Then the fix from https://reviews.llvm.org/D108859 is adapted
to the unit test Lit files with some modifications.
Fixes PR52158.
Differential Revision: https://reviews.llvm.org/D111756
Trace pointers accessed very rarely and don't need to
be in hot data.
Depends on D111613.
Reviewed By: dvyukov
Differential Revision: https://reviews.llvm.org/D111614
We are running `ls -lh` in gcov-execlp.c test in Posix folder.
However `-h` is not a POSIX option,ls on some POSIX system (eg: AIX)
may not support it.
This patch remove this option to avoid break.
Reviewed By: anhtuyen
Differential Revision: https://reviews.llvm.org/D111807
Fuchsia Clang code coverage pipeline started to use binary ids that are
embedded in profiles. This patch removes emitting symbolizer markup,
which is not necessary in the coverage pipeline anymore.
Differential Revision: https://reviews.llvm.org/D111674
This lets us reduce size of Node, similar to D111183 proposal.
Depends on D111610.
Reviewed By: dvyukov
Differential Revision: https://reviews.llvm.org/D111612
unlock corrupted backets by using s set by loop to nullptr.
Also StackDepot supports iterating without locking.
Reviewed By: dvyukov
Differential Revision: https://reviews.llvm.org/D111599
Reducing the number of iterations in that test with D111342 helped,
but the failure still occured flakily when the test is ran as part
of a large test suite.
Reducing further the number of iterations might not be good enough,
so we will skip the test if the `max_map_count` variable can be
read, and if lower than a given threshold.
Differential Revision: https://reviews.llvm.org/D111465
Test sometimes fails on buildbot (after two non-Origins executions):
/usr/bin/ld: warning: Cannot export local symbol 'dfsan_flush'
RSS at start: 4620, after mmap: 107020, after mmap+set label: 209424, after fixed map: 4624, after another mmap+set label: 209424, after munmap: 4624
/usr/bin/ld: warning: Cannot export local symbol 'dfsan_flush'
RSS at start: 4620, after mmap: 107020, after mmap+set label: 209424, after fixed map: 4624, after another mmap+set label: 209424, after munmap: 4624
/usr/bin/ld: warning: Cannot export local symbol 'dfsan_flush'
RSS at start: 4620, after mmap: 107020, after mmap+set label: 317992, after fixed map: 10792, after another mmap+set label: 317992, after munmap: 10792
release_shadow_space.c.tmp: /b/sanitizer-x86_64-linux/build/llvm-project/compiler-rt/test/dfsan/release_shadow_space.c:91: int main(int, char **): Assertion `after_fixed_mmap <= before + delta' failed.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D111522
Teresa Johnson