Commit Graph

10615 Commits

Author SHA1 Message Date
Matt Morehouse 373d6ed7cf [MSan] Add functions to enable/disable interceptor checks.
Summary:
The motivation for this change is to make libFuzzer+MSan possible
without instrumenting libFuzzer.

See https://github.com/google/sanitizers/issues/958.

Reviewers: eugenis

Reviewed By: eugenis

Subscribers: llvm-commits, kcc

Differential Revision: https://reviews.llvm.org/D48890

llvm-svn: 336447
2018-07-06 17:10:51 +00:00
Yvan Roux becbb87553 [libFuzzer] Disable hanging tests on AArch64
Disable problematic tests which broke AArch64 bots.
Details available in Bugzilla: https://bugs.llvm.org/show_bug.cgi?id=38034

Differential Revision: https://reviews.llvm.org/D49011

llvm-svn: 336446
2018-07-06 17:06:01 +00:00
Kostya Kortchinsky 2efb847b6f [scudo] Add some logs for Android
Summary:
Namely, set the abort message, and allow to write the message to syslog if the
option is enabled.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D48902

llvm-svn: 336445
2018-07-06 16:50:12 +00:00
Marco Castelluccio 32d18beb8c Make __gcov_flush flush counters for all shared libraries
Summary:
This will make the behavior of __gcov_flush match the GCC behavior.

I would like to rename __gcov_flush to __llvm_gcov_flush (in case of programs linking to libraries built with different compilers), but I guess we can't for compatibility reasons.

Reviewers: davidxl

Reviewed By: davidxl

Subscribers: samsonov, vitalybuka, pcc, kcc, junbuml, glider, fhahn, eugenis, dvyukov, davidxl, srhines, chh, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D48538

llvm-svn: 336365
2018-07-05 15:52:59 +00:00
Yvan Roux 01c0d860f3 Revert "[CMake] Run libFuzzer tests with check-all."
Revert due to AArch64 bots breakage, upstream PR raised to track the
issue: https://bugs.llvm.org/show_bug.cgi?id=38034

llvm-svn: 336341
2018-07-05 12:19:03 +00:00
George Karpenkov aeeac6d41c [libFuzzer] [NFC] Inline static variable to avoid the linker warning.
Differential Revision: https://reviews.llvm.org/D48650

llvm-svn: 336238
2018-07-04 00:37:45 +00:00
Kostya Serebryany 51ddb88300 [libFuzzer] add one more value profile metric, under a flag (experimental)
llvm-svn: 336234
2018-07-03 22:33:09 +00:00
Kostya Serebryany dcac0a3b5e [libFuzzer] remove stale code, as suggested in https://reviews.llvm.org/D48800
llvm-svn: 336230
2018-07-03 21:22:44 +00:00
Kostya Serebryany 0bc9143de8 [libFuzzer] add a tiny and surprisingly hard puzzle
llvm-svn: 336229
2018-07-03 21:17:55 +00:00
Kostya Kortchinsky e64a81475c [scudo] Get rid of builtin-declaration-mismatch warnings
Summary:
The C interceptors were using `SIZE_T` defined in the interception library as
a `__sanitizer::uptr`. On some 32-bit platforms, this lead to the following
warning:
```
warning: declaration of ‘void* malloc(SIZE_T)’ conflicts with built-in declaration ‘void* malloc(unsigned int)’ [-Wbuiltin-declaration-mismatch]
INTERCEPTOR_ATTRIBUTE void *malloc(SIZE_T size) {
```
`__sanitizer::uptr` is indeed defined as an `unsigned long` on those.

So just include `stddef.h` and use `size_t` instead.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D48885

llvm-svn: 336221
2018-07-03 19:03:46 +00:00
Kostya Kortchinsky 02827792b5 [scudo] Enable Scudo on PPC64
Summary:
In conjunction with the clang side change D48833, this will enable Scudo on
PPC64. I tested `check-scudo` on a powerpc64le box and everything passes.

Reviewers: eugenis, alekseyshl

Reviewed By: alekseyshl

Subscribers: mgorny, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D48834

llvm-svn: 336213
2018-07-03 16:09:18 +00:00
Kostya Kortchinsky 4bc7f3d4d6 [scudo] Enable Scudo memory hooks for Fuchsia.
Summary:
    It would be useful for Flutter apps, especially, to be able to use
    malloc hooks to debug memory leaks on Fuchsia. They're not able to do
    this right now, so it'd be a nice bonus to throw in with the Scudo
    switchover.


Reviewers: cryptoad, alekseyshl

Reviewed By: cryptoad

Differential Revision: https://reviews.llvm.org/D48618

llvm-svn: 336139
2018-07-02 19:48:01 +00:00
Fangrui Song 5f27e0c021 [asan] Fix deadlock issue on FreeBSD, caused by use of .preinit_array in rL325240
Summary:
Without this patch,
clang -fsanitize=address -xc =(printf 'int main(){}') -o a; ./a => deadlock in __asan_init>AsanInitInternal>AsanTSDInit>...>__getcontextx_size>_rtld_bind>rlock_acquire(rtld_bind_lock, &lockstate)

libexec/rtld-elf/rtld.c
  wlock_acquire(rtld_bind_lock, &lockstate);
  if (obj_main->crt_no_init)
    preinit_main(); // unresolved PLT functions cannot be called here

lib/libthr/thread/thr_rtld.c
  uc_len = __getcontextx_size(); // unresolved PLT function in libthr.so.3

check-xray tests currently rely on .preinit_array so we special case in
xray_init.cc

Subscribers: srhines, kubamracek, krytarowski, delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D48806

llvm-svn: 336067
2018-07-01 17:52:41 +00:00
David Carlier c33f35ff0f [UBsan] Enable subset of unit tests for OpenBSD
Reviewers: kubamracek, krytarowski

Reviewed By: krytarowski

Differential Revision: https://reviews.llvm.org/D48805

llvm-svn: 336053
2018-06-30 21:35:05 +00:00
Fangrui Song da891dc6b4 [asan] Use MADV_NOCORE for use_madv_dontdump on FreeBSD.
Currently in FreeBSD 12.0-CURRENT with trunk clang+compiler-rt, faulty -fsanitize=address executable hangs at 'urdlck' state.

Ka Ho Ng has verified that by backporting this to llvm 6.0.1, with use_madv_dontdump=1, shadow memory is not dumped.

ASAN_OPTIONS=abort_on_error=1:disable_coredump=0:use_madv_dontdump=1 ./a

Reviewers: dimitry, kcc, dvyukov, emaste, khng300

Subscribers: kubamracek, delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D48257

llvm-svn: 336046
2018-06-30 08:27:48 +00:00
Chih-Hung Hsieh 7222e8e30b [profile] Add llvm_gcov_flush to be called outside a shared library
__gcov_flush is hidden.
For applications to dump profiling data of selected .so files,
they can use dlsym to find and call llvm_gcov_flush in each .so file.

Differential Revision: https://reviews.llvm.org/D45454

llvm-svn: 336019
2018-06-29 21:45:55 +00:00
Evgeniy Stepanov d77e7ba74e [cfi] Use __builtin version of __clear_cache.
__builtin___clear_cache is also present on X86 and does the right thing (i.e. nop) there.

llvm-svn: 335997
2018-06-29 17:22:58 +00:00
Evgeniy Stepanov ac9b6adf77 [scs] Disable negative test in shadowcallstack.
The test checks that scs does NOT work correctly w/o runtime support.
That's a strange thing to test, and it is also flaky, because things
may just work if x18 happens to point to a writable page.

llvm-svn: 335982
2018-06-29 15:16:45 +00:00
Kostya Kortchinsky e92d7878e5 [scudo] Add some runtime tests for the minimal runtime
Summary:
As well as some tests to ensure that various combinations of the clang command
line flags work (shared/static/minimal).

Reviewers: eugenis, alekseyshl, vitalybuka

Reviewed By: vitalybuka

Subscribers: srhines, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D48553

llvm-svn: 335981
2018-06-29 14:56:25 +00:00
Evgeniy Stepanov 6d310651a2 [cfi] Reset i-cache after copying code in test.
Fixes intermittent crashes on Android.
Also add PROT_READ to the code mapping for no real reason.

llvm-svn: 335943
2018-06-29 00:38:26 +00:00
Evgeniy Stepanov 3702f91287 [asan] Change 3 tests from XFAIL to UNSUPPORTED.
The failure in https://github.com/google/sanitizers/issues/981 is flaky.

llvm-svn: 335941
2018-06-28 23:27:55 +00:00
Evgeniy Stepanov 730c985d56 [asan] Disable 3 tests on Android O+.
https://github.com/google/sanitizers/issues/981

llvm-svn: 335937
2018-06-28 22:54:55 +00:00
Evgeniy Stepanov e40ae2e3a2 [asan] Fix one more test on Android.
This change makes it pass both on libstdc++ and libc++-based toolchains.

The error was:
error: function-like macro '__GLIBC_PREREQ' is not defined
                               ^

llvm-svn: 335936
2018-06-28 22:54:42 +00:00
Evgeniy Stepanov 59ef3dbbc6 [asan] un-XFAIL one test
NDK r16 provides glob.h, which makes this test pass.
Supporting different test outcomes depending on the version of NDK
add unnecessary complexity to the test harness. IMHO, it's OK to require
the latest stable release.

llvm-svn: 335935
2018-06-28 22:54:30 +00:00
Evgeniy Stepanov 71dc39823c [fuzzer] Don't run tests on Android.
Need better build system support.

llvm-svn: 335933
2018-06-28 22:16:59 +00:00
Alex Shlyapnikov fb1644835b [TSan] More detailed error message on failed sahdow memory madvise
Summary:
Report errno value on failed shadow memory madvise attempt and add a
hint message with the possible workaround.

Reviewers: eugenis

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D48668

llvm-svn: 335928
2018-06-28 21:38:34 +00:00
David Carlier a8c5041bdf [UBsan] minimal unit tests removing OpenBSD spacial case
The OpenBSD driver now handles sanitisers with ld.lld automatically.

Reviewers: vitalybuka

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D48740

llvm-svn: 335923
2018-06-28 21:10:52 +00:00
Vedant Kumar c8735fbb4c [Darwin] Add an integration test for PGO + symbol exports
rdar://41470205

llvm-svn: 335891
2018-06-28 17:53:43 +00:00
Kuba Mracek df93fe46e5 Skip building TSan on platforms where there are no 64-bit architectures.
llvm-svn: 335873
2018-06-28 16:43:43 +00:00
Haojian Wu d93a1e5271 Correct the test modified in rL335777.
llvm-svn: 335865
2018-06-28 15:24:34 +00:00
Petr Hosek 887f26d470 Support for multiarch runtimes layout
This change adds a support for multiarch style runtimes layout, so in
addition to the existing layout where runtimes get installed to:

lib/clang/$version/lib/$os

Clang now allows runtimes to be installed to:

lib/clang/$version/$target/lib

This also includes libc++, libc++abi and libunwind; today those are
assumed to be in Clang library directory built for host, with the
new layout it is possible to install libc++, libc++abi and libunwind
into the runtime directory built for different targets.

The use of new layout is enabled by setting the
LLVM_ENABLE_RUNTIME_TARGET_DIR CMake variable and is supported by both
projects and runtimes layouts. The runtimes CMake build has been further
modified to use the new layout when building runtimes for multiple
targets.

Differential Revision: https://reviews.llvm.org/D45604

llvm-svn: 335809
2018-06-28 03:11:52 +00:00
George Karpenkov 348dd9ddd1 [libFuzzer] [Tests] [NFC] Change seed for reduce_inputs.test
Differential Revision: https://reviews.llvm.org/D48686

llvm-svn: 335802
2018-06-28 00:39:50 +00:00
George Karpenkov 0cad9cb9a4 [libFuzzer] [NFC] [Tests] Mark signal handling tests as UNSUPPORTED on Darwin
Under load, these tests tend to fail sporadically on our bots.
In my understanding, the signal handling is not guaranteed to happen
within 2 seconds, and the test is inherently flaky.

llvm-svn: 335792
2018-06-27 23:11:24 +00:00
Petr Hosek 61574fdb06 [sanitizer] zx_cprng_draw no longer returns any value
Remove the return value check.

Differential Revision: https://reviews.llvm.org/D48671

llvm-svn: 335790
2018-06-27 21:25:21 +00:00
Vlad Tsyrklevich 600a2df289 Another shot at fixing android r335644 failure
The android buildbot moves the build outputs to a different directory
and rewrites the executable path, the DSO passed as an argument does not
get re-written. Use rpaths to load the DSO the same way the
test/cfi/cross-dso/ tests do and test the DSO name differently.

llvm-svn: 335777
2018-06-27 20:00:55 +00:00
Matt Morehouse 520748f01e [UBSan] Add silence_unsigned_overflow flag.
Summary:
Setting UBSAN_OPTIONS=silence_unsigned_overflow=1 will silence all UIO
reports.  This feature, combined with
-fsanitize-recover=unsigned-integer-overflow, is useful for providing
fuzzing signal without the excessive log output.

Helps with https://github.com/google/oss-fuzz/issues/910.

Reviewers: kcc, vsk

Reviewed By: vsk

Subscribers: vsk, kubamracek, Dor1s, llvm-commits

Differential Revision: https://reviews.llvm.org/D48660

llvm-svn: 335762
2018-06-27 18:24:46 +00:00
Petr Hosek 3209b28aa9 [sanitizer] zx_cprng_draw no longer takes the output argument
The zx_cprng_draw system call no longer takes the output argument.

Differential Revision: https://reviews.llvm.org/D48657

llvm-svn: 335755
2018-06-27 16:49:37 +00:00
Vedant Kumar 7eb8119798 [asan] Relax the null_deref.cc test even further
On Darwin/x86_64, asan may report the crashing line of NullDeref as line
19 (i.e the closing brace of the function), whereas on other targets we
see line 15 ("ptr[10]++").

The optimized debug info here isn't reliable enough to check.

rdar://problem/41526369

llvm-svn: 335747
2018-06-27 16:06:15 +00:00
Vedant Kumar 43eec242e0 [asan] Relax a test which depends on optimized debug info
On some ARM platforms this test depends on debug locations being
present on constant materialization code, which was eliminated in
r335497.

Relax the test to allow two outcomes: the backtrace either contains the
right line numbers, or no line numbers.

llvm-svn: 335741
2018-06-27 14:57:59 +00:00
Dan Liew bb78eef6b6 [CMake] Tidy up the organisation of compiler-rt when configured as a standalone
build with an IDE (e.g. Xcode) as the generator.

Previously the global `USE_FOLDERS` property wasn't set in standalone
builds leading to existing settings of FOLDER not being respected.

In addition to this there were several targets that appeared at the top
level that were not interesting and clustered up the view. These have
been changed to be displayed in "Compiler-RT Misc".

Now when an Xcode project is generated from a standalone compiler-rt
build the project navigator is much less cluttered. The interesting
libraries should appear in "Compiler-RT Libraries" in the IDE.

Differential Revision: https://reviews.llvm.org/D48378

llvm-svn: 335728
2018-06-27 12:56:34 +00:00
Vlad Tsyrklevich 5417225745 Fix another bot broken by r335644
The android sanitizer bot can't resolve the function name in the DSO and
it's not relevant to the test.

llvm-svn: 335693
2018-06-27 00:31:17 +00:00
Vlad Tsyrklevich ee59c5e922 Fix test broken by r335644
llvm-svn: 335657
2018-06-26 20:21:33 +00:00
Vlad Tsyrklevich e745cf9bf3 CFI: Print DSO names for failed cross-DSO icalls
Reviewers: pcc

Reviewed By: pcc

Subscribers: kubamracek, delcypher, llvm-commits, kcc, #sanitizers

Differential Revision: https://reviews.llvm.org/D48583

llvm-svn: 335644
2018-06-26 18:51:04 +00:00
Matt Morehouse 0948bc2086 [libFuzzer] Do not turn unittest warnings into errors.
Some warnings originating from googletest were causing bots to fail
while bulding unit tests.  The sanitizers address this issue by not
using -Werror.  We adopt this approach for libFuzzer.

llvm-svn: 335640
2018-06-26 18:37:37 +00:00
Matt Morehouse 0ab788e271 [libFuzzer] Mark several tests UNSUPPORTED for aarch64.
Now that check-fuzzer runs as part of check-all, some aarch64 bots had
tests failing.

llvm-svn: 335639
2018-06-26 18:37:09 +00:00
Peter Collingbourne f99bc0b2f6 Mark mfcall.cpp as UNSUPPORTED: win32.
llvm-svn: 335573
2018-06-26 02:59:22 +00:00
Peter Collingbourne e44acadf6a Implement CFI for indirect calls via a member function pointer.
Similarly to CFI on virtual and indirect calls, this implementation
tries to use program type information to make the checks as precise
as possible.  The basic way that it works is as follows, where `C`
is the name of the class being defined or the target of a call and
the function type is assumed to be `void()`.

For virtual calls:
- Attach type metadata to the addresses of function pointers in vtables
  (not the functions themselves) of type `void (B::*)()` for each `B`
  that is a recursive dynamic base class of `C`, including `C` itself.
  This type metadata has an annotation that the type is for virtual
  calls (to distinguish it from the non-virtual case).
- At the call site, check that the computed address of the function
  pointer in the vtable has type `void (C::*)()`.

For non-virtual calls:
- Attach type metadata to each non-virtual member function whose address
  can be taken with a member function pointer. The type of a function
  in class `C` of type `void()` is each of the types `void (B::*)()`
  where `B` is a most-base class of `C`. A most-base class of `C`
  is defined as a recursive base class of `C`, including `C` itself,
  that does not have any bases.
- At the call site, check that the function pointer has one of the types
  `void (B::*)()` where `B` is a most-base class of `C`.

Differential Revision: https://reviews.llvm.org/D47567

llvm-svn: 335569
2018-06-26 02:15:47 +00:00
David Carlier 7a0bf90dcf [UBsan] Enable ubsan minimal unit tests on OpenBSD
OpenBSD needs lld linker for sanitisers.
Disabling lint checking as some symbols cannot be defined and block the proper unit tests launch.

Reviewers: lebedev.ri, vitalybuka

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D48528

llvm-svn: 335524
2018-06-25 21:10:25 +00:00
Vedant Kumar 487bef376b [ubsan] Mark a test case as unsupported on Windows
__ubsan_on_report isn't defined as weak, and redefining it in a test is
not supported on Windows.

See the error message here: https://reviews.llvm.org/D48446

llvm-svn: 335523
2018-06-25 20:56:23 +00:00
Matt Morehouse 3077c82498 [CMake] Run libFuzzer tests with check-all.
Reviewers: kcc

Reviewed By: kcc

Subscribers: llvm-commits, mgorny

Differential Revision: https://reviews.llvm.org/D48200

llvm-svn: 335519
2018-06-25 20:30:03 +00:00
Alex Shlyapnikov d42edb28d0 [HWASan] Initalize shadow earler.
Summary:
Initialize shadow memory before calling more libc functions to allow
for HWASan-instrumented libc.

Reviewers: eugenis

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D48551

llvm-svn: 335502
2018-06-25 17:27:13 +00:00
Matt Morehouse 988f2da613 [libFuzzer] Use Vector rather than std::vector.
llvm-svn: 335487
2018-06-25 15:59:24 +00:00
Vedant Kumar c835306ac5 [ubsan] Fix __ubsan_on_report interface definition
Speculative fix for the interface definition of __ubsan_on_report for
the Windows bots:

  http://lab.llvm.org:8011/builders/sanitizer-windows/builds/30528
  lib\ubsan\ubsan_interface.inc(55): error C2065: '__ubsan_on_report':
  undeclared identifier

INTERCEPT_SANITIZER_WEAK_FUNCTION was the wrong macro to use to begin
with because __ubsan_on_report isn't weak. Reading through that macro,
it's still not clear to me why there is an undefined reference, though,
because it appears to define a dummy __ubsan_on_report shim.

llvm-svn: 335383
2018-06-22 20:15:33 +00:00
Vedant Kumar 059d20360a [ubsan] Add support for reporting diagnostics to a monitor process
Add support to the ubsan runtime for reporting diagnostics to a monitor
process (e.g a debugger).

The Xcode IDE uses this by setting a breakpoint on __ubsan_on_report and
collecting diagnostic information via __ubsan_get_current_report_data,
which it then surfaces to users in the editor UI.

Testing for this functionality already exists in upstream lldb, here:
lldb/packages/Python/lldbsuite/test/functionalities/ubsan

Apart from that, this is `ninja check-{a,ub}san` clean.

Differential Revision: https://reviews.llvm.org/D48446

llvm-svn: 335371
2018-06-22 17:21:17 +00:00
Dmitry Vyukov ac5fa6605c tsan: fix deficiency in MutexReadOrWriteUnlock
MutexUnlock uses ReleaseStore on s->clock, which is the right thing to do.
However MutexReadOrWriteUnlock for writers uses Release on s->clock.
Make MutexReadOrWriteUnlock also use ReleaseStore for consistency and performance.
Unfortunately, I don't think any test can detect this as this only potentially
affects performance.

llvm-svn: 335322
2018-06-22 08:27:52 +00:00
Kostya Kortchinsky 307c2eb94f [scudo] Add a minimal runtime for -fsanitize-minimal-runtime compatibility
Summary:
This patch follows D48373.

The point is to be able to use Scudo with `-fsanitize-minimal-runtime`. For that
we need a runtime that doesn't embed the UBSan one. This results in binaries
that can be compiled with `-fsanitize=scudo,integer -fsanitize-minimal-runtime`.

Reviewers: eugenis

Reviewed By: eugenis

Subscribers: mgorny, delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D48377

llvm-svn: 335296
2018-06-21 21:48:04 +00:00
George Karpenkov b085ca5095 [libFuzzer] Filter architectures for testing on Apple platforms.
This is done in all other sanitizers, and was missing on libFuzzer.

llvm-svn: 335290
2018-06-21 21:19:43 +00:00
George Karpenkov e1a50ca120 [libFuzzer] Provide more descriptive names for testing targets.
llvm-svn: 335289
2018-06-21 21:19:24 +00:00
Kuba Mracek 14c786a960 [tsan] Use DARWIN_osx_LINK_FLAGS when building unit tests to match ASan behavior.
llvm-svn: 335265
2018-06-21 18:00:51 +00:00
Kuba Mracek bb9dedfa8a [sanitizer] Stop running tests against 32-bit iOS simulator
llvm-svn: 335245
2018-06-21 15:21:24 +00:00
David Carlier 7d796ececd [TSan] fix build and couple of unit tests on FreeBSD
Similarly to Msan adding -pie provokes linkage issue, was spotted with pie_test.cc
Set to XFAIL for couple of unit tests.

Reviewers: vitalybuka, krytarowski, dim

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D48317

llvm-svn: 335166
2018-06-20 20:17:44 +00:00
David Carlier 8979d44d31 [Lsan] intercept thr_exit on FreeBSD
Intercepts thr_exit call on FreeBSD.
Disable pthread key workflow.
The pthread key create approach does not function under FreeBSD as the libpthread is not initialised enough at this stage.

Reviewers: vitalybuka, krytarowski, dim

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D48268

llvm-svn: 335164
2018-06-20 20:13:25 +00:00
Alex Shlyapnikov 7e067ab1af [Sanitizers] Remove OOM/BadRequest allocator error handling policies.
Summary:
Remove the generic error nadling policies and handle each allocator error
explicitly. Although more verbose, it allows for more comprehensive, precise
and actionable allocator related failure reports.

This finishes up the series of changes of the particular sanitizer
allocators, improves the internal allocator error reporting and removes
now unused policies.

Reviewers: vitalybuka, cryptoad

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D48328

llvm-svn: 335147
2018-06-20 17:10:33 +00:00
Kuba Mracek dd51ef2867 Fix-up for r335123: Set default apple_platform to "osx".
llvm-svn: 335133
2018-06-20 14:59:16 +00:00
Kuba Mracek badaa15c88 [sanitizer] Unify and generalize Apple platforms in CMake and lit test configs
There's more platforms than just "ios" and "iossim" that we should support, and adding more lit config variables for each platform isn't great. Let's generalize and have a single value that determines what the platform under test is.

Differential Revision: https://reviews.llvm.org/D48309

llvm-svn: 335123
2018-06-20 13:33:42 +00:00
Reid Kleckner 3f573cab15 [asan] Add flushes to try to fix test
llvm-svn: 335089
2018-06-20 01:09:51 +00:00
Reid Kleckner b3ba1cc82a [asan] Add Windows test for handle_segv and SetUnhandledExceptionFilter
llvm-svn: 335087
2018-06-20 00:45:54 +00:00
Fangrui Song 6fa3bf9588 [XRay] Fix error message. NFC
Reviewers: dberris

Subscribers: delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D48313

llvm-svn: 335055
2018-06-19 15:39:19 +00:00
Kostya Kortchinsky 63c33b1c9b [scudo] Move noinline functions definitions out of line
Summary:
Mark `isRssLimitExceeded` as `NOINLINE`, and move it's definition as well as
the one of `performSanityChecks` out of the class definition, as requested.

Reviewers: filcab, alekseyshl

Reviewed By: alekseyshl

Subscribers: delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D48228

llvm-svn: 335054
2018-06-19 15:36:30 +00:00
Fangrui Song cad40b1a39 [XRay] rm GLOB || true -> rm -f
Summary: `rm -f` does not write diagnostic message when there is no file argument.

Reviewers: dberris

Subscribers: delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D48311

llvm-svn: 335025
2018-06-19 07:11:33 +00:00
Francis Visoiu Mistrih 972a462a7e Fix bots after r334981
llvm-svn: 335013
2018-06-19 02:56:24 +00:00
Reid Kleckner b8a5ee2b00 [asan] Avoid deadlock when initializing the symbolizer CHECK fails
llvm-svn: 335007
2018-06-19 00:36:47 +00:00
Chris Bieneman feb4537b44 Fixing os_version_check.c to be actual C source
The initial implementaiton was using the C++ typeof keyword. This
causes the compiler to spew warnings unnecissarilly. This patch removes
the uses of typeof and replaces them with explicit typedefs of the
function types.

llvm-svn: 334981
2018-06-18 20:56:19 +00:00
Alex Shlyapnikov c75d47b52d [TSan] Report proper error on allocator failures instead of CHECK(0)-ing
Summary:
Following up on and complementing D44404 and other sanitizer allocators.

Currently many allocator specific errors (OOM, for example) are reported as
a text message and CHECK(0) termination, no stack, no details, not too
helpful nor informative. To improve the situation, detailed and structured
common errors were defined and reported under the appropriate conditions.

Common tests were generalized a bit to cover a slightly different TSan
stack reporting format, extended to verify errno value and returned
pointer value check is now explicit to facilitate debugging.

Reviewers: dvyukov

Subscribers: srhines, kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D48087

llvm-svn: 334975
2018-06-18 20:03:31 +00:00
Francis Visoiu Mistrih 1197f01bb9 [sanitizer] Guard call to internal_lseek when SANITIZER_MAC is true
r334881 breaks macOS bots because internal_lseek is not defined (neither
used on macOS):
http://green.lab.llvm.org/green/job/clang-stage1-configure-RA/46240/consoleFull.

See discussion from r334881: https://reviews.llvm.org/rL334881

llvm-svn: 334944
2018-06-18 13:53:51 +00:00
Paul Robinson 7555c589af Update copyright year to 2018.
llvm-svn: 334936
2018-06-18 12:22:17 +00:00
Petr Hosek 068246886a [Fuzzer] Set an explicit libc++ dependency when needed
On targets that don't link internal libc++ (Fuchsia and Linux) but
use libc++ as their C++ library and libFuzzer is being built using
the just built compiler together with libc++ as part of runtimes, we
need an explicit dependency from libFuzzer object library to libc++ to
make sure the headers are available by the time we start building
libFuzzer.

Differential Revision: https://reviews.llvm.org/D48261

llvm-svn: 334928
2018-06-18 08:01:57 +00:00
Vitaly Buka 7fd0d388e0 [sanitizer] Suppress unused function warning
llvm-svn: 334923
2018-06-18 05:55:44 +00:00
Vitaly Buka 87ea4487f7 [sanitizer] Fix tsan GO build
llvm-svn: 334914
2018-06-17 17:58:08 +00:00
Vitaly Buka 4647c8b93d [sanitizer] Fix s390 build after r334900
llvm-svn: 334913
2018-06-17 17:40:38 +00:00
Vitaly Buka 755f0e8174 [sanitizer] Use confstr to check libc version in InitTlsSize
Reviewers: Lekensteyn, jakubjelinek

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D48265

llvm-svn: 334912
2018-06-17 17:31:22 +00:00
Jonas Hahnfeld 71befacd6c Revert "[CMake] Use a different source depending on C++ support"
This reverts commit r332924 and followup r332936 silencing a warning.

The change breaks the build on x86 if there is no 32-bit version of the
C++ libraries, see discussion in https://reviews.llvm.org/D47169.

llvm-svn: 334903
2018-06-17 09:51:33 +00:00
Vitaly Buka 4d6b33c510 [fuzzer] Python 3 print fixes
llvm-svn: 334902
2018-06-17 09:11:56 +00:00
Vitaly Buka b213a345e1 [fuzzer] Fix collect_data_flow.py for python 3
llvm-svn: 334901
2018-06-17 08:41:56 +00:00
Vitaly Buka b957f99d57 [sanitizer] Use const char* in internal_simple_strtoll
llvm-svn: 334900
2018-06-17 08:41:45 +00:00
Fangrui Song 87bcd4abef [sanitizer_common] Fix windows build caused by r334881
llvm-svn: 334884
2018-06-16 05:05:19 +00:00
Vitaly Buka b0a3e6f127 [asan] Enable fgets_fputs test on Android
"echo data" didn't work because %run on android executes test on the device
when lit shell command on the host system.

https://github.com/google/sanitizers/issues/952

llvm-svn: 334883
2018-06-16 04:01:08 +00:00
Fangrui Song d40fc6019b [sanitizer_common] Use O_TRUNC for WrOnly access mode.
Summary: Otherwise if the file existed and was larger than the write size before the OpenFile call, the file will not be truncated and contain garbage in trailing bytes.

Reviewers: glider, kcc, vitalybuka

Subscribers: kubamracek, delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D48250

llvm-svn: 334881
2018-06-16 03:32:59 +00:00
Vitaly Buka 572e442b6d [asan] Move long-object-path.cc test to Linux
Reviewers: Lekensteyn

Reviewed By: Lekensteyn

Subscribers: srhines, kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D48186

llvm-svn: 334866
2018-06-15 21:25:42 +00:00
Matt Morehouse 668dd03c8d [libFuzzer] Avoid -fuse-ld=lld on gc-sections.
The bot doesn't recognize lld as a linker even though it has the
property lld-available.

llvm-svn: 334864
2018-06-15 21:01:56 +00:00
Matt Morehouse 0ea9a90b3d [SanitizerCoverage] Add associated metadata to pc-tables.
Summary:
Using associated metadata rather than llvm.used allows linkers to
perform dead stripping with -fsanitize-coverage=pc-table.  Unfortunately
in my local tests, LLD was the only linker that made use of this metadata.

Partially addresses https://bugs.llvm.org/show_bug.cgi?id=34636 and fixes
https://github.com/google/sanitizers/issues/971.

Reviewers: eugenis

Reviewed By: eugenis

Subscribers: Dor1s, hiraditya, llvm-commits, kcc

Differential Revision: https://reviews.llvm.org/D48203

llvm-svn: 334858
2018-06-15 20:12:58 +00:00
Petr Hosek 5e6bc090ed [Fuzzer] Don't hardcode target architecture for Fuzzer tests
Don't hardcode the architecture for Fuzzer tests which breaks when
compiler-rt is being compiled for architectures other than x86_64.

Differential Revision: https://reviews.llvm.org/D48207

llvm-svn: 334852
2018-06-15 18:21:02 +00:00
Kostya Kortchinsky 4adf24502e [scudo] Add verbose failures in place of CHECK(0)
Summary:
The current `FailureHandler` mechanism was fairly opaque with regard to the
failure reason due to using `CHECK(0)`. Scudo is a bit different from the other
Sanitizers as it prefers to avoid spurious processing in its failure path. So
we just `dieWithMessage` using a somewhat explicit string.

Adapted the tests for the new strings.

While this takes care of the `OnBadRequest` & `OnOOM` failures, the next step
is probably to migrate the other Scudo failures in the same failes (header
corruption, invalid state and so on).

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: filcab, mgorny, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D48199

llvm-svn: 334843
2018-06-15 16:45:19 +00:00
Eli Friedman 16c4456022 [compiler-rt] [builtins] Don't build __atomic_* by default.
The locks need to be implemented in a shared library to work correctly,
so they shouldn't be part of libclang_rt.builtins.a, except in
specialized scenarios where the user can prove it will only be linked
once.

Differential Revision: https://reviews.llvm.org/D47606

llvm-svn: 334779
2018-06-14 23:22:53 +00:00
George Karpenkov 9afe1fa7e2 [libFuzzer] [NFC] XFAIL one of the tests on iOS.
llvm-svn: 334775
2018-06-14 22:18:18 +00:00
George Karpenkov d7e1a9488b [libFuzzer] [NFC] Remaining minor fixes to support testing on devices.
XFAIL's and adding %run commands.

llvm-svn: 334774
2018-06-14 22:13:36 +00:00
George Karpenkov d1e0365ce3 [libFuzzer] [NFC] Support multi-arch and multi-OS building and testing
Differential Revision: https://reviews.llvm.org/D47296

llvm-svn: 334768
2018-06-14 20:46:07 +00:00
George Karpenkov 46b0df649f [libFuzzer] [NFC] Generalize DSO tests to work even when files are moved.
Differential Revision: https://reviews.llvm.org/D47292

llvm-svn: 334766
2018-06-14 20:30:04 +00:00
George Karpenkov 320c690df3 [NFC] Generalize flags for linking in shared objects in compiler-rt tests to support using multiple shared objects at once
Differential Revision: https://reviews.llvm.org/D48156

llvm-svn: 334765
2018-06-14 20:29:47 +00:00
Alex Shlyapnikov c9b41354ff [MSan] Linker-initialize static fallback_mutex in msan_allocator.cc
Summary:
static fallback_mutex in msan_allocator.cc does not need the ctor
call and can be linker initialized.

Issue: https://github.com/google/sanitizers/issues/194

Reviewers: morehouse, eugenis

Subscribers: delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D48145

llvm-svn: 334749
2018-06-14 18:30:51 +00:00
Alex Shlyapnikov 241b758690 [ASan] Linker-initialize static ScopedInErrorReport::current_error_.
Summary:
Static ScopedInErrorReport::current_error_ can be linker initialized to
shave one global ctor call on application startup and be __asan_init-safe.

Global constructors in ASan runtime are bad because __asan_init runs
from preinit_array, before any such constructors.

Issue: https://github.com/google/sanitizers/issues/194

Reviewers: eugenis, morehouse

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D48141

llvm-svn: 334748
2018-06-14 18:22:23 +00:00
Peter Wu 4ff2d87f30 [ASAN] fix typos and disable long-object-path test for win32
Glob patterns seem unsupported for commands executed by the emulated
shell (LIT_USE_INTERNAL_SHELL=1). Disable the test while that is being
addressed (a workaround such as "cd a-*" also does not work).

llvm-svn: 334719
2018-06-14 15:12:46 +00:00
Kostya Kortchinsky 1ee741bdb6 [scudo] Make Secondary linker-initialized compliant
Summary:
As a follow up to D48142 for Scudo, switch the `SpinMutex` to its static
counterpart, and ensure zero-initialization by memset'ing the whole class.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D48148

llvm-svn: 334716
2018-06-14 14:33:28 +00:00
Peter Wu 1bbab1e558 [ASAN] fix startup crash in dlsym for long paths since glibc 2.27
Summary:
Error messages for dlsym used to be stored on the stack, but since
commit 2449ae7b ("ld.so: Introduce struct dl_exception") in glibc 2.27
these are now stored on the heap (and thus use the dlsym alloc pool).

Messages look like "undefined symbol: __isoc99_printf\0/path/to/a.out".
With many missing library functions and long object paths, the pool is
quickly exhausted. Implement a simple mechanism to return freed memory
to the pool (clear it in case it is used for calloc).

Fixes https://github.com/google/sanitizers/issues/957

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D47995

llvm-svn: 334703
2018-06-14 10:42:54 +00:00
David Carlier 706ea47bdc [Sanitizer] fix compilation warning
In most of systems, this field is a signed type but in some it is an unsigned.

Reviewers: vitalybuka

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D48118

llvm-svn: 334686
2018-06-14 04:51:26 +00:00
Peter Wu 0569660af3 [ASAN] Fix fputs interception for 32-bit macOS
On 32-bit macOS, "_fputs$UNIX2003" is called instead of "fputs" and the
"fgets_fputs" test fails. Apparently previous versions still passed the
test due to the internal implementation calling "strlen", but that does
not seem to be the case with macOS 10.13.3. Fixes r334450.

llvm-svn: 334670
2018-06-13 23:19:21 +00:00
Alex Shlyapnikov 3fc66075c7 [Sanitizers] Make sanitizer allocator linker-initialize compliant.
Summary:
These four SpinMutex ctors was the only code executed in the ctor for
the static __asan::Allocator instance (same for the other sanitizers
allocators), which is supposed to be fully linker-initialized.

Also, when the global ctor for this allocator instance is executed,
this instance might already be initialized by __asan_init called from
.preinit_array.

Issue: https://github.com/google/sanitizers/issues/194

Reviewers: morehouse, eugenis, cryptoad

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D48142

llvm-svn: 334660
2018-06-13 21:45:01 +00:00
Dimitry Andric a00c372cd3 Disable MSan tests of prlimit on FreeBSD
Like NetBSD, FreeBSD does not have prlimit(2), so do not attempt to use
it during the MSan tests.

llvm-svn: 334659
2018-06-13 21:37:49 +00:00
Shoaib Meenai b682276f48 [compiler-rt] Use CMAKE_LINKER instead of hardcoding ld
Respect a custom linker path provided by the user if one is present
(otherwise CMAKE_LINKER will have been set to the right value by CMake).

llvm-svn: 334654
2018-06-13 20:48:30 +00:00
Ahmed Bougacha 61631b1b8f [ASAN] Re-enable fgets_fputs test for darwin.
It seems to be passing, at least on the bots I've seen.

llvm-svn: 334626
2018-06-13 17:46:17 +00:00
Alex Shlyapnikov 4bf9b5612a [TSan] Fix madvise(MADV_NOHUGEPAGE) for meta shadow memory
Summary:
Move madvise(MADV_NOHUGEPAGE) for the meta shadow memory after the meta
shadow memory is mapped (currently it silently fails with ENOMEM).

Add a diagnostic message to detect similar problems in the future.

Reviewers: dvyukov

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D48097

llvm-svn: 334624
2018-06-13 17:18:41 +00:00
Petr Hosek 7252e2a8bd [XRay] Set an explicit dependency on libc++ when needed
When XRay is being built as part of the just built compiler together
with libc++ as part of the runtimes build, we need an explicit
dependency from XRay to libc++ to make sure that the library is
available by the time we start building XRay.

Differential Revision: https://reviews.llvm.org/D48113

llvm-svn: 334575
2018-06-13 07:08:28 +00:00
Petr Hosek 22f8825dca Reland "Passthrough additional flags to custom libcxx CMake build"
This is needed when we're cross-compiling compiler-rt.

Differential Revision: https://reviews.llvm.org/D47834

llvm-svn: 334570
2018-06-13 05:32:22 +00:00
Peter Wu fa60e6966c [ASAN] disable fgets_fputs test for android and darwin
aarch64/aosp_marlin-userdebug/OPR4.170623.016] builder unexpectedly
failed the fgets test (`assertion "fp" failed`). macOS unexpectedly
passes the fputs test without triggering ASAN.

llvm-svn: 334558
2018-06-12 23:05:03 +00:00
George Karpenkov 077c75bfd1 [libFuzzer] [NFC] Do not use hardcoded relative paths in tests.
For paths without "%t" inside uniqueness is not guaranteed, and
potential collisions might be present.

Differential Revision: https://reviews.llvm.org/D47288

llvm-svn: 334546
2018-06-12 21:14:33 +00:00
George Karpenkov 43a0f71aa4 [libFuzzer] [NFC] Make compiler command generation more readable.
Use config.clang as a more general option than config.c_compiler.

Differential Revision: https://reviews.llvm.org/D47295

llvm-svn: 334545
2018-06-12 21:14:11 +00:00
Matt Morehouse e3300d2ff3 Revert r334458, r334220, r334212, r334139.
Reverts changes to AddCompilerRT.cmake due to breakage of
http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux/.

llvm-svn: 334528
2018-06-12 19:43:40 +00:00
Walter Lee 0a58f9c953 [asan, myriad] Support environment variables
Provide a buffer that the test harness can write into to provide
values for the environment variables.  Format is a null-separated list
of VAR=value pairs; this is sufficent for our purpose.

Differential Revision: https://reviews.llvm.org/D47960

llvm-svn: 334522
2018-06-12 18:34:12 +00:00
David Carlier 301855fb0d [Fuzzer] Afl driver changing iterations handling
Handling differently the iterations with the type limit and eventually an error message.

Reviewers: morehouse, kcc

Reviewed By: morehouse

Differential Revision: https://reviews.llvm.org/D47880

llvm-svn: 334510
2018-06-12 15:47:58 +00:00
Kostya Kortchinsky 76969eaf3d [scudo] Add C++17 aligned new/delete operators support
Summary:
This CL adds support for aligned new/delete operators (C++17). Currently we
do not support alignment inconsistency detection on deallocation, as this
requires a header change, but the APIs are introduced and are functional.

Add a smoke test for the aligned version of the operators.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D48031

llvm-svn: 334505
2018-06-12 14:42:40 +00:00
Peter Wu 14db2509ac [ASAN] fix fgets and fgets_fputs tests failure
Some systems (Android) might not have /etc/passwd. Fixes r334450.

llvm-svn: 334487
2018-06-12 09:27:18 +00:00
Dean Michael Berris aea8ece102 [XRay][profiler] Part 5: Profiler File Writing
Summary:
This is part of the larger XRay Profiling Mode effort.

This patch implements the profile writing mechanism, to allow profiles
collected through the profiler mode to be persisted to files.

Follow-on patches would allow us to load these profiles and start
converting/analysing them through the `llvm-xray` tool.

Depends on D44620.

Reviewers: echristo, kpw, pelikan

Reviewed By: kpw

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D45998

llvm-svn: 334472
2018-06-12 04:06:25 +00:00
Dean Michael Berris cfd7eec3d8 [XRay][profiler] Part 4: Profiler Mode Wiring
Summary:
This is part of the larger XRay Profiling Mode effort.

This patch implements the wiring required to enable us to actually
select the `xray-profiling` mode, and install the handlers to start
measuring the time and frequency of the function calls in call stacks.
The current way to get the profile information is by working with the
XRay API to `__xray_process_buffers(...)`.

In subsequent changes we'll implement profile saving to files, similar
to how the FDR and basic modes operate, as well as means for converting
this format into those that can be loaded/visualised as flame graphs. We
will also be extending the accounting tool in LLVM to support
stack-based function call accounting.

We also continue with the implementation to support building small
histograms of latencies for the `FunctionCallTrie::Node` type, to allow
us to actually approximate the distribution of latencies per function.

Depends on D45758 and D46998.

Reviewers: eizan, kpw, pelikan

Reviewed By: kpw

Subscribers: llvm-commits, mgorny

Differential Revision: https://reviews.llvm.org/D44620

llvm-svn: 334469
2018-06-12 03:29:39 +00:00
Ryan Prichard 1e216a59f6 [builtins] Delay emutls deallocation for one round
Summary:
With Android/Bionic, delay deallocation to round 2 of 4. It must run after
C++ thread_local destructors have been called, but before the final 2
rounds, because emutls calls free, and jemalloc then needs another 2
rounds to free its thread-specific data.

Fixes https://github.com/android-ndk/ndk/issues/687

Reviewers: cmtice, srhines, jyknight, chh, echristo

Reviewed By: srhines, chh, echristo

Subscribers: echristo, delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D46978

llvm-svn: 334463
2018-06-12 01:32:26 +00:00
Ryan Prichard 31e4a550a3 [builtins] emutls cleanup: determine header size using sizeof
Summary: Also add a few post-#else/#endif comments

Reviewers: echristo, srhines

Reviewed By: echristo

Subscribers: chh, delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D47861

llvm-svn: 334462
2018-06-12 01:32:25 +00:00
Matt Morehouse a4374ab222 [CMake] Remove -Wno-maybe-uninitialized from passthrough vars.
Fixes http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux/
buildbot.

llvm-svn: 334458
2018-06-12 00:37:06 +00:00
Peter Wu 1c05c95739 [sanitizer] Add fgets, fputs and puts into sanitizer_common
Summary:
Add fgets, fputs and puts to sanitizer_common. This adds ASAN coverage
for these functions, extends MSAN support from fgets to fputs/puts and
extends TSAN support from puts to fputs.

Fixes: https://github.com/google/sanitizers/issues/952

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D46545

llvm-svn: 334450
2018-06-11 22:58:04 +00:00
Alex Shlyapnikov ed32baa84b [Sanitizers] %tool_options -> %env_tool_opts in allocator_returns_null.cc
Use proper substitution in the common allocator_returns_null.cc test.

llvm-svn: 334438
2018-06-11 21:04:24 +00:00
Alex Shlyapnikov 406385505d [Sanitizers] Move allocator_returns_null.cc test to common.
Summary:
Add allocator_returns_null.cc test to sanitizer_common and
remove all sanitizer-specific ones except:
- HWASan is not covered by sanitizer_common
- TSan allocator does not have comprehensive error reporting yet

Reviewers: vitalybuka

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D47971

llvm-svn: 334433
2018-06-11 19:45:59 +00:00
Alex Shlyapnikov 789494026e [Sanitizers] Move pvalloc overflow tests to common.
Summary:
Now all sanitizers with improved allocator error reporting are covered
by these common tests.

Also, add pvalloc-specific checks to LSan.

HWASan is not covered by sanitizer_common, hence its own pvalloc
and other allocator tests.

Reviewers: vitalybuka

Subscribers: srhines, kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D47970

llvm-svn: 334424
2018-06-11 17:33:53 +00:00
Kostya Kortchinsky 4410e2c43f [scudo] Improve the scalability of the shared TSD model
Summary:
The shared TSD model in its current form doesn't scale. Here is an example of
rpc2-benchmark (with default parameters, which is threading heavy) on a 72-core
machines (defaulting to a `CompactSizeClassMap` and no Quarantine):
- with tcmalloc: 337K reqs/sec, peak RSS of 338MB;
- with scudo (exclusive): 321K reqs/sec, peak RSS of 637MB;
- with scudo (shared): 241K reqs/sec, peak RSS of 324MB.

This isn't great, since the exclusive model uses a lot of memory, while the
shared model doesn't even come close to be competitive.

This is mostly due to the fact that we are consistently scanning the TSD pool
starting at index 0 for an available TSD, which can result in a lot of failed
lock attempts, and touching some memory that needs not be touched.

This CL attempts to make things better in most situations:
- first, use a thread local variable on Linux (intead of pthread APIs) to store
  the current TSD in the shared model;
- move the locking boolean out of the TSD: this allows the compiler to use a
  register and potentially optimize out a branch instead of reading it from the
  TSD everytime (we also save a tiny bit of memory per TSD);
- 64-bit atomic operations on 32-bit ARM platforms happen to be expensive: so
  store the `Precedence` in a `uptr` instead of a `u64`. We lose some
  nanoseconds of precision and we'll wrap around at some point, but the benefit
  is worth it;
- change a `CHECK` to a `DCHECK`: this should never happen, but if something is
  ever terribly wrong, we'll crash on a near null AV if the TSD happens to be
  null;
- based on an idea by dvyukov@, we are implementing a bound random scan for
  an available TSD. This requires computing the coprimes for the number of TSDs,
  and attempting to lock up to 4 TSDs in an random order before falling back to
  the current one. This is obviously slightly more expansive when we have just
  2 TSDs (barely noticeable) but is otherwise beneficial. The `Precedence` still
  basically corresponds to the moment of the first contention on a TSD. To seed
  on random choice, we use the precedence of the current TSD since it is very
  likely to be non-zero (since we are in the slow path after a failed `tryLock`)

With those modifications, the benchmark yields to:
- with scudo (shared): 330K reqs/sec, peak RSS of 327MB.

So the shared model for this specific situation not only becomes competitive but
outperforms the exclusive model. I experimented with some values greater than 4
for the number of TSDs to attempt to lock and it yielded a decrease in QPS. Just
sticking with the current TSD is also a tad slower. Numbers on platforms with
less cores (eg: Android) remain similar.

Reviewers: alekseyshl, dvyukov, javed.absar

Reviewed By: alekseyshl, dvyukov

Subscribers: srhines, kristof.beyls, delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D47289

llvm-svn: 334410
2018-06-11 14:50:31 +00:00
Peter Wu 1db004ebdd [ASAN] Fix crash on i?86-linux (32-bit) against glibc 2.27 and later
Summary:
Running sanitized 32-bit x86 programs on glibc 2.27 crashes at startup, with:

    ERROR: AddressSanitizer: SEGV on unknown address 0xf7a8a250 (pc 0xf7f807f4 bp 0xff969fc8 sp 0xff969f7c T16777215)
    The signal is caused by a WRITE memory access.
    #0 0xf7f807f3 in _dl_get_tls_static_info (/lib/ld-linux.so.2+0x127f3)
    #1 0xf7a92599  (/lib/libasan.so.5+0x112599)
    #2 0xf7a80737  (/lib/libasan.so.5+0x100737)
    #3 0xf7f7e14f in _dl_init (/lib/ld-linux.so.2+0x1014f)
    #4 0xf7f6eb49  (/lib/ld-linux.so.2+0xb49)

The problem is that glibc changed the calling convention for the GLIBC_PRIVATE
symbol that sanitizer uses (even when it should not, GLIBC_PRIVATE is exactly
for symbols that can change at any time, be removed etc.), see
https://sourceware.org/ml/libc-alpha/2017-08/msg00497.html

Fixes https://github.com/google/sanitizers/issues/954

Patch By: Jakub Jelinek

Reviewed By: vitalybuka, Lekensteyn

Differential Revison: https://reviews.llvm.org/D44623

llvm-svn: 334363
2018-06-10 11:17:47 +00:00
Alex Shlyapnikov b24e6a9905 [MSan] Switch to common aligned allocs tests.
Summary:
Switch to aligned_alloc and posix_memalign tests in sanitizer_common
and delete the MSan-specific ones.

Reviewers: vitalybuka

Subscribers: srhines, kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D47968

llvm-svn: 334340
2018-06-09 00:47:37 +00:00
Alex Shlyapnikov 643600566d [MSan] Report proper error on allocator failures instead of CHECK(0)-ing
Summary:
Following up on and complementing D44404.

Currently many allocator specific errors (OOM, for example) are reported as
a text message and CHECK(0) termination, not stack, no details, not too
helpful nor informative. To improve the situation, detailed and
structured errors were defined and reported under the appropriate conditions.

Reviewers: eugenis

Subscribers: srhines, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D47793

llvm-svn: 334338
2018-06-08 23:31:42 +00:00
Alex Shlyapnikov 84f8ec5c72 [Sanitizers] Use proper substitution in common allocator test.
Use %env_tool_ops instead of %tool_options.

llvm-svn: 334332
2018-06-08 21:58:22 +00:00
Walter Lee 5a780ee9a9 [asan, myriad] Use local pool for new/delete when ASan run-time is not up
This can happen on Myriad RTEMS so needs to be handled.

Differential Revision: https://reviews.llvm.org/D47916

llvm-svn: 334329
2018-06-08 21:49:38 +00:00
Vitaly Buka 683e2d636d [asan] Re-enable no_asan_gen_globals for 32-bit
PR22682. Bug in gold was fixed 3 years ago.

llvm-svn: 334319
2018-06-08 21:04:44 +00:00
Alex Shlyapnikov dcf0097962 [Sanitizers] Check alignment != 0 for aligned_alloc and posix_memalign
Summary:
Move the corresponding tests to the common folder (as all of the
sanitizer allocators will support this feature soon) and add the checks
specific to aligned_alloc to ASan and LSan allocators.

Reviewers: vitalybuka

Subscribers: srhines, kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D47924

llvm-svn: 334316
2018-06-08 20:40:35 +00:00
David Carlier 6dedf2b5d8 [Xray] logging forgotten header
pthread.h missing for pthread_key* functions.

Reviewers: dberris

Reviewed By: dberris

Differential Revison: https://reviews.llvm.org/D47933

llvm-svn: 334272
2018-06-08 08:42:37 +00:00
Dean Michael Berris b81372b56b [XRay][compiler-rt] Cleanup some internal XRay utilities
This change uses 'const' for the retryingWriteAll(...) API and removes
unnecessary 'static' local variables in getting the temporary filename.

llvm-svn: 334267
2018-06-08 07:48:03 +00:00
Dean Michael Berris af4d583ea8 [XRay][compiler-rt] Remove reliance on C++ ABI features
Summary:
This fixes http://llvm.org/PR32274.

This change adds a test to ensure that we're able to link XRay modes and
the runtime to binaries that don't need to depend on the C++ standard
library or a C++ ABI library. In particular, we ensure that this will work
with C programs compiled+linked with XRay.

To make the test pass, we need to change a few things in the XRay
runtime implementations to remove the reliance on C++ ABI features. In
particular, we change the thread-safe function-local-static
initialisation to use pthread_* instead of the C++ features that ensure
non-trivial thread-local/function-local-static initialisation.

Depends on D47696.

Reviewers: dblaikie, jfb, kpw, eizan

Reviewed By: kpw

Subscribers: echristo, eizan, kpw, mgorny, llvm-commits

Differential Revision: https://reviews.llvm.org/D46998

llvm-svn: 334262
2018-06-08 04:00:07 +00:00
Alex Shlyapnikov a2670df602 [HWASan] Disable allocator_returns_null.cc test on Android.
Disable test added in D47798 on Android and investigate why "new"
behaves differently there.

llvm-svn: 334255
2018-06-08 00:56:49 +00:00
Alex Shlyapnikov af12729185 [HWASan] Report proper error on allocator failures instead of CHECK(0)-ing
Summary:
Currently many allocator specific errors (OOM, for example) are reported as
a text message and CHECK(0) termination, not stack, no details, not too
helpful nor informative. To improve the situation, detailed and
structured errors were defined and reported under the appropriate conditions.

Reviewers: eugenis

Subscribers: kubamracek, delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D47798

llvm-svn: 334248
2018-06-07 23:33:33 +00:00
Kostya Serebryany 111d0b2aec [libFuzzer] When printing NEW_FUNC, use 1-base indexing.
Summary: Otherwise we print things like [0/1] which is visually confusing.

Reviewers: kcc

Reviewed By: kcc

Subscribers: delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D47837

llvm-svn: 334234
2018-06-07 21:15:24 +00:00
Eric Fiselier 84be761332 [CMake] Fix Libc++ Modules build.
When building the dylib, the C++ headers are fundamentally non-module.
They require special versions of the headers in order to provide C++03 and
legacy ABI definitions. This causes ODR issues when modules are enabled
during both the build and the usage of the libc++ headers.

This patch fixes the build error by disabling modules when building the
libc++ sources.

llvm-svn: 334220
2018-06-07 19:57:43 +00:00
Douglas Yung 20fa3d2504 Silence a -Wconstant-logical-operand warning.
llvm-svn: 334214
2018-06-07 18:51:47 +00:00
Petr Hosek 157a5c2f09 [Fuzzer] Update the header path for fdio/spawn.h on Fuchsia
The path now includes lib/ prefix.

Differential Revision: https://reviews.llvm.org/D47866

llvm-svn: 334213
2018-06-07 18:41:35 +00:00
Petr Hosek 6367bca339 [CMake] Filter out -z,defs when building custom libc++
-z,defs is incompatible with sanitizers so we need to filter it out
from the linker flags before passing them to the libc++ build.

Differential Revision: https://reviews.llvm.org/D47865

llvm-svn: 334212
2018-06-07 18:27:27 +00:00
Kamil Rytarowski b9c78e41ba Disable recursive interceptors in signal(3)/MSan
Summary:
signal(3) on NetBSD calls internally sigaction(2).

Without disabling the recursive interceptor, there are
false positives about uninitialized memory reads inside libc.

This change fixes execution of such programs as sh(1) and
vmstat(1) in the NetBSD userland.

Sponsored by <The NetBSD Foundation>

Reviewers: eugenis, vitalybuka, joerg

Reviewed By: vitalybuka

Subscribers: llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D47650

llvm-svn: 334168
2018-06-07 07:55:20 +00:00
Kostya Serebryany 67af99235f [libFuzzer] make the corpus elements aware of their data flow traces
llvm-svn: 334158
2018-06-07 01:40:20 +00:00
Kostya Serebryany 7a88f19099 [libFuzzer] simplify a test, hopefully to fix the bot
llvm-svn: 334156
2018-06-07 01:18:43 +00:00
Vitaly Buka 661d81ef61 [sanitizer] Don't use internal_unlink on Windows
llvm-svn: 334152
2018-06-07 00:26:06 +00:00
Kostya Serebryany 4d9fd7a266 [libFuzzer] remove an experimental flag -use_feature_frequency
llvm-svn: 334146
2018-06-06 23:24:41 +00:00
Petr Hosek 3a32b41024 [Fuzzer] Use private libc++ even for Fuchsia
On Fuchsia, we use libc++ compiled with ASan for our ASan built
executable which means we cannot use the same libc++ for libFuzzer when
building fuzz targets, instead we'll link a custom internal libc++ into
Fuchsia's build of libFuzzer like we already do on Linux.

Differential Revision: https://reviews.llvm.org/D47835

llvm-svn: 334144
2018-06-06 22:59:14 +00:00
Petr Hosek 3cf6db2d56 [CMake] Passthrough additional flags to custom libcxx CMake build
This is needed when we're cross-compiling compiler-rt.

Differential Revision: https://reviews.llvm.org/D47834

llvm-svn: 334139
2018-06-06 22:10:12 +00:00
Vitaly Buka 72fd23c9c5 [sanitizer] Replace deprecated mktemp with mkstemp
llvm-svn: 334138
2018-06-06 22:05:48 +00:00
Vitaly Buka 4ef9117b49 [sanitizer] Cleanup ReadFileToVector and ReadFileToBuffer
Summary:
Added unit-test.
Fixed behavior of max_len argument.
Call read syscall with all available buffer, not just a page.

Reviewers: eugenis

Subscribers: kubamracek, mgorny, llvm-commits

Differential Revision: https://reviews.llvm.org/D46618

llvm-svn: 334130
2018-06-06 20:53:43 +00:00
Walter Lee 381072c417 [asan] Fix Myraid RTEMS port broken by r333985
Add nop CheckASLR() function.

llvm-svn: 334102
2018-06-06 15:39:47 +00:00
Ilya Biryukov b7c3745006 Fix compile error with libstdc++.
By adding a ctor to create fuzzer_allocator<T> from fuzzer_allocator<U>.
This mimics construcotrs of std::allocator<T>.

Without the constructors, some versions of libstdc++ can't compile
`vector<bool, fuzzer_allocator<bool>>`.

llvm-svn: 334077
2018-06-06 09:22:19 +00:00
Dean Michael Berris c561970b27 [XRay][compiler-rt] Refactor recursion guard for Basic and FDR Mode
Summary:
This change extracts the recursion guard implementation from FDR Mode
and updates it to do the following:

- Do the atomic operation correctly to be signal-handler safe.

- Make it usable in both FDR and Basic Modes.

Before this change, the recursion guard relied on an unsynchronised read
and write on a volatile thread-local. A signal handler could then run in
between the read and the write, and then be able to run instrumented
code as part of the signal handling. Using an atomic exchange instead
fixes that by doing a proper mutual exclusion even in the presence of
signal handling.

Reviewers: kpw, eizan, jfb

Reviewed By: eizan

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D47696

llvm-svn: 334064
2018-06-06 06:07:48 +00:00
Kostya Serebryany 1fd005f552 [libFuzzer] initial implementation of -data_flow_trace. It parses the data flow trace and prints the summary, but doesn't use the information in any other way yet
llvm-svn: 334058
2018-06-06 01:23:29 +00:00
Vitaly Buka b89704fa6f [lsan] Do not check for leaks in the forked process
Summary:
If calling process had threads then forked process will fail to detect
references from them.

Fixes https://github.com/google/sanitizers/issues/836

Reviewers: alekseyshl

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D47751

llvm-svn: 334036
2018-06-05 18:15:57 +00:00
Alex Shlyapnikov 236c3f9c4a [LSan] Report proper error on allocator failures instead of CHECK(0)-ing
Summary:
Following up on and complementing D44404.

Currently many allocator specific errors (OOM, for example) are reported as
a text message and CHECK(0) termination, not stack, no details, not too
helpful nor informative. To improve the situation, detailed and
structured errors were defined and reported under the appropriate conditions.

Reviewers: eugenis

Subscribers: srhines, mgorny, delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D47645

llvm-svn: 334034
2018-06-05 18:02:09 +00:00
Dean Michael Berris 4c1fdcfb97 [XRay][compiler-rt] Use DCHECK instead of assert(...) (NFC)
Use DCHECK instead of assert(...) in the XRay runtime.

llvm-svn: 334002
2018-06-05 10:27:20 +00:00
Dean Michael Berris 13b2fcc4e6 [XRay][compiler-rt] Use static instead of inline (NFC)
We don't actually need to support multiple definitions of the functions
in FDR mode, but rather want to make sure that the implementation-detail
functions are marked as 'static' instead. This allows the inliner to do
its magic better for these functions too, since inline functions must
have a unique address across translation units.

llvm-svn: 334001
2018-06-05 10:18:39 +00:00
Dean Michael Berris abbeb4cbf7 [XRay][compiler-rt] Remove __xray:: in some places (NFC)
This is a cosmetic change to remove unnecessary full-qualifications of
types/functions that are already in the __xray:: namespace.

llvm-svn: 334000
2018-06-05 10:12:58 +00:00
Dean Michael Berris bd37b57b4d [XRay][compiler-rt] Remove namespace __xray_fdr_internal (NFC)
We no longer need the __xray_fdr_internal namespace.

llvm-svn: 333998
2018-06-05 10:01:45 +00:00
Dean Michael Berris 7f88ea1fa2 [XRay][compiler-rt] Merge XRay FDR mode into a single file (NFC)
We planned to have FDR mode's internals unit-tested but it turns out
that we can just use end-to-end testing to verify the implementation.
We're going to move towards that approach more and more going forward,
so we're merging the implementation details of FDR mode into a single
.cc file.

We also avoid globbing in the XRay test helper macro, and instead list
down the files from the lib directory.

llvm-svn: 333986
2018-06-05 08:20:54 +00:00
Kamil Rytarowski 7d260775f3 Introduce CheckASLR() in sanitizers
Summary:
At least the ASan, MSan, TSan sanitizers require disabled ASLR on a NetBSD.

Introduce a generic CheckASLR() routine, that implements a check for the
current process. This flag depends on the global or per-process settings.

There is no simple way to disable ASLR in the build process from the
level of a sanitizer or during the runtime execution.

With ASLR enabled sanitizers that operate over the process virtual address
space can misbehave usually breaking with cryptic messages.

This check is dummy for !NetBSD.

Sponsored by <The NetBSD Foundation>

Reviewers: vitalybuka, joerg

Reviewed By: vitalybuka

Subscribers: cryptoad, kubamracek, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D47442

llvm-svn: 333985
2018-06-05 07:29:23 +00:00
Dean Michael Berris 5eaaff6095 [XRay][compiler-rt] Remove __sanitizer:: from namespace __xray (NFC)
This is a non-functional change that removes the full qualification of
functions in __sanitizer:: being used in __xray.

llvm-svn: 333983
2018-06-05 06:12:42 +00:00
Dean Michael Berris 8ba925d2c3 [XRay][compiler-rt] Remove reliance on C++ ABI from BufferQueue
Summary:
This is part of the work to address http://llvm.org/PR32274.

We remove the calls to array-placement-new and array-delete. This allows
us to rely on the internal memory management provided by
sanitizer_common/sanitizer_internal_allocator.h.

Reviewers: eizan, kpw

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D47695

llvm-svn: 333982
2018-06-05 03:46:54 +00:00
David Carlier a6303a9866 [TSan] FreeBSD / intercept thr_exit
intercepting thr_exit to terminate threads under FreeBSD.
Unblock few unit tests hanging.

Reviewers: krytarowski, vitalybuka, emaste

Reviewed By: dvyukov		

Differential Revision: https://reviews.llvm.org/D47677

M    lib/tsan/rtl/tsan_interceptors.cc

llvm-svn: 333870
2018-06-04 06:02:38 +00:00
Dean Michael Berris d12cb59c7a [XRay][compiler-rt] Remove RTTI requirement from XRay Builds
XRay doesn't use RTTI and doesn't need it. We disable it explicitly in
the CMake config, similar to how the other sanitizers already do it.

Part of the work to address http://llvm.org/PR32274.

llvm-svn: 333867
2018-06-04 04:22:26 +00:00
Petr Hosek 8908aa4e30 [Fuzzer] Migrate Fuchsia port from launchpad to fdio_spawn
fdio_spawn is the new public API for launching processes in Fuchsia.

Differential Revision: https://reviews.llvm.org/D47649

llvm-svn: 333809
2018-06-02 01:17:10 +00:00
Kostya Serebryany d74d04a6c5 Add weak definitions of trace-cmp hooks to dfsan
Summary:
This allows to build and link the code with e.g.
-fsanitize=dataflow -fsanitize-coverage=trace-pc-guard,pc-table,func,trace-cmp
w/o providing (all) the definitions of trace-cmp hooks.

This is similar to dummy hooks provided by asan/ubsan/msan for the same purpose,
except that some of the hooks need to have the __dfsw_ prefix
since we need dfsan to replace them.

Reviewers: pcc

Reviewed By: pcc

Subscribers: delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D47605

llvm-svn: 333796
2018-06-01 21:59:25 +00:00
Walter Lee e2f884d73f [asan, myriad] Implement aligned local pool allocation
Extend the local pool allocation support to posix_memalign.

Differential Revision: https://reviews.llvm.org/D47642

llvm-svn: 333788
2018-06-01 21:29:05 +00:00
Walter Lee bf03bfb0a3 [asan, myriad] Configure platform interceptors
Myriad only uses the platform interceptors for memory allocation
routines.  Configure them properly.

Also add a missing guard around aligned alloc interceptor.

Differential Revision: https://reviews.llvm.org/D47641

llvm-svn: 333784
2018-06-01 19:47:41 +00:00
David Carlier 45cdbdac06 [XRay] Fix supposely failing test for FreeBSD
One test was marked as XFAIL for FreeBSD however still running when launching make check-xray

Reviewers: krytarowski, vitalybuka

Reviewed By: dberris

Differential Revision: https://reviews.llvm.org/D47622

llvm-svn: 333733
2018-06-01 09:15:44 +00:00
Yvan Roux 3b480d1858 [ASAN] Sanitize testsuite for ARM.
Address failures exhibited by ARMv8 bot in Thumb mode:

- Fix logic for fast unwinding support (i.e feature is not available for Thumb)
- Fix Unsupported and Requires rules to handle armv8 as well as soft and hard
  float targets
- Un-xfail passing tests

Differential Revision: https://reviews.llvm.org/D47575

llvm-svn: 333729
2018-06-01 06:23:36 +00:00
Reid Kleckner d44a6eab40 Fix GCC 4.8.5 build by avoiding braced default member initializer
Use internal_memset instead.  Should revive all Linux Chromium ToT bots.

llvm-svn: 333678
2018-05-31 18:26:36 +00:00
Sylvestre Ledru ac804c2238 Define SIZEOF_STRUCT_USTAT for 32bit sparc.
Patch landed on gcc upstream:
27453e962b
Patch by Matthias Klose

llvm-svn: 333644
2018-05-31 12:41:15 +00:00
Stephan Bergmann af3226752b Missing include
(For some reason, my Mac build complained about unknown pthread_once_t and
PTHREAD_ONCE_INIT, but not about pthread_once itself.)

llvm-svn: 333638
2018-05-31 09:26:31 +00:00
Dean Michael Berris d1fe506694 [XRay] Fixup: Remove unnecessary type alias
Follow-up to D45758.

llvm-svn: 333628
2018-05-31 05:25:47 +00:00
Dean Michael Berris 16c865b071 [XRay] Fixup: Explicitly call std::make_tuple(...)
Follow-up to D45758.

llvm-svn: 333627
2018-05-31 05:02:11 +00:00
Dean Michael Berris ca856e07de [XRay] Fixup: Address some warnings breaking build
Follow-up to D45758.

llvm-svn: 333625
2018-05-31 04:55:11 +00:00
Dean Michael Berris 1eb8c206cd [XRay][profiler] Part 3: Profile Collector Service
Summary:
This is part of the larger XRay Profiling Mode effort.

This patch implements a centralised collector for `FunctionCallTrie`
instances, associated per thread. It maintains a global set of trie
instances which can be retrieved through the XRay API for processing
in-memory buffers (when registered). Future changes will include the
wiring to implement the actual profiling mode implementation.

This central service provides the following functionality:

*  Posting a `FunctionCallTrie` associated with a thread, to the central
   list of tries.

*  Serializing all the posted `FunctionCallTrie` instances into
   in-memory buffers.

*  Resetting the global state of the serialized buffers and tries.

Depends on D45757.

Reviewers: echristo, pelikan, kpw

Reviewed By: kpw

Subscribers: llvm-commits, mgorny

Differential Revision: https://reviews.llvm.org/D45758

llvm-svn: 333624
2018-05-31 04:33:52 +00:00
Kostya Serebryany 980e45fe55 [libFuzzer] add collect_data_flow.py that allows to run the data-flow tracer several times on subsets of inputs bytes, to overcome DFSan out-of-label failures
llvm-svn: 333616
2018-05-31 01:27:07 +00:00
Reid Kleckner b54ac414d1 [asan] Remove unneeded VirtualQuery from exception handler
We don't use the result of the query, and all tests pass if I remove it.
During startup, ASan spends a fair amount of time in this handler, and
the query is much more expensive than the call to commit the memory.

llvm-svn: 333595
2018-05-30 21:21:18 +00:00
Walter Lee 3dc6cea0e3 [asan, myriad] Simplify main thread handling
On Myriad RTEMS, we don't need to treat the main thread differently.
The existing thread hooks will do the right thing, so get rid of all
the unneeded special logic.

Differential Revision: https://reviews.llvm.org/D47502

llvm-svn: 333504
2018-05-30 04:57:29 +00:00
Walter Lee 0a70e9db03 [asan, myriad] Reset shadow memory during exit
Reset shadow memory during exit.  Also update a cut-and-paste comment,
and do some minor refactoring of InitializeShadowMemory.

Differential Revision: https://reviews.llvm.org/D47501

llvm-svn: 333503
2018-05-30 04:57:04 +00:00
George Burgess IV d30bcab70e Fix a typo; NFC
llvm-svn: 333331
2018-05-26 02:28:43 +00:00
Vitaly Buka 4add8a1f4e [safestack] Lazy initialization of interceptors
Interceptors initialization may try to allocate memory and to call not
initialized allocator.

It's similar to r326025 for CFI.

llvm-svn: 333329
2018-05-26 01:18:32 +00:00
Petr Hosek 796bb7d4ce [Fuzzer] Update _zx_port_wait function use in Fuchsia port
Fuchsia's _zx_port_wait no longer takes the count argument. This
change also updates all symbol uses to use the underscored versions.

Differential Revision: https://reviews.llvm.org/D47403

llvm-svn: 333328
2018-05-26 01:02:34 +00:00
H.J. Lu 11c1c266cc Revert the last test commit
llvm-svn: 333310
2018-05-25 20:51:45 +00:00
H.J. Lu caab7c8cca This is a test commit to verify repository access
llvm-svn: 333309
2018-05-25 20:51:10 +00:00
David Carlier f6311c669f [MemorySanitizer] fix mmap test for oses not implementing MAP_NORESERVE flag
Reviewers: krytarowski, eugenis

Reviewed By: eugenis

Differential Revision: https://review.llvm.org/D47146

llvm-svn: 333306
2018-05-25 20:30:39 +00:00
Vitaly Buka 540d7e528d [libFuzzer] Avoid optimization of "abs(x) < 0"
llvm-svn: 333295
2018-05-25 18:27:12 +00:00
Vitaly Buka b161db099d Revert "[libFuzzer] [NFC] Generalize DSO tests to work even when files are moved."
Breaks libFuzzer tests.

This reverts commit r333243.

llvm-svn: 333257
2018-05-25 07:18:29 +00:00
George Karpenkov 698b0a674d [libFuzzer] [NFC] Generalize fdmask test.
The ordering of stderr/stdout writes might not be guaranteed,
and is irrelevant for this test.

Differential Revision: https://reviews.llvm.org/D47294

llvm-svn: 333245
2018-05-24 23:55:54 +00:00
George Karpenkov b4ccb3ab08 [libFuzzer] [NFC] Do not remove SanitizerLintCheck from libFuzzer tests
With %run commands being present now, we want to make sure that they
stay present.

Differential Revision: https://reviews.llvm.org/D47293

llvm-svn: 333244
2018-05-24 23:55:54 +00:00
George Karpenkov fe44eb4cf1 [libFuzzer] [NFC] Generalize DSO tests to work even when files are moved.
Differential Revision: https://reviews.llvm.org/D47292

llvm-svn: 333243
2018-05-24 23:55:53 +00:00
George Karpenkov 464d1415b2 [libFuzzer] Fix multi-arch build support for fuzzer tests.
The bug was not previously exposed as only one architecture is currently
supported for libFuzzer.

Differential Revision: https://reviews.llvm.org/D47287

llvm-svn: 333242
2018-05-24 23:55:53 +00:00
George Karpenkov 02c85f1489 [libFuzzer] Run libFuzzer unit tests only on host architecture.
LIT tests are used to test other cross-compiled architectures,
unit tests are only run on the host.
NFC now as currently only a single architecture is supported.

Differential Revision: https://reviews.llvm.org/D47286

llvm-svn: 333241
2018-05-24 23:55:52 +00:00
George Karpenkov 0dcb6902f1 [libFuzzer] XFAIL a few libFuzzer tests on iOS.
NFC for now.

Differential Revision: https://reviews.llvm.org/D47284

llvm-svn: 333240
2018-05-24 23:55:52 +00:00
George Karpenkov e7e3fc26ee [libFuzzer] Fix a typo in CMake configuration.
NFC now when libFuzzer supports only one architecture,
will stop being NFC after multiple architectures are supported.

Differential Revision: https://reviews.llvm.org/D47283

llvm-svn: 333239
2018-05-24 23:55:51 +00:00
Azharuddin Mohammed 06e0fd3bc1 Revert "[cmake] [ARM] Check if VFP is supported before including any VFP builtins"
This reverts commit 2a10f5da5acb1b51d0a2ecf13dca0bf1de859db2.

llvm-svn: 333232
2018-05-24 21:36:27 +00:00
Azharuddin Mohammed 80968a9aba [cmake] [ARM] Check if VFP is supported before including any VFP builtins
Summary:
rL325492 disables FPU features when using soft floating point
(-mfloat-abi=soft), which is used internally when building for armv7. This
causes errors with builtins that utililize VFP instructions. With this change
we first check if VFP is enabled (by checking if the preprocessor macro
__VFP_FP__ is defined) before including such builtins.

Reviewers: rengolin, samsonov, compnerd, smeenai, javed.absar, peter.smith

Reviewed By: peter.smith

Subscribers: peter.smith, mgorny, kristof.beyls, chrib, llvm-commits

Differential Revision: https://reviews.llvm.org/D47217

llvm-svn: 333216
2018-05-24 18:53:16 +00:00
Craig Topper 383fe5c866 sanitizer: Use pre-computed size of struct ustat for Linux
<sys/ustat.h> has been removed from glibc 2.28 by:

commit cf2478d53ad7071e84c724a986b56fe17f4f4ca7
Author: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Date: Sun Mar 18 11:28:59 2018 +0800

Deprecate ustat syscall interface
This patch uses pre-computed size of struct ustat for Linux to fix

https://bugs.llvm.org/show_bug.cgi?id=37418

Patch by H.J. Lu.

Differential Revision: https://reviews.llvm.org/D47281

llvm-svn: 333213
2018-05-24 17:59:47 +00:00
Igor Kudrin 13f8a77d79 [tsan] Fix a failure in test java_race_pc on AArch64 after r333071
llvm-svn: 333180
2018-05-24 12:26:20 +00:00
Kostya Serebryany 4d53b744ca [libFuzzer] DataFlow tracer now tags a subset of the input. A separate script merges traces from the subsets
llvm-svn: 333149
2018-05-24 01:43:48 +00:00
Kostya Serebryany 500ca8713c [libFuzzer] fix two off-by-ones (!!) in the data flow tracer
llvm-svn: 333142
2018-05-23 23:55:54 +00:00
Jake Ehrlich 767d92e3fb [fuchsia] Add line buffering in RawWrite
This change causes RawWrite to buffer upto 128 bytes or until
a line is reached. This helps group calls into more readable
lines.

llvm-svn: 333136
2018-05-23 22:27:12 +00:00
Kostya Serebryany 4925392897 [libFuzzer] change the output format for the DataFlow tracer
llvm-svn: 333122
2018-05-23 20:57:11 +00:00
Kostya Serebryany 28fe54fac8 [libFuzzer] add a stress test for the DataFlow tracer
llvm-svn: 333119
2018-05-23 20:23:33 +00:00
Max Moroz fe97441c2c [libFuzzer] Don't complain about lack of interesting inputs when -runs=0.
Summary:
The most common usecase for -runs=0 is for generating code coverage
over some corpus. Coverage reports based on sancov are about to be deprecated,
which means some external coverage solution will be used, e.g. Clang source
based code coverage, which does not use any sancov instrumentations and thus
libFuzzer would consider any input to be not interesting in that case.

Reviewers: kcc

Reviewed By: kcc

Subscribers: alex, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D47271

llvm-svn: 333116
2018-05-23 19:42:30 +00:00
Kostya Kortchinsky 08c630457a [sanitizer] Define UINTPTR_MAX
Summary:
`sanitizer_internal_defs.h` didn't have this define, which will be useful in
an upcoming CL.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: kubamracek, delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D47270

llvm-svn: 333109
2018-05-23 18:13:21 +00:00
Kamil Rytarowski 5b1b594f64 Fix internal_mmap() on 32-bit NetBSD platforms
There is need to use internal_syscall64() instead of internal_syscall_ptr().
The offset argument of type off_t is always 64-bit.

llvm-svn: 333075
2018-05-23 10:37:00 +00:00
Igor Kudrin c0099f9be7 [sanitizer] Reland "Be more accurate when calculating the previous instruction address on ARM."
Differential Revision: https://reviews.llvm.org/D46004

llvm-svn: 333071
2018-05-23 09:18:10 +00:00
Kostya Serebryany 7f5bf83f53 [libFuzzer] modify -print_corpus_stats to print whether the input reaches the focus function
llvm-svn: 333048
2018-05-23 01:42:53 +00:00
Petr Hosek aa6d13e501 [CMake] Support builtins as Clang default rtlib in compiler-rt
Use compiler-rt builtins when selected as default Clang rtlib and avoid
explicitly passing -rtlib= flag to avoid the "argument unused during
compilation" warning.

This is a partial alternative to D47094 that does not rely on compiler
runtime checks.

Differential Revision: https://reviews.llvm.org/D47115

llvm-svn: 333037
2018-05-22 22:58:48 +00:00
Evgeniy Stepanov 45f8986b80 [asan] Use dynamic allocator space address on Android/AArch64.
Summary:
We need one library to support all of 39, 42 and 48 bit VMAs, and
there is no common address that works for all of them.

Reviewers: kcc, alekseyshl, javed.absar

Subscribers: rengolin, srhines, kubamracek, kristof.beyls, llvm-commits, cryptoad

Differential Revision: https://reviews.llvm.org/D47160

llvm-svn: 333025
2018-05-22 20:44:45 +00:00
Petr Hosek 1ffaa48d4a [CMake] Support libc++ as Clang default stdlib in compiler-rt
Use libc++ when selected as default Clang stdlib and avoid checking
C++ compiler when using the in-tree version of libc++.

This is a partial alternative to D47094 that does not rely on compiler
runtime checks.

Differential Revision: https://reviews.llvm.org/D47100

llvm-svn: 333010
2018-05-22 18:33:27 +00:00
Petr Hosek 4ca5af0721 [CMake] Silence unused variable warning in compiler check
This is breaking the compiler check.

llvm-svn: 332936
2018-05-22 02:53:32 +00:00
Petr Hosek f6dda7cb44 [CMake] Use a different source depending on C++ support
When using system C++ library, assume we have a working C++ compiler and
try to compile a complete C++ program. When using in tree C++ library,
only check the C compiler since the C++ library likely won't have been
built yet at time of running the check.

Differential Revision: https://reviews.llvm.org/D47169

llvm-svn: 332924
2018-05-22 01:01:56 +00:00
Walter Lee ead3b3487b [asan] Make GetCurrentThread RTEMS-friendly
On RTEMS, system and user code all live in a single binary and address
space. There is no clean separation, and instrumented code may
execute before the ASan run-time is initialized (or after it has been
destroyed).

Currently, GetCurrentThread() may crash if it's called before ASan
run-time is initialized. Make it return nullptr instead.

Similarly, fix __asan_handle_no_return so that it gives up rather than
try something that may crash.

Differential Revision: https://reviews.llvm.org/D46459

llvm-svn: 332888
2018-05-21 20:43:36 +00:00
Kostya Serebryany 69c2b71a51 [libFuzzer] reinstate -dump_coverage, which is still in use (reverts r332036)
llvm-svn: 332876
2018-05-21 19:47:00 +00:00
Petr Hosek 702d073642 Revert "[CMake] Reland "Make simple source used for CMake checks a C file""
This reverts commit rCRT332679 which doesn't quite work and will be
replaced by D47100 and D47115 which is a cleaner solution.

llvm-svn: 332871
2018-05-21 19:32:43 +00:00
Kamil Rytarowski 434606c8e4 Align ClearShadowForThreadStackAndTLS for NetBSD/i386
Summary:
The static TLS vector for the main thread on NetBSD/i386 can be
unaligned in terms of the shadow granularity. Align the start of it with
Round Down and end of it with Round Up operations for the shadow
granularity shift.

Example static TLS vector ranges on NetBSD/i386:
tls_begin_=0xfbee7244 tls_end_=0xfbee726c.

ClearShadowForThreadStackAndTLS() is called from the Main Thread
bootstrap functions.

This change restores the NetBSD x86 32-bit (i386) support.

Sponsored by <The NetBSD Foundation>

Reviewers: vitalybuka, joerg

Reviewed By: vitalybuka

Subscribers: kubamracek, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D46585

llvm-svn: 332792
2018-05-19 01:20:00 +00:00
Kamil Rytarowski 51daee08cd Port msan_test.cc to NetBSD
Summary:
The changes allows building this file on NetBSD, mostly by
disabling the unsupported functions and adding OS-specific
system includes.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, kcc, vitalybuka

Reviewed By: vitalybuka

Subscribers: llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D46712

llvm-svn: 332791
2018-05-19 01:10:25 +00:00
Kamil Rytarowski c7fe74576f Do not link with -ldl in NetBSD/ASan tests
Summary:
The dlopen(3) features on NetBSD are in libc.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, vitalybuka, kcc

Reviewed By: vitalybuka

Subscribers: kubamracek, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D46713

llvm-svn: 332790
2018-05-19 01:02:51 +00:00
Evgeniy Stepanov 28f330fd6f [msan] Don't check divisor shadow in fdiv.
Summary:
Floating point division by zero or even undef does not have undefined
behavior and may occur due to optimizations.

Fixes https://bugs.llvm.org/show_bug.cgi?id=37523.

Reviewers: kcc

Subscribers: hiraditya, llvm-commits

Differential Revision: https://reviews.llvm.org/D47085

llvm-svn: 332761
2018-05-18 20:19:53 +00:00
Reid Kleckner 4f9516d617 Revert r332683 & r332684 relating to compiler runtime checks
r332683 passes flags to the compiler without first checking if they are
supported.

llvm-svn: 332754
2018-05-18 18:44:37 +00:00
Walter Lee 0edca4f505 [asan] Explicitly declare memintrinsics interceptors to have C linkage
This is needed on RTEMS.  Also update a comment.

Differential Revision: https://reviews.llvm.org/D47079

llvm-svn: 332746
2018-05-18 18:01:08 +00:00
Walter Lee 927f4ec1ff [asan] Remove an unsigned compare >= 0
This is not needed after we've forked the Myriad version.  Not to
mention it produces a compiler warning.

Differential Revision: https://reviews.llvm.org/D47054

llvm-svn: 332744
2018-05-18 17:52:12 +00:00
Walter Lee e35f57f023 [asan] Add target-specific files for Myriad RTEMS port
Differential Revision: https://reviews.llvm.org/D46468

llvm-svn: 332691
2018-05-18 04:10:12 +00:00
Walter Lee ce26498e3f [asan] Add support for Myriad RTEMS memory map
The Myriad RTEMS memory system has a few unique aspects that
require support in the ASan run-time.

- A limited amount of memory (currently 512M).

- No virtual memory, no memory protection.

- DRAM starts at address 0x80000000.  Other parts of memory may be
  used for MMIO, etc.

- The second highest address bit is the "cache" bit, and 0x80000000
  and 0x84000000 alias to the same memory.

To support the above, we make the following changes:

- Use a ShadowScale of 5, to reduce shadow memory overhead.

- Adjust some existing macros to remove assumption that the lowest
  memory address is 0.

- add a RawAddr macro that on Myriad strips the cache bit from the
  input address, before using the address for shadow memory (for other
  archs this does nothing).

- We must check that an address is in DRAM range before using it to
  index into shadow memory.

Differential Revision: https://reviews.llvm.org/D46456

llvm-svn: 332690
2018-05-18 04:09:45 +00:00
Petr Hosek 83061cc4aa [CMake] Use <UNIX|WINDOWS>_COMMAND with separate_arguments
NATIVE_COMMAND is only available since CMake 3.9.

llvm-svn: 332684
2018-05-18 01:20:47 +00:00
Petr Hosek 8b5fe57d3e [CMake] Detect the compiler runtime and standard library
Rather then requiring the user to specify runtime the compiler
runtime and C++ standard library, or trying to guess them which is
error-prone, use auto-detection by parsing the compiler link output.

Differential Revision: https://reviews.llvm.org/D46857

llvm-svn: 332683
2018-05-18 01:10:09 +00:00
Walter Lee b134dbb3c3 [sanitizer] Trivial portion of the port to Myriad RTEMS
This commit contains the trivial portion of the port of ASan to
Myriad RTEMS.
- Whitelist platform in sanitizer_platform.h, ubsan_platform.h
- Turn off general interception
- Use memset for FastPoisonShadow
- Define interception wrappers
- Set errno symbol correctly
- Enable ASAN_LOW_MEMORY
- Enable preinit array
- Disable slow unwinding
- Use fuchsia offline symbolizer
- Disable common code for: InitializeShadowMemory, CreateMainThread,
  AsanThread::ThreadStart, StartReportDeadlySignal,
  MaybeReportNonExecRegion.

Differential Revision: https://reviews.llvm.org/D46454

llvm-svn: 332681
2018-05-18 00:43:54 +00:00
Petr Hosek 3eb71831e5 [CMake] Reland "Make simple source used for CMake checks a C file"
The source being compiled is plain C, but using .cc extension forces it
to be compiled as C++ which requires a working C++ compiler including
C++ library which may not be the case when we're building compiler-rt
together with libcxx as part of runtimes build.

Differential Revision: https://reviews.llvm.org/D47031

llvm-svn: 332679
2018-05-17 23:50:35 +00:00
Petr Hosek 5a668a054b [CMake] Cleanup find_compiler_rt_library function [NFC]
Rename the output variable and remove the unnecessary set call.

llvm-svn: 332661
2018-05-17 20:35:55 +00:00
Alex Lorenz a54e218d4a [builtins][macos] bump up the the macOS version min in os_version_check tests
This ensures that the tests link with the latest OS.

llvm-svn: 332647
2018-05-17 18:41:38 +00:00
Kostya Serebryany a52d30b2bc [libFuzzer] rename a test from .c to .cpp
llvm-svn: 332558
2018-05-16 23:38:53 +00:00
Walter Lee c5368c230e [asan] Add a magic shadow value for shadw gap
This gives us something to insert into the shadow gap for systems that
don't have memory protection turned on there (i.e. on Myriad).

Differential Revision: https://reviews.llvm.org/D46457

llvm-svn: 332557
2018-05-16 23:36:01 +00:00
Kostya Serebryany e9c6f06cce [libFuzzer] add an experimental flag -focus_function: libFuzzer will try to focus on inputs that trigger that function
llvm-svn: 332554
2018-05-16 23:26:37 +00:00
Walter Lee 3bf63c9201 [asan] Restore check removed by r332033
Needed by fiber handling code, and possibly other code paths.

Differential Revision: https://reviews.llvm.org/D46981

llvm-svn: 332553
2018-05-16 23:23:56 +00:00
Max Moroz d63396097d [libFuzzer] add a symbolic execution puzzle (difficult for today's libFuzzer).
Summary:
This can be solved just in seconds with KLEE. Current libFuzzer
is able to satistfy 101 constraints out of 410 constraints presented during
the first hour of running with -use_value_profile=1 and -max_len=20.
During the next 3 hours, libFuzzer is able to generate ~50 NEW inputs,
bot none of those solve any new constraint.
During the next 20 hours, it didn't find any NEW inputs.

This test might be interesting for experimenting with the data flow tracing
approach started in https://reviews.llvm.org/D46666.

For the solution with KLEE and other information, see
https://github.com/Dor1s/codegate2017-quals-angrybird

Reviewers: kcc

Reviewed By: kcc

Subscribers: delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D46924

llvm-svn: 332507
2018-05-16 18:19:30 +00:00
Kostya Kortchinsky e5c9e9f0bd [scudo] Quarantine optimization
Summary:
It turns out that the previous code construct was not optimizing the allocation
and deallocation of batches. The class id was read as a class member (even
though a precomputed one) and nothing else was optimized. By changing the
construct this way, the compiler actually optimizes most of the allocation and
deallocation away to only work with a single class id, which not only saves some
CPU but also some code footprint.

Reviewers: alekseyshl, dvyukov

Reviewed By: dvyukov

Subscribers: dvyukov, delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D46961

llvm-svn: 332502
2018-05-16 18:12:31 +00:00
Kostya Kortchinsky 561228b2fa [sanitizer] Minor 32-bit primary improvements
Summary:
For the 32-bit TransferBatch:
- `SetFromArray` callers have bounds `count`, so relax the `CHECK` to `DCHECK`;
- same for `Add`;
- mark `CopyToArray` as `const`;
For the 32-bit Primary:
- `{Dea,A}llocateBatch` are only called from places that check `class_id`,
  relax the `CHECK` to `DCHECK`;
- same for `AllocateRegion`;
- remove `GetRegionBeginBySizeClass` that is not used;
- use a local variable for the random shuffle state, so that the compiler can
  use a register instead of reading and writing to the `SizeClassInfo` at every
  iteration;
For the 32-bit local cache:
- pass the count to drain instead of doing a `Min` everytime which is at times
  superfluous.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D46657

llvm-svn: 332478
2018-05-16 15:13:26 +00:00
Vitaly Buka 75e01fa00b [sanitizer] Intercept __pthread_mutex_lock and __pthread_mutex_unlock
Reviewers: eugenis, dvyukov

Subscribers: srhines, kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D46793

llvm-svn: 332320
2018-05-15 01:39:25 +00:00