Commit Graph

10615 Commits

Author SHA1 Message Date
Kostya Kortchinsky f22f5fe910 [scudo] Adding a public Scudo interface
Summary:
The first and only function to start with allows to set the soft or hard RSS
limit at runtime. Add associated tests.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: mgorny, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D41128

llvm-svn: 320611
2017-12-13 20:41:35 +00:00
Kostya Kortchinsky f50246da65 [sanitizer] Introduce a vDSO aware timing function
Summary:
See D40657 & D40679 for previous versions of this patch & description.

A couple of things were fixed here to have it not break some bots.
Weak symbols can't be used with `SANITIZER_GO` so the previous version was
breakin TsanGo. I set up some additional local tests and those pass now.

I changed the workaround for the glibc vDSO issue: `__progname` is initialized
after the vDSO and is actually public and of known type, unlike
`__vdso_clock_gettime`. This works better, and with all compilers.

The rest is the same.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: srhines, kubamracek, krytarowski, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D41121

llvm-svn: 320594
2017-12-13 16:23:54 +00:00
Kostya Kortchinsky 4ac0b1e6e9 [scudo] Inline getScudoChunk function.
Summary:
getScudoChunk function is implicitly inlined for optimized builds on
clang, but not on gcc. It's a small enough function that it seems
sensible enough to just inline it by default.

Reviewers: cryptoad, alekseyshl

Reviewed By: cryptoad

Differential Revision: https://reviews.llvm.org/D41138

llvm-svn: 320592
2017-12-13 16:10:39 +00:00
Dean Michael Berris eec462f0e8 [XRay][compiler-rt] Reduce XRay log spam
This change makes XRay print the log file output only when the verbosity
level is higher than 0. It reduces the log spam in the default case when
we want XRay running silently, except when there are actual
fatal/serious errors.

We also update the documentation to show how to get the information
after the change to the default behaviour.

llvm-svn: 320550
2017-12-13 06:37:13 +00:00
Evgeniy Stepanov ecb48e523e [hwasan] Inline instrumentation & fixed shadow.
Summary: This brings CPU overhead on bzip2 down from 5.5x to 2x.

Reviewers: kcc, alekseyshl

Subscribers: kubamracek, hiraditya, llvm-commits

Differential Revision: https://reviews.llvm.org/D41137

llvm-svn: 320538
2017-12-13 01:16:34 +00:00
Kostya Serebryany e9ed2327b6 [libFuzzer] change the strategy for -experimental_len_control to grow max_len slower
llvm-svn: 320531
2017-12-12 23:11:28 +00:00
Roman Lebedev 1b318b1016 [msan] LIT: Add lld testing config
Summary: A follow-up for D39508, with memory sanitizer changes.

Reviewers: eugenis, vitalybuka

Reviewed By: eugenis

Subscribers: mgorny, mehdi_amini, kcc, #sanitizers, llvm-commits

Tags: #sanitizers, #lld

Differential Revision: https://reviews.llvm.org/D40768

llvm-svn: 320514
2017-12-12 19:17:52 +00:00
Petr Hosek 95d7a4197a [CMake] Support runtimes and monorepo layouts when looking for libcxx
This also slightly refactors the code that's checking the directory
presence which allows eliminating one unnecessary variable.

Differential Revision: https://reviews.llvm.org/D40637

llvm-svn: 320446
2017-12-12 01:20:52 +00:00
Kostya Kortchinsky ab5f6aaa75 [sanitizer] Revert rL320409
Summary: D40679 broke a couple of builds, reverting while investigating.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: srhines, kubamracek, krytarowski, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D41088

llvm-svn: 320417
2017-12-11 21:03:12 +00:00
Kostya Kortchinsky d276d72441 [sanitizer] Introduce a vDSO aware time function, and use it in the allocator [redo]
Summary:
Redo of D40657, which had the initial discussion. The initial code had to move
into a libcdep file, and things had to be shuffled accordingly.

`NanoTime` is a time sink when checking whether or not to release memory to
the OS. While reducing the amount of calls to said function is in the works,
another solution that was found to be beneficial was to use a timing function
that can leverage the vDSO.

We hit a couple of snags along the way, like the fact that the glibc crashes
when clock_gettime is called from a preinit_array, or the fact that
`__vdso_clock_gettime` is mangled (for security purposes) and can't be used
directly, and also that clock_gettime can be intercepted.

The proposed solution takes care of all this as far as I can tell, and
significantly improve performances and some Scudo load tests with memory
reclaiming enabled.

@mcgrathr: please feel free to follow up on
https://reviews.llvm.org/D40657#940857 here. I posted a reply at
https://reviews.llvm.org/D40657#940974.

Reviewers: alekseyshl, krytarowski, flowerhack, mcgrathr, kubamracek

Reviewed By: alekseyshl, krytarowski

Subscribers: #sanitizers, mcgrathr, srhines, llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D40679

llvm-svn: 320409
2017-12-11 19:23:12 +00:00
Ahmed Bougacha 7008c6bc03 Update another sanitizer test for C++14.
Follow-up to r320251.

llvm-svn: 320284
2017-12-10 05:50:23 +00:00
Kostya Serebryany de891a1946 [libFuzzer] even less aggressive step function for stack depth. Also don't use asan in one test to speed it up
llvm-svn: 320259
2017-12-09 19:18:10 +00:00
Tim Northover ad850f83ee Update sanitizer tests for C++14 default in Clang
llvm-svn: 320251
2017-12-09 13:35:52 +00:00
Kostya Serebryany f94d87aadf [libFuzzer] make the stack depth signal less agressive, otherwise we are sometimes exploding the corpus size. This still needs more analysis and tuning
llvm-svn: 320237
2017-12-09 02:45:23 +00:00
Kamil Rytarowski df8147e30d Revert change in test/msan/textdomain.cc for NetBSD
SVN r. 320226

This breaks Linux.

llvm-svn: 320236
2017-12-09 02:38:58 +00:00
Kamil Rytarowski f6510a01a6 Correct handling of the TLS/NetBSD block of the main program
Summary:
Include <sys/tls.h> for:

 - struct tls_tcb - thread control block structure
 - __HAVE___LWP_GETTCB_FAST - __lwp_gettcb_fast() is available
 - __HAVE___LWP_GETPRIVATE_FAST -  __lwp_getprivate_fast() is available
 - __HAVE_TLS_VARIANT_I - TLS Variant I for this architecture
 - __HAVE_TLS_VARIANT_II - TLS Variant II for this architecture

Rename ThreadSelfSegbase() to ThreadSelfTlsTcb and switch it
to retrieve in a portable way TCB.

Switch ThreadSelf() to retrieve pthread from struct tcb_tls.

Use dl_iterate_phdr() to find out the size of TLS block of
the main program.

Correct the index of the  TLS block of the main program
(dlpi_tls_modid); it's 1, not 2.

New NetBSD code is now CPU (NetBSD port) agnostic.

Stop sharing the same code with FreeBSD.

Sponsored by <The NetBSD Foundation>

Reviewers: dvyukov, joerg, vitalybuka

Reviewed By: vitalybuka

Subscribers: emaste, kubamracek, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D40159

llvm-svn: 320234
2017-12-09 02:23:33 +00:00
Evgeniy Stepanov 9595d16893 Hardware-assisted AddressSanitizer (compiler-rt)
Summary:
Runtime library for HWASan, initial commit.
Does not randomize tags yet, does not handle stack or globals.

Reviewers: kcc, pcc, alekseyshl

Subscribers: srhines, kubamracek, dberris, mgorny, llvm-commits, krytarowski

Differential Revision: https://reviews.llvm.org/D40935

llvm-svn: 320231
2017-12-09 01:31:51 +00:00
Kamil Rytarowski f318d95524 Fix test/msan/iconv.cc on NetBSD
Summary:
NetBSD still uses the old POSIX iconv(3) signature with the 2nd const argument.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, eugenis, vitalybuka

Reviewed By: vitalybuka

Subscribers: llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D41017

llvm-svn: 320228
2017-12-09 00:41:59 +00:00
Kamil Rytarowski 79a8c282e3 Fix test/msan/textdomain.cc on NetBSD
Summary:
This tests must be linked with -lintl for the gettext(3) features.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, eugenis, vitalybuka

Reviewed By: vitalybuka

Subscribers: llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D41013

llvm-svn: 320226
2017-12-09 00:39:04 +00:00
Kamil Rytarowski ff98462be3 Disable test/msan/tsearch.cc on NetBSD
Summary:
This test uses GNU-specific extension to libc: tdestroy() and as-is is not compatible with NetBSD.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, eugenis, vitalybuka

Reviewed By: vitalybuka

Subscribers: llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D41011

llvm-svn: 320225
2017-12-09 00:37:58 +00:00
Kamil Rytarowski a85d348111 Disable test/msan/pvalloc.cc on NetBSD
Summary:
The pvalloc(3) function is a non-standard extension missing on NetBSD.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, eugenis, vitalybuka

Reviewed By: vitalybuka

Subscribers: llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D41014

llvm-svn: 320224
2017-12-09 00:37:20 +00:00
Kamil Rytarowski 35542495db Fix test/msan/ifaddrs.cc for NetBSD
Summary:
NetBSD requires to include <sys/socket.h> for struct sockaddr.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, eugenis, vitalybuka

Reviewed By: vitalybuka

Subscribers: llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D41015

llvm-svn: 320223
2017-12-09 00:36:49 +00:00
Kamil Rytarowski 008f63c3c9 Disable test/msan/ftime.cc on NetBSD
Summary:
ftime(3) has been removed from libc/NetBSD.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, eugenis, vitalybuka

Reviewed By: vitalybuka

Subscribers: llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D41018

llvm-svn: 320222
2017-12-09 00:36:17 +00:00
Kamil Rytarowski fb22504594 NetBSD ships with POSIX strerror_r(3)
Summary:
Switch NetBSD from GNU to the POSIX strerror_r(3) interceptor.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, eugenis, vitalybuka

Reviewed By: vitalybuka

Subscribers: srhines, kubamracek, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D41019

llvm-svn: 320221
2017-12-09 00:35:44 +00:00
Kamil Rytarowski 9ecd493129 Add NetBSD/x86_64 mapping in MSan
Summary:
Reuse the Linux new mapping as it is.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, vitalybuka, eugenis

Reviewed By: vitalybuka

Subscribers: llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D41023

llvm-svn: 320218
2017-12-09 00:27:52 +00:00
Matt Morehouse ffb1132e52 [libFuzzer] Add support for Fuchsia OS.
Summary:
This patch adds the initial support for Fuchsia.

- LIBFUZZER_FUCHSIA is added as an OS type in FuzzerDefs.h
- Fuchsia is, by design, not POSIX compliant. However, it does use ELF and
  supports common POSIX I/O functions. Thus, FuzzerExtFunctions.h and
  FuzzerIO.h are implemented by extending the header guards in
  FuzzerExtFunctionsWeak.cpp and FuzzerIOPosix.cpp to include
  LIBFUZZER_FUCHSIA.
- The platform-specific portions of FuzzerUtil.h are implemented by
  FuzzerUtilFuchsia.cpp, which makes use of exception ports, syscalls, and
  the launchpad library.
- The experimental equivalence server is not currently supported, so
  FuzzerShmem.h is implemented by stub methods in FuzzerShmemFuchsia.cpp.
  Any future implementation will likely involve VMOs.

Tested with ASAN/SanCov on Fuchsia/x86-64 with the canonical toy fuzzer.

Patch By: aarongreen

Reviewers: kcc, morehouse, flowerhack, phosek

Reviewed By: kcc, phosek, Eugene.Zelenko

Subscribers: srhines, mgorny, Eugene.Zelenko

Differential Revision: https://reviews.llvm.org/D40974

llvm-svn: 320210
2017-12-08 22:54:44 +00:00
Kostya Serebryany 2659c63e2e [libFuzzer] honor -use_counters, sligntly change the meaning of -experimental_len_control, call UpdateFeatureFrequency only if instructed by the flag
llvm-svn: 320205
2017-12-08 22:21:42 +00:00
Xinliang David Li 3905953582 Update test case for r320180
llvm-svn: 320181
2017-12-08 19:38:42 +00:00
Kostya Kortchinsky 9fcb91b3eb [scudo] Minor code generation improvement
Summary:
It looks like clang was generating somewhat weird assembly with the current
code. `FromPrimary`, even though `const`,  was replaced every time with the code
generated for `size <= SizeClassMap::kMaxSize` instead of using a variable or
register, and `FromPrimary` didn't induce `ClassId != 0` for the compiler, so a
dead branch was generated for `getActuallyAllocatedSize(Ptr, ClassId)` since
it's never called for `ClassId = 0` (Secondary backed allocations) [this one
was more wishful thinking on my side than anything else].

I rearranged the code bit so that the generated assembly is less clunky.

Also changed 2 whitespace inconsistencies that were bothering me.

Reviewers: alekseyshl, flowerhack

Reviewed By: flowerhack

Subscribers: llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D40976

llvm-svn: 320160
2017-12-08 16:36:37 +00:00
Vedant Kumar 75850f57fd [ubsan] Test for pass_object_size bounds checks
llvm-svn: 320129
2017-12-08 01:51:51 +00:00
Bill Seurer c776a9f5a8 [PowerPC][asan] Update asan to handle changed memory layouts in newer kernels
In more recent Linux kernels with 47 bit VMAs the layout of virtual memory
for powerpc64 changed causing the address sanitizer to not work properly. This
patch adds support for 47 bit VMA kernels for powerpc64 and fixes up test
cases.

https://reviews.llvm.org/D40908

There is an associated patch for trunk.

Tested on several 4.x and 3.x kernel releases.

llvm-svn: 320110
2017-12-07 22:53:49 +00:00
Xinliang David Li b0c97e566f Test case update for D40873
llvm-svn: 320105
2017-12-07 22:23:43 +00:00
Evgeniy Stepanov be437e72ef [sanitizer] Simplify android_run.py.
A test-only change to pass all *SAN_OPTIONS to the device without
listing them individually.

llvm-svn: 319998
2017-12-07 01:28:44 +00:00
Kostya Serebryany a97bd9a7d3 [libFuzzer] Decrease stack usage in unit tests
Summary: With 3 Dictionary objects, each containing space of ~16k DictionaryEntry objects, the MutationDispatcher object is fairly memory heavy.  On platforms with a lower default stack size, this can cause panics in FuzzerUnittest as those tests stack-allocate the MutationDispatcher.  This may be especially problematic for platforms that do not (yet) have a way to programmatically change their stack size, aside from link-time flags.  In general, it seems more prudent to use the heap for an object of this size.

Reviewers: kcc, morehouse

Reviewed By: kcc

Differential Revision: https://reviews.llvm.org/D40926

llvm-svn: 319988
2017-12-06 23:35:02 +00:00
Vlad Tsyrklevich 5407edd375 Update other SizeEnums to be of type uptr as well
llvm-svn: 319984
2017-12-06 23:02:02 +00:00
Kamil Rytarowski 0f5cb013d0 Revert SVN r, 319967
"Correct atexit(3) support in MSan/NetBSD"

This causes failures on Linux.

llvm-svn: 319981
2017-12-06 22:50:12 +00:00
Vlad Tsyrklevich 03ad4e7f46 Fix broken windows sanitizer buildbot
r319875 caused a sign comparison build failure. Explicitly set the
enum's type to be unsigned.

llvm-svn: 319977
2017-12-06 22:40:23 +00:00
Kostya Serebryany 7ac58ee3e1 [libFuzzer] fix a minor regression in printing
llvm-svn: 319975
2017-12-06 22:12:24 +00:00
Kamil Rytarowski 0192818ffe Correct atexit(3) support in MSan/NetBSD
Summary:
The NetBSD specific implementation of cxa_atexit() does not
preserve the 2nd argument if dso is equal to NULL.

Changes:

 - Split paths of handling intercepted __cxa_atexit() and atexit(3).
   This affects all supported Operating Systems.
 - Add a local stack-like structure to hold the __cxa_atexit() context.
   atexit(3) is documented in the C standard as calling callback from the
   earliest to the oldest entry. This path also fixes potential ABI
   problem of passing an argument to a function from the atexit(3)
   callback mechanism.
 - Allow usage of global vars with ctors in interceptors.
   This allows to use Vector without automatic cleaning up the structures.

This code has been modeled after TSan implementation for the same functions.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, dvyukov, eugenis, vitalybuka, kcc

Reviewed By: vitalybuka

Subscribers: llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D40714

llvm-svn: 319967
2017-12-06 21:57:39 +00:00
Kamil Rytarowski bf228b9200 Handle NetBSD symbol renaming in msan_interceptors.cc
Summary:
NetBSD renames symbols for historical and compat reasons.

Add required symbol renames in sanitizer_common_interceptors.inc:

 - gettimeofday -> __gettimeofday50
 - getrusage -> __getrusage50
 - shmctl -> __shmctl50

Additionally handle sigaction symbol mangling.
Rename the function symbol in the file to SIGACTION_SYMNAME and define
it as __sigaction14 for NetBSD and sigaction for !NetBSD. We cannot use
simple renaming with the proprocessor, as there are valid fields named
sigaction and they must be left intact.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, eugenis, vitalybuka, dvyukov

Reviewed By: vitalybuka

Subscribers: kubamracek, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D40766

llvm-svn: 319966
2017-12-06 21:32:57 +00:00
Ben Hamilton f20525bc87 [compiler-rt] Set up .arcconfig to point to new Diffusion CRT repository
Summary:
We want to automatically copy the appropriate mailing list
as well as #sanitizers for review requests to the compiler-rt repository.

For context, see the proposal and discussion here:

http://lists.llvm.org/pipermail/cfe-dev/2017-November/056032.html

Similar to D40179, I set up a new Diffusion repository with callsign
"CRT" for compiler-rt:

https://reviews.llvm.org/source/compiler-rt/

This explicitly updates compiler-rt's .arcconfig to point to the new
CRT repository in Diffusion, which will let us use Herald rules H270
and H271.

Reviewers: krytarowski, joerg, dvyukov, vitalybuka, sammccall

Reviewed By: sammccall

Subscribers: dlj, bkramer, dberris, llvm-commits, sammccall, klimek, #sanitizers

Differential Revision: https://reviews.llvm.org/D40498

llvm-svn: 319954
2017-12-06 20:55:32 +00:00
Alex Shlyapnikov e85b4df435 [TSan] Make more TSan interceptors symbolizer-aware.
Summary:
Switching the rest of intercepted allocs to InternalAlloc (well, except
__libc_memalign) when current thread is 'in_symbolizer'. Symbolizer
might (and does) use allocation functions other than malloc/calloc/realloc.

posix_memalign is the one actually used, others switched just in case
(since the failure is obscure and not obvious to diagnose).

Reviewers: dvyukov

Subscribers: llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D40877

llvm-svn: 319929
2017-12-06 19:12:57 +00:00
Kamil Rytarowski 90b46353e7 [Sanitizers] Use SANITIZER_* macros in lib/interception
Summary:
Unlike the rest of the sanitizer code, lib/interception uses native macros like __linux__
to check for specific targets instead of the common ones like SANITIZER_LINUX.

When working on the Solaris port of the sanitizers, the current style was found to not
only be inconsistent, but clumsy to use because the canonical way to check for Solaris
is to check for __sun__ && __svr4__ which is a mouthful.

Therefore, this patch switches to use SANITIZER_* macros instead.

Tested on x86_64-pc-linux-gnu.

Reviewers: kcc, vitalybuka

Reviewed By: vitalybuka

Subscribers: #sanitizers, srhines, krytarowski, llvm-commits, fedor.sergeev

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D39798

llvm-svn: 319906
2017-12-06 17:02:00 +00:00
Kostya Kortchinsky ddf4ef3959 [scudo] Correct performance regression in Secondary
Summary:
This wasn't noticed: `RoundUpTo` doesn't produce a constant expression, so the
sizes were not constant either. Enforce them to be static const, replace
`RoundUpTo` by its expression. The compiler can now optimize the associated
computations accordingly.

Also looking at the produced assembly, `PageSize` was fetched multiple times
during `Allocate`, so keep a local value of it. As a result it's fetched once
and kept in a register.

Reviewers: alekseyshl, flowerhack

Reviewed By: alekseyshl

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D40862

llvm-svn: 319903
2017-12-06 16:53:24 +00:00
Simon Dardis b0f8e3511b [compiler-rt][xray][cmake] Fix a build issue caused by set/item mixup
r319165 introduced a change to CMakeLists.txt for xray where the set of supported
architectures for XRay was iterated over, tested if they could be targeted then
passed to add_compiler_rt_object_libraries. However all targets were passed,
rather than the architecture that was just tested. For cases such as MIPS, where
mips and mips64 are supported, cmake would then test if mips64 could be targetted
resulting in an attempt to produce multiple identical logical target names, falling
afowl of CMP0002.

Reviewers: dberris

Differential Revision: https://reviews.llvm.org/D40890

llvm-svn: 319893
2017-12-06 14:03:41 +00:00
Kamil Rytarowski a149c1a517 Fix typo fcvr -> fcvt and attempt to unbreak MSan/!NetBSD
llvm-svn: 319872
2017-12-06 01:44:41 +00:00
Kamil Rytarowski b6778ff9d2 Fix typo gcvr -> gcvt and attempt to unbreak MSan/!NetBSD
llvm-svn: 319871
2017-12-06 01:43:38 +00:00
Kamil Rytarowski 8cec32cc5f Support pthread_key_create symbol alias in MSan/NetBSD
Summary:
NetBSD uses indirection symbol for a set of threading functions.

Add alias to handle __libc_thr_keycreate the same way as pthread_key_create.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, dvyukov, eugenis, vitalybuka

Reviewed By: vitalybuka

Subscribers: llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D40767

llvm-svn: 319868
2017-12-06 01:16:49 +00:00
Kamil Rytarowski 993447c830 Disable absent functions in MSan/NetBSD interceptors
Summary:
Disable for NetBSD missing functions missing in this OS:

 - mempcpy,
 - __libc_memalign,
 - malloc_usable_size,
 - stpcpy,
 - gcvt,
 - wmempcpy,
 - fcvt.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, dvyukov, eugenis, vitalybuka, kcc

Reviewed By: vitalybuka

Subscribers: llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D40764

llvm-svn: 319866
2017-12-06 01:03:11 +00:00
Vitaly Buka b791cf3e46 [msan] Fix formatting
llvm-svn: 319844
2017-12-05 22:10:01 +00:00
Vitaly Buka f7b63c5f2b [msan] add strtouq msan interceptor
Summary: Fixes https://github.com/google/sanitizers/issues/892

Reviewers: eugenis

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D40853

llvm-svn: 319843
2017-12-05 22:07:26 +00:00
Xinliang David Li 52967ea5b1 Revert test case change for r319794
llvm-svn: 319842
2017-12-05 21:54:20 +00:00
Xinliang David Li 06be6debbc [PGO] fix test case for D40702
Differential Revision: http://reviews.llvm.org/D40702

llvm-svn: 319795
2017-12-05 17:19:49 +00:00
Matt Morehouse 056774d13b [libFuzzer] Make redirects happen in proper sequence.
"> file" must come before "2>&1" to have redirection occur correctly in
all cases.  Fixes a regression on minimize_two_crashes.test.

llvm-svn: 319792
2017-12-05 17:13:17 +00:00
Kostya Kortchinsky df6ba242bf [scudo] Get rid of the thread local PRNG & header salt
Summary:
It was deemed that the salt in the chunk header didn't improve security
significantly (and could actually decrease it). The initial idea was that the
same chunk would different headers on different allocations, allowing for less
predictability. The issue is that gathering the same chunk header with different
salts can give information about the other "secrets" (cookie, pointer), and that
if an attacker leaks a header, they can reuse it anyway for that same chunk
anyway since we don't enforce the salt value.

So we get rid of the salt in the header. This means we also get rid of the
thread local Prng, and that we don't need a global Prng anymore as well. This
makes everything faster.

We reuse those 8 bits to store the `ClassId` of a chunk now (0 for a secondary
based allocation). This way, we get some additional speed gains:
- `ClassId` is computed outside of the locked block;
- `getActuallyAllocatedSize` doesn't need the `GetSizeClass` call;
- same for `deallocatePrimary`;
We add a sanity check at init for this new field (all sanity checks are moved
in their own function, `init` was getting crowded).

Reviewers: alekseyshl, flowerhack

Reviewed By: alekseyshl

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D40796

llvm-svn: 319791
2017-12-05 17:08:29 +00:00
Dean Michael Berris 21d0d531e5 [XRay][compiler-rt] Use __sanitizer::Atexit() instead of atexit()
Follow-up to D40828.

llvm-svn: 319764
2017-12-05 13:40:01 +00:00
Dean Michael Berris 52517d7cf7 [XRay][compiler-rt] Implement XRay Basic Mode Filtering
Summary:
This change implements the basic mode filtering similar to what we do in
FDR mode. The implementation is slightly simpler in basic-mode filtering
because we have less details to remember, but the idea is the same. At a
high level, we do the following to decide when to filter function call
records:

  - We maintain a per-thread "shadow stack" which keeps track of the
    XRay instrumented functions we've encountered in a thread's
    execution.
  - We push an entry onto the stack when we enter an XRay instrumented
    function, and note the CPU, TSC, and type of entry (whether we have
    payload or not when entering).
  - When we encounter an exit event, we determine whether the function
    being exited is the same function we've entered recently, was
    executing in the same CPU, and the delta of the recent TSC and the
    recorded TSC at the top of the stack is less than the equivalent
    amount of microseconds we're configured to ignore -- then we un-wind
    the record offset an appropriate number of times (so we can
    overwrite the records later).

We also support limiting the stack depth of the recorded functions,
so that we don't arbitrarily write deep function call stacks.

Reviewers: eizan, pelikan, kpw, dblaikie

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D40828

llvm-svn: 319762
2017-12-05 12:21:14 +00:00
Dean Michael Berris c360f41097 [XRay][compiler-rt] Implement logging implementation registration
Summary:
This change allows for registration of multiple logging implementations
through a central mechanism in XRay, mapping an implementation to a
"mode". Modes are strings that are used as keys to determine which
implementation to install through a single API. This mechanism allows
users to choose which implementation to install either from the
environment variable 'XRAY_OPTIONS' with the `xray_mode=` flag, or
programmatically using the `__xray_select_mode(...)` function.

Here, we introduce two API functions for the XRay logging:

__xray_log_register_mode(Mode, Impl): Associates an XRayLogImpl to a
string Mode. We can only have one implementation associated with a given
Mode.

__xray_log_select_mode(Mode): Finds the associated Impl for Mode and
installs it as if by calling `__xray_set_log_impl(...)`.

Along with these changes, we also deprecate the xray_naive_log and
xray_fdr_log flags and encourage users to instead use the xray_mode
flag.

Reviewers: kpw, dblaikie, eizan, pelikan

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D40703

llvm-svn: 319759
2017-12-05 12:08:56 +00:00
Matt Morehouse e911a5d356 [libFuzzer] Remove const from ignoreRemainingArgs return value.
In this case const does nothing but trigger a warning.

llvm-svn: 319685
2017-12-04 20:06:52 +00:00
Matt Morehouse dc81fd6cec [libFuzzer] Remove FuzzerCommand.cpp from build sources.
FuzzerCommand.cpp was not introduced in r319680.  Instead, it was
implemented in the header file.

llvm-svn: 319682
2017-12-04 19:54:14 +00:00
Alex Shlyapnikov 9842821461 [ASan] Disable invalid-pointer-pairs-threads.cc on Darwin.
pthread barriers are not available on OS X

Differential revision: https://reviews.llvm.org/D40600

llvm-svn: 319681
2017-12-04 19:40:39 +00:00
Matt Morehouse 04304d129b [libFuzzer] Encapsulate commands in a class.
Summary:
To be more portable (especially w.r.t. platforms without system()),
commands should be managed programmatically rather than via string
manipulation on the command line. This change introduces
Fuzzer::Command, with methods to manage arguments and flags, set output
options, and execute the command.

Patch By: aarongreen

Reviewers: kcc, morehouse

Reviewed By: kcc, morehouse

Subscribers: llvm-commits, mgorny

Differential Revision: https://reviews.llvm.org/D40103

llvm-svn: 319680
2017-12-04 19:25:59 +00:00
Kostya Kortchinsky d4a774d57f [sanitizer] 64-bit allocator's PopulateFreeArray partial refactor
Summary:
This is an attempt at making `PopulateFreeArray` less obscure, more consistent,
and a tiny bit faster in some circumstances:
- use more consistent variable names, that work both for the user & the metadata
  portions of the code; the purpose of the code is mostly the same for both
  regions, so it makes sense that the code should be mostly similar as well;
- replace the while sum loops with a single `RoundUpTo`;
- mask most of the metadata computations behind kMetadataSize, allowing some
  blocks to be completely optimized out if not use metadata;
- `const` the constant variables;
- add a `LIKELY` as the branch it applies to will almost always be taken.

Reviewers: alekseyshl, flowerhack

Reviewed By: alekseyshl

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D40754

llvm-svn: 319673
2017-12-04 18:56:38 +00:00
Alex Shlyapnikov ebbbf0eb8c [ASan] Fix test/asan/TestCases/Posix/invalid-pointer-pairs-threads.cc
Explicitly add pthread for asan-dynamic config.

Differential revision: https://reviews.llvm.org/D40600

llvm-svn: 319671
2017-12-04 18:35:25 +00:00
Kamil Rytarowski 8e0eca0e48 Handle NetBSD symbol renaming in sanitizer_common_interceptors.inc
Summary:
NetBSD renames symbols for historical and compat reasons.

Add required symbol renames in sanitizer_common_interceptors.inc:

 - clock_gettime -> __clock_gettime50
 - clock_getres -> __clock_getres50
 - clock_settime -> __clock_settime50
 - setitimer -> __setitimer50
 - getitimer -> __getitimer50
 - opendir -> __opendir30
 - readdir -> __readdir30
 - time -> __time50
 - localtime_r -> __localtime_r50
 - gmtime_r -> __gmtime_r50
 - gmtime -> __gmtime50
 - ctime -> __ctime50
 - ctime_r -> __ctime_r50
 - mktime -> __mktime50
 - getpwnam -> __getpwnam50
 - getpwuid -> __getpwuid50
 - getpwnam_r -> __getpwnam_r50
 - getpwuid_r -> __getpwuid_r50
 - getpwent -> __getpwent50
 - glob -> __glob30
 - wait3 -> __wait350
 - wait4 -> __wait450
 - readdir_r -> __readdir_r30
 - setlocale -> __setlocale50
 - scandir -> __scandir30
 - sigtimedwait -> __sigtimedwait50
 - sigemptyset -> __sigemptyset14
 - sigfillset -> __sigfillset14
 - sigpending -> __sigpending14
 - sigprocmask -> __sigprocmask14
 - shmctl -> __shmctl50
 - times -> __times13
 - stat -> __stat50
 - getutent -> __getutent50
 - getutxent -> __getutxent50
 - getutxid -> __getutxid50
 - getutxline -> __getutxline50

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, vitalybuka, cryptoad, dvyukov

Reviewed By: cryptoad

Subscribers: llvm-commits, kubamracek, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D40765

llvm-svn: 319670
2017-12-04 18:24:15 +00:00
Sterling Augustine 83df523330 These tests require x86 80-bit doubles. Mark them so.
llvm-svn: 319669
2017-12-04 18:18:51 +00:00
Alex Shlyapnikov c73d1e28f1 [ASan] Enhance libsanitizer support for invalid-pointer-pair.
Following patch adds support of all memory origins in
CheckForInvalidPointerPair function. For small difference of pointers,
it's directly done in shadow memory (the limit was set to 2048B).
Then we search for origin of first pointer and verify that the second
one has the same origin. If so, we verify that it points either to a same
variable (in case of stack memory or a global variable), or to a same
heap segment.

Committing on behanf of marxin and jakubjelinek.

Reviewers: alekseyshl, kcc

Subscribers: llvm-commits

Differential revision: https://reviews.llvm.org/D40600

llvm-svn: 319668
2017-12-04 18:00:24 +00:00
Kamil Rytarowski 64fc9cf2e5 Move __tsan::Vector to __sanitizer
Summary:
The low-fat STL-like vector container will be reused in MSan.

It is needed to implement an atexit(3) interceptor on NetBSD/amd64 in MSan.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, dvyukov, eugenis, vitalybuka, kcc

Reviewed By: dvyukov

Subscribers: kubamracek, mgorny, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D40726

llvm-svn: 319650
2017-12-04 12:30:09 +00:00
Kostya Serebryany de9bafb162 [libFuzzer] add a flag -malloc_limit_mb
llvm-svn: 319590
2017-12-01 22:12:04 +00:00
Roman Lebedev d45054dbd2 [ubsan] Re-commit: lit changes for lld testing, future lto testing.
Summary:
As discussed in https://github.com/google/oss-fuzz/issues/933,
it would be really awesome to be able to use ThinLTO for fuzzing.
However, as @kcc has pointed out, it is currently undefined (untested)
whether the sanitizers actually function properly with LLD and/or LTO.

This patch is inspired by the cfi test, which already do test with LTO
(and/or LLD), since LTO is required for CFI to function.

I started with UBSan, because it's cmakelists / lit.* files appeared
to be the cleanest. This patch adds the infrastructure to easily add
LLD and/or LTO sub-variants of the existing lit test configurations.

Also, this patch adds the LLD flavor, that explicitly does use LLD to link.
The check-ubsan does pass on my machine. And to minimize the [initial]
potential buildbot breakage i have put some restrictions on this flavour.

Please review carefully, i have not worked with lit/sanitizer tests before.

The original attempt, r319525 was reverted in r319526 due
to the failures in compiler-rt standalone builds.

Reviewers: eugenis, vitalybuka

Reviewed By: eugenis

Subscribers: #sanitizers, pcc, kubamracek, mgorny, llvm-commits, mehdi_amini, inglorion, kcc

Differential Revision: https://reviews.llvm.org/D39508

llvm-svn: 319575
2017-12-01 19:36:29 +00:00
Kostya Serebryany 6afa7a54be [libFuzzer] remove stale flags; NFC
llvm-svn: 319572
2017-12-01 19:24:06 +00:00
Kostya Serebryany ad05ee0512 [libFuzzer] add an experimental search heuristic flag -reduce_depth
llvm-svn: 319571
2017-12-01 19:18:38 +00:00
Shoaib Meenai 5cd0f44abc [compiler-rt] Remove out of date comment
Per beanz, building compiler-rt standalone is a pretty important use
case, so the comment is very out of date.

Differential Revision: https://reviews.llvm.org/D40740

llvm-svn: 319570
2017-12-01 19:06:30 +00:00
Shoaib Meenai 7f963b4cee [compiler-rt] Add install-*-stripped targets
These targets strip during installation, and are required to support
install-distribution-stripped in LLVM (to support a stripped
distribution). LLVM has an add_llvm_install_targets function for this
purpose, but we can't rely on LLVM being present.

Differential Revision: https://reviews.llvm.org/D40687

llvm-svn: 319569
2017-12-01 19:06:29 +00:00
Ismail Donmez b8dd5078ee Add missing signal.h header:
/havana/work/llvm/projects/compiler-rt/test/asan/TestCases/Linux/swapcontext_annotation.cc:158:20: error: variable has incomplete type 'struct sigaction'
  struct sigaction act = {};
                   ^
/havana/work/llvm/projects/compiler-rt/test/asan/TestCases/Linux/swapcontext_annotation.cc:158:10: note: forward declaration of 'sigaction'
  struct sigaction act = {};
         ^
/havana/work/llvm/projects/compiler-rt/test/asan/TestCases/Linux/swapcontext_annotation.cc:160:17: error: use of undeclared identifier 'SIGPROF'
  if (sigaction(SIGPROF, &act, 0)) {
                ^
2 errors generated.

llvm-svn: 319532
2017-12-01 11:12:58 +00:00
Roman Lebedev 85653e3db9 Revert "[ubsan] lit changes for lld testing, future lto testing."
This reverts commit r319525.

This change has introduced a problem with the Lit tests build for compiler-rt using Gold: http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux/builds/6047/steps/test%20standalone%20compiler-rt/logs/stdio

llvm-lit: /b/sanitizer-x86_64-linux/build/llvm/utils/lit/lit/TestingConfig.py:101: fatal: unable to parse config file '/b/sanitizer-x86_64-linux/build/llvm/projects/compiler-rt/test/profile/Linux/lit.local.cfg', traceback: Traceback (most recent call last):
  File "/b/sanitizer-x86_64-linux/build/llvm/utils/lit/lit/TestingConfig.py", line 88, in load_from_path
    exec(compile(data, path, 'exec'), cfg_globals, None)
  File "/b/sanitizer-x86_64-linux/build/llvm/projects/compiler-rt/test/profile/Linux/lit.local.cfg", line 37, in <module>
    if root.host_os not in ['Linux'] or not is_gold_linker_available():
  File "/b/sanitizer-x86_64-linux/build/llvm/projects/compiler-rt/test/profile/Linux/lit.local.cfg", line 27, in is_gold_linker_available
    stderr = subprocess.PIPE)
  File "/usr/lib/python2.7/subprocess.py", line 390, in __init__
    errread, errwrite)
  File "/usr/lib/python2.7/subprocess.py", line 1024, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory
llvm-svn: 319529
2017-12-01 10:09:27 +00:00
Roman Lebedev 0d044dc090 [ubsan] lit changes for lld testing, future lto testing.
Summary:
As discussed in https://github.com/google/oss-fuzz/issues/933,
it would be really awesome to be able to use ThinLTO for fuzzing.
However, as @kcc has pointed out, it is currently undefined (untested)
whether the sanitizers actually function properly with LLD and/or LTO.

This patch is inspired by the cfi test, which already do test with LTO
(and/or LLD), since LTO is required for CFI to function.

I started with UBSan, because it's cmakelists / lit.* files appeared
to be the cleanest. This patch adds the infrastructure to easily add
LLD and/or LTO sub-variants of the existing lit test configurations.

Also, this patch adds the LLD flavor, that explicitly does use LLD to link.
The check-ubsan does pass on my machine. And to minimize the [initial]
potential buildbot breakage i have put some restrictions on this flavour.

Please review carefully, i have not worked with lit/sanitizer tests before.

Reviewers: eugenis, vitalybuka

Reviewed By: eugenis

Subscribers: #sanitizers, pcc, kubamracek, mgorny, llvm-commits, mehdi_amini, inglorion, kcc

Differential Revision: https://reviews.llvm.org/D39508

llvm-svn: 319525
2017-12-01 08:38:25 +00:00
Vitaly Buka fbb4bace66 [msan] Fix return type of mbrtowc
Summary: Fixes https://github.com/google/oss-fuzz/issues/1009

Reviewers: eugenis

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D40676

llvm-svn: 319484
2017-11-30 22:15:39 +00:00
Sterling Augustine 9b6943f103 Add powerpc64 to compiler-rt build infrastructure.
Now that we have disabled the run-forever tests, and cleaned up the
intel 80-bit float based tests, we should be able to enable testing
compiler-rt for powerpc64.

llvm-svn: 319474
2017-11-30 21:04:11 +00:00
Sterling Augustine 9750f662f7 Move x86-specific sources to x86-specific source lists.
llvm-svn: 319464
2017-11-30 19:39:33 +00:00
Xinliang David Li 8b97870c44 [PGO] Add a test case for infinite loops
Differential Revision: http://reviews.llvm.org/D40663

llvm-svn: 319463
2017-11-30 19:37:56 +00:00
Alexander Richardson 23e70e6f57 Fix the MIPS baremetal build
Summary:
Currently sys/cachectl.h is used unconditionally on MIPS although it is
only available on Linux and will fail the build when targeting baremetal

Reviewers: petarj

Reviewed By: petarj

Subscribers: sdardis, krytarowski

Differential Revision: https://reviews.llvm.org/D40659

llvm-svn: 319455
2017-11-30 18:37:04 +00:00
Stephan Bergmann 1c14e86a26 Linux needs to include sys/uio.h for readv, preadv
...at least when building against glibc-2.26-16.fc27.x86_64

llvm-svn: 319412
2017-11-30 09:22:12 +00:00
Dean Michael Berris 0e508d69fd [XRay][compiler-rt][Darwin] Use dynamic initialisation as an alternative
Summary:
In cases where we can't use the .preinit_array section (as in Darwin for
example) we instead use dynamic initialisation. We know that this
alternative approach will race with the initializers of other objects at
global scope, but this is strictly better than nothing.

Reviewers: kubamracek, nglevin

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D40599

llvm-svn: 319366
2017-11-29 22:06:12 +00:00
Kuba Mracek ceea5466eb [sanitizer] Add 'strip_env' flag to enable/disable removing sanitizer dylib from DYLD_INSERT_LIBRARIES
On macOS, we usually don't require launching the target with DYLD_INSERT_LIBRARIES anymore. However, it is still necessary when running a target that is not instrumented (and e.g. dlopen's an instrument library later). In any case, ASan and TSan currently remove themselves from the DYLD_INSERT_LIBRARIES environment variable to avoid passing it onto children. This works well e.g. when instrumenting a shell. A problem arises when the target is a non-instrumented shim (e.g. "xcrun") that either re-execs or launches a child that is supposed to get DYLD_INSERT_LIBRARIES propagated. To support this mode, this patch introduces 'strip_env' flag that can be used to keep DYLD_INSERT_LIBRARIES untouched.

Differential Revision: https://reviews.llvm.org/D39991

llvm-svn: 319365
2017-11-29 21:42:48 +00:00
Kostya Kortchinsky cf5b4af820 [scudo] Allow for compile-time choice of the SizeClassMap
Summary:
With this change, we allow someone to chose the `SizeClassMap` they want to use
at compile time via a define.

I feel somewhat unimaginative with the name of the defines, so if someone has a
better idea, let me know. I have been alternating between those and
`SCUDO_USE_xxx_SIZECLASSMAP` which is clearer but also longer. The issue with
those is that it wouldn't be consistent with `SCUDO_TSD_EXCLUSIVE` that should
probably become `SCUDO_USE_EXCLUSIVE_TSD` maybe?

Anyway, naming is hard, and I am not sure what makes more sense!

Reviewers: alekseyshl, flowerhack

Reviewed By: alekseyshl

Subscribers: llvm-commits, srhines

Differential Revision: https://reviews.llvm.org/D40521

llvm-svn: 319350
2017-11-29 19:52:09 +00:00
Kuba Mracek 2183c32444 [compiler-rt] Switch from deprecated TARGET_IPHONE_SIMULATOR to TARGET_OS_SIMULATOR
Differential Revision: https://reviews.llvm.org/D39987

llvm-svn: 319349
2017-11-29 19:47:14 +00:00
Kuba Mracek d102535cf8 [asan] Fix macOS FindDynamicShadowStart to consider the last gap in the VM map
It looks FindDynamicShadowStart has a bug: When iterating over the memory map, we will not consider the very last gap in the address space. Let's fix that.

Differential Revision: https://reviews.llvm.org/D39989

llvm-svn: 319348
2017-11-29 19:44:52 +00:00
Kuba Mracek e73d1f13b6 [asan] Don't crash on fclose(NULL)
It's explicitly forbidden to call fclose with NULL, but at least on Darwin, this succeeds and doesn't segfault. To maintain binary compatibility, ASan should survice fclose(NULL) as well.

Differential Revision: https://reviews.llvm.org/D40053

llvm-svn: 319347
2017-11-29 19:43:11 +00:00
Kuba Mracek 21e6efcb51 [asan] Allow getpwnam(NULL) for binary compatibility
Calling getpwnam(NULL) is probably a bug, but at least on Darwin, such a call succeeds without segfaulting. I have some existing code that relies on that. To maintain binary compatibility, ASan should also survive a call to getpwnam with NULL.

Differential Revision: https://reviews.llvm.org/D40052

llvm-svn: 319344
2017-11-29 19:33:35 +00:00
Kuba Mracek aa4d9e2a66 [sanitizer] Refactor how assembly files are handled
This renames ASM_TSAN_SYMBOL and ASM_TSAN_SYMBOL_INTERCEPTOR to just ASM_SYMBOL and ASM_SYMBOL_INTERCEPTOR, because they can be useful in more places than just TSan. Also introduce a CMake function to add ASM sources to a target.

Differential Revision: https://reviews.llvm.org/D40143

llvm-svn: 319339
2017-11-29 19:27:25 +00:00
Kamil Rytarowski 8e2192adc5 Defer StartBackgroundThread() and StopBackgroundThread() in TSan
Summary:
NetBSD cannot spawn new POSIX thread entities in early
libc and libpthread initialization stage. Defer this to the point
of intercepting the first pthread_create(3) call.

This is the last change that makes Thread Sanitizer functional
on NetBSD/amd64 without downstream patches.

********************
Testing Time: 64.91s
********************
Failing Tests (5):
    ThreadSanitizer-x86_64 :: dtls.c
    ThreadSanitizer-x86_64 :: ignore_lib5.cc
    ThreadSanitizer-x86_64 :: ignored-interceptors-mmap.cc
    ThreadSanitizer-x86_64 :: mutex_lock_destroyed.cc
    ThreadSanitizer-x86_64 :: vfork.cc

  Expected Passes    : 290
  Expected Failures  : 1
  Unsupported Tests  : 83
  Unexpected Failures: 5

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, eugenis, dvyukov, vitalybuka

Reviewed By: dvyukov

Subscribers: kubamracek, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D40583

llvm-svn: 319305
2017-11-29 10:23:59 +00:00
Yi Kong bac0a4fb08 Revert change for LibFuzzer target archs
Broke buildbot.

llvm-svn: 319296
2017-11-29 07:03:11 +00:00
Yi Kong 971c6f32fe [LibFuzzer] Add Android to LibFuzzer's supported OSes
... and a trivial fix that x86_64h arch should also be supported.

Differential Revision: https://reviews.llvm.org/D40592

llvm-svn: 319288
2017-11-29 03:52:44 +00:00
Weiming Zhao e4ba19efdc [compiler-rt] Avoid unnecessarily hiding inline visibility [NFC]
Summary:
having fvisibility=hidden obviates the need for
fvisibility-inlines-hidden.

Reviewers: cryptoad, weimingz, mgorny, vsk, compnerd, peter.smith, nikhgupt

Reviewed By: vsk, nikhgupt

Subscribers: dberris, mgorny

Differential Revision: https://reviews.llvm.org/D40269

llvm-svn: 319257
2017-11-28 23:41:42 +00:00
Dean Michael Berris 1d39d1817b [XRay][compiler-rt] Fix armhf build
rL319241 was a bit too aggressive removing sources dependencies. This
restores the actual required dependency for armhf.

Follow-up to D39114.

llvm-svn: 319255
2017-11-28 23:38:18 +00:00
Dean Michael Berris 28832d2b36 [XRay][compiler-rt] Fix runtime build
This isolates the per-architecture files from the common files
implementing the XRay facilities. Because of the refactoring done in
D39114, we were including the definition of the sources in the archive
twice, causing link-time failures.

Follow-up to D39114.

llvm-svn: 319241
2017-11-28 22:33:07 +00:00
Alex Shlyapnikov 15b71ea646 [LSan] Fix one source of stale segments in the process memory mapping.
Summary:
Load process memory map after updating the same cache to reflect the
umap happening in the process of updating.
Also clear out the buffer in case of failed read of /proc/self/maps (not
the source of stale segments, but can lead to the similar crash).

Reviewers: eugenis

Subscribers: llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D40529

llvm-svn: 319237
2017-11-28 22:15:27 +00:00
Dan Liew 37516b527b [LibFuzzer] Improve comments on `CounterToFeature()` function.
This is based on discussion in https://reviews.llvm.org/D40376 .

The comments try to explain the reason for the current implementation
and note that it might change in the future, so clients should not
rely on this particular implementation.

Differential Revision: https://reviews.llvm.org/D40565

llvm-svn: 319190
2017-11-28 17:41:58 +00:00
Kamil Rytarowski 4e9e9bf0cf Support the setjmp(3) family of functions in TSan/NetBSD
Summary:
This change adds support for the setjmp(3)/longjmp(3)
family of functions on NetBSD.

There are three types of them on NetBSD:

 - setjmp(3) / longjmp(3)
 - sigsetjmp(3) / sigsetjmp(3)
 - _setjmp(3) / _longjmp(3)

Due to historical and compat reasons the symbol
names are mangled:

 - setjmp -> __setjmp14
 - longjmp -> __longjmp14
 - sigsetjmp -> __sigsetjmp14
 - siglongjmp -> __siglongjmp14
 - _setjmp -> _setjmp
 - _longjmp -> _longjmp

This leads to symbol renaming in the existing codebase.

There is no such symbol as __sigsetjmp/__longsetjmp
on NetBSD

Add a comment that GNU-style executable stack
note is not needed on NetBSD. The stack is not
executable without it.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, dvyukov, vitalybuka

Reviewed By: dvyukov

Subscribers: llvm-commits, kubamracek, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D40337

llvm-svn: 319189
2017-11-28 17:35:35 +00:00
Bill Seurer 785294d497 [PowerPC][tsan] Update tsan to handle changed memory layouts in newer kernels
In more recent Linux kernels with 47 bit VMAs the layout of virtual memory
for powerpc64 changed causing the thread sanitizer to not work properly. This
patch adds support for 47 bit VMA kernels for powerpc64.

(second part)

Tested on several 4.x and 3.x kernel releases.

llvm-svn: 319180
2017-11-28 16:28:54 +00:00
Dean Michael Berris 542485f29c [XRay][compiler-rt][Darwin] Minimal XRay build support in Darwin
This change is the first in a series of changes to get the XRay runtime
building on macOS. This first allows us to build the minimal parts of
XRay to get us started on supporting macOS development. These include:

  - CMake changes to allow targeting x86_64 initially.
  - Allowing for building the initialisation routines without
    `.preinit_array` support.
  - Use __sanitizer::SleepForMillis() to work around the lack of
    clock_nanosleep on macOS.
  - Deprecate the xray_fdr_log_grace_period_us flag, and introduce
    the xray_fdr_log_grace_period_ms flag instead, to use
    milliseconds across platforms.

Reviewers: kubamracek

Subscribers: llvm-commits, krytarowski, nglevin, mgorny

Differential Review: https://reviews.llvm.org/D39114

llvm-svn: 319165
2017-11-28 11:49:22 +00:00
Kamil Rytarowski b789ab35b3 Correct mangled_sp on NetBSD/amd64 in TSan
The proper index is 6, not 2.

Patch extracted from https://reviews.llvm.org/D40337

Reviewed and accepted by <dvyukov>.

Sponsored by <The NetBSD Foundation>

llvm-svn: 319163
2017-11-28 11:21:27 +00:00
Kamil Rytarowski 1d67a48124 Handle symbol renaming of sigaction for NetBSD
Summary:
NetBSD uses the __sigaction14 symbol name for historical and compat
reasons for the sigaction(2) function name.

Rename the interceptors and users of sigaction to sigaction_symname
and reuse it in the code base.

This change fixes 4 failing tests in TSan/NetBSD:

 - ThreadSanitizer-x86_64 :: signal_errno.cc
 - ThreadSanitizer-x86_64 :: signal_malloc.cc
 - ThreadSanitizer-x86_64 :: signal_sync2.cc
 - ThreadSanitizer-x86_64 :: signal_thread.cc

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, vitalybuka, eugenis, dvyukov, kcc

Reviewed By: dvyukov

Subscribers: kubamracek, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D40341

llvm-svn: 319160
2017-11-28 10:54:58 +00:00
Evgeniy Stepanov c181d2e773 [msan] Avoid shadowing a variable in common interceptors.
"offset" declared in a macro may shadow a variable with the same name
in the caller which is used in a macro argument. We are quite lucky
that it does not actually happen, but rename the variable anyway to
be on the safe side.

llvm-svn: 319115
2017-11-27 23:25:38 +00:00
Matt Morehouse c5330c240b [TSan] Do not run cond_cancel.c test on ppc64.
After r319004, the expected failure on ppc64 manifests as an infinite
loop.

llvm-svn: 319114
2017-11-27 23:20:47 +00:00
Kostya Kortchinsky 06b891f693 [scudo] Workaround for uninitialized Bionic globals
Summary:
Bionic doesn't initialize its globals early enough. This causes issues when
trying to access them from a preinit_array (b/25751302) or from another
constructor called before the libc one (b/68046352). __progname is initialized
after the other globals, so we can check its value to know if calling
`getauxval` is safe.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: srhines, llvm-commits

Differential Revision: https://reviews.llvm.org/D40504

llvm-svn: 319099
2017-11-27 21:34:43 +00:00
Kostya Kortchinsky fbff7fcdf4 [fuchsia] Update Fuchsia with a new mmap implementation.
Summary:
    Now that the sanitizer_common interface for MmapNoAccess / MmapFixed
    have been refactored to allow a more OO-esque access pattern, update the
    Fuchsia mmap implementation to take advantage of this.
    
    Previously MmapNoAccess / MmapFixed relied on a global allocator_vmar,
    since the sanitizer_allocator only called MmapNoAccess once.  Now, we
    create a new VMAR per ReservedAddressRange object.
    
    This allows the sanitizer allocator to work in tandem with the Scudo
    secondary allocator.
    
    This is part 4 of a 4 part changeset:
    * part 1 https://reviews.llvm.org/D38593
    * part 2 https://reviews.llvm.org/D38592
    * part 3 https://reviews.llvm.org/D38593

Reviewers: mcgrathr, cryptoad

Reviewed By: cryptoad

Subscribers: alekseyshl, mcgrathr, kubamracek, mehdi_amini

Differential Revision: https://reviews.llvm.org/D38595

llvm-svn: 319083
2017-11-27 19:53:53 +00:00
Kamil Rytarowski 27fb9cc994 Build more sanitizers for NetBSD
Summary:
Enable for NetBSD:

 - MSan,
 - TSan,
 - LSan.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, eugenis, dvyukov, vitalybuka

Reviewed By: eugenis

Subscribers: srhines, mgorny, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D40464

llvm-svn: 319060
2017-11-27 17:54:26 +00:00
Kamil Rytarowski 7160c2f770 Prevent Thread Exited/Joined events race
Summary:
Add atomic verification to ensure that Thread is Joined after marking it
Finished.

It is required for NetBSD in order to prevent Thread Exited/Joined race,
that may occur when native system libpthread(3) cannot be reliably traced
in a way to guarantee that the mentioned events happen one after another.

This change fixes at least TSan and LSan on NetBSD.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, dvyukov, vitalybuka

Reviewed By: dvyukov

Subscribers: llvm-commits, kubamracek, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D40294

llvm-svn: 319004
2017-11-26 20:20:42 +00:00
Kamil Rytarowski 20e97ae9a0 Detect thread termination in LSan/NetBSD
Summary:
Stop using the Linux solution with pthread_key_create(3).
This approach does not work on NetBSD, because calling
the thread destructor is not the latest operation on a POSIX
thread entity.

Detect _lwp_exit(2) call as it is really the latest operation
called from a detaching POSIX thread.

The pthread_key_create(3) solution also cannot be used
in early libc/libpthread initialization on NetBSD as the
system libraries are not bootstrapped enough.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, vitalybuka, kcc, dvyukov

Reviewed By: dvyukov

Subscribers: llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D40457

llvm-svn: 318994
2017-11-26 09:42:01 +00:00
Kamil Rytarowski e36f5cb480 Plug dlerror() leak for swift_demangle
Summary:
InitializeSwiftDemangler() attempts to resolve the
swift_demangle symbol. If this is not available, we
observe dlerror message leak.

Caught on NetBSD/amd64 in TSan.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, kubamracek, vitalybuka, dvyukov, eugenis

Reviewed By: dvyukov

Subscribers: llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D40382

llvm-svn: 318980
2017-11-25 16:47:20 +00:00
Kostya Kortchinsky 0207b6fbbf [scudo] Overhaul hardware CRC32 feature detection
Summary:
This patch aims at condensing the hardware CRC32 feature detection and making
it slightly more effective on Android.

The following changes are included:
- remove the `CPUFeature` enum, and get rid of one level of nesting of
  functions: we only used CRC32, so we just implement and use
  `hasHardwareCRC32`;
- allow for a weak `getauxval`: the Android toolchain is compiled at API level
  14 for Android ARM, meaning no `getauxval` at compile time, yet we will run
  on API level 27+ devices. The `/proc/self/auxv` fallback can work but is
  worthless for a process like `init` where the proc filesystem doesn't exist
  yet. If a weak `getauxval` doesn't exist, then fallback.
- couple of extra corrections.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: kubamracek, aemerson, srhines, kristof.beyls, llvm-commits

Differential Revision: https://reviews.llvm.org/D40322

llvm-svn: 318859
2017-11-22 18:30:44 +00:00
Kostya Kortchinsky 5a3fdbd829 [scudo] Make getNumberOfCPUs Fuchsia compliant v2
Summary:
This change allows Fuchsia to boot properly using the Scudo allocator.

A first version of this commit was reverted by rL317834 because it broke Android
builds for toolchains generated with older NDKs. This commit introduces a
fall back to solve that issue.

Reviewers: cryptoad, krytarowski, rnk, alekseyshl

Reviewed By: cryptoad, krytarowski, alekseyshl

Subscribers: llvm-commits, srhines, kubamracek, krytarowski

Differential Revision: https://reviews.llvm.org/D40121

llvm-svn: 318802
2017-11-21 21:14:00 +00:00
Kostya Kortchinsky 2e96469465 [sanitizer] Define SANITIZER_USE_GETAUXVAL for Android
Summary:
Android for API level >= 21 has `getauxval`. Enable `SANITIZER_USE_GETAUXVAL`
when those requirements are met. Correct a typo in the header.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: srhines, llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D40260

llvm-svn: 318775
2017-11-21 16:08:07 +00:00
Kamil Rytarowski d7c85137aa Correct NetBSD support in pthread_once(3)/TSan
Summary:
The pthread_once(3)/NetBSD type is built with the following structure:

struct __pthread_once_st {
 pthread_mutex_t pto_mutex;
 int pto_done;
};

Set the pto_done position as shifted by __sanitizer::pthread_mutex_t_sz
from the beginning of the pthread_once struct.

This corrects deadlocks when the pthread_once(3) function
is used.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, dvyukov, vitalybuka

Reviewed By: dvyukov

Subscribers: llvm-commits, kubamracek, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D40262

llvm-svn: 318742
2017-11-21 09:36:07 +00:00
Dean Michael Berris 364f11cdd3 [XRay][compiler-rt] Migrate basic mode logging to the XRay framework
Summary:
Before this patch, XRay's basic (naive mode) logging would be
initialised and installed in an adhoc manner. This patch ports the
implementation of the basic (naive mode) logging implementation to use
the common XRay framework.

We also make the following changes to reduce the variance between the
usage model of basic mode from FDR (flight data recorder) mode:

  - Allow programmatic control of the size of the buffers dedicated to
    per-thread records. This removes some hard-coded constants and turns
    them into runtime-controllable flags and through an Options
    structure.

  - Default the `xray_naive_log` option to false. For now, the only way
    to start basic mode is to set the environment variable, or set the
    default at build-time compiler options. Because of this change we've
    had to update a couple of tests relying on basic mode being always
    on.

  - Removed the reliance on a non-trivially destructible per-thread
    resource manager. We use a similar trick done in D39526 to use
    pthread_key_create() and pthread_setspecific() to ensure that the
    per-thread cleanup handling is performed at thread-exit time.

We also radically simplify the code structure for basic mode, to move
most of the implementation in the `__xray` namespace.

Reviewers: pelikan, eizan, kpw

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D40164

llvm-svn: 318734
2017-11-21 07:29:21 +00:00
Dean Michael Berris 6ec72625f2 [XRay] Use optimistic logging model for FDR mode
Summary:
Before this change, the FDR mode implementation relied on at thread-exit
handling to return buffers back to the (global) buffer queue. This
introduces issues with the initialisation of the thread_local objects
which, even through the use of pthread_setspecific(...) may eventually
call into an allocation function. Similar to previous changes in this
line, we're finding that there is a huge potential for deadlocks when
initialising these thread-locals when the memory allocation
implementation is also xray-instrumented.

In this change, we limit the call to pthread_setspecific(...) to provide
a non-null value to associate to the key created with
pthread_key_create(...). While this doesn't completely eliminate the
potential for the deadlock(s), it does allow us to still clean up at
thread exit when we need to. The change is that we don't need to do more
work when starting and ending a thread's lifetime. We also have a test
to make sure that we actually can safely recycle the buffers in case we
end up re-using the buffer(s) available from the queue on multiple
thread entry/exits.

This change cuts across both LLVM and compiler-rt to allow us to update
both the XRay runtime implementation as well as the library support for
loading these new versions of the FDR mode logging. Version 2 of the FDR
logging implementation makes the following changes:

  * Introduction of a new 'BufferExtents' metadata record that's outside
    of the buffer's contents but are written before the actual buffer.
    This data is associated to the Buffer handed out by the BufferQueue
    rather than a record that occupies bytes in the actual buffer.

  * Removal of the "end of buffer" records. This is in-line with the
    changes we described above, to allow for optimistic logging without
    explicit record writing at thread exit.

The optimistic logging model operates under the following assumptions:

  * Threads writing to the buffers will potentially race with the thread
    attempting to flush the log. To avoid this situation from occuring,
    we make sure that when we've finalized the logging implementation,
    that threads will see this finalization state on the next write, and
    either choose to not write records the thread would have written or
    write the record(s) in two phases -- first write the record(s), then
    update the extents metadata.

  * We change the buffer queue implementation so that once it's handed
    out a buffer to a thread, that we assume that buffer is marked
    "used" to be able to capture partial writes. None of this will be
    safe to handle if threads are racing to write the extents records
    and the reader thread is attempting to flush the log. The optimism
    comes from the finalization routine being required to complete
    before we attempt to flush the log.

This is a fairly significant semantics change for the FDR
implementation. This is why we've decided to update the version number
for FDR mode logs. The tools, however, still need to be able to support
older versions of the log until we finally deprecate those earlier
versions.

Reviewers: dblaikie, pelikan, kpw

Subscribers: llvm-commits, hiraditya

Differential Revision: https://reviews.llvm.org/D39526

llvm-svn: 318733
2017-11-21 07:16:57 +00:00
Walter Lee 6c529f7e96 [sanitizers] Add init function to set alignment of low level allocator
ASan requires that the min alignment be at least the shadow
granularity, so add an init function to do that.

Differential Revision: https://reviews.llvm.org/D39473

llvm-svn: 318717
2017-11-21 01:01:32 +00:00
Vitaly Buka 7f6bb4ab1f [tsan] Fix sigaction implementation when it's called only to get handler
Reviewers: eugenis

Subscribers: kubamracek, llvm-commits, krytarowski

Differential Revision: https://reviews.llvm.org/D40272

llvm-svn: 318707
2017-11-20 23:37:46 +00:00
Kamil Rytarowski 37d3288cf5 Try to fix syntax for !NetBSD in TSAN_INTERCEPTORs
llvm-svn: 318676
2017-11-20 18:27:39 +00:00
Kamil Rytarowski 60cc1d3218 Handle NetBSD specific indirection of libpthread functions
Summary:
Correct handling of libpthread(3) functions in TSan/NetBSD:

 - pthread_cond_init(3),
 - pthread_cond_signal(3),
 - pthread_cond_broadcast(3),
 - pthread_cond_wait(3),
 - pthread_cond_destroy(3),
 - pthread_mutex_init(3),
 - pthread_mutex_destroy(3),
 - pthread_mutex_trylock(3),
 - pthread_rwlock_init(3),
 - pthread_rwlock_destroy(3),
 - pthread_rwlock_rdlock(3),
 - pthread_rwlock_tryrdlock(3),
 - pthread_rwlock_wrlock(3),
 - pthread_rwlock_trywrlock(3),
 - pthread_rwlock_unlock(3),
 - pthread_once(3).

Code out of the libpthread(3) context uses the libc symbols
that are prefixed with __libc_, for example: __libc_cond_init.

This caused that these functions were invisible to sanitizers on NetBSD.
Intercept the libc-specific ones and add them as NetBSD-specific aliases
for the common pthread(3) ones.

NetBSD needs to intercept both functions, as the regularly named ones
are used internally in libpthread(3).

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, dvyukov, vitalybuka

Reviewed By: dvyukov

Subscribers: kubamracek, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D40243

llvm-svn: 318673
2017-11-20 18:07:52 +00:00
Kamil Rytarowski 00ab6fbb2e Add DemangleFunctionName for backtracing on NetBSD
Summary:
NetBSD uses indirection for old threading functions for historical reasons
The mangled names are internal implementation detail and should not be
exposed even in backtraces.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, vitalybuka, dvyukov

Reviewed By: dvyukov

Subscribers: kubamracek, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D40251

llvm-svn: 318671
2017-11-20 18:06:26 +00:00
Evgeniy Stepanov 8e7018d92f [asan] Use dynamic shadow on 32-bit Android, try 2.
Summary:
This change reverts r318575 and changes FindDynamicShadowStart() to
keep the memory range it found mapped PROT_NONE to make sure it is
not reused. We also skip MemoryRangeIsAvailable() check, because it
is (a) unnecessary, and (b) would fail anyway.

Reviewers: pcc, vitalybuka, kcc

Subscribers: srhines, kubamracek, mgorny, llvm-commits, hiraditya

Differential Revision: https://reviews.llvm.org/D40203

llvm-svn: 318666
2017-11-20 17:41:57 +00:00
Kostya Kortchinsky a695e418f3 [sanitizer] New attempt at using runtime checks for Android logging
Summary:
This is a second attempt after D40100 induced racey crashes with ASan
(due to `__android_log_write` and the `strncpy` interceptor on API >= 21).

This new version checks the runtime API level to be <= `ANDROID_KITKAT` for
the use  of `__android_log_write`, otherwise we use `syslog`, which should
conform with the previous behavior.

Unfortunately despite numerous efforts I couldn't reproduce the original
crashes in my environments so I couldn't test that the fix was actually
preventing crashes.

Reviewers: eugenis

Reviewed By: eugenis

Subscribers: llvm-commits, srhines, kubamracek

Differential Revision: https://reviews.llvm.org/D40149

llvm-svn: 318659
2017-11-20 16:39:49 +00:00
Kamil Rytarowski 2419452505 Correct return types of NetBSD specific functions
The __libc_mutex_lock, __libc_mutex_unlock and __libc_thr_setcancelstate
functions return int, not void.

This does not seem to introduce a functional change, however it looks
better with fixed the function prototype.

Sponsored by <The NetBSD Foundation>

llvm-svn: 318654
2017-11-20 15:39:30 +00:00
Kamil Rytarowski 86bff788eb Handle NetBSD specific indirection of libpthread functions
Summary:
Correct handling of three libpthread(3) functions on NetBSD:

 - pthread_mutex_lock(3),
 - pthread_mutex_unlock(3),
 - pthread_setcancelstate(3).

Code out of the libpthread(3) context uses the libc symbols:

 - __libc_mutex_lock,
 - __libc_mutex_unlock,
 - __libc_thr_setcancelstate.

The threading library (libpthread(3)) defines strong aliases:

 - __strong_alias(__libc_mutex_lock,pthread_mutex_lock)
 - __strong_alias(__libc_mutex_unlock,pthread_mutex_unlock)
 - __strong_alias(__libc_thr_setcancelstate,pthread_setcancelstate)

This caused that these functions were invisible to sanitizers on NetBSD.
Intercept the libc-specific ones and add them as NetBSD-specific aliases
for the common pthread(3) ones.

NetBSD needs to intercept both functions, as the regularly named ones
are used internally in libpthread(3).

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, dvyukov, vitalybuka

Reviewed By: dvyukov

Subscribers: llvm-commits, kubamracek, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D40241

llvm-svn: 318646
2017-11-20 12:21:11 +00:00
Craig Topper 436c9263fb [X86] Add cpu detection for cannonlake.
This uses the same encoding for cannonlake in the proposed gcc patches here. https://gcc.gnu.org/ml/gcc-patches/2017-11/msg00551.html

llvm-svn: 318610
2017-11-19 00:46:21 +00:00
Evgeniy Stepanov 9d564cdcb0 Revert "[asan] Use dynamic shadow on 32-bit Android" and 3 more.
Revert the following commits:
  r318369 [asan] Fallback to non-ifunc dynamic shadow on android<22.
  r318235 [asan] Prevent rematerialization of &__asan_shadow.
  r317948 [sanitizer] Remove unnecessary attribute hidden.
  r317943 [asan] Use dynamic shadow on 32-bit Android.

MemoryRangeIsAvailable() reads /proc/$PID/maps into an mmap-ed buffer
that may overlap with the address range that we plan to use for the
dynamic shadow mapping. This is causing random startup crashes.

llvm-svn: 318575
2017-11-18 00:22:34 +00:00
Evgeniy Stepanov 0b44f44bcf [asan] Fix asan_device_setup on KitKat.
"ln" from toybox does not understand -f (force) flag.

llvm-svn: 318573
2017-11-17 23:52:34 +00:00
Peter Collingbourne dd01ddbe67 Enable PDB generation with lld in asan and cfi tests on Windows.
PDB emission now works well enough that we can rely on it for these
tests to pass.

Differential Revision: https://reviews.llvm.org/D40188

llvm-svn: 318546
2017-11-17 19:49:41 +00:00
Vitaly Buka 065421f0cc [fuzzer] Initialize PcDescr buffer before calling __sanitizer_symbolize_pc
__sanitizer_symbolize_pc is not instrumented so msan assumes that PcDescr
was not initialized.

llvm-svn: 318488
2017-11-17 00:46:59 +00:00
Matt Morehouse 6497193cfe [libFuzzer] Fix flaky merge.test.
llvm-svn: 318475
2017-11-17 00:11:12 +00:00
Walter Lee 513fc069f0 [asan] Properly mark or disable tests that only work with shadow scale of 3
Differential Revision: https://reviews.llvm.org/D39774

llvm-svn: 318471
2017-11-16 23:29:19 +00:00
Walter Lee 00b4931d5d [asan] Port tests to shadow scale of 5
The tests are ported as follows:

contiguous_container_crash.cc
use-after-delete.cc
use-after-free.cc
  Replace hardwired shadow granularity in CHECK statements with regex.

max_redzone.cc
  Bump max_redzone parameter to 32.

memset_test.cc
  Bump size parameter of __asan_poison_memory_region to 32.

scariness_score_test.cc
  For "far-from-bounds" heap overflow, make sure overflow is more than
  one shadow granularity away.

  At large shadow granularity, there is not enough redzone between
  stack elements to detect far-from-bounds, so fake out that test.

Differential Revision: https://reviews.llvm.org/D39773

llvm-svn: 318470
2017-11-16 23:28:50 +00:00
Walter Lee 9af3b173c0 [asan] Add lit feature to indicate compiler_rt's shadow scale value
This will be used to mark tests that require a specific shadow scale.

Differential Revision: https://reviews.llvm.org/D39772

llvm-svn: 318469
2017-11-16 23:28:25 +00:00
Matt Morehouse 12615ec49b [libFuzzer] Make merge-sigusr.test work on Mac.
llvm-svn: 318465
2017-11-16 23:06:17 +00:00
Walter Lee ed8b52c8bc [asan] Avoid assert failure for non-default shadow scale
Rather than assertion failing, we can fall back to the
non-optimized version which works for any shadow scale.

Differential Revision: https://reviews.llvm.org/D39474

llvm-svn: 318460
2017-11-16 22:02:58 +00:00
Kostya Serebryany dc3c9eb0cf [libFuzzer] add more tests for merge
llvm-svn: 318453
2017-11-16 20:25:45 +00:00
Kostya Kortchinsky 76879dcc91 [sanitizer] Revert rL318410
Summary:
The patch seems to have turned some Android tests flaky. The reason is unclear.
This reverts D40100 in case we can't figure out what is happening.

Reviewers: eugenis

Reviewed By: eugenis

Subscribers: srhines, kubamracek

Differential Revision: https://reviews.llvm.org/D40138

llvm-svn: 318438
2017-11-16 18:53:18 +00:00
Kamil Rytarowski 13007b2220 Implement GetTls() for NetBSD
Summary:
Reuse the existing code for FreeBSD that is compatible with NetBSD.

Add NetBSD support in tests: tls_race.cc and tls_race2.cc.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, dvyukov, vitalybuka, kcc, eugenis

Reviewed By: dvyukov

Subscribers: srhines, emaste, kubamracek, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D40105

llvm-svn: 318431
2017-11-16 17:54:14 +00:00
Walter Lee 13a3cf3b6a [asan] Ensure that the minimum redzone is at least SHADOW_GRANULARITY
This is required by the Asan run-time.

Differential Revision: https://reviews.llvm.org/D39472

llvm-svn: 318422
2017-11-16 17:03:25 +00:00
Walter Lee 8f1545c629 [asan] Fix small X86_64 ShadowOffset for non-default shadow scale
The requirement is that shadow memory must be aligned to page
boundaries (4k in this case).  Use a closed form equation that always
satisfies this requirement.

Differential Revision: https://reviews.llvm.org/D39471

llvm-svn: 318421
2017-11-16 17:03:00 +00:00
Kostya Kortchinsky 8562eb32c9 [sanitizer] Use runtime checks instead of API level for Android logging
Summary:
Recent Bionic have a slew of `async_safe_*` logging functions that are
basically the liblog ones but included within the libc. They have the advantage
of not allocating memory. `async_safe_write_log` does no formatting and is
likely the best candidate for logging.

Use a weak definition to try and use it. Also, avoid API level checks (as
the toolchain is compiled at a rather low API level) for `__android_log_write`
in favor of a weak definition as well.

Keep the fallback to `syslog` if nothing else was found.

I tried to overhaul the code block to only have a single #if SANITIZER_ANDROID
but I am not particularly attached to the form. LMKWYT.

Reviewers: eugenis

Reviewed By: eugenis

Subscribers: srhines, kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D40100

llvm-svn: 318410
2017-11-16 16:19:44 +00:00
Peter Smith a98efd2b81 [builtins][ARM] re-enable mulsc3_test.c on Arm
The mulsc3_test.c was marked as unsupported due to PR32457, the underlying
cause of this PR was fixed in PR28164 so we can remove the unsupported as
it is no longer needed.

Differential Revision: https://reviews.llvm.org/D40076

llvm-svn: 318396
2017-11-16 13:00:17 +00:00
Kostya Serebryany 0e194faeea [libFuzzer] disable test/fuzzer/merge-sigusr.test on Mac while investigating the bot failure
llvm-svn: 318376
2017-11-16 04:52:05 +00:00
Evgeniy Stepanov 396ed67950 [asan] Fallback to non-ifunc dynamic shadow on android<22.
Summary: Android < 22 does not support ifunc.

Reviewers: pcc

Subscribers: srhines, kubamracek, hiraditya, llvm-commits

Differential Revision: https://reviews.llvm.org/D40116

llvm-svn: 318369
2017-11-16 02:52:19 +00:00
Alex Shlyapnikov 7fb4d3d16e [LSan] Export __lsan_init
Summary:
lsan_preinit.cc is meant to be linked into executable and calls
lsan_init from .preinit_array section. But if liblsan is a shared library,
then this doesn't work, because the symbol is not exported. This patch fixes
that. The counterparts like asan_init or __tsan_init already do have
SANITIZER_INTERFACE_ATTRIBUTE.

Committing on behalf of jakubjelinek.

Reviewers: alekseyshl, kcc

Subscribers: llvm-commits

Differential revision: https://reviews.llvm.org/D39889

llvm-svn: 318349
2017-11-15 23:42:48 +00:00
Francis Ricci 18074ab1c5 Fix some typos in a comment.
llvm-svn: 318336
2017-11-15 21:19:20 +00:00
Kostya Serebryany cb74c65f5e libfuzzer: Fix file listing on some filesystems
Summary:
For some filesystems, readdir will not populate dirent::d_type with valuable information. This causes libfuzzer to proceed with an empty corpus, instead of the file it contains.

This has been tested on a server using XFS.

It should fix https://bugs.llvm.org//show_bug.cgi?id=25991

Reviewers: kcc

Reviewed By: kcc

Differential Revision: https://reviews.llvm.org/D40028

llvm-svn: 318303
2017-11-15 16:45:17 +00:00
Kostya Serebryany 8eb9ca2580 [libFuzzer] respect max_len during merge
llvm-svn: 318302
2017-11-15 16:42:52 +00:00
Kostya Kortchinsky 58f2656d7e [scudo] Soft and hard RSS limit checks
Summary:
This implements an opportunistic check for the RSS limit.

For ASan, this was implemented thanks to a background thread checking the
current RSS vs the set limit every 100ms. This was deemed problematic for Scudo
due to potential Android concerns (Zygote as pointed out by Aleksey) as well as
the general inconvenience of having a permanent background thread.

If a limit (soft or hard) is specified, we will attempt to update the RSS limit
status (exceeded or not) every 100ms. This is done in an opportunistic way: if
we can update it, we do it, if not we return the current status, mostly because
we don't need it to be fully consistent (it's done every 100ms anyway). If the
limit is exceeded `allocate` will act as if OOM for a soft limit, or just die
for a hard limit.

We use the `common_flags()`'s `hard_rss_limit_mb` & `soft_rss_limit_mb` for
configuration of the limits.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D40038

llvm-svn: 318301
2017-11-15 16:40:27 +00:00
Dean Michael Berris 6a40b2d0a5 [XRay][compiler-rt][x86_64] Align the stack before and after calling handlers
Summary:
This change fixes the XRay trampolines aside from the __xray_CustomEvent
trampoline to align the stack to 16-byte boundaries before calling the
handler. Before this change we've not been explicitly aligning the stack
to 16-byte boundaries, which makes it dangerous when calling handlers
that leave the stack in a state that isn't strictly 16-byte aligned
after calling the handlers.

We add a test that makes sure we can handle these cases appropriately
after the changes, and prevents us from regressing the state moving
forward.

Fixes http://llvm.org/PR35294.

Reviewers: pelikan, pcc

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D40004

llvm-svn: 318261
2017-11-15 03:35:42 +00:00
Vedant Kumar ede2449fd1 [profile] Update InstrProfData.inc to sync with llvm
llvm-svn: 318230
2017-11-14 23:57:58 +00:00
Adam Nemet c1ffcec3ae Adjust test after r318159
llvm-svn: 318170
2017-11-14 19:00:08 +00:00
Kostya Kortchinsky a2b715f883 [scudo] Simplify initialization and flags
Summary:
This is mostly some cleanup and shouldn't affect functionalities.

Reviewing some code for a future addition, I realized that the complexity of
the initialization path was unnecessary, and so was maintaining a structure
for the allocator options throughout the initialization.

So we get rid of that structure, of an extraneous level of nesting for the
`init` function, and correct a couple of related code inaccuracies in the
flags cpp.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D39974

llvm-svn: 318157
2017-11-14 16:14:53 +00:00
Martin Storsjo 5fe3b4b266 [builtins] Include GENERIC_SOURCES in arm_SOURCES for MinGW
It is included in the built sources for all other arches supported
for MinGW currently, except for arm.

Differential Revision: https://reviews.llvm.org/D39938

llvm-svn: 318139
2017-11-14 07:07:01 +00:00
Kuba Mracek 90e01c364b Update TSan/ARM64 max VM to 0xfc0000000 to reflect a kernel change.
llvm-svn: 318103
2017-11-13 23:04:47 +00:00
Vitaly Buka fba6547f2a [sanitizer] Relax stack check in assert.cc even more
assert implementations can be very different

llvm-svn: 318089
2017-11-13 21:27:58 +00:00
Tom de Vries 66b43dcab5 [lsan] Remove semicolon after do {} while (0)
Remove semicolon after "do {} while (0)" in LOG_POINTERS and LOG_THREADS.

Reviewed by: kcc

llvm-svn: 318085
2017-11-13 20:59:26 +00:00
Tom de Vries a91707224d [asan] Remove semicolon after do {} while (0)
Remove semicolon after "do {} while (0)" in in CHECK_SMALL_REGION

llvm-svn: 318084
2017-11-13 20:59:20 +00:00
Vitaly Buka c691d4eef2 [tsan] Fix signal chaining
Summary: Return saved values only if installed sigaction is our wrapper.

Reviewers: eugenis, dvyukov

Subscribers: llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D39935

llvm-svn: 318082
2017-11-13 20:49:14 +00:00
Kostya Kortchinsky 2ba105a7da [sanitizer] Update scudo to use new API
Summary:
The ScudoAllocator uses a SecondaryHeader to keep track of the size and base address of each mmap'd chunk.

This aligns well with what the ReservedAddressRange is trying to do.  This changeset converts the scudo allocator from using the MmapNoAccess/MmapFixed APIs to the ReservedAddressRange::Init and ::Map APIs.  In doing so, it replaces the SecondayHeader struct with the ReservedAddressRange object.

This is part 3 of a 4 part changeset; part 1 https://reviews.llvm.org/D39072 and part 2 https://reviews.llvm.org/D38592

Reviewers: alekseyshl, mcgrathr, cryptoad, phosek

Reviewed By: cryptoad

Subscribers: llvm-commits, cryptoad, kubamracek

Differential Revision: https://reviews.llvm.org/D38593

llvm-svn: 318080
2017-11-13 20:38:22 +00:00
Vitaly Buka 014be10717 [tsan] Deadly signal handler for tsan
Summary: https://github.com/google/sanitizers/issues/637

Reviewers: eugenis

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D39929

llvm-svn: 318078
2017-11-13 19:56:20 +00:00
Vitaly Buka 2a46e8d1c2 [sanitizer] Relax stack check
-NEXT sometimes does not work as LLVMSymbolizer warning can appear there.

llvm-svn: 318077
2017-11-13 19:56:06 +00:00
Bill Seurer 3e3ee1282b [PowerPC][tsan] Update tsan to handle changed memory layouts in newer kernels
In more recent Linux kernels with 47 bit VMAs the layout of virtual memory
for powerpc64 changed causing the thread sanitizer to not work properly. This
patch adds support for 47 bit VMA kernels for powerpc64.

Tested on several 4.x and 3.x kernel releases.

llvm-svn: 318044
2017-11-13 15:42:28 +00:00
Walter Lee 52b2bd7845 [asan] Add CMake hook to override shadow scale in compiler_rt
Allow user to override shadow scale in compiler_rt by passing
-DCOMPILER_RT_ASAN_SHADOW_SCALE=n to CMake.  Propagate the override
shadow scale value via a compiler define to compiler-rt and asan
tests.  Tests will use the define to partially disable unsupported
tests.  Set "-mllvm -asan-mapping-scale=<n>" for compiler_rt tests.

Differential Revision: https://reviews.llvm.org/D39469

llvm-svn: 318038
2017-11-13 14:02:27 +00:00
Greg Bedwell d6b0ecb795 Allow compiler-rt test targets to work with multi-config CMake generators
Multi-config CMake generators need lit to be able to resolve paths of
artifacts from previous build steps at lit time, rather than expect them
to be fully resolved at CMake time as they may contain the build mode.

Differential Revision: https://reviews.llvm.org/D38471

llvm-svn: 318037
2017-11-13 12:57:54 +00:00
Vitaly Buka 8b9d6be24d [sanitizer] Simplify stack check in accert.cc
Somehow on arm bots stack does not include main.

llvm-svn: 318002
2017-11-12 21:15:19 +00:00
Vitaly Buka 1925591925 [sanitizer] Try to see test output on armv7
llvm-svn: 318001
2017-11-12 20:25:14 +00:00
Kamil Rytarowski 32ee3342d9 Correct NetBSD build issue
Cast SIG_IGN to (uptr).

SIG_IGN is defined on NetBSD as a function pointer and cannot be
assigned to an integer as it is.

sys/signal.h:#define       SIG_IGN         ((void (*)(int))  1)

llvm-svn: 317978
2017-11-11 09:29:21 +00:00
Vitaly Buka 5f767113c5 [msan] Fix signal chaining
Return internally stored handlers only if handlers is set to wrapper

llvm-svn: 317970
2017-11-11 03:03:34 +00:00
Evgeniy Stepanov 24bc8d5905 [ubsan-minimal] Get rid of the libc++ dependency.
Summary:
Use -nodefaultlibs.
Replace std:atomic with sanitizer atomics.

Reviewers: vitalybuka, kongyi, EricWF

Subscribers: mgorny, llvm-commits

Differential Revision: https://reviews.llvm.org/D39934

llvm-svn: 317969
2017-11-11 02:32:02 +00:00
Vitaly Buka 2aa04b4a0f [sanitizer] Include stack trace check into signal tests
llvm-svn: 317963
2017-11-11 01:30:03 +00:00
Vitaly Buka 8f90f8b812 [ubsan] Fix ubsan tests broken by linking as C instead of C++
llvm-svn: 317958
2017-11-11 01:01:09 +00:00
Evgeniy Stepanov ad2c50f9b9 [sanitizer] Remove unnecessary attribute hidden.
This should fix windows build of compiler-rt broken in r317943.

The attribute is unnecessary because since GetMaxVirtualAddress was split in two,
we no longer use common_flags() in the ifunc resolver context.

llvm-svn: 317948
2017-11-10 22:52:44 +00:00
Vitaly Buka 8c6917872c [tsan] Move out more types to sanitizer_common
https://github.com/google/sanitizers/issues/637

llvm-svn: 317946
2017-11-10 22:41:52 +00:00
Evgeniy Stepanov 989299c42b [asan] Use dynamic shadow on 32-bit Android.
Summary:
The following kernel change has moved ET_DYN base to 0x4000000 on arm32:
https://marc.info/?l=linux-kernel&m=149825162606848&w=2

Switch to dynamic shadow base to avoid such conflicts in the future.

Reserve shadow memory in an ifunc resolver, but don't use it in the instrumentation
until PR35221 is fixed. This will eventually let use save one load per function.

Reviewers: kcc

Subscribers: aemerson, srhines, kubamracek, kristof.beyls, hiraditya, llvm-commits

Differential Revision: https://reviews.llvm.org/D39393

llvm-svn: 317943
2017-11-10 22:27:48 +00:00
Peter Collingbourne 0f43b92980 sanitizer_common: Try looking up symbols with RTLD_DEFAULT if RTLD_NEXT does not work.
If the lookup using RTLD_NEXT failed, the sanitizer runtime library
is later in the library search order than the DSO that we are trying
to intercept, which means that we cannot intercept this function. We
still want the address of the real definition, though, so look it up
using RTLD_DEFAULT.

Differential Revision: https://reviews.llvm.org/D39779

llvm-svn: 317930
2017-11-10 22:09:37 +00:00
Kuba Mracek ada45dfde7 [compiler-rt] Fix const and volatile qualifier warnings
Building with a new clang produces a bunch of warnings about dropped 'const' and 'volatile' qualifiers on pointers. Let's fix them.

Differential Revision: https://reviews.llvm.org/D39861

llvm-svn: 317929
2017-11-10 21:19:20 +00:00
Vitaly Buka 84d11cac28 [tsan] Move code of sigaction_impl and signal_impl
They need to be after sanitizer_signal_interceptors.inc to use READ function

Part of https://github.com/google/sanitizers/issues/637

llvm-svn: 317914
2017-11-10 19:23:02 +00:00
Vitaly Buka 01a7e4ca6b [tsan] Extract sigaction_impl and signal_impl
Preparation for switching to sanitizer_signal_interceptors.inc

Part of https://github.com/google/sanitizers/issues/637

llvm-svn: 317913
2017-11-10 19:22:51 +00:00
Kostya Kortchinsky 36e56785b4 [scudo] Bump the Android API level requirement to 21 for getauxval
Summary:
`getauxval` was introduced in 18 & 21 depending on the architecture. Bump the
requirement to 21.

It also turns out that the NDK is finicky: NDK r13b doesn't include sys/auxv.h
when creating a standalone toolchain at API level 19 for ARM. So 18 didn't work
well with older NDKs.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: aemerson, srhines, llvm-commits, kristof.beyls

Differential Revision: https://reviews.llvm.org/D39905

llvm-svn: 317907
2017-11-10 19:01:17 +00:00
Vitaly Buka db5757e68b [msan] Remove INTERCEPT_FUNCTION for sigaction and signal
Already done in InitializeSignalInterceptors()

llvm-svn: 317906
2017-11-10 18:58:59 +00:00
Dean Michael Berris 46d0cd3413 Revert "[XRay][darwin] Initial XRay in Darwin Support"
This reverts r317875.

llvm-svn: 317877
2017-11-10 07:00:55 +00:00
Dean Michael Berris bfd111bca5 [XRay][darwin] Initial XRay in Darwin Support
Summary:
This change implements the changes required in both clang and
compiler-rt to allow building XRay-instrumented binaries in Darwin. For
now we limit this to x86_64. We also start building the XRay runtime
library in compiler-rt for osx.

A caveat to this is that we don't have the tests set up and running
yet, which we'll do in a set of follow-on changes.

This patch uses the monorepo layout for the coordinated change across
multiple projects.

Reviewers: kubamracek

Subscribers: mgorny, cfe-commits, llvm-commits

Differential Revision: https://reviews.llvm.org/D39114

llvm-svn: 317875
2017-11-10 05:50:13 +00:00
Vitaly Buka 3e3f3cf54d [esan] Fix compilation of signal handlers
llvm-svn: 317874
2017-11-10 05:41:13 +00:00
Vitaly Buka 8b689f4092 [tsan] Use __sanitizer_siginfo from sanitizer_common
llvm-svn: 317872
2017-11-10 04:27:47 +00:00
Vitaly Buka 529e321e49 [sanitizer] Remove unused <signal.h> which conflicts with asan declarations
llvm-svn: 317871
2017-11-10 04:12:35 +00:00
Vitaly Buka 748c180fb1 [sanitizer] Forward declaration of __sanitizer_sigaction without signal interceptors
llvm-svn: 317870
2017-11-10 04:05:38 +00:00
Vitaly Buka a10d7a9fcc [sanitizer] Remove unneeded forward declaration of real_sigaction
llvm-svn: 317869
2017-11-10 04:00:58 +00:00
Vitaly Buka 084d365c90 [sanitizer] Remove unneeded forward declarations
llvm-svn: 317868
2017-11-10 03:35:26 +00:00
Vitaly Buka bad0b3ea7e [sanitizer] Remove unused <signal.h> which conflicts with asan declarations
llvm-svn: 317867
2017-11-10 03:18:53 +00:00
Vitaly Buka 112b2d1625 [sanitizer] Fix "cast from 'const void *' to 'sigaction *'" on Darwin
llvm-svn: 317866
2017-11-10 03:13:42 +00:00
Vitaly Buka f2827e133b [sanitizer] Remove references and hide CommonSanitizerReportMutex
llvm-svn: 317865
2017-11-10 02:07:11 +00:00
Vitaly Buka bec32e9ac4 [msan] Deadly signal handler for msan
Summary: Part of https://github.com/google/sanitizers/issues/637

Reviewers: eugenis, alekseyshl

Subscribers: llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D39826

llvm-svn: 317864
2017-11-10 02:06:59 +00:00
Vitaly Buka a452f9cc37 [msan] Move sigaction_impl and signal_impl
llvm-svn: 317863
2017-11-10 02:06:50 +00:00
Kostya Serebryany c183e2c33b [libFuzzer] trying to make sigusr tests more reliable
llvm-svn: 317852
2017-11-10 00:26:23 +00:00
Kostya Serebryany e8637f45ea [libFuzzer] split a test into two
llvm-svn: 317851
2017-11-10 00:18:13 +00:00
Vitaly Buka bc55ab6397 [sanitizer] Remove unneeded "struct"
llvm-svn: 317847
2017-11-09 23:06:59 +00:00
Vitaly Buka 893bf354eb [sanitizer] Fix signal interceptors build on Windows
llvm-svn: 317846
2017-11-09 23:05:01 +00:00
Vitaly Buka 8e92025718 [msan] Extract signal_impl and sigaction_impl
Summary: Preparation for using interceptor from sanitizer_common.

Reviewers: eugenis

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D39824

llvm-svn: 317844
2017-11-09 22:52:15 +00:00
Vitaly Buka 43c3e6ae1c [sanitizer] Allow sanitizers to redefine implementation of signal interceptors
Reviewers: eugenis

Subscribers: llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D39870

llvm-svn: 317843
2017-11-09 22:48:54 +00:00
Vitaly Buka 771e3995a4 [msan] Replace CommonSanitizerReportMutex with ScopedErrorReportLock
llvm-svn: 317842
2017-11-09 22:48:45 +00:00
Kostya Serebryany 533cbf749c [libFuzzer] use pkill instead of killall in a test since the bots don't have killall
llvm-svn: 317840
2017-11-09 22:01:06 +00:00
Kostya Serebryany 54a8456005 [libFuzzer] make sure to flush IO when done merging one file
llvm-svn: 317835
2017-11-09 21:30:33 +00:00
Kostya Kortchinsky 5604ad1c9b [sanitizer] Revert rL317822
Summary:
This reverts D39490.

For toolchains generated with older NDKs (<=r13b as far as we tested),
`cpu_set_t` doesn't exist in `sched.h`.
We have to figure out another way to get the number of CPUs without this.

Reviewers: rnk

Reviewed By: rnk

Subscribers: kubamracek, llvm-commits, krytarowski

Differential Revision: https://reviews.llvm.org/D39867

llvm-svn: 317834
2017-11-09 21:26:07 +00:00
Matt Morehouse 947838c9c5 [libFuzzer] Don't add leaking inputs to corpus.
Reviewers: kcc

Reviewed By: kcc

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D39850

llvm-svn: 317831
2017-11-09 20:44:08 +00:00
Kostya Serebryany a2ca2dcc46 [libFuzzer] handle SIGUSR1/SIGUSR2 and try to exit grafully on these signals
llvm-svn: 317829
2017-11-09 20:30:19 +00:00
Teresa Johnson 179b8fbba6 Revert new ThinLTO ASAN test until lit support added
This reverts commit r317723 and r317728. Will be re-added when
support for LTO/ThinLTO added to test/asan/lit.cfg.

llvm-svn: 317823
2017-11-09 19:26:54 +00:00
Kostya Kortchinsky 6458216b28 [scudo] Make getNumberOfCPUs Fuchsia compliant
Summary: This change allows Fuchsia to boot properly using the Scudo allocator.

Reviewers: cryptoad, alekseyshl, krytarowski

Reviewed By: cryptoad, krytarowski

Subscribers: rnk, krytarowski, kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D39490

llvm-svn: 317822
2017-11-09 19:18:55 +00:00
Weiming Zhao e58bb5dc4e [Builtins] Do not use tailcall for Thumb1
Summary:
The `b` instruction in Thumb1 has limited range, which may cause link-time errors if the jump target is far away.
This patch guards the tailcalls for non-Thumb1

Reviewers: peter.smith, compnerd, rengolin, eli.friedman

Reviewed By: rengolin

Subscribers: joerg, dalias, javed.absar, llvm-commits

Differential Revision: https://reviews.llvm.org/D39700

llvm-svn: 317814
2017-11-09 17:32:57 +00:00
Bill Seurer 823338d1fe [PowerPC][msan] Update msan to handle changed memory layouts in newer kernels
In more recent Linux kernels (including those with 47 bit VMAs) the layout of
virtual memory for powerpc64 changed causing the memory sanitizer to not
work properly.  This patch adjusts the memory ranges in the tables for the
memory sanitizer to work on the newer kernels while continuing to work on the
older ones as well.

Tested on several 4.x and 3.x kernel releases.

llvm-svn: 317802
2017-11-09 16:14:57 +00:00
Vitaly Buka 66f32fc431 [sanitizers] Rename GetStackTraceWithPcBpAndContext
Name does not need to enumerate arguments.

llvm-svn: 317774
2017-11-09 07:53:06 +00:00
Vitaly Buka 2b2d3aaa04 [msan] Add context argument into GetStackTrace
llvm-svn: 317773
2017-11-09 07:48:53 +00:00
Vitaly Buka cb4b2c0ffc [lsan] Add "static" to internal function
llvm-svn: 317772
2017-11-09 07:46:30 +00:00
Kostya Serebryany 3ac31d4453 [libFuzzer] allow merge to resume after being preempted
llvm-svn: 317767
2017-11-09 05:49:28 +00:00
Peter Collingbourne c6721f580d ubsan: Allow programs to use setenv to configure ubsan_standalone.
Previously ubsan_standalone used the GetEnv function to read the
environment variables UBSAN_OPTIONS and UBSAN_SYMBOLIZER_PATH. The
problem with GetEnv is that it does not respect changes to the
environment variables made using the libc setenv function, which
prevents clients from setting environment variables to configure
ubsan before loading ubsan-instrumented libraries.

The reason why we have GetEnv is that some runtimes need to read
environment variables while they initialize using .preinit_array,
and getenv does not work while .preinit_array functions are being
called. However, it is unnecessary for ubsan_standalone to initialize
that early. So this change switches ubsan_standalone to using getenv
and removes the .preinit_array entry. The static version of the runtime
still ends up being initialized using a C++ constructor that exists
to support the shared runtime.

Differential Revision: https://reviews.llvm.org/D39827

llvm-svn: 317757
2017-11-09 02:22:07 +00:00
Kostya Serebryany e1d121d715 [libFuzzer] mechanically simplify a test, NFC
llvm-svn: 317756
2017-11-09 02:13:43 +00:00
Kostya Serebryany 8a94f220aa [libFuzzer] fix a test (and hopefully, the bot)
llvm-svn: 317749
2017-11-09 01:45:59 +00:00
Kostya Serebryany 68fdef1f7a [libFuzzer] allow user to specify the merge control file
llvm-svn: 317747
2017-11-09 01:05:29 +00:00
Evgeniy Stepanov 75a35179de [Sanitizers, CMake] Also use version script for libclang_rt.asan-i386.so
When building LLVM on x86_64-pc-linux-gnu (Fedora 25) with the bundled gcc 6.4.1
which uses gld 2.26.1-1.fc25, the dynamic/Asan-i386-calls-Dynamic-Test and
dynamic/Asan-i386-inline-Dynamic-Test tests failed to link with

/usr/bin/ld: /var/scratch/gcc/llvm/dist/lib/clang/6.0.0/lib/linux/libclang_rt.asan-i386.so: fork: invalid version 21 (max 0)
/var/scratch/gcc/llvm/dist/lib/clang/6.0.0/lib/linux/libclang_rt.asan-i386.so: error adding symbols: Bad value

I tried building with a self-compiled gcc 7.1.0 using gld 2.28, but the error remained.

It seems the error has been hit before (cf. https://reviews.llvm.org/rL314085), but
no real explanation has been found.

However, the problem goes away when linking the i386 libclang_rt.asan with a version
script just like every other variant is. Not using the version script in this single case
dates back to the initial introduction of the version script in r236551, but this change
was just checked in without any explanation AFAICT.

Since I've not found any other workaround and no reason for not always using the
version script, I propose to do so.

Tested on x86_64-pc-linux-gnu.

Patch by Rainer Orth.

Differential Revision: https://reviews.llvm.org/D39795

llvm-svn: 317738
2017-11-08 22:51:09 +00:00
Kamil Rytarowski 2fd314e2e2 Correct atexit(3) support in TSan/NetBSD
Summary:
The NetBSD specific implementation of cxa_atexit() does not
preserve the 2nd argument if dso is equal to NULL.

Changes:

 - Split paths of handling intercepted __cxa_atexit() and atexit(3).
   This affects all supported Operating Systems.
 - Add a local stack-like structure to hold the __cxa_atexit() context.
   atexit(3) is documented in the C standard as calling callback from the
   earliest to the oldest entry. This path also fixes potential ABI
   problem of passing an argument to a function from the atexit(3)
   callback mechanism.
 - Add new test to ensure LIFO style of atexit(3) callbacks: atexit3.cc

Proposal to change the behavior of __cxa_atexit() in NetBSD has been rejected.

With the above changes TSan/NetBSD with the current tsan_interceptors.cc
can bootstrap into operation.

Sponsored by <The NetBSD Foundation>

Reviewers: vitalybuka, dvyukov, joerg, kcc, eugenis

Reviewed By: dvyukov

Subscribers: kubamracek, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D39619

llvm-svn: 317735
2017-11-08 22:34:17 +00:00
Teresa Johnson c707c6f3a7 [ThinLTO] New test needs to require LTO
Fix buildbot failures:
http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux/builds/5262/steps/annotate/logs/stdio

llvm-svn: 317728
2017-11-08 21:48:27 +00:00
Teresa Johnson edf782f0d0 [ThinLTO] Ensure sanitizer passes are run
Recommit new test as linux-only.

llvm-svn: 317723
2017-11-08 20:27:28 +00:00
Teresa Johnson 67a781c4fc Revert "[ThinLTO] Ensure sanitizer passes are run"
This reverts commit r317715. It failed a Windows buildbot since
ThinLTO is presumably not supported, leading to a corrupt file error
on the object file:
http://lab.llvm.org:8011/builders/sanitizer-windows/builds/19395/steps/run%20check-asan/logs/stdio

Will re-commit the new ThinLTO part of the test to a linux only test
file.

llvm-svn: 317717
2017-11-08 20:08:15 +00:00
Teresa Johnson 0cfd530b2b [ThinLTO] Ensure sanitizer passes are run
Summary:
Test fix to pass manager for ThinLTO.

Depends on D39565.

Reviewers: pcc

Subscribers: kubamracek, mehdi_amini, llvm-commits, inglorion

Differential Revision: https://reviews.llvm.org/D39566

llvm-svn: 317715
2017-11-08 19:46:25 +00:00
Marco Castelluccio ff4da375ae Implement flock for Windows in compiler-rt
Summary:
This patch implements flock for Windows, needed to make gcda writing work in a multiprocessing scenario.

Fixes https://bugs.llvm.org/show_bug.cgi?id=34923.

Reviewers: zturner

Reviewed By: zturner

Subscribers: rnk, zturner, llvm-commits

Differential Revision: https://reviews.llvm.org/D38891

llvm-svn: 317705
2017-11-08 19:11:54 +00:00
Kostya Kortchinsky 6edadae34a [sanitizer] Add Scudo to the sanitizer lint checks.
Summary:
Scudo abides by the coding style enforced by the sanitizer_common
linter, but as of right now, it's not linter-enforced.

Add Scudo to the list of directories checked by check_lint.sh.

Also: fixes some linter errors found after getting this running.

Reviewers: cryptoad

Reviewed By: cryptoad

Subscribers: llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D39757

llvm-svn: 317699
2017-11-08 16:42:29 +00:00
Sylvestre Ledru a8c5460c8a Always use prctl(PR_SET_PTRACER)
Summary:
Sufficiently old Linux kernel headers don't provide the PR_SET_PTRACER, but we can still call prctl with it if the runtime kernel is newer. Even if it's not, prctl will only return EINVAL.

Patch by Mike Hommey <mh-llvm@glandium.org>

Reviewers: eugenis

Reviewed By: eugenis

Subscribers: sylvestre.ledru, cfe-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D39717

llvm-svn: 317668
2017-11-08 07:25:19 +00:00
Reid Kleckner f7fdac4508 Revert "[scudo] Make getNumberOfCPUs Fuchsia compliant"
This reverts commit r317604.

Android doesn't have cpu_set_t.

llvm-svn: 317655
2017-11-08 01:33:15 +00:00
Evgeniy Stepanov 98fe207986 [sanitizer] Asm implementation of syscall() for arm32.
Summary:
These will be used in an ifunc resolver, when the binary may not be
completely relocated, and syscall() function from libc could not be
used.

Reviewers: dvyukov, vitalybuka

Subscribers: aemerson, kubamracek, javed.absar, llvm-commits, kristof.beyls

Differential Revision: https://reviews.llvm.org/D39701

llvm-svn: 317640
2017-11-08 00:15:12 +00:00
Evgeniy Stepanov 0379d3f844 (NFC) Rename GetMax{,User}VirtualAddress.
Future change will introduce GetMaxVirtualAddress that will not take
the kernel area into account.

llvm-svn: 317638
2017-11-07 23:51:22 +00:00
Vitaly Buka fdc0795098 [sanitizer] Fix sanitizer_common compile errors on gcc trunk
Summary:
When testing a merge of compiler-rt r304709 into gcc trunk on x86-64-pc-linux-gnu,
I ran into two compile errors:
sanitizer_common/sanitizer_symbolizer_libbacktrace.cc:96:73: error: no matching function for call to '__sanitizer::AddressInfo::FillModuleInfo(char*&, __sanitizer::uptr&)'

All other files in sanitizer_common (with the exception of sanitizer_malloc_mac.inc
which is special) include sanitizer_platform.h without directory name.

Patch by Mike Jongen

Reviewers: kcc, vitalybuka

Reviewed By: kcc

Subscribers: kubamracek, fedor.sergeev, llvm-commits

Differential Revision: https://reviews.llvm.org/D33933

llvm-svn: 317608
2017-11-07 20:10:17 +00:00
Kostya Kortchinsky 4e8ce0225f [scudo] Make getNumberOfCPUs Fuchsia compliant
Summary: This change allows Fuchsia to boot properly using the Scudo allocator.

Reviewers: cryptoad, alekseyshl, krytarowski

Reviewed By: cryptoad, krytarowski

Subscribers: krytarowski, kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D39490

llvm-svn: 317604
2017-11-07 19:30:08 +00:00
Sam Clegg 12166ea1f2 [WebAssembly] Include GENERIC_TF_SOURCES in wasm builds
Differential Revision: https://reviews.llvm.org/D39748

llvm-svn: 317601
2017-11-07 19:03:11 +00:00
Dmitry Vyukov 3cd4022c39 tsan: allow usage of global vars with ctors in interceptors
We allow usage of global/per-thread data with non-trivial ctors/dtors
throughout tsan code base by placing all global/per-thread data into
Context/ThreadState and then explicitly constructing them with
placement new. This greatly simplifies code by restricting the
"linker initialized plague" to only these 2 objects.

Do the same for interceptors data.

This allows to use Vector instead of bunch of hand-written code in:
https://reviews.llvm.org/D39619

Reviewed in: https://reviews.llvm.org/D39721

llvm-svn: 317587
2017-11-07 16:31:08 +00:00
Kostya Kortchinsky c6f0ade6c3 Update sanitizer_allocator to use new API.
Summary:
Update sanitizer_allocator to use new API.
    
Second patch in a series.  First patch https://reviews.llvm.org/D39072
    
Updates MmapNoAccess / MmapFixed call sites in the saniziter_allocator
to use the new Init/Map APIs instead.


Reviewers: alekseyshl, cryptoad, phosek, mcgrathr, dvyukov

Reviewed By: alekseyshl, cryptoad

Subscribers: dvyukov, mcgrathr, kubamracek

Differential Revision: https://reviews.llvm.org/D38592

llvm-svn: 317586
2017-11-07 16:19:24 +00:00
Alex Shlyapnikov 2093b94e70 [LSan] Detect dynamic loader by its base address.
Summary:
Relanding D38600, which was reverted due to various PPC bot failures.

If it breaks something again, please provide some pointers to broken
bots, not just revert it, otherwise it's very hard to reason what's
wrong with this commit.

Whenever possible (Linux + glibc 2.16+), detect dynamic loader module by
its base address, not by the module name matching. The current name
matching approach fails on some configurations.

Reviewers: eugenis

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D39275

llvm-svn: 317512
2017-11-06 21:27:06 +00:00
Alex Shlyapnikov adbf476a88 [Sanitizers] Check pthread_setcancel{state|type} interceptor arguments for != nullptr.
Summary:
According to man, pthread_setcancelstate's oldstate and
pthread_setcanceltype's oldtype parameters can be nullptr.
Check these parameters for != nullptr before attempting to
access their shadow memory.

Reviewers: dvyukov

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D39626

llvm-svn: 317494
2017-11-06 17:43:28 +00:00
Mehdi Amini a2a539e4c5 Fix CMake definitions of tsan runtime to make it installed by "install-compiler-rt"
Summary: The PARENT_TARGET was correctly set under APPLE but not under linux.

Reviewers: kubamracek, samsonov

Subscribers: dberris, mgorny, llvm-commits

Differential Revision: https://reviews.llvm.org/D39621

llvm-svn: 317391
2017-11-04 00:07:20 +00:00
Kostya Kortchinsky 4a0ebbfe97 [scudo] Rearrange #include order
Summary:
To be compliant with https://llvm.org/docs/CodingStandards.html#include-style,
system headers have to come after local headers.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D39623

llvm-svn: 317390
2017-11-03 23:48:25 +00:00
Alex Shlyapnikov 32224fe842 [Sanitizers] Call NanoTime() conditionally.
Summary:
Call NanoTime() in primary 64 bit allocator only when necessary,
otherwise the unwarranted syscall causes problems in sandbox environments.
ReleaseToOSIntervalMs() conditional allows them to turn the feature off
with allocator_release_to_os_interval_ms=-1 flag.

Reviewers: eugenis

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D39624

llvm-svn: 317386
2017-11-03 23:31:00 +00:00
Kamil Rytarowski e6b53dad42 Late fixup in _lwp_exit on TSan/NetBSD
Call DestroyThreadState() before REAL(_lwp_exit)();

This variation is less racy.

Sponsored by <The NetBSD Foundation>

llvm-svn: 317369
2017-11-03 21:08:52 +00:00
Kamil Rytarowski 55435b7427 Correct detection of a thread termination
Summary:
Stop using the Linux solution with pthread_key_create(3).
This approach does not work on NetBSD, because calling
the thread destructor is not the latest operation on a POSIX
thread entity. NetBSD's libpthread still calls at least
pthread_mutex_lock and pthread_mutex_unlock.

Detect _lwp_exit(2) call as it is really the latest operation
called from a detaching POSIX thread.

This resolves one set of crashes observed in
the Thread Sanitizer execution.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, kcc, vitalybuka, dvyukov, eugenis

Reviewed By: vitalybuka

Subscribers: llvm-commits, kubamracek, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D39618

llvm-svn: 317363
2017-11-03 20:48:19 +00:00
Kamil Rytarowski 99a91f6744 Disable detection of on_exit()/TSan on NetBSD
Summary:
NetBSD does not ship with on_exit() function.

Introduce TSAN_MAYBE_INTERCEPT_ON_EXIT.
It looks like this addition fixes build for Darwin.

Sponsored by <The NetBSD Foundation>

Reviewers: vitalybuka, joerg, eugenis, dvyukov, kcc

Reviewed By: vitalybuka

Subscribers: llvm-commits, kubamracek, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D39617

llvm-svn: 317361
2017-11-03 20:39:51 +00:00
Vitaly Buka 7d22324b87 [fuzzer] Fix nested mallocs
Summary: Nested mallocs are possible with internal symbolizer.

Reviewers: kcc

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D39397

llvm-svn: 317186
2017-11-02 04:12:10 +00:00
Vitaly Buka ca9075272b [fuzzer] Script to detect unbalanced allocation in -trace_malloc output
Reviewers: kcc

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D39466

llvm-svn: 317119
2017-11-01 20:27:06 +00:00
Kostya Kortchinsky a2cdb2d541 [scudo] Fix standlone build -lrt requirement
Summary:
The split in D39461 introduced separate C++ flags, but `cxx_flags` needs `-lrt` as well for the standalone build.


Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D39497

llvm-svn: 317103
2017-11-01 17:00:26 +00:00
Kostya Kortchinsky d937b0a10c [scudo] Implement stricter separation of C vs C++
Summary:
Initially, Scudo had a monolithic design where both C and C++ functions were
living in the same library. This was not necessarily ideal, and with the work
on -fsanitize=scudo, it became more apparent that this needed to change.

We are splitting the new/delete interceptor in their own C++ library. This
allows more flexibility, notably with regard to std::bad_alloc when the work is
done. This also allows us to not link new & delete when using pure C.

Additionally, we add the UBSan runtimes with Scudo, in order to be able to have
a -fsanitize=scudo,undefined in Clang (see work in D39334).

The changes in this patch:
- split the cxx specific code in the scudo cmake file into a new library;
  (remove the spurious foreach loop, that was not necessary)
- add the UBSan runtimes (both C and C++);
- change the test cmake file to allow for specific C & C++ tests;
- make C tests pure C, rename their extension accordingly.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: srhines, mgorny, llvm-commits

Differential Revision: https://reviews.llvm.org/D39461

llvm-svn: 317097
2017-11-01 15:28:20 +00:00
Vitaly Buka 7dbc1d8433 [fuzzer] Fix threaded stack printing
Reviewers: kcc

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D39397

llvm-svn: 317071
2017-11-01 03:02:59 +00:00
Vitaly Buka df00e89cdc Revert "[fuzzer] Fix threaded stack printing and nested mallocs"
Fails on darwin

Revert "[fuzzer] Script to detect unbalanced allocation in -trace_malloc output"
Needs previous one.

This reverts commit r317034, r317036.

llvm-svn: 317061
2017-11-01 00:55:52 +00:00
Vitaly Buka 0747700ce1 [fuzzer] Script to detect unbalanced allocation in -trace_malloc output
Reviewers: kcc

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D39466

llvm-svn: 317036
2017-10-31 20:50:07 +00:00
Vitaly Buka 58da33e35c [fuzzer] Fix threaded stack printing and nested mallocs
Summary: Nested mallocs are possible with internal symbolizer.

Reviewers: kcc

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D39397

llvm-svn: 317034
2017-10-31 20:49:48 +00:00
Petr Hosek 29476f759b [sanitizer][Fuchsia] Add a missing semicolon
Differential Revision: https://reviews.llvm.org/D39433

llvm-svn: 316959
2017-10-30 21:29:26 +00:00
Kostya Kortchinsky 264e1b73eb Fix warning + death test + failing test on Windows (D39072).
Summary: Fixes https://reviews.llvm.org/D39072

Reviewers: cryptoad

Reviewed By: cryptoad

Subscribers: kubamracek

Differential Revision: https://reviews.llvm.org/D39427

llvm-svn: 316943
2017-10-30 19:06:59 +00:00
Kostya Kortchinsky 8c39ccf10c [sanitizer] Fixing an error introduced in D39072
Summary: This should fix the Windows bots after D39072.

Reviewers: alekseyshl, flowerhack

Reviewed By: flowerhack

Subscribers: llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D39426

llvm-svn: 316937
2017-10-30 18:16:05 +00:00
Kostya Kortchinsky 96da9fa4ca Introduce ReservedAddressRange to sanitizer_common.
Summary:
Fixed version of https://reviews.llvm.org/D38437 (fixes Win/Fuchsia failures).

Creating a new revision, since the old one was getting a bit old/crowded.

    In Fuchsia, MmapNoAccess/MmapFixedOrDie are implemented using a global
    VMAR, which means that MmapNoAccess can only be called once. This works
    for the sanitizer allocator but *not* for the Scudo allocator.
    
    Hence, this changeset introduces a new ReservedAddressRange object to
    serve as the new API for these calls. In this changeset, the object
    still calls into the old Mmap implementations.
    
    The next changeset two changesets will convert the sanitizer and scudo
    allocators to use the new APIs, respectively. (ReservedAddressRange will
    replace the SecondaryHeader in Scudo.)
    
    Finally, a last changeset will update the Fuchsia implementation.


Reviewers: alekseyshl, cryptoad, phosek

Reviewed By: alekseyshl, cryptoad

Subscribers: kubamracek

Differential Revision: https://reviews.llvm.org/D39072

llvm-svn: 316934
2017-10-30 17:56:24 +00:00
Reid Kleckner 86fca5d058 Fix clang warnings in winasan code
There are two instances of -Wcast-qual and one of -Wsign-compare.

llvm-svn: 316930
2017-10-30 17:26:57 +00:00
Reid Kleckner af86cba0cf [asan] Intercept heap routines in VS2010 CRT
Users have requested that we add it to the list:
https://github.com/google/sanitizers/issues/864

llvm-svn: 316929
2017-10-30 17:26:13 +00:00
Martin Pelikan f5f4c009fe [XRay] [compiler-rt] fix build by including errno.h into FDR mode
The build got broken after D39277 (and rL316816) deleted <cerrno>.

llvm-svn: 316821
2017-10-28 01:35:07 +00:00
Dean Michael Berris e8fec1955a [XRay][compiler-rt] Remove more STL dependenices from FDR mode
Summary:
This change removes dependencies on STL types:

  - std::aligned_storage -- we're using manually-aligned character
    buffers instead for metadata and function records.

  - std::tuple -- use a plain old struct instead.

This is an incremental step in removing all STL references from the
compiler-rt implementation of XRay (llvm.org/PR32274).

Reviewers: dblaikie, pelikan, kpw

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D39277

llvm-svn: 316816
2017-10-27 23:59:41 +00:00
Kostya Kortchinsky 91b7558ca8 [scudo] Allow to specify the maximum number of TSDs at compile time
Summary:
This introduces `SCUDO_MAX_CACHES` allowing to define an upper bound to the
number of `ScudoTSD` created in the Shared TSD model (by default 32U).
This name felt clearer than `SCUDO_MAX_TSDS` which is technically what it really
is. I am opened to suggestions if that doesn't feel right.

Additionally change `getNumberOfCPUs` to return a `u32` to be more consistent.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D39338

llvm-svn: 316788
2017-10-27 20:10:14 +00:00
Alex Shlyapnikov f280eff080 [LSan] Disable a couple of failing tests on PPC64 (pending investigation).
llvm-svn: 316720
2017-10-27 00:29:40 +00:00
Alex Shlyapnikov e4b9ae666e [LSan] Enable LSan tests on PPC64 Linux.
Summary:
LSan is functional on PPC64 Linux now, let's enable all tests.

One test required ppc specific changes: use_registers.cc.

Reviewers: eugenis

Subscribers: mgorny, llvm-commits

Differential Revision: https://reviews.llvm.org/D39316

llvm-svn: 316698
2017-10-26 20:59:04 +00:00
Alex Shlyapnikov 9b4e32785a [Sanitizers] Set default allocator_release_to_os_interval_ms to 5 seconds
Summary:
With new release to OS approach (see D38245) it's reasonable to enable
it by default. Setting allocator_release_to_os_interval_ms to 5000 seems
to be a reasonable default (might be tuned later, based on the
feedback).

Also delaying the first release to OS in each bucket for at least
allocator_release_to_os_interval_ms after the first allocation to
prevent just allocated memory to be madvised back to OS and let short
lived processes to avoid release to OS overhead altogether.

Reviewers: cryptoad

Subscribers: kubamracek, llvm-commits, mehdi_amini

Differential Revision: https://reviews.llvm.org/D39318

llvm-svn: 316683
2017-10-26 17:59:24 +00:00
Vitaly Buka f3e0f76196 [sanitizer] Fix internal symbolized build on Debian 9
llvm-svn: 316637
2017-10-26 06:08:01 +00:00
Alex Shlyapnikov 02bda37492 [LSan] Adjust LSan allocator limits for PPC64.
Summary: Now the limits are the same as for ASan allocator.

Reviewers: cryptoad

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D39309

llvm-svn: 316633
2017-10-26 01:22:48 +00:00
Kostya Serebryany 3c9e59d0f6 [libFuzzer] trying to make a test more stable on Mac
llvm-svn: 316627
2017-10-25 23:24:45 +00:00
Kostya Kortchinsky f9008a3a06 [scudo] Remove comment about security of the 32-bit allocator
Summary:
The 32-bit allocator is now on par with the 64-bit in terms of security (chunks
randomization is done, batches separation is done).

Unless objection, the comment can go away.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D39303

llvm-svn: 316620
2017-10-25 22:00:26 +00:00
Evgeniy Stepanov 0b8602791b [msan] Intercept __strxfrm_l.
llvm-svn: 316613
2017-10-25 21:40:17 +00:00
Kamil Rytarowski 175e775e0f Try to unbreak Linux sanitizers
Add fallback definition of internal_syscall_ptr and internal_syscall64
for Linux/x86_64 and Linux/aarch64.

llvm-svn: 316598
2017-10-25 17:56:36 +00:00
Kostya Kortchinsky c484912b06 [sanitizer] Random shuffling of chunks for the 32-bit Primary Allocator
Summary:
The 64-bit primary has had random shuffling of chunks for a while, this
implements it for the 32-bit primary. Scudo is currently the only user of
`kRandomShuffleChunks`.

This change consists of a few modifications:
- move the random shuffling functions out of the 64-bit primary to
  `sanitizer_common.h`. Alternatively I could move them to
  `sanitizer_allocator.h` as they are only used in the allocator, I don't feel
  strongly either way;
- small change in the 64-bit primary to make the `rand_state` initialization
  `UNLIKELY`;
- addition of a `rand_state` in the 32-bit primary's `SizeClassInfo` and
  shuffling of chunks when populating the free list.
- enabling the `random_shuffle.cpp` test on platforms using the 32-bit primary
  for Scudo.

Some comments on why the shuffling is done that way. Initially I just
implemented a `Shuffle` function in the `TransferBatch` which was simpler but I
came to realize this wasn't good enough: for chunks of 10000 bytes for example,
with a `CompactSizeClassMap`, a batch holds only 1 chunk, meaning shuffling the
batch has no effect, while a region is usually 1MB, eg: 104 chunks of that size.
So I decided to "stage" the newly gathered chunks in a temporary array that
would be shuffled prior to placing the chunks in batches.
The result is looping twice through n_chunks even if shuffling is not enabled,
but I didn't notice any significant significant performance impact.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: srhines, llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D39244

llvm-svn: 316596
2017-10-25 17:24:56 +00:00
Alex Shlyapnikov a53b55f66c [Sanitizers] ASan: detect new/delete calls with mismatched alignment.
ASan allocator stores the requested alignment for new and new[] calls
and on delete and delete[] verifies that alignments do match.

The representable alignments are: default alignment, 8, 16, 32, 64, 128,
256 and 512 bytes. Alignments > 512 are stored as 512, hence two
different alignments > 512 will pass the check (possibly masking the bug),
but limited memory requirements deemed to be a resonable tradeoff for
relaxed conditions.

The feature is controlled by new_delete_type_mismatch flag, the same one
protecting new/delete matching size check.

Differential revision: https://reviews.llvm.org/D38574

Issue: https://github.com/google/sanitizers/issues/799
llvm-svn: 316595
2017-10-25 17:21:37 +00:00
Kamil Rytarowski 9c1eeaca80 Add NetBSD improvements in sanitizers
Summary:
Changes:

 * Add initial msan stub support.
 * Handle NetBSD specific pthread_setname_np(3).
 * NetBSD supports __attribute__((tls_model("initial-exec"))),
   define it in SANITIZER_TLS_INITIAL_EXEC_ATTRIBUTE.
 * Add ReExec() specific bits for NetBSD.
 * Simplify code and add syscall64 and syscall_ptr for !NetBSD.
 * Correct bunch of syscall wrappers for NetBSD.
 * Disable test/tsan/map32bit on NetBSD as not applicable.
 * Port test/tsan/strerror_r to a POSIX-compliant OSes.
 * Disable __libc_stack_end on NetBSD.
 * Disable ReadNullSepFileToArray() on NetBSD.
 * Define struct_ElfW_Phdr_sz, detected missing symbol by msan.
 * Change type of __sanitizer_FILE from void to char. This helps
   to reuse this type as an array. Long term it will be properly
   implemented along with SANITIZER_HAS_STRUCT_FILE setting to 1.
 * Add initial NetBSD support in lib/tsan/go/buildgo.sh.
 * Correct referencing stdout and stderr in tsan_interceptors.cc
   on NetBSD.
 * Document NetBSD x86_64 specific virtual memory layout in
   tsan_platform.h.
 * Port tests/rtl/tsan_test_util_posix.cc to NetBSD.
 * Enable NetBSD tests in test/msan/lit.cfg.
 * Enable NetBSD tests in test/tsan/lit.cfg.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, vitalybuka, eugenis, kcc, dvyukov

Reviewed By: dvyukov

Subscribers: #sanitizers, llvm-commits, kubamracek

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D39124

llvm-svn: 316591
2017-10-25 17:09:05 +00:00
Reid Kleckner 03d02a0e1d [asan] Don't print rows of shadow bytes outside shadow memory
Summary:
They might not be mapped on some platforms such as Win64.  In
particular, this happens if the user address is null. There will not be
any shadow memory 5*16 bytes before the user address. This happens on
Win64 in the error_report_callback.cc test case. It's not clear why this
isn't a problem on Linux as well.

Fixes PR35058

Reviewers: vitalybuka

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D39260

llvm-svn: 316589
2017-10-25 16:54:12 +00:00
Dmitry Vyukov 60d8ad2849 [tsan] Fix warnings in tsan_interceptors.cc from expansion of variadic macros
C99 technically requires the rest arguments to be used in C variadic macros.
This presents a problem with the macro SCOPED_TSAN_INTERCEPTOR when func
takes no arguments. This happens with the function pause. Like other void
argument functions, we pass in a fake argument to avoid this warning.

Author: Alex Langford (xiaobai)
Reviewed in: https://reviews.llvm.org/D39151

llvm-svn: 316558
2017-10-25 08:05:13 +00:00
Dean Michael Berris 389ebc0c4f [XRay][compiler-rt][NFC] Clean up xray log files before running test
Improves the test behaviour in the face of failure. Without this change
the fdr-single-thread.cc test may leave around artefacts of a previous
failing run since the cleanup doesn't happen if any of the intermediary
steps fail.

Non-functional change.

Subscribers: llvm-commits
llvm-svn: 316548
2017-10-25 05:19:20 +00:00
Dimitry Andric 95af9e654f On FreeBSD, skip the first entry in the dl_iterate_phdr list.
Summary:
Similar to NetBSD, in FreeBSD, the first returned entry when callbacks
are done via dl_iterate_phdr will return the main program.  Ignore that
entry when checking that the dynamic ASan lib is loaded first.

Reviewers: eugenis, krytarowski, emaste, joerg

Reviewed By: eugenis, krytarowski

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D39253

llvm-svn: 316487
2017-10-24 19:45:59 +00:00
Reid Kleckner 80db37ec25 [asan] Add more x86 prologue decodings to handle x64 VC 2017 CRT
Fixes atoi and strtol interception.

llvm-svn: 316482
2017-10-24 18:22:07 +00:00
Dean Michael Berris 23e54d85be [XRay][compiler-rt] More fixups.
Follow-up to D39175.

llvm-svn: 316410
2017-10-24 02:43:49 +00:00
Dean Michael Berris 2e592ea7ed [XRay][compiler-rt] Fixup shadowing
Follow-up to D39175.

llvm-svn: 316409
2017-10-24 02:36:32 +00:00
Dean Michael Berris 0b1cce2036 [XRay][compiler-rt] Remove C++ STL from the buffer queue implementation
Summary:
This change removes the dependency on C++ standard library
types/functions in the implementation of the buffer queue. This is an
incremental step in resolving llvm.org/PR32274.

Reviewers: dblaikie, pelikan

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D39175

llvm-svn: 316406
2017-10-24 01:39:59 +00:00
Alex Shlyapnikov 5ded0701a9 [Sanitizers-libFuzzer] Addressing coding style issues.
Summary: The result of clang-format and few manual changes (as prompted on D39155).

Reviewers: vitalybuka

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D39211

llvm-svn: 316395
2017-10-23 23:24:33 +00:00
Alex Shlyapnikov 6f1c26f222 [libFuzzer] Periodically purge allocator's quarantine to prolong fuzzing sessions.
Summary:
Fuzzing targets that allocate/deallocate a lot of memory tend to consume
a lot of RSS when ASan quarantine is enabled. Purging quarantine between
iterations and returning memory to OS keeps RSS down and should not
reduce the quarantine effectiveness provided the fuzz target does not
preserve state between iterations (in this case this feature can be turned off).

Based on D39153.

Reviewers: vitalybuka

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D39155

llvm-svn: 316382
2017-10-23 22:04:30 +00:00
Alex Shlyapnikov 1108d072f4 [Sanitizers] Add total primary allocator RSS to allocator report.
Summary: .

Reviewers: cryptoad

Subscribers: llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D39131

llvm-svn: 316356
2017-10-23 17:58:16 +00:00
Evgeniy Stepanov aedda3bafc Revert "[Compiler-rt][MIPS] Fix cross build for XRAY."
Breaks build:
http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux/builds/4677/steps/build%20with%20ninja/logs/stdio

In file included from compiler-rt/lib/xray/xray_fdr_logging.cc:34:
In file included from compiler-rt/lib/xray/xray_fdr_logging_impl.h:36:
In file included from compiler-rt/lib/xray/xray_flags.h:18:
compiler-rt/lib/xray/../sanitizer_common/sanitizer_flag_parser.h:23:7: error: '__sanitizer::FlagHandlerBase' has virtual functions but non-virtual destructor [-Werror,-Wnon-virtual-dtor]
class FlagHandlerBase {

llvm-svn: 316348
2017-10-23 17:13:24 +00:00
Alex Shlyapnikov 028c4cddad [Sanitizers] New sanitizer API to purge allocator quarantine.
Summary:
Purging allocator quarantine and returning memory to OS might be desired
between fuzzer iterations since, most likely, the quarantine is not
going to catch bugs in the code under fuzz, but reducing RSS might
significantly prolong the fuzzing session.

Reviewers: cryptoad

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D39153

llvm-svn: 316347
2017-10-23 17:12:07 +00:00
Kostya Kortchinsky 9e917a13fb [scudo] Add a shared runtime
Summary:
Up to now, the Scudo cmake target only provided a static library that had to be
linked to an executable to benefit from the hardened allocator.
This introduces a shared library as well, that can be LD_PRELOAD'ed.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: srhines, mgorny, llvm-commits

Differential Revision: https://reviews.llvm.org/D38980

llvm-svn: 316342
2017-10-23 16:27:47 +00:00
Nitesh Jain cf8a5c26f9 [Compiler-rt][MIPS] Fix cross build for XRAY.
Reviewers: dberris, sdardis

Subscribers: jaydeep, bhushan, llvm-commits

Differential Revision: https://reviews.llvm.org/D38021

llvm-svn: 316286
2017-10-22 09:37:50 +00:00
Dmitry Vyukov 3d34d877cb tsan: add tests missed in r316209
llvm-svn: 316210
2017-10-20 12:10:21 +00:00
Dmitry Vyukov 9e2cd1c125 [tsan] Add Mutex annotation flag for constant-initialized __tsan_mutex_linker_init behavior
Add a new flag, _⁠_tsan_mutex_not_static, which has the opposite sense
of _⁠_tsan_mutex_linker_init. When the new _⁠_tsan_mutex_not_static flag
is passed to _⁠_tsan_mutex_destroy, tsan ignores the destruction unless
the mutex was also created with the _⁠_tsan_mutex_not_static flag.

This is useful for constructors that otherwise woud set
_⁠_tsan_mutex_linker_init but cannot, because they are declared constexpr.

Google has a custom mutex with two constructors, a "linker initialized"
constructor that relies on zero-initialization and sets
⁠_⁠_tsan_mutex_linker_init, and a normal one which sets no tsan flags.
The "linker initialized" constructor is morally constexpr, but we can't
declare it constexpr because of the need to call into tsan as a side effect.

With this new flag, the normal c'tor can set _⁠_tsan_mutex_not_static,
the "linker initialized" constructor can rely on tsan's lazy initialization,
and _⁠_tsan_mutex_destroy can still handle both cases correctly.

Author: Greg Falcon (gfalcon)
Reviewed in: https://reviews.llvm.org/D39095

llvm-svn: 316209
2017-10-20 12:08:53 +00:00
Marco Castelluccio 17103fc12f Use O_BINARY when opening GCDA file on Windows
Summary:
Fixes https://bugs.llvm.org/show_bug.cgi?id=34922.

Apparently, the mode in **fdopen** gets simply ignored and Windows only cares about the mode of the original **open**.

I have verified this both with the simple case from bug 34922 and with a full Firefox build.

Reviewers: zturner

Reviewed By: zturner

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D38984

llvm-svn: 316048
2017-10-18 00:22:01 +00:00
Evgeniy Stepanov db8d53cdad Revert "[asan] Deflake one test by running it 3 times."
Disable this test on Android/x86 only.

llvm-svn: 316023
2017-10-17 19:57:48 +00:00
Dean Michael Berris 1d2f9e2bdd [XRay][compiler-rt] Always place the CPU record first for every buffer
Summary:
In FDR Mode, when we set up a new buffer for a thread that's just
overflowed, we must place the CPU identifier with the TSC record as the
first record. This is so that we can reconstruct all the function
entry/exit with deltas rooted on a TSC record for the CPU at the
beginning of the buffer.

Without doing this, the tools are rejecting the log for cases when we've
overflown and have different buffers that don't have the CPU and TSC
records as the first entry in the buffers.

Reviewers: pelikan

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D38995

llvm-svn: 315987
2017-10-17 10:33:24 +00:00
Evgeniy Stepanov 1993de54ee [cfi] Test cross-dso CFI on Android.
Reviewers: vitalybuka, pcc

Subscribers: llvm-commits, srhines

Differential Revision: https://reviews.llvm.org/D38911

llvm-svn: 315922
2017-10-16 18:03:11 +00:00
Kostya Kortchinsky 73a80c5493 [scudo] Do not include sanitizer_posix.h if not on a Posix platform
Summary:
Move the `sanitizer_posix.h` include within the `SANITIZER_ANDROID` `#if`,
otherwise this errors when built on non-Posix platforms (eg: Fuchsia).

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D38956

llvm-svn: 315917
2017-10-16 17:06:13 +00:00
Vitaly Buka ac03fb616f [asan] Increase kHandlerStackSize for TracerThreadSignalHandler
4096 is not enough on some platform, e.g. Debian 4.9.0-3-amd64

llvm-svn: 315844
2017-10-15 04:18:29 +00:00
George Karpenkov 1b11460610 [xray] Fix CMake for X-RAY tests
Correctly depend on llvm-xray, make sure unit tests are being run.

Differential Revision: https://reviews.llvm.org/D38917

llvm-svn: 315827
2017-10-14 21:38:13 +00:00
Kostya Serebryany fc3079963f [libFuzzer] print a better warning if we hit the ld bug
llvm-svn: 315778
2017-10-14 00:07:11 +00:00
Evgeniy Stepanov 4fb209ede2 [asan] Deflake one test by running it 3 times.
The test seems to trigger an android platform bug under load.

llvm-svn: 315777
2017-10-13 23:57:08 +00:00
Evgeniy Stepanov f156c932ae [cfi] Use %ld_flags_rpath_* substitutions in tests (NFC)
llvm-svn: 315775
2017-10-13 23:33:12 +00:00
Alex Shlyapnikov 29d3ba7576 [Sanitizers] Always build libFuzzer with frame pointers.
Lacking frame pointers, sanitized fuzzers collect bogus stack
traces and the set of stack traces grows indefinitely, leading to OOMs.

llvm-svn: 315770
2017-10-13 21:57:43 +00:00
Peter Collingbourne 868783e855 LowerTypeTests: Give imported symbols a type with size 0 so that they are not assumed not to alias.
It is possible for both a base and a derived class to be satisfied
with a unique vtable. If a program contains casts of the same pointer
to both of those types, the CFI checks will be lowered to this
(with ThinLTO):

if (p != &__typeid_base_global_addr)
  trap();
if (p != &__typeid_derived_global_addr)
  trap();

The optimizer may then use the first condition combined
with the assumption that __typeid_base_global_addr and
__typeid_derived_global_addr may not alias to optimize away the second
comparison, resulting in an unconditional trap.

This patch fixes the bug by giving imported globals the type [0 x i8]*,
which prevents the optimizer from assuming that they do not alias.

Differential Revision: https://reviews.llvm.org/D38873

llvm-svn: 315753
2017-10-13 21:02:16 +00:00
Kostya Kortchinsky f4c11e353a [scudo] Allow for non-Android Shared TSD platforms, part 2
Summary:
Follow up to D38826.

We introduce `pthread_{get,set}specific` versions of `{get,set}CurrentTSD` to
allow for non Android platforms to use the Shared TSD model.
We now allow `SCUDO_TSD_EXCLUSIVE` to be defined at compile time.

A couple of things:
- I know that `#if SANITIZER_ANDROID` is not ideal within a function, but in
  the end I feel it looks more compact and clean than going the .inc route; I
  am open to an alternative if anyone has one;
- `SCUDO_TSD_EXCLUSIVE=1` requires ELF TLS support (and not emutls as this uses
  malloc). I haven't found anything to enforce that, so it's currently not
  checked.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: srhines, llvm-commits

Differential Revision: https://reviews.llvm.org/D38854

llvm-svn: 315751
2017-10-13 20:55:31 +00:00
Alex Shlyapnikov 8a8d945df7 [Sanitizers] Add more details to ASan allocator stats report.
Summary: .

Reviewers: cryptoad

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D38834

llvm-svn: 315730
2017-10-13 18:38:10 +00:00
Kostya Serebryany 51823d3aae [libFuzzer] tweam use_feature_frequency to be less aggressive; run a dummy input before the seed corpus
llvm-svn: 315657
2017-10-13 01:12:23 +00:00
Evgeniy Stepanov 150b7d6f55 [sanitizer] Workaround a Linux kernel bug in hard_rss_limit_mb_test.
llvm-svn: 315632
2017-10-12 22:24:58 +00:00
Michal Gorny 4e9e939b4f [cmake] [asan] Remove unnecessary gtest dep from dynamic tests
Remove the redundant dependency on 'gtest' target from the dynamic tests
in non-MSVC environment. The tests reuse compiled objects
from ASAN_INST_TEST_OBJECTS, and therefore they have been built against
gtest already.

This both fixes the spurious dependency on 'gtest' target that breaks
stand-alone builds, and brings the dynamic tests more in line with
regular tests which do not pass this dependency
to add_compiler_rt_test() through generate_compiler_rt_tests().

Differential Revision: https://reviews.llvm.org/D38840

llvm-svn: 315620
2017-10-12 21:07:43 +00:00
Michal Gorny 007b540feb [cmake] [interception] Remove duplicate gtest from test COMPILE_DEPS
Fix the gtest dependency to be included in DEPS only, rather than
in COMPILE_DEPS + DEPS. The former variable is apparently used to
provide unconditional dependencies, while the latter are only used
for non-standalone builds. Since they are concatenated, specifying gtest
in both is redundant. Furthermore, including it in COMPILE_DEPS causes
build failure for standalone builds where 'gtest' target is not present.

Differential Revision: https://reviews.llvm.org/D38839

llvm-svn: 315605
2017-10-12 18:51:41 +00:00
Michal Gorny 858509cfb3 [cmake] Fix skipping DEPS (typo) in sanitizer_test_compile()
Fix typo in variable assignment inside sanitizer_test_compile() that
resulted in TEST_DEPS parameter not being included in the clang_compile()
call. Spotted by George Karpenkov in D38444.

Differential Revision: https://reviews.llvm.org/D38838

llvm-svn: 315604
2017-10-12 18:51:37 +00:00
Evgeniy Stepanov fe35898b42 [asan] Disable a flaky test on android.
llvm-svn: 315602
2017-10-12 18:34:20 +00:00
Kostya Kortchinsky 8d4ba5fd23 [scudo] Allow for non-Android Shared TSD platforms, part 1
Summary:
This first part just prepares the grounds for part 2 and doesn't add any new
functionality. It mostly consists of small refactors:
- move the `pthread.h` include higher as it will be used in the headers;
- use `errno.h` in `scudo_allocator.cpp` instead of the sanitizer one, update
  the `errno` assignments accordingly (otherwise it creates conflicts on some
  platforms due to `pthread.h` including `errno.h`);
- introduce and use `getCurrentTSD` and `setCurrentTSD` for the shared TSD
  model code;

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: llvm-commits, srhines

Differential Revision: https://reviews.llvm.org/D38826

llvm-svn: 315583
2017-10-12 15:01:09 +00:00
Hans Wennborg eb2218ceae Revert r315533 "Reland "[sanitizer] Introduce ReservedAddressRange to sanitizer_common""
The SanitizerCommon.ReservedAddressRangeUnmap test fails on Windows:

FAIL: SanitizerCommon-Unit :: ./Sanitizer-x86_64-Test.exe/SanitizerCommon.ReservedAddressRangeUnmap (34003 of 35554)
******************** TEST 'SanitizerCommon-Unit :: ./Sanitizer-x86_64-Test.exe/SanitizerCommon.ReservedAddressRangeUnmap' FAILED ********************
Note: Google Test filter = SanitizerCommon.ReservedAddressRangeUnmap
[==========] Running 1 test from 1 test case.
[----------] Global test environment set-up.
[----------] 1 test from SanitizerCommon
[ RUN      ] SanitizerCommon.ReservedAddressRangeUnmap
==3780==ERROR: SanitizerTool failed to deallocate 0x1000 (4096) bytes at address 0x0000000c3000 (error code: 487)
==3780==Sanitizer CHECK failed: E:\b\build\slave\win_upload_clang\build\src\third_party\llvm\projects\compiler-rt\lib\sanitizer_common\sanitizer_win.cc:129 (("unable to unmap" && 0)) != (0) (0, 0)
********************
Testing: 0 .. 10.. 20.. 30.. 40.. 50.. 60.. 70.. 80.. 90..
Testing Time: 299.76s
********************
Failing Tests (1):
    SanitizerCommon-Unit :: ./Sanitizer-x86_64-Test.exe/SanitizerCommon.ReservedAddressRangeUnmap

> In Fuchsia, MmapNoAccess/MmapFixedOrDie are implemented using a global
> VMAR, which means that MmapNoAccess can only be called once. This works
> for the sanitizer allocator but *not* for the Scudo allocator.
>
> Hence, this changeset introduces a new ReservedAddressRange object to
> serve as the new API for these calls. In this changeset, the object
> still calls into the old Mmap implementations.
>
> The next changeset two changesets will convert the sanitizer and scudo
> allocators to use the new APIs, respectively. (ReservedAddressRange will
> replace the SecondaryHeader in Scudo.)
>
> Finally, a last changeset will update the Fuchsia implementation.
>
> Patch by Julia Hansbrough
>
> Differential Revision: https://reviews.llvm.org/D38437

llvm-svn: 315553
2017-10-12 03:23:31 +00:00
Petr Hosek 4f213ae3a7 Reland "[sanitizer] Introduce ReservedAddressRange to sanitizer_common"
In Fuchsia, MmapNoAccess/MmapFixedOrDie are implemented using a global
VMAR, which means that MmapNoAccess can only be called once. This works
for the sanitizer allocator but *not* for the Scudo allocator.

Hence, this changeset introduces a new ReservedAddressRange object to
serve as the new API for these calls. In this changeset, the object
still calls into the old Mmap implementations.

The next changeset two changesets will convert the sanitizer and scudo
allocators to use the new APIs, respectively. (ReservedAddressRange will
replace the SecondaryHeader in Scudo.)

Finally, a last changeset will update the Fuchsia implementation.

Patch by Julia Hansbrough

Differential Revision: https://reviews.llvm.org/D38437

llvm-svn: 315533
2017-10-11 23:41:32 +00:00
Evgeniy Stepanov 1961c6cb2c Disable TSan tests on Android.
They never passed. This change excludes them from 'check-all'.

llvm-svn: 315512
2017-10-11 21:22:45 +00:00
Evgeniy Stepanov 3747b826c4 Disable profile tests on Android.
They never actually worked, but this way they are not included in "check-all".

llvm-svn: 315511
2017-10-11 21:22:32 +00:00
Kostya Kortchinsky 6a45f9ce82 [sanitizer] Move the errno/ENOMEM allocator checks logic to separate .cc
Summary:
This is a new attempt at D38706, which had 2 issues.

The first one was that it broke TSan, because `sanitizer_errno.h` was not
directly included in `tsan_mman.cc`. This fixes the include.

The second one was that it broke the nolibc build, because `__errno_location`
couldn't be found. This adds the new .cc to the libcdep list instead of the
base one.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: kubamracek, mgorny, llvm-commits

Differential Revision: https://reviews.llvm.org/D38743

llvm-svn: 315509
2017-10-11 21:20:04 +00:00
Evgeniy Stepanov 450ae592b4 [ubsan] Support ubsan-minimal tests on Android.
llvm-svn: 315507
2017-10-11 20:46:13 +00:00
Craig Topper 705b969f80 [X86] Add Knights Mill CPU to cpu_indicator support to match libgcc.
llvm-svn: 315505
2017-10-11 20:35:43 +00:00
Kostya Serebryany 75480e3871 [libFuzzer] disable use_feature_frequency as it degrades some of the benchmarks too much :(
llvm-svn: 315503
2017-10-11 20:31:01 +00:00
Petr Hosek c530f497b8 Revert "[sanitizer] Introduce ReservedAddressRange to sanitizer_common"
This reverts commit r315493 which is failing to build on sanitizer-windows.

llvm-svn: 315494
2017-10-11 19:29:14 +00:00
Petr Hosek 2bd0123afc [sanitizer] Introduce ReservedAddressRange to sanitizer_common
In Fuchsia, MmapNoAccess/MmapFixedOrDie are implemented using a global
VMAR, which means that MmapNoAccess can only be called once. This works
for the sanitizer allocator but *not* for the Scudo allocator.

Hence, this changeset introduces a new ReservedAddressRange object to
serve as the new API for these calls. In this changeset, the object
still calls into the old Mmap implementations.

The next changeset two changesets will convert the sanitizer and scudo
allocators to use the new APIs, respectively. (ReservedAddressRange will
replace the SecondaryHeader in Scudo.)

Finally, a last changeset will update the Fuchsia implementation.

Patch by Julia Hansbrough

Differential Revision: https://reviews.llvm.org/D38759

llvm-svn: 315493
2017-10-11 19:17:35 +00:00
Kostya Serebryany 198d385a77 [libFuzzer] make -use_feature_frequency less aggressive and enable by default
llvm-svn: 315490
2017-10-11 19:01:35 +00:00
Evgeniy Stepanov b751082d56 [asan] Tweak test output to diagnose buildbot failures.
llvm-svn: 315479
2017-10-11 17:32:38 +00:00
Kostya Serebryany 4083d54675 [libFuzzer] experimental flag to tweak the corpus distribution. Seems to improve the situation dramatically on the png benchmark and make things worse on a number of micro-puzzles. Needs more A/B testing
llvm-svn: 315407
2017-10-11 01:44:26 +00:00
Evgeniy Stepanov 922973a466 [sanitizer] Re-disable several tests on Android.
The tests have been enabled by accident in r315389.

llvm-svn: 315396
2017-10-11 00:32:03 +00:00
Evgeniy Stepanov 0bf7717a02 Factor out "stable-runtime" feature and enable it on all android.
This is a very poorly named feature. I think originally it meant to cover linux only, but the use of it in msan
seems to be about any aarch64 platform. Anyway, this change should be NFC on everything except Android.

llvm-svn: 315389
2017-10-10 23:37:26 +00:00
Hans Wennborg 4a6ee430bb XFAIL ubsan/TestCases/TypeCheck/Function/function.cpp on Windows
I think it got accidentally enabled in r315105 or thereabouts.

llvm-svn: 315374
2017-10-10 22:11:25 +00:00
Kostya Kortchinsky ea2f800bbc [sanitizer] Revert D38706
Summary:
D38706 breaks tsan and the nolibc build.
Reverting while working on a fix.

Reviewers: alekseyshl

Subscribers: kubamracek, mgorny, llvm-commits

Differential Revision: https://reviews.llvm.org/D38739

llvm-svn: 315320
2017-10-10 15:35:11 +00:00
Kostya Kortchinsky 03f9e76790 [sanitizer] Move the errno/ENOMEM allocator checks logic to separate .cc
Summary:
The fact that `sanitizer_allocator_checks.h` is including `sanitizer_errno.h`
creates complications for future changes, where it would conflict with `errno.h`
definitions on Android and Fuchsia (macro redefinition).

By moving the portion that sets errno in the checks to a separate compilation
unit, we avoid the inclusion of the header there, which solves the issue.

Not that it is not vital to have that function in a header as it is called as a
result of an unlikely event, and doesn't need to be inlined.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: kubamracek, llvm-commits, mgorny

Differential Revision: https://reviews.llvm.org/D38706

llvm-svn: 315319
2017-10-10 14:58:09 +00:00
Dean Michael Berris 9ba7401a7b [XRay][compiler-rt] Fix rdtscp support check for x86_64
Follow-up to D29438.

llvm-svn: 315306
2017-10-10 12:44:20 +00:00
Evgeniy Stepanov cb993304ac [ubsan] Fix Asan internal alloc corruption in PR33221 test.
MAP_FIXED discards the existing mapping at the given address.

llvm-svn: 315247
2017-10-09 22:52:13 +00:00
Petr Hosek 1c0c35d034 [sanitizer] Don't intercept signal and sigaction on Fuchsia
Fuchsia doesn't support signals, so don't use interceptors for signal or
sigaction.

Differential Revision: https://reviews.llvm.org/D38669

llvm-svn: 315227
2017-10-09 18:29:52 +00:00
Evgeniy Stepanov 8804483b05 [ubsan] Disable one test on Android.
llvm-svn: 315220
2017-10-09 17:45:03 +00:00
Vlad Tsyrklevich a0b02833bc Limit CFI blacklist entries to specific CFI mode
Summary:
Since D37924 and D37925 were merged, it's now possible to specify
individual sanitizers or CFI modes in sanitizer blacklists. Update the
CFI blacklist entries to only apply to cfi-unrelated-cast checks.

Reviewers: eugenis, pcc

Reviewed By: eugenis

Subscribers: kcc

Differential Revision: https://reviews.llvm.org/D38385

llvm-svn: 315216
2017-10-09 17:11:44 +00:00
Peter Collingbourne 107a9f624d Make the cfi target available on more platforms.
On non-Linux targets it just installs the blacklist.

Differential Revision: https://reviews.llvm.org/D38661

llvm-svn: 315215
2017-10-09 17:07:47 +00:00
Vedant Kumar 2465e64846 cmake: Fix one more usage of append()
append() isn't available with some cmake versions, so I need to use a
different construct. I missed this case in r315144.

http://lab.llvm.org:8080/green/job/clang-stage1-configure-RA/39355

llvm-svn: 315157
2017-10-07 20:20:42 +00:00
Vedant Kumar 16d41609ab Use list(APPEND) instead of append()
append() seems to be available with the version of cmake I'm using, but
not on the bots:

http://green.lab.llvm.org/green//job/clang-stage1-configure-RA/39354

llvm-svn: 315144
2017-10-07 03:44:46 +00:00
Vedant Kumar 074a997b13 [ubsan] Add a static runtime on Darwin
As a follow-up to r315142, this makes it possible to use ubsan with a
static runtime on Darwin. I've also added a new StandaloneStatic testing
configuration so the new setup can be tested.

llvm-svn: 315143
2017-10-07 01:46:36 +00:00
Evgeniy Stepanov 729bcd1f6e [asan] Disable wcslen test on 32-bit Android.
llvm-svn: 315132
2017-10-07 00:04:24 +00:00
Evgeniy Stepanov 0a4217c14b Factor out default_(a|ub)sanitizer_opts in lit.
Reviewers: vitalybuka

Subscribers: srhines, llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D38644

llvm-svn: 315106
2017-10-06 20:53:40 +00:00
Evgeniy Stepanov f29ee9a050 [sanitizer] Test ubsan and cfi on android.
Summary:
Enable check-cfi and check-ubsan on Android.
Check-ubsan includes standalone and ubsan+asan, but not tsan or msan.
Cross-dso cfi tests are disabled for now.

Reviewers: vitalybuka, pcc

Subscribers: srhines, kubamracek, llvm-commits, mgorny

Differential Revision: https://reviews.llvm.org/D38608

llvm-svn: 315105
2017-10-06 20:51:51 +00:00
Evgeniy Stepanov 505260449e [asan] Use gold linker in android tests.
Replace a partial workaround for ld.bfd strangeness with the ultimate one: -fuse-ld=gold.
Reason: ld.bfd problem gets worse with libc++-based NDK toolchain.

llvm-svn: 315039
2017-10-06 00:00:53 +00:00
Alex Shlyapnikov 84d16165d4 Revert "[LSan] Detect dynamic loader by its base address."
This reverts commit r315024.

Breaks sysconf_interceptor_bypass_test.cc

llvm-svn: 315031
2017-10-05 22:53:17 +00:00
Max Moroz 330496c3fc [libFuzzer] Disable experimental clang coverage support by default.
Summary:
It can be enabled via "-use_clang_coverage=1" flag. Reason for disabling:
libFuzzer resets Clang Counters and makes it impossible to generate coverage
report for a regular fuzz target (i.e. not standalone build).

Reviewers: kcc

Reviewed By: kcc

Subscribers: kcc

Differential Revision: https://reviews.llvm.org/D38604

llvm-svn: 315029
2017-10-05 22:41:03 +00:00
Evgeniy Stepanov 0d0325606b [cfi] Disable tests with lld on i386.
bin/ld.lld: error: ubsan_handlers.cc:(.debug_info+0x80D5D): has non-ABS reloc

Bug pending.

llvm-svn: 315027
2017-10-05 22:07:21 +00:00
Alex Shlyapnikov 44f9376347 [LSan] Detect dynamic loader by its base address.
Summary:
Relanding D33859, which was reverted because it has "broken LOTS of
ARM/AArch64 bots for two days".

If it breaks something again, please provide some pointers to broken
bots, not just revert it, otherwise it's very hard to reason what's
wrong with this commit.

Whenever possible (Linux + glibc 2.16+), detect dynamic loader module by
its base address, not by the module name matching. The current name
matching approach fails on some configurations.

Reviewers: eugenis

Subscribers: aemerson, kubamracek, kristof.beyls, llvm-commits

Differential Revision: https://reviews.llvm.org/D38600

llvm-svn: 315024
2017-10-05 21:38:33 +00:00
Evgeniy Stepanov 1917176d47 [asan] Add --enable-new-dtags in tests on Android.
The dynamic loader does not accept DT_RPATH; it wants DT_RUNPATH.

This is a temporary fix until D38430 lands.

llvm-svn: 315020
2017-10-05 20:45:12 +00:00
Evgeniy Stepanov 4a3e502d5d [asan] Use full binary path in the Android test runner.
Summary:
This prevents the confusion when there are similarly named tests in
different configurations (like in test/sanitizer_common).

Reviewers: vitalybuka

Subscribers: srhines, llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D38526

llvm-svn: 315011
2017-10-05 20:00:19 +00:00
Evgeniy Stepanov 3b87cc910e [sanitizer] Move %ld_flags_rpath_exe to common and use it in more tests.
Reviewers: vitalybuka

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D38527

llvm-svn: 315010
2017-10-05 20:00:07 +00:00
Evgeniy Stepanov ca6c8e85d1 [cfi] Mark a test as unsupported on darwin.
llvm-svn: 315007
2017-10-05 19:36:15 +00:00
Evgeniy Stepanov f7e14f4c72 [cfi] Test on all available targets.
Summary:
Run CFI tests on all targets current toolchain can target.
On multiarch Linux, this will run all CFI tests with -m32 and -m64.

Reviewers: pcc

Subscribers: mgorny, llvm-commits

Differential Revision: https://reviews.llvm.org/D38572

llvm-svn: 315001
2017-10-05 18:28:04 +00:00
Dean Michael Berris 8dcba551d9 [XRay][compiler-rt] Write out arg1 payload in naive mode logging
Summary:
This change allows the XRay basic (naive) mode logging implementation to
start writing the payload entries through the arg1 logging handler. This
implementation writes out the records that the llvm-xray tool and the
trace reader library will start processing in D38550.

This introduces a new payload record type which logs the data through
the in-memory buffer. It uses the same size/alignment that the normal
XRay record entries use. We use a new record type to indicate these new
entries, so that the trace reader library in LLVM can start reading
these entries.

Depends on D38550.

Reviewers: pelikan

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D38551

llvm-svn: 314968
2017-10-05 05:45:51 +00:00
Evgeniy Stepanov 928bc08247 [sanitizer] Move cxx-abi library earlier in link flags.
Summary:
This change moves cxx-abi library in asan/ubsan/dd link command line
ahead of other libraries, such as pthread/rt/dl/c/gcc. Given that
cxx-abi may be the full libstdc++/libc++, it makes sense for it to be
ahead of libc and libgcc, at least.

The real motivation is Android, where in the arm32 NDK toolchain
libstdc++.a is actually a linker script that tries to sneak LLVM's
libunwind ahead of libgcc's. Wrong library order breaks unwinding.

Reviewers: srhines, danalbert

Subscribers: aemerson, kubamracek, mgorny, kristof.beyls, llvm-commits

Differential Revision: https://reviews.llvm.org/D38520

llvm-svn: 314948
2017-10-04 23:35:14 +00:00
Dean Michael Berris f279d9b409 [XRay][compiler-rt] Fix logical failure in BufferQueue::getBuffer()
Follow-up to D38119.

llvm-svn: 314878
2017-10-04 06:02:12 +00:00
Dean Michael Berris d06e917b9e [XRay][compiler-rt] Use a hand-written circular buffer in BufferQueue
Summary:
This change removes the dependency on using a std::deque<...> for the
storage of the buffers in the buffer queue. We instead implement a
fixed-size circular buffer that's resilient to exhaustion, and preserves
the semantics of the BufferQueue.

We're moving away from using std::deque<...> for two reasons:

  - We want to remove dependencies on the STL for data structures.

  - We want the data structure we use to not require re-allocation in
    the normal course of operation.

The internal implementation of the buffer queue uses heap-allocated
arrays that are initialized once when the BufferQueue is created, and
re-uses slots in the buffer array as buffers are returned in order.

We also change the lock used in the implementation to a spinlock
instead of a blocking mutex. We reason that since the release operations
now take very little time in the critical section, that a spinlock would
be appropriate.

This change is related to D38073.

This change is a re-submit with the following changes:

  - Keeping track of the live buffers with a counter independent of the
    pointers keeping track of the extents of the circular buffer.

  - Additional documentation of what the data members are meant to
    represent.

Reviewers: dblaikie, kpw, pelikan

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D38119

llvm-svn: 314877
2017-10-04 05:20:13 +00:00
Martin Pelikan 68ea360ed1 [XRay] [compiler-rt] make sure single threaded programs get traced too
Summary:
When the XRay user calls the API to finish writing the log, the thread
which is calling the API still hasn't finished and therefore won't get
its trace written.  Add a test for only the main thread to check this.

Reviewers: dberris

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D38493

llvm-svn: 314875
2017-10-04 05:12:00 +00:00
Eli Friedman 0d586d06a7 [compiler-rt] Add back ARM EABI aliases where legal.
r303188 removed all the uses of aliases for EABI functions from
compiler-rt, because some of them had mismatched calling conventions.
Obviously, we can't use aliases for functions which don't have the same
calling convention, but that's only an issue for floating-point
functions with the hardfloat ABI.  In other cases, the stubs increase
size and reduce performance for no benefit.

This patch adds back the aliases, with appropriate checks to make sure
they're only used in cases where the calling convention matches.

llvm-svn: 314851
2017-10-03 21:25:07 +00:00
Dean Michael Berris 11415ac44e Revert "[XRay][compiler-rt] Use a hand-written circular buffer in BufferQueue"
This reverts r314766 (rL314766). Unit tests fail in multiple bots.

llvm-svn: 314786
2017-10-03 11:40:54 +00:00
Dean Michael Berris a1b8e0352f [XRay][compiler-rt] Use a hand-written circular buffer in BufferQueue
Summary:
This change removes the dependency on using a std::deque<...> for the
storage of the buffers in the buffer queue. We instead implement a
fixed-size circular buffer that's resilient to exhaustion, and preserves
the semantics of the BufferQueue.

We're moving away from using std::deque<...> for two reasons:

  - We want to remove dependencies on the STL for data structures.

  - We want the data structure we use to not require re-allocation in
    the normal course of operation.

The internal implementation of the buffer queue uses heap-allocated
arrays that are initialized once when the BufferQueue is created, and
re-uses slots in the buffer array as buffers are returned in order.

We also change the lock used in the implementation to a spinlock
instead of a blocking mutex. We reason that since the release operations
now take very little time in the critical section, that a spinlock would
be appropriate.

This change is related to D38073.

Reviewers: dblaikie, kpw, pelikan

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D38119

llvm-svn: 314766
2017-10-03 06:15:34 +00:00
Dean Michael Berris 4204464c76 fixup: use UNUSED, restore alignment for cache-line friendliness, and report on errors found when pthread_create_key fails
llvm-svn: 314765
2017-10-03 06:11:20 +00:00
Dean Michael Berris 1dbf45dc27 [XRay][compiler-rt] Use pthread for initializing thread-local data
Summary:
We avoid using C++11's thread_local keyword on non-trivially
destructible objects because it may introduce deadlocks when the C++
runtime registers destructors calling std::malloc(...). The deadlock may
happen when the allocator implementation is itself XRay instrumented.

To avoid having to call malloc(...) and free(...) in particular, we use
pthread_once, pthread_create_key, and pthread_setspecific to instead
manually register the cleanup implementation we want.

The code this replaces used an RAII type that implements the cleanup
functionality in the destructor, that was then initialized as a
function-local thread_local object. While it works in usual situations,
unfortunately it breaks when using a malloc implementation that itself
is XRay-instrumented.

Reviewers: dblaikie, kpw, pelikan

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D38073

llvm-svn: 314764
2017-10-03 06:11:13 +00:00
Manoj Gupta e2ff2ba57d [builtins] ARM: Reland fix for assembling builtins in thumb state.
Summary:
clang does not assemble files in thumb mode unless .thumb declaration
is present. Add .thumb/.arm decl to _FUNCTION macros to ensure 	that
files are assembled correctly.

Also add a fix to ensure that armv7k-watchos can assemble the
aeabi_c{f|d}cmp.S files.

Fixes PR 34715.

Reviewers: compnerd, peter.smith, srhines, weimingz, rengolin, efriedma, t.p.northover, fjricci

Reviewed By: compnerd

Subscribers: aemerson, javed.absar, llvm-commits, kristof.beyls

Differential Revision: https://reviews.llvm.org/D38390

llvm-svn: 314718
2017-10-02 20:56:49 +00:00
Francis Ricci daf210f7b6 Add support for custom loaders to the sanitizer symbolizer
Summary:
Adds a fallback mode to procmaps when the symbolizer
fails to locate a module for a given address by using
dl_iterate_phdr.

Reviewers: kubamracek, rnk, vitalybuka, eugenis

Reviewed By: eugenis

Subscribers: srhines, llvm-commits

Differential Revision: https://reviews.llvm.org/D37269

llvm-svn: 314713
2017-10-02 20:22:16 +00:00
Francis Ricci b7055bb813 Revert "Add support for custom loaders to the sanitizer symbolizer"
This reverts commit r314671, which hangs on the gcc sanitizer buildbot.

llvm-svn: 314684
2017-10-02 15:55:11 +00:00
Francis Ricci e2aa5b2ace Add support for custom loaders to the sanitizer symbolizer
Summary:
Adds a fallback mode to procmaps when the symbolizer
fails to locate a module for a given address by using
dl_iterate_phdr.

Reviewers: kubamracek, rnk, vitalybuka, eugenis

Reviewed By: eugenis

Subscribers: srhines, llvm-commits

Differential Revision: https://reviews.llvm.org/D37269

llvm-svn: 314671
2017-10-02 14:30:58 +00:00
Filipe Cabecinhas 3ccd23e116 Use %run for running CFI tests
Reviewers: pcc, krasin, eugenis

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D38412

llvm-svn: 314659
2017-10-02 10:21:26 +00:00
Michal Gorny 806b8d56a2 [cmake] Add a separate CMake var to control profile runtime
Make it possible to control building profile runtime separately from
other options. Before r313549, the profile runtime building was
controlled along with sanitizers. However, since that commit it is built
unconditionally which results in multiple builds for people building
different runtimes separately.

Differential Revision: https://reviews.llvm.org/D38441

llvm-svn: 314646
2017-10-02 05:03:55 +00:00
Francis Ricci 345187338e Move LoadedModule list to a NoCtor vector and initialize on demand.
Unreverting this patch because llvm-clang-lld-x86_64-debian-fast started
passing again before the revert hit. Must've been just a flake.

llvm-svn: 314556
2017-09-29 20:55:06 +00:00
Francis Ricci 74c93de659 Revert "Move LoadedModule list to a NoCtor vector and initialize on demand."
I think this may have introduced a failure on
llvm-clang-lld-x86_64-debian-fast

This reverts commit r314533

llvm-svn: 314552
2017-09-29 20:04:29 +00:00
Francis Ricci 113708b7cb Move LoadedModule list to a NoCtor vector and initialize on demand.
llvm-svn: 314533
2017-09-29 16:47:02 +00:00
Alex Shlyapnikov 54ea394963 Fix cmake file broken by D38277.
llvm-svn: 314528
2017-09-29 16:02:39 +00:00
Francis Ricci 8fb270c691 Add missing header definition
llvm-svn: 314521
2017-09-29 15:17:23 +00:00
Francis Ricci c48e9487a5 Remove recursion from FindModuleForAddress. NFC.
llvm-svn: 314520
2017-09-29 15:14:31 +00:00
Francis Ricci 5207630d7e Refactor android fallback procmaps init. NFC.
llvm-svn: 314518
2017-09-29 15:06:47 +00:00
Jonas Hahnfeld ddbb0926a0 [CMake] Fix configuration on PowerPC with sanitizers
TEST_BIG_ENDIAN() performs compile tests that will fail with
-nodefaultlibs when building under LLVM_USE_SANITIZER.

Differential Revision: https://reviews.llvm.org/D38277

llvm-svn: 314512
2017-09-29 13:32:39 +00:00
Dean Michael Berris fc064af181 [XRay][compiler-rt][NFC] Remove loggingInitialized() convenience function
The function was introduced as a convenience that used to be called in
multiple places. Recent refactorings have removed the need to call this
function in multiple places, so inlined the implementation in the single
place it's defined.

Broken out from D38119.

llvm-svn: 314489
2017-09-29 04:28:11 +00:00
Francis Ricci 052ec5f92a Revert "Add support for custom loaders to the sanitizer symbolizer"
This causes the gcc sanitizer buildbot to timeout.

This reverts commit 81f388fe570e5b6460dd5bc9b9a36b72714eeb68.

llvm-svn: 314453
2017-09-28 19:37:17 +00:00
Evgeniy Stepanov a41b578635 [asan] Fix the bug number in the error message.
The link in the "Shadow memory range interleaves with an existing
memory mapping" error message was pointing to the wrong bug.

llvm-svn: 314441
2017-09-28 18:19:44 +00:00
Francis Ricci b9a32d470a Add support for custom loaders to the sanitizer symbolizer
Summary:
Adds a fallback mode to procmaps when the symbolizer
fails to locate a module for a given address by using
dl_iterate_phdr.

Reviewers: kubamracek, rnk, vitalybuka, eugenis

Reviewed By: eugenis

Subscribers: srhines, llvm-commits

Differential Revision: https://reviews.llvm.org/D37269

llvm-svn: 314431
2017-09-28 16:58:35 +00:00
Benjamin Kramer c4e652f3aa [asan] Unpoison global metadata on dlclose.
dlclose itself might touch it, so better return it to the state it was
before. I don't know how to create a test for this as it would require
chaning dlclose itself.

llvm-svn: 314415
2017-09-28 13:38:58 +00:00
Dmitry Vyukov 47ad1ef2db tsan: handle signals in pause call
llvm-svn: 314384
2017-09-28 07:32:00 +00:00
Martin Pelikan 2e9cd5693f [XRay] [compiler-rt] FDR logging arg1 handler
Summary:
Write out records about logged function call first arguments. D32840
implements the reading of this in llvm-xray.

Reviewers: dberris

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D32844

llvm-svn: 314378
2017-09-28 05:29:59 +00:00
Evgeniy Stepanov d122381288 [ubsan] Merge ubsan_standalone_cxx shared library.
Summary:
Link everything, including the C++ bits, in the single
ubsan_standalone SHARED library. This matches ASan setup.

Reviewers: vitalybuka

Subscribers: kubamracek, llvm-commits, mgorny

Differential Revision: https://reviews.llvm.org/D38340

llvm-svn: 314369
2017-09-28 00:31:09 +00:00
Dmitry Mikulin d2224e17c3 ASan allocates a global data initialization array at the tail end of each
compunit's .data section. This vector is not poisoned. Because of this the
first symbol of the following section has no left red zone. As a result, ASan
cannot detect underflow for such symbols.

Poison ASan allocated metadata, it should not be accessible to user code.

This fix does not eliminate the problem with missing left red zones but it
reduces the set of vulnerable symbols from first symbols in each input data
section to first symbols in the output section of the binary.

Differential Revision: https://reviews.llvm.org/D38056

llvm-svn: 314365
2017-09-27 23:32:01 +00:00
Francis Ricci cb15e22b3b Revert "Add support for custom loaders to symbolizer"
This broke the windows buildbots, revert for now.

This reverts commit 24050b5ddef42f6f3306aa94d4a1f42a7893a9a7.

llvm-svn: 314347
2017-09-27 21:35:33 +00:00
Kostya Kortchinsky fa1ae3e862 [scudo] Temporary disabling the valloc test on armhf
Summary:
Weird failure where `errno != ENOMEM` on valloc failure. The returned pointer
is null since it passes the previous assert, so this shouldn't happen.

http://lab.llvm.org:8011/builders/clang-cmake-armv7-a15-full/builds/10931
http://lab.llvm.org:8011/builders/clang-cmake-thumbv7-a15-full-sh/builds/2469

Disabling until we figure out what's going on.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: aemerson, srhines, llvm-commits, kristof.beyls

Differential Revision: https://reviews.llvm.org/D38324

llvm-svn: 314344
2017-09-27 21:10:25 +00:00
Francis Ricci f80c42e589 Add support for custom loaders to symbolizer
Change-Id: I5594bd6b216deca2c73cf0a7001f9aec1e803c60
llvm-svn: 314342
2017-09-27 20:56:04 +00:00
Manoj Gupta bfa662ae4e [builtins] ARM: Revert r314284, r314285 and r314289
Revert r314284, r314285 and r314289 because of a reported
breakage in armv7k watchos builder.

llvm-svn: 314333
2017-09-27 19:06:47 +00:00
Manoj Gupta a2eacb3bfc [ARM] builtins: Replace abort by assert in clear_cache.
Summary:
__builtion___clear_cache maps to clear_cache function. On Linux,
clear_cache functions makes a syscall and does an abort if syscall fails.
Replace the abort by an assert so that non-debug builds do not abort
if the syscall fails.

Fixes PR34588.

Reviewers: rengolin, compnerd, srhines, peter.smith, joerg

Reviewed By: rengolin

Subscribers: aemerson, kristof.beyls, llvm-commits

Differential Revision: https://reviews.llvm.org/D37788

llvm-svn: 314322
2017-09-27 17:36:25 +00:00
Alex Shlyapnikov fb238e5907 [Sanitizer] Disable compact size class tests on Android
Fixing test failure on Android introduced in D38245. Compact size class
maps defined there are not to be used on Android.

llvm-svn: 314318
2017-09-27 17:10:49 +00:00
Alex Shlyapnikov 04ce5ac306 [Sanitizers] Allocator: new "release memory to OS" implementation
Summary:
The current implementation of the allocator returning freed memory
back to OS (controlled by allocator_release_to_os_interval_ms flag)
requires sorting of the free chunks list, which has two major issues,
first, when free list grows to millions of chunks, sorting, even the
fastest one, is just too slow, and second, sorting chunks in place
is unacceptable for Scudo allocator as it makes allocations more
predictable and less secure.

The proposed approach is linear in complexity (altough requires quite
a bit more temporary memory). The idea is to count the number of free
chunks on each memory page and release pages containing free chunks
only. It requires one iteration over the free list of chunks and one
iteration over the array of page counters. The obvious disadvantage
is the allocation of the array of the counters, but even in the worst
case we support (4T allocator space, 64 buckets, 16 bytes bucket size,
full free list, which leads to 2 bytes per page counter and ~17M page
counters), requires just about 34Mb of the intermediate buffer (comparing
to ~64Gb of actually allocated chunks) and usually it stays under 100K
and released after each use. It is expected to be a relatively rare event,
releasing memory back to OS, keeping the buffer between those runs
and added complexity of the bookkeeping seems unnesessary here (it can
always be improved later, though, never say never).

The most interesting problem here is how to calculate the number of chunks
falling into each memory page in the bucket. Skipping all the details,
there are three cases when the number of chunks per page is constant:
  1) P >= C, P % C == 0 --> N = P / C
  2) C > P , C % P == 0 --> N = 1
  3) C <= P, P % C != 0 && C % (P % C) == 0 --> N = P / C + 1
where P is page size, C is chunk size and N is the number of chunks per
page and the rest of the cases, where the number of chunks per page is
calculated on the go, during the page counter array iteration.

Among the rest, there are still cases where N can be deduced from the
page index, but they require not that much less calculations per page
than the current "brute force" way and 2/3 of the buckets fall into
the first three categories anyway, so, for the sake of simplicity,
it was decided to stick to those two variations. It can always be
refined and improved later, should we see that brute force way slows
us down unacceptably.

Reviewers: eugenis, cryptoad, dvyukov

Subscribers: kubamracek, mehdi_amini, llvm-commits

Differential Revision: https://reviews.llvm.org/D38245

llvm-svn: 314311
2017-09-27 15:38:05 +00:00
Manoj Gupta 92c21d54a4 [builtins] fix build error on non-ARM for r314285.
llvm-svn: 314289
2017-09-27 09:49:32 +00:00
Manoj Gupta 6992817a0e [Builtins] ARM: Fix assembling files in thumb mode.
Summary:
clang does not assemble files in thumb mode unless .thumb declaration
is present. Add .thumb/.arm decl to _FUNCTION macros to ensure that
files are assembled correctly.

Fixes PR 34715.

Reviewers: compnerd, peter.smith, srhines, weimingz, rengolin

Reviewed By: compnerd

Subscribers: aemerson, javed.absar, llvm-commits, kristof.beyls

Differential Revision: https://reviews.llvm.org/D38227

llvm-svn: 314285
2017-09-27 09:33:14 +00:00
Manoj Gupta 9c277321e3 [Builtins] ARM: Fix msr assembly instruction use for Thumb2.
Summary:
MSR instruction in Thumb2 does not support immediate operand.
Fix this by moving the condition for V7-M to Thumb2 since V7-M support
Thumb2 only. With this change, aeabi_cfcmp.s and aeabi_cdcmp.S files can
be assembled in Thumb2 mode. (This is split out from the review D38227).

Reviewers: compnerd, peter.smith, srhines, weimingz, rengolin, kristof.beyls

Reviewed By: compnerd

Subscribers: aemerson, javed.absar, llvm-commits

Differential Revision: https://reviews.llvm.org/D38268

llvm-svn: 314284
2017-09-27 09:29:57 +00:00
Manoj Gupta 347ff633f7 [Builtins] Use 4 byte alignment for __aeabi_memclr.
Summary:
Align __aeabi_memclr to 4 bytes. All other ARM functions are already aligned to
4-bytes in compiler-rt.
(Split off from review D38227)

Reviewers: compnerd, peter.smith, srhines, weimingz, rengolin, kristof.beyls

Reviewed By: compnerd

Subscribers: aemerson, javed.absar, llvm-commits

Differential Revision: https://reviews.llvm.org/D38271

llvm-svn: 314255
2017-09-26 22:37:08 +00:00
Kostya Kortchinsky b59abb2590 [scudo] Scudo thread specific data refactor, part 3
Summary:
Previous parts: D38139, D38183.

In this part of the refactor, we abstract the Linux vs Android TSD dissociation
in favor of a Exclusive vs Shared one, allowing for easier platform introduction
and configuration.

Most of this change consist of shuffling the files around to reflect the new
organization.

We introduce `scudo_platform.h` where platform specific definition lie. This
involves the TSD model and the platform specific allocator parameters. In an
upcoming CL, those will be configurable via defines, but we currently stick
with conservative defaults.

Reviewers: alekseyshl, dvyukov

Reviewed By: alekseyshl, dvyukov

Subscribers: srhines, llvm-commits, mgorny

Differential Revision: https://reviews.llvm.org/D38244

llvm-svn: 314224
2017-09-26 17:20:02 +00:00
Francis Ricci b4f6c0eca2 Invalidate symbolizer module list from dlopen/dlclose interceptors
Summary:
The module list should only be invalidated by dlopen and dlclose,
so the symbolizer should only re-generate it when we've hit one of those functions.

Reviewers: kubamracek, rnk, vitalybuka

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D37268

llvm-svn: 314219
2017-09-26 16:12:56 +00:00
Simon Dardis 5a14bc84e1 [mips][asan] Fix preinstalled_signal.cc test for mips
Linux for mips has a non-standard layout for the kernel sigaction struct.
Adjust the layout by the minimally amount to get the test to pass, as we
don't require the usage of the restorer function.

llvm-svn: 314200
2017-09-26 11:49:04 +00:00
Francis Ricci 92fa15d01c Add missing include to sanitizer procmaps
llvm-svn: 314162
2017-09-25 21:51:04 +00:00
Francis Ricci 7bde17f3ec Fix style in sanitizer_procmaps.h
llvm-svn: 314157
2017-09-25 21:26:34 +00:00
Francis Ricci 6e4dfccc31 [sanitizer_common] Don't provide sanitizer_procmaps API functions where not defined
Summary:
Platforms that don't implement procmaps (primarily fuchsia and windows) still expose
the procmaps API when including sanitizer_procmaps.h, despite not implementing the functions
provided by that header. Ensure that the API is only exposed on platforms that implement it.

Reviewers: vitalybuka, alekseyshl, kubamracek

Subscribers: llvm-commits, krytarowski

Differential Revision: https://reviews.llvm.org/D38187

llvm-svn: 314149
2017-09-25 20:48:51 +00:00
Kostya Kortchinsky 22396c2f47 [scudo] Scudo thread specific data refactor, part 2
Summary:
Following D38139, we now consolidate the TSD definition, merging the shared
TSD definition with the exclusive TSD definition. We introduce a boolean set
at initializaton denoting the need for the TSD to be unlocked or not. This
adds some unused members to the exclusive TSD, but increases consistency and
reduces the definitions fragmentation.

We remove the fallback mechanism from `scudo_allocator.cpp` and add a fallback
TSD in the non-shared version. Since the shared version doesn't require one,
this makes overall more sense.

There are a couple of additional cosmetic changes: removing the header guards
from the remaining `.inc` files, added error string to a `CHECK`.

Question to reviewers: I thought about friending `getTSDAndLock` in `ScudoTSD`
so that the `FallbackTSD` could `Mutex.Lock()` directly instead of `lock()`
which involved zeroing out the `Precedence`, which is unused otherwise. Is it
worth doing?

Reviewers: alekseyshl, dvyukov, kcc

Reviewed By: dvyukov

Subscribers: srhines, llvm-commits

Differential Revision: https://reviews.llvm.org/D38183

llvm-svn: 314110
2017-09-25 15:12:08 +00:00
Simon Dardis 46f5df1a20 [mips][compiler-rt] Disable sem_init_glibc.cc for MIPS64.
This test can't pass on MIPS64 due to the lack of versioned interceptors
for asan and company. The interceptors bind to the earlier version of
sem_init rather than the latest version. For MIPS64el this causes an
accidental pass while MIPS64 big endian fails due reading back a
different 32bit word to what sem_init wrote when the test is corrected
to use 64bit atomics.

llvm-svn: 314100
2017-09-25 13:25:34 +00:00
Vitaly Buka dcbae8631f [ubsan] Fix assert.cc test by compiling it as C++
llvm-svn: 314057
2017-09-23 05:09:45 +00:00
Vitaly Buka 5b81dfc76e [ubsan] Replace CommonSanitizerReportMutex with ScopedErrorReportLock
Reviewers: eugenis, alekseyshl

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D38194

llvm-svn: 314053
2017-09-23 02:47:21 +00:00
Vitaly Buka c3f431fba3 [ubsan] Support signal specific options in ubsan
Summary:
Part of https://github.com/google/sanitizers/issues/637

Standalone ubsan needs signal and sigaction handlers and interceptors.
Plugin mode should rely on parent tool.

Reviewers: eugenis, alekseyshl

Subscribers: kubamracek, llvm-commits, mgorny

Differential Revision: https://reviews.llvm.org/D37895

llvm-svn: 314052
2017-09-23 02:47:11 +00:00
Vitaly Buka 1b73bde182 [lsan] Add __lsan_default_options
For consistency with asan, msan, tsan and ubsan.

llvm-svn: 314048
2017-09-22 23:49:49 +00:00
Vitaly Buka 15d7e604d9 [lsan] Deadly signal handler for lsan
Summary: Part of https://github.com/google/sanitizers/issues/637

Reviewers: eugenis, alekseyshl

Subscribers: llvm-commits, dberris, kubamracek, krytarowski

Differential Revision: https://reviews.llvm.org/D37608

llvm-svn: 314041
2017-09-22 22:57:48 +00:00
Vitaly Buka a33e156f81 [sanitizer] Replace thread id with GetThreadSelf
This allows to avoid constructor parameter

llvm-svn: 314040
2017-09-22 22:36:21 +00:00
Vitaly Buka a18f2ebdea [sanitizer] Move ScopedErrorReportLock into libcdep version
llvm-svn: 314039
2017-09-22 22:36:11 +00:00
Francis Ricci 3aae099702 Fix fuchsia builds broken by r313999
llvm-svn: 314021
2017-09-22 19:22:08 +00:00
Vitaly Buka 1c06f9a127 [sanitizer] Replace AddressSanitizer with correct tool name
llvm-svn: 314015
2017-09-22 18:50:18 +00:00
Kuba Mracek 0e68358cbf Deflake the "xpc-race.mm" test.
llvm-svn: 314014
2017-09-22 18:49:56 +00:00
Vitaly Buka d8e18a4cf6 [sanitizer] Move report locking code from asan into common
llvm-svn: 314008
2017-09-22 18:32:05 +00:00
Vitaly Buka 03ac15fffb [asan] Fix unlocking order for CommonSanitizerReportMutex and reporting_thread_tid_
llvm-svn: 314007
2017-09-22 18:31:51 +00:00
Vitaly Buka 4a98c85008 [sanitizer] Move CommonSanitizerReportMutex from _print.cc to _common.cc
llvm-svn: 314006
2017-09-22 18:31:37 +00:00
Francis Ricci c007c42ee9 Fix windows buildbot broken by r313999
llvm-svn: 314001
2017-09-22 18:17:26 +00:00
Francis Ricci fbccb0a8d2 Removed platform-specific ifdefs from sanitizer_procmaps.h
Summary: Removed platform-specific ifdefs for linux, mac, freebsd and netbsd from sanitizer_procmaps.h

Patch by Yicheng Wang <yichengfb@fb.com>

Reviewers: kcc, kubamracek, alekseyshl, fjricci, vitalybuka

Reviewed By: fjricci, vitalybuka

Subscribers: vitalybuka, emaste, krytarowski, llvm-commits

Differential Revision: https://reviews.llvm.org/D38098

llvm-svn: 313999
2017-09-22 17:48:24 +00:00
Kostya Kortchinsky 392480952c [scudo] Scudo thread specific data refactor, part 1
Summary:
We are going through an overhaul of Scudo's TSD, to allow for new platforms
to be integrated more easily, and make the code more sound.

This first part is mostly renaming, preferring some shorter names, correcting
some comments. I removed `getPrng` and `getAllocatorCache` to directly access
the members, there was not really any benefit to them (and it was suggested by
Dmitry in D37590).

The only functional change is in `scudo_tls_android.cpp`: we enforce bounds to
the `NumberOfTSDs` and most of the logic in `getTSDAndLockSlow` is skipped if we
only have 1 TSD.

Reviewers: alekseyshl, dvyukov, kcc

Reviewed By: dvyukov

Subscribers: llvm-commits, srhines

Differential Revision: https://reviews.llvm.org/D38139

llvm-svn: 313987
2017-09-22 15:35:37 +00:00
Maxim Ostapenko d0d900d850 [asan/lsan] Trying to fix PPC64 and x380x buildbots after r313966
llvm-svn: 313974
2017-09-22 10:44:28 +00:00
Maxim Ostapenko c1f0b0ecab [asan/lsan] Trying to fix buildbots after r313966
llvm-svn: 313967
2017-09-22 08:23:16 +00:00
Maxim Ostapenko 9d016b773b [asan/lsan] Make LSan compliant with recovery mode when running on top of ASan
Don't overwrite exit code in LSan when running on top of ASan in recovery mode
to avoid breakage of users code due to found leaks.

Patch by Slava Barinov.

Differential Revision: https://reviews.llvm.org/D38026

llvm-svn: 313966
2017-09-22 07:11:43 +00:00
Akira Hatanaka 475a38fc5c [tsan] Annotate function parameters with attribute 'noescape'.
This commit annotates the block parameters of the following functions
declared in compiler-rt with 'noescape':

- dispatch_sync
- dispatch_barrier_sync
- dispatch_once
- dispatch_apply

This is needed to commit the patch that adds support for 'noescape' in
clang (see https://reviews.llvm.org/D32210) since these functions are
annotated with 'noescape' in the SDK header files.

Differential Revision: https://reviews.llvm.org/D32210

llvm-svn: 313929
2017-09-21 22:16:50 +00:00
Dean Michael Berris c14b5f210f [XRay][compiler-rt] Remove non-trivial globals from xray_log_interface.cc
Summary:
Remove dependency on std::unique_ptr<...> for the global representing
the installed XRay implementation.

Reviewers: dblaikie, kpw, pelikan

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D38121

llvm-svn: 313871
2017-09-21 10:16:56 +00:00
Vitaly Buka be88ae0eb0 [ubsan] Split ubsan_init_standalone
Summary:
On Linux we may need preinit_array in static lib and
ubsan_standalone_initializer in shared lib.

Reviewers: eugenis

Subscribers: kubamracek, llvm-commits, mgorny

Differential Revision: https://reviews.llvm.org/D38013

llvm-svn: 313851
2017-09-21 00:35:22 +00:00
Vitaly Buka 08706542e9 [asan] Remove trailing spaces
llvm-svn: 313845
2017-09-21 00:14:17 +00:00
Vitaly Buka d0d025ef62 [asan] Remove "COLLISION" workaround for datarace in asan
"nested bug in the same thread" is not expected in case like this and was caused
by https://github.com/google/sanitizers/issues/858

llvm-svn: 313844
2017-09-21 00:11:30 +00:00
Vitaly Buka d7f0361ab1 [asan] Remove "rm -f" in tests where it was needed only because of >>
llvm-svn: 313843
2017-09-21 00:06:08 +00:00
Vitaly Buka 3191a2ac1f [asan] Fix nested error detection
Summary: Fixes https://github.com/google/sanitizers/issues/858

Reviewers: eugenis, dvyukov

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D38019

llvm-svn: 313835
2017-09-20 23:27:38 +00:00
Vitaly Buka 8a65209d0b [asan] Try to fix windows test by fflush(stderr)
llvm-svn: 313728
2017-09-20 07:16:08 +00:00
Vitaly Buka cf096c2af3 [asan] Resolve FIXME by converting gtest into lit test
llvm-svn: 313727
2017-09-20 07:01:19 +00:00
Francis Ricci d327254b1b Set ANDROID when any android abi is used, not just androideabi
Reviewers: compnerd, beanz

Subscribers: srhines, mgorny, llvm-commits

Differential Revision: https://reviews.llvm.org/D38044

Change-Id: Idab521f187aba18977818d91503763e0e9d3aa0e
llvm-svn: 313652
2017-09-19 17:56:27 +00:00
Francis Ricci ba064be7af Ensure that armhf builtins library is created when using an hf abi
Reviewers: beanz, compnerd

Reviewed By: compnerd

Subscribers: aemerson, mgorny, kristof.beyls, llvm-commits

Differential Revision: https://reviews.llvm.org/D38045

llvm-svn: 313650
2017-09-19 17:54:11 +00:00
Simon Dardis 3efb88695f [mips][compiler-rt] UnXFAIL test.
lsan and asan were reporting leaks caused by a glibc configuration issue.

llvm-svn: 313645
2017-09-19 17:26:02 +00:00
Petr Hosek 5995e71a37 [sanitizer] Don't define common ReportDeadlySignal on Fuchsia
This causes a linker error because of duplicate symbol since
ReportDeadlySignal is defined both in sanitizer_common_libcdep and
sanitizer_fuchsia.

Differential Revision: https://reviews.llvm.org/D37952

llvm-svn: 313641
2017-09-19 17:00:22 +00:00
Vedant Kumar 515cffb8f6 [ubsan-minimal] Test exported symbol set against RTUBsan
Check that the symbol sets exported by the minimal runtime and the full
runtime match (making exceptions for special cases as needed).

This test uses some possibly non-standard nm options, and needs to
inspect the symbols in runtime dylibs. I haven't found a portable way to
do this, so it's limited to x86-64/Darwin for now.

llvm-svn: 313615
2017-09-19 06:46:36 +00:00
Vedant Kumar c539795bc3 [ubsan-minimal] Make the interface more compatible with RTUBSan
This eliminates a few inconsistencies between the symbol sets exported
by RTUBSan and RTUBSan_minimal:

  * Handlers for nonnull_return were missing from the minimal RT, and
    are now added in.

  * The minimal runtime exported recoverable handlers for
    builtin_unreachable and missing_return. These are not supposed to
    exist, and are now removed.

llvm-svn: 313614
2017-09-19 06:46:36 +00:00
Vedant Kumar 648752ab14 [profile] Update Linux-only tests after r313597
Addresses bot failure:
http://lab.llvm.org:8011/builders/clang-ppc64le-linux/builds/9803

llvm-svn: 313602
2017-09-19 00:15:18 +00:00
Vedant Kumar 4265d16d45 [profile] Update InstrProfData.inc
llvm-svn: 313599
2017-09-18 23:37:32 +00:00
Vitaly Buka 6863972b48 Revert "[ubsan] Split ubsan_init_standalone"
Breaks build.

This reverts commit r313583 and r313584.

llvm-svn: 313585
2017-09-18 22:47:36 +00:00
Vitaly Buka 3c1c34d5c5 [ubsan] Add file missing from r313583
llvm-svn: 313584
2017-09-18 22:43:28 +00:00
Vitaly Buka eb0c26168f [ubsan] Split ubsan_init_standalone
On Linux we may need preinit_array in static lib and
ubsan_standalone_initializer in shared lib.

llvm-svn: 313583
2017-09-18 22:38:18 +00:00
Vitaly Buka 59a1275be1 [ubsan] Fix conflict with previous declaration on Mac
llvm-svn: 313572
2017-09-18 21:35:49 +00:00
Kostya Kortchinsky 58c9a79bc2 [scudo] Additional modifications for Android tests support
Summary:
With the recent move of `android_commands` to `sanitizer_common`, some things
have to be updated with regard to Scudo on Android.

Notably:
- `config.android` is dealt with in the common code
- `config.compile_wrapper` can be prepended to allow for the use of the android
  commands
- `SCUDO_OPTIONS` must be passed with the environment when running a test
- `preinit.cpp` fails with some API levels, not sure why, I will have to dig
  into this later.

Note that `check-scudo` is not enabled yet in the bots. It's all local testing
for now until everything looks good.

Reviewers: alekseyshl, vitalybuka

Reviewed By: vitalybuka

Subscribers: srhines, kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D37990

llvm-svn: 313561
2017-09-18 20:31:57 +00:00
Vitaly Buka a4f62c62d3 [ubsan] Fix interface_symbols_windows test
Summary:
1. Update ubsan_interface.inc to make the test happy.
2. Switch interface_symbols_linux and interface_symbols_darwin to C++ to import __ubsan_handle_dynamic_type_cache_miss
3. Switch interface_symbols_windows to C++ for consistency.

Reviewers: rnk, zturner

Subscribers: llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D37986

llvm-svn: 313551
2017-09-18 18:46:19 +00:00
Vedant Kumar 4a10504e1f [cmake] Make it possible to build and test profile without sanitizers
This should fix an issue which arises when running check-compiler-rt on
the coverage bot:
http://green.lab.llvm.org/green/job/clang-stage2-coverage-R_build/1590/

The bot doesn't build the sanitizers, but the check-compiler-rt target
always expects the profile runtime to exist.

llvm-svn: 313549
2017-09-18 18:13:47 +00:00
Kostya Kortchinsky 8a0b40a8b3 [scudo] Android build support
Summary:
Mark Android as supported in the cmake configuration for Scudo.

Scudo is not added yet in the Android build bots, but code builds and tests
pass locally. It is for a later CL. I also checked that Scudo builds as part
of the Android toolchain.

A few modifications had to be made:
- Android defaults to `abort_on_error=1`, which doesn't work well with the
  current tests. So change the default way to pass `SCUDO_OPTIONS` to the tests
  to account for this, setting it to 0 by default;
- Disable the `valloc.cpp` & `random_shuffle.cpp` tests on Android;
- There is a bit of gymnatic to be done with the `SCUDO_TEST_TARGET_ARCH`
  string, due to android using the `-android` suffix, and `i686` instead of
  `i386`;
- Android doesn't need `-lrt`.

Reviewers: alekseyshl, eugenis

Reviewed By: alekseyshl

Subscribers: srhines, mgorny, llvm-commits

Differential Revision: https://reviews.llvm.org/D37907

llvm-svn: 313538
2017-09-18 15:40:53 +00:00
Vitaly Buka e66dc01bbf Revert "[ubsan] Add RTUbsan_standalone into UBSAN standalone shared"
Error: .preinit_array section is not allowed in DSO

This reverts commit r313519.

llvm-svn: 313520
2017-09-18 08:15:00 +00:00
Vitaly Buka d2b4022c0b [ubsan] Add RTUbsan_standalone into UBSAN standalone shared
Same as for Apple.
This also fixes flags related tests on Android as without this flags are not
initialized.

llvm-svn: 313519
2017-09-18 08:04:03 +00:00
Vitaly Buka 7119b57a9d [sanitizer] Move StartReportDeadlySignal into sanitizer_common_libcdep
llvm-svn: 313518
2017-09-18 07:36:32 +00:00
Vitaly Buka 86dd088f4d [asan] Remove ScopedDeadlySignal
This is used only to make fast = true in GetStackTraceWithPcBpAndContext
on SANITIZER_FREEBSD and SANITIZER_NETBSD and can be done explicitly.

llvm-svn: 313517
2017-09-18 06:56:57 +00:00
Dean Michael Berris 484fe0a9fb [XRay][compiler-rt] Handle tail-call exits in the XRay runtime
Summary:
This change starts differentiating tail exits from normal exits. We also
increase the version number of the "naive" log to version 2, which will
be the starting version where these records start appearing. In FDR mode
we treat the tail exits as normal exits, and are thus subject to the
same treatment with regard to record unwriting.

Updating the version number is important to signal older builds of the
llvm-xray tool that do not deal with the tail exit records must fail
early (and that users should only use the llvm-xray tool built after
the support for tail exits to get accurate handling of these records).

Depends on D37964.

Reviewers: kpw, pelikan

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D37965

llvm-svn: 313515
2017-09-18 06:18:03 +00:00
Vitaly Buka 9b7657fbc4 [sanitizer] Use SI_ instead of SANITIZER_ in sanitizer_platform_interceptors
llvm-svn: 313505
2017-09-18 00:12:12 +00:00
Kuba Mracek 8db966a097 Mark various failing tests with "UNSUPPORTED: ios".
llvm-svn: 313498
2017-09-17 20:00:43 +00:00
Ulrich Weigand 6a38d4c7c7 [compiler-rt] Fix build break after r313277 on s390x
Commit r313277 moved IsStackOverflow to inside the SignalContext
class, but didn't update a code block in #ifdef s390x accordingly.

llvm-svn: 313480
2017-09-17 09:38:55 +00:00
Vitaly Buka 9f1c0d4970 [sanitizer] Fix check for i386 Android in lit tests
llvm-svn: 313452
2017-09-16 07:56:06 +00:00
Vitaly Buka 6c196978eb [sanitizer] Move signal interceptors from asan to sanitizer_common
Summary: Part of https://github.com/google/sanitizers/issues/637

Reviewers: eugenis, alekseyshl

Subscribers: srhines, kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D37889

llvm-svn: 313449
2017-09-16 07:16:29 +00:00
Vitaly Buka 6979b85b79 [sanitizer] Disable sanitizer test which already fails on Android i386
llvm-svn: 313447
2017-09-16 07:08:23 +00:00
Vitaly Buka cd2d9a6b98 [sanitizer] Support check-asan on Android
This patch enabled asan tests from sanitizer_common.

llvm-svn: 313444
2017-09-16 05:14:05 +00:00
Vitaly Buka 0dc3ef3734 [sanitizer] Move android_commoands from asan into sanitizer_common
llvm-svn: 313443
2017-09-16 05:13:56 +00:00
Vitaly Buka 011aec2cf1 [builtins] Remove one more missed not-android
llvm-svn: 313441
2017-09-16 03:47:19 +00:00
Vitaly Buka 20d78186c2 [asan] Remove not-android
Replaced with !android

llvm-svn: 313440
2017-09-16 03:41:16 +00:00
Vitaly Buka 3d8c362f94 Revert "[ubsan] Update ubsan_interface.inc"
This brakes interface_symbols_linux.c test.

This reverts commit r313432.

llvm-svn: 313439
2017-09-16 03:26:12 +00:00
Vitaly Buka 581fbc61b2 [asan] Enable asan_and_llvm_coverage_test.cc on Android
Test just needs profile.

llvm-svn: 313438
2017-09-16 03:26:03 +00:00
Vitaly Buka 3ab63e34f6 [ubsan] Update ubsan_interface.inc
llvm-svn: 313432
2017-09-16 01:21:04 +00:00
Peter Collingbourne 823bb7eff6 Try to fix check-asan.
llvm-svn: 313423
2017-09-15 23:37:22 +00:00
Kostya Serebryany 80e9a6ccd1 [libFuzzer] add linux-specific test for gc-sections
llvm-svn: 313421
2017-09-15 23:07:18 +00:00
Kostya Serebryany ad66fefdca [libFuzzer] test fix
llvm-svn: 313411
2017-09-15 22:29:20 +00:00
Zachary Turner ce92db13ea Resubmit "[lit] Force site configs to run before source-tree configs"
This is a resubmission of r313270.  It broke standalone builds of
compiler-rt because we were not correctly generating the llvm-lit
script in the standalone build directory.

The fixes incorporated here attempt to find llvm/utils/llvm-lit
from the source tree returned by llvm-config.  If present, it
will generate llvm-lit into the output directory.  Regardless,
the user can specify -DLLVM_EXTERNAL_LIT to point to a specific
lit.py on their file system.  This supports the use case of
someone installing lit via a package manager.  If it cannot find
a source tree, and -DLLVM_EXTERNAL_LIT is either unspecified or
invalid, then we print a warning that tests will not be able
to run.

Differential Revision: https://reviews.llvm.org/D37756

llvm-svn: 313407
2017-09-15 22:10:46 +00:00
Kostya Serebryany bcd78491ef [libFuzzer] minor refactoring, NFC
llvm-svn: 313406
2017-09-15 22:10:36 +00:00
Kostya Serebryany 97b8b2daff [libFuzzer] reduce the size of the merge control file by not dumping redundant features into it
llvm-svn: 313403
2017-09-15 22:02:26 +00:00
Peter Collingbourne d13d61fdb4 ubsan: Unbreak ubsan_cxx runtime library on Windows.
This was originally broken by r258744 which introduced a weak reference
from ubsan to ubsan_cxx. This reference does not work directly on
Windows because COFF has no direct concept of weak symbols. The fix is
to use /alternatename to create a weak external reference to ubsan_cxx.

Also fix the definition (and the name, so that we drop cached values)
of the cmake flag that controls whether to build ubsan_cxx. Now the
user-controllable flag is always on, and we turn it off internally
depending on whether we support building it.

Differential Revision: https://reviews.llvm.org/D37882

llvm-svn: 313391
2017-09-15 20:24:12 +00:00
Peter Collingbourne a02c2eaf52 cfi: Enable ThinLTO tests on Windows.
We now avoid using absolute symbols on Windows (D37407 and D37408),
so this should work.

Fixes PR32770.

Differential Revision: https://reviews.llvm.org/D37883

llvm-svn: 313379
2017-09-15 18:55:35 +00:00
Peter Collingbourne 7d218f7532 ubsan: Stop building the DLL version of the runtime library on Windows.
As far as I know we never use it.

Differential Revision: https://reviews.llvm.org/D37884

llvm-svn: 313378
2017-09-15 18:54:37 +00:00
Petar Jovanovic 16807db3d4 [sanitizer][mips64] fix MIPS64 kernel_stat_to_stat()
This patch tackles with two issues:

Output stat st_[a|m|c]time fields were holding wrong values.
st_[a|m|c]time fields should have contained value of seconds and instead
these are filled with st_[a|m|c]time_nsec fields which hold nanoseconds.
Build fails for MIPS64 if SANITIZER_ANDROID. Recently <sys/stat.h> from
bionic introduced st_[a|m|c]time_nsec macros for compatibility with old NDKs
and those clashed with the field names of the <asm/stat.h> kernel_stat
structure.
To fix both issues and make sure sanitizer builds on all platforms, we must
un-define all compatibility macros and access the fields directly when
copying the 'time' fields.

Patch by Miodrag Dinic <miodrag.dinic@imgtec.com>

Differential Revision: https://reviews.llvm.org/D35671

llvm-svn: 313360
2017-09-15 15:18:51 +00:00
Vitaly Buka 5fbd91df56 [ubsan] Extract GetStackTraceWithPcBpAndContext similar to asan version
llvm-svn: 313350
2017-09-15 08:11:53 +00:00
Dmitry Vyukov b63a3ab5bf tsan: respect LDFLAGS when build Go test
Reported at:
https://bugs.llvm.org/show_bug.cgi?id=27597

Some platforms need additional LDFLAGS when building the test
(e.g. -no-pie). Respect LDFLAGS.

llvm-svn: 313347
2017-09-15 06:51:37 +00:00
Vitaly Buka 3e45e10819 [sanitizer] Simplify checks in allow_user_segv.cc
llvm-svn: 313342
2017-09-15 04:48:02 +00:00
Vitaly Buka 461e8c1de8 [sanitizer] Use __sanitizer:: in CHECK_IMPL on both sides of assignment
llvm-svn: 313338
2017-09-15 04:05:15 +00:00
Zachary Turner 83dcb68468 Revert "[lit] Force site configs to run before source-tree configs"
This patch is still breaking several multi-stage compiler-rt bots.
I already know what the fix is, but I want to get the bots green
for now and then try re-applying in the morning.

llvm-svn: 313335
2017-09-15 02:56:40 +00:00
Vitaly Buka f716931bb2 [lsan] Disable clang-format on few RUN: statements
llvm-svn: 313321
2017-09-15 00:02:30 +00:00
Vitaly Buka 21ddc6219b [sanitizer] Move stack overflow and signal reporting from Asan into common.
Summary: Part of https://github.com/google/sanitizers/issues/637

Reviewers: eugenis, alekseyshl

Subscribers: kubamracek

Differential Revision: https://reviews.llvm.org/D37844

llvm-svn: 313310
2017-09-14 22:44:03 +00:00
Vitaly Buka 846a217bfc [asan] Remove ErrorStackOverflow
Summary:
The only difference from ErrorDeadlySignal is reporting code and it lives in
sanitizer common.

Part of https://github.com/google/sanitizers/issues/637

Reviewers: eugenis, alekseyshl, filcab

Subscribers: llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D37868

llvm-svn: 313309
2017-09-14 22:43:53 +00:00
Eric Fiselier 4aa0ecf86b [ASAN] Revert r313303 - Add macro denoting availability of new `__asan_handle_no_return()` function.
It was pointed out that compiler-rt has always defined the symbol, but only
recently added it to the public headers. Meaning that libc++abi can re-declare
it instead of needing this macro.

llvm-svn: 313306
2017-09-14 22:31:34 +00:00
Eric Fiselier 6892062f7c [ASAN] Add macro denoting availability of new `__asan_handle_no_return()` function.
Summary:
Libc++abi attempts to use the newly added `__asan_handle_no_return()` when built under ASAN. Unfortunately older versions of compiler-rt do not provide this symbol, and so libc++abi needs a way to detect if `asan_interface.h` actually provides the function.

This patch adds the macro `SANITIZER_ASAN_INTERFACE_HAS_HANDLE_NO_RETURN` which can be used to detect the availability of the new function.

Reviewers: phosek, kcc, vitalybuka, alekseyshl

Reviewed By: phosek

Subscribers: mclow.lists, cfe-commits

Differential Revision: https://reviews.llvm.org/D37871

llvm-svn: 313303
2017-09-14 22:19:10 +00:00
Kostya Kortchinsky 26e689f0c5 [scudo] Fix bad request handling when allocator has not been initialized
Summary:
In a few functions (`scudoMemalign` and the like), we would call
`ScudoAllocator::FailureHandler::OnBadRequest` if the parameters didn't check
out. The issue is that if the allocator had not been initialized (eg: if this
is the first heap related function called), we would use variables like
`allocator_may_return_null` and `exitcode` that still had their default value
(as opposed to the one set by the user or the initialization path).

To solve this, we introduce `handleBadRequest` that will call `initThreadMaybe`,
allowing the options to be correctly initialized.

Unfortunately, the tests were passing because `exitcode` was still 0, so the
results looked like success. Change those tests to do what they were supposed
to.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D37853

llvm-svn: 313294
2017-09-14 20:34:32 +00:00
Simon Dardis b2388c52e8 Revert "[mips] Fix sem_init_glibc test for MIPS."
The commit did not fix the failing test and instead exposed an inconsistency
between lsan and (t|m|a)san. I'm reverting the patch as it causes more failures
and the original patch had a '||' instead of '&&', which meant that an N32 build
of test would have be incorrect w.r.t. __HAVE_64B_ATOMICS for glibc.

This reverts commit r313248.

llvm-svn: 313291
2017-09-14 19:58:04 +00:00
Vitaly Buka 2cda290ec8 [asan] Disable two dynamic tests on armhf
This is not an regression. Tests are old and we just recently started to
run them on bots with dynamic runtime.

llvm-svn: 313283
2017-09-14 18:19:28 +00:00
Zachary Turner a0e55b6403 [lit] Force site configs to be run before source-tree configs
This patch simplifies LLVM's lit infrastructure by enforcing an ordering
that a site config is always run before a source-tree config.

A significant amount of the complexity from lit config files arises from
the fact that inside of a source-tree config file, we don't yet know if
the site config has been run.  However it is *always* required to run
a site config first, because it passes various variables down through
CMake that the main config depends on.  As a result, every config
file has to do a bunch of magic to try to reverse-engineer the location
of the site config file if they detect (heuristically) that the site
config file has not yet been run.

This patch solves the problem by emitting a mapping from source tree
config file to binary tree site config file in llvm-lit.py. Then, during
discovery when we find a config file, we check to see if we have a
target mapping for it, and if so we use that instead.

This mechanism is generic enough that it does not affect external users
of lit. They will just not have a config mapping defined, and everything
will work as normal.

On the other hand, for us it allows us to make many simplifications:

* We are guaranteed that a site config will be executed first
* Inside of a main config, we no longer have to assume that attributes
  might not be present and use getattr everywhere.
* We no longer have to pass parameters such as --param llvm_site_config=<path>
  on the command line.
* It is future-proof, meaning you don't have to edit llvm-lit.in to add
  support for new projects.
* All of the duplicated logic of trying various fallback mechanisms of
  finding a site config from the main config are now gone.

One potentially noteworthy thing that was required to implement this
change is that whereas the ninja check targets previously used the first
method to spawn lit, they now use the second. In particular, you can no
longer run lit.py against the source tree while specifying the various
`foo_site_config=<path>` parameters.  Instead, you need to run
llvm-lit.py.

Differential Revision: https://reviews.llvm.org/D37756

llvm-svn: 313270
2017-09-14 16:47:58 +00:00
Simon Dardis c720ee5914 [mips] Fix sem_init_glibc test for MIPS.
glibc changed the implementation of semaphores for glibc 2.21 requiring
some target specific changes for this compiler-rt test. Modify the test
to cope with MIPS64 and do some future/correctness work by tying the
define for MIPS64 to exactly the define of __HAVE_64B_ATOMICS in glibc.

Contributions from Nitesh Jain.

Reviewers: eugenis

Differential Revision: https://reviews.llvm.org/D37829

llvm-svn: 313248
2017-09-14 10:36:04 +00:00
Vitaly Buka 561f5408c9 [sanitizer] Mark allow_user_segv as XFAIL instead of UNSUPPORTED
llvm-svn: 313241
2017-09-14 08:15:13 +00:00
Vitaly Buka 62f02d5247 [sanitizer] Add empty Fuchsia and Win versions of StartReportDeadlySignal
llvm-svn: 313240
2017-09-14 08:14:56 +00:00
Vitaly Buka 2b077ecb91 [lsan] Extract GetStackTraceWithPcBpAndContext similar to asan version
llvm-svn: 313239
2017-09-14 08:14:38 +00:00
Vitaly Buka 9a4c73e20c [sanitizer] Move IsStackOverflow into SignalContext
llvm-svn: 313227
2017-09-14 03:23:02 +00:00
Vitaly Buka 5d53e050ca [sanitizer] Add BufferedStackTrace::Reset()
llvm-svn: 313226
2017-09-14 03:06:35 +00:00
Vitaly Buka 9c1fa35e5f [asan] Add const into ScarinessScoreBase::Print
llvm-svn: 313225
2017-09-14 03:06:20 +00:00
Vitaly Buka cd851f8085 [compiler-rt] Fix Windows build
llvm-svn: 313224
2017-09-14 02:58:18 +00:00
Vitaly Buka 73c101613d [compiler-rt] Cleanup SignalContext initialization
Reviewers: eugenis, alekseyshl

Subscribers: kubamracek, dberris

Differential Revision: https://reviews.llvm.org/D37827

llvm-svn: 313223
2017-09-14 02:48:41 +00:00
Vedant Kumar afe2bdd773 [ubsan-minimal] Filter targets to test by host arch on Darwin
This reverts r313189, and adds a use of darwin_filter_host_archs() for
ubsan-minimal.

llvm-svn: 313206
2017-09-13 21:57:47 +00:00
Zachary Turner 1929ffd452 Revert "Determine up front which projects are enabled."
This was intended to be a generic CMake solution to a problem
shared across several projects.  It turns out it doesn't interact
very well certain CMake configurations, and furthermore the
"problem" is actually not a problem, as the problematic code
is never executed to begin with.  So this really isn't solving
anything.

llvm-svn: 313191
2017-09-13 20:49:25 +00:00
Vedant Kumar cb835d3fdb [ubsan-minimal] Temporarily disable x86_64h testing on Darwin
We're seeing strange issues on the public GreenDragon Darwin bots which
we don't understand. x86_64h tests are still being run on pre-Haswell
bots despite the added checks in test/ubsan_minimal/lit.common.cfg,
which were verified on our internal bots.

I'm unable to ssh into the affected public bot, so for now am trying a
more aggressive check which disables all x86_64h testing for
ubsan-minimal on Darwin.

rdar://problem/34409349

llvm-svn: 313189
2017-09-13 20:46:26 +00:00
Vitaly Buka 72e8b55bab [asan] Fix Windows build
llvm-svn: 313177
2017-09-13 19:39:06 +00:00
Vitaly Buka dbde6f337c [compiler-rt] Use SignalContext in ErrorStackOverflow and ErrorDeadlySignal
Summary: Part of https://github.com/google/sanitizers/issues/637

Reviewers: eugenis, alekseyshl, filcab

Subscribers: kubamracek, llvm-commits, dberris

Differential Revision: https://reviews.llvm.org/D37793

llvm-svn: 313168
2017-09-13 18:30:16 +00:00
Vitaly Buka b215e90fb0 [compiler-rt] Add siginfo into SignalContext
Summary:
Information stored there is often been passed along with SignalContext.

Part of https://github.com/google/sanitizers/issues/637

Reviewers: eugenis, alekseyshl

Subscribers: kubamracek, llvm-commits, dberris

Differential Revision: https://reviews.llvm.org/D37792

llvm-svn: 313167
2017-09-13 18:30:06 +00:00
Vitaly Buka c40f5ceca0 [compiler-rt] Compile signal specific functions only for !SANITIZER_GO
llvm-svn: 313130
2017-09-13 08:10:16 +00:00
Vitaly Buka 6570c1d8b8 [compiler-rt] Add test for not fully implemented dump_registers
Reviewers: eugenis, alekseyshl

Subscribers: kubamracek, dberris, llvm-commits

Differential Revision: https://reviews.llvm.org/D37765

llvm-svn: 313120
2017-09-13 06:33:43 +00:00
Vitaly Buka 20562abc1c Fix line breaks.
llvm-svn: 313118
2017-09-13 06:25:09 +00:00
Vitaly Buka ecc44ecdbc [compiler-rt] Move dump_instruction_bytes and dump_registers into sanitizer_common
Summary: Part of https://github.com/google/sanitizers/issues/637

Reviewers: eugenis, alekseyshl

Subscribers: kubamracek, llvm-commits, dberris

Differential Revision: https://reviews.llvm.org/D37766

llvm-svn: 313117
2017-09-13 06:24:59 +00:00
Vitaly Buka d9d2a89e50 [compiler-rt] Move *Sanitizer:DEADLYSIGNAL printing into common part
Summary: Part of https://github.com/google/sanitizers/issues/637

Reviewers: eugenis, alekseyshl

Subscribers: kubamracek, llvm-commits, dberris

Differential Revision: https://reviews.llvm.org/D37764

llvm-svn: 313115
2017-09-13 04:46:37 +00:00
Petr Hosek d8328f13ac [Fuchsia] Magenta -> Zircon
Fuchsia's lowest API layer has been renamed from Magenta to Zircon.

Patch by Roland McGrath

Differential Revision: https://reviews.llvm.org/D37770

llvm-svn: 313106
2017-09-13 01:18:15 +00:00
Vitaly Buka 97ec1451a8 [compiler-rt] Move IsStackOverflow into sanitizer_posix_libcdep.cc to the rest
of instrumentation code.

llvm-svn: 313100
2017-09-13 00:24:44 +00:00
Vedant Kumar 5fb5094a4a [ubsan] Enable -fsanitize=function test on Darwin
Differential Revision: https://reviews.llvm.org/D37598

llvm-svn: 313097
2017-09-13 00:04:35 +00:00
Zachary Turner c7fc93946c Determine up front which projects are enabled.
Some projects need to add conditional dependencies on other projects.
compiler-rt is already doing this, and I attempted to add this to
debuginfo-tests when I ran into the ordering problem, that you can't
conditionally add a dependency unless that dependency's CMakeLists.txt
has already been run (which would allow you to say if (TARGET foo).

The solution to this seems to be to determine very early on the entire
set of projects which is enabled. This is complicated by the fact that
there are multiple ways to enable projects, and different tree layouts
(e.g. mono-repo, out of -tree, external, etc). This patch attempts to
centralize all of this into one place, and then updates compiler-rt to
demonstrate as a proof of concept how this can simplify code.

Differential Revision: https://reviews.llvm.org/D37637

llvm-svn: 313091
2017-09-12 23:32:34 +00:00
Kostya Serebryany 93679be037 [libFuzzer] factor out some code into GetSizedFilesFromDir; NFC
llvm-svn: 313081
2017-09-12 21:58:07 +00:00
Vitaly Buka 2890105eab [tsan] Disable user_malloc test which fails glibc 2.24
llvm-svn: 313069
2017-09-12 20:03:39 +00:00
Vedant Kumar 57e8a2e504 [ubsan-minimal] Fix the x86_64h config check
Checking if config.target_arch is x86_64h doesn't work (the 'h' suffix
is dropped here, and I didn't account for that). Instead, check to see
if '-arch x86_64h' is in the cflags.

Tested on a pre-Haswell bot.

rdar://problem/34378605

llvm-svn: 313053
2017-09-12 17:32:25 +00:00
Max Moroz 1a33da2401 [libFuzzer] Fix lit files to make running tests more straightforward on Mac OS.
Summary:
Current implementation does not work if CMAKE_OSX_SYSROOT is not specified.

It silently generates invalid command with the following flags:

`-std=c++11 -lc++ -gline-tables-only -isysroot  -fsanitize=address,fuzzer`

and then fails with the following error:

```
warning: no such sysroot directory: '-fsanitize=address,fuzzer' [-Wmissing-sysroot]"
<...>/RepeatedBytesTest.cpp:5:10: fatal error: 'assert.h' file not found
#include <assert.h>
         ^~~~~~~~~~
1 error generated.
```

However, if you have Command Line Tools installed, you have '/usr/include' dir.
In that case, it is not necessary to specify isysroot path.

Also, with the patch, in case of '/usr/include' does not exist, the '-sysroot'
path would be resolved automatically in compiler-rt/cmake/base-config-ix.cmake.

For more context, see the comment at `compiler-rt/cmake/base-config-ix.cmake#L76`

Reviewers: kcc, george.karpenkov

Reviewed By: kcc, george.karpenkov

Differential Revision: https://reviews.llvm.org/D37721

llvm-svn: 313033
2017-09-12 15:02:10 +00:00
Max Moroz 3f26dac416 [libfuzzer] Compare TotalNumberOfRuns with MaxNumberOfRuns when testing a memory leak.
Summary:
Fuzzer::TryDetectingAMemoryLeak may call ExecuteCallback which would
increment TotalNumberOfRuns, but it doesn't respect Options.MaxNumberOfRuns
value specified by a user.

Context: https://github.com/google/oss-fuzz/issues/822#issuecomment-328153970

Reviewers: kcc

Reviewed By: kcc

Differential Revision: https://reviews.llvm.org/D37632

llvm-svn: 312993
2017-09-12 02:01:54 +00:00
Dean Michael Berris d56b90fb4c [XRay][compiler-rt] Use a single global volatile recursion guard for FDR handlers
Summary:
Before this change, the recursion guard for the flight data recorder
(FDR) mode handlers were independent. This change makes it so that when
a handler is already in the process of running and somehow the same or
another handler starts running -- say in a signal handler, while the
XRay handler is executing -- then we can use the same thread-local
recursion guard to stop the second handler from running.

Reviewers: kpw, eizan

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D37612

llvm-svn: 312992
2017-09-12 01:37:59 +00:00
Vitaly Buka f6fc68c7d1 [compiler-rt] Move IsStackOverflow from asan into sanitizer_common
Summary: Part of https://github.com/google/sanitizers/issues/637

Reviewers: eugenis, alekseyshl

Subscribers: kubamracek, dberris, llvm-commits

Differential Revision: https://reviews.llvm.org/D37536

llvm-svn: 312987
2017-09-12 00:44:23 +00:00
Vitaly Buka 7af8e4b146 Revert "[compiler-rt] Move IsStackOverflow from asan into sanitizer_common"
Windows is broken.

This reverts commit r312951

llvm-svn: 312984
2017-09-12 00:14:33 +00:00
Vedant Kumar 661e2422d7 [ubsan-minimal] Disable x86_64h tests when not on x86_64h
llvm-svn: 312982
2017-09-12 00:01:13 +00:00
Evgeniy Stepanov 93f2edc215 Runtime detection of android_set_abort_message.
Summary:
Use runtime detection (with a weak-undef symbol) of
android_set_abort_message availability. Android NDK provides a single
version of the ASan runtime library to be used for any target API
level, which makes compile-time feature detection impossible (the
library itself is built at API level 9).

Reviewers: vitalybuka

Subscribers: srhines, llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D37716

llvm-svn: 312973
2017-09-11 23:27:58 +00:00
Vedant Kumar 7669679e82 [ubsan-minimal] Enable on Darwin
Testing: check-ubsan-minimal

Differential Revision: https://reviews.llvm.org/D37646

llvm-svn: 312959
2017-09-11 21:37:53 +00:00
Vitaly Buka 54cc3603de [ubsan] Make ubsan version of __sanitizer_print_stack_trace consistent with other sanitizers
Summary: Other sanitizers include __sanitizer_print_stack_trace into stack trace.

Reviewers: eugenis, alekseyshl

Subscribers: llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D37657

llvm-svn: 312954
2017-09-11 21:25:22 +00:00
Vitaly Buka bdcc82d7ed [ubsan] Save binary name before parsing options
Summary: To parser "include" we may need to do binary name substitution.

Reviewers: eugenis, alekseyshl

Subscribers: llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D37658

llvm-svn: 312953
2017-09-11 21:13:06 +00:00
Vitaly Buka 4084a583d9 Revert "[ubsan] Save binary name before parsing options"
Patch was corrupted by rebase.

This reverts commit r312933

llvm-svn: 312952
2017-09-11 21:12:43 +00:00
Vitaly Buka ecba46d0e6 [compiler-rt] Move IsStackOverflow from asan into sanitizer_common
Summary: Part of https://github.com/google/sanitizers/issues/637

Reviewers: eugenis, alekseyshl

Subscribers: kubamracek, dberris, llvm-commits

Differential Revision: https://reviews.llvm.org/D37536

llvm-svn: 312951
2017-09-11 21:00:24 +00:00
Vitaly Buka 36266b6b0d [compiler-rt] Cleanup decorators
Summary:
Removed redundant End*() methods which defined same way.
Removed redundant Warning() methods.

Reviewers: eugenis

Subscribers: kubamracek, llvm-commits, dberris

Differential Revision: https://reviews.llvm.org/D37549

llvm-svn: 312950
2017-09-11 20:55:49 +00:00
Kostya Kortchinsky 040c211bc4 [scudo] Fix improper TSD init after TLS destructors are called
Summary:
Some of glibc's own thread local data is destroyed after a user's thread local
destructors are called, via __libc_thread_freeres. This might involve calling
free, as is the case for strerror_thread_freeres.
If there is no prior heap operation in the thread, this free would end up
initializing some thread specific data that would never be destroyed properly
(as user's pthread destructors have already been called), while still being
deallocated when the TLS goes away. As a result, a program could SEGV, usually
in __sanitizer::AllocatorGlobalStats::Unregister, where one of the doubly linked
list links would refer to a now unmapped memory area.

To prevent this from happening, we will not do a full initialization from the
deallocation path. This means that the fallback cache & quarantine will be used
if no other heap operation has been called, and we effectively prevent the TSD
being initialized and never destroyed. The TSD will be fully initialized for all
other paths.

In the event of a thread doing only frees and nothing else, a TSD would never
be initialized for that thread, but this situation is unlikely and we can live
with that.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D37697

llvm-svn: 312939
2017-09-11 19:59:40 +00:00
Vitaly Buka 87d1f9ce93 [asan] Include asan-dynamic into check-all
Summary: It's adds just 1k to about 45k tests.

Reviewers: eugenis, alekseyshl

Subscribers: kubamracek, mgorny, llvm-commits

Differential Revision: https://reviews.llvm.org/D37666

llvm-svn: 312937
2017-09-11 19:41:17 +00:00
Vitaly Buka 57ff314368 [ubsan] Save binary name before parsing options
Summary: To parser "include" we may need to do binary name substitution.

Reviewers: eugenis, alekseyshl

Subscribers: llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D37658

llvm-svn: 312933
2017-09-11 18:32:51 +00:00
Vitaly Buka f8e944121e [asan] Fix tests broken by r312858
llvm-svn: 312872
2017-09-09 20:07:45 +00:00
Vitaly Buka fd5960a653 [ubsan] Enable UBsan in sanitizer_common tests
Summary:
Failing tests just marked as UNSUPPORTED or XFAIL.
Some of them can be easily supported, but I'll do this in separate patches.

Reviewers: eugenis, alekseyshl

Subscribers: srhines, kubamracek, llvm-commits, mgorny

Differential Revision: https://reviews.llvm.org/D37630

llvm-svn: 312860
2017-09-09 06:10:58 +00:00
Vitaly Buka b4f2d13dc6 [compiler-rt] Move allow_user_segv.cc into sanitizer_common
Summary: Part of https://github.com/google/sanitizers/issues/637

Reviewers: eugenis

Subscribers: kubamracek, dberris, llvm-commits

Differential Revision: https://reviews.llvm.org/D37537

llvm-svn: 312859
2017-09-09 06:04:23 +00:00
Vitaly Buka e7e7194a73 [asan] Use more generic string in error message
Summary: Part of https://github.com/google/sanitizers/issues/637

Reviewers: eugenis, alekseyshl

Subscribers: llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D37609

llvm-svn: 312858
2017-09-09 05:58:52 +00:00
Kostya Serebryany 25d6b95679 [libFuzzer] remove a couple of reduntant includes
llvm-svn: 312848
2017-09-09 00:56:34 +00:00
Francis Ricci f36d34a4d4 Revert "Prevent DCE on __lsan_is_turned_off and re-enable test case"
This doesn't fix the failing test. Leave in the comment and the
attribute, since the used attribute is still required.

This partially reverts commit r312824

llvm-svn: 312827
2017-09-08 21:09:43 +00:00
Francis Ricci 4a327e1c5c Prevent DCE on __lsan_is_turned_off and re-enable test case
Summary:
-dead_strip in ld64 strips weak interface symbols, which I believe
is most likely the cause of this test failure. Re-enable after marking the interface
function as used.

Reviewers: alekseyshl, kubamracek, kcc

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D37635

llvm-svn: 312824
2017-09-08 19:43:53 +00:00
Kostya Serebryany 754e584076 [libFuzzer] simplify CustomCrossOverTest even more
llvm-svn: 312697
2017-09-07 05:33:05 +00:00
Kostya Serebryany 57c03648e1 [libFuzzer] simplify CustomCrossOverTest a bit more
llvm-svn: 312695
2017-09-07 05:23:23 +00:00
Kostya Serebryany d0386fac26 [libFuzzer] simplify and re-enable CustomCrossOverTest
llvm-svn: 312689
2017-09-07 02:04:06 +00:00
Vedant Kumar b6d2fe5c88 [cmake] Work around more -Wunused-driver-argument warnings
add_compiler_rt_object_libraries should strip out the -msse3 option on
non-macOS Apple platforms.

llvm-svn: 312688
2017-09-07 01:36:47 +00:00
Evgeniy Stepanov 7d1ab555ae [asan] Add a note to shadow memory setup error.
Point to https://github.com/google/sanitizers/issues/856 as a possible cause of the failed mapping.

llvm-svn: 312687
2017-09-07 01:35:59 +00:00
Kostya Kortchinsky 6bc7b26d18 [scudo] getauxval alternative for Android
Summary:
`getauxval` was introduced with API level 18. In order to get things to work
at lower API levels (for the toolchain itself which is built at 14 for 32-bit),
we introduce an alternative implementation reading directly from
`/proc/self/auxv`.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: srhines, llvm-commits

Differential Revision: https://reviews.llvm.org/D37488

llvm-svn: 312653
2017-09-06 17:51:32 +00:00
Simon Dardis 10d0ae85dd [compiler-rt][xray][mips] Mark some tests as unsupported.
Thesee tests require the integrated assembler which is still in
development / testing for MIPS64. GAS doesn't understand the
section directives produced by XRay, so marking the relevant
tests as unsupported.

llvm-svn: 312628
2017-09-06 10:17:29 +00:00
Evgeniy Stepanov 9566d28997 [msan] Remove a stale fixme (NFC).
It was fixed in 312576.

llvm-svn: 312597
2017-09-06 00:28:52 +00:00
Petr Hosek 4f4bdc3c20 [sanitizer_common][Fuchsia] Update Fuchsia sanitizer markup
Include URLs to the markup format specification in code comments.
Use sanitizer markup in the sancov message about a dump just produced.

Patch by Roland McGrath

Differential Revision: https://reviews.llvm.org/D37273

llvm-svn: 312596
2017-09-06 00:00:46 +00:00
Kostya Serebryany 79cdf36a2c [libFuzzer] remporary disable an unstable test
llvm-svn: 312593
2017-09-05 23:45:54 +00:00
Evgeniy Stepanov 29c7487167 Remove ld.config.txt for Android O.
ld.config.txt defines linker namespaces in a way that is incompatible
with ASan. Remove the file when installing ASan on an Android O
(8.0.x) device.

Patch by Jiyong Park.

llvm-svn: 312581
2017-09-05 21:51:20 +00:00
Evgeniy Stepanov 8b80b328d1 [msan] Check sigset_t and sigaction arguments.
Summary:
Check sigset_t arguments in ppoll, sig*wait*, sigprocmask
interceptors, and the entire "struct sigaction" in sigaction. This
can be done because sigemptyset/sigfullset are intercepted and
signal masks should be correctly marked as initialized.

Reviewers: vitalybuka

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D37367

llvm-svn: 312576
2017-09-05 21:08:56 +00:00
Evgeniy Stepanov 00dedc208f (NFC) Fix the use of do{}while(0) in a macro.
llvm-svn: 312396
2017-09-02 00:09:57 +00:00
David Blaikie 387d0964ba Fix constant-logical-operand warning.
llvm-svn: 312394
2017-09-01 23:57:34 +00:00
Vedant Kumar f5addb18f2 [cmake] Remove accidentally committed warning messages
llvm-svn: 312393
2017-09-01 23:49:22 +00:00
Vedant Kumar 6b1de0e673 [cmake] Work around -Wunused-driver-argument warnings
Fix the Darwin logic so that -msse3 is only used on macOS, and
-fomit-frame-pointer is not used on armv7/armv7k/armv7s.

llvm-svn: 312390
2017-09-01 23:23:59 +00:00
Kostya Serebryany 07490f8415 [libFuzzer] use more iterations for a test
llvm-svn: 312356
2017-09-01 19:45:08 +00:00
George Karpenkov 2928375cd2 [libFuzzer] Enable 8-bit counters test on macOS
llvm-svn: 312339
2017-09-01 17:13:26 +00:00
Evgeniy Stepanov adfa21087f Revert "[ubsan] Make check-ubsan depend on check-ubsan-minimal."
Breaks buildbot with
CMake Error at projects/compiler-rt/test/CMakeLists.txt:76 (add_dependencies):
  The dependency target "check-ubsan-minimal" of target "check-ubsan" does
  not exist.

llvm-svn: 312295
2017-08-31 23:34:01 +00:00
Evgeniy Stepanov a865658a74 [ubsan] Make check-ubsan depend on check-ubsan-minimal.
Summary: This way we don't need to add check-ubsan-minimal steps to all the bots.

Reviewers: vitalybuka

Subscribers: mgorny, llvm-commits

Differential Revision: https://reviews.llvm.org/D37350

llvm-svn: 312291
2017-08-31 22:26:34 +00:00
Kostya Serebryany 082e9a7528 [libFuzzer] tolerate missing files when loading the seed corpus
llvm-svn: 312269
2017-08-31 19:17:15 +00:00
Alex Lorenz 341317fda4 Revert r312240
The buildbots have shown that -Wstrict-prototypes behaves differently in GCC
and Clang so we should keep it disabled until Clang follows GCC's behaviour

llvm-svn: 312246
2017-08-31 15:51:23 +00:00
Alex Lorenz fbac1ae062 Build LLVM with -Wstrict-prototypes enabled
Clang 5 supports -Wstrict-prototypes. We should use it to catch any C
declarations that declare a non-prototype function.

rdar://33705313

Differential Revision: https://reviews.llvm.org/D36669

llvm-svn: 312240
2017-08-31 13:23:24 +00:00
Martin Storsjo 376cce2ec8 [cmake] Fix the list of arm32 architectures
This was accidentally changed in SVN r311924, which was only
supposed to change the behaviour for x86.

llvm-svn: 312230
2017-08-31 10:01:36 +00:00
Evgeniy Stepanov 84614f851c [ubsan] Give ubsan-minimal lit test suite a name.
Otherwise llvm-lit -v prints this:
PASS: <unnamed> :: TestCases/recover-dedup-limit.cpp (1 of 3)
PASS: <unnamed> :: TestCases/recover-dedup.cpp (2 of 3)
PASS: <unnamed> :: TestCases/uadd-overflow.cpp (3 of 3)

llvm-svn: 312203
2017-08-31 00:54:10 +00:00
Dean Michael Berris 1704f6289a [XRay][compiler-rt] Enable the XRay compiler-rt unit tests.
Summary:
Before this change we seemed to not be running the unit tests, and therefore we
set out to run them. In the process of making this happen we found a divergence
between the implementation and the tests.

This includes changes to both the CMake files as well as the implementation and
headers of the XRay runtime. We've also updated documentation on the changed
functions.

Reviewers: kpw, eizan

Subscribers: mgorny, llvm-commits

Differential Revision: https://reviews.llvm.org/D37290

llvm-svn: 312202
2017-08-31 00:50:12 +00:00
Kamil Rytarowski d92f3982ce Finalize ASAN/NetBSD
Summary:
This revision contains various cleanups.

Sponsored by <The NetBSD Foundation>

Reviewers: kcc, vitalybuka, joerg, eugenis

Reviewed By: kcc

Subscribers: emaste, srhines, llvm-commits, kubamracek, mgorny, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D37244

llvm-svn: 312188
2017-08-30 23:02:36 +00:00
Matt Morehouse 034126e507 [SanitizeCoverage] Enable stack-depth coverage for -fsanitize=fuzzer
Summary:
- Don't sanitize __sancov_lowest_stack.
- Don't instrument leaf functions.
- Add CoverageStackDepth to Fuzzer and FuzzerNoLink.
- Only enable on Linux.

Reviewers: vitalybuka, kcc, george.karpenkov

Reviewed By: kcc

Subscribers: kubamracek, cfe-commits, llvm-commits, hiraditya

Differential Revision: https://reviews.llvm.org/D37156

llvm-svn: 312185
2017-08-30 22:49:31 +00:00
Kamil Rytarowski 06f2f80791 Add NetBSD support in lsan_interceptors.cc
Summary:
NetBSD is a modern POSIX-like UNIX-like Operating System derived from 4.4BSD/386BSD.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, vitalybuka, kcc, dvyukov

Reviewed By: kcc

Subscribers: llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D37307

llvm-svn: 312184
2017-08-30 22:47:05 +00:00
Kamil Rytarowski 8da2722a9f Add preliminary NetBSD support in libfuzzer
Summary:
This code already works and passes some number of tests.

There is need to finish remaining sanitizers to get better coverage.

Many tests fail due to overly long file names of executables (>31).
This is a current shortcoming of the NetBSD 8(beta) kernel, as
certain functions can fail (like retrieving file name of executable).

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, kcc, vitalybuka, george.karpenkov

Reviewed By: kcc

Subscribers: mgorny, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D37304

llvm-svn: 312183
2017-08-30 22:44:11 +00:00
Kamil Rytarowski 5ec32d4f96 Add NetBSD support in tsan_interceptors.cc
Summary:
NetBSD is a POSIX-like BSD Operating System.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, kcc, vitalybuka, dvyukov, eugenis

Reviewed By: dvyukov

Subscribers: srhines, kubamracek, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D37305

llvm-svn: 312160
2017-08-30 19:41:30 +00:00
Kamil Rytarowski 08e525ea06 Add NetBSD support in test/tsan/thread_name*.cc
Summary:
A snipped from the documentation of thread_setname_np(3):

NAME
     pthread_getname_np - get and set descriptive name of a thread

LIBRARY
     POSIX Threads Library (libpthread, -lpthread)

SYNOPSIS
     #include <pthread.h>

     int
     pthread_getname_np(pthread_t thread, char *name, size_t len);

     int
     pthread_setname_np(pthread_t thread, const char *name, void *arg);

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, dvyukov, eugenis, vitalybuka, kcc

Reviewed By: dvyukov

Subscribers: kubamracek, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D37306

llvm-svn: 312159
2017-08-30 19:40:47 +00:00
Vedant Kumar 637e321e35 [fuzzer] Don't enable tests when the fuzzer isn't built
Should fix:
http://green.lab.llvm.org/green/job/clang-stage2-coverage-R_build/1527

llvm-svn: 312157
2017-08-30 19:29:11 +00:00
Francis Ricci e3b018f6e5 [builtins] Prevent duplicate definitions for overridden functions
Summary:
Some architecture-specific function overrides (for example, i386/ashrdi3.S)
duplicate generic functions (in that case, ashrdi3.c). Prevent duplicate definitions
by filtering out the generic files before compiling.

Reviewers: compnerd, beanz

Subscribers: llvm-commits, mgorny

Differential Revision: https://reviews.llvm.org/D37166

llvm-svn: 312140
2017-08-30 17:12:57 +00:00
George Karpenkov 5a2898ae03 Fix for TSan unit-tests:
Previous refactoring has left unit-tests in a buggy state,
where they were not launched at all.

llvm-svn: 312094
2017-08-30 02:24:31 +00:00
Evgeniy Stepanov c6daf73c72 Restore clang_rt library name on i686-android.
Summary:
Recent changes canonicalized clang_rt library names to refer to
"i386" on all x86 targets. Android historically uses i686.

This change adds a special case to keep i686 in all clang_rt
libraries when targeting Android.

Reviewers: hans, mgorny, beanz

Subscribers: srhines, cfe-commits, llvm-commits

Differential Revision: https://reviews.llvm.org/D37278

llvm-svn: 312048
2017-08-29 22:12:31 +00:00
Matt Morehouse ba2e61b357 Revert "[SanitizeCoverage] Enable stack-depth coverage for -fsanitize=fuzzer"
This reverts r312026 due to bot breakage.

llvm-svn: 312047
2017-08-29 21:56:56 +00:00
Kostya Kortchinsky 913d038b29 Enable GetRandom for Fuchsia sanitizer.
Summary: Adds a true implementation of GetRandom, to be used by scudo_utils.h.

Reviewers: mcgrathr, phosek, kcc, vitalybuka, cryptoad

Reviewed By: mcgrathr

Subscribers: kubamracek

Differential Revision: https://reviews.llvm.org/D37218

llvm-svn: 312046
2017-08-29 21:52:56 +00:00
Vitaly Buka 07196d3145 [asan] Add use-after-scope test which fails because of bug in clang
Reviewers: kcc, eugenis

Subscribers: llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D37242

llvm-svn: 312039
2017-08-29 21:23:44 +00:00
Matt Morehouse 2edac86cdb Re-enable stack depth instrumentation on Windows.
Specified tls_model attribute properly. Should compile on Windows
now.

llvm-svn: 312037
2017-08-29 21:15:33 +00:00
Evgeniy Stepanov 3bafa546d0 Disable ubsan-minimal on Darwin.
Should un-break this bot:
http://green.lab.llvm.org/green//job/clang-stage1-configure-RA_build/38264/consoleFull

llvm-svn: 312036
2017-08-29 21:15:08 +00:00
Kostya Serebryany 4faeb87ebe [libFUzzer] change the way we load the seed corpora: instead of loading all files and these executing all files, load and execute them one-by-one. This should reduce the memory usage in many cases
llvm-svn: 312033
2017-08-29 20:51:24 +00:00
Matt Morehouse c29c2c9b0c Disable stack depth tracking on Windows.
Windows doesn't support the tls_model attribute.

llvm-svn: 312032
2017-08-29 20:44:41 +00:00
Evgeniy Stepanov 6d2b6f0a5f Minimal runtime for UBSan.
Summary:
An implementation of ubsan runtime library suitable for use in production.

Minimal attack surface.
* No stack traces.
* Definitely no C++ demangling.
* No UBSAN_OPTIONS=log_file=/path (very suid-unfriendly). And no UBSAN_OPTIONS in general.
* as simple as possible

Minimal CPU and RAM overhead.
* Source locations unnecessary in the presence of (split) debug info.
* Values and types (as in A+B overflows T) can be reconstructed from register/stack dumps, once you know what type of error you are looking at.
* above two items save 3% binary size.

When UBSan is used with -ftrap-function=abort, sometimes it is hard to reason about failures. This library replaces abort with a slightly more informative message without much extra overhead. Since ubsan interface in not stable, this code must reside in compiler-rt.

Reviewers: pcc, kcc

Subscribers: srhines, mgorny, aprantl, krytarowski, llvm-commits

Differential Revision: https://reviews.llvm.org/D36810

llvm-svn: 312029
2017-08-29 20:03:51 +00:00
Matt Morehouse 2ad8d948b2 [SanitizeCoverage] Enable stack-depth coverage for -fsanitize=fuzzer
Summary:
- Don't sanitize __sancov_lowest_stack.
- Don't instrument leaf functions.
- Add CoverageStackDepth to Fuzzer and FuzzerNoLink.
- Disable stack depth tracking on Mac.

Reviewers: vitalybuka, kcc, george.karpenkov

Reviewed By: kcc

Subscribers: kubamracek, cfe-commits, llvm-commits, hiraditya

Differential Revision: https://reviews.llvm.org/D37156

llvm-svn: 312026
2017-08-29 19:48:12 +00:00
Dean Michael Berris eca980396e [XRay][compiler-rt][NFC] Refactor global TLS variables behind an accessor function.
Summary:
This change hides all the initialization of thread_local variables used
by the XRay FDR mode implementation behind a function call. This makes
initialization of thread-local data to be done lazily, instead of
eagerly when they're done as globals. It also gives us an isolation
mechanism if/when we want to change the TLS implementation from using
the C++ thread_local keyword, for something more ad-hoc (potentialy
using pthread directly) on some platforms or set-ups where we cannot use
the C++ thread_local variables.

Reviewers: kpw, eizan

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D37248

llvm-svn: 311997
2017-08-29 12:21:45 +00:00
Kostya Serebryany 3a8e3c8336 [libFuzzer] refactoring: move reading the seed corpus closer to where it's consumed; NFC
llvm-svn: 311972
2017-08-29 02:05:01 +00:00
Kamil Rytarowski 802afb4c0a Shorten filenames of tests (-with-calls to -calls)
Summary:
The NetBSD's 8(beta) versions of kernel functions to retrieve
program name (vnode to path translator) and process memory
map have internal limit of processing filenames with maximum
of 31 characters.

Filenames like Asan-x86_64-with-calls-Noinst-Test break this
limit and affect tests. Rename "-with-calls" to "-calls".

This changes fixes all issues for the Address Sanitizer test
target (check-asan) on the current NetBSD support caused
by long filenames.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, vitalybuka, filcab, fjricci, kcc

Reviewed By: vitalybuka

Subscribers: kubamracek, mgorny, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D37149

llvm-svn: 311966
2017-08-29 01:03:13 +00:00
Kostya Serebryany c07008653c [libFuzzer] allow -print_funcs=N: N is the max number of new covered function printed
llvm-svn: 311945
2017-08-28 22:52:22 +00:00
Kamil Rytarowski 357bbc57f9 Add NetBSD ASAN shadow mapping for x86-64
Summary:
The maximal virtual address on NetBSD/amd64 is 0x7f7ffffff000.
Define shadow offset 0x400000000000 (1ULL << 46).

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, vitalybuka, filcab, fjricci, kcc, eugenis, pcc

Reviewed By: kcc

Subscribers: llvm-commits, kubamracek, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D36587

llvm-svn: 311937
2017-08-28 21:41:04 +00:00
Kamil Rytarowski 5abb0ac6ea Add NetBSD specific version of sanitizer_platform_limits_posix
Summary:
NetBSD is an Open-Source POSIX-like BSD Operating System.

Part of the code inspired by the original work on libsanitizer in GCC 5.4 by Christos Zoulas.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, kcc, vitalybuka, filcab, fjricci

Reviewed By: kcc

Subscribers: llvm-commits, kubamracek, mgorny, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D37193

llvm-svn: 311933
2017-08-28 21:03:23 +00:00
Michal Gorny 0e52de05db Reland r311842 - [cmake] Remove i686 target that is duplicate to i386
Remove the explicit i686 target that is completely duplicate to
the i386 target, with the latter being used more commonly.

1. The runtime built for i686 will be identical to the one built for
i386.

2. Supporting both -i386 and -i686 suffixes causes unnecessary confusion
on the clang end which has to expect either of them.

3. The checks are based on wrong assumption that __i686__ is defined for
all newer x86 CPUs. In fact, it is only declared when -march=i686 is
explicitly used. It is not available when a more specific (or newer)
-march is used.

Curious enough, if CFLAGS contain -march=i686, the runtime will be built
both for i386 and i686. For any other value, only i386 variant will be
built.

Differential Revision: https://reviews.llvm.org/D26764

llvm-svn: 311924
2017-08-28 20:30:12 +00:00
George Karpenkov 52a3e48e9f [libFuzzer] Exclude a test failing on OS X.
llvm-svn: 311916
2017-08-28 20:10:30 +00:00
George Karpenkov dacfd66dfd [libFuzzer] Fix libFuzzer flag propagation for standalone builds.
Under the previous configurations, flags from SANITIZER_COMMON were not
propagated for standalone builds.

Differential Revision: https://reviews.llvm.org/D37225

llvm-svn: 311912
2017-08-28 19:44:19 +00:00
George Karpenkov 16a05f2dfe Proper dependency check for clang in compiler_rt.
- Not having a dependency does not work in standalone build, as Clang does not exist.
 - if (TARGET clang) check is useless, as it is order-dependent,
   and Clang may not be registered yet.

Differential Revision: https://reviews.llvm.org/D37228

llvm-svn: 311911
2017-08-28 19:39:05 +00:00
Kostya Kortchinsky 476f21d87e [sanitizer] Re-introduce kUseSeparateSizeClassForBatch for the 32-bit Primary
Summary:
Currently `TransferBatch` are located within the same memory regions as
"regular" chunks. This is not ideal for security: they make for an interesting
target to overwrite, and are not protected by the frontend (namely, Scudo).

To solve this, we re-introduce `kUseSeparateSizeClassForBatch` for the 32-bit
Primary allowing for `TransferBatch` to end up in their own memory region.
Currently only Scudo would use this new feature, the default behavior remains
unchanged. The separate `kBatchClassID` was used for a brief period of time
previously but removed when the 64-bit ended up using the "free array".

Reviewers: alekseyshl, kcc, eugenis

Reviewed By: alekseyshl

Subscribers: llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D37082

llvm-svn: 311891
2017-08-28 15:20:02 +00:00
Dean Michael Berris 66faaceeb1 [XRay][compiler-rt] Return the pointer associated with the function instead of the sled
Summary:
XRay has erroneously been returning the address of the first sled in the
instrumentation map for a function id instead of the (runtime-relocated)
functison address. This causes confusion and issues for applications
where:

  - The first sled in the function may not be an entry sled (due to
    re-ordering or some other reason).
  - The caller attempts to find a symbol associated with the pointer at
    runtime, because the sled may not be exactly where the function's
    known address is (in case of inlined functions or those that have an
    external definition for symbols).

This fixes http://llvm.org/PR34340.

Reviewers: eizan

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D37202

llvm-svn: 311871
2017-08-28 03:58:23 +00:00
Petr Hosek 00b760ed48 [asan] Move __asan_handle_no_return to public header
Heretofore asan_handle_no_return was used only by interceptors,
i.e. code private to the ASan runtime. However, on systems without
interceptors, code like libc++abi is built with -fsanitize=address
itself and should call asan_handle_no_return directly from
__cxa_throw so that no interceptor is required.

Patch by Roland McGrath

Differential Revision: https://reviews.llvm.org/D36811

llvm-svn: 311869
2017-08-28 00:45:12 +00:00
George Karpenkov bebcbfb46d [libFuzzer] Use custom allocators for STL containers in libFuzzer.
Avoids ODR violations causing spurious ASAN warnings.

Differential Revision: https://reviews.llvm.org/D37086

llvm-svn: 311866
2017-08-27 23:20:09 +00:00
Michal Gorny 7298595a2c Revert r311842 - [cmake] Remove i686 target that is duplicate to i386
The required change in clang is being reverted because of the Android
build bot failure.

llvm-svn: 311859
2017-08-27 20:37:06 +00:00
Michal Gorny 34ca7168ae [cmake] Remove i686 target that is duplicate to i386
Remove the explicit i686 target that is completely duplicate to
the i386 target, with the latter being used more commonly.

1. The runtime built for i686 will be identical to the one built for
i386.

2. Supporting both -i386 and -i686 suffixes causes unnecessary confusion
on the clang end which has to expect either of them.

3. The checks are based on wrong assumption that __i686__ is defined for
all newer x86 CPUs. In fact, it is only declared when -march=i686 is
explicitly used. It is not available when a more specific (or newer)
-march is used.

Curious enough, if CFLAGS contain -march=i686, the runtime will be built
both for i386 and i686. For any other value, only i386 variant will be
built.

Differential Revision: https://reviews.llvm.org/D26764

llvm-svn: 311842
2017-08-27 07:44:41 +00:00
George Karpenkov 24069a0424 Revert "[libFuzzer] Use custom allocators for STL containers in libFuzzer"
This reverts commit 3539efc2f2218dba2bcbd645d0fe276f2b5cf588.

llvm-svn: 311831
2017-08-26 17:50:35 +00:00
George Karpenkov d50410bfb1 [libFuzzer] Use custom allocators for STL containers in libFuzzer
Avoids ODR violations causing spurious ASAN container overflow warnings.

Differential Revision: https://reviews.llvm.org/D37086

llvm-svn: 311830
2017-08-26 17:17:37 +00:00
Evgeniy Stepanov c28cedcfe8 Automatically pick up new sanitizers in cmake.
Change the default of COMPILER_RT_SANITIZERS_TO_BUILD to "all" in
order to automatically pick up new sanitizers in existing build
trees.

llvm-svn: 311824
2017-08-26 01:13:33 +00:00
Matt Morehouse 6ec7595b1e Revert "[SanitizeCoverage] Enable stack-depth coverage for -fsanitize=fuzzer"
This reverts r311801 due to a bot failure.

llvm-svn: 311803
2017-08-25 22:01:21 +00:00
Matt Morehouse f42bd31323 [SanitizeCoverage] Enable stack-depth coverage for -fsanitize=fuzzer
Summary:
- Don't sanitize __sancov_lowest_stack.
- Don't instrument leaf functions.
- Add CoverageStackDepth to Fuzzer and FuzzerNoLink.

Reviewers: vitalybuka, kcc

Reviewed By: kcc

Subscribers: cfe-commits, llvm-commits, hiraditya

Differential Revision: https://reviews.llvm.org/D37156

llvm-svn: 311801
2017-08-25 21:18:29 +00:00
Kostya Serebryany 09c4161029 [libFuzzer] prepare tests for switching from -fsanitize-coverage=trace-pc-guard to -fsanitize-coverage=inline-8bit-counters
llvm-svn: 311798
2017-08-25 20:20:46 +00:00
Kostya Serebryany 2eef816e6e [libFuzzer] add -print_funcs=1 (on bey default): print newly discovered functions during fuzzing
llvm-svn: 311797
2017-08-25 20:09:25 +00:00
Max Moroz 1dbb7578ff [compiler-rt] Test commit: remove some trailing white spaces.
Summary: [compiler-rt] Test commit: remove some trailing white spaces.

Reviewers: Dor1s

Reviewed By: Dor1s

Subscribers: dberris

Differential Revision: https://reviews.llvm.org/D37161

llvm-svn: 311796
2017-08-25 19:36:30 +00:00
Kostya Serebryany d3e4b7e24a [sanitizer-coverage] extend fsanitize-coverage=pc-table with flags for every PC
llvm-svn: 311794
2017-08-25 19:29:47 +00:00
Dmitry Vyukov 780b462840 tsan: fix darwin build
Runtime hooks do not seem to work on darwin. The test failed on builder:
http://lab.llvm.org:8080/green/job/clang-stage1-configure-RA_check/34882/consoleFull#7286766538254eaf0-7326-4999-85b0-388101f2d404
Move the test to Linux dir.

llvm-svn: 311776
2017-08-25 15:18:48 +00:00
Dmitry Vyukov 714ac2e781 tsan: don't pass bogus PCs to __tsan_symbolize_external
See the added comment for an explanation.

Reviewed in https://reviews.llvm.org/D37107

llvm-svn: 311768
2017-08-25 08:52:28 +00:00
Adrian Prantl 1d6ae01d4c Revert "Add Clang dependency to the check for Clang C++ headers."
This temporarily reverts commit r311733, because of bot breakage.
http://lab.llvm.org:8080/green/job/clang-stage1-configure-RA_build/38139/consoleFull#-256426522e9a0fee5-ebcc-4238-a641-c5aa112c323e

llvm-svn: 311757
2017-08-25 02:36:36 +00:00
Sean Fertile 6e5e0bcd80 [cfi] only add cfi tests on supported arches.
Differential Revision:  https://reviews.llvm.org/D35101

llvm-svn: 311745
2017-08-25 01:50:53 +00:00
George Karpenkov cdb8341e62 Add Clang dependency to the check for Clang C++ headers.
The problem is that CMake is mostly imperative and the result of
processing "if (TARGET blah)" checks depends on the order of import of
CMake files.
In this case, "projects" folder is registered before "tools",
and calling "CheckClangHeaders" [renamed to have a better name]
errors out without even giving Clang a chance to be built.

This, in turn, leads to libFuzzer bot failures in some circumstances on
some machines (depends on whether LIT or UNIT tests are scheduled
first).

Differential Revision: https://reviews.llvm.org/D37126

llvm-svn: 311733
2017-08-24 23:43:17 +00:00
Hans Wennborg c7d8b01774 Mark allocator_oom_test.cc unsupported on arm & aarch64 (PR33972)
The buildbots don't seem to like it.

llvm-svn: 311674
2017-08-24 17:00:36 +00:00
Xinliang David Li c7c5303fa3 [Profile] create a copy of profile file name from environment
Original patch by Max Moroz.

Differential Revsion: http://reviews.llvm.org/D36903

llvm-svn: 311607
2017-08-23 21:39:33 +00:00
Oleg Ranevskyy 6c75b51bb5 [ARM][Compiler-rt] Fix AEABI builtins to correctly pass arguments to non-AEABI functions on HF targets
Summary:
This is a patch for PR34167.

On HF targets functions like `__{eq,lt,le,ge,gt}df2` and `__{eq,lt,le,ge,gt}sf2` expect their arguments to be passed in d/s registers, while some of the AEABI builtins pass them in r registers.

Reviewers: compnerd, peter.smith, asl

Reviewed By: peter.smith, asl

Subscribers: peter.smith, aemerson, dberris, javed.absar, llvm-commits, asl, kristof.beyls

Differential Revision: https://reviews.llvm.org/D36675

llvm-svn: 311555
2017-08-23 14:26:31 +00:00
Dean Michael Berris 71f88a955d [XRay][compiler-rt] Support sled versioning for custom event sleds
Summary:
This change introduces versions to the instrumentation map entries we
emit for XRay instrumentaiton points. The status quo for the version is
currently set to 0 (as emitted by the LLVM back-end), and versions will
count up to 255 (unsigned char).

This change is in preparation for supporting the newer version of the
custom event sleds that will be emitted by the LLVM compiler.

While we're here, we take the opportunity to stash more registers and
align the stack properly in the __xray_CustomEvent trampoline.

Reviewers: kpw, pcc, dblaikie

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D36816

llvm-svn: 311524
2017-08-23 04:42:37 +00:00
Hans Wennborg c5e7cee0a2 [profile] Fix warning about C++ style comment in C file
llvm-svn: 311496
2017-08-22 21:54:37 +00:00
Hans Wennborg 67ef655026 [esan] Use stack_t instead of struct sigaltstack (PR34011)
The struct tag is going away in soon-to-be-released glibc 2.26 and the
stack_t typedef seems to have been there forever.

Patch by Bernhard Rosenkraenzer!

llvm-svn: 311495
2017-08-22 21:54:37 +00:00
Matt Morehouse b1fa8255db [SanitizerCoverage] Optimize stack-depth instrumentation.
Summary:
Use the initialexec TLS type and eliminate calls to the TLS
wrapper.  Fixes the sanitizer-x86_64-linux-fuzzer bot failure.

Reviewers: vitalybuka, kcc

Reviewed By: kcc

Subscribers: hiraditya, llvm-commits

Differential Revision: https://reviews.llvm.org/D37026

llvm-svn: 311490
2017-08-22 21:28:29 +00:00
George Karpenkov 6bb45f793f [libFuzzer] Move check for thread_local back into libFuzzer's CMake,
as it breaks builtin standalone build on some bots.

llvm-svn: 311482
2017-08-22 18:34:28 +00:00
Kostya Serebryany 868e69091c [libFuzzer] disable a test failing on the bot
llvm-svn: 311427
2017-08-22 05:15:57 +00:00
Saleem Abdulrasool 84315ec513 Revert "builtins: erase `struct` modifier for EH personality"
This reverts SVN r311425 which broke one of the buildbots.  It is
unclear what header is being used there.  Revert it until that can be
handled properly.

llvm-svn: 311426
2017-08-22 04:19:51 +00:00
Saleem Abdulrasool 94fd3c41fb builtins: erase `struct` modifier for EH personality
On ARM, the `_Unwind_Exception` is an alias for
`struct _Unwind_Control_Block`.  The extra `struct` modifier causes a
warning due to the locally scoped type.  Special case this to avoid the
warning.  NFC.

llvm-svn: 311425
2017-08-22 04:05:50 +00:00
Kostya Serebryany 33fb36c353 [libFuzzer] fix the stack-depth initialization, add a lit test for DeepRecursionTest.cpp
llvm-svn: 311421
2017-08-22 01:50:00 +00:00
Kostya Serebryany f65cf64fa2 [libFuzzer] apply changes lost during the migration to compiler-rt
llvm-svn: 311420
2017-08-22 01:28:32 +00:00
George Karpenkov e4c5322992 [NFC] do not run linter on libFuzzer's tests.
llvm-svn: 311419
2017-08-22 01:19:17 +00:00
Kostya Serebryany 055f441bd2 [libFuzzer] better README.txt
llvm-svn: 311418
2017-08-22 01:15:40 +00:00
Kostya Serebryany 0977643039 [libFuzzer] remove stale file
llvm-svn: 311417
2017-08-22 01:14:53 +00:00
George Karpenkov 92d8000487 Remove check-fuzzer from check-all, as tests don't pass on some bots.
llvm-svn: 311415
2017-08-22 00:54:57 +00:00
George Karpenkov 10ab2ace13 Move libFuzzer to compiler_rt.
Resulting library binaries will be named libclang_rt.fuzzer*, and will
be placed in Clang toolchain, allowing redistribution.

Differential Revision: https://reviews.llvm.org/D36908

llvm-svn: 311407
2017-08-21 23:25:50 +00:00
Evgeniy Stepanov 753297e634 Fix ASan version list dependency in multi-arch builds.
Fixes PR32390.

Patch by Andrey Krayniak.

llvm-svn: 311394
2017-08-21 22:31:31 +00:00
Kostya Kortchinsky 0d3439a29c [sanitizer] Do not over-dup string flags
Summary:
String flags values appear to be duped twice. Once in `FlagParser::parse_flag`
using the `LowLevelAllocator` via `ll_strndup`, once in
`FlagHandler<const char *>::Parse` using the `InternalAllocator` via
`internal_strdup`. It looks like the second one is redundant, as the memory
for the first one is never freed and not used for anything else.

Assigning the value to the flag instead of duping it has a few advantages:
- if it was the only use of the `InternalAllocator` (which is the case for
  Scudo), then the related code will not be compiled it, which saves us a
  whole instantiation of the CombinedAllocator worth of extra code;
- in the event a string flag is parsed, the `InternalAllocator` would have
  created a whole SizeClassAllocator32 region for a single allocation, which is
  kind of wasteful.
- also, the string is dup'ed twice for the whole lifetime of a process.

I tested check-{sanitizer,asan,tsan,ubsan,scudo} successfully, so as far as I
can tell this doesn't appear to have bad side effects.

Reviewers: eugenis, alekseyshl

Reviewed By: eugenis

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D36970

llvm-svn: 311386
2017-08-21 21:25:38 +00:00
George Karpenkov 93e9e8a0d3 [NFC CMake] Do not relink test targets every time in compiler-rt
CMake's add_custom_target is considered to be *always* out of date.
This patch changes it to a combination of add_custom_target and
add_custom_command which actually tracks dependencies' timestamps.

On my machine this reliably saves 6-7 seconds on each test group.
This can be a large difference when debugging small tests.

Differential Revision: https://reviews.llvm.org/D36912

llvm-svn: 311384
2017-08-21 21:19:13 +00:00
George Karpenkov 9d7a75d617 Fix multi-architecture build for lib/xray.
Differential Revision: https://reviews.llvm.org/D36881

llvm-svn: 311379
2017-08-21 20:28:32 +00:00
Kuba Mracek b17fd11e09 Remove "%T" from ASan Darwin tests.
llvm-svn: 311298
2017-08-20 18:31:00 +00:00
Vlad Tsyrklevich 599b11784a [TSan] Update test values
Summary:
This test was broken by the tail duplication logic being changed in
r311139, update the test values and add a note about how to properly run
a benchmark to verify that the values are safe to update.

Reviewers: vitalybuka

Reviewed By: vitalybuka

Subscribers: dvyukov, kubamracek

Differential Revision: https://reviews.llvm.org/D36889

llvm-svn: 311189
2017-08-18 19:22:39 +00:00
Matt Morehouse 5c7fc76983 [SanitizerCoverage] Add stack depth tracing instrumentation.
Summary:
Augment SanitizerCoverage to insert maximum stack depth tracing for
use by libFuzzer.  The new instrumentation is enabled by the flag
-fsanitize-coverage=stack-depth and is compatible with the existing
trace-pc-guard coverage.  The user must also declare the following
global variable in their code:
  thread_local uintptr_t __sancov_lowest_stack

https://bugs.llvm.org/show_bug.cgi?id=33857

Reviewers: vitalybuka, kcc

Reviewed By: vitalybuka

Subscribers: kubamracek, hiraditya, cfe-commits, llvm-commits

Differential Revision: https://reviews.llvm.org/D36839

llvm-svn: 311186
2017-08-18 18:43:30 +00:00
Dean Michael Berris 7eaaa0f0f2 [XRay][compiler-rt][NFC] Expand the PIC test case for XRay
Summary:
Here we add a build with -ffunction-sections -fdata-sections and
-Wl,--gc-sections to ensure that we're still able to generate XRay
traces.

This is just adding a test, no functional changes.

Differential Revision: https://reviews.llvm.org/D36863

llvm-svn: 311145
2017-08-18 05:24:32 +00:00
Francis Ricci 69639c24f9 Add C++17 aligned new/delete interceptors to standalone lsan
Summary: Based on r282019.

Reviewers: kcc, jakubjelinek, alekseyshl

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D36757

llvm-svn: 311030
2017-08-16 18:09:29 +00:00
Kostya Kortchinsky 43917720a7 [scudo] Application & platform compatibility changes
Summary:
This patch changes a few (small) things around for compatibility purposes for
the current Android & Fuchsia work:
- `realloc`'ing some memory that was not allocated with `malloc`, `calloc` or
  `realloc`, while UB according to http://pubs.opengroup.org/onlinepubs/009695399/functions/realloc.html
  is more common that one would think. We now only check this if
  `DeallocationTypeMismatch` is set; change the "mismatch" error
  messages to be more homogeneous;
- some sketchily written but widely used libraries expect a call to `realloc`
  to copy the usable size of the old chunk to the new one instead of the
  requested size. We have to begrundingly abide by this de-facto standard.
  This doesn't seem to impact security either way, unless someone comes up with
  something we didn't think about;
- the CRC32 intrinsics for 64-bit take a 64-bit first argument. This is
  misleading as the upper 32 bits end up being ignored. This was also raising
  `-Wconversion` errors. Change things to take a `u32` as first argument.
  This also means we were (and are) only using 32 bits of the Cookie - not a
  big thing, but worth mentioning.
- Includes-wise: prefer `stddef.h` to `cstddef`, move `scudo_flags.h` where it
  is actually needed.
- Add tests for the memalign-realloc case, and the realloc-usable-size one.

(Edited typos)

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D36754

llvm-svn: 311018
2017-08-16 16:40:48 +00:00
George Karpenkov 4c26922662 Quickfix to the refactoring commit: typo in the link flags variable
name.

llvm-svn: 310973
2017-08-15 23:22:52 +00:00
George Karpenkov 769124dc5e [sanitizers CMake] NFC Refactor the logic for compiling and generating tests
into a function.

Most CMake configuration under compiler-rt/lib/*/tests have
almost-the-same-but-not-quite functions of the form add_X_[unit]tests
for compiling and running the tests.
Much of the logic is duplicated with minor variations across different
sub-folders.
This can harm productivity for multiple reasons:

For newcomers, resulting CMake files are very large, hard to understand,
and hide the intention of the code.
Changes for enabling certain architectures end up being unnecessarily
large, as they get duplicated across multiple folders.
Adding new sub-projects requires more effort than it should, as a
developer has to again copy-n-paste the configuration, and it's not even
clear from which sub-project it should be copy-n-pasted.
With this change the logic of compile-and-generate-a-set-of-tests is
extracted into a function, which hopefully makes writing and reading
CMake much easier.

Differential Revision: https://reviews.llvm.org/D36116

llvm-svn: 310971
2017-08-15 22:56:10 +00:00
George Karpenkov 83ea28116e [CMake compiler-rt] NFC: Minor CMake refactoring.
Detect ObjC files in `clang_compile` and pass an appropriate flag to a
compiler, also change `clang_compile` to a function.

Differential Revision: https://reviews.llvm.org/D36727

llvm-svn: 310945
2017-08-15 18:38:14 +00:00
George Karpenkov cb6be4dd90 [compiler-rt CMake] NFC: Minor CMake refactoring.
Change macro to a function, and use a generic variable instead of
branching for handling multi-output build with
CMAKE_CONFIGURATION_TYPES.

Differential Revision: https://reviews.llvm.org/D36725

llvm-svn: 310944
2017-08-15 18:35:02 +00:00
George Karpenkov d46f17db06 [compiler-rt CMake] CMake refactoring: create directories in helper func.
Change macro to a function, move creating test directory into
`add_compiler_rt_test`.

Differential Revision: https://reviews.llvm.org/D36724

llvm-svn: 310943
2017-08-15 18:32:28 +00:00
Kamil Rytarowski d341194d68 Revert: Enable profile on NetBSD
Requested by V.Kumar.

Not all tests pass.

llvm-svn: 310912
2017-08-15 09:56:47 +00:00
Xinliang David Li 1c9320cd4d Revert r310857 due to internal test failure
llvm-svn: 310907
2017-08-15 03:13:01 +00:00
Weiming Zhao 1c617be054 [builtins] fix build error on non-ARM for r310884
llvm-svn: 310890
2017-08-14 21:44:33 +00:00
Weiming Zhao 8bf15723ae [builtins][ARM] Select correct code fragments when compiling for Thumb1/Thum2/ARM ISA
Summary:
Value of __ARM_ARCH_ISA_THUMB isn't based on the actual compilation
 mode (-mthumb, -marm), it reflect's capability of given CPU.

Due to this:
•use tbumb and thumb2 insteand of __ARM_ARCH_ISA_THUMB
•use '.thumb' directive consistently in all affected files
•decorate all thumb functions using DEFINE_COMPILERRT_THUMB_FUNCTION()

(This is based off  Michal's patch https://reviews.llvm.org/D30938)

Reviewers: dim, rengolin, compnerd, strejda

Reviewed By: compnerd

Subscribers: peter.smith, kubamracek, mgorny, javed.absar, kristof.beyls, jamesduley, aemerson, llvm-commits

Differential Revision: https://reviews.llvm.org/D31220

llvm-svn: 310884
2017-08-14 20:48:47 +00:00
Evgeniy Stepanov b52b141dd0 Fix a cmake typo.
Also add "libc++" to list of property values (AFAIK that only affects the cmake gui).

llvm-svn: 310883
2017-08-14 20:42:43 +00:00
Xinliang David Li 533d4c90bd [PGO] Add support for relocate profile dumping directory
Differential Revsion: http://reviews.llvm.org/D36648

llvm-svn: 310857
2017-08-14 16:51:05 +00:00
Kostya Kortchinsky e1dde07640 [sanitizers] Add a blocking boolean to GetRandom prototype
Summary:
On platforms with `getrandom`, the system call defaults to blocking. This
becomes an issue in the very early stage of the boot for Scudo, when the RNG
source is not set-up yet: the syscall will block and we'll stall.

Introduce a parameter to specify that the function should not block, defaulting
to blocking as the underlying syscall does.

Update Scudo to use the non-blocking version.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D36399

llvm-svn: 310839
2017-08-14 14:53:47 +00:00
Sean Eveson cf9525eac2 [llvm-cov] Fix compiler-rt tests failing in build bots after rL310827.
The compiler-rt tests used llvm-cov with -filename-equivelence, which was
replaced with the new option -path-equivalence in rL310827.

llvm-svn: 310836
2017-08-14 12:43:05 +00:00
Kamil Rytarowski c4edc0705c Enable profile on NetBSD
Summary:
make check-profile:

Failing Tests (2):
    Profile-i386 :: instrprof-dlopen.test
    Profile-x86_64 :: instrprof-dlopen.test

  Expected Passes    : 64
  Unsupported Tests  : 42
  Unexpected Failures: 2

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, vitalybuka, kcc, filcab, fjricci

Reviewed By: vitalybuka

Subscribers: vsk, llvm-commits, srhines, mgorny, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D36603

llvm-svn: 310800
2017-08-13 20:18:15 +00:00
Evgeniy Stepanov 3072b539da [compiler-rt] Add SANITIZER_CXX_ABI_LIBNAME=libc++ option.
Summary: This is to support Android where libc++abi is part of libc++.

Reviewers: srhines, EricWF

Subscribers: dberris, mgorny, llvm-commits

Differential Revision: https://reviews.llvm.org/D36640

llvm-svn: 310769
2017-08-11 22:28:02 +00:00
Kamil Rytarowski 7a6b680622 Enable ASAN on NetBSD
Summary:
This enables also static runtime option.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, vitalybuka, filcab, kcc, fjricci

Reviewed By: vitalybuka

Subscribers: mgorny, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D36490

llvm-svn: 310651
2017-08-10 19:08:39 +00:00
Kamil Rytarowski 4ad374ea76 Add NetBSD support in asan_linux.cc
Summary:
Part of the code inspired by the original work on libsanitizer in GCC 5.4 by Christos Zoulas.

Sponsored by <The NetBSD Foundation>

Reviewers: fjricci, vitalybuka, joerg, kcc, filcab

Reviewed By: vitalybuka

Subscribers: llvm-commits, kubamracek, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D36488

llvm-svn: 310647
2017-08-10 18:51:51 +00:00
Kamil Rytarowski fbd51b954d Enable SafeStack on NetBSD
Summary:
make check-safestack:

-- Testing: 8 tests, 8 threads --
Testing: 0 .. 10.. 20.. 30.. 40.. 50.. 60.. 70.. 80.. 90..
Testing Time: 0.44s
  Expected Passes    : 7
  Unsupported Tests  : 1

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, vitalybuka, kcc, fjricci, filcab

Reviewed By: vitalybuka

Subscribers: mgorny, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D36542

llvm-svn: 310646
2017-08-10 18:40:09 +00:00
Alexander Potapenko 925c0741ab [sanitizer_common] Update sanitizers w.r.t. the new comparisons instrumentation API
Added declarations of __sanitizer_cov_trace_const_cmp[1248] callbacks.
For more details, please see https://reviews.llvm.org/D36465.

Patch by Victor Chibotaru.

llvm-svn: 310596
2017-08-10 14:22:57 +00:00
Oleg Ranevskyy a02114b651 [compiler-rt][ARM] Fix filtering of ARM targets
Summary:
Similarly to i686, the ARM build target has multiple names, such as armhf, armv7 and so on. Currently we get duplicated symbol definitions for these targets while compiling the library. Each duplicated definition has its generic version from `lib/builtins` and an ARM-specialized version from `lib/builtins/arm`. 

This patch fixes filtering for ARM to ignore the generic definitions if they have their ARM specializations.

Reviewers: compnerd

Reviewed By: compnerd

Subscribers: aemerson, dberris, llvm-commits, mgorny, asl, kristof.beyls

Differential Revision: https://reviews.llvm.org/D35336

llvm-svn: 310588
2017-08-10 13:27:29 +00:00
Vitaly Buka 7030354744 [asan] Refactor thread creation bookkeeping
Summary:
This is a pure refactoring change.  It paves the way for OS-specific
implementations, such as Fuchsia's, that can do most of the
per-thread bookkeeping work in the creator thread before the new
thread actually starts.  This model is simpler and cleaner, avoiding
some race issues that the interceptor code for thread creation has
to do for the existing OS-specific implementations.

Submitted on behalf of Roland McGrath.

Reviewers: vitalybuka, alekseyshl, kcc

Reviewed By: alekseyshl

Subscribers: phosek, filcab, llvm-commits, kubamracek

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D36385

llvm-svn: 310432
2017-08-09 00:38:57 +00:00
Vitaly Buka 83832fe7db [asan] Complete the Fuchsia port
Submitted on behalf of Roland McGrath.

Reviewers: kcc, eugenis, alekseyshl, vitalybuka

Reviewed By: vitalybuka

Subscribers: filcab, vitalybuka, srhines, kubamracek, mgorny, phosek, llvm-commits

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D35865

llvm-svn: 310431
2017-08-09 00:21:45 +00:00
Kamil Rytarowski e5a9d9f17c Try to appease compiler and break multiline comment.
llvm-svn: 310428
2017-08-08 23:31:21 +00:00
Reid Kleckner c3fc3bc5af [winasan] Fix hotpatching ntdll!strcpy for Win10 creators edition
The 9 byte nop is a suffix of the 10 byte nop, and we need at most 6
bytes.

ntdll's version of strcpy is written in assembly and is very clever.
strcat tail calls strcpy but with a slightly different arrangement of
argument registers at an alternate entry point. It looks like this:

  ntdll!strcpy:
  00007ffd`64e8a7a0 4c8bd9          mov     r11,rcx
  ntdll!__entry_from_strcat_in_strcpy:
  00007ffd`64e8a7a3 482bca          sub     rcx,rdx
  00007ffd`64e8a7a6 f6c207          test    dl,7

If we overwrite more than two bytes in our interceptor, that label will
no longer be a valid instruction boundary.

By recognizing the 9 byte nop, we use the two byte backwards branch to
start our trampoline, avoiding this issue.

Fixes https://github.com/google/sanitizers/issues/829

Patch by David Major

llvm-svn: 310419
2017-08-08 21:18:36 +00:00
Kamil Rytarowski a1e4edd155 Add NetBSD support in asan_malloc_linux.cc
Summary:
Part of the code inspired by the original work on libsanitizer in GCC 5.4 by Christos Zoulas.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, kcc, fjricci, vitalybuka, filcab

Reviewed By: fjricci

Subscribers: llvm-commits, kubamracek, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D36376

llvm-svn: 310414
2017-08-08 20:52:54 +00:00
Kamil Rytarowski c5c9ba1ca9 Add NetBSD support in asan_symbolize.py
Summary:
Part of the code inspired by the original work on libsanitizer in GCC 5.4 by Christos Zoulas.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, filcab, vitalybuka, kcc, fjricci

Reviewed By: fjricci

Subscribers: kubamracek, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D36484

llvm-svn: 310413
2017-08-08 20:50:07 +00:00
Kamil Rytarowski c9bcc8dc9d Enable ubsan on NetBSD
Summary:
Part of the code inspired by the original work on libsanitizer in GCC 5.4 by Christos Zoulas.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, vitalybuka, kcc, filcab, fjricci

Reviewed By: fjricci

Subscribers: srhines, kubamracek, mgorny, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D36483

llvm-svn: 310412
2017-08-08 20:49:20 +00:00
Kamil Rytarowski c187cabe41 Reuse sanitizer_linux for NetBSD
Summary:
Follow FreeBSD and reuse sanitizer_linux for NetBSD.

Part of the code inspired by the original work on libsanitizer in GCC 5.4 by Christos Zoulas.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, kcc, filcab, vitalybuka, fjricci, dvyukov

Reviewed By: fjricci

Subscribers: dvyukov, emaste, kubamracek, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D36325

llvm-svn: 310411
2017-08-08 20:36:10 +00:00
Kamil Rytarowski f5266f03a8 Add NetBSD support in asan_interceptors.cc
Summary:
Part of the code inspired by the original work on libsanitizer in GCC 5.4 by Christos Zoulas.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, kcc, vitalybuka, filcab, fjricci

Reviewed By: vitalybuka

Subscribers: kubamracek, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D36470

llvm-svn: 310400
2017-08-08 19:09:48 +00:00
Kamil Rytarowski 367c63dc11 NetBSD compatibility nit in asan_test_utils.h
Summary:
Do not include <malloc.h> on NetBSD, as this header
serves on this OS backward compatibility with K&R alias
for <stdlib.h>.

Sponsored by <The NetBSD Foundation>

Reviewers: vitalybuka, kcc, joerg, filcab, fjricci

Reviewed By: vitalybuka

Subscribers: kubamracek, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D36469

llvm-svn: 310391
2017-08-08 18:16:35 +00:00
Kamil Rytarowski 741d21f958 Enable COMPILER_RT_HAS_SANITIZER_COMMON on NetBSD
Summary:
Temporarily keep disabled COMPILER_RT_HAS_ASAN on NetBSD.

Part of the code inspired by the original work on libsanitizer in GCC 5.4 by Christos Zoulas.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, filcab, kcc, vitalybuka

Reviewed By: vitalybuka

Subscribers: srhines, mgorny, #sanitizers, llvm-commits

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D36312

llvm-svn: 310370
2017-08-08 15:25:26 +00:00
Kamil Rytarowski 70a3511bd5 Reuse interception_linux for NetBSD
Summary:
Part of the code inspired by the original work on libsanitizer in GCC 5.4 by Christos Zoulas.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, kcc, vitalybuka, filcab

Reviewed By: vitalybuka

Subscribers: llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D36321

llvm-svn: 310351
2017-08-08 12:10:08 +00:00
Kamil Rytarowski e528bd2193 Define OFF_T as 64-bit integer on NetBSD
Summary:
All 32 and 64 bit NetBSD platforms define off_t as 64-bit integer.

Part of the code inspired by the original work on libsanitizer in GCC 5.4 by Christos Zoulas.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, filcab, kcc, vitalybuka

Reviewed By: vitalybuka

Subscribers: emaste, kubamracek, llvm-commits

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D35553

llvm-svn: 310349
2017-08-08 11:40:15 +00:00
Vitaly Buka 4bc6c466b8 [asan] Restore dead-code-elimination optimization for Fuchsia
Summary:
r310244 fixed a bug introduced by r309914 for non-Fuchsia builds.
In doing so it also reversed the intended effect of the change for
Fuchsia builds, which was to allow all the AllocateFromLocalPool
code and its variables to be optimized away entirely.

This change restores that optimization for Fuchsia builds, but
doesn't have the original change's bug because the comparison
arithmetic now takes into account the size of the elements.

Submitted on behalf of Roland McGrath.

Reviewers: vitalybuka, alekseyshl

Reviewed By: alekseyshl

Subscribers: llvm-commits, kubamracek

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D36430

llvm-svn: 310330
2017-08-08 01:01:59 +00:00
Kamil Rytarowski 1b39be7867 Fix asan_test.cc build on NetBSD
Summary:
Include <stdarg.h> for variable argument list macros (va_list, va_start etc).

Add fallback definition of _LIBCPP_GET_C_LOCALE, this is required for
GNU libstdc++ compatibility. Define new macro SANITIZER_GET_C_LOCALE.
This value is currently required for FreeBSD and NetBSD for printf_l(3) tests.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, kcc, vitalybuka, filcab, fjricci

Reviewed By: vitalybuka

Subscribers: llvm-commits, emaste, kubamracek, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D36406

llvm-svn: 310323
2017-08-07 23:38:14 +00:00
Kamil Rytarowski 123f62d515 Add NetBSD support in asan_stack.h
Summary:
Part of the code inspired by the original work on libsanitizer in GCC 5.4 by Christos Zoulas.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, kcc, vitalybuka, filcab, fjricci

Reviewed By: vitalybuka

Subscribers: davide, kubamracek, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D36377

llvm-svn: 310322
2017-08-07 23:34:45 +00:00
Kuba Mracek 507b1e280c [tsan] Fix format string in WriteMemoryProfile
The current format string is broken and fails to parse.

Differential Revision: https://reviews.llvm.org/D36192

llvm-svn: 310276
2017-08-07 18:12:59 +00:00
Kuba Mracek ac102b6977 [asan] Make dump_registers.cc more stable
Differential Revision: https://reviews.llvm.org/D36231

llvm-svn: 310273
2017-08-07 18:12:01 +00:00
Kuba Mracek b0d208a0ab [sanitizer] Remove use of task_for_pid from sanitizer_stoptheworld_mac.cc
Using task_for_pid to get the "self" task is not necessary, and it can fail (e.g. for sandboxed processes). Let's just use mach_task_self().

Differential Revision: https://reviews.llvm.org/D36284

llvm-svn: 310271
2017-08-07 18:07:20 +00:00
Kamil Rytarowski dc213718db Add NetBSD support in sanitizer_test_utils.h
Summary:
NetBSD ships with printf_l(3) like FreeBSD.

NetBSD does not ship with memalign, pvalloc, malloc with "usable size"
and is the same here as Darwin, Android, FreeBSD and Windows.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, vitalybuka, kcc, fjricci, filcab

Reviewed By: vitalybuka

Subscribers: srhines, llvm-commits, emaste, kubamracek, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D36373

llvm-svn: 310248
2017-08-07 10:59:44 +00:00
Kamil Rytarowski b0ca299cfe Add NetBSD support in asan_errors.cc
Summary:
Part of the code inspired by the original work on libsanitizer in GCC 5.4 by Christos Zoulas.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, fjricci, vitalybuka, filcab, kcc

Reviewed By: vitalybuka

Subscribers: llvm-commits, kubamracek, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D36374

llvm-svn: 310247
2017-08-07 10:58:48 +00:00
Kamil Rytarowski 767960bf86 Add NetBSD support in asan_interceptors.h
Summary:
Part of the code inspired by the original work on libsanitizer in GCC 5.4 by Christos Zoulas.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, filcab, kcc, fjricci, vitalybuka

Reviewed By: vitalybuka

Subscribers: kubamracek, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D36375

llvm-svn: 310246
2017-08-07 10:57:58 +00:00
Vitaly Buka bdd455f0d5 [asan] Return sizeof missed by r309914
llvm-svn: 310244
2017-08-07 09:08:44 +00:00
Kamil Rytarowski c827f014c3 Add NetBSD support in sanitizer_errno.h
Summary:
NetBSD ships with __errno (value for __errno_location) like Android.

Part of the code inspired by the original work on libsanitizer in GCC 5.4 by Christos Zoulas.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, vitalybuka, fjricci, kcc, filcab

Reviewed By: vitalybuka

Subscribers: llvm-commits, srhines, kubamracek, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D36360

llvm-svn: 310182
2017-08-05 18:10:51 +00:00
Kamil Rytarowski 4c69d68ea0 Add NetBSD support in sanitizer_unwind_linux_libcdep.cc
Summary:
NetBSD is a POSIX-like and BSD-family system.

Reuse FreeBSD and Linux code.

NetBSD uses DWARF ExceptionHandler.

Part of the code inspired by the original work on libsanitizer in GCC 5.4 by Christos Zoulas.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, kcc, filcab, vitalybuka

Reviewed By: vitalybuka

Subscribers: srhines, emaste, llvm-commits, kubamracek, aprantl, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D36314

llvm-svn: 310179
2017-08-05 13:36:49 +00:00
Kamil Rytarowski 886b30c4ff Add NetBSD support in sanitizer_linux_libcdep.cc
Summary:
When possible reuse FreeBSD and Linux code.

Part of the code inspired by the original work on libsanitizer in GCC 5.4 by Christos Zoulas.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, kcc, vitalybuka, filcab

Reviewed By: vitalybuka

Subscribers: srhines, emaste, kubamracek, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D36320

llvm-svn: 310143
2017-08-04 22:32:46 +00:00
Kamil Rytarowski d1dc704c87 Add NetBSD support in interception.h
Summary:
Part of the code inspired by the original work on libsanitizer in GCC 5.4 by Christos Zoulas.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, kcc, filcab, vitalybuka

Reviewed By: vitalybuka

Subscribers: llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D36323

llvm-svn: 310140
2017-08-04 22:27:01 +00:00
Kamil Rytarowski 6c68c2e2c5 Add NetBSD support in sanitizer_syscall_generic.inc
Summary:
This adds:
 - NetBSD specific aliases for renamed syscalls,
 - differentiate internal_syscall, internal_syscall64, internal_syscall_ptr as there are various types of syscalls on NetBSD.

Part of the code inspired by the original work on libsanitizer in GCC 5.4 by Christos Zoulas.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, kcc, vitalybuka, filcab

Reviewed By: vitalybuka

Subscribers: kubamracek, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D36316

llvm-svn: 310139
2017-08-04 22:23:52 +00:00
Kostya Kortchinsky a53de0bf0f [asan] Fix unsupported test on Android
Summary:
`pvalloc` appears to not be available on Android. Mark the failing test as
unsupported on that platform.

Reviewers: alekseyshl, vitalybuka

Reviewed By: alekseyshl, vitalybuka

Subscribers: srhines, kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D36339

llvm-svn: 310133
2017-08-04 21:26:20 +00:00
Kostya Kortchinsky d7d1681a0e [asan] Check for pvalloc overlow
Summary:
Last one of the `pvalloc` overflow checks!

`CheckForPvallocOverflow` was introduced with D35818 to detect when `pvalloc`
would wrap when rounding up to the next multiple of the page size.

Add this check to ASan's `pvalloc` implementation.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D36257

llvm-svn: 310119
2017-08-04 20:28:59 +00:00
Kamil Rytarowski fcf8e5e345 Add NetBSD support in sanitizer_procmaps_freebsd.cc
Summary:
This adds NetBSD specific:
 - ReadProcMaps()
 - MemoryMappingLayout::Next()

This code is largely shared with FreeBSD.

Part of the code inspired by the original work on libsanitizer in GCC 5.4 by Christos Zoulas.

Sponsored by <The NetBSD Foundation>

Reviewers: kcc, joerg, filcab, vitalybuka, fjricci

Reviewed By: fjricci

Subscribers: emaste, kubamracek, mgorny, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D35551

llvm-svn: 310116
2017-08-04 20:17:24 +00:00
Vlad Tsyrklevich 8d7755e7aa CFI: Move STL allocator blacklist to clang
Summary:
The regular expression to match STL allocators can't easily account for
C++ mangling compression and fails to match some valid instances of STL
allocators. Perform this logic in clang instead.

Motivated by crbug.com/751385.

Reviewers: pcc, kcc, llvm-commits

Reviewed By: pcc

Differential Revision: https://reviews.llvm.org/D36291

llvm-svn: 310109
2017-08-04 20:04:01 +00:00
Alex Shlyapnikov d053135d89 [compiler-rt] Check for empty buffer in Addr2LineProcess::ReadFromSymbolizer
This fixes a bug in the ReadFromSymbolizer method of the
Addr2LineProcess class; if the input is too large, the returned buffer
will be null and will consequently fail the CHECK. The proposed fix is
to simply check if the buffer consists of only a null-terminator and
return if so (in effect skipping that frame). I tested by running one of
the unit tests both before and after my change.

Submitted on behalf of david-y-lam.

Reviewers: eugenis, alekseyshl, kcc

Reviewed By: alekseyshl

Differential Revision: https://reviews.llvm.org/D36207

llvm-svn: 310089
2017-08-04 18:39:36 +00:00
Benjamin Kramer 7fba72e97b [msan] Switch the pvalloc overflow test to a lit test
The test was not passing on targets where allocator_may_return_null
defaults to true. Change the test to a lit test so that we can test both
situations.

Patch by Kostya Kortchinsky!

Differential Revision: https://reviews.llvm.org/D36302

llvm-svn: 310033
2017-08-04 07:32:10 +00:00
Vedant Kumar 696e3847dd coverage: Update tests to reflect changes from r310012
llvm-svn: 310015
2017-08-04 01:22:19 +00:00
Martin Storsjo becd2ef602 [builtins] Use Interlocked* intrinsics for atomics on MSVC
Tested on MSVC 2013, 2015 and 2017 targeting X86, X64 and ARM.

This fixes building emutls.c for Windows for ARM (both with clang
which don't need these atomics fallbacks at all, but just failed
due to the immintrin.h include before, and with MSVC).

Differential Revision: https://reviews.llvm.org/D36071

llvm-svn: 309974
2017-08-03 19:04:28 +00:00
Sterling Augustine 3eaab9620d These tests use 80-bit long doubles, which are x86 only. Mark them so.
This avoids having each new target need to mark them as unsupported.

llvm-svn: 309973
2017-08-03 18:56:54 +00:00
Vitaly Buka 993bbbf19e [asan] Allocator support for Fuchsia
Submitted on behalf of Roland McGrath.

Reviewers: vitalybuka, alekseyshl, kcc

Reviewed By: alekseyshl

Subscribers: srhines, cryptoad, kubamracek, phosek, filcab, llvm-commits

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D36190

llvm-svn: 309914
2017-08-03 02:22:11 +00:00
Dean Michael Berris 449fdf816e [XRay][compiler-rt] Allow for building the XRay runtime without PREINIT initialization.
Summary:
Define a build-time configuration option for the XRay runtime to
determine whether the archive will add an entry to the `.preinit_array`
section of the binary. We also allow for initializing the XRay data
structures with an explicit call to __xray_init(). This allows us to
give users the capability to initialize the XRay data structures on
demand.

This can allow us to start porting XRay to platforms where
`.preinit_array` isn't a supported section. It also allows us to limit
the effects of XRay in the initialization sequence for applications that
are sensitive to this kind of interference (i.e. large binaries) or
those that want to package XRay control in libraries.

Future changes should allow us to build two different library archives
for the XRay runtime, and allow clang users to determine which version
to link.

Reviewers: dblaikie, kpw, pelikan

Subscribers: mgorny, llvm-commits

Differential Revision: https://reviews.llvm.org/D36080

llvm-svn: 309909
2017-08-03 00:58:45 +00:00
Kostya Kortchinsky 0357e8de3b [tsan] Check for pvalloc overlow
Summary:
`CheckForPvallocOverflow` was introduced with D35818 to detect when pvalloc
would wrap when rounding up to the next multiple of the page size.

Add this check to TSan's pvalloc implementation.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D36245

llvm-svn: 309897
2017-08-02 22:47:54 +00:00
Sterling Augustine a0213850e5 Use a more standard method to mark these tests as unsupported on powerpc64.
llvm-svn: 309892
2017-08-02 21:52:23 +00:00
Kostya Kortchinsky 94380ed406 [msan] Check for pvalloc overflow
Summary:
CheckForPvallocOverflow was introduced with D35818 to detect when pvalloc
would wrap when rounding up to the next multiple of the page size.

Add this check to MSan's pvalloc implementation.

This time I made sure I was actually running (and writing) the correct tests,
and that they are passing...

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D36164

llvm-svn: 309883
2017-08-02 20:32:12 +00:00
Kostya Serebryany 1ca948a2b4 Add new ASAN_OPTION: sleep_after_init.
Summary: As mentioned in https://github.com/google/sanitizers/issues/834, suggested option can be handy for debugging.

Reviewers: kcc

Reviewed By: kcc

Subscribers: llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D35409

llvm-svn: 309854
2017-08-02 18:48:45 +00:00
Sterling Augustine f67036b62c This ppc64 implementation of clear_cache works for both big and little endian.
llvm-svn: 309848
2017-08-02 18:13:59 +00:00
Vedant Kumar 8adb8d5d0d [ubsan] Test -fsanitize=vptr without -fsanitize=null
This reverts commit r309042, thereby adding a test for -fsanitize=vptr
functionality without -fsanitize=null. It also removes -fsanitize=null
from another -fsanitize=vptr test.

llvm-svn: 309847
2017-08-02 18:10:36 +00:00
Vitaly Buka 411533009d [asan] Interceptors for Fuchsia
Summary:
Fuchsia uses the "memintrinsics" interceptors, though not via any
generalized interception mechanism.  It doesn't use any other interceptors.

Submitted on behalf of Roland McGrath.

Reviewers: vitalybuka, alekseyshl, kcc

Reviewed By: vitalybuka

Subscribers: kubamracek, phosek, filcab, llvm-commits

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D36189

llvm-svn: 309798
2017-08-02 07:59:30 +00:00
Vitaly Buka 137c9c2a6a [sanitizer_common] Fuchsia-specific implementation of SanitizerCoverage
Submitted on behalf of Roland McGrath.

Reviewers: kcc, eugenis, alekseyshl, vitalybuka

Reviewed By: kcc

Subscribers: filcab, vitalybuka, phosek, llvm-commits, kubamracek, mgorny

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D35866

llvm-svn: 309797
2017-08-02 07:51:38 +00:00
Dean Michael Berris 9952d95921 [XRay][compiler-rt] Remove use of std::mutex and std::shared_ptr from global scope.
Summary:
This change attempts to remove all the dependencies we have on
std::mutex and any std::shared_ptr construction in global variables. We
instead use raw pointers to these objects, and construct them on the
heap. In cases where it's possible, we lazily initialize these pointers.

While we do not have a replacement for std::shared_ptr yet in
compiler-rt, we use this work-around to avoid having to statically
initialize the objects as globals. Subsequent changes should allow us to
completely remove our dependency on std::shared_ptr and instead have our
own implementation of the std::shared_ptr and std::weak_ptr semantics
(or completely rewrite the implementaton to not need these
standard-library provided abstractions).

Reviewers: dblaikie, kpw, pelikan

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D36078

llvm-svn: 309792
2017-08-02 04:51:40 +00:00
Vitaly Buka f4891c2a66 [sanitizer_common] Fuchsia-specific symbolizer
Summary:
Fuchsia doesn't support built-in symbolization per se at all.
Instead, it always emits a Fuchsia-standard "symbolizer markup"
format that makes it possible for a post-processing filter to
massage the logs into symbolized format.  Hence, it does not
support user-specified formatting options for backtraces or other
symbolization.

Reviewers: vitalybuka, alekseyshl, kcc

Subscribers: kubamracek, mgorny, phosek, filcab, llvm-commits

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D36032

llvm-svn: 309760
2017-08-01 22:54:51 +00:00
Vitaly Buka 5d960ec420 [sanitizer_common] Fuchsia OS support code
Submitted on behalf of Roland McGrath.

Reviewers: vitalybuka, alekseyshl, kcc

Reviewed By: vitalybuka

Subscribers: cryptoad, srhines, kubamracek, mgorny, phosek, filcab, llvm-commits

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D36031

llvm-svn: 309756
2017-08-01 22:22:25 +00:00
Vitaly Buka 51ca757e61 [sanitizer_common] Disable filesystem-related code for Fuchsia
Summary:
Fuchsia doesn't support filesystem access per se at low level.
So it won't use any of the filesystem-oriented code in sanitizer_common.

Submitted on behalf of Roland McGrath.

Reviewers: vitalybuka, alekseyshl, kcc

Reviewed By: vitalybuka

Subscribers: kubamracek, phosek, filcab, llvm-commits

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D36029

llvm-svn: 309749
2017-08-01 21:28:39 +00:00
Vitaly Buka d4e03d59cb [sanitizer_common] Fuchsia support for interceptors
Summary:
Actually Fuchsia non-support for interceptors.  Fuchsia doesn't use
interceptors in the common sense at all.  Almost all system library
functions don't need interception at all, because the system
libraries are just themselves compiled with sanitizers enabled and
have specific hook interfaces where needed to inform the sanitizer
runtime about thread lifetimes and the like.  For the few functions
that do get intercepted, they don't use a generic mechanism like
dlsym with RTLD_NEXT to find the underlying system library function.
Instead, they use specific extra symbol names published by the
system library (e.g. __unsanitized_memcpy).

Submitted on behalf of Roland McGrath.

Reviewers: vitalybuka, alekseyshl, kcc, filcab

Reviewed By: filcab

Subscribers: kubamracek, phosek, filcab, llvm-commits

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D36028

llvm-svn: 309745
2017-08-01 21:15:19 +00:00
Vitaly Buka 6886061dab [ubsan] Enable UBSan build for Fuchsia
Submitted on behalf of Roland McGrath.

Reviewers: vitalybuka, alekseyshl, kcc

Reviewed By: vitalybuka

Subscribers: srhines, kubamracek, mgorny, phosek, filcab, llvm-commits

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D36033

llvm-svn: 309742
2017-08-01 20:48:36 +00:00
Sterling Augustine f6b2e770ea Revert rL309634 until upstream buildbots have upgraded libc.
llvm-svn: 309704
2017-08-01 17:16:05 +00:00
Vitaly Buka 2ae1edd139 [sanitizer-coverage] Fix Windows build broken by r309655
llvm-svn: 309665
2017-08-01 07:45:46 +00:00
Kostya Serebryany d6cfed6060 [sanitizer-coverage] dummy definitions for __sanitizer_cov_8bit_counters_init and __sanitizer_cov_pcs_init
llvm-svn: 309655
2017-08-01 04:24:05 +00:00
Evgeniy Stepanov 6ff4412375 [sancov] Fix coverage-reset test on Android/i686.
DSO coverage may be dumped in any order.

llvm-svn: 309639
2017-07-31 23:08:27 +00:00
Petr Hosek a14a2cc5ce [sanitizer] Fix the sanitizer build on Android
Android uses libgcc name even for shared library unlike other platforms
which use libgcc_s. Furthemore, Android libstdc++ has a dependency on
libdl. These need to be handled while performing CMake checks.

Differential Revision: https://reviews.llvm.org/D36035

llvm-svn: 309638
2017-07-31 22:46:43 +00:00
Kostya Kortchinsky 9d52cedb24 [msan] Reverting D36093
Summary:
Reverting D36093 until I can figure out how to launch the correct tests :/
My apologies.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D36120

llvm-svn: 309637
2017-07-31 22:46:01 +00:00
Sterling Augustine 81fd449238 Add powerpc64 to compiler-rt build infrastructure.
Summary: Add powerpc64 to compiler-rt build infrastructure.

Reviewers: timshen

Reviewed By: timshen

Subscribers: nemanjai, dberris, mgorny, aheejin, cfe-commits

Differential Revision: https://reviews.llvm.org/D36108

llvm-svn: 309634
2017-07-31 22:23:17 +00:00
Peter Collingbourne 36df72b50a Remove STL/microsoft-specific CFI blacklist entries
Patch by Vlad Tsyrklevich!

Differential Revision: https://reviews.llvm.org/D35855

llvm-svn: 309617
2017-07-31 20:39:32 +00:00
Kostya Kortchinsky 56f5f17349 [msan] Check for pvalloc overflow
Summary:
`CheckForPvallocOverflow` was introduced with D35818 to detect when pvalloc
would wrap when rounding up to the next multiple of the page size.

Add this check to MSan's pvalloc implementation.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D36093

llvm-svn: 309601
2017-07-31 18:45:17 +00:00
Dean Michael Berris 432fec23c5 [XRay][compiler-rt] Fix test to not be too strict with output order.
Follow-up to D35789.

llvm-svn: 309543
2017-07-31 06:58:09 +00:00
Vitaly Buka 0aeb0fe2ab [asan] Move shadow memory setup into its own file
Submitted on behalf of Roland McGrath.

Reviewers: vitalybuka, alekseyshl, kcc

Subscribers: kubamracek, mgorny, phosek, filcab, llvm-commits

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D36037

llvm-svn: 309542
2017-07-31 06:48:34 +00:00
Dean Michael Berris 8443f7ba64 [XRay][compiler-rt] Fix typo for REQUIRES.
Follow-up on D35789.

llvm-svn: 309540
2017-07-31 06:21:38 +00:00
Vitaly Buka 0a6464aa98 [sanitizer_common] Add SANITIZER_FUCHSIA
Summary: More changes to follow will add the Fuchsia port.

Submitted on behalf of Roland McGrath.

Reviewers: vitalybuka, alekseyshl, kcc

Reviewed By: vitalybuka

Subscribers: kubamracek, llvm-commits, phosek, filcab

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D36027

llvm-svn: 309539
2017-07-31 06:21:13 +00:00
Dean Michael Berris 598bc37e27 [XRay][compiler-rt] Require build-in-tree and x86_64-linux.
The quiet-start.cc test currently fails for arm (and potentially other
platforms). This change limits it to x86_64-linux.

Follow-up to D35789.

llvm-svn: 309538
2017-07-31 06:09:57 +00:00
Martin Storsjo adf4489009 [builtins] Fix mingw-w64 cross compilation
Lowercase the Windows.h include in enable_execute_stack.c, just as in
emutls.c in SVN r302340.

Differential Revision: https://reviews.llvm.org/D36066

llvm-svn: 309537
2017-07-31 06:01:39 +00:00
Vitaly Buka 8c809eb7cd [sanitizer_common] Rename SI_NOT_WINDOWS to SI_POSIX
Summary:
New systems might be neither Windows nor POSIX.  The SI_NOT_WINDOWS
macro in sanitizer_platform_interceptors.h was already effectively
the same as SI_POSIX, so just use SI_POSIX instead.

Submitted on behalf of Roland McGrath.

Reviewers: vitalybuka, alekseyshl, kcc

Reviewed By: vitalybuka

Subscribers: phosek, filcab, llvm-commits, kubamracek

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D36038

llvm-svn: 309536
2017-07-31 05:58:15 +00:00
Dean Michael Berris 5ca1955627 [XRay][compiler-rt] Do not print the warning when the binary is not XRay instrumented.
Summary:
Currently when the XRay runtime is linked into a binary that doesn't
have the instrumentation map, we print a warning unconditionally. This
change attempts to make this behaviour more quiet.

Reviewers: kpw, pelikan

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D35789

llvm-svn: 309534
2017-07-31 05:16:20 +00:00
Vedant Kumar 4fb45077c4 [ubsan] Diagnose invalid uses of builtins (compiler-rt)
Differential Revision: https://reviews.llvm.org/D34591

llvm-svn: 309461
2017-07-29 00:20:02 +00:00
Reid Kleckner 3bcad2b2e6 Try to fix asan test on sanitizer-windows
llvm-svn: 309440
2017-07-28 21:43:23 +00:00
Krzysztof Parzyszek 48a34c684b [compiler-rt] Add missing quotation marks to msan_compile invocation
llvm-svn: 309430
2017-07-28 20:29:29 +00:00
Matt Morehouse 74ddba0c95 Add end-to-end tests for overflows of byval arguments.
Summary:
Included is one test for passing structs by value and one test for
passing C++
objects by value.

Reviewers: eugenis, vitalybuka

Reviewed By: eugenis

Subscribers: srhines, kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D34827

llvm-svn: 309424
2017-07-28 19:52:31 +00:00
Sterling Augustine dd9173420f Add clear_cache implementation for ppc64. Fix buffer to meet ppc64 alignment.
llvm-svn: 309423
2017-07-28 19:49:22 +00:00
Matt Morehouse 6848e05d8b Revert r308677.
Incorrect directories were created by the patch.

llvm-svn: 309420
2017-07-28 19:11:16 +00:00
George Karpenkov a6f43a68d3 [sanitizers test CMake] further refactor testing CMake for tsan
TSan tests on Darwin first link all libraries into a static archive file.
With this change, the linking is done once per all architecture,
and previously the linking step was repeated per each architecture per
each add_tsan_test call.
Furthermore, the code is cleared up.

Differential Revision: https://reviews.llvm.org/D35913

llvm-svn: 309406
2017-07-28 17:38:44 +00:00
George Karpenkov 831875b4bf [sanitizer tests CMake] Factor out CMake logic for compiling sanitizer tests
Currently there's a large amount of CMake logic duplication for
compiling sanitizer tests.
If we add more sanitizers, the duplication will get even worse.

This change factors out common compilation commands into a macro
available to all sanitizers.

llvm-svn: 309405
2017-07-28 17:32:37 +00:00
Dehao Chen ab360d6015 Change INSTR_PROF_DEFAULT_NUM_VAL_PER_SITE from 8 to 16.
Summary: In the current implementation, the defaul number of values per site tracked by value profiler is 8, which is too small and could introduce inaccuracies to profile. Changing it to 16 will be able to gain more accurate value profiler.

Reviewers: davidxl, tejohnson

Reviewed By: tejohnson

Subscribers: sanjoy, llvm-commits

Differential Revision: https://reviews.llvm.org/D35964

llvm-svn: 309388
2017-07-28 15:00:30 +00:00
Petr Hosek a7a9ca4ad1 Support libc++abi in addition to libstdc++
This change adds sanitizer support for LLVM's libunwind and libc++abi
as an alternative to libstdc++. This allows using the in tree version
of libunwind and libc++abi which is useful when building a toolchain
for different target.

Differential Revision: https://reviews.llvm.org/D34501

llvm-svn: 309362
2017-07-28 03:39:39 +00:00
Petr Hosek 2bbb6ad217 Support compiler-rt builtins
This change adds support for compiler-rt builtins as an alternative
compiler runtime to libgcc.

Differential Revision: https://reviews.llvm.org/D35165

llvm-svn: 309361
2017-07-28 03:39:38 +00:00
Kostya Serebryany ce5e65ee93 [sanitizer-coverage] add a run-time test for -fsanitize-coverage=inline-8bit-counters,pc-table
llvm-svn: 309351
2017-07-28 01:38:43 +00:00
George Karpenkov 50dd3fe903 [sanitizers] Sanitizer tests CMake clean up: try #2
This patch addresses two issues:

Most of the time, hacks with `if/else` in order to get support for
multi-configuration builds are superfluous.
The variable `CMAKE_CFG_INTDIR` was created precisely for this purpose: it
expands to `.`  on all single-configuration builds, and to a configuration
name otherwise.
The `if/else` hacks for the library name generation should also not be
done, as CMake has `TARGET_FILE` generator expression precisely for this
purpose, as it expands to the exact filename of the resulting target.

Differential Revision: https://reviews.llvm.org/D35952

llvm-svn: 309341
2017-07-28 00:50:56 +00:00
Evgeniy Stepanov fea50c286a [sancov] Add missing line breaks in test. NFC.
llvm-svn: 309339
2017-07-28 00:22:42 +00:00
Evgeniy Stepanov 59b5e9c00c [sancov] Implement __sanitizer_cov_reset.
Summary: Clears all collected coverage.

Reviewers: kcc

Subscribers: srhines, kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D35958

llvm-svn: 309333
2017-07-27 23:22:37 +00:00
George Karpenkov 1bf535daae Revert "[sanitizers] Sanitizer tests CMake clean up"
This reverts commit 0ab44db2aa1cd3710355ad79b04f954ce68c0b3a.

Fails on some bots, reverting until I can fix it.

llvm-svn: 309318
2017-07-27 20:44:33 +00:00
George Karpenkov 165a1edc85 [sanitizers] Sanitizer tests CMake clean up
This patch addresses two issues:

Most of the time, hacks with `if/else` in order to get support for
multi-configuration builds are superfluous.
The variable `CMAKE_CFG_INTDIR` was created precisely for this purpose: it
expands to `.`  on all single-configuration builds, and to a configuration
name otherwise.
The `if/else` hacks for the library name generation should also not be
done, as CMake has `TARGET_FILE` generator expression precisely for this
purpose, as it expands to the exact filename of the resulting target.

Differential Revision: https://reviews.llvm.org/D35952

llvm-svn: 309306
2017-07-27 18:40:38 +00:00
Leo Li f10459893f Add liblog to ubsan `UBSAN_DYNAMIC_LIBS`
Summary: liblog is still required to create ubsan runtimes on Android when __ANDROID_API__ < 21.

Reviewers: eugenis, vsk

Subscribers: kubamracek, mgorny, pirama, srhines, llvm-commits

Differential Revision: https://reviews.llvm.org/D35915

llvm-svn: 309180
2017-07-26 21:16:15 +00:00
Kostya Serebryany f539bd84b6 [sanitizer-coverage] remove stale code
llvm-svn: 309173
2017-07-26 20:40:25 +00:00
Petr Hosek 3243a13f0a Revert "[sanitizer] Support compiler-rt builtins"
This reverts commit fd63314d6770e0da62572a3fea2c41c4cc0fc58a.

llvm-svn: 309083
2017-07-26 06:46:11 +00:00
Petr Hosek cc9559f515 Revert "[sanitizer] Support libc++abi in addition to libstdc++"
This reverts commit d1997bff31cf6b484eb59c2ee1fc3155442e338c.

llvm-svn: 309082
2017-07-26 06:46:10 +00:00
Petr Hosek c2c3d6b87b [sanitizer] Support libc++abi in addition to libstdc++
This change adds sanitizer support for LLVM's libunwind and libc++abi
as an alternative to libstdc++. This allows using the in tree version
of libunwind and libc++abi which is useful when building a toolchain
for different target.

Differential Revision: https://reviews.llvm.org/D34501

llvm-svn: 309074
2017-07-26 01:43:02 +00:00
Petr Hosek b1305ce038 [sanitizer] Support compiler-rt builtins
This change adds support for compiler-rt builtins as an alternative
compiler runtime to libgcc.

Differential Revision: https://reviews.llvm.org/D35165

llvm-svn: 309060
2017-07-25 23:38:25 +00:00
Vedant Kumar 2d8de203f7 [ubsan] Update a test missed in r309008, NFC
llvm-svn: 309042
2017-07-25 22:33:28 +00:00
Kostya Kortchinsky 65fdf677f2 [scudo] Check for pvalloc overflow
Summary:
Previously we were rounding up the size passed to `pvalloc` to the next
multiple of page size no matter what. There is an overflow possibility that
wasn't accounted for. So now, return null in the event of an overflow. The man
page doesn't seem to indicate the errno to set in this particular situation,
but the glibc unit tests go for ENOMEM (https://code.woboq.org/userspace/glibc/malloc/tst-pvalloc.c.html#54)
so we'll do the same.
Update the aligned allocation funtions tests to check for properly aligned
returned pointers, and the `pvalloc` corner cases.

@alekseyshl: do you want me to do the same in the other Sanitizers?

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: kubamracek, alekseyshl, llvm-commits

Differential Revision: https://reviews.llvm.org/D35818

llvm-svn: 309033
2017-07-25 21:18:02 +00:00
Vedant Kumar 612d918435 [ubsan] -fsanitize=vptr now requires -fsanitize=null, update tests
See: https://bugs.llvm.org/show_bug.cgi?id=33881
llvm-svn: 309008
2017-07-25 19:34:27 +00:00
Francis Ricci 0d920c4418 Only scan global sections containing data in LSan on darwin
Summary:
__DATA segments on Darwin contain a large number of separate sections,
many of which cannot actually contain pointers, and contain const values or
objc metadata. Not scanning sections which cannot contain pointers significantly
improves performance.

On a medium-sized (~4000 files) internal project, I saw a speedup of about 30%
in standalone LSan's execution time (30% improvement in the time spent running
LSan, not the total program time).

Reviewers: kcc, kubamracek, alekseyshl

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D35432

llvm-svn: 308999
2017-07-25 18:16:58 +00:00
Francis Ricci dafea67abd Fix unused variable warning with MemoryMappedSegment private data
llvm-svn: 308992
2017-07-25 17:28:41 +00:00
Vitaly Buka 2cb6f208b6 Revert "[compiler-rt] Include thread ID into sanitizers logs"
This improvement introduce additional dependencies on sandboxed environments.

This reverts commit r308637.

llvm-svn: 308984
2017-07-25 16:56:22 +00:00
Francis Ricci ec4ac0f0c6 Add address ranges for individual macho sections on darwin
Summary:
This is a re-upload of the reverted commit r308644. It has changed quite
a bit to reflect post-commit comments by kcc, so I'm re-uploading as
a new review.

Reviewers: kubamracek, alekseyshl, kcc

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D35799

llvm-svn: 308977
2017-07-25 15:27:32 +00:00
George Karpenkov c7d30e9c0a Add .pyc files to .gitignore to compiler-rt
During testing .pyc temporary files appear, which may be annoying.
Did not change SVN ignore, as it was heavily out of sync with GIT one.

Differential Revision: D35815

llvm-svn: 308931
2017-07-24 21:51:12 +00:00
Alex Shlyapnikov 132689243e [Sanitizers] TSan allocator set errno on failure.
Summary:
Set proper errno code on allocation failures and change realloc, pvalloc,
aligned_alloc, memalign and posix_memalign implementation to satisfy
their man-specified requirements.

Modify allocator API implementation to bring it closer to other
sanitizers allocators.

Reviewers: dvyukov

Subscribers: llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D35690

llvm-svn: 308929
2017-07-24 21:22:59 +00:00
George Karpenkov 680d102023 Splitting out test for Darwin for print-stack-trace:
New default symbolizer can not symbolize inlined function which appear
under -O3.

llvm-svn: 308925
2017-07-24 20:35:20 +00:00
Stephen Hines aba18768f4 [mips] Switch asm to __asm__ for non-GNU compiles.
Summary:
Using asm works fine for gnu11, but fails if the compiler uses C11.
Switch to the more consistent __asm__, since that is what the rest of
the source is using.

Reviewers: petarj

Reviewed By: petarj

Subscribers: llvm-commits, sdardis, arichardson, pirama

Differential Revision: https://reviews.llvm.org/D35756

llvm-svn: 308922
2017-07-24 20:25:08 +00:00
George Karpenkov b87543c704 Prefer atos to llvm-symbolizer on Darwin
atos is the default symbolizer on Apple's compiler for quite a few years now.
llvm-symbolizer is quite fragile on Darwin: for example, unless a .dSYM
file was explicitly generated symbolication would not work.
It is also very convenient when the behavior of LLVM open source
compiler matches to that of Apple's compiler on Apple's platform.
Furthermore, llvm-symbolizer is not installed on Apple's platform by
default, which leads to strange behavior during debugging: the test
might fail under lit (where it has llvm-symbolizer) but would run
properly when launched on the command line (where it does not, and atos
would be used).

Indeed, there's a downside: atos does not work properly with inlined
functions, hence the test change.
We do not think that this is a major problem, as users would often
compile with -O0 when debugging, and in any case it is preferable to
symbolizer not being able to symbolize.

Differential Revision: https://reviews.llvm.org/D35745

llvm-svn: 308908
2017-07-24 18:24:08 +00:00
Kostya Kortchinsky e6f1b7281f [scudo] Fix QuarantineChunksUpToSize failing test on AArch64
Summary:
Warm-up the other 2 sizes used by the tests, which should get rid of a failure
on AArch64.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: aemerson, rengolin, llvm-commits, kristof.beyls

Differential Revision: https://reviews.llvm.org/D35806

llvm-svn: 308907
2017-07-24 18:22:33 +00:00
Kostya Kortchinsky 2d94405a32 [scudo] Quarantine overhaul
Summary:
First, some context.

The main feedback we get about the quarantine is that it's too memory hungry.
A single MB of quarantine will have an impact of 3 to 4MB of PSS/RSS, and
things quickly get out of hand in terms of memory usage, and the quarantine
ends up disabled.

The main objective of the quarantine is to protect from use-after-free
exploitation by making it harder for an attacker to reallocate a controlled
chunk in place of the targeted freed chunk. This is achieved by not making it
available to the backend right away for reuse, but holding it a little while.

Historically, what has usually been the target of such attacks was objects,
where vtable pointers or other function pointers could constitute a valuable
targeti to replace. Those are usually on the smaller side. There is barely any
advantage in putting the quarantine several megabytes of RGB data or the like.

Now for the patch.

This patch introduces a new way the Quarantine behaves in Scudo. First of all,
the size of the Quarantine will be defined in KB instead of MB, then we
introduce a new option: the size up to which (lower than or equal to) a chunk
will be quarantined. This way, we only quarantine smaller chunks, and the size
of the quarantine remains manageable. It also prevents someone from triggering
a recycle by allocating something huge. We default to 512 bytes on 32-bit and
2048 bytes on 64-bit platforms.

In details, the patches includes the following:
- introduce `QuarantineSizeKb`, but honor `QuarantineSizeMb` if set to fall
  back to the old behavior (meaning no threshold in that case);
  `QuarantineSizeMb` is described as deprecated in the options descriptios;
  documentation update will follow;
- introduce `QuarantineChunksUpToSize`, the new threshold value;
- update the `quarantine.cpp` test, and other tests using `QuarantineSizeMb`;
- remove `AllocatorOptions::copyTo`, it wasn't used;
- slightly change the logic around `quarantineOrDeallocateChunk` to accomodate
  for the new logic; rename a couple of variables there as well;

Rewriting the tests, I found a somewhat annoying bug where non-default aligned
chunks would account for more than needed when placed in the quarantine due to
`<< MinAlignment` instead of `<< MinAlignmentLog`. This is fixed and tested for
now.

Reviewers: alekseyshl, kcc

Reviewed By: alekseyshl

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D35694

llvm-svn: 308884
2017-07-24 15:29:38 +00:00
Francis Ricci 4f82717643 [compiler-rt] Add missing const specifier to MemoryMappedSegment functions
llvm-svn: 308881
2017-07-24 14:31:01 +00:00
Vitaly Buka d4abe9e7d3 [sanitizer_common] Move filesystem-related code out of sanitizer_common.cc
Summary:
This is a pure refactoring change.  It just moves code that is
related to filesystem operations from sanitizer_common.{cc,h} to
sanitizer_file.{cc,h}.  This makes it cleaner to disable the
filesystem-related code for a new port that doesn't want it.

Submitted on behalf of Roland McGrath.

Reviewers: kcc, eugenis, alekseyshl

Reviewed By: alekseyshl

Subscribers: vitalybuka, llvm-commits, kubamracek, mgorny, phosek

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D35591

llvm-svn: 308819
2017-07-22 01:46:40 +00:00
Erich Keane a021743cfd Remove Bitrig: CompilerRT Changes
Bitrig code has been merged back to OpenBSD, thus the OS has been abandoned.

Differential Revision: https://reviews.llvm.org/D35709

llvm-svn: 308798
2017-07-21 22:47:46 +00:00
Dean Michael Berris d8e12128dc [XRay][compiler-rt] Update test to account for change in logging format.
Fixes build breakage for some bots after we've started logging both the
process id and the thread id.

llvm-svn: 308701
2017-07-21 00:30:04 +00:00
Vitaly Buka 6d8fb107b2 Add end-to-end tests for overflows of byval arguments.
Summary:
Included is one test for passing structs by value and one test for passing C++
objects by value.

Submitted on behalf of Matt Morehouse.

Reviewers: eugenis, vitalybuka

Reviewed By: eugenis

Subscribers: srhines, kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D34827

llvm-svn: 308677
2017-07-20 21:40:53 +00:00
Francis Ricci d08a0864e4 Revert "Add MemoryMappedSection struct for two-level memory map iteration"
This probably broke lib0 tsan unit test on 10.11 buildbots

This reverts commit 35ad307c385e384f47a7fb348c14b3602d3a33c4.

llvm-svn: 308676
2017-07-20 21:23:14 +00:00
Vitaly Buka 04787bbc98 [tsan] Update unittest for logging changes of r308637
llvm-svn: 308660
2017-07-20 20:06:00 +00:00
Vitaly Buka fddf535d48 [compiler-rt] Use NOINLE to shrink stack frames
llvm-svn: 308654
2017-07-20 18:54:26 +00:00
Vitaly Buka a19a2762e3 [compiler-rt] Reorder functions to shrink stack frames
llvm-svn: 308652
2017-07-20 18:47:01 +00:00
Vitaly Buka 72b40fbf4d Revert "[compiler-rt] Reorder functions to have smaller stack frames"
Does not compile.

This reverts commit r308650.

llvm-svn: 308651
2017-07-20 18:43:56 +00:00
Vitaly Buka b49c933338 [compiler-rt] Reorder functions to have smaller stack frames
llvm-svn: 308650
2017-07-20 18:43:09 +00:00
Vitaly Buka 094684c155 Revert "[sanitizer_common] Move filesystem-related code out of sanitizer_common.cc"
Breaks Windows build.

This reverts commit r308640.

llvm-svn: 308648
2017-07-20 18:24:45 +00:00
Vitaly Buka 1eb9595c53 [asan] Update unittest for changes in logging r308637
llvm-svn: 308647
2017-07-20 18:20:12 +00:00
Francis Ricci 0ba09182bd Add MemoryMappedSection struct for two-level memory map iteration
Summary: This will allow sanitizer_procmaps on mac to expose section information.

Reviewers: kubamracek, alekseyshl, kcc

Subscribers: llvm-commits, emaste

Differential Revision: https://reviews.llvm.org/D35422

llvm-svn: 308644
2017-07-20 18:06:02 +00:00
Alex Shlyapnikov 89e85d1717 [sanitizer_common] Move filesystem-related code out of sanitizer_common.cc
This is a pure refactoring change. It just moves code that is
related to filesystem operations from sanitizer_common.{cc,h} to
sanitizer_file.{cc,h}. This makes it cleaner to disable the
filesystem-related code for a new port that doesn't want it.

Commiting for mcgrathr.

Reviewers: alekseyshl

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D35591

llvm-svn: 308640
2017-07-20 17:48:42 +00:00
Vitaly Buka bedb2601c1 [compiler-rt] Include thread ID into sanitizers logs
Reviewers: kcc, alekseyshl

Subscribers: kubamracek, llvm-commits, dberris

Differential Revision: https://reviews.llvm.org/D35654

llvm-svn: 308637
2017-07-20 17:31:08 +00:00
Kamil Rytarowski fc1651cb1a Honour 80-character line limit
llvm-svn: 308620
2017-07-20 14:15:00 +00:00
Kamil Rytarowski d9e9e44baa Add NetBSD support in sanitizer_libignore.cc
Summary:
Reuse Linux, FreeBSD and Apple code - no NetBSD specific changes.

Part of the code inspired by the original work on libsanitizer in GCC 5.4 by Christos Zoulas.
    
Sponsored by <The NetBSD Foundation>

Reviewers: joerg, vitalybuka, filcab, kcc

Reviewed By: filcab

Subscribers: emaste, kubamracek, llvm-commits

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D35628

llvm-svn: 308616
2017-07-20 13:33:06 +00:00
Kamil Rytarowski 734acfda0f Add NetBSD support in sanitizer_procmaps.h
Summary:
Reuse Linux and FreeBSD - no NetBSD specific changes.

Part of the code inspired by the original work on libsanitizer in GCC 5.4 by Christos Zoulas.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, filcab, kcc, vitalybuka

Reviewed By: filcab

Subscribers: llvm-commits, emaste, kubamracek, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D35629

llvm-svn: 308615
2017-07-20 13:31:06 +00:00
Kamil Rytarowski bd82d28413 Add NetBSD support in sanitizer_procmaps_common.cc
Summary:
Reuse Linux and FreeBSD code - no NetBSD specific changes.

Part of the code inspired by the original work on libsanitizer in GCC 5.4 by Christos Zoulas.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, kcc, vitalybuka, filcab

Reviewed By: filcab

Subscribers: emaste, kubamracek, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D35632

llvm-svn: 308614
2017-07-20 13:28:28 +00:00
Vitaly Buka 7af8fa41d3 [compiler-rt] Replace VPrintf with VReport in sanitizer_tls_get_addr.cc
Summary:
Thread id will be added to VRerort. Having thread here is useful.
This is also common place for logging for all sanitizers, so I can use this in
common test.

Reviewers: kcc, alekseyshl

Subscribers: kubamracek, llvm-commits, dberris

Differential Revision: https://reviews.llvm.org/D35655

llvm-svn: 308578
2017-07-20 01:36:16 +00:00
Alex Shlyapnikov 36a1c17dff [asan] Move memcpy, memmove, memset code out of asan_interceptors.cc
This is a pure refactoring change. It simply moves all the code and
macros related to defining the ASan interceptor versions of memcpy,
memmove, and memset into a separate file. This makes it cleaner to
disable all the other interceptor code while still using these three,
for a port that defines these but not the other common interceptors.

Reviewers: alekseyshl

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D35590

llvm-svn: 308575
2017-07-20 01:29:01 +00:00
Craig Topper b77279083c [X86][builtins] Sync getX86CpuIDAndInfoEx with llvm's Host.cpp again.
We now use __cpuidex intrinsics intead of inline asm on 32-bit Windows. We already used it on 64-bit.

llvm-svn: 308420
2017-07-19 05:11:20 +00:00
Francis Ricci bf60f68c2c Revert "Add MemoryMappedSection struct for two-level memory map iteration"
This reverts commit c8095ce74118dee8544b0f1ffaba8f46aa10215c.

Reverted due to some buildbot timeouts, perhaps due to 10.11 issues.

llvm-svn: 308395
2017-07-18 23:51:47 +00:00
Francis Ricci 1b5f773859 Revert "Only scan global sections containing data in LSan on darwin"
This reverts commit 7e46d78d47832f03ce42adcf56417fbfd47cbaad.

llvm-svn: 308394
2017-07-18 23:51:44 +00:00
Francis Ricci e43f27666d Don't call exit() from atexit handlers on Darwin
Summary:
Calling exit() from an atexit handler is undefined behavior.
On Linux, it's unavoidable, since we cannot intercept exit (_exit isn't called
if a user program uses return instead of exit()), and I haven't
seen it cause issues regardless.

However, on Darwin, I have a fairly complex internal test that hangs roughly
once in every 300 runs after leak reporting finishes, which is resolved with
this patch, and is presumably due to the undefined behavior (since the Die() is
the only thing that happens after the end of leak reporting).

In addition, this is the way TSan works as well, where an atexit handler+Die()
is used on Linux, and an _exit() interceptor is used on Darwin. I'm not sure if it's
intentionally structured that way in TSan, since TSan sets up the atexit handler and the
_exit() interceptor on both platforms, but I have observed that on Darwin, only the
_exit() interceptor is used, and on Linux the atexit handler is used.

There is some additional related discussion here: https://reviews.llvm.org/D35085

Reviewers: alekseyshl, kubamracek

Subscribers: eugenis, vsk, llvm-commits

Differential Revision: https://reviews.llvm.org/D35513

llvm-svn: 308353
2017-07-18 20:18:32 +00:00
Alex Shlyapnikov 42bea018af [Sanitizers] ASan/MSan/LSan allocators set errno on failure.
Summary:
ASan/MSan/LSan allocators set errno on allocation failures according to
malloc/calloc/etc. expected behavior.

MSan allocator was refactored a bit to make its structure more similar
with other allocators.

Also switch Scudo allocator to the internal errno definitions.

TSan allocator changes will follow.

Reviewers: eugenis

Subscribers: llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D35275

llvm-svn: 308344
2017-07-18 19:11:04 +00:00
Vitaly Buka 013317d9d5 [asan] Remove recent asan tests which expect death in allocator
These tests assume allocator_may_return_null=false
If allocator_may_return_null=true, gtest would not be able to switch it.
Tests needs to be re-implemented as lit tests.

llvm-svn: 308254
2017-07-18 01:39:56 +00:00
Francis Ricci 7096b08cd7 Only scan global sections containing data in LSan on darwin
Summary:
__DATA segments on Darwin contain a large number of separate sections,
most of which cannot actually contain pointers, and contain const values or
objc metadata. Only scanning sections which can contain pointers greatly improves
performance.

On a medium-sized (~4000 files) internal project, I saw a speedup of about 50%
in standalone LSan's execution time (50% improvement in the time spent running
LSan, not the total program time).

Reviewers: kcc, kubamracek, alekseyshl

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D35432

llvm-svn: 308231
2017-07-17 23:03:03 +00:00
Vitaly Buka 4a8f16ec9d [sanitizers] Make sure that all complex macros uses parenthesis
Summary:
Without them expressions like this may have different values.
(SANITIZER_INTERCEPT_MEMRCHR && SANITIZER_INTERCEPT_PREADV)

Reviewers: alekseyshl

Subscribers: srhines, kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D35512

llvm-svn: 308228
2017-07-17 22:49:46 +00:00
Kamil Rytarowski e9140e5b95 Add missing && to fix syntax.
llvm-svn: 308221
2017-07-17 21:09:34 +00:00
Kamil Rytarowski 20f5a5c64c Add NetBSD support in platform_interceptors.h
Summary:
Introduce SI_NETBSD for NetBSD.

Add NetBSD support for appropriate `SANITIZER_INTERCEPT_*`.

Part of the code inspired by the original work on libsanitizer in GCC 5.4 by Christos Zoulas.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, dim, kcc, alekseyshl, filcab, eugenis, vitalybuka

Reviewed By: vitalybuka

Subscribers: srhines, kubamracek, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D35468

llvm-svn: 308217
2017-07-17 20:49:13 +00:00
Kamil Rytarowski 27169a7709 Introduce SANITIZER_NETBSD in sanitizer_platform.h
Summary:
Add defines for new NetBSD: SANITIZER_NETBSD,
it will be used across the codebase for sanitizers.

NetBSD is a POSIX-like platform, add it to SANITIZER_POSIX.

Part of the code inspired by the original work on libsanitizer in GCC 5.4 by Christos Zoulas.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, kcc, dim, alekseyshl, filcab, eugenis, vitalybuka

Reviewed By: kcc

Subscribers: kubamracek, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D35467

llvm-svn: 308216
2017-07-17 20:47:53 +00:00
Francis Ricci 455ba3fd79 Add MemoryMappedSection struct for two-level memory map iteration
Summary: This will allow sanitizer_procmaps on mac to expose section information.

Reviewers: kubamracek, alekseyshl, kcc

Subscribers: llvm-commits, emaste

Differential Revision: https://reviews.llvm.org/D35422

llvm-svn: 308210
2017-07-17 20:09:20 +00:00
Vitaly Buka 923ddad4c6 [asan] Disable not working new test on Windows
llvm-svn: 308123
2017-07-16 01:28:40 +00:00
Vitaly Buka d8afaeb866 [msan] Add missing include for fix test on Windows
llvm-svn: 308122
2017-07-16 01:19:35 +00:00
Vitaly Buka a016feb7a1 [tsan] Disable test with debug runtime
Test expects at least -O1 compiled runtime.

llvm-svn: 308121
2017-07-16 01:05:37 +00:00
Vitaly Buka 3f25b53485 [asan] Attempt to fix test in Windows after r308064
llvm-svn: 308120
2017-07-16 00:40:40 +00:00
Vitaly Buka 50648af623 [compiler-rt] Fix fix format specifies type in test
llvm-svn: 308117
2017-07-16 00:17:11 +00:00
George Karpenkov e89d0a8f5c [compiler-rt] [CMake] Build compiler-rt with no optimizations if the flag says so
Differential Revision: https://reviews.llvm.org/D35400

llvm-svn: 308072
2017-07-15 00:30:46 +00:00
Alex Shlyapnikov 31e8173c94 [Sanitizers] ASan and LSan allocator set errno on failure.
Summary:
Set proper errno code on alloction failures and change some
implementations to satisfy their man-specified requirements:

LSan: valloc and memalign
ASan: pvalloc, memalign and posix_memalign

Changing both allocators in one patch since LSan depends on ASan allocator in some configurations.

Reviewers: vitalybuka

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D35440

llvm-svn: 308064
2017-07-14 22:23:47 +00:00
Alex Shlyapnikov d08c32b2f4 [Sanitizers] LSan allocator set errno on failure.
Set proper errno code on alloction failures and change valloc and
memalign implementations to satisfy their man-specified requirements.

llvm-svn: 308063
2017-07-14 22:23:46 +00:00
Alex Shlyapnikov df18cbba55 [Sanitizers] Scudo allocator set errno on failure.
Summary:
Set proper errno code on alloction failure and change pvalloc and
posix_memalign implementation to satisfy their man-specified
requirements.

Reviewers: cryptoad

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D35429

llvm-svn: 308053
2017-07-14 21:17:16 +00:00
Dmitry Vyukov 9f2c6207d5 tsan: optimize sync clock memory consumption
This change implements 2 optimizations of sync clocks that reduce memory consumption:

Use previously unused first level block space to store clock elements.
Currently a clock for 100 threads consumes 3 512-byte blocks:

2 64-bit second level blocks to store clock elements
+1 32-bit first level block to store indices to second level blocks
Only 8 bytes of the first level block are actually used.
With this change such clock consumes only 2 blocks.

Share similar clocks differing only by a single clock entry for the current thread.
When a thread does several release operations on fresh sync objects without intervening
acquire operations in between (e.g. initialization of several fields in ctor),
the resulting clocks differ only by a single entry for the current thread.
This change reuses a single clock for such release operations. The current thread time
(which is different for different clocks) is stored in dirty entries.

We are experiencing issues with a large program that eats all 64M clock blocks
(32GB of non-flushable memory) and crashes with dense allocator overflow.
Max number of threads in the program is ~170 which is currently quite unfortunate
(consume 4 blocks per clock). Currently it crashes after consuming 60+ GB of memory.
The first optimization brings clock block consumption down to ~40M and
allows the program to work. The second optimization further reduces block consumption
to "modest" 16M blocks (~8GB of RAM) and reduces overall RAM consumption to ~30GB.

Measurements on another real world C++ RPC benchmark show RSS reduction
from 3.491G to 3.186G and a modest speedup of ~5%.

Go parallel client/server HTTP benchmark:
https://github.com/golang/benchmarks/blob/master/http/http.go
shows RSS reduction from 320MB to 240MB and a few percent speedup.

Reviewed in https://reviews.llvm.org/D35323

llvm-svn: 308018
2017-07-14 11:30:06 +00:00
Kostya Serebryany c56d444b75 Fix sanitizer build against latest glibc
Summary:
libsanitizer doesn't build against latest glibc anymore, see https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81066 for details.
One of the changes is that stack_t changed from typedef struct sigaltstack { ... } stack_t; to typedef struct { ... } stack_t; for conformance reasons.
And the other change is that the glibc internal __need_res_state macro is now ignored, so when doing
```
#define __need_res_state
#include <resolv.h>
```
the effect is now the same as just
```
#include <resolv.h>
```
and thus one doesn't get just the
```
struct __res_state { ... };
```
definition, but newly also the
```
extern struct __res_state *__res_state(void) __attribute__ ((__const__));
```
prototype.  So __res_state is no longer a type, but a function.

Reviewers: kcc, ygribov

Reviewed By: kcc

Subscribers: kubamracek

Differential Revision: https://reviews.llvm.org/D35246

llvm-svn: 307969
2017-07-13 21:59:01 +00:00
Kostya Kortchinsky b44364dd15 [scudo] Do not grab a cache for secondary allocation & per related changes
Summary:
Secondary backed allocations do not require a cache. While it's not necessary
an issue when each thread has its cache, it becomes one with a shared pool of
caches (Android), as a Secondary backed allocation or deallocation holds a
cache that could be useful to another thread doing a Primary backed allocation.

We introduce an additional PRNG and its mutex (to avoid contention with the
Fallback one for Primary allocations) that will provide the `Salt` needed for
Secondary backed allocations.

I changed some of the code in a way that feels more readable to me (eg: using
some values directly rather than going  through ternary assigned variables,
using directly `true`/`false` rather than `FromPrimary`). I will let reviewers
decide if it actually is.

An additional change is to mark `CheckForCallocOverflow` as `UNLIKELY`.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D35358

llvm-svn: 307958
2017-07-13 21:01:19 +00:00
Vedant Kumar 965f23fe84 [ubsan] Teach the pointer overflow check that "p - <unsigned> <= p" (compiler-rt)
Compiler-rt changes associated with: D34121

llvm-svn: 307956
2017-07-13 20:55:41 +00:00
Kuba Mracek f0c652a52e For Darwin's GetTaskInfoMaxAddress, define the real structure with real fields to avoid confusion. NFC.
llvm-svn: 307945
2017-07-13 20:02:45 +00:00
Craig Topper e6b07073ca [compiler-rt][X86] Match the detection of cpu's for __cpu_model to the latest version of gcc
Summary:
We were missing many feature flags that newer gcc supports and we had our own set of feature flags that gcc didnt' support that were overlapping. Clang's implementation assumes gcc's features list so a mismatch here is problematic.

I've also matched the cpu type/subtype lists with gcc and removed all the cpus that gcc doesn't support. I've also removed the fallback autodetection logic that was taken from Host.cpp. It was the main reason we had extra feature flags relative to gcc. I don't think gcc does this in libgcc.

Once this support is in place we can consider implementing __builtin_cpu_is in clang. This could also be needed for function dispatching that Erich Keane is working on.

Reviewers: echristo, asbirlea, RKSimon, erichkeane, zvi

Reviewed By: asbirlea

Subscribers: dberris, llvm-commits

Differential Revision: https://reviews.llvm.org/D35214

llvm-svn: 307878
2017-07-13 02:56:24 +00:00
Xinliang David Li 028c78b593 Fix broken test
llvm-svn: 307869
2017-07-13 00:22:01 +00:00
Vitaly Buka 9fd492e71e [asan] Avoid recompilation of the same code in the test
llvm-svn: 307868
2017-07-12 23:59:22 +00:00
Kuba Mracek c1e903be19 [asan] For iOS/AArch64, if the dynamic shadow doesn't fit, restrict the VM space
On iOS/AArch64, the address space is very limited and has a dynamic maximum address based on the configuration of the device. We're already using a dynamic shadow, and we find a large-enough "gap" in the VM where we place the shadow memory. In some cases and some device configuration, we might not be able to find a large-enough gap: E.g. if the main executable is linked against a large number of libraries that are not part of the system, these libraries can fragment the address space, and this happens before ASan starts initializing.

This patch has a solution, where we have a "backup plan" when we cannot find a large-enough gap: We will restrict the address space (via MmapFixedNoAccess) to a limit, for which the shadow limit will fit.

Differential Revision: https://reviews.llvm.org/D35098

llvm-svn: 307865
2017-07-12 23:29:21 +00:00
Xinliang David Li 07df59b7b6 [PGO] Add a test for 2-deep loop nest
llvm-svn: 307864
2017-07-12 23:28:45 +00:00
Kuba Mracek 7004b7cf0d Add explicit CMake targets for ASan/TSan iOS Simulator testing and update the instructions how to run them.
llvm-svn: 307844
2017-07-12 21:02:48 +00:00
Kuba Mracek 4c4ea1d1ba On Darwin, start building the TSan iOS dylib by default.
llvm-svn: 307839
2017-07-12 20:25:14 +00:00
Petr Hosek 969518b442 [builtins] Better Fuchsia support
Add Fuchsia support to some builtings and avoid building builtins
that are not and will never be used on Fuchsia.

Differential Revision: https://reviews.llvm.org/D34075

llvm-svn: 307832
2017-07-12 19:33:30 +00:00
Kuba Mracek 46f2b8485a On Darwin, start building the TSan dylib for the iOS simulator.
llvm-svn: 307816
2017-07-12 17:30:54 +00:00
Kuba Mracek 476c3e9ba0 Fix the declaration of DARWIN_PREFER_PUBLIC_SDK cmake variable (move before the return).
llvm-svn: 307815
2017-07-12 17:11:53 +00:00
Kostya Kortchinsky 00582563be [scudo] PRNG makeover
Summary:
This follows the addition of `GetRandom` with D34412. We remove our
`/dev/urandom` code and use the new function. Additionally, change the PRNG for
a slightly faster version. One of the issues with the old code is that we have
64 full bits of randomness per "next", using only 8 of those for the Salt and
discarding the rest. So we add a cached u64 in the PRNG that can serve up to
8 u8 before having to call the "next" function again.

During some integration work, I also realized that some very early processes
(like `init`) do not benefit from `/dev/urandom` yet. So if there is no
`getrandom` syscall as well, we have to fallback to some sort of initialization
of the PRNG.

Now a few words on why XoRoShiRo and not something else. I have played a while
with various PRNGs on 32 & 64 bit platforms. Some results are below. LCG 32 & 64
are usually faster but produce respectively 15 & 31 bits of entropy, meaning
that to get a full 64-bit, you would need to call them several times. The simple
XorShift is fast, produces 32 bits but is mediocre with regard to PRNG test
suites, PCG is slower overall, and XoRoShiRo is faster than XorShift128+ and
produces full 64 bits.

%%%
root@tulip-chiphd:/data # ./randtest.arm
[+] starting xs32...
[?] xs32 duration: 22431833053ns
[+] starting lcg32...
[?] lcg32 duration: 14941402090ns
[+] starting pcg32...
[?] pcg32 duration: 44941973771ns
[+] starting xs128p...
[?] xs128p duration: 48889786981ns
[+] starting lcg64...
[?] lcg64 duration: 33831042391ns
[+] starting xos128p...
[?] xos128p duration: 44850878605ns

root@tulip-chiphd:/data # ./randtest.aarch64
[+] starting xs32...
[?] xs32 duration: 22425151678ns
[+] starting lcg32...
[?] lcg32 duration: 14954255257ns
[+] starting pcg32...
[?] pcg32 duration: 37346265726ns
[+] starting xs128p...
[?] xs128p duration: 22523807219ns
[+] starting lcg64...
[?] lcg64 duration: 26141304679ns
[+] starting xos128p...
[?] xos128p duration: 14937033215ns
%%%

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: aemerson, kristof.beyls, llvm-commits

Differential Revision: https://reviews.llvm.org/D35221

llvm-svn: 307798
2017-07-12 15:29:08 +00:00
Dmitry Vyukov 1fa729999b tsan: remove some clock-related stats
The stats are too dependent on implementation
and won't be relevant in future.

llvm-svn: 307786
2017-07-12 12:54:38 +00:00
Dmitry Vyukov 62b9ad718f tsan: refactor SyncClock code
1. Add SyncClock::ResetImpl which removes code
   duplication between ctor and Reset.
2. Move SyncClock::Resize to SyncClock methods,
   currently it's defined between ThreadClock methods.

llvm-svn: 307785
2017-07-12 12:50:36 +00:00
Dmitry Vyukov 5f924089e5 tsan: prepare clock for future changes
Pass ClockCache to ThreadClock::set and introduce ThreadCache::ResetCached.
For now both are unused, but will reduce future diffs.

llvm-svn: 307784
2017-07-12 12:45:20 +00:00
Dmitry Vyukov 293b4fc04e tsan: s/-1/kInvalidTid/
llvm-svn: 307781
2017-07-12 12:36:44 +00:00
Dmitry Vyukov 5abf9bb1ca tsan: give debug names to dense allocators
Improves crash message on dense alloc overflow.
Allows to understand what alloc overflowed.

llvm-svn: 307780
2017-07-12 12:34:12 +00:00
Dmitry Vyukov 7bf9ee041a tsan: don't create sync objects on acquire-load
Don't create sync object if it does not exist yet. For example, an atomic
pointer is initialized to nullptr and then periodically acquire-loaded.

llvm-svn: 307778
2017-07-12 12:28:23 +00:00
Dmitry Vyukov 0a9265c26c tsan: add another test for clock growth
llvm-svn: 307777
2017-07-12 12:25:03 +00:00
Dmitry Vyukov 05d297080e tsan: add test for __tsan_java_find
The test should have been added in 289682
"tsan: allow Java VM iterate over allocated objects"
but I forgot to avn add.

Author: Alexander Smundak (asmundak)
Reviewed in https://reviews.llvm.org/D27720

llvm-svn: 307776
2017-07-12 12:23:31 +00:00
Vitaly Buka 11fe7beda9 [tsan] Update test to r307338
r307338 enabled new optimization reducing number of operation in tested functions.
There is no any performance regression detectable with TsanRtlTest DISABLED_BENCH.Mop* tests.

llvm-svn: 307739
2017-07-12 00:14:05 +00:00
Francis Ricci 9fd8a6381c Use internal_strncpy to copy filename in linux procmaps
Cleaner than using a while loop to copy the string character by character.

Reviewers: alekseyshl, glider

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D35136

llvm-svn: 307696
2017-07-11 19:40:54 +00:00
Francis Ricci edd53cb652 Inline function to get mac segment address range
Summary:
This function is only called once and is fairly simple. Inline to
keep API simple.

Reviewers: alekseyshl, kubamracek

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D35270

llvm-svn: 307695
2017-07-11 19:40:53 +00:00
Francis Ricci f6a4329b7d Refactor MemoryMappingLayout::Next to use a single struct instead of output parameters. NFC.
Summary:
This is the first in a series of patches to refactor sanitizer_procmaps
to allow MachO section information to be exposed on darwin.

In addition, grouping all segment information in a single struct is
cleaner than passing it through a large set of output parameters, and
avoids the need for annotations of NULL parameters for unneeded
information.

The filename string is optional and must be managed and supplied by the
calling function. This is to allow the MemoryMappedSegment struct to be
stored on the stack without causing overly large stack sizes.

Reviewers: alekseyshl, kubamracek, glider

Subscribers: emaste, llvm-commits

Differential Revision: https://reviews.llvm.org/D35135

llvm-svn: 307688
2017-07-11 18:54:00 +00:00
Evgeniy Stepanov b76e4d12f4 [asan] Fix asan_device_setup --use-su mode on Android 7.x.
mount command does not accept -o remount,rw flag on some versions of Android.
mount -o rw,remount works everywhere.

llvm-svn: 307685
2017-07-11 18:18:50 +00:00
Petar Jovanovic adf4ce3a2c Remove duplicate assignments in stat64/kernel_stat_to_stat functions
Remove duplicate assignments in stat64_to_stat() and kernel_stat_to_stat().

llvm-svn: 307657
2017-07-11 13:03:48 +00:00
George Karpenkov 0b4ebb1d0b Do not crash with missing symbolication when running in DEDUP mode
Printing stacktrace from ASAN crashes with a segfault in DEDUP mode when
symbolication is missing.

Differential Revision: https://reviews.llvm.org/D34914

llvm-svn: 307577
2017-07-10 20:06:06 +00:00
Kuba Mracek 7fa20ac847 Add an #if SANITIZER_MAC and a comment to lsan_common's suppression for "_os_trace".
llvm-svn: 307567
2017-07-10 18:55:33 +00:00
Craig Topper 76ac813790 [X86] Sync a few more things from llvm's Host.cpp to cpu_model.
This syncs the cpuid functions and adds breaks to the appropriate spots in the CPU decoding switches. Also a few formatting tweaks.

llvm-svn: 307561
2017-07-10 17:47:23 +00:00
Craig Topper 61d8450277 [X86] Resync cpu_model.c with llvm's Host.cpp in preparation for making it compatible with newer gcc.
llvm-svn: 307558
2017-07-10 17:30:20 +00:00
Kuba Mracek 21ad3fd3c5 Fix-up for r307537: We need to #include stdint.h to get int32_t.
llvm-svn: 307557
2017-07-10 17:11:52 +00:00
Kuba Mracek dbfd7f35ff Fix a build failure due to r307541 (tsan_rtl_aarch64.S:54: Error: unknown pseudo-op: `.').
llvm-svn: 307549
2017-07-10 15:58:22 +00:00
Kuba Mracek 6496d92dd2 [lsan] Add _os_trace into LSan's suppression list
Differential Revision: https://reviews.llvm.org/D35173

llvm-svn: 307548
2017-07-10 15:55:25 +00:00
Kuba Mracek 9ae34c2d3f Fix-up for r307540.
llvm-svn: 307547
2017-07-10 15:52:30 +00:00
Kuba Mracek 7cd7c1a7b5 [tsan] Add comments for the bool argument of ThreadIgnoreBegin/ThreadIgnoreSyncBegin, NFC.
Differential Revision: https://reviews.llvm.org/D35134

llvm-svn: 307545
2017-07-10 15:37:13 +00:00
Kuba Mracek f20cb690b5 [tsan] Add a mapping for Darwin/AArch64
This patch defines the TSan memory map and offsets for Darwin on AArch64.

Differential Revision: https://reviews.llvm.org/D35147

llvm-svn: 307544
2017-07-10 15:35:04 +00:00
Kuba Mracek 8bd2cca391 [tsan] Port setjmp/longjmp assembly to Darwin/AArch64
This patch ports the assembly file implementing TSan's setjmp support to AArch64 on Darwin.

Differential Revision: https://reviews.llvm.org/D35143

llvm-svn: 307541
2017-07-10 15:28:16 +00:00
Kuba Mracek 8b225456c2 [tsan] Add a max VM address check for Darwin/AArch64
Differential Revision: https://reviews.llvm.org/D35154

llvm-svn: 307540
2017-07-10 15:24:48 +00:00
Kuba Mracek 989513be94 Reverting an accidentally landed change.
llvm-svn: 307539
2017-07-10 15:20:50 +00:00
Kuba Mracek 872cb7086c Fixup whitespace.
llvm-svn: 307538
2017-07-10 15:19:14 +00:00
Kuba Mracek 91f3fa5f31 [tsan] Add support for running TSan tests on iOS simulator and devices
Differential Revision: https://reviews.llvm.org/D35157

llvm-svn: 307537
2017-07-10 15:00:55 +00:00
Kuba Mracek 5c2e3e2b75 Make sure SANITIZER_MIN_OSX_VERSION is defined before using it.
llvm-svn: 307448
2017-07-07 22:40:13 +00:00
Kuba Mracek afbde43a8f Update buildgo.sh to pass -isysroot on Darwin.
llvm-svn: 307443
2017-07-07 21:17:29 +00:00
Kuba Mracek 0fa9d68740 Fix-up for r307307: vm_info.max_address is the first non-addressable pointer, so we need to subtract one.
llvm-svn: 307408
2017-07-07 15:32:44 +00:00
Evgeniy Stepanov 7d3eeaaa96 Revert r307342, r307343.
Revert "Copy arguments passed by value into explicit allocas for ASan."
Revert "[asan] Add end-to-end tests for overflows of byval arguments."

Build failure on lldb-x86_64-ubuntu-14.04-buildserver.
Test failure on clang-cmake-aarch64-42vma and sanitizer-x86_64-linux-android.

llvm-svn: 307345
2017-07-07 01:31:23 +00:00
Kuba Mracek 7a493393b3 [cmake] Cache results of find_darwin_sdk_dir
This improves find_darwin_sdk_dir to cache the results of executing xcodebuild to find the SDK. Should significantly reduce the CMake re-configure time.

Differential Revision: https://reviews.llvm.org/D34736

llvm-svn: 307344
2017-07-07 01:06:20 +00:00
Evgeniy Stepanov 03542db81c [asan] Add end-to-end tests for overflows of byval arguments.
Included is one test for passing structs by value and one test for passing C++
objects by value.

Patch by Matt Morehouse.

Differential revision: https://reviews.llvm.org/D34827

llvm-svn: 307343
2017-07-07 00:48:38 +00:00
Kuba Mracek 935243486c [cmake] Add an option to prefer public SDK in find_darwin_sdk_dir
Adds a CMake option DARWIN_PREFER_PUBLIC_SDK, off by default. When on, this prefers to use the public SDK, even when an internal one is present. With this, it's easy to emulate a build that the public buildbots are doing.

Differential Revision: https://reviews.llvm.org/D35071

llvm-svn: 307330
2017-07-06 23:09:16 +00:00
Reid Kleckner da565bc97c [asan] Fix -Winvalid-paste error with clang-cl
We don't need to paste tokens here. String literal concatenation works
just fine here with MSVC and Clang.

llvm-svn: 307321
2017-07-06 21:51:32 +00:00
Kuba Mracek 560440975a Fix whitespace lint issue (introduced in r307307).
llvm-svn: 307309
2017-07-06 20:38:33 +00:00
Kuba Mracek e9b5857db3 [sanitizer] Use TASK_VM_INFO to get the maximum VM address on iOS/AArch64
We currently hardcode the maximum VM address on iOS/AArch64, which is not really correct and this value changes between device configurations. Let's use TASK_VM_INFO to retrieve the maximum VM address dynamically.

Differential Revision: https://reviews.llvm.org/D35032

llvm-svn: 307307
2017-07-06 20:30:09 +00:00
Kuba Mracek c7f895351e More fixup for r307281: Move the #includes to sanitizer_linux.cc
llvm-svn: 307284
2017-07-06 17:45:01 +00:00
Kuba Mracek 47e994c4f1 Fixup for r307281: Also move GetKernelAreaSize into sanitizer_linux.cc
llvm-svn: 307282
2017-07-06 17:17:50 +00:00
Kuba Mracek 72665d6c15 [sanitizer] Split GetMaxVirtualAddress into separate Linux version and Mac version [NFC]
The logic in GetMaxVirtualAddress is already pretty complex, and I want to get rid of the hardcoded value for iOS/AArch64, which would need adding more Darwin-specific code, so let's split the implementation into sanitizer_linux.cc and sanitizer_mac.cc files. NFC.

Differential Revision: https://reviews.llvm.org/D35031

llvm-svn: 307281
2017-07-06 17:13:40 +00:00
Alex Shlyapnikov 35adb43950 [Sanitizers] Consolidate internal errno definitions.
Move internal errno definitions to common to be shared by all sanitizers
and to be used by allocators.

llvm-svn: 307233
2017-07-06 00:50:57 +00:00
Kuba Mracek 7149ca04ad [tsan] Use pthread_sigmask instead of sigprocmask to block signals in a thread on Darwin
On Darwin, sigprocmask changes the signal mask for the entire process. This has some unwanted consequences, because e.g. internal_start_thread wants to disable signals only in the current thread (to make the new thread inherit the signal mask), which is currently broken on Darwin. This patch switches to pthread_sigmask.

Differential Revision: https://reviews.llvm.org/D35016

llvm-svn: 307212
2017-07-05 22:17:44 +00:00
Jonas Hahnfeld 5b14061848 [asan] Remove check for stack size
This has been introduced in r304598 and fails for increased stack sizes.

Differential Revision: https://reviews.llvm.org/D34876

llvm-svn: 307127
2017-07-05 06:54:43 +00:00
Hiroshi Inoue 7e6d29535d fix trivial typos in comments; NFC
llvm-svn: 307124
2017-07-05 05:43:31 +00:00
Hiroshi Inoue 03656aa3dc fix trivial typos in comments; NFC
llvm-svn: 307005
2017-07-03 06:44:05 +00:00
Hiroshi Inoue 85c529c988 fix trivial typo; NFC
llvm-svn: 306955
2017-07-01 08:58:47 +00:00
Vitaly Buka 080118351b [asan] This Android lit workaround should not be needed as bug is fixed
llvm-svn: 306942
2017-07-01 04:23:47 +00:00
Vitaly Buka d29d3da349 [asan] Fix test on Android i686/fugu
printf from .preinit_array may crash.

llvm-svn: 306940
2017-07-01 03:54:19 +00:00
Vitaly Buka c1c17833ff [asan] Disable test which fails on Android x86
Other Android CPUs probably pass just by luck as ulimit was not executed.

llvm-svn: 306914
2017-06-30 23:47:03 +00:00
Alex Shlyapnikov d3213c792e [LSan] Make LSan allocator allocator_may_return_null compliant
Summary:
An attempt to reland D34786 (which caused bot failres on Mac), now with
properly intercepted operators new() and delete().

LSan allocator used to always return nullptr on too big allocation requests
(the definition of "too big" depends on platform and bitness), now it
follows policy configured by allocator_may_return_null flag

Reviewers: eugenis

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D34845

llvm-svn: 306845
2017-06-30 17:21:34 +00:00
Kuba Mracek c41ba09433 [objc] Don't require null-check and don't emit memset when result is ignored for struct-returning method calls [compiler-rt part]
This fixes an issue with the emission of lifetime markers for struct-returning Obj-C msgSend calls. When the result of a struct-returning call is ignored, the temporary storage is only marked with lifetime markers in one of the two branches of the nil-receiver-check. The check is, however, not required when the result is unused. If we still need to emit the check (due to consumer arguments), let's not emit the memset to zero out the result if it's unused. This fixes a use-after-scope false positive with AddressSanitizer.

Differential Revision: https://reviews.llvm.org/D34834

llvm-svn: 306838
2017-06-30 16:29:43 +00:00
Kostya Kortchinsky b138ab810f [sanitizer] Small tweaks and fixes to allocator related functions
Summary:
In `sanitizer_allocator_primary32.h`:
- rounding up in `MapWithCallback` is not needed as `MmapOrDie` does it. Note
  that the 64-bit counterpart doesn't round up, this keeps the behavior
  consistent;
- since `IsAligned` exists, use it in `AllocateRegion`;
- in `PopulateFreeList`:
  - checking `b->Count` to be greater than 0 when `b->Count() == max_count` is
    redundant when done more than once. Just check that `max_count` is greater
    than 0 out of the loop; the compiler (at least on ARM) didn't optimize it;
  - mark the batch creation failure as `UNLIKELY`;

In `sanitizer_allocator_primary64.h`:
- in `MapWithCallback`, mark the failure condition as `UNLIKELY`;

In `sanitizer_posix.h`:
- mark a bunch of Mmap related failure conditions as `UNLIKELY`;
- in `MmapAlignedOrDieOnFatalError`, we have `IsAligned`, so use it; rearrange
  the conditions as one test was redudant;
- in `MmapFixedImpl`, 30 chars was not large enough to hold the message and a
  full 64-bit address (or at least a 48-bit usermode address), increase to 40.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: aemerson, kubamracek, kristof.beyls, llvm-commits

Differential Revision: https://reviews.llvm.org/D34840

llvm-svn: 306834
2017-06-30 16:05:40 +00:00
Hiroshi Inoue 32f2183483 fix trivial typos, NFC
llvm-svn: 306807
2017-06-30 08:28:50 +00:00
Alex Shlyapnikov 346988bf02 Merge
llvm-svn: 306748
2017-06-29 21:54:38 +00:00
Alex Shlyapnikov cb823f6a19 [Sanitizers] Rename CallocShouldReturnNullDueToOverflow to CheckForCallocOverflow
Summary:
Due to changes in semantics, CheckForCallocOverflow makes much more sense
now.

Reviewers: eugenis

Subscribers: llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D34799

llvm-svn: 306747
2017-06-29 21:54:37 +00:00
Alex Shlyapnikov 93c1a8c7c7 Merge
llvm-svn: 306746
2017-06-29 21:54:36 +00:00
Vedant Kumar ff3227e77d [profile] Move __llvm_profile_filename into a separate object
Users can specify the path a raw profile is written to by passing
-fprofile-instr-generate=<path>, but this functionality broke on Darwin
after __llvm_profile_filename was made weak [1], resulting in profiles
being written to "default.profraw" even when <path> is specified.

The situation is that instrumented programs provide a weak definition of
__llvm_profile_filename, which conflicts with a weak redefinition
provided by the profiling runtime.

The linker appears to pick the 'winning' definition arbitrarily: on
Darwin, it usually prefers the larger definition, which is probably why
the instrprof-override-filename.c test has been passing.

The fix is to move the runtime's definition into a separate object file
within the archive. This means that the linker won't "see" the runtime's
definition unless the user program has not provided one. I couldn't
think of a great way to test this other than to mimic the Darwin
failure: use -fprofile-instr-generate=<some-small-path>.

Testing: check-{clang,profile}, modified instrprof-override-filename.c.

[1] [Profile] deprecate __llvm_profile_override_default_filename
https://reviews.llvm.org/D22613
https://reviews.llvm.org/D22614

Differential Revision: https://reviews.llvm.org/D34797

llvm-svn: 306710
2017-06-29 17:42:24 +00:00
Reid Kleckner 4f98ab369d Export the nothrow overload of operator new
This missing export was causing allocator_returns_null.cc to fail on
Windows with a dynamic ASan runtime.

llvm-svn: 306707
2017-06-29 17:39:53 +00:00
Reid Kleckner 4acf36beb6 Fix WinASan after moving wcslen interceptor to sanitizer_common
Do this by removing SANITIZER_INTERCEPT_WCSLEN and intercept wcslen
everywhere. Before this change, we were already intercepting wcslen on
Windows, but the interceptor was in asan, not sanitizer_common. After
this change, we stopped intercepting wcslen on Windows, which broke
asan_dll_thunk.c, which attempts to thunk to __asan_wcslen in the ASan
runtime.

llvm-svn: 306706
2017-06-29 17:15:53 +00:00
Kostya Kortchinsky 0ce4999002 [scudo] Change aligned alloc functions to be more compliant & perf changes
Summary:
We were not following the `man` documented behaviors for invalid arguments to
`memalign` and associated functions. Using `CHECK` for those was a bit extreme,
so we relax the behavior to return null pointers as expected when this happens.
Adapt the associated test.

I am using this change also to change a few more minor performance improvements:
- mark as `UNLIKELY` a bunch of unlikely conditions;
- the current `CHECK` in `__sanitizer::RoundUpTo` is redundant for us in *all*
  calls. So I am introducing our own version without said `CHECK`.
- change our combined allocator `GetActuallyAllocatedSize`. We already know if
  the pointer is from the Primary or Secondary, so the `PointerIsMine` check is
  redundant as well, and costly for the 32-bit Primary. So we get the size by
  directly using the available Primary functions.

Finally, change a `int` to `uptr` to avoid a warning/error when compiling on
Android.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D34782

llvm-svn: 306698
2017-06-29 16:45:20 +00:00
Michael Zolotukhin 1576571112 Revert "[LSan] Make LSan allocator allocator_may_return_null compliant"
This reverts commit r306624.

The committed test failed on various bots (e.g. on green dragon).

llvm-svn: 306644
2017-06-29 04:39:17 +00:00
Vitaly Buka 1631129834 Revert "[asan] Control location of symbolizer on device using ANDROID_SYMBOLIZER_PATH"
Not needed, I am going to put symbolizer into tests dir.

This reverts commit r306627.

llvm-svn: 306630
2017-06-29 02:48:06 +00:00
Vitaly Buka 72c24da063 [asan] Control location of symbolizer on device using ANDROID_SYMBOLIZER_PATH
llvm-svn: 306627
2017-06-29 01:04:32 +00:00
Alex Shlyapnikov 17277f13f0 [LSan] Make LSan allocator allocator_may_return_null compliant
Summary:
LSan allocator used to always return nullptr on too big allocation requests
(the definition of "too big" depends on platform and bitness), now it
follows policy configured by allocator_may_return_null flag.

Reviewers: eugenis

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D34786

llvm-svn: 306624
2017-06-29 01:02:40 +00:00
Vitaly Buka 2af2fd5f76 [asan] Fix try to fix test on Android
%T is dir on host system, device does not have it

llvm-svn: 306621
2017-06-29 00:19:29 +00:00
Vitaly Buka 0c981cf69b [asan] Disable tests which do no work on Android
llvm-svn: 306620
2017-06-29 00:19:28 +00:00
Vitaly Buka 3681a55137 [asan] This test now passes
llvm-svn: 306619
2017-06-29 00:19:27 +00:00
Alex Shlyapnikov 4b450685d3 [Sanitizers] Operator new() interceptors always die on allocation error
Summary:
Operator new interceptors behavior is now controlled by their nothrow
property as well as by allocator_may_return_null flag value:

- allocator_may_return_null=* + new()        - die on allocation error
- allocator_may_return_null=0 + new(nothrow) - die on allocation error
- allocator_may_return_null=1 + new(nothrow) - return null

Ideally new() should throw std::bad_alloc exception, but that is not
trivial to achieve, hence TODO.

Reviewers: eugenis

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D34731

llvm-svn: 306604
2017-06-28 21:58:57 +00:00
Xinliang David Li f50cc3ed8a [PGO] Reduce IO in profile dumping with merging
Differential Revision: http://reviews.llvm.org/D34709

llvm-svn: 306561
2017-06-28 16:46:06 +00:00
Kuba Mracek ce65982ea1 Revert r306504: Re-enable wait.cc, wait4.cc, waitid.cc tests on Darwin.
llvm-svn: 306551
2017-06-28 15:13:09 +00:00
Dean Michael Berris 66e08e7781 [XRay][compiler-rt][NFC] Move test case into correct directory.
Followup to D34669.

llvm-svn: 306506
2017-06-28 05:21:15 +00:00
Dean Michael Berris c3881436cf [XRay][compiler-rt] Only run test in x86_64 linux.
Followup to D34669.

llvm-svn: 306505
2017-06-28 05:19:59 +00:00
Kuba Mracek 70e8e9d8f6 Re-enable wait.cc, wait4.cc, waitid.cc tests on Darwin. They used to be flaky, something to do with LeakSanitizer now being enabled on Darwin. Let's re-enable them and see if they are still flaky or not.
llvm-svn: 306504
2017-06-28 05:03:28 +00:00
Dean Michael Berris 261d97332d [XRay][compiler-rt][NFC] Add example always/never instrument files.
Summary:
This change introduces two files that show exaples of the
always/never instrument files that can be provided to clang. We don't
add these as defaults yet in clang, which we can do later on (in a
separate change).

We also add a test that makes sure that these apply in the compiler-rt
project tests, and that changes in clang don't break the expectations in
compiler-rt.

Reviewers: pelikan, kpw

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D34669

llvm-svn: 306502
2017-06-28 04:44:36 +00:00
Evgeniy Stepanov 9e4519db1e [asan] Fix windows build.
llvm-svn: 306493
2017-06-28 00:37:29 +00:00
Evgeniy Stepanov 6f75e2dd48 [msan] Intercept wcscat, wcsncat.
Also move wcslen, wscnlen to common interceptors.

Reviewers: vitalybuka

Reviewed By: vitalybuka

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D34656

llvm-svn: 306482
2017-06-27 22:52:38 +00:00
Xinliang David Li 1f0d0b2a4c [Profile] Remove redundant call
llvm-svn: 306480
2017-06-27 22:42:24 +00:00
Francis Ricci fa0e536521 Don't build tsan/dd when COMPILER_RT_HAS_TSAN is false
llvm-svn: 306463
2017-06-27 21:10:46 +00:00
Francis Ricci 23aae3b554 Don't double-include cfi tests on linux
llvm-svn: 306455
2017-06-27 19:52:35 +00:00
Francis Ricci d379d1c2ee Loop directly over sanitizers to build in cmake
Summary: Cleaner than computing the intersection for each possible sanitizer

Reviewers: compnerd, beanz

Subscribers: llvm-commits, mgorny

Differential Revision: https://reviews.llvm.org/D34693

llvm-svn: 306453
2017-06-27 19:32:39 +00:00
Francis Ricci 57a3f4584d Only test sanitizers that are built when COMPILER_RT_SANITIZERS_TO_BUILD is used
Summary: This allows check-all to be used when only a subset of the sanitizers are built.

Reviewers: beanz, compnerd, rnk, pcc

Subscribers: llvm-commits, mgorny

Differential Revision: https://reviews.llvm.org/D34644

llvm-svn: 306450
2017-06-27 19:18:01 +00:00
Xinliang David Li 967669f6c1 [PGO] Refactor file/buffer writer callback interfaces /NFC
Introduces a 'owner' struct to include the overridable write
method and the write context in C.

This allows easy introdution of new member API to help reduce
profile merge time in the follow up patch.

llvm-svn: 306432
2017-06-27 17:28:01 +00:00
Francis Ricci 07fa7942fd Revert "Only test sanitizers that are built when COMPILER_RT_SANITIZERS_TO_BUILD is used"
This breaks cfi testing in cases when the cfi runtime isn't built.

This reverts commit 1c6a7b07545f0c9ce68e8b226f5397694ce48af7.

llvm-svn: 306431
2017-06-27 17:24:26 +00:00
Francis Ricci 80296ee7f3 Only test sanitizers that are built when COMPILER_RT_SANITIZERS_TO_BUILD is used
Summary: This allows check-all to be used when only a subset of the sanitizers are built.

Reviewers: beanz, compnerd

Subscribers: llvm-commits, mgorny

Differential Revision: https://reviews.llvm.org/D34644

llvm-svn: 306415
2017-06-27 15:22:56 +00:00
Francis Ricci 2189fe1870 [tsan] Add missing include directory for test unittests
Summary: Required to fix standalone builds in some configurations

Reviewers: kubamracek, zaks.anna

Subscribers: mgorny, llvm-commits

Differential Revision: https://reviews.llvm.org/D34631

llvm-svn: 306411
2017-06-27 14:56:59 +00:00
Alex Shlyapnikov 01676883cd [Sanitizers] 64 bit allocator respects allocator_may_return_null flag
Summary:
Make SizeClassAllocator64 return nullptr when it encounters OOM, which
allows the entire sanitizer's allocator to follow
allocator_may_return_null=1 policy
(LargeMmapAllocator: D34243, SizeClassAllocator64: D34433).

Reviewers: eugenis

Subscribers: srhines, kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D34540

llvm-svn: 306342
2017-06-26 22:54:10 +00:00
Kuba Mracek 495371d6df [asan] Flag 'asan_gen_prefixes.cc' as unsupported on iOS. The ARM and ARM64 assemblers can use different label prefixes than the expected.
llvm-svn: 306335
2017-06-26 21:37:40 +00:00
Derek Bruening 92e4443c32 [esan] Disable flaky tests for PR33590
Disables 3 esan workingset tests until their underlying failures are
determined and resolved.

llvm-svn: 306259
2017-06-26 01:02:54 +00:00
Xinliang David Li 95ce779185 [PGO] Implementate profile counter regiser promotion (test case)
Differential Revision: http://reviews.llvm.org/D34085

llvm-svn: 306232
2017-06-25 00:27:09 +00:00
Vitaly Buka b768d5b863 [asan] Regression test for PR33372
Reviewers: eugenis

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D34315

llvm-svn: 306195
2017-06-24 01:40:41 +00:00
Evgeniy Stepanov 90e5c30836 [asan] Add support for Android debug message.
Add ASan report to the "debug message" field in Android tombstones.

llvm-svn: 306184
2017-06-23 23:38:20 +00:00
Vedant Kumar d1c3780cf6 [ubsan] Improve diagnostics for return value checks (compiler-rt)
Differential Revision: https://reviews.llvm.org/D34298

llvm-svn: 306164
2017-06-23 21:32:48 +00:00
Alex Shlyapnikov f3cc7cc3d8 [Sanitizers] 32 bit allocator respects allocator_may_return_null flag
Summary:
Make SizeClassAllocator32 return nullptr when it encounters OOM, which
allows the entire sanitizer's allocator to follow allocator_may_return_null=1
policy, even for small allocations (LargeMmapAllocator is already fixed
by D34243).

Will add a test for OOM in primary allocator later, when
SizeClassAllocator64 can gracefully handle OOM too.

Reviewers: eugenis

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D34433

llvm-svn: 305972
2017-06-22 00:02:37 +00:00
Justin Bogner dd862f9106 [sanitizer-coverage] Stop marking this test as unsupported on Darwin
The bug that was causing this to fail was fixed in r305429.

llvm-svn: 305942
2017-06-21 19:04:59 +00:00
Kostya Kortchinsky 2b053b1c28 [sanitizer] Add a function to gather random bytes
Summary:
AFAICT compiler-rt doesn't have a function that would return 'good' random
bytes to seed a PRNG. Currently, the `SizeClassAllocator64` uses addresses
returned by `mmap` to seed its PRNG, which is not ideal, and  
`SizeClassAllocator32` doesn't benefit from the entropy offered by its 64-bit
counterpart address space, so right now it has nothing. This function aims at
solving this, allowing to implement good 32-bit chunk randomization. Scudo also
has a function that does this for Cookie purposes, which would go away in a
later CL once this lands.

This function will try the `getrandom` syscall if available, and fallback to
`/dev/urandom` if not.

Unfortunately, I do not have a way to implement and test a Mac and Windows
version, so those are unimplemented as of now. Note that `kRandomShuffleChunks`
is only used on Linux for now.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: zturner, rnk, llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D34412

llvm-svn: 305922
2017-06-21 15:56:03 +00:00
Simon Dardis f62399945c [mips][compiler-rt] Fix build breakage.
Change some reinterpret_casts to c-style casts due to template instantiation
restrictions and build breakage due to missing paranthesises.

llvm-svn: 305899
2017-06-21 11:29:15 +00:00
Vitaly Buka ff7b8ea2a0 Revert "[compiler-rt] Don't reset non-default user handler if allow_user_segv_handler is true."
Summary:
On Android we still need to reset preinstalled handlers and allow use handlers later.

This reverts commit r304039.

Reviewers: eugenis

Subscribers: kubamracek, dberris, llvm-commits

Differential Revision: https://reviews.llvm.org/D34434

llvm-svn: 305871
2017-06-21 01:10:23 +00:00
Evgeniy Stepanov 827cd74618 [asan] Fix android compiler wrapper lost in r301617.
llvm-svn: 305870
2017-06-21 00:56:31 +00:00
Alex Shlyapnikov ccab11b0e8 [Sanitizers] Move cached allocator_may_return_null flag to sanitizer_allocator
Summary:
Move cached allocator_may_return_null flag to sanitizer_allocator.cc and
provide API to consolidate and unify the behavior of all specific allocators.

Make all sanitizers using CombinedAllocator to follow
AllocatorReturnNullOrDieOnOOM() rules to behave the same way when OOM
happens.

When OOM happens, turn allocator_out_of_memory flag on regardless of
allocator_may_return_null flag value (it used to not to be set when
allocator_may_return_null == true).

release_to_os_interval_ms and rss_limit_exceeded will likely be moved to
sanitizer_allocator.cc too (later).

Reviewers: eugenis

Subscribers: srhines, kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D34310

llvm-svn: 305858
2017-06-20 21:23:02 +00:00
Alex Shlyapnikov a26d10802b [ASan] Disable allocator_oom_test.cc on s390
Summary:
ASan shadow memory on s390 is larger than other configurations, let's
disable this test for now (will revisit it later).

Reviewers: eugenis

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D34414

llvm-svn: 305822
2017-06-20 19:16:41 +00:00
Evgeniy Stepanov a2049b5210 [asan] Fix android setup script to overwrite the symlink.
This fixes asan_device_setup failing to update an existing asan
installation.

llvm-svn: 305746
2017-06-19 21:42:44 +00:00
Francis Ricci 42175cb0d7 Add lsan interceptors for libdispatch functions on darwin
Summary:
This is required for standalone LSan to work with libdispatch worker threads,
and is a slimmed down version of the functionality provided for ASan
in asan_mac.cc.

Re-commit of r305695 with use_stacks=0 to get around a racy lingering pointer.

Reviewers: alekseyshl, kubamracek, glider, kcc

Subscribers: mgorny, llvm-commits

Differential Revision: https://reviews.llvm.org/D34247

llvm-svn: 305732
2017-06-19 19:21:31 +00:00
Francis Ricci 214b354f2e Revert "Add lsan interceptors for libdispatch functions on darwin"
This reverts r305695

llvm-svn: 305712
2017-06-19 17:08:55 +00:00
Francis Ricci 4f1e047a6d Add lsan interceptors for libdispatch functions on darwin
Summary:
This is required for standalone LSan to work with libdispatch worker threads,
and is a slimmed down version of the functionality provided for ASan
in asan_mac.cc.

Reviewers: alekseyshl, kubamracek, glider, kcc

Subscribers: mgorny, llvm-commits

Differential Revision: https://reviews.llvm.org/D34247

llvm-svn: 305695
2017-06-19 14:09:10 +00:00
Sagar Thakur 6478d14a0d [scudo] Enabling MIPS support for Scudo
Adding MIPS 32-bit and 64-bit support for Scudo.

Reviewed by cryptoad, sdardis.
Differential: D31803

llvm-svn: 305682
2017-06-19 11:28:59 +00:00
Dean Michael Berris 8d6f9c24d3 [XRay][compiler-rt][NFC] Add a test for both arg1 and arg0 handling in the same binary
This test makes sure we can handle both arg0 and arg1 handling in the
same binary, and making sure that the XRay runtime calls the correct
trampoline when handlers for both of these cases are installed.

llvm-svn: 305660
2017-06-19 03:52:25 +00:00
Dean Michael Berris 043d3f8682 [XRay][compiler-rt][NFC] Add a name for argument to __xray_set_handler_arg1(...)
Just makes the interface consistent with the other functions in
include/xray/xray_interface.h.

llvm-svn: 305658
2017-06-19 01:30:04 +00:00
Pirama Arumuga Nainar 31b825d834 Call cmake_minimum_required at the top of CMakeLists.txt
Summary:
Since r298413, the NEW behavior of the CMake policy CMP0056 is followed.
However, it is only effective after the call to cmake_minimum_required.
This causes CMAKE_EXE_LINKER_FLAGS etc. to be unused when CMake tries to
check compilers for languages specified in the 'project' declaration.

Set cmake_minimum_required(VERSION) at the top of the file and ahead of
the project declaration.

Reviewers: beanz

Subscribers: mgorny, srhines, llvm-commits

Differential Revision: https://reviews.llvm.org/D34282

llvm-svn: 305593
2017-06-16 21:14:45 +00:00
Alex Shlyapnikov 5a308f24c3 [Sanitizer] Remove CombinedAllocator::Allocate's 'cleared' parameter
Summary:
CombinedAllocator::Allocate cleared parameter is not used anywhere and
seem to be obsolete.

Reviewers: eugenis

Subscribers: llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D34289

llvm-svn: 305590
2017-06-16 21:00:03 +00:00
Reid Kleckner f1b9f3a23f [WinASan] Fix hotpatching new Win 10 build 1703 x64 strnlen prologue
The first instruction of the new ucrtbase!strnlen implementation loads a
global, presumably to dispatch between SSE and non-SSE optimized strnlen
implementations.

Fixes PR32895 and probably
https://github.com/google/sanitizers/issues/818

llvm-svn: 305581
2017-06-16 20:44:00 +00:00
Alex Shlyapnikov d340605a37 [Sanitizers] Fix allocator OOM test on Windows.
Summary:
Point of failure is different after D34243, hence the change of the
message.

Reviewers: eugenis

Subscribers: llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D34292

llvm-svn: 305580
2017-06-16 20:36:39 +00:00
Alex Shlyapnikov 9092fe6f4b [Sanitizers] Secondary allocator respects allocator_may_return_null=1.
Summary:
Context: https://github.com/google/sanitizers/issues/740.

Making secondary allocator to respect allocator_may_return_null=1 flag
and return nullptr when "out of memory" happens.

More changes in primary allocator and operator new will follow.

Reviewers: eugenis

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D34243

llvm-svn: 305569
2017-06-16 18:48:08 +00:00
Dean Michael Berris 10bbc4f99e Add test for logging the implicit "this" argument for C++ member functions.
Summary:
This allows us to do more interesting things with the data available to
C++ methods, to log the `this` pointer.

Depends on D34050.

Reviewers: pelikan

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D34051

llvm-svn: 305545
2017-06-16 03:24:07 +00:00
Evgeniy Stepanov 2aed0fb07e [cfi] Enable icall tests with thinlto.
llvm-svn: 305534
2017-06-16 00:18:35 +00:00
Vedant Kumar f7e804157e [ubsan] Fix a faulty memory accessibility check
The dynamic type check needs to inspect vtables, but could crash if it
encounters a vtable pointer to inaccessible memory. In the first attempt
to fix the issue (r304437), we performed a memory accessibility check on
the wrong range of memory. This should *really* fix the problem.

Patch by Max Moroz!

Differential Revision: https://reviews.llvm.org/D34215

llvm-svn: 305489
2017-06-15 18:23:16 +00:00
Vitaly Buka 4a2cc96f1f [asan] Fix typo in doc string.
llvm-svn: 305436
2017-06-15 00:31:59 +00:00
Vitaly Buka 7901b47094 [asan] Return allow_user_segv_handler=0 to fix compatibility issues.
Summary:
After r303941 it was not possible to setup ASAN_OPTIONS to have the same
behavior for pre r303941 and post r303941 builds.
Pre r303941 Asan does not accept handle_sigbus=2.
Post r303941 Asan does not accept allow_user_segv_handler.

This fix ignores allow_user_segv_handler=1, but for allow_user_segv_handler=0
it will upgrade flags like handle_sigbus=1 to handle_sigbus=2. So user can set
ASAN_OPTIONS=allow_user_segv_handler=0 and have same behavior on old and new
clang builds (except range from r303941 to this revision).

In future users which need to prevent third party handlers should switch to
handle_sigbus=2 and remove allow_user_segv_handler as soon as suport of older
builds is not needed.

Related bugs:
  https://github.com/google/oss-fuzz/issues/675
  https://bugs.chromium.org/p/chromium/issues/detail?id=731130

Reviewers: eugenis

Subscribers: llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D34227

llvm-svn: 305433
2017-06-15 00:19:13 +00:00
Kostya Kortchinsky 77f30c9c31 [sanitizer] Reverting D34152
Summary:
This broke thread_local_quarantine_pthread_join.cc on some architectures, due
to the overhead of the stashed regions. Reverting while figuring out the best
way to deal with it.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D34213

llvm-svn: 305404
2017-06-14 17:32:26 +00:00
Kostya Kortchinsky eca926ab3a [sanitizer] MmapAlignedOrDie changes to reduce fragmentation
Summary:
The reasoning behind this change is explained in D33454, which unfortunately
broke the Windows version (due to the platform not supporting partial unmapping
of a memory region).

This new approach changes `MmapAlignedOrDie` to allow for the specification of
a `padding_chunk`. If non-null, and the initial allocation is aligned, this
padding chunk will hold the address of the extra memory (of `alignment` bytes).
This allows `AllocateRegion` to get 2 regions if the memory is aligned
properly, and thus help reduce fragmentation (and saves on unmapping
operations). As with the initial D33454, we use a stash in the 32-bit Primary
to hold those extra regions and return them on the fast-path.

The Windows version of `MmapAlignedOrDie` will always return a 0
`padding_chunk` if one was requested.

Reviewers: alekseyshl, dvyukov, kcc

Reviewed By: alekseyshl

Subscribers: llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D34152

llvm-svn: 305391
2017-06-14 15:32:17 +00:00
Alex Shlyapnikov 8531fa3028 [ASan] Move rss_limit_is_exceeded_ flag to ASan.
Summary:
Move the OOM decision based on RSS limits out of generic allocator to
ASan allocator, where it makes more sense at the moment.

Reviewers: eugenis

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D34180

llvm-svn: 305342
2017-06-13 23:57:24 +00:00
Evgeniy Stepanov cde90a66b5 [sanitize] Remove stack size limits from secondary threads.
If pthread_attr_getstack tell us the stack is 2G, why would we doubt that?

Differential Revision: https://reviews.llvm.org/D34169

llvm-svn: 305330
2017-06-13 22:29:15 +00:00
Dmitry Vyukov dc2a38cdf2 tsan: fix reading of mutex flags
SyncVar::IsFlagSet returns true if any flag is set.
This is wrong. Check the actual requested flag.

llvm-svn: 305281
2017-06-13 09:37:51 +00:00
Dmitry Vyukov f5b3b84826 tsan: fix pedantic warnings
ISO C++ does not allow ?: with omitted middle operand

llvm-svn: 305273
2017-06-13 07:09:56 +00:00
Vedant Kumar 8c31c2a546 [ubsan] Detect invalid unsigned pointer index expression (compiler-rt)
Compiler-rt part of: https://reviews.llvm.org/D33910

Differential Revision: https://reviews.llvm.org/D33911

llvm-svn: 305217
2017-06-12 18:42:51 +00:00
Vedant Kumar c7672d66ba Mark a sancov test as unsupported on x86_64h-darwin
Failing bot: http://lab.llvm.org:8080/green/job/clang-stage1-cmake-RA-expensive/6891

llvm-svn: 305133
2017-06-10 00:12:11 +00:00
Xinliang David Li 384f704834 Minor clean up of profile rt tests
llvm-svn: 305113
2017-06-09 21:14:19 +00:00
Maxim Ostapenko d8c47ca836 [asan] Fix ASan internal failure in AllocateFromLocalPool
This patch addresses PR 33206. There might be a situation when dynamic ASan runtime initializes later
than shared library which has malloc in static constructor (rtld doesn't provide an order of shared libs initialization).
In this case ASan hasn't yet initialized interceptors, but already intercepts malloc.
If malloc is too big to be handled by static local pool, ASan will die with error: 
Sanitizer CHECK failed: lib/asan/asan_malloc_linux.cc:40 ((allocated_for_dlsym)) < ((kDlsymAllocPoolSize)) (1036, 1024)

Patch by Denis Khalikov.

Differential Revision: https://reviews.llvm.org/D33784

llvm-svn: 305058
2017-06-09 07:47:38 +00:00
Kostya Serebryany 2c2fb8896b [sanitizer-coverage] one more flavor of coverage: -fsanitize-coverage=inline-8bit-counters. Experimental so far, not documenting yet. Reapplying revisions 304630, 304631, 304632, 304673, see PR33308
llvm-svn: 305026
2017-06-08 22:58:19 +00:00
Vedant Kumar cecb1121bb Mark the tsan strerror_r test as unsupported on Darwin
llvm-svn: 304944
2017-06-07 21:10:35 +00:00
Vedant Kumar 3e95c498aa Revert "Fix tsan test for Darwin. NFCI."
This reverts commit r304941. Vitaly Buka writes:
"Actually it depends on return value.
Test is for char* version of function. It will probably fail for int
version."

llvm-svn: 304943
2017-06-07 21:10:33 +00:00
Vedant Kumar a302ae096e Fix tsan test for Darwin. NFCI.
On Darwin, strerror_r returns an int, not a char*. I don't think this
test really depends on what strerror_r returns, so I've used something
else in place of the result of the call to strerror_r.

llvm-svn: 304941
2017-06-07 20:43:15 +00:00
Evgeniy Stepanov 7922ac23f1 [sanitizer] Fix symbolizer build after r304864.
r304864 moved code into a new BinaryFormat library. Update the
symbolizer build script.

llvm-svn: 304930
2017-06-07 17:24:58 +00:00
Vitaly Buka d9bc851fb3 [tsan]: Fix GNU version of strerror_r interceptor
GNU version of strerror_r returns a result pointer that doesn't match the input
buffer. The result pointer is in fact a pointer to some internal storage.
TSAN was recording a write to this location, which was incorrect.

Fixed https://github.com/google/sanitizers/issues/696

llvm-svn: 304858
2017-06-07 01:53:38 +00:00
Alex Shlyapnikov effb794346 [ASan] A speculative attempt to fix a flaky test on ppc64be.
Summary:
As mentioned in test/msan/fork.cc, if test output is redirected to a file
(as opposed to being piped directly to FileCheck), we may lose some "done"s due to
a kernel bug: https://lkml.org/lkml/2014/2/17/324, so let's pipe the
output of the test.

Reviewers: eugenis

Subscribers: llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D33915

llvm-svn: 304744
2017-06-05 22:23:15 +00:00
Evgeniy Stepanov 790838110f Revert r304285, r304297.
r304285 - [sanitizer] Avoid possible deadlock in child process after fork
r304297 - [sanitizer] Trying to fix MAC buildbots after r304285

These changes create deadlock when Tcl calls pthread_create from a
pthread_atfork child handler. More info in the original review at
https://reviews.llvm.org/D33325

llvm-svn: 304735
2017-06-05 21:20:55 +00:00
Alex Shlyapnikov abe8af9fac [asan] Use asan exitcode=0 option to always succeed a test run.
Summary:
halt_on_error-torture.cc intermittently fails on ppc64be, let's try to
collect more info on failures.

Reviewers: eugenis

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D33912

llvm-svn: 304731
2017-06-05 20:36:57 +00:00
Tim Northover ada043541d CMake: don't try to use lld if we're not building it.
Monorepo version!

llvm-svn: 304716
2017-06-05 15:10:04 +00:00
Renato Golin 77a12b3972 Revert "[sanitizer-coverage] test for -fsanitize-coverage=inline-8bit-counters"
Revert "Mark sancov test as unsupported on Darwin"
Revert "[LSan] Detect dynamic loader by its base address."

This reverts commit r304633.
This reverts commit r304673.
This reverts commit r304632.

Those commit have broken LOTS of ARM/AArch64 bots for two days.

llvm-svn: 304699
2017-06-05 07:36:02 +00:00
Vedant Kumar 921e4ddbad Mark the atos-symbolizer test as unsupported on i386-darwin
atos is apparently not able to resolve symbol addresses properly on
i386-darwin reliably any more. This is causing bot flakiness:
http://lab.llvm.org:8080/green/job/clang-stage1-cmake-RA-expensive/6841

There have not been any SDK changes on the bot as of late.

/Users/buildslave/jenkins/sharedspace/clang-stage1-cmake-RA_workspace/llvm/projects/compiler-rt/test/asan/TestCases/Darwin/atos-symbolizer.cc:20:12: error: expected string not found in input
 // CHECK: #1 0x{{.*}} in main {{.*}}atos-symbolizer.cc:[[@LINE-4]]
           ^
<stdin>:35:27: note: scanning from here
 #0 0x112f56 in wrap_free (/Users/buildslave/jenkins/sharedspace/clang-stage1-cmake-RA_workspace/clang-build/lib/clang/5.0.0/lib/darwin/libclang_rt.asan_osx_dynamic.dylib:i386+0x56f56)
                          ^
<stdin>:35:27: note: with expression "@LINE-4" equal to "16"
 #0 0x112f56 in wrap_free (/Users/buildslave/jenkins/sharedspace/clang-stage1-cmake-RA_workspace/clang-build/lib/clang/5.0.0/lib/darwin/libclang_rt.asan_osx_dynamic.dylib:i386+0x56f56)
                          ^
<stdin>:36:168: note: possible intended match here
 #1 0xb6f20 in main (/Users/buildslave/jenkins/sharedspace/clang-stage1-cmake-RA_workspace/clang-build/tools/clang/runtime/compiler-rt-bins/test/asan/I386DarwinConfig/TestCases/Darwin/Output/atos-symbolizer.cc.tmp:i386+0x1f20)

llvm-svn: 304674
2017-06-04 02:18:45 +00:00
Vedant Kumar d3322c1ed7 Mark sancov test as unsupported on Darwin
This test has been failing on all Darwin bots since it was introduced:
http://lab.llvm.org:8080/green/job/clang-stage1-configure-RA_check/32111

fatal error: error in backend: Global variable '__sancov_gen_' has an invalid section specifier '__DATA,__sancov_counters': mach-o section specifier requires a section whose length is between 1 and 16 characters.
Target: x86_64-apple-darwin15.6.0

llvm-svn: 304673
2017-06-04 01:56:44 +00:00
Dimitry Andric 6c00c9950c Adjust sanitizers for FreeBSD 64-bit inode update
Summary:
Very recently, FreeBSD 12 has been updated to use 64-bit inode numbers:
<https://svnweb.freebsd.org/changeset/base/318737>.  This entails many
user-visible changes, but for the sanitizers the modifications are
limited in scope:
* The `stat` and `lstat` syscalls were removed, and should be replaced
  with calls to `fstatat`.
* The `getdents` syscall was removed, and should be replaced with calls
  to `getdirentries`.
* The layout of `struct dirent` was changed to accomodate 64-bit inode
  numbers, and a new `d_off` field was added.
* The system header <sys/_types.h> now contains a macro `__INO64` to
  determine whether the system uses 64-bit inode numbers.

I tested these changes on both FreeBSD 12.0-CURRENT (after r318959,
which adds the `__INO64` macro), and FreeBSD 11.0-STABLE (which still
uses 32-bit inode numbers).

Reviewers: emaste, kcc, vitalybuka, kubamracek

Reviewed By: vitalybuka

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D33600

llvm-svn: 304658
2017-06-03 11:11:36 +00:00
Alex Shlyapnikov 2090504453 [LSan] Detect dynamic loader by its base address.
Summary:
Whenever possible (Linux + glibc 2.16+), detect dynamic loader module by
its base address, not by the module name matching. The current name
matching approach fails on some configurations.

Reviewers: eugenis

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D33859

llvm-svn: 304633
2017-06-03 01:43:44 +00:00
Kostya Serebryany edc20a10c1 [sanitizer-coverage] test for -fsanitize-coverage=inline-8bit-counters
llvm-svn: 304632
2017-06-03 01:36:53 +00:00
Kostya Serebryany ad272b0861 [asan] fix one more case where stack-use-after-return is not async-signal-safe (during thread startup). beef-up the test to give it a chance to catch regressions. Also relax the lint to make C++11 more usable.
llvm-svn: 304598
2017-06-02 21:32:04 +00:00
Kostya Serebryany 124c2ae4fa [sanitizer-coverage] nuke more stale code
llvm-svn: 304508
2017-06-02 01:17:04 +00:00
Kostya Serebryany 1800814b41 [sanitizer-coverage] nuke more stale code
llvm-svn: 304504
2017-06-02 00:52:35 +00:00
Kostya Serebryany c1a56baa3e [sanitizer-coverage] nuke more stale code
llvm-svn: 304503
2017-06-02 00:17:54 +00:00
Kostya Serebryany 70c64869b5 [sanitizer-coverage] nuke more stale code
llvm-svn: 304500
2017-06-01 23:56:49 +00:00
Vedant Kumar 41dfc4f1fa [ubsan] Runtime support for pointer overflow checking
Patch by John Regehr and Will Dietz!

Differential Revision: https://reviews.llvm.org/D20323

llvm-svn: 304461
2017-06-01 19:40:59 +00:00
Vedant Kumar 8f5073446f Tighten up test to address bot failure. NFC.
http://lab.llvm.org:8080/green/job/clang-stage1-configure-RA_check/32035

llvm-svn: 304440
2017-06-01 17:56:12 +00:00
Vedant Kumar 9af30e3541 Bug 33221 [UBSAN] segfault with -fsanitize=undefined
There is can be a situation when vptr is not initializing
by constructor of the object, and has a junk data which should
be properly checked, because c++ standard says:

"if default constructor is not specified
16 (7.3) no initialization is performed."

Patch by Denis Khalikov!

Differential Revision: https://reviews.llvm.org/D33712

llvm-svn: 304437
2017-06-01 16:44:11 +00:00
Pierre Gousseau 183d1368f3 [asan] Add strndup/__strndup interceptors.
Recommit of r302781 with Vitaly Buka's fix for non zero terminated strings.

Differential Revision: https://reviews.llvm.org/D31457

llvm-svn: 304399
2017-06-01 09:37:22 +00:00
Kostya Serebryany 9c50876120 [sanitizer-coverage] remove stale code (old coverage); compiler-rt part
llvm-svn: 304318
2017-05-31 18:26:32 +00:00
Maxim Ostapenko b1f0a346d6 [sanitizer] Trying to fix MAC buildbots after r304285
It seems that on MAC allocator already locks on fork thus adding another ForceLock
in fork interceptor will cause a deadlock.

llvm-svn: 304297
2017-05-31 11:40:57 +00:00
Maxim Ostapenko 62a0f55930 [sanitizer] Avoid possible deadlock in child process after fork
This patch addresses https://github.com/google/sanitizers/issues/774. When we
fork a multi-threaded process it's possible to deadlock if some thread acquired
StackDepot or allocator internal lock just before fork. In this case the lock
will never be released in child process causing deadlock on following memory alloc/dealloc
routine. While calling alloc/dealloc routines after multi-threaded fork is not allowed,
most of modern allocators (Glibc, tcmalloc, jemalloc) are actually fork safe. Let's do the same
for sanitizers except TSan that has complex locking rules.

Differential Revision: https://reviews.llvm.org/D33325

llvm-svn: 304285
2017-05-31 07:28:09 +00:00
Alex Shlyapnikov 0145dee366 [sanitizer] Add "isapla" to symbolizer's global symbols whitelist.
Summary: D33637 introduced isalpha, whitelist need to reflect that.

Reviewers: eugenis

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D33687

llvm-svn: 304234
2017-05-30 19:52:34 +00:00
Alex Shlyapnikov f8f8f49a24 [asan] Enable back halt_on_error-torture.cc disabled on PowerPC.
Summary:
D33521 addressed a memory ordering issue in BlockingMutex, which seems
to be the cause of a flakiness of a few ASan tests on PowerPC.

Reviewers: eugenis

Subscribers: kubamracek, nemanjai, llvm-commits

Differential Revision: https://reviews.llvm.org/D33611

llvm-svn: 304045
2017-05-26 23:14:06 +00:00
Vitaly Buka e8b09195fa [compiler-rt] Don't reset non-default user handler if allow_user_segv_handler is true.
Reviewers: eugenis, kcc

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D32457

llvm-svn: 304039
2017-05-26 21:51:26 +00:00
Renato Golin 5ade1f1c86 [LSAN-ARM] Marking new test unsupported on ARMHF due to bot failures
The test was meant for Darwin anyway, so I'm not even sure it's supposed
to run on Linux. If it was, then we need time to investigate, but since
the test is new, there's no point in reverting the whole patch because
of it.

llvm-svn: 304010
2017-05-26 17:31:33 +00:00
Kostya Kortchinsky db18e4d993 [scudo] Check the return values of the pthread_* functions
Summary:
Currently we are not enforcing the success of `pthread_once`, and
`pthread_setspecific`. Errors could lead to harder to debug issues later in
the thread's life. This adds checks for a 0 return value for both.
If `pthread_setspecific` fails in the teardown path, opt for an immediate
teardown as opposed to a fatal failure.

Reviewers: alekseyshl, kcc

Reviewed By: alekseyshl

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D33555

llvm-svn: 303998
2017-05-26 15:39:22 +00:00
Alex Shlyapnikov 23ff882e45 [asan] Enable back some ASan tests disabled on PowerPC.
Summary:
D33521 addressed a memory ordering issue in BlockingMutex, which seems
to be the cause of a flakiness of a few ASan tests on PowerPC.

Reviewers: eugenis

Subscribers: kubamracek, nemanjai, llvm-commits

Differential Revision: https://reviews.llvm.org/D33569

llvm-svn: 303995
2017-05-26 14:49:42 +00:00
Vitaly Buka a05da1fca9 [compiler-rt] Replace allow_user_segv_handler=0 with kHandleSignalExclusive
Summary:
allow_user_segv_handler had confusing name did not allow to control behavior for
signals separately.

Reviewers: eugenis, alekseyshl, kcc

Subscribers: llvm-commits, dberris, kubamracek

Differential Revision: https://reviews.llvm.org/D33371

llvm-svn: 303941
2017-05-25 23:42:33 +00:00
Kostya Serebryany f3509b6d9b [asan] relax sanbox_read_proc_self_maps_test to pass even if unshare() fails.
llvm-svn: 303911
2017-05-25 20:50:36 +00:00
Francis Ricci 3bfbd70840 Fix typo in tls patch
llvm-svn: 303906
2017-05-25 19:55:44 +00:00
Vitaly Buka 40d54d408b [compiler-rt] Make print_module_map description consistent with the rest.
Reviewers: eugenis

Subscribers: llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D33160

llvm-svn: 303892
2017-05-25 18:07:48 +00:00
Francis Ricci 86e070f7e9 Implement tls scanning for darwin LSan
Summary:
This required for any users who call exit() after creating
thread-specific data, as tls destructors are only called when
pthread_exit() or pthread_cancel() are used. This should also
match tls behavior on linux.

Getting the base address of the tls section is straightforward,
as it's stored as a section offset in %gs. The size is a bit trickier
to work out, as there doesn't appear to be any official documentation
or source code referring to it. The size used in this patch was determined
by taking the difference between the base address and the address of the
subsequent memory region returned by vm_region_recurse_64, which was
1024 * sizeof(uptr) on all threads except the main thread, where it was
larger. Since the section must be the same size on all of the threads,
1024 * sizeof(uptr) seemed to be a reasonable size to use, barring
a more programtic way to get the size.

1024 seems like a reasonable number, given that PTHREAD_KEYS_MAX
is 512 on darwin, so pthread keys will fit inside the region while
leaving space for other tls data. A larger size would overflow the
memory region returned by vm_region_recurse_64, and a smaller size
wouldn't leave room for all the pthread keys. In addition, the
stress test added here passes, which means that we are scanning at
least the full set of possible pthread keys, and probably
the full tls section.

Reviewers: alekseyshl, kubamracek

Subscribers: krytarowski, llvm-commits

Differential Revision: https://reviews.llvm.org/D33215

llvm-svn: 303887
2017-05-25 17:41:13 +00:00
Francis Ricci 75ca300f2b Don't require ThreadState to be contained within tls on all platforms
The existing implementation ran CHECKs to assert that the thread state
was stored inside the tls. However, the mac implementation of tsan doesn't
store the thread state in tls, so these checks fail once darwin tls support
is added to the sanitizers. Only run these checks on platforms where
the thread state is expected to be contained in the tls.

llvm-svn: 303886
2017-05-25 17:41:10 +00:00
Adam Nemet 14205b4a76 Disable two more flaky ASan wait* tests temporarily on Darwin
llvm-svn: 303885
2017-05-25 17:24:54 +00:00