They need to have same AddressSpaceView and MapUnmapCallback.
Reviewers: eugenis
Subscribers: kubamracek, #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D61168
llvm-svn: 359719
to reflect the new license.
We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.
Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.
llvm-svn: 351636
Summary:
This is a follow up patch to r349138.
This patch makes a `AddressSpaceView` a type declaration in the
allocator parameters used by `SizeClassAllocator64`. For ASan, LSan, and
the unit tests the AP64 declarations have been made templated so that
`AddressSpaceView` can be changed at compile time. For the other
sanitizers we just hard-code `LocalAddressSpaceView` because we have no
plans to use these allocators in an out-of-process manner.
rdar://problem/45284065
Reviewers: kcc, dvyukov, vitalybuka, cryptoad, eugenis, kubamracek, george.karpenkov
Subscribers: #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D55764
llvm-svn: 349954
Summary:
This is a follow up patch to r346956 for the `SizeClassAllocator32`
allocator.
This patch makes `AddressSpaceView` a template parameter both to the
`ByteMap` implementations (but makes `LocalAddressSpaceView` the
default), some `AP32` implementations and is used in `SizeClassAllocator32`.
The actual changes to `ByteMap` implementations and
`SizeClassAllocator32` are very simple. However the patch is large
because it requires changing all the `AP32` definitions, and users of
those definitions.
For ASan and LSan we make `AP32` and `ByteMap` templateds type that take
a single `AddressSpaceView` argument. This has been done because we will
instantiate the allocator with a type that isn't `LocalAddressSpaceView`
in the future patches. For the allocators used in the other sanitizers
(i.e. HWAsan, MSan, Scudo, and TSan) use of `LocalAddressSpaceView` is
hard coded because we do not intend to instantiate the allocators with
any other type.
In the cases where untemplated types have become templated on a single
`AddressSpaceView` parameter (e.g. `PrimaryAllocator`) their name has
been changed to have a `ASVT` suffix (Address Space View Type) to
indicate they are templated. The only exception to this are the `AP32`
types due to the desire to keep the type name as short as possible.
In order to check that template is instantiated in the correct a way a
`static_assert(...)` has been added that checks that the
`AddressSpaceView` type used by `Params::ByteMap::AddressSpaceView` matches
the `Params::AddressSpaceView`. This uses the new `sanitizer_type_traits.h`
header.
rdar://problem/45284065
Reviewers: kcc, dvyukov, vitalybuka, cryptoad, eugenis, kubamracek, george.karpenkov
Subscribers: mgorny, llvm-commits, #sanitizers
Differential Revision: https://reviews.llvm.org/D54904
llvm-svn: 349138
Summary:
static fallback_mutex in msan_allocator.cc does not need the ctor
call and can be linker initialized.
Issue: https://github.com/google/sanitizers/issues/194
Reviewers: morehouse, eugenis
Subscribers: delcypher, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D48145
llvm-svn: 334749
Summary:
Following up on and complementing D44404.
Currently many allocator specific errors (OOM, for example) are reported as
a text message and CHECK(0) termination, not stack, no details, not too
helpful nor informative. To improve the situation, detailed and
structured errors were defined and reported under the appropriate conditions.
Reviewers: eugenis
Subscribers: srhines, delcypher, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D47793
llvm-svn: 334338
Summary:
Make common allocator agnostic to failure handling modes and move the
decision up to the particular sanitizer's allocator, where the context
is available (call stack, parameters, return nullptr/crash mode etc.)
It simplifies the common allocator and allows the particular sanitizer's
allocator to generate more specific and detailed error reports (which
will be implemented later).
The behavior is largely the same, except one case, the violation of the
common allocator's check for "size + alignment" overflow is now reportied
as OOM instead of "bad request". It feels like a worthy tradeoff and
"size + alignment" is huge in this case anyway (thus, can be interpreted
as not enough memory to satisfy the request). There's also a Report()
statement added there.
Reviewers: eugenis
Subscribers: kubamracek, llvm-commits, #sanitizers
Differential Revision: https://reviews.llvm.org/D42198
llvm-svn: 322784
Summary:
Reuse the Linux new mapping as it is.
Sponsored by <The NetBSD Foundation>
Reviewers: joerg, vitalybuka, eugenis
Reviewed By: vitalybuka
Subscribers: llvm-commits, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D41023
llvm-svn: 320218
Summary:
CheckForPvallocOverflow was introduced with D35818 to detect when pvalloc
would wrap when rounding up to the next multiple of the page size.
Add this check to MSan's pvalloc implementation.
This time I made sure I was actually running (and writing) the correct tests,
and that they are passing...
Reviewers: alekseyshl
Reviewed By: alekseyshl
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D36164
llvm-svn: 309883
Summary:
Reverting D36093 until I can figure out how to launch the correct tests :/
My apologies.
Reviewers: alekseyshl
Reviewed By: alekseyshl
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D36120
llvm-svn: 309637
Summary:
`CheckForPvallocOverflow` was introduced with D35818 to detect when pvalloc
would wrap when rounding up to the next multiple of the page size.
Add this check to MSan's pvalloc implementation.
Reviewers: alekseyshl
Reviewed By: alekseyshl
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D36093
llvm-svn: 309601
Summary:
ASan/MSan/LSan allocators set errno on allocation failures according to
malloc/calloc/etc. expected behavior.
MSan allocator was refactored a bit to make its structure more similar
with other allocators.
Also switch Scudo allocator to the internal errno definitions.
TSan allocator changes will follow.
Reviewers: eugenis
Subscribers: llvm-commits, kubamracek
Differential Revision: https://reviews.llvm.org/D35275
llvm-svn: 308344
Summary:
Move cached allocator_may_return_null flag to sanitizer_allocator.cc and
provide API to consolidate and unify the behavior of all specific allocators.
Make all sanitizers using CombinedAllocator to follow
AllocatorReturnNullOrDieOnOOM() rules to behave the same way when OOM
happens.
When OOM happens, turn allocator_out_of_memory flag on regardless of
allocator_may_return_null flag value (it used to not to be set when
allocator_may_return_null == true).
release_to_os_interval_ms and rss_limit_exceeded will likely be moved to
sanitizer_allocator.cc too (later).
Reviewers: eugenis
Subscribers: srhines, kubamracek, llvm-commits
Differential Revision: https://reviews.llvm.org/D34310
llvm-svn: 305858
Summary:
CombinedAllocator::Allocate cleared parameter is not used anywhere and
seem to be obsolete.
Reviewers: eugenis
Subscribers: llvm-commits, kubamracek
Differential Revision: https://reviews.llvm.org/D34289
llvm-svn: 305590
r304285 - [sanitizer] Avoid possible deadlock in child process after fork
r304297 - [sanitizer] Trying to fix MAC buildbots after r304285
These changes create deadlock when Tcl calls pthread_create from a
pthread_atfork child handler. More info in the original review at
https://reviews.llvm.org/D33325
llvm-svn: 304735
This patch addresses https://github.com/google/sanitizers/issues/774. When we
fork a multi-threaded process it's possible to deadlock if some thread acquired
StackDepot or allocator internal lock just before fork. In this case the lock
will never be released in child process causing deadlock on following memory alloc/dealloc
routine. While calling alloc/dealloc routines after multi-threaded fork is not allowed,
most of modern allocators (Glibc, tcmalloc, jemalloc) are actually fork safe. Let's do the same
for sanitizers except TSan that has complex locking rules.
Differential Revision: https://reviews.llvm.org/D33325
llvm-svn: 304285
Summary:
With rL279771, SizeClassAllocator64 was changed to accept only one template
instead of 5, for the following reasons: "First, this will make the mangled
names shorter. Second, this will make adding more parameters simpler". This
patch mirrors that work for SizeClassAllocator32.
This is in preparation for introducing the randomization of chunks in the
32-bit SizeClassAllocator in a later patch.
Reviewers: kcc, alekseyshl, dvyukov
Reviewed By: alekseyshl
Subscribers: llvm-commits, kubamracek
Differential Revision: https://reviews.llvm.org/D33141
llvm-svn: 303071
Summary:
The current code was sometimes attempting to release huge chunks of
memory due to undesired RoundUp/RoundDown interaction when the requested
range is fully contained within one memory page.
Reviewers: eugenis
Subscribers: kubabrecka, llvm-commits
Patch by Aleksey Shlyapnikov.
Differential Revision: https://reviews.llvm.org/D27228
llvm-svn: 288271
Summary:
In order to avoid starting a separate thread to return unused memory to
the system (the thread interferes with process startup on Android,
Zygota waits for all threads to exit before fork, but this thread never
exits), try to return it right after free.
Reviewers: eugenis
Subscribers: cryptoad, filcab, danalbert, kubabrecka, llvm-commits
Patch by Aleksey Shlyapnikov.
Differential Revision: https://reviews.llvm.org/D27003
llvm-svn: 288091
This is an implementation of
https://github.com/google/sanitizers/issues/579
It has a number of advantages over the current mapping:
* Works for non-PIE executables.
* Does not require ASLR; as a consequence, debugging MSan programs in
gdb no longer requires "set disable-randomization off".
* Supports linux kernels >=4.1.2.
* The code is marginally faster and smaller.
This is an ABI break. We never really promised ABI stability, but
this patch includes a courtesy escape hatch: a compile-time macro
that reverts back to the old mapping layout.
llvm-svn: 249754
- Trim spaces.
- Use nullptr in place of 0 for pointer variables.
- Use '!p' in place of 'p == 0' for null pointer checks.
- Add blank lines to separate function definitions.
- Add 'extern "C"' or 'namespace foo' comments after the appropriate
closing brackets
This is a continuation of work from 409b7b82. The focus here is on the
various sanitizers (not sanitizer_common, as before).
Patch by Eugene Zelenko!
Differential Revision: http://reviews.llvm.org/D13225
llvm-svn: 248966
This patch enabled msan for aarch64 with 39-bit VMA and 42-bit VMA.
As defined by lib/msan/msan.h the memory layout used is for 39-bit is:
00 0000 0000 - 40 0000 0000: invalid
40 0000 0000 - 43 0000 0000: shadow
43 0000 0000 - 46 0000 0000: origin
46 0000 0000 - 55 0000 0000: invalid
55 0000 0000 - 56 0000 0000: app (low)
56 0000 0000 - 70 0000 0000: invalid
70 0000 0000 - 80 0000 0000: app (high)
And for 42-bit VMA:
000 0000 0000 - 100 0000 0000: invalid
100 0000 0000 - 11b 0000 0000: shadow
11b 0000 0000 - 120 0000 0000: invalid
120 0000 0000 - 13b 0000 0000: origin
13b 0000 0000 - 2aa 0000 0000: invalid
2aa 0000 0000 - 2ab 0000 0000: app (low)
2ab 0000 0000 - 3f0 0000 0000: invalid
3f0 0000 0000 - 400 0000 0000: app (high)
Most of tests are passing with exception of:
* Linux/mallinfo.cc
* chained_origin_limits.cc
* dlerror.cc
* param_tls_limit.cc
* signal_stress_test.cc
* nonnull-arg.cpp
The 'Linux/mallinfo.cc' is due the fact AArch64 returns the sret in 'x8'
instead of default first argument 'x1'. So a function prototype that
aims to mimic (by using first argument as the return of function) won't
work. For GCC one can make a register alias (register var asm ("r8")), but
for clang it detects is an unused variable and generate wrong code.
The 'chained_origin_limits' is probably due a wrong code generation,
since it fails only when origin memory is used
(-fsanitize-memory-track-origins=2) and only in the returned code
(return buf[50]).
The 'signal_streess_test' and 'nonnull-arg' are due currently missing variadic
argument handling in memory sanitizer code instrumentation on LLVM side.
Both 'dlerror' and 'param_tls_test' are unknown failures that require
further investigation.
All the failures are XFAIL for aarch64 for now.
llvm-svn: 247809
Summary:
This patch adds basic memory sanitizer support for PPC64. PR23219.
I have further patches ready to enable it in LLVM and Clang, and to fix
most of the many failing tests in check-msan.
Reviewers: kcc, willschm, samsonov, wschmidt, eugenis
Reviewed By: eugenis
Subscribers: wschmidt, llvm-commits
Differential Revision: http://reviews.llvm.org/D10648
llvm-svn: 240623
By attaching an extra integer tag to heap origins, we are able
to distinguish between uninits
- created by heap allocation,
- created by heap deallocation (i.e. use-after-free),
- created by __msan_allocated_memory call,
- etc.
See https://code.google.com/p/memory-sanitizer/issues/detail?id=35.
llvm-svn: 226821
Fixes 2 issues in origins arising from realloc() calls:
* In the in-place grow case origin for the new memory is not set at all.
* In the copy-realloc case __msan_memcpy is used, which unwinds stack from
inside the MSan runtime. This does not generally work (as we may be built
w/o frame pointers), and produces "bad" stack trace anyway, with several
uninteresting (internal) frames on top.
This change also makes realloc() honor "zeroise" and "poison_in_malloc" flags.
See https://code.google.com/p/memory-sanitizer/issues/detail?id=73.
llvm-svn: 226674
Summary:
Turn "allocator_may_return_null" common flag into an
Allocator::may_return_null bool flag. We want to make sure
that common flags are immutable after initialization. There
are cases when we want to change this flag in the allocator
at runtime: e.g. in unit tests and during ASan activation
on Android.
Test Plan: regression test suite, real-life applications
Reviewers: kcc, eugenis
Subscribers: llvm-commits
Differential Revision: http://reviews.llvm.org/D6623
llvm-svn: 224148
Previously, all origin ids were "chained" origins, i.e values of
ChainedOriginDepot. This added a level of indirection for simple
stack and heap allocation, which were represented as chains of
length 1. This costs both RAM and CPU, but provides a joined 2**29
origin id space. It also made function (any instrumented function)
entry non-async-signal-safe, but that does not really matter because
memory stores in track-origins=2 mode are not async-signal-safe anyway.
With this change, the type of the origin is encoded in origin id.
See comment in msan_origin.h for more details. This reduces chained and stack
origin id range to 2**28 each, but leaves extra 2**31 for heap origins.
This change should not have any user-visible effects.
llvm-svn: 223233
Introduce new public header <sanitizer/allocator_interface.h> and a set
of functions __sanitizer_get_ownership(), __sanitizer_malloc_hook() etc.
that will eventually replace their tool-specific equivalents
(__asan_get_ownership(), __msan_get_ownership() etc.). Tool-specific
functions are now deprecated and implemented as stubs redirecting
to __sanitizer_ versions (which are implemented differently in each tool).
Replace all uses of __xsan_ versions with __sanitizer_ versions in unit
and lit tests.
llvm-svn: 212469
Generalize StackDepot and create a new specialized instance of it to
efficiently (i.e. without duplicating stack trace data) store the
origin history tree.
This reduces memory usage for chained origins roughly by an order of
magnitude.
Most importantly, this new design allows us to put two limits on
stored history data (exposed in MSAN_OPTIONS) that help avoid
exponential growth in used memory on certain workloads.
See comments in lib/msan/msan_origin.h for more details.
llvm-svn: 209284
Vitaly Buka