to reflect the new license.
We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.
Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.
llvm-svn: 351636
Add a CheckMPROTECT() routine to detect when pax MPROTECT is enabled
on NetBSD, and error xray out when it is. The solution is adapted
from existing CheckASLR().
Differential Revision: https://reviews.llvm.org/D56049
llvm-svn: 350030
This speeds up process startup and teardown and also reduces lock contention when running multiple ASanified/TSanified processes simultaneously. Should greatly improve lit testing time.
Differential Revision: https://reviews.llvm.org/D48445
llvm-svn: 346262
This allows users of static libraries (such as ubsan) to link without
knowing about this transitive dependency, if using the psapi functions
that require linking to a separate psapi library. Since Windows 7,
these functions (EnumProcessModules, GetModuleInformation,
GetProcessMemoryInfo) are remapped to K32- prefixed ones, available in
kernel32.dll.
Differential Revision: https://reviews.llvm.org/D53012
llvm-svn: 344126
Summary:
Before my change, BlockingMutex used Windows critial sections. Critical
sections can only be initialized by calling InitializeCriticalSection,
dynamically.
The primary sanitizer allocator expects to be able to reinterpret zero
initialized memory as a BlockingMutex and immediately lock it.
RegionInfo contains a mutex, and it placement new is never called for
it. These objects are accessed via:
RegionInfo *GetRegionInfo(uptr class_id) const {
DCHECK_LT(class_id, kNumClasses);
RegionInfo *regions = reinterpret_cast<RegionInfo *>(SpaceEnd());
return ®ions[class_id];
}
The memory comes from the OS without any other initialization.
For various reasons described in the comments, BlockingMutex::Lock would
check if the object appeared to be zero-initialized, and it would lazily
call the LinkerInitialized constructor to initialize the critical
section. This pattern is obviously racy, and the code had a bunch of
FIXMEs about it.
The best fix here is to use slim reader writer locks, which can start
out zero-initialized. They are available starting in Windows Vista. I
think it's safe to go ahead and use them today.
Reviewers: kcc, vitalybuka
Subscribers: kubamracek, llvm-commits
Differential Revision: https://reviews.llvm.org/D49893
llvm-svn: 338331
MmapFixedNoReserve does not terminate process on failure.
Failure to check its result and die will always lead to harder
to debug crashes later in execution. This was observed in Go
processes due to some address space conflicts.
Consistently check result of MmapFixedNoReserve.
While we are here also add warn_unused_result attribute
to prevent such bugs in future and change return type to bool
as that's what all callers want.
Reviewed in https://reviews.llvm.org/D49367
llvm-svn: 337531
Summary:
Move madvise(MADV_NOHUGEPAGE) for the meta shadow memory after the meta
shadow memory is mapped (currently it silently fails with ENOMEM).
Add a diagnostic message to detect similar problems in the future.
Reviewers: dvyukov
Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D48097
llvm-svn: 334624
Summary:
At least the ASan, MSan, TSan sanitizers require disabled ASLR on a NetBSD.
Introduce a generic CheckASLR() routine, that implements a check for the
current process. This flag depends on the global or per-process settings.
There is no simple way to disable ASLR in the build process from the
level of a sanitizer or during the runtime execution.
With ASLR enabled sanitizers that operate over the process virtual address
space can misbehave usually breaking with cryptic messages.
This check is dummy for !NetBSD.
Sponsored by <The NetBSD Foundation>
Reviewers: vitalybuka, joerg
Reviewed By: vitalybuka
Subscribers: cryptoad, kubamracek, llvm-commits, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D47442
llvm-svn: 333985
The profile library was missing some includes and was erroneously using
ftruncate. WinASan was using `= {0}` to initialize structs, which
creates -Wmissing-field-initializers and -Wmissing-braces warnings with
clang. Use `= {}` instead, since this is C++.
llvm-svn: 330616
Summary:
Some of the functions had spurious conditional statements and checks, and some
intermediary variables that I feel made the code more complicated than it needs
to be. Also, when unmapping the whole range, the range size would be 0, but
the base was set to the address of the end of the range, which sounds prone to
error. I think nulling out the base in this scenario is a better way to go.
Reviewers: alekseyshl, flowerhack
Reviewed By: alekseyshl
Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D45775
llvm-svn: 330355
Summary:
The purpose of this set of changes is to separate stackframe/symbolizer support into their own RT within sanitizer_common.
Sanitizers with no use for those could then be built without the extraneous dependencies pulled in by the default visibility interface functions.
I am aiming to do small changes for specific platforms.
In this one, we split the unwind functions from sanitizer_win.cc into their own sanitizer_unwind_win.cc.
Reviewers: alekseyshl, rnk
Reviewed By: alekseyshl, rnk
Subscribers: delcypher, #sanitizers, kubamracek, mgorny, llvm-commits
Differential Revision: https://reviews.llvm.org/D44799
llvm-svn: 329204
Summary:
`Symbolizer::PrepareForSandboxing` is empty for all platforms and apparently
has been for a while (D10213). Remove it, and shuffle things around so that the
platform specific code is now in `PlatformPrepareForSandboxing`.
This allows to have one less symbolizer dependency in a common file, which
helps for the upcoming split.
Also remove `SymbolizerPrepareForSandboxing` in tsan_go which appears to not
be used anywhere.
Reviewers: alekseyshl, eugenis, dvyukov, mcgrathr
Reviewed By: alekseyshl
Subscribers: kubamracek, delcypher, llvm-commits, #sanitizers
Differential Revision: https://reviews.llvm.org/D44953
llvm-svn: 329094
FindAvailableMemoryRange can currently overwrite existing memory (by restricting the VM below addresses that are already used). This patch adds a check to make sure we don't restrict the VM space too much. We are also now more explicit about why the lookup failed and print out verbose values.
Differential Revision: https://reviews.llvm.org/D43318
llvm-svn: 326106
Summary:
Pretty straightforward, returning the `WorkingSetSize` of a
`PROCESS_MEMORY_COUNTERS` structure. AFAIU, `GetProcessMemoryInfo` is in
`kernel32.lib` for Windows 7 and above. Support for earlier Windows versions
would require `psapi.lib`, but I don't think those are supported by ASan?
Reviewers: alekseyshl, rnk, vitalybuka
Reviewed By: vitalybuka
Subscribers: vitalybuka, kubamracek, delcypher, llvm-commits, #sanitizers
Differential Revision: https://reviews.llvm.org/D42822
llvm-svn: 325020
Summary:
Implement `MonotonicNanoTime` using `QueryPerformanceCounter`.
This function is used by Scudo & the 64-bit Primary allocator. Implementing it
now means that the release-to-OS mechanism of the Primary will kick in (it
never did since the function returned 0 always), but `ReleaseMemoryPagesToOS` is
still not currently implemented for Windows.
Performance wise, this adds a syscall & a 64-bit division per call to
`MonotonicNanoTime` so the impact might not be negligible, but I don't think
there is a way around it.
Reviewers: rnk, alekseyshl, amccarth
Reviewed By: alekseyshl, amccarth
Subscribers: amccarth, flowerhack, kubamracek, delcypher, llvm-commits, #sanitizers
Differential Revision: https://reviews.llvm.org/D42579
llvm-svn: 324011
Summary:
Implement `GetNumberOfCPUs` using `GetNativeSystemInfo`.
The only consummer of this function is Scudo which is not functional on
Windows yet.
Reviewers: rnk, zturner
Reviewed By: zturner
Subscribers: zturner, kubamracek, delcypher, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D42547
llvm-svn: 323462
Summary:
See D40657 & D40679 for previous versions of this patch & description.
A couple of things were fixed here to have it not break some bots.
Weak symbols can't be used with `SANITIZER_GO` so the previous version was
breakin TsanGo. I set up some additional local tests and those pass now.
I changed the workaround for the glibc vDSO issue: `__progname` is initialized
after the vDSO and is actually public and of known type, unlike
`__vdso_clock_gettime`. This works better, and with all compilers.
The rest is the same.
Reviewers: alekseyshl
Reviewed By: alekseyshl
Subscribers: srhines, kubamracek, krytarowski, llvm-commits, #sanitizers
Differential Revision: https://reviews.llvm.org/D41121
llvm-svn: 320594
Summary:
Redo of D40657, which had the initial discussion. The initial code had to move
into a libcdep file, and things had to be shuffled accordingly.
`NanoTime` is a time sink when checking whether or not to release memory to
the OS. While reducing the amount of calls to said function is in the works,
another solution that was found to be beneficial was to use a timing function
that can leverage the vDSO.
We hit a couple of snags along the way, like the fact that the glibc crashes
when clock_gettime is called from a preinit_array, or the fact that
`__vdso_clock_gettime` is mangled (for security purposes) and can't be used
directly, and also that clock_gettime can be intercepted.
The proposed solution takes care of all this as far as I can tell, and
significantly improve performances and some Scudo load tests with memory
reclaiming enabled.
@mcgrathr: please feel free to follow up on
https://reviews.llvm.org/D40657#940857 here. I posted a reply at
https://reviews.llvm.org/D40657#940974.
Reviewers: alekseyshl, krytarowski, flowerhack, mcgrathr, kubamracek
Reviewed By: alekseyshl, krytarowski
Subscribers: #sanitizers, mcgrathr, srhines, llvm-commits, kubamracek
Differential Revision: https://reviews.llvm.org/D40679
llvm-svn: 320409
Summary:
Now that the sanitizer_common interface for MmapNoAccess / MmapFixed
have been refactored to allow a more OO-esque access pattern, update the
Fuchsia mmap implementation to take advantage of this.
Previously MmapNoAccess / MmapFixed relied on a global allocator_vmar,
since the sanitizer_allocator only called MmapNoAccess once. Now, we
create a new VMAR per ReservedAddressRange object.
This allows the sanitizer allocator to work in tandem with the Scudo
secondary allocator.
This is part 4 of a 4 part changeset:
* part 1 https://reviews.llvm.org/D38593
* part 2 https://reviews.llvm.org/D38592
* part 3 https://reviews.llvm.org/D38593
Reviewers: mcgrathr, cryptoad
Reviewed By: cryptoad
Subscribers: alekseyshl, mcgrathr, kubamracek, mehdi_amini
Differential Revision: https://reviews.llvm.org/D38595
llvm-svn: 319083
Summary:
This change allows Fuchsia to boot properly using the Scudo allocator.
A first version of this commit was reverted by rL317834 because it broke Android
builds for toolchains generated with older NDKs. This commit introduces a
fall back to solve that issue.
Reviewers: cryptoad, krytarowski, rnk, alekseyshl
Reviewed By: cryptoad, krytarowski, alekseyshl
Subscribers: llvm-commits, srhines, kubamracek, krytarowski
Differential Revision: https://reviews.llvm.org/D40121
llvm-svn: 318802
Summary:
This change reverts r318575 and changes FindDynamicShadowStart() to
keep the memory range it found mapped PROT_NONE to make sure it is
not reused. We also skip MemoryRangeIsAvailable() check, because it
is (a) unnecessary, and (b) would fail anyway.
Reviewers: pcc, vitalybuka, kcc
Subscribers: srhines, kubamracek, mgorny, llvm-commits, hiraditya
Differential Revision: https://reviews.llvm.org/D40203
llvm-svn: 318666
Revert the following commits:
r318369 [asan] Fallback to non-ifunc dynamic shadow on android<22.
r318235 [asan] Prevent rematerialization of &__asan_shadow.
r317948 [sanitizer] Remove unnecessary attribute hidden.
r317943 [asan] Use dynamic shadow on 32-bit Android.
MemoryRangeIsAvailable() reads /proc/$PID/maps into an mmap-ed buffer
that may overlap with the address range that we plan to use for the
dynamic shadow mapping. This is causing random startup crashes.
llvm-svn: 318575
Summary:
The ScudoAllocator uses a SecondaryHeader to keep track of the size and base address of each mmap'd chunk.
This aligns well with what the ReservedAddressRange is trying to do. This changeset converts the scudo allocator from using the MmapNoAccess/MmapFixed APIs to the ReservedAddressRange::Init and ::Map APIs. In doing so, it replaces the SecondayHeader struct with the ReservedAddressRange object.
This is part 3 of a 4 part changeset; part 1 https://reviews.llvm.org/D39072 and part 2 https://reviews.llvm.org/D38592
Reviewers: alekseyshl, mcgrathr, cryptoad, phosek
Reviewed By: cryptoad
Subscribers: llvm-commits, cryptoad, kubamracek
Differential Revision: https://reviews.llvm.org/D38593
llvm-svn: 318080
Summary:
The following kernel change has moved ET_DYN base to 0x4000000 on arm32:
https://marc.info/?l=linux-kernel&m=149825162606848&w=2
Switch to dynamic shadow base to avoid such conflicts in the future.
Reserve shadow memory in an ifunc resolver, but don't use it in the instrumentation
until PR35221 is fixed. This will eventually let use save one load per function.
Reviewers: kcc
Subscribers: aemerson, srhines, kubamracek, kristof.beyls, hiraditya, llvm-commits
Differential Revision: https://reviews.llvm.org/D39393
llvm-svn: 317943
Summary:
This reverts D39490.
For toolchains generated with older NDKs (<=r13b as far as we tested),
`cpu_set_t` doesn't exist in `sched.h`.
We have to figure out another way to get the number of CPUs without this.
Reviewers: rnk
Reviewed By: rnk
Subscribers: kubamracek, llvm-commits, krytarowski
Differential Revision: https://reviews.llvm.org/D39867
llvm-svn: 317834
Summary:
Update sanitizer_allocator to use new API.
Second patch in a series. First patch https://reviews.llvm.org/D39072
Updates MmapNoAccess / MmapFixed call sites in the saniziter_allocator
to use the new Init/Map APIs instead.
Reviewers: alekseyshl, cryptoad, phosek, mcgrathr, dvyukov
Reviewed By: alekseyshl, cryptoad
Subscribers: dvyukov, mcgrathr, kubamracek
Differential Revision: https://reviews.llvm.org/D38592
llvm-svn: 317586
Summary: This should fix the Windows bots after D39072.
Reviewers: alekseyshl, flowerhack
Reviewed By: flowerhack
Subscribers: llvm-commits, kubamracek
Differential Revision: https://reviews.llvm.org/D39426
llvm-svn: 316937
Summary:
Fixed version of https://reviews.llvm.org/D38437 (fixes Win/Fuchsia failures).
Creating a new revision, since the old one was getting a bit old/crowded.
In Fuchsia, MmapNoAccess/MmapFixedOrDie are implemented using a global
VMAR, which means that MmapNoAccess can only be called once. This works
for the sanitizer allocator but *not* for the Scudo allocator.
Hence, this changeset introduces a new ReservedAddressRange object to
serve as the new API for these calls. In this changeset, the object
still calls into the old Mmap implementations.
The next changeset two changesets will convert the sanitizer and scudo
allocators to use the new APIs, respectively. (ReservedAddressRange will
replace the SecondaryHeader in Scudo.)
Finally, a last changeset will update the Fuchsia implementation.
Reviewers: alekseyshl, cryptoad, phosek
Reviewed By: alekseyshl, cryptoad
Subscribers: kubamracek
Differential Revision: https://reviews.llvm.org/D39072
llvm-svn: 316934
The SanitizerCommon.ReservedAddressRangeUnmap test fails on Windows:
FAIL: SanitizerCommon-Unit :: ./Sanitizer-x86_64-Test.exe/SanitizerCommon.ReservedAddressRangeUnmap (34003 of 35554)
******************** TEST 'SanitizerCommon-Unit :: ./Sanitizer-x86_64-Test.exe/SanitizerCommon.ReservedAddressRangeUnmap' FAILED ********************
Note: Google Test filter = SanitizerCommon.ReservedAddressRangeUnmap
[==========] Running 1 test from 1 test case.
[----------] Global test environment set-up.
[----------] 1 test from SanitizerCommon
[ RUN ] SanitizerCommon.ReservedAddressRangeUnmap
==3780==ERROR: SanitizerTool failed to deallocate 0x1000 (4096) bytes at address 0x0000000c3000 (error code: 487)
==3780==Sanitizer CHECK failed: E:\b\build\slave\win_upload_clang\build\src\third_party\llvm\projects\compiler-rt\lib\sanitizer_common\sanitizer_win.cc:129 (("unable to unmap" && 0)) != (0) (0, 0)
********************
Testing: 0 .. 10.. 20.. 30.. 40.. 50.. 60.. 70.. 80.. 90..
Testing Time: 299.76s
********************
Failing Tests (1):
SanitizerCommon-Unit :: ./Sanitizer-x86_64-Test.exe/SanitizerCommon.ReservedAddressRangeUnmap
> In Fuchsia, MmapNoAccess/MmapFixedOrDie are implemented using a global
> VMAR, which means that MmapNoAccess can only be called once. This works
> for the sanitizer allocator but *not* for the Scudo allocator.
>
> Hence, this changeset introduces a new ReservedAddressRange object to
> serve as the new API for these calls. In this changeset, the object
> still calls into the old Mmap implementations.
>
> The next changeset two changesets will convert the sanitizer and scudo
> allocators to use the new APIs, respectively. (ReservedAddressRange will
> replace the SecondaryHeader in Scudo.)
>
> Finally, a last changeset will update the Fuchsia implementation.
>
> Patch by Julia Hansbrough
>
> Differential Revision: https://reviews.llvm.org/D38437
llvm-svn: 315553
In Fuchsia, MmapNoAccess/MmapFixedOrDie are implemented using a global
VMAR, which means that MmapNoAccess can only be called once. This works
for the sanitizer allocator but *not* for the Scudo allocator.
Hence, this changeset introduces a new ReservedAddressRange object to
serve as the new API for these calls. In this changeset, the object
still calls into the old Mmap implementations.
The next changeset two changesets will convert the sanitizer and scudo
allocators to use the new APIs, respectively. (ReservedAddressRange will
replace the SecondaryHeader in Scudo.)
Finally, a last changeset will update the Fuchsia implementation.
Patch by Julia Hansbrough
Differential Revision: https://reviews.llvm.org/D38437
llvm-svn: 315533
In Fuchsia, MmapNoAccess/MmapFixedOrDie are implemented using a global
VMAR, which means that MmapNoAccess can only be called once. This works
for the sanitizer allocator but *not* for the Scudo allocator.
Hence, this changeset introduces a new ReservedAddressRange object to
serve as the new API for these calls. In this changeset, the object
still calls into the old Mmap implementations.
The next changeset two changesets will convert the sanitizer and scudo
allocators to use the new APIs, respectively. (ReservedAddressRange will
replace the SecondaryHeader in Scudo.)
Finally, a last changeset will update the Fuchsia implementation.
Patch by Julia Hansbrough
Differential Revision: https://reviews.llvm.org/D38759
llvm-svn: 315493
Summary:
Adds a fallback mode to procmaps when the symbolizer
fails to locate a module for a given address by using
dl_iterate_phdr.
Reviewers: kubamracek, rnk, vitalybuka, eugenis
Reviewed By: eugenis
Subscribers: srhines, llvm-commits
Differential Revision: https://reviews.llvm.org/D37269
llvm-svn: 314713
Summary:
Adds a fallback mode to procmaps when the symbolizer
fails to locate a module for a given address by using
dl_iterate_phdr.
Reviewers: kubamracek, rnk, vitalybuka, eugenis
Reviewed By: eugenis
Subscribers: srhines, llvm-commits
Differential Revision: https://reviews.llvm.org/D37269
llvm-svn: 314671
Unreverting this patch because llvm-clang-lld-x86_64-debian-fast started
passing again before the revert hit. Must've been just a flake.
llvm-svn: 314556
Summary:
Adds a fallback mode to procmaps when the symbolizer
fails to locate a module for a given address by using
dl_iterate_phdr.
Reviewers: kubamracek, rnk, vitalybuka, eugenis
Reviewed By: eugenis
Subscribers: srhines, llvm-commits
Differential Revision: https://reviews.llvm.org/D37269
llvm-svn: 314431
Chandler Carruth