Commit Graph

13616 Commits

Author SHA1 Message Date
Luís Marques 43fa23a01f [Sanitizer][NFC] Fix typo 2021-02-19 17:46:02 +00:00
Matthew Malcomson c1653b8cc7 Hwasan InitPrctl check for error using internal_iserror
When adding this function in https://reviews.llvm.org/D68794 I did not
notice that internal_prctl has the API of the syscall to prctl rather
than the API of the glibc (posix) wrapper.

This means that the error return value is not necessarily -1 and that
errno is not set by the call.

For InitPrctl this means that the checks do not catch running on a
kernel *without* the required ABI (not caught since I only tested this
function correctly enables the ABI when it exists).
This commit updates the two calls which check for an error condition to
use internal_iserror. That function sets a provided integer to an
equivalent errno value and returns a boolean to indicate success or not.

Tested by running on a kernel that has this ABI and on one that does
not. Verified that running on the kernel without this ABI the current
code prints the provided error message and does not attempt to run the
program. Verified that running on the kernel with this ABI the current
code does not print an error message and turns on the ABI.
This done on an x86 kernel (where the ABI does not exist), an AArch64
kernel without this ABI, and an AArch64 kernel with this ABI.

In order to keep running the testsuite on kernels that do not provide
this new ABI we add another option to the HWASAN_OPTIONS environment
variable, this option determines whether the library kills the process
if it fails to enable the relaxed syscall ABI or not.
This new flag is `fail_without_syscall_abi`.
The check-hwasan testsuite results do not change with this patch on
either x86, AArch64 without a kernel supporting this ABI, and AArch64
with a kernel supporting this ABI.

Differential Revision: https://reviews.llvm.org/D96964
2021-02-19 16:30:56 +00:00
Vitaly Buka df2940aa85 [android] Fix testing adb call
Looks like some debug leftover which force another retry.
2021-02-18 17:24:07 -08:00
Petr Hosek 5fbd1a333a [Coverage] Store compilation dir separately in coverage mapping
We currently always store absolute filenames in coverage mapping.  This
is problematic for several reasons. It poses a problem for distributed
compilation as source location might vary across machines.  We are also
duplicating the path prefix potentially wasting space.

This change modifies how we store filenames in coverage mapping. Rather
than absolute paths, it stores the compilation directory and file paths
as given to the compiler, either relative or absolute. Later when
reading the coverage mapping information, we recombine relative paths
with the working directory. This approach is similar to handling
ofDW_AT_comp_dir in DWARF.

Finally, we also provide a new option, -fprofile-compilation-dir akin
to -fdebug-compilation-dir which can be used to manually override the
compilation directory which is useful in distributed compilation cases.

Differential Revision: https://reviews.llvm.org/D95753
2021-02-18 14:34:39 -08:00
Petr Hosek fbf8b957fd Revert "[Coverage] Store compilation dir separately in coverage mapping"
This reverts commit 97ec8fa5bb since
the test is failing on some bots.
2021-02-18 12:50:24 -08:00
Petr Hosek 97ec8fa5bb [Coverage] Store compilation dir separately in coverage mapping
We currently always store absolute filenames in coverage mapping.  This
is problematic for several reasons. It poses a problem for distributed
compilation as source location might vary across machines.  We are also
duplicating the path prefix potentially wasting space.

This change modifies how we store filenames in coverage mapping. Rather
than absolute paths, it stores the compilation directory and file paths
as given to the compiler, either relative or absolute. Later when
reading the coverage mapping information, we recombine relative paths
with the working directory. This approach is similar to handling
ofDW_AT_comp_dir in DWARF.

Finally, we also provide a new option, -fprofile-compilation-dir akin
to -fdebug-compilation-dir which can be used to manually override the
compilation directory which is useful in distributed compilation cases.

Differential Revision: https://reviews.llvm.org/D95753
2021-02-18 12:27:42 -08:00
Jianzhou Zhao 063a6fa87e [dfsan] Add origin tls/move/read APIs
This is a part of https://reviews.llvm.org/D95835.

Added
1) TLS storage
2) a weak global used to set by instrumented code
3) move origins

These APIs are similar to MSan's APIs
  https://github.com/llvm/llvm-project/blob/main/compiler-rt/lib/msan/msan_poisoning.cpp
We first improved MSan's by https://reviews.llvm.org/D94572 and https://reviews.llvm.org/D94552.
So the correctness has been verified by MSan.
After the DFSan instrument code is ready, we wil be adding more test
cases

4) read

To reduce origin tracking cost, some of the read APIs return only
the origin from the first taint data.

Note that we did not add origin set APIs here because they are related
to code instrumentation, will be added later with IR transformation
code.

Reviewed-by: morehouse

Differential Revision: https://reviews.llvm.org/D96564
2021-02-18 17:48:20 +00:00
Fangrui Song 833d4d8e89 [profile] Make {__start_,__stop_}__llvm_prf_* symbols undefined weak
To make a kind of metadata section usage work, we want to drop the
`__start_/__stop_ references retain C identifier name sections` rule from LLD (see D96914).

If an application has no `__llvm_prf_data` input section surviving --gc-sections,
LLD will error for undefined hidden `{__start_,__stop_}__llvm_prf_*` from `libclang_rt.profile-*`.
Other `__llvm_prf_*` sections have similar issues.

Making the references weak can address the problem.
This probably enables the opportunity to drop zero size dummy sections in `InstrProfilingPlatformLinux.c`.

Reviewed By: davidxl

Differential Revision: https://reviews.llvm.org/D96936
2021-02-17 23:33:13 -08:00
Fangrui Song da59c2e4dc [GWP-ASan] Change sys/cdefs.h to features.h
sys/cdefs.h is a glibc internal header which is not supposed to be included by applications.
(Some libc implementations provide this file for compatibility.)
Android features.h includes sys/cdefs.h, so we can include features.h instead.

This change makes `ninja gwp_asan` build on musl.
2021-02-17 20:03:16 -08:00
Fangrui Song 58ecfccd0d [profile] Add __attribute__((used)) to zero size dummy sections
D14468 added these dummy sections. This patch adds `__attribute__((used))` so
that when compiled by GCC>=11 or (expected, D96838) Clang>=13 on some ELF platforms,
these sections will get SHF_GNU_RETAIN to make sure they will not be discarded
by ld --gc-sections.

We are trying to get rid of LLD's "__start_/__stop_ references retain C identifier name sections" rule.
If LLD drops the rule in the future (we will retain compatibility for `__llvm_prf_*` for a while),
`__llvm_prf_*` will need to have the SHF_GNU_RETAIN flag, otherwise:

```
// __llvm_prf_cnts/__llvm_prf_data usually exist, but {names,vnds} may not exist.
// Such diagnostics will happen with {cnts,data} as well if no input object file is instrumented.
% clang++ -fprofile-generate a.cc -fuse-ld=lld -Wl,--gc-sections
ld.lld: error: undefined hidden symbol: __start___llvm_prf_names
>>> referenced by InstrProfilingPlatformLinux.c
>>>               InstrProfilingPlatformLinux.c.o:(__llvm_profile_begin_names) in archive /tmp/RelA/lib/clang/13.0.0/lib/linux/libclang_rt.profile-x86_64.a
...
```

Differential Revision: https://reviews.llvm.org/D96902
2021-02-17 19:22:25 -08:00
Aaron Green 10993bf072 Bugfix for collecting features from very small DSOs.
During unit tests, it was observed that crafting an artificially small DSO could cause OOB memory to be accessed. This change fixes that (but again, the affected DSOs are unlikely to ever occur outside unit tests).

Reviewed By: morehouse, charco

Differential Revision: https://reviews.llvm.org/D94507
2021-02-17 13:04:49 -08:00
Dmitry Vyukov fb19400d4e tsan: fix mmap_lots test
If tsan runtime will try to allocate something during exit handling,
the allocation will fail because there is no VA whatsoever.
It's observed to fail with the following error in some cases:
failed to allocate 0x1000 (4096) bytes of DTLS_NextBlock.
So terminate the process immediately.

Reviewed-in: https://reviews.llvm.org/D96874
2021-02-17 19:03:17 +01:00
Jonny Farley f8ed31cd99 [Fuzzer][Test] Use %python substitution for trace-malloc-unbalanced.test
This test was found to fail for some of our downstream builds, on
computers where python was not on the default $PATH. Therefore
add a %python substitution to use sys.executable, based on similar
solutions for python calls in tests elsewhere in LLVM.

Differential Revision: https://reviews.llvm.org/D96799
2021-02-17 15:00:46 +00:00
Alex Richardson ca9815fc24 [sanitizers] Define SANITIZER_INTERCEPTOR_HOOKS on FreeBSD
This fixes the weak_hooks.cpp test on FreeBSD. Since this feature appears
to be supported on almost all platforms, it might also make sense to turn
it into an opt-out list instead of being opt-in.

Reviewed By: krytarowski

Differential Revision: https://reviews.llvm.org/D96255
2021-02-17 10:36:47 +00:00
Joachim Protze ac6c13bfc4 [sanitizers] Pass CMAKE_C_FLAGS into TSan buildgo script
When compiling with ccache, compiler commands get split into smaller steps
and clang's default -Wunused-command-line-argument complains about unused
include directory arguments. In combination -Werror, compilation aborts.

This patch passes the CMAKE_C_FLAGS into the build script. Configuring with
-DCMAKE_C_FLAGS=-Wno-unused-command-line-argument allows successful testing.

Differential Revision: https://reviews.llvm.org/D96762
2021-02-17 09:14:23 +01:00
Dmitry Vyukov 0984b8de0b tsan: don't leave unmapped hole in non-app memory
If an app mmaps lots of memory, a user mmap may end up
in the tsan region for traces. Shadow for this range
overlaps with shadow for other user regions.
This causes havok: from false positives to crashes.
Don't leave unmapped holes in the traces region.

Reviewed-in: https://reviews.llvm.org/D96697
2021-02-17 08:37:04 +01:00
Adhemerval Zanella b9d3234c70 [sanitizer] [arm] Disable some LSAN tests for arm-linux-gnueabihf
Reinstate D90628 since the fix done by D96337 does not change the outcome
of the https://bugs.llvm.org/show_bug.cgi?id=48052
2021-02-15 09:35:02 -03:00
Amy Huang 60a55337e0 Fix test in external_symbolizer_path.cpp, by adding a REQUIRES: static-lib.
Follow-up to https://reviews.llvm.org/D94563.
2021-02-12 14:04:43 -08:00
Amy Huang 1e92b1730f Fix test failure for external_symbolizer_path.cpp 2021-02-12 12:49:41 -08:00
Amy Huang 5815b71eac Disable test in external_symbolizer_path.cpp temporarily to debug test failures. 2021-02-12 11:51:13 -08:00
Marco Vanotti 0fe4701e51 Expand unit tests for fuzzer::Merger
This change adds additional unit tests for fuzzer::Merger::Parse and fuzzer::Merger::Merge in anticipation of additional changes to the merge control file format to support cross-process fuzzing.

It modifies the parameter handling of Merge slightly in order to make NewFeatures and NewCov consistent with NewFiles; namely, Merge *replaces* the contents of these output parameters rather than accumulating them (thereby fixing a buggy return value).

This is change 1 of (at least) 18 for cross-process fuzzing support.

Reviewed By: morehouse

Differential Revision: https://reviews.llvm.org/D94506
2021-02-12 11:18:34 -08:00
Amy Huang 394913fdb9 Try to fix external_symbolizer_path.cpp test to stop breaking on buildbots.
Not sure what the issue is, but it might be because the test copies
llvm-symbolizer to a different directory, and it can't find libc++.
Try to add some REQUIRES that we use in other tests where we copy
llvm tools out of their original directories.
2021-02-12 10:16:49 -08:00
Jianzhou Zhao a7538fee3a [dfsan] Comment out ChainOrigin temporarily
It was added by D96160, will be used by D96564.
Some OS got errors if it is not used.
Comment it out for the time being.
2021-02-12 18:13:24 +00:00
Matthew G McGovern 81b1d3da09 [sanitizers][Windows] Implement __sanitizer_purge_allocator for Win64
Windows' memory unmapping has to be explicit, there is no madvise.
Similarly, re-mapping memory has to be explicit as well. This patch
implements a basic method for remapping memory which was previously
returned to the OS on Windows.

Patch by Matthew G. McGovern and Jordyn Puryear
2021-02-12 09:49:04 -08:00
Jianzhou Zhao 7590c0078d [dfsan] Turn off THP at dfsan_flush
https://reviews.llvm.org/D89662 turned this off at dfsan_init.
dfsan_flush also needs to turn it off.
W/o this a program may get more and more memory usage after hours.

Reviewed-by: morehouse

Differential Revision: https://reviews.llvm.org/D96569
2021-02-12 17:10:09 +00:00
Vitaly Buka 0b3d31222d [ASAN][NFC] Improve language 2021-02-12 02:55:58 -08:00
Vitaly Buka fc05b2d9e5 [NFC][ProfileData] Improve language 2021-02-12 02:55:58 -08:00
Jianzhou Zhao 083d45b21c [dfsan] Fix building OriginAddr at non-linux OS
Fix the broken build by D96545
2021-02-12 05:02:14 +00:00
Vitaly Buka f2133f2e31 [NFC,memprof] Update test after D96319 2021-02-11 16:36:16 -08:00
Julian Lettner 9360f1a191 [Sanitizer] Fix sanitizer tests without reducing optimization levels
As discussed, these tests are compiled with optimization to mimic real
sanitizer usage [1].

Let's mark relevant functions with `noinline` so we can continue to
check against the stack traces in the report.

[1] https://reviews.llvm.org/D96198

This reverts commit 04af72c542.

Differential Revision: https://reviews.llvm.org/D96357
2021-02-11 15:22:20 -08:00
Jianzhou Zhao 5ebbc5802f [dfsan] Introduce memory mapping for origin tracking
Reviewed-by: morehouse

Differential Revision: https://reviews.llvm.org/D96545
2021-02-11 22:33:16 +00:00
Jianzhou Zhao 2d9c6e10e9 [dfsan] Add origin chain utils
This is a part of https://reviews.llvm.org/D95835.

The design is based on MSan origin chains.

An 4-byte origin is a hash of an origin chain. An origin chain is a
pair of a stack hash id and a hash to its previous origin chain. 0 means
no previous origin chains exist. We limit the length of a chain to be
16. With origin_history_size = 0, the limit is removed.

The change does not have any test cases yet. The following change
will be adding test cases when the APIs are used.

Reviewed-by: morehouse

Differential Revision: https://reviews.llvm.org/D96160
2021-02-11 19:10:11 +00:00
Fangrui Song 0dc6122dd3 [asan][test] Fix Linux/odr-violation.cpp on gcc 2021-02-11 09:53:48 -08:00
Adhemerval Zanella 88d1724d9b [sanitizer] Fix suffix-log-path_test.c on arm-linux-gnu
The recent suffix-log-path_test.c checks for a full stacktrace and
since on some arm-linux-gnu configuration the slow unwinder is used
on default (when the compiler emits thumb code as default), it
requires -funwind-tables on tests.

It also seems to fix the issues disable by d025df3c1d.

Reviewed By: ostannard

Differential Revision: https://reviews.llvm.org/D96337
2021-02-11 14:26:04 -03:00
Jianzhou Zhao b4993cf54d [sanitizer] Move MSan's chained_origin_depot to sanitizer_common
https://reviews.llvm.org/D95835 implements origin tracking for DFSan.
It reuses the chained origin depot of MSan.

This change moves the utility to sanitizer_common to share between
MSan and DFSan.

Reviewed-by: eugenis, morehouse

Differential Revision: https://reviews.llvm.org/D96319
2021-02-11 01:25:56 +00:00
Vedant Kumar 897bec937e [test/ubsan] Allow unused FileCheck prefixes in TypeCheck/vptr.cpp
This test started failing after https://reviews.llvm.org/D95849
defaulted --allow-unused-prefixes to false.

Taking a look at the test, I didn't see an obvious need to add
OS-specific check lines for each supported value of %os.

rdar://74207657
2021-02-10 16:06:51 -08:00
Fangrui Song 096f547208 [asan][test] Fix Linux/odr-violation.cpp after symbolizer change D95927 2021-02-10 14:27:12 -08:00
Mitch Phillips b93786907c [GWP-ASan] Add back some headers removed by IWYU.
These headers are required for Android.

Reviewed By: eugenis

Differential Revision: https://reviews.llvm.org/D96374
2021-02-10 09:13:45 -08:00
Todd Lipcon 5dd29d9922
Fix xray fdr mode to allow multiple flushes
Reviewed By: dberris

Differential Revision: https://reviews.llvm.org/D96382
2021-02-10 12:57:24 +11:00
Nico Weber 23cd8d51ad Try to fix compiler-rt tests after 87dbdd2e3b on mac and win
These tests use `--check-prefix=CHECK-%os` but then didn't have
a CHECK line for every os.

In most tests, the linux expectations were sufficient (they match
the "wrap_" prefix with .*), so just remove the check-prefix there.
In the places where this didn't easily work, make sure there are
at least CHECK-Windows and CHECK-Darwin lines.
2021-02-09 09:28:52 -05:00
Roland McGrath 4c9adbb287 [scudo/standalone] Use .arch_extension memtag, not mte
GNU binutils accepts only `.arch_extension memtag` while Clang
accepts either that or `.arch_extension mte` to mean the same thing.

Reviewed By: pcc

Differential Revision: https://reviews.llvm.org/D95996
2021-02-08 12:24:47 -08:00
Mitch Phillips 3d8823b8e4 [GWP-ASan] Add aligned allocations.
Adds a new allocation API to GWP-ASan that handles size+alignment
restrictions.

Reviewed By: cryptoad, eugenis

Differential Revision: https://reviews.llvm.org/D94830
2021-02-08 11:22:29 -08:00
Julian Lettner 04af72c542 [Sanitizer] Fix failing sanitizer tests
The new pass manager was enabled by default [1].

The commit message states the following relevant differences:
  * The inliner works slightly differently
  * -O1 does some amount of inlining

These tests are affected because they specify `-O1` and then check the
reported stack trace.

[1] https://reviews.llvm.org/D95380

Differential Revision: https://reviews.llvm.org/D96198
2021-02-08 09:56:32 -08:00
Jianzhou Zhao 0f3fd3b281 [dfsan] Add thread registration
This is a part of https://reviews.llvm.org/D95835.

This change is to address two problems
1) When recording stacks in origin tracking, libunwind is not async signal safe. Inside signal callbacks, we need
to use fast unwind. Fast unwind needs threads
2) StackDepot used by origin tracking is not async signal safe, we set a flag per thread inside
a signal callback to prevent from using it.

The thread registration is similar to ASan and MSan.

Related MSan changes are
* 98f5ea0dba
* f653cda269
* 5a7c364343

Some changes in the diff are used in the next diffs
1) The test case pthread.c is not very interesting for now. It will be
  extended to test origin tracking later.
2) DFsanThread::InSignalHandler will be used by origin tracking later.

Reviewed-by: morehouse

Differential Revision: https://reviews.llvm.org/D95963
2021-02-05 17:38:59 +00:00
Emily Shi 039567b664 [Darwin] Switch to new logging api for sanitizers
Switch to new logging api added in [[ https://developer.apple.com/documentation/os/os_log_error | macOS 10.12 ]] that is more memory safe and enables us to label the log messages in the future. Falls back to old API if ran on older OS versions.

Commited by Dan Liew on behalf of Emily Shi.

rdar://25181524

Reviewed By: delcypher, yln

Differential Revision: https://reviews.llvm.org/D95977
2021-02-04 21:04:51 -08:00
Vitaly Buka 758928e3b8 [sanitizer] Fix the test on android 2021-02-04 18:02:02 -08:00
Amy Huang 8d7d2deb37 Fix test case from D94563.
The added test case failed on ppc, android, and other buildbots,
so require x86 targets.
2021-02-04 16:34:39 -08:00
Amy Huang 4737aab53d Fix a test case after committing D94563.
D94563 implemented `ReadBinaryName` on Windows, which causes a test case
to now pass, so remove the `XFAIL: windows-msvc` line.
2021-02-04 16:06:51 -08:00
Amy Huang 9ba623c655 [asan] Add %d variable to external_symbolizer_path option, so that user can specify paths relative to the location of the binary.
We want way to set a path to llvm-symbolizer that isn't relative
to the current working directory; this change adds a variable that
expands to the path relative to the current binary.
This approach came from comments in https://reviews.llvm.org/D93070

Differential Revision: https://reviews.llvm.org/D94563
2021-02-04 15:43:02 -08:00
Bill Torpey dd5c2b8de9 [sanitizer] Add suffix to report file name
For those using a GUI, it can be very helpful to have a
particular suffix appended to the report file name, so
it can be opened with a double-click.

(see also: https://github.com/google/sanitizers/issues/951)

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D46546
2021-02-04 13:50:11 -08:00