Commit Graph

86 Commits

Author SHA1 Message Date
Kostya Serebryany 59bf1e8b36 [libFuzzer] remove the experimental support for clang coverage instrumentation. This mode has not been used and our experiments with https://github.com/google/fuzzer-test-suite show that this signal is weaker than the SanitizerCoverage
llvm-svn: 332034
2018-05-10 20:12:15 +00:00
Matt Morehouse 14cf71a3a5 [libFuzzer] Guard symbolization with try-lock.
Summary:
When out-of-memory or timeout occurs, threads can be stopped during
symbolization, thereby causing a deadlock when the OOM/TO handlers
attempt symbolization.  We avoid this deadlock by skipping symbolization
if another thread is symbolizing.

Reviewers: kcc

Reviewed By: kcc

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D46605

llvm-svn: 331825
2018-05-08 23:45:05 +00:00
Matt Morehouse 7764a04af0 [libFuzzer] Don't short-circuit from CrashCallback.
Short-circuiting causes tests to fail on Mac since libFuzzer crashes
rather than exiting with an error code when an unexpected signal
happens.

llvm-svn: 331324
2018-05-02 02:55:28 +00:00
Matt Morehouse 52fd169035 [libFuzzer] Report at most one crash per input.
Summary:
Fixes https://github.com/google/sanitizers/issues/788/, a deadlock
caused by multiple crashes happening at the same time.  Before printing
a crash report, we now test and set an atomic flag.  If the flag was
already set, the crash handler returns immediately.

Reviewers: kcc

Reviewed By: kcc

Subscribers: llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D46277

llvm-svn: 331310
2018-05-01 21:01:53 +00:00
Matt Morehouse ddf352b953 [libFuzzer] Include TEMP_MAX_LEN in Fuzzer::PrintStats.
Reviewers: kcc

Reviewed By: kcc

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D43597

llvm-svn: 325817
2018-02-22 19:00:17 +00:00
Matt Morehouse 36c89b3c77 [libFuzzer] Set -experimental_len_control=1000 as default.
Summary:
Experiments using
https://github.com/google/fuzzer-test-suite/tree/master/engine-comparison
show a significant increase in coverage and reduction in corpus size
with this option enabled.

Addresses https://llvm.org/pr36371.

Reviewers: kcc

Reviewed By: kcc

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D42932

llvm-svn: 325050
2018-02-13 20:52:15 +00:00
Petr Hosek eac2b47b9f Reland "[libFuzzer] Support using libc++"
This is needed in case the users of libFuzzer use libc++ in their
code, which the fuzz target (libFuzzer) will be linked against.
When libc++ source is available, we build a private version of it
and link it against libFuzzer which allows using the same static
library against codebases which use both libc++ and libstdc++.

Differential Revision: https://reviews.llvm.org/D37631

llvm-svn: 322755
2018-01-17 20:39:14 +00:00
Petr Hosek 68bc4bd6e5 Revert "[libFuzzer] Support using libc++"
This reverts commit r322604: test is failing for standalone compiler-rt.

llvm-svn: 322689
2018-01-17 17:24:56 +00:00
Petr Hosek a1b57e694e [libFuzzer] Support using libc++
This is needed in case the users of libFuzzer use libc++ in their
code, which the fuzz target (libFuzzer) will be linked against.
When libc++ source is available, we build a private version of it
and link it against libFuzzer which allows using the same static
library against codebases which use both libc++ and libstdc++.

Differential Revision: https://reviews.llvm.org/D37631

llvm-svn: 322604
2018-01-17 00:42:48 +00:00
Kostya Serebryany e9ed2327b6 [libFuzzer] change the strategy for -experimental_len_control to grow max_len slower
llvm-svn: 320531
2017-12-12 23:11:28 +00:00
Kostya Serebryany 2659c63e2e [libFuzzer] honor -use_counters, sligntly change the meaning of -experimental_len_control, call UpdateFeatureFrequency only if instructed by the flag
llvm-svn: 320205
2017-12-08 22:21:42 +00:00
Kostya Serebryany de9bafb162 [libFuzzer] add a flag -malloc_limit_mb
llvm-svn: 319590
2017-12-01 22:12:04 +00:00
Kostya Serebryany 6afa7a54be [libFuzzer] remove stale flags; NFC
llvm-svn: 319572
2017-12-01 19:24:06 +00:00
Kostya Serebryany ad05ee0512 [libFuzzer] add an experimental search heuristic flag -reduce_depth
llvm-svn: 319571
2017-12-01 19:18:38 +00:00
Matt Morehouse 947838c9c5 [libFuzzer] Don't add leaking inputs to corpus.
Reviewers: kcc

Reviewed By: kcc

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D39850

llvm-svn: 317831
2017-11-09 20:44:08 +00:00
Kostya Serebryany a2ca2dcc46 [libFuzzer] handle SIGUSR1/SIGUSR2 and try to exit grafully on these signals
llvm-svn: 317829
2017-11-09 20:30:19 +00:00
Vitaly Buka 7d22324b87 [fuzzer] Fix nested mallocs
Summary: Nested mallocs are possible with internal symbolizer.

Reviewers: kcc

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D39397

llvm-svn: 317186
2017-11-02 04:12:10 +00:00
Vitaly Buka 7dbc1d8433 [fuzzer] Fix threaded stack printing
Reviewers: kcc

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D39397

llvm-svn: 317071
2017-11-01 03:02:59 +00:00
Vitaly Buka df00e89cdc Revert "[fuzzer] Fix threaded stack printing and nested mallocs"
Fails on darwin

Revert "[fuzzer] Script to detect unbalanced allocation in -trace_malloc output"
Needs previous one.

This reverts commit r317034, r317036.

llvm-svn: 317061
2017-11-01 00:55:52 +00:00
Vitaly Buka 58da33e35c [fuzzer] Fix threaded stack printing and nested mallocs
Summary: Nested mallocs are possible with internal symbolizer.

Reviewers: kcc

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D39397

llvm-svn: 317034
2017-10-31 20:49:48 +00:00
Alex Shlyapnikov 5ded0701a9 [Sanitizers-libFuzzer] Addressing coding style issues.
Summary: The result of clang-format and few manual changes (as prompted on D39155).

Reviewers: vitalybuka

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D39211

llvm-svn: 316395
2017-10-23 23:24:33 +00:00
Alex Shlyapnikov 6f1c26f222 [libFuzzer] Periodically purge allocator's quarantine to prolong fuzzing sessions.
Summary:
Fuzzing targets that allocate/deallocate a lot of memory tend to consume
a lot of RSS when ASan quarantine is enabled. Purging quarantine between
iterations and returning memory to OS keeps RSS down and should not
reduce the quarantine effectiveness provided the fuzz target does not
preserve state between iterations (in this case this feature can be turned off).

Based on D39153.

Reviewers: vitalybuka

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D39155

llvm-svn: 316382
2017-10-23 22:04:30 +00:00
Kostya Serebryany 51823d3aae [libFuzzer] tweam use_feature_frequency to be less aggressive; run a dummy input before the seed corpus
llvm-svn: 315657
2017-10-13 01:12:23 +00:00
Kostya Serebryany 4083d54675 [libFuzzer] experimental flag to tweak the corpus distribution. Seems to improve the situation dramatically on the png benchmark and make things worse on a number of micro-puzzles. Needs more A/B testing
llvm-svn: 315407
2017-10-11 01:44:26 +00:00
Max Moroz 330496c3fc [libFuzzer] Disable experimental clang coverage support by default.
Summary:
It can be enabled via "-use_clang_coverage=1" flag. Reason for disabling:
libFuzzer resets Clang Counters and makes it impossible to generate coverage
report for a regular fuzz target (i.e. not standalone build).

Reviewers: kcc

Reviewed By: kcc

Subscribers: kcc

Differential Revision: https://reviews.llvm.org/D38604

llvm-svn: 315029
2017-10-05 22:41:03 +00:00
Kostya Serebryany 93679be037 [libFuzzer] factor out some code into GetSizedFilesFromDir; NFC
llvm-svn: 313081
2017-09-12 21:58:07 +00:00
Max Moroz 3f26dac416 [libfuzzer] Compare TotalNumberOfRuns with MaxNumberOfRuns when testing a memory leak.
Summary:
Fuzzer::TryDetectingAMemoryLeak may call ExecuteCallback which would
increment TotalNumberOfRuns, but it doesn't respect Options.MaxNumberOfRuns
value specified by a user.

Context: https://github.com/google/oss-fuzz/issues/822#issuecomment-328153970

Reviewers: kcc

Reviewed By: kcc

Differential Revision: https://reviews.llvm.org/D37632

llvm-svn: 312993
2017-09-12 02:01:54 +00:00
Kostya Serebryany 082e9a7528 [libFuzzer] tolerate missing files when loading the seed corpus
llvm-svn: 312269
2017-08-31 19:17:15 +00:00
Kostya Serebryany 4faeb87ebe [libFUzzer] change the way we load the seed corpora: instead of loading all files and these executing all files, load and execute them one-by-one. This should reduce the memory usage in many cases
llvm-svn: 312033
2017-08-29 20:51:24 +00:00
Kostya Serebryany 3a8e3c8336 [libFuzzer] refactoring: move reading the seed corpus closer to where it's consumed; NFC
llvm-svn: 311972
2017-08-29 02:05:01 +00:00
George Karpenkov bebcbfb46d [libFuzzer] Use custom allocators for STL containers in libFuzzer.
Avoids ODR violations causing spurious ASAN warnings.

Differential Revision: https://reviews.llvm.org/D37086

llvm-svn: 311866
2017-08-27 23:20:09 +00:00
George Karpenkov 24069a0424 Revert "[libFuzzer] Use custom allocators for STL containers in libFuzzer"
This reverts commit 3539efc2f2218dba2bcbd645d0fe276f2b5cf588.

llvm-svn: 311831
2017-08-26 17:50:35 +00:00
George Karpenkov d50410bfb1 [libFuzzer] Use custom allocators for STL containers in libFuzzer
Avoids ODR violations causing spurious ASAN container overflow warnings.

Differential Revision: https://reviews.llvm.org/D37086

llvm-svn: 311830
2017-08-26 17:17:37 +00:00
Kostya Serebryany 2eef816e6e [libFuzzer] add -print_funcs=1 (on bey default): print newly discovered functions during fuzzing
llvm-svn: 311797
2017-08-25 20:09:25 +00:00
Kostya Serebryany f65cf64fa2 [libFuzzer] apply changes lost during the migration to compiler-rt
llvm-svn: 311420
2017-08-22 01:28:32 +00:00
George Karpenkov 10ab2ace13 Move libFuzzer to compiler_rt.
Resulting library binaries will be named libclang_rt.fuzzer*, and will
be placed in Clang toolchain, allowing redistribution.

Differential Revision: https://reviews.llvm.org/D36908

llvm-svn: 311407
2017-08-21 23:25:50 +00:00