dfbe59b03d
[libFuzzer] add InsertRepeatedBytes and EraseBytes.
...
New mutation: InsertRepeatedBytes.
Updated mutation: EraseByte => EraseBytes.
This helps https://github.com/google/sanitizers/issues/710
where libFuzzer was not able to find a known bug.
Now it finds it in minutes.
Hopefully, the change is general enough to help other targets.
llvm-svn: 278687
2016-08-15 17:48:28 +00:00
b4bbc3bb7a
[sanitizers] trace buffer API to use user-allocated buffer.
...
Differential Revision: https://reviews.llvm.org/D23185
llvm-svn: 277859
2016-08-05 20:09:53 +00:00
c135b55ae0
[libFuzzer] add hooks for strstr, strcasestr, strcasecmp, strncasecmp
...
llvm-svn: 275648
2016-07-15 23:27:19 +00:00
f0b3e85f4e
[libfuzzer] moving is_ascii handler inside mutation dispatcher.
...
Summary: It also fixes a bug, when first random might not be ascii.
Differential Revision: http://reviews.llvm.org/D21573
llvm-svn: 273611
2016-06-23 20:44:48 +00:00
1873a496e2
[LibFuzzer] Declare and use sanitizer functions in ``fuzzer::ExternalFunctions``
...
This fixes linking problems on OSX.
Unfortunately it turns out we need to use an instance of the
``fuzzer::ExternalFunctions`` object in several places so this
commit also replaces all instances with a single global instance.
It also turns out initializing a global ``fuzzer::ExternalFunctions``
before main is entered (i.e. letting the object be initialised by the
global initializers) is not safe (on OSX the call to ``Printf()`` in the
CTOR crashes if it is called from a global initializer) so we instead
have a global ``fuzzer::ExternalFunctions*`` and initialize it inside
``FuzzerDriver()``.
Multiple unit tests depend also depend on the
``fuzzer::ExternalFunctions*`` global so a ``main()`` function has been
added that initializes it before running any tests.
Differential Revision: http://reviews.llvm.org/D20943
llvm-svn: 272072
2016-06-07 23:32:50 +00:00
41d6683c39
[libfuzzer] custom crossover interface function.
...
Differential Revision: http://reviews.llvm.org/D21089
llvm-svn: 272054
2016-06-07 20:22:15 +00:00
1f88b12272
[libfuzzer] prune_corpus option for disabling pruning during the load.
...
Summary:
The option is very useful for testing, plus I intend to measure
its effect on fuzzer effectiveness.
Differential Revision: http://reviews.llvm.org/D21084
llvm-svn: 272035
2016-06-07 18:16:32 +00:00
70fd3e412a
[libfuzzer] hiding custom mutator handling in MutationDispatcher.
...
Summary: Refactoring, no functional changes.
Differential Revision: http://reviews.llvm.org/D20975
llvm-svn: 271740
2016-06-03 21:34:29 +00:00
d3c33116fd
[LibFuzzer] Reimplement how the optional user functions are called.
...
The motivation for this change is to fix linking issues on OSX.
However this only partially fixes linking issues (the uninstrumented
tests and a few others won't succesfully link yet).
This change introduces a struct of function pointers
(``fuzzer::ExternalFuntions``) which when initialised will point to the
optional functions if they are available. Currently these
``LLVMFuzzerInitialize`` and ``LLVMFuzzerCustomMutator`` functions.
Two implementations of ``fuzzer::ExternalFunctions`` constructor are
provided one for Linux and one for OSX.
The OSX implementation uses ``dlsym()`` because the prior implementation
using weak symbols does not work unless the additional flags are passed
to the linker.
The Linux implementation continues to use weak symbols because the
``dlsym()`` approach does not work unless additional flags are passed
to the linker.
Differential Revision: http://reviews.llvm.org/D20741
llvm-svn: 271491
2016-06-02 05:48:02 +00:00
0edb563f27
[libFuzzer] make OOM-handling more portable. Instead of sending a signal to the main fuzzing thread, print the message in the getrusage thread and exit.
...
llvm-svn: 270945
2016-05-27 00:54:15 +00:00
8fc3a27c5c
[libFuzzer] more refactoring: make sure CurrentUnitData is awlays a valid pointer to read from
...
llvm-svn: 270942
2016-05-27 00:21:33 +00:00
d8384122a3
[libFuzzer] more refactoring around CurrentUnit. Also add a threading test on which we currently have a race (when reporting bugs from multiple threads)
...
llvm-svn: 270929
2016-05-26 22:17:32 +00:00
f26017baf9
[libFuzzer] refactor: hide CurrentUnitData inside an interface function. NFC
...
llvm-svn: 270922
2016-05-26 21:32:30 +00:00
4b92326b17
[libFuzzer] when there is a leak in the existing corpus report the reproducer properly
...
llvm-svn: 270905
2016-05-26 20:25:49 +00:00
f1f3f93c9e
[libFuzzer] reimplement the way we do -only_ascii to allow more 'const' in function declarations. Add a test for -only_ascii. NFC intended
...
llvm-svn: 270900
2016-05-26 20:03:02 +00:00
af432a45e3
[libfuzzer] Trying random unit prefixes during corpus load.
...
Differential Revision: http://reviews.llvm.org/D20301
llvm-svn: 270632
2016-05-24 23:14:29 +00:00
3868e468fe
[LibFuzzer]
...
Work around crashes in ``__sanitizer_malloc_hook()`` under Mac OSX.
Under Mac OSX we intercept calls to malloc before thread local
storage is initialised leading to a crash when accessing
``AllocTracer``. To workaround this ``AllocTracer`` is only accessed
in the hook under Linux. For symmetry ``__sanitizer_free_hook()``
is also modified in the same way.
To support this change a set of new macros
LIBFUZZER_LINUX and LIBFUZZER_APPLE has been defined which can be
used to check the target being compiled for.
Differential Revision: http://reviews.llvm.org/D20402
llvm-svn: 270145
2016-05-19 22:00:33 +00:00
8b0d90a6d4
[libFuzzer] simplify FuzzerInterface.h
...
llvm-svn: 269448
2016-05-13 18:04:35 +00:00
1aa501e7e8
[libfuzzer] Refactoring coverage state-management code.
...
It is now less state-dependent and will allow easier comparing of
coverages of different units.
Differential Revision: http://reviews.llvm.org/D20085
llvm-svn: 269140
2016-05-10 23:43:15 +00:00
8b8f7a3cda
[libFuzzer] enhance -rss_limit_mb and enable by default. Now it will print the OOM reproducer.
...
llvm-svn: 268821
2016-05-06 23:38:07 +00:00
52b394e981
[libFuzzer] add exeprimental -rss_limit_mb flag to fight against OOMs
...
llvm-svn: 268807
2016-05-06 21:58:35 +00:00
baf7fd0b16
[libFuzzer] print stats after running individual inputs
...
llvm-svn: 268547
2016-05-04 20:44:50 +00:00
2fe9304d62
[libFuzzer] enable detect_leaks=1, add proper docs
...
llvm-svn: 268088
2016-04-29 18:49:55 +00:00
7018a1aaa4
[libFuzzer] disable leak detection if we have tried it for 1000 times w/o finding a leak
...
llvm-svn: 267770
2016-04-27 19:52:34 +00:00
9ba19182be
[libFuzzer] remove dead code
...
llvm-svn: 267455
2016-04-25 19:41:45 +00:00
1bfd583d82
[libFuzzer] added -detect_leaks flag (0 by default for now). When enabled, it will help finding leaks while fuzzing
...
llvm-svn: 266838
2016-04-20 00:24:21 +00:00
ebb932d060
[libFuzzer] try to print correct time in seconds when reporting a timeout. Don't report timeouts while still loading the corpus.
...
llvm-svn: 266693
2016-04-18 22:50:39 +00:00
f389ae12c1
[libFuzzer] handle SIGTERM
...
llvm-svn: 264338
2016-03-24 21:03:58 +00:00
49e409068a
[libFuzzer] add a flag close_fd_mask so that we can silence spammy targets by closing stderr/stdout
...
llvm-svn: 263831
2016-03-18 20:58:29 +00:00
945761b8c2
[libFuzzer] improve -merge functionality
...
llvm-svn: 263769
2016-03-18 00:23:29 +00:00
c5575aabd6
[libFuzzer] deprecate several flags
...
llvm-svn: 263739
2016-03-17 19:59:39 +00:00
64d24578d8
[libFuzzer] try to use max_len based on the items of the corpus instead of blindly defaulting to 64 bytes.
...
llvm-svn: 263323
2016-03-12 01:57:04 +00:00
3d95dd9149
[libFuzzer] deprecate exit_on_first flag
...
llvm-svn: 262417
2016-03-01 22:33:14 +00:00
228d5b1ce4
[libFuzzer] add generic signal handlers so that libFuzzer can report at least something if ASan is not handlig the signals for us. Remove abort_on_timeout flag.
...
llvm-svn: 262415
2016-03-01 22:19:21 +00:00
66ff0756e4
[libFuzzer] add -print_final_stats=1 flag
...
llvm-svn: 262084
2016-02-26 22:42:23 +00:00
da63c1d09a
[libFuzzer] initial implementation of path coverage based on -fsanitize-coverage=trace-pc. This does not scale well yet, but already cracks FullCoverageSetTest in seconds
...
llvm-svn: 262073
2016-02-26 21:33:56 +00:00
a35f7d383f
[libFuzzer] only read MaxLen bytes from every file in the corpus to speedup loading the corpus
...
llvm-svn: 261267
2016-02-18 21:49:10 +00:00
8a5bef0fcf
[libFuzzer] remove std::vector operations from hot paths, NFC
...
llvm-svn: 260829
2016-02-13 17:56:51 +00:00
1deb0498f5
[libFuzzer] don't require seed in fuzzer::Mutate, instead use the global Fuzzer object for fuzzer::Mutate. This makes custom mutators fast
...
llvm-svn: 260810
2016-02-13 06:24:18 +00:00
23194963f7
[libFuzzer] simplify CTOR of MutationDispatcher
...
llvm-svn: 260800
2016-02-13 03:46:26 +00:00
292cf0379c
[libFuzzer] get rid of MutationDispatcher::Impl (simplify the code; NFC)
...
llvm-svn: 260799
2016-02-13 03:37:24 +00:00
7ec0c56e07
[libFuzzer] get rid of UserSuppliedFuzzer; NFC
...
llvm-svn: 260798
2016-02-13 03:25:16 +00:00
a399221c32
[libFuzzer] simplify the code around Random. NFC
...
llvm-svn: 260797
2016-02-13 03:00:53 +00:00
ecab57b3ce
[libFuzzer] remove UserSuppliedFuzzer from the interface (it was a bad idea).
...
llvm-svn: 260796
2016-02-13 02:39:30 +00:00
bfbe7fc404
[libFuzzer] allow passing 1 or more files as individual inputs
...
llvm-svn: 259459
2016-02-02 03:03:47 +00:00
54a6363a8f
[libFuzzer] add -timeout_exitcode option
...
llvm-svn: 259265
2016-01-29 23:30:07 +00:00
9768e7f06b
[libFuzzer] add -abort_on_timeout option
...
llvm-svn: 258631
2016-01-23 19:34:19 +00:00
160dcba81f
[libFuzzer] add more fields to DictionaryEntry to count the number of uses and successes
...
llvm-svn: 258589
2016-01-22 23:55:14 +00:00
df91910bd4
Use std::piecewise_constant_distribution instead of ad-hoc binary search.
...
Summary:
Fix the issue with the most recently discovered unit receiving much less attention.
Note: this is the second attempt (prev: r258473). Now, libc++ build is fixed.
Reviewers: aizatsky, kcc
Subscribers: llvm-commits
Differential Revision: http://reviews.llvm.org/D16487
llvm-svn: 258571
2016-01-22 22:28:27 +00:00
d84f74cab7
Revert r258473 as it's breaking the build with libc++
...
Reviewers: kcc
Differential Revision: http://reviews.llvm.org/D16441
llvm-svn: 258479
2016-01-22 03:21:52 +00:00
b008fd4d89
Use std::piecewise_constant_distribution instead of ad-hoc binary search.
...
Summary:
Fix the issue with the most recently discovered unit receiving much less attention.
Note: I had to change the seed for one test to make it pass. Alternatively,
the number of runs could be increased. I believe that the average time of
'foo' discovery is not increased, just seed=1 was particularly convenient
for the previous PRNG scheme used.
Reviewers: aizatsky, kcc
Subscribers: llvm-commits, kcc
Differential Revision: http://reviews.llvm.org/D16419
llvm-svn: 258473
2016-01-22 01:32:34 +00:00
476f0ce31a
[libFuzzer] replace vector with a simpler data structure in the Dictionaries to avoid memory allocations on hot path
...
llvm-svn: 257985
2016-01-16 03:53:32 +00:00
aca7696f4d
[libFuzzer] introduce LLVMFuzzerInitialize
...
llvm-svn: 257980
2016-01-16 01:23:12 +00:00
628bc3ec00
[libFuzzer] move some code from public interface header to a non-public header. NFC
...
llvm-svn: 257963
2016-01-16 00:04:36 +00:00
ae5b9567bc
[libFuzzer] do mutations based on memcmp/strcmp interceptors under a separate flag (-use_memcmp, default=1)
...
llvm-svn: 257873
2016-01-15 06:24:05 +00:00
98abb2c90a
[libFuzzer] make CurrentUnit a POD object instead of vector to avoid extra allocations
...
llvm-svn: 257713
2016-01-13 23:46:01 +00:00
d50a3eedb4
[libFuzzer] make sure we find buffer overflow in the input buffer. Previously, re-using the same vector object was hiding buffer overflows (unless we used annotated vector)
...
llvm-svn: 257701
2016-01-13 23:02:30 +00:00
4174005622
[libFuzzer] when a new unit is discovered using a dictionary, print all used dictionary entries
...
llvm-svn: 257435
2016-01-12 02:36:59 +00:00
b65805a939
[libFuzzer] change the way trace-based mutations are applied. Instead of a custom code just rely on the automatically created dictionary
...
llvm-svn: 257248
2016-01-09 03:08:58 +00:00
8b11f877e4
[libfuzzer] print_new_cov_pcs experimental option.
...
Differential Revision: http://reviews.llvm.org/D15901
llvm-svn: 256882
2016-01-06 00:21:22 +00:00
27ab2d759f
[libFuzzer] make CrossOver just one of the other mutations
...
llvm-svn: 256081
2015-12-19 02:49:09 +00:00
9e48cda9bc
[libFuzzer] compute base64 in-process instead of using an external lib. Since libFuzzer should not depend on anything, just re-implement base64 encoder. PR25746
...
llvm-svn: 254784
2015-12-04 22:29:39 +00:00
2d0ef14f5d
[libFuzzer] add a flag -exact_artifact_path
...
llvm-svn: 254100
2015-11-25 21:40:46 +00:00
a9c2387192
output_csv libfuzzer option
...
Summary:
The option outputs statistics in CSV format preceded by 1 header line.
This is intended for machine processing of the output.
-verbosity=0 should likely be set.
Differential Revision: http://reviews.llvm.org/D14600
llvm-svn: 252856
2015-11-12 04:38:40 +00:00
dc3135db05
[libFuzzer] experimental flag -drill (another search heuristic; Mike Aizatsky's idea)
...
llvm-svn: 252838
2015-11-12 01:02:01 +00:00
e692621a9d
[libFuzzer] when choosing the next unit to mutate, give some preference to the most recent units (they are more likely to be interesting)
...
llvm-svn: 252097
2015-11-04 23:22:25 +00:00
9cc3b0ddb6
[libFuzzer] add -merge flag to merge corpora
...
llvm-svn: 251168
2015-10-24 01:16:40 +00:00
94660b3c36
[libFuzzer] remove some old code; also make __sanitizer_get_total_unique_caller_callee_pairs weak so that newer libFuzzer works with older asan
...
llvm-svn: 251133
2015-10-23 18:37:58 +00:00
2e9fca9f88
[libFuzzer] use the indirect caller-callee counter as an independent search heuristic
...
llvm-svn: 251078
2015-10-22 23:55:39 +00:00
09d2a5f6e1
[libFuzzer] more refactoring the code that checks the coverage. NFC
...
llvm-svn: 251075
2015-10-22 22:56:45 +00:00
007c9b25f4
[libFuzzer] refactoring the code that checks the coverage. NFC
...
llvm-svn: 251074
2015-10-22 22:50:47 +00:00
b36025619c
[libFuzzer] remove the deprecated 'tokens' feature
...
llvm-svn: 251069
2015-10-22 21:48:09 +00:00
fed509e73d
[libFuzzer] add -shuffle flag
...
llvm-svn: 250603
2015-10-17 04:38:26 +00:00
d6edce97fb
[libFuzzer] print a stack trace on timeout
...
llvm-svn: 250571
2015-10-16 23:04:31 +00:00
b91c62b1f3
[libFuzzer] When -test_single_input crashes the test it is not necessary to write crash-file because input is already known to the user. Patch by Mike Aizatsky
...
llvm-svn: 250564
2015-10-16 22:41:47 +00:00
bd5d1cdbb9
[libFuzzer] add -artifact_prefix flag
...
llvm-svn: 249807
2015-10-09 03:57:59 +00:00
20bb5e71b2
[libFuzzer] make LLVMFuzzerTestOneInput (the fuzzer target function) return int instead of void. The actual return value is not *yet* used (and expected to be 0). This change is API breaking, so the fuzzers will need to be updated.
...
llvm-svn: 249214
2015-10-02 23:34:06 +00:00
65d0a1458f
[libFuzzer] remove experimental flag and functionality
...
llvm-svn: 249194
2015-10-02 22:00:32 +00:00
b85db178a0
[libFuzzer] add a flag -max_total_time
...
llvm-svn: 249181
2015-10-02 20:47:55 +00:00
95e82d5b48
[LibFuzzer] test_single_input option to run a single test case.
...
-test_single_input flag specifies a file name with test data.
Review URL: http://reviews.llvm.org/D13359
Patch by Mike Aizatsky!
llvm-svn: 249096
2015-10-01 23:23:06 +00:00
468ed78434
[libFuzzer] remove -iterations as redundant (there is also -num_runs)
...
llvm-svn: 247030
2015-09-08 17:30:35 +00:00
9838b2be87
[libFuzzer] adding a parser for AFL-style dictionaries + tests.
...
llvm-svn: 246800
2015-09-03 20:23:46 +00:00
a9346c2e65
[libFuzzer] honour -only_ascii=1 when reading the initial corpus. Also, remove ugly #ifdef
...
llvm-svn: 246689
2015-09-02 19:08:08 +00:00
12c7837381
[libFuzzer] add two flags, -tbm_depth and -tbm_width to control how the trace-based-mutations are applied
...
llvm-svn: 244712
2015-08-12 01:55:37 +00:00
bc7c0ad24d
[libFuzzer] add -only_ascii flag
...
llvm-svn: 244559
2015-08-11 01:44:42 +00:00
347663b214
Add missing include guard to FuzzerInternal.h, NFC.
...
llvm-svn: 244457
2015-08-10 16:37:40 +00:00
242ca930e8
[libFuzzer] move the mutators to public interface so that custom mutators may reuse these functions directly
...
llvm-svn: 244250
2015-08-06 19:19:55 +00:00
bf29ff2fa5
[libFuzzer] add one more mutation strategy: byte shuffling
...
llvm-svn: 244188
2015-08-06 01:29:13 +00:00
70926aed6b
[libFuzzer] add option -report_slow_units=Nsec to control when slow units are printed
...
llvm-svn: 244152
2015-08-05 21:43:48 +00:00
86a5fba71d
[libFuzzer] more refactoring of the Mutator and adding tests to it
...
llvm-svn: 243818
2015-08-01 02:23:06 +00:00
8ce7424e9c
[libFuzzer] start refactoring the Mutator and adding tests to it
...
llvm-svn: 243817
2015-08-01 01:42:51 +00:00
404c69f2c8
[libFuzzer] allow users to supply their own implementation of rand
...
llvm-svn: 243078
2015-07-24 01:06:40 +00:00
2b7d2e91cc
[libFuzzer] dump long running units to disk
...
llvm-svn: 243031
2015-07-23 18:37:22 +00:00
20e9bcbfc8
[lib/Fuzzer] start getting rid of std::cerr. Sadly, these parts of C++ library used in libFuzzer badly interract with the same code used in the target function and also with dfsan. It's easier to just not use std::cerr than to defeat these issues.
...
llvm-svn: 238078
2015-05-23 01:07:46 +00:00
f3c7cb464e
[lib/Fuzzer] remove -use_coverage_pairs=1, an experimental feature that is unlikely to ever scale
...
llvm-svn: 238063
2015-05-22 22:47:03 +00:00
f342459aa4
[lib/Fuzzer] extend the fuzzer interface to allow user-supplied mutators
...
llvm-svn: 238059
2015-05-22 22:35:31 +00:00
490bbd6fa4
[lib/Fuzzer] change the meaning of -timeout flag: now timeout is applied to every unit of work separately
...
llvm-svn: 237735
2015-05-19 22:12:57 +00:00
cbb2334b7a
[lib/Fuzzer] more efficient reload logic; also don't spam git too much
...
llvm-svn: 237649
2015-05-19 01:06:07 +00:00
2da7b84852
[lib/Fuzzer] when -sync_command=<CMD> is given, periodically execute 'CMD CORPUS' to synchronize with other processes
...
llvm-svn: 237617
2015-05-18 21:34:20 +00:00
96eab65d81
[lib/Fuzzer] Add SHA1 implementation from public domain.
...
Summary:
This adds a SHA1 implementation taken from public domain code.
The change is trivial, but as it involves third-party code I'd like
a second pair of eyes before commit.
LibFuzzer can not use SHA1 from openssl because openssl may not be available
and because we may be fuzzing openssl itself.
Using sha1sum via a pipe is too slow.
Test Plan: n/a
Reviewers: chandlerc
Reviewed By: chandlerc
Subscribers: majnemer, llvm-commits
Differential Revision: http://reviews.llvm.org/D9733
llvm-svn: 237400
2015-05-14 22:41:49 +00:00
Kostya Serebryany