Commit Graph

123 Commits

Author SHA1 Message Date
Zachary Turner 6fa57ad9bd Resubmit "[LibFuzzer] Split FuzzerUtil for Posix and Windows."
This resubmits r288529, which was resubmitted because it broke a
fuzzer bot.  According to kcc@ the test that broke was flakey
and it is unlikely to be a result of this patch.

llvm-svn: 288549
2016-12-02 23:02:01 +00:00
Zachary Turner 3cfeab7059 Revert "[LibFuzzer] Split FuzzerUtil for Posix and Windows."
This reverts commit r288529, as it seems to introduce some
problems on the Linux bots.

llvm-svn: 288533
2016-12-02 20:54:56 +00:00
Zachary Turner 34dcfb9294 [LibFuzzer] Split FuzzerUtil for Posix and Windows.
Pave the way for separating out platform specific
utility functions into separate files.

Patch by Marcos Pividori
Differential Revision: https://reviews.llvm.org/D27234

llvm-svn: 288529
2016-12-02 19:38:19 +00:00
Zachary Turner 24a148b1d4 [LibFuzzer] Split up some functions among different headers.
In an effort to get libfuzzer working on Windows, we need to make
a distinction between what functions require platform specific
code (e.g. different code on Windows vs Linux) and what code
doesn't.  IO functions, for example, tend to be platform
specific.

This patch separates out some of the functions which will need
to have platform specific implementations into different headers,
so that we can then provide different implementations for each
platform.

Aside from that, this patch contains no functional change.  It
is purely a re-organization.

Patch by Marcos Pividori
Differential Revision: https://reviews.llvm.org/D27230

llvm-svn: 288264
2016-11-30 19:06:14 +00:00
Kostya Serebryany 6c77811a29 [libFuzzer] replace 'auto' with 'auto *' to better follow the LLVM style
llvm-svn: 286870
2016-11-14 19:21:38 +00:00
Kostya Serebryany 53c894d257 [libFuzzer] use a valid ASCII string for a dummy seed corpus
llvm-svn: 286702
2016-11-12 02:27:21 +00:00
Kostya Serebryany fc1c405f98 [libFuzzer] use less stack
llvm-svn: 286689
2016-11-12 00:24:35 +00:00
Kostya Serebryany 8a56917492 [libFuzzer] fix -error_exitcode=N, now with a test
llvm-svn: 285958
2016-11-03 19:31:18 +00:00
Kostya Serebryany bb59ef77ca [libFuzzer] detect leaks after every run when executing fixed inputs (./fuzzer -runs=1000000 my-file)
llvm-svn: 284514
2016-10-18 18:38:08 +00:00
Kostya Serebryany f9b8e8b117 [libFuzzer] better algorithm for -minimize_crash
llvm-svn: 284299
2016-10-15 01:00:24 +00:00
Kostya Serebryany a5f94fb6c9 [libFuzzer] add -trace_cmp=1 (guiding mutations based on the observed CMP instructions). This is a reincarnation of the previously deleted -use_traces, but using a different approach for collecting traces. Still a toy, but at least it scales well. Also fix -merge in trace-pc-guard mode
llvm-svn: 284273
2016-10-14 20:20:33 +00:00
Kostya Serebryany a17d23eaa7 [libFuzzer] add -trace_malloc= flag
llvm-svn: 284149
2016-10-13 19:06:46 +00:00
Kostya Serebryany c5325ed29d [libFuzzer] when shrinking the corpus, delete evicted files previously created by the current process
llvm-svn: 283682
2016-10-08 23:24:45 +00:00
Kostya Serebryany 9adc7c8b4a [libFuzzer] control the reload interval by a flag, make it 10 seconds by default
llvm-svn: 283676
2016-10-08 22:12:14 +00:00
Kostya Serebryany 936b1e774f [libFuzzer] be more careful with memory usage, print peak rss in status lines
llvm-svn: 283418
2016-10-06 05:14:00 +00:00
Kostya Serebryany 1c73f1bf27 [libFuzzer] refactoring to make -shrink=1 work for value profile, added a test.
llvm-svn: 283409
2016-10-05 22:56:21 +00:00
Kostya Serebryany 2455f0d013 [libFuzzer] clear the corpus elements if they are evicted (i.e. smaller elements with proper coverage are found). Make sure we never try to mutate empty element. Print the corpus size in bytes in the status lines
llvm-svn: 283279
2016-10-05 00:25:17 +00:00
Kostya Serebryany d216922a80 [libFuzzer] implement the -shrink=1 option that tires to make elements of the corpus smaller, off by default
llvm-svn: 282995
2016-10-01 01:04:29 +00:00
Kostya Serebryany e7e790bad6 [libFuzzer] remove unused option
llvm-svn: 282971
2016-09-30 22:29:57 +00:00
Kostya Serebryany 5ff481fd9e [libFuzzer] add -exit_on_src_pos to test libFuzzer itself, add a test script for RE2 that uses this flag
llvm-svn: 282458
2016-09-27 00:10:20 +00:00
Kostya Serebryany 16a145fd0f [libFuzzer] fix merging with trace-pc-guard
llvm-svn: 282224
2016-09-23 01:58:51 +00:00
Kostya Serebryany ab73c6924f [libFuzzer] move value profiling logic into TracePC
llvm-svn: 282219
2016-09-23 00:46:18 +00:00
Kostya Serebryany be0ed59cdc [libFuzzer] simplify the crash minimizer; split MaxLen into two: MaxInputLen and MaxMutationLen, allow MaxMutationLen to be less than MaxInputLen
llvm-svn: 282211
2016-09-22 23:16:36 +00:00
Kostya Serebryany 624f59f4d8 [libFuzzer] add 'features' to the corpus elements, allow mutations with Size > MaxSize, fix sha1 in corpus stats; various refactorings
llvm-svn: 282129
2016-09-22 01:34:58 +00:00
Kostya Serebryany 29bb664075 [libFuzzer] add stats to the corpus; more refactoring
llvm-svn: 282121
2016-09-21 22:42:17 +00:00
Kostya Serebryany 6f5a804cdb [libFuzzer] refactoring: split the large header into many; NFC
llvm-svn: 282044
2016-09-21 01:50:50 +00:00
Kostya Serebryany 09aa01a6f8 [libFuzzer] refactoring: move the Corpus into a separate class; delete two unused experimental features
llvm-svn: 282042
2016-09-21 01:04:43 +00:00
Kostya Serebryany b706b481ba [libFuzzer] add -print_coverage=1 flag to print coverage directly from libFuzzer
llvm-svn: 281866
2016-09-18 21:47:08 +00:00
Kostya Serebryany 4529960a3b [libFuzzer] don't print help for internal flags
llvm-svn: 281124
2016-09-10 00:35:30 +00:00
Kostya Serebryany 5c04bd250e [libFuzzer] remove use_traces=1 since use_value_profile seems to be strictly better
llvm-svn: 281007
2016-09-09 01:17:03 +00:00
Kostya Serebryany e2d0f63654 [libFuzzer] add -minimize_crash flag (to minimize crashers). also add two tests that I failed to commit last time
llvm-svn: 280332
2016-09-01 01:22:27 +00:00
Kostya Serebryany a016a45d60 [libFuzzer] fix a bug when running a single unit of N bytes with -max_len=M, M<N, caused a buffer overflow
llvm-svn: 280098
2016-08-30 14:52:05 +00:00
Kostya Serebryany 0f0fa4faf2 [libFizzer] rename -print_new_cov_pcs=1 into -print_pcs=1 and make it more useful: print PCs only after the initial corpus has been read and symbolize them
llvm-svn: 279787
2016-08-25 22:35:08 +00:00
Kostya Serebryany d46a59fac4 [libFuzzer] new experimental feature: value profiling. Profiles values that affect control flow and treats new values as new coverage.
llvm-svn: 278839
2016-08-16 19:33:51 +00:00
Kostya Serebryany bdb220c7a0 [libFuzzer] print a verbose message after executing inputs in non-fuzzing mode
llvm-svn: 278724
2016-08-15 19:44:04 +00:00
Kostya Serebryany c135b55ae0 [libFuzzer] add hooks for strstr, strcasestr, strcasecmp, strncasecmp
llvm-svn: 275648
2016-07-15 23:27:19 +00:00
Mike Aizatsky f0b3e85f4e [libfuzzer] moving is_ascii handler inside mutation dispatcher.
Summary: It also fixes a bug, when first random might not be ascii.

Differential Revision: http://reviews.llvm.org/D21573

llvm-svn: 273611
2016-06-23 20:44:48 +00:00
Kostya Serebryany 042d1a7b04 [libFuzzer] make the single-run output more reliable
llvm-svn: 272998
2016-06-17 13:07:06 +00:00
Dan Liew 1873a496e2 [LibFuzzer] Declare and use sanitizer functions in ``fuzzer::ExternalFunctions``
This fixes linking problems on OSX.

Unfortunately it turns out we need to use an instance of the
``fuzzer::ExternalFunctions`` object in several places so this
commit also replaces all instances with a single global instance.

It also turns out initializing a global ``fuzzer::ExternalFunctions``
before main is entered (i.e. letting the object be initialised by the
global initializers) is not safe (on OSX the call to ``Printf()`` in the
CTOR crashes if it is called from a global initializer) so we instead
have a global ``fuzzer::ExternalFunctions*`` and initialize it inside
``FuzzerDriver()``.

Multiple unit tests depend also depend on the
``fuzzer::ExternalFunctions*`` global so a ``main()`` function has been
added that initializes it before running any tests.

Differential Revision: http://reviews.llvm.org/D20943

llvm-svn: 272072
2016-06-07 23:32:50 +00:00
Mike Aizatsky 1f88b12272 [libfuzzer] prune_corpus option for disabling pruning during the load.
Summary:
The option is very useful for testing, plus I intend to measure
its effect on fuzzer effectiveness.

Differential Revision: http://reviews.llvm.org/D21084

llvm-svn: 272035
2016-06-07 18:16:32 +00:00
Dan Liew d3c33116fd [LibFuzzer] Reimplement how the optional user functions are called.
The motivation for this change is to fix linking issues on OSX.
However this only partially fixes linking issues (the uninstrumented
tests and a few others  won't succesfully link yet).

This change introduces a struct of function pointers
(``fuzzer::ExternalFuntions``) which when initialised will point to the
optional functions if they are available.  Currently these
``LLVMFuzzerInitialize`` and ``LLVMFuzzerCustomMutator`` functions.

Two implementations of ``fuzzer::ExternalFunctions`` constructor are
provided one for Linux and one for OSX.

The OSX implementation uses ``dlsym()`` because the prior implementation
using weak symbols does not work unless the additional flags are passed
to the linker.

The Linux implementation continues to use weak symbols because the
``dlsym()`` approach does not work unless additional flags are passed
to the linker.

Differential Revision: http://reviews.llvm.org/D20741

llvm-svn: 271491
2016-06-02 05:48:02 +00:00
Kostya Serebryany f817731a19 [libFuzzer] when an invalid flag is given, warn, but don't crash
llvm-svn: 271404
2016-06-01 16:41:12 +00:00
Mike Aizatsky af432a45e3 [libfuzzer] Trying random unit prefixes during corpus load.
Differential Revision: http://reviews.llvm.org/D20301

llvm-svn: 270632
2016-05-24 23:14:29 +00:00
Kostya Serebryany 6289536922 [libFuzzer] print the file name before executing the input so that if there is a crash we know which files has caused it
llvm-svn: 269450
2016-05-13 18:10:33 +00:00
Kostya Serebryany 8b8f7a3cda [libFuzzer] enhance -rss_limit_mb and enable by default. Now it will print the OOM reproducer.
llvm-svn: 268821
2016-05-06 23:38:07 +00:00
Kostya Serebryany 52b394e981 [libFuzzer] add exeprimental -rss_limit_mb flag to fight against OOMs
llvm-svn: 268807
2016-05-06 21:58:35 +00:00
Kostya Serebryany baf7fd0b16 [libFuzzer] print stats after running individual inputs
llvm-svn: 268547
2016-05-04 20:44:50 +00:00
Kostya Serebryany 1bfd583d82 [libFuzzer] added -detect_leaks flag (0 by default for now). When enabled, it will help finding leaks while fuzzing
llvm-svn: 266838
2016-04-20 00:24:21 +00:00
Kostya Serebryany 09087bba4d [libFuzzer] warn if the corpus is empty
llvm-svn: 266670
2016-04-18 21:14:11 +00:00
Mehdi Amini b550cb1750 [NFC] Header cleanup
Removed some unused headers, replaced some headers with forward class declarations.

Found using simple scripts like this one:
clear && ack --cpp -l '#include "llvm/ADT/IndexedMap.h"' | xargs grep -L 'IndexedMap[<]' | xargs grep -n --color=auto 'IndexedMap'

Patch by Eugene Kosov <claprix@yandex.ru>

Differential Revision: http://reviews.llvm.org/D19219

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 266595
2016-04-18 09:17:29 +00:00