Summary:
This patch changes how the SMT bug refutation runs in an equivalent bug report class.
Now, all other visitor are executed until they find a valid bug or mark all bugs as invalid. When the one valid bug is found (and crosscheck is enabled), the SMT refutation checks the satisfiability of this single bug.
If the bug is still valid after checking with Z3, it is returned and a bug report is created. If the bug is found to be invalid, the next bug report in the equivalent class goes through the same process, until we find a valid bug or all bugs are marked as invalid.
Massive speedups when verifying redis/src/rax.c, from 1500s to 10s.
Reviewers: NoQ, george.karpenkov
Reviewed By: george.karpenkov
Subscribers: xazax.hun, szepet, a.sidorin
Differential Revision: https://reviews.llvm.org/D49693
llvm-svn: 337920
In the current implementation, we run visitors until the fixed point is
reached.
That is, if a visitor adds another visitor, the currently processed path
is destroyed, all diagnostics is discarded, and it is regenerated again,
until it's no longer modified.
This pattern has a few negative implications:
- This loop does not even guarantee to terminate.
E.g. just imagine two visitors bouncing a diagnostics around.
- Performance-wise, e.g. for sqlite3 all visitors are being re-run at
least 10 times for some bugs.
We have already seen a few reports where it leads to timeouts.
- If we want to add more computationally intense visitors, this will
become worse.
- From architectural standpoint, the current layout requires copying
visitors, which is conceptually wrong, and can be annoying (e.g. no
unique_ptr on visitors allowed).
The proposed change is a much simpler architecture: the outer loop
processes nodes upwards, and whenever the visitor is added it only
processes current nodes and above, thus guaranteeing termination.
Differential Revision: https://reviews.llvm.org/D47856
llvm-svn: 335666
BugReporter.cpp is already severely overloaded, and those dump methods
are on PathDiagnostics and should belong in the corresponding
implementation file.
Differential Revision: https://reviews.llvm.org/D48035
llvm-svn: 334541
getEndPath is a problematic API, because it's not clear when it's called
(hint: not always at the end of the path), it crashes at runtime with
more than one non-nullptr returning implementation, and diagnostics
internal depend on it being called at some exact place.
However, most visitors don't actually need that: all they want is a
function consistently called after all nodes are traversed, to perform
finalization and to decide whether invalidation is needed.
Differential Revision: https://reviews.llvm.org/D48042
llvm-svn: 334540
Once we removed AlternateExtensive, I've looked closer into the
difference between Minimal and Extensive, and turns out, the difference
was not that large.
Differential Revision: https://reviews.llvm.org/D47756
llvm-svn: 334525
Rename AlternateExtensive to Extensive.
In 2013, five years ago, we have switched to AlternateExtensive
diagnostics by default, and Extensive was available under unused,
undocumented flag.
This change remove the flag, renames the Alternate
diagnostic to Extensive (as it's no longer Alternate), and ports the
test.
Differential Revision: https://reviews.llvm.org/D47670
llvm-svn: 334524
Summary: This is a prototype of a bug reporter visitor that invalidates bug reports by re-checking constraints of certain states on the bug path using the Z3 constraint manager backend. The functionality is available under the `crosscheck-with-z3` analyzer config flag.
Reviewers: george.karpenkov, NoQ, dcoughlin, rnkovacs
Reviewed By: george.karpenkov
Subscribers: rnkovacs, NoQ, george.karpenkov, dcoughlin, xbolva00, ddcc, mikhail.ramalho, MTC, fhahn, whisperity, baloghadamsoftware, szepet, a.sidorin, gsd, dkrupp, xazax.hun, cfe-commits
Differential Revision: https://reviews.llvm.org/D45517
llvm-svn: 333903
This is similar to the LLVM change https://reviews.llvm.org/D46290.
We've been running doxygen with the autobrief option for a couple of
years now. This makes the \brief markers into our comments
redundant. Since they are a visual distraction and we don't want to
encourage more \brief markers in new code either, this patch removes
them all.
Patch produced by
for i in $(git grep -l '\@brief'); do perl -pi -e 's/\@brief //g' $i & done
for i in $(git grep -l '\\brief'); do perl -pi -e 's/\\brief //g' $i & done
Differential Revision: https://reviews.llvm.org/D46320
llvm-svn: 331834
When a '>>' token is split into two '>' tokens (in C++11 onwards), or (as an
extension) when we do the same for other tokens starting with a '>', we can't
just use a location pointing to the first '>' as the location of the split
token, because that would result in our miscomputing the length and spelling
for the token. As a consequence, for example, a refactoring replacing 'A<X>'
with something else would sometimes replace one character too many, and
similarly diagnostics highlighting a template-id source range would highlight
one character too many.
Fix this by creating an expansion range covering the first character of the
'>>' token, whose spelling is '>'. For this to work, we generalize the
expansion range of a macro FileID to be either a token range (the common case)
or a character range (used in this new case).
llvm-svn: 331155
removeUnneededCalls() is responsible for removing path diagnostic pieces within
functions that don't contain "interesting" events. It makes bug reports
much tidier.
When a stack frame is known to be interesting, the function doesn't descend
into it to prune anything within it, even other callees that are totally boring.
Fix the function to prune boring callees in interesting stack frames.
Differential Revision: https://reviews.llvm.org/D45117
llvm-svn: 329102
r327219 added wrappers to std::sort which randomly shuffle the container before
sorting. This will help in uncovering non-determinism caused due to undefined
sorting order of objects having the same key.
To make use of that infrastructure we need to invoke llvm::sort instead of
std::sort.
llvm-svn: 328636
When viewing the report in the collapsed mode the label signifying where
did the execution go is often necessary for properly understanding the
context.
Differential Revision: https://reviews.llvm.org/D43145
llvm-svn: 325975
HTML diagnostics can be an overwhelming blob of pages of code.
This patch adds a checkbox which filters this list down to only the
lines *relevant* to the counterexample by e.g. skipping branches which
analyzer has assumed to be infeasible at a time.
The resulting amount of output is much smaller, and often fits on one
screen, and also provides a much more readable diagnostics.
Differential Revision: https://reviews.llvm.org/D41378
llvm-svn: 322612
This addresses an issue introduced in r183451: since
`removePiecesWithInvalidLocations` is called *after* `adjustCallLocations`,
it is not necessary, and in fact harmful, to have this assertion in
adjustCallLocations.
Addresses rdar://36170689
Differential Revision: https://reviews.llvm.org/D41680
llvm-svn: 321682
The bugreporter::trackNullOrUndefValue() mechanism contains a system of bug
reporter visitors that recursively call each other in order to track where a
null or undefined value came from, where each visitor represents a particular
tracking mechanism (track how the value was stored, track how the value was
returned from a function, track how the value was constrained to null, etc.).
Each visitor is only added once per value it needs to track. Almost. One
exception from this rule would be FindLastStoreBRVisitor that has two operation
modes: it contains a flag that indicates whether null stored values should be
suppressed. Two instances of FindLastStoreBRVisitor with different values of
this flag are considered to be different visitors, so they can be added twice
and produce the same diagnostic twice. This was indeed the case in the affected
test.
With the current logic of this whole machinery, such duplication seems
unavoidable. We should be able to safely add visitors with different flag
values without constructing duplicate diagnostic pieces. Hence the effort
in this commit to de-duplicate diagnostics regardless of what visitors
have produced them.
Differential Revision: https://reviews.llvm.org/D41258
llvm-svn: 321135
Because since r308957 the suppress-on-sink feature contains its own
mini-analysis, it also needs to become aware that C++ unhandled exceptions
cause sinks. Unfortunately, for now we treat all exceptions as unhandled in
the analyzer, so suppress-on-sink needs to do the same.
rdar://problem/28157554
Differential Revision: https://reviews.llvm.org/D35674
llvm-svn: 308961
If a certain memory leak (or other similar bug) found by the analyzer is known
to be happening only before abnormal termination of the program ("sink", eg.
assertion failure in the code under analysis, or another bug that introduces
undefined behavior), such leak warning is discarded. However, if the analysis
has never reaches completion (due to complexity of the code), it may be
failing to notice the sink.
This commit further extends the partial solution introduced in r290341 to cover
cases when a complicated control flow occurs before encountering a no-return
statement (which anyway inevitably leads to such statement(s)) by traversing
the respective section of the CFG in a depth-first manner. A complete solution
still seems elusive.
rdar://problem/28157554
Differential Revision: https://reviews.llvm.org/D35673
llvm-svn: 308957
In plist output mode with alternate path diagnostics, when entering a function,
we draw an arrow from the caller to the beginning of the callee's declaration.
Upon exiting, however, we draw the arrow from the last statement in the
callee function. The former makes little sense when the declaration is
not a definition, i.e. has no body, which may happen in case the body
is coming from a body farm, eg. Objective-C autosynthesized property accessor.
Differential Revision: https://reviews.llvm.org/D33671
llvm-svn: 304713
Warnings with suppress-on-sink are discarded during FlushReports when
BugReporter notices that all paths in ExplodedGraph that pass through the
warning eventually run into a sink node.
However, suppress-on-sink fails to filter out false positives when the analysis
terminates too early - by running into analyzer limits, such as block count
limits or graph size limits - and the interruption hits the narrow window
between throwing the leak report and reaching the no-return function call. In
such case the report is there, however suppression-on-sink doesn't work, because
the sink node was never constructed in the incomplete ExplodedGraph.
This patch implements a very partial solution: also suppress reports thrown
against a statement-node that corresponds to a statement that belongs to a
no-return block of the CFG.
rdar://problem/28832541
Differential Revision: https://reviews.llvm.org/D28023
llvm-svn: 290341
The problem that caused the msvc crash has been indentified and fixed
in the previous commit. This patch contains the rest of r283092.
llvm-svn: 283584
Define PathDiagnosticNotePiece. The next commit would be able to address the
BugReport class code that is pointed to by the msvc crash message.
llvm-svn: 283566
In the analyzer's path-sensitive reports, when a report goes through a branch
and the branch condition cannot be decided to be definitely true or false
(based on the previous execution path), an event piece is added that tells the
user that a new assumption is added upon the symbolic value of the branch
condition. For example, "Assuming 'a' is equal to 3".
The text of the assumption is hand-crafted in various manners depending on
the AST expression. If the AST expression is too complex and the text of
the assumption fails to be constructed, the event piece is omitted.
This causes loss of information and misunderstanding of the report.
Do not omit the event piece even if the expression is too complex;
add a piece with a generic text instead.
Differential Revision: https://reviews.llvm.org/D23300
llvm-svn: 283301
These diagnostics are separate from the path-sensitive engine's path notes,
and can be added manually on top of path-sensitive or path-insensitive reports.
The new note diagnostics would appear as note:-diagnostic on console and
as blue bubbles in scan-build. In plist files they currently do not appear,
because format needs to be discussed with plist file users.
The analyzer option "-analyzer-config notes-as-events=true" would convert
notes to normal path notes, and put them at the beginning of the path.
This is a temporary hack to show the new notes in plist files.
A few checkers would be updated in subsequent commits,
including tests for this new feature.
Differential Revision: https://reviews.llvm.org/D24278
llvm-svn: 283092
This checker checks copy and move assignment operators whether they are
protected against self-assignment. Since C++ core guidelines discourages
explicit checking for `&rhs==this` in general we take a different approach: in
top-frame analysis we branch the exploded graph for two cases, where &rhs==this
and &rhs!=this and let existing checkers (e.g. unix.Malloc) do the rest of the
work. It is important that we check all copy and move assignment operator in top
frame even if we checked them already since self-assignments may happen
undetected even in the same translation unit (e.g. using random indices for an
array what may or may not be the same).
This reapplies r275820 after fixing a string-lifetime issue discovered by the
bots.
A patch by Ádám Balogh!
Differential Revision: https://reviews.llvm.org/D19311
llvm-svn: 276365
This checker checks copy and move assignment operators whether they are
protected against self-assignment. Since C++ core guidelines discourages
explicit checking for `&rhs==this` in general we take a different approach: in
top-frame analysis we branch the exploded graph for two cases, where &rhs==this
and &rhs!=this and let existing checkers (e.g. unix.Malloc) do the rest of the
work. It is important that we check all copy and move assignment operator in top
frame even if we checked them already since self-assignments may happen
undetected even in the same translation unit (e.g. using random indices for an
array what may or may not be the same).
A patch by Ádám Balogh!
Differential Revision: https://reviews.llvm.org/D19311
llvm-svn: 275820
During the core analysis, ExplodedNodes are added to the
ExplodedGraph, and those nodes are cached for deduplication purposes.
After core analysis, reports are generated. Here, trimmed copies of
the ExplodedGraph are made. Since the ExplodedGraph has already been
deduplicated, there is no need to deduplicate again.
This change makes it possible to add ExplodedNodes to an
ExplodedGraph without the overhead of deduplication. "Uncached" nodes
also cannot be iterated over, but none of the report generation code
attempts to iterate over all nodes. This change reduces the analysis
time of a large .C file from 3m43.941s to 3m40.256s (~1.6% speedup).
It should slightly reduce memory consumption. Gains should be roughly
proportional to the number (and path length) of static analysis
warnings.
This patch enables future work that should remove the need for an
InterExplodedGraphMap inverse map. I plan on using the (now unused)
ExplodedNode link to connect new nodes to the original nodes.
http://reviews.llvm.org/D21229
llvm-svn: 273572
Summary: It breaks the build for the ASTMatchers
Subscribers: klimek, cfe-commits
Differential Revision: http://reviews.llvm.org/D13893
llvm-svn: 250827
The analyzer trims unnecessary nodes from the exploded graph before reporting
path diagnostics. However, in some cases it can trim all nodes (including the
error node), leading to an assertion failure (see
https://llvm.org/bugs/show_bug.cgi?id=24184).
This commit addresses the issue by adding two new APIs to CheckerContext to
explicitly create error nodes. Unless the client provides a custom tag, these
APIs tag the node with the checker's tag -- preventing it from being trimmed.
The generateErrorNode() method creates a sink error node, while
generateNonFatalErrorNode() creates an error node for a path that should
continue being explored.
The intent is that one of these two methods should be used whenever a checker
creates an error node.
This commit updates the checkers to use these APIs. These APIs
(unlike addTransition() and generateSink()) do not take an explicit Pred node.
This is because there are not any error nodes in the checkers that were created
with an explicit different than the default (the CheckerContext's Pred node).
It also changes generateSink() to require state and pred nodes (previously
these were optional) to reduce confusion.
Additionally, there were several cases where checkers did check whether a
generated node could be null; we now explicitly check for null in these places.
This commit also includes a test case written by Ying Yi as part of
http://reviews.llvm.org/D12163 (that patch originally addressed this issue but
was reverted because it introduced false positive regressions).
Differential Revision: http://reviews.llvm.org/D12780
llvm-svn: 247859
Stephen Kelly