To illustrate our current understanding, let's start with the following program:
https://godbolt.org/z/33f6vheh1
```lang=c++
void clang_analyzer_printState();
struct C {
int x;
int y;
int more_padding;
};
struct D {
C c;
int z;
};
C foo(D d, int new_x, int new_y) {
d.c.x = new_x; // B1
assert(d.c.x < 13); // C1
C c = d.c; // L
assert(d.c.y < 10); // C2
assert(d.z < 5); // C3
d.c.y = new_y; // B2
assert(d.c.y < 10); // C4
return c; // R
}
```
In the code, we create a few bindings to subregions of root region `d` (`B1`, `B2`), a constrain on the values (`C1`, `C2`, ….), and create a `lazyCompoundVal` for the part of the region `d` at point `L`, which is returned at point `R`.
Now, the question is which of these should remain live as long the return value of the `foo` call is live. In perfect a word we should preserve:
# only the bindings of the subregions of `d.c`, which were created before the copy at `L`. In our example, this includes `B1`, and not `B2`. In other words, `new_x` should be live but `new_y` shouldn’t.
# constraints on the values of `d.c`, that are reachable through `c`. This can be created both before the point of making the copy (`L`) or after. In our case, that would be `C1` and `C2`. But not `C3` (`d.z` value is not reachable through `c`) and `C4` (the original value of`d.c.y` was overridden at `B2` after the creation of `c`).
The current code in the `RegionStore` covers the use case (1), by using the `getInterestingValues()` to extract bindings to parts of the referred region present in the store at the point of copy. This also partially covers point (2), in case when constraints are applied to a location that has binding at the point of the copy (in our case `d.c.x` in `C1` that has value `new_x`), but it fails to preserve the constraints that require creating a new symbol for location (`d.c.y` in `C2`).
We introduce the concept of //lazily copied// locations (regions) to the `SymbolReaper`, i.e. for which a program can access the value stored at that location, but not its address. These locations are constructed as a set of regions referred to by `lazyCompoundVal`. A //readable// location (region) is a location that //live// or //lazily copied// . And symbols that refer to values in regions are alive if the region is //readable//.
For simplicity, we follow the current approach to live regions and mark the base region as //lazily copied//, and consider any subregions as //readable//. This makes some symbols falsy live (`d.z` in our example) and keeps the corresponding constraints alive.
The rename `Regions` to `LiveRegions` inside `RegionStore` is NFC change, that was done to make it clear, what is difference between regions stored in this two sets.
Regression Test: https://reviews.llvm.org/D134941
Co-authored-by: Balazs Benics <benicsbalazs@gmail.com>
Reviewed By: martong, xazax.hun
Differential Revision: https://reviews.llvm.org/D134947
This patch adds a new descendant to the SymExpr hierarchy. This way, now
we can assign constraints to symbolic unary expressions. Only the unary
minus and bitwise negation are handled.
Differential Revision: https://reviews.llvm.org/D125318
Under the hood this prints the same as `QualType::getAsString()` but cuts out the middle-man when that string is sent to another raw_ostream.
Also cleaned up all the call sites where this occurs.
Reviewed By: aaron.ballman
Differential Revision: https://reviews.llvm.org/D123926
It turns out llvm::isa<> is variadic, and we could have used this at a
lot of places.
The following patterns:
x && isa<T1>(x) || isa<T2>(x) ...
Will be replaced by:
isa_and_non_null<T1, T2, ...>(x)
Sometimes it caused further simplifications, when it would cause even
more code smell.
Aside from this, keep in mind that within `assert()` or any macro
functions, we need to wrap the isa<> expression within a parenthesis,
due to the parsing of the comma.
Reviewed By: martong
Differential Revision: https://reviews.llvm.org/D111982
The summary and very short discussion in D82122 summarizes whats happening here.
In short, liveness talks about variables, or expressions, anything that
has a value. Well, statements just simply don't have a one.
Differential Revision: https://reviews.llvm.org/D82598
Previously, it was a tedious task to comprehend Z3 dumps.
We will use the same name prefix just as we use in the corresponding dump method
For all `SymbolData` values:
`$###` -> `conj_$###`
`$###` -> `derived_$###`
`$###` -> `extent_$###`
`$###` -> `meta_$###`
`$###` -> `reg_$###`
Reviewed By: xazax.hun,mikhail.ramalho
Differential Revision: https://reviews.llvm.org/D86223
Based on the discussion in D82598#2171312. Thanks @NoQ!
D82598 is titled "Get rid of statement liveness, because such a thing doesn't
exist", and indeed, expressions express a value, non-expression statements
don't.
if (a && get() || []{ return true; }())
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ has a value
~ has a value
~~~~~~~~~~ has a value
~~~~~~~~~~~~~~~~~~~~ has a value
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ doesn't have a value
That is simple enough, so it would only make sense if we only assigned symbolic
values to expressions in the static analyzer. Yet the interface checkers can
access presents, among other strange things, the following two methods:
ProgramState::BindExpr(const Stmt *S, const LocationContext *LCtx, SVal V,
bool Invalidate=true)
ProgramState::getSVal(const Stmt *S, const LocationContext *LCtx)
So, what gives? Turns out, we make an exception for ReturnStmt (which we'll
leave for another time) and ObjCForCollectionStmt. For any other loops, in order
to know whether we should analyze another iteration, among other things, we
evaluate it's condition. Which is a problem for ObjCForCollectionStmt, because
it simply doesn't have one (CXXForRangeStmt has an implicit one!). In its
absence, we assigned the actual statement with a concrete 1 or 0 to indicate
whether there are any more iterations left. However, this is wildly incorrect,
its just simply not true that the for statement has a value of 1 or 0, we can't
calculate its liveness because that doesn't make any sense either, so this patch
turns it into a GDM trait.
Fixing this allows us to reinstate the assert removed in
https://reviews.llvm.org/rG032b78a0762bee129f33e4255ada6d374aa70c71.
Differential Revision: https://reviews.llvm.org/D86736
Summary:
SymIntExpr, IntSymExpr, and SymSymExpr share a big portion of logic
that used to be duplicated across all three classes. New
implementation also adds an easy way of introducing another type of
operands into the mix.
Differential Revision: https://reviews.llvm.org/D79156
So far we've been dropping coverage every time we've encountered
a CXXInheritedCtorInitExpr. This patch attempts to add some
initial support for it.
Constructors for arguments of a CXXInheritedCtorInitExpr are still
not fully supported.
Differential Revision: https://reviews.llvm.org/D74735
Summary:
This patch introduces a placeholder for representing the dynamic size of
regions. It also moves the `getExtent()` method of `SubRegions` to the
`MemRegionManager` as `getStaticSize()`.
Reviewed By: NoQ
Differential Revision: https://reviews.llvm.org/D69540
to reflect the new license.
We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.
Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.
llvm-svn: 351636
SymbolReaper now realizes that our liveness analysis isn't sharp enough
to discriminate between liveness of, say, variables and their fields.
Surprisingly, this didn't quite work before: having a variable live only
through Environment (eg., calling a C++ method on a local variable
as the last action ever performed on that variable) would not keep the
region value symbol of a field of that variable alive.
It would have been broken in the opposite direction as well, but both
Environment and RegionStore use the scanReachableSymbols mechanism for finding
live symbols regions within their values, and due to that they accidentally
end up marking the whole chain of super-regions as live when at least one
sub-region is known to be live.
It is now a direct responsibility of SymbolReaper to maintain this invariant,
and a unit test was added in order to make sure it stays that way.
Differential Revision: https://reviews.llvm.org/D56632
rdar://problem/46914108
llvm-svn: 351499
It's an old bug that consists in stale references to symbols remaining in the
GDM if they disappear from other program state sections as a result of any
operation that isn't the actual dead symbol collection. The most common example
here is:
FILE *fp = fopen("myfile.txt", "w");
fp = 0; // leak of file descriptor
In this example the leak were not detected previously because the symbol
disappears from the public part of the program state due to evaluating
the assignment. For that reason the checker never receives a notification
that the symbol is dead, and never reports a leak.
This patch not only causes leak false negatives, but also a number of other
problems, including false positives on some checkers.
What's worse, even though the program state contains a finite number of symbols,
the set of symbols that dies is potentially infinite. This means that is
impossible to compute the set of all dead symbols to pass off to the checkers
for cleaning up their part of the GDM.
No longer compute the dead set at all. Disallow iterating over dead symbols.
Disallow querying if any symbols are dead. Remove the API for marking symbols
as dead, as it is no longer necessary. Update checkers accordingly.
Differential Revision: https://reviews.llvm.org/D18860
llvm-svn: 347953
Summary:
This patch introduces a new member to SymExpr, which stores the symbol complexity, avoiding recalculating it every time computeComplexity() is called.
Also, increase the complexity of conjured Symbols by one, so it's clear that it has a greater complexity than its underlying symbols.
Reviewers: NoQ, george.karpenkov
Reviewed By: NoQ, george.karpenkov
Subscribers: xazax.hun, szepet, a.sidorin
Differential Revision: https://reviews.llvm.org/D49232
llvm-svn: 337472
Like SymbolConjured, SymbolMetadata also needs to be uniquely
identified by the moment of its birth.
Such moments are coded by the (Statement, LocationContext, Block count) triples.
Each such triple represents the moment of analyzing a statement with a certain
call backtrace, with corresponding CFG block having been entered a given amount
of times during analysis of the current code body.
The LocationContext information was accidentally omitted for SymbolMetadata,
which leads to reincarnation of SymbolMetadata upon re-entering a code body
with a different backtrace; the new symbol is incorrectly unified with
the old symbol, which leads to unsound assumptions.
Patch by Alexey Sidorin!
Differential Revision: https://reviews.llvm.org/D21978
llvm-svn: 278937
The purpose of these changes is to simplify introduction of definition files
for the three hierarchies.
1. For every sub-class C of these classes, its kind in the relevant enumeration
is changed to "CKind" (or C##Kind in preprocessor-ish terms), eg:
MemRegionKind -> MemRegionValKind
RegionValueKind -> SymbolRegionValueKind
CastSymbolKind -> SymbolCastKind
SymIntKind -> SymIntExprKind
2. MemSpaceRegion used to be inconsistently used as both an abstract base and
a particular region. This region class is now an abstract base and no longer
occupies GenericMemSpaceRegionKind. Instead, a new class, CodeSpaceRegion,
is introduced for handling the unique use case for MemSpaceRegion as
"the generic memory space" (when it represents a memory space that holds all
executable code).
3. BEG_ prefixes in memory region kind ranges are renamed to BEGIN_ for
consisitency with symbol kind ranges.
4. FunctionTextRegion and BlockTextRegion are renamed to FunctionCodeRegion and
BlockCodeRegion, respectively. The term 'code' is less jargony than 'text' and
we already refer to BlockTextRegion as a 'code region' in BlockDataRegion.
Differential Revision: http://reviews.llvm.org/D16062
llvm-svn: 257598
SymbolReaper was destroying the symbol too early when it was referenced only
from an index SVal of a live ElementRegion.
In order to test certain aspects of this patch, extend the debug.ExprInspection
checker to allow testing SymbolReaper in a direct manner.
Differential Revision: http://reviews.llvm.org/D12726
llvm-svn: 255236
(return by value is in ExprEngine::processPointerEscapedOnBind and any
other call to the scanReachableSymbols function template used there)
Protect the special members in the base class to avoid slicing, and make
derived classes final so these special members don't accidentally become
public on an intermediate base which would open up the possibility of
slicing again.
llvm-svn: 244975
When casting the address of a FunctionTextRegion to bool, or when adding
constraints to such an address, use a stand-in symbol to represent the
presence or absence of the function if the function is weakly linked.
This is groundwork for possible simple availability testing checks, and
can already catch mistakes involving inverted null checks for
weakly-linked functions.
Currently, the implementation reuses the "extent" symbols, originally created
for tracking the size of a malloc region. Since FunctionTextRegions cannot
be dereferenced, the extent symbol will never be used for anything else.
Still, this probably deserves a refactoring in the future.
This patch does not attempt to support testing the presence of weak
/variables/ (global variables), which would likely require much more of
a change and a generalization of "region structure metadata", like the
current "extents", vs. "region contents metadata", like CStringChecker's
"string length".
Patch by Richard <tarka.t.otter@googlemail.com>!
llvm-svn: 189492
Previously, the analyzer used isIntegerType() everywhere, which uses the C
definition of "integer". The C++ predicate with the same behavior is
isIntegerOrUnscopedEnumerationType().
However, the analyzer is /really/ using this to ask if it's some sort of
"integrally representable" type, i.e. it should include C++11 scoped
enumerations as well. hasIntegerRepresentation() sounds like the right
predicate, but that includes vectors, which the analyzer represents by its
elements.
This commit audits all uses of isIntegerType() and replaces them with the
general isIntegerOrEnumerationType(), except in some specific cases where
it makes sense to exclude scoped enumerations, or any enumerations. These
cases now use isIntegerOrUnscopedEnumerationType() and getAs<BuiltinType>()
plus BuiltinType::isInteger().
isIntegerType() is hereby banned in the analyzer - lib/StaticAnalysis and
include/clang/StaticAnalysis. :-)
Fixes real assertion failures. PR15703 / <rdar://problem/12350701>
llvm-svn: 179081
Fixes a FIXME, improves dead symbol collection, suppresses a false positive,
which resulted from reusing the same symbol twice for simulation of 2 calls to the same function.
Fixing this lead to 2 possible false negatives in CString checker. Since the checker is still alpha and
the solution will not require revert of this commit, move the tests to a FIXME section.
llvm-svn: 177206
This will simplify checkers that need to register for leaks. Currently,
they have to register for both: check dead and check end of path.
I've modified the SymbolReaper to consider everything on the stack dead
if the input StackLocationContext is 0.
(This is a bit disruptive, so I'd like to flash out all the issues
asap.)
llvm-svn: 167352
Previously, we'd just keep constraints around forever, which means we'd
never be able to merge paths that differed only in constraints on dead
symbols.
Because we now allow constraints on symbolic expressions, not just single
symbols, this requires changing SymExpr::symbol_iterator to include
intermediate symbol nodes in its traversal, not just the SymbolData leaf
nodes.
This depends on the previous commit to be correct. Originally applied in
r163444, reverted in r164275, now being re-applied.
llvm-svn: 164622
No tests, but this allows the optimization of removing dead constraints.
We can then add tests that we don't do this prematurely.
<rdar://problem/12333297>
Note: the added FIXME to investigate SymbolRegionValue liveness is
tracked by <rdar://problem/12368183>. This patch does not change the
existing behavior.
llvm-svn: 164621
While we definitely want this optimization in the future, we're not
currently handling constraints on symbolic /expressions/ correctly.
These should stay live even if the SymExpr itself is no longer referenced
because could recreate an identical SymExpr later. Only once the SymExpr
can no longer be recreated -- i.e. a component symbol is dead -- can we
safely remove the constraints on it.
This liveness issue is tracked by <rdar://problem/12333297>.
This reverts r163444 / 24c7f98828e039005cff3bd847e7ab404a6a09f8.
llvm-svn: 164275
Previously, we'd just keep constraints around forever, which means we'd
never be able to merge paths that differed only in constraints on dead
symbols.
Because we now allow constraints on symbolic expressions, not just single
symbols, this requires changing SymExpr::symbol_iterator to include
intermediate symbol nodes in its traversal, not just the SymbolData leaf
nodes.
llvm-svn: 163444
This turned out to have many implications, but what eventually seemed to
make it unworkable was the fact that we can get struct values (as
LazyCompoundVals) from other places besides return-by-value function calls;
that is, we weren't actually able to "treat all struct values as regions"
consistently across the entire analyzer core.
Hopefully we'll be able to come up with an alternate solution soon.
This reverts r163066 / 02df4f0aef142f00d4637cd851e54da2a123ca8e.
llvm-svn: 163218
This allows us to correctly symbolicate the fields of structs returned by
value, as well as get the proper 'this' value for when methods are called
on structs returned by value.
This does require a moderately ugly hack in the StoreManager: if we assign
a "struct value" to a struct region, that now appears as a Loc value being
bound to a region of struct type. We handle this by simply "dereferencing"
the struct value region, which should create a LazyCompoundVal.
This should fix recent crashes analyzing LLVM and on our internal buildbot.
<rdar://problem/12137950>
llvm-svn: 163066
No need to have the "get", the word "conjure" is a verb too!
Getting a conjured symbol is the same as conjuring one up.
This shortening is largely cosmetic, but just this simple changed
cleaned up a handful of lines, making them less verbose.
llvm-svn: 162348
This is needed to ensure that we always report issues in the correct
function. For example, leaks are identified when we call remove dead
bindings. In order to make sure we report a callee's leak in the callee,
we have to run the operation in the callee's context.
This change required quite a bit of infrastructure work since:
- We used to only run remove dead bindings before a given statement;
here we need to run it after the last statement in the function. For
this, we added additional Program Point and special mode in the
SymbolReaper to remove all symbols in context lower than the current
one.
- The call exit operation turned into a sequence of nodes, which are
now guarded by CallExitBegin and CallExitEnd nodes for clarity and
convenience.
(Sorry for the long diff.)
llvm-svn: 155244
(Stmt*,LocationContext*) pairs to SVals instead of Stmt* to SVals.
This is needed to support basic IPA via inlining. Without this, we cannot tell
if a Stmt* binding is part of the current analysis scope (StackFrameContext) or
part of a parent context.
This change introduces an uglification of the use of getSVal(), and thus takes
two steps forward and one step back. There are also potential performance implications
of enlarging the Environment. Both can be addressed going forward by refactoring the
APIs and optimizing the internal representation of Environment. This patch
mainly introduces the functionality upon when we want to build upon (and clean up).
llvm-svn: 147688
Fix a bug in SimpleSValBuilder, where we should swap lhs and rhs when calling generateUnknownVal(), - the function which creates symbolic expressions when data is tainted. The issue is not visible when we only create the expressions for taint since all expressions are commutative from taint perspective.
Refactor SymExpr::symbol_iterator::expand() to use a switch instead of a chain of ifs.
llvm-svn: 146336
Tomasz Kamiński