This patch mechanically replaces None with std::nullopt where the
compiler would warn if None were deprecated. The intent is to reduce
the amount of manual work required in migrating from Optional to
std::optional.
This is part of an effort to migrate from llvm::Optional to
std::optional:
https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716
constraints
In this patch I add a new NoteTag for each applied argument constraint.
This way, any other checker that reports a bug - where the applied
constraint is relevant - will display the corresponding note. With this
change we provide more information for the users to understand some
bug reports easier.
Differential Revision: https://reviews.llvm.org/D101526
Reviewed By: NoQ
Some of the code used in StdLibraryFunctionsChecker is applicable to
other checkers, this is put into common functions. Errno related
parts of the checker are simplified and renamed. Documentations in
errno_modeling functions are updated.
This change makes it available to have more checkers that perform
modeling of some standard functions. These can set the errno state
with common functions and the bug report messages (note tags) can
look similar.
Reviewed By: steakhal, martong
Differential Revision: https://reviews.llvm.org/D131879
The functions 'mkdir', 'mknod', 'mkdirat', 'mknodat' return 0 on success
and -1 on failure. The checker modeled these functions with a >= 0
return value on success which is changed to 0 only. This fix makes
ErrnoChecker work better for these functions.
Reviewed By: steakhal
Differential Revision: https://reviews.llvm.org/D127277
This updates StdLibraryFunctionsChecker to set the state of 'errno'
by using the new errno_modeling functionality.
The errno value is set in the PostCall callback. Setting it in call::Eval
did not work for some reason and then every function should be
EvalCallAsPure which may be bad to do. Now the errno value and state
is not allowed to be checked in any PostCall checker callback because
it is unspecified if the errno was set already or will be set later
by this checker.
Reviewed By: martong, steakhal
Differential Revision: https://reviews.llvm.org/D125400
The patch is straightforward except the tiny fix in BugReporterVisitors.cpp
that suppresses a default note for "Assuming pointer value is null" when
a note tag from the checker is present. This is probably the right thing to do
but also definitely not a complete solution to the problem of different sources
of path notes being unaware of each other, which is a large and annoying issue
that we have to deal with. Note tags really help there because they're nicely
introspectable. The problem is demonstrated by the newly added getenv() test.
Differential Revision: https://reviews.llvm.org/D122285
A recent review emphasized the preference to use DefaultBool instead of
bool for checker options. This change is a NFC and cleans up some of the
instances where bool was used, and could be changed to DefaultBool.
Reviewed By: steakhal
Differential Revision: https://reviews.llvm.org/D123464
If the `assume-controlled-environment` is `true`, we should expect `getenv()`
to succeed, and the result should not be considered tainted.
By default, the option will be `false`.
Reviewed By: NoQ, martong
Differential Revision: https://reviews.llvm.org/D111296
The `getenv()` function might return `NULL` just like any other function.
However, in case of `getenv()` a state-split seems justified since the
programmer should expect the failure of this function.
`secure_getenv(const char *name)` behaves the same way but is not handled
right now.
Note that `std::getenv()` is also not handled.
Reviewed By: martong
Differential Revision: https://reviews.llvm.org/D111245
When we report an argument constraint violation, we should track those
other arguments that participate in the evaluation of the violation. By
default, we depend only on the argument that is constrained, however,
there are some special cases like the buffer size constraint that might
be encoded in another argument(s).
Differential Revision: https://reviews.llvm.org/D101358
In this patch, I provide a detailed explanation for each argument
constraint. This explanation is added in an extra 'note' tag, which is
displayed alongside the warning.
Since these new notes describe clearly the constraint, there is no need
to provide the number of the argument (e.g. 'Arg3') within the warning.
However, I decided to keep the name of the constraint in the warning (but
this could be a subject of discussion) in order to be able to identify
the different kind of constraint violations easily in a bug database
(e.g. CodeChecker).
Differential Revision: https://reviews.llvm.org/D101060
The idiom:
```
DeclContext::lookup_result R = DeclContext::lookup(Name);
for (auto *D : R) {...}
```
is not safe when in the loop body we trigger deserialization from an AST file.
The deserialization can insert new declarations in the StoredDeclsList whose
underlying type is a vector. When the vector decides to reallocate its storage
the pointer we hold becomes invalid.
This patch replaces a SmallVector with an singly-linked list. The current
approach stores a SmallVector<NamedDecl*, 4> which is around 8 pointers.
The linked list is 3, 5, or 7. We do better in terms of memory usage for small
cases (and worse in terms of locality -- the linked list entries won't be near
each other, but will be near their corresponding declarations, and we were going
to fetch those memory pages anyway). For larger cases: the vector uses a
doubling strategy for reallocation, so will generally be between half-full and
full. Let's say it's 75% full on average, so there's N * 4/3 + 4 pointers' worth
of space allocated currently and will be 2N pointers with the linked list. So we
break even when there are N=6 entries and slightly lose in terms of memory usage
after that. We suspect that's still a win on average.
Thanks to @rsmith!
Differential revision: https://reviews.llvm.org/D91524
`initFunctionSummaries` lazily initializes a data structure with
function summaries for standard library functions. It is called for
every pre-, post-, and eval-call events, i.e. 3 times for each call on
the path. If the initialization doesn't find any standard library
functions in the translation unit, it will get re-tried (with the same
effect) many times even for small translation units.
For projects not using standard libraries, the speed-up can reach 50%
after this patch.
Differential Revision: https://reviews.llvm.org/D98244
This time, we add contraints to functions that either return with [0, -1] or
with a file descriptor.
Differential Revision: https://reviews.llvm.org/D92771
close:
It is quite often that users chose to call close even if the fd is
negative. Theoretically, it would be nicer to close only valid fds, but
in practice the implementations of close just returns with EBADF in case
of a non-valid fd param. So, we can eliminate many false positives if we
let close to take -1 as an fd. Other negative values are very unlikely,
because open and other fd factories return with -1 in case of failure.
mmap:
In the case of MAP_ANONYMOUS flag (which is supported e.g. in Linux) the
mapping is not backed by any file; its contents are initialized to zero.
The fd argument is ignored; however, some implementations require fd to
be -1 if MAP_ANONYMOUS (or MAP_ANON) is specified, and portable
applications should ensure this.
Consequently, we must allow -1 as the 4th arg.
Differential Revision: https://reviews.llvm.org/D92764
The fd parameter of
```
void *mmap(void *addr, size_t length, int prot, int flags, int fd, off_t offset)
```
should be constrained to the range [0, IntMax] as that is of type int.
Constraining to the range [0, Off_tMax] would result in a crash as that is
of a signed type with the value of 0xff..f (-1).
The crash would happen when we try to apply the arg constraints.
At line 583: assert(Min <= Max), as 0 <= -1 is not satisfied
The mmap64 is fixed for the same reason.
Reviewed By: martong, vsavchenko
Differential Revision: https://reviews.llvm.org/D92307
The signature should not be part of the summaries as many FIXME comments
suggests. By separating the signature, we open up the way to a generic
matching implementation which could be used later under the hoods of
CallDescriptionMap.
Differential Revision: https://reviews.llvm.org/D88100
It is no longer needed to add summaries of 'getline' for different
possible underlying types of ssize_t. We can just simply lookup the
type.
Differential Revision: https://reviews.llvm.org/D88092
Add the BufferSize argument constraint to fread and fwrite. This change
itself makes it possible to discover a security critical case, described
in SEI-CERT ARR38-C.
We also add the not-null constraint on the 3rd arguments.
In this patch, I also remove those lambdas that don't take any
parameters (Fwrite, Fread, Getc), thus making the code better
structured.
Differential Revision: https://reviews.llvm.org/D87081
There are 2 reasons to remove strcasecmp and strncasecmp.
1) They are also modeled in CStringChecker and the related argumentum
contraints are checked there.
2) The argument constraints are checked in CStringChecker::evalCall.
This is fundamentally flawed, they should be checked in checkPreCall.
Even if we set up CStringChecker as a weak dependency for
StdLibraryFunctionsChecker then the latter reports the warning always.
Besides, CStringChecker fails to discover the constraint violation
before the call, so, its evalCall returns with `true` and then
StdCLibraryFunctions also tries to evaluate, this causes an assertion
in CheckerManager.
Either we fix CStringChecker to handle the call prerequisites in
checkPreCall, or we must not evaluate any pure functions in
StdCLibraryFunctions that are also handled in CStringChecker.
We do the latter in this patch.
Differential Revision: https://reviews.llvm.org/D87239
The "restrict" keyword is illegal in C++, however, many libc
implementations use the "__restrict" compiler intrinsic in functions
prototypes. The "__restrict" keyword qualifies a type as a restricted type
even in C++.
In case of any non-C99 languages, we don't want to match based on the
restrict qualifier because we cannot know if the given libc implementation
qualifies the paramter type or not.
Differential Revision: https://reviews.llvm.org/D87097
By using optionals, we no longer have to check the validity of types that we
get from a lookup. This way, the definition of the summaries have a declarative
form, there are no superflous conditions in the source code.
Differential Revision: https://reviews.llvm.org/D86531
Summary:
Adding networking functions from the POSIX standard (2017). This includes
functions that deal with sockets from socket.h, netdb.h.
In 'socket.h' of some libc implementations (e.g. glibc) with C99, sockaddr
parameter is a transparent union of the underlying sockaddr_ family of pointers
instead of being a pointer to struct sockaddr. In these cases, the standardized
signature will not match, thus we try to match with another signature that has
the joker Irrelevant type. In the case of transparent unions, we also not add
those constraints which require pointer types for the sockaddr param.
Interestingly, in 'netdb.h' sockaddr is not handled as a transparent union.
Tags: #clang
Differential Revision: https://reviews.llvm.org/D83407
Adding file handling functions from the POSIX standard (2017).
A new checker option is introduced to enable them.
In follow-up patches I am going to upstream networking, pthread, and other
groups of POSIX functions.
Differential Revision: https://reviews.llvm.org/D82288
Summary:
In this patch I am trying to get rid of the `Irrelevant` types from the
signatures of the functions from the standard C library. For that I've
introduced `lookupType()` to be able to lookup arbitrary types in the global
scope. This makes it possible to define the signatures precisely.
Note 1) `fread`'s signature is now fixed to have the proper `FILE *restrict`
type when C99 is the language.
Note 2) There are still existing `Irrelevant` types, but they are all from
POSIX. I am planning to address those together with the missing POSIX functions
(in D79433).
Reviewers: xazax.hun, NoQ, Szelethus, balazske
Subscribers: whisperity, baloghadamsoftware, szepet, rnkovacs, a.sidorin, mikhail.ramalho, donat.nagy, dkrupp, gamesh411, Charusso, steakhal, ASDenysPetrov, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D80016
Summary:
Once we found a matching FunctionDecl for the given summary then we
validate the given constraints against that FunctionDecl. E.g. we
validate that a NotNull constraint is applied only on arguments that
have pointer types.
This is needed because when we matched the signature of the summary we
were working with incomplete function types, i.e. some intricate type
could have been marked as `Irrelevant` in the signature.
Reviewers: NoQ, Szelethus, balazske
Subscribers: whisperity, xazax.hun, baloghadamsoftware, szepet, rnkovacs, a.sidorin, mikhail.ramalho, donat.nagy, dkrupp, gamesh411, Charusso, steakhal, ASDenysPetrov, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D77658
Summary:
Further develop the buffer size argumentum constraint so it can handle sizes
that we can get by multiplying two variables.
Reviewers: Szelethus, NoQ, steakhal
Subscribers: whisperity, xazax.hun, baloghadamsoftware, szepet, rnkovacs, a.sidorin, mikhail.ramalho, donat.nagy, dkrupp, gamesh411, Charusso, ASDenysPetrov, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D77148
Summary:
Introducing a new argument constraint to confine buffer sizes. It is typical in
C APIs that a parameter represents a buffer and another param holds the size of
the buffer (or the size of the data we want to handle from the buffer).
Reviewers: NoQ, Szelethus, Charusso, steakhal
Subscribers: whisperity, xazax.hun, baloghadamsoftware, szepet, rnkovacs, a.sidorin, mikhail.ramalho, donat.nagy, dkrupp, gamesh411, ASDenysPetrov, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D77066
Summary:
Currently we map function summaries to names (i.e. strings). We can
associate more summaries with different signatures to one name, this way
we support overloading. During a call event we check whether the
signature of the summary matches the signature of the callee and we
apply the summary only in that case.
In this patch we change this mapping to associate a summary to a
FunctionDecl. We do lookup operations when the summary map is
initialized. We lookup the given name and we match the signature of the
given summary against the lookup results. If the summary matches the
FunctionDecl (got from the lookup result) then we add that to the
summary map. During a call event we compare FunctionDecl pointers.
Advantages of this new refactor:
- Cleaner mapping and structure for the checker.
- Possibly way more efficient handling of call events.
- A summary is added only if that is relevant for the given TU.
- We can get the concrete FunctionDecl by the time when we create the
summary, this opens up possibilities of further sanity checks
regarding the summary cases and argument constraints.
- Opens up to future work when we'd like to store summaries from IR to a
FunctionDecl (or from the Attributor results of the given
FunctionDecl).
Note, we cannot support old C functions without prototypes.
Reviewers: NoQ, Szelethus, balazske, jdoerfert, sstefan1, uenoku
Subscribers: whisperity, xazax.hun, baloghadamsoftware, szepet, rnkovacs, a.sidorin, mikhail.ramalho, donat.nagy, dkrupp, gamesh411, Charusso, steakhal, uenoku, ASDenysPetrov, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D77641
Summary:
Currently we match the summary signature based on the arguments in the CallExpr.
There are a few problems with this approach.
1) Variadic arguments are handled badly. Consider the below code:
int foo(void *stream, const char *format, ...);
void test_arg_constraint_on_variadic_fun() {
foo(0, "%d%d", 1, 2); // CallExpr
}
Here the call expression holds 4 arguments, whereas the function declaration
has only 2 `ParmVarDecl`s. So there is no way to create a summary that
matches the call expression, because the discrepancy in the number of
arguments causes a mismatch.
2) The call expression does not handle the `restrict` type qualifier.
In C99, fwrite's signature is the following:
size_t fwrite(const void *restrict, size_t, size_t, FILE *restrict);
However, in a call expression, like below, the type of the argument does not
have the restrict qualifier.
void test_fread_fwrite(FILE *fp, int *buf) {
size_t x = fwrite(buf, sizeof(int), 10, fp);
}
This can result in an unmatches signature, so the summary is not applied.
The solution is to match the summary against the referened callee
`FunctionDecl` that we can query from the `CallExpr`.
Further patches will continue with additional refactoring where I am going to
do a lookup during the checker initialization and the signature match will
happen there. That way, we will not check the signature during every call,
rather we will compare only two `FunctionDecl` pointers.
Reviewers: NoQ, Szelethus, gamesh411, baloghadamsoftware
Subscribers: whisperity, xazax.hun, kristof.beyls, szepet, rnkovacs, a.sidorin, mikhail.ramalho, donat.nagy, dkrupp, Charusso, steakhal, danielkiss, ASDenysPetrov, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D77410
Summary:
Previously we induced a state split if there were multiple argument
constraints given for a function. This was because we called
`addTransition` inside the for loop.
The fix is to is to store the state and apply the next argument
constraint on that. And once the loop is finished we call `addTransition`.
Reviewers: NoQ, Szelethus, baloghadamsoftware
Subscribers: whisperity, xazax.hun, szepet, rnkovacs, a.sidorin, mikhail.ramalho, donat.nagy, dkrupp, gamesh411, C
Tags: #clang
Differential Revision: https://reviews.llvm.org/D76790
Summary:
This check was causing a crash in a test case where the 0th argument was
uninitialized ('Assertion `T::isKind(*this)' at line SVals.h:104). This
was happening since the argument was actually undefined, but the castAs
assumes the value is DefinedOrUnknownSVal.
The fix appears to be simply to check for an undefined value and skip
the check allowing the uninitalized value checker to detect the error.
I included a test case that I verified to produce the negative case
prior to the fix, and passes with the fix.
Reviewers: martong, NoQ
Subscribers: xazax.hun, szepet, rnkovacs, a.sidorin, mikhail.ramalho, Szelethus, donat.nagy, Charusso, ASDenysPetrov, baloghadamsoftware, dkrupp, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D77012
Some checkers may not only depend on language options but also analyzer options.
To make this possible this patch changes the parameter of the shouldRegister*
function to CheckerManager to be able to query the analyzer options when
deciding whether the checker should be registered.
Differential Revision: https://reviews.llvm.org/D75271
Kazu Hirata