There can be Macros that are tagged with `modifiable`. Thus verifying
`canModifyAllDescendants` is not sufficient to avoid macros when deep
copying.
We think the `TokenBuffer` could inform us whether a `Token` comes from
a macro. We'll look into that when we can surface this information
easily, for instance in unit tests for `ComputeReplacements`.
Differential Revision: https://reviews.llvm.org/D88034
* Introduce `TreeTest.cpp` to unit test `Tree.h`
* Add `generateAllTreesWithShape` to generating test cases
* Add tests for `findFirstLeaf` and `findLastLeaf`
* Fix implementations of `findFirstLeaf` and `findLastLeaf` that had
been broken when empty `Tree` were present.
Differential Revision: https://reviews.llvm.org/D87779
Prior to this change `createTree` could not create arbitrary syntax
trees. Now it dispatches to the constructor of the concrete syntax tree
according to the `NodeKind` passed as argument. This allows reuse inside
the Synthesis API. # Please enter the commit message for your changes.
Lines starting
Differential Revision: https://reviews.llvm.org/D87820
Building on Mac OS with clang 12:
```
jhemphill@jhemphill-mbp build % clang --version
Apple clang version 12.0.0 (clang-1200.0.26.2)
Target: x86_64-apple-darwin19.6.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin
```
yields one warning:
```
/Users/jhemphill/oss/llvm-project/clang/lib/Tooling/Syntax/BuildTree.cpp:1126:22: warning: loop variable 'Arg' is always a copy because the range of type 'llvm::iterator_range<clang::Stmt::CastIterator<clang::Expr, clang::Expr *, clang::Stmt *> >' does not return a reference [-Wrange-loop-analysis]
for (const auto &Arg : Args) {
^
/Users/jhemphill/oss/llvm-project/clang/lib/Tooling/Syntax/BuildTree.cpp:1126:10: note: use non-reference type 'clang::Expr *'
for (const auto &Arg : Args) {
```
It appears that `Arg` is an `Expr*`, passed by value rather than by const reference.
Reviewed By: eduucaldas, gribozavr2
Differential Revision: https://reviews.llvm.org/D87482
In a future patch
* Implement helper function to generate Trees for tests
* and test Tree methods, namely `findFirstLeaf` and `findLastLeaf`
Differential Revision: https://reviews.llvm.org/D87533
* Do not visit `CXXDefaultArgExpr`
* To build `CallArguments` nodes, just go through non-default arguments
Differential Revision: https://reviews.llvm.org/D87249
Previously a NodeRole would generally be prefixed with the `NodeKind`,
we remove this prefix, as it we redundant and made tests more noisy.
Differential Revision: https://reviews.llvm.org/D86636
* Generate `CallExpression` syntax node for all semantic nodes inheriting from
`CallExpr` with call-expression syntax - except `CUDAKernelCallExpr`.
* Implement all the accessors
* Arguments of `CallExpression` have their own syntax node which is based on
the `List` base API
Differential Revision: https://reviews.llvm.org/D86544
We should see `NodeRole` information in the dump because that exposes how the
accessors will behave.
Functional changes in the dump:
* Surround Leaf tokens with `'`
* Append `Node` dumps with `NodeRole` information, except for unknown roles
* Append marks to `Node` dumps, instead of prepending
Non-functional changes:
* `::dumpTokens(llvm::raw_ostream, ArrayRef<syntax::Token>, const
SourceManager &SM)` always received as parameter a `syntax::Token *`
pointing to `Leaf::token()`. Changed the function to
`dumpLeaf(llvm::raw_ostream, syntax::Leaf *, const SourceManager&)`
* `dumpTree` acted on a Node, rename to `dumpNode`
Differential Revision: https://reviews.llvm.org/D85330
For an user define `<`, `x < y` would yield the syntax tree:
```
BinaryOperatorExpression
|-IdExpression
| `-UnqualifiedId
| `-x
|-IdExpression
| `-UnqualifiedId
| `-<
`-IdExpression
`-UnqualifiedId
`-y
```
But there is no syntatic difference at call site between call site or
built-in `<`. As such they should generate the same syntax tree, namely:
```
BinaryOperatorExpression
|-IdExpression
| `-UnqualifiedId
| `-x
|-<
`-IdExpression
`-UnqualifiedId
`-y
```
Differential Revision: https://reviews.llvm.org/D85750
Summary:
We want NestedNameSpecifier syntax nodes to be generally supported, not
only for `DeclRefExpr` and `DependentScopedDeclRefExpr`.
To achieve this we:
* Use the `RecursiveASTVisitor`'s API to traverse
`NestedNameSpecifierLoc`s and automatically create its syntax nodes
* Add links from the `NestedNameSpecifierLoc`s to their syntax nodes.
In this way, from any semantic construct that has a `NestedNameSpecifier`,
we implicitly generate its syntax node via RAV and we can easily access
this syntax node via the links we added.
We defined a List construct to help with the implementation of list-like
grammar rules. This is a first implementation of this API.
Differential Revision: https://reviews.llvm.org/D85295
This is our grammar rule for nested-name-specifiers:
globalbal-specifier:
/*empty*/
simple-template-specifier:
template_opt simple-template-id
name-specifier:
global-specifier
decltype-specifier
identifier
simple-template-specifier
nested-name-specifier:
list(name-specifier, ::, non-empty, terminated)
It is a relaxed version of C++ [expr.prim.id] and quite simpler to map to our API.
TODO: refine name specifiers, `simple-template-name-specifier` and
decltype-name-specifier` are token soup for now.
We use inheritance to model the grammar's disjunction rule:
literal:
integer-literal
character-literal
floating-point-literal
string-literal
boolean-literal
pointer-literal
user-defined-literal
Differential Revision: https://reviews.llvm.org/D85186
Currently an Arena can only be built while consuming a TokenBuffer,
some users (like clangd) might want to share a TokenBuffer with multiple
compenents. This patch changes Arena's TokenBuffer member to be a reference so
that it can be created with read-only token buffers.
Differential Revision: https://reviews.llvm.org/D84973
Summary:
This allows efficiently accessing all expansions (without iterating over each
token and searching), and also identifying tokens within a range that are
affected by the preprocessor (which is how clangd will use it).
Subscribers: ilya-biryukov, kadircet, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D84009
* as we are using them only for integer and floating literals they have
the same behavior
* FileRange::text is simpler to call and is within the context of
syntax trees
Summary:
Given an UserDefinedLiteral `1.2_w`:
Problem: Lexer generates one Token for the literal, but ClangAST
references two source locations
Fix: Ignore the operator and interpret it as the underlying literal.
e.g.: `1.2_w` token generates syntax node IntegerLiteral(1.2_w)
Subscribers: cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D82157
Summary:
How does RecursiveASTVisitor call the WalkUp callback for expressions?
* In pre-order traversal mode, RecursiveASTVisitor calls the WalkUp
callback from the default implementation of Traverse callbacks.
* In post-order traversal mode when we don't have a DataRecursionQueue,
RecursiveASTVisitor also calls the WalkUp callback from the default
implementation of Traverse callbacks.
* However, in post-order traversal mode when we have a DataRecursionQueue,
RecursiveASTVisitor calls the WalkUp callback from PostVisitStmt.
As a result, when the user overrides the Traverse callback, in pre-order
traversal mode they never get the corresponding WalkUp callback. However
in the post-order traversal mode the WalkUp callback is invoked or not
depending on whether the data recursion optimization could be applied.
I had to adjust the implementation of TraverseCXXForRangeStmt in the
syntax tree builder to call the WalkUp method directly, as it was
relying on this behavior. There is an existing test for this
functionality and it prompted me to make this extra fix.
In addition, I had to fix the default implementation implementation of
RecursiveASTVisitor::TraverseSynOrSemInitListExpr to call WalkUpFrom in
the same manner as the implementation generated by the DEF_TRAVERSE_STMT
macro. Without this fix, the InitListExprIsPostOrderNoQueueVisitedTwice
test was failing because WalkUpFromInitListExpr was never called.
Reviewers: eduucaldas, ymandel
Reviewed By: eduucaldas, ymandel
Subscribers: gribozavr2, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D82486
Eduardo Caldas