Commit Graph

498 Commits

Author SHA1 Message Date
Johannes Doerfert f9d558c871 [OpenMP] "UnFix" layering problem with FrontendOpenMP
This reverts commit 97aa593a83 as it
causes problems (PR45453) https://reviews.llvm.org/D77574#1966321.

This additionally adds an explicit reference to FrontendOpenMP to
clang-tidy where ASTMatchers is used.

This is hopefully just a temporary solution. The dependence on
`FrontendOpenMP` from `ASTMatchers` should be handled by CMake
implicitly, not us explicitly.

Reviewed By: aheejin

Differential Revision: https://reviews.llvm.org/D77666
2020-04-07 14:41:18 -05:00
Marcel Hlopko 1bf055c989 [Syntax] Add mapping from spelled to expanded tokens for TokenBuffer
Summary:
Same restrictions apply as in the other direction: macro arguments are
not supported yet, only full macro expansions can be mapped.

Taking over from https://reviews.llvm.org/D72581.

Reviewers: gribozavr2, sammccall

Reviewed By: gribozavr2

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D77209
2020-04-07 15:07:16 +02:00
Sam McCall d66afd6dde [Syntax] Merge overlapping top-level macros in TokenBuffer
Summary:
Our previous definition of "top-level" was too informal, and didn't
allow for overlapping macros that each directly produce expanded tokens.
See D77507 for previous discussion.

Fixes http://bugs.llvm.org/show_bug.cgi?id=45428

Reviewers: kadircet, vabridgers

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D77615
2020-04-07 15:06:04 +02:00
Johannes Doerfert 97aa593a83 [OpenMP] Fix layering problem with FrontendOpenMP
Summary:
ASTMatchers is used in various places and it now exposes the
LLVMFrontendOpenMP library to its users without them needing to depend
on it explicitly.

Reviewers: lebedev.ri

Subscribers: mgorny, yaxunl, bollu, guansong, martong, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D77574
2020-04-06 13:04:26 -05:00
Johannes Doerfert 931c0cd713 [OpenMP][NFC] Move and simplify directive -> allowed clause mapping
Move the listing of allowed clauses per OpenMP directive to the new
macro file in `llvm/Frontend/OpenMP`. Also, use a single generic macro
that specifies the directive and one allowed clause explicitly instead
of a dedicated macro per directive.

We save 800 loc and boilerplate for all new directives/clauses with no
functional change. We also need to include the macro file only once and
not once per directive.

Depends on D77112.

Reviewed By: JonChesterfield

Differential Revision: https://reviews.llvm.org/D77113
2020-04-06 00:04:08 -05:00
Marcel Hlopko cdce2fe561 [Syntax] Remove delayed folding from tree building.
Summary:
This patch removes delayed folding and replaces it with forward peeking.

Delayed folding was previously used as a solution to the problem that
declaration doesn't have a representation in the AST. For example following
code:

```
int a,b;
```

is expressed in the AST as:

```
TranslationUnitDecl
|-...
|-VarDecl `int a`
`-VarDecl `int b`
```

And in the syntax tree we need:

```
*: TranslationUnit
`-SimpleDeclaration
  |-int
  |-SimpleDeclarator
  | `-a
  |-,
  |-SimpleDeclarator
  | `-b
  |-;
```

So in words, we need to create SimpleDeclaration to be a parent of
SimpleDeclarator nodes. Previously we used delayed folding to make sure SimpleDeclarations will be
eventually created. And in case multiple declarators requested declaration
creation, declaration range was extended to cover all declarators.

This design started to be hard to reason about, so we decided to replace it with
forward peeking. The last declarator node in the chain is responsible for creating
SimpleDeclaration for the whole chain. Range of the declaration corresponds to
the source range of the declarator node. Declarator decides whether its the last
one by peeking to the next AST node (see `isResponsibleForCreatingDeclaration`).

This patch does following:
* Removed delayed folding logic
* Tweaks Token.dumpForTests
* Moves getQualifiedNameStart inside BuildTreeVisitor
* Extracts BuildTreeVisitor.ProcessDeclaratorAndDeclaration
* Renames Builder.getDeclRange to Builder.getDeclarationRange and uses the
  method in all places.
* Adds a bunch of tests

Reviewers: gribozavr2

Reviewed By: gribozavr2

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D76922
2020-03-31 09:47:50 +02:00
Kadir Cetinkaya 9619c2cc9a
[clang][Syntax] Handle macro arguments in spelledForExpanded
Reviewers: sammccall

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D75446
2020-03-28 16:35:46 +01:00
Marcel Hlopko eddede9d51 [Syntax] Test both the default and windows target platforms in unittests
Summary:
This increases the coverage for things that differ between Linux and Windows, such as -fdelayed-template-parsing. This would have prevented the rollback of https://reviews.llvm.org/D76346.

While at it, update -std=c++11 to c++17 for the test.

Reviewers: gribozavr2

Reviewed By: gribozavr2

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D76497
2020-03-20 16:33:58 +01:00
Dmitri Gribenko 9967352a03 Revert "[Syntax] Test both the default and windows target platforms in unittests"
This reverts commit fd7300f717. The fix
in this patch didn't help and the Windows buildbot broke:
http://45.33.8.238/win/10881/step_7.txt
2020-03-20 12:13:49 +01:00
Marcel Hlopko fd7300f717 [Syntax] Test both the default and windows target platforms in unittests
Summary:
This increases the coverage for things that differ between Linux and Windows, such as `-fdelayed-template-parsing`. This would have prevented the rollback of https://reviews.llvm.org/D76346.

While at it, update -std=c++11 to c++17 for the test.

Reviewers: gribozavr2

Reviewed By: gribozavr2

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D76433
2020-03-20 11:42:18 +01:00
Marcel Hlopko e9630630ff [Syntax] Split syntax tests
Summary:
This patch split Basic test into multple individual tests to allow simpler
filtering and clearer signal into what's broken when it's broken.

Reviewers: gribozavr2

Reviewed By: gribozavr2

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D76366
2020-03-20 09:53:43 +01:00
Marcel Hlopko 88bf9b3d26 [Syntax] Build template declaration nodes
Summary:
Rollforward of
https://reviews.llvm.org/rGdd12826808f9079e164b82e64b0697a077379241 after
temporarily adding -fno-delayed-template-parsing to the TreeTest.

Original summary:

> Copy of https://reviews.llvm.org/D72334, submitting with Ilya's permission.
>
> Handles template declaration of all kinds.
>
> Also builds template declaration nodes for specializations and explicit
> instantiations of classes.
>
> Some missing things will be addressed in the follow-up patches:
>
> * specializations of functions and variables,
> * template parameters.

Reviewers: gribozavr2

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D76418
2020-03-19 17:43:07 +01:00
Nico Weber 881f5b5a7b Revert "[Syntax] Build template declaration nodes"
This reverts commit dd12826808.
Breaks tests on Windows, see https://reviews.llvm.org/D76346#1929208
2020-03-18 12:57:55 -04:00
Marcel Hlopko dd12826808 [Syntax] Build template declaration nodes
Summary:
Copy of https://reviews.llvm.org/D72334, submitting with Ilya's permission.

Handles template declaration of all kinds.

Also builds template declaration nodes for specializations and explicit
instantiations of classes.

Some missing things will be addressed in the follow-up patches:

specializations of functions and variables,
template parameters.

Reviewers: gribozavr2

Reviewed By: gribozavr2

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D76346
2020-03-18 16:16:59 +01:00
Marcel Hlopko 7d382dcd46 [Syntax] Build declarator nodes
Summary:
Copy of https://reviews.llvm.org/D72089 with Ilya's permission. See
https://reviews.llvm.org/D72089 for the first batch of comments.

Reviewers: gribozavr2

Reviewed By: gribozavr2

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D76220
2020-03-16 19:13:59 +01:00
Reid Kleckner 213aea4c58 Remove unused Endian.h includes, NFC
Mainly avoids including Host.h everywhere:

$ diff -u <(sort thedeps-before.txt) <(sort thedeps-after.txt) \
    | grep '^[-+] ' | sort | uniq -c | sort -nr
   3141 - /usr/local/google/home/rnk/llvm-project/llvm/include/llvm/Support/Host.h
2020-03-11 15:45:34 -07:00
Kadir Cetinkaya cd9b2e18bd
[clang][Syntax] Add spelledTokenAt helper to TokenBuffer
Reviewers: sammccall

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D75503
2020-03-03 14:30:41 +01:00
Adrian McCarthy 0a3117832c Need /bigobj to build SourceCodeTest.cpp with MSVC. 2020-02-27 14:56:29 -08:00
Joe Turner b26c88e3c6 [clang-tidy] Store all ranges in clang::tooling::Diagnostic
Summary: Instead of dropping all the ranges associated with a Diagnostic when
converting them to a ClangTidy error, instead attach them to the ClangTidyError,
so they can be consumed by other APIs.

Patch by Joe Turner <joturner@google.com>.
Differential Revision: https://reviews.llvm.org/D69782
2020-02-27 19:39:42 +01:00
Yitzhak Mandelbaum 38b4516de8 [libTooling] Add function to determine associated text of a declaration.
Summary:
Second attempt -- the first was reverted in commit 0e480b39c6, because of test breakages. This revision fixes the cause of the test breakages.

Original description follows:
This patch adds `getAssociatedRange` which, for a given decl, computes preceding
and trailing text that would conceptually be associated with the decl by the
reader. This includes comments, whitespace, and separators like ';'.

Reviewers: gribozavr

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D72153
2020-02-26 09:56:48 -05:00
Kadir Cetinkaya 8c2cf499e6
[clang][Tooling] Add a way to tokenize a FileRange
Reviewers: sammccall

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D74962
2020-02-26 13:50:41 +01:00
Nico Weber 0e480b39c6 Revert "[libTooling] Add function to determine associated text of a declaration."
This reverts commit 9c54f6154f.
Breaks two tests on Windows, see e.g.
http://lab.llvm.org:8011/builders/clang-x64-windows-msvc/builds/14505/steps/stage%201%20check/logs/stdio
2020-02-25 19:21:52 -05:00
Yitzhak Mandelbaum 9c54f6154f [libTooling] Add function to determine associated text of a declaration.
Summary:
This patch adds `getAssociatedRange` which, for a given decl, computes preceding
and trailing text that would conceptually be associated with the decl by the
reader. This includes comments, whitespace, and separators like ';'.

Reviewers: gribozavr

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D72153
2020-02-25 16:37:20 -05:00
Yitzhak Mandelbaum 523cae324d [libTooling] Add option for `buildAST` to report diagnostics.
Summary:
Currently, `buildAST[WithArgs]` either succeeds or fails.  This patch adds
support for the caller to pass a `DiagnosticConsumer` to receive all relevant
diagnostics.

Reviewers: gribozavr

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D74763
2020-02-18 09:11:51 -05:00
Justin Lebar 027eb71696 Use std::foo_t rather than std::foo in clang.
Summary: No functional change.

Reviewers: bkramer, MaskRay, martong, shafik

Subscribers: martong, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D74414
2020-02-11 10:37:08 -08:00
Benjamin Kramer 757bdc64d3 Fix clang unnittest build with GCC 5 2020-01-29 10:30:36 +01:00
Benjamin Kramer adcd026838 Make llvm::StringRef to std::string conversions explicit.
This is how it should've been and brings it more in line with
std::string_view. There should be no functional change here.

This is mostly mechanical from a custom clang-tidy check, with a lot of
manual fixups. It uncovers a lot of minor inefficiencies.

This doesn't actually modify StringRef yet, I'll do that in a follow-up.
2020-01-28 23:25:25 +01:00
Yitzhak Mandelbaum b9d2bf38e8 [libTooling] Fix bug in Stencil handling of macro ranges
Summary: Currently, an attempt to rewrite source code inside a macro expansion succeeds, but results in empty text, rather than failing with an error.  This patch restructures to the code to explicitly validate ranges before attempting to edit them.

Reviewers: gribozavr

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D72274
2020-01-17 12:11:25 -05:00
Dmitri Gribenko 0406b4fab9 Renamed traverseDecl to TraverseDecl in a test
RecursiveASTVisitor expects TraverseDecl to be implemented by
subclasses.
2020-01-17 17:12:23 +01:00
Ilya Biryukov 013c07f697 [Syntax] Unset IsOriginal flag on nodes removed from the tree
And add a corresponding test.
Only nodes inside the TranslationUnit subtree can be marked as original,
computeReplacements() relies on this.
2020-01-14 17:00:33 +01:00
Ilya Biryukov 07a41018e9 [Syntax] Mark synthesized nodes as modifiable
This was an oversight in the original patch.
Also add corresponding tests.
2020-01-14 16:41:09 +01:00
Ilya Biryukov 04f627f6b9 [Syntax] Build spanning SimpleDecalration for classes, structs, etc
When they are free-standing, e.g. `struct X;` or `struct X {};`.
Although this complicates the common case (of free-standing class
declarations), this ensures the less common case (e.g. `struct X {} a;`)
are handled uniformly and produce similar syntax trees.
2020-01-03 12:33:11 +01:00
Alexey Bader 128f39da93 Fix crash in getFullyQualifiedName for inline namespace
Summary: The ICE happens when the most outer namespace is an inline namespace.

Reviewers: bkramer, ilya-biryukov

Reviewed By: ilya-biryukov

Subscribers: ebevhan, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D71962
2019-12-28 16:35:51 +03:00
Kadir Cetinkaya c2377eae28
[clang][Tooling] Prefer -x over -std when interpolating
Summary:
Currently interpolation logic prefers -std over -x. But the latter is a
more strong signal, so this patch inverts the order and only makes use of -std
if -x didn't exist.

Fixes https://github.com/clangd/clangd/issues/185

Thanks @sammccall for tracking this down!

Reviewers: sammccall

Subscribers: ilya-biryukov, usaxena95, cfe-commits, sammccall

Tags: #clang

Differential Revision: https://reviews.llvm.org/D71727
2019-12-20 11:10:36 +01:00
Ilya Biryukov c1bbefef9d [Syntax] Use a hash table to search for tokens by their location
This is both more efficient and avoids corner cases in
`SourceManager::isBeforeInTranslationUnit`.

The change is trivial and clearly a performance improvement on the hot
path of building the syntax tree, so sending without review.
2019-12-18 12:24:00 +01:00
Ilya Biryukov 038f53882e [Syntax] Uppercase the first letter of the test name. NFC
To match naming style of other tests.
2019-12-18 12:20:30 +01:00
Ilya Biryukov 1ad15046dc [Syntax] Allow to mutate syntax trees
Summary:
This patch adds facilities to mutate the syntax trees and produce
corresponding text replacements.

The public interface of the syntax library now includes facilities to:
    1. perform type-safe modifications of syntax trees,
    2. compute textual replacements to apply the modifications,
    3. create syntax trees not backed by the source code.

For each of the three, we only add a few example transformations in this
patch to illustrate the idea, support for more kinds of nodes and
transformations will be done in follow-up patches.

The high-level mutation operations are implemented on top of operations
that allow to arbitrarily change the trees. They are considered to be
implementation details and are not available to the users of the
library.

Reviewers: sammccall, gribozavr2

Reviewed By: gribozavr2

Subscribers: merge_guards_bot, mgorny, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D64573
2019-12-18 12:19:03 +01:00
Sam McCall 3f8da5d091 [Tooling/Syntax] Helpers to find spelled tokens touching a location.
Summary: Useful when positions are used to target nodes, with before/after ambiguity.

Reviewers: ilya-biryukov, kbobyrev

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D71356
2019-12-12 12:59:50 +01:00
Ilya Biryukov 7d7789899f [Tooling] Move raw string literal out of a macro call. NFC
Should fix buildbots with some older gcc versions.
2019-12-12 10:53:20 +01:00
Ilya Biryukov be14a22b47 [Syntax] Build nodes for simple cases of top level declarations
Summary:
More complicated nodes (e.g. template declarations) will be implemented
in the follow-up patches.

Reviewers: gribozavr2

Reviewed By: gribozavr2

Subscribers: merge_guards_bot, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D70856
2019-12-12 08:04:22 +01:00
Haojian Wu 6d5c273500 Reland "[AST] Traverse the class type loc inside the member type loc.""
Summary: added a unittest which causes "TL.getClassTInfo" is null.

Reviewers: ilya-biryukov

Subscribers: mgorny, jkorous, arphaman, kadircet, usaxena95, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D71186
2019-12-09 11:18:12 +01:00
Sterling Augustine 6470497817 Revert "[AST] Traverse the class type loc inside the member type loc."
This reverts commit 7f93cb6228.

The assertion at RecursiveASTVisitor.h:1169 fails when passed a TypeLocNode.
Not sure if the correct fix is to use getTypeLocClass or something else.
2019-12-05 16:48:18 -08:00
Ilya Biryukov 242649c17c [Syntax] Show input code on tests failures. NFC 2019-12-05 16:25:35 +01:00
Haojian Wu 7f93cb6228 [AST] Traverse the class type loc inside the member type loc.
Summary:
We are missing this currently.

This would fix https://github.com/clangd/clangd/issues/216.

Reviewers: ilya-biryukov

Subscribers: mgorny, kadircet, usaxena95, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D70849
2019-12-05 15:03:30 +01:00
Kadir Cetinkaya 45ef055d4f
[clang][Tooling] Add support for .rsp files in compile_commands.json
Summary:
Add support for .rsp files.

Fixes https://github.com/clangd/clangd/issues/81

Patch By: liu hui(@lh123)

Reviewers: sammccall, ilya-biryukov, hokein, kadircet

Reviewed By: kadircet

Subscribers: merge_guards_bot, mgorny, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits

Tags: #clang-tools-extra, #clang

Differential Revision: https://reviews.llvm.org/D70222
2019-12-04 15:13:12 +01:00
Sam McCall c9c714c705 Reland [clangd] Rethink how SelectionTree deals with macros and #includes.
This reverts commit 905b002c13.

Avoid tricky (and invalid) comparator for std::set.
2019-12-03 17:53:43 +01:00
Sam McCall 905b002c13 Revert "[clangd] Rethink how SelectionTree deals with macros and #includes."
This reverts commit 19daa21f84.

It causes a bunch of failures on a bot that I've been unable to
reproduce so far:
http://45.33.8.238/mac/3308/step_7.txt
2019-11-29 19:59:02 +01:00
Sam McCall 19daa21f84 [clangd] Rethink how SelectionTree deals with macros and #includes.
Summary:
The exclusive-claim model is successful at resolving conflicts over tokens
between parent/child or siblings. However claims at the spelled-token
level do the wrong thing for macro expansions, where siblings can be
equally associated with the macro invocation.
Moreover, any model that only uses the endpoints in a range can fail when
a macro invocation occurs inside the node.

To address this, we use the existing TokenBuffer in more depth.
Claims are expressed in terms of expanded tokens, so there is no need to worry
about macros, includes etc.

Once we know which expanded tokens were claimed, they are mapped onto
spelled tokens for hit-testing.
This mapping is fairly flexible, currently the handling of macros is
pretty simple (map macro args onto spellings, other macro expansions onto the
macro name token).
This mapping is in principle token-by-token for correctness (though
there's some batching for performance).

The aggregation of the selection enum is now more principled as we need to be
able to aggregate several hit-test results together.

For simplicity i removed the ability to determine selectedness of TUDecl.
(That was originally implemented in 90a5bf92ff97b1, but doesn't seem to be very
important or worth the complexity any longer).

The expandedTokens(SourceLocation) helper could be added locally, but seems to
make sense on TokenBuffer.

Fixes https://github.com/clangd/clangd/issues/202
Fixes https://github.com/clangd/clangd/issues/126

Reviewers: hokein

Subscribers: MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits, ilya-biryukov

Tags: #clang

Differential Revision: https://reviews.llvm.org/D70512
2019-11-29 15:21:13 +01:00
Ilya Biryukov e702bdb859 [Syntax] Build SimpleDeclaration node that groups multiple declarators
Summary:
Also remove the temporary TopLevelDeclaration node and add
UnknownDeclaration to represent other unknown nodes.

See the follow-up change for building more top-level declarations.
Adding declarators is also pretty involved and will be done in another
follow-up patch.

Reviewers: gribozavr2

Reviewed By: gribozavr2

Subscribers: merge_guards_bot, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D70787
2019-11-29 11:39:45 +01:00
Yitzhak Mandelbaum 01e8dd2e7a [libTooling] Add stencil combinators for nodes that may be pointers or values.
Summary:
Adds combinators `maybeDeref` and `maybeAddressOf` to provide a uniform way to handle
nodes which may be bound to either a pointer or a value (most often in the
context of member expressions). Such polymorphism is already supported by
`access`; these combinators extend it to more general uses.

Reviewers: gribozavr

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D70554
2019-11-22 12:36:40 -05:00