Add support for concepts and requires expression in the clang index.
Genarate USRs for concepts.
Also change how `RecursiveASTVisitor` handles return type requirement in
requires expressions. The new code unpacks the synthetic template parameter
list used for storing the actual expression. This simplifies
implementation of the indexing. No code seems to depend on the original
traversal anyway and the synthesized template parameter list is easily
accessible from inside the requires expression if needed.
Add tests in the clangd codebase.
Fixes https://github.com/clangd/clangd/issues/1103.
Reviewed By: sammccall
Differential Revision: https://reviews.llvm.org/D124441
- Inline SymbolID hashing to header
- Don't collect references for symbols without a SymbolID
- Store referenced symbols, rather than separately storing decls and
macros.
- Don't defer ref collection to end of translation unit
- Perform const_cast when updating reference counts (~0.5% saving)
- Introduce caching for getSymbolID in SymbolCollector. (~30% saving)
- Don't modify symbolslab if there's no definition location
- Don't lex the whole file to deduce spelled tokens, just lex the
relevant piece (~8%)
Overall this achieves ~38% reduction in time spent inside
SymbolCollector compared to baseline (on my machine :)).
I'd expect the last optimization to affect dynamic index a lot more, I
was testing with clangd-indexer on clangd subfolder of LLVM. As
clangd-indexer runs indexing of whole TU at once, we indeed see almost
every token from every source included in the TU (hence lexing full
files vs just lexing referenced tokens are almost the same), whereas
during dynamic indexing we mostly index main file symbols, but we would
touch the files defining/declaring those symbols, and lex complete files
for nothing, rather than just the token location.
The last optimization is also a functional change (added test),
previously we used raw tokens from syntax::tokenize, which didn't
canonicalize trigraphs/newlines in identifiers, wheres
Lexer::getSpelling canonicalizes them.
Differential Revision: https://reviews.llvm.org/D122894
This patch switches CanonicalInclude mappings to use `llvm::sys::fs::UniqueID` for a stable file representation because the `FileEntry::getName()` results turn out to be changing throughout the lifetime of a program (exposed in D120306). This patch makes it possible for D120306 to be re-landed and increases overall stability.
Reviewed By: sammccall
Differential Revision: https://reviews.llvm.org/D123031
It's deprecated; migrate to FileEntryRef::getName where it doesn't matter.
Also change one subtle case of implicit FileEntry::getName to be explicit.
After this patch, all the remaining FileEntry::getName calls are subtle
cases where we may be relying on exactly which filename variant is returned
(for indexing, IWYU directive handling, etc).
With this change, clangd now computes framework-style includes
for framework headers at indexing time.
Differential Revision: https://reviews.llvm.org/D117056
This is a cleanup of all llvm-qualified-auto findings.
This patch was created by automatically applying the fixes from
clang-tidy.
Differential Revision: https://reviews.llvm.org/D113898
Main use of these is in the standard library, where they generally clutter up
the index.
Certain macros are also common, we don't touch indexing of macros in this patch.
Differential Revision: https://reviews.llvm.org/D115301
Also give CanonicalIncludes a less powerful interface (canonicalizes
symbols vs headers separately) so we can cache its results better.
Prior to this:
- path->uri conversions were not consistently cached, this is
particularly cheap when we start from a FileEntry* (which we often can)
- only a small fraction of header-to-include calculation was cached
This is a significant speedup at least for dynamic indexing of preambles.
On my machine, opening XRefs.cpp:
```
PreambleCallback 1.208 -> 1.019 (-15.7%)
BuildPreamble 5.538 -> 5.214 (-5.8%)
```
Differential Revision: https://reviews.llvm.org/D98371
Previously we did not record local class declarations. Now with features like
findImplementation and typeHierarchy, we have a need to index such local
classes to accurately report subclasses and implementations of methods.
Performance testing results:
- No changes in indexing timing.
- No significant change in memory usage.
- **1%** increase in #relations.
- **0.17%** increase in #refs.
- **0.22%** increase #symbols.
**New index stats**
Time to index: **4:13 min**
memory usage **543MB**
number of symbols: **521.5K**
number of refs: **8679K**
number of relations: **49K**
**Base Index stats**
Time to index: **4:15 min**
memory usage **542MB**
number of symbols: **520K**
number of refs: **8664K**
number of relations: **48.5K**
Fixes: https://github.com/clangd/clangd/issues/644
Differential Revision: https://reviews.llvm.org/D94785
This patch is a try to fix `WorkspaceSymbols.Macros` test after D93796.
If a macro definition is in the preamble section, then it appears to be in the preamble (static) index and not in the main-file (dynamic) index.
Thus, a such macro could not be found at a symbol search according to the logic that we skip symbols from the static index if the location of these symbols is inside the dynamic index files.
To fix this behavior this patch adds main file macros into the main-file (dynamic) index.
Reviewed By: sammccall
Differential Revision: https://reviews.llvm.org/D94477
We were default initializing SymbolIDs before, which would leave
indeterminate values in underlying std::array.
This patch updates the underlying data initalization to be value-init and adds a
way to check for validness of a SymbolID.
Differential Revision: https://reviews.llvm.org/D90397
Up until now, we relied on matching the filename.
This depends on unstable details of libstdc++ and doesn't work well on other
stdlibs. Also we'd like to remove it (see D88204).
Differential Revision: https://reviews.llvm.org/D88885
Summary: This will be needed to support call hierarchy
Reviewers: kadircet
Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D83536
Summary:
Without this patch clangd does not collect references for main-file symbols if there is no public declaration in preamble.
Example:
`test1.c`
```
void f1() {}
```
`test2.c`
```
extern void f1();
void f2() {
f^1();
}
```
`Find all references` does not show definition of f1() in the result, but GTD works OK.
Reviewers: sammccall, kadircet
Reviewed By: kadircet
Subscribers: ilya-golovenko, ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D84513
Summary:
This was resulting in macros coming from preambles vanishing when user
have opened the source header. For example:
```
// test.h:
#define X
```
and
```
// test.cc
#include "test.h
^
```
If user only opens test.cc, we'll get `X` as a completion candidate,
since it is indexed as part of the preamble. But if the user opens
test.h afterwards we would index it as part of the main file and lose
the symbol (as new index shard for test.h will override the existing one
in dynamic index).
Also we were not setting origins for macros correctly, this patch also
fixes it.
Fixes https://github.com/clangd/clangd/issues/461
Reviewers: hokein
Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D84297
It's not clear whether clangd should really consider #import to mark a
header as self-contained or not, but this change preserves the old (and
unit-tested) behavior.
Summary:
This reduces memory usage by dynamic index from more than 400MB to 32MB
when all files in clang-tools-extra/clangd/*.cpp are active in clangd.
Reviewers: sammccall
Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D77732
Summary:
DeclarationName for cxx constructor is special, it is not an identifier.
thus the "Spelled" flag are not set for all ctor references, this patch
fixes it.
Reviewers: kbobyrev
Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D74125
This patch allows the index does to provide a way to distinguish
implicit references (e.g. coming from macro expansions) from the spelled
ones. The corresponding flag was added to RefKind and symbols that are
referenced without spelling their name explicitly are now marked
implicit. This allows fixing incorrect behavior when renaming a symbol
that was referenced in macro expansions would try to rename macro
invocations.
Differential Revision: D72746
Reviewed by: hokein
Summary:
Adds macro references to the dynamic index.
Tests added.
Also exposed a new API to convert path to URI in URI.h
Reviewers: hokein
Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D71406
`findName` was always used in conjuction with `spellingLocIfSpelled`.
This patch replaces patterns of the form:
spellingLocIfSpelled(findName(&ND), SM)
With a new helper function:
nameLocation(ND, SM)
And removes `spellingLocIfSpelled` and `findName`. Both are never used
anywhere else and the latter is an equivalent of `Decl::getLocation` if
we ever need it again.
Summary:
Previously, xrefs has inconsistent behavior when the reference is inside
macro body:
- AST-based xrefs (for main file) uses the expansion location;
- our index uses the spelling location;
This patch makes our index use file locations for references, which is
consistent with AST-based xrefs, and kythe as well.
After this patch, memory usage of static index on LLVM increases ~5%.
Reviewers: ilya-biryukov
Subscribers: merge_guards_bot, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D70480
Summary:
This adds the references for macros to the SymbolCollector (used for static index).
Enabled if `CollectMacro` option is set.
Reviewers: hokein
Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D70489
Summary:
We use the name from the IdentifierInfo of the Macro to compute its
SymbolID. It is better to just take the Name as a parameter to avoid
storing the IdentifierInfo whenever we need the SymbolID for the Macro.
Patch by UTKARSH SAXENA!
Reviewers: hokein
Reviewed By: hokein
Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D69937
Summary:
This is used for cross-file rename. When renaming a class, we expect to
rename all related constructors/destructors.
Reviewers: kadircet, ilya-biryukov
Subscribers: MaskRay, jkorous, arphaman, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D69338
Summary:
This significantly improves performance of background indexing.
We do not collect references and declarations inside the processed
files, so this does not affect the final indexing results.
The idea is borrowed from libclang, which has a similar optimization in
its indexing functionality.
Measurements show a nice decrease in indexing time, up to ~40% for
building the whole index. These are not proper benchmarks, so one should
not rely on these results too much.
1. Rebuilding the whole index for LLVM:
- Before. Total time: 14m58s.
./bin/clangd -pch-storage=memory < ./clangd.input 23917.67s user 515.86s system 2718% cpu 14:58.68 total
- After. Total time: 8m41s.
./bin/clangd -pch-storage=memory < ./clangd.input 13627.29s user 288.10s system 2672% cpu 8:40.67 total
2. Rebuilding index after removing shards matching '*clangd*' (case-insensitively):
- Before. Total time: 30s.
./bin/clangd -pch-storage=memory < ./clangd.input 130.94s user 6.82s system 452% cpu 30.423 total
- After. Total time: 26s.
./bin/clangd -pch-storage=memory < ./clangd.input 80.51s user 5.40s system 333% cpu 25.777 total
Reviewers: kadircet, sammccall
Reviewed By: kadircet
Subscribers: MaskRay, jkorous, arphaman, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D66226
llvm-svn: 369349
Now that we've moved to C++14, we no longer need the llvm::make_unique
implementation from STLExtras.h. This patch is a mechanical replacement
of (hopefully) all the llvm::make_unique instances across the monorepo.
Differential revision: https://reviews.llvm.org/D66259
llvm-svn: 368944
The FileManager has been updated to return llvm::ErrorOr from getFile
and getDirectory, this commit updates all the callers of those APIs from
clang.
llvm-svn: 367617
Summary:
Previously, we only collect refs of the symbols which are declared in
the preamble and referenced in the main file, it works well when the
main file is .cpp file.
However, when the main file is .h file (when opening a .h file in the
editor), we don't collect refs of the symbol declared in this file, so we miss
these refs in our dynamic index.
A typical scenario:
1. Open Foo.h (which contains class Foo)
2. Open Foo.cpp, call find references for Foo
And we only get refs from Foo.cpp.
Reviewers: ilya-biryukov
Subscribers: MaskRay, jkorous, arphaman, kadircet, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D63818
llvm-svn: 364893
Summary:
The only relation currently collected is RelationBaseOf, because this is
all we need for type hierarchy subtypes. Additional relations can be
collected in the future as the need arises.
This patch builds on D59407 and D62459.
Reviewers: kadircet
Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D62471
llvm-svn: 362467
Ilya Biryukov