Summary:
This reduces memory usage by dynamic index from more than 400MB to 32MB
when all files in clang-tools-extra/clangd/*.cpp are active in clangd.
Reviewers: sammccall
Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D77732
Summary:
DeclarationName for cxx constructor is special, it is not an identifier.
thus the "Spelled" flag are not set for all ctor references, this patch
fixes it.
Reviewers: kbobyrev
Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D74125
This patch allows the index does to provide a way to distinguish
implicit references (e.g. coming from macro expansions) from the spelled
ones. The corresponding flag was added to RefKind and symbols that are
referenced without spelling their name explicitly are now marked
implicit. This allows fixing incorrect behavior when renaming a symbol
that was referenced in macro expansions would try to rename macro
invocations.
Differential Revision: D72746
Reviewed by: hokein
Summary:
Adds macro references to the dynamic index.
Tests added.
Also exposed a new API to convert path to URI in URI.h
Reviewers: hokein
Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D71406
`findName` was always used in conjuction with `spellingLocIfSpelled`.
This patch replaces patterns of the form:
spellingLocIfSpelled(findName(&ND), SM)
With a new helper function:
nameLocation(ND, SM)
And removes `spellingLocIfSpelled` and `findName`. Both are never used
anywhere else and the latter is an equivalent of `Decl::getLocation` if
we ever need it again.
Summary:
Previously, xrefs has inconsistent behavior when the reference is inside
macro body:
- AST-based xrefs (for main file) uses the expansion location;
- our index uses the spelling location;
This patch makes our index use file locations for references, which is
consistent with AST-based xrefs, and kythe as well.
After this patch, memory usage of static index on LLVM increases ~5%.
Reviewers: ilya-biryukov
Subscribers: merge_guards_bot, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D70480
Summary:
This adds the references for macros to the SymbolCollector (used for static index).
Enabled if `CollectMacro` option is set.
Reviewers: hokein
Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D70489
Summary:
We use the name from the IdentifierInfo of the Macro to compute its
SymbolID. It is better to just take the Name as a parameter to avoid
storing the IdentifierInfo whenever we need the SymbolID for the Macro.
Patch by UTKARSH SAXENA!
Reviewers: hokein
Reviewed By: hokein
Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D69937
Summary:
This is used for cross-file rename. When renaming a class, we expect to
rename all related constructors/destructors.
Reviewers: kadircet, ilya-biryukov
Subscribers: MaskRay, jkorous, arphaman, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D69338
Summary:
This significantly improves performance of background indexing.
We do not collect references and declarations inside the processed
files, so this does not affect the final indexing results.
The idea is borrowed from libclang, which has a similar optimization in
its indexing functionality.
Measurements show a nice decrease in indexing time, up to ~40% for
building the whole index. These are not proper benchmarks, so one should
not rely on these results too much.
1. Rebuilding the whole index for LLVM:
- Before. Total time: 14m58s.
./bin/clangd -pch-storage=memory < ./clangd.input 23917.67s user 515.86s system 2718% cpu 14:58.68 total
- After. Total time: 8m41s.
./bin/clangd -pch-storage=memory < ./clangd.input 13627.29s user 288.10s system 2672% cpu 8:40.67 total
2. Rebuilding index after removing shards matching '*clangd*' (case-insensitively):
- Before. Total time: 30s.
./bin/clangd -pch-storage=memory < ./clangd.input 130.94s user 6.82s system 452% cpu 30.423 total
- After. Total time: 26s.
./bin/clangd -pch-storage=memory < ./clangd.input 80.51s user 5.40s system 333% cpu 25.777 total
Reviewers: kadircet, sammccall
Reviewed By: kadircet
Subscribers: MaskRay, jkorous, arphaman, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D66226
llvm-svn: 369349
Now that we've moved to C++14, we no longer need the llvm::make_unique
implementation from STLExtras.h. This patch is a mechanical replacement
of (hopefully) all the llvm::make_unique instances across the monorepo.
Differential revision: https://reviews.llvm.org/D66259
llvm-svn: 368944
The FileManager has been updated to return llvm::ErrorOr from getFile
and getDirectory, this commit updates all the callers of those APIs from
clang.
llvm-svn: 367617
Summary:
Previously, we only collect refs of the symbols which are declared in
the preamble and referenced in the main file, it works well when the
main file is .cpp file.
However, when the main file is .h file (when opening a .h file in the
editor), we don't collect refs of the symbol declared in this file, so we miss
these refs in our dynamic index.
A typical scenario:
1. Open Foo.h (which contains class Foo)
2. Open Foo.cpp, call find references for Foo
And we only get refs from Foo.cpp.
Reviewers: ilya-biryukov
Subscribers: MaskRay, jkorous, arphaman, kadircet, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D63818
llvm-svn: 364893
Summary:
The only relation currently collected is RelationBaseOf, because this is
all we need for type hierarchy subtypes. Additional relations can be
collected in the future as the need arises.
This patch builds on D59407 and D62459.
Reviewers: kadircet
Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D62471
llvm-svn: 362467
Summary:
Both of these attempt to check whether a header guard exists while parsing the
file. However the file is only marked as guarded once clang finishes processing
it. We defer the checks and work until SymbolCollector::finish().
This is ugly and ad-hoc, deferring *all* work might be cleaner.
Reviewers: kadircet
Subscribers: ilya-biryukov, MaskRay, jkorous, mgrang, arphaman, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D61442
llvm-svn: 359880
Summary:
This is a tricky case (we baked the assumption that symbols come from
the preamble xor mainfile pretty deeply) and the fix is a bit of a hack:
We look at the code to guess the macro names, and deserialize them from
the preamble "by hand".
Reviewers: ilya-biryukov
Subscribers: ioeric, MaskRay, jkorous, arphaman, kadircet, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D60937
........
Fix buildbots http://lab.llvm.org:8011/builders/llvm-clang-lld-x86_64-scei-ps4-ubuntu-fast/builds/47684/
llvm-svn: 359796
Summary:
This is a tricky case (we baked the assumption that symbols come from
the preamble xor mainfile pretty deeply) and the fix is a bit of a hack:
We look at the code to guess the macro names, and deserialize them from
the preamble "by hand".
Reviewers: ilya-biryukov
Subscribers: ioeric, MaskRay, jkorous, arphaman, kadircet, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D60937
llvm-svn: 359778
Summary:
llvm::Regex is really slow, and regex evaluation during preamble indexing was
showing up as 25% on a profile of clangd in a codebase with large preambles.
Reviewers: ilya-biryukov
Subscribers: MaskRay, jkorous, arphaman, kadircet, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D61120
llvm-svn: 359214
Summary:
Typically used with umbrella headers, e.g. GTK:
#if !defined (__GTK_H_INSIDE__) && !defined (GTK_COMPILATION)
#error "Only <gtk/gtk.h> can be included directly."
#endif
Heuristic is fairly conservative, a quick code search over github showed
a fair number of hits and few/no false positives. (Not all were umbrella
headers, but I'd be happy avoiding include insertion for all of them).
We may want to relax the heuristic later to catch more cases.
Reviewers: ioeric
Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D60815
llvm-svn: 358605
Summary:
We do have some reports of include insertion behaving badly in some
codebases. Requiring header guards both makes sense in principle, and is
likely to disable this "nice-to-have" feature in codebases where headers don't
follow the expected pattern.
With this we can drop some other heuristics, such as looking at file
extensions to detect known non-headers - implementation files have no guards.
One wrinkle here is #import - objc headers may not have guards because
they're intended to be used via #import. If the header is the main file
or is #included, we won't collect locations - merge should take care of
this if we see the file #imported somewhere. Seems likely to be OK.
Headers which have a canonicalization (stdlib, IWYU) are exempt from this check.
*.inc files continue to be handled by looking up to the including file.
This patch also adds *.def here - tablegen wants this pattern too.
In terms of code structure, the division between SymbolCollector and
CanonicalIncludes has shifted: SymbolCollector is responsible for more.
This is because SymbolCollector has all the SourceManager/HeaderSearch access
needed for checking for guards, and we interleave these checks with the *.def
checks in a loop (potentially).
We could hand all the info into CanonicalIncludes and put the logic there
if that's preferable.
Reviewers: ioeric
Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D60316
llvm-svn: 358571
Summary:
Some implicit/built-in decls lack the source location
information. Fallback to OrigD that we've seen in the source code
instead of the canonical one in those cases.
Reviewers: sammccall
Subscribers: cfe-commits, arphaman, jkorous, MaskRay, ioeric, ilya-biryukov
Tags: #clang
Differential Revision: https://reviews.llvm.org/D60689
llvm-svn: 358413
Summary:
Part of re-landing rC356541 with D59599. Changes the way we store
template arguments, previous patch was storing them inside Name field of Symbol.
Which was violating the assumption:
```Symbol::Scope+Symbol::Name == clang::clangd::printQualifiedName```
which was made in multiple places inside codebase. This patch instead moves
those arguments into their own field. Currently the field is meant to be
human-readable, can be made structured if need be.
Reviewers: ioeric, ilya-biryukov, gribozavr
Subscribers: MaskRay, jkorous, arphaman, jdoerfert, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D59640
llvm-svn: 358273
Summary:
This introduces ~4k new symbols, and ~10k refs for LLVM. We need that
information for providing better code navigation support:
- When references for a class template is requested, we should return these specializations as well.
- When children of a specialization is requested, we should be able to query for those symbols(instead of just class template)
Number of symbols: 378574 -> 382784
Number of refs: 5098857 -> 5110689
Reviewers: hokein, gribozavr
Reviewed By: gribozavr
Subscribers: nridge, ilya-biryukov, ioeric, MaskRay, jkorous, arphaman, jdoerfert, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D59083
llvm-svn: 356125
Summary:
This is a further optimization of r350803, we drop docs in static index for
symbols not being indexed for completion, while keeping the docs in dynamic
index (we rely on dynamic index to get docs for class members).
Reviewers: ilya-biryukov
Reviewed By: ilya-biryukov
Subscribers: ioeric, MaskRay, jkorous, arphaman, kadircet, cfe-commits
Differential Revision: https://reviews.llvm.org/D56539
llvm-svn: 354792
With ObjCPropertyDecl, ASTNode.OrigD can be a ObjCPropertyImplDecl
which is not a NamedDecl, leading to a crash since the code
incorrectly assumes ASTNode.OrigD will always be a NamedDecl.
Change by dgoldman (David Goldman)!
Differential Revision: https://reviews.llvm.org/D56916
llvm-svn: 351941
to reflect the new license.
We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.
Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.
llvm-svn: 351636
Summary:
This would save us some memory and disk space:
- Dex usage (261 MB vs 266 MB)
- Disk (75 MB vs 76 MB)
It would save more when we index the main file symbol D55185.
Reviewers: ilya-biryukov
Reviewed By: ilya-biryukov
Subscribers: nridge, ioeric, MaskRay, jkorous, arphaman, kadircet, cfe-commits
Differential Revision: https://reviews.llvm.org/D56314
llvm-svn: 350803
The new guideline is to qualify with 'llvm::' explicitly both in
'.h' and '.cpp' files. This simplifies moving the code between
header and source files and is easier to keep consistent.
llvm-svn: 350531
Summary:
There were a few different places where we canonicalized paths, each
one had its own flavor. This patch tries to unify them all under one place.
Reviewers: ilya-biryukov
Subscribers: ioeric, MaskRay, jkorous, arphaman, cfe-commits
Differential Revision: https://reviews.llvm.org/D55818
llvm-svn: 349618
Summary:
And add a hidden option to control whether the types are collected.
For experiments, will be removed when expected types implementation
is stabilized.
The index size is almost unchanged, e.g. the YAML index for all clangd
sources increased from 53MB to 54MB.
Reviewers: ioeric, sammccall
Reviewed By: sammccall
Subscribers: MaskRay, jkorous, arphaman, kadircet, cfe-commits
Differential Revision: https://reviews.llvm.org/D52274
llvm-svn: 347560
Summary:
Instead of passing around a list of supported URI schemes in clangd, we
expose an interface to convert a path to URI using any compatible scheme
that has been registered. It favors customized schemes and falls
back to "file" when no other scheme works.
Changes in this patch are:
- URI::create(AbsPath, URISchemes) -> URI::create(AbsPath). The new API finds a
compatible scheme from the registry.
- Remove URISchemes option everywhere (ClangdServer, SymbolCollecter, FileIndex etc).
- Unit tests will use "unittest" by default.
- Move "test" scheme from ClangdLSPServer to ClangdMain.cpp, and only
register the test scheme when lit-test or enable-lit-scheme is set.
(The new flag is added to make lit protocol.test work; I wonder if there
is alternative here.)
Reviewers: sammccall
Reviewed By: sammccall
Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits
Differential Revision: https://reviews.llvm.org/D54800
llvm-svn: 347467
Summary:
This would save us 8 bytes per ref, and buy us ~40MB in total
for llvm index (from ~300MB to ~260 MB).
The char pointer must be null-terminated, and llvm::StringSaver
guarantees it.
Reviewers: sammccall
Subscribers: ilya-biryukov, ioeric, MaskRay, jkorous, arphaman, kadircet, cfe-commits
Differential Revision: https://reviews.llvm.org/D53427
llvm-svn: 346852
Summary:
Namespace references is less useful compared with other symbols, and
they contribute large part of the index. This patch drops them.
The number of refs is reduced from 5.4 million to 4.7 million.
| | Before | After |
|file size | 78 MB | 71MB |
|memory | 330MB | 300MB|
Reviewers: sammccall
Subscribers: ilya-biryukov, ioeric, MaskRay, jkorous, arphaman, kadircet, cfe-commits
Differential Revision: https://reviews.llvm.org/D54202
llvm-svn: 346319
Kadir Cetinkaya