I went over the output of the following mess of a command:
`(ulimit -m 2000000; ulimit -v 2000000; git ls-files -z | parallel --xargs -0 cat | aspell list --mode=none --ignore-case | grep -E '^[A-Za-z][a-z]*$' | sort | uniq -c | sort -n | grep -vE '.{25}' | aspell pipe -W3 | grep : | cut -d' ' -f2 | less)`
and proceeded to spend a few days looking at it to find probable typos
and fixed a few hundred of them in all of the llvm project (note, the
ones I found are not anywhere near all of them, but it seems like a
good start).
Reviewed By: kadircet
Differential Revision: https://reviews.llvm.org/D130826
It's more natural to use uint8_t * (std::byte needs C++17 and llvm has
too much uint8_t *) and most callers use uint8_t * instead of char *.
The functions are recently moved into `llvm::compression::zlib::`, so
downstream projects need to make adaption anyway.
* Refactor compression namespaces across the project, making way for a possible
introduction of alternatives to zlib compression.
Changes are as follows:
* Relocate the `llvm::zlib` namespace to `llvm::compression::zlib`.
Reviewed By: MaskRay, leonardchan, phosek
Differential Revision: https://reviews.llvm.org/D128953
Before commit b3a991df3c SystemHeaderMap used to be a vector.
Commit b3a991df3c changed it into a map, but neglected to remove
duplicate keys (e.g. "bits/typesizes.h", "include/stdint.h", etc.).
To prevent confusion, remove all duplicates, build HeaderMapping
one pair at a time and assert() that no duplicates are found.
Change by Paul Pluzhnikov (ppluzhnikov)!
Reviewed By: ilya-biryukov
Differential Revision: https://reviews.llvm.org/D125742
This allows index implementations to fill container details when required specially when computing containerID is expensive.
Differential Revision: https://reviews.llvm.org/D125925
Tracked down the crash, which was argument-evaluation-order UB
in the wrapping indexStandardLibrary().
Sorry for the churn!
This reverts commit 77533ea443.
This reverts commit ccdb56ac10.
Still seeing windows failures on GN bots: http://45.33.8.238/win/58316/step_9.txt
Unfortunately I can't debug these at all - it's a bare unsymbolized
stacktrace, and I can't reproduce the failure.
This is a followup to D124715, which changed the default, and it anticipates
future patches raising the priority of Low (which is currently equal to
Background on Windows & Linux).
The main point is to allow users to restore the old behavior, which e.g.
allows efficiency cores to remain idle.
I did consider making this a config setting, this is a more complicated change:
- needs to touch queue priorities as well as thread priorities
- we don't know the priority until evaluating the config inside the task
- users would want the ability to prioritize background indexing tasks relative
to each other without necessarily affecting thread priority, so using one
option for both may be confusing
I don't really have a use case, so I prefer the simpler thing.
Differential Revision: https://reviews.llvm.org/D125673
This provides a nice "warm start" with all headers indexed, not just
those included so far.
The standard library is indexed after a preamble is parsed, using that
file's configuration. The result is pushed into the dynamic index.
If we later see a higher language version, we reindex it.
It's configurable as Index.StandardLibrary, off by default for now.
Based on D105177 by @kuhnel
Fixes https://github.com/clangd/clangd/issues/618
Differential Revision: https://reviews.llvm.org/D115232
On Apple Silicon Macs, using a Darwin thread priority of PRIO_DARWIN_BG seems to
map directly to the QoS class Background. With this priority, the thread is
confined to efficiency cores only, which makes background indexing take forever.
Introduce a new ThreadPriority "Low" that sits in the middle between Background
and Default, and maps to QoS class "Utility" on Mac. Make this new priority the
default for indexing. This makes the thread run on all cores, but still lowers
priority enough to keep the machine responsive, and not interfere with
user-initiated actions.
I didn't change the implementations for Windows and Linux; on these systems,
both ThreadPriority::Background and ThreadPriority::Low map to the same thread
priority. This could be changed as a followup (e.g. by using SCHED_BATCH for Low
on Linux).
See also https://github.com/clangd/clangd/issues/1119.
Reviewed By: sammccall, dgoldman
Differential Revision: https://reviews.llvm.org/D124715
Add support for concepts and requires expression in the clang index.
Genarate USRs for concepts.
Also change how `RecursiveASTVisitor` handles return type requirement in
requires expressions. The new code unpacks the synthetic template parameter
list used for storing the actual expression. This simplifies
implementation of the indexing. No code seems to depend on the original
traversal anyway and the synthesized template parameter list is easily
accessible from inside the requires expression if needed.
Add tests in the clangd codebase.
Fixes https://github.com/clangd/clangd/issues/1103.
Reviewed By: sammccall
Differential Revision: https://reviews.llvm.org/D124441
Reduce peak memory usage by tearing down the intermediate representation
as we build the final one. Rather than deleting it in the end.
Differential Revision: https://reviews.llvm.org/D124240
This patch changes type of the `File` parameter in `PPCallbacks::InclusionDirective()` from `const FileEntry *` to `Optional<FileEntryRef>`.
With the API change in place, this patch then removes some uses of the deprecated `FileEntry::getName()` (e.g. in `DependencyGraph.cpp` and `ModuleDependencyCollector.cpp`).
Reviewed By: dexonsmith, bnbarham
Differential Revision: https://reviews.llvm.org/D123574
- Inline SymbolID hashing to header
- Don't collect references for symbols without a SymbolID
- Store referenced symbols, rather than separately storing decls and
macros.
- Don't defer ref collection to end of translation unit
- Perform const_cast when updating reference counts (~0.5% saving)
- Introduce caching for getSymbolID in SymbolCollector. (~30% saving)
- Don't modify symbolslab if there's no definition location
- Don't lex the whole file to deduce spelled tokens, just lex the
relevant piece (~8%)
Overall this achieves ~38% reduction in time spent inside
SymbolCollector compared to baseline (on my machine :)).
I'd expect the last optimization to affect dynamic index a lot more, I
was testing with clangd-indexer on clangd subfolder of LLVM. As
clangd-indexer runs indexing of whole TU at once, we indeed see almost
every token from every source included in the TU (hence lexing full
files vs just lexing referenced tokens are almost the same), whereas
during dynamic indexing we mostly index main file symbols, but we would
touch the files defining/declaring those symbols, and lex complete files
for nothing, rather than just the token location.
The last optimization is also a functional change (added test),
previously we used raw tokens from syntax::tokenize, which didn't
canonicalize trigraphs/newlines in identifiers, wheres
Lexer::getSpelling canonicalizes them.
Differential Revision: https://reviews.llvm.org/D122894
This patch switches CanonicalInclude mappings to use `llvm::sys::fs::UniqueID` for a stable file representation because the `FileEntry::getName()` results turn out to be changing throughout the lifetime of a program (exposed in D120306). This patch makes it possible for D120306 to be re-landed and increases overall stability.
Reviewed By: sammccall
Differential Revision: https://reviews.llvm.org/D123031
It's deprecated; migrate to FileEntryRef::getName where it doesn't matter.
Also change one subtle case of implicit FileEntry::getName to be explicit.
After this patch, all the remaining FileEntry::getName calls are subtle
cases where we may be relying on exactly which filename variant is returned
(for indexing, IWYU directive handling, etc).
With a sufficiently large output buffer, the only failure is Z_MEM_ERROR.
Check it and call the noreturn report_bad_alloc_error if applicable.
resize_for_overwrite may call report_bad_alloc_error as well.
Now that there is no other error type, we can replace the return type with void
and simplify call sites.
Reviewed By: ikudrin
Differential Revision: https://reviews.llvm.org/D121512
Preparing for the cl::opt reset fix proposed on D115433 this
patch fixes the dexp tool to preserve its three command line
options (IndexLocation, ExecCommand, ProjectRoot) from reset
that is done before parsing query options.
Tags: #clang
Preparing for the cl::opt reset fix proposed on D115433 this
patch fixes the dexp tool to preserve its three command line
options (IndexLocation, ExecCommand, ProjectRoot) from reset
that is done before parsing query options.
Tags: #clang
This will allow moving the IncludeCleaner library essentials to Clang
and decoupling them from the majority of clangd.
The patch itself just moves the code, it doesn't change existing
functionality.
Reviewed By: sammccall
Differential Revision: https://reviews.llvm.org/D119130
With this change, clangd now computes framework-style includes
for framework headers at indexing time.
Differential Revision: https://reviews.llvm.org/D117056
Auto-generated patch based on clang-tidy readability-identifier-naming.
Only some manual cleanup for `extern "C"` declarations and a GTest change was required.
I'm not sure if this cleanup is actually very useful. It cleans up clang-tidy findings to the number of warnings from clang-tidy should be lower. Since it was easy to do and required only little cleanup I thought I'd upload it for discussion.
One pattern that keeps recurring: Test **matchers** are also supposed to start with a lowercase letter as per LLVM convention. However GTest naming convention for matchers start with upper case. I would propose to keep stay consistent with the GTest convention there. However that would imply a lot of `//NOLINT` throughout these files.
To re-product this patch run:
```
run-clang-tidy -checks="-*,readability-identifier-naming" -fix -format ./clang-tools-extra/clangd
```
To convert the macro names, I was using this script with some manual cleanup afterwards:
https://gist.github.com/ChristianKuehnel/a01cc4362b07c58281554ab46235a077
Differential Revision: https://reviews.llvm.org/D115634
This is a cleanup of all llvm-qualified-auto findings.
This patch was created by automatically applying the fixes from
clang-tidy.
Differential Revision: https://reviews.llvm.org/D113898
These errors are non-harmful and should be transient. They either
imply:
- compilation database returned stale results for TUs and it'll be fixed once
it's updated to match project state.
- a TUs dependencies has changed and some headers no longer exist. this should
be fixed with the next indexing cycle.
In either case the user will have some stale symbols in their index until clangd
restarts and the underlying issue is resolved. On the downside these logs are
confusing users when there's another issue.
Differential Revision: https://reviews.llvm.org/D117792
New values:
- Split Dynamic into Open/Preamble
- Add Background (previously was just Unknown)
- Soon: stdlib index
This requires extending to 16 bits, which fits within the padding of Symbol.
Unfortunately we're also *serializing* SymbolOrigin as a fixed 8 bits.
Stop serializing SymbolOrigin:
- conceptually, the source is whoever indexes or *deserializes* a symbol
- deserialization takes SymbolOrigin as a parameter and stamps it on each sym
- this is a breaking format change
Differential Revision: https://reviews.llvm.org/D115243
Main use of these is in the standard library, where they generally clutter up
the index.
Certain macros are also common, we don't touch indexing of macros in this patch.
Differential Revision: https://reviews.llvm.org/D115301
Kazu Hirata