Commit Graph

353 Commits

Author SHA1 Message Date
Sam McCall 4dc8365f80 [clangd] Remove support for pre-standard semanticHighlighting notification
This is obsoleted by the standard semanticTokens request family.
As well as the protocol details, this allows us to remove a bunch of plumbing
around pushing highlights to clients.

This should not land until the new protocol has feature parity, see D77702.

Differential Revision: https://reviews.llvm.org/D95576
2021-02-10 22:09:03 +01:00
Sam McCall 1eb7fd089e [clangd] Remove some obsolete options that are now always on
- always collect main-file refs when indexing
 - always build preambles asynchronously
 - always use dex for fast preamble index

Retire associated flags

Differential Revision: https://reviews.llvm.org/D95571
2021-02-01 17:24:03 +01:00
Quentin Chateau 3680cb99a7 [clangd] ignore parallelism level for quick tasks
This allows quick tasks without dependencies that
need to run fast to run ASAP. This is mostly useful
for code formatting.

----------------------------

This fixes something that's been annoying me:
- Open your IDE workspace and its 20 open files
- Clangd spends 5 minutes parsing it all
- In the meantime you start to work
- Save a file, trigger format-on-save, which hangs because clangd is busy
- You're stuck waiting until clangd is done parsing your files before the formatting and save takes place

Reviewed By: sammccall

Differential Revision: https://reviews.llvm.org/D94875
2021-01-25 23:03:43 +01:00
Sam McCall 60cd75a098 [clangd] Inject context provider rather than config into ClangdServer. NFC
This is a step towards allowing CDB behavior to being configurable.

Previously ClangdServer itself created the configs and installed them into
contexts. This was natural as it knows how to deal with resulting diagnostics.

However this prevents config being used in CDB, which must be created before
ClangdServer. So we extract the context provider (config loader) as a separate
object, which publishes diagnostics to a ClangdServer::Callbacks itself.

Now initialization looks like:
 - First create the config::Provider
 - Then create the ClangdLSPServer, passing config provider
 - Next, create the context provider, passing config provider + diagnostic callbacks
 - now create the CDB, passing context provider
 - finally create ClangdServer, passing CDB, context provider, and diagnostic callbacks

Differential Revision: https://reviews.llvm.org/D95087
2021-01-22 14:34:30 +01:00
Sam McCall 2ab5fd2c85 [clangd] Retire some flags for uncontroversial, stable features.
And mark a couple to be retired afther the next release branch.

Differential Revision: https://reviews.llvm.org/D94727
2021-01-20 11:47:12 +01:00
Sam McCall e6be5c7cd6 [clangd] Remove the recovery-ast options.
These force a couple of flags or that are now on by default.
So the flags don't currently do anything unless the compile command has
-fno-recovery-ast explicitly.

(For turning recovery *off* for debugging we can inject the flag with config)

This leaves the command-line flags around with no effect, I'm planning to add
a "retired flag" mechanism shortly in a separate patch.

Differential Revision: https://reviews.llvm.org/D94724
2021-01-20 11:23:57 +01:00
Sam McCall 536a1b0ea2 [clangd] Allow CDBs to have background work to block on.
In preparation for moving DirectoryBasedCompilationDatabase broadcasting off
the main thread.

Differential Revision: https://reviews.llvm.org/D94603
2021-01-20 11:11:01 +01:00
Utkarsh Saxena 275716d6db [clangd] Derive new signals in CC from ASTSignals.
This patch only introduces new signals but does not use their value
in scoring a CC candidate. Usage of these signals in CC ranking in both
heiristics and ML model will be introduced in later patches.

Differential Revision: https://reviews.llvm.org/D94473
2021-01-18 17:37:27 +01:00
Sam McCall 17fb21f875 [clangd] Remove another option that was effectively always true. NFC 2021-01-14 17:19:47 +01:00
Nathan James 3505d8dc07 [clangd][NFC] Use PathRef for getCorrespondingHeaderOrSource 2021-01-07 02:41:19 +00:00
Quentin Chateau 3fa2d37eb3 [clangd][NFC] Improve clangd status messages
clangd actions have various naming schemes, the most
common being PascalCase. This commit applies PascalCase
to all clangd actions, and fix the status rendering
in `renderTUAction` to look more consistent.

Reviewed By: sammccall

Differential Revision: https://reviews.llvm.org/D93546
2020-12-21 20:19:25 +01:00
Sam McCall fed9af29c2 [clangd] Publish config file errors over LSP
We don't act as a language server for these files (e.g. don't get open/close
notifications for them), but just blindly publish diagnostics for them.

This works reasonably well in coc.nvim and vscode: they show up in the
workspace diagnostic list and when you open the file.
The only update after the file is reparsed, not as you type which is a bit
janky, but seems a lot better than nothing.

Fixes https://github.com/clangd/clangd/issues/614

Differential Revision: https://reviews.llvm.org/D92704
2020-12-07 11:07:32 +01:00
Nathan James 73fdd99870
[clangd] Implement clang-tidy options from config
Added some new ClangTidyOptionsProvider like classes designed for clangd work flow.
These providers are designed to source the options on the worker thread but in a thread safe manner.
This is done through making the options getter take a pointer to the filesystem used by the worker thread which natuarally is from a ThreadsafeFS.
Internal caching in the providers is also guarded.

The providers don't inherit from `ClangTidyOptionsProvider` instead they share a base class which is able to create a provider for the `ClangTidyContext` using a specific FileSystem.
This approach means one provider can be used for multiple contexts even though `ClangTidyContext` owns its provider.

Depends on D90531

Reviewed By: sammccall

Differential Revision: https://reviews.llvm.org/D91029
2020-11-25 18:35:35 +00:00
Nathan Ridge dced150375 [clangd] Use WorkScheduler.run() in ClangdServer::resolveTypeHierarchy()
Differential Revision: https://reviews.llvm.org/D91941
2020-11-23 20:44:14 -05:00
Nathan Ridge 4cb976e014 [clangd] Call hierarchy (ClangdServer layer)
Differential Revision: https://reviews.llvm.org/D91123
2020-11-23 20:43:41 -05:00
Utkarsh Saxena b31486ad97 [clangd] textDocument/implementation (LSP layer)
Differential Revision: https://reviews.llvm.org/D91721
2020-11-23 13:50:44 +01:00
Sam McCall 8adc4d1ec7 [clangd] Add textDocument/ast extension method to dump the AST
This is a mass-market version of the "dump AST" tweak we have behind
-hidden-features.
I think in this friendlier form it'll be useful for people outside clang
developers, which would justify making it a real feature.
It could be useful as a step towards lightweight clang-AST tooling in clangd
itself (like matcher-based search).

Advantages over the tweak:
 - simplified information makes it more accessible, likely somewhat useful
   without learning too much clang internals
 - can be shown in a tree view
 - structured information gives some options for presentation (e.g.
   icon + two text colors + tooltip in vscode)
 - clickable nodes jump to the corresponding code
Disadvantages:
 - a bunch of code to handle different node types
 - likely missing some important info vs dump-ast due to brevity/oversight
 - may end up chasing/maintaining support for the long tail of nodes

Demo with VSCode support: https://imgur.com/a/6gKfyIV

Differential Revision: https://reviews.llvm.org/D89571
2020-11-20 01:13:28 +01:00
Nathan Ridge cb3c13fab6 [clangd] Propagate CollectMainFileRefs to BackgroundIndex
This appears to have been an omission in D83536.

Differential Revision: https://reviews.llvm.org/D89284
2020-10-13 09:20:18 -04:00
Kadir Cetinkaya 35871fde55
[clangd] Record memory usages after each notification
Depends on D88415

Differential Revision: https://reviews.llvm.org/D88417
2020-10-12 15:25:29 +02:00
Sam McCall 7530b254e9 [clangd] Make the tweak filter a parameter to enumerateTweaks. NFC
(Required for CodeActionContext.only)

Differential Revision: https://reviews.llvm.org/D88724
2020-10-09 14:11:19 +02:00
Haojian Wu 9c09e2055e [clangd] Add a NewName optional parameter to clangdServer::prepareRename.
If the NewName is provided, prepareRename would perform a name
validation.

The motivation is to allow our internal embeder implement the customized
"canRenameInto" functionality on top of prepareRename.

Differential Revision: https://reviews.llvm.org/D88881
2020-10-07 21:18:51 +02:00
Haojian Wu 8a3cbb1535 [clangd] Add basic keyword-name-validation in rename.
Differential Revision: https://reviews.llvm.org/D88875
2020-10-06 15:47:57 +02:00
Haojian Wu 0f0cbcc4b1 [clangd] Extend the rename API.
several changes:
- return a structure result in rename API;
- prepareRename now returns more information (main-file occurrences);
- remove the duplicated detecting-touch-identifier code in prepareRename (which is implemented in rename API);

Differential Revision: https://reviews.llvm.org/D88634
2020-10-02 16:03:44 +02:00
Sam McCall 17747d2ec8 [clangd] Remove Tweak::Intent, use CodeAction kind directly. NFC
Intent was a nice idea but it ends up being a bit awkward/heavyweight
without adding much.

In particular, it makes it hard to implement `CodeActionParams.only` properly
(there's an inheritance hierarchy for kinds).

Differential Revision: https://reviews.llvm.org/D88427
2020-10-02 11:14:23 +02:00
Kadir Cetinkaya 574dd60547
[clangd] Track tweaks that fail the apply stage
Differential Revision: https://reviews.llvm.org/D87501
2020-09-14 11:24:02 +02:00
Sam McCall 30667c967d [clangd] Add error() function for creating formatv-style llvm::Errors. NFC
Summary:
This is considerably terser than the makeStringError and friends, and
avoids verbosity cliffs that discourage adding log information.

It follows the syntax used in log/elog/vlog/dlog that have been successful.

The main caveats are:
 - it's strictly out-of-place in logger.h, though kind of fits thematically and
   in implementation
 - it claims the "error" identifier, which seems a bit too opinionated
   to put higher up in llvm

I've updated some users of StringError mostly at random - there are lots
more mechanical changes but I'd like to get this reviewed before making
them all.

Reviewers: kbobyrev, hokein

Subscribers: mgorny, ilya-biryukov, javed.absar, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D83419
2020-09-14 10:43:42 +02:00
Nathan Ridge 15673d748a [clangd] Index refs to main-file symbols as well
Summary: This will be needed to support call hierarchy

Reviewers: kadircet

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, usaxena95, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D83536
2020-08-18 00:30:07 -04:00
Kadir Cetinkaya 0464acd019
[clangd] Move clang-tidy check modifications into ClangdServer
Summary:
This enables sharing the logic between standalone clangd and embedders
of it. The new approach should be same performance-wise, as it is only called
once per addDocument call.

This patch also introduces a blacklisting code path for disabling crashy or
high-noise tests, until we figure out a way to make them work with clangd-setup.

The biggest difference is the way we make use of preambles, hence those checks
can't see directives coming from the preamble section of the file. The second
thing is the fact that code might-not be compiling while clangd is trying to
build an AST, hence some checks might choke on those incomplete ASTs.

Reviewers: sammccall

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, usaxena95, aaron.ballman, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D83224
2020-08-13 18:32:59 +02:00
Sam McCall b36e22d644 [clangd] Extract BackgroundIndex::Options struct. NFC
I've dropped the background context parameter, since we in practice just pass the
current context there, and we now have a different way to specify context too.
While here, clean up a couple of comments.

Reviewed By: kadircet

Differential Revision: https://reviews.llvm.org/D83157
2020-08-13 18:12:54 +02:00
Sam McCall 41d0edd54e [clangd] Express dumpAST in tests as a customAction() 2020-08-13 14:27:32 +02:00
Sam McCall 32626bcc0b [clangd] Add ClangdServer::customAction() extension point
This lets basic AST-consuming actions be defined outside clangdserver.
(it essentially exposes TUScheduler::runWithAST).

Two use cases for now:
 - replace ClangdServer::dumpAST, which doesn't really belong in the public
   interface (a followup patch will do this)
 - allow embedders to add/experiment with extra features
   (e.g. I know some who want to try crazy AST+ML code completion...)

Maybe in the future we should use this sort of mechanism to make ClangdServer
less of a monolith, but that's not in scope for now.
This would probably require a richer and more carefully-thought-out API.

Differential Revision: https://reviews.llvm.org/D85883
2020-08-13 12:26:32 +02:00
Sam McCall ff616f74c3 [clangd] Cache config files for 5 seconds, without revalidating with stat.
Summary:
This is motivated by:
 - code completion: nice to do no i/o on the request path
 - background index: deciding whether to enqueue each file would stat the config
   file thousands of times in quick succession.

Currently it's applied uniformly to all requests though.

This gives up on performing stat() outside the lock, all this achieves is
letting multiple threads stat concurrently (and thus finish without contention
for nonexistent files).
The ability to finish without IO (just mutex lock + integer check) should
outweigh this, and is less sensitive to platform IO characteristics.

Reviewers: kadircet

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, usaxena95, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D83755
2020-07-14 17:00:41 +02:00
Kirill Bobyrev 7a514c9bf8
[clangd] Implement textDocument/foldingRange
Summary:
This patch introduces basic textDocument/foldingRange support. It relies on
textDocument/documentSymbols to collect all symbols and uses takes ranges
to create folds.

The next steps for textDocument/foldingRange support would be:

* Implementing FoldingRangeClientCapabilities and respecting respect client
  preferences
* Specifying folding range kind
* Migrating from DocumentSymbol implementation to custom RecursiveASTVisitor flow that will allow more flexibility
* Supporting more folding range types: comments, PP conditional regions, includes and other code regions (e.g. public/private/protected sections of classes, control flow statement bodies)

Tested: (Neo)Vim (coc-clangd) and VSCode.

Related issue: https://github.com/clangd/clangd/issues/310

Reviewers: sammccall

Reviewed By: sammccall

Subscribers: nridge, ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D82436
2020-07-14 09:28:42 +02:00
Benjamin Kramer edba2864a7 [clangd] Fix stack-use-after-scope
Found by asan.
2020-07-05 17:09:08 +02:00
Sam McCall 15a60fe09f [clangd] Config: compute config in TUScheduler and BackgroundIndex
Summary:
ClangdServer owns the question of exactly which config to create, but
TUScheduler/BackgroundIndex control threads and so decide at which point
to inject it.

Reviewers: kadircet

Subscribers: ilya-biryukov, javed.absar, MaskRay, jkorous, arphaman, usaxena95, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D83095
2020-07-04 11:18:14 +02:00
Sam McCall ffa63dde8e [clangd] Run formatting operations asynchronously.
Summary:
They don't need ASTs or anything, so they should still run immediately.

These were sync for historical reasons (they predate clangd having a pervasive
threading model). This worked ok as they were "cheap".
Aside for consistency, there are a couple of reasons to make them async:
 - they do IO (finding .clang-format) so aren't trivially cheap
 - having TUScheduler involved in running these tasks means we can use it as
   an injection point for configuration.
   (TUScheduler::run will need to learn about which file is being operated on,
   but that's an easy change).
 - adding the config system adds more potential IO, too

Reviewers: kbobyrev

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D82642
2020-07-01 00:52:08 +02:00
Kadir Cetinkaya 8d654df5b9
[clangd] Rename FSProvider to TFS in case of ThreadsafeFS
Summary: Depends on D81998

Reviewers: sammccall

Subscribers: ilya-biryukov, javed.absar, MaskRay, jkorous, arphaman, usaxena95, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D82024
2020-06-19 12:12:03 +02:00
Kadir Cetinkaya 0628705efa
[clangd][NFC] Rename FSProvider and getFileSystem
Summary:
Clangd uses FSProvider to get threadsafe views into file systems. This
patch changes naming to make that more explicit.

Depends on D81920

Reviewers: sammccall

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, usaxena95, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D81998
2020-06-19 12:12:03 +02:00
Kadir Cetinkaya 2dc2e47e3c
[clangd] Change FSProvider::getFileSystem to take CurrentWorkingDirectory
Summary:
We've faced a couple of problems when the returned FS didn't have the
proper working directory. New signature makes the API safer against such
problems.

Reviewers: sammccall

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, usaxena95, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D81920
2020-06-19 12:12:02 +02:00
Kadir Cetinkaya f693ce4aa9
[clangd] Change ParseInputs to store FSProvider rather than VFS
Summary: This ensures ParseInputs provides a read-only access to FS.

Reviewers: sammccall

Subscribers: ilya-biryukov, javed.absar, MaskRay, jkorous, arphaman, usaxena95, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D81173
2020-06-08 13:23:55 +02:00
Haojian Wu a7534dc819 [clangd] Populate the parse options to CodeCompletion/SignatureHelp.
Summary: A followup of D79938.

Reviewers: sammccall

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D81066
2020-06-05 10:32:07 +02:00
Kadir Cetinkaya 1f4ba66ecc
[clangd] Run PreambleThread in async mode behind a flag
Summary: Depends on D80198.

This patch implies ASTs might be built with stale preambles without
blocking for a fresh one. It also drops any guarantees on every preamble
version being built. In case of multiple preamble build requests, in
addition to being debounced.

Any preamble requested with a WantDiags::Yes will always be built, this
is ensured by blocking enqueueing of any subsequent reqest.

AST worker will still block for initial preamble to reduce duplicate
work.

Subscribers: ilya-biryukov, javed.absar, MaskRay, jkorous, arphaman, jfb, usaxena95, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D80293
2020-05-29 13:20:46 +02:00
Haojian Wu 0320ce8916 [clangd] Add a flag to preserve type for recovery expression.
Reviewers: sammccall

Reviewed By: sammccall

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D79938
2020-05-19 15:21:57 +02:00
Kadir Cetinkaya e64f99c51a
[clangd] Metric tracking through Tracer
Summary: Introduces an endpoint to Tracer for tracking metrics on
internal events.

Reviewers: sammccall

Subscribers: ilya-biryukov, javed.absar, MaskRay, jkorous, arphaman, usaxena95, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D78429
2020-05-03 10:50:32 +02:00
Sam McCall fa1f4cf843 [clangd] Rename FormattedString -> Markup, move to support. NFC 2020-05-02 14:53:47 +02:00
Sam McCall ad97ccf6b2 [clangd] Move non-clang base pieces into separate support/ lib. NFCI
Summary:
This enforces layering, reduces a sprawling clangd/ directory, and makes life
easier for embedders.

Reviewers: kbobyrev

Subscribers: mgorny, ilya-biryukov, javed.absar, MaskRay, jkorous, arphaman, jfb, kadircet, usaxena95, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D79014
2020-04-29 15:57:12 +02:00
Kadir Cetinkaya 2214b9076f
[clangd] Make signatureHelp work with stale preambles
Summary:
This is achieved by calculating newly added includes and implicitly
parsing them as if they were part of the main file.

This also gets rid of the need for consistent preamble reads.

Reviewers: sammccall

Subscribers: ilya-biryukov, javed.absar, MaskRay, jkorous, mgrang, arphaman, jfb, usaxena95, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D77392
2020-04-21 10:27:26 +02:00
Sam McCall 808c2855e1 [clangd] Add tests that no-op changes are cheap
Summary:
We want to be sure they don't cause AST rebuilds or evict items from the cache.
D77847 is going to start sending spurious no-op changes (in case the preamble
was invalidated), this is cheap enough but we shouldn't regress that in future.

Reviewers: kadircet

Subscribers: ilya-biryukov, javed.absar, MaskRay, jkorous, arphaman, jfb, usaxena95, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D78048
2020-04-14 16:15:23 +02:00
Sam McCall 31db1e0bd1 [clangd] Send the correct error code when cancelling requests.
Summary:
I couldn't quite bring myself to make Cancellation depend on LSP ErrorCode.
Magic numbers instead...

Reviewers: kadircet

Subscribers: ilya-biryukov, javed.absar, MaskRay, jkorous, arphaman, jfb, usaxena95, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D77947
2020-04-13 19:42:38 +02:00
Kadir Cetinkaya 43aa04eb7a
[clangd] Run semaCodeComplete only with a preamble
Summary:
It is used by code completion and signature help. Code completion
already uses a special no-compile mode for missing preambles, so this change is
a no-op for that.

As for signature help, it already blocks for a preamble and missing it implies
clang has failed to parse the preamble and retrying it in signature help likely
will fail again. And even if it doesn't, request latency will be too high to be
useful as parsing preambles is expensive.

Reviewers: sammccall

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, usaxena95, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D77204
2020-04-01 13:02:47 +02:00
Haojian Wu 72439b6b95 [clangd] Add a flag to turn on recovery-expr.
Reviewers: sammccall

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D77142
2020-04-01 09:03:56 +02:00
Sam McCall 71177ac168 [clangd] Support new semanticTokens request from LSP 3.16.
Summary:
This is a simpler request/response protocol.

Reference: https://github.com/microsoft/vscode-languageserver-node/blob/master/protocol/src/protocol.semanticTokens.proposed.ts

No attempt to support incremental formatting (yet).

Reviewers: hokein

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D76663
2020-03-31 15:14:35 +02:00
Sam McCall 8f237f9b09 [clangd] Support multiple cursors in selectionRange.
Summary:
One change: because there's no way to signal failure individually for
each cursor, we now "succeed" with an empty range with no parent if a
cursor doesn't point at anything.

Reviewers: usaxena95

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D76741
2020-03-25 17:59:09 +01:00
Sam McCall edf6a19adf [clangd] Rename theia-derived semantic highlighting protocol. NFC
This feature is being added to LSP with a different (request-response)
protocol in 3.16, so this should avoid some confusion.
2020-03-24 00:39:47 +01:00
Sam McCall 2cd33e6fe6 [clangd] Track document versions, include them with diags, enhance logs
Summary:
This ties to an LSP feature (diagnostic versioning) but really a lot
of the value is in being able to log what's happening with file versions
and queues more descriptively and clearly.

As such it's fairly invasive, for a logging patch :-\

Key decisions:
 - at the LSP layer, we don't reqire the client to provide versions (LSP
   makes it mandatory but we never enforced it). If not provided,
   versions start at 0 and increment. DraftStore handles this.
 - don't propagate magically using contexts, but rather manually:
   addDocument -> ParseInputs -> (ParsedAST, Preamble, various callbacks)
   Context-propagation would hide the versions from ClangdServer, which
   would make producing good log messages hard
 - within ClangdServer, treat versions as opaque and unordered.
   std::string is a convenient type for this, and allows richer versions
   for embedders. They're "mandatory" but "null" is a reasonable default.

Subscribers: ilya-biryukov, javed.absar, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D75582
2020-03-05 01:22:32 +01:00
Sam McCall c627b120eb [clangd] Cancel certain operations if the file changes before we start.
Summary:
Otherwise they can force us to build lots of snapshots that we don't need.
Particularly, try to do this for operations that are frequently
generated by editors without explicit user interaction, and where
editing the file makes the result less useful. (Code action
enumeration is a good example).

https://github.com/clangd/clangd/issues/298

This doesn't return the "right" LSP error code (ContentModified) to the client,
we need to teach the cancellation API to distinguish between different causes.

Reviewers: kadircet

Subscribers: ilya-biryukov, javed.absar, MaskRay, jkorous, arphaman, jfb, usaxena95, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D75602
2020-03-05 00:10:07 +01:00
Sam McCall c0b27c4891 [clangd] Remove unused getDocument() API 2020-03-03 14:53:59 +01:00
Haojian Wu c443b610bf [clangd] Remove the deprecated clangdServer::rename API, NFC.
There is no actual user of it now.
2020-03-02 09:48:25 +01:00
Haojian Wu 34d0e1bd6d [clangd] Expose the rename LimitFiles option to the C++ API, NFC.
Reviewers: sammccall

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D74834
2020-02-26 09:33:58 +01:00
Sam McCall e9997cfb4d [clangd] Try to fix buildbots - copy elision not happening here? 2020-02-23 21:12:26 +01:00
Sam McCall be6d07c920 [clangd] Reapply b60896fad9 Fall back to selecting token-before-cursor if token-after-cursor fails.
This reverts commit b4b9706d5d.
Now avoiding expected<vector<selection>> in favor of expected<vector<unique_ptr<selection>>>
2020-02-23 20:17:30 +01:00
Sam McCall b4b9706d5d Revert "[clangd] Reapply b60896fad9 Fall back to selecting token-before-cursor if token-after-cursor fails."
This reverts commit a2ce807eb7.

Buildbot failures on GCC due to SelectionTree not being copyable, and
instantiating vector<Selection> in the tweak-handling in ClangdServer.
2020-02-23 16:34:49 +01:00
Sam McCall a2ce807eb7 [clangd] Reapply b60896fad9 Fall back to selecting token-before-cursor if token-after-cursor fails.
This reverts commit 6af1ad20d6.
2020-02-23 16:17:46 +01:00
David Goldman 6ff0228c6d [clang] Add `forceReload` clangd extension to 'textDocument/didChange'
Summary:
- This option forces a preamble rebuild to handle the odd case
  of a missing header file being added

Reviewers: sammccall

Subscribers: ilya-biryukov, javed.absar, MaskRay, jkorous, arphaman, jfb, kadircet, usaxena95, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D73916
2020-02-10 14:02:02 -05:00
Sam McCall 2f4c4d0a78 Reland "[clangd] Mechanism to make update debounce responsive to rebuild speed."
This reverts commit ed98994f64.
Removed the accidental double-mutex-unlock.
2020-02-04 17:17:53 +01:00
Sam McCall ed98994f64 Revert "[clangd] Mechanism to make update debounce responsive to rebuild speed."
This reverts commit 92570718a8.
Breaking tests: http://45.33.8.238/linux/9296/step_9.txt
2020-02-04 15:34:10 +01:00
Sam McCall 92570718a8 [clangd] Mechanism to make update debounce responsive to rebuild speed.
Summary:
Currently we delay AST rebuilds by 500ms after each edit, to wait for
further edits. This is a win if a rebuild takes 5s, and a loss if it
takes 50ms.

This patch sets debouncepolicy = clamp(min, ratio * rebuild_time, max).
However it sets min = max = 500ms so there's no policy change or actual
customizability - will do that in a separate patch.

See https://github.com/clangd/clangd/issues/275

Reviewers: hokein

Subscribers: ilya-biryukov, javed.absar, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D73873
2020-02-04 12:28:20 +01:00
Sam McCall b79cb54712 [clangd] Refactor TUScheduler options into a struct. NFC 2020-02-03 11:20:48 +01:00
Haojian Wu e864f93766 [clangd] Replace raw lexer code with token buffer in prepare rename.
Summary:
there is a slight behavior change in this patch:

- before: `in^t a;`, returns our internal error message (no symbol at given location)
- after: `in^t a, returns null, and client displays their message (e.g.
   e.g. "the element can't be renamed" in vscode).

both are sensible according LSP, and we'd save one `rename` call in the later case.

Reviewers: sammccall

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D73610
2020-01-29 12:57:18 +01:00
Benjamin Kramer adcd026838 Make llvm::StringRef to std::string conversions explicit.
This is how it should've been and brings it more in line with
std::string_view. There should be no functional change here.

This is mostly mechanical from a custom clang-tidy check, with a lot of
manual fixups. It uncovers a lot of minor inefficiencies.

This doesn't actually modify StringRef yet, I'll do that in a follow-up.
2020-01-28 23:25:25 +01:00
Sam McCall 6ef1ccecf7 [clangd] Rename DiagnosticsConsumer -> ClangdServer::Callbacks, and make it optional
Summary:
This reflects its current function better and avoids confusion with clang::DiagnosticConsumer.

The old name/constructor is left around temporarily for compatibility.
(Metagame: merging with out-of-tree changes is harder than usual this month)

Reviewers: hokein

Subscribers: ilya-biryukov, javed.absar, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D73346
2020-01-24 15:59:32 +01:00
Sam McCall 7d20e80225 [clangd] Show background index status using LSP 3.15 work-done progress notifications
Summary:
It simply shows the completed/total items on the background queue, e.g.
 indexing: 233/1000
The denominator is reset to zero every time the queue goes idle.

The protocol is fairly complicated here (requires creating a remote "progress"
resource before sending updates). We implement the full protocol, but I've added
an extension allowing it to be skipped to reduce the burden on clients - in
particular the lit test takes this shortcut.

The addition of background index progress to DiagnosticConsumer seems ridiculous
at first glance, but I believe that interface is trending in the direction of
"ClangdServer callbacks" anyway. It's due for a rename, but otherwise actually
fits.

Reviewers: kadircet, usaxena95

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, jfb, cfe-commits, llvm-commits

Tags: #clang, #llvm

Differential Revision: https://reviews.llvm.org/D73218
2020-01-24 12:21:08 +01:00
Sam McCall 8d7ecc1629 Revert "Revert "[clangd] Implement "textDocument/documentLink" protocol support""
This reverts commit 079ef783dd.

The revert describes a test failure without details, after offline
discussion this in in a private/unsupported build system and doesn't
seem to reflect a real upstream bug.
2020-01-02 16:36:21 +01:00
Sam McCall 6af1ad20d6 Revert "[clangd] Reapply b60896fad9 Fall back to selecting token-before-cursor if token-after-cursor fails."
This reverts commit a0ff8cd631.
Buildbot failures I can't chase further tonight.
2019-12-16 19:07:49 +01:00
Sam McCall a0ff8cd631 [clangd] Reapply b60896fad9 Fall back to selecting token-before-cursor if token-after-cursor fails.
This reverts commit 8f876d5105.
2019-12-16 15:58:51 +01:00
Sam McCall 8f876d5105 Revert "[clangd] Reapply b60896fad9 Fall back to selecting token-before-cursor if token-after-cursor fails."
This reverts commit 2500a8d5d8.
2019-12-16 15:57:43 +01:00
Sam McCall 2500a8d5d8 [clangd] Reapply b60896fad9 Fall back to selecting token-before-cursor if token-after-cursor fails.
This reverts commit f0604e73a4
The issue with movability of Tweak::Selection was addressed in 7dc388bd95
2019-12-16 15:55:16 +01:00
Dmitri Gribenko 079ef783dd Revert "[clangd] Implement "textDocument/documentLink" protocol support"
This reverts commit d6417f5584. The tests
depend on builtin headers, which is not intentionally supported in
clangd tests; these tests are broken in some build environments.
2019-12-16 15:21:51 +01:00
Kirill Bobyrev 3b9715cb21 [NFC] Fix typos in Clangd and Clang
Reviewed by: Jim

Differential Revision: https://reviews.llvm.org/D71455
2019-12-16 10:54:40 +01:00
Nico Weber f0604e73a4 Revert "[clangd] Fall back to selecting token-before-cursor if token-after-cursor fails."
This reverts commit b60896fad9.
Breaks building with gcc:

/usr/include/c++/7/bits/stl_construct.h:75:7: error: use of deleted function ‘clang::clangd::Tweak::Selection::Selection(const clang::clangd::Tweak::Selection&)’
     { ::new(static_cast<void*>(__p)) _T1(std::forward<_Args>(__args)...); }
       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /home/buildslave/buildslave/clang-cmake-armv7-selfhost-neon/llvm/clang-tools-extra/clangd/ClangdServer.h:28:0,
                 from /home/buildslave/buildslave/clang-cmake-armv7-selfhost-neon/llvm/clang-tools-extra/clangd/ClangdServer.cpp:9:
/home/buildslave/buildslave/clang-cmake-armv7-selfhost-neon/llvm/clang-tools-extra/clangd/refactor/Tweak.h:49:10: note: ‘clang::clangd::Tweak::Selection::Selection(const clang::clangd::Tweak::Selection&)’ is implicitly deleted because the default definition would be ill-formed:
   struct Selection {
          ^~~~~~~~~
/home/buildslave/buildslave/clang-cmake-armv7-selfhost-neon/llvm/clang-tools-extra/clangd/refactor/Tweak.h:49:10: error: use of deleted function ‘clang::clangd::SelectionTree::SelectionTree(const clang::clangd::SelectionTree&)’
In file included from /home/buildslave/buildslave/clang-cmake-armv7-selfhost-neon/llvm/clang-tools-extra/clangd/refactor/Tweak.h:25:0,
                 from /home/buildslave/buildslave/clang-cmake-armv7-selfhost-neon/llvm/clang-tools-extra/clangd/ClangdServer.h:28,
                 from /home/buildslave/buildslave/clang-cmake-armv7-selfhost-neon/llvm/clang-tools-extra/clangd/ClangdServer.cpp:9:
/home/buildslave/buildslave/clang-cmake-armv7-selfhost-neon/llvm/clang-tools-extra/clangd/Selection.h:96:3: note: declared here
   SelectionTree(const SelectionTree &) = delete;
   ^~~~~~~~~~~~~

e.g. here:
http://lab.llvm.org:8011/builders/clang-cmake-armv7-selfhost-neon/builds/2714
http://lab.llvm.org:8011/builders/clang-ppc64be-linux/builds/41866
2019-12-14 08:13:29 -05:00
Sam McCall b60896fad9 [clangd] Fall back to selecting token-before-cursor if token-after-cursor fails.
Summary:
The problem:

LSP specifies that Positions are between characters. Therefore when a position
(or an empty range) is used to target elements of the source code, there is an
ambiguity - should we look left or right of the cursor?

Until now, SelectionTree resolved this to the right except in trivial cases
(where there's whitespace, semicolon, or eof on the right).
This meant that it's unable to e.g. out-line `int foo^()` today.

Complicating this, LSP notwithstanding the cursor is *on* a character in many
editors (mostly terminal-based). In these cases there's no ambiguity - we must
"look right" - but there's also no way to tell in LSP.

(Several features currently resolve this by using getBeginningOfIdentifier,
which tries to rewind and supports end-of-identifier. But this relies on
raw lexing and is limited and buggy).

Precedent: well - most other languages aren't so full of densely packed symbols
that we might want to target. Bias-towards-identifier works well enough.
MS C++ for vscode seems to mostly use bias-toward-identifier too.
The problem with this solution is it doesn't provide any way to target some
things such as the constructor call in Foo^(bar());

Presented solution:

When an ambiguous selection is found, we generate *both* possible selection
trees. We try to run the feature on the rightward tree first, and then on the
leftward tree if it fails.

This is basically do-what-I-mean, the main downside is the need to do this on
a feature-by-feature basis (because each feature knows what "fail" means).
The most complicated instance of this is Tweaks, where the preferred selection
may vary tweak-by-tweak.

Wrinkles:

While production behavior is pretty consistent, this introduces some
inconsistency in testing, depending whether the interface we're testing is
inside or outside the "retry" wrapper.

In particular, for many features like Hover, the unit tests will show production
behavior, while for Tweaks the harness would have to run the loop itself if
we want this.

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D71345
2019-12-13 16:57:03 +01:00
Michael Forster d6417f5584 [clangd] Implement "textDocument/documentLink" protocol support
Summary:
This adds an implementation for the "textDocument/documentLink" LSP request.

It returns links for all `#include` directives to the resolved target files.

Fixes https://github.com/clangd/clangd/issues/217.

Reviewers: sammccall

Reviewed By: sammccall

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D70872
2019-12-12 14:55:20 +01:00
Alex Lorenz c0ee0224c4 [clangd] NFC, add getLangOpts helper to ParsedAST
The addition of the helper is split out from https://reviews.llvm.org/D69543
as suggested by Kadir. I also updated the existing uses to use the new API.
2019-12-04 16:55:25 -08:00
Haojian Wu 852bafae2b [clangd] Implement cross-file rename.
Summary:
This is the initial version. The cross-file rename is purely based on
the index.

It is hidden under a command-line flag, and only available for a small set
of symbols.

Reviewers: ilya-biryukov, sammccall

Subscribers: merge_guards_bot, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D69263
2019-11-26 10:04:31 +01:00
Haojian Wu 5181adab61 [clangd] Expose the xref's incomplete flag to clangdServer API.
Summary: so that clangd C++ API users (via ClangdServer) can access it.

Reviewers: sammccall

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D70380
2019-11-18 13:27:19 +01:00
Kadir Cetinkaya f4d7caf6df
[clangd] Reland Store Index in Tweak::Selection
Summary:
Incoming define out-of-line tweak requires access to index.

This patch simply propogates the index in ClangdServer to Tweak::Selection while
passing the AST. Also updates TweakTest to accommodate this change.

Reviewers: ilya-biryukov

Subscribers: MaskRay, jkorous, arphaman, usaxena95, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D69165
2019-10-28 07:55:58 +01:00
Kadir Cetinkaya 8e567b0730
[clangd] Revert define-inline action changes to un-break windows build-bots 2019-10-25 18:40:01 +02:00
Kadir Cetinkaya 8aa84ad37d
[clangd] Store Index in Tweak::Selection
Summary:
Incoming define out-of-line tweak requires access to index.

This patch simply propogates the index in ClangdServer to Tweak::Selection while
passing the AST. Also updates TweakTest to accommodate this change.

Reviewers: ilya-biryukov

Subscribers: MaskRay, jkorous, arphaman, usaxena95, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D69165
2019-10-25 12:15:20 +02:00
Haojian Wu 77c97002dc [clangd] Catch an unchecked "Expected<T>" in HeaderSourceSwitch.
Summary: Also fixes a potential user-after-scope issue of "Path".

Reviewers: kadircet

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, usaxena95, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D68564

llvm-svn: 373897
2019-10-07 11:37:25 +00:00
Haojian Wu d6d5eddcdb [clangd] Use the index-based API to do the header-source switch.
Summary:
If the file heuristic fails, we try to use the index&AST to do the
header/source inference.

Reviewers: kadircet

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, usaxena95, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D68211

llvm-svn: 373320
2019-10-01 10:21:15 +00:00
Haojian Wu e259a5fd0d [clangd] Move the existing heder-source-switch implemenation out of clangdServer.
Summary: This is a NFC change.

Reviewers: kadircet

Reviewed By: kadircet

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, usaxena95, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D67960

llvm-svn: 372841
2019-09-25 10:30:22 +00:00
Utkarsh Saxena 79d19bdf81 Add SemanticRanges to Clangd server.
Summary:
Adds Semantic Ranges capabilities to Clangd server.
Also adds tests for running it via clangd server.

This differs from the LSP spec as the spec needs this to be evaluated on multiple 'pos' and the expected output is an list of list of semantic ranges.
This is majorly for multi cursor and assuming this is a rare thing, we don't want to optimize make things complicated just for this.
This should be done in the LSP level by queueing one request per 'pos' in the input.

LSP Spec:
https://github.com/microsoft/language-server-protocol/blob/dbaeumer/3.15/specification.md#textDocument_selectionRange

Reviewers: hokein

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D67650

llvm-svn: 372102
2019-09-17 10:28:05 +00:00
Kadir Cetinkaya 5b270932cc [clangd] Support multifile edits as output of Tweaks
Summary:
First patch for propogating multifile changes from tweak outputs to LSP
WorkspaceEdits.

Uses SM to convert tooling::Replacements to TextEdits.
Errors out if there are any inconsistencies between the draft version and the
version generated the edits.

Reviewers: sammccall

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D66637

llvm-svn: 371392
2019-09-09 12:28:44 +00:00
Sam McCall 915f9785e0 [clangd] Rename ClangdUnit.h -> ParsedAST.h. NFC
This much better reflects what is (now) in this header.
Maybe a rename to ParsedTU would be an improvement, but that's a much
more invasive change and life is too short.

ClangdUnit is dead, long live ClangdUnitTests!

llvm-svn: 370862
2019-09-04 09:46:06 +00:00
Sam McCall cf3a585fff [clangd] Split Preamble.h out of ClangdUnit.h. NFC
Summary:
Add comment describing use of preamble in clangd.
Remove deps on ClangdUnit.h where possible.

Subscribers: mgorny, ilya-biryukov, javed.absar, MaskRay, jkorous, mgrang, arphaman, kadircet, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D67117

llvm-svn: 370843
2019-09-04 07:35:00 +00:00
Sam McCall 19cefc2878 [clangd] Move getBeginningOfIdentifier from ClangdUnit to SourceCode. Drop dependencies on ClangdUnit from some headers. NFC
llvm-svn: 370768
2019-09-03 15:34:47 +00:00
Ilya Biryukov d73ac96d80 [clangd] Surface errors from command-line parsing
Summary:
Those errors are exposed at the first character of a file,
for a lack of a better place.

Previously, all errors were stored inside the AST and report
accordingly. However, errors in command-line argument parsing could
result in failure to produce the AST, so we need an alternative ways to
report those errors.

We take the following approach in this patch:
  - buildCompilerInvocation() now requires an explicit DiagnosticConsumer.
  - TUScheduler and TestTU now collect the diagnostics produced when
    parsing command line arguments.
    If pasing of the AST failed, diagnostics are reported via a new
    ParsingCallbacks::onFailedAST method.
    If parsing of the AST succeeded, any errors produced during
    command-line parsing are stored alongside the AST inside the
    ParsedAST instance and reported as previously by calling the
    ParsingCallbacks::onMainAST method;
  - The client code that uses ClangdServer's DiagnosticConsumer
    does not need to change, it will receive new diagnostics in the
    onDiagnosticsReady() callback

Errors produced when parsing command-line arguments are collected using
the same StoreDiags class that is used to collect all other errors. They
are recognized by their location being invalid. IIUC, the location is
invalid as there is no source manager at this point, it is created at a
later stage.

Although technically we might also get diagnostics that mention the
command-line arguments FileID with after the source manager was created
(and they have valid source locations), we choose to not handle those
and they are dropped as not coming from the main file. AFAICT, those
diagnostics should always be notes, therefore it's safe to drop them
without loosing too much information.

Reviewers: kadircet

Reviewed By: kadircet

Subscribers: nridge, javed.absar, MaskRay, jkorous, arphaman, cfe-commits, gribozavr

Tags: #clang

Differential Revision: https://reviews.llvm.org/D66759

llvm-svn: 370177
2019-08-28 09:24:55 +00:00
Haojian Wu 0a6000f2cb [clangd] Send highlighting diff beyond the end of the file.
Summary: This would make the client life (tracking the changes) easier.

Reviewers: jvikstrom

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D66541

llvm-svn: 369884
2019-08-26 08:38:45 +00:00
Benjamin Kramer 15ee5ba6e7 [clangd] Fold string copy into lambda capture. NFC.
llvm-svn: 369679
2019-08-22 18:09:46 +00:00
Benjamin Kramer 9880b5dd21 [clangd] Don't use Bind() where C++14 move capture works
llvm-svn: 369005
2019-08-15 14:16:06 +00:00