The function was only counting lines that included tokens and not empty lines,
but MaxLines (mainly initiated to the line where the code-completion point resides)
is a count of overall lines (even empty ones).
llvm-svn: 139085
Preprocessor, eliminating the constructor that was used by ASTUnit
(which didn't provide an ASTContext or Prepreprocessor). Ensuring that
both objects are non-NULL will simplify module loading (but none of
that is done yet).
llvm-svn: 138986
builtin types (When requested). This is another step toward making
ASTUnit build the ASTContext as needed when loading an AST file,
rather than doing so after the fact. No actual functionality change (yet).
llvm-svn: 138985
and language-specific initialization. Use this to allow ASTUnit to
create a preprocessor object *before* loading the AST file. No actual
functionality change.
llvm-svn: 138983
LangOptions, rather than making distinct copies of
LangOptions. Granted, LangOptions doesn't actually get modified, but
this will eventually make it easier to construct ASTContext and
Preprocessor before we know all of the LangOptions.
llvm-svn: 138959
include guards don't show up as macro definitions in every translation
unit that imports a module. Macro definitions can, however, be
exported with the intentionally-ugly #__export_macro__
directive. Implement this feature by not even bothering to serialize
non-exported macros to a module, because clients of that module need
not (should not) know that these macros even exist.
llvm-svn: 138943
, such as list of forward @class decls, in a DeclGroup
node. Deal with its consequence throught clang. This
is in preparation for more Sema work ahead. // rdar://8843851.
Feel free to reverse if it breaks something important
and I am unavailable.
llvm-svn: 138709
loads the named module. The syntax itself is intentionally hideous and
will be replaced at some later point with something more
palatable. For now, we're focusing on the semantics:
- Module imports are handled first by the preprocessor (to get macro
definitions) and then the same tokens are also handled by the parser
(to get declarations). If both happen (as in normal compilation),
the second one is redundant, because we currently have no way to
hide macros or declarations when loading a module. Chris gets credit
for this mad-but-workable scheme.
- The Preprocessor now holds on to a reference to a module loader,
which is responsible for loading named modules. CompilerInstance is
the only important module loader: it now knows how to create and
wire up an AST reader on demand to actually perform the module load.
- We search for modules in the include path, using the module name
with the suffix ".pcm" (precompiled module) for the file name. This
is a temporary hack; we hope to improve the situation in the
future.
llvm-svn: 138679
from the given source. -emit-module behaves similarly to -emit-pch,
except that Sema is somewhat more strict about the contents of
-emit-module. In the future, there are likely to be more interesting
differences.
llvm-svn: 138595
all AST files have a normal METADATA record that has the same form
regardless of whether we refer to a chained PCH or any other kind of
AST file.
Introduce the IMPORTS record, which describes all of the AST files
that are imported by this AST file, and how (as a module, a PCH file,
etc.). Currently, we emit at most one entry to this record, to support
chained PCH.
llvm-svn: 137869
already-defined and forward-declared results. Already-defined results
are fine because they could be the start of a category. Fixes
<rdar://problem/9811691>.
llvm-svn: 136559
For PCH files, have only one open/close for temporary + rename to be safe from race conditions.
For all other output files open/close the output file directly.
Depends on llvm r136310. rdar://9082880 & http://llvm.org/PR9374.
llvm-svn: 136315
entities generated directly by the preprocessor from those loaded from
the external source (e.g., the ASTReader). By separating these two
sets of entities into different vectors, we allow both to grow
independently, and eliminate the need for preallocating all of the
loaded preprocessing entities. This is similar to the way the recent
SourceManager refactoring treats FileIDs and the source location
address space.
As part of this, switch over to building a continuous range map to
track preprocessing entities.
llvm-svn: 135646
source locations from source locations loaded from an AST/PCH file.
Previously, loading an AST/PCH file involved carefully pre-allocating
space at the beginning of the source manager for the source locations
and FileIDs that correspond to the prefix, and then appending the
source locations/FileIDs used for parsing the remaining translation
unit. This design forced us into loading PCH files early, as a prefix,
whic has become a rather significant limitation.
This patch splits the SourceManager space into two parts: for source
location "addresses", the lower values (growing upward) are used to
describe parsed code, while upper values (growing downward) are used
for source locations loaded from AST/PCH files. Similarly, positive
FileIDs are used to describe parsed code while negative FileIDs are
used to file/macro locations loaded from AST/PCH files. As a result,
we can load PCH/AST files even during parsing, making various
improvemnts in the future possible, e.g., teaching #include <foo.h> to
look for and load <foo.h.gch> if it happens to be already available.
This patch was originally written by Sebastian Redl, then brought
forward to the modern age by Jonathan Turner, and finally
polished/finished by me to be committed.
llvm-svn: 135484
This is switches all the interfaces points (and most of the commenst
/ local variables I saw on my way through) regarding the
NestedMacroInstantiations bit.
The libclang enums corresponding to this state were renamed, but
a legacy enum was added with the old name, and the same value to keep
existing clients working. I've added a documentation blurb for it, but
let me know if there is a canonical way to document legacy elemenst of
the libclang interface.
No functionality changed here, even in tests.
llvm-svn: 135141
clang_codeCompleteGetContexts(), that provides the client with
information about the context in which code completion has occurred
and what kinds of entities make sense as completions at that
point. Patch by Connor Wakamo!
llvm-svn: 134615
CompilerInvocation on the stack, because other objects (e.g., the
CompilerInstance) maintain an intrusive reference-counted pointer to
the CompilerInvocation. This doesn't matter in the normal case,
because we take back the CompilerInvocation. However, during crash
recovery, this leads to us trying to free an object on the stack, and
hilarity ensues. Fixes <rdar://problem/9652540>.
llvm-svn: 134245
CXTranslationUnit_NestedMacroInstantiations, which indicates whether
we want to see "nested" macro instantiations (e.g., those that occur
inside other macro instantiations) within the detailed preprocessing
record. Many clients (e.g., those that only care about visible tokens)
don't care about this information, and in code that uses preprocessor
metaprogramming, this information can have a very high cost.
Addresses <rdar://problem/9389320>.
llvm-svn: 130990
This change requires making a bunch of fundamental Clang structures (optionally) reference counted to allow correct
ownership semantics of these objects (e.g., ASTContext) to play out between an active ASTUnit and CompilerInstance
object.
llvm-svn: 128011
should report the original file name for contents of files that were overriden by other files,
otherwise it should report the name of the new file. Default is true.
Also add similar field in PreprocessorOptions and pass similar parameter in ASTUnit::LoadFromCommandLine.
llvm-svn: 127289
Allow remapping a file by specifying another filename whose contents should be loaded if the original
file gets loaded. This allows to override files without having to create & load buffers in advance.
llvm-svn: 127052
enumeration type, prioritize the enumeration constants and don't
provide completions for any other expressions. Fixes <rdar://problem/7283668>.
llvm-svn: 125991
code-completion results accessed via libclang, to extend the lifetime
of the allocator used for cached global code-completion results at
least until these completion results are destroyed. Fixes
<rdar://problem/8997369>.
llvm-svn: 125678
completions. We now compute a hash of the names of all top-level
declarations and macro definitions, and invalidate the cache when the
hash value changes.
llvm-svn: 125670
Store in PCH the directory that the PCH was originally created in.
If a header file is not found at the path that we expect it to be and the PCH file
was moved from its original location, try to resolve the file by assuming that
header+PCH were moved together and the header is in the same place relative to the PCH.
llvm-svn: 125576
overridden via remapping. Thus, when we create a "virtual" file in the
file manager, we still stat() the real file that lives behind it so
that we can provide proper uniquing based on inodes. This helps keep
the file manager much more consistent.
To take advantage of this when reparsing files in libclang, we disable
the use of the stat() cache when reparsing or performing code
completion, since the stat() cache is very likely to be out of date in
this use case.
llvm-svn: 124971
BumpPtrAllocator, rather than manually new/delete'ing them. This
optimization also allows us to avoid allocating memory for and copying
constant strings (e.g., "return", "class").
This also required embedding the priority and availability of results
within the code completion string, to avoid extra memory allocation
within libclang.
llvm-svn: 124673
trap the serialized preprocessing records (macro definitions, macro
instantiations, macro definitions) from the generation of the
precompiled preamble, then replay those when walking the list of
preprocessed entities. This eliminates a bug where clang_getCursor()
wasn't able to find preprocessed-entity cursors in the preamble.
llvm-svn: 120396
two copies, since they are fundamentally different
operations and the StringRef one should go away
(it shouldn't be part of FileManager at least).
Remove some dead arguments.
llvm-svn: 120013
FileSystemOpts through a ton of apis, simplifying a lot of code.
This also fixes a latent bug in ASTUnit where it would invoke
methods on FileManager without creating one in some code paths
in cindextext.
llvm-svn: 120010
This patch completely defeated the "passing in a prestat'd size
to MemoryBuffer" optimization, leading to an extra fstat call for
every buffer opened, in order to find out if the datestamp and size
of the file on disk matches what is in the stat cache.
I fully admit that I don't completely understand what is going on here:
why punish code when a stat cache isn't in use? what is the point of a
stat cache if you have to turn around and stat stuff to validate it?
To resolve both these issues, just drop the modtime check and check the
file size, which is the important thing anyway. This should also resolve
PR6812, because presumably windows is stable when it comes to file sizes.
If the modtime is actually important, we should get it and keep it on the
first stat.
This eliminates 833 fstat syscalls when processing Cocoa.h, speeding up
system time on -Eonly Cocoa.h from 0.041 to 0.038s.
llvm-svn: 120001
we were just getting a range covering only the property name, which is
certainly not correct (and broke token annotation, among other
things).
Also, teach libclang about the relationship between
@synthesize/@dynamic and @property, so we get property name and
cursor-reference information for @synthesize and @dynamic.
llvm-svn: 119409
caching global code-completion results. In particular, don't perform
either operation the first time we parse, but do both after the first
reparse.
llvm-svn: 119285
diagnostic-capturing client lives as long as the ASTUnit itself
does. Otherwise, we can end up with crashes when we get a diagnostic
outside of parsing/code completion. The circumstances under which this
happen are really hard to reproduce, because a file needs to change
from under us.
llvm-svn: 118751
When -working-directory is passed in command line, file paths are resolved relative to the specified directory.
This helps both when using libclang (where we can't require the user to actually change the working directory)
and to help reproduce test cases when the reproduction work comes along.
--FileSystemOptions is introduced which controls how file system operations are performed (currently it just contains
the working directory value if set).
--FileSystemOptions are passed around to various interfaces that perform file operations.
--Opening & reading the content of files should be done only through FileManager. This is useful in general since
file operations will be abstracted in the future for the reproduction mechanism.
FileSystemOptions is independent of FileManager so that we can have multiple translation units sharing the same
FileManager but with different FileSystemOptions.
Addresses rdar://8583824.
llvm-svn: 118203
timers to be dumped whenever the ASTUnit is destroyed. Instead, just
print the time elapsed for each operation after we perform the
operation.
llvm-svn: 117550
diagnostics produced by the driver itself. Previously, we were
allowing these to either be dropped or to slip through to stderr.
Fixes <rdar://problem/7595339>.
llvm-svn: 116285
clang_codeCompleteAt(). This uncovered a few issues with the latter:
- ASTUnit wasn't saving/restoring diagnostic state appropriately between
reparses and code completions.
- "Overload" completions weren't being passed through to the client
llvm-svn: 116241
preambles:
- When we rebuild a precompiled preamble, make sure to disable
skipping anything in the main file; we may have had leftover
preamble-skipping values in the lexer, which leads to very empty
preamble. This is a correctness issue.
- When we rebuild a precompiled preamble, clear out any prior state
in the Diagnostic object. Otherwise, we might think that there
were errors when we were building the preamble itself, and
therefore reject the resulting preamble. This is mainly a
performance issue.
- Don't remove old remappings when digging out the remapping for the
main file. Having the old mappings around does not hurt in the
common case (later remappings will just overwrite them), and is
important when we fail to find a preamble: we don't want to have
removed the remapping, because we'll need it later.
llvm-svn: 116041
identifier, we may have a Sema object but no translation unit scope
(because parsing is finished). In this case, we still need to update
the IdResolver, which might still be used when writing a PCH
containing another PCH (without chaining). This bug manifested as a
failure with precompiled preambles.
Also, add a little environment-variable-sensitive logging for
libclang.
llvm-svn: 114774
provided when the optimization is disabled. In particular, split
the completion context CCC_Other into two contexts: CCC_Other, which
means that it's an undisclosed context for which any other results are
unwelcome, and CCC_Recovery, which is used in recovery cases.
Since we're now using the completion context within the completion
results builder, make sure that it's always set to something.
Fixes <rdar://problem/8470644>.
llvm-svn: 114704
- In Objective-C, we prefer BOOL to bool for historic reasons;
slightly penalize "bool".
- Treat Nil macro as a NULL pointer constant.
- Treat YES, NO, true, and false macros as constants.
- Treat the bool macro as a type.
llvm-svn: 114356
expression, e.g., after the '(' that could also be a type cast. Here,
we provide types as code-completion results in C/Objective-C (C++
already had them), although we wouldn't in a normal expression context.
llvm-svn: 113904
preambles end up leaving the precompiled preambles around. This is by
design, since we do minimal cleanup during crash recovery. However,
it's unfortunate for testing, so introduce a hook that allows these
two tests to put the precompiled preamble somewhere where we can
delete them after testing.
llvm-svn: 113698
just abort creation of the precompiled preamble rather than doing
silly things. This is the second part of the fix for the weird
preamble-related failures on Windows. Big thanks to Francois Pichet
for the great detective work!
llvm-svn: 113697
Windows GetTempPath() function, and be sure to create the directory in
which the precompiled preamble will reside before creating the
temporary file itself.
llvm-svn: 113695
into the clients, e.g., the printing code-completion consumer and
c-index-test. Clients may want to re-sort the results anyway.
Provide a libclang function that sorts the results.
3rd try. How embarrassing.
llvm-svn: 112180
into the clients, e.g., the printing code-completion consumer and
c-index-test. Clients may want to re-sort the results anyway.
Provide a libclang function that sorts the results.
llvm-svn: 112149
code-completion results cached by ASTUnit, sort the resulting result
set. This makes testing far, far easier, so this commit also includes
tests for the previous few fixes.
llvm-svn: 112070
declarator. Here, we can only see a few things (e.g., cvr-qualifiers,
nested name specifiers) and we do not want to provide other non-macro
completions. Previously, we would end up in recovery mode and would
provide a large number of non-relevant completions.
llvm-svn: 111818
duplication between the constant and non-constant paths in all of this.
Implement ARM ABI semantics for member pointer constants and conversion.
llvm-svn: 111772
sure to (1) actually use the remapped files we were given rather
than old data, and (2) keep the remapped files alive until the
code-completion results are destroyed. Big thanks to Daniel for the
test case.
llvm-svn: 111597
and reenable crash recovery test.
- Reparsing is still very crashy / weird, so I had to sprinkle random code into
the remapped input to get it to do what I want (i.e., crash!).
llvm-svn: 111550
false (not true), so that the CompilerInstance will actually free data
structures when it's done. This fixes a major leak with libclang's
in-process code completion.
llvm-svn: 111457
translation unit, refresh code-completion results because they've
probably changed. However, enforce a cooldown period between
refreshes, to avoid thrashing.
llvm-svn: 111218
nested-name-specifiers. Also includes fixes to the generation of
nested-name-specifier result in the non-cached case; we were producing
lame results for namespaces and namespace aliases, which (1) didn't
always have nested-name-specifiers when we want them, and (2) did not
have the necessary "::" as part of the completion.
llvm-svn: 111203
the usage type of each declaration result, then compare those types to
the preferred type of the completion. This provides parity in the
priority calculation between the code-completion results produced
directly from Sema and those cached by ASTUnit.
For the standard Cocoa.h (+ others) example, there's a penalty of 3-4
hundredeths of a second when caching the global results (for ~31,000
results), because we need an ASTContext-agnostic representation of
types for the comparison, and therefore we use... strings. Eventually,
we'd like to implement a more efficient ASTContext-agnostic encoding
of types.
llvm-svn: 111165
type class, so that we can adjust priorities appropriately when the
preferred type for the context and the actual type of the completion
are similar.
This gets us one step closer to parity of the cached completion
results with the non-cached completion results.
llvm-svn: 111139
declarations (in addition to macros). Each kind of declaration maps to
a certain set of completion contexts, and the ASTUnit completion logic
introduces the completion strings for those declarations if the actual
code-completion occurs in one of the contexts where it matters.
There are a few new code-completion-context kinds. Without these,
certain completions (e.g., after "using namespace") would need to
suppress all global completions, which would be unfortunate.
Note that we don't get the priorities right for global completions,
because we don't have enough type information. We'll need a way to
compare types in an ASTContext-agnostic way before this can be
implemented.
llvm-svn: 111093
when the CXTranslationUnit_CacheCompletionResults option is given to
clang_parseTranslationUnit(). Essentially, we compute code-completion
results for macro definitions after we have parsed the file, then
store an ASTContext-agnostic version of those results (completion
string, cursor kind, priority, and active contexts) in the
ASTUnit. When performing code completion in that ASTUnit, we splice
the macro definition results into the results provided by the actual
code-completion (which has had macros turned off) before libclang gets
those results. We use completion context information to only splice in
those results that make sense for that context.
With a completion involving all of the macros from Cocoa.h and a few other
system libraries (totally ~8500 macro definitions) living in a
precompiled header, we get about a 9% performance improvement from
code completion, since we no longer have to deserialize all of the
macro definitions from the precompiled header.
Note that macro definitions are merely the canary; the cache is
designed to also support other top-level declarations, which should be
a bigger performance win. That optimization will be next.
Note also that there is no mechanism for determining when to throw
away the cache and recompute its contents.
llvm-svn: 111051
used when parsing (or re-parsing) a file. Also, when loading a
precompiled header into ASTUnit, create a Sema object that holds onto
semantic-analysis information.
llvm-svn: 111003
and create separate decl nodes for forward declarations and the
definition," which appears to be causing significant Objective-C
breakage.
llvm-svn: 110803
Douglas Gregor