Change ASTFileSignature from a random 32-bit number to the hash of the
PCM content.
- Move definition ASTFileSignature to Basic/Module.h so Module and
ASTSourceDescriptor can use it.
- Change the signature from uint64_t to std::array<uint32_t,5>.
- Stop using (saving/reading) the size and modification time of PCM
files when there is a valid SIGNATURE.
- Add UNHASHED_CONTROL_BLOCK, and use it to store the SIGNATURE record
and other records that shouldn't affect the hash. Because implicit
modules reuses the same file for multiple levels of -Werror, this
includes DIAGNOSTIC_OPTIONS and DIAG_PRAGMA_MAPPINGS.
This helps to solve a PCH + implicit Modules dependency issue: PCH files
are handled by the external build system, whereas implicit modules are
handled by internal compiler build system. This prevents invalidating a
PCH when the compiler overwrites a PCM file with the same content
(modulo the diagnostic differences).
Design and original patch by Manman Ren!
llvm-svn: 297655
Modules/preambles/PCH files can contain diagnostics, which, when used,
are added to the current ASTUnit. For that to work, they are translated
to use the current FileManager's FileIDs. When the entry is not the
main file, all local source locations will be checked by a linear
search. Now this is a problem, when there are lots of diagnostics (say,
25000) and lots of local source locations (say, 440000), and end up
taking seconds when using such a preamble.
The fix is to cache the last FileID, because many subsequent diagnostics
refer to the same file. This reduces the time spent in
ASTUnit::TranslateStoredDiagnostics from seconds to a few milliseconds
for files with many slocs/diagnostics.
This fixes PR31353.
Differential Revision: https://reviews.llvm.org/D29755
llvm-svn: 295301
If the preamble had diagnostic state this would leave behind invalid
state in the DiagnosticsEngine and crash later. The test case runs into
an assertion in DiagnosticsEngine::setSourceManager.
llvm-svn: 294963
Aleksey Shlypanikov pointed out my mistake in migrating an explicit
unique_ptr to auto - I was expecting the function returned a unique_ptr,
but instead it returned a raw pointer - introducing a leak.
Thanks Aleksey!
This reapplies r291184, reverted in r291249.
llvm-svn: 291270
In this mode, there is no need to load any module map and the programmer can
simply use "@import" syntax to load the module directly from a prebuilt
module path. When loading from prebuilt module path, we don't support
rebuilding of the module files and we ignore compatible configuration
mismatches.
rdar://27290316
Differential Revision: http://reviews.llvm.org/D23125
llvm-svn: 279096
This changes the CompilerInstance::createOutputFile function to return
a std::unique_ptr<llvm::raw_ostream>, rather than an llvm::raw_ostream
implicitly owned by the CompilerInstance. This in most cases required that
I move ownership of the output stream to the relevant ASTConsumer.
The motivation for this change is to allow BackendConsumer to be a client
of interfaces such as D20268 which take ownership of the output stream.
Differential Revision: http://reviews.llvm.org/D21537
llvm-svn: 275507
When remapped files were changed, they would not always cause the preamble's PCH to be invalidated, because the remapped path didn't necessarily match the include path (e.g. slash direction -- this happens a lot on Windows). I fixed this by moving to a llvm::sys::fs::UniqueID-based map instead of comparing paths stringwise.
Differential Revision: http://reviews.llvm.org/D20137
llvm-svn: 269769
Revert the two changes to thread CodeGenOptions into the TargetInfo allocation
and to fix the layering violation by moving CodeGenOptions into Basic.
Code Generation is arguably not particularly "basic". This addresses Richard's
post-commit review comments. This change purely does the mechanical revert and
will be followed up with an alternate approach to thread the desired information
into TargetInfo.
llvm-svn: 265806
This threads CodeGenOptions into the TargetInfo hierarchy. This is motivated by
ARM which can change some target information based on the EABI selected
(-meabi). Similar options exist for other platforms (e.g. MIPS) and thus is
generally useful. NFC.
llvm-svn: 265640
option. Previously these options could both be used to specify that you were
compiling the implementation file of a module, with a different set of minor
bugs in each case.
This change removes -fmodule-implementation-of, and instead tracks a flag to
determine whether we're currently building a module. -fmodule-name now behaves
the same way that -fmodule-implementation-of previously did.
llvm-svn: 261372
Summary:
The current default is to create the preamble on the first reparse, aka
second parse. This is useful for clients that do not want to block when
opening a file because serializing the preamble takes a bit of time.
However, this makes the reparse much more expensive and that may be on the
critical path as it's the first interaction a user has with the source code.
YouCompleteMe currently optimizes for the first code interaction by parsing
the file twice when loaded. That's just unnecessarily slow and this flag
helps to avoid that.
Reviewers: doug.gregor, klimek
Subscribers: cfe-commits
Differential Revision: http://reviews.llvm.org/D15490
llvm-svn: 255635
Introduce the notion of a module file extension, which introduces
additional information into a module file at the time it is built that
can then be queried when the module file is read. Module file
extensions are identified by a block name (which must be unique to the
extension) and can write any bitstream records into their own
extension block within the module file. When a module file is loaded,
any extension blocks are matched up with module file extension
readers, that are per-module-file and are given access to the input
bitstream.
Note that module file extensions can only be introduced by
programmatic clients that have access to the CompilerInvocation. There
is only one such extension at the moment, which is used for testing
the module file extension harness. As a future direction, one could
imagine allowing the plugin mechanism to introduce new module file
extensions.
llvm-svn: 251955
Summary: It breaks the build for the ASTMatchers
Subscribers: klimek, cfe-commits
Differential Revision: http://reviews.llvm.org/D13893
llvm-svn: 250827
ASTUnit was creating multiple FileManagers and throwing them away. Reuse
the one from Tooling. No functionality change now but necessary for
VFSifying tooling.
llvm-svn: 249410
to enable the use of external type references in the debug info
(a.k.a. module debugging).
The driver expands -gmodules to "-g -fmodule-format=obj -dwarf-ext-refs"
and passes that to cc1. All this does at the moment is set a flag
codegenopts.
http://reviews.llvm.org/D11958
llvm-svn: 246192
- introduces a new cc1 option -fmodule-format=[raw,obj]
with 'raw' being the default
- supports arbitrary module container formats that libclang is agnostic to
- adds the format to the module hash to avoid collisions
- splits the old PCHContainerOperations into PCHContainerWriter and
a PCHContainerReader.
Thanks to Richard Smith for reviewing this patch!
llvm-svn: 242499
The patch is generated using this command:
$ tools/extra/clang-tidy/tool/run-clang-tidy.py -fix \
-checks=-*,llvm-namespace-comment -header-filter='llvm/.*|clang/.*' \
work/llvm/tools/clang
To reduce churn, not touching namespaces spanning less than 10 lines.
llvm-svn: 240270
A PCHContainerOperations abstract interface provides operations for
creating and unwrapping containers for serialized ASTs (precompiled
headers and clang modules). The default implementation is
RawPCHContainerOperations, which uses a flat file for the output.
The main application for this interface will be an
ObjectFilePCHContainerOperations implementation that uses LLVM to
wrap the module in an ELF/Mach-O/COFF container to store debug info
alongside the AST.
rdar://problem/20091852
llvm-svn: 240225
If the type isn't trivially moveable emplace can skip a potentially
expensive move. It also saves a couple of characters.
Call sites were found with the ASTMatcher + some semi-automated cleanup.
memberCallExpr(
argumentCountIs(1), callee(methodDecl(hasName("push_back"))),
on(hasType(recordDecl(has(namedDecl(hasName("emplace_back")))))),
hasArgument(0, bindTemporaryExpr(
hasType(recordDecl(hasNonTrivialDestructor())),
has(constructExpr()))),
unless(isInTemplateInstantiation()))
No functional change intended.
llvm-svn: 238601
Now that SmallString is a first-class citizen, most SmallString::str()
calls are not required. This patch removes a whole bunch of them, yet
there are lots more.
There are two use cases where str() is really needed:
1) To use one of StringRef member functions which is not available in
SmallString.
2) To convert to std::string, as StringRef implicitly converts while
SmallString do not. We may wish to change this, but it may introduce
ambiguity.
llvm-svn: 232622
with a subset of the existing target CPU features or mismatched CPU
names.
While we can't check that the CPU name used to build the module will end
up being able to codegen correctly for the translation unit, we actually
check that the imported features are a subset of the existing features.
While here, rewrite the code to use std::set_difference and have it
diagnose all of the differences found.
Test case added which walks the set relationships and ensures we
diagnose all the right cases and accept the others.
No functional change for implicit modules here, just better diagnostics.
llvm-svn: 232248
This is a necessary prerequisite for debugging with modules.
The .pcm files become containers that hold the serialized AST which allows
us to store debug information in the module file that can be shared by all
object files that were built importing the module.
This reapplies r230044 with a fixed configure+make build and updated
dependencies and testcase requirements. Over the last iteration this
version adds
- missing target requirements for testcases that specify an x86 triple,
- a missing clangCodeGen.a dependency to libClang.a in the make build.
rdar://problem/19104245
llvm-svn: 230423
This is a necessary prerequisite for debugging with modules.
The .pcm files become containers that hold the serialized AST which allows
us to store debug information in the module file that can be shared by all
object files that were built importing the module.
rdar://problem/19104245
This reapplies r230044 with a fixed configure+make build and updated
dependencies. Take 3.
llvm-svn: 230305
This is a necessary prerequisite for debugging with modules.
The .pcm files become containers that hold the serialized AST which allows
us to store debug information in the module file that can be shared by all
object files that were built importing the module.
rdar://problem/19104245
This reapplies r230044 with a fixed configure+make build and updated
dependencies. Take 2.
llvm-svn: 230089
This is a necessary prerequisite for debugging with modules.
The .pcm files become containers that hold the serialized AST which allows
us to store debug information in the module file that can be shared by all
object files that were built importing the module.
rdar://problem/19104245
This reapplies r230044 with a fixed configure+make build and updated
dependencies.
llvm-svn: 230067
This is a necessary prerequisite for debugging with modules.
The .pcm files become containers that hold the serialized AST which allows
us to store debug information in the module file that can be shared by all
object files that were built importing the module.
rdar://problem/19104245
llvm-svn: 230044
Summary:
Make DiagnosticsEngine::takeClient return std::unique_ptr<>. Updated
callers to store conditional ownership using a pair of pointer and unique_ptr
instead of a pointer + bool. Updated code that temporarily registers clients to
use the non-owning registration (+ removed extra calls to takeClient).
Reviewers: dblaikie
Reviewed By: dblaikie
Subscribers: cfe-commits
Differential Revision: http://reviews.llvm.org/D6294
llvm-svn: 222193
Implicit module builds are not well-suited to a lot of build systems. In
particular, they fare badly in distributed build systems, and they lead to
build artifacts that are not tracked as part of the usual dependency management
process. This change allows explicitly-built module files (which are already
supported through the -emit-module flag) to be explicitly loaded into a build,
allowing build systems to opt to manage module builds and dependencies
themselves.
This is only the first step in supporting such configurations, and it should
be considered experimental and subject to change or removal for now.
llvm-svn: 220359
The various ways to create an ASTUnit all take a refcounted pointer to
a diagnostics engine as an argument, and if it isn't pointing at
anything they initialize it. This is a pretty confusing API, and it
really makes more sense for the caller to initialize the thing since
they control the lifetime anyway.
This fixes the one caller that didn't bother initializing the pointer
and asserts that the argument is initialized.
llvm-svn: 219752
Rather than having a pair of pairs and a reference out parameter, build
a structure with everything together and named. A raw pointer and a
unique_ptr, rather than a raw pointer and a boolean, are used to
communicate ownership transfer.
It's possible one day we'll end up with a conditional pointer (probably
represented by a raw pointer and a boolean) abstraction to use in places
like this. Conditional ownership seems to be coming up more often than
I'd hoped...
llvm-svn: 216712
This change is the last in the pack of five commits
(also see r216691, r216694, r216695, and r216696) that reduces the number
of test failures in "check-clang" invocation in UBSan bootstrap
from 2443 down to 5.
llvm-svn: 216697
(dropping const from the reference as MemoryBuffer is immutable already,
so const is just redundant - and while I'd personally put const
everywhere, that's not the LLVM Way (see llvm::Type for another example
of an immutable type where "const" is omitted for brevity))
Changing the pointer argument to a reference parameter makes call sites
identical between callers with unique_ptrs or raw pointers, minimizing
the churn in a pending unique_ptr migrations.
llvm-svn: 215391
After post-commit review and community discussion, this seems like a
reasonable direction to continue, making ownership semantics explicit in
the source using the type system.
llvm-svn: 215323
This reverts commit r213307.
Reverting to have some on-list discussion/confirmation about the ongoing
direction of smart pointer usage in the LLVM project.
llvm-svn: 213325
(after fixing a bug in MultiplexConsumer I noticed the ownership of the
nested consumers was implemented with raw pointers - so this fixes
that... and follows the source back to its origin pushing unique_ptr
ownership up through there too)
llvm-svn: 213307
Add module dependencies (header files, module map files) to the list of
files to check when deciding whether to rebuild a preamble. That fixes
using preambles with module imports so long as they are in
non-overridden files.
My intent is to use to unify the existing dependency collectors to the
new “DependencyCollectory” interface from this commit, starting with the
DependencyFileGenerator.
llvm-svn: 212060
We were using old stat values for any files that had previously been
looked up, leading to badness. There might be a more elegant solution in
invalidating the cache for those file (since we already know which ones
they are), but it seems too likely there are existing references to
them hiding somewhere.
llvm-svn: 211504
Having various possible states of initialization following construction doesn't
add value here.
Also remove the unused size_reserve parameter.
llvm-svn: 207897
The Preprocessor::Initialize() function already offers a clear interface to
achieve this, further reducing the confusing number of states a newly
constructed preprocessor can have.
llvm-svn: 207825
We don't need the ASTContext for the diagnostics, only the language
options, which we can get from the compiler invocation. It worries me
how many categorically different states the ASTUnit class can be in
depending on how it is being constructed/used.
llvm-svn: 206909
Padding does not seem to be useful currently, and it leads to bogus location if an error
points to the end of the file.
rdar://15836513
llvm-svn: 203370
With r197755 we started reading the contents of buffer file entries, but the
buffers may point to ASTReader blobs that have been disposed.
Fix this by having the CompilerInstance object keep a reference to the ASTReader
as well as having the ASTContext keep reference to the ExternalASTSource.
This was very difficult to construct a test case for.
rdar://16149782
llvm-svn: 202346
Previously reverted in r201755 due to causing an assertion failure.
I've removed the offending assertion, and taught the CompilerInstance to
create a default virtual file system inside createFileManager. In the
future, we should be able to reach into the CompilerInvocation to
customize this behaviour without breaking clients that don't care.
llvm-svn: 201818
llvm::sys::cas_flag is 'long' instead of 'uint32_t' on win32, because
that's what InterlockedIncrement is defined to accept.
I still don't know if we should be calling fprintf from ASTUnit.cpp
behind a getenv check.
llvm-svn: 200718
files to tell if they were changed since the last time we have computed the
preamble
We used to check only the buffer size, so if the new remapped buffer has the
same size as the previous one, we would think that the buffer did not change,
and we did not rebuild the preambule, which sometimes caused us to crash.
llvm-svn: 197755
ASTUnit instances are allocated infrequently so it's fine to keep this field
around in all build configurations.
Assigns null to silence -Wunused-private-field in Release.
llvm-svn: 195419
This allows using virtual file mappings on the original SourceManager to
map in virtual module.map files. Without this patch, the ModuleMap
search will find a module.map file (as the FileEntry exists in the
FileManager), but will be unable to get the content from the
SourceManager (as ModuleMap previously created its own SourceManager).
Two problems needed to be fixed which this patch exposed:
1. Storing the inferred module map
When writing out a module, the ASTWriter stores the names of the files
in the main source manager; when loading the AST again, the ASTReader
errs out if such a file is found missing, unless it is overridden.
Previously CompilerInstance's compileModule method would store the
inferred module map to a temporary file; the problem with this approach
is that now that the module map is handled by the main source manager,
the ASTWriter stores the name of the temporary module map as source to
the compilation; later, when the module is loaded, the temporary file
has already been deleted, which leads to a compilation error. This patch
changes the inferred module map to instead inject a virtual file into
the source manager. This both saves some disk IO, and works with how the
ASTWriter/ASTReader handle overridden source files.
2. Changing test input in test/Modules/Inputs/*
Now that the module map file is handled by the main source manager, the
VerifyDiagnosticConsumer will not ignore diagnostics created while
parsing the module map file. The module test test/Modules/renamed.m uses
-I test/Modules/Inputs and triggers recursive loading of all module maps
in test/Modules/Inputs, some of which had conflicting names, thus
leading errors while parsing the module maps. Those diagnostics already
occur on trunk, but before this patch they would not break the test, as
they were ignored by the VerifyDiagnosticConsumer. This patch thus
changes the module maps that have been recently introduced which broke
the invariant of compatible modules maps in test/Modules/Inputs.
llvm-svn: 193314
Let me tell you a tale...
Within some twisted maze of debug info I've ended up implementing an
insane man's Include What You Use device. When the debugger emits debug
info it really shouldn't, I find out why & then realize the code could
be improved too.
In this instance CIndexDiagnostics.cpp had a lot more debug info with
Clang than GCC. Upon inspection a major culprit was all the debug info
describing clang::Sema. This was emitted because clang::Sema is
befriended by DiagnosticEngine which was rightly required, but GCC
doesn't emit debug info for friends so it never emitted anything for
Clang. Clang does emit debug info for friends (will be fixed/changed to
reduce debug info size).
But why didn't Clang just emit a declaration of Sema if this entire TU
didn't require a definition?
1) Diagnostic.h did the right thing, only using a declaration of Sema
and not including Sema.h at all.
2) Some other dependency of CIndexDiagnostics.cpp didn't do the right
thing. ASTUnit.h, only needing a declaration, still included Sema.h
(hence this commit which removes that include and adds the necessary
includes to the cpp files that were relying on this)
3) -flimit-debug-info didn't save us because of
EnterExpressionEvaluationContext, defined inline in Sema.h which fires
the "requiresCompleteType" check/flag (since it uses nested types from
Sema and calls Sema member functions) and thus, if debug info is ever
emitted for the type, the whole type is emitted and not just a
declaration.
Improving -flimit-debug-info to account for this would be... hard.
Modifying the code so that's not 'required to be complete' might be
possible, but probably only by moving EnterExpressionEvaluationContext
either into Sema, or out of Sema.h. That might be a bit too much of a
contortion to be bothered with.
Also, this is only one of the cases where emitting debug info for
friends caused us to emit a lot more debug info (this change reduces
Clang's DWO size by 0.93%, dropping friends entirely reduces debug info
by 3.2%) - I haven't hunted down the other cases, but I assume they
might be similar (Sema or something like it). IWYU or a similar tool
might help us reduce build times a bit, but analyzing debug info to find
these differences isn't worthwhile. I'll take the 3.2% win, provide this
small improvement to the code itself, and move on.
llvm-svn: 190715
The problem was that an enum without closing semicolon could be associated as a forward enum
in an erroneous declaration, leading to the identifier being associated with the enum decl but
without a declaration actually referencing it.
This resulted in not having it serialized before serializing the identifier that is associated with.
Also prevent the ASTUnit from querying the serialized DeclID for an invalid top-level decl; it may not
have been serialized.
rdar://14539667
llvm-svn: 187914
The top-level hash is used to determine if we need to update the global code-completion results.
ImportDecls did not affect the hash so a newly introduced ImportDecl would not trigger an update of the global results.
rdar://14202797
llvm-svn: 184782
A while ago we allowed libclang to build a PCH that had compiler errors; this was to retain the performance
afforded by a PCH even if the user's code is in an intermediate state.
Extend this for the precompiled preamble as well.
rdar://14109828
llvm-svn: 183717
Duncan P. N. Exon Smith