Summary:
Review comments on {D68767} asked that this duplicated code in clang-format was moved to one central location that being SourceManager (where it had originally be copied from I assume)
Moved function into static function ContentCache::getInvalidBOM(...) - (closest class to where it was defined before)
Updated clang-format to call this static function
Added unit tests for said new function in BasicTests
Sorry not my normal code area so may have the wrong reviewers. (but your names were on the recent history)
Reviewers: bruno, arphaman, klimek, owenpan, mitchell-stellar, dexonsmith
Reviewed By: owenpan
Subscribers: cfe-commits
Tags: #clang, #clang-format, #clang-tools-extra
Differential Revision: https://reviews.llvm.org/D68914
If contents of a file that is part of a PCM are overridden when reading
it, but weren't overridden when the PCM was being built, the ASTReader
will emit an error. Now it creates a separate FileEntry for recovery,
bypassing the overridden content instead of discarding it. The
pre-existing testcase clang/test/PCH/remap-file-from-pch.cpp confirms
that the new recovery method works correctly.
This resolves a long-standing FIXME to avoid hypothetically invalidating
another precompiled module that's already using the overridden contents.
This also removes ContentCache-related API that would be unsafe to use
across `CompilerInstance`s in an implicit modules build. This helps to
unblock us sinking it from SourceManager into FileManager in the future,
which would allow us to delete `InMemoryModuleCache`.
https://reviews.llvm.org/D66710
llvm-svn: 370546
Refactor ContentCache::IsSystemFile to IsFileVolatile, checking
SourceManager::userFilesAreVolatile at construction time. This is a
step toward lowering ContentCache down from SourceManager to
FileManager.
No functionality change intended.
https://reviews.llvm.org/D66713
llvm-svn: 369958
when the FileManager is reused across invocations
This commit introduces a parallel API to FileManager's getFile: getFileEntryRef, which returns
a reference to the FileEntry, and the name that was used to access the file. In the case of
a VFS with 'use-external-names', the FileEntyRef contains the external name of the file,
not the filename that was used to access it.
The new API is adopted only in the HeaderSearch and Preprocessor for include file lookup, so that the
accessed path can be propagated to SourceManager's FileInfo. SourceManager's FileInfo now can report this accessed path, using
the new getName method. This API is then adopted in the dependency collector, which now correctly reports dependencies when a file
is included both using a symlink and a real path in the case when the FileManager is reused across multiple Preprocessor invocations.
Note that this patch does not fix all dependency collector issues, as the same problem is still present in other cases when dependencies
are obtained using FileSkipped, InclusionDirective, and HasInclude. This will be fixed in follow-up commits.
Differential Revision: https://reviews.llvm.org/D65907
llvm-svn: 369680
I noticed that SourceManager::translateFile has code that doesn't really make sense.
In particular, if it fails to find a FileID by comparing FileEntry * values, it tries to
look through files that have the same filename, to see if they have a matching inode to try to
find the right FileID. However, the inode comparison seem redundant, as Clang's FileManager
already deduplicates FileEntry * values by inode.
Thus the comparisons between inodes should never actually succeed, and the comparison between FileEntry * values should be sufficient here.
Differential Revision: https://reviews.llvm.org/D65481
llvm-svn: 369585
Now that we've moved to C++14, we no longer need the llvm::make_unique
implementation from STLExtras.h. This patch is a mechanical replacement
of (hopefully) all the llvm::make_unique instances across the monorepo.
Differential revision: https://reviews.llvm.org/D66259
llvm-svn: 368942
Update the callers of FileManager::getFile and FileManager::getDirectory to handle the new llvm::ErrorOr-returning methods.
Signed-off-by: Harlan Haskins <harlan@apple.com>
llvm-svn: 367616
crashing.
Ideally we wouldn't care about the size of a file so long as it fits in
memory, but in practice we have lots of hardocded assumptions that
unsigned can be used to index files, string literals, and so on.
llvm-svn: 364103
If a header file was processed for the second time, we could end up with a
wrong conditional stack and skipped ranges:
In the particular example, if the header guard is evaluated the second time and
it is decided to skip the conditional block, the corresponding "#endif" is
never seen since the preamble does not include it and we end up in the
Tok.is(tok::eof) case with a wrong conditional stack.
Detect the circular inclusion, emit a diagnostic and stop processing the
inclusion.
llvm-svn: 360418
Requires making the llvm::MemoryBuffer* stored by SourceManager const,
which in turn requires making the accessors for that return const
llvm::MemoryBuffer*s and updating all call sites.
The original motivation for this was to use it and fix the TODO in
CodeGenAction.cpp's ConvertBackendLocation() by using the UnownedTag
version of createFileID, and since llvm::SourceMgr* hands out a const
llvm::MemoryBuffer* this is required. I'm not sure if fixing the TODO
this way actually works, but this seems like a good change on its own
anyways.
No intended behavior change.
Differential Revision: https://reviews.llvm.org/D60247
llvm-svn: 357724
to reflect the new license.
We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.
Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.
llvm-svn: 351636
Summary:
SSE2 vectorization was added in 2012, but it is 2018 now and I can't
observe any performance boost (testing clang -E [all Sema/* CodeGen/* with proper -I options]) with the existing _mm_movemask_epi8+countTrailingZeros or the following SSE4.2 (compiling with -msse4.2):
__m128i C = _mm_setr_epi8('\r','\n',0,0,0,0,0,0,0,0,0,0,0,0,0,0);
_mm_cmpestri(C, 2, Chunk, 16, _SIDD_UBYTE_OPS | _SIDD_CMP_EQUAL_ANY | _SIDD_POSITIVE_POLARITY | _SIDD_LEAST_SIGNIFICANT)
Delete the vectorization to simplify the code.
Also simplify the code a bit and don't check the line ending sequence \n\r
Reviewers: bkramer, #clang
Reviewed By: bkramer
Subscribers: cfe-commits
Differential Revision: https://reviews.llvm.org/D55484
llvm-svn: 348777
This patch moves the virtual file system form clang to llvm so it can be
used by more projects.
Concretely the patch:
- Moves VirtualFileSystem.{h|cpp} from clang/Basic to llvm/Support.
- Moves the corresponding unit test from clang to llvm.
- Moves the vfs namespace from clang::vfs to llvm::vfs.
- Formats the lines affected by this change, mostly this is the result of
the added llvm namespace.
RFC on the mailing list:
http://lists.llvm.org/pipermail/llvm-dev/2018-October/126657.html
Differential revision: https://reviews.llvm.org/D52783
llvm-svn: 344140
Summary: This can be used to create a virtual environment (incl. VFS, source manager) for code snippets.
Reviewers: sammccall, klimek
Reviewed By: sammccall
Subscribers: klimek, mgorny, cfe-commits
Differential Revision: https://reviews.llvm.org/D46176
llvm-svn: 331923
This is similar to the LLVM change https://reviews.llvm.org/D46290.
We've been running doxygen with the autobrief option for a couple of
years now. This makes the \brief markers into our comments
redundant. Since they are a visual distraction and we don't want to
encourage more \brief markers in new code either, this patch removes
them all.
Patch produced by
for i in $(git grep -l '\@brief'); do perl -pi -e 's/\@brief //g' $i & done
for i in $(git grep -l '\\brief'); do perl -pi -e 's/\\brief //g' $i & done
Differential Revision: https://reviews.llvm.org/D46320
llvm-svn: 331834
When a '>>' token is split into two '>' tokens (in C++11 onwards), or (as an
extension) when we do the same for other tokens starting with a '>', we can't
just use a location pointing to the first '>' as the location of the split
token, because that would result in our miscomputing the length and spelling
for the token. As a consequence, for example, a refactoring replacing 'A<X>'
with something else would sometimes replace one character too many, and
similarly diagnostics highlighting a template-id source range would highlight
one character too many.
Fix this by creating an expansion range covering the first character of the
'>>' token, whose spelling is '>'. For this to work, we generalize the
expansion range of a macro FileID to be either a token range (the common case)
or a character range (used in this new case).
llvm-svn: 331155
Found via codespell -q 3 -I ../clang-whitelist.txt
Where whitelist consists of:
archtype
cas
classs
checkk
compres
definit
frome
iff
inteval
ith
lod
methode
nd
optin
ot
pres
statics
te
thru
Patch by luzpaz! (This is a subset of D44188 that applies cleanly with a few
files that have dubious fixes reverted.)
Differential revision: https://reviews.llvm.org/D44188
llvm-svn: 329399
The code for going up the macro arg expansion is duplicated in many
places (and we need it for the analyzer as well, so I did not want to
duplicate it two more times).
This patch is an NFC, so the semantics should remain the same.
Differential Revision: https://reviews.llvm.org/D42458
llvm-svn: 324780
Summary:
llvm has grown a WritableMemoryBuffer class, which is convertible
(inherits from) a MemoryBuffer. We can use it to avoid conts_casting the
buffer contents when we want to write to it.
Reviewers: dblaikie, rsmith
Subscribers: cfe-commits
Differential Revision: https://reviews.llvm.org/D41387
llvm-svn: 321167
This fixes a possible crash on certain kinds of corrupted AST file, but
checking in an AST file corrupted in just the right way will be a maintenance
nightmare because the format changes frequently.
llvm-svn: 312851
The first user of this API will be the cross translation unit
functionality of the Static Analyzer which will be committed in a
follow-up patch.
Differential Revision: https://reviews.llvm.org/D34506
llvm-svn: 306648
When giving a ContentCache a null buffer, ignore the DoNotFree flag rather than
inheriting it onto whatever buffer we end up using for the file. Also ensure
that the main buffer is properly destroyed.
llvm-svn: 304740
replay the steps taken to create the AST file with the preprocessor-only action
installed to produce preprocessed output.
This can be used to produce the preprocessed text for an existing .pch or .pcm
file.
llvm-svn: 304726
This allows #line directives to appear in system headers that have code
that clang would normally warn on. This is compatible with GCC, which is
easy to test by running `gcc -E`.
Fixes PR30752
llvm-svn: 303582
When fixing a Clang-Tidy bug in D31406,
reuse of FileID enabled the missing highlightRange function.
Assertion in highlightRange failed because the end-of-range column
number was 2 + the last column of a line on Windows.
This fix is required to enable D31406.
Differential Revision: https://reviews.llvm.org/D31713
llvm-svn: 299681
than reusing the "overridden buffer" mechanism. This will allow us to make
embedded files and overridden files behave differently in future.
llvm-svn: 254121
This flag causes all files that were read by the compilation to be embedded
into a produced module file. This is useful for distributed build systems that
use an include scanning system to determine which files are "needed" by a
compilation, and only provide those files to remote compilation workers. Since
using a module can require any file that is part of that module (or anything it
transitively includes), files that are not found by an include scanner can be
required in a regular build using explicit modules. With this flag, only files
that are actually referenced by transitively-#included files are required to be
present on the build machine.
llvm-svn: 253950
Summary: It breaks the build for the ASTMatchers
Subscribers: klimek, cfe-commits
Differential Revision: http://reviews.llvm.org/D13893
llvm-svn: 250827
Trace the ranges through the macro backtrace better. This allows better
range highlighting through all levels of the macro bracktrace. Also some
improvements to backtrace printer for omitting different backtraces.
Patch by Zhengkai Wu.
Differential Revision: http://reviews.llvm.org/D12379
llvm-svn: 248454
file in the .pcm files. This allows a smaller set of files to be sent to a
remote build worker when building with explicit modules (for instance, module
map files need not be sent along with the corresponding precompiled modules).
This doesn't actually make the embedded files visible to header search, so
it's not useful as a packaging format for public header files.
llvm-svn: 245028
location data is available. If pragma handling wants to look up the
position, it finds the LLVM buffer and wants to compare it with the
special built-in buffer, failing badly. Extend to the special handling
of the built-in buffer to also check for the inline asm buffer. Expect
only a single asm buffer. Sort it between the built-in buffers and the
normal file buffers.
Fixes the assert part of PR 22576.
llvm-svn: 232389
This removes a const_cast added in r211884 that occurred due to an
inconsistency in how MemoryBuffers are handled between some parts of
clang and LLVM.
MemoryBuffers are immutable and the general convention in the LLVM
project is to omit const from immutable types as it's simply
redundant/verbose (see llvm::Type, for example). While this change
doesn't remove "const" from /every/ MemoryBuffer, it at least makes this
chain of ownership/usage consistent.
llvm-svn: 211915
alignment constraint rather than using the allocator function's over
alignment "feature". This was the only use of the "feature" which I plan
to remove next. =] Attaching the alignment to the type seems cleaner and
more principled anyways.
llvm-svn: 206324
- We really shouldn't compute line numbers for every file that is asked if it's
the main file, it destroys the lazy computation.
- Invalid locations are no longer accounted to the main file, no other
functionality change.
llvm-svn: 191535
The main benefit is to speed-up SourceManager::isBeforeInTranslationUnit which is common to query
the included/expanded location of the same FileID multiple times.
llvm-svn: 179435
isBeforeInTranslationUnit() uses a cache to reduce the expensive work
to compute a common ancestor for two FileIDs. This work is very
expensive, so even caching the latest used FileIDs was a big win.
A closer analysis of the cache before, however, shows that the cache
access pattern would oscillate between a working set of FileIDs, and
thus caching more pairs would be profitable.
This patch adds a side table for extending caching. This side table
is bounded in size (experimentally determined in this case from
a simple Objective-C project), and when the table gets too large
we fall back to the single entry caching before as before.
On Sketch (a small example Objective-C project), this optimization
reduces -fsyntax-only time on SKTGraphicView.m by 5%. This is
for a project that is already using PCH.
Fixes <rdar://problem/13299847>
llvm-svn: 176142
This may seem counter-intuitive but the POD-like optimization helps when the
vectors grow into multimegabyte buffers. SmallVector calls realloc which knows
how to twiddle virtual memory bits and avoids large copies.
llvm-svn: 175906
Previously, -Wunused-comparison ignored comparisons in both macro bodies and
macro arguments, but we would still emit a -Wunused-value warning for either.
Now we correctly emit -Wunused-comparison for expressions in macro arguments.
Also, add isMacroBodyExpansion to SourceManager, to go along with
isMacroArgExpansion.
llvm-svn: 172279
uncovered.
This required manually correcting all of the incorrect main-module
headers I could find, and running the new llvm/utils/sort_includes.py
script over the files.
I also manually added quite a few missing headers that were uncovered by
shuffling the order or moving headers up to be main-module-headers.
llvm-svn: 169237
working with preprocessed testcases. This causes source locations in
diagnostics to point at the spelling location instead of the presumed location,
while still keeping the semantic effects of the line directives (entering and
leaving system-header mode, primarily).
llvm-svn: 168004
macro expansion ranges, make sure to check all the FileID
entries that are contained in the spelling range of the
expansion for the macro argument.
Fixes rdar://12537982
llvm-svn: 166359
as "volatile", meaning there's a high enough chance that they may
change while we are trying to use them.
This flag is only enabled by libclang.
Currently "volatile" source files will be stat'ed immediately
before opening them, because the file size stat info
may not be accurate since when we got it (e.g. from the PCH).
This avoids crashes when trying to reference mmap'ed memory
from a file whose size is not what we expect.
Note that there's still a window for a racing issue to occur
but the window for it should be way smaller than before.
We can consider later on to avoid mmap completely on such files.
rdar://11612916
llvm-svn: 160074
Jan Korous