Commit Graph

962 Commits

Author SHA1 Message Date
Richard Smith b648399f9f PR27754: CXXRecordDecl::data() needs to perform an update even if it's called
on a declaration that already knows the location of the DefinitionData object.

llvm-svn: 269858
2016-05-17 22:44:15 +00:00
Manman Ren e4a5d37d6b Modules: set SystemHeader to true if we are building a system module.
If we are processing a #include from a module build, we should treat it
as a system header if we're building a system module. Passing an optional
flag to HeaderSearch::LookupFile.

Before this, the testing case will crash when accessing a freed FileEntry.

rdar://26214027

llvm-svn: 269730
2016-05-17 02:15:12 +00:00
Bruno Cardoso Lopes b171a59bfd [Modules] Use vfs for (recursive) directory iteration
Clang performs directory walk while searching headers inside modules by
using the ::sys::fs instead of ::vfs. This prevents any code that uses
the VFS (e.g, reproducer scripts) to actually find such headers, since
the VFS will never be searched for those.

Change these places to use vfs::recursive_directory_iterator and
vfs::directory_iterator instead.

Differential Revision: http://reviews.llvm.org/D20266

rdar://problem/25880368

llvm-svn: 269661
2016-05-16 16:46:01 +00:00
Bruno Cardoso Lopes b3a0fa4863 [ModuleMap][CrashReproducer] Collect headers from inner frameworks
(1) Collect headers under inner frameworks (frameworks inside other
other frameworks).
(2) Make sure we also collect the right header files inside them.

More info on (2):

Consider a dummy framework module B, with header Frameworks/B/B.h. Now
consider that another framework A, with header Frameworks/A/A.h, has a
layout with a inner framework Frameworks/A/Frameworks/B/B.h, where the
"B/B.h" part is a symlink for Frameworks/B/B.h. Also assume that
Frameworks/A/A.h includes <B/B.h>.

When parsing header Frameworks/A/A.h, framework module lookup is
performed in search for B, and it happens that
"Frameworks/A/Frameworks/B/B.h" path is registered in the module instead
of real "Frameworks/B/B.h". This occurs because
"Frameworks/A/Frameworks/B/B.h" is scanned first by the FileManager,
when looking for inner framework modules under Frameworks/A/Frameworks.
This makes Frameworks/A/Frameworks/B/B.h the default cached named inside
the FileManager for the B.h file UID.

This leads to modules being built without consistent paths to underlying
header files. This is usually not a problem in regular compilation flow,
but it's an issue when running the crash reproducer. The issue is that
clangs collect "Frameworks/A/Frameworks/B/B.h" but not
"Frameworks/B/B.h" into the VFS, leading to err_mmap_umbrella_clash. So
make sure we also collect the original header.

Differential Revision: http://reviews.llvm.org/D20194

rdar://problem/25880368

llvm-svn: 269502
2016-05-13 22:21:51 +00:00
Bruno Cardoso Lopes 0df3e04460 [CrashReproducer] Always use realpath for destination
When running reproducer scripts we need that original symlinks from the
source filesystem are reproduced in the VFS so that different virtual
paths can map to the same file, allowing the FileManager to share the
same UID between these virtual entries. This avoids all sorts of module
redefinition errors when using frameworks.

llvm-svn: 268825
2016-05-06 23:58:58 +00:00
Richard Smith 6739a10cec [modules] Enforce the rules that an explicit or partial specialization must be
declared before it is used. Because we don't use normal name lookup to find
these, the normal code to filter out non-visible names from name lookup results
does not apply.

llvm-svn: 268585
2016-05-05 00:56:12 +00:00
Ben Langmuir 537c5b5b62 Fix CodeCompletion & TypoCorrection when combining a PCH with Modules
This commit fixes the IdentifierIterator to actually include identifiers
from a PCH or precompiled preamble when there is also a global module
index. This was causing code-completion (outside of C++) and
typo-correction to be missing global identifiers defined in the
PCH/preamble. Typo-correction has been broken since we first started
using the module index, whereas code-completion only started relying on
identifier iterator in r232793.

rdar://problem/25642879

llvm-svn: 268471
2016-05-04 00:53:13 +00:00
David Blaikie af09f4ac72 [modules][debuginfo] Only include imported modules when targeting LLDB
These constructs are only applicable to a debugger capable of loading a
Clang AST, so omit them for brevity when not doing so.

We could potentially propagate more of CodeGenOptions through the
ObjectFilePCGContainerOperations for consistency (so the next person who
adds some CodeGenOpts feature that tweaks debug info output doesn't get
caught by this), so I'm open to objections/alternatives there, but went
with this for now.

Tested just a couple of basic cases (one direct, one indirect (through
the ObjectFilePCHContainerOperations) & fixed up other cases to pass the
-debugger-tuning flag as appropriate.

llvm-svn: 268460
2016-05-03 23:06:40 +00:00
Douglas Yung d58d9fa573 Adding a test for a compiler crash that was fixed in r248069.
Differential Revision: http://reviews.llvm.org/D19048

llvm-svn: 268344
2016-05-03 00:29:56 +00:00
Manman Ren a0f31a01f3 Method Pool in modules: we make sure that if a module contains an entry for
a selector, the entry should be complete, containing everything introduced by
that module and all modules it imports.

Before writing out the method pool of a module, we sync up the out of date
selectors by pulling in methods for the selectors, from all modules it imports.

In ReadMethodPool, after pulling in the method pool entry for module A, this
lets us skip the modules that module A imports.

rdar://problem/25900131

llvm-svn: 268091
2016-04-29 19:04:05 +00:00
Vassil Vassilev 928c8254a9 Reland r267691 fixing PR27535.
llvm-svn: 267882
2016-04-28 14:13:28 +00:00
Richard Smith 4eb8393c63 [modules] When diagnosing a missing module import, suggest adding a #include if
the current language doesn't have an import syntax and we can figure out a
suitable file to include.

llvm-svn: 267802
2016-04-27 21:57:05 +00:00
Nico Weber 3a94763101 Revert r267691, it caused PR27535.
llvm-svn: 267744
2016-04-27 17:26:08 +00:00
Vassil Vassilev a4d7d783d0 [modules] Fix Decl's Used invariant.
The Decl::isUsed has a value for every decl. In non-module builds it is very
difficult (but possible) to break this invariant but when we walk up the redecl
chain we find the neccessary information.

When deserializing the decls from a module it is much more difficult to update
correctly this invariant. The patch centralizes the information whether a decl
is used in the canonical decl marking the entire entity as being used.

Fixes https://llvm.org/bugs/show_bug.cgi?id=27401

Patch by Cristina Cristescu and me.

Thanks to Richard Smith who helped to debug and understand the issue!

Reviewed by Richard Smith.

llvm-svn: 267691
2016-04-27 10:46:06 +00:00
Richard Smith d49941b726 PR27513: When determining which declaration to put into an exported lookup
table for a module / PCH, never map from a normal declaration of a class to an
injected-class-name declaration (or vice versa). Those declarations live in
distinct lookup tables and should not be confused.

We really shouldn't be using a CXXRecordDecl to represent an
injected-class-name in the first place; I've filed PR27532 so we don't forget.

llvm-svn: 267632
2016-04-26 23:40:43 +00:00
Adrian Prantl 0dabc2adba Module debugging: Also correctly handle typedef'd foward-declared members.
Thanks again to Richard Smith for pointing this out.

llvm-svn: 267630
2016-04-26 23:37:38 +00:00
Adrian Prantl 90931c4853 Module debugging: Add testcase for member classes of class template specializations.
llvm-svn: 267612
2016-04-26 21:58:23 +00:00
Adrian Prantl 88d7917970 Module debugging: Use the definition to determine module-defined types.
Follow-up to r267464. Thanks to Richard Smith for pointing this out!

llvm-svn: 267611
2016-04-26 21:58:18 +00:00
Adrian Prantl 05fefa4a85 Module Debugging: Fix the condition for determining whether a template
instantiation is in a module.

This patch fixes the condition for determining whether the debug info for a
template instantiation will exist in an imported clang module by:

- checking whether the ClassTemplateSpecializationDecl is complete and
- checking that the instantiation was in a module by looking at the first field.

I also added a negative check to make sure that a typedef to a forward-declared
template (with the definition outside of the module) is handled correctly.

http://reviews.llvm.org/D19443
rdar://problem/25553724

llvm-svn: 267464
2016-04-25 20:52:40 +00:00
Duncan P. N. Exon Smith 383f8413cf DebugInfo: Adapt to loss of DITypeRef in LLVM r267296
LLVM stopped using MDString-based type references, and DIBuilder no
longer fills 'retainedTypes:' with every DICompositeType that has an
'identifier:' field.   There are just minor changes to keep the same
behaviour in CFE.

Leaving 'retainedTypes:' unfilled has a dramatic impact on the output
order of the IR though.  There are a huge number of testcase changes,
which were unfortunately not really scriptable.

llvm-svn: 267297
2016-04-23 21:08:27 +00:00
Adrian Prantl 50fd1a87c4 Module Debugging: Emit the canonical debug info for Objective-C classes
in the compile unit that contains their implementation even if their
interface is declared in a module.

The private @implementation of an @interface may have additional
hidden ivars so we should not defer to the public version of the
type that is found in the module.

<rdar://problem/25541798>

llvm-svn: 266937
2016-04-20 23:59:32 +00:00
Duncan P. N. Exon Smith f9521b0bb7 DebugInfo: Make DICompositeTypes distinct most of the time
Since elements of most kinds of DICompositeType have back references,
most are involved in uniquing cycles.  Except via the ODR 'identifier:'
field, which doesn't care about the storage type (see r266549),
they have no hope of being uniqued.

Distinct nodes are far more efficient, so use them for most kinds of
DICompositeType definitions (i.e., when DIType::isForwardDecl is false).
The exceptions:

  - DW_TAG_array_type, since their elements never have back-references
    and they never have ODR 'identifier:' fields;

  - DW_TAG_enumeration_type when there is no ODR 'identifier:' field,
    since their elements usually don't have back-references.

This breaks the last major uniquing cycle I'm aware of in the debug info
graph.  The impact won't be enormous for C++ because references to
ODR-uniqued nodes still use string-based DITypeRefs; but this should
prevent a regression in C++ when we drop the string-based references.

This wouldn't have been reasonable until r266549, when composite types
stopped relying on being uniqued by structural equivalence to prevent
blow-ups at LTO time.

llvm-svn: 266556
2016-04-17 07:45:08 +00:00
Adrian Prantl e76bda544b Update to match LLVM changes for PR27284.
(Reverse the ownership between DICompileUnit and DISubprogram.)

http://reviews.llvm.org/D19034
<rdar://problem/25256815>

llvm-svn: 266445
2016-04-15 15:55:45 +00:00
Richard Smith 9c4fb0a833 Fix off-by-one error in worst-case number of offsets needed for an AST record.
llvm-svn: 266353
2016-04-14 18:32:54 +00:00
Bruno Cardoso Lopes 50688beb10 [CrashReproducer] Add test to run the reproducer script + modules
Now that we have basic support in place, make sure the reproducer script
works with modules for a simple testcase.

llvm-svn: 266235
2016-04-13 19:28:24 +00:00
Bruno Cardoso Lopes fc8644cd62 [CrashReproducer] Setup 'use-external-names' in YAML files.
Hide the real paths when rebuilding from VFS by setting up the crash
reproducer to use 'use-external-names' = false. This way we avoid
module redifinition errors and consistently use the same paths against
all modules.

With this change on Darwin we are able to simulate a crash for a simple
application using "Foundation/Foundation.h" (which relies on a bunch of
different frameworks and headers) and successfully rebuild all the
modules by relying solely at the VFS overlay.

llvm-svn: 266234
2016-04-13 19:28:21 +00:00
Richard Smith 6893570853 [modules] Extend r266113 to cope with submodules.
llvm-svn: 266116
2016-04-12 20:20:33 +00:00
Richard Smith 58df343b76 [modules] When an incompatible module file is explicitly provided for a module,
and we fall back to textual inclusion, don't require the module as a whole to
be marked available; it's OK if some other file in the same module is missing,
just as it would be if the header were explicitly marked textual.

llvm-svn: 266113
2016-04-12 19:58:30 +00:00
Richard Smith 1e8e91b379 PR25501: Delay loading visible updates for a declaration until after we've
processed update records. If an update record adds a definition, we need to
merge that with any pre-existing definition to determine which the canonical
definition is before we apply the visible update, otherwise we wouldn't know
where to apply it.

Thanks to Vassil Vassilev for help reducing this and tracking down the problem.

llvm-svn: 265848
2016-04-08 20:53:26 +00:00
Richard Smith fe1bc708b5 [modules] Handle merged fields in designated initializers.
llvm-svn: 265838
2016-04-08 19:57:40 +00:00
Richard Smith 6a6747d9dd [modules] Add a comment to explain why -E leaves some #includes in the preprocessed output.
llvm-svn: 265766
2016-04-08 01:23:59 +00:00
Richard Smith 466a15ef7f [modules] Don't write @import in -E output if the current language mode doesn't
support @import; use the form as written instead.

llvm-svn: 265756
2016-04-08 00:09:53 +00:00
Richard Smith 351241c83e Replace Sema-level implementation of -fassume-sane-operator-new with a
CodeGen-level implementation. Instead of adding an attribute to clang's
FunctionDecl, add the IR attribute directly. This means a module built with
this flag is now compatible with code built without it and vice versa.

This change also results in the 'noalias' attribute no longer being added to
calls to operator new in the IR; it's now only added to the declaration. It
also fixes a bug where we failed to add the attribute to the 'nothrow' versions
(because we didn't implicitly declare them, there was no good time to inject a
fake attribute).

llvm-svn: 265728
2016-04-07 21:46:12 +00:00
Richard Smith a1ddf5e744 [modules] Allow differences in flags that only affect preprocessor predefines
(and __has_feature checks) between explicitly-specified module files and the
current compilation.

llvm-svn: 265718
2016-04-07 20:47:37 +00:00
Bruno Cardoso Lopes 4c20bef1ef [CrashReproducer] Setup 'case-sensitive' in YAML files.
The crash reproducer was not setting up case sensitivity in the
VFS yaml files, which defaults to true. Make the crash reproducer
explicitly set that flag based on the case sensitivity of the .cache
path where vfs and modules are dumped.

llvm-svn: 265622
2016-04-07 00:00:57 +00:00
Manman Ren 8f42e69b18 Keep -fmodule-implementation-of as an alias of -fmodule-name.
This helps us transitioning to -fmodule-name. Once the transitioning is done,
we can remove this alias.

rdar://24800983

llvm-svn: 265616
2016-04-06 23:28:26 +00:00
Vassil Vassilev 59f6586da0 [modules] Add forgotten test case to r265597.
llvm-svn: 265599
2016-04-06 20:58:12 +00:00
Bruno Cardoso Lopes b810b02b85 [CrashReproducer] Pass -I, -F and -resource-dir to the reproducer script when using modules/vfs
The reproducer should use -I/-F/-resource-dir in the same way as the
original command. The VFS already collects the right headers but without
these flags the reproducer will fail to do the right thing.

llvm-svn: 265343
2016-04-04 20:26:57 +00:00
Bruno Cardoso Lopes f854b042e7 [CrashReproducer] Add -fmodule-cache-path to reproducer script
The cc1 invocation in the reproducer script should contain a valid path in
-fmodule-cache-path; for that reuse "<name>.cache/module" dir we already
use to dump the vfs and modules.

llvm-svn: 265162
2016-04-01 17:39:08 +00:00
Bruno Cardoso Lopes e62cfd7c17 [CrashReproducer] Add a module map callback for added headers
The current ModuleDependencyCollector has a AST listener to collect
header files present in loaded modules, but this isn't enough to collect
all headers needed in the crash reproducer. One of the reasons is that
the AST writer doesn't write symbolic link header paths in the pcm modules,
this makes the listeners on the reader only able to collect the real files.

Since the module maps could contain submodules that use headers which
are symbolic links, not collecting those forbid the reproducer scripts
to regen the modules.

For instance:

usr/include/module.map:
  ...
  module pthread {
    header "pthread.h"
    export *

    module impl {
      header "pthread_impl.h"
      export *
    }
  }
  ...

usr/include/pthread/pthread_impl.h
usr/include/pthread_impl.h -> pthread/pthread_impl.h

The AST dump for the module above:

  <SUBMODULE_HEADER abbrevid=6/> blob data = 'pthread_impl.h'
  <SUBMODULE_TOPHEADER abbrevid=7/> blob data = '/<path_to_sdk>/usr/include/pthread/pthread_impl.h'

Note that we don't have "usr/include/pthread_impl.h" which is requested
by the module.map in case we want to reconstruct the module in the
reproducer. The reason the original symbolic link path isn't used is
because the headers are kept by name and requested through the
FileManager, which unique files and returns the real path only.

To fix that, add a callback to be invoked everytime a header is added
while parsing module maps and hook that up to the module dependecy
collector. This callback is only registered when generating the
reproducer.

Differential Revision: http://reviews.llvm.org/D18585

rdar://problem/24499339

llvm-svn: 264971
2016-03-30 23:54:25 +00:00
Bruno Cardoso Lopes d712b34167 [VFS] Handle empty entries in directory traversal
The VFS YAML files contain empty directory entries to describe that it's
returning from a subdirectory before describing new files in the parent.
In the future, we should properly sort and write YAML files avoiding
such empty dirs and mitigate the extra recurson cost. However, since
this is used by previous existing YAMLs, make the traversal work in
their presence.

rdar://problem/24499339

llvm-svn: 264970
2016-03-30 23:54:00 +00:00
Vassil Vassilev 85daf65c5a Add -emit-llvm-only to the regression test for PR21547.
llvm-svn: 264939
2016-03-30 22:22:50 +00:00
Vassil Vassilev bab6f96fff Canonicalize UnaryTransformType types when they don't have a known underlying type.
Fixes https://llvm.org/bugs/show_bug.cgi?id=26014

Reviewed by Richard Smith.

llvm-svn: 264937
2016-03-30 22:18:29 +00:00
Vassil Vassilev 262f41e91c [modules] Write out identifiers if the ID is local, too.
In some cases a slot for an identifier is requested but it gets written to
another module, causing an assertion.

At the point when we start serializing Rtypes, we have no imported IdentifierID
for float_round_style. We start serializing stuff and allocate an ID for it.
Then, during the serialization process, we pull in the identifier info for it
from TSchemaHelper. Finally, WriteIdentifierTable decides that the identifier
has not changed since it was deserialized, so doesn't emit it.

Fixes https://llvm.org/bugs/show_bug.cgi?id=27041

Discussed on IRC with Richard Smith. Agreed on post commit review if needed.

llvm-svn: 264913
2016-03-30 20:16:03 +00:00
Vassil Vassilev a3cbf2a738 [modules] Add a regression test for PR21547.
llvm-svn: 264908
2016-03-30 20:10:07 +00:00
Richard Smith 8a3e39ab67 [modules] If both a module file and a module map for the same module are
explicitly provided, and the module map lists a header that does not exist,
unmark the module as 'unavailable' when loading its .pcm file. (Use of the
module might still fail if the relevant headers aren't embedded, but this
behavior is now consistent with how we behave if the module map is not
provided, and with the desired behavior for embedding headers in modules.)

llvm-svn: 264664
2016-03-28 21:31:09 +00:00
Richard Smith 8d83d6db7c [modules] Don't invent a module cache path unless implicit module builds are enabled.
llvm-svn: 263970
2016-03-21 19:06:06 +00:00
Bruno Cardoso Lopes 936a054522 [VFS] Fix test to use more restrict set of headers
llvm-svn: 263912
2016-03-20 18:08:32 +00:00
Bruno Cardoso Lopes d878e28e67 Reapply [2] [VFS] Add 'overlay-relative' field to YAML files
This reapplies r261552 and r263748. Fixed testcase to reapply.

The VFS overlay mapping between virtual paths and real paths is done through
the 'external-contents' entries in YAML files, which contains hardcoded paths
to the real files.

When a module compilation crashes, headers are dumped into <name>.cache/vfs
directory and are mapped via the <name>.cache/vfs/vfs.yaml. The script
generated for reproduction uses -ivfsoverlay pointing to file to gather the
mapping between virtual paths and files inside <name>.cache/vfs. Currently, we
are only capable of reproducing such crashes in the same machine as they
happen, because of the hardcoded paths in 'external-contents'.

To be able to reproduce a crash in another machine, this patch introduces a new
option in the VFS yaml file called 'overlay-relative'. When it's equal to
'true' it means that the provided path to the YAML file through the
-ivfsoverlay option should also be used to prefix the final path for every
'external-contents'.

Example, given the invocation snippet "... -ivfsoverlay
<name>.cache/vfs/vfs.yaml" and the following entry in the yaml file:

"overlay-relative": "true",
"roots": [
...
  "type": "directory",
  "name": "/usr/include",
  "contents": [
    {
      "type": "file",
      "name": "stdio.h",
      "external-contents": "/usr/include/stdio.h"
    },
...

Here, a file manager request for virtual "/usr/include/stdio.h", that will map
into real path "/<absolute_path_to>/<name>.cache/vfs/usr/include/stdio.h.

This is a useful feature for debugging module crashes in machines other than
the one where the error happened.

Differential Revision: http://reviews.llvm.org/D17457

rdar://problem/24499339

llvm-svn: 263893
2016-03-20 02:08:48 +00:00
Bruno Cardoso Lopes 9e2f9d818f Revert "Reapply [VFS] Add 'overlay-relative' field to YAML files"
Tests failing on
http://bb.pgr.jp/builders/cmake-clang-x86_64-linux/builds/46102

This reverts commit a1683cd6c9e07359c09f86e98a4db6b4e1bc51fc.

llvm-svn: 263750
2016-03-17 21:30:55 +00:00
Bruno Cardoso Lopes 016b2d0ddc Reapply [VFS] Add 'overlay-relative' field to YAML files
This reapplies r261552.

The VFS overlay mapping between virtual paths and real paths is done through
the 'external-contents' entries in YAML files, which contains hardcoded paths
to the real files.

When a module compilation crashes, headers are dumped into <name>.cache/vfs
directory and are mapped via the <name>.cache/vfs/vfs.yaml. The script
generated for reproduction uses -ivfsoverlay pointing to file to gather the
mapping between virtual paths and files inside <name>.cache/vfs. Currently, we
are only capable of reproducing such crashes in the same machine as they
happen, because of the hardcoded paths in 'external-contents'.

To be able to reproduce a crash in another machine, this patch introduces a new
option in the VFS yaml file called 'overlay-relative'. When it's equal to
'true' it means that the provided path to the YAML file through the
-ivfsoverlay option should also be used to prefix the final path for every
'external-contents'.

Example, given the invocation snippet "... -ivfsoverlay
<name>.cache/vfs/vfs.yaml" and the following entry in the yaml file:

"overlay-relative": "true",
"roots": [
...
  "type": "directory",
  "name": "/usr/include",
  "contents": [
    {
      "type": "file",
      "name": "stdio.h",
      "external-contents": "/usr/include/stdio.h"
    },
...

Here, a file manager request for virtual "/usr/include/stdio.h", that will map
into real path "/<absolute_path_to>/<name>.cache/vfs/usr/include/stdio.h.

This is a useful feature for debugging module crashes in machines other than
the one where the error happened.

Differential Revision: http://reviews.llvm.org/D17457

rdar://problem/24499339

llvm-svn: 263748
2016-03-17 21:11:23 +00:00
Benjamin Kramer f8bff1c31c Use a simpler set of mock headers for the vfs+modules crash recovery tests.
The System/ mock is large and too complex for this test. It can cause
the tests to fail in mysterious ways as it depends on the resource dir
being present, which is not really supported for driver tests (using
%clang instead of %clang_cc1). Copy the tree and trim out all the
%unnecessary fat.

llvm-svn: 263718
2016-03-17 16:19:51 +00:00
Bruno Cardoso Lopes b76c027717 Reapply [2]: [VFS] Add support for handling path traversals
This was applied twice r261551 and 263617 and later reverted because:

(1) Windows bot failing on unittests. Change the current behavior to do
not handle path traversals on windows.

(2) Windows bot failed to include llvm/Config/config.h in order to use
HAVE_REALPATH. Use LLVM_ON_UNIX instead, as done in lib/Basic/FileManager.cpp.

Handle ".", ".." and "./" with trailing slashes while collecting files
to be dumped into the vfs overlay directory.

Include the support for symlinks into components. Given the path:

/install-dir/bin/../lib/clang/3.8.0/include/altivec.h, if "bin"
component is a symlink, it's not safe to use `path::remove_dots` here,
and `realpath` is used to get the right answer. Since `realpath`
is expensive, we only do it at collecting time (which only happens
during the crash reproducer) and cache the base directory for fast lookups.

Overall, this makes the input to the VFS YAML file to be canonicalized
to never contain traversal components.

Differential Revision: http://reviews.llvm.org/D17104

rdar://problem/24499339

llvm-svn: 263686
2016-03-17 02:20:43 +00:00
NAKAMURA Takumi 9ed443ccca Revert r263617, "Reapply: [VFS] Add support for handling path traversals"
It broke standalone clang build.

llvm-svn: 263636
2016-03-16 12:15:29 +00:00
Vassil Vassilev 632eac3428 [modules] Fix adding a templated friend functions to a namespace from another module.
When clang adds argument dependent lookup candidates, it can perform template
instantiation. For example, it can instantiate a templated friend function and
register it in the enclosing namespace's lookup table.

Fixes https://llvm.org/bugs/show_bug.cgi?id=24954

Reviewed by Richard Smith.

llvm-svn: 263634
2016-03-16 11:17:04 +00:00
Bruno Cardoso Lopes c9daaaedb6 Reapply: [VFS] Add support for handling path traversals
This is originally r261551, reverted because of windows bots failing on
unittests. Change the current behavior to do not handle path traversals
on windows.

Handle ".", ".." and "./" with trailing slashes while collecting files
to be dumped into the vfs overlay directory.

Include the support for symlinks into components. Given the path:

/install-dir/bin/../lib/clang/3.8.0/include/altivec.h, if "bin"
component is a symlink, it's not safe to use `path::remove_dots` here,
and `realpath` is used to get the right answer. Since `realpath`
is expensive, we only do it at collecting time (which only happens
during the crash reproducer) and cache the base directory for fast lookups.

Overall, this makes the input to the VFS YAML file to be canonicalized
to never contain traversal components.

Differential Revision: http://reviews.llvm.org/D17104

rdar://problem/24499339

llvm-svn: 263617
2016-03-16 04:39:38 +00:00
Richard Smith 8d4e90b319 [modules] Don't diagnose non-modular includes from modular files that are
implementation units of modules rather than interface units.

llvm-svn: 263449
2016-03-14 17:52:37 +00:00
Adrian Prantl 5a9a42778a Module Debugging: Fix a crash when emitting debug info for nested tag types
whose DeclContext is not yet complete by deferring their emission.

rdar://problem/24918680

llvm-svn: 262851
2016-03-07 20:58:52 +00:00
Davide Italiano 5d29dee003 [Modules] Don't swallow errors when parsing optional attributes.
Differential Revision:	http://reviews.llvm.org/D17787

llvm-svn: 262789
2016-03-06 04:20:05 +00:00
Vassil Vassilev 4d75e8d676 [modules] Prefer more complete array types.
If we import a module that has a complete array type and one that has an
incomplete array type, the declaration found by name lookup might be the one with
the incomplete type, possibly resulting in rejects-valid.

Now, the name lookup prefers decls with a complete array types. Also,
diagnose cases when the redecl chain has array bound, different from the merge
candidate.

Reviewed by Richard Smith.

llvm-svn: 262189
2016-02-28 19:08:24 +00:00
Bruno Cardoso Lopes be056b1667 Revert "[VFS] Add support for handling path traversals"
This reverts commit r261551 due to failing tests in windows bots:

http://lab.llvm.org:8011/builders/clang-x64-ninja-win7/builds/10054

Failing Tests (4):
    Clang-Unit :: Basic/BasicTests.exe/VFSFromYAMLTest.CaseInsensitive
    Clang-Unit :: Basic/BasicTests.exe/VFSFromYAMLTest.DirectoryIteration
    Clang-Unit :: Basic/BasicTests.exe/VFSFromYAMLTest.MappedFiles
    Clang-Unit :: Basic/BasicTests.exe/VFSFromYAMLTest.UseExternalName

llvm-svn: 261654
2016-02-23 17:06:50 +00:00
Bruno Cardoso Lopes b7eb8db023 Revert "[VFS] Add 'overlay-relative' field to YAML files" and "[VFS] Fix call to getVFSFromYAML in unittests"
This reverts commit r261552 and r261556 because of failing unittests on
windows:

Failing Tests (4):
    Clang-Unit :: Basic/BasicTests.exe/VFSFromYAMLTest.CaseInsensitive
    Clang-Unit :: Basic/BasicTests.exe/VFSFromYAMLTest.DirectoryIteration
    Clang-Unit :: Basic/BasicTests.exe/VFSFromYAMLTest.MappedFiles
    Clang-Unit :: Basic/BasicTests.exe/VFSFromYAMLTest.UseExternalName

llvm-svn: 261613
2016-02-23 07:06:12 +00:00
Bruno Cardoso Lopes 8c03c54200 [VFS] Add 'overlay-relative' field to YAML files
The VFS overlay mapping between virtual paths and real paths is done through
the 'external-contents' entries in YAML files, which contains hardcoded paths
to the real files.

When a module compilation crashes, headers are dumped into <name>.cache/vfs
directory and are mapped via the <name>.cache/vfs/vfs.yaml. The script
generated for reproduction uses -ivfsoverlay pointing to file to gather the
mapping between virtual paths and files inside <name>.cache/vfs. Currently, we
are only capable of reproducing such crashes in the same machine as they
happen, because of the hardcoded paths in 'external-contents'.

To be able to reproduce a crash in another machine, this patch introduces a new
option in the VFS yaml file called 'overlay-relative'. When it's equal to
'true' it means that the provided path to the YAML file through the
-ivfsoverlay option should also be used to prefix the final path for every
'external-contents'.

Example, given the invocation snippet "... -ivfsoverlay
<name>.cache/vfs/vfs.yaml" and the following entry in the yaml file:

"overlay-relative": "true",
"roots": [
...
  "type": "directory",
  "name": "/usr/include",
  "contents": [
    {
      "type": "file",
      "name": "stdio.h",
      "external-contents": "/usr/include/stdio.h"
    },
...

Here, a file manager request for virtual "/usr/include/stdio.h", that will map
into real path "/<absolute_path_to>/<name>.cache/vfs/usr/include/stdio.h.

This is a useful feature for debugging module crashes in machines other than
the one where the error happened.

Differential Revision: http://reviews.llvm.org/D17457

rdar://problem/24499339

llvm-svn: 261552
2016-02-22 18:41:09 +00:00
Bruno Cardoso Lopes 956e6a0dc6 [VFS] Add support for handling path traversals
Handle ".", ".." and "./" with trailing slashes while collecting files
to be dumped into the vfs overlay directory.

Include the support for symlinks into components. Given the path:

/install-dir/bin/../lib/clang/3.8.0/include/altivec.h, if "bin"
component is a symlink, it's not safe to use `path::remove_dots` here,
and `realpath` is used to get the right answer. Since `realpath`
is expensive, we only do it at collecting time (which only happens
during the crash reproducer) and cache the base directory for fast lookups.

Overall, this makes the input to the VFS YAML file to be canonicalized
to never contain traversal components.

Differential Revision: http://reviews.llvm.org/D17104

rdar://problem/24499339

llvm-svn: 261551
2016-02-22 18:41:01 +00:00
Richard Smith 7e82e019c6 [modules] Flatten -fmodule-name= and -fmodule-implementation-of= into a single
option. Previously these options could both be used to specify that you were
compiling the implementation file of a module, with a different set of minor
bugs in each case.

This change removes -fmodule-implementation-of, and instead tracks a flag to
determine whether we're currently building a module. -fmodule-name now behaves
the same way that -fmodule-implementation-of previously did.

llvm-svn: 261372
2016-02-19 22:25:36 +00:00
Ben Langmuir 9b1e442e67 [Modules] Early-exit if ReadOptionsBlock fails to avoid crashing
If we didn't tell ReadOptionsBlock to allow failures then we can't
assume that the stream is not in the middle of a block if it returns
out-of-date. This was causing a crash when we tried to continue reading.

Also, it's just generally a good idea to early-exit if we're doing
implicit module builds, since we will want to immediately rebuild this
module anyway and there's no reason to waste time continuing after
failure.

rdar://problem/24114938

llvm-svn: 260563
2016-02-11 18:54:02 +00:00
Justin Lebar af94c0f2b3 Remove some unnecessary CHECK-SAMEs in ModuleDebugInfo.{cpp,m}.
Suggested by Paul Robinson.

llvm-svn: 260346
2016-02-10 01:34:17 +00:00
Justin Lebar ae06ca0c7e Get rid of CHECK-SAME-NOT in tests.
Summary: This isn't a FileCheck directive; it does nothing.

Reviewers: jroelofs

Subscribers: cfe-commits, majnemer

Differential Revision: http://reviews.llvm.org/D17051

llvm-svn: 260334
2016-02-10 00:38:15 +00:00
Richard Smith aada85c5f7 [modules] Compress files embedded into a .pcm file, to reduce the disk usage of -fembed-all-files mode.
llvm-svn: 259976
2016-02-06 02:06:43 +00:00
Richard Smith d79514e24b [modules] Separately track whether an identifier's preprocessor information and
name lookup information have changed since deserialization. For a C++ modules
build, we do not need to re-emit the identifier into the serialized identifier
table if only the name lookup information has changed (and in all cases, we
don't need to re-emit the macro information if only the name lookup information
has changed).

llvm-svn: 259901
2016-02-05 19:03:40 +00:00
Ben Langmuir e7d67575f2 Don't synthesize an ImportDecl for a module named in -fmodule-implementation-of
When building a PCH with modules enabled this import would assert in the
ASTWriter and (if assertions were disabled) sometimes crash the compiler
that loaded the resulting PCH when trying to lookup the submodule ID.

rdar://problem/24137448

llvm-svn: 259859
2016-02-05 01:10:05 +00:00
Ben Langmuir f5416740fc Fix predefine for __NSConstantString struct type
Per review feedback the name was wrong and it can be used outside
Objective-C.

Unfortunately, making the internal struct visible broke some ASTMatchers
tests that assumed that the first record decl would be from user code,
rather than a builtin type.  I'm worried that this will also affect
users' code.  So this patch adds a typedef to wrap the internal struct
and only makes the typedef visible to namelookup.  This is sufficient to
allow the ASTReader to merge the decls we need without making the struct
itself visible.

rdar://problem/24425801

llvm-svn: 259734
2016-02-04 00:55:24 +00:00
Quentin Colombet 043406b87f Reapply r259624, it is likely not the commit causing the bot failures.
Original message:
Make CF constant string decl visible to name lookup to fix module errors

The return type of the __builtin___*StringMakeConstantString functions
is a pointer to a struct, so we need that struct to be visible to name
lookup so that we will correctly merge multiple declarations of that
type if they come from different modules.

Incidentally, to make this visible to name lookup we need to rename the
type to __NSConstantString, since the real NSConstantString is an
Objective-C interface type.  This shouldn't affect anyone outside the
compiler since users of the constant string builtins cast the result
immediately to CFStringRef.

Since this struct type is otherwise implicitly created by the AST
context and cannot access namelookup, we make this a predefined type
and initialize it in Sema.

Note: this issue of builtins that refer to types not visible to name
lookup technically also affects other builtins (e.g. objc_msgSendSuper),
but in all other cases the builtin is a library builtin and the issue
goes away if you include the library that defines the types it uses,
unlike for these constant string builtins.

rdar://problem/24425801

llvm-svn: 259721
2016-02-03 22:41:00 +00:00
Quentin Colombet 0cdb86bd38 Revert r259624 - Make CF constant string decl visible to name lookup to fix module errors.
This breaks some internal bots in stage2: clang seg fault.
Looking with Ben to see what is going on.

llvm-svn: 259715
2016-02-03 22:14:53 +00:00
Ben Langmuir 6a34e10514 Make CF constant string decl visible to name lookup to fix module errors
The return type of the __builtin___*StringMakeConstantString functions
is a pointer to a struct, so we need that struct to be visible to name
lookup so that we will correctly merge multiple declarations of that
type if they come from different modules.

Incidentally, to make this visible to name lookup we need to rename the
type to __NSConstantString, since the real NSConstantString is an
Objective-C interface type.  This shouldn't affect anyone outside the
compiler since users of the constant string builtins cast the result
immediately to CFStringRef.

Since this struct type is otherwise implicitly created by the AST
context and cannot access namelookup, we make this a predefined type
and initialize it in Sema.

Note: this issue of builtins that refer to types not visible to name
lookup technically also affects other builtins (e.g. objc_msgSendSuper),
but in all other cases the builtin is a library builtin and the issue
goes away if you include the library that defines the types it uses,
unlike for these constant string builtins.

rdar://problem/24425801

llvm-svn: 259624
2016-02-03 03:26:19 +00:00
Adrian Prantl 3a2d4947a6 Module Debugging: Canonicalize the file names used as PCH module names
by stripping the path. Follow-up to r258555.

This is safe because only one PCH per CU is currently supported for
module debugging.

rdar://problem/24301262

llvm-svn: 258582
2016-01-22 23:30:56 +00:00
Adrian Prantl aa5d08d04b Module debugging: Create a parent DIModule with the PCH name for types
emitted into a precompiled header to mirror the debug info emitted for
object files importing the PCH.

rdar://problem/24290667

llvm-svn: 258555
2016-01-22 21:14:41 +00:00
Adrian Prantl d4e73e7412 Rephrase this test to help debug a buildbot issue
llvm-svn: 258526
2016-01-22 19:14:24 +00:00
Yaron Keren e94da64e65 Merge templated static member variables, fixes http://llvm.org/pr26179.
Patch by Vassil Vassilev!
Reviewed by Richard Smith.

llvm-svn: 258524
2016-01-22 19:03:27 +00:00
Adrian Prantl ae108c4704 Fix a typo in r258507 and change the PCH dwoid constant to ~1UL.
rdar://problem/24290667

llvm-svn: 258519
2016-01-22 18:46:45 +00:00
Adrian Prantl e1c2ad8aa7 Fix 80-column violations.
llvm-svn: 258518
2016-01-22 18:46:40 +00:00
Adrian Prantl c96da8faa6 Module Debugging: Use a nonzero DWO id for precompiled headers.
PCH files don't have a module signature and LLVM uses a nonzero DWO id as
an indicator for skeleton / module CUs. This change pins the DWO id for PCH
files to a known constant value.
The correct long-term solution here is to implement a module signature
that is an actual dterministic hash (at the moment module signatures are
just random nonzero numbers) and then enable this for PCH files as well.

<rdar://problem/24290667>

llvm-svn: 258507
2016-01-22 17:43:43 +00:00
Adrian Prantl 8f55b66a53 Module Debugging: Fine-tune the condition that determines whether a type
can be found in a module.

There are externally visible anonymous types that can be found:
  typedef struct { } s; // I can be found via the typedef.
There are anonymous internal types that can be found:
  namespace { struct s {}; } // I can be found by name.

rdar://problem/24199640

llvm-svn: 258272
2016-01-20 01:29:34 +00:00
Adrian Prantl 8c0e50519b Module Debugging: Add Objective-C testcases for anonymous tag decls. (NFC)
rdar://problem/24199640

llvm-svn: 258252
2016-01-19 23:42:56 +00:00
Adrian Prantl 43e0081740 Module Debugging: Don't emit external type references to anonymous types.
Even if they exist in the module, they can't be matched with the forward
declaration in the object file.
<rdar://problem/24199640>

llvm-svn: 258251
2016-01-19 23:42:53 +00:00
Adrian Prantl cd975018fe Module Debugging: Make sure that anonymous tag decls that define global
variables are visited.

This shouldn't encourage anyone to put global variables into clang modules.
rdar://problem/24199640

llvm-svn: 258250
2016-01-19 23:42:44 +00:00
Adrian Prantl e5238d2a82 Module Debugging: Defer the emission of anonymous tag decls
until we are visiting their declcontext.

This fixes a regression introduced in r256962:
When building debug info for a typdef'd anonymous tag type, we would be
visiting the inner anonymous type first thus creating a "typedef changes
linkage of anonymous type, but linkage was already computed" error.

rdar://problem/24199640

llvm-svn: 258152
2016-01-19 18:02:47 +00:00
Richard Smith 2de44e658d [modules] Don't diagnose a conflict between two using-declarations that name equivalent internal linkage entities.
llvm-svn: 257512
2016-01-12 20:34:32 +00:00
Richard Smith 38c202e97e When a tag is declared in prototype scope in C, if we've decided that it
redeclares an existing tag but are creating a new declaration anyway (because
it has attributes or changes the visibility of the name), don't warn that it
won't be visible outside the current scope. That's not true.

Also narrow down the set of cases where we create these extra declarations when
building modules; previously, all tag declarations but the first in a module
header would get this treatment if -fmodules-local-submodule-visibility. (This
isn't a functional change, but we try to avoid creating these extra
declarations whenever we can.)

llvm-svn: 257403
2016-01-11 22:41:53 +00:00
Richard Smith e123cd2f40 [modules] If we're treating an elaborated-type-specifier as if it introduces a
tag (because the previous declaration was found in a different module), inject
the tag into the appropriate scope (that is, the enclosing scope if we're in a
function prototype scope in C++).

llvm-svn: 257251
2016-01-09 06:58:48 +00:00
Adrian Prantl 35e765baff Module debugging: Add a testcase for standalone forward declarations.
llvm-svn: 257241
2016-01-09 01:11:35 +00:00
Richard Smith f7aeda1f07 [modules] Make sure we always include the contents of private headers when
building a module. Prior to this change, the private header's content would
only be included if the header were included by another header in the same
module. If not (if the private header is only used by the .cc files of the
module, or is included from outside the module via -Wno-private-header),
a #include of that file would be silently ignored.

llvm-svn: 257222
2016-01-08 22:36:45 +00:00
Adrian Prantl b2a8b35349 Fix a typo.
llvm-svn: 256963
2016-01-06 19:22:23 +00:00
Adrian Prantl b3b821f1df Module debugging: Defer emitting tag types until their definition
was visited and all decls have been merged.

We only get a single chance to emit the types for virtual classes because
CGDebugInfo::completeRequiredType() categorically doesn't complete them.

llvm-svn: 256962
2016-01-06 19:22:19 +00:00
Richard Smith 26dfbace82 [modules] When a tag type that was imported from a module is referenced via an
elaborated-type-specifier, create a declaration of it to track that the current
module makes it visible too.

llvm-svn: 256907
2016-01-06 03:52:10 +00:00
Adrian Prantl aac97c93ac Fix a typo in testcase and increase its coverage!
llvm-svn: 256874
2016-01-05 23:54:01 +00:00
Richard Smith 82b8d4e6fd [modules] Don't try to use the definition of a class if
RequireCompleteType(..., 0) says we're not permitted to do so. The definition
might not be visible, even though we know what it is.

llvm-svn: 256045
2015-12-18 22:19:11 +00:00
Richard Smith 84824edd0b Allow non-defining declarations of class template partial specializations to
have a nested name specifier. Strictly speaking, forward declarations of class
template partial specializations are not permitted at all, but that seems like
an obvious wording defect, and if we allow them without a nested name specifier
we should also allow them with a nested name specifier.

llvm-svn: 255383
2015-12-11 22:39:52 +00:00
Ben Langmuir 237ccb165e Reapply "[Modules] Fix regression when an elaborated-type-specifier mentions a hidden tag"
Now not trying to use a C++ lookup mechanism in C (d'oh).  Unqualified
lookup is actually fine for this case in C.

llvm-svn: 255377
2015-12-11 22:05:13 +00:00
Ben Langmuir 38f2bfbbe4 Revert "[Modules] Fix regression when an elaborated-type-specifier mentions a hidden tag"
This is causing assertion failures; reverting until I can fix.

This reverts commit r255267

llvm-svn: 255324
2015-12-11 01:44:43 +00:00