Without this patch, clang will not wrap in an ElaboratedType node types written
without a keyword and nested name qualifier, which goes against the intent that
we should produce an AST which retains enough details to recover how things are
written.
The lack of this sugar is incompatible with the intent of the type printer
default policy, which is to print types as written, but to fall back and print
them fully qualified when they are desugared.
An ElaboratedTypeLoc without keyword / NNS uses no storage by itself, but still
requires pointer alignment due to pre-existing bug in the TypeLoc buffer
handling.
---
Troubleshooting list to deal with any breakage seen with this patch:
1) The most likely effect one would see by this patch is a change in how
a type is printed. The type printer will, by design and default,
print types as written. There are customization options there, but
not that many, and they mainly apply to how to print a type that we
somehow failed to track how it was written. This patch fixes a
problem where we failed to distinguish between a type
that was written without any elaborated-type qualifiers,
such as a 'struct'/'class' tags and name spacifiers such as 'std::',
and one that has been stripped of any 'metadata' that identifies such,
the so called canonical types.
Example:
```
namespace foo {
struct A {};
A a;
};
```
If one were to print the type of `foo::a`, prior to this patch, this
would result in `foo::A`. This is how the type printer would have,
by default, printed the canonical type of A as well.
As soon as you add any name qualifiers to A, the type printer would
suddenly start accurately printing the type as written. This patch
will make it print it accurately even when written without
qualifiers, so we will just print `A` for the initial example, as
the user did not really write that `foo::` namespace qualifier.
2) This patch could expose a bug in some AST matcher. Matching types
is harder to get right when there is sugar involved. For example,
if you want to match a type against being a pointer to some type A,
then you have to account for getting a type that is sugar for a
pointer to A, or being a pointer to sugar to A, or both! Usually
you would get the second part wrong, and this would work for a
very simple test where you don't use any name qualifiers, but
you would discover is broken when you do. The usual fix is to
either use the matcher which strips sugar, which is annoying
to use as for example if you match an N level pointer, you have
to put N+1 such matchers in there, beginning to end and between
all those levels. But in a lot of cases, if the property you want
to match is present in the canonical type, it's easier and faster
to just match on that... This goes with what is said in 1), if
you want to match against the name of a type, and you want
the name string to be something stable, perhaps matching on
the name of the canonical type is the better choice.
3) This patch could expose a bug in how you get the source range of some
TypeLoc. For some reason, a lot of code is using getLocalSourceRange(),
which only looks at the given TypeLoc node. This patch introduces a new,
and more common TypeLoc node which contains no source locations on itself.
This is not an inovation here, and some other, more rare TypeLoc nodes could
also have this property, but if you use getLocalSourceRange on them, it's not
going to return any valid locations, because it doesn't have any. The right fix
here is to always use getSourceRange() or getBeginLoc/getEndLoc which will dive
into the inner TypeLoc to get the source range if it doesn't find it on the
top level one. You can use getLocalSourceRange if you are really into
micro-optimizations and you have some outside knowledge that the TypeLocs you are
dealing with will always include some source location.
4) Exposed a bug somewhere in the use of the normal clang type class API, where you
have some type, you want to see if that type is some particular kind, you try a
`dyn_cast` such as `dyn_cast<TypedefType>` and that fails because now you have an
ElaboratedType which has a TypeDefType inside of it, which is what you wanted to match.
Again, like 2), this would usually have been tested poorly with some simple tests with
no qualifications, and would have been broken had there been any other kind of type sugar,
be it an ElaboratedType or a TemplateSpecializationType or a SubstTemplateParmType.
The usual fix here is to use `getAs` instead of `dyn_cast`, which will look deeper
into the type. Or use `getAsAdjusted` when dealing with TypeLocs.
For some reason the API is inconsistent there and on TypeLocs getAs behaves like a dyn_cast.
5) It could be a bug in this patch perhaps.
Let me know if you need any help!
Signed-off-by: Matheus Izvekov <mizvekov@gmail.com>
Differential Revision: https://reviews.llvm.org/D112374
Clang has traditionally allowed C programs to implicitly convert
integers to pointers and pointers to integers, despite it not being
valid to do so except under special circumstances (like converting the
integer 0, which is the null pointer constant, to a pointer). In C89,
this would result in undefined behavior per 3.3.4, and in C99 this rule
was strengthened to be a constraint violation instead. Constraint
violations are most often handled as an error.
This patch changes the warning to default to an error in all C modes
(it is already an error in C++). This gives us better security posture
by calling out potential programmer mistakes in code but still allows
users who need this behavior to use -Wno-error=int-conversion to retain
the warning behavior, or -Wno-int-conversion to silence the diagnostic
entirely.
Differential Revision: https://reviews.llvm.org/D129881
This reverts commit 7c51f02eff because it
stills breaks the LLDB tests. This was re-landed without addressing the
issue or even agreement on how to address the issue. More details and
discussion in https://reviews.llvm.org/D112374.
Without this patch, clang will not wrap in an ElaboratedType node types written
without a keyword and nested name qualifier, which goes against the intent that
we should produce an AST which retains enough details to recover how things are
written.
The lack of this sugar is incompatible with the intent of the type printer
default policy, which is to print types as written, but to fall back and print
them fully qualified when they are desugared.
An ElaboratedTypeLoc without keyword / NNS uses no storage by itself, but still
requires pointer alignment due to pre-existing bug in the TypeLoc buffer
handling.
---
Troubleshooting list to deal with any breakage seen with this patch:
1) The most likely effect one would see by this patch is a change in how
a type is printed. The type printer will, by design and default,
print types as written. There are customization options there, but
not that many, and they mainly apply to how to print a type that we
somehow failed to track how it was written. This patch fixes a
problem where we failed to distinguish between a type
that was written without any elaborated-type qualifiers,
such as a 'struct'/'class' tags and name spacifiers such as 'std::',
and one that has been stripped of any 'metadata' that identifies such,
the so called canonical types.
Example:
```
namespace foo {
struct A {};
A a;
};
```
If one were to print the type of `foo::a`, prior to this patch, this
would result in `foo::A`. This is how the type printer would have,
by default, printed the canonical type of A as well.
As soon as you add any name qualifiers to A, the type printer would
suddenly start accurately printing the type as written. This patch
will make it print it accurately even when written without
qualifiers, so we will just print `A` for the initial example, as
the user did not really write that `foo::` namespace qualifier.
2) This patch could expose a bug in some AST matcher. Matching types
is harder to get right when there is sugar involved. For example,
if you want to match a type against being a pointer to some type A,
then you have to account for getting a type that is sugar for a
pointer to A, or being a pointer to sugar to A, or both! Usually
you would get the second part wrong, and this would work for a
very simple test where you don't use any name qualifiers, but
you would discover is broken when you do. The usual fix is to
either use the matcher which strips sugar, which is annoying
to use as for example if you match an N level pointer, you have
to put N+1 such matchers in there, beginning to end and between
all those levels. But in a lot of cases, if the property you want
to match is present in the canonical type, it's easier and faster
to just match on that... This goes with what is said in 1), if
you want to match against the name of a type, and you want
the name string to be something stable, perhaps matching on
the name of the canonical type is the better choice.
3) This patch could exposed a bug in how you get the source range of some
TypeLoc. For some reason, a lot of code is using getLocalSourceRange(),
which only looks at the given TypeLoc node. This patch introduces a new,
and more common TypeLoc node which contains no source locations on itself.
This is not an inovation here, and some other, more rare TypeLoc nodes could
also have this property, but if you use getLocalSourceRange on them, it's not
going to return any valid locations, because it doesn't have any. The right fix
here is to always use getSourceRange() or getBeginLoc/getEndLoc which will dive
into the inner TypeLoc to get the source range if it doesn't find it on the
top level one. You can use getLocalSourceRange if you are really into
micro-optimizations and you have some outside knowledge that the TypeLocs you are
dealing with will always include some source location.
4) Exposed a bug somewhere in the use of the normal clang type class API, where you
have some type, you want to see if that type is some particular kind, you try a
`dyn_cast` such as `dyn_cast<TypedefType>` and that fails because now you have an
ElaboratedType which has a TypeDefType inside of it, which is what you wanted to match.
Again, like 2), this would usually have been tested poorly with some simple tests with
no qualifications, and would have been broken had there been any other kind of type sugar,
be it an ElaboratedType or a TemplateSpecializationType or a SubstTemplateParmType.
The usual fix here is to use `getAs` instead of `dyn_cast`, which will look deeper
into the type. Or use `getAsAdjusted` when dealing with TypeLocs.
For some reason the API is inconsistent there and on TypeLocs getAs behaves like a dyn_cast.
5) It could be a bug in this patch perhaps.
Let me know if you need any help!
Signed-off-by: Matheus Izvekov <mizvekov@gmail.com>
Differential Revision: https://reviews.llvm.org/D112374
This reverts commit bdc6974f92 because it
breaks all the LLDB tests that import the std module.
import-std-module/array.TestArrayFromStdModule.py
import-std-module/deque-basic.TestDequeFromStdModule.py
import-std-module/deque-dbg-info-content.TestDbgInfoContentDequeFromStdModule.py
import-std-module/forward_list.TestForwardListFromStdModule.py
import-std-module/forward_list-dbg-info-content.TestDbgInfoContentForwardListFromStdModule.py
import-std-module/list.TestListFromStdModule.py
import-std-module/list-dbg-info-content.TestDbgInfoContentListFromStdModule.py
import-std-module/queue.TestQueueFromStdModule.py
import-std-module/stack.TestStackFromStdModule.py
import-std-module/vector.TestVectorFromStdModule.py
import-std-module/vector-bool.TestVectorBoolFromStdModule.py
import-std-module/vector-dbg-info-content.TestDbgInfoContentVectorFromStdModule.py
import-std-module/vector-of-vectors.TestVectorOfVectorsFromStdModule.py
https://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake/45301/
Without this patch, clang will not wrap in an ElaboratedType node types written
without a keyword and nested name qualifier, which goes against the intent that
we should produce an AST which retains enough details to recover how things are
written.
The lack of this sugar is incompatible with the intent of the type printer
default policy, which is to print types as written, but to fall back and print
them fully qualified when they are desugared.
An ElaboratedTypeLoc without keyword / NNS uses no storage by itself, but still
requires pointer alignment due to pre-existing bug in the TypeLoc buffer
handling.
Signed-off-by: Matheus Izvekov <mizvekov@gmail.com>
Differential Revision: https://reviews.llvm.org/D112374
Based on post-commit review discussion on
2bd8493847 with Richard Smith.
Other uses of forcing HasEmptyPlaceHolder to false seem OK to me -
they're all around pointer/reference types where the pointer/reference
token will appear at the rightmost side of the left side of the type
name, so they make nested types (eg: the "int" in "int *") behave as
though there is a non-empty placeholder (because the "*" is essentially
the placeholder as far as the "int" is concerned).
This was originally committed in 277623f4d5
Reverted in f9ad1d1c77 due to breakages
outside of clang - lldb seems to have some strange/strong dependence on
"char [N]" versus "char[N]" when printing strings (not due to that name
appearing in DWARF, but probably due to using clang to stringify type
names) that'll need to be addressed, plus a few other odds and ends in
other subprojects (clang-tools-extra, compiler-rt, etc).
Fixes observed warning running `ninja check-all`:
llvm-project/clang/bindings/python/tests/cindex/test_diagnostics.py💯
DeprecationWarning: Please use assertRegex instead.
self.assertRegexpMatches(children[0].spelling
Looks like unittest.assertRegexpMatches has been deprecated in favor of
unittest.assertRegex since Python 3.2, according to:
https://docs.python.org/3/library/unittest.html#deprecated-aliases
Reviewed By: MaskRay
Differential Revision: https://reviews.llvm.org/D85692
Summary:
Wraps JSON compilation database with a target and mode adding database
wrapper. So that driver can correctly figure out which toolchain to use.
Note that clients that wants to make use of this target discovery mechanism
needs to link in TargetsInfos and initialize them at startup.
Reviewers: ilya-biryukov
Subscribers: mgorny, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D63755
llvm-svn: 364386
check-all invokes check-clang-python which prints the annoying message:
LIBCLANG TOOLING ERROR: fixed-compilation-database: Error while opening fixed database: No such file or directory
json-compilation-database: Error while opening JSON database: No such file or directory
Let's fix it now with os.dup os.dup2 trick.
llvm-svn: 357562
StringIO is obsoleted in Python3, replaced by io.BytesIO or io.StringIO depending on the use.
Differential Revision: https://reviews.llvm.org/D55196
llvm-svn: 350318
This reverts commit r349064.
This wasn't updating the right test. Causing (not the different line number
from the previous revert):
======================================================================
FAIL: test_diagnostic_warning (tests.cindex.test_diagnostics.TestDiagnostics)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/buildslave/jenkins/workspace/clang-stage1-configure-RA/llvm/tools/clang/bindings/python/tests/cindex/test_diagnostics.py", line 18, in test_diagnostic_warning
self.assertEqual(len(tu.diagnostics), 2)
AssertionError: 1 != 2
llvm-svn: 349118
Python 3.6 introduced a file system path protocol (PEP 519[1]).
The standard library APIs accepting file system paths now accept path
objects too. It could be useful to add this here as well
for convenience.
[1] https://www.python.org/dev/peps/pep-0519
Authored by: jstasiak (Jakub Stasiak)
Differential Revision: https://reviews.llvm.org/D54120
llvm-svn: 346586
Remove the test checking for compilation db lookup failure.
Since r342228, JSONCompilationDatabasePlugin infers compile commands for
missing files, therefore making the lookup always succeed.
Differential Revision: https://reviews.llvm.org/D53202
llvm-svn: 344364
Support a new CLANG_LIBRARY_PATH environment variable for the Python
binding tests. This variable can be used to force the bindings to load
libclang.* from a specific directory.
I plan to use this when integrating Python binding tests with the CMake
build system. Currently, those tests load libclang.so from default
search paths, so I would have to rely on platform-specific mechanics
such as LD_LIBRARY_PATH. Instead of copying the whole logic necessary
to handle platform differences into yet another place, it's easier to
just add a dedicated variable for this purpose.
Differential Revision: https://reviews.llvm.org/D52806
llvm-svn: 344240
Expose the C bindings for clang_Type_getNumTemplateArguments() and
clang_Type_getTemplateArgumentAsType() in the python API.
Patch by kjteske (Kyle Teske).
Reviewed By: jbcoe
Subscribers: cfe-commits
Differential Revision: https://reviews.llvm.org/D51299
llvm-svn: 341930
Summary:
This fixes all but one of the test cases for Windows. TestCDB will
take more work to debug, as CompilationDatabase seems not to work correctly.
Reviewers: bkramer, wanders, jbcoe
Reviewed By: bkramer, jbcoe
Subscribers: cfe-commits
Differential Revision: https://reviews.llvm.org/D47864
Patch written by ethanhs (Ethan)
llvm-svn: 335282
Summary:
adding function: `Cursor.get_included_file` , so the C API's `clang_getIncludedFile` function is available on the python binding interface
also adding test to unittests
related ticket: https://bugs.llvm.org/show_bug.cgi?id=15223
Reviewers: mgorny, arphaman, jbcoe
Reviewed By: jbcoe
Subscribers: cfe-commits
Differential Revision: https://reviews.llvm.org/D46383
Patch by jlaz (József Láz)
llvm-svn: 332045
This patch allows checking whether a C++ record declaration is abstract through
libclang and clang.cindex (Python).
Patch by Johann Klähn!
Differential Revision: https://reviews.llvm.org/D36952
llvm-svn: 320748
Starting with r314037, anonymous namespaces no longer give
unique-external linkage to variables. However, this linkage can still be
achieved by using a type which is not exterally visible,
e.g. through being declared in an anonymous namespace but used outside
it. Fix the test to take advantage of that.
Differential Revision: https://reviews.llvm.org/D39810
llvm-svn: 317986
Rewrite the tests from using plain 'assert' mixed with some nosetests
methods to the standard unittest module layout. Improve the code
to use the most canonical assertion methods whenever possible.
This has a few major advantages:
- the code uses standard methods now, resulting in a reduced number
of WTFs whenever someone with basic Python knowledge gets to read it,
- completely unnecessary dependency on nosetests is removed since
the standard library supplies all that is necessary for the tests
to run,
- the tests can be run via any test runner, including the one built-in
in Python,
- the failure output for most of the tests is improved from 'assertion
x == y failed' to actually telling the values.
Differential Revision: https://reviews.llvm.org/D39763
llvm-svn: 317897
The priority for destructors and operators was reduced in r314019.
Adjust the values used in the test appropriately to fix the test
failure.
Differential Revision: https://reviews.llvm.org/D39838
llvm-svn: 317828
Since cfe commit r237337, '__declspec(thread)' and 'thread_local' have
been the same since MSVC 2015. i.e. they are both considered to supply
a dynamic TLS kind, not a static TLS kind.
This test originally did not specify which version of MS compatibility
to assume. As a result, the test was brittle, since changing the
default compatibility version could break the test.
This commit adds a specific version when building up the flags used to
parse the translation unit, and tests both versions.
llvm-svn: 317706
o) Add a 'Location' class that represents the four properties of a
physical location
o) Enhance 'SourceLocation' to provide 'expansion' and 'spelling'
locations, maintaining backwards compatibility with existing code by
forwarding the four properties to 'expansion'.
o) Update the implementation to use 'clang_getExpansionLocation'
instead of the deprecated 'clang_getInstantiationLocation', which
has been present since 2011.
o) Update the implementation of 'clang_getSpellingLocation' to actually
obtain spelling location instead of file location.
llvm-svn: 316278
Some API calls accept 'NULL' instead of a char array (e.g. the second
argument to 'clang_ParseTranslationUnit'). For Python 3 compatibility,
all strings are passed through 'c_interop_string' which expects to
receive only 'bytes' or 'str' objects. This change extends this
behavior to additionally allow 'None' to be supplied.
A test case was added which breaks in Python 3, and is fixed by this
change. All the test cases pass in both, Python 2 and Python 3.
llvm-svn: 316264
Summary:
This patch allows checking the availability of cursors through libclang and clang.cindex (Python).
This e.g. allows to check whether a C++ member function has been marked as deleted.
Reviewers: arphaman, jbcoe
Reviewed By: jbcoe
Subscribers: cfe-commits
Tags: #clang
Patch by jklaehn (Johann Klähn)
Differential Revision: https://reviews.llvm.org/D36973
llvm-svn: 315959
Summary: Previously, `VisitAttributes` was not called for function and class templates and thus their attributes were not accessible using libclang.
Reviewers: bkramer, arphaman, rsmith, jbcoe
Reviewed By: jbcoe
Subscribers: cfe-commits
Tags: #clang
Patch by jklaehn (Johann Klähn)
Differential Revision: https://reviews.llvm.org/D36955
llvm-svn: 315958
Summary:
Previously the `_tu` was not propagated to the returned cursor, leading to errors when calling any
method on that cursor (e.g. `cursor.referenced`).
Reviewers: jbcoe, rsmith
Reviewed By: jbcoe
Subscribers: cfe-commits
Tags: #clang
Patch by jklaehn (Johann Klähn)
Differential Revision: https://reviews.llvm.org/D36953
llvm-svn: 313913
Introduce the 'TLS Kind' property of variable declarations through
libclang. Additionally, provide a Python accessor for it, and test that
functionality.
Patch by Masud Rahman!
llvm-svn: 313111
This commit allows checking whether an enum declaration is scoped
through libclang and clang.cindex (Python).
Patch by Johann Klähn!
Differential Revision: https://reviews.llvm.org/D35187
llvm-svn: 307771
This commit allows checking whether an enum declaration is scoped
through libclang and clang.cindex (Python).
Differential Revision: https://reviews.llvm.org/D35187
llvm-svn: 307769
Expose the following functions:
- clang_getTypedefName
- clang_getAddressSpace
Patch by Simon Perretta.
Differential Revision: https://reviews.llvm.org/D33598
llvm-svn: 304978
Summary:
Introduce an interop string to convert from unicode to c-strings where needed.
Add missing conversions from _CXString to strings in function registrations.
Explicitly evaluate lists where Python 3's lazy iterators would not otherwise do so.
This is an improvement upon the reverted change proposed in https://reviews.llvm.org/D26082
Reviewers: compnerd, skalinichev, modocache, MathieuDuponchelle
Reviewed By: compnerd
Subscribers: cfe-commits
Tags: #clang-c
Differential Revision: https://reviews.llvm.org/D31568
llvm-svn: 300829
This reverts commit 4464581bb63e9789e9ee231a8c8800be5f614743.
Memory access issues on Linux were reported by Mathieu Duponchelle and
discussed here: https://reviews.llvm.org/D26082.
llvm-svn: 291907
Summary:
Python bindings tests now pass in Python 3.
`map` in Python 3 is lazily evaluated so the method by which functions are registered needed updating.
Strings are unicode in Python 3 not UTF-8, I've tried to create an new c_types-like class (c_string_p) to automate the conversion.
String conversions made explicit where required.
Reviewers: eliben, nemanjai, skalinichev, compnerd
Subscribers: mgorny, cfe-commits
Differential Revision: https://reviews.llvm.org/D26082
llvm-svn: 285909
Matheus Izvekov