Commit Graph

337 Commits

Author SHA1 Message Date
Med Ismail Bennani 41f3e6b74d [lldb/Plugins] Fix build failure with GCC in ScriptedPythonInterface::Dispatch
This patch should fix the build failures following 7e01924 when building
with GCC. These failures were mostly caused by GCC's poor support of C++
templates (namely, partial template specialization).

To work around that problem, this patch makes use of overloading and get
rid of the templated structs and specialized structs.

Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
2022-11-19 13:24:47 -08:00
Med Ismail Bennani 27a26e7ab7 [lldb/Python] Unify PythonFormat & GetPythonValueFormatString (NFC)
This patch removes all occurences to GetPythonValueFormatString and
use the template specialization of PythonFormat structs instead.

Differential Revision: https://reviews.llvm.org/D134033

Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
2022-11-18 13:56:48 -08:00
Med Ismail Bennani 7e01924e4e [lldb/Plugins] Improve error reporting with reading memory in Scripted Process
This patch improves the ScriptedPythonInterface::Dispatch method to
support passing lldb_private types to the python implementation.

This will allow, for instance, the Scripted Process python implementation
to report errors when reading memory back to lldb.

To do so, the Dispatch method will transform the private types in the
parameter pack into `PythonObject`s to be able to pass them down to the
python methods.

Then, if the call succeeded, the transformed arguments will be converted
back to their original type and re-assigned in the parameter pack, to
ensure pointers and references behaviours are preserved.

This patch also updates various scripted process python class and tests
to reflect this change.

rdar://100030995

Differential Revision: https://reviews.llvm.org/D134033

Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
2022-11-18 13:56:48 -08:00
Med Ismail Bennani 288843a161 [lldb/Python] Make use of PythonObject and PythonFormat in callbacks (NFC)
This patch extends the template specialization of PythonFormat structs
and makes use of the pre-existing PythonObject class to format arguments
and pass them to the right method, before calling it.

This is a preparatory patch to merge PythonFormat with SWIGPythonBridge's
GetPythonValueFormatString methods.

Differential Revision: https://reviews.llvm.org/D138248

Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
2022-11-18 13:56:48 -08:00
Med Ismail Bennani 78d6e1d1d4 [lldb/crashlog] Add support for Application Specific Backtraces & Information
For an exception crashlog, the thread backtraces aren't usually very helpful
and instead, developpers look at the "Application Specific Backtrace" that
was generated by `objc_exception_throw`.

LLDB could already parse and symbolicate these Application Specific Backtraces
for regular textual-based crashlog, so this patch adds support to parse them
in JSON crashlogs, and materialize them a HistoryThread extending the
crashed ScriptedThread.

This patch also includes the Application Specific Information messages
as part of the process extended crash information log. To do so, the
ScriptedProcess Python interface has a new GetMetadata method that
returns an arbitrary dictionary with data related to the process.

rdar://93207586

Differential Revision: https://reviews.llvm.org/D126260

Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
2022-11-03 14:44:53 -07:00
Jorge Gorbe Moya d76566417e [lldb] Add matching based on Python callbacks for data formatters.
This patch adds a new matching method for data formatters, in addition
to the existing exact typename and regex-based matching. The new method
allows users to specify the name of a Python callback function that
takes a `SBType` object and decides whether the type is a match or not.

Here is an overview of the changes performed:

- Add a new `eFormatterMatchCallback` matching type, and logic to handle
  it in `TypeMatcher` and `SBTypeNameSpecifier`.

- Extend `FormattersMatchCandidate` instances with a pointer to the
  current `ScriptInterpreter` and the `TypeImpl` corresponding to the
  candidate type, so we can run registered callbacks and pass the type
  to them. All matcher search functions now receive a
  `FormattersMatchCandidate` instead of a type name.

- Add some glue code to ScriptInterpreterPython and the SWIG bindings to
  allow calling a formatter matching callback. Most of this code is
  modeled after the equivalent code for watchpoint callback functions.

- Add an API test for the new callback-based matching feature.

For more context, please check the RFC thread where this feature was
originally discussed:
https://discourse.llvm.org/t/rfc-python-callback-for-data-formatters-type-matching/64204/11

Differential Revision: https://reviews.llvm.org/D135648
2022-10-19 12:53:38 -07:00
Jonas Devlieghere 70599d7027
[lldb] Remove LLDB reproducers
This patch removes the remaining reproducer code. The SBReproducer class
remains for ABI stability but is just an empty shell. This completes the
removal process outlined on the mailing list [1].

[1] https://lists.llvm.org/pipermail/lldb-dev/2021-September/017045.html
2022-09-19 14:43:31 -07:00
Michał Górny 9823d42557 [lldb] [Core] Split read thread support into ThreadedCommunication
Split the read thread support from Communication into a dedicated
ThreadedCommunication subclass.  The read thread support is used only
by a subset of Communication consumers, and it adds a lot of complexity
to the base class.  Furthermore, having a dedicated subclass makes it
clear whether a particular consumer needs to account for the possibility
of read thread being running or not.

The modules currently calling `StartReadThread()` are updated to use
`ThreadedCommunication`.  The remaining modules use the simplified
`Communication` class.

`SBCommunication` is changed to use `ThreadedCommunication` in order
to avoid changing the public API.

`CommunicationKDP` is updated in order to (hopefully) compile with
the new code.  However, I do not have a Darwin box to test it, so I've
limited the changes to the bare minimum.

`GDBRemoteCommunication` is updated to become a `Broadcaster` directly.
Since it does not inherit from `ThreadedCommunication`, its event
support no longer collides with the one used for read thread and can
be implemented cleanly.  The support for
`eBroadcastBitReadThreadDidExit` is removed from the code -- since
the read thread was not used, this event was never reported.

Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.llvm.org/D133251
2022-09-06 13:09:42 +02:00
Kazu Hirata 920ffab9cc [lldb] Use nullptr instead of NULL (NFC)
Identified with modernize-use-nullptr.
2022-08-27 21:21:07 -07:00
Kazu Hirata 0660249cca [lldb] Remove a redundaunt return statement (NFC)
Identified with readability-redundant-control-flow.
2022-08-27 21:21:05 -07:00
Dave Lee 56f9cfe30c [lldb] Remove uses of six module (NFC)
With lldb (& llvm) requiring Python 3.6+, use of the `six` module can be removed.

Differential Revision: https://reviews.llvm.org/D131304
2022-08-11 19:06:15 -07:00
Med Ismail Bennani a07a75180c [lldb/crashlog] Surface error using SBCommandReturnObject argument
This patch allows the crashlog script to surface its errors to lldb by
using the provided SBCommandReturnObject argument.

rdar://95048193

Differential Revision: https://reviews.llvm.org/D129614

Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
2022-08-09 21:01:37 -07:00
Greg Clayton 529a3d87a7 [NFC] Improve FileSpec internal APIs and usage in preparation for adding caching of resolved/absolute.
Resubmission of https://reviews.llvm.org/D130309 with the 2 patches that fixed the linux buildbot, and new windows fixes.

The FileSpec APIs allow users to modify instance variables directly by getting a non const reference to the directory and filename instance variables. This makes it impossible to control all of the times the FileSpec object is modified so we can clear cached member variables like m_resolved and with an upcoming patch caching if the file is relative or absolute. This patch modifies the APIs of FileSpec so no one can modify the directory or filename instance variables directly by adding set accessors and by removing the get accessors that are non const.

Many clients were using FileSpec::GetCString(...) which returned a unique C string from a ConstString'ified version of the result of GetPath() which returned a std::string. This caused many locations to use this convenient function incorrectly and could cause many strings to be added to the constant string pool that didn't need to. Most clients were converted to using FileSpec::GetPath().c_str() when possible. Other clients were modified to use the newly renamed version of this function which returns an actualy ConstString:

ConstString FileSpec::GetPathAsConstString(bool denormalize = true) const;

This avoids the issue where people were getting an already uniqued "const char *" that came from a ConstString only to put the "const char *" back into a "ConstString" object. By returning the ConstString instead of a "const char *" clients can be more efficient with the result.

The patch:
- Removes the non const GetDirectory() and GetFilename() get accessors
- Adds set accessors to replace the above functions: SetDirectory() and SetFilename().
- Adds ClearDirectory() and ClearFilename() to replace usage of the FileSpec::GetDirectory().Clear()/FileSpec::GetFilename().Clear() call sites
- Fixed all incorrect usage of FileSpec::GetCString() to use FileSpec::GetPath().c_str() where appropriate, and updated other call sites that wanted a ConstString to use the newly returned ConstString appropriately and efficiently.

Differential Revision: https://reviews.llvm.org/D130549
2022-07-28 13:28:26 -07:00
Nico Weber 1b4b12a340 Revert "[NFC] Improve FileSpec internal APIs and usage in preparation for adding caching of resolved/absolute." and follow-ups
This reverts commit 9429b67b8e.

It broke the build on Windows, see comments on https://reviews.llvm.org/D130309

It also reverts these follow-ups:

Revert "Fix buildbot breakage after https://reviews.llvm.org/D130309."
This reverts commit f959d815f4.

Revert "Fix buildbot breakage after https://reviews.llvm.org/D130309."
This reverts commit 0bbce7a4c2.

Revert "Cache the value for absolute path in FileSpec."
This reverts commit dabe877248.
2022-07-23 12:35:48 -04:00
Greg Clayton f959d815f4 Fix buildbot breakage after https://reviews.llvm.org/D130309. 2022-07-22 13:24:26 -07:00
Greg Clayton 9429b67b8e [NFC] Improve FileSpec internal APIs and usage in preparation for adding caching of resolved/absolute.
The FileSpect APIs allow users to modify instance variables directly by getting a non const reference to the directory and filename instance variables. This makes it impossibly to control all of the times the FileSpec object is modified so we can clear the cache. This patch modifies the APIs of FileSpec so no one can modify the directory or filename directly by adding set accessors and by removing the get accessors that are non const.

Many clients were using FileSpec::GetCString(...) which returned a unique C string from a ConstString'ified version of the result of GetPath() which returned a std::string. This caused many locations to use this convenient function incorrectly and could cause many strings to be added to the constant string pool that didn't need to. Most clients were converted to using FileSpec::GetPath().c_str() when possible. Other clients were modified to use the newly renamed version of this function which returns an actualy ConstString:
    ConstString FileSpec::GetPathAsConstString(bool denormalize = true) const;

This avoids the issue where people were getting an already uniqued "const char *" that came from a ConstString only to put the "const char *" back into a "ConstString" object. By returning the ConstString instead of a "const char *" clients can be more efficient with the result.

The patch:
- Removes the non const GetDirectory() and GetFilename() get accessors
- Adds set accessors to replace the above functions: SetDirectory() and SetFilename().
- Adds ClearDirectory() and ClearFilename() to replace usage of the FileSpec::GetDirectory().Clear()/FileSpec::GetFilename().Clear() call sites
- Fixed all incorrect usage of FileSpec::GetCString() to use FileSpec::GetPath().c_str() where appropriate, and updated other call sites that wanted a ConstString to use the newly returned ConstString appropriately and efficiently.

Differential Revision: https://reviews.llvm.org/D130309
2022-07-22 10:12:31 -07:00
Jim Ingham e83d47f6b7 When the module path for `command script import` is invalid, echo the path.
We were just emitting "invalid module" w/o saying which module.  That's
not particularly helpful.

Differential Revision: https://reviews.llvm.org/D129338
2022-07-18 14:49:07 -07:00
Jonas Devlieghere 9916633997
[lldb] Fix modernize-use-override warnings (NFC)
Fix modernize-use-override warnings. Because this check is listed in
LLDB's top level .clang-tidy configuration, the check is enabled by
default and the resulting warnings show up in my editor.

I've audited the modified lines. This is not a blind change.
2022-06-17 15:08:02 -07:00
Tobias Ribizel b1aed14bfe [llvm][lldb] use FindLibEdit.cmake everywhere
Currently, LLVM's LineEditor and LLDB both use libedit, but find them in different (inconsistent) ways.
This causes issues e.g. when you are using a locally installed version of libedit, which will not be used
by clang-query, but by lldb if picked up by FindLibEdit.cmake

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D124673
2022-05-12 15:59:41 -07:00
Jonas Devlieghere 9053767330
Remove Python 2 support from the ScriptInterpreter plugin
We dropped downstream support for Python 2 in the previous release. Now
that we have branched for the next release the window where this kind of
change could introduce conflicts is closing too. Start by getting rid of
Python 2 support in the Script Interpreter plugin.

Differential revision: https://reviews.llvm.org/D124429
2022-04-27 08:26:25 -07:00
Med Ismail Bennani 680ca7f21a [lldb/Plugins] Add ability to load modules to Scripted Processes
This patch introduces a new way to load modules programatically with
Scripted Processes. To do so, the scripted process blueprint holds a
list of dictionary describing the modules to load, which their path or
uuid, load address and eventually a slide offset.

LLDB will fetch that list after launching the ScriptedProcess, and
iterate over each entry to create the module that will be loaded in the
Scripted Process' target.

The patch also refactors the StackCoreScriptedProcess test to stop
inside the `libbaz` module and make sure it's loaded correctly and that
we can fetch some variables from it.

rdar://74520238

Differential Revision: https://reviews.llvm.org/D120969

Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
2022-03-04 13:35:28 -08:00
Jonas Devlieghere bf414cfbf7
[lldb] Fix the build after 8b3b66ea63
Remove remaining calls to FileSystem::Collect.
2022-03-03 13:57:33 -08:00
Jonas Devlieghere 8b3b66ea63
[lldb] Remove FileSystem::Initialize from FileCollector
This patch removes the ability to instantiate the LLDB FileSystem class
with a FileCollector. It keeps the ability to collect files, but uses
the FileCollectorFileSystem to do that transparently.

Because the two are intertwined, this patch also removes the
finalization logic which copied the files over out of process.
2022-03-03 13:22:38 -08:00
Med Ismail Bennani 7066584491 [lldb/Plugin] Add artificial stackframe loading in ScriptedThread
This patch adds the ability for ScriptedThread to load artificial stack
frames. To do so, the interpreter instance can create a list that will
contain the frame index and its pc address.

Then, when the Scripted Process plugin stops, it will refresh its
Scripted Threads state by invalidating their register context and load
to list from the interpreter object and reconstruct each frame.

This patch also removes all of the default implementation for
`get_stackframes` from the derived ScriptedThread classes, and add the
interface code for the Scripted Thread Interface.

rdar://88721095

Differential Revision: https://reviews.llvm.org/D119388

Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
2022-02-16 11:44:07 -08:00
Pavel Labath c34698a811 [lldb] Rename Logging.h to LLDBLog.h and clean up includes
Most of our code was including Log.h even though that is not where the
"lldb" log channel is defined (Log.h defines the generic logging
infrastructure). This worked because Log.h included Logging.h, even
though it should.

After the recent refactor, it became impossible the two files include
each other in this direction (the opposite inclusion is needed), so this
patch removes the workaround that was put in place and cleans up all
files to include the right thing. It also renames the file to LLDBLog to
better reflect its purpose.
2022-02-03 14:47:01 +01:00
Pavel Labath a007a6d844 [lldb] Convert "LLDB" log channel to the new API 2022-02-02 14:13:08 +01:00
Jonas Devlieghere 16bff06790 [lldb] Make PythonDataObjects work with Python 2
I considered keeping this change strictly downstream. Since we still
have a bunch of places that check for Python 2, I figured it doesn't
harm to land it upstream and avoid the conflict when I eventually do
remove them (hopefully soon!).
2022-01-24 17:23:09 -08:00
Med Ismail Bennani 45148bfe8a [lldb/Plugins] Fix ScriptedThread IndexID reporting
When listing all the Scripted Threads of a ScriptedProcess, we can see that all
have the thread index set to 1. This is caused by the lldb_private::Thread
constructor, which sets the m_index_id member using the provided thread id `tid`.

Because the call to the super constructor is done before instantiating
the `ScriptedThreadInterface`, lldb can't fetch the thread id from the
script instance, so it uses `LLDB_INVALID_THREAD_ID` instead.

To mitigate this, this patch takes advantage of the `ScriptedThread::Create`
fallible constructor idiom to defer calling the `ScriptedThread` constructor
(and the `Thread` super constructor with it), until we can fetch a valid
thread id `tid` from the `ScriptedThreadInterface`.

rdar://87432065

Differential Revision: https://reviews.llvm.org/D117076

Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
2022-01-24 20:25:54 +01:00
Med Ismail Bennani d3e0f7e150 [lldb/Plugins] Add support of multiple ScriptedThreads in a ScriptedProcess
This patch adds support of multiple Scripted Threads in a ScriptedProcess.

This is done by fetching the Scripted Threads info dictionary at every
ScriptedProcess::DoUpdateThreadList and iterate over each element to
create a new ScriptedThread using the object instance, if it was not
already available.

This patch also adds the ability to pass a pointer of a script interpreter
object instance to initialize a ScriptedInterface instead of having to call
the script object initializer in the ScriptedInterface constructor.

This is used to instantiate the ScriptedThreadInterface from the
ScriptedThread constructor, to be able to perform call on that script
interpreter object instance.

Finally, the patch also updates the scripted process test to check for
multiple threads.

rdar://84507704

Differential Revision: https://reviews.llvm.org/D117071

Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
2022-01-24 20:25:53 +01:00
Med Ismail Bennani 1b86344fa8 [lldb/Plugins] Move ScriptedThreadInterface to ScriptedThread
Since we can have multiple Scripted Threads per Scripted Process, having
only a single ScriptedThreadInterface (with a single object instance)
will cause the method calls to be done on the wrong object.

Instead, this patch creates a separate ScriptedThreadInterface for each
new lldb_private::ScriptedThread to make sure we interact with the right
instance.

rdar://87427911

Differential Revision: https://reviews.llvm.org/D117070

Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
2022-01-24 20:25:53 +01:00
Med Ismail Bennani 4858fe04a1 [lldb/Plugins] Add ScriptedProcess::GetThreadsInfo interface
This patch adds a new method to the Scripted Process interface to
retrive a dictionary of Scripted Threads. It uses the thread ID as a key
and the Scripted Thread instance as the value.

This dictionary will be used to create Scripted Threads in lldb and
perform calls to the python scripted thread object.

rdar://87427126

Differential Revision: https://reviews.llvm.org/D117068

Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
2022-01-24 20:25:53 +01:00
Jonas Devlieghere 1755f5b1d7 [lldb] Decouple instrumentation from the reproducers
Remove the last remaining references to the reproducers from the
instrumentation. This patch renames the relevant files and macros.

Differential revision: https://reviews.llvm.org/D117712
2022-01-20 18:06:14 -08:00
Adrian Prantl 36cb29cbbe Work around a module build failure on the bots.
This patch works around what looks like a bug in Clang itself.

The error on the bot is:

https://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake/40466/consoleText

In module 'LLVM_Utils' imported from /Users/buildslave/jenkins/workspace/lldb-cmake/llvm-project/lldb/source/Plugins/ScriptInterpreter/Python/lldb-python.h:18:
/Users/buildslave/jenkins/workspace/lldb-cmake/llvm-project/llvm/include/llvm/Support/Error.h:720:3: error: 'llvm::Expected<bool>::(anonymous)' from module 'LLVM_Utils.Support.Error' is not present in definition of 'llvm::Expected<bool>' in module 'LLVM_Utils.Support.Error'
  union {
  ^
/Users/buildslave/jenkins/workspace/lldb-cmake/llvm-project/llvm/include/llvm/Support/Error.h:720:3: note: declaration of '' does not match
/Users/buildslave/jenkins/workspace/lldb-cmake/llvm-project/llvm/include/llvm/Support/Error.h:720:3: note: declaration of '' does not match
1 error generated.

The intention is to revert this as soon as a proper fix has been identified!

rdar://87845391
2022-01-20 13:39:48 -08:00
Jonas Devlieghere eb5c0ea681 [lldb] Initialize Python exactly once
We got a few crash reports that showed LLDB initializing Python on two
separate threads. Make sure Python is initialized exactly once.

rdar://87287005

Differential revision: https://reviews.llvm.org/D117601
2022-01-19 09:56:55 -08:00
Pavel Labath 6ff4af8e18 [lldb] Fix D114722 for python<=3.6
_Py_IsFinalizing was called _Py_Finalizing back then (and it was a
variable instead of a function).
2022-01-19 12:49:46 +01:00
Pavel Labath c154f397ee [lldb/python] Use PythonObject in LLDBSwigPython functions
Return our PythonObject wrappers instead of raw PyObjects (obfuscated as
void *). This ensures that ownership (reference counts) of python
objects is automatically tracked.

Differential Revision: https://reviews.llvm.org/D117462
2022-01-18 10:28:58 +01:00
Ralf Grosse-Kunstleve a6598575f4 [LLDB] Fix Python GIL-not-held issues
The GIL must be held when calling any Python C API functions. In multithreaded applications that use callbacks this requirement can easily be violated by accident. A general tool to ensure GIL health is not available, but patching Python Py_INCREF to add an assert provides a basic health check:

```
+int PyGILState_Check(void); /* Include/internal/pystate.h */
+
 #define Py_INCREF(op) (                         \
+    assert(PyGILState_Check()),                 \
     _Py_INC_REFTOTAL  _Py_REF_DEBUG_COMMA       \
     ((PyObject *)(op))->ob_refcnt++)

 #define Py_DECREF(op)                                   \
     do {                                                \
+        assert(PyGILState_Check());                     \
         PyObject *_py_decref_tmp = (PyObject *)(op);    \
         if (_Py_DEC_REFTOTAL  _Py_REF_DEBUG_COMMA       \
         --(_py_decref_tmp)->ob_refcnt != 0)             \
```

Adding this assertion causes around 50 test failures in LLDB. Adjusting the scope of things guarded by `py_lock` fixes them.

More background: https://docs.python.org/3/glossary.html#term-global-interpreter-lock

Patch by Ralf Grosse-Kunstleve

Differential Revision: https://reviews.llvm.org/D114722
2022-01-17 10:32:19 +01:00
Jonas Devlieghere 049ae93097 [lldb] Fix that the embedded Python REPL crashes if it receives SIGINT
When LLDB receives a SIGINT while running the embedded Python REPL it
currently just crashes in ScriptInterpreterPythonImpl::Interrupt with an
error such as the one below:

  Fatal Python error: PyThreadState_Get: the function must be called
  with the GIL held, but the GIL is released (the current Python thread
  state is NULL)

The faulty code that causes this error is this part of
ScriptInterpreterPythonImpl::Interrupt:

  PyThreadState *state = PyThreadState_GET();
  if (!state)
    state = GetThreadState();
  if (state) {
    long tid = state->thread_id;
    PyThreadState_Swap(state);
    int num_threads = PyThreadState_SetAsyncExc(tid, PyExc_KeyboardInterrupt);

The obvious fix I tried is to just acquire the GIL before this code is
running which fixes the crash but the KeyboardInterrupt we want to raise
immediately is actually just queued and would only be raised once the
next line of input has been parsed (which e.g. won't interrupt Python
code that is currently waiting on a timer or IO from what I can see).
Also none of the functions we call here is marked as safe to be called
from a signal handler from what I can see, so we might still end up
crashing here with some bad timing.

Python 3.2 introduced PyErr_SetInterrupt to solve this and the function
takes care of all the details and avoids doing anything that isn't safe
to do inside a signal handler. The only thing we need to do is to
manually setup our own fake SIGINT handler that behaves the same way as
the standalone Python REPL signal handler (which raises a
KeyboardInterrupt).

From what I understand the old code used to work with Python 2 so I kept
the old code around until we officially drop support for Python 2.

There is a small gap here with Python 3.0->3.1 where we might still be
crashing, but those versions have reached their EOL more than a decade
ago so I think we don't need to bother about them.

Differential revision: https://reviews.llvm.org/D104886
2022-01-13 15:27:38 -08:00
Jonas Devlieghere d51402ac6b [lldb] Remove reproducer instrumentation
This patch removes most of the reproducer instrumentation. It keeps
around the LLDB_RECORD_* macros for logging. See [1] for more details.

[1] https://lists.llvm.org/pipermail/lldb-dev/2021-September/017045.html

Differential revision: https://reviews.llvm.org/D116847
2022-01-09 21:40:55 -08:00
Qiu Chaofan c2cc70e4f5 [NFC] Fix endif comments to match with include guard 2022-01-07 15:52:59 +08:00
Pavel Labath 7406d236d8 [lldb/python] Fix (some) dangling pointers in our glue code
This starts to fix the other half of the lifetime problems in this code
-- dangling references. SB objects created on the stack will go away
when the function returns, which is a problem if the python code they
were meant for stashes a reference to them somewhere.  Most of the time
this goes by unnoticed, as the code rarely has a reason to store these,
but in case it does, we shouldn't respond by crashing.

This patch fixes the management for a couple of SB objects (Debugger,
Frame, Thread). The SB objects are now created on the heap, and
their ownership is immediately passed on to SWIG, which will ensure they
are destroyed when the last python reference goes away. I will handle
the other objects in separate patches.

I include one test which demonstrates the lifetime issue for SBDebugger.
Strictly speaking, one should create a test case for each of these
objects and each of the contexts they are being used. That would require
figuring out how to persist (and later access) each of these objects.
Some of those may involve a lot of hoop-jumping (we can run python code
from within a frame-format string). I don't think that is
necessary/worth it since the new wrapper functions make it very hard to
get this wrong.

Differential Revision: https://reviews.llvm.org/D115925
2021-12-20 09:42:08 +01:00
Pavel Labath 82de8df26f [lldb] Clarify StructuredDataImpl ownership
StructuredDataImpl ownership semantics is unclear at best. Various
structures were holding a non-owning pointer to it, with a comment that
the object is owned somewhere else. From what I was able to gather that
"somewhere else" was the SBStructuredData object, but I am not sure that
all created object eventually made its way there. (It wouldn't matter
even if they did, as we are leaking most of our SBStructuredData
objects.)

Since StructuredDataImpl is just a collection of two (shared) pointers,
there's really no point in elaborate lifetime management, so this patch
replaces all StructuredDataImpl pointers with actual objects or
unique_ptrs to it. This makes it much easier to resolve SBStructuredData
leaks in a follow-up patch.

Differential Revision: https://reviews.llvm.org/D114791
2021-12-13 21:04:51 +01:00
Pavel Labath 85578db68a [lldb/lua] Add a file that should have been a part of a52af6d3 2021-12-06 14:58:39 +01:00
Pavel Labath a52af6d371 [lldb] Remove extern "C" from lldb-swig-lua interface
This is the lua equivalent of 9a14adeae0.
2021-12-06 14:57:44 +01:00
Pavel Labath 9a14adeae0 [lldb] Remove 'extern "C"' from the lldb-swig-python interface
The LLDBSWIGPython functions had (at least) two problems:
- There wasn't a single source of truth (a header file) for the
  prototypes of these functions. This meant that subtle differences
  in copies of function declarations could go by undetected. And
  not-so-subtle differences would result in strange runtime failures.
- All of the declarations had to have an extern "C" interface, because
  the function definitions were being placed inside and extert "C" block
  generated by swig.

This patch fixes both problems by moving the function definitions to the
%header block of the swig files. This block is not surrounded by extern
"C", and seems more appropriate anyway, as swig docs say it is meant for
"user-defined support code" (whereas the previous %wrapper code was for
automatically-generated wrappers).

It also puts the declarations into the SWIGPythonBridge header file
(which seems to have been created for this purpose), and ensures it is
included by all code wishing to define or use these functions. This
means that any differences in the declaration become a compiler error
instead of a runtime failure.

Differential Revision: https://reviews.llvm.org/D114369
2021-11-30 11:06:09 +01:00
Pavel Labath 7f09ab08de [lldb] Fix [some] leaks in python bindings
Using an lldb_private object in the bindings involves three steps
- wrapping the object in it's lldb::SB variant
- using swig to convert/wrap that to a PyObject
- wrapping *that* in a lldb_private::python::PythonObject

Our SBTypeToSWIGWrapper was only handling the middle part. This doesn't
just result in increased boilerplate in the callers, but is also a
functionality problem, as it's very hard to get the lifetime of of all
of these objects right. Most of the callers are creating the SB object
(step 1) on the stack, which means that we end up with dangling python
objects after the function terminates. Most of the time this isn't a
problem, because the python code does not need to persist the objects.
However, there are legitimate cases where they can do it (and even if
the use case is not completely legitimate, crashing is not the best
response to that).

For this reason, some of our code creates the SB object on the heap, but
it has another problem -- it never gets cleaned up.

This patch begins to add a new function (ToSWIGWrapper), which does all
of the three steps, while properly taking care of ownership. In the
first step, I have converted most of the leaky code (except for
SBStructuredData, which needs a bit more work).

Differential Revision: https://reviews.llvm.org/D114259
2021-11-22 15:14:52 +01:00
Lawrence D'Anna 4c2cf3a314 [lldb] fix -print-script-interpreter-info on windows
Apparently "{sys.prefix}/bin/python3" isn't where you find the
python interpreter on windows, so the test I wrote for
-print-script-interpreter-info is failing.

We can't rely on sys.executable at runtime, because that will point
to lldb.exe not python.exe.

We can't just record sys.executable from build time, because python
could have been moved to a different location.

But it should be OK to apply relative path from sys.prefix to sys.executable
from build-time to the sys.prefix at runtime.

Reviewed By: JDevlieghere

Differential Revision: https://reviews.llvm.org/D113650
2021-11-16 13:50:20 -08:00
Raphael Isemann c3a3e65ecc Revert "[lldb] Fix that the embedded Python REPL crashes if it receives SIGINT"
This reverts commit cef1e07cc6.

It broke the windows bot.
2021-11-13 18:18:24 +01:00
Raphael Isemann cef1e07cc6 [lldb] Fix that the embedded Python REPL crashes if it receives SIGINT
When LLDB receives a SIGINT while running the embedded Python REPL it currently
just crashes in `ScriptInterpreterPythonImpl::Interrupt` with an error such as
the one below:

```

Fatal Python error: PyThreadState_Get: the function must be called with the GIL
held, but the GIL is released (the current Python thread state is NULL)

```

The faulty code that causes this error is this part of `ScriptInterpreterPythonImpl::Interrupt`:
```
    PyThreadState *state = PyThreadState_GET();
    if (!state)
      state = GetThreadState();
    if (state) {
      long tid = state->thread_id;
      PyThreadState_Swap(state);
      int num_threads = PyThreadState_SetAsyncExc(tid, PyExc_KeyboardInterrupt);
```

The obvious fix I tried is to just acquire the GIL before this code is running
which fixes the crash but the `KeyboardInterrupt` we want to raise immediately
is actually just queued and would only be raised once the next line of input has
been parsed (which e.g. won't interrupt Python code that is currently waiting on
a timer or IO from what I can see). Also none of the functions we call here is
marked as safe to be called from a signal handler from what I can see, so we
might still end up crashing here with some bad timing.

Python 3.2 introduced `PyErr_SetInterrupt` to solve this and the function takes
care of all the details and avoids doing anything that isn't safe to do inside a
signal handler. The only thing we need to do is to manually setup our own fake
SIGINT handler that behaves the same way as the standalone Python REPL signal
handler (which raises a KeyboardInterrupt).

From what I understand the old code used to work with Python 2 so I kept the old
code around until we officially drop support for Python 2.

There is a small gap here with Python 3.0->3.1 where we might still be crashing,
but those versions have reached their EOL more than a decade ago so I think we
don't need to bother about them.

Reviewed By: JDevlieghere

Differential Revision: https://reviews.llvm.org/D104886
2021-11-12 14:19:15 +01:00
Lawrence D'Anna bbef51eb43 [lldb] make it easier to find LLDB's python
It is surprisingly difficult to write a simple python script that
can reliably `import lldb` without failing, or crashing.   I'm
currently resorting to convolutions like this:

    def find_lldb(may_reexec=False):
		if prefix := os.environ.get('LLDB_PYTHON_PREFIX'):
			if os.path.realpath(prefix) != os.path.realpath(sys.prefix):
				raise Exception("cannot import lldb.\n"
					f"  sys.prefix should be: {prefix}\n"
					f"  but it is: {sys.prefix}")
		else:
			line1, line2 = subprocess.run(
				['lldb', '-x', '-b', '-o', 'script print(sys.prefix)'],
				encoding='utf8', stdout=subprocess.PIPE,
				check=True).stdout.strip().splitlines()
			assert line1.strip() == '(lldb) script print(sys.prefix)'
			prefix = line2.strip()
			os.environ['LLDB_PYTHON_PREFIX'] = prefix

		if sys.prefix != prefix:
			if not may_reexec:
				raise Exception(
					"cannot import lldb.\n" +
					f"  This python, at {sys.prefix}\n"
					f"  does not math LLDB's python at {prefix}")
			os.environ['LLDB_PYTHON_PREFIX'] = prefix
			python_exe = os.path.join(prefix, 'bin', 'python3')
			os.execl(python_exe, python_exe, *sys.argv)

		lldb_path = subprocess.run(['lldb', '-P'],
			check=True, stdout=subprocess.PIPE,
				encoding='utf8').stdout.strip()

		sys.path = [lldb_path] + sys.path

This patch aims to replace all that with:

  #!/usr/bin/env lldb-python
  import lldb
  ...

... by adding the following features:

* new command line option: --print-script-interpreter-info.  This
   prints language-specific information about the script interpreter
   in JSON format.

* new tool (unix only): lldb-python which finds python and exec's it.

Reviewed By: JDevlieghere

Differential Revision: https://reviews.llvm.org/D112973
2021-11-10 10:33:34 -08:00