This patch refactors Scripted Process and Scripted Thread related
classes to use LLVM_PRETTY_FUNCTION instead of the compiler macro.
Differential Revision: https://reviews.llvm.org/D111452
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This patch adds support for memory regions in Scripted Processes.
This is necessary to read the stack memory region in order to
reconstruct each stackframe of the program.
In order to do so, this patch makes some changes to the SBAPI, namely:
- Add a new constructor for `SBMemoryRegionInfo` that takes arguments
such as the memory region name, address range, permissions ...
This is used when reading memory at some address to compute the offset
in the binary blob provided by the user.
- Add a `GetMemoryRegionContainingAddress` method to `SBMemoryRegionInfoList`
to simplify the access to a specific memory region.
With these changes, lldb is now able to unwind the stack and reconstruct
each frame. On top of that, reloading the target module at offset 0 allows
lldb to symbolicate the `ScriptedProcess` using debug info, similarly to an
ordinary Process.
To test this, I wrote a simple program with multiple function calls, ran it in
lldb, stopped at a leaf function and read the registers values and copied
the stack memory into a binary file. These are then used in the python script.
Differential Revision: https://reviews.llvm.org/D108953
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This patch introduces the `ScriptedThread` class with its python
interface.
When used with `ScriptedProcess`, `ScriptedThreaad` can provide various
information such as the thread state, stop reason or even its register
context.
This can be used to reconstruct the program stack frames using lldb's unwinder.
rdar://74503836
Differential Revision: https://reviews.llvm.org/D107585
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
Refactor TerminalState to make the code simpler. Move 'struct termios'
to a PImpl-style subclass. Add an RAII interface to automatically store
and restore the state.
Differential revision: https://reviews.llvm.org/D110721
In all these years, we haven't found a use for this function (it has
zero callers). Lets just remove the boilerplate.
Differential Revision: https://reviews.llvm.org/D109600
This patch should fix the build failure that surfaced when build llvm
with GCC: https://lab.llvm.org/staging/#/builders/16/builds/10450
GCC complained that I explicitely specialized
`ScriptedPythonInterface::ExtractValueFromPythonObject` in a
in non-namespace scope, which is tolerated by Clang.
To solve this issue, the specialization were declared out of the class
and implemented in the source file.
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This patch splits the previous `ScriptedProcessPythonInterface` into
multiple specific classes:
1. The `ScriptedInterface` abstract class that carries the interface
instance object and its virtual pure abstract creation method.
2. The `ScriptedPythonInterface` that holds a generic `Dispatch` method that
can be used by various interfaces to call python methods and also keeps a
reference to the Python Script Interpreter instance.
3. The `ScriptedProcessInterface` that describes the base Scripted
Process model with all the methods used in the underlying script.
All these components are used to refactor the `ScriptedProcessPythonInterface`
class, making it more modular.
This patch is also a requirement for the upcoming work on `ScriptedThread`.
Differential Revision: https://reviews.llvm.org/D107521
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
Due to CMake cache, find_package in FindLuaAndSwig.cmake
will be ignored. This commit adds EXACT and REQUIRED flags
to it and removes find_package in Lua ScriptInterpreter.
Signed-off-by: Siger Yang <sigeryeung@gmail.com>
Reviewed By: tammela, JDevlieghere
Differential Revision: https://reviews.llvm.org/D108515
Modify OpenOptions enum to open the future path into synchronizing
vFile:open bits with GDB. Currently, LLDB and GDB use different flag
models effectively making it impossible to match bits. Notably, LLDB
uses two bits to indicate read and write status, and uses union of both
for read/write. GDB uses a value of 0 for read-only, 1 for write-only
and 2 for read/write.
In order to future-proof the code for the GDB variant:
1. Add a distinct eOpenOptionReadWrite constant to be used instead
of (eOpenOptionRead | eOpenOptionWrite) when R/W access is required.
2. Rename eOpenOptionRead and eOpenOptionWrite to eOpenOptionReadOnly
and eOpenOptionWriteOnly respectively, to make it clear that they
do not mean to be combined and require update to all call sites.
3. Use the intersection of all three flags when matching against
the three possible values.
This commit does not change the actual bits used by LLDB.
Differential Revision: https://reviews.llvm.org/D106984
This patch updates the `ScriptedProcess::GetGenericInteger` return type
to `llvm::Optional<unsigned long long>` to match implementation.
Differential Revision: https://reviews.llvm.org/D105788
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This patch should address the compiler warnings due to mismatch type
comparaison.
Differential Revision: https://reviews.llvm.org/D105788
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This patch introduces Scripted Processes to lldb.
The goal, here, is to be able to attach in the debugger to fake processes
that are backed by script files (in Python, Lua, Swift, etc ...) and
inspect them statically.
Scripted Processes can be used in cooperative multithreading environments
like the XNU Kernel or other real-time operating systems, but it can
also help us improve the debugger testing infrastructure by writting
synthetic tests that simulates hard-to-reproduce process/thread states.
Although ScriptedProcess is not feature-complete at the moment, it has
basic execution capabilities and will improve in the following patches.
rdar://65508855
Differential Revision: https://reviews.llvm.org/D100384
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
Add the ability to silence command script import. The motivation for
this change is being able to add command script import -s
lldb.macosx.crashlog to your ~/.lldbinit without it printing the
following message at the beginning of every debug session.
"malloc_info", "ptr_refs", "cstr_refs", "find_variable", and
"objc_refs" commands have been installed, use the "--help" options on
these commands for detailed help.
In addition to forwarding the silent option to LoadScriptingModule, this
also changes ScriptInterpreterPythonImpl::ExecuteOneLineWithReturn and
ScriptInterpreterPythonImpl::ExecuteMultipleLines to honor the enable IO
option in ExecuteScriptOptions, which until now was ignored.
Note that IO is only enabled (or disabled) at the start of a session,
and for this particular use case, that's done when taking the Python
lock in LoadScriptingModule, which means that the changes to these two
functions are not strictly necessary, but (IMO) desirable nonetheless.
Differential revision: https://reviews.llvm.org/D105327
This is an NFC cleanup.
Many of the API's that returned BreakpointOptions always returned valid ones.
Internally the BreakpointLocations usually have null BreakpointOptions, since they
use their owner's options until an option is set specifically on the location.
So the original code used pointers & unique_ptr everywhere for consistency.
But that made the code hard to reason about from the outside.
This patch changes the code so that everywhere an API is guaranteed to
return a non-null BreakpointOption, it returns it as a reference to make
that clear.
It also changes the Breakpoint to hold a BreakpointOption
member where it previously had a UP. Since we were always filling the UP
in the Breakpoint constructor, having the UP wasn't helping anything.
Differential Revision: https://reviews.llvm.org/D104162
This converts a default constructor's member initializers into C++11
default member initializers. This patch was automatically generated with
clang-tidy and the modernize-use-default-member-init check.
$ run-clang-tidy.py -header-filter='lldb' -checks='-*,modernize-use-default-member-init' -fix
This is a mass-refactoring patch and this commit will be added to
.git-blame-ignore-revs.
Differential revision: https://reviews.llvm.org/D103483
The C headers are deprecated so as requested in D102845, this is replacing them
all with their (not deprecated) C++ equivalent.
Reviewed By: shafik
Differential Revision: https://reviews.llvm.org/D103084
This patch adds a ScriptedProcess interface to the ScriptInterpreter and
more specifically, to the ScriptInterpreterPython.
This interface will be used in the C++ `ScriptProcess` Process Plugin to
call the script methods.
At the moment, not all methods are implemented, they will upstreamed in
upcoming patches.
This patch also adds helper methods to the ScriptInterpreter to
convert `SBAPI` Types (SBData & SBError) to `lldb_private` types
(DataExtractor & Status).
rdar://65508855
Differential Revision: https://reviews.llvm.org/D95711
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This function would fail in debug builds, as the two usages of the
LLDB_PYTHON_RELATIVE_LIBDIR macro would expand to two distinct strings.
The path iterator macros don't support that.
Use a temporary variable to ensure everything points to a single string.
This patch adds a ScriptedProcess interface to the ScriptInterpreter and
more specifically, to the ScriptInterpreterPython.
This interface will be used in the C++ `ScriptProcess` Process Plugin to
call the script methods.
At the moment, not all methods are implemented, they will upstreamed in
upcoming patches.
This patch also adds helper methods to the ScriptInterpreter to
convert `SBAPI` Types (SBData & SBError) to `lldb_private` types
(DataExtractor & Status).
rdar://65508855
Differential Revision: https://reviews.llvm.org/D95711
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
Our code for locating the shared library directory works via dladdr (or
the windows equivalent) to locate the path of an address known to reside
in liblldb. This works great for C++ programs, but there's a catch.
When (lib)lldb is used from python (like in our test suite), this dladdr
call will return a path to the _lldb.so (or such) file in the python
directory. To compensate for this, we have code which attempts to
resolve this symlink, to ensure we get the canonical location. However,
here's the second catch.
On windows, this file is not a symlink (but a copy), so this logic
fails. Since most of our other paths are derived from the liblldb
location, all of these paths will be wrong, when running the test suite.
One effect of this was the failure to find lldb-server in D96202.
To fix this issue, I add some windows-specific code to locate the
liblldb directory. Since it cannot rely on symlinks, it works by
manually walking the directory tree -- essentially doing the opposite of
what we do when computing the python directory.
To avoid python leaking back into the host code, I implement this with
the help of a callback which can be passed to HostInfo::Initialize in
order to assist with the directory location. The callback lives inside
the python plugin.
I also strenghten the existing path test to ensure the returned path is
the right one.
Differential Revision: https://reviews.llvm.org/D96779
LLDB does not like to import Python files with dashes or dots in their
name. While the former are technically allowed, it is discouraged. Dots
are allowed for subpackages but not in module names. This patch improves
the user experience by printing a useful error.
Before this patch:
error: module importing failed: SyntaxError('invalid syntax',
('<string>', 1, 11, 'import foo-bar\n'))
After this patch:
error: module importing failed: Python discourages dashes in module
names: foo-bar
rdar://74263511
[1] https://www.python.org/dev/peps/pep-0008/#package-and-module-names
Differential revision: https://reviews.llvm.org/D96833
Adds support for running a Lua function when a breakpoint is hit.
Example:
breakpoint command add -s lua -F abc
The above runs the Lua function 'abc' passing 2 arguments. 'frame', 'bp_loc' and 'extra_args'.
A third parameter 'extra_args' is only present when there is structured data
declared in the command line.
Example:
breakpoint command add -s lua -F abc -k foo -v bar
Differential Revision: https://reviews.llvm.org/D93649
1 - Partial Statements
The interpreter loop runs every line it receives, so partial
Lua statements are not being handled properly. This is a problem for
multiline breakpoint scripts since the interpreter loop, for this
particular case, is just an abstraction to a partially parsed function
body declaration.
This patch addresses this issue and as a side effect improves the
general Lua interpreter loop as well. It's now possible to write partial
statements in the 'script' command.
Example:
(lldb) script
>>> do
..> local a = 123
..> print(a)
..> end
123
The technique implemented is the same as the one employed by Lua's own REPL implementation.
Partial statements always errors out with the '<eof>' tag in the error
message.
2 - CheckSyntax in Lua.h
In order to support (1), we need an API for just checking the syntax of string buffers.
3 - Multiline scripted breakpoints
Finally, with all the base features implemented this feature is
straightforward. The interpreter loop behaves exactly the same, the
difference is that it will aggregate all Lua statements into the body of
the breakpoint function. An explicit 'quit' statement is needed to exit the
interpreter loop.
Example:
(lldb) breakpoint command add -s lua
Enter your Lua command(s). Type 'quit' to end.
The commands are compiled as the body of the following Lua function
function (frame, bp_loc, ...) end
..> print(456)
..> a = 123
..> quit
Differential Revision: https://reviews.llvm.org/D93481
This patch introduces a LLDB_SCOPED_TIMER macro to hide the needlessly
repetitive creation of scoped timers in LLDB. It's similar to the
LLDB_LOG(F) macro.
Differential revision: https://reviews.llvm.org/D93663
The reproducers currently use a static variable to track the API
boundary. This is obviously incorrect when the SB API is used
concurrently. While I do not plan to support that use-case (right now),
I do want to avoid us crashing. As a first step, correctly track API
boundaries across multiple threads.
Before this patch SB API calls made by the embedded script interpreter
would be considered "behind the API boundary" and correctly ignored.
After this patch, we need to tell the reproducers to ignore the
scripting thread as a "private thread".
Differential revision: https://reviews.llvm.org/D92811
LLDB is ignoring compilation errors for one-line breakpoint scripts.
This patch fixes the issues and now the error message of the
ScriptInterpreter is shown to the user.
I had to remove a new-line character for the Lua interpreter since it
was duplicated.
Differential Revision: https://reviews.llvm.org/D92729
The code that gets the ScriptInterpreter was not considering the
case that it receives a Lua interpreter.
Differential Revision: https://reviews.llvm.org/D92249
These callbacks are set using the following:
breakpoint command add -s lua -o "print('hello world!')"
The user supplied script is executed as:
function (frame, bp_loc, ...)
<body>
end
So the local variables 'frame', 'bp_loc' and vararg are all accessible.
Any global variables declared will persist in the Lua interpreter.
A user should never hold 'frame' and 'bp_loc' in a global variable as
these userdatas are context dependent.
Differential Revision: https://reviews.llvm.org/D91508
This patch changes the implementation of Lua's `print()` function to
respect `io.stdout`.
The original implementation uses `lua_writestring()` internally, which is
hardcoded to `stdout`.
Reviewed By: JDevlieghere
Differential Revision: https://reviews.llvm.org/D90787
This patch is a minor suggestion to not rely on the fact
that the `LUA_OK` macro is 0.
This assumption could change in future versions of the C API.
Differential Revision: https://reviews.llvm.org/D90556
This patch calls `lua_close()` on Lua dtor.
This guarantees that the Lua GC finalizers are honored, aside from the
usual internal clean up.
It also guarantees a call to the `__close` metamethod of any active
to-be-closed variable in Lua 5.4.
Since the previous `luaL_openlibs()` was a noop, because the standard
library is cached internally, I've removed it.
Differential Revision: https://reviews.llvm.org/D90557
Make it possible to use a relative path in command script import to the
location of the file being sourced. This allows the user to put Python
scripts next to LLDB command files and importing them without having to
specify an absolute path.
To enable this behavior pass `-c` to `command script import`. The
argument can only be used when sourcing the command from a file.
rdar://68310384
Differential revision: https://reviews.llvm.org/D89334
Currently we only strip the Python extension when the file exists on
disk because we assumed that if it didn't exist it was a module.
However, with the change from D89334 this is no longer the case as we
want to be able to import a relative path to a .py as a module. Since we
always import a scripting module as a "python module" we should always
strip the extension if present.
Differential revision: https://reviews.llvm.org/D89352
This reverts commit f775fe5964.
I fixed a return type error in the original patch that was causing a test failure.
Also added a REQUIRES: python to the shell test so we'll skip this for
people who build lldb w/o Python.
Also added another test for the error printing.
This patch is a big sed to rename the following variables:
s/PYTHON_LIBRARIES/Python3_LIBRARIES/g
s/PYTHON_INCLUDE_DIRS/Python3_INCLUDE_DIRS/g
s/PYTHON_EXECUTABLE/Python3_EXECUTABLE/g
s/PYTHON_RPATH/Python3_RPATH/g
I've also renamed the CMake module to better express its purpose and for
consistency with FindLuaAndSwig.
Differential revision: https://reviews.llvm.org/D85976
This patch takes the IO redirection logic from ScriptInterpreterPython
and moves it into the interpreter library so that it can be used by
other script interpreters. I've turned it into a RAII object so that we
don't have to worry about cleaning up in the calling code.
Differential revision: https://reviews.llvm.org/D82396
Add support for changing the stdout and stderr file in Lua's I/O library
and hook it up with the debugger's output and error file respectively
for the interactive Lua interpreter.
https://reviews.llvm.org/D82273
Add a way to quit the interactive script interpreter from a shell tests.
Currently, the only way (that I know) to exit the interactive Lua
interpreter is to send a EOF with CTRL-D. I noticed that the embedded
Python script interpreter accepts quit (while the regular python
interpreter doesn't). I've added a special case to the Lua interpreter
to do the same.
Differential revision: https://reviews.llvm.org/D82272
This was reverted due to a python2-specific bug. Re-landing with a fix
for python2.
Summary:
One small step in my long running quest to improve python exception handling in
LLDB. Replace GetInteger() which just returns an int with As<long long> and
friends, which return Expected types that can track python exceptions
Reviewers: labath, jasonmolenda, JDevlieghere, vadimcn, omjavaid
Reviewed By: labath, omjavaid
Subscribers: omjavaid, lldb-commits
Tags: #lldb
Differential Revision: https://reviews.llvm.org/D78462
Summary:
One small step in my long running quest to improve python exception handling in
LLDB. Replace GetInteger() which just returns an int with As<long long> and
friends, which return Expected types that can track python exceptions
Reviewers: labath, jasonmolenda, JDevlieghere, vadimcn
Reviewed By: labath
Subscribers: lldb-commits
Tags: #lldb
Differential Revision: https://reviews.llvm.org/D78462
Discussed on lldb-dev with Pavel Labath. This doesn't work for
background processes [causes Python to be stuck forever], and it's
unclear whether it's needed. There's no test, also. If this turns
out to be useful, it can be recommitted with a functional implementation
and a test.
Files imported by the script interpreter aren't opened by LLDB so they
don't end up in the reproducer. The solution is to explicitly add them
to the FileCollector.
Differential revision: https://reviews.llvm.org/D76626
Summary:
This is another attempt of 0bb90628b5.
The difference is that g_python_home is not declared as const. Since
some versions of python do not expect that.
Subscribers: mgorny, lldb-commits
Tags: #lldb
Differential Revision: https://reviews.llvm.org/D74998
Summary:
This change allows a hard coded relative PYTHONHOME setting. So that
python can easily be packaged together with lldb.
The change includes:
1. Extend LLDB_RELOCATABLE_PYTHON to all platforms. It defaults to ON
for platforms other than Windows, to keep the behavior compatible.
2. Allows to customize LLDB_PYTHON_HOME. But still defaults to
PYTHON_HOME.
3. LLDB_PYTHON_HOME can be a path relative to liblldb. If it is
relative, we will resolve it before send it to Py_DecodeLocale.
Subscribers: mgorny, lldb-commits
Tags: #lldb
Differential Revision: https://reviews.llvm.org/D74727
LLDB has a few different styles of header guards and they're not very
consistent because things get moved around or copy/pasted. This patch
unifies the header guards across LLDB and converts everything to match
LLVM's style.
Differential revision: https://reviews.llvm.org/D74743
This is a step towards making the initialize and terminate calls be
generated by CMake, which in turn is towards making it possible to
disable plugins at configuration time.
Differential revision: https://reviews.llvm.org/D74245
This is how it should've been and brings it more in line with
std::string_view. There should be no functional change here.
This is mostly mechanical from a custom clang-tidy check, with a lot of
manual fixups. It uncovers a lot of minor inefficiencies.
This doesn't actually modify StringRef yet, I'll do that in a follow-up.
Summary:
A *.cpp file header in LLDB (and in LLDB) should like this:
```
//===-- TestUtilities.cpp -------------------------------------------------===//
```
However in LLDB most of our source files have arbitrary changes to this format and
these changes are spreading through LLDB as folks usually just use the existing
source files as templates for their new files (most notably the unnecessary
editor language indicator `-*- C++ -*-` is spreading and in every review
someone is pointing out that this is wrong, resulting in people pointing out that this
is done in the same way in other files).
This patch removes most of these inconsistencies including the editor language indicators,
all the different missing/additional '-' characters, files that center the file name, missing
trailing `===//` (mostly caused by clang-format breaking the line).
Reviewers: aprantl, espindola, jfb, shafik, JDevlieghere
Reviewed By: JDevlieghere
Subscribers: dexonsmith, wuzish, emaste, sdardis, nemanjai, kbarton, MaskRay, atanasyan, arphaman, jfb, abidh, jsji, JDevlieghere, usaxena95, lldb-commits
Tags: #lldb
Differential Revision: https://reviews.llvm.org/D73258
The 'asynchronously' argument to both GetLLDBCommandsFromIOHandler and
GetPythonCommandsFromIOHandler is true for all call sites. This commit
simplifies the API by dropping it and giving the baton a default
argument.
The Python script interpreter makes the current debugger, target,
process, thread and frame available to interactive scripting sessions
through convenience variables. This patch does the same for Lua.
Differential revision: https://reviews.llvm.org/D71801
The current FOUND_VAR for FindLibEdit is libedit_FOUND but wasn't set by
find_package_handle_standard_args. However this isn't valid for the
package name.
The argument for FOUND_VAR is "libedit_FOUND", but only "LibEdit_FOUND" and
"LIBEDIT_FOUND" are valid names.
This fixes all the variables set by FindLibEdit to match the desired
naming scheme.
PYTHON_LIBRARIES is the canonical variable set by FindPythonLibs while
PYTHON_LIBRARY is an implementation detail. This replaces the uses of
the latter with the former.
The `-r` option for `command script import` is there for legacy
compatibility, however the can_reload flag is always set to true. This
patch removes the flag and any code that relies on it being false.
Don't create a new lua state on every operation. Share a single state
across the lifetime of the script interpreter. Add simple locking to
prevent two threads from modifying the state concurrently.
This implements a very elementary Lua script interpreter. It supports
running a single command as well as running interactively. It uses
editline if available. It's still missing a bunch of stuff though. Some
things that I intentionally ingored for now are that I/O isn't properly
hooked up (so every print goes to stdout) and the non-editline support
which is not handling a bunch of corner cases. The latter is a matter of
reusing existing code in the Python interpreter.
Discussion on the mailing list:
http://lists.llvm.org/pipermail/lldb-dev/2019-December/015812.html
Differential revision: https://reviews.llvm.org/D71234
As suggested by Pavel in a code review:
> Can we replace this (and maybe python too, while at it) with a
> Host/Config.h entry? A global definition means that one has to
> recompile everything when these change in any way, whereas in
> practice only a handful of files need this..
Differential revision: https://reviews.llvm.org/D71280
Restrict building the readline override to Linux only. It both does not
build on *BSD systems, and is largely irrelevant since they default to
using libedit over readline anyway. This restores the behavior
of the old readline override that also was built only on Linux.
Differential Revision: https://reviews.llvm.org/D69846
Fix https://bugs.llvm.org/show_bug.cgi?id=43830 while avoiding polluting the
global Python namespace.
This both reverts r357277 to rebundle a version of Python's readline module
based on libedit.
However, this patch also provides two improvements over the previous
implementation:
1. use PyMem_RawMalloc instead of PyMem_Malloc, as expected by PyOS_Readline
(prevents to segfault upon exit of interactive session)
2. patch the readline module upon embedded interpreter loading, instead of
patching it globally, which should prevent any side effect on other
modules/packages
3. only activate the patched module if libedit is actually linked in lldb
Differential Revision: https://reviews.llvm.org/D69793
Summary:
This patch updates the last user of ArgInfo::count and deletes
it. I also delete `GetNumInitArguments()` and `GetInitArgInfo()`.
Classess are callables and `GetArgInfo()` should work on them.
On python 3 it already works, of course. `inspect` is good.
On python 2 we have to add yet another special case. But hey if
python 2 wasn't crufty we wouln't need python 3.
I also delete `is_bound_method` becuase it is unused.
This path is tested in `TestStepScripted.py`
Reviewers: labath, mgorny, JDevlieghere
Reviewed By: labath, JDevlieghere
Subscribers: lldb-commits
Tags: #lldb
Differential Revision: https://reviews.llvm.org/D69742
Med Ismail Bennani