This patch improves the ScriptedPythonInterface::Dispatch method to
support passing lldb_private types to the python implementation.
This will allow, for instance, the Scripted Process python implementation
to report errors when reading memory back to lldb.
To do so, the Dispatch method will transform the private types in the
parameter pack into `PythonObject`s to be able to pass them down to the
python methods.
Then, if the call succeeded, the transformed arguments will be converted
back to their original type and re-assigned in the parameter pack, to
ensure pointers and references behaviours are preserved.
This patch also updates various scripted process python class and tests
to reflect this change.
rdar://100030995
Differential Revision: https://reviews.llvm.org/D134033
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
Allow `in_call_stack` to be imported in either of the following ways:
```
command script import path/to/in_call_stack.py
command script import lldb.utils.in_call_stack
```
rdar://102249295
Differential Revision: https://reviews.llvm.org/D137860
This patch replaces the backing file path key to "file_path" to keep it
consistent.
rdar://101652618
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
It can happen that the originator of a crash report doesn't have access
to certain images. When that's the case, ReportCrash won't show the
source info in the crash report stack frames, but only the stack address
and image name.
This patch fixes a bug in the crashlog stackframe parser regular
expression to optionally match the source info group.
rdar://101934135
Differential Revision: https://reviews.llvm.org/D137466
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
For an exception crashlog, the thread backtraces aren't usually very helpful
and instead, developpers look at the "Application Specific Backtrace" that
was generated by `objc_exception_throw`.
LLDB could already parse and symbolicate these Application Specific Backtraces
for regular textual-based crashlog, so this patch adds support to parse them
in JSON crashlogs, and materialize them a HistoryThread extending the
crashed ScriptedThread.
This patch also includes the Application Specific Information messages
as part of the process extended crash information log. To do so, the
ScriptedProcess Python interface has a new GetMetadata method that
returns an arbitrary dictionary with data related to the process.
rdar://93207586
Differential Revision: https://reviews.llvm.org/D126260
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This patch updates the image_regex_uuid matcher to match null-UUID
images in the plain text crashlog parser.
It updates the regex to match one or more '?' characters or the image
full path.
rdar://100904019
Differential Revision: https://reviews.llvm.org/D135482
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This patch adds support for 32bit stack frame addresses in the `crashlog`
command.
For crash reports that are generated from a arm64_32 process, `PAGEZERO`
is loaded at 0x00004000 so no code address will be less than 0x4000.
This patch changes the crashlog frame address regex group to match
addresses as small as 4 hex characters.
rdar://100805026
Differential Revision: https://reviews.llvm.org/D135310
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
The python "open" function will use the default encoding for the
locale (the result of "locale.getpreferredencoding()"). Explicitly set
the locale to utf-8 when opening the crashlog for writing, as there may
be non-ascii symbols in there (for example, Swift uses "τ" to indicate
generic parameters).
rdar://101402755
Differential Revision: https://reviews.llvm.org/D136798
I went over the output of the following mess of a command:
(ulimit -m 2000000; ulimit -v 2000000; git ls-files -z | parallel
--xargs -0 cat | aspell list --mode=none --ignore-case | grep -E
'^[A-Za-z][a-z]*$' | sort | uniq -c | sort -n | grep -vE '.{25}' |
aspell pipe -W3 | grep : | cut -d' ' -f2 | less)
and proceeded to spend a few days looking at it to find probable typos
and fixed a few hundred of them in all of the llvm project (note, the
ones I found are not anywhere near all of them, but it seems like a
good start).
Differential revision: https://reviews.llvm.org/D131122
This patch improve exception reporting when loading a crash report in a
scripted process. Now, we parse the `exception` dictionary from the
crash report use it the create a higher fidelity `MachException` stop info.
This patch also updates the test to reflect that change.
rdar://97096486
Differential Revision: https://reviews.llvm.org/D131086
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This patch parses CrashLog exception data from the raw
text format and adapts it to the new JSON format.
This is necessary for feature parity between the 2 formats.
Differential Revision: https://reviews.llvm.org/D131719
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
A spiritual follow up to D131032. I noticed some regex could be simplified.
This does some of the following:
1. Removes unused capture groups
2. Uses non-capturing `(?:...)` groups where grouping is needed but capturing isn't
3. Removes trailing `.*`
4. Uses `\d` over `[0-9]`
5. Uses raw strings
6. Uses `{N,}` to indicate N-or-more
Also improves the call site of a `re.findall`.
Differential Revision: https://reviews.llvm.org/D131305
This patch introduces a new option to the crashlog command to get the
the script version.
Since `crashlog.py` is not actually versioned, this returns lldb's
version instead.
rdar://98392669
Differential Revision: https://reviews.llvm.org/D131542
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This patch changes the CrashLogParser class to be both the base class
and a Factory for the JSONCrashLogParser & TextCrashLogParser.
That should help remove some code duplication and ensure both class
have a parse method.
Differential Revision: https://reviews.llvm.org/D131085
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
Sometimes, it can happen that a crash report has null images in its list
of used binaries. This manifests like such:
```
0x0 - 0xffffffffffffffff ??? (*) <00000000-0000-0000-0000-000000000000> ???
```
When fetching debug symbols to symbolicate the crashlog stackframe,
having null images causes `dsymForUUID` to hang for few seconds.
This patch addresses that by skipping null images from being load by the
scripted process.
rdar://97419487
Differential Revision: https://reviews.llvm.org/D131038
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This patch introduces a new option for the interactive crashlog mode,
that will prevent it from dumping the `process status` & `thread backtrace`
output to the debugger console.
This is necessary when lldb in running from an IDE, to prevent flooding
the console with information that should be already present in the UI.
rdar://96813296
Differential Revision: https://reviews.llvm.org/D131036
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
In can happen when creating stackshot crash report that that key is missing.
Moreover, we try to parse that key but don't use it, or need it, since we
fetch images and symbolicate the stackframes using the binaries UUIDs.
This is why this patch removes everything that is related to the
`process_path`/`procPath` parsing.
rdar://95054188
Differential Revision: https://reviews.llvm.org/D131033
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This patch updates the regular expression matching stackframes in
crashlog to allow addresses that are 7 characters long and more (vs. 8
characters previously).
It changes the `0x[0-9a-fA-F]{7}[0-9a-fA-F]+` by `0x[0-9a-fA-F]{7,}`.
rdar://97684839
Differential Revision: https://reviews.llvm.org/D131032
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This patch allows the crashlog script to surface its errors to lldb by
using the provided SBCommandReturnObject argument.
rdar://95048193
Differential Revision: https://reviews.llvm.org/D129614
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This patch introduces a new flag for the interactive crashlog mode, that
allow the user to specify, which target to use to create the scripted
process.
This can be very useful when lldb already have few targets created:
Instead of taking the first one (zeroth index), we will use that flag to
create a new target. If the user didn't provide a target path, we will rely
on the symbolicator to create a targer.If that fails and there are already
some targets loaded in lldb, we use the first one.
rdar://94682869
Differential Revision: https://reviews.llvm.org/D129611
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This patch changes the `crashlog` command behavior to print the help
message if no argument was provided with the command.
rdar://94576026
Differential Revision: https://reviews.llvm.org/D127362
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This patch subtracts 1 to the pc of any frame above frame 0 to get the
previous line entry and display the right line in the debugger.
This also rephrase some old comment from `48d157dd4`.
rdar://92686666
Differential Revision: https://reviews.llvm.org/D125928
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
Avoid a OverflowError (an underflow really) when the pc is zero. This
can happen for "unknown frames" where the crashlog generator reports a
zero pc. We could omit them altogether, but if they're part of the
crashlog it seems fair to display them in lldb as well.
rdar://92686666
Differential revision: https://reviews.llvm.org/D125716
crashlog.py catches every exception in order to format them. This
results in both the exception name as well as the backtrace getting
swallowed.
Here's an example of the current output:
error: python exception: in method 'SBTarget_ResolveLoadAddress', argument 2 of type 'lldb::addr_t'
Compare this to the output without the custom exception handling:
Traceback (most recent call last):
File "[...]/site-packages/lldb/macosx/crashlog.py", line 929, in __call__
SymbolicateCrashLogs(debugger, shlex.split(command))
File "[...]/site-packages/lldb/macosx/crashlog.py", line 1239, in SymbolicateCrashLogs
SymbolicateCrashLog(crash_log, options)
File "[...]/site-packages/lldb/macosx/crashlog.py", line 1006, in SymbolicateCrashLog
thread.dump_symbolicated(crash_log, options)
File "[...]/site-packages/lldb/macosx/crashlog.py", line 124, in dump_symbolicated
symbolicated_frame_addresses = crash_log.symbolicate(
File "[...]/site-packages/lldb/utils/symbolication.py", line 540, in symbolicate
if symbolicated_address.symbolicate(verbose):
File "[...]/site-packages/lldb/utils/symbolication.py", line 98, in symbolicate
sym_ctx = self.get_symbol_context()
File "[...]/site-packages/lldb/utils/symbolication.py", line 77, in get_symbol_context
sb_addr = self.resolve_addr()
File "[...]/site-packages/lldb/utils/symbolication.py", line 69, in resolve_addr
self.so_addr = self.target.ResolveLoadAddress(self.load_addr)
File "[...]/site-packages/lldb/__init__.py", line 10675, in ResolveLoadAddress
return _lldb.SBTarget_ResolveLoadAddress(self, vm_addr)
OverflowError: in method 'SBTarget_ResolveLoadAddress', argument 2 of type 'lldb::addr_t'
This patch removes the custom exception handling and lets LLDB or the
default exception handler deal with it instead.
Differential revision: https://reviews.llvm.org/D125589
When using dsymForUUID, the majority of time symbolication a crashlog
with crashlog.py is spent waiting for it to complete. Currently, we're
calling dsymForUUID sequentially when iterating over the modules. We can
drastically cut down this time by calling dsymForUUID in parallel. This
patch uses Python's ThreadPoolExecutor (introduced in Python 3.2) to
parallelize this IO-bound operation.
The performance improvement is hard to benchmark, because even with an
empty local cache, consecutive calls to dsymForUUID for the same UUID
complete faster. With warm caches, I'm seeing a ~30% performance
improvement (~90s -> ~60s). I suspect the gains will be much bigger for
a cold cache.
dsymForUUID supports batching up multiple UUIDs. I considered going that
route, but that would require more intrusive changes. It would require
hoisting the logic out of locate_module_and_debug_symbols which we
explicitly document [1] as a feature of Symbolication.py to locate
symbol files.
[1] https://lldb.llvm.org/use/symbolication.html
Differential reviison: https://reviews.llvm.org/D125107
On arm64 targets, when the crashing pc is 0, the caller
frame can be found by looking at $lr, but the crash
reports don't use that trick to show the actual crashing
frame. This patch adds that stack frame that lldb shows.
Also fix an issue where some register names were printed
as having a prefix of 'None'.
Differential Revision: https://reviews.llvm.org/D125042
rdar://92631787
Previously, the ScriptedThread used the thread index as the thread id.
This patch parses the crashlog json to extract the actual thread "id" value,
and passes this information to the Crashlog ScriptedProcess blueprint,
to create a higher fidelity ScriptedThreaad.
It also updates the blueprint to show the thread name and thread queue.
Finally, this patch updates the interactive crashlog test to reflect
these changes.
rdar://90327854
Differential Revision: https://reviews.llvm.org/D122422
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This patch pipes down the `-a|--load-all` crashlog command option to the
Scripted Process initializer to load all the images used by crashed
process instead of only loading the images related to the crashed
thread.
This allows us to recreate artificial frames also for the non-crashed
scripted threads.
rdar://90396265
Differential Revision: https://reviews.llvm.org/D121826
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This patch makes the crashlog interactive mode show the scripted process
status with the crashed scripted thread backtrace after launching it.
rdar://89634338
Differential Revision: https://reviews.llvm.org/D121038
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This patch introduces a new way to load modules programatically with
Scripted Processes. To do so, the scripted process blueprint holds a
list of dictionary describing the modules to load, which their path or
uuid, load address and eventually a slide offset.
LLDB will fetch that list after launching the ScriptedProcess, and
iterate over each entry to create the module that will be loaded in the
Scripted Process' target.
The patch also refactors the StackCoreScriptedProcess test to stop
inside the `libbaz` module and make sure it's loaded correctly and that
we can fetch some variables from it.
rdar://74520238
Differential Revision: https://reviews.llvm.org/D120969
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This patch should fix the test failure on scripted_crashlog_json.test.
The failure is happening because crash reporter will obfuscate the
executable path in the crashlog, if it is located inside the user's
home directory and replace it with `/USER/*/` as a placeholder.
To fix that, we can patch the placeholder with the executable path
before loading the crashlog in lldb.
This also fixes a bug where we would create another target when loading
the crashlog in a scripted process, even if lldb already had a target
for it. Now, crashlog will only create a target if there is none in lldb.
Differential Revision: https://reviews.llvm.org/D120598
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
In some cases, it can happen that crashlogs don't have any signal in
the exception, which causes the parser to crash.
This fixes the parsing by checking if the `signal` field is in the
`exception` dictionary before trying to access it.
rdar://84552251
Differential Revision: https://reviews.llvm.org/D119504
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This patch introduces a new type of ScriptedProcess: CrashLogScriptedProcess.
It takes advantage of lldb's crashlog parsers and Scripted Processes to
reconstruct a static debugging session with symbolicated stackframes, instead
of just dumping out everything in the user's terminal.
The crashlog command also has an interactive mode that only provide a
very limited experience. This is why this patch removes all the logic
for this interactive mode and creates CrashLogScriptedProcess instead.
This will fetch and load all the libraries that were used by the crashed
thread and re-create all the frames artificially.
rdar://88721117
Differential Revision: https://reviews.llvm.org/D119501
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This patch adds the ability for ScriptedThread to load artificial stack
frames. To do so, the interpreter instance can create a list that will
contain the frame index and its pc address.
Then, when the Scripted Process plugin stops, it will refresh its
Scripted Threads state by invalidating their register context and load
to list from the interpreter object and reconstruct each frame.
This patch also removes all of the default implementation for
`get_stackframes` from the derived ScriptedThread classes, and add the
interface code for the Scripted Thread Interface.
rdar://88721095
Differential Revision: https://reviews.llvm.org/D119388
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This patch removes the `my_scripted_process.py` blueprint since it's not
used anymore.
The patch also updates the base ScriptedProcess and ScriptedThread
initializers to automatically initialize convinience variables, to
access debugger from the ScriptedProcess, access the SBProcess and
ScriptedProcess object from a ScriptedThread instance.
Differential Revision: https://reviews.llvm.org/D119386
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
The symbolicator assumes that the first image in the image list is the
main image. That isn't always the case. For JSON crashlogs we can use
the procName to move the main image to the front of the list.
rdar://83907760
Instead trying to pro-actively determine if the first line in a
crashlog contains meta data, change the heuristic to do the following:
1. To trying to parse the whole file. If that fails, then:
2. Strip the first line and try parsing the remainder of the file. If
that fails, then:
3. Fall back to the textual crashlog parser.
rdar://88580543
Differential revision: https://reviews.llvm.org/D119755
This patch fixes the register parser for arm64 crashlogs.
Compared to x86_64 crashlogs, the arm64 crashlogs nests the general
purpose registers into a separate dictionary within `thread_state`
dictionary. It uses the dictionary key as the the register number.
Differential Revision: https://reviews.llvm.org/D119168
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This patch fixes a timeout issue on the ScriptedProcess test that was
happening on intel platforms. The timeout was due to a misreporting of
the StopInfo in the ScriptedThread that caused the ScriptedProcess to
never stop.
To solve this, this patch changes the way a ScriptedThread reports its
stop reason by making it more architecture specific. In order to do so,
this patch also refactors the ScriptedProcess & ScriptedThread
initializer methods to provide an easy access to the target architecture.
Differential Revision: https://reviews.llvm.org/D118484
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
Previously, importing `crashlog` resulted in a message being printed. The
message was about other commands (those in heap.py), not `crashlog`. The
changes in D117237 made it so that the heap.py messages were printed only when
importing `lldb.macosx.heap`, not when importing `lldb.macosx.crashlog`. Some
users may see no output and think `crashlog` wasn't successfully loaded. This
ensures users see that `crashlog` is loaded.
rdar://88283132
Differential Revision: https://reviews.llvm.org/D119155
Rosetta crashlogs can have their own thread register state. Unlike the
other registers which ware directly listed under "threadState", the
Rosetta registers are nested under their own key in the JSON, as
illustrated below:
{
"threadState":
{
"rosetta":
{
"tmp2":
{
"value": 4935057216
},
"tmp1":
{
"value": 4365863188
},
"tmp0":
{
"value": 18446744073709551615
}
}
}
}
Med Ismail Bennani