This patch mechanically replaces None with std::nullopt where the
compiler would warn if None were deprecated. The intent is to reduce
the amount of manual work required in migrating from Optional to
std::optional.
This is part of an effort to migrate from llvm::Optional to
std::optional:
https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716
This adds a new line between the real thread and the extended backtrace
thread when it's available. This should improve readability for the user.
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
The JSON dumper is very minimalistic. It pretty much only shows the
delimiting instruction IDs of every segment, so that further queries to
the SBCursor can be used to make sense of the data. It's main purpose is
to be serialized somewhat cheaply.
I also renamed untracedSegment to untracedPrefixSegment, in case in the
future we add an untracedSuffixSegment. In any case, this new name is
more explicit, which I like.
Differential Revision: https://reviews.llvm.org/D136034
This diff implements the reconstruction algorithm for the call tree and
add tests.
See TraceDumper.h for documentation and explanations.
One important detail is that the tree objects are in TraceDumper, even
though Trace.h is a better home. I'm leaving that as future work.
Another detail is that this code is as slow as dumping the entire
symolicated trace, which is not that bad tbh. The reason is that we use
symbols throughout the algorithm and we are not being careful about
memory and speed. This is also another area for future improvement.
Lastly, I made sure that incomplete traces work, i.e. you start tracing
very deep in the stack or failures randomly appear in the trace.
Differential Revision: https://reviews.llvm.org/D135917
The command is thread trace dump function-calls and as minimum will
require printing to a file in json and non-json format
I added a test
Differential Revision: https://reviews.llvm.org/D135521
This patch fixes:
lldb/source/Commands/CommandObjectThread.cpp:66:61: warning:
comparison of unsigned expression in ‘< 0’ is always false
[-Wtype-limits]
The per-PSB packet decoding logic was wrong because it was assuming that pt_insn_get_sync_offset was being udpated after every PSB. Silly me, that is not true. It returns the offset of the PSB packet after invoking pt_insn_sync_forward regardless of how many PSBs are visited later. Instead, I'm now following the approach described in https://github.com/intel/libipt/blob/master/doc/howto_libipt.md#parallel-decode for parallel decoding, which is basically what we need.
A nasty error that happened because of this is that when we had two PSBs (A and B), the following was happening
1. PSB A was processed all the way up to the end of the trace, which includes PSB B.
2. PSB B was then processed until the end of the trace.
The instructions emitted by step 2. were also emitted as part of step 1. so our trace had duplicated chunks. This problem becomes worse when you many PSBs.
As part of making sure this diff is correct, I added some other features that are very useful.
- Added a "synchronization point" event to the TraceCursor, so we can inspect when PSBs are emitted.
- Removed the single-thread decoder. Now the per-cpu decoder and single-thread decoder use the same code paths.
- Use the query decoder to fetch PSBs and timestamps. It turns out that the pt_insn_sync_forward of the instruction decoder can move past several PSBs (this means that we could skip some TSCs). On the other hand, the pt_query_sync_forward method doesn't skip PSBs, so we can get more accurate sync events and timing information.
- Turned LibiptDecoder into PSBBlockDecoder, which decodes single PSB blocks. It is the fundamental processing unit for decoding.
- Added many comments, asserts and improved error handling for clarity.
- Improved DecodeSystemWideTraceForThread so that a TSC is emitted always before a cpu change event. This was a bug that was annoying me before.
- SplitTraceInContinuousExecutions and FindLowestTSCInTrace are now using the query decoder, which can identify precisely each PSB along with their TSCs.
- Added an "only-events" option to the trace dumper to inspect only events.
I did extensive testing and I think we should have an in-house testing CI. The LLVM buildbots are not capable of supporting testing post-mortem traces of hundreds of megabytes. I'll leave that for later, but at least for now the current tests were able to catch most of the issues I encountered when doing this task.
A sample output of a program that I was single stepping is the following. You can see that only one PSB is emitted even though stepping happened!
```
thread #1: tid = 3578223
0: (event) trace synchronization point [offset = 0x0xef0]
a.out`main + 20 at main.cpp:29:20
1: 0x0000000000402479 leaq -0x1210(%rbp), %rax
2: (event) software disabled tracing
3: 0x0000000000402480 movq %rax, %rdi
4: (event) software disabled tracing
5: (event) software disabled tracing
6: 0x0000000000402483 callq 0x403bd4 ; std::vector<int, std::allocator<int>>::vector at stl_vector.h:391:7
7: (event) software disabled tracing
a.out`std::vector<int, std::allocator<int>>::vector() at stl_vector.h:391:7
8: 0x0000000000403bd4 pushq %rbp
9: (event) software disabled tracing
10: 0x0000000000403bd5 movq %rsp, %rbp
11: (event) software disabled tracing
```
This is another trace of a long program with a few PSBs.
```
(lldb) thread trace dump instructions -E -f thread #1: tid = 3603082
0: (event) trace synchronization point [offset = 0x0x80]
47417: (event) software disabled tracing
129231: (event) trace synchronization point [offset = 0x0x800]
146747: (event) software disabled tracing
246076: (event) software disabled tracing
259068: (event) trace synchronization point [offset = 0x0xf78]
259276: (event) software disabled tracing
259278: (event) software disabled tracing
no more data
```
Differential Revision: https://reviews.llvm.org/D131630
Add bindings for the `TraceCursor` to allow for programatic traversal of
traces.
This diff adds bindings for all public `TraceCursor` methods except
`GetHwClock` and also adds `SBTrace::CreateNewCursor`. A new unittest
has been added to TestTraceLoad.py that uses the new `SBTraceCursor` API
to test that the sequential and random access APIs of the `TraceCursor`
are equivalent.
This diff depends on D130925.
Test Plan:
`ninja lldb-dotest && ./bin/lldb-dotest -p TestTraceLoad`
Differential Revision: https://reviews.llvm.org/D130930
The use of `std::unique_ptr` with `TraceCursor` adds unnecessary complexity to adding `SBTraceCursor` bindings
Specifically, since `TraceCursor` is an abstract class there's no clean
way to provide "deep clone" semantics for `TraceCursorUP` short of
creating a pure virtual `clone()` method (afaict).
After discussing with @wallace, we decided there is no strong reason to
favor wrapping `TraceCursor` with `std::unique_ptr` over `std::shared_ptr`, thus this diff
replaces all usages of `std::unique_ptr<TraceCursor>` with `std::shared_ptr<TraceCursor>`.
This sets the stage for future diffs to introduce `SBTraceCursor`
bindings in a more clean fashion.
Test Plan:
Differential Revision: https://reviews.llvm.org/D130925
Refactor the command option enum values and the command argument table
to connect the two. This has two benefits:
- We guarantee that two options that use the same argument type have
the same accepted values.
- We can print the enum values and their description in the help
output. (D129707)
Differential revision: https://reviews.llvm.org/D129703
Thanks to ymeng@fb.com for coming up with this change.
`thread trace dump info` can dump some metrics that can be useful for
analyzing the performance and quality of a trace. This diff adds a --json
option for dumping this information in json format that can be easily
understood my machines.
Differential Revision: https://reviews.llvm.org/D129332
The requirements for "thread until <line number>" are:
a) If any code contributed by <line number> or the nearest subsequent of <line number> is executed before leaving the function, stop
b) If you end up leaving the function w/o triggering (a), then stop
In case of (a), since the <line number> may have multiple entries in the line table and the compiler might have scheduled/moved the relevant code across, and the lldb does not know the control flow, set breakpoints on all the line table entries of best match of <line number> i.e. exact or the nearest subsequent line.
Along with the above, currently, CommandObjectThreadUntil is also setting the breakpoints on all the subsequent line numbers after the best match and this latter part is wrong.
This issue is discussed at http://lists.llvm.org/pipermail/lldb-dev/2018-August/013979.html.
In fact, currently `TestStepUntil.py` is not actually testing step until scenarios and `test_missing_one` test fails without this patch if tests are made to run. Fixed the test as well.
Reviewed By: jingham
Differential Revision: https://reviews.llvm.org/D50304
We want to include events with metadata, like context switches, and this
requires the API to handle events with payloads (e.g. information about
such context switches). Besides this, we want to support multiple
similar events between two consecutive instructions, like multiple
context switches. However, the current implementation is not good for this because
we are defining events as bitmask enums associated with specific
instructions. Thus, we need to decouple instructions from events and
make events actual items in the trace, just like instructions and
errors.
- Add accessors in the TraceCursor to know if an item is an event or not
- Modify from the TraceDumper all the way to DecodedThread to support
- Renamed the paused event to disabled.
- Improved the tsc handling logic. I was using an API for getting the tsc from libipt, but that was an overkill that should be used when not processing events manually, but as we are already processing events, we can more easily get the tscs.
event items. Fortunately this simplified many things
- As part of this refactor, I also fixed and long stating issue, which is that some non decoding errors were being inserted in the decoded thread. I changed this so that TraceIntelPT::Decode returns an error if the decoder couldn't be set up proplerly. Then, errors within a trace are actual anomalies found in between instrutions.
All test pass
Differential Revision: https://reviews.llvm.org/D128576
https://reviews.llvm.org/D128453 recently added some safety checks for
command arguments. Unfortunately, some few commands started failing due
to that, and this diff fixes it. But fortunately, the fix is trivial, which is
simply declaring the argument that these commands will receive.
Differential Revision: https://reviews.llvm.org/D128775
The current way ot traversing the cursor is a bit uncommon and it can't handle empty traces, in fact, its invariant is that it shold always point to a valid item. This diff simplifies the cursor API and allows it to point to invalid items, thus being able to handle empty traces or to know it ran out of data.
- Removed all the granularity functionalities, because we are not actually making use of that. We can bring them back when they are actually needed.
- change the looping logic to the following:
```
for (; cursor->HasValue(); cursor->Next()) {
if (cursor->IsError()) {
.. do something for error
continue;
}
.. do something for instruction
}
```
- added a HasValue method that can be used to identify if the cursor ran out of data, the trace is empty, or the user tried to move to an invalid position via SetId() or Seek()
- made several simplifications to severals parts of the code.
Differential Revision: https://reviews.llvm.org/D128543
The requirements for "thread until <line number>" are:
a) If any code contributed by <line number> or the nearest subsequent of <line number> is executed before leaving the function, stop
b) If you end up leaving the function w/o triggering (a), then stop
In case of (a), since the <line number> may have multiple entries in the line table and the compiler might have scheduled/moved the relevant code across, and the lldb does not know the control flow, set breakpoints on all the line table entries of best match of <line number> i.e. exact or the nearest subsequent line.
Along with the above, currently, CommandObjectThreadUntil is also setting the breakpoints on all the subsequent line numbers after the best match and this latter part is wrong.
This issue is discussed at http://lists.llvm.org/pipermail/lldb-dev/2018-August/013979.html.
In fact, currently `TestStepUntil.py` is not actually testing step until scenarios and `test_missing_one` test fails without this patch if tests are made to run. Fixed the test as well.
Reviewed By: jingham
Differential Revision: https://reviews.llvm.org/D50304
In order to provide simple scripting support on top of instruction traces, a simple solution is to enhance the `dump instructions` command and allow printing in json and directly to a file. The format is verbose and not space efficient, but it's not supposed to be used for really large traces, in which case the TraceCursor API is the way to go.
- add a -j option for printing the dump in json
- add a -J option for pretty printing the json output
- add a -F option for specifying an output file
- add a -a option for dumping all the instructions available starting at the initial point configured with the other flags
- add tests for all cases
- refactored the instruction dumper and abstracted the actual "printing" logic. There are two writer implementations: CLI and JSON. This made the dumper itself much more readable and maintanable
sample output:
```
(lldb) thread trace dump instructions -t -a --id 100 -J
[
{
"id": 100,
"tsc": "43591204528448966"
"loadAddress": "0x407a91",
"module": "a.out",
"symbol": "void std::deque<Foo, std::allocator<Foo>>::_M_push_back_aux<Foo>(Foo&&)",
"mnemonic": "movq",
"source": "/usr/include/c++/8/bits/deque.tcc",
"line": 492,
"column": 30
},
...
```
Differential Revision: https://reviews.llvm.org/D128316
For the 'thread until' command, the selected thread ID, to perform the operation on, could be of the current thread or the specified thread.
Reviewed By: jingham
Differential Revision: https://reviews.llvm.org/D48865
A trace might contain events traced during the target's execution. For
example, a thread might be paused for some period of time due to context
switches or breakpoints, which actually force a context switch. Not only
that, a trace might be paused because the CPU decides to trace only a
specific part of the target, like the address filtering provided by
intel pt, which will cause pause events. Besides this case, other kinds
of events might exist.
This patch adds the method `TraceCursor::GetEvents()`` that returns the
list of events that happened right before the instruction being pointed
at by the cursor. Some refactors were done to make this change simpler.
Besides this new API, the instruction dumper now supports the -e flag
which shows pause events, like in the following example, where pauses
happened due to breakpoints.
```
thread #1: tid = 2717361
a.out`main + 20 at main.cpp:27:20
0: 0x00000000004023d9 leaq -0x1200(%rbp), %rax
[paused]
1: 0x00000000004023e0 movq %rax, %rdi
[paused]
2: 0x00000000004023e3 callq 0x403a62 ; std::vector<int, std::allocator<int> >::vector at stl_vector.h:391:7
a.out`std::vector<int, std::allocator<int> >::vector() at stl_vector.h:391:7
3: 0x0000000000403a62 pushq %rbp
4: 0x0000000000403a63 movq %rsp, %rbp
```
The `dump info` command has also been updated and now it shows the
number of instructions that have associated events.
Differential Revision: https://reviews.llvm.org/D123982
In order to support quick arbitrary access to instructions in the trace, we need
each instruction to have an id. It could be an index or any other value that the
trace plugin defines.
This will be useful for reverse debugging or for creating callstacks, as each
frame will need an instruction id associated with them.
I've updated the `thread trace dump instructions` command accordingly. It now
prints the instruction id instead of relative offset. I've also added a new --id
argument that allows starting the dump from an arbitrary position.
Differential Revision: https://reviews.llvm.org/D122254
Minor fixes needed and now `./bin/lldb-dotest -p TestTrace` passes
correctly.
- There was an incorrect iteration.
- Some error messages changed.
- The way repeat commands are handled changed a bit, so I had to create
a new --continue arg in "thread trace dump instructions" to handle this
correctly.
Differential Revision: https://reviews.llvm.org/D122023
Applied modernize-use-default-member-init clang-tidy check over LLDB.
It appears in many files we had already switched to in class member init but
never updated the constructors to reflect that. This check is already present in
the lldb/.clang-tidy config.
Differential Revision: https://reviews.llvm.org/D121481
Some dyld cross library stubs can have line information but no function. Make sure you
check that you have a valid Function object before asking it questions.
Differential Revision: https://reviews.llvm.org/D119297
This way if you have a long stack, you can issue "thread backtrace --count 10"
and then subsequent <Return>-s will page you through the stack.
This took a little more effort than just adding the repeat command, since
the GetRepeatCommand API was returning a "const char *". That meant the command
had to keep the repeat string alive, which is inconvenient. The original
API returned either a nullptr, or a const char *, so I changed the private API to
return an llvm::Optional<std::string>. Most of the patch is propagating that change.
Also, there was a little thinko in fetching the repeat command. We don't
fetch repeat commands for commands that aren't being added to history, which
is in general reasonable. And we don't add repeat commands to the history -
also reasonable. But we do want the repeat command to be able to generate
the NEXT repeat command. So I adjusted the logic in HandleCommand to work
that way.
Differential Revision: https://reviews.llvm.org/D119046
[NFC] As part of using inclusive language within the llvm project, this patch
renames master plan to controlling plan in lldb.
Reviewed By: jingham
Differential Revision: https://reviews.llvm.org/D113019
There is no reason why this function should be returning a ConstString.
While modifying these files, I also fixed several instances where
GetPluginName and GetPluginNameStatic were returning different strings.
I am not changing the return type of GetPluginNameStatic in this patch, as that
would necessitate additional changes, and this patch is big enough as it is.
Differential Revision: https://reviews.llvm.org/D111877
We added some support for this mode back in 2015, but the feature was
never productionized. It is completely untested, and there are known
major structural lldb issues that need to be resolved before this
feature can really be supported.
It also complicates making further changes to stop reply packet
handling, which is what I am about to do.
Differential Revision: https://reviews.llvm.org/D110553
Copying from the inline documentation:
```
Trace exporter plug-ins operate on traces, converting the trace data provided by an \a lldb_private::TraceCursor into a different format that can be digested by other tools, e.g. Chrome Trace Event Profiler.
Trace exporters are supposed to operate on an architecture-agnostic fashion, as a TraceCursor, which feeds the data, hides the actual trace technology being used.
```
I want to use this to make the code in https://reviews.llvm.org/D105741 a plug-in. I also imagine that there will be more and more exporters being implemented, as an exporter creates something useful out of trace data. And tbh I don't want to keep adding more stuff to the lldb/Target folder.
This is the minimal definition for a TraceExporter plugin. I plan to use this with the following commands:
- thread trace export <plug-in name> [plug-in specific args]
- This command would support autocompletion of plug-in names
- thread trace export list
- This command would list the available trace exporter plug-ins
I don't plan to create yet a "process trace export" because it's easier to start analyzing the trace of a given thread than of the entire process. When we need a process-level command, we can implement it.
I also don't plan to force each "export" command implementation to support multiple threads (for example, "thread trace start 1 2 3" or "thread trace start all" operate on many threads simultaneously). The reason is that the format used by the exporter might or might not support multiple threads, so I'm leaving this decision to each trace exporter plug-in.
Differential Revision: https://reviews.llvm.org/D106501
When the user types that command 'thread trace dump info' and there's a running Trace session in LLDB, a raw trace in bytes should be printed; the command 'thread trace dump info all' should print the info for all the threads.
Original Author: hanbingwang
Reviewed By: clayborg, wallace
Differential Revision: https://reviews.llvm.org/D105717
D104422 added the interface for TraceCursor, which is the main way to traverse instructions in a trace. This diff implements the corresponding cursor class for Intel PT and deletes the now obsolete code.
Besides that, the logic for the "thread trace dump instructions" was adapted to use this cursor (pretty much I ended up moving code from Trace.cpp to TraceCursor.cpp). The command by default traverses the instructions backwards, and if the user passes --forwards, then it's not forwards. More information about that is in the Options.td file.
Regarding the Intel PT cursor. All Intel PT cursors for the same thread share the same DecodedThread instance. I'm not yet implementing lazy decoding because we don't need it. That'll be for later. For the time being, the entire thread trace is decoded when the first cursor for that thread is requested.
Differential Revision: https://reviews.llvm.org/D105531
Replacing existing uses with AppendError.
SetError is also part of the SBI API. This remains
but instead of calling the underlying SetError it
will call AppendError.
Reviewed By: teemperor
Differential Revision: https://reviews.llvm.org/D104768
This adds a basic SB API for creating and stopping traces.
Note: This doesn't add any APIs for inspecting individual instructions. That'd be a more complicated change and it might be better to enhande the dump functionality to output the data in binary format. I'll leave that for a later diff.
This also enhances the existing tests so that they test the same flow using both the command interface and the SB API.
I also did some cleanup of legacy code.
Differential Revision: https://reviews.llvm.org/D103500
This is part 2, covering the commands source.
Some uses remain where it's tricky to see what the
logic is or they are not used with AppendError.
Reviewed By: teemperor
Differential Revision: https://reviews.llvm.org/D104448
This converts a default constructor's member initializers into C++11
default member initializers. This patch was automatically generated with
clang-tidy and the modernize-use-default-member-init check.
$ run-clang-tidy.py -header-filter='lldb' -checks='-*,modernize-use-default-member-init' -fix
This is a mass-refactoring patch and this commit will be added to
.git-blame-ignore-revs.
Differential revision: https://reviews.llvm.org/D103483
This implements the interactive trace start and stop methods.
This diff ended up being much larger than I anticipated because, by doing it, I found that I had implemented in the beginning many things in a non optimal way. In any case, the code is much better now.
There's a lot of boilerplate code due to the gdb-remote protocol, but the main changes are:
- New tracing packets: jLLDBTraceStop, jLLDBTraceStart, jLLDBTraceGetBinaryData. The gdb-remote packet definitions are quite comprehensive.
- Implementation of the "process trace start|stop" and "thread trace start|stop" commands.
- Implementaiton of an API in Trace.h to interact with live traces.
- Created an IntelPTDecoder for live threads, that use the debugger's stop id as checkpoint for its internal cache.
- Added a functionality to stop the process in case "process tracing" is enabled and a new thread can't traced.
- Added tests
I have some ideas to unify the code paths for post mortem and live threads, but I'll do that in another diff.
Differential Revision: https://reviews.llvm.org/D91679
Depends on D90490.
The stop command is simple and invokes the new method Trace::StopTracingThread(thread).
On the other hand, the start command works by delegating its implementation to a CommandObject provided by the Trace plugin. This is necessary because each trace plugin needs different options for this command. There's even the chance that a Trace plugin can't support live tracing, but instead supports offline decoding and analysis, which means that "thread trace dump instructions" works but "thread trace start" doest. Because of this and a few other reasons, it's better to have each plugin provide this implementation.
Besides, I'm using the GetSupportedTraceType method introduced in D90490 to quickly infer what's the trace plug-in that works for the current process.
As an implementation note, I moved CommandObjectIterateOverThreads to its header so that I can use it from the IntelPT plugin. Besides, the actual start and stop logic for intel-pt is not part of this diff.
Reviewed By: clayborg
Differential Revision: https://reviews.llvm.org/D90729
Kazu Hirata