This is the first part of support for reading MTE tags from Linux
core files. The format is documented here:
https://www.kernel.org/doc/html/latest/arm64/memory-tagging-extension.html#core-dump-support
This patch adds a method to unpack from the format the core
file uses, which is different to the one chosen for GDB packets.
MemoryTagManagerAArch64MTE is not tied one OS so another OS
might choose a different format in future. However, infrastructure
to handle that would go untested until then so I've chosen not to
attempt to handle that.
Reviewed By: omjavaid
Differential Revision: https://reviews.llvm.org/D129487
As discusses offline with @jj10305, we are updating some naming used throughout the code, specially in the json schema
- traceBuffer -> iptTrace
- core -> cpu
Differential Revision: https://reviews.llvm.org/D127817
This is the final functional patch to support intel pt decoding per cpu.
It works by doing the following:
- First, all context switches are split by tid and sorted in order. This produces a list of continuous executes per thread per core.
- Then, all intel pt subtraces are split by PSB boundaries and assigned to individual thread continuous executions on the same core by doing simple TSC-based comparisons.
- With this, we have, per thread, a sorted list of continuous executions each one with a list of intel pt subtraces. Up to this point, this is really fast because no instructions were actually decoded.
- Then, each thread can be decoded by traversing their continuous executions and intel pt subtraces. An advantage of having these continuous executions is that we can identify if a continuous exexecution doesn't have intel pt data, and thus has a gap in it. We can later to more sofisticated comparisons to identify if within a continuous execution there are gaps.
I'm adding a test as well.
Differential Revision: https://reviews.llvm.org/D126394
:q!
This diff is massive, but it's because it connects the client with lldb-server
and also ensures that the postmortem case works.
- Flatten the postmortem trace schema. The reason is that the schema has become quite complex due to the new multicore case, which defeats the original purpose of having a schema that could work for every trace plug-in. At this point, it's better that each trace plug-in defines it's own full schema. This means that the only common field is "type".
-- Because of this new approach, I merged the "common" trace load and saving functionalities into the IntelPT one. This simplified the code quite a bit. If we eventually implement another trace plug-in, we can see then what we could reuse.
-- The new schema, which is flattened, has now better comments and is parsed better. A change I did was to disallow hex addresses, because they are a bit error prone. I'm asking now to print the address in decimal.
-- Renamed "intel" to "GenuineIntel" in the schema because that's what you see in /proc/cpuinfo.
- Implemented reading the context switch trace data buffer. I had to do
some refactors to do that cleanly.
-- A major change that I did here was to simplify the perf_event circular buffer reading logic. It was too complex. Maybe the original Intel author had something different in mind.
- Implemented all the necessary bits to read trace.json files with per-core data.
- Implemented all the necessary bits to save to disk per-core trace session.
- Added a test that ensures that parsing and saving to disk works.
Differential Revision: https://reviews.llvm.org/D126015
- Add a warnings field in the jLLDBGetState response, for warnings to be delivered to the client for troubleshooting. This removes the need to silently log lldb-server's llvm::Errors and not expose them easily to the user
- Simplify the tscPerfZeroConversion struct and schema. It used to extend a base abstract class, but I'm doubting that we'll ever add other conversion mechanisms because all modern kernels support perf zero. It is also the one who is supposed to work with the timestamps produced by the context switch trace, so expecting it is imperative.
- Force tsc collection for cpu tracing.
- Add a test checking that tscPerfZeroConversion is returned by the GetState request
- Add a pre-check for cpu tracing that makes sure that perf zero values are available.
Differential Revision: https://reviews.llvm.org/D125932
This diffs implements per-core tracing on lldb-server. It also includes tests that ensure that tracing can be initiated from the client and that the jLLDBGetState ppacket returns the list of trace buffers per core.
This doesn't include any decoder changes.
Finally, this makes some little changes here and there improving the existing code.
A specific piece of code that can't reliably be tested is when tracing
per core fails due to permissions. In this case we add a
troubleshooting message and this is the manual test:
```
/proc/sys/kernel/perf_event_paranoid set to 1
(lldb) process trace start --per-core-tracing error: perf event syscall failed: Permission denied
You might need that /proc/sys/kernel/perf_event_paranoid has a value of 0 or -1.
``
Differential Revision: https://reviews.llvm.org/D124858
I'm refactoring IntelPTThreadTrace into IntelPTSingleBufferTrace so that it can
both single threads or single cores. In this diff I'm basically renaming the
class, moving it to its own file, and removing all the pieces that are not used
along with some basic cleanup.
Differential Revision: https://reviews.llvm.org/D124648
In order to open perf events per core, we need to first get the list of
core ids available in the system. So I'm adding a function that does
that by parsing /proc/cpuinfo. That seems to be the simplest and most
portable way to do that.
Besides that, I made a few refactors and renames to reflect better that
the cpu info that we use in lldb-server comes from procfs.
Differential Revision: https://reviews.llvm.org/D124573
Update the response schema of the TraceGetState packet and add
Intel PT specific response structure that contains the TSC conversion,
if it exists. The IntelPTCollector loads the TSC conversion and caches
it to prevent unnecessary calls to perf_event_open. Move the TSC conversion
calculation from Perf.h to TraceIntelPTGDBRemotePackets.h to remove
dependency on Linux specific headers.
Differential Revision: https://reviews.llvm.org/D122246
- Add PerfEvent class to handle creating ring buffers and handle the resources associated with a perf_event
- Refactor IntelPT collection code to use this new API
- Add TSC to timestamp conversion logic with unittest
Differential Revision: https://reviews.llvm.org/D121734
- Rename IntelPTManager class and files to IntelPTCollector
- Change GetTimestampCounter API to general trace counter API,
GetCounter
Differential Revision: https://reviews.llvm.org/D121711
Applied modernize-use-default-member-init clang-tidy check over LLDB.
It appears in many files we had already switched to in class member init but
never updated the constructors to reflect that. This check is already present in
the lldb/.clang-tidy config.
Differential Revision: https://reviews.llvm.org/D121481
This is to be used when you want to know what subranges
of a larger range have memory tagging. Like MakeTaggedRange
but memory without tags is skipped and you get a list of ranges back.
Will be used later by DumpDataExtractor to show memory tags.
MakeTaggedRanges assumes that the memory regions it is
given are sorted in ascending order and do not overlap.
For the current use case where you get regions from
GetMemoryRegions and are on some Linux like OS, this is
reasonable to assume.
I've used asserts to check those conditions. In future
any API binding will check them up front to prevent a crash.
Reviewed By: omjavaid
Differential Revision: https://reviews.llvm.org/D112824
This better describes the intent of the method. Which for AArch64
is removing the top byte which includes the memory tags.
It does not include pointer signatures, for those we need to use
the ABI plugin. The rename makes this a little more clear.
It's a bit awkward that the memory tag manager is removing the whole
top byte not just the memory tags but it's an improvement for now.
Reviewed By: omjavaid
Differential Revision: https://reviews.llvm.org/D117671
DataEncoder was previously made to modify data within an existing buffer. As the code progressed, new clients started using DataEncoder to create binary data. In these cases the use of this class was possibly, but only if you knew exactly how large your buffer would be ahead of time. This patchs adds the ability for DataEncoder to own a buffer that can be dynamically resized as data is appended to the buffer.
Change in this patch:
- Allow a DataEncoder object to be created that owns a DataBufferHeap object that can dynamically grow as data is appended
- Add new methods that start with "Append" to append data to the buffer and grow it as needed
- Adds full testing of the API to assure modifications don't regress any functionality
- Has two constructors: one that uses caller owned data and one that creates an object with object owned data
- "Append" methods only work if the object owns it own data
- Removes the ability to specify a shared memory buffer as no one was using this functionality. This allows us to switch to a case where the object owns its own data in a DataBufferHeap that can be resized as data is added
"Put" methods work on both caller and object owned data.
"Append" methods work on only object owned data where we can grow the buffer. These methods will return false if called on a DataEncoder object that has caller owned data.
The main reason for these modifications is to be able to use the DateEncoder objects instead of llvm::gsym::FileWriter in https://reviews.llvm.org/D113789. This patch wants to add the ability to create symbol table caching to LLDB and the code needs to build binary caches and save them to disk.
Reviewed By: labath
Differential Revision: https://reviews.llvm.org/D115073
Although I cannot find any mention of this in the specification, both
gdb and lldb agree on sending an initial + packet after establishing the
connection.
OTOH, gdbserver and lldb-server behavior is subtly different. While
lldb-server *expects* the initial ack, and drops the connection if it is
not received, gdbserver will just ignore a spurious ack at _any_ point
in the connection.
This patch changes lldb's behavior to match that of gdb. An ACK packet
is ignored at any point in the connection (except when expecting an ACK
packet, of course). This is inline with the "be strict in what you
generate, and lenient in what you accept" philosophy, and also enables
us to remove some special cases from the server code. I've extended the
same handling to NAK (-) packets, mainly because I don't see a reason to
treat them differently here.
(The background here is that we had a stub which was sending spurious
+ packets. This bug has since been fixed, but I think this change makes
sense nonetheless.)
Differential Revision: https://reviews.llvm.org/D114520
[NFC] This patch fixes URLs containing "master". Old URLs were either broken or
redirecting to the new URL.
Reviewed By: #libc, ldionne, mehdi_amini
Differential Revision: https://reviews.llvm.org/D113186
Unify the listen and connect code inside lldb-server to use
ConnectionFileDescriptor uniformly rather than a mix of it and Acceptor.
This involves:
- adding a function to map legacy values of host:port parameter
(including legacy server URLs) into CFD-style URLs
- adding a callback to return "local socket id" (i.e. UNIX socket path
or TCP port number) between listen() and accept() calls in CFD
- adding a "unix-abstract-accept" scheme to CFD
As an additional advantage, this permits lldb-server to accept any URL
known to CFD including the new serial:// scheme. Effectively,
lldb-server can now listen on the serial port. Tests for connecting
over a pty are added to test that.
Differential Revision: https://reviews.llvm.org/D111964
There is no reason why this function should be returning a ConstString.
While modifying these files, I also fixed several instances where
GetPluginName and GetPluginNameStatic were returning different strings.
I am not changing the return type of GetPluginNameStatic in this patch, as that
would necessitate additional changes, and this patch is big enough as it is.
Differential Revision: https://reviews.llvm.org/D111877
Move DynamicRegisterInfo from the internal lldbPluginProcessUtility
library to the public lldbTarget library. This is a prerequisite
towards ABI plugin changes that are going to pass DynamicRegisterInfo
parameters.
Differential Revision: https://reviews.llvm.org/D110942
Add a convenience method to add supplementary registers that takes care
of adding invalidate_regs to all (potentially) overlapping registers.
Differential Revision: https://reviews.llvm.org/D110023
Set value_regs and invalidate_regs in RegisterInfo pushed onto m_regs
to nullptr, to ensure that the temporaries passed there are not
accidentally used.
Differential Revision: https://reviews.llvm.org/D109879
In all these years, we haven't found a use for this function (it has
zero callers). Lets just remove the boilerplate.
Differential Revision: https://reviews.llvm.org/D109600
The type field is a signed integer.
(https://sourceware.org/gdb/current/onlinedocs/gdb/General-Query-Packets.html)
However it's not packed in the packet in the way
you might think. For example the type -1 should be:
qMemTags:<addr>,<len>:ffffffff
Instead of:
qMemTags:<addr>,<len>:-1
This change makes lldb-server's parsing more strict
and adds more tests to check that we handle negative types
correctly in lldb and lldb-server.
We only support one tag type value at this point,
for AArch64 MTE, which is positive. So this doesn't change
any of those interactions. It just brings us in line with GDB.
Also check that the test target has MTE. Previously
we just checked that we were AArch64 with a toolchain
that supports MTE.
Finally, update the tag type check for QMemTags to use
the same conversion steps that qMemTags now does.
Using static_cast can invoke UB and though we do do a limit
check to avoid this, I think it's clearer with the new method.
Reviewed By: omjavaid
Differential Revision: https://reviews.llvm.org/D104914
This adds memory tag writing to Process and the
GDB remote code. Supporting work for the
"memory tag write" command. (to follow)
Process WriteMemoryTags is similair to ReadMemoryTags.
It will pack the tags then call DoWriteMemoryTags.
That function will send the QMemTags packet to the gdb-remote.
The QMemTags packet follows the GDB specification in:
https://sourceware.org/gdb/current/onlinedocs/gdb/General-Query-Packets.html#General-Query-Packets
Note that lldb-server will be treating partial writes as
complete failures. So lldb doesn't need to handle the partial
write case in any special way.
Reviewed By: omjavaid
Differential Revision: https://reviews.llvm.org/D105181
Change SetCurrentThread*() logic not to include the zero padding
in PID/TID that was a side effect of 02ef0f5ab4. This should fix
problems caused by sending 64-bit integers to 32-bit servers. Reported
by Ted Woodward.
Differential Revision: https://reviews.llvm.org/D106832
PackTags is used by to compress tags to go in the QMemTags packet
and be passed to ptrace when writing memory tags.
The behaviour of RepeatTagsForRange matches that described for QMemTags
in the GDB documentation:
https://sourceware.org/gdb/current/onlinedocs/gdb/General-Query-Packets.html#General-Query-Packets
In addition, unpacking tags with number of tags 0 now means
do not check that number of tags matches the range.
This will be used by lldb-server to unpack tags before repeating
them to fill the requested range.
Reviewed By: omjavaid
Differential Revision: https://reviews.llvm.org/D105179
Previously GetMemoryTagManager checked many things in one:
* architecture supports memory tagging
* process supports memory tagging
* memory range isn't inverted
* memory range is all tagged
Since writing follow up patches for tag writing (in review
at the moment) it has become clear that this gets unwieldy
once we add the features needed for that.
It also implies that the memory tag manager is tied to the
range you used to request it with but it is not. It's a per
process object.
Instead:
* GetMemoryTagManager just checks architecture and process.
* Then the MemoryTagManager can later be asked to check a
memory range.
This is better because:
* We don't imply that range and manager are tied together.
* A slightly diferent range calculation for tag writing
doesn't add more code to Process.
* Range checking code can now be unit tested.
Reviewed By: omjavaid
Differential Revision: https://reviews.llvm.org/D105630
This reverts commit 82a3883715.
The original version had a copy-paste error: using the Interrupt timeout
for the ResumeSynchronous wait, which is clearly wrong. This error would
have been evident with real use, but the interrupt is long enough that it
only caused one testsuite failure (in the Swift fork).
Anyway, I found that mistake and fixed it and checked all the other places
where I had to plumb through a timeout, and added a test with a short
interrupt timeout stepping over a function that takes 3x the interrupt timeout
to complete, so that should detect a similar mistake in the future.
Refactor SetCurrentThread() and SetCurrentThreadForRun() to reduce code
duplication and simplify it. Both methods now call common
SendSetCurrentThreadPacket() that implements the common protocol
exchange part (the only variable is sending `Hg` vs `Hc`) and returns
the selected TID. The logic is rewritten to use a StreamString
instead of snprintf().
A side effect of the change is that thread-id sent is now zero-padded.
However, this should not have practical impact on the server as both
forms are equivalent.
Differential Revision: https://reviews.llvm.org/D100459
These were disabled in 473a3a773e
because they failed on 32 bit platforms. (Arm for sure but I assume
any 32 bit)
This was due to the printf formatter used. These assumed
that types like uint64_t/size_t would be certain size/type and
that changes on 32 bit.
Instead use "z" to print the size_t and PRI<...> formatters
for the addr_t (always uint64_t) and the int32_t.
This adds GDB client support for the qMemTags packet
which reads memory tags. Following the design
which was recently committed to GDB.
https://sourceware.org/gdb/current/onlinedocs/gdb/General-Query-Packets.html#General-Query-Packets
(look for qMemTags)
lldb commands will use the new Process methods
GetMemoryTagManager and ReadMemoryTags.
The former takes a range and checks that:
* The current process architecture has an architecture plugin
* That plugin provides a MemoryTagManager
* That the range of memory requested lies in a tagged range
(it will expand it to granules for you)
If all that was true you get a MemoryTagManager you
can give to ReadMemoryTags.
This two step process is done to allow commands to get the
tag manager without having to read tags as well. For example
you might just want to remove a logical tag, or error early
if a range with tagged addresses is inverted.
Note that getting a MemoryTagManager doesn't mean that the process
or a specific memory range is tagged. Those are seperate checks.
Having a tag manager just means this architecture *could* have
a tagging feature enabled.
An architecture plugin has been added for AArch64 which
will return a MemoryTagManagerAArch64MTE, which was added in a
previous patch.
Reviewed By: omjavaid
Differential Revision: https://reviews.llvm.org/D95602
This adds the MemoryTagManager class and a specialisation
of that class for AArch64 MTE tags. It provides a generic
interface for various tagging operations.
Adding/removing tags, diffing tagged pointers, etc.
Later patches will use this manager to handle memory tags
in generic code in both lldb and lldb-server.
Since it will be used in both, the base class header is in
lldb/Target.
(MemoryRegionInfo is another example of this pattern)
Reviewed By: omjavaid
Differential Revision: https://reviews.llvm.org/D97281
This adds a basic SB API for creating and stopping traces.
Note: This doesn't add any APIs for inspecting individual instructions. That'd be a more complicated change and it might be better to enhande the dump functionality to output the data in binary format. I'll leave that for a later diff.
This also enhances the existing tests so that they test the same flow using both the command interface and the SB API.
I also did some cleanup of legacy code.
Differential Revision: https://reviews.llvm.org/D103500
This reverts commit bd5751f3d2.
This patch series is causing us to every so often miss switching
the state from eStateRunning to eStateStopped when we get the stop
packet from the debug server.
Reverting till I can figure out how that could be happening.
David Spickett