Thanks to visit codestin.com
Credit goes to github.com

Skip to content

[DRAFT][lldb][RPC] Design doc for upstreaming PR #138612

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

chelcassanova
Copy link
Contributor

This mainly adds as design doc to help follow with the current PRs up for upstreaming the lldb-rpc-gen tool and emitters.

@llvmbot
Copy link
Member

llvmbot commented May 5, 2025

@llvm/pr-subscribers-lldb

Author: Chelsea Cassanova (chelcassanova)

Changes

This mainly adds as design doc to help follow with the current PRs up for upstreaming the lldb-rpc-gen tool and emitters.


Full diff: https://github.com/llvm/llvm-project/pull/138612.diff

1 Files Affected:

  • (added) lldb/docs/rpc-design-doc.rst (+94)
diff --git a/lldb/docs/rpc-design-doc.rst b/lldb/docs/rpc-design-doc.rst
new file mode 100644
index 0000000000000..97b51e1dd4854
--- /dev/null
+++ b/lldb/docs/rpc-design-doc.rst
@@ -0,0 +1,94 @@
+LLDB RPC Upstreaming Design Doc
+===============================
+
+This document aims to explain the general structure of the upstreaming patches for adding LLDB RPC. The 2 primary concepts explained here will be:
+
+* How LLDB RPC is used
+* How the ``lldb-rpc-gen`` works and what it outputs
+
+LLDB RPC
+*********
+
+LLDB RPC is a framework by which processes can communicate with LLDB out of process while maintaining compatibility with the SB API. More details are explained in the `RFC<https://discourse.llvm.org/t/rfc-upstreaming-lldb-rpc/85804>`_ for upstreaming LLDB RPC, but the main focus in this doc for this section will be how exactly the code is structured for the PRs that will upstream this code.
+
+The ``lldb-rpc-gen`` tool
+*************************
+
+``lldb-rpc-gen`` is the tool that generates the main client and server interfaces for LLDB RPC. It is a ``ClangTool`` that reads all SB API header files and their functions and outputs the client/server interfaces and certain other pieces of code, such as RPC-specfic versions of Python bindings used for the test suite. There's 3 main components behind ``lldb-rpc-gen``:
+
+1. The ``lldb-rpc-gen`` tool itself, which contains the main driver that uses the ``ClangTool``.
+2. The code that generates all interfaces, which we call "emitters". All generated code for the interfaces are in C++, so the server side has one emitter for its generated source code and another for its generated header code. The client side has the same.
+3. All common code shared between all emitters, such as helper methods and information about exceptions to take when emitting.
+
+The `current PR<https://github.com/llvm/llvm-project/pull/136748>`_ up for upstreaming LLDB RPC upstreams a subset of the code used for the tool. It upstreams the ``lldb-rpc-gen`` tool and all code needed for the server side emitters. Here's an example of what ``lldb-rpc-gen`` will output for the server side interface:
+
+Input
+-----
+
+We'll use ``SBDebugger::CreateTarget(const char *filename)`` as an example. ``lldb-rpc-gen`` will read this method from ``SBDebugger.h``. The output is as follows.
+
+Source Code Output
+------------------
+
+::
+
+   bool rpc_server::_ZN4lldb10SBDebugger12CreateTargetEPKc::HandleRPCCall(rpc_common::Connection &connection, RPCStream &send, RPCStream &response) {
+   // 1) Make local storage for incoming function arguments
+   lldb::SBDebugger *this_ptr = nullptr;
+   rpc_common::ConstCharPointer filename;
+   // 2) Decode all function arguments
+   this_ptr = RPCServerObjectDecoder<lldb::SBDebugger>(send, rpc_common::RPCPacket::ValueType::Argument);
+   if (!this_ptr)
+   return false;
+   if (!RPCValueDecoder(send, rpc_common::RPCPacket::ValueType::Argument, filename))
+   return false;
+   // 3) Call the method and encode the return value
+   lldb::SBTarget && __result = this_ptr->CreateTarget(filename.c_str());
+   RPCServerObjectEncoder(response, rpc_common::RPCPacket::ValueType::ReturnValue, std::move(__result));
+   return true;
+   }
+
+Function signature
+~~~~~~~~~~~~~~~~~~
+
+All server-side source code functions have a function signature that take the format ``bool rpc_server::<mangled-function-name>::HandleRPCCall(rpc_common::Connection &connection, RPCStream &send, RPCStream &response)``. Here the ``connection`` is what's maintained between the client and server. The ``send`` variable is a byte stream that carries information sent from the client. ``response`` is also a byte stream that will be populated with the return value obtained from the call into the SB API function that will be sent back to the client.
+
+Local variable storage
+~~~~~~~~~~~~~~~~~~~~~~
+
+First, variables are created to hold all arguments coming in from the client side. These variables will be a pointer for the SB API class in question, and corresponding variables for all parameters that the function has. Since this signature for ``SBDebugger::CreateTarget()`` only has one parameter, a ``const char *``, 2 local variables get created. A pointer for an ``SBDebugger`` object, and an ``RPCCommon::ConstCharPointer`` for the ``const char * filename`` parameter. The ``ConstCharPointer`` is a typedef over ``const char *`` in the main RPC core code.
+
+Incoming stream decoding
+~~~~~~~~~~~~~~~~~~~~~~~~
+
+Following this, ``RPCServerObjectDecoder`` is used to decode the ``send`` byte stream. In this case, we're decoding this stream into the ``SBDebugger`` pointer we created earlier. We then decode the ``send`` stream again to obtain the ``const char * filename`` sent by the client. Each decoded argument from the client is checked for validity and the function will exit early if any are invalid.
+
+SB API function call
+~~~~~~~~~~~~~~~~~~~~
+
+Once all arguments have been decoded, the underlying SB API function called with the decoded arguments. ``RPCServerObjectEncoder`` is then used to encode the return value from the SB API call into the ``response`` stream, and this is then sent back to the client.
+
+Header Code Output
+------------------
+::
+
+   class _ZN4lldb10SBDebugger12CreateTargetEPKc : public rpc_common::RPCFunctionInstance {
+   public:
+   _ZN4lldb10SBDebugger12CreateTargetEPKc() : RPCFunctionInstance("_ZN4lldb10SBDebugger12CreateTargetEPKc") {}
+   ~_ZN4lldb10SBDebugger12CreateTargetEPKc() override {}
+   bool HandleRPCCall(rpc_common::Connection &connection, rpc_common::RPCStream &send, rpc_common::RPCStream &response) override;
+   };
+
+Class definition and ``HandleRPCCall``
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+ALL RPC server side functions are subclasses of ``RPCFunctionInstance``. Each class will then define their ``HandleRPCCall`` function that is seen in the source code above. This subclassing and ``HandleRPCCall`` definition is what is emitted in the header code for server.
+
+The ``lldb-rpc-gen`` emitters
+*****************************
+
+The bulk of the code is generated using the emitters. For the server side, we have ``RPCServerSourceEmitter`` and ``RPCServerHeaderEmitter``. The former handles generation of the source code and the latter handles generation of the header code seen above.
+
+Emitters largely have similar structure. Constituent sections of code, such as function headers, function bodies and others are typically given their own method. As an example, the function to emit a function header is ``EmitFunctionHeader()`` and the function to emit a function body is ``EmitFunctionBody()``. Information that will be written to the output file is written using ``EmitLine()``, which uses ``llvm::raw_string_ostreams`` and is defined in ``RPCCommon.h``.
+
+Since this is a ``ClangTool``, information about each method is obtained from Clang itself and stored in the ``Method`` struct located in ``RPCCommon.h`` in ``lldb-rpc-gen``'s directory. ``Method`` is used for simplicity and abstraction, and other information that would be needed from the SB API is obtained from Clang directly.

Copy link
Collaborator

@DavidSpickett DavidSpickett left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for writing this up.

When we get to a point where PRs have landed you can just change "will be" / "will do" language to "is" / "does" and it should be a good reference for the future.

One thing that is missing right now is how to test it. Basic things just like what format of test does it use (shell I think) and is it added to a ninja check target.

Also, is there a way for a developer to set up an lldb-rpc based session? If we have an issue that only happens over lldb-rpc. Which in theory we won't, but we might think we do and need to check that.

In other words - if the only way to do that is to use XCode - that would not be good.

Function signature
~~~~~~~~~~~~~~~~~~

All server-side source code functions have a function signature that take the format ``bool rpc_server::<mangled-function-name>::HandleRPCCall(rpc_common::Connection &connection, RPCStream &send, RPCStream &response)``. Here the ``connection`` is what's maintained between the client and server. The ``send`` variable is a byte stream that carries information sent from the client. ``response`` is also a byte stream that will be populated with the return value obtained from the call into the SB API function that will be sent back to the client.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why mangled function name not just function name? It's in rpc_server:: so at first glance, it doesn't seem like it needs to be mangled.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we have interfaces for all functions, we want to differentiate them in the event of overloaded functions, e.g. SBDebugger::CreateTarget has 2 signatures, so each signature gets registered on the server side as:

  • bool rpc_server::_ZN4lldb10SBDebugger12CreateTargetEPKc::HandleRPCCall
  • bool rpc_server::_ZN4lldb10SBDebugger12CreateTargetEPKcS2_S2_bRNS_7SBErrorE::HandleRPCCall

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For context, this choice was originally made by @clayborg when he was at Apple. I worked on the earliest incarnation of lldb-rpc-gen and decided to keep the mangled name for the reason Chelsea mentioned, but the choice is indeed arbitrary.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok so the thing you need to do is differentiate overloads, and mangled names are designed to do exactly that. Got it.

Please add that justification to the doc.

@DavidSpickett
Copy link
Collaborator

Also we will want the conversion process (using the Python scripts) to be documented here eventually.

@chelcassanova
Copy link
Contributor Author

Thanks for responding! I'll add a section on testing the tool itself (which we do with shell tests) and testing the interfaces (which we do by running the main LLDB API test suite against liblldbrpc).

Also, is there a way for a developer to set up an lldb-rpc based session? If we have an issue that only happens over lldb-rpc. Which in theory we won't, but we might think we do and need to check that.

Both the RPC client and server are built as binaries that each have a main. The RPC core code contains the functionality for a client binary to connect to RPC and then interface with the lldb-rpc-server binary, so Xcode isn't the only way to create a session. This is how we set up the API test suite for RPC, as Python is used as the client binary for the RPC session.

@DavidSpickett
Copy link
Collaborator

Ok so if had a very specific situation I wanted to run via lldb-rpc, I could write an API test for it? That's cool, if I do find a bug I'd need to write a test case anyway.

Please add that as a suggestion in the document where you describe / will describe the testing process.

@DavidSpickett
Copy link
Collaborator

But there's no lldb-rpc client built into lldb, right? Because that would be my first instinct to try.

@chelcassanova chelcassanova force-pushed the lldb-rpc/design-doc branch from fe7cf15 to 5bc66fd Compare May 6, 2025 16:59
@chelcassanova
Copy link
Contributor Author

Ok so if had a very specific situation I wanted to run via lldb-rpc, I could write an API test for it? That's cool, if I do find a bug I'd need to write a test case anyway.

Yes, you could have an API test that could then get run against liblldbrpc. We also have decorators to exclude/include tests that run against liblldbrpc.

But there's no lldb-rpc client built into lldb, right? Because that would be my first instinct to try.

Not directly in lldb itself as the lldb-rpc-client binary is a separate build object.

@bulbazord
Copy link
Member

But there's no lldb-rpc client built into lldb, right? Because that would be my first instinct to try.

Not directly in lldb itself as the lldb-rpc-client binary is a separate build object.

To add to this, LLDBRPC's API should be almost the same as LLDB's. The major difference is that some methods and constructors will need to take an additional connection parameter. It would be possible to write some lldb-rpc driver, but we haven't done.

Function signature
~~~~~~~~~~~~~~~~~~

All server-side source code functions have a function signature that take the format ``bool rpc_server::<mangled-function-name>::HandleRPCCall(rpc_common::Connection &connection, RPCStream &send, RPCStream &response)``. Here the ``connection`` is what's maintained between the client and server. The ``send`` variable is a byte stream that carries information sent from the client. ``response`` is also a byte stream that will be populated with the return value obtained from the call into the SB API function that will be sent back to the client.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For context, this choice was originally made by @clayborg when he was at Apple. I worked on the earliest incarnation of lldb-rpc-gen and decided to keep the mangled name for the reason Chelsea mentioned, but the choice is indeed arbitrary.

Local variable storage
~~~~~~~~~~~~~~~~~~~~~~

First, variables are created to hold all arguments coming in from the client side. These variables will be a pointer for the SB API class in question, and corresponding variables for all parameters that the function has. Since this signature for ``SBDebugger::CreateTarget()`` only has one parameter, a ``const char *``, 2 local variables get created. A pointer for an ``SBDebugger`` object, and an ``RPCCommon::ConstCharPointer`` for the ``const char * filename`` parameter. The ``ConstCharPointer`` is a typedef over ``const char *`` in the main RPC core code.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: ConstCharPointer is not a typedef but a full-fledged class. It's backed by a std::string but has lots of convenience methods for operating at the RPC layer.

Class definition and ``HandleRPCCall``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

ALL RPC server side functions are subclasses of ``RPCFunctionInstance``. Each class will then define their ``HandleRPCCall`` function that is seen in the source code above. This subclassing and ``HandleRPCCall`` definition is what is emitted in the header code for server.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: ALL -> All

This mainly adds as design doc to help follow with the current PRs up
for upstreaming the `lldb-rpc-gen` tool and emitters.
@chelcassanova chelcassanova force-pushed the lldb-rpc/design-doc branch from 5bc66fd to 40106f6 Compare May 6, 2025 20:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants