-
Notifications
You must be signed in to change notification settings - Fork 13.4k
[DRAFT][lldb][RPC] Design doc for upstreaming PR #138612
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
@llvm/pr-subscribers-lldb Author: Chelsea Cassanova (chelcassanova) ChangesThis mainly adds as design doc to help follow with the current PRs up for upstreaming the Full diff: https://github.com/llvm/llvm-project/pull/138612.diff 1 Files Affected:
diff --git a/lldb/docs/rpc-design-doc.rst b/lldb/docs/rpc-design-doc.rst
new file mode 100644
index 0000000000000..97b51e1dd4854
--- /dev/null
+++ b/lldb/docs/rpc-design-doc.rst
@@ -0,0 +1,94 @@
+LLDB RPC Upstreaming Design Doc
+===============================
+
+This document aims to explain the general structure of the upstreaming patches for adding LLDB RPC. The 2 primary concepts explained here will be:
+
+* How LLDB RPC is used
+* How the ``lldb-rpc-gen`` works and what it outputs
+
+LLDB RPC
+*********
+
+LLDB RPC is a framework by which processes can communicate with LLDB out of process while maintaining compatibility with the SB API. More details are explained in the `RFC<https://discourse.llvm.org/t/rfc-upstreaming-lldb-rpc/85804>`_ for upstreaming LLDB RPC, but the main focus in this doc for this section will be how exactly the code is structured for the PRs that will upstream this code.
+
+The ``lldb-rpc-gen`` tool
+*************************
+
+``lldb-rpc-gen`` is the tool that generates the main client and server interfaces for LLDB RPC. It is a ``ClangTool`` that reads all SB API header files and their functions and outputs the client/server interfaces and certain other pieces of code, such as RPC-specfic versions of Python bindings used for the test suite. There's 3 main components behind ``lldb-rpc-gen``:
+
+1. The ``lldb-rpc-gen`` tool itself, which contains the main driver that uses the ``ClangTool``.
+2. The code that generates all interfaces, which we call "emitters". All generated code for the interfaces are in C++, so the server side has one emitter for its generated source code and another for its generated header code. The client side has the same.
+3. All common code shared between all emitters, such as helper methods and information about exceptions to take when emitting.
+
+The `current PR<https://github.com/llvm/llvm-project/pull/136748>`_ up for upstreaming LLDB RPC upstreams a subset of the code used for the tool. It upstreams the ``lldb-rpc-gen`` tool and all code needed for the server side emitters. Here's an example of what ``lldb-rpc-gen`` will output for the server side interface:
+
+Input
+-----
+
+We'll use ``SBDebugger::CreateTarget(const char *filename)`` as an example. ``lldb-rpc-gen`` will read this method from ``SBDebugger.h``. The output is as follows.
+
+Source Code Output
+------------------
+
+::
+
+ bool rpc_server::_ZN4lldb10SBDebugger12CreateTargetEPKc::HandleRPCCall(rpc_common::Connection &connection, RPCStream &send, RPCStream &response) {
+ // 1) Make local storage for incoming function arguments
+ lldb::SBDebugger *this_ptr = nullptr;
+ rpc_common::ConstCharPointer filename;
+ // 2) Decode all function arguments
+ this_ptr = RPCServerObjectDecoder<lldb::SBDebugger>(send, rpc_common::RPCPacket::ValueType::Argument);
+ if (!this_ptr)
+ return false;
+ if (!RPCValueDecoder(send, rpc_common::RPCPacket::ValueType::Argument, filename))
+ return false;
+ // 3) Call the method and encode the return value
+ lldb::SBTarget && __result = this_ptr->CreateTarget(filename.c_str());
+ RPCServerObjectEncoder(response, rpc_common::RPCPacket::ValueType::ReturnValue, std::move(__result));
+ return true;
+ }
+
+Function signature
+~~~~~~~~~~~~~~~~~~
+
+All server-side source code functions have a function signature that take the format ``bool rpc_server::<mangled-function-name>::HandleRPCCall(rpc_common::Connection &connection, RPCStream &send, RPCStream &response)``. Here the ``connection`` is what's maintained between the client and server. The ``send`` variable is a byte stream that carries information sent from the client. ``response`` is also a byte stream that will be populated with the return value obtained from the call into the SB API function that will be sent back to the client.
+
+Local variable storage
+~~~~~~~~~~~~~~~~~~~~~~
+
+First, variables are created to hold all arguments coming in from the client side. These variables will be a pointer for the SB API class in question, and corresponding variables for all parameters that the function has. Since this signature for ``SBDebugger::CreateTarget()`` only has one parameter, a ``const char *``, 2 local variables get created. A pointer for an ``SBDebugger`` object, and an ``RPCCommon::ConstCharPointer`` for the ``const char * filename`` parameter. The ``ConstCharPointer`` is a typedef over ``const char *`` in the main RPC core code.
+
+Incoming stream decoding
+~~~~~~~~~~~~~~~~~~~~~~~~
+
+Following this, ``RPCServerObjectDecoder`` is used to decode the ``send`` byte stream. In this case, we're decoding this stream into the ``SBDebugger`` pointer we created earlier. We then decode the ``send`` stream again to obtain the ``const char * filename`` sent by the client. Each decoded argument from the client is checked for validity and the function will exit early if any are invalid.
+
+SB API function call
+~~~~~~~~~~~~~~~~~~~~
+
+Once all arguments have been decoded, the underlying SB API function called with the decoded arguments. ``RPCServerObjectEncoder`` is then used to encode the return value from the SB API call into the ``response`` stream, and this is then sent back to the client.
+
+Header Code Output
+------------------
+::
+
+ class _ZN4lldb10SBDebugger12CreateTargetEPKc : public rpc_common::RPCFunctionInstance {
+ public:
+ _ZN4lldb10SBDebugger12CreateTargetEPKc() : RPCFunctionInstance("_ZN4lldb10SBDebugger12CreateTargetEPKc") {}
+ ~_ZN4lldb10SBDebugger12CreateTargetEPKc() override {}
+ bool HandleRPCCall(rpc_common::Connection &connection, rpc_common::RPCStream &send, rpc_common::RPCStream &response) override;
+ };
+
+Class definition and ``HandleRPCCall``
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+ALL RPC server side functions are subclasses of ``RPCFunctionInstance``. Each class will then define their ``HandleRPCCall`` function that is seen in the source code above. This subclassing and ``HandleRPCCall`` definition is what is emitted in the header code for server.
+
+The ``lldb-rpc-gen`` emitters
+*****************************
+
+The bulk of the code is generated using the emitters. For the server side, we have ``RPCServerSourceEmitter`` and ``RPCServerHeaderEmitter``. The former handles generation of the source code and the latter handles generation of the header code seen above.
+
+Emitters largely have similar structure. Constituent sections of code, such as function headers, function bodies and others are typically given their own method. As an example, the function to emit a function header is ``EmitFunctionHeader()`` and the function to emit a function body is ``EmitFunctionBody()``. Information that will be written to the output file is written using ``EmitLine()``, which uses ``llvm::raw_string_ostreams`` and is defined in ``RPCCommon.h``.
+
+Since this is a ``ClangTool``, information about each method is obtained from Clang itself and stored in the ``Method`` struct located in ``RPCCommon.h`` in ``lldb-rpc-gen``'s directory. ``Method`` is used for simplicity and abstraction, and other information that would be needed from the SB API is obtained from Clang directly.
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for writing this up.
When we get to a point where PRs have landed you can just change "will be" / "will do" language to "is" / "does" and it should be a good reference for the future.
One thing that is missing right now is how to test it. Basic things just like what format of test does it use (shell I think) and is it added to a ninja check target.
Also, is there a way for a developer to set up an lldb-rpc based session? If we have an issue that only happens over lldb-rpc. Which in theory we won't, but we might think we do and need to check that.
In other words - if the only way to do that is to use XCode - that would not be good.
Function signature | ||
~~~~~~~~~~~~~~~~~~ | ||
|
||
All server-side source code functions have a function signature that take the format ``bool rpc_server::<mangled-function-name>::HandleRPCCall(rpc_common::Connection &connection, RPCStream &send, RPCStream &response)``. Here the ``connection`` is what's maintained between the client and server. The ``send`` variable is a byte stream that carries information sent from the client. ``response`` is also a byte stream that will be populated with the return value obtained from the call into the SB API function that will be sent back to the client. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why mangled function name not just function name? It's in rpc_server::
so at first glance, it doesn't seem like it needs to be mangled.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since we have interfaces for all functions, we want to differentiate them in the event of overloaded functions, e.g. SBDebugger::CreateTarget
has 2 signatures, so each signature gets registered on the server side as:
bool rpc_server::_ZN4lldb10SBDebugger12CreateTargetEPKc::HandleRPCCall
bool rpc_server::_ZN4lldb10SBDebugger12CreateTargetEPKcS2_S2_bRNS_7SBErrorE::HandleRPCCall
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For context, this choice was originally made by @clayborg when he was at Apple. I worked on the earliest incarnation of lldb-rpc-gen and decided to keep the mangled name for the reason Chelsea mentioned, but the choice is indeed arbitrary.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok so the thing you need to do is differentiate overloads, and mangled names are designed to do exactly that. Got it.
Please add that justification to the doc.
Also we will want the conversion process (using the Python scripts) to be documented here eventually. |
Thanks for responding! I'll add a section on testing the tool itself (which we do with shell tests) and testing the interfaces (which we do by running the main LLDB API test suite against liblldbrpc).
Both the RPC client and server are built as binaries that each have a main. The RPC core code contains the functionality for a client binary to connect to RPC and then interface with the |
Ok so if had a very specific situation I wanted to run via lldb-rpc, I could write an API test for it? That's cool, if I do find a bug I'd need to write a test case anyway. Please add that as a suggestion in the document where you describe / will describe the testing process. |
But there's no lldb-rpc client built into |
fe7cf15
to
5bc66fd
Compare
Yes, you could have an API test that could then get run against liblldbrpc. We also have decorators to exclude/include tests that run against liblldbrpc.
Not directly in lldb itself as the |
To add to this, LLDBRPC's API should be almost the same as LLDB's. The major difference is that some methods and constructors will need to take an additional |
Function signature | ||
~~~~~~~~~~~~~~~~~~ | ||
|
||
All server-side source code functions have a function signature that take the format ``bool rpc_server::<mangled-function-name>::HandleRPCCall(rpc_common::Connection &connection, RPCStream &send, RPCStream &response)``. Here the ``connection`` is what's maintained between the client and server. The ``send`` variable is a byte stream that carries information sent from the client. ``response`` is also a byte stream that will be populated with the return value obtained from the call into the SB API function that will be sent back to the client. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For context, this choice was originally made by @clayborg when he was at Apple. I worked on the earliest incarnation of lldb-rpc-gen and decided to keep the mangled name for the reason Chelsea mentioned, but the choice is indeed arbitrary.
lldb/docs/rpc-design-doc.rst
Outdated
Local variable storage | ||
~~~~~~~~~~~~~~~~~~~~~~ | ||
|
||
First, variables are created to hold all arguments coming in from the client side. These variables will be a pointer for the SB API class in question, and corresponding variables for all parameters that the function has. Since this signature for ``SBDebugger::CreateTarget()`` only has one parameter, a ``const char *``, 2 local variables get created. A pointer for an ``SBDebugger`` object, and an ``RPCCommon::ConstCharPointer`` for the ``const char * filename`` parameter. The ``ConstCharPointer`` is a typedef over ``const char *`` in the main RPC core code. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Note: ConstCharPointer
is not a typedef but a full-fledged class. It's backed by a std::string
but has lots of convenience methods for operating at the RPC layer.
lldb/docs/rpc-design-doc.rst
Outdated
Class definition and ``HandleRPCCall`` | ||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
|
||
ALL RPC server side functions are subclasses of ``RPCFunctionInstance``. Each class will then define their ``HandleRPCCall`` function that is seen in the source code above. This subclassing and ``HandleRPCCall`` definition is what is emitted in the header code for server. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: ALL
-> All
This mainly adds as design doc to help follow with the current PRs up for upstreaming the `lldb-rpc-gen` tool and emitters.
5bc66fd
to
40106f6
Compare
This mainly adds as design doc to help follow with the current PRs up for upstreaming the
lldb-rpc-gen
tool and emitters.