Thanks to visit codestin.com
Credit goes to github.com

Skip to content

[lldb-dap] Implement runInTerminal for Windows #121269

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

SuibianP
Copy link

Currently, the named pipe is passed by name and a transient ofstream is constructed at each I/O request. This assumes,

  • Blocking semantics: FIFO I/O waits for the other side to connect.
  • Buffered semantics: Closing one side does not discard existing data.

The former can be replaced by WaitNamedPipe/ConnectNamedPipe on Win32, but the second cannot be easily worked around. It is also impossible to have another "keep-alive" pipe server instance, as server-client pairs are fixed on connection on Win32 and the client may get connected to it instead of the real one.

Refactor FifoFile[IO] to use an open file handles rather than file name.

Copy link

Thank you for submitting a Pull Request (PR) to the LLVM Project!

This PR will be automatically labeled and the relevant teams will be notified.

If you wish to, you can add reviewers by using the "Reviewers" section on this page.

If this is not working for you, it is probably because you do not have write permissions for the repository. In which case you can instead tag reviewers by name in a comment by using @ followed by their GitHub username.

If you have received no comments on your PR for a week, you can request a review by "ping"ing the PR by adding a comment “Ping”. The common courtesy "ping" rate is once a week. Please remember that you are asking for valuable time from other developers.

If you have further questions, they may be answered by the LLVM GitHub User Guide.

You can also ask questions in a comment on this PR, on the LLVM Discord or on the forums.

@SuibianP
Copy link
Author

SuibianP commented Dec 28, 2024

The patch currently suffers from a critical issue that I cannot diagnose. Help is much appreciated.

When the debug adapter attempts to attach to the launcher,

dap.target.Attach(attach_info, error);

it blocks at

if (::WaitForSingleObject(m_session_data->m_initial_stop_event, INFINITE) ==
WAIT_OBJECT_0) {

although adding the following snippet at the front of RunInTerminalLauncherCommChannel::WaitUntilDebugAdaptorAttaches indicates that the launcher process does get paused and never resumed.

for (int i = 0; !::IsDebuggerPresent(); ++i)
  LLVM_LOGV(log, "Waiting to be attached... {0}", i);
LLVM_LOGV(log, "Attached");

CC @JDevlieghere

@SuibianP SuibianP force-pushed the lldb-dap-runinterminal-windows branch from 4c8c18c to b980d11 Compare December 28, 2024 15:47
@SuibianP
Copy link
Author

SuibianP commented Jan 7, 2025

Ping

@SuibianP SuibianP force-pushed the lldb-dap-runinterminal-windows branch 2 times, most recently from 35d7391 to b475e34 Compare January 7, 2025 15:02
@SuibianP
Copy link
Author

Hi @JDevlieghere, sorry for pinging again but could you please kindly help take a look at this when you have time? Thanks in advance!

@JDevlieghere
Copy link
Member

Sorry, I didn't get the ping (I think because this is marked as a draft). I'm not familiar enough with Windows to review this, so I'm adding some folks that are or work on related stuff in lldb-dap.

@ashgti
Copy link
Contributor

ashgti commented Jan 17, 2025

Is the process stopped? I think lldb is waiting for the process to be in a stopped state. In posix terms a SIGSTOP, although I'm not as familiar with the win32 equiv (DebugActiveProcess maybe).

@SuibianP SuibianP force-pushed the lldb-dap-runinterminal-windows branch from b475e34 to c23b994 Compare January 18, 2025 16:49
@SuibianP SuibianP marked this pull request as ready for review January 18, 2025 16:49
@llvmbot llvmbot added the lldb label Jan 18, 2025
@llvmbot
Copy link
Member

llvmbot commented Jan 18, 2025

@llvm/pr-subscribers-lldb

Author: Hu Jialun (SuibianP)

Changes

Currently, the named pipe is passed by name and a transient ofstream is constructed at each I/O request. This assumes,

  • Blocking semantics: FIFO I/O waits for the other side to connect.
  • Buffered semantics: Closing one side does not discard existing data.

The former can be replaced by WaitNamedPipe/ConnectNamedPipe on Win32, but the second cannot be easily worked around. It is also impossible to have another "keep-alive" pipe server instance, as server-client pairs are fixed on connection on Win32 and the client may get connected to it instead of the real one.

Refactor FifoFile[IO] to use an open file handles rather than file name.


Full diff: https://github.com/llvm/llvm-project/pull/121269.diff

5 Files Affected:

  • (modified) lldb/tools/lldb-dap/FifoFiles.cpp (+102-19)
  • (modified) lldb/tools/lldb-dap/FifoFiles.h (+20-15)
  • (modified) lldb/tools/lldb-dap/RunInTerminal.cpp (+26-9)
  • (modified) lldb/tools/lldb-dap/RunInTerminal.h (+4-2)
  • (modified) lldb/tools/lldb-dap/lldb-dap.cpp (+39-10)
diff --git a/lldb/tools/lldb-dap/FifoFiles.cpp b/lldb/tools/lldb-dap/FifoFiles.cpp
index 1f1bba80bd3b11..8bca006e63d657 100644
--- a/lldb/tools/lldb-dap/FifoFiles.cpp
+++ b/lldb/tools/lldb-dap/FifoFiles.cpp
@@ -9,7 +9,13 @@
 #include "FifoFiles.h"
 #include "JSONUtils.h"
 
-#if !defined(_WIN32)
+#include "llvm/Support/FileSystem.h"
+
+#if defined(_WIN32)
+#include <Windows.h>
+#include <fcntl.h>
+#include <io.h>
+#else
 #include <sys/stat.h>
 #include <sys/types.h>
 #include <unistd.h>
@@ -24,27 +30,73 @@ using namespace llvm;
 
 namespace lldb_dap {
 
-FifoFile::FifoFile(StringRef path) : m_path(path) {}
+std::error_code EC;
 
+FifoFile::FifoFile(StringRef path)
+    : m_path(path), m_file(fopen(path.data(), "r+")) {
+  if (m_file == nullptr) {
+    EC = std::error_code(errno, std::generic_category());
+    llvm::errs() << "Failed to open fifo file: " << path << EC.message()
+                 << "\n";
+    std::terminate();
+  }
+  if (setvbuf(m_file, NULL, _IONBF, 0))
+    llvm::errs() << "Error setting unbuffered mode on C FILE\n";
+}
+FifoFile::FifoFile(StringRef path, FILE *f) : m_path(path), m_file(f) {}
+FifoFile::FifoFile(FifoFile &&other)
+    : m_path(other.m_path), m_file(other.m_file) {
+  other.m_file = nullptr;
+}
 FifoFile::~FifoFile() {
+  if (m_file)
+    fclose(m_file);
 #if !defined(_WIN32)
+  // Unreferenced named pipes are deleted automatically on Win32
   unlink(m_path.c_str());
 #endif
 }
 
-Expected<std::shared_ptr<FifoFile>> CreateFifoFile(StringRef path) {
-#if defined(_WIN32)
-  return createStringError(inconvertibleErrorCode(), "Unimplemented");
+// This probably belongs to llvm::sys::fs as another FSEntity type
+std::error_code createNamedPipe(const Twine &Prefix, StringRef Suffix,
+                                int &ResultFd,
+                                SmallVectorImpl<char> &ResultPath) {
+  const char *Middle = Suffix.empty() ? "-%%%%%%" : "-%%%%%%.";
+  auto EC = sys::fs::getPotentiallyUniqueFileName(
+#ifdef _WIN32
+      "\\\\.\\pipe\\LOCAL\\"
+#else
+      "/tmp/"
+#endif
+          + Prefix + Middle + Suffix,
+      ResultPath);
+  if (EC)
+    return EC;
+  ResultPath.push_back(0);
+  const char *path = ResultPath.data();
+#ifdef _WIN32
+  HANDLE h = ::CreateNamedPipeA(
+      path, PIPE_ACCESS_DUPLEX | FILE_FLAG_FIRST_PIPE_INSTANCE,
+      PIPE_TYPE_BYTE | PIPE_READMODE_BYTE | PIPE_WAIT, 1, 1024, 1024, 0, NULL);
+  if (h == INVALID_HANDLE_VALUE)
+    return std::error_code(::GetLastError(), std::system_category());
+  ResultFd = _open_osfhandle((intptr_t)h, _O_TEXT | _O_RDWR);
+  if (ResultFd == -1)
+    return std::error_code(::GetLastError(), std::system_category());
 #else
-  if (int err = mkfifo(path.data(), 0600))
-    return createStringError(std::error_code(err, std::generic_category()),
-                             "Couldn't create fifo file: %s", path.data());
-  return std::make_shared<FifoFile>(path);
+  if (mkfifo(path, 0600) == -1)
+    return std::error_code(errno, std::generic_category());
+  EC = openFileForWrite(ResultPath, ResultFd, sys::fs::CD_OpenExisting,
+                        sys::fs::OF_None, 0600);
+  if (EC)
+    return EC;
 #endif
+  return std::error_code();
 }
 
-FifoFileIO::FifoFileIO(StringRef fifo_file, StringRef other_endpoint_name)
-    : m_fifo_file(fifo_file), m_other_endpoint_name(other_endpoint_name) {}
+FifoFileIO::FifoFileIO(FifoFile &&fifo_file, StringRef other_endpoint_name)
+    : m_fifo_file(std::move(fifo_file)),
+      m_other_endpoint_name(other_endpoint_name) {}
 
 Expected<json::Value> FifoFileIO::ReadJSON(std::chrono::milliseconds timeout) {
   // We use a pointer for this future, because otherwise its normal destructor
@@ -52,13 +104,20 @@ Expected<json::Value> FifoFileIO::ReadJSON(std::chrono::milliseconds timeout) {
   std::optional<std::string> line;
   std::future<void> *future =
       new std::future<void>(std::async(std::launch::async, [&]() {
-        std::ifstream reader(m_fifo_file, std::ifstream::in);
-        std::string buffer;
-        std::getline(reader, buffer);
-        if (!buffer.empty())
-          line = buffer;
+        rewind(m_fifo_file.m_file);
+        constexpr size_t buffer_size = 2048;
+        char buffer[buffer_size];
+        char *ptr = fgets(buffer, buffer_size, m_fifo_file.m_file);
+        if (ptr == nullptr || *ptr == 0)
+          return;
+        size_t len = strlen(buffer);
+        if (len <= 1)
+          return;
+        buffer[len - 1] = '\0'; // remove newline
+        line = buffer;
       }));
-  if (future->wait_for(timeout) == std::future_status::timeout || !line)
+
+  if (future->wait_for(timeout) == std::future_status::timeout)
     // Indeed this is a leak, but it's intentional. "future" obj destructor
     //  will block on waiting for the worker thread to join. And the worker
     //  thread might be stuck in blocking I/O. Intentionally leaking the  obj
@@ -69,6 +128,11 @@ Expected<json::Value> FifoFileIO::ReadJSON(std::chrono::milliseconds timeout) {
     return createStringError(inconvertibleErrorCode(),
                              "Timed out trying to get messages from the " +
                                  m_other_endpoint_name);
+  if (!line) {
+    return createStringError(inconvertibleErrorCode(),
+                             "Failed to get messages from the " +
+                                 m_other_endpoint_name);
+  }
   delete future;
   return json::parse(*line);
 }
@@ -78,8 +142,12 @@ Error FifoFileIO::SendJSON(const json::Value &json,
   bool done = false;
   std::future<void> *future =
       new std::future<void>(std::async(std::launch::async, [&]() {
-        std::ofstream writer(m_fifo_file, std::ofstream::out);
-        writer << JSONToString(json) << std::endl;
+        rewind(m_fifo_file.m_file);
+        auto payload = JSONToString(json);
+        if (fputs(payload.c_str(), m_fifo_file.m_file) == EOF ||
+            fputc('\n', m_fifo_file.m_file) == EOF) {
+          return;
+        }
         done = true;
       }));
   if (future->wait_for(timeout) == std::future_status::timeout || !done) {
@@ -98,4 +166,19 @@ Error FifoFileIO::SendJSON(const json::Value &json,
   return Error::success();
 }
 
+Error FifoFileIO::WaitForPeer() {
+#ifdef _WIN32
+  llvm::errs() << m_fifo_file.m_file << " ; " << fileno(m_fifo_file.m_file)
+               << "\n";
+  if (!::ConnectNamedPipe((HANDLE)_get_osfhandle(fileno(m_fifo_file.m_file)),
+                          NULL) &&
+      GetLastError() != ERROR_PIPE_CONNECTED) {
+    return createStringError(
+        std::error_code(GetLastError(), std::system_category()),
+        "Failed to connect to the " + m_other_endpoint_name);
+  }
+#endif
+  return Error::success();
+}
+
 } // namespace lldb_dap
diff --git a/lldb/tools/lldb-dap/FifoFiles.h b/lldb/tools/lldb-dap/FifoFiles.h
index 633ebeb2aedd45..5aa3466f3a620f 100644
--- a/lldb/tools/lldb-dap/FifoFiles.h
+++ b/lldb/tools/lldb-dap/FifoFiles.h
@@ -11,8 +11,10 @@
 
 #include "llvm/Support/Error.h"
 #include "llvm/Support/JSON.h"
+#include "llvm/Support/raw_ostream.h"
 
 #include <chrono>
+#include <fstream>
 
 namespace lldb_dap {
 
@@ -21,21 +23,22 @@ namespace lldb_dap {
 /// The file is destroyed when the destructor is invoked.
 struct FifoFile {
   FifoFile(llvm::StringRef path);
+  FifoFile(llvm::StringRef path, FILE *f);
+  // FifoFile(llvm::StringRef path, FILE *f);
+  FifoFile(FifoFile &&other);
+
+  FifoFile(const FifoFile &) = delete;
+  FifoFile &operator=(const FifoFile &) = delete;
 
   ~FifoFile();
 
   std::string m_path;
+  FILE *m_file;
 };
 
-/// Create a fifo file in the filesystem.
-///
-/// \param[in] path
-///     The path for the fifo file.
-///
-/// \return
-///     A \a std::shared_ptr<FifoFile> if the file could be created, or an
-///     \a llvm::Error in case of failures.
-llvm::Expected<std::shared_ptr<FifoFile>> CreateFifoFile(llvm::StringRef path);
+std::error_code createNamedPipe(const llvm::Twine &Prefix,
+                                llvm::StringRef Suffix, int &ResultFd,
+                                llvm::SmallVectorImpl<char> &ResultPath);
 
 class FifoFileIO {
 public:
@@ -45,7 +48,7 @@ class FifoFileIO {
   /// \param[in] other_endpoint_name
   ///     A human readable name for the other endpoint that will communicate
   ///     using this file. This is used for error messages.
-  FifoFileIO(llvm::StringRef fifo_file, llvm::StringRef other_endpoint_name);
+  FifoFileIO(FifoFile &&fifo_file, llvm::StringRef other_endpoint_name);
 
   /// Read the next JSON object from the underlying input fifo file.
   ///
@@ -71,12 +74,14 @@ class FifoFileIO {
   /// \return
   ///     An \a llvm::Error object indicating whether the data was consumed by
   ///     a reader or not.
-  llvm::Error SendJSON(
-      const llvm::json::Value &json,
-      std::chrono::milliseconds timeout = std::chrono::milliseconds(20000));
+  llvm::Error
+  SendJSON(const llvm::json::Value &json,
+           std::chrono::milliseconds timeout = std::chrono::milliseconds(2000));
+
+  llvm::Error WaitForPeer();
 
-private:
-  std::string m_fifo_file;
+  // private:
+  FifoFile m_fifo_file;
   std::string m_other_endpoint_name;
 };
 
diff --git a/lldb/tools/lldb-dap/RunInTerminal.cpp b/lldb/tools/lldb-dap/RunInTerminal.cpp
index 4fe09e2885a8e5..5bf123c689404f 100644
--- a/lldb/tools/lldb-dap/RunInTerminal.cpp
+++ b/lldb/tools/lldb-dap/RunInTerminal.cpp
@@ -97,7 +97,7 @@ static Error ToError(const RunInTerminalMessage &message) {
 
 RunInTerminalLauncherCommChannel::RunInTerminalLauncherCommChannel(
     StringRef comm_file)
-    : m_io(comm_file, "debug adaptor") {}
+    : m_io(FifoFile(comm_file), "debug adaptor") {}
 
 Error RunInTerminalLauncherCommChannel::WaitUntilDebugAdaptorAttaches(
     std::chrono::milliseconds timeout) {
@@ -111,8 +111,10 @@ Error RunInTerminalLauncherCommChannel::WaitUntilDebugAdaptorAttaches(
     return message.takeError();
 }
 
-Error RunInTerminalLauncherCommChannel::NotifyPid() {
-  return m_io.SendJSON(RunInTerminalMessagePid(getpid()).ToJSON());
+Error RunInTerminalLauncherCommChannel::NotifyPid(lldb::pid_t pid) {
+  if (pid == 0)
+    pid = getpid();
+  return m_io.SendJSON(RunInTerminalMessagePid(pid).ToJSON());
 }
 
 void RunInTerminalLauncherCommChannel::NotifyError(StringRef error) {
@@ -122,8 +124,12 @@ void RunInTerminalLauncherCommChannel::NotifyError(StringRef error) {
 }
 
 RunInTerminalDebugAdapterCommChannel::RunInTerminalDebugAdapterCommChannel(
-    StringRef comm_file)
-    : m_io(comm_file, "runInTerminal launcher") {}
+    FifoFile &comm_file)
+    : m_io(std::move(comm_file), "runInTerminal launcher") {}
+
+Error RunInTerminalDebugAdapterCommChannel::WaitForLauncher() {
+  return m_io.WaitForPeer();
+}
 
 // Can't use \a std::future<llvm::Error> because it doesn't compile on Windows
 std::future<lldb::SBError>
@@ -158,13 +164,24 @@ std::string RunInTerminalDebugAdapterCommChannel::GetLauncherError() {
 }
 
 Expected<std::shared_ptr<FifoFile>> CreateRunInTerminalCommFile() {
+  int comm_fd;
   SmallString<256> comm_file;
-  if (std::error_code EC = sys::fs::getPotentiallyUniqueTempFileName(
-          "lldb-dap-run-in-terminal-comm", "", comm_file))
+  if (std::error_code EC = createNamedPipe("lldb-dap-run-in-terminal-comm", "",
+                                           comm_fd, comm_file))
     return createStringError(EC, "Error making unique file name for "
                                  "runInTerminal communication files");
-
-  return CreateFifoFile(comm_file.str());
+  FILE *cf = fdopen(comm_fd, "r+");
+  if (setvbuf(cf, NULL, _IONBF, 0))
+    return createStringError(std::error_code(errno, std::generic_category()),
+                             "Error setting unbuffered mode on C FILE");
+  // There is no portable way to conjure an ofstream from HANDLE, so use FILE *
+  // llvm::raw_fd_stream does not support getline() and there is no
+  // llvm::buffer_istream
+
+  if (cf == NULL)
+    return createStringError(std::error_code(errno, std::generic_category()),
+                             "Error converting file descriptor to C FILE");
+  return std::make_shared<FifoFile>(comm_file, cf);
 }
 
 } // namespace lldb_dap
diff --git a/lldb/tools/lldb-dap/RunInTerminal.h b/lldb/tools/lldb-dap/RunInTerminal.h
index b20f8beb6071dd..235108dbb08d89 100644
--- a/lldb/tools/lldb-dap/RunInTerminal.h
+++ b/lldb/tools/lldb-dap/RunInTerminal.h
@@ -87,7 +87,7 @@ class RunInTerminalLauncherCommChannel {
   /// \return
   ///     An \a llvm::Error object in case of errors or if this operation times
   ///     out.
-  llvm::Error NotifyPid();
+  llvm::Error NotifyPid(lldb::pid_t pid = 0);
 
   /// Notify the debug adaptor that there's been an error.
   void NotifyError(llvm::StringRef error);
@@ -98,7 +98,7 @@ class RunInTerminalLauncherCommChannel {
 
 class RunInTerminalDebugAdapterCommChannel {
 public:
-  RunInTerminalDebugAdapterCommChannel(llvm::StringRef comm_file);
+  RunInTerminalDebugAdapterCommChannel(FifoFile &comm_file);
 
   /// Notify the runInTerminal launcher that it was attached.
   ///
@@ -118,6 +118,8 @@ class RunInTerminalDebugAdapterCommChannel {
   /// default error message if a certain timeout if reached.
   std::string GetLauncherError();
 
+  llvm::Error WaitForLauncher();
+
 private:
   FifoFileIO m_io;
 };
diff --git a/lldb/tools/lldb-dap/lldb-dap.cpp b/lldb/tools/lldb-dap/lldb-dap.cpp
index 7e8f7b5f6df679..5c38de393e9673 100644
--- a/lldb/tools/lldb-dap/lldb-dap.cpp
+++ b/lldb/tools/lldb-dap/lldb-dap.cpp
@@ -2007,7 +2007,7 @@ llvm::Error request_runInTerminal(DAP &dap,
     return comm_file_or_err.takeError();
   FifoFile &comm_file = *comm_file_or_err.get();
 
-  RunInTerminalDebugAdapterCommChannel comm_channel(comm_file.m_path);
+  RunInTerminalDebugAdapterCommChannel comm_channel(comm_file);
 
   lldb::pid_t debugger_pid = LLDB_INVALID_PROCESS_ID;
 #if !defined(_WIN32)
@@ -2025,6 +2025,9 @@ llvm::Error request_runInTerminal(DAP &dap,
                            }
                          });
 
+  auto err = comm_channel.WaitForLauncher();
+  if (err)
+    return err;
   if (llvm::Expected<lldb::pid_t> pid = comm_channel.GetLauncherPid())
     attach_info.SetProcessID(*pid);
   else
@@ -4860,11 +4863,6 @@ static void printHelp(LLDBDAPOptTable &table, llvm::StringRef tool_name) {
 static void LaunchRunInTerminalTarget(llvm::opt::Arg &target_arg,
                                       llvm::StringRef comm_file,
                                       lldb::pid_t debugger_pid, char *argv[]) {
-#if defined(_WIN32)
-  llvm::errs() << "runInTerminal is only supported on POSIX systems\n";
-  exit(EXIT_FAILURE);
-#else
-
   // On Linux with the Yama security module enabled, a process can only attach
   // to its descendants by default. In the runInTerminal case the target
   // process is launched by the client so we need to allow tracing explicitly.
@@ -4873,8 +4871,37 @@ static void LaunchRunInTerminalTarget(llvm::opt::Arg &target_arg,
     (void)prctl(PR_SET_PTRACER, debugger_pid, 0, 0, 0);
 #endif
 
+  const char *target = target_arg.getValue();
+
+#ifdef _WIN32
+  /* Win32 provides no way to replace the process image. exec* are misnomers.
+     Neither is the adapter notified of new processes due to DebugActiveProcess
+     semantics. Hence, we create the new process in a suspended state and resume
+     it after attach.
+   */
+  std::string cmdline;
+  for (char **arg = argv; *arg != nullptr; ++arg) {
+    cmdline += *arg;
+    cmdline += ' ';
+  }
+  llvm::errs() << "Executing cmdline: " << cmdline << "\n";
+  STARTUPINFOA si = {};
+  si.cb = sizeof(si);
+  PROCESS_INFORMATION pi = {};
+  bool res = CreateProcessA(target, cmdline.data(), NULL, NULL, FALSE,
+                            CREATE_SUSPENDED, NULL, NULL, &si, &pi);
+  if (!res) {
+    llvm::errs() << "Failed to create process: " << GetLastError() << "\n";
+    exit(EXIT_FAILURE);
+  }
+#endif
+
   RunInTerminalLauncherCommChannel comm_channel(comm_file);
-  if (llvm::Error err = comm_channel.NotifyPid()) {
+  if (llvm::Error err = comm_channel.NotifyPid(
+#ifdef _WIN32
+          pi.dwProcessId
+#endif
+          )) {
     llvm::errs() << llvm::toString(std::move(err)) << "\n";
     exit(EXIT_FAILURE);
   }
@@ -4891,15 +4918,17 @@ static void LaunchRunInTerminalTarget(llvm::opt::Arg &target_arg,
     llvm::errs() << llvm::toString(std::move(err)) << "\n";
     exit(EXIT_FAILURE);
   }
-
-  const char *target = target_arg.getValue();
+#ifdef _WIN32
+  assert(ResumeThread(pi.hThread) != -1);
+  exit(EXIT_SUCCESS);
+#else
   execvp(target, argv);
+#endif
 
   std::string error = std::strerror(errno);
   comm_channel.NotifyError(error);
   llvm::errs() << error << "\n";
   exit(EXIT_FAILURE);
-#endif
 }
 
 /// used only by TestVSCode_redirection_to_console.py

@SuibianP
Copy link
Author

SuibianP commented Jan 18, 2025

As it turns out that Win32 execvp actually creates a new process instead of replace the current process image, I changed the implementation to use CreateProcess before reporting the PID of the target.

The implementation works with both dape and VSCode now, so I marked it ready for review.


This is not relevant anymore but for the sake of completion: In the old implementation (wait for attach then execvp), the launcher process does get resumed after some 3 minutes. I still do not understand how it happened.

@SuibianP SuibianP force-pushed the lldb-dap-runinterminal-windows branch from c23b994 to c32fb82 Compare January 19, 2025 10:21
Copy link
Contributor

@omjavaid omjavaid left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be able make lldb\test\API\tools\lldb-dap\runInTerminal\TestDAP_runInTerminal.py pass all tests. Please tests and remove the skipping decorators.

Copy link
Member

@walter-erquinigo walter-erquinigo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be honest it's a bit hard to review this because I'm not versed in these low level APIs, let along Windows. Could you make sure that your changes pass mac, linux and windows CI? IIRC you can trigger manually buildbots if you want to run custom tests.

/// A \a std::shared_ptr<FifoFile> if the file could be created, or an
/// \a llvm::Error in case of failures.
llvm::Expected<std::shared_ptr<FifoFile>> CreateFifoFile(llvm::StringRef path);
std::error_code createNamedPipe(const llvm::Twine &Prefix,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This API is unnecessarily complex. The Prefix can just be a StringRef. Also use lower case for names. You also never use the Suffix.
Finally, please write some documentation mentioning that a unique named pipe with the given prefix will be created.
You may consider renaming this function as createUniqueNamedPipe

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was mimicking the llvm::sys::fs::createTemporaryFile API with the idea that this abstraction might be melded in there at some later time.

Is this considered too broad?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added doxygen comments, changed to snake_case and renamed function to createUniqueNamedPipe.

@SuibianP SuibianP force-pushed the lldb-dap-runinterminal-windows branch from c32fb82 to dc866c2 Compare January 24, 2025 12:23
@SuibianP
Copy link
Author

I enabled runInTerminal tests for Windows (will fix line 93 later) except for the 4 launcher tests that requires os.mkfifo (python/cpython#103510) and they pass on my local computer. This should be sufficient to guarantee the common functionalities. I can do a simple polyfill using pywin32 for Windows if the ignored test is a blocker for this PR.

All LLDB tests still pass on Linux CI, so there seems to be no regression https://buildkite.com/llvm-project/github-pull-requests/builds/140240

@SuibianP SuibianP force-pushed the lldb-dap-runinterminal-windows branch from dc866c2 to 41650cf Compare February 4, 2025 15:08
@@ -9,7 +9,13 @@
#include "FifoFiles.h"
#include "JSONUtils.h"

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: no need for a space between headers

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed one empty line and preserved those around ifdef guarded includes, the same way lldb-dap.cpp does it.

std::getline(reader, buffer);
if (!buffer.empty())
line = buffer;
rewind(m_fifo_file.m_file);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to rewind the file? Shouldn't the position be at the beginning of the file already?

Copy link
Author

@SuibianP SuibianP Feb 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seemed to help on Windows, but I did not investigate further, as it is technically also required by ISO C standard (C23 draft N3220 §7.23.5.3.7) which C++ delegate to for C libraries ([library.c]).

When a file is opened with update mode (’+’ as the second or third character in the previously
described list of mode argument values), both input and output may be performed on the associated
stream. However, output shall not be directly followed by input without an intervening call to the
fflush function or to a file positioning function (fseek, fsetpos, or rewind), and input shall not
be directly followed by output without an intervening call to a file positioning function, unless the
input operation encounters end-of-file. Opening (or creating) a text file with update mode may
instead open (or create) a binary stream in some implementations.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the flow of the logic here is:

  • lldb-dap (pid 1) opens the fifo
  • pid 1 sends sends the runInTerminal command.
  • pid 1 reads from the fifo (waiting for data)
  • lldb-dap --launch-target (pid 2) starts
  • pid 2 opens the fifo
  • pid 2 spawns the new process
  • pid 2 writes to the fifo (something like {"pid":2}\n)
  • pid 1 finishes the initial read from the fifo
  • pid 1 lldb attaches to the process started by pid 2

This should mean that the parent side only reads from the file and the child side only writes to the file. Additionally, I think they're only each performing 1 write/read on each side so their respective file positions should still be at the beginning of the file after opening.

Copy link
Author

@SuibianP SuibianP Feb 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The DA side ("PID 1") sends a RunInTerminalMessageDidAttach message in RunInTerminalDebugAdapterCommChannel::NotifyDidAttach when it attaches to the target PID received from the launcher. This is also why the FIFO must be PIPE_ACCESS_DUPLEX. The launcher process waits for this message before proceeding with executing the target.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think seeking isn't support on a fifo (lseek would return ESPIPE, see man 2 lseek or the Errors section of https://man7.org/linux/man-pages/man2/lseek.2.html). So I'm not sure if you need to do these rewind on non-windows platforms.

Copy link
Author

@SuibianP SuibianP Feb 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Practically it should not be needed, but I am tempted to leave the rewind there. It helps with standard compliance and does not harm anyways as long as its errno is ignored. There might as well be other non-Windows platforms requiring this invocation to properly function.

Citing C89 rationale §4.9.5.3

A change of input/output direction on an update file is only allowed following a fsetpos, fseek, rewind, or fflush operation, since these are precisely the functions which assure that the I/O buffer has been flushed.

FIFO can also possibly have buffering, and the requirement makes sense. I suppose the timeout I observed on Windows with rewind omitted is due to a deadlock condition created by unflushed buffer.

@SuibianP SuibianP force-pushed the lldb-dap-runinterminal-windows branch from 41650cf to c656de0 Compare February 5, 2025 14:04
@SuibianP SuibianP requested a review from ashgti February 10, 2025 11:05
#endif
result_path.pop_back();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any reason why this is being removed from the end of the path? I think thats the \0 char (pushed on line 75).

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it is the NUL. It would be encoded into JSON if not popped.

I think it might have always been buggy and just happened to work, as NUL termination from StringRef::data is explicitly not guaranteed.

if (!buffer.empty())
line = buffer;
rewind(m_fifo_file.m_file);
constexpr size_t buffer_size = 2048;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here the buffer is 2048 but when the file is created its 1024. Should we have a common default buffer size?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The two sizes are not related. The one passed to CreateNamedPipe is the kernel buffer, while this one is for the receiving user buffer. I do need to rethink how long the buffer needs to be, though.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed at 512 which should be longer than any single message that passes through the FIFO. Mentioned in code comment as well.

Comment on lines +110 to +129
char buffer[buffer_size];
char *ptr = fgets(buffer, buffer_size, m_fifo_file.m_file);
if (ptr == nullptr || *ptr == 0)
return;
size_t len = strlen(buffer);
if (len <= 1)
return;
buffer[len - 1] = '\0'; // remove newline
line = buffer;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The previous version would get a full line, however this new version will read until the buffer is full or EOF. The JSON used is smaller than the buffer size, so this should be okay, but if the JSON value was changed then this could be an issue.

Should we read until EOF? Or Should we read until we find a newline specifically?

Copy link
Author

@SuibianP SuibianP Feb 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can reinvent POSIX getline, but it is probably not worth the effort. Any JSON schema change that exceeds the buffer length will absolutely get truncated without actually overrunning the buffer, break the test visibly and get caught during review.

std::getline(reader, buffer);
if (!buffer.empty())
line = buffer;
rewind(m_fifo_file.m_file);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the flow of the logic here is:

  • lldb-dap (pid 1) opens the fifo
  • pid 1 sends sends the runInTerminal command.
  • pid 1 reads from the fifo (waiting for data)
  • lldb-dap --launch-target (pid 2) starts
  • pid 2 opens the fifo
  • pid 2 spawns the new process
  • pid 2 writes to the fifo (something like {"pid":2}\n)
  • pid 1 finishes the initial read from the fifo
  • pid 1 lldb attaches to the process started by pid 2

This should mean that the parent side only reads from the file and the child side only writes to the file. Additionally, I think they're only each performing 1 write/read on each side so their respective file positions should still be at the beginning of the file after opening.

Comment on lines +169 to +167
// There is no portable way to conjure an ofstream from HANDLE, so use FILE *
// llvm::raw_fd_stream does not support getline() and there is no
// llvm::buffer_istream
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you have the path, can't you use that to open the file with std::ofstream?

Copy link
Author

@SuibianP SuibianP Feb 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On the client (launcher) side it should be possible, but on the server end I suppose not. Windows has the distinction of "server" (from CreateNamedPipe) and "client" (from CreateFile) ends of FIFOs; and unlike on POSIX where FIFOs can read data from any previous write, clients can only communicate with server but not with fellow clients.

Opening a std::ofstream (which underlyingly is likely CreateFile) would fail to create the file ERROR_FILE_NOT_FOUND. The server side must first use CreateNamedPipe which gives out a HANDLE, thus the issue of turning that into some platform independent file object.

@SuibianP SuibianP force-pushed the lldb-dap-runinterminal-windows branch 2 times, most recently from 9f89e9c to c08879c Compare February 26, 2025 13:24
@SuibianP SuibianP requested a review from ashgti February 26, 2025 13:26
std::getline(reader, buffer);
if (!buffer.empty())
line = buffer;
rewind(m_fifo_file.m_file);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think seeking isn't support on a fifo (lseek would return ESPIPE, see man 2 lseek or the Errors section of https://man7.org/linux/man-pages/man2/lseek.2.html). So I'm not sure if you need to do these rewind on non-windows platforms.

@SuibianP SuibianP requested a review from ashgti February 27, 2025 12:12
@SuibianP SuibianP force-pushed the lldb-dap-runinterminal-windows branch 4 times, most recently from fd85cd7 to 20fbed5 Compare March 1, 2025 08:39
@SuibianP SuibianP force-pushed the lldb-dap-runinterminal-windows branch from 20fbed5 to 0b7cc85 Compare March 7, 2025 05:50
@SuibianP
Copy link
Author

SuibianP commented Mar 7, 2025

Hi @ashgti, pinging just in case GitHub had its notification fall through the cracks again :)

I have left several possibly debatable review chains open for your kind attention, and would appreciate instructions on next steps.

@SuibianP SuibianP force-pushed the lldb-dap-runinterminal-windows branch from 0b7cc85 to babc347 Compare April 3, 2025 06:49
@SuibianP
Copy link
Author

SuibianP commented Apr 3, 2025

Pinging @ashgti and @walter-erquinigo for review before I forget again

Currently, the named pipe is passed by name and a transient ofstream is
constructed at each I/O request. This assumes,
  - Blocking semantics: FIFO I/O waits for the other side to connect.
  - Buffered semantics: Closing one side does not discard existing data.

The former can be replaced by WaitNamedPipe/ConnectNamedPipe on Win32,
but the second cannot be easily worked around. It is also impossible to
have another "keep-alive" pipe server instance, as server-client pairs
are fixed on connection on Win32 and the client may get connected to it
instead of the real one.

Refactor FifoFile[IO] to use an open file handles rather than file name.

---

Win32 provides no way to replace the process image. Under the hood exec*
actually creates a new process with a new PID. DebugActiveProcess also
cannot get notified of process creations.

Create the new process in a suspended state and resume it after attach.
@SuibianP SuibianP force-pushed the lldb-dap-runinterminal-windows branch from d13e94b to f9675fb Compare May 5, 2025 10:19
@SuibianP
Copy link
Author

SuibianP commented May 6, 2025

Pinging again for review @ashgti @omjavaid @walter-erquinigo

Also want to highlight SuibianP@f9675fb#diff-ed319b24adfa654beeff8b32f9d8f975c62299fbe5cb6fbe1028aa6bbaa369e7. Not sure how it is passing CI though; I might as well be understanding it wrongly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants