Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

@Fang-
Copy link
Collaborator

@Fang- Fang- commented Feb 19, 2021

Bunch of small goodies in here, but the majority of this revolves around decoupling term.c, dill and drum.

Previously, they all knew about and played a role in drum's "prompt at the bottom" semantics.
With these changes, only drum knows about the prompt. term.c just remembers whatever the line at the bottom of the screen contains to aid in spinner drawing. It also whimsically inserts printfs between the second-to-last and bottom lines of the screen, but those only show up in the default session, which is always linked to drum, so it works out fine.

Other fixes and improvements include:

  • Support for 2D cursor movement, and by extension 2D screen printing,
  • Better (but still incomplete) support for multiple sessions in dill,
  • Minimal X10-style mouse-click support,
  • A cleaned up dill api,
  • General cleanup and smaller improvements.

There's some remaining work here, but this is probably good to go for a first review while I work on that. While the order of these commits is a bit all over the place, the commits themselves should be well-contained changes, often paired with additional commentary, and as such probably the best way to review all of this.
Please also keep an eye out for //REVIEW comments strewn about, I'd love some input on those.

Some remaining work:

  • Fix/improve -t mode behavior. See also CI output, it's kinda not good in some places.
  • Update webterm and its dependencies to reflect these changes. (The blit mark doesn't even build rn, but CI passes anyway! [distant screaming]) Updated, but not yet perfect. Remaining work includes:
    • Restructuring and cleanup of codebase, follow best practices.
    • Improved subscription management. (Pending broader landscape subscription reform.)
    • Respond to window resizing.
    • Respond to landscape theme (light vs dark).
  • Update KH to reflect these changes.
  • Make clay and gall send proper printout requests to dill, instead of having dill handle their updates locally.
  • Experiment with multi-session dill in webterm, settle on an interface. May or may not get Settled on an (as of yet unreviewed) interface. Further drum & webterm work punted to a separate PR. Fixes unused Dill tasks #2646.

Currently targeting release/next-sys release/next-dill, which is the merge of next-sys and next-js, so we have a cleaner diff here.
The arvo-side changes depend on the runtime-side changes. So maybe we want release/next-step or w/e for that? Also depends on urbit/landscape#188 if we want users to actually know about the strict need for a runtime upgrade.

Might fix #353, if that wasn't already the case.

Fang- added 29 commits January 26, 2021 18:37
Likewise for belt. This necessitates renaming the %mor blit for newlines
to %nel, making this require a new runtime version. That's fine, more
breaking changes are to follow.
Instead of relative, character-based position. In anticipation of
greater degrees of cursor freedom, and so that we don't have to
track invisible characters all the time.
Current era runs exclusively on v1.0 and up.
Instead of confining you to just the bottom row. 0,0 is bottom left.

Doesn't behave exactly as expected for non-zero column coordinates yet,
but all in due time.
We want to decouple drum state & semantics from term.c. But we also want
to continue showing printfs without mangling whatever the arvo side is
doing. Short of formalizing various "terminal modes", the best we can do
it assume that the main session is always in drum mode, and use that
assumption to squeeze into the actual content stream when doing printfs.

In essence, we insert the printf between the prompt and content, pushing
the content up into terminal scrollback. Both the prompt and cursor
position are maintained without knowledge of their original states.

Because this logic relies on knowing the (accurate) terminal height, and
this logic being the *only* thing in term.c that reads from the height,
we no longer initialize the terminal size with a sane height. Instead,
we set it to zero, and check for that to determine whether we're ready
to use this logic or not.

Due to the way this inserting works, trailing newlines are no longer
required from the output. For consistency, we manually add trailing
newlines when this logic cannot be used, both in the above-described
circumstance, and the less-common case of u3l_log.
This will of course need to be accounted for in all existing calls to
u3l_log, but we move that into the next commit for readability's sake.
As of dc2c990 these are redundant, and keeping them in would result
in trailing whiteline.

The result is that sometimes we call u3l_log with the empty string. We
simply want it to insert a newline into the scrollback for us. GCC
complains about this, because it leads to using an empty format string,
but we consider this sane behavior. We update the flags accordingly.
Only detects mouse clicks. Though, "9" mode seems broken, or unsupported
or something? Probably need to upgrade to "1000" mode or higher, but
that also reports scrolling events and such, which don't want to steal
from the context we're running in just yet.
No longer inserts newlines or redraws the prompt post-print, pushing
this responsibility down to drum where it belongs.

Additionally, separates the flow for dill's own output, from that of the
console application. This lets us keep the desired behavior for now, and
will ease reworking in the future.

Last-printed-line and cursor position are still kept around in dill
state, in order to respond to the relevant scry endpoints. These should
either be refactored to scry into the underlying console app, or be
removed entirely in favor of %hey.
As follow-up to 3fdef14, we now no longer store the prompt or cursor
in state. These were still used for view initialization, but it's more
appropriate to %hey the underlying console application in those cases.
The scry endpoints we remove, expecting clients that depended on them
to send a %hail instead.
Previously, these affected dill state. Now, they're equivalent to %lin
and %klr blits.
This prepares us for actually making use of multiple session in a sane
way.

Notable implicit change is that we no longer crash on an "unrecognized
duct", instead always handling it as destined for the default session.
Before recent dill changes, this wouldn't always be visible, since it
would get drawn in place of (and subsequently get overwritten by) the
prompt. Now that it displays consistently again, it should look a bit
better than just a noun dump.

This is somewhat redundant with gall's own "reloading agent" printfs,
but you know what they say: printfs don't real!
As per the note a couple lines up, +fore depends on drum semantics being
active. We can only guarantee those being present for the default
session, not for any others. So, we print a warning when appropriate.
Some of the remainder are still _presently_ unused, but point at
functionality we want to support again in the near future. The ones
removed here are either redundant or have no clear purpose.
styx and stub are both defined in lull. Having functions for dealing
with them in zuse rather than userspace is fitting.

While not a _common_ format per se, it still seems best at home in
+format, instead of on its own.
Adds the vertical position of the cursor to mirror state, and caps the
cursor movement to the edges of the known window.
Instead, rely on the console application to send whatever is needed
directly following a %clr blit.
This simplifies the behavior of individual blits, making their
implementation simpler and giving arvo more control.

This lets us write on top of existing content, instead of completely
replacing the affected row. Additionally, lets us draw starting at the
cursor position, instead of the leftmost column.

To retain the previous behavior, preface with [%hop 0] to move the
cursor to the start of the line, [%wyp ~] to clear the existing content,
and finally your %lin to render it.
%url blits are meant for "activating" urls. Of course, opening a new
browser tab from within a C program is difficult, so we don't do it.

This is still better than doing the faux activation by just printing
the url. term.c no longer really knows where/how to draw it, and it's
meant as interactive behavior rather than visual output, anyway.
This keeps the cursor position at the last %hop location. And being able
to go back to that position without relying on state is just plain
convenient.

The exception here is %nel blits, which move the cursor (down and) to
the start of the line, as expected of "new line" semantics.
The bottom line is where we draw the spinner, so we're only interested
in screen draws that affect that part of the screen (so that we may
restore it after having drawn the spinner there).

Given recent changes, redraws should be less frequent than updates, so
we lazily store the utf32 encoded line contents in state, instead of
the eagerly converted utf8 version.
Without the \r, the first couple printfs (generally, regarding replay)
would not align on the lefthand side.
That's the one line we track. If it gets drawn anywhere else, we can't
restore the overwritten contents.
To hint to developers that this form of output isn't as concrete or
reliable as they might assume it to be.
@Fang- Fang- added the dill label Feb 19, 2021
further propagating temporary space leak mitigation
@joemfb joemfb changed the base branch from release/next-dill to release/next-vere September 9, 2021 18:09
@joemfb
Copy link
Collaborator

joemfb commented Sep 10, 2021

This PR is now ready, the only thing still blocking it is an issue with the mingw windows build.

We're hitting an assertion in libuv, but only when running with -t (or -d`):

Assertion failed: req->write_buffer.base, file src/win/pipe.c, line 1063

Notably, this is the same assertion that we would hit if we were building against an unpatched libuv. @locpyl-tidnyd, do you have any insight into what might be going wrong here?

Copy link
Collaborator

@joemfb joemfb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

I've added backwards-compatibility for -t, and comments describing where to add further compatibility shims (in the case of a new vere running an old pier). There's still work to be done to make versioning info straightforwardly available to i/o drivers. We need straightforward patterns for simultaneously supporting multiple versions of the arvo/vere interface. But this work needn't block release, it can be added at any future point.

@Fang-, please take a look at these last couple comments, and see if there are any other compatibility issues that should be noted for the future. If not, feel free to push the big green button. Sorry for the many delays, and thank you for your patience!

Comment on lines +667 to +669
// XX for backwards compatibility, check kelvin version
// and fallback to [%met @c]
//
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@joemfb I don't think these are necessary? Current dill, as it exists on the network, is already forward-compatible with the inputs from the new runtime. See also #4942.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But previous dill isn't, so people with old piers could end up in a situation where they have to juggle binaries to get upgraded (or even just poke around). (Since non-modified %txt input will still work, they can probably get unstuck by just typing |merge ... -- these concerns are a little more theoretical.)

In general, dropping support for old protocols should happen after some longer period time, and it should be straightforward to continue supporting multiple versions. But those patterns are yet to be established, which is why I'm settling for comments.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair game. I'm relatively comfortable (and relieved (^: ) to merge this as is, but if any poor sap happens to get hurt by this: blame me.

@Fang- Fang- merged commit 39667fc into release/next-vere Sep 15, 2021
@Fang- Fang- deleted the m/next-gen-term branch September 15, 2021 16:43
@joemfb joemfb restored the m/next-gen-term branch September 24, 2021 20:17
joemfb added a commit that referenced this pull request Sep 24, 2021
This reverts commit 39667fc, reversing
changes made to 698fd6f.
joemfb added a commit that referenced this pull request Sep 24, 2021
…ranches 'frodwith/keccak' and 'botter-nidnul/release/next-vere' into jb/rnv-merge

* jb/rnv-restage:
  build: patch windows uv_pipe impl to allow null writes
  build: update pmnsh to support cachix text/x-nix-narinfo responses
  vere: renames terminal streams for clarity, removes unused uv_tcp_t
  vere: suppress null writes in term.c
  Revert "Merge pull request #4463 from urbit/m/next-gen-term"

* jb/rnv-fixes:
  vere: bumps versions
  vere: fixes crash (null deref) in comet booting
  u3: fixes printf in c3_assert()

* frodwith/keccak:
  jets: ice keccak
  working keccak jets
  urcrypt: fixing some typos - now builds
  Squashed 'pkg/urcrypt/keccak-tiny/' content from commit 9b438dc0f7
  urcrypt,jets: initial commit of keccak material

* botter-nidnul/release/next-vere:
  urcrypt: argon2 only optimize for x86_64 on x86_64
joemfb added a commit that referenced this pull request Sep 25, 2021
joemfb added a commit that referenced this pull request Sep 28, 2021
* release/next-vere: (2627 commits)
  vere: bumps versions
  vere: fixes crash (null deref) in comet booting
  u3: fixes printf in c3_assert()
  build: patch windows uv_pipe impl to allow null writes
  build: update pmnsh to support cachix text/x-nix-narinfo responses
  vere: renames terminal streams for clarity, removes unused uv_tcp_t
  vere: suppress null writes in term.c
  Revert "Merge pull request #4463 from urbit/m/next-gen-term"
  urcrypt: argon2 only optimize for x86_64 on x86_64
  jets: ice keccak
  working keccak jets
  urcrypt: fixing some typos - now builds
  Squashed 'pkg/urcrypt/keccak-tiny/' content from commit 9b438dc0f7
  urcrypt,jets: initial commit of keccak material
  tests: remove unnecessary runtime config from hashtable_tests
  u3: restore hashtable root struct layout for compatibility
  serf: fix incorrect conditional (integer precision) in event tracing
  vere: adds backwards-compatibility TODO comments for terminal inputs
  vere: adds backwards-compatibility to blit handling under -t
  u3: replaces spurious bloq-size bail:exit's in +rap and +rep jets
  ...
joemfb added a commit that referenced this pull request Oct 5, 2021
* release/next-vere: (329 commits)
  pill: all
  serf: updates grab to fallback to old output on error
  serf: disable hashboard in meld
  u3: fix big noun equality
  vere: bumps versions
  vere: fixes crash (null deref) in comet booting
  u3: fixes printf in c3_assert()
  build: patch windows uv_pipe impl to allow null writes
  build: update pmnsh to support cachix text/x-nix-narinfo responses
  vere: renames terminal streams for clarity, removes unused uv_tcp_t
  vere: suppress null writes in term.c
  Revert "Merge pull request #4463 from urbit/m/next-gen-term"
  urcrypt: argon2 only optimize for x86_64 on x86_64
  jets: ice keccak
  working keccak jets
  urcrypt: fixing some typos - now builds
  Squashed 'pkg/urcrypt/keccak-tiny/' content from commit 9b438dc0f7
  urcrypt,jets: initial commit of keccak material
  tests: remove unnecessary runtime config from hashtable_tests
  u3: restore hashtable root struct layout for compatibility
  ...
@Fang- Fang- mentioned this pull request Nov 25, 2021
@Fang- Fang- deleted the m/next-gen-term branch August 24, 2022 17:16
@zalberico zalberico mentioned this pull request Oct 25, 2022
@tacryt-socryp
Copy link
Contributor

So how do you use any of the new terminal features such as session support?

@Fang-
Copy link
Collaborator Author

Fang- commented Jan 27, 2023

@tacryt-socryp Depends on what context you're talking about.

Drum/sole/shoe agents can distinguish between connecting sessions from the subscription path. For +on-watch calls where the path starts with /sole, you can call +path-to-id from the /lib/sole to receive a sole-id identifying the session. Use this to register it in your state. Inputs for that session will have that same id in their %sole-action pokes. If you use /lib/shoe, these ids just come in as arguments on the shoe-specific arms.
The rest of the sole protocol has not changed. You do not get any new output affordances.

If you're developing a standalone application, it's possible for you to speak the dill protocol directly, like drum does. If a session connects to an agent directly, it gets watched on a /dill/[session-id] path. For inputs, it will receive pokes with a %dill-poke mark, containing a [session=@ta belt=dill-belt:dill]. To send outputs, produce %dill-blit facts containing a single blit:dill.
The blit:dill includes new affordances, like moving the cursor around on the screen, to draw at arbitrary locations. It's also very primitive, in practice this means you'll write a lot of dumb rendering logic.

The web terminal can create and connect to different sessions. To connect directly to a custom agent, use a session name with the following format: agent!session-name. If the target agent does not support the dill protocol, the session will be unusable.
I have ideas on how to let apps bring you into specific webterm sessions like this in a more user-friendly way, stay tuned.

The runtime currently does not support connecting to anything but the default session, as it always has.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.