(rsh 3 q.tub) reallocates the whole rest of the string, which will indeed be worse than cdr on a tape. You want the input to be like, an offset-and-cord "slice", not a cord alone.

Though also, when I did some small-scale testing of that, having to chase the tape pointers got swamped by having to chase the formula pointers for tree-walking the combinators themselves, to no visible improvement. (Even when q.tub was like, a dummy number denoting "n unspecified characters"). Not sure if the bytecode interpreter is clever enough to solve that at this point.

This is a worthy experiment -- I'm surprised at how small this diff is!

@ohAitch is right, you'll want to keep a cursor into the input cord and +cut bytes out of it to avoid reallocating every intermediate stage (this will probably make the diff quite a bit bigger). See #5224 for this same transformation being applied to the naive rollups.

Better allocation patterns should result in significant performance improvements, but I don't know how it will compare to the current tape handling (especially when those tapes have decent locality due to inner-road slab allocation). There will be significantly more function calls ...

Yeah I wouldn't be too surprised if this was still straight-up slower unjetted due to the extra tuples and function calls; though once you get to multiple kb of input text probably locality or no the 24x memory overhead of tapes starts to sting. (Certainly tripping hoon.hoon will flush everything else out of multiple layers of cache, though that's an extreme case and in practice probably most bytes parsed are from inputs <200 characters.)
In a jet ofc, the cut is directly grabbing a byte from an array.

For a perhaps more direct comparison(and to keep diff size manageable), you could write

++  tx  ta  ::/tc
++  ta
  |%
  ++  type  tape
  ++  take  |=(a=cord ^-(type (trip a)))
  ++  done  |=(a=type ^-(? ?=(~ a)))
  ++  look  |=(a=type ^-(char ?~(a '' i.a)))
  ++  next  |=(a=type ^-(type t.+.a))  :: unchecked
  --
++  tc
  |%
  ++  type  (pair @u cord)
  ++  take  |=(a=cord ^-(type [0 a])))
  ++  done  |=(a=type ^-(? =(p.a (met 3 q.a))))
  ++  look  |=(a=type ^-(char (cut 3 [p.a 1] q.a)))
  ++  next  |=(a=type ^-(type a(p +(p.a))))  :: unchecked
  --
::
+$  nail  [p=hair q=type:tx]  :: etc   

(You might want to cache the met / support slices that also omit part of the end of the cord, though for final jetted performance that's presumably worse bc it's just a slot lookup)

Closing this one since it's an old RFC and there's no plan to work on it or merge it for now.

In lieu of a better place to put and discuss "draft RFC with working code", I want to keep this kind of thing open.

Four years later, time to take another stab at this!

Changes the parsers' "continuation" from a tape representing the remainder of the parse input, to the original parse input paired with a pointer into it. We +cut bytes out of this input to process them.

Includes a rewritten +inde (and deduplicates +iny by pointing to it explicitly), because the original implementation was difficult to make work in this new context, and was somewhat questionable anyway. (Degenerate cases would reparse the entire rest of the input...)

Note that for converting inputs from tapes, we must be cautious around two things:

• +crip would drop null characters, so we must +rep instead.
• Trailing null characters cannot be represented by the cord itself so must instead be represented by the .m in the parser input. Therefore, when converting from tape, measure the tape length first, instead of measuring the resulting atom. (It's not common by any means, but without this the wasm tests break.)

Despite best-effort changes, /lib/der gets broken here. I spent some time on it and thought I had it right, tests did pass! But for some reason now they don't anymore. Probably stupid, but I'll punt on it for now.

Updated jets are on vere's m/cord-parsers.

I took some measurements, and suspect the juice may not be worth the squeeze.

Using %bout, the time it takes to parse hoon.hoon gets reduced by at best 2%. Really, the difference is negligible and can easily be chalked up to other factors.

I wasn't sure how to measure the memory situation accurately here. I put a %meme hint before and after a +ream call. Only the "solid heap" displays any significant change.

In stock hoon, the solid heap grows by about 21 MB after parsing hoon.hoon.
With these changes, the heap grows by about 29 MB instead.

So it seems this implementation is worse than our baseline, memory-wise.

Maybe there is (again!) something stupid I've overlooked. It's certainly been too long since I wrote serious vere code, maybe I'm neglecting to refcount properly in the updated jets. Once again: RFC!