-
Notifications
You must be signed in to change notification settings - Fork 362
zuse: changes in preparation for JSON jets #6463
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Release urbit-os-v2.135
…umber. Resolves urbit#6377 - stuck %eth-watcher due to get-block-by-number zero-padded number RPC failure
Per suggestion at https://github.com/urbit/urbit/pull/6377/files/f5eeffef587b06bc93540536cf88a10e49dc77fd#r1128704330 Moves x-co:co into num-to-hex-minimal.
Fix stuck %eth-watcher due to get-block-by-number zero-padded number RPC failure.
This commit refactors `sur/hood.hoon` and changes the signature of the `+vats` generator thus: ```hoon |= $: [now=@da eny=@uvj bec=beak] $@(~ [?(%suspended %running %blocking %nonexistent) ~]) $: verb=? show-suspended=? show-running=? show-blocking=? show-nonexistent=? == == ``` Called with a single positional argument, `+vats` will show only those desks fitting the description given, while keyword parameters allow finer-grained control over which desks are described. The `verb` parameter determines whether the full load of desk information be shown, or a subset thereof. Resolves urbit#6297.
We were retrying failed kelvin upgrades as many times as we had apps that needed to be suspended, because suspending an app triggers an attempt to run the next kelvin upgrade. This suspends all those apps in one batch move, and then tries the next kelvin upgrade only once at the end. Fixes urbit#6407 Partially addresses urbit#6285
hotfix clay rebuild loop
+vats options
sss: Unsubscribing and permissions
…-desk arvo: add more files to |new-desk generator
Release urbit-os-v2.136
…-contact-scry ames: add last-contact scry endpoint
Merge master back to develop after urbit-os-v2.136 release
Instead of including wrapped-task as-is in most call traces, we now only include it in traces for crashing (harden task) calls. For everything else, we include only the tag of the resulting $task. Closes urbit#6444.
dill: exclude wrapped-task from most +call traces
ames: ping faster to sponsors
arvo: print module compilation times
+vats: fix crash on nonexistent desks
behn: propagate errors in deferred moves
|
@joemfb says |
The unit tests for de-json:html and en-json:html in zuse.hoon were copied from the JSON Test Suite (https://github.com/nst/JSONTestSuite). While a good test suite if usable as-is, it has a lot of problems with it, namely: inconsistent test names; repeated tests; and mixing tests for multiple cases into one file. Since the tests need to be manually added to Urbit anyway, this commit uses the opportunity to clean up the test suite by making the following changes: - Renaming tests to use a consistent naming scheme - Removing repetetive test cases - Splitting multi-case tests into multiple single-purpose tests
The JSON standard rejects all un-escaped, un-printable characters EXCEPT for the delete character (ID 127, i.e. 0x7F). See: https://www.crockford.com/mckeeman.html Via: https://www.json.org/json-en.html
JSON supports Unicode as both UTF8 sequences and escaped UTF16. Unicode points U+10000 to U+10FFFF are encoded as two consecutive escaped UTF16 units known as a surrogate pair. The JSON decoder was previously treating surrogate pairs as two individual escaped UTF16 units. In addition, the JSON standard allows hexadecimal letters to be either uppercase or lowercase, which the parser was not respecting. See: - https://www.crockford.com/mckeeman.html - https://www.json.org/json-en.html - https://en.wikipedia.org/wiki/UTF-16#Code_points_from_U+010000_to_U+10FFFF Git issue: urbit#1776
JSON supports Unicode as both UTF8 sequences and escaped UTF16. UTF8 is encoded as between 2 and 4 bytes, the first byte of which governs the length. The JSON parser was previously accepting any non-ASCII bytes as UTF8 in any order. See: - https://www.crockford.com/mckeeman.html - https://www.json.org/json-en.html - https://en.wikipedia.org/wiki/UTF-8#Encoding
The backspace character was not being escaped like other invisible characters.
Encode JSON to cord instead of tape. Code copied from @joemfb 's version.
joemfb
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this will do
my motivation to not change +teff and +sane is that @t is not exactly utf-8 -- it imposes additional constraints on whitespace (rejecting newlines). that may or may not be a good idea (and it's violated in more than a few places), but it is longstanding and was intentional (i'll see if i can pull up links, maybe from the old urbit-dev list). it's a different kind of change than the others here, and should be decided on for the whole system with context and care. separate tracking issues will be good to have.
|
@joemfb Sounds good - I'll make some additional tracking issues and include what context I have now. @jalehman @belisarius222 I leave this one for you guys to merge whenever and into where ever it needs to go. |
Updated version of changes to Vere from [#5566](urbit/urbit#5566) These changes jet the JSON encoding / decoding arms in [this PR](urbit/urbit#6463). **TESTS** - JSON unit tests in Arvo pass with jets enabled - `bazel test --build_tests_only ...` passes - Both of the above hold true when `ice` flag disabled - Wall-clock unit test times from previous tests replicated - [Previous tests](urbit/urbit#5566 (comment)) = `28.9%` - Current tests = `27.7%` - Let's be pessimistic and say "70% reduction in time to parse JSON" **TODO** - [x] Confirm wall-clock test times from previous PR still hold true - [ ] Redo Memcheck - [ ] Redo AddressSanitizer check
Updated version of changes to Arvo from #5566
These changes resolve several issues with how JSON is handled and prepare Arvo for JSON encoding/decoding jets.
See #342 for test results.
Resolves:
#1775
#1776
#5456