Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Support stripping away layers in iterate_many #2252

@dominiklohmann

Description

@dominiklohmann

I have a potentially infinite array of objects I want to parse:

[{…}, {…}, {…}, …]

I do not really care about the outer array, but am much rather looking for the fastest way to stream the objects contained in the array.

The popular command-line tool jq supports a way to "strip" layers of JSON while streaming the output without requiring the outer array's structure to be fully available, i.e., it continues to work even if the end of the array never arrives.

jq -ercn --stream 'fromstream(1|truncate_stream(inputs))'

I am looking to do something just like this in my own application, which uses simdjson's iterate_many API. My (perhaps naïve) thinking is that the API likely needs an additional parameter that specifies how many layers to strip away.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions