Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

andrewleech
Copy link

This significantly speeds up readline on files opened directly from a mpremote mount.

With a local file hello.py that's 5741 bytes, using mpremote mount . with a pyboard-d:

import time;
def test_readline():
    s = time.ticks_ms();
    [l for l in open("hello.py")]; 
    print(time.ticks_diff(time.ticks_ms(), s))
test_readline()

Without this MR, this reports 16650 (ms)
With this MR, it drops to 1415

jimmo and others added 7 commits June 19, 2023 11:53
This is a step towards making the transport expose a Python API rather than
functions that mostly print to stdout.

Most use cases of `transport.eval()` are to get some state back from the
device, so have it return as a value directly by default.

Updates uses of `transport.eval()` to remove the parse argument where it
now isn't needed, make the `rtc` command use eval/exec, and update the
`mip` command to use eval's parsing.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <[email protected]>
This introduces a Python filesystem API on `Transport` that is implemented
entirely with eval/exec provided by the underlying transport subclass.

Updates existing mpremote filesystem commands (and `edit) to use this API.

Also re-implements recursive `cp` to allow arbitrary source / destination.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <[email protected]>
- Adds transport API `fs_hashfile` to compute the hash of a file with given
algorithm.
- Adds command `mpremote hash file` to compute and print sha256 hash.
- Uses the hash computation to improve speed of recursive file copy to
  avoid copying a file where the target is identical.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <[email protected]>
These tests are specifically for the command-line interface and cover:
 - resume/soft-reset/connect/disconnect
 - mount
 - fs cp,touch,mkdir,cat,hash,rm,rmdir
 - eval/exec/run

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <[email protected]>
If possible it will use the board's support (e.g. built-in hashlib or
hashlib from micropython-lib), but will fall back to downloading the
file and using the local implementation.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <[email protected]>
Makes the filesystem command give standard error messages rather than
just printing the exception from the device.

Makes the distinction between CommandError and TransportError clearer.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <[email protected]>
This significantly speeds up readline on files opened
directly from a mpremote mount.

Signed-off-by: Andrew Leech <[email protected]>
@dpgeorge
Copy link

dpgeorge commented Nov 1, 2023

You probably want to open this on the upstream repo.

@andrewleech
Copy link
Author

I made the change on top of tools/mpremote: Recursive directory sync, regression tests, transport filesystem API, hash command. #11777 so pushed it here for first pass / guage interest.

@dpgeorge
Copy link

dpgeorge commented Nov 1, 2023

Ah, I see!


If your code is doing a lot of readline's then this is definitely an improvement. But if you don't use readline you are unnecessarily paying for the size of the VFS hook code. Although the change in size of that VFS hook code is pretty small. And the existing loop that did read(1) is allocating a lot of strings by accumulating the single characters.

So, I'd say this is a good improvement, worth doing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants