v0.5.0: Added direct access to PJRT buffers when PJRT running in CPU; Benchmarks. #20
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Added BufferToHost benchmarks; Consolidated CGO calls into fewer for Buffer.ToHost for significant gains for small tensors.
Added benchmarks and its results.
Added arena benchmarks. Updated Add1 benchmarks.
Updated benchmark and CHANGELOG.
…tableAlive and BuffersAlive.
- Fixed client.CreateViewOfDeviceBuffer: despite documentation, it seems to require setting a on_detele_callback needs to be set to something != nullptr.
Updated coverage.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
install_linux_amd64_amazonlinux.shand pre-built libraries for amazonlinux (built using old glibc support).xargsin installation script for Linux.go.yaml) to only change the README.md with the result of the change, if pushing to themainbranch.prjt.arenato avoid costly allocations for CGO calls, and merged some of CGO calls for general speed-ups.The following functions had > 50% improvements on their fixed-cost (measured on transfers with 1 value, and minimal programs)
execution time (not the variable part):
Buffer.ToHost()Client.BufferFromHost()LoadedExecutable.Execute()BufferToHostandBufferFromHostbenchmarks.XLA_DEBUG_OPTIONS: if set, it is parsed as aDebugOptionsproto thatis passed to the JIT-compilation of a computation graph.
LoadedExecutable.Execute()now waits for the end of the execution (by settingPJRT_LoadedExecutable_Execute_Args.device_complete_events).Previous behavior lead to odd behavior and was undefined (not documented).
dtypes:SizeForDimensions()to be used for dtypes that uses fractions of bytes (like 4 bits).Client.NewSharedBuffer(and the lower levelclient.CreateViewOfDeviceBuffer()) to create buffers with sharedmemory with the host, for faster input.
AlignedAllocandAlignedFreerequired byclient.CreateViewOfDeviceBuffer.Buffer.Datafor direct access to a buffer's data. Undocumented in PJRT, and likely only works on CPU.