Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Implement local shuffle optimization for broadcast relations #549

@senderista

Description

@senderista

@stechu had a simple and beautiful idea: for broadcast inputs to a hash-partitioned shuffle, we can just filter them in-place to retain only tuples satisfying the hash partition condition for that worker. The same would apply to round-robin shuffles, except we would partition the tuple count on uniform boundaries. This might require support on the MyriaX side.

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions