Thanks to visit codestin.com
Credit goes to github.com

Skip to content

stream pipelining #3095

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
Jan 17, 2022
Merged

stream pipelining #3095

merged 2 commits into from
Jan 17, 2022

Conversation

@mangas
Copy link
Contributor

@mangas mangas commented Dec 22, 2021

No description provided.

@mangas mangas force-pushed the filipe/stream-pipeline branch 3 times, most recently from 3d2a5ee to 0870ece Compare December 23, 2021 13:36
leoyvens
leoyvens reviewed Dec 23, 2021
View reviewed changes
Copy link
Collaborator

@leoyvens leoyvens left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A more encapsulated approach which I'd suggest is to introduce a BufferedBlockStream, which takes as input a block stream and a buffer size, and returns

@mangas mangas force-pushed the filipe/stream-pipeline branch 2 times, most recently from 1bacbbe to 72d5aaf Compare December 23, 2021 17:02
@mangas mangas marked this pull request as ready for review December 23, 2021 17:27
@mangas mangas force-pushed the filipe/stream-pipeline branch 2 times, most recently from e518cf7 to abf15a3 Compare December 31, 2021 00:50
@mangas mangas requested a review from leoyvens December 31, 2021 00:50
@mangas mangas force-pushed the filipe/stream-pipeline branch 3 times, most recently from 3f6b299 to cc1e71a Compare January 5, 2022 16:52
evaporei
evaporei reviewed Jan 5, 2022
View reviewed changes
Copy link
Contributor

@evaporei evaporei left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, still I think it's worth to have @leoyvens take a last look

@mangas mangas force-pushed the filipe/stream-pipeline branch 10 times, most recently from 126af66 to 4e5652b Compare January 10, 2022 11:23
leoyvens
leoyvens requested changes Jan 12, 2022
View reviewed changes
@mangas mangas force-pushed the filipe/stream-pipeline branch 2 times, most recently from a6013a8 to ded67bf Compare January 13, 2022 16:28
leoyvens
leoyvens reviewed Jan 14, 2022
View reviewed changes
core/src/subgraph/instance_manager.rs
use tokio::task;

const MINUTE: Duration = Duration::from_secs(60);
const BUFFERED_BLOCK_STREAM_SIZE: usize = 100;
Copy link
Collaborator

@leoyvens leoyvens Jan 14, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BUFFERED_POLLING_BLOCK_STREAM_SIZE would be more accurate.

@mangas mangas force-pushed the filipe/stream-pipeline branch from ded67bf to 448970a Compare January 14, 2022 15:15
@mangas mangas force-pushed the filipe/stream-pipeline branch 3 times, most recently from 76f8966 to b0802c4 Compare January 17, 2022 14:40
@mangas mangas force-pushed the filipe/stream-pipeline branch from b0802c4 to 74cd3b7 Compare January 17, 2022 14:47
@mangas mangas merged commit 6f348ad into master Jan 17, 2022
@mangas mangas deleted the filipe/stream-pipeline branch January 17, 2022 15:19
@leoyvens leoyvens mentioned this pull request Mar 7, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@leoyvens leoyvens leoyvens approved these changes

+1 more reviewer

@evaporei evaporei evaporei left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@mangas @leoyvens @evaporei